2024-06-28T14:46:41.519 INFO:root:teuthology version: 0.0.1.dev329+g544fecb 2024-06-28T14:46:41.520 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_cbodley@teuthology --archive /home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037 --name cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi --description rgw/verify/{0-install accounts$/{tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} --verbose -- /home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037/orig.config.yaml 2024-06-28T14:46:41.571 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-28T14:46:41.662 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037 branch: wip-cbodley-testing description: rgw/verify/{0-install accounts$/{tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} email: null first_in_suite: false job_id: '7778037' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-cbodley-testing ceph: conf: client: debug rgw: 20 debug rgw notification: 20 rgw bucket counters cache: true rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw d3n l1 datacache persistent path: /tmp/rgw_datacache/ rgw d3n l1 datacache size: 10737418240 rgw d3n l1 local datacache enabled: true rgw enable ops log: true rgw max chunk size: 4194304 rgw obj stripe size: 6291456 rgw s3 auth use sts: true rgw sts key: abcdefghijklmnop rgw torrent flag: true rgw user counters cache: true setgroup: ceph setuser: ceph global: mon client directed command retry: 5 ms inject socket failures: 5000 osd heartbeat grace: 40 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 rgw override bucket index max shards: 1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd fast shutdown: false osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled - \(OSD_SLOW_PING_TIME sha1: ec2a38ba1b69b5357de8f0e8f96b00b4a577169d ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ec2a38ba1b69b5357de8f0e8f96b00b4a577169d rgw: client.0: valgrind: - --tool=memcheck - --max-threads=1024 compression type: random datacache: true datacache_path: /tmp/rgw_datacache ec-data-pool: true erasure_code_profile: crush-failure-domain: osd k: 3 m: 1 name: testprofile frontend: beast inline data: false storage classes: LUKEWARM, FROZEN s3tests: accounts: iam alt root: RGW99999999999999999 iam root: RGW88888888888888888 s3 tenant: RGW33333333333333333 force-branch: ceph-master slow_backend: true storage classes: LUKEWARM, FROZEN thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-cbodley-testing sha1: ec2a38ba1b69b5357de8f0e8f96b00b4a577169d owner: scheduled_cbodley@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 8526 sha1: ec2a38ba1b69b5357de8f0e8f96b00b4a577169d sleep_before_teardown: 0 suite: rgw suite_branch: wip-cbodley-testing suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: ec2a38ba1b69b5357de8f0e8f96b00b4a577169d targets: smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFwhDo0ebMNmb/A01I7bjDnzquAQgYCuAT3VMgMwd32ijenqYq2JwS91Hm2ikkCrvijXPE3oeNArhF2jWLa83AQ= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIeW5AMP9KxVexvh1h1g+7SB5yjzlLb3/Ib+ea2Eyfv98+KHfDZ11Ut/FYyV/4cz+T75rd5cUfGe75dKdvrCBh4= tasks: - install: extra_system_packages: deb: - s3cmd - maven rpm: - s3cmd - maven - ceph: null - openssl_keys: null - rgw: client.0: null - tox: - client.0 - workunit: clients: client.0: - rgw/run-datacache.sh env: RGW_DATACACHE_PATH: /tmp/rgw_datacache - workunit: clients: client.0: - rgw/run-bucket-check.sh - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - cls/test_cls_rgw_gc.sh - cls/test_cls_rgw_stats.sh - cls/test_cls_cmpomap.sh - cls/test_cls_2pc_queue.sh - cls/test_cls_user.sh - rgw/test_rgw_gc_log.sh - rgw/test_rgw_obj.sh - rgw/test_librgw_file.sh - rgw/test_awssdkv4_sig.sh - workunit: clients: client.0: - rgw/test_rgw_s3_mp_reupload.sh - ragweed: client.0: default-branch: ceph-master rgw_server: client.0 stages: prepare,check - workunit: clients: client.0: - rgw/run-reshard.sh - s3tests-java: client.0: force-branch: ceph-master force-repo: https://github.com/ceph/java_s3tests.git - s3tests: client.0: rgw_server: client.0 - workunit: clients: client.0: - rgw/run-versioning.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 544fecbcd55f3d2b6f478823823ce40cbefef1d4 timestamp: 2024-06-28_14:14:22 tube: smithi user: cbodley verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698093 2024-06-28T14:46:41.662 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa; will attempt to use it 2024-06-28T14:46:41.663 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks 2024-06-28T14:46:41.663 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-28T14:46:41.665 INFO:teuthology.task.internal:Checking packages... 2024-06-28T14:46:41.684 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'ec2a38ba1b69b5357de8f0e8f96b00b4a577169d' 2024-06-28T14:46:41.684 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-28T14:46:41.684 INFO:teuthology.packaging:ref: None 2024-06-28T14:46:41.684 INFO:teuthology.packaging:tag: None 2024-06-28T14:46:41.684 INFO:teuthology.packaging:branch: wip-cbodley-testing 2024-06-28T14:46:41.684 INFO:teuthology.packaging:sha1: ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T14:46:41.684 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-cbodley-testing 2024-06-28T14:46:42.061 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-4592-gec2a38ba-1jammy 2024-06-28T14:46:42.062 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-28T14:46:42.069 INFO:teuthology.task.internal:no buildpackages task found 2024-06-28T14:46:42.069 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-28T14:46:42.105 INFO:teuthology.task.internal:Saving configuration 2024-06-28T14:46:42.122 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-28T14:46:42.156 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-28T14:46:42.183 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-28 14:40:43.375556', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJgjtSprrHYDSA+gChq4ReFrx5AmU7innhVXnJ4Ef2Mn'} 2024-06-28T14:46:42.209 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-28 14:40:43.377420', 'locked_by': 'scheduled_cbodley@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIALIR2hgcQU4tZY93RbBn5hsFKdEcIMu7Y4pZfDCwB2I'} 2024-06-28T14:46:42.209 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-28T14:46:42.215 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-06-28T14:46:42.215 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-06-28T14:46:42.215 INFO:teuthology.run_tasks:Running task console_log... 2024-06-28T14:46:42.306 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2d7f017ac0>, signals=[15]) 2024-06-28T14:46:42.306 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-28T14:46:42.312 INFO:teuthology.task.internal:Opening connections... 2024-06-28T14:46:42.312 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-06-28T14:46:42.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-28T14:46:42.391 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2024-06-28T14:46:42.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-28T14:46:42.467 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-28T14:46:42.473 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-06-28T14:46:42.478 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-06-28T14:46:42.478 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-06-28T14:46:42.523 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-06-28T14:46:42.523 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Ubuntu" 2024-06-28T14:46:42.523 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="22.04" 2024-06-28T14:46:42.523 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-06-28T14:46:42.523 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_CODENAME=jammy 2024-06-28T14:46:42.523 INFO:teuthology.orchestra.run.smithi178.stdout:ID=ubuntu 2024-06-28T14:46:42.523 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE=debian 2024-06-28T14:46:42.523 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.ubuntu.com/" 2024-06-28T14:46:42.523 INFO:teuthology.orchestra.run.smithi178.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-06-28T14:46:42.523 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-06-28T14:46:42.523 INFO:teuthology.orchestra.run.smithi178.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-06-28T14:46:42.524 INFO:teuthology.orchestra.run.smithi178.stdout:UBUNTU_CODENAME=jammy 2024-06-28T14:46:42.524 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-06-28T14:46:42.552 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2024-06-28T14:46:42.557 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2024-06-28T14:46:42.557 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2024-06-28T14:46:42.604 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-06-28T14:46:42.604 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="Ubuntu" 2024-06-28T14:46:42.605 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="22.04" 2024-06-28T14:46:42.605 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-06-28T14:46:42.605 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_CODENAME=jammy 2024-06-28T14:46:42.605 INFO:teuthology.orchestra.run.smithi195.stdout:ID=ubuntu 2024-06-28T14:46:42.605 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE=debian 2024-06-28T14:46:42.605 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://www.ubuntu.com/" 2024-06-28T14:46:42.605 INFO:teuthology.orchestra.run.smithi195.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-06-28T14:46:42.605 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-06-28T14:46:42.605 INFO:teuthology.orchestra.run.smithi195.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-06-28T14:46:42.605 INFO:teuthology.orchestra.run.smithi195.stdout:UBUNTU_CODENAME=jammy 2024-06-28T14:46:42.605 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2024-06-28T14:46:42.627 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-28T14:46:42.636 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-28T14:46:42.679 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-28T14:46:42.680 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-06-28T14:46:42.682 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2024-06-28T14:46:42.687 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-28T14:46:42.729 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-28T14:46:42.730 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-06-28T14:46:42.732 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2024-06-28T14:46:42.798 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-28T14:46:42.857 INFO:teuthology.run_tasks:Running task kernel... 2024-06-28T14:46:42.871 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-06-28T14:46:42.872 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-06-28T14:46:42.872 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-28T14:46:42.872 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-28T14:46:42.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T14:46:42.877 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-06-28T14:46:42.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T14:46:42.878 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-06-28T14:46:42.923 INFO:teuthology.orchestra.run.smithi178.stdout:5.15.0-56-generic 2024-06-28T14:46:42.923 INFO:teuthology.task.kernel:Running kernel on smithi178: 5.15.0-56-generic 2024-06-28T14:46:42.923 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get clean 2024-06-28T14:46:42.925 INFO:teuthology.orchestra.run.smithi195.stdout:5.15.0-56-generic 2024-06-28T14:46:42.925 INFO:teuthology.task.kernel:Running kernel on smithi195: 5.15.0-56-generic 2024-06-28T14:46:42.925 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get clean 2024-06-28T14:46:43.058 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-06-28T14:46:43.106 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get update 2024-06-28T14:46:43.271 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-06-28T14:46:43.316 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-06-28T14:46:43.325 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-06-28T14:46:43.327 INFO:teuthology.orchestra.run.smithi178.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-06-28T14:46:43.358 INFO:teuthology.orchestra.run.smithi178.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-06-28T14:46:43.387 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-06-28T14:46:43.586 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,568 kB] 2024-06-28T14:46:43.612 INFO:teuthology.orchestra.run.smithi178.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-06-28T14:46:43.643 INFO:teuthology.orchestra.run.smithi178.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,568 kB] 2024-06-28T14:46:43.717 INFO:teuthology.orchestra.run.smithi178.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [650 kB] 2024-06-28T14:46:43.747 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-06-28T14:46:43.880 INFO:teuthology.orchestra.run.smithi178.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,776 kB] 2024-06-28T14:46:43.887 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,776 kB] 2024-06-28T14:46:43.939 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [493 kB] 2024-06-28T14:46:43.957 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [266 kB] 2024-06-28T14:46:43.968 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-06-28T14:46:43.969 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.3 kB] 2024-06-28T14:46:43.970 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,010 kB] 2024-06-28T14:46:43.972 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [493 kB] 2024-06-28T14:46:43.992 INFO:teuthology.orchestra.run.smithi178.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [266 kB] 2024-06-28T14:46:44.076 INFO:teuthology.orchestra.run.smithi178.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-06-28T14:46:44.076 INFO:teuthology.orchestra.run.smithi178.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.3 kB] 2024-06-28T14:46:44.078 INFO:teuthology.orchestra.run.smithi178.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,010 kB] 2024-06-28T14:46:44.080 INFO:teuthology.orchestra.run.smithi178.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [323 kB] 2024-06-28T14:46:44.091 INFO:teuthology.orchestra.run.smithi178.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-06-28T14:46:44.092 INFO:teuthology.orchestra.run.smithi178.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,067 kB] 2024-06-28T14:46:44.129 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [342 kB] 2024-06-28T14:46:44.137 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-06-28T14:46:44.137 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [875 kB] 2024-06-28T14:46:44.138 INFO:teuthology.orchestra.run.smithi178.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [342 kB] 2024-06-28T14:46:44.141 INFO:teuthology.orchestra.run.smithi178.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-06-28T14:46:44.142 INFO:teuthology.orchestra.run.smithi178.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [875 kB] 2024-06-28T14:46:44.202 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [650 kB] 2024-06-28T14:46:44.222 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [323 kB] 2024-06-28T14:46:44.236 INFO:teuthology.orchestra.run.smithi178.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [612 kB] 2024-06-28T14:46:44.236 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-06-28T14:46:44.237 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,067 kB] 2024-06-28T14:46:44.240 INFO:teuthology.orchestra.run.smithi178.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.9 kB] 2024-06-28T14:46:44.241 INFO:teuthology.orchestra.run.smithi178.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [352 kB] 2024-06-28T14:46:44.247 INFO:teuthology.orchestra.run.smithi178.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [170 kB] 2024-06-28T14:46:44.248 INFO:teuthology.orchestra.run.smithi178.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-06-28T14:46:44.248 INFO:teuthology.orchestra.run.smithi178.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,092 kB] 2024-06-28T14:46:44.249 INFO:teuthology.orchestra.run.smithi178.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-06-28T14:46:44.249 INFO:teuthology.orchestra.run.smithi178.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-06-28T14:46:44.251 INFO:teuthology.orchestra.run.smithi178.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-06-28T14:46:44.251 INFO:teuthology.orchestra.run.smithi178.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-06-28T14:46:44.251 INFO:teuthology.orchestra.run.smithi178.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-06-28T14:46:44.256 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [612 kB] 2024-06-28T14:46:44.268 INFO:teuthology.orchestra.run.smithi178.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [711 kB] 2024-06-28T14:46:44.274 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [170 kB] 2024-06-28T14:46:44.279 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-06-28T14:46:44.280 INFO:teuthology.orchestra.run.smithi178.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [253 kB] 2024-06-28T14:46:44.281 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-06-28T14:46:44.281 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-06-28T14:46:44.281 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-06-28T14:46:44.281 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-06-28T14:46:44.284 INFO:teuthology.orchestra.run.smithi178.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-06-28T14:46:44.285 INFO:teuthology.orchestra.run.smithi178.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-06-28T14:46:44.286 INFO:teuthology.orchestra.run.smithi178.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,744 B] 2024-06-28T14:46:44.286 INFO:teuthology.orchestra.run.smithi178.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-06-28T14:46:44.298 INFO:teuthology.orchestra.run.smithi178.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-06-28T14:46:44.298 INFO:teuthology.orchestra.run.smithi178.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-06-28T14:46:44.299 INFO:teuthology.orchestra.run.smithi178.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-06-28T14:46:44.303 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.9 kB] 2024-06-28T14:46:44.304 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [352 kB] 2024-06-28T14:46:44.312 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-06-28T14:46:44.312 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [711 kB] 2024-06-28T14:46:44.315 INFO:teuthology.orchestra.run.smithi178.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-06-28T14:46:44.316 INFO:teuthology.orchestra.run.smithi178.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-06-28T14:46:44.316 INFO:teuthology.orchestra.run.smithi178.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.6 kB] 2024-06-28T14:46:44.317 INFO:teuthology.orchestra.run.smithi178.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-06-28T14:46:44.317 INFO:teuthology.orchestra.run.smithi178.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-06-28T14:46:44.325 INFO:teuthology.orchestra.run.smithi178.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-06-28T14:46:44.326 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,092 kB] 2024-06-28T14:46:44.361 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [253 kB] 2024-06-28T14:46:44.370 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-06-28T14:46:44.371 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-06-28T14:46:44.372 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,744 B] 2024-06-28T14:46:44.444 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-06-28T14:46:44.445 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-06-28T14:46:44.445 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-06-28T14:46:44.447 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-06-28T14:46:44.448 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-06-28T14:46:44.452 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-06-28T14:46:44.452 INFO:teuthology.orchestra.run.smithi195.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.6 kB] 2024-06-28T14:46:44.533 INFO:teuthology.orchestra.run.smithi195.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-06-28T14:46:44.534 INFO:teuthology.orchestra.run.smithi195.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-06-28T14:46:44.534 INFO:teuthology.orchestra.run.smithi195.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-06-28T14:46:49.853 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 14.4 MB in 2s (6,090 kB/s) 2024-06-28T14:46:50.002 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 14.4 MB in 2s (6,190 kB/s) 2024-06-28T14:46:51.207 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-06-28T14:46:51.225 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-06-28T14:46:51.288 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-06-28T14:46:51.402 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-06-28T14:46:51.421 DEBUG:teuthology.orchestra.run.smithi178:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-06-28T14:46:51.482 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-06-28T14:46:51.486 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-06-28T14:46:51.487 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-06-28T14:46:51.670 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-06-28T14:46:51.670 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-06-28T14:46:51.670 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-06-28T14:46:51.671 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-06-28T14:46:51.671 INFO:teuthology.orchestra.run.smithi195.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-06-28T14:46:51.671 INFO:teuthology.orchestra.run.smithi195.stdout: python2.7-minimal 2024-06-28T14:46:51.671 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-06-28T14:46:51.672 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2024-06-28T14:46:51.672 INFO:teuthology.orchestra.run.smithi195.stdout: linux-generic linux-headers-5.15.0-113 linux-headers-5.15.0-113-generic 2024-06-28T14:46:51.672 INFO:teuthology.orchestra.run.smithi195.stdout: linux-headers-generic linux-image-5.15.0-113-generic 2024-06-28T14:46:51.673 INFO:teuthology.orchestra.run.smithi195.stdout: linux-modules-5.15.0-113-generic linux-modules-extra-5.15.0-113-generic 2024-06-28T14:46:51.674 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2024-06-28T14:46:51.674 INFO:teuthology.orchestra.run.smithi195.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-06-28T14:46:51.674 INFO:teuthology.orchestra.run.smithi195.stdout:Recommended packages: 2024-06-28T14:46:51.674 INFO:teuthology.orchestra.run.smithi195.stdout: thermald 2024-06-28T14:46:51.687 INFO:teuthology.orchestra.run.smithi178.stdout:Building dependency tree... 2024-06-28T14:46:51.687 INFO:teuthology.orchestra.run.smithi178.stdout:Reading state information... 2024-06-28T14:46:51.721 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2024-06-28T14:46:51.721 INFO:teuthology.orchestra.run.smithi195.stdout: linux-headers-5.15.0-113 linux-headers-5.15.0-113-generic 2024-06-28T14:46:51.722 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.15.0-113-generic linux-modules-5.15.0-113-generic 2024-06-28T14:46:51.722 INFO:teuthology.orchestra.run.smithi195.stdout: linux-modules-extra-5.15.0-113-generic 2024-06-28T14:46:51.723 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages will be upgraded: 2024-06-28T14:46:51.723 INFO:teuthology.orchestra.run.smithi195.stdout: linux-generic linux-headers-generic linux-image-generic 2024-06-28T14:46:51.845 INFO:teuthology.orchestra.run.smithi195.stdout:3 upgraded, 5 newly installed, 0 to remove and 328 not upgraded. 2024-06-28T14:46:51.845 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 113 MB of archives. 2024-06-28T14:46:51.845 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 583 MB of additional disk space will be used. 2024-06-28T14:46:51.845 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-113-generic amd64 5.15.0-113.123 [22.6 MB] 2024-06-28T14:46:51.858 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages were automatically installed and are no longer required: 2024-06-28T14:46:51.858 INFO:teuthology.orchestra.run.smithi178.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-06-28T14:46:51.858 INFO:teuthology.orchestra.run.smithi178.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-06-28T14:46:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-06-28T14:46:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-06-28T14:46:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: python2.7-minimal 2024-06-28T14:46:51.859 INFO:teuthology.orchestra.run.smithi178.stdout:Use 'sudo apt autoremove' to remove them. 2024-06-28T14:46:51.860 INFO:teuthology.orchestra.run.smithi178.stdout:The following additional packages will be installed: 2024-06-28T14:46:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: linux-generic linux-headers-5.15.0-113 linux-headers-5.15.0-113-generic 2024-06-28T14:46:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: linux-headers-generic linux-image-5.15.0-113-generic 2024-06-28T14:46:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: linux-modules-5.15.0-113-generic linux-modules-extra-5.15.0-113-generic 2024-06-28T14:46:51.862 INFO:teuthology.orchestra.run.smithi178.stdout:Suggested packages: 2024-06-28T14:46:51.862 INFO:teuthology.orchestra.run.smithi178.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-06-28T14:46:51.862 INFO:teuthology.orchestra.run.smithi178.stdout:Recommended packages: 2024-06-28T14:46:51.862 INFO:teuthology.orchestra.run.smithi178.stdout: thermald 2024-06-28T14:46:51.913 INFO:teuthology.orchestra.run.smithi178.stdout:The following NEW packages will be installed: 2024-06-28T14:46:51.913 INFO:teuthology.orchestra.run.smithi178.stdout: linux-headers-5.15.0-113 linux-headers-5.15.0-113-generic 2024-06-28T14:46:51.913 INFO:teuthology.orchestra.run.smithi178.stdout: linux-image-5.15.0-113-generic linux-modules-5.15.0-113-generic 2024-06-28T14:46:51.914 INFO:teuthology.orchestra.run.smithi178.stdout: linux-modules-extra-5.15.0-113-generic 2024-06-28T14:46:51.914 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages will be upgraded: 2024-06-28T14:46:51.915 INFO:teuthology.orchestra.run.smithi178.stdout: linux-generic linux-headers-generic linux-image-generic 2024-06-28T14:46:52.038 INFO:teuthology.orchestra.run.smithi178.stdout:3 upgraded, 5 newly installed, 0 to remove and 328 not upgraded. 2024-06-28T14:46:52.039 INFO:teuthology.orchestra.run.smithi178.stdout:Need to get 113 MB of archives. 2024-06-28T14:46:52.039 INFO:teuthology.orchestra.run.smithi178.stdout:After this operation, 583 MB of additional disk space will be used. 2024-06-28T14:46:52.039 INFO:teuthology.orchestra.run.smithi178.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-113-generic amd64 5.15.0-113.123 [22.6 MB] 2024-06-28T14:46:53.086 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-113-generic amd64 5.15.0-113.123 [11.5 MB] 2024-06-28T14:46:53.351 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-113-generic amd64 5.15.0-113.123 [63.9 MB] 2024-06-28T14:46:53.628 INFO:teuthology.orchestra.run.smithi178.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-113-generic amd64 5.15.0-113.123 [11.5 MB] 2024-06-28T14:46:54.018 INFO:teuthology.orchestra.run.smithi178.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-113-generic amd64 5.15.0-113.123 [63.9 MB] 2024-06-28T14:46:55.399 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.113.113 [1,692 B] 2024-06-28T14:46:55.399 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.113.113 [2,488 B] 2024-06-28T14:46:55.400 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-113 all 5.15.0-113.123 [12.3 MB] 2024-06-28T14:46:55.760 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-113-generic amd64 5.15.0-113.123 [2,895 kB] 2024-06-28T14:46:55.816 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.113.113 [2,346 B] 2024-06-28T14:46:55.994 INFO:teuthology.orchestra.run.smithi178.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.113.113 [1,692 B] 2024-06-28T14:46:55.994 INFO:teuthology.orchestra.run.smithi178.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.113.113 [2,488 B] 2024-06-28T14:46:55.995 INFO:teuthology.orchestra.run.smithi178.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-113 all 5.15.0-113.123 [12.3 MB] 2024-06-28T14:46:56.213 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 113 MB in 4s (27.7 MB/s) 2024-06-28T14:46:56.284 INFO:teuthology.orchestra.run.smithi178.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-113-generic amd64 5.15.0-113.123 [2,895 kB] 2024-06-28T14:46:56.327 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.113.113 [2,346 B] 2024-06-28T14:46:56.396 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-modules-5.15.0-113-generic. 2024-06-28T14:46:56.715 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 113 MB in 4s (25.7 MB/s) 2024-06-28T14:46:56.874 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-modules-5.15.0-113-generic. 2024-06-28T14:46:58.429 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-06-28T14:46:58.433 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../0-linux-modules-5.15.0-113-generic_5.15.0-113.123_amd64.deb ... 2024-06-28T14:46:58.502 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-modules-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:46:58.896 INFO:teuthology.orchestra.run.smithi178.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-06-28T14:46:58.901 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../0-linux-modules-5.15.0-113-generic_5.15.0-113.123_amd64.deb ... 2024-06-28T14:46:58.979 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-modules-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:00.970 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-image-5.15.0-113-generic. 2024-06-28T14:47:00.991 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../1-linux-image-5.15.0-113-generic_5.15.0-113.123_amd64.deb ... 2024-06-28T14:47:01.081 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-image-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:01.523 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-113-generic. 2024-06-28T14:47:01.544 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-113-generic_5.15.0-113.123_amd64.deb ... 2024-06-28T14:47:01.581 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-modules-extra-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:01.647 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-image-5.15.0-113-generic. 2024-06-28T14:47:01.668 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../1-linux-image-5.15.0-113-generic_5.15.0-113.123_amd64.deb ... 2024-06-28T14:47:01.749 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-image-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:02.225 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-113-generic. 2024-06-28T14:47:02.245 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-113-generic_5.15.0-113.123_amd64.deb ... 2024-06-28T14:47:02.283 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-modules-extra-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:07.673 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../3-linux-generic_5.15.0.113.113_amd64.deb ... 2024-06-28T14:47:07.775 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-generic (5.15.0.113.113) over (5.15.0.56.54) ... 2024-06-28T14:47:08.084 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.113.113_amd64.deb ... 2024-06-28T14:47:08.212 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-image-generic (5.15.0.113.113) over (5.15.0.56.54) ... 2024-06-28T14:47:08.515 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-headers-5.15.0-113. 2024-06-28T14:47:08.538 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../5-linux-headers-5.15.0-113_5.15.0-113.123_all.deb ... 2024-06-28T14:47:08.573 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../3-linux-generic_5.15.0.113.113_amd64.deb ... 2024-06-28T14:47:08.581 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-5.15.0-113 (5.15.0-113.123) ... 2024-06-28T14:47:08.700 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-generic (5.15.0.113.113) over (5.15.0.56.54) ... 2024-06-28T14:47:09.058 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.113.113_amd64.deb ... 2024-06-28T14:47:09.186 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-image-generic (5.15.0.113.113) over (5.15.0.56.54) ... 2024-06-28T14:47:09.489 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-headers-5.15.0-113. 2024-06-28T14:47:09.512 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../5-linux-headers-5.15.0-113_5.15.0-113.123_all.deb ... 2024-06-28T14:47:09.547 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-headers-5.15.0-113 (5.15.0-113.123) ... 2024-06-28T14:47:14.026 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-headers-5.15.0-113-generic. 2024-06-28T14:47:14.064 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../6-linux-headers-5.15.0-113-generic_5.15.0-113.123_amd64.deb ... 2024-06-28T14:47:14.101 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:15.008 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-headers-5.15.0-113-generic. 2024-06-28T14:47:15.045 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../6-linux-headers-5.15.0-113-generic_5.15.0-113.123_amd64.deb ... 2024-06-28T14:47:15.083 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-headers-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:17.253 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.113.113_amd64.deb ... 2024-06-28T14:47:17.379 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-generic (5.15.0.113.113) over (5.15.0.56.54) ... 2024-06-28T14:47:17.701 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.113.113_amd64.deb ... 2024-06-28T14:47:17.720 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-5.15.0-113 (5.15.0-113.123) ... 2024-06-28T14:47:17.822 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-headers-generic (5.15.0.113.113) over (5.15.0.56.54) ... 2024-06-28T14:47:17.834 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:17.935 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-generic (5.15.0.113.113) ... 2024-06-28T14:47:18.037 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-modules-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:18.156 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-headers-5.15.0-113 (5.15.0-113.123) ... 2024-06-28T14:47:18.277 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-headers-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:18.404 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-headers-generic (5.15.0.113.113) ... 2024-06-28T14:47:18.515 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-modules-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:19.541 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-image-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:20.645 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-image-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:20.928 INFO:teuthology.orchestra.run.smithi195.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-06-28T14:47:20.928 INFO:teuthology.orchestra.run.smithi195.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-06-28T14:47:20.928 INFO:teuthology.orchestra.run.smithi195.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-113-generic 2024-06-28T14:47:20.928 INFO:teuthology.orchestra.run.smithi195.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-113-generic 2024-06-28T14:47:21.060 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-modules-extra-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:22.025 INFO:teuthology.orchestra.run.smithi178.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-06-28T14:47:22.025 INFO:teuthology.orchestra.run.smithi178.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-06-28T14:47:22.025 INFO:teuthology.orchestra.run.smithi178.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-113-generic 2024-06-28T14:47:22.025 INFO:teuthology.orchestra.run.smithi178.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-113-generic 2024-06-28T14:47:22.172 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-modules-extra-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:22.443 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-image-generic (5.15.0.113.113) ... 2024-06-28T14:47:22.544 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-generic (5.15.0.113.113) ... 2024-06-28T14:47:22.654 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for linux-image-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:22.714 INFO:teuthology.orchestra.run.smithi195.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-06-28T14:47:22.714 INFO:teuthology.orchestra.run.smithi195.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-113-generic 2024-06-28T14:47:23.530 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-image-generic (5.15.0.113.113) ... 2024-06-28T14:47:23.667 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-generic (5.15.0.113.113) ... 2024-06-28T14:47:23.791 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for linux-image-5.15.0-113-generic (5.15.0-113.123) ... 2024-06-28T14:47:23.856 INFO:teuthology.orchestra.run.smithi178.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-06-28T14:47:23.856 INFO:teuthology.orchestra.run.smithi178.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-113-generic 2024-06-28T14:47:36.467 INFO:teuthology.orchestra.run.smithi195.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-06-28T14:47:36.468 INFO:teuthology.orchestra.run.smithi195.stdout:Sourcing file `/etc/default/grub' 2024-06-28T14:47:36.476 INFO:teuthology.orchestra.run.smithi195.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-06-28T14:47:36.500 INFO:teuthology.orchestra.run.smithi195.stdout:Generating grub configuration file ... 2024-06-28T14:47:36.787 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.15.0-113-generic 2024-06-28T14:47:36.809 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.15.0-113-generic 2024-06-28T14:47:37.110 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-06-28T14:47:37.116 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-06-28T14:47:37.180 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-06-28T14:47:37.185 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-06-28T14:47:37.352 INFO:teuthology.orchestra.run.smithi195.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-06-28T14:47:37.352 INFO:teuthology.orchestra.run.smithi195.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-06-28T14:47:37.353 INFO:teuthology.orchestra.run.smithi195.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-06-28T14:47:37.390 INFO:teuthology.orchestra.run.smithi195.stdout:done 2024-06-28T14:47:37.775 INFO:teuthology.orchestra.run.smithi178.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-06-28T14:47:37.775 INFO:teuthology.orchestra.run.smithi178.stdout:Sourcing file `/etc/default/grub' 2024-06-28T14:47:37.782 INFO:teuthology.orchestra.run.smithi178.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-06-28T14:47:37.796 INFO:teuthology.orchestra.run.smithi178.stdout:Generating grub configuration file ... 2024-06-28T14:47:37.840 DEBUG:teuthology.orchestra.run.smithi195:> dpkg -s linux-image-generic 2024-06-28T14:47:37.866 INFO:teuthology.orchestra.run.smithi195.stdout:Package: linux-image-generic 2024-06-28T14:47:37.867 INFO:teuthology.orchestra.run.smithi195.stdout:Status: install ok installed 2024-06-28T14:47:37.867 INFO:teuthology.orchestra.run.smithi195.stdout:Priority: optional 2024-06-28T14:47:37.867 INFO:teuthology.orchestra.run.smithi195.stdout:Section: kernel 2024-06-28T14:47:37.867 INFO:teuthology.orchestra.run.smithi195.stdout:Installed-Size: 21 2024-06-28T14:47:37.867 INFO:teuthology.orchestra.run.smithi195.stdout:Maintainer: Ubuntu Kernel Team 2024-06-28T14:47:37.867 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: amd64 2024-06-28T14:47:37.867 INFO:teuthology.orchestra.run.smithi195.stdout:Source: linux-meta 2024-06-28T14:47:37.867 INFO:teuthology.orchestra.run.smithi195.stdout:Version: 5.15.0.113.113 2024-06-28T14:47:37.867 INFO:teuthology.orchestra.run.smithi195.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-113), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-06-28T14:47:37.867 INFO:teuthology.orchestra.run.smithi195.stdout:Depends: linux-image-5.15.0-113-generic, linux-modules-extra-5.15.0-113-generic, linux-firmware, intel-microcode, amd64-microcode 2024-06-28T14:47:37.867 INFO:teuthology.orchestra.run.smithi195.stdout:Recommends: thermald 2024-06-28T14:47:37.867 INFO:teuthology.orchestra.run.smithi195.stdout:Description: Generic Linux kernel image 2024-06-28T14:47:37.867 INFO:teuthology.orchestra.run.smithi195.stdout: This package will always depend on the latest generic kernel image 2024-06-28T14:47:37.868 INFO:teuthology.orchestra.run.smithi195.stdout: available. 2024-06-28T14:47:37.868 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-113-generic 2024-06-28T14:47:37.868 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-28T14:47:37.868 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-28T14:47:37.868 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-06-28T14:47:37.868 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi195.front.sepia.ceph.com, path=None, version=distro) 2024-06-28T14:47:37.868 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get clean 2024-06-28T14:47:38.042 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-06-28T14:47:38.045 INFO:teuthology.orchestra.run.smithi178.stdout:Found linux image: /boot/vmlinuz-5.15.0-113-generic 2024-06-28T14:47:38.064 INFO:teuthology.orchestra.run.smithi178.stdout:Found initrd image: /boot/initrd.img-5.15.0-113-generic 2024-06-28T14:47:38.227 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-06-28T14:47:38.263 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-06-28T14:47:38.274 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-06-28T14:47:38.320 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-06-28T14:47:38.362 INFO:teuthology.orchestra.run.smithi178.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-06-28T14:47:38.368 INFO:teuthology.orchestra.run.smithi178.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-06-28T14:47:38.436 INFO:teuthology.orchestra.run.smithi178.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-06-28T14:47:38.441 INFO:teuthology.orchestra.run.smithi178.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-06-28T14:47:38.591 INFO:teuthology.orchestra.run.smithi178.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-06-28T14:47:38.591 INFO:teuthology.orchestra.run.smithi178.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-06-28T14:47:38.591 INFO:teuthology.orchestra.run.smithi178.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-06-28T14:47:38.637 INFO:teuthology.orchestra.run.smithi178.stdout:done 2024-06-28T14:47:39.123 DEBUG:teuthology.orchestra.run.smithi178:> dpkg -s linux-image-generic 2024-06-28T14:47:39.146 INFO:teuthology.orchestra.run.smithi178.stdout:Package: linux-image-generic 2024-06-28T14:47:39.146 INFO:teuthology.orchestra.run.smithi178.stdout:Status: install ok installed 2024-06-28T14:47:39.146 INFO:teuthology.orchestra.run.smithi178.stdout:Priority: optional 2024-06-28T14:47:39.146 INFO:teuthology.orchestra.run.smithi178.stdout:Section: kernel 2024-06-28T14:47:39.146 INFO:teuthology.orchestra.run.smithi178.stdout:Installed-Size: 21 2024-06-28T14:47:39.146 INFO:teuthology.orchestra.run.smithi178.stdout:Maintainer: Ubuntu Kernel Team 2024-06-28T14:47:39.146 INFO:teuthology.orchestra.run.smithi178.stdout:Architecture: amd64 2024-06-28T14:47:39.146 INFO:teuthology.orchestra.run.smithi178.stdout:Source: linux-meta 2024-06-28T14:47:39.146 INFO:teuthology.orchestra.run.smithi178.stdout:Version: 5.15.0.113.113 2024-06-28T14:47:39.147 INFO:teuthology.orchestra.run.smithi178.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-113), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-06-28T14:47:39.147 INFO:teuthology.orchestra.run.smithi178.stdout:Depends: linux-image-5.15.0-113-generic, linux-modules-extra-5.15.0-113-generic, linux-firmware, intel-microcode, amd64-microcode 2024-06-28T14:47:39.147 INFO:teuthology.orchestra.run.smithi178.stdout:Recommends: thermald 2024-06-28T14:47:39.147 INFO:teuthology.orchestra.run.smithi178.stdout:Description: Generic Linux kernel image 2024-06-28T14:47:39.147 INFO:teuthology.orchestra.run.smithi178.stdout: This package will always depend on the latest generic kernel image 2024-06-28T14:47:39.147 INFO:teuthology.orchestra.run.smithi178.stdout: available. 2024-06-28T14:47:39.147 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-113-generic 2024-06-28T14:47:39.147 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-28T14:47:39.147 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-28T14:47:39.147 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-06-28T14:47:39.148 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi178.front.sepia.ceph.com, path=None, version=distro) 2024-06-28T14:47:39.148 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get clean 2024-06-28T14:47:39.330 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get update 2024-06-28T14:47:39.590 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-06-28T14:47:39.593 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-06-28T14:47:39.593 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-06-28T14:47:39.608 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-06-28T14:47:39.654 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-06-28T14:47:39.668 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-06-28T14:47:39.736 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-06-28T14:47:39.874 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-06-28T14:47:39.875 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-06-28T14:47:40.047 INFO:teuthology.orchestra.run.smithi195.stdout:linux-image-generic is already the newest version (5.15.0.113.113). 2024-06-28T14:47:40.047 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-06-28T14:47:40.047 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-06-28T14:47:40.047 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-06-28T14:47:40.047 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-06-28T14:47:40.048 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-06-28T14:47:40.048 INFO:teuthology.orchestra.run.smithi195.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-06-28T14:47:40.048 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-06-28T14:47:40.089 INFO:teuthology.orchestra.run.smithi195.stdout:0 upgraded, 0 newly installed, 0 to remove and 328 not upgraded. 2024-06-28T14:47:40.092 DEBUG:teuthology.orchestra.run.smithi195:> dpkg -s linux-image-generic 2024-06-28T14:47:40.107 INFO:teuthology.orchestra.run.smithi195.stdout:Package: linux-image-generic 2024-06-28T14:47:40.107 INFO:teuthology.orchestra.run.smithi195.stdout:Status: install ok installed 2024-06-28T14:47:40.107 INFO:teuthology.orchestra.run.smithi195.stdout:Priority: optional 2024-06-28T14:47:40.107 INFO:teuthology.orchestra.run.smithi195.stdout:Section: kernel 2024-06-28T14:47:40.108 INFO:teuthology.orchestra.run.smithi195.stdout:Installed-Size: 21 2024-06-28T14:47:40.108 INFO:teuthology.orchestra.run.smithi195.stdout:Maintainer: Ubuntu Kernel Team 2024-06-28T14:47:40.108 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: amd64 2024-06-28T14:47:40.108 INFO:teuthology.orchestra.run.smithi195.stdout:Source: linux-meta 2024-06-28T14:47:40.108 INFO:teuthology.orchestra.run.smithi195.stdout:Version: 5.15.0.113.113 2024-06-28T14:47:40.108 INFO:teuthology.orchestra.run.smithi195.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-113), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-06-28T14:47:40.108 INFO:teuthology.orchestra.run.smithi195.stdout:Depends: linux-image-5.15.0-113-generic, linux-modules-extra-5.15.0-113-generic, linux-firmware, intel-microcode, amd64-microcode 2024-06-28T14:47:40.108 INFO:teuthology.orchestra.run.smithi195.stdout:Recommends: thermald 2024-06-28T14:47:40.108 INFO:teuthology.orchestra.run.smithi195.stdout:Description: Generic Linux kernel image 2024-06-28T14:47:40.108 INFO:teuthology.orchestra.run.smithi195.stdout: This package will always depend on the latest generic kernel image 2024-06-28T14:47:40.108 INFO:teuthology.orchestra.run.smithi195.stdout: available. 2024-06-28T14:47:40.109 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-06-28T14:47:40.152 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.Bv7cGbFxti 2024-06-28T14:47:40.152 DEBUG:teuthology.orchestra.run.smithi195:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Bv7cGbFxti 2024-06-28T14:47:40.209 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.Bv7cGbFxti 2024-06-28T14:47:40.329 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.Bv7cGbFxti is 10KB 2024-06-28T14:47:40.342 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.Bv7cGbFxti 2024-06-28T14:47:40.347 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-06-28T14:47:40.399 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:47:40.400 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-06-28T14:47:40.400 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-06-28T14:47:40.467 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-113-generic 2024-06-28T14:47:40.467 DEBUG:teuthology.orchestra.run.smithi195:> sudo update-grub 2024-06-28T14:47:41.031 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-06-28T14:47:41.049 DEBUG:teuthology.orchestra.run.smithi178:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-06-28T14:47:41.101 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-06-28T14:47:41.283 INFO:teuthology.orchestra.run.smithi178.stdout:Building dependency tree... 2024-06-28T14:47:41.283 INFO:teuthology.orchestra.run.smithi178.stdout:Reading state information... 2024-06-28T14:47:41.450 INFO:teuthology.orchestra.run.smithi178.stdout:linux-image-generic is already the newest version (5.15.0.113.113). 2024-06-28T14:47:41.451 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages were automatically installed and are no longer required: 2024-06-28T14:47:41.451 INFO:teuthology.orchestra.run.smithi178.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-06-28T14:47:41.451 INFO:teuthology.orchestra.run.smithi178.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-06-28T14:47:41.451 INFO:teuthology.orchestra.run.smithi178.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-06-28T14:47:41.451 INFO:teuthology.orchestra.run.smithi178.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-06-28T14:47:41.451 INFO:teuthology.orchestra.run.smithi178.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-06-28T14:47:41.452 INFO:teuthology.orchestra.run.smithi178.stdout:Use 'sudo apt autoremove' to remove them. 2024-06-28T14:47:41.490 INFO:teuthology.orchestra.run.smithi178.stdout:0 upgraded, 0 newly installed, 0 to remove and 328 not upgraded. 2024-06-28T14:47:41.492 DEBUG:teuthology.orchestra.run.smithi178:> dpkg -s linux-image-generic 2024-06-28T14:47:41.549 INFO:teuthology.orchestra.run.smithi178.stdout:Package: linux-image-generic 2024-06-28T14:47:41.550 INFO:teuthology.orchestra.run.smithi178.stdout:Status: install ok installed 2024-06-28T14:47:41.550 INFO:teuthology.orchestra.run.smithi178.stdout:Priority: optional 2024-06-28T14:47:41.550 INFO:teuthology.orchestra.run.smithi178.stdout:Section: kernel 2024-06-28T14:47:41.550 INFO:teuthology.orchestra.run.smithi178.stdout:Installed-Size: 21 2024-06-28T14:47:41.550 INFO:teuthology.orchestra.run.smithi178.stdout:Maintainer: Ubuntu Kernel Team 2024-06-28T14:47:41.550 INFO:teuthology.orchestra.run.smithi178.stdout:Architecture: amd64 2024-06-28T14:47:41.550 INFO:teuthology.orchestra.run.smithi178.stdout:Source: linux-meta 2024-06-28T14:47:41.550 INFO:teuthology.orchestra.run.smithi178.stdout:Version: 5.15.0.113.113 2024-06-28T14:47:41.550 INFO:teuthology.orchestra.run.smithi178.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-113), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-06-28T14:47:41.550 INFO:teuthology.orchestra.run.smithi178.stdout:Depends: linux-image-5.15.0-113-generic, linux-modules-extra-5.15.0-113-generic, linux-firmware, intel-microcode, amd64-microcode 2024-06-28T14:47:41.550 INFO:teuthology.orchestra.run.smithi178.stdout:Recommends: thermald 2024-06-28T14:47:41.550 INFO:teuthology.orchestra.run.smithi178.stdout:Description: Generic Linux kernel image 2024-06-28T14:47:41.551 INFO:teuthology.orchestra.run.smithi178.stdout: This package will always depend on the latest generic kernel image 2024-06-28T14:47:41.551 INFO:teuthology.orchestra.run.smithi178.stdout: available. 2024-06-28T14:47:41.551 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-06-28T14:47:41.594 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.xBp4RP0pub 2024-06-28T14:47:41.594 DEBUG:teuthology.orchestra.run.smithi178:> sudo cp /boot/grub/grub.cfg /tmp/tmp.xBp4RP0pub 2024-06-28T14:47:41.650 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.xBp4RP0pub 2024-06-28T14:47:41.667 INFO:teuthology.orchestra.run.smithi195.stderr:Sourcing file `/etc/default/grub' 2024-06-28T14:47:41.668 INFO:teuthology.orchestra.run.smithi195.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-06-28T14:47:41.671 INFO:teuthology.orchestra.run.smithi195.stderr:Generating grub configuration file ... 2024-06-28T14:47:41.763 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.xBp4RP0pub is 10KB 2024-06-28T14:47:41.776 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.xBp4RP0pub 2024-06-28T14:47:41.782 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-06-28T14:47:41.833 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:47:41.833 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-06-28T14:47:41.834 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-06-28T14:47:41.857 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.15.0-113-generic 2024-06-28T14:47:41.863 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.15.0-113-generic 2024-06-28T14:47:41.904 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-113-generic 2024-06-28T14:47:41.904 DEBUG:teuthology.orchestra.run.smithi178:> sudo update-grub 2024-06-28T14:47:42.150 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-06-28T14:47:42.156 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-06-28T14:47:42.197 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-06-28T14:47:42.202 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-06-28T14:47:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-06-28T14:47:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-06-28T14:47:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-06-28T14:47:42.366 INFO:teuthology.orchestra.run.smithi195.stderr:done 2024-06-28T14:47:42.368 DEBUG:teuthology.orchestra.run.smithi195:> sudo shutdown -r now 2024-06-28T14:47:43.171 INFO:teuthology.orchestra.run.smithi178.stderr:Sourcing file `/etc/default/grub' 2024-06-28T14:47:43.173 INFO:teuthology.orchestra.run.smithi178.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-06-28T14:47:43.175 INFO:teuthology.orchestra.run.smithi178.stderr:Generating grub configuration file ... 2024-06-28T14:47:43.348 INFO:teuthology.orchestra.run.smithi178.stderr:Found linux image: /boot/vmlinuz-5.15.0-113-generic 2024-06-28T14:47:43.354 INFO:teuthology.orchestra.run.smithi178.stderr:Found initrd image: /boot/initrd.img-5.15.0-113-generic 2024-06-28T14:47:43.584 INFO:teuthology.orchestra.run.smithi178.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-06-28T14:47:43.588 INFO:teuthology.orchestra.run.smithi178.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-06-28T14:47:43.630 INFO:teuthology.orchestra.run.smithi178.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-06-28T14:47:43.635 INFO:teuthology.orchestra.run.smithi178.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-06-28T14:47:43.755 INFO:teuthology.orchestra.run.smithi178.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-06-28T14:47:43.755 INFO:teuthology.orchestra.run.smithi178.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-06-28T14:47:43.755 INFO:teuthology.orchestra.run.smithi178.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-06-28T14:47:43.766 INFO:teuthology.orchestra.run.smithi178.stderr:done 2024-06-28T14:47:43.782 DEBUG:teuthology.orchestra.run.smithi178:> sudo shutdown -r now 2024-06-28T14:48:12.398 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-06-28T14:48:12.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-06-28T14:48:12.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-28T14:48:13.786 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-06-28T14:48:13.787 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-06-28T14:48:13.787 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-28T14:48:30.754 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.195 2024-06-28T14:48:32.290 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-06-28T14:48:39.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-06-28T14:48:39.764 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-28T14:48:41.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-06-28T14:48:41.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-28T14:48:44.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-06-28T14:48:56.362 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-06-28T14:48:56.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-28T14:48:59.426 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-06-28T14:49:14.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-06-28T14:49:14.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-28T14:49:39.790 DEBUG:teuthology.orchestra.remote:timed out 2024-06-28T14:49:46.434 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-06-28T14:49:46.971 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-06-28T14:49:46.971 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-113-generic"... 2024-06-28T14:49:46.972 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-06-28T14:49:47.019 INFO:teuthology.orchestra.run.smithi178.stdout:5.15.0-113-generic 2024-06-28T14:49:47.020 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-113-generic vs 5.15.0-113-generic 2024-06-28T14:49:47.020 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-28T14:49:47.020 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-06-28T14:49:48.020 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-06-28T14:49:48.021 DEBUG:teuthology.orchestra.run.smithi178:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-28T14:49:48.333 INFO:teuthology.orchestra.run.smithi178.stdout:ttyS1 2024-06-28T14:49:48.361 DEBUG:teuthology.parallel:result is None 2024-06-28T14:49:51.791 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-06-28T14:49:51.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-28T14:49:52.080 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-06-28T14:49:52.654 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-06-28T14:49:52.655 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-113-generic"... 2024-06-28T14:49:52.655 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-06-28T14:49:52.703 INFO:teuthology.orchestra.run.smithi195.stdout:5.15.0-113-generic 2024-06-28T14:49:52.703 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-113-generic vs 5.15.0-113-generic 2024-06-28T14:49:52.703 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-28T14:49:52.703 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-06-28T14:49:53.704 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-06-28T14:49:53.704 DEBUG:teuthology.orchestra.run.smithi195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-28T14:49:53.789 INFO:teuthology.orchestra.run.smithi195.stdout:ttyS1 2024-06-28T14:49:53.816 DEBUG:teuthology.parallel:result is None 2024-06-28T14:49:53.817 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-28T14:49:53.824 INFO:teuthology.task.internal:Creating test directory... 2024-06-28T14:49:53.825 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-28T14:49:53.827 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-28T14:49:53.833 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-28T14:49:53.911 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-28T14:49:53.986 INFO:teuthology.task.internal:Creating archive directory... 2024-06-28T14:49:53.986 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-28T14:49:53.988 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-28T14:49:54.010 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-28T14:49:54.144 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-28T14:49:54.144 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-28T14:49:54.146 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-28T14:49:54.166 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-28T14:49:54.175 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-28T14:49:54.175 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-28T14:49:54.183 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-28T14:49:54.185 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-28T14:49:54.220 INFO:teuthology.task.internal:Configuring sudo... 2024-06-28T14:49:54.221 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-28T14:49:54.225 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-28T14:49:54.243 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-28T14:49:54.297 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-28T14:49:54.297 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-28T14:49:54.300 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-28T14:49:54.306 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-28T14:49:54.352 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-28T14:49:54.400 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:49:54.400 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-28T14:49:54.464 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-28T14:49:54.469 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-28T14:49:54.515 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:49:54.515 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-28T14:49:54.596 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-06-28T14:49:54.599 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2024-06-28T14:49:54.686 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-28T14:49:54.695 INFO:teuthology.task.internal:Starting timer... 2024-06-28T14:49:54.695 INFO:teuthology.run_tasks:Running task pcp... 2024-06-28T14:49:54.732 INFO:teuthology.run_tasks:Running task selinux... 2024-06-28T14:49:54.816 DEBUG:teuthology.task.selinux:Excluding smithi178: OS 'ubuntu' does not support SELinux 2024-06-28T14:49:54.816 DEBUG:teuthology.task.selinux:Excluding smithi195: OS 'ubuntu' does not support SELinux 2024-06-28T14:49:54.817 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-28T14:49:54.817 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-06-28T14:49:54.817 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-28T14:49:54.817 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-28T14:49:54.905 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-28T14:49:54.934 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-28T14:49:54.936 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi178.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-28T14:56:56.993 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi178.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2024-06-28T14:56:56.995 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-06-28T14:56:56.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-28T14:56:57.078 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-06-28T14:56:57.158 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-06-28T14:56:57.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-06-28T14:56:57.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-28T14:56:57.236 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-06-28T14:56:57.314 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-06-28T14:56:57.315 INFO:teuthology.run_tasks:Running task clock... 2024-06-28T14:56:57.327 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-28T14:56:57.327 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-28T14:56:57.328 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-28T14:56:57.330 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-28T14:56:57.330 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-28T14:56:57.356 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-06-28T14:56:57.356 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: Command line: ntpd -gq 2024-06-28T14:56:57.356 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: ---------------------------------------------------- 2024-06-28T14:56:57.357 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: ntp-4 is maintained by Network Time Foundation, 2024-06-28T14:56:57.357 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-06-28T14:56:57.357 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: corporation. Support and training for ntp-4 are 2024-06-28T14:56:57.357 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: available at https://www.nwtime.org/support 2024-06-28T14:56:57.357 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: ---------------------------------------------------- 2024-06-28T14:56:57.358 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: proto: precision = 0.045 usec (-24) 2024-06-28T14:56:57.358 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: basedate set to 2022-02-04 2024-06-28T14:56:57.358 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: gps base set to 2022-02-06 (week 2196) 2024-06-28T14:56:57.358 INFO:teuthology.orchestra.run.smithi178.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-06-28T14:56:57.359 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-06-28T14:56:57.360 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: restrict ::: KOD does nothing without LIMITED. 2024-06-28T14:56:57.360 INFO:teuthology.orchestra.run.smithi178.stderr:restrict ::: KOD does nothing without LIMITED. 2024-06-28T14:56:57.361 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: Listen and drop on 0 v6wildcard [::]:123 2024-06-28T14:56:57.361 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-06-28T14:56:57.361 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: Listen normally on 2 lo 127.0.0.1:123 2024-06-28T14:56:57.362 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: Listen normally on 3 enp3s0f1 172.21.15.178:123 2024-06-28T14:56:57.362 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: Listen normally on 4 lo [::1]:123 2024-06-28T14:56:57.362 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7ef9%5]:123 2024-06-28T14:56:57.362 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:56:57 ntpd[17575]: Listening on routing socket on fd #22 for interface updates 2024-06-28T14:56:57.383 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-06-28T14:56:57.383 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: Command line: ntpd -gq 2024-06-28T14:56:57.383 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: ---------------------------------------------------- 2024-06-28T14:56:57.383 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: ntp-4 is maintained by Network Time Foundation, 2024-06-28T14:56:57.383 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-06-28T14:56:57.383 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: corporation. Support and training for ntp-4 are 2024-06-28T14:56:57.383 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: available at https://www.nwtime.org/support 2024-06-28T14:56:57.383 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: ---------------------------------------------------- 2024-06-28T14:56:57.384 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: proto: precision = 0.061 usec (-24) 2024-06-28T14:56:57.384 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: basedate set to 2022-02-04 2024-06-28T14:56:57.384 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: gps base set to 2022-02-06 (week 2196) 2024-06-28T14:56:57.384 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-06-28T14:56:57.385 INFO:teuthology.orchestra.run.smithi195.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-06-28T14:56:57.385 INFO:teuthology.orchestra.run.smithi195.stderr:restrict ::: KOD does nothing without LIMITED. 2024-06-28T14:56:57.386 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: restrict ::: KOD does nothing without LIMITED. 2024-06-28T14:56:57.386 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: Listen and drop on 0 v6wildcard [::]:123 2024-06-28T14:56:57.386 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-06-28T14:56:57.386 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: Listen normally on 2 lo 127.0.0.1:123 2024-06-28T14:56:57.386 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: Listen normally on 3 enp3s0f1 172.21.15.195:123 2024-06-28T14:56:57.386 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: Listen normally on 4 lo [::1]:123 2024-06-28T14:56:57.386 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8081%5]:123 2024-06-28T14:56:57.386 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:56:57 ntpd[17596]: Listening on routing socket on fd #22 for interface updates 2024-06-28T14:56:58.356 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:56:58 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:56:58.356 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:56:58 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:56:58.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:56:58 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:56:58.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:56:58 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:56:58.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:56:58 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:56:58.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:56:58 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:56:58.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:56:58 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:56:58.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:56:58 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:56:58.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:56:58 ntpd[17596]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:56:58.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:56:58 ntpd[17596]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:56:58.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:56:58 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:56:58.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:56:58 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:56:58.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:56:58 ntpd[17596]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:56:58.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:56:58 ntpd[17596]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:56:59.356 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:56:59 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:56:59.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:56:59 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:56:59.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:56:59 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:56:59.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:56:59 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:56:59.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:56:59 ntpd[17575]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:56:59.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:56:59 ntpd[17575]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:56:59.383 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:56:59 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:56:59.383 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:56:59 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:00.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:00 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:00.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:00 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:00.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:00 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:00.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:00 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:00.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:00 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:00.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:00 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:00.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:00 ntpd[17575]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:00.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:00 ntpd[17575]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:00.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:00 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:00.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:00 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:00.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:00 ntpd[17575]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:00.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:00 ntpd[17575]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:00.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:00 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:00.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:00 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:00.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:00 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:00.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:00 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:00.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:00 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:00.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:00 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:00.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:00 ntpd[17596]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:00.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:00 ntpd[17596]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:00.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:00 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:00.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:00 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:00.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:00 ntpd[17596]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:00.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:00 ntpd[17596]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:01.356 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:01 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:01.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:01 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:01.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:01 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:01.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:01 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:01.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:01 ntpd[17575]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:01.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:01 ntpd[17575]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:02.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:02 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:02.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:02 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:02.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:02 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:02.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:02 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:02.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:02 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:02.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:02 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:02.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:02 ntpd[17575]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:02.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:02 ntpd[17575]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:02.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:02 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:02.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:02 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:02.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:02 ntpd[17575]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:02.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:02 ntpd[17575]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:02.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:02.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:02.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:02.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:02.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:02.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:02.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:02.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:02.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:02.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:02.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:02.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:02.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:02.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:02.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:02.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:02.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:02.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:02 ntpd[17596]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:03.356 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:03 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:03.356 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:03 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:03.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:03 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:03.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:03 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:03.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:03 ntpd[17575]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:03.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:03 ntpd[17575]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:04.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:04 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:04.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:04 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:04.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:04 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:04.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:04 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:04.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:04 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:04.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:04 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:04.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:04 ntpd[17575]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:04.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:04 ntpd[17575]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:04.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:04 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:04.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:04 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:04.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:04 ntpd[17575]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:04.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:04 ntpd[17575]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:04.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:04 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:04.384 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:04 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:04.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:04 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:04.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:04 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:04.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:04 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:04.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:04 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:04.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:04 ntpd[17596]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:04.385 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:04 ntpd[17596]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:04.386 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:04 ntpd[17596]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:04.386 INFO:teuthology.orchestra.run.smithi195.stdout:28 Jun 14:57:04 ntpd[17596]: ntpd: time slew -0.000425 s 2024-06-28T14:57:04.386 INFO:teuthology.orchestra.run.smithi195.stdout:ntpd: time slew -0.000425s 2024-06-28T14:57:04.387 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:04 ntpd[17596]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:04.387 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:04 ntpd[17596]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-06-28T14:57:04.387 INFO:teuthology.orchestra.run.smithi195.stderr:28 Jun 14:57:04 ntpd[17596]: can't open /var/log/ntpstats/loopstats.20240628: Permission denied 2024-06-28T14:57:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: remote refid st t when poll reach delay offset jitter 2024-06-28T14:57:04.438 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================== 2024-06-28T14:57:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-06-28T14:57:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-06-28T14:57:04.438 INFO:teuthology.orchestra.run.smithi195.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-06-28T14:57:04.439 INFO:teuthology.orchestra.run.smithi195.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-06-28T14:57:05.356 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:05 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:05 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:05 ntpd[17575]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-28T14:57:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:05 ntpd[17575]: can't open /var/log/ntpstats/rawstats.20240628: Permission denied 2024-06-28T14:57:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:05 ntpd[17575]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-28T14:57:05.357 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:05 ntpd[17575]: can't open /var/log/ntpstats/peerstats.20240628: Permission denied 2024-06-28T14:57:05.357 INFO:teuthology.orchestra.run.smithi178.stdout:28 Jun 14:57:05 ntpd[17575]: ntpd: time slew -0.000104 s 2024-06-28T14:57:05.357 INFO:teuthology.orchestra.run.smithi178.stdout:ntpd: time slew -0.000104s 2024-06-28T14:57:05.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:05 ntpd[17575]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-06-28T14:57:05.358 INFO:teuthology.orchestra.run.smithi178.stderr:28 Jun 14:57:05 ntpd[17575]: can't open /var/log/ntpstats/loopstats.20240628: Permission denied 2024-06-28T14:57:05.406 INFO:teuthology.orchestra.run.smithi178.stdout: remote refid st t when poll reach delay offset jitter 2024-06-28T14:57:05.406 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================== 2024-06-28T14:57:05.406 INFO:teuthology.orchestra.run.smithi178.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-06-28T14:57:05.406 INFO:teuthology.orchestra.run.smithi178.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-06-28T14:57:05.406 INFO:teuthology.orchestra.run.smithi178.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-06-28T14:57:05.406 INFO:teuthology.orchestra.run.smithi178.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-06-28T14:57:05.407 INFO:teuthology.run_tasks:Running task install... 2024-06-28T14:57:05.416 DEBUG:teuthology.task.install:project ceph 2024-06-28T14:57:05.416 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ec2a38ba1b69b5357de8f0e8f96b00b4a577169d'}} 2024-06-28T14:57:05.416 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'flavor': 'default', 'sha1': 'ec2a38ba1b69b5357de8f0e8f96b00b4a577169d'} 2024-06-28T14:57:05.416 INFO:teuthology.task.install:Using flavor: default 2024-06-28T14:57:05.423 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-28T14:57:05.423 INFO:teuthology.task.install:extra packages: [] 2024-06-28T14:57:05.424 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-key list | grep Ceph 2024-06-28T14:57:05.424 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-key list | grep Ceph 2024-06-28T14:57:05.487 INFO:teuthology.orchestra.run.smithi195.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-06-28T14:57:05.503 INFO:teuthology.orchestra.run.smithi178.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-06-28T14:57:05.514 INFO:teuthology.orchestra.run.smithi195.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-06-28T14:57:05.514 INFO:teuthology.orchestra.run.smithi195.stdout:uid [ unknown] Ceph.com (release key) 2024-06-28T14:57:05.515 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-06-28T14:57:05.516 INFO:teuthology.task.install.deb:Installing system (non-project) packages: s3cmd, maven on remote deb x86_64 2024-06-28T14:57:05.516 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T14:57:05.529 INFO:teuthology.orchestra.run.smithi178.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-06-28T14:57:05.529 INFO:teuthology.orchestra.run.smithi178.stdout:uid [ unknown] Ceph.com (release key) 2024-06-28T14:57:05.529 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-06-28T14:57:05.530 INFO:teuthology.task.install.deb:Installing system (non-project) packages: s3cmd, maven on remote deb x86_64 2024-06-28T14:57:05.530 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T14:57:05.671 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default/ 2024-06-28T14:57:05.671 INFO:teuthology.task.install.deb:Package version is 19.0.0-4592-gec2a38ba-1jammy 2024-06-28T14:57:05.780 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:57:05.780 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-06-28T14:57:05.794 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get update 2024-06-28T14:57:05.860 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default/ 2024-06-28T14:57:05.860 INFO:teuthology.task.install.deb:Package version is 19.0.0-4592-gec2a38ba-1jammy 2024-06-28T14:57:06.087 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-06-28T14:57:06.088 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-06-28T14:57:06.164 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:57:06.165 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-06-28T14:57:06.165 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-06-28T14:57:06.179 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-06-28T14:57:06.208 INFO:teuthology.orchestra.run.smithi178.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy InRelease 2024-06-28T14:57:06.246 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-06-28T14:57:06.269 INFO:teuthology.orchestra.run.smithi178.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-06-28T14:57:06.332 INFO:teuthology.orchestra.run.smithi178.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy Release.gpg 2024-06-28T14:57:06.392 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-06-28T14:57:06.408 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-06-28T14:57:06.460 INFO:teuthology.orchestra.run.smithi178.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-06-28T14:57:06.467 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-06-28T14:57:06.538 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 28.5 kB in 1s (50.8 kB/s) 2024-06-28T14:57:06.549 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-06-28T14:57:06.575 INFO:teuthology.orchestra.run.smithi195.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy InRelease 2024-06-28T14:57:06.631 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-06-28T14:57:06.647 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-06-28T14:57:06.719 INFO:teuthology.orchestra.run.smithi195.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy Release.gpg 2024-06-28T14:57:06.794 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-06-28T14:57:06.880 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-06-28T14:57:06.973 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 28.5 kB in 1s (48.0 kB/s) 2024-06-28T14:57:07.501 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-06-28T14:57:07.518 DEBUG:teuthology.orchestra.run.smithi178:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-4592-gec2a38ba-1jammy cephadm=19.0.0-4592-gec2a38ba-1jammy ceph-mds=19.0.0-4592-gec2a38ba-1jammy ceph-mgr=19.0.0-4592-gec2a38ba-1jammy ceph-common=19.0.0-4592-gec2a38ba-1jammy ceph-fuse=19.0.0-4592-gec2a38ba-1jammy ceph-test=19.0.0-4592-gec2a38ba-1jammy ceph-volume=19.0.0-4592-gec2a38ba-1jammy radosgw=19.0.0-4592-gec2a38ba-1jammy python3-rados=19.0.0-4592-gec2a38ba-1jammy python3-rgw=19.0.0-4592-gec2a38ba-1jammy python3-cephfs=19.0.0-4592-gec2a38ba-1jammy python3-rbd=19.0.0-4592-gec2a38ba-1jammy libcephfs2=19.0.0-4592-gec2a38ba-1jammy libcephfs-dev=19.0.0-4592-gec2a38ba-1jammy librados2=19.0.0-4592-gec2a38ba-1jammy librbd1=19.0.0-4592-gec2a38ba-1jammy rbd-fuse=19.0.0-4592-gec2a38ba-1jammy 2024-06-28T14:57:07.572 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-06-28T14:57:07.752 INFO:teuthology.orchestra.run.smithi178.stdout:Building dependency tree... 2024-06-28T14:57:07.752 INFO:teuthology.orchestra.run.smithi178.stdout:Reading state information... 2024-06-28T14:57:07.933 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages were automatically installed and are no longer required: 2024-06-28T14:57:07.933 INFO:teuthology.orchestra.run.smithi178.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-06-28T14:57:07.933 INFO:teuthology.orchestra.run.smithi178.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-06-28T14:57:07.933 INFO:teuthology.orchestra.run.smithi178.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-06-28T14:57:07.934 INFO:teuthology.orchestra.run.smithi178.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-06-28T14:57:07.934 INFO:teuthology.orchestra.run.smithi178.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-06-28T14:57:07.934 INFO:teuthology.orchestra.run.smithi178.stdout:Use 'sudo apt autoremove' to remove them. 2024-06-28T14:57:07.934 INFO:teuthology.orchestra.run.smithi178.stdout:The following additional packages will be installed: 2024-06-28T14:57:07.934 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-06-28T14:57:07.935 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-06-28T14:57:07.935 INFO:teuthology.orchestra.run.smithi178.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-06-28T14:57:07.935 INFO:teuthology.orchestra.run.smithi178.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-06-28T14:57:07.935 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-06-28T14:57:07.935 INFO:teuthology.orchestra.run.smithi178.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-06-28T14:57:07.936 INFO:teuthology.orchestra.run.smithi178.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-06-28T14:57:07.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-06-28T14:57:07.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-06-28T14:57:07.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-06-28T14:57:07.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-06-28T14:57:07.936 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-06-28T14:57:07.937 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-06-28T14:57:07.937 INFO:teuthology.orchestra.run.smithi178.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-06-28T14:57:07.937 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-06-28T14:57:07.937 INFO:teuthology.orchestra.run.smithi178.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-06-28T14:57:07.937 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-06-28T14:57:07.938 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-06-28T14:57:07.938 INFO:teuthology.orchestra.run.smithi178.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-06-28T14:57:07.938 INFO:teuthology.orchestra.run.smithi178.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-06-28T14:57:07.938 INFO:teuthology.orchestra.run.smithi178.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-06-28T14:57:07.938 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-06-28T14:57:07.938 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-06-28T14:57:07.939 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet zip 2024-06-28T14:57:07.939 INFO:teuthology.orchestra.run.smithi178.stdout:Suggested packages: 2024-06-28T14:57:07.939 INFO:teuthology.orchestra.run.smithi178.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-06-28T14:57:07.939 INFO:teuthology.orchestra.run.smithi178.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-06-28T14:57:07.939 INFO:teuthology.orchestra.run.smithi178.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-06-28T14:57:07.939 INFO:teuthology.orchestra.run.smithi178.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-06-28T14:57:07.940 INFO:teuthology.orchestra.run.smithi178.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-06-28T14:57:07.940 INFO:teuthology.orchestra.run.smithi178.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-06-28T14:57:07.940 INFO:teuthology.orchestra.run.smithi178.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-06-28T14:57:07.940 INFO:teuthology.orchestra.run.smithi178.stdout:Recommended packages: 2024-06-28T14:57:07.940 INFO:teuthology.orchestra.run.smithi178.stdout: btrfs-tools 2024-06-28T14:57:08.009 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-06-28T14:57:08.025 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-4592-gec2a38ba-1jammy cephadm=19.0.0-4592-gec2a38ba-1jammy ceph-mds=19.0.0-4592-gec2a38ba-1jammy ceph-mgr=19.0.0-4592-gec2a38ba-1jammy ceph-common=19.0.0-4592-gec2a38ba-1jammy ceph-fuse=19.0.0-4592-gec2a38ba-1jammy ceph-test=19.0.0-4592-gec2a38ba-1jammy ceph-volume=19.0.0-4592-gec2a38ba-1jammy radosgw=19.0.0-4592-gec2a38ba-1jammy python3-rados=19.0.0-4592-gec2a38ba-1jammy python3-rgw=19.0.0-4592-gec2a38ba-1jammy python3-cephfs=19.0.0-4592-gec2a38ba-1jammy python3-rbd=19.0.0-4592-gec2a38ba-1jammy libcephfs2=19.0.0-4592-gec2a38ba-1jammy libcephfs-dev=19.0.0-4592-gec2a38ba-1jammy librados2=19.0.0-4592-gec2a38ba-1jammy librbd1=19.0.0-4592-gec2a38ba-1jammy rbd-fuse=19.0.0-4592-gec2a38ba-1jammy 2024-06-28T14:57:08.032 INFO:teuthology.orchestra.run.smithi178.stdout:The following NEW packages will be installed: 2024-06-28T14:57:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-06-28T14:57:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-06-28T14:57:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-06-28T14:57:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-06-28T14:57:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-06-28T14:57:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-06-28T14:57:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-06-28T14:57:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-06-28T14:57:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-06-28T14:57:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-06-28T14:57:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-06-28T14:57:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-06-28T14:57:08.085 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-06-28T14:57:08.156 INFO:teuthology.orchestra.run.smithi178.stdout:0 upgraded, 118 newly installed, 0 to remove and 315 not upgraded. 2024-06-28T14:57:08.156 INFO:teuthology.orchestra.run.smithi178.stdout:Need to get 248 MB of archives. 2024-06-28T14:57:08.156 INFO:teuthology.orchestra.run.smithi178.stdout:After this operation, 916 MB of additional disk space will be used. 2024-06-28T14:57:08.156 INFO:teuthology.orchestra.run.smithi178.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-06-28T14:57:08.172 INFO:teuthology.orchestra.run.smithi178.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-4592-gec2a38ba-1jammy [3,561 kB] 2024-06-28T14:57:08.287 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-06-28T14:57:08.287 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-06-28T14:57:08.418 INFO:teuthology.orchestra.run.smithi178.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-06-28T14:57:08.427 INFO:teuthology.orchestra.run.smithi178.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-06-28T14:57:08.475 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-06-28T14:57:08.475 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-06-28T14:57:08.475 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-06-28T14:57:08.475 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-06-28T14:57:08.475 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-06-28T14:57:08.476 INFO:teuthology.orchestra.run.smithi195.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-06-28T14:57:08.476 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-06-28T14:57:08.476 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2024-06-28T14:57:08.476 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-06-28T14:57:08.476 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-06-28T14:57:08.477 INFO:teuthology.orchestra.run.smithi195.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-06-28T14:57:08.477 INFO:teuthology.orchestra.run.smithi195.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-06-28T14:57:08.477 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-06-28T14:57:08.478 INFO:teuthology.orchestra.run.smithi195.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-06-28T14:57:08.478 INFO:teuthology.orchestra.run.smithi195.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-06-28T14:57:08.478 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-06-28T14:57:08.478 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-06-28T14:57:08.478 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-06-28T14:57:08.478 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-06-28T14:57:08.478 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-06-28T14:57:08.478 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-06-28T14:57:08.478 INFO:teuthology.orchestra.run.smithi195.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-06-28T14:57:08.478 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-06-28T14:57:08.478 INFO:teuthology.orchestra.run.smithi195.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-06-28T14:57:08.478 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-06-28T14:57:08.478 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-06-28T14:57:08.479 INFO:teuthology.orchestra.run.smithi195.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-06-28T14:57:08.479 INFO:teuthology.orchestra.run.smithi195.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-06-28T14:57:08.479 INFO:teuthology.orchestra.run.smithi195.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-06-28T14:57:08.479 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-06-28T14:57:08.479 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-06-28T14:57:08.479 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet zip 2024-06-28T14:57:08.479 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2024-06-28T14:57:08.479 INFO:teuthology.orchestra.run.smithi195.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-06-28T14:57:08.479 INFO:teuthology.orchestra.run.smithi195.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-06-28T14:57:08.479 INFO:teuthology.orchestra.run.smithi195.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-06-28T14:57:08.480 INFO:teuthology.orchestra.run.smithi195.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-06-28T14:57:08.480 INFO:teuthology.orchestra.run.smithi195.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-06-28T14:57:08.480 INFO:teuthology.orchestra.run.smithi195.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-06-28T14:57:08.480 INFO:teuthology.orchestra.run.smithi195.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-06-28T14:57:08.480 INFO:teuthology.orchestra.run.smithi195.stdout:Recommended packages: 2024-06-28T14:57:08.480 INFO:teuthology.orchestra.run.smithi195.stdout: btrfs-tools 2024-06-28T14:57:08.484 INFO:teuthology.orchestra.run.smithi178.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-06-28T14:57:08.579 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2024-06-28T14:57:08.579 INFO:teuthology.orchestra.run.smithi195.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-06-28T14:57:08.579 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-06-28T14:57:08.579 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-06-28T14:57:08.579 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-06-28T14:57:08.579 INFO:teuthology.orchestra.run.smithi195.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-06-28T14:57:08.579 INFO:teuthology.orchestra.run.smithi195.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-06-28T14:57:08.581 INFO:teuthology.orchestra.run.smithi195.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-06-28T14:57:08.581 INFO:teuthology.orchestra.run.smithi195.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-06-28T14:57:08.581 INFO:teuthology.orchestra.run.smithi195.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-06-28T14:57:08.581 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-06-28T14:57:08.582 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-06-28T14:57:08.582 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-06-28T14:57:08.582 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-06-28T14:57:08.582 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-06-28T14:57:08.583 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-06-28T14:57:08.583 INFO:teuthology.orchestra.run.smithi195.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-06-28T14:57:08.583 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-06-28T14:57:08.583 INFO:teuthology.orchestra.run.smithi195.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-06-28T14:57:08.583 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-06-28T14:57:08.583 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-06-28T14:57:08.583 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-06-28T14:57:08.583 INFO:teuthology.orchestra.run.smithi195.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-06-28T14:57:08.583 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-06-28T14:57:08.583 INFO:teuthology.orchestra.run.smithi195.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-06-28T14:57:08.584 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-06-28T14:57:08.584 INFO:teuthology.orchestra.run.smithi195.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-06-28T14:57:08.627 INFO:teuthology.orchestra.run.smithi178.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-4592-gec2a38ba-1jammy [3,230 kB] 2024-06-28T14:57:08.651 INFO:teuthology.orchestra.run.smithi178.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-06-28T14:57:08.659 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-06-28T14:57:08.717 INFO:teuthology.orchestra.run.smithi195.stdout:0 upgraded, 118 newly installed, 0 to remove and 315 not upgraded. 2024-06-28T14:57:08.717 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 248 MB of archives. 2024-06-28T14:57:08.717 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 916 MB of additional disk space will be used. 2024-06-28T14:57:08.717 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-4592-gec2a38ba-1jammy [3,561 kB] 2024-06-28T14:57:08.727 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-06-28T14:57:08.753 INFO:teuthology.orchestra.run.smithi178.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-06-28T14:57:08.760 INFO:teuthology.orchestra.run.smithi178.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-06-28T14:57:08.761 INFO:teuthology.orchestra.run.smithi178.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-06-28T14:57:08.762 INFO:teuthology.orchestra.run.smithi178.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-06-28T14:57:08.777 INFO:teuthology.orchestra.run.smithi178.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-06-28T14:57:08.781 INFO:teuthology.orchestra.run.smithi178.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-06-28T14:57:08.787 INFO:teuthology.orchestra.run.smithi178.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-06-28T14:57:08.789 INFO:teuthology.orchestra.run.smithi178.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-06-28T14:57:08.789 INFO:teuthology.orchestra.run.smithi178.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-06-28T14:57:08.808 INFO:teuthology.orchestra.run.smithi178.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-06-28T14:57:08.813 INFO:teuthology.orchestra.run.smithi178.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-06-28T14:57:08.813 INFO:teuthology.orchestra.run.smithi178.stdout:Get:20 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-4592-gec2a38ba-1jammy [715 kB] 2024-06-28T14:57:08.816 INFO:teuthology.orchestra.run.smithi178.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-06-28T14:57:08.839 INFO:teuthology.orchestra.run.smithi178.stdout:Get:22 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-4592-gec2a38ba-1jammy [356 kB] 2024-06-28T14:57:08.855 INFO:teuthology.orchestra.run.smithi178.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-06-28T14:57:08.855 INFO:teuthology.orchestra.run.smithi178.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-06-28T14:57:08.856 INFO:teuthology.orchestra.run.smithi178.stdout:Get:25 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-4592-gec2a38ba-1jammy [32.8 kB] 2024-06-28T14:57:08.858 INFO:teuthology.orchestra.run.smithi178.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-06-28T14:57:08.858 INFO:teuthology.orchestra.run.smithi178.stdout:Get:27 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-4592-gec2a38ba-1jammy [180 kB] 2024-06-28T14:57:08.859 INFO:teuthology.orchestra.run.smithi178.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-06-28T14:57:08.859 INFO:teuthology.orchestra.run.smithi178.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-06-28T14:57:08.860 INFO:teuthology.orchestra.run.smithi178.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-06-28T14:57:08.860 INFO:teuthology.orchestra.run.smithi178.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-06-28T14:57:08.863 INFO:teuthology.orchestra.run.smithi178.stdout:Get:32 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-4592-gec2a38ba-1jammy [67.4 kB] 2024-06-28T14:57:08.871 INFO:teuthology.orchestra.run.smithi178.stdout:Get:33 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-4592-gec2a38ba-1jammy [334 kB] 2024-06-28T14:57:08.880 INFO:teuthology.orchestra.run.smithi178.stdout:Get:34 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-4592-gec2a38ba-1jammy [7,898 kB] 2024-06-28T14:57:08.900 INFO:teuthology.orchestra.run.smithi178.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-06-28T14:57:08.901 INFO:teuthology.orchestra.run.smithi178.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-06-28T14:57:08.906 INFO:teuthology.orchestra.run.smithi178.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-06-28T14:57:08.951 INFO:teuthology.orchestra.run.smithi178.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-06-28T14:57:08.952 INFO:teuthology.orchestra.run.smithi178.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-06-28T14:57:08.952 INFO:teuthology.orchestra.run.smithi178.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-06-28T14:57:08.954 INFO:teuthology.orchestra.run.smithi178.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-06-28T14:57:08.955 INFO:teuthology.orchestra.run.smithi178.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-06-28T14:57:08.956 INFO:teuthology.orchestra.run.smithi178.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-06-28T14:57:08.958 INFO:teuthology.orchestra.run.smithi178.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-06-28T14:57:08.992 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-06-28T14:57:09.000 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-06-28T14:57:09.009 INFO:teuthology.orchestra.run.smithi178.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-06-28T14:57:09.009 INFO:teuthology.orchestra.run.smithi178.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-06-28T14:57:09.021 INFO:teuthology.orchestra.run.smithi178.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-06-28T14:57:09.071 INFO:teuthology.orchestra.run.smithi178.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-06-28T14:57:09.122 INFO:teuthology.orchestra.run.smithi178.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-4592-gec2a38ba-1jammy [112 kB] 2024-06-28T14:57:09.124 INFO:teuthology.orchestra.run.smithi178.stdout:Get:50 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-4592-gec2a38ba-1jammy [470 kB] 2024-06-28T14:57:09.133 INFO:teuthology.orchestra.run.smithi178.stdout:Get:51 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-4592-gec2a38ba-1jammy [27.9 MB] 2024-06-28T14:57:09.144 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-06-28T14:57:09.146 INFO:teuthology.orchestra.run.smithi178.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-06-28T14:57:09.147 INFO:teuthology.orchestra.run.smithi178.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-06-28T14:57:09.150 INFO:teuthology.orchestra.run.smithi178.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-06-28T14:57:09.153 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-4592-gec2a38ba-1jammy [3,230 kB] 2024-06-28T14:57:09.154 INFO:teuthology.orchestra.run.smithi178.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-06-28T14:57:09.155 INFO:teuthology.orchestra.run.smithi178.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-06-28T14:57:09.174 INFO:teuthology.orchestra.run.smithi178.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-06-28T14:57:09.175 INFO:teuthology.orchestra.run.smithi178.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-06-28T14:57:09.180 INFO:teuthology.orchestra.run.smithi178.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-06-28T14:57:09.181 INFO:teuthology.orchestra.run.smithi178.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-06-28T14:57:09.203 INFO:teuthology.orchestra.run.smithi178.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-06-28T14:57:09.245 INFO:teuthology.orchestra.run.smithi178.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-06-28T14:57:09.382 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-4592-gec2a38ba-1jammy [715 kB] 2024-06-28T14:57:09.404 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-06-28T14:57:09.410 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-06-28T14:57:09.420 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-4592-gec2a38ba-1jammy [356 kB] 2024-06-28T14:57:09.427 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-06-28T14:57:09.432 INFO:teuthology.orchestra.run.smithi178.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-06-28T14:57:09.433 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-06-28T14:57:09.433 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-06-28T14:57:09.434 INFO:teuthology.orchestra.run.smithi178.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-06-28T14:57:09.434 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-06-28T14:57:09.437 INFO:teuthology.orchestra.run.smithi178.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-06-28T14:57:09.438 INFO:teuthology.orchestra.run.smithi178.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-06-28T14:57:09.451 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-4592-gec2a38ba-1jammy [32.8 kB] 2024-06-28T14:57:09.453 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-4592-gec2a38ba-1jammy [180 kB] 2024-06-28T14:57:09.457 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-4592-gec2a38ba-1jammy [67.4 kB] 2024-06-28T14:57:09.459 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-4592-gec2a38ba-1jammy [334 kB] 2024-06-28T14:57:09.491 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-4592-gec2a38ba-1jammy [7,898 kB] 2024-06-28T14:57:09.507 INFO:teuthology.orchestra.run.smithi178.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-06-28T14:57:09.511 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-06-28T14:57:09.516 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-06-28T14:57:09.547 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-06-28T14:57:09.550 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-06-28T14:57:09.551 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-06-28T14:57:09.552 INFO:teuthology.orchestra.run.smithi178.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-06-28T14:57:09.555 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-06-28T14:57:09.556 INFO:teuthology.orchestra.run.smithi178.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-06-28T14:57:09.557 INFO:teuthology.orchestra.run.smithi178.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-06-28T14:57:09.584 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-06-28T14:57:09.585 INFO:teuthology.orchestra.run.smithi178.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-06-28T14:57:09.585 INFO:teuthology.orchestra.run.smithi178.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-06-28T14:57:09.586 INFO:teuthology.orchestra.run.smithi178.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-06-28T14:57:09.587 INFO:teuthology.orchestra.run.smithi178.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-06-28T14:57:09.588 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-06-28T14:57:09.588 INFO:teuthology.orchestra.run.smithi178.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-06-28T14:57:09.589 INFO:teuthology.orchestra.run.smithi178.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-06-28T14:57:09.589 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-06-28T14:57:09.590 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-06-28T14:57:09.592 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-06-28T14:57:09.593 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-06-28T14:57:09.593 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-06-28T14:57:09.609 INFO:teuthology.orchestra.run.smithi178.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-06-28T14:57:09.610 INFO:teuthology.orchestra.run.smithi178.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-06-28T14:57:09.620 INFO:teuthology.orchestra.run.smithi178.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-06-28T14:57:09.620 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-06-28T14:57:09.620 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-06-28T14:57:09.621 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-06-28T14:57:09.622 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-06-28T14:57:09.628 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-06-28T14:57:09.639 INFO:teuthology.orchestra.run.smithi178.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-06-28T14:57:09.641 INFO:teuthology.orchestra.run.smithi178.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-06-28T14:57:09.642 INFO:teuthology.orchestra.run.smithi178.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-06-28T14:57:09.650 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-06-28T14:57:09.651 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-06-28T14:57:09.652 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-06-28T14:57:09.652 INFO:teuthology.orchestra.run.smithi178.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-06-28T14:57:09.654 INFO:teuthology.orchestra.run.smithi195.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-06-28T14:57:09.654 INFO:teuthology.orchestra.run.smithi195.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-06-28T14:57:09.661 INFO:teuthology.orchestra.run.smithi178.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-06-28T14:57:09.663 INFO:teuthology.orchestra.run.smithi178.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-06-28T14:57:09.663 INFO:teuthology.orchestra.run.smithi178.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-06-28T14:57:09.675 INFO:teuthology.orchestra.run.smithi178.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-06-28T14:57:09.676 INFO:teuthology.orchestra.run.smithi178.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-06-28T14:57:09.685 INFO:teuthology.orchestra.run.smithi195.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-06-28T14:57:09.688 INFO:teuthology.orchestra.run.smithi195.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-06-28T14:57:09.689 INFO:teuthology.orchestra.run.smithi178.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-06-28T14:57:09.690 INFO:teuthology.orchestra.run.smithi195.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-06-28T14:57:09.691 INFO:teuthology.orchestra.run.smithi195.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-06-28T14:57:09.702 INFO:teuthology.orchestra.run.smithi195.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-06-28T14:57:09.709 INFO:teuthology.orchestra.run.smithi178.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-06-28T14:57:09.710 INFO:teuthology.orchestra.run.smithi178.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-06-28T14:57:09.710 INFO:teuthology.orchestra.run.smithi178.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-06-28T14:57:09.713 INFO:teuthology.orchestra.run.smithi178.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-06-28T14:57:09.714 INFO:teuthology.orchestra.run.smithi178.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-06-28T14:57:09.716 INFO:teuthology.orchestra.run.smithi195.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-06-28T14:57:09.717 INFO:teuthology.orchestra.run.smithi195.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-06-28T14:57:09.719 INFO:teuthology.orchestra.run.smithi195.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-06-28T14:57:09.721 INFO:teuthology.orchestra.run.smithi195.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-06-28T14:57:09.726 INFO:teuthology.orchestra.run.smithi195.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-06-28T14:57:09.750 INFO:teuthology.orchestra.run.smithi195.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-06-28T14:57:09.753 INFO:teuthology.orchestra.run.smithi195.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-06-28T14:57:09.755 INFO:teuthology.orchestra.run.smithi195.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-06-28T14:57:09.760 INFO:teuthology.orchestra.run.smithi195.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-06-28T14:57:09.761 INFO:teuthology.orchestra.run.smithi195.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-06-28T14:57:09.789 INFO:teuthology.orchestra.run.smithi195.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-06-28T14:57:09.794 INFO:teuthology.orchestra.run.smithi195.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-06-28T14:57:09.801 INFO:teuthology.orchestra.run.smithi178.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-06-28T14:57:09.801 INFO:teuthology.orchestra.run.smithi178.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-06-28T14:57:09.834 INFO:teuthology.orchestra.run.smithi178.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-06-28T14:57:09.873 INFO:teuthology.orchestra.run.smithi178.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-06-28T14:57:09.873 INFO:teuthology.orchestra.run.smithi178.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-4592-gec2a38ba-1jammy [5,096 kB] 2024-06-28T14:57:09.874 INFO:teuthology.orchestra.run.smithi178.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-06-28T14:57:09.891 INFO:teuthology.orchestra.run.smithi195.stdout:Get:60 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-4592-gec2a38ba-1jammy [112 kB] 2024-06-28T14:57:09.893 INFO:teuthology.orchestra.run.smithi195.stdout:Get:61 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-4592-gec2a38ba-1jammy [470 kB] 2024-06-28T14:57:09.902 INFO:teuthology.orchestra.run.smithi195.stdout:Get:62 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-4592-gec2a38ba-1jammy [27.9 MB] 2024-06-28T14:57:09.932 INFO:teuthology.orchestra.run.smithi195.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-06-28T14:57:09.933 INFO:teuthology.orchestra.run.smithi195.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-06-28T14:57:09.935 INFO:teuthology.orchestra.run.smithi195.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-06-28T14:57:09.936 INFO:teuthology.orchestra.run.smithi195.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-06-28T14:57:09.937 INFO:teuthology.orchestra.run.smithi195.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-06-28T14:57:09.951 INFO:teuthology.orchestra.run.smithi178.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-4592-gec2a38ba-1jammy [247 kB] 2024-06-28T14:57:09.954 INFO:teuthology.orchestra.run.smithi178.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-4592-gec2a38ba-1jammy [124 kB] 2024-06-28T14:57:09.955 INFO:teuthology.orchestra.run.smithi178.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-4592-gec2a38ba-1jammy [1,547 kB] 2024-06-28T14:57:09.962 INFO:teuthology.orchestra.run.smithi195.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-06-28T14:57:09.965 INFO:teuthology.orchestra.run.smithi195.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-06-28T14:57:09.965 INFO:teuthology.orchestra.run.smithi195.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-06-28T14:57:09.979 INFO:teuthology.orchestra.run.smithi178.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-4592-gec2a38ba-1jammy [6,268 kB] 2024-06-28T14:57:10.036 INFO:teuthology.orchestra.run.smithi195.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-06-28T14:57:10.036 INFO:teuthology.orchestra.run.smithi195.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-06-28T14:57:10.037 INFO:teuthology.orchestra.run.smithi195.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-06-28T14:57:10.038 INFO:teuthology.orchestra.run.smithi195.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-06-28T14:57:10.038 INFO:teuthology.orchestra.run.smithi195.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-06-28T14:57:10.039 INFO:teuthology.orchestra.run.smithi195.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-06-28T14:57:10.039 INFO:teuthology.orchestra.run.smithi195.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-06-28T14:57:10.040 INFO:teuthology.orchestra.run.smithi195.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-06-28T14:57:10.043 INFO:teuthology.orchestra.run.smithi195.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-06-28T14:57:10.070 INFO:teuthology.orchestra.run.smithi195.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-06-28T14:57:10.078 INFO:teuthology.orchestra.run.smithi195.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-06-28T14:57:10.079 INFO:teuthology.orchestra.run.smithi195.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-06-28T14:57:10.084 INFO:teuthology.orchestra.run.smithi195.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-06-28T14:57:10.088 INFO:teuthology.orchestra.run.smithi195.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-06-28T14:57:10.090 INFO:teuthology.orchestra.run.smithi195.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-06-28T14:57:10.090 INFO:teuthology.orchestra.run.smithi195.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-06-28T14:57:10.096 INFO:teuthology.orchestra.run.smithi195.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-06-28T14:57:10.096 INFO:teuthology.orchestra.run.smithi195.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-06-28T14:57:10.112 INFO:teuthology.orchestra.run.smithi195.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-06-28T14:57:10.125 INFO:teuthology.orchestra.run.smithi178.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-4592-gec2a38ba-1jammy [22.8 MB] 2024-06-28T14:57:10.131 INFO:teuthology.orchestra.run.smithi195.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-06-28T14:57:10.132 INFO:teuthology.orchestra.run.smithi195.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-06-28T14:57:10.133 INFO:teuthology.orchestra.run.smithi195.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-06-28T14:57:10.135 INFO:teuthology.orchestra.run.smithi195.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-06-28T14:57:10.136 INFO:teuthology.orchestra.run.smithi195.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-06-28T14:57:10.147 INFO:teuthology.orchestra.run.smithi195.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-06-28T14:57:10.148 INFO:teuthology.orchestra.run.smithi195.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-06-28T14:57:10.149 INFO:teuthology.orchestra.run.smithi195.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-06-28T14:57:10.152 INFO:teuthology.orchestra.run.smithi195.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-06-28T14:57:10.189 INFO:teuthology.orchestra.run.smithi195.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-06-28T14:57:10.774 INFO:teuthology.orchestra.run.smithi178.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-4592-gec2a38ba-1jammy [14.0 kB] 2024-06-28T14:57:10.775 INFO:teuthology.orchestra.run.smithi178.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-4592-gec2a38ba-1jammy [871 kB] 2024-06-28T14:57:10.794 INFO:teuthology.orchestra.run.smithi178.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-4592-gec2a38ba-1jammy [2,446 kB] 2024-06-28T14:57:10.831 INFO:teuthology.orchestra.run.smithi178.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-4592-gec2a38ba-1jammy [793 kB] 2024-06-28T14:57:10.841 INFO:teuthology.orchestra.run.smithi178.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-4592-gec2a38ba-1jammy [149 kB] 2024-06-28T14:57:10.868 INFO:teuthology.orchestra.run.smithi178.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-4592-gec2a38ba-1jammy [60.3 MB] 2024-06-28T14:57:11.128 INFO:teuthology.orchestra.run.smithi195.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-4592-gec2a38ba-1jammy [5,096 kB] 2024-06-28T14:57:11.202 INFO:teuthology.orchestra.run.smithi195.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-4592-gec2a38ba-1jammy [247 kB] 2024-06-28T14:57:11.205 INFO:teuthology.orchestra.run.smithi195.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-4592-gec2a38ba-1jammy [124 kB] 2024-06-28T14:57:11.206 INFO:teuthology.orchestra.run.smithi195.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-4592-gec2a38ba-1jammy [1,547 kB] 2024-06-28T14:57:11.220 INFO:teuthology.orchestra.run.smithi195.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-4592-gec2a38ba-1jammy [6,268 kB] 2024-06-28T14:57:11.345 INFO:teuthology.orchestra.run.smithi195.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-4592-gec2a38ba-1jammy [22.8 MB] 2024-06-28T14:57:11.924 INFO:teuthology.orchestra.run.smithi195.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-4592-gec2a38ba-1jammy [14.0 kB] 2024-06-28T14:57:11.924 INFO:teuthology.orchestra.run.smithi195.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-4592-gec2a38ba-1jammy [871 kB] 2024-06-28T14:57:11.939 INFO:teuthology.orchestra.run.smithi195.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-4592-gec2a38ba-1jammy [2,446 kB] 2024-06-28T14:57:11.978 INFO:teuthology.orchestra.run.smithi195.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-4592-gec2a38ba-1jammy [793 kB] 2024-06-28T14:57:11.988 INFO:teuthology.orchestra.run.smithi195.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-4592-gec2a38ba-1jammy [149 kB] 2024-06-28T14:57:11.990 INFO:teuthology.orchestra.run.smithi195.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-4592-gec2a38ba-1jammy [60.3 MB] 2024-06-28T14:57:13.634 INFO:teuthology.orchestra.run.smithi195.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-4592-gec2a38ba-1jammy [8,625 kB] 2024-06-28T14:57:13.801 INFO:teuthology.orchestra.run.smithi195.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-4592-gec2a38ba-1jammy [14.2 kB] 2024-06-28T14:57:13.802 INFO:teuthology.orchestra.run.smithi195.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-4592-gec2a38ba-1jammy [54.8 MB] 2024-06-28T14:57:14.134 INFO:teuthology.orchestra.run.smithi178.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-4592-gec2a38ba-1jammy [8,625 kB] 2024-06-28T14:57:14.411 INFO:teuthology.orchestra.run.smithi178.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-4592-gec2a38ba-1jammy [14.2 kB] 2024-06-28T14:57:14.412 INFO:teuthology.orchestra.run.smithi178.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-4592-gec2a38ba-1jammy [54.8 MB] 2024-06-28T14:57:15.534 INFO:teuthology.orchestra.run.smithi195.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-4592-gec2a38ba-1jammy [125 kB] 2024-06-28T14:57:15.616 INFO:teuthology.orchestra.run.smithi195.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-4592-gec2a38ba-1jammy [38.9 kB] 2024-06-28T14:57:15.617 INFO:teuthology.orchestra.run.smithi195.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-4592-gec2a38ba-1jammy [15.7 MB] 2024-06-28T14:57:16.073 INFO:teuthology.orchestra.run.smithi195.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-4592-gec2a38ba-1jammy [91.3 kB] 2024-06-28T14:57:16.350 INFO:teuthology.orchestra.run.smithi178.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-4592-gec2a38ba-1jammy [125 kB] 2024-06-28T14:57:16.352 INFO:teuthology.orchestra.run.smithi178.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-4592-gec2a38ba-1jammy [38.9 kB] 2024-06-28T14:57:16.354 INFO:teuthology.orchestra.run.smithi178.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-4592-gec2a38ba-1jammy [15.7 MB] 2024-06-28T14:57:16.651 INFO:teuthology.orchestra.run.smithi178.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-cbodley-testing/ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-4592-gec2a38ba-1jammy [91.3 kB] 2024-06-28T14:57:16.655 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 248 MB in 7s (33.1 MB/s) 2024-06-28T14:57:16.819 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-06-28T14:57:16.920 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165982 files and directories currently installed.) 2024-06-28T14:57:16.925 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-06-28T14:57:17.010 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-06-28T14:57:17.227 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 248 MB in 9s (28.8 MB/s) 2024-06-28T14:57:17.330 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-06-28T14:57:17.356 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-06-28T14:57:17.391 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-06-28T14:57:17.396 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-06-28T14:57:17.491 INFO:teuthology.orchestra.run.smithi178.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165982 files and directories currently installed.) 2024-06-28T14:57:17.496 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-06-28T14:57:17.581 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-06-28T14:57:17.741 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-06-28T14:57:17.767 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-06-28T14:57:17.808 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-06-28T14:57:17.960 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-06-28T14:57:17.986 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-06-28T14:57:18.051 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-06-28T14:57:18.236 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-06-28T14:57:18.263 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-06-28T14:57:18.311 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-06-28T14:57:18.429 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-06-28T14:57:18.456 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-06-28T14:57:18.496 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-06-28T14:57:18.706 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-06-28T14:57:18.732 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-06-28T14:57:18.772 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-06-28T14:57:18.941 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-06-28T14:57:18.968 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-06-28T14:57:19.016 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-06-28T14:57:19.126 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-06-28T14:57:19.152 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-06-28T14:57:19.192 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-06-28T14:57:19.486 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-06-28T14:57:19.513 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-06-28T14:57:19.537 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-06-28T14:57:19.561 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-06-28T14:57:19.563 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-06-28T14:57:19.603 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-06-28T14:57:19.915 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librados2. 2024-06-28T14:57:19.939 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-06-28T14:57:19.941 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../007-librados2_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:19.966 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-06-28T14:57:19.982 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librados2 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:20.006 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-06-28T14:57:20.351 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-06-28T14:57:20.377 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-06-28T14:57:20.417 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-06-28T14:57:20.426 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librbd1. 2024-06-28T14:57:20.452 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../008-librbd1_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:20.492 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librbd1 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:20.795 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package librados2. 2024-06-28T14:57:20.822 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../007-librados2_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:20.870 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking librados2 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:20.954 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcephfs2. 2024-06-28T14:57:20.980 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:21.020 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcephfs2 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:21.281 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rados. 2024-06-28T14:57:21.297 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../010-python3-rados_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:21.331 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package librbd1. 2024-06-28T14:57:21.331 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rados (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:21.358 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../008-librbd1_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:21.398 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking librbd1 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:21.634 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-06-28T14:57:21.660 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:57:21.709 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-ceph-argparse (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:21.859 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libcephfs2. 2024-06-28T14:57:21.885 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:21.925 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libcephfs2 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:21.961 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cephfs. 2024-06-28T14:57:21.984 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:22.028 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cephfs (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:22.236 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-rados. 2024-06-28T14:57:22.263 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../010-python3-rados_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:22.311 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-rados (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:22.347 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-ceph-common. 2024-06-28T14:57:22.373 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:57:22.414 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-ceph-common (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:22.689 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-06-28T14:57:22.716 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:57:22.734 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-wcwidth. 2024-06-28T14:57:22.760 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-06-28T14:57:22.763 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-ceph-argparse (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:22.800 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-06-28T14:57:23.025 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-cephfs. 2024-06-28T14:57:23.051 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:23.091 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-cephfs (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:23.095 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-prettytable. 2024-06-28T14:57:23.121 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-06-28T14:57:23.153 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-06-28T14:57:23.397 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rbd. 2024-06-28T14:57:23.402 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-ceph-common. 2024-06-28T14:57:23.423 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:23.429 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:57:23.469 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-ceph-common (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:23.472 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rbd (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:23.797 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-wcwidth. 2024-06-28T14:57:23.824 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-06-28T14:57:23.833 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-06-28T14:57:23.859 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-06-28T14:57:23.863 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-06-28T14:57:23.900 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-06-28T14:57:24.144 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-06-28T14:57:24.159 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-06-28T14:57:24.202 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-06-28T14:57:24.242 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-prettytable. 2024-06-28T14:57:24.268 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-06-28T14:57:24.316 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-06-28T14:57:24.522 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-06-28T14:57:24.548 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-06-28T14:57:24.588 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-06-28T14:57:24.611 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-rbd. 2024-06-28T14:57:24.637 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:24.677 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-rbd (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:24.934 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua5.1. 2024-06-28T14:57:24.960 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-06-28T14:57:25.000 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-06-28T14:57:25.022 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-06-28T14:57:25.048 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-06-28T14:57:25.088 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-06-28T14:57:25.287 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua-any. 2024-06-28T14:57:25.302 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-06-28T14:57:25.337 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua-any (27ubuntu1) ... 2024-06-28T14:57:25.433 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-06-28T14:57:25.460 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-06-28T14:57:25.507 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-06-28T14:57:25.564 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package zip. 2024-06-28T14:57:25.579 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-06-28T14:57:25.631 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking zip (3.0-12build2) ... 2024-06-28T14:57:25.861 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-06-28T14:57:25.887 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-06-28T14:57:25.936 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-06-28T14:57:26.201 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package unzip. 2024-06-28T14:57:26.227 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-06-28T14:57:26.267 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-06-28T14:57:26.280 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package lua5.1. 2024-06-28T14:57:26.307 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-06-28T14:57:26.347 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-06-28T14:57:26.658 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package lua-any. 2024-06-28T14:57:26.671 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package luarocks. 2024-06-28T14:57:26.674 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-06-28T14:57:26.697 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-06-28T14:57:26.717 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking lua-any (27ubuntu1) ... 2024-06-28T14:57:26.738 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-06-28T14:57:27.028 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package zip. 2024-06-28T14:57:27.054 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-06-28T14:57:27.102 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking zip (3.0-12build2) ... 2024-06-28T14:57:27.112 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librgw2. 2024-06-28T14:57:27.139 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../025-librgw2_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:27.178 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librgw2 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:27.456 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package unzip. 2024-06-28T14:57:27.483 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-06-28T14:57:27.523 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-06-28T14:57:27.790 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rgw. 2024-06-28T14:57:27.816 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:27.856 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rgw (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:27.918 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package luarocks. 2024-06-28T14:57:27.944 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-06-28T14:57:27.988 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-06-28T14:57:28.159 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liboath0:amd64. 2024-06-28T14:57:28.174 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-06-28T14:57:28.209 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-06-28T14:57:28.401 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package librgw2. 2024-06-28T14:57:28.427 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../025-librgw2_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:28.492 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking librgw2 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:28.529 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libradosstriper1. 2024-06-28T14:57:28.555 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:28.595 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libradosstriper1 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:29.041 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-common. 2024-06-28T14:57:29.067 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../029-ceph-common_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:29.107 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-common (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:29.178 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-rgw. 2024-06-28T14:57:29.205 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:29.253 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-rgw (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:29.606 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package liboath0:amd64. 2024-06-28T14:57:29.632 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-06-28T14:57:29.672 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-06-28T14:57:29.967 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libradosstriper1. 2024-06-28T14:57:29.993 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:30.033 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libradosstriper1 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:30.529 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-base. 2024-06-28T14:57:30.555 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../030-ceph-base_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:30.621 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-base (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:30.699 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-common. 2024-06-28T14:57:30.726 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../029-ceph-common_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:30.870 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-common (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:31.200 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-06-28T14:57:31.226 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-06-28T14:57:31.266 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-06-28T14:57:31.569 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cheroot. 2024-06-28T14:57:31.596 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-06-28T14:57:31.635 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-06-28T14:57:31.955 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-06-28T14:57:31.982 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-06-28T14:57:32.022 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-06-28T14:57:32.291 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.text. 2024-06-28T14:57:32.307 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-06-28T14:57:32.341 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-06-28T14:57:32.400 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-base. 2024-06-28T14:57:32.427 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../030-ceph-base_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:32.476 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-base (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:32.628 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-06-28T14:57:32.654 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-06-28T14:57:32.694 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-06-28T14:57:32.980 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-tempora. 2024-06-28T14:57:33.006 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-06-28T14:57:33.046 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-06-28T14:57:33.047 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-06-28T14:57:33.073 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-06-28T14:57:33.114 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-06-28T14:57:33.358 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-portend. 2024-06-28T14:57:33.385 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-06-28T14:57:33.399 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-cheroot. 2024-06-28T14:57:33.425 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-06-28T14:57:33.425 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-06-28T14:57:33.465 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-06-28T14:57:33.669 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-06-28T14:57:33.695 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-06-28T14:57:33.736 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-06-28T14:57:33.776 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-06-28T14:57:33.803 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-06-28T14:57:33.843 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-06-28T14:57:34.064 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cherrypy3. 2024-06-28T14:57:34.090 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-06-28T14:57:34.113 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-jaraco.text. 2024-06-28T14:57:34.130 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-06-28T14:57:34.139 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-06-28T14:57:34.179 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-06-28T14:57:34.450 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-natsort. 2024-06-28T14:57:34.477 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-06-28T14:57:34.482 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-06-28T14:57:34.508 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-06-28T14:57:34.509 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-06-28T14:57:34.549 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-06-28T14:57:34.803 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-logutils. 2024-06-28T14:57:34.830 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-06-28T14:57:34.851 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-tempora. 2024-06-28T14:57:34.870 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-06-28T14:57:34.878 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-06-28T14:57:34.918 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-06-28T14:57:35.158 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-markupsafe. 2024-06-28T14:57:35.171 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-portend. 2024-06-28T14:57:35.182 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-06-28T14:57:35.186 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-06-28T14:57:35.221 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-06-28T14:57:35.222 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-06-28T14:57:35.507 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-06-28T14:57:35.526 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-mako. 2024-06-28T14:57:35.533 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-06-28T14:57:35.552 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-06-28T14:57:35.574 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-06-28T14:57:35.592 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-06-28T14:57:35.876 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-cherrypy3. 2024-06-28T14:57:35.887 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-simplegeneric. 2024-06-28T14:57:35.903 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-06-28T14:57:35.913 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-06-28T14:57:35.945 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-06-28T14:57:35.968 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-06-28T14:57:36.215 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-singledispatch. 2024-06-28T14:57:36.241 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-06-28T14:57:36.281 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-06-28T14:57:36.297 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-natsort. 2024-06-28T14:57:36.323 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-06-28T14:57:36.363 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-06-28T14:57:36.586 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webob. 2024-06-28T14:57:36.611 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-06-28T14:57:36.651 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-06-28T14:57:36.674 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-logutils. 2024-06-28T14:57:36.701 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-06-28T14:57:36.741 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-06-28T14:57:36.998 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-waitress. 2024-06-28T14:57:37.022 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-06-28T14:57:37.052 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-markupsafe. 2024-06-28T14:57:37.065 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-06-28T14:57:37.078 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-06-28T14:57:37.119 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-06-28T14:57:37.406 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-mako. 2024-06-28T14:57:37.407 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-tempita. 2024-06-28T14:57:37.431 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-06-28T14:57:37.433 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-06-28T14:57:37.463 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-06-28T14:57:37.475 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-06-28T14:57:37.778 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-paste. 2024-06-28T14:57:37.783 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-simplegeneric. 2024-06-28T14:57:37.804 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-06-28T14:57:37.810 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-06-28T14:57:37.841 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-06-28T14:57:37.844 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-06-28T14:57:38.131 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-06-28T14:57:38.144 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-singledispatch. 2024-06-28T14:57:38.146 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-06-28T14:57:38.172 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-06-28T14:57:38.181 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-06-28T14:57:38.211 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-06-28T14:57:38.484 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pastedeploy. 2024-06-28T14:57:38.496 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-webob. 2024-06-28T14:57:38.510 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-06-28T14:57:38.524 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-06-28T14:57:38.550 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-06-28T14:57:38.563 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-06-28T14:57:38.853 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webtest. 2024-06-28T14:57:38.880 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-06-28T14:57:38.908 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-waitress. 2024-06-28T14:57:38.920 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-06-28T14:57:38.935 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-06-28T14:57:38.978 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-06-28T14:57:39.231 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pecan. 2024-06-28T14:57:39.257 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-06-28T14:57:39.277 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-tempita. 2024-06-28T14:57:39.297 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-06-28T14:57:39.304 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-06-28T14:57:39.336 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-06-28T14:57:39.638 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-paste. 2024-06-28T14:57:39.665 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-06-28T14:57:39.669 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-certifi. 2024-06-28T14:57:39.696 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-06-28T14:57:39.705 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-06-28T14:57:39.735 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-06-28T14:57:40.038 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-idna. 2024-06-28T14:57:40.041 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-06-28T14:57:40.057 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-06-28T14:57:40.065 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-06-28T14:57:40.100 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-06-28T14:57:40.104 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-06-28T14:57:40.394 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pastedeploy. 2024-06-28T14:57:40.421 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-06-28T14:57:40.461 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-06-28T14:57:40.739 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-webtest. 2024-06-28T14:57:40.754 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-06-28T14:57:40.788 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-06-28T14:57:41.025 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pecan. 2024-06-28T14:57:41.047 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-06-28T14:57:41.091 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-06-28T14:57:41.292 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-urllib3. 2024-06-28T14:57:41.319 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-06-28T14:57:41.567 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-06-28T14:57:41.895 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-requests. 2024-06-28T14:57:41.922 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-06-28T14:57:41.961 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-06-28T14:57:42.248 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-werkzeug. 2024-06-28T14:57:42.274 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-06-28T14:57:42.306 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-06-28T14:57:42.341 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-certifi. 2024-06-28T14:57:42.368 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-06-28T14:57:42.407 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-06-28T14:57:42.575 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-06-28T14:57:42.591 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:57:42.634 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-modules-core (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:42.735 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-idna. 2024-06-28T14:57:42.762 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-06-28T14:57:42.802 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-06-28T14:57:43.022 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-06-28T14:57:43.049 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:43.088 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:43.130 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-urllib3. 2024-06-28T14:57:43.157 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-06-28T14:57:43.197 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-06-28T14:57:43.350 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-bcrypt. 2024-06-28T14:57:43.376 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-06-28T14:57:43.424 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-06-28T14:57:43.500 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-requests. 2024-06-28T14:57:43.527 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-06-28T14:57:43.567 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-06-28T14:57:43.727 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-openssl. 2024-06-28T14:57:43.754 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-06-28T14:57:43.794 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-06-28T14:57:43.878 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-werkzeug. 2024-06-28T14:57:43.905 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-06-28T14:57:43.944 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-06-28T14:57:44.164 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr. 2024-06-28T14:57:44.190 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:44.230 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:44.248 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-06-28T14:57:44.275 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:57:44.315 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-modules-core (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:44.642 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mon. 2024-06-28T14:57:44.668 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:44.678 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-06-28T14:57:44.705 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:44.708 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mon (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:44.744 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:45.065 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-bcrypt. 2024-06-28T14:57:45.091 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-06-28T14:57:45.148 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-06-28T14:57:45.278 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-osd. 2024-06-28T14:57:45.305 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:45.345 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-osd (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:45.409 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-openssl. 2024-06-28T14:57:45.424 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-06-28T14:57:45.459 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-06-28T14:57:45.812 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr. 2024-06-28T14:57:45.839 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:45.878 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:46.282 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mon. 2024-06-28T14:57:46.309 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:46.349 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mon (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:46.406 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph. 2024-06-28T14:57:46.422 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../066-ceph_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:46.473 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:46.751 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-fuse. 2024-06-28T14:57:46.778 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:46.809 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-fuse (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:46.936 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-osd. 2024-06-28T14:57:46.962 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:46.994 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-osd (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:47.170 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mds. 2024-06-28T14:57:47.197 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:47.253 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mds (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:47.656 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephadm. 2024-06-28T14:57:47.684 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../069-cephadm_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:47.723 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephadm (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:48.000 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-asyncssh. 2024-06-28T14:57:48.016 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-06-28T14:57:48.056 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph. 2024-06-28T14:57:48.059 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-06-28T14:57:48.071 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../066-ceph_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:48.114 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:48.378 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-babel-localedata. 2024-06-28T14:57:48.405 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-06-28T14:57:48.433 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-fuse. 2024-06-28T14:57:48.444 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-06-28T14:57:48.460 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:48.500 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-fuse (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:48.837 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mds. 2024-06-28T14:57:48.863 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:48.895 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mds (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:49.307 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package cephadm. 2024-06-28T14:57:49.333 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../069-cephadm_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:57:49.373 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking cephadm (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:49.400 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-babel. 2024-06-28T14:57:49.427 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-06-28T14:57:49.457 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-06-28T14:57:49.701 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-asyncssh. 2024-06-28T14:57:49.728 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-06-28T14:57:49.752 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jinja2. 2024-06-28T14:57:49.767 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-06-28T14:57:49.780 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-06-28T14:57:49.819 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-06-28T14:57:50.088 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-06-28T14:57:50.104 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python-babel-localedata. 2024-06-28T14:57:50.104 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:57:50.130 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-06-28T14:57:50.147 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-cephadm (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:50.170 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-06-28T14:57:50.500 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-repoze.lru. 2024-06-28T14:57:50.527 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-06-28T14:57:50.583 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-06-28T14:57:50.878 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-routes. 2024-06-28T14:57:50.904 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-06-28T14:57:50.944 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-06-28T14:57:51.126 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-babel. 2024-06-28T14:57:51.153 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-06-28T14:57:51.192 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-06-28T14:57:51.289 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-06-28T14:57:51.316 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:57:51.355 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-dashboard (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:51.520 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-jinja2. 2024-06-28T14:57:51.547 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-06-28T14:57:51.587 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-06-28T14:57:51.940 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-06-28T14:57:51.967 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:57:52.007 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-cephadm (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:52.360 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-repoze.lru. 2024-06-28T14:57:52.387 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-06-28T14:57:52.427 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-06-28T14:57:52.713 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-routes. 2024-06-28T14:57:52.740 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-06-28T14:57:52.780 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-06-28T14:57:53.117 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-06-28T14:57:53.143 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:57:53.183 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-dashboard (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:54.335 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-06-28T14:57:54.352 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-06-28T14:57:54.385 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-06-28T14:57:54.872 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-joblib. 2024-06-28T14:57:54.899 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-06-28T14:57:54.938 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-06-28T14:57:55.309 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-06-28T14:57:55.336 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-06-28T14:57:55.384 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-06-28T14:57:55.653 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-sklearn. 2024-06-28T14:57:55.680 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-06-28T14:57:55.711 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-06-28T14:57:55.839 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-06-28T14:57:55.857 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-06-28T14:57:55.897 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-06-28T14:57:56.427 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-joblib. 2024-06-28T14:57:56.428 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-06-28T14:57:56.454 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-06-28T14:57:56.456 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:57:56.580 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:56.701 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-06-28T14:57:57.046 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-06-28T14:57:57.074 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-06-28T14:57:57.113 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-06-28T14:57:57.399 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-sklearn. 2024-06-28T14:57:57.427 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-06-28T14:57:57.465 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-06-28T14:57:57.575 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cachetools. 2024-06-28T14:57:57.603 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-06-28T14:57:57.642 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-06-28T14:57:57.937 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyasn1. 2024-06-28T14:57:57.964 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-06-28T14:57:57.995 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-06-28T14:57:58.158 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-06-28T14:57:58.186 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:57:58.298 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-06-28T14:57:58.310 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:57:58.326 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-06-28T14:57:58.364 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-06-28T14:57:58.676 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rsa. 2024-06-28T14:57:58.704 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-06-28T14:57:58.742 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rsa (4.8-1) ... 2024-06-28T14:57:59.055 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-google-auth. 2024-06-28T14:57:59.082 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-06-28T14:57:59.121 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-06-28T14:57:59.323 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-cachetools. 2024-06-28T14:57:59.350 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-06-28T14:57:59.397 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-06-28T14:57:59.399 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-06-28T14:57:59.427 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-06-28T14:57:59.466 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-06-28T14:57:59.727 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-websocket. 2024-06-28T14:57:59.755 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-06-28T14:57:59.767 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pyasn1. 2024-06-28T14:57:59.794 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-06-28T14:57:59.794 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-06-28T14:57:59.833 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-06-28T14:58:00.105 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-kubernetes. 2024-06-28T14:58:00.133 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-06-28T14:58:00.162 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-06-28T14:58:00.171 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-06-28T14:58:00.190 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-06-28T14:58:00.229 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-06-28T14:58:00.540 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-rsa. 2024-06-28T14:58:00.568 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-06-28T14:58:00.607 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-rsa (4.8-1) ... 2024-06-28T14:58:00.869 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-06-28T14:58:00.898 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:58:00.935 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:00.952 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-google-auth. 2024-06-28T14:58:00.980 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-06-28T14:58:01.052 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-06-28T14:58:01.230 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libonig5:amd64. 2024-06-28T14:58:01.258 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-06-28T14:58:01.288 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-06-28T14:58:01.373 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-06-28T14:58:01.400 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-06-28T14:58:01.439 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-06-28T14:58:01.608 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjq1:amd64. 2024-06-28T14:58:01.636 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-06-28T14:58:01.674 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-06-28T14:58:01.751 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-websocket. 2024-06-28T14:58:01.779 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-06-28T14:58:01.917 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-06-28T14:58:02.111 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package jq. 2024-06-28T14:58:02.138 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-06-28T14:58:02.177 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-06-28T14:58:02.230 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-kubernetes. 2024-06-28T14:58:02.256 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-06-28T14:58:02.304 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-06-28T14:58:02.430 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package socat. 2024-06-28T14:58:02.458 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-06-28T14:58:02.513 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-06-28T14:58:02.824 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package xmlstarlet. 2024-06-28T14:58:02.840 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-06-28T14:58:02.882 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-06-28T14:58:03.002 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-06-28T14:58:03.031 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:58:03.068 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:03.212 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-test. 2024-06-28T14:58:03.228 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../097-ceph-test_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:58:03.270 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-test (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:03.446 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libonig5:amd64. 2024-06-28T14:58:03.475 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-06-28T14:58:03.521 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-06-28T14:58:03.933 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libjq1:amd64. 2024-06-28T14:58:03.961 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-06-28T14:58:03.999 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-06-28T14:58:04.243 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package jq. 2024-06-28T14:58:04.272 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-06-28T14:58:04.310 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-06-28T14:58:04.595 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package socat. 2024-06-28T14:58:04.623 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-06-28T14:58:04.671 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-06-28T14:58:04.992 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package xmlstarlet. 2024-06-28T14:58:05.008 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-06-28T14:58:05.050 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-06-28T14:58:05.413 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-test. 2024-06-28T14:58:05.441 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../097-ceph-test_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:58:05.488 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-test (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:05.749 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-volume. 2024-06-28T14:58:05.766 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:58:05.799 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-volume (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:06.070 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcephfs-dev. 2024-06-28T14:58:06.098 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:58:06.136 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcephfs-dev (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:06.465 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua-socket:amd64. 2024-06-28T14:58:06.492 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-06-28T14:58:06.530 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-06-28T14:58:06.885 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua-sec:amd64. 2024-06-28T14:58:06.913 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-06-28T14:58:06.952 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-06-28T14:58:07.331 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package nvme-cli. 2024-06-28T14:58:07.359 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-06-28T14:58:07.397 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-06-28T14:58:07.802 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package pkg-config. 2024-06-28T14:58:07.831 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-06-28T14:58:07.859 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-volume. 2024-06-28T14:58:07.869 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-06-28T14:58:07.887 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-4592-gec2a38ba-1jammy_all.deb ... 2024-06-28T14:58:07.925 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-volume (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:08.130 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-06-28T14:58:08.146 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-06-28T14:58:08.188 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-06-28T14:58:08.263 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libcephfs-dev. 2024-06-28T14:58:08.291 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:58:08.329 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libcephfs-dev (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:08.607 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package lua-socket:amd64. 2024-06-28T14:58:08.616 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-iniconfig. 2024-06-28T14:58:08.623 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-06-28T14:58:08.644 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-06-28T14:58:08.657 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-06-28T14:58:08.682 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-06-28T14:58:08.986 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pastescript. 2024-06-28T14:58:08.987 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package lua-sec:amd64. 2024-06-28T14:58:09.013 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-06-28T14:58:09.015 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-06-28T14:58:09.052 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-06-28T14:58:09.053 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-06-28T14:58:09.347 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pluggy. 2024-06-28T14:58:09.363 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-06-28T14:58:09.398 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-06-28T14:58:09.416 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package nvme-cli. 2024-06-28T14:58:09.444 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-06-28T14:58:09.482 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-06-28T14:58:09.683 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-psutil. 2024-06-28T14:58:09.711 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-06-28T14:58:09.749 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-06-28T14:58:09.888 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package pkg-config. 2024-06-28T14:58:09.916 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-06-28T14:58:09.946 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-06-28T14:58:10.103 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-py. 2024-06-28T14:58:10.130 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-06-28T14:58:10.169 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-py (1.10.0-1) ... 2024-06-28T14:58:10.174 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-06-28T14:58:10.191 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-06-28T14:58:10.233 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-06-28T14:58:10.439 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pygments. 2024-06-28T14:58:10.467 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-06-28T14:58:10.512 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-06-28T14:58:10.619 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-iniconfig. 2024-06-28T14:58:10.647 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-06-28T14:58:10.686 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-06-28T14:58:10.985 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyinotify. 2024-06-28T14:58:10.997 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pastescript. 2024-06-28T14:58:11.013 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-06-28T14:58:11.025 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-06-28T14:58:11.051 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-06-28T14:58:11.063 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-06-28T14:58:11.384 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pluggy. 2024-06-28T14:58:11.387 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-toml. 2024-06-28T14:58:11.412 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-06-28T14:58:11.416 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-06-28T14:58:11.450 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-06-28T14:58:11.453 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-06-28T14:58:11.731 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pytest. 2024-06-28T14:58:11.737 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-psutil. 2024-06-28T14:58:11.759 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-06-28T14:58:11.765 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-06-28T14:58:11.798 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-06-28T14:58:12.054 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-06-28T14:58:12.143 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-simplejson. 2024-06-28T14:58:12.171 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-06-28T14:58:12.218 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-06-28T14:58:12.407 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-py. 2024-06-28T14:58:12.435 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-06-28T14:58:12.473 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-py (1.10.0-1) ... 2024-06-28T14:58:12.479 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-06-28T14:58:12.507 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-06-28T14:58:12.545 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-06-28T14:58:12.785 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pygments. 2024-06-28T14:58:12.814 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-06-28T14:58:12.851 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-06-28T14:58:13.241 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package radosgw. 2024-06-28T14:58:13.270 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../116-radosgw_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:58:13.281 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pyinotify. 2024-06-28T14:58:13.307 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking radosgw (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:13.309 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-06-28T14:58:13.347 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-06-28T14:58:13.634 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-toml. 2024-06-28T14:58:13.662 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-06-28T14:58:13.700 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-06-28T14:58:14.003 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pytest. 2024-06-28T14:58:14.031 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-06-28T14:58:14.069 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-06-28T14:58:14.219 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package rbd-fuse. 2024-06-28T14:58:14.248 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:58:14.285 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking rbd-fuse (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:14.407 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-simplejson. 2024-06-28T14:58:14.435 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-06-28T14:58:14.489 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-06-28T14:58:14.590 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-06-28T14:58:14.743 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-06-28T14:58:14.759 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-06-28T14:58:14.801 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-06-28T14:58:14.848 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-06-28T14:58:14.974 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-06-28T14:58:15.241 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-06-28T14:58:15.522 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package radosgw. 2024-06-28T14:58:15.550 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../116-radosgw_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:58:15.583 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-06-28T14:58:15.588 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking radosgw (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:16.022 INFO:teuthology.orchestra.run.smithi195.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-06-28T14:58:16.310 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephadm (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:16.492 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package rbd-fuse. 2024-06-28T14:58:16.520 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-4592-gec2a38ba-1jammy_amd64.deb ... 2024-06-28T14:58:16.558 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking rbd-fuse (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:16.769 INFO:teuthology.orchestra.run.smithi195.stdout:Adding system user cephadm....done 2024-06-28T14:58:16.871 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-06-28T14:58:16.924 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-06-28T14:58:17.130 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-06-28T14:58:17.201 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-06-28T14:58:17.256 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-06-28T14:58:17.477 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-06-28T14:58:17.521 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-06-28T14:58:17.603 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-06-28T14:58:17.836 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-06-28T14:58:17.854 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-06-28T14:58:18.105 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-06-28T14:58:18.223 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-py (1.10.0-1) ... 2024-06-28T14:58:18.242 INFO:teuthology.orchestra.run.smithi178.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-06-28T14:58:18.308 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up cephadm (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:18.516 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-06-28T14:58:18.743 INFO:teuthology.orchestra.run.smithi178.stdout:Adding system user cephadm....done 2024-06-28T14:58:18.862 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-06-28T14:58:18.898 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-06-28T14:58:19.104 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-06-28T14:58:19.200 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-06-28T14:58:19.389 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-06-28T14:58:19.467 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-06-28T14:58:19.593 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-06-28T14:58:19.844 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-06-28T14:58:19.924 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-06-28T14:58:20.104 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-06-28T14:58:20.166 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-06-28T14:58:20.230 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-py (1.10.0-1) ... 2024-06-28T14:58:20.409 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-06-28T14:58:20.557 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-06-28T14:58:20.752 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-ceph-argparse (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:20.918 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-06-28T14:58:21.028 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-06-28T14:58:21.129 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-06-28T14:58:21.169 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-06-28T14:58:21.230 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-06-28T14:58:21.348 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-06-28T14:58:21.429 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-06-28T14:58:21.599 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-06-28T14:58:21.697 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-06-28T14:58:21.931 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-06-28T14:58:21.976 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-06-28T14:58:22.013 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-06-28T14:58:22.077 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-06-28T14:58:22.182 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-06-28T14:58:22.212 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-06-28T14:58:22.442 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-ceph-argparse (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:22.563 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-06-28T14:58:22.664 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-06-28T14:58:22.726 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-06-28T14:58:22.940 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-06-28T14:58:23.011 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-06-28T14:58:23.139 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-06-28T14:58:23.183 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-06-28T14:58:23.240 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-06-28T14:58:23.484 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up zip (3.0-12build2) ... 2024-06-28T14:58:23.499 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-06-28T14:58:23.610 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-06-28T14:58:23.894 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-06-28T14:58:23.913 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-06-28T14:58:23.977 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-06-28T14:58:24.103 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-06-28T14:58:24.320 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-06-28T14:58:24.463 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-06-28T14:58:24.571 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-06-28T14:58:24.589 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-06-28T14:58:24.697 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-06-28T14:58:24.807 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-06-28T14:58:24.865 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-06-28T14:58:25.124 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-06-28T14:58:25.125 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-06-28T14:58:25.434 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up zip (3.0-12build2) ... 2024-06-28T14:58:25.526 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-06-28T14:58:25.560 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-06-28T14:58:25.777 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-06-28T14:58:26.187 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-06-28T14:58:26.237 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-06-28T14:58:26.464 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-06-28T14:58:26.496 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-06-28T14:58:26.622 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-06-28T14:58:26.757 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-06-28T14:58:26.798 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-06-28T14:58:27.049 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-06-28T14:58:27.049 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-06-28T14:58:27.192 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-ceph-common (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:27.493 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-06-28T14:58:27.637 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-06-28T14:58:27.811 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-06-28T14:58:27.993 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-06-28T14:58:28.361 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-06-28T14:58:28.637 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-06-28T14:58:28.964 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-06-28T14:58:28.983 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-06-28T14:58:29.206 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-06-28T14:58:29.243 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-06-28T14:58:29.333 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-ceph-common (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:29.386 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-toml (0.10.2-1) ... 2024-06-28T14:58:29.642 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-06-28T14:58:29.654 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-06-28T14:58:29.755 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-06-28T14:58:29.872 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-06-28T14:58:29.976 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-06-28T14:58:30.157 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-06-28T14:58:30.408 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-06-28T14:58:30.676 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-06-28T14:58:30.802 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-06-28T14:58:31.065 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-06-28T14:58:31.112 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-06-28T14:58:31.317 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-06-28T14:58:31.363 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-06-28T14:58:31.451 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-toml (0.10.2-1) ... 2024-06-28T14:58:31.630 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-06-28T14:58:31.710 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-06-28T14:58:31.836 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-06-28T14:58:31.865 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-06-28T14:58:31.954 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-06-28T14:58:32.124 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-06-28T14:58:32.230 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-06-28T14:58:32.384 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-06-28T14:58:32.498 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-06-28T14:58:32.510 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-06-28T14:58:32.766 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-06-28T14:58:32.803 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-06-28T14:58:32.892 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-06-28T14:58:32.929 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-06-28T14:58:33.193 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-06-28T14:58:33.213 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-06-28T14:58:33.461 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-06-28T14:58:33.514 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-06-28T14:58:33.704 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-06-28T14:58:33.757 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-06-28T14:58:33.908 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-06-28T14:58:33.971 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-06-28T14:58:34.034 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-06-28T14:58:34.123 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-06-28T14:58:34.214 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-06-28T14:58:34.386 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-06-28T14:58:34.473 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-06-28T14:58:34.495 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-06-28T14:58:34.599 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-06-28T14:58:34.875 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-06-28T14:58:34.880 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-06-28T14:58:34.986 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-06-28T14:58:35.148 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua-any (27ubuntu1) ... 2024-06-28T14:58:35.249 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-portend (3.0.0-1) ... 2024-06-28T14:58:35.269 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-06-28T14:58:35.500 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-06-28T14:58:35.570 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-06-28T14:58:35.626 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-06-28T14:58:35.821 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-06-28T14:58:35.935 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-06-28T14:58:35.956 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-06-28T14:58:36.082 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-06-28T14:58:36.171 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-06-28T14:58:36.237 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-06-28T14:58:36.408 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-06-28T14:58:36.513 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-06-28T14:58:36.534 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-06-28T14:58:36.630 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rsa (4.8-1) ... 2024-06-28T14:58:36.898 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-06-28T14:58:36.944 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-06-28T14:58:37.182 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-06-28T14:58:37.195 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up lua-any (27ubuntu1) ... 2024-06-28T14:58:37.304 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-portend (3.0.0-1) ... 2024-06-28T14:58:37.536 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-06-28T14:58:37.547 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-06-28T14:58:37.673 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-06-28T14:58:37.913 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-06-28T14:58:38.024 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-06-28T14:58:38.206 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-06-28T14:58:38.325 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-06-28T14:58:38.575 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-06-28T14:58:38.609 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-06-28T14:58:38.701 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librados2 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:38.735 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-rsa (4.8-1) ... 2024-06-28T14:58:38.827 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libsqlite3-mod-ceph (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:38.945 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-06-28T14:58:39.028 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-06-28T14:58:39.247 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcephfs2 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:39.297 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-06-28T14:58:39.373 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libradosstriper1 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:39.499 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-06-28T14:58:39.624 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-06-28T14:58:39.776 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librbd1 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:39.902 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-modules-core (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:40.017 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-06-28T14:58:40.028 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-fuse (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:40.250 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-06-28T14:58:40.250 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-06-28T14:58:40.319 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-06-28T14:58:40.672 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-06-28T14:58:40.738 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcephfs-dev (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:40.797 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up librados2 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:40.855 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rados (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:40.923 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libsqlite3-mod-ceph (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:40.981 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librgw2 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:41.049 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-06-28T14:58:41.107 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-06-28T14:58:41.469 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libcephfs2 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:41.620 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libradosstriper1 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:41.754 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-06-28T14:58:42.039 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up librbd1 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:42.165 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-modules-core (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:42.291 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-fuse (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:42.303 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rbd (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:42.412 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up rbd-fuse (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:42.501 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-06-28T14:58:42.502 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-06-28T14:58:42.513 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rgw (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:42.639 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cephfs (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:42.765 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-common (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:42.951 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libcephfs-dev (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:43.077 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-rados (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:43.202 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up librgw2 (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:43.308 INFO:teuthology.orchestra.run.smithi195.stdout:Adding group ceph....done 2024-06-28T14:58:43.328 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-06-28T14:58:43.693 INFO:teuthology.orchestra.run.smithi195.stdout:Adding system user ceph....done 2024-06-28T14:58:43.879 INFO:teuthology.orchestra.run.smithi195.stdout:Setting system user ceph properties....done 2024-06-28T14:58:43.892 INFO:teuthology.orchestra.run.smithi195.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-06-28T14:58:44.030 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-06-28T14:58:44.372 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-06-28T14:58:44.482 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-rbd (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:44.633 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up rbd-fuse (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:44.784 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-rgw (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:44.834 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-test (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:44.935 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-cephfs (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:44.959 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up radosgw (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:45.086 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-common (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:45.434 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-06-28T14:58:45.434 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-06-28T14:58:45.604 INFO:teuthology.orchestra.run.smithi178.stdout:Adding group ceph....done 2024-06-28T14:58:45.895 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-base (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:45.955 INFO:teuthology.orchestra.run.smithi178.stdout:Adding system user ceph....done 2024-06-28T14:58:46.124 INFO:teuthology.orchestra.run.smithi178.stdout:Setting system user ceph properties....done 2024-06-28T14:58:46.140 INFO:teuthology.orchestra.run.smithi178.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-06-28T14:58:46.232 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-06-28T14:58:46.274 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-06-28T14:58:46.574 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-06-28T14:58:46.688 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mds (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:46.941 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-06-28T14:58:46.941 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-06-28T14:58:47.037 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-test (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:47.147 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up radosgw (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:47.407 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:47.601 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-06-28T14:58:47.601 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-06-28T14:58:47.621 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-06-28T14:58:47.621 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-06-28T14:58:48.024 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-base (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:48.108 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-osd (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:48.393 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-06-28T14:58:48.393 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-06-28T14:58:48.478 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-06-28T14:58:48.893 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-k8sevents (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:48.918 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mds (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:49.027 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:49.178 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mon (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:49.232 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-06-28T14:58:49.232 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-06-28T14:58:49.392 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-06-28T14:58:49.393 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-06-28T14:58:49.687 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:49.854 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-cephadm (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:49.900 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-06-28T14:58:49.900 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-06-28T14:58:49.989 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:50.115 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-dashboard (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:50.266 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-volume (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:50.346 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-osd (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:50.595 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-06-28T14:58:50.596 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-06-28T14:58:50.620 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-06-28T14:58:50.771 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-06-28T14:58:51.040 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-06-28T14:58:51.048 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-k8sevents (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:51.188 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:51.341 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mon (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:51.555 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-06-28T14:58:51.556 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-06-28T14:58:51.993 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-cephadm (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:52.144 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:52.249 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-06-28T14:58:52.269 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-dashboard (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:52.421 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-volume (19.0.0-4592-gec2a38ba-1jammy) ... 2024-06-28T14:58:52.782 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-06-28T14:58:52.874 INFO:teuthology.orchestra.run.smithi195.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-06-28T14:58:52.875 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-06-28T14:58:52.877 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install s3cmd maven 2024-06-28T14:58:52.940 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-06-28T14:58:53.026 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-06-28T14:58:53.143 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-06-28T14:58:53.144 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-06-28T14:58:53.327 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-06-28T14:58:53.327 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-06-28T14:58:53.328 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-06-28T14:58:53.328 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-06-28T14:58:53.328 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-06-28T14:58:53.328 INFO:teuthology.orchestra.run.smithi195.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-06-28T14:58:53.329 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-06-28T14:58:53.330 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2024-06-28T14:58:53.330 INFO:teuthology.orchestra.run.smithi195.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-06-28T14:58:53.330 INFO:teuthology.orchestra.run.smithi195.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-06-28T14:58:53.330 INFO:teuthology.orchestra.run.smithi195.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-06-28T14:58:53.330 INFO:teuthology.orchestra.run.smithi195.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-06-28T14:58:53.330 INFO:teuthology.orchestra.run.smithi195.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-06-28T14:58:53.330 INFO:teuthology.orchestra.run.smithi195.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-06-28T14:58:53.330 INFO:teuthology.orchestra.run.smithi195.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-06-28T14:58:53.330 INFO:teuthology.orchestra.run.smithi195.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-06-28T14:58:53.330 INFO:teuthology.orchestra.run.smithi195.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-06-28T14:58:53.331 INFO:teuthology.orchestra.run.smithi195.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-06-28T14:58:53.331 INFO:teuthology.orchestra.run.smithi195.stdout: libwagon-http-shaded-java libwagon-provider-api-java python3-magic 2024-06-28T14:58:53.331 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2024-06-28T14:58:53.331 INFO:teuthology.orchestra.run.smithi195.stdout: libaopalliance-java-doc libatinject-jsr330-api-java-doc libel-api-java 2024-06-28T14:58:53.331 INFO:teuthology.orchestra.run.smithi195.stdout: libcommons-io-java-doc libcommons-lang3-java-doc libasm-java libcglib-java 2024-06-28T14:58:53.331 INFO:teuthology.orchestra.run.smithi195.stdout: libjsr305-java-doc libmaven-shared-utils-java-doc liblogback-java 2024-06-28T14:58:53.331 INFO:teuthology.orchestra.run.smithi195.stdout: libplexus-classworlds-java-doc libplexus-sec-dispatcher-java-doc 2024-06-28T14:58:53.331 INFO:teuthology.orchestra.run.smithi195.stdout: libplexus-utils2-java-doc testng libcommons-logging-java liblog4j1.2-java 2024-06-28T14:58:53.370 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2024-06-28T14:58:53.370 INFO:teuthology.orchestra.run.smithi195.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-06-28T14:58:53.370 INFO:teuthology.orchestra.run.smithi195.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-06-28T14:58:53.370 INFO:teuthology.orchestra.run.smithi195.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-06-28T14:58:53.370 INFO:teuthology.orchestra.run.smithi195.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-06-28T14:58:53.370 INFO:teuthology.orchestra.run.smithi195.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-06-28T14:58:53.370 INFO:teuthology.orchestra.run.smithi195.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-06-28T14:58:53.370 INFO:teuthology.orchestra.run.smithi195.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-06-28T14:58:53.370 INFO:teuthology.orchestra.run.smithi195.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-06-28T14:58:53.370 INFO:teuthology.orchestra.run.smithi195.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-06-28T14:58:53.370 INFO:teuthology.orchestra.run.smithi195.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-06-28T14:58:53.371 INFO:teuthology.orchestra.run.smithi195.stdout: libwagon-http-shaded-java libwagon-provider-api-java maven python3-magic 2024-06-28T14:58:53.371 INFO:teuthology.orchestra.run.smithi195.stdout: s3cmd 2024-06-28T14:58:53.458 INFO:teuthology.orchestra.run.smithi195.stdout:0 upgraded, 35 newly installed, 0 to remove and 315 not upgraded. 2024-06-28T14:58:53.458 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 10.4 MB of archives. 2024-06-28T14:58:53.459 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 13.9 MB of additional disk space will be used. 2024-06-28T14:58:53.459 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libapache-pom-java all 18-1 [4,720 B] 2024-06-28T14:58:53.489 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libatinject-jsr330-api-java all 1.0+ds1-5 [5,348 B] 2024-06-28T14:58:53.489 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-interceptor-3.0-spec-java all 1.0.1-4fakesync [8,616 B] 2024-06-28T14:58:53.497 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcdi-api-java all 1.2-3 [54.3 kB] 2024-06-28T14:58:53.589 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-cli-java all 1.4-2 [55.8 kB] 2024-06-28T14:58:53.658 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-parent-java all 43-1 [10.8 kB] 2024-06-28T14:58:53.659 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-io-java all 2.11.0-2 [297 kB] 2024-06-28T14:58:53.744 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-lang3-java all 3.11-1 [526 kB] 2024-06-28T14:58:53.787 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-annotation-1.3-spec-java all 1.3-1 [11.2 kB] 2024-06-28T14:58:53.788 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjsr305-java all 0.1~+svn49-11 [27.0 kB] 2024-06-28T14:58:53.789 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguava-java all 29.0-6 [2,418 kB] 2024-06-28T14:58:53.835 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-06-28T14:58:53.877 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libaopalliance-java all 20070526-6 [9,084 B] 2024-06-28T14:58:53.877 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguice-java all 4.2.3-2 [1,434 kB] 2024-06-28T14:58:53.908 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libhawtjni-runtime-java all 1.17-1 [28.8 kB] 2024-06-28T14:58:53.909 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-native-java all 1.8-1 [23.8 kB] 2024-06-28T14:58:53.910 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-java all 1.18-1 [56.8 kB] 2024-06-28T14:58:53.911 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-parent-java all 31-2 [5,140 B] 2024-06-28T14:58:53.912 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-utils2-java all 3.3.0-1 [250 kB] 2024-06-28T14:58:53.917 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-provider-api-java all 3.3.4-1 [48.5 kB] 2024-06-28T14:58:53.918 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-resolver-java all 1.4.2-3build1 [555 kB] 2024-06-28T14:58:53.940 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libmaven-shared-utils-java all 3.3.0-1ubuntu0.22.04.1 [148 kB] 2024-06-28T14:58:53.967 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-cipher-java all 1.8-2 [15.1 kB] 2024-06-28T14:58:53.968 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-classworlds-java all 2.6.0-1 [49.4 kB] 2024-06-28T14:58:53.969 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-component-annotations-java all 2.1.0-1 [6,564 B] 2024-06-28T14:58:53.969 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-interpolation-java all 1.26-1 [76.8 kB] 2024-06-28T14:58:53.971 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-sec-dispatcher-java all 1.4-4 [28.1 kB] 2024-06-28T14:58:53.972 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libslf4j-java all 1.7.32-1 [141 kB] 2024-06-28T14:58:53.974 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-inject-java all 0.3.4-2 [347 kB] 2024-06-28T14:58:53.987 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-plexus-java all 0.3.4-3 [181 kB] 2024-06-28T14:58:53.990 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven3-core-java all 3.6.3-5 [1,535 kB] 2024-06-28T14:58:54.030 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-file-java all 3.3.4-1 [7,928 B] 2024-06-28T14:58:54.030 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-http-shaded-java all 3.3.4-1 [1,855 kB] 2024-06-28T14:58:54.057 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 maven all 3.6.3-5 [17.8 kB] 2024-06-28T14:58:54.057 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-magic all 2:0.4.24-2 [12.6 kB] 2024-06-28T14:58:54.057 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/universe amd64 s3cmd all 2.2.0-1 [120 kB] 2024-06-28T14:58:54.499 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 10.4 MB in 1s (15.2 MB/s) 2024-06-28T14:58:54.504 INFO:teuthology.orchestra.run.smithi178.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-06-28T14:58:54.507 DEBUG:teuthology.orchestra.run.smithi178:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install s3cmd maven 2024-06-28T14:58:54.579 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libapache-pom-java. 2024-06-28T14:58:54.608 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-06-28T14:58:54.654 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176228 files and directories currently installed.) 2024-06-28T14:58:54.659 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../00-libapache-pom-java_18-1_all.deb ... 2024-06-28T14:58:54.703 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libapache-pom-java (18-1) ... 2024-06-28T14:58:54.812 INFO:teuthology.orchestra.run.smithi178.stdout:Building dependency tree... 2024-06-28T14:58:54.813 INFO:teuthology.orchestra.run.smithi178.stdout:Reading state information... 2024-06-28T14:58:54.964 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libatinject-jsr330-api-java. 2024-06-28T14:58:54.980 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../01-libatinject-jsr330-api-java_1.0+ds1-5_all.deb ... 2024-06-28T14:58:55.004 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages were automatically installed and are no longer required: 2024-06-28T14:58:55.004 INFO:teuthology.orchestra.run.smithi178.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-06-28T14:58:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-06-28T14:58:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-06-28T14:58:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-06-28T14:58:55.005 INFO:teuthology.orchestra.run.smithi178.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-06-28T14:58:55.005 INFO:teuthology.orchestra.run.smithi178.stdout:Use 'sudo apt autoremove' to remove them. 2024-06-28T14:58:55.007 INFO:teuthology.orchestra.run.smithi178.stdout:The following additional packages will be installed: 2024-06-28T14:58:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-06-28T14:58:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-06-28T14:58:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-06-28T14:58:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-06-28T14:58:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-06-28T14:58:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-06-28T14:58:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-06-28T14:58:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-06-28T14:58:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-06-28T14:58:55.007 INFO:teuthology.orchestra.run.smithi178.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-06-28T14:58:55.008 INFO:teuthology.orchestra.run.smithi178.stdout: libwagon-http-shaded-java libwagon-provider-api-java python3-magic 2024-06-28T14:58:55.008 INFO:teuthology.orchestra.run.smithi178.stdout:Suggested packages: 2024-06-28T14:58:55.008 INFO:teuthology.orchestra.run.smithi178.stdout: libaopalliance-java-doc libatinject-jsr330-api-java-doc libel-api-java 2024-06-28T14:58:55.008 INFO:teuthology.orchestra.run.smithi178.stdout: libcommons-io-java-doc libcommons-lang3-java-doc libasm-java libcglib-java 2024-06-28T14:58:55.008 INFO:teuthology.orchestra.run.smithi178.stdout: libjsr305-java-doc libmaven-shared-utils-java-doc liblogback-java 2024-06-28T14:58:55.008 INFO:teuthology.orchestra.run.smithi178.stdout: libplexus-classworlds-java-doc libplexus-sec-dispatcher-java-doc 2024-06-28T14:58:55.008 INFO:teuthology.orchestra.run.smithi178.stdout: libplexus-utils2-java-doc testng libcommons-logging-java liblog4j1.2-java 2024-06-28T14:58:55.022 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-06-28T14:58:55.046 INFO:teuthology.orchestra.run.smithi178.stdout:The following NEW packages will be installed: 2024-06-28T14:58:55.046 INFO:teuthology.orchestra.run.smithi178.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-06-28T14:58:55.046 INFO:teuthology.orchestra.run.smithi178.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-06-28T14:58:55.046 INFO:teuthology.orchestra.run.smithi178.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-06-28T14:58:55.046 INFO:teuthology.orchestra.run.smithi178.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-06-28T14:58:55.046 INFO:teuthology.orchestra.run.smithi178.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-06-28T14:58:55.046 INFO:teuthology.orchestra.run.smithi178.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-06-28T14:58:55.046 INFO:teuthology.orchestra.run.smithi178.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-06-28T14:58:55.046 INFO:teuthology.orchestra.run.smithi178.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-06-28T14:58:55.046 INFO:teuthology.orchestra.run.smithi178.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-06-28T14:58:55.047 INFO:teuthology.orchestra.run.smithi178.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-06-28T14:58:55.047 INFO:teuthology.orchestra.run.smithi178.stdout: libwagon-http-shaded-java libwagon-provider-api-java maven python3-magic 2024-06-28T14:58:55.047 INFO:teuthology.orchestra.run.smithi178.stdout: s3cmd 2024-06-28T14:58:55.227 INFO:teuthology.orchestra.run.smithi178.stdout:0 upgraded, 35 newly installed, 0 to remove and 315 not upgraded. 2024-06-28T14:58:55.227 INFO:teuthology.orchestra.run.smithi178.stdout:Need to get 10.4 MB of archives. 2024-06-28T14:58:55.227 INFO:teuthology.orchestra.run.smithi178.stdout:After this operation, 13.9 MB of additional disk space will be used. 2024-06-28T14:58:55.227 INFO:teuthology.orchestra.run.smithi178.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libapache-pom-java all 18-1 [4,720 B] 2024-06-28T14:58:55.300 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libgeronimo-interceptor-3.0-spec-java. 2024-06-28T14:58:55.309 INFO:teuthology.orchestra.run.smithi178.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libatinject-jsr330-api-java all 1.0+ds1-5 [5,348 B] 2024-06-28T14:58:55.309 INFO:teuthology.orchestra.run.smithi178.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-interceptor-3.0-spec-java all 1.0.1-4fakesync [8,616 B] 2024-06-28T14:58:55.326 INFO:teuthology.orchestra.run.smithi178.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcdi-api-java all 1.2-3 [54.3 kB] 2024-06-28T14:58:55.327 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../02-libgeronimo-interceptor-3.0-spec-java_1.0.1-4fakesync_all.deb ... 2024-06-28T14:58:55.374 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-06-28T14:58:55.490 INFO:teuthology.orchestra.run.smithi178.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-cli-java all 1.4-2 [55.8 kB] 2024-06-28T14:58:55.560 INFO:teuthology.orchestra.run.smithi178.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-parent-java all 43-1 [10.8 kB] 2024-06-28T14:58:55.568 INFO:teuthology.orchestra.run.smithi178.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-io-java all 2.11.0-2 [297 kB] 2024-06-28T14:58:55.702 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-lang3-java all 3.11-1 [526 kB] 2024-06-28T14:58:55.745 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcdi-api-java. 2024-06-28T14:58:55.773 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../03-libcdi-api-java_1.2-3_all.deb ... 2024-06-28T14:58:55.794 INFO:teuthology.orchestra.run.smithi178.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-annotation-1.3-spec-java all 1.3-1 [11.2 kB] 2024-06-28T14:58:55.795 INFO:teuthology.orchestra.run.smithi178.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjsr305-java all 0.1~+svn49-11 [27.0 kB] 2024-06-28T14:58:55.799 INFO:teuthology.orchestra.run.smithi178.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguava-java all 29.0-6 [2,418 kB] 2024-06-28T14:58:55.820 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcdi-api-java (1.2-3) ... 2024-06-28T14:58:55.943 INFO:teuthology.orchestra.run.smithi178.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libaopalliance-java all 20070526-6 [9,084 B] 2024-06-28T14:58:55.943 INFO:teuthology.orchestra.run.smithi178.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguice-java all 4.2.3-2 [1,434 kB] 2024-06-28T14:58:55.983 INFO:teuthology.orchestra.run.smithi178.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libhawtjni-runtime-java all 1.17-1 [28.8 kB] 2024-06-28T14:58:55.984 INFO:teuthology.orchestra.run.smithi178.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-native-java all 1.8-1 [23.8 kB] 2024-06-28T14:58:55.985 INFO:teuthology.orchestra.run.smithi178.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-java all 1.18-1 [56.8 kB] 2024-06-28T14:58:55.987 INFO:teuthology.orchestra.run.smithi178.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-parent-java all 31-2 [5,140 B] 2024-06-28T14:58:55.987 INFO:teuthology.orchestra.run.smithi178.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-utils2-java all 3.3.0-1 [250 kB] 2024-06-28T14:58:55.993 INFO:teuthology.orchestra.run.smithi178.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-provider-api-java all 3.3.4-1 [48.5 kB] 2024-06-28T14:58:55.994 INFO:teuthology.orchestra.run.smithi178.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-resolver-java all 1.4.2-3build1 [555 kB] 2024-06-28T14:58:56.040 INFO:teuthology.orchestra.run.smithi178.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libmaven-shared-utils-java all 3.3.0-1ubuntu0.22.04.1 [148 kB] 2024-06-28T14:58:56.043 INFO:teuthology.orchestra.run.smithi178.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-cipher-java all 1.8-2 [15.1 kB] 2024-06-28T14:58:56.087 INFO:teuthology.orchestra.run.smithi178.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-classworlds-java all 2.6.0-1 [49.4 kB] 2024-06-28T14:58:56.088 INFO:teuthology.orchestra.run.smithi178.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-component-annotations-java all 2.1.0-1 [6,564 B] 2024-06-28T14:58:56.089 INFO:teuthology.orchestra.run.smithi178.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-interpolation-java all 1.26-1 [76.8 kB] 2024-06-28T14:58:56.091 INFO:teuthology.orchestra.run.smithi178.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-sec-dispatcher-java all 1.4-4 [28.1 kB] 2024-06-28T14:58:56.092 INFO:teuthology.orchestra.run.smithi178.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libslf4j-java all 1.7.32-1 [141 kB] 2024-06-28T14:58:56.096 INFO:teuthology.orchestra.run.smithi178.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-inject-java all 0.3.4-2 [347 kB] 2024-06-28T14:58:56.105 INFO:teuthology.orchestra.run.smithi178.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-plexus-java all 0.3.4-3 [181 kB] 2024-06-28T14:58:56.122 INFO:teuthology.orchestra.run.smithi178.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven3-core-java all 3.6.3-5 [1,535 kB] 2024-06-28T14:58:56.131 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcommons-cli-java. 2024-06-28T14:58:56.159 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../04-libcommons-cli-java_1.4-2_all.deb ... 2024-06-28T14:58:56.203 INFO:teuthology.orchestra.run.smithi178.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-file-java all 3.3.4-1 [7,928 B] 2024-06-28T14:58:56.203 INFO:teuthology.orchestra.run.smithi178.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-http-shaded-java all 3.3.4-1 [1,855 kB] 2024-06-28T14:58:56.206 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcommons-cli-java (1.4-2) ... 2024-06-28T14:58:56.238 INFO:teuthology.orchestra.run.smithi178.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 maven all 3.6.3-5 [17.8 kB] 2024-06-28T14:58:56.239 INFO:teuthology.orchestra.run.smithi178.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-magic all 2:0.4.24-2 [12.6 kB] 2024-06-28T14:58:56.239 INFO:teuthology.orchestra.run.smithi178.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/universe amd64 s3cmd all 2.2.0-1 [120 kB] 2024-06-28T14:58:56.483 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcommons-parent-java. 2024-06-28T14:58:56.499 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../05-libcommons-parent-java_43-1_all.deb ... 2024-06-28T14:58:56.541 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcommons-parent-java (43-1) ... 2024-06-28T14:58:56.682 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 10.4 MB in 1s (8,737 kB/s) 2024-06-28T14:58:56.765 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libapache-pom-java. 2024-06-28T14:58:56.802 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcommons-io-java. 2024-06-28T14:58:56.830 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../06-libcommons-io-java_2.11.0-2_all.deb ... 2024-06-28T14:58:56.846 INFO:teuthology.orchestra.run.smithi178.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176228 files and directories currently installed.) 2024-06-28T14:58:56.852 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../00-libapache-pom-java_18-1_all.deb ... 2024-06-28T14:58:56.869 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcommons-io-java (2.11.0-2) ... 2024-06-28T14:58:56.889 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libapache-pom-java (18-1) ... 2024-06-28T14:58:57.150 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libatinject-jsr330-api-java. 2024-06-28T14:58:57.178 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../01-libatinject-jsr330-api-java_1.0+ds1-5_all.deb ... 2024-06-28T14:58:57.180 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcommons-lang3-java. 2024-06-28T14:58:57.208 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../07-libcommons-lang3-java_3.11-1_all.deb ... 2024-06-28T14:58:57.216 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-06-28T14:58:57.255 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcommons-lang3-java (3.11-1) ... 2024-06-28T14:58:57.469 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libgeronimo-interceptor-3.0-spec-java. 2024-06-28T14:58:57.486 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../02-libgeronimo-interceptor-3.0-spec-java_1.0.1-4fakesync_all.deb ... 2024-06-28T14:58:57.527 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-06-28T14:58:57.599 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libgeronimo-annotation-1.3-spec-java. 2024-06-28T14:58:57.634 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../08-libgeronimo-annotation-1.3-spec-java_1.3-1_all.deb ... 2024-06-28T14:58:57.674 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-06-28T14:58:57.772 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libcdi-api-java. 2024-06-28T14:58:57.800 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../03-libcdi-api-java_1.2-3_all.deb ... 2024-06-28T14:58:57.880 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libcdi-api-java (1.2-3) ... 2024-06-28T14:58:57.934 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjsr305-java. 2024-06-28T14:58:57.951 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../09-libjsr305-java_0.1~+svn49-11_all.deb ... 2024-06-28T14:58:57.993 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjsr305-java (0.1~+svn49-11) ... 2024-06-28T14:58:58.266 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libcommons-cli-java. 2024-06-28T14:58:58.283 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../04-libcommons-cli-java_1.4-2_all.deb ... 2024-06-28T14:58:58.324 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libcommons-cli-java (1.4-2) ... 2024-06-28T14:58:58.329 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libguava-java. 2024-06-28T14:58:58.358 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../10-libguava-java_29.0-6_all.deb ... 2024-06-28T14:58:58.396 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libguava-java (29.0-6) ... 2024-06-28T14:58:58.882 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libaopalliance-java. 2024-06-28T14:58:58.910 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../11-libaopalliance-java_20070526-6_all.deb ... 2024-06-28T14:58:58.957 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libaopalliance-java (20070526-6) ... 2024-06-28T14:58:59.161 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libcommons-parent-java. 2024-06-28T14:58:59.189 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../05-libcommons-parent-java_43-1_all.deb ... 2024-06-28T14:58:59.228 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libcommons-parent-java (43-1) ... 2024-06-28T14:58:59.259 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libguice-java. 2024-06-28T14:58:59.288 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../12-libguice-java_4.2.3-2_all.deb ... 2024-06-28T14:58:59.326 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libguice-java (4.2.3-2) ... 2024-06-28T14:58:59.489 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libcommons-io-java. 2024-06-28T14:58:59.505 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../06-libcommons-io-java_2.11.0-2_all.deb ... 2024-06-28T14:58:59.547 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libcommons-io-java (2.11.0-2) ... 2024-06-28T14:58:59.705 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libhawtjni-runtime-java. 2024-06-28T14:58:59.721 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../13-libhawtjni-runtime-java_1.17-1_all.deb ... 2024-06-28T14:58:59.763 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libhawtjni-runtime-java (1.17-1) ... 2024-06-28T14:58:59.774 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libcommons-lang3-java. 2024-06-28T14:58:59.802 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../07-libcommons-lang3-java_3.11-1_all.deb ... 2024-06-28T14:58:59.832 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libcommons-lang3-java (3.11-1) ... 2024-06-28T14:59:00.016 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjansi-native-java. 2024-06-28T14:59:00.051 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../14-libjansi-native-java_1.8-1_all.deb ... 2024-06-28T14:59:00.099 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjansi-native-java (1.8-1) ... 2024-06-28T14:59:00.118 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libgeronimo-annotation-1.3-spec-java. 2024-06-28T14:59:00.146 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../08-libgeronimo-annotation-1.3-spec-java_1.3-1_all.deb ... 2024-06-28T14:59:00.185 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-06-28T14:59:00.395 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libjsr305-java. 2024-06-28T14:59:00.402 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjansi-java. 2024-06-28T14:59:00.412 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../09-libjsr305-java_0.1~+svn49-11_all.deb ... 2024-06-28T14:59:00.430 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../15-libjansi-java_1.18-1_all.deb ... 2024-06-28T14:59:00.462 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libjsr305-java (0.1~+svn49-11) ... 2024-06-28T14:59:00.485 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjansi-java (1.18-1) ... 2024-06-28T14:59:00.715 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libguava-java. 2024-06-28T14:59:00.731 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../10-libguava-java_29.0-6_all.deb ... 2024-06-28T14:59:00.773 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libguava-java (29.0-6) ... 2024-06-28T14:59:00.848 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libmaven-parent-java. 2024-06-28T14:59:00.876 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../16-libmaven-parent-java_31-2_all.deb ... 2024-06-28T14:59:00.906 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libmaven-parent-java (31-2) ... 2024-06-28T14:59:01.141 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libplexus-utils2-java. 2024-06-28T14:59:01.158 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../17-libplexus-utils2-java_3.3.0-1_all.deb ... 2024-06-28T14:59:01.200 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libplexus-utils2-java (3.3.0-1) ... 2024-06-28T14:59:01.201 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libaopalliance-java. 2024-06-28T14:59:01.229 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../11-libaopalliance-java_20070526-6_all.deb ... 2024-06-28T14:59:01.267 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libaopalliance-java (20070526-6) ... 2024-06-28T14:59:01.486 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libwagon-provider-api-java. 2024-06-28T14:59:01.502 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../18-libwagon-provider-api-java_3.3.4-1_all.deb ... 2024-06-28T14:59:01.511 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libguice-java. 2024-06-28T14:59:01.528 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../12-libguice-java_4.2.3-2_all.deb ... 2024-06-28T14:59:01.552 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libwagon-provider-api-java (3.3.4-1) ... 2024-06-28T14:59:01.570 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libguice-java (4.2.3-2) ... 2024-06-28T14:59:01.797 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libmaven-resolver-java. 2024-06-28T14:59:01.813 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../19-libmaven-resolver-java_1.4.2-3build1_all.deb ... 2024-06-28T14:59:01.847 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libmaven-resolver-java (1.4.2-3build1) ... 2024-06-28T14:59:01.965 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libhawtjni-runtime-java. 2024-06-28T14:59:01.982 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../13-libhawtjni-runtime-java_1.17-1_all.deb ... 2024-06-28T14:59:02.024 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libhawtjni-runtime-java (1.17-1) ... 2024-06-28T14:59:02.168 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libmaven-shared-utils-java. 2024-06-28T14:59:02.184 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../20-libmaven-shared-utils-java_3.3.0-1ubuntu0.22.04.1_all.deb ... 2024-06-28T14:59:02.226 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-06-28T14:59:02.276 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libjansi-native-java. 2024-06-28T14:59:02.292 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../14-libjansi-native-java_1.8-1_all.deb ... 2024-06-28T14:59:02.334 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libjansi-native-java (1.8-1) ... 2024-06-28T14:59:02.528 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libplexus-cipher-java. 2024-06-28T14:59:02.557 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../21-libplexus-cipher-java_1.8-2_all.deb ... 2024-06-28T14:59:02.570 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libjansi-java. 2024-06-28T14:59:02.586 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../15-libjansi-java_1.18-1_all.deb ... 2024-06-28T14:59:02.604 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libplexus-cipher-java (1.8-2) ... 2024-06-28T14:59:02.628 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libjansi-java (1.18-1) ... 2024-06-28T14:59:02.864 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libmaven-parent-java. 2024-06-28T14:59:02.880 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../16-libmaven-parent-java_31-2_all.deb ... 2024-06-28T14:59:02.889 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libplexus-classworlds-java. 2024-06-28T14:59:02.914 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libmaven-parent-java (31-2) ... 2024-06-28T14:59:02.917 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../22-libplexus-classworlds-java_2.6.0-1_all.deb ... 2024-06-28T14:59:02.956 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libplexus-classworlds-java (2.6.0-1) ... 2024-06-28T14:59:03.133 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libplexus-utils2-java. 2024-06-28T14:59:03.149 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../17-libplexus-utils2-java_3.3.0-1_all.deb ... 2024-06-28T14:59:03.191 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libplexus-utils2-java (3.3.0-1) ... 2024-06-28T14:59:03.208 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libplexus-component-annotations-java. 2024-06-28T14:59:03.237 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../23-libplexus-component-annotations-java_2.1.0-1_all.deb ... 2024-06-28T14:59:03.292 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libplexus-component-annotations-java (2.1.0-1) ... 2024-06-28T14:59:03.485 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libwagon-provider-api-java. 2024-06-28T14:59:03.501 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../18-libwagon-provider-api-java_3.3.4-1_all.deb ... 2024-06-28T14:59:03.544 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libwagon-provider-api-java (3.3.4-1) ... 2024-06-28T14:59:03.561 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libplexus-interpolation-java. 2024-06-28T14:59:03.577 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../24-libplexus-interpolation-java_1.26-1_all.deb ... 2024-06-28T14:59:03.628 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libplexus-interpolation-java (1.26-1) ... 2024-06-28T14:59:03.813 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libmaven-resolver-java. 2024-06-28T14:59:03.830 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../19-libmaven-resolver-java_1.4.2-3build1_all.deb ... 2024-06-28T14:59:03.871 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libmaven-resolver-java (1.4.2-3build1) ... 2024-06-28T14:59:03.930 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libplexus-sec-dispatcher-java. 2024-06-28T14:59:03.958 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../25-libplexus-sec-dispatcher-java_1.4-4_all.deb ... 2024-06-28T14:59:04.005 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libplexus-sec-dispatcher-java (1.4-4) ... 2024-06-28T14:59:04.224 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libslf4j-java. 2024-06-28T14:59:04.240 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../26-libslf4j-java_1.7.32-1_all.deb ... 2024-06-28T14:59:04.276 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libmaven-shared-utils-java. 2024-06-28T14:59:04.282 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libslf4j-java (1.7.32-1) ... 2024-06-28T14:59:04.304 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../20-libmaven-shared-utils-java_3.3.0-1ubuntu0.22.04.1_all.deb ... 2024-06-28T14:59:04.343 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-06-28T14:59:04.561 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libsisu-inject-java. 2024-06-28T14:59:04.578 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../27-libsisu-inject-java_0.3.4-2_all.deb ... 2024-06-28T14:59:04.603 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libplexus-cipher-java. 2024-06-28T14:59:04.620 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libsisu-inject-java (0.3.4-2) ... 2024-06-28T14:59:04.620 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../21-libplexus-cipher-java_1.8-2_all.deb ... 2024-06-28T14:59:04.662 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libplexus-cipher-java (1.8-2) ... 2024-06-28T14:59:04.914 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libsisu-plexus-java. 2024-06-28T14:59:04.915 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libplexus-classworlds-java. 2024-06-28T14:59:04.942 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../28-libsisu-plexus-java_0.3.4-3_all.deb ... 2024-06-28T14:59:04.943 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../22-libplexus-classworlds-java_2.6.0-1_all.deb ... 2024-06-28T14:59:04.980 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libsisu-plexus-java (0.3.4-3) ... 2024-06-28T14:59:04.981 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libplexus-classworlds-java (2.6.0-1) ... 2024-06-28T14:59:05.242 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libmaven3-core-java. 2024-06-28T14:59:05.242 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libplexus-component-annotations-java. 2024-06-28T14:59:05.258 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../23-libplexus-component-annotations-java_2.1.0-1_all.deb ... 2024-06-28T14:59:05.258 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../29-libmaven3-core-java_3.6.3-5_all.deb ... 2024-06-28T14:59:05.291 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libmaven3-core-java (3.6.3-5) ... 2024-06-28T14:59:05.300 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libplexus-component-annotations-java (2.1.0-1) ... 2024-06-28T14:59:05.544 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libplexus-interpolation-java. 2024-06-28T14:59:05.572 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../24-libplexus-interpolation-java_1.26-1_all.deb ... 2024-06-28T14:59:05.610 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libplexus-interpolation-java (1.26-1) ... 2024-06-28T14:59:05.671 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libwagon-file-java. 2024-06-28T14:59:05.699 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../30-libwagon-file-java_3.3.4-1_all.deb ... 2024-06-28T14:59:05.737 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libwagon-file-java (3.3.4-1) ... 2024-06-28T14:59:05.880 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libplexus-sec-dispatcher-java. 2024-06-28T14:59:05.908 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../25-libplexus-sec-dispatcher-java_1.4-4_all.deb ... 2024-06-28T14:59:05.938 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libplexus-sec-dispatcher-java (1.4-4) ... 2024-06-28T14:59:06.032 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libwagon-http-shaded-java. 2024-06-28T14:59:06.048 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../31-libwagon-http-shaded-java_3.3.4-1_all.deb ... 2024-06-28T14:59:06.090 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libwagon-http-shaded-java (3.3.4-1) ... 2024-06-28T14:59:06.165 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libslf4j-java. 2024-06-28T14:59:06.182 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../26-libslf4j-java_1.7.32-1_all.deb ... 2024-06-28T14:59:06.224 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libslf4j-java (1.7.32-1) ... 2024-06-28T14:59:06.494 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libsisu-inject-java. 2024-06-28T14:59:06.522 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../27-libsisu-inject-java_0.3.4-2_all.deb ... 2024-06-28T14:59:06.551 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package maven. 2024-06-28T14:59:06.560 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libsisu-inject-java (0.3.4-2) ... 2024-06-28T14:59:06.579 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../32-maven_3.6.3-5_all.deb ... 2024-06-28T14:59:06.618 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking maven (3.6.3-5) ... 2024-06-28T14:59:06.855 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libsisu-plexus-java. 2024-06-28T14:59:06.883 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../28-libsisu-plexus-java_0.3.4-3_all.deb ... 2024-06-28T14:59:06.896 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-magic. 2024-06-28T14:59:06.922 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libsisu-plexus-java (0.3.4-3) ... 2024-06-28T14:59:06.924 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../33-python3-magic_2%3a0.4.24-2_all.deb ... 2024-06-28T14:59:06.963 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-magic (2:0.4.24-2) ... 2024-06-28T14:59:07.241 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libmaven3-core-java. 2024-06-28T14:59:07.257 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../29-libmaven3-core-java_3.6.3-5_all.deb ... 2024-06-28T14:59:07.290 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package s3cmd. 2024-06-28T14:59:07.291 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libmaven3-core-java (3.6.3-5) ... 2024-06-28T14:59:07.319 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../34-s3cmd_2.2.0-1_all.deb ... 2024-06-28T14:59:07.357 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking s3cmd (2.2.0-1) ... 2024-06-28T14:59:07.653 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libwagon-file-java. 2024-06-28T14:59:07.670 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../30-libwagon-file-java_3.3.4-1_all.deb ... 2024-06-28T14:59:07.711 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libwagon-file-java (3.3.4-1) ... 2024-06-28T14:59:07.738 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libslf4j-java (1.7.32-1) ... 2024-06-28T14:59:07.839 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libplexus-utils2-java (3.3.0-1) ... 2024-06-28T14:59:07.964 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libwagon-http-shaded-java. 2024-06-28T14:59:07.965 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libplexus-classworlds-java (2.6.0-1) ... 2024-06-28T14:59:07.980 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../31-libwagon-http-shaded-java_3.3.4-1_all.deb ... 2024-06-28T14:59:08.022 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libwagon-http-shaded-java (3.3.4-1) ... 2024-06-28T14:59:08.091 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjsr305-java (0.1~+svn49-11) ... 2024-06-28T14:59:08.217 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libaopalliance-java (20070526-6) ... 2024-06-28T14:59:08.343 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcommons-cli-java (1.4-2) ... 2024-06-28T14:59:08.469 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libplexus-component-annotations-java (2.1.0-1) ... 2024-06-28T14:59:08.483 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package maven. 2024-06-28T14:59:08.512 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../32-maven_3.6.3-5_all.deb ... 2024-06-28T14:59:08.550 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking maven (3.6.3-5) ... 2024-06-28T14:59:08.595 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libplexus-cipher-java (1.8-2) ... 2024-06-28T14:59:08.721 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-06-28T14:59:08.830 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-magic. 2024-06-28T14:59:08.839 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-06-28T14:59:08.844 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../33-python3-magic_2%3a0.4.24-2_all.deb ... 2024-06-28T14:59:08.878 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-magic (2:0.4.24-2) ... 2024-06-28T14:59:08.940 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libapache-pom-java (18-1) ... 2024-06-28T14:59:09.066 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-06-28T14:59:09.192 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libplexus-interpolation-java (1.26-1) ... 2024-06-28T14:59:09.223 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package s3cmd. 2024-06-28T14:59:09.252 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../34-s3cmd_2.2.0-1_all.deb ... 2024-06-28T14:59:09.289 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking s3cmd (2.2.0-1) ... 2024-06-28T14:59:09.318 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libplexus-sec-dispatcher-java (1.4-4) ... 2024-06-28T14:59:09.444 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libwagon-http-shaded-java (3.3.4-1) ... 2024-06-28T14:59:09.569 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-magic (2:0.4.24-2) ... 2024-06-28T14:59:09.686 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libslf4j-java (1.7.32-1) ... 2024-06-28T14:59:09.811 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libplexus-utils2-java (3.3.0-1) ... 2024-06-28T14:59:09.837 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcdi-api-java (1.2-3) ... 2024-06-28T14:59:09.937 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libplexus-classworlds-java (2.6.0-1) ... 2024-06-28T14:59:09.964 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libhawtjni-runtime-java (1.17-1) ... 2024-06-28T14:59:10.057 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libjsr305-java (0.1~+svn49-11) ... 2024-06-28T14:59:10.064 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libwagon-provider-api-java (3.3.4-1) ... 2024-06-28T14:59:10.174 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libaopalliance-java (20070526-6) ... 2024-06-28T14:59:10.182 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libmaven-parent-java (31-2) ... 2024-06-28T14:59:10.300 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libcommons-cli-java (1.4-2) ... 2024-06-28T14:59:10.308 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcommons-parent-java (43-1) ... 2024-06-28T14:59:10.426 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libplexus-component-annotations-java (2.1.0-1) ... 2024-06-28T14:59:10.451 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libsisu-inject-java (0.3.4-2) ... 2024-06-28T14:59:10.552 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libplexus-cipher-java (1.8-2) ... 2024-06-28T14:59:10.577 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up s3cmd (2.2.0-1) ... 2024-06-28T14:59:10.678 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-06-28T14:59:10.804 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-06-28T14:59:10.870 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libsisu-plexus-java (0.3.4-3) ... 2024-06-28T14:59:10.930 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libapache-pom-java (18-1) ... 2024-06-28T14:59:11.039 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-06-28T14:59:11.157 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libplexus-interpolation-java (1.26-1) ... 2024-06-28T14:59:11.266 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libplexus-sec-dispatcher-java (1.4-4) ... 2024-06-28T14:59:11.392 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libwagon-http-shaded-java (3.3.4-1) ... 2024-06-28T14:59:11.413 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libmaven-resolver-java (1.4.2-3build1) ... 2024-06-28T14:59:11.518 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-magic (2:0.4.24-2) ... 2024-06-28T14:59:11.523 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libguava-java (29.0-6) ... 2024-06-28T14:59:11.624 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcommons-lang3-java (3.11-1) ... 2024-06-28T14:59:11.750 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjansi-native-java (1.8-1) ... 2024-06-28T14:59:11.769 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libcdi-api-java (1.2-3) ... 2024-06-28T14:59:11.876 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libwagon-file-java (3.3.4-1) ... 2024-06-28T14:59:11.895 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libhawtjni-runtime-java (1.17-1) ... 2024-06-28T14:59:12.002 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcommons-io-java (2.11.0-2) ... 2024-06-28T14:59:12.021 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libwagon-provider-api-java (3.3.4-1) ... 2024-06-28T14:59:12.128 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libguice-java (4.2.3-2) ... 2024-06-28T14:59:12.147 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libmaven-parent-java (31-2) ... 2024-06-28T14:59:12.254 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjansi-java (1.18-1) ... 2024-06-28T14:59:12.281 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libcommons-parent-java (43-1) ... 2024-06-28T14:59:12.379 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-06-28T14:59:12.407 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libsisu-inject-java (0.3.4-2) ... 2024-06-28T14:59:12.497 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libmaven3-core-java (3.6.3-5) ... 2024-06-28T14:59:12.533 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up s3cmd (2.2.0-1) ... 2024-06-28T14:59:12.598 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up maven (3.6.3-5) ... 2024-06-28T14:59:12.826 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libsisu-plexus-java (0.3.4-3) ... 2024-06-28T14:59:12.847 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/share/maven/bin/mvn to provide /usr/bin/mvn (mvn) in auto mode 2024-06-28T14:59:12.919 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-06-28T14:59:12.952 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libmaven-resolver-java (1.4.2-3build1) ... 2024-06-28T14:59:13.095 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libguava-java (29.0-6) ... 2024-06-28T14:59:13.221 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libcommons-lang3-java (3.11-1) ... 2024-06-28T14:59:13.347 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libjansi-native-java (1.8-1) ... 2024-06-28T14:59:13.456 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libwagon-file-java (3.3.4-1) ... 2024-06-28T14:59:13.565 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libcommons-io-java (2.11.0-2) ... 2024-06-28T14:59:13.683 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libguice-java (4.2.3-2) ... 2024-06-28T14:59:13.809 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libjansi-java (1.18-1) ... 2024-06-28T14:59:13.935 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-06-28T14:59:14.061 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libmaven3-core-java (3.6.3-5) ... 2024-06-28T14:59:14.187 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up maven (3.6.3-5) ... 2024-06-28T14:59:14.291 INFO:teuthology.orchestra.run.smithi195.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-06-28T14:59:14.297 DEBUG:teuthology.parallel:result is None 2024-06-28T14:59:14.622 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/share/maven/bin/mvn to provide /usr/bin/mvn (mvn) in auto mode 2024-06-28T14:59:14.693 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-06-28T14:59:16.040 INFO:teuthology.orchestra.run.smithi178.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-06-28T14:59:16.045 DEBUG:teuthology.parallel:result is None 2024-06-28T14:59:16.046 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T14:59:16.311 DEBUG:teuthology.orchestra.run.smithi178:> dpkg-query -W -f '${Version}' ceph 2024-06-28T14:59:16.330 INFO:teuthology.orchestra.run.smithi178.stdout:19.0.0-4592-gec2a38ba-1jammy 2024-06-28T14:59:16.331 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-4592-gec2a38ba-1jammy 2024-06-28T14:59:16.331 INFO:teuthology.task.install:The correct ceph version 19.0.0-4592-gec2a38ba-1jammy is installed. 2024-06-28T14:59:16.333 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T14:59:16.470 DEBUG:teuthology.orchestra.run.smithi195:> dpkg-query -W -f '${Version}' ceph 2024-06-28T14:59:16.490 INFO:teuthology.orchestra.run.smithi195.stdout:19.0.0-4592-gec2a38ba-1jammy 2024-06-28T14:59:16.490 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-4592-gec2a38ba-1jammy 2024-06-28T14:59:16.490 INFO:teuthology.task.install:The correct ceph version 19.0.0-4592-gec2a38ba-1jammy is installed. 2024-06-28T14:59:16.491 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-28T14:59:16.492 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:16.492 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-28T14:59:16.507 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:16.507 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-28T14:59:16.545 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-28T14:59:16.546 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:16.546 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-06-28T14:59:16.562 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-28T14:59:16.618 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:16.618 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2024-06-28T14:59:16.631 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-28T14:59:16.688 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-28T14:59:16.689 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:16.689 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-28T14:59:16.701 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-28T14:59:16.753 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:16.753 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-28T14:59:16.766 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-28T14:59:16.820 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-28T14:59:16.820 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:16.820 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-06-28T14:59:16.833 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-28T14:59:16.886 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:16.886 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2024-06-28T14:59:16.899 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-28T14:59:16.952 INFO:teuthology.run_tasks:Running task ceph... 2024-06-28T14:59:17.032 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-06-28T14:59:17.032 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /var/log/ceph 2024-06-28T14:59:17.034 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /var/log/ceph 2024-06-28T14:59:17.047 INFO:tasks.ceph:Disabling ceph logrotate... 2024-06-28T14:59:17.047 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/logrotate.d/ceph 2024-06-28T14:59:17.091 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/logrotate.d/ceph 2024-06-28T14:59:17.103 INFO:tasks.ceph:Creating extra log directories... 2024-06-28T14:59:17.103 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-06-28T14:59:17.152 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-06-28T14:59:17.175 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-06-28T14:59:17.175 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'debug rgw notification': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw enable ops log': True, 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-06-28T14:59:17.175 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037', 'branch': 'wip-cbodley-testing', 'description': 'rgw/verify/{0-install accounts$/{tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}', 'email': None, 'first_in_suite': False, 'job_id': '7778037', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-cbodley-testing'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'debug rgw notification': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw enable ops log': True, 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME'], 'sha1': 'ec2a38ba1b69b5357de8f0e8f96b00b4a577169d'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ec2a38ba1b69b5357de8f0e8f96b00b4a577169d'}}, 'rgw': {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'compression type': 'random', 'datacache': True, 'datacache_path': '/tmp/rgw_datacache', 'ec-data-pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 3, 'm': 1, 'name': 'testprofile'}, 'frontend': 'beast', 'inline data': False, 'storage classes': 'LUKEWARM, FROZEN'}, 's3tests': {'accounts': {'iam alt root': 'RGW99999999999999999', 'iam root': 'RGW88888888888888888', 's3 tenant': 'RGW33333333333333333'}, 'force-branch': 'ceph-master', 'slow_backend': True, 'storage classes': 'LUKEWARM, FROZEN'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-cbodley-testing', 'sha1': 'ec2a38ba1b69b5357de8f0e8f96b00b4a577169d'}}, 'owner': 'scheduled_cbodley@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 8526, 'sha1': 'ec2a38ba1b69b5357de8f0e8f96b00b4a577169d', 'sleep_before_teardown': 0, 'suite': 'rgw', 'suite_branch': 'wip-cbodley-testing', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'ec2a38ba1b69b5357de8f0e8f96b00b4a577169d', 'targets': {'smithi178.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFwhDo0ebMNmb/A01I7bjDnzquAQgYCuAT3VMgMwd32ijenqYq2JwS91Hm2ikkCrvijXPE3oeNArhF2jWLa83AQ=', 'smithi195.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIeW5AMP9KxVexvh1h1g+7SB5yjzlLb3/Ib+ea2Eyfv98+KHfDZ11Ut/FYyV/4cz+T75rd5cUfGe75dKdvrCBh4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'flavor': 'default', 'sha1': 'ec2a38ba1b69b5357de8f0e8f96b00b4a577169d'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': None}}, {'tox': ['client.0']}, {'workunit': {'clients': {'client.0': ['rgw/run-datacache.sh']}, 'env': {'RGW_DATACACHE_PATH': '/tmp/rgw_datacache'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-bucket-check.sh']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'cls/test_cls_user.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh']}}}, {'workunit': {'clients': {'client.0': ['rgw/test_rgw_s3_mp_reupload.sh']}}}, {'ragweed': {'client.0': {'default-branch': 'ceph-master', 'rgw_server': 'client.0', 'stages': 'prepare,check'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-reshard.sh']}}}, {'s3tests-java': {'client.0': {'force-branch': 'ceph-master', 'force-repo': 'https://github.com/ceph/java_s3tests.git'}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-versioning.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '544fecbcd55f3d2b6f478823823ce40cbefef1d4', 'timestamp': '2024-06-28_14:14:22', 'tube': 'smithi', 'user': 'cbodley', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2698093'} 2024-06-28T14:59:17.176 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-06-28T14:59:17.208 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-06-28T14:59:17.214 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/run/ceph 2024-06-28T14:59:17.255 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/run/ceph 2024-06-28T14:59:17.269 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:17.269 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-06-28T14:59:17.315 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-28T14:59:17.316 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-06-28T14:59:17.367 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-28T14:59:17.367 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-28T14:59:17.367 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-06-28T14:59:17.367 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-28T14:59:17.367 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-06-28 14:53:38.937340325 +0000 2024-06-28T14:59:17.367 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-06-28 14:53:38.681342598 +0000 2024-06-28T14:59:17.367 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-06-28 14:53:38.681342598 +0000 2024-06-28T14:59:17.367 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2024-06-28T14:59:17.367 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-28T14:59:17.421 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-06-28T14:59:17.421 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-06-28T14:59:17.421 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000320488 s, 1.6 MB/s 2024-06-28T14:59:17.424 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-28T14:59:17.472 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-06-28T14:59:17.518 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-28T14:59:17.518 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-28T14:59:17.518 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-06-28T14:59:17.518 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-28T14:59:17.519 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-06-28 14:53:39.417336061 +0000 2024-06-28T14:59:17.519 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-06-28 14:53:39.181338157 +0000 2024-06-28T14:59:17.519 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-06-28 14:53:39.181338157 +0000 2024-06-28T14:59:17.519 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2024-06-28T14:59:17.519 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-28T14:59:17.572 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-06-28T14:59:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-06-28T14:59:17.573 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000309514 s, 1.7 MB/s 2024-06-28T14:59:17.574 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-28T14:59:17.624 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-06-28T14:59:17.670 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-28T14:59:17.671 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-28T14:59:17.671 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 923 Links: 1 2024-06-28T14:59:17.671 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-28T14:59:17.671 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-06-28 14:53:39.885331903 +0000 2024-06-28T14:59:17.671 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-06-28 14:53:39.633334141 +0000 2024-06-28T14:59:17.671 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-06-28 14:53:39.633334141 +0000 2024-06-28T14:59:17.671 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2024-06-28T14:59:17.671 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-28T14:59:17.724 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-06-28T14:59:17.724 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-06-28T14:59:17.724 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000260804 s, 2.0 MB/s 2024-06-28T14:59:17.726 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-28T14:59:17.776 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-06-28T14:59:17.823 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-28T14:59:17.823 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-28T14:59:17.823 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 940 Links: 1 2024-06-28T14:59:17.823 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-28T14:59:17.823 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-06-28 14:53:40.385327461 +0000 2024-06-28T14:59:17.823 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-06-28 14:53:40.125329771 +0000 2024-06-28T14:59:17.823 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-06-28 14:53:40.125329771 +0000 2024-06-28T14:59:17.824 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2024-06-28T14:59:17.824 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-28T14:59:17.876 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-06-28T14:59:17.876 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-06-28T14:59:17.877 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000251046 s, 2.0 MB/s 2024-06-28T14:59:17.878 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-28T14:59:17.928 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-06-28T14:59:17.928 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:17.928 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2024-06-28T14:59:17.934 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-28T14:59:17.934 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2024-06-28T14:59:17.982 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-28T14:59:17.982 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-28T14:59:17.982 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 883 Links: 1 2024-06-28T14:59:17.982 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-28T14:59:17.983 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-06-28 14:53:38.942804379 +0000 2024-06-28T14:59:17.983 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-06-28 14:53:38.686806662 +0000 2024-06-28T14:59:17.983 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-06-28 14:53:38.690806626 +0000 2024-06-28T14:59:17.983 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-06-28T14:59:17.983 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-28T14:59:18.036 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-06-28T14:59:18.036 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-06-28T14:59:18.036 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000251328 s, 2.0 MB/s 2024-06-28T14:59:18.038 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-28T14:59:18.087 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2024-06-28T14:59:18.134 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-28T14:59:18.134 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-28T14:59:18.134 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 894 Links: 1 2024-06-28T14:59:18.134 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-28T14:59:18.134 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-06-28 14:53:39.182802238 +0000 2024-06-28T14:59:18.135 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-06-28 14:53:39.178802274 +0000 2024-06-28T14:59:18.135 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-06-28 14:53:39.178802274 +0000 2024-06-28T14:59:18.135 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-06-28T14:59:18.135 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-28T14:59:18.187 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-06-28T14:59:18.187 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-06-28T14:59:18.187 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000300257 s, 1.7 MB/s 2024-06-28T14:59:18.188 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-28T14:59:18.235 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2024-06-28T14:59:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-28T14:59:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-28T14:59:18.282 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 916 Links: 1 2024-06-28T14:59:18.282 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-28T14:59:18.282 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-06-28 14:53:39.666797920 +0000 2024-06-28T14:59:18.282 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-06-28 14:53:39.662797956 +0000 2024-06-28T14:59:18.282 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-06-28 14:53:39.662797956 +0000 2024-06-28T14:59:18.282 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-06-28T14:59:18.283 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-28T14:59:18.335 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-06-28T14:59:18.336 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-06-28T14:59:18.336 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000238357 s, 2.1 MB/s 2024-06-28T14:59:18.337 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-28T14:59:18.388 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2024-06-28T14:59:18.434 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-28T14:59:18.434 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-28T14:59:18.434 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 932 Links: 1 2024-06-28T14:59:18.434 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-28T14:59:18.434 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-06-28 14:53:40.450790926 +0000 2024-06-28T14:59:18.434 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-06-28 14:53:40.190793245 +0000 2024-06-28T14:59:18.434 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-06-28 14:53:40.190793245 +0000 2024-06-28T14:59:18.434 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-06-28T14:59:18.435 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-28T14:59:18.488 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-06-28T14:59:18.488 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-06-28T14:59:18.488 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000310635 s, 1.6 MB/s 2024-06-28T14:59:18.489 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-28T14:59:18.539 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-06-28T14:59:18.539 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-06-28T14:59:18.539 INFO:tasks.ceph:Generating config... 2024-06-28T14:59:18.540 INFO:tasks.ceph:[client] debug rgw = 20 2024-06-28T14:59:18.540 INFO:tasks.ceph:[client] debug rgw notification = 20 2024-06-28T14:59:18.540 INFO:tasks.ceph:[client] rgw bucket counters cache = True 2024-06-28T14:59:18.540 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2024-06-28T14:59:18.541 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2024-06-28T14:59:18.541 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-06-28T14:59:18.541 INFO:tasks.ceph:[client] rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2024-06-28T14:59:18.541 INFO:tasks.ceph:[client] rgw d3n l1 datacache size = 10737418240 2024-06-28T14:59:18.541 INFO:tasks.ceph:[client] rgw d3n l1 local datacache enabled = True 2024-06-28T14:59:18.541 INFO:tasks.ceph:[client] rgw enable ops log = True 2024-06-28T14:59:18.541 INFO:tasks.ceph:[client] rgw max chunk size = 4194304 2024-06-28T14:59:18.541 INFO:tasks.ceph:[client] rgw obj stripe size = 6291456 2024-06-28T14:59:18.541 INFO:tasks.ceph:[client] rgw s3 auth use sts = True 2024-06-28T14:59:18.541 INFO:tasks.ceph:[client] rgw sts key = abcdefghijklmnop 2024-06-28T14:59:18.541 INFO:tasks.ceph:[client] rgw torrent flag = True 2024-06-28T14:59:18.541 INFO:tasks.ceph:[client] rgw user counters cache = True 2024-06-28T14:59:18.541 INFO:tasks.ceph:[client] setgroup = ceph 2024-06-28T14:59:18.541 INFO:tasks.ceph:[client] setuser = ceph 2024-06-28T14:59:18.542 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-06-28T14:59:18.542 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-06-28T14:59:18.542 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2024-06-28T14:59:18.542 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2024-06-28T14:59:18.542 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2024-06-28T14:59:18.542 INFO:tasks.ceph:[global] rgw override bucket index max shards = 1 2024-06-28T14:59:18.542 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-06-28T14:59:18.542 INFO:tasks.ceph:[mgr] debug ms = 1 2024-06-28T14:59:18.542 INFO:tasks.ceph:[mon] debug mon = 20 2024-06-28T14:59:18.542 INFO:tasks.ceph:[mon] debug ms = 1 2024-06-28T14:59:18.542 INFO:tasks.ceph:[mon] debug paxos = 20 2024-06-28T14:59:18.542 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2024-06-28T14:59:18.542 INFO:tasks.ceph:[osd] bdev async discard = True 2024-06-28T14:59:18.543 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-06-28T14:59:18.543 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-06-28T14:59:18.543 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-06-28T14:59:18.543 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-06-28T14:59:18.543 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-06-28T14:59:18.543 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-06-28T14:59:18.543 INFO:tasks.ceph:[osd] debug ms = 1 2024-06-28T14:59:18.543 INFO:tasks.ceph:[osd] debug osd = 20 2024-06-28T14:59:18.543 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-06-28T14:59:18.543 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-06-28T14:59:18.543 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-06-28T14:59:18.543 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-06-28T14:59:18.543 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-06-28T14:59:18.543 INFO:tasks.ceph:[osd] osd fast shutdown = False 2024-06-28T14:59:18.544 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-06-28T14:59:18.544 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-06-28T14:59:18.544 INFO:tasks.ceph:Setting up mon.a... 2024-06-28T14:59:18.544 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-06-28T14:59:18.579 INFO:teuthology.orchestra.run.smithi178.stdout:creating /etc/ceph/ceph.keyring 2024-06-28T14:59:18.586 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-06-28T14:59:18.667 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-06-28T14:59:18.721 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.178'), ('mon.c', '[v2:172.21.15.178:3301,v1:172.21.15.178:6790]'), ('mon.b', '172.21.15.195')] 2024-06-28T14:59:18.721 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790],172.21.15.195', 'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10, 'rgw override bucket index max shards': 1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon osd crush smoke test': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': True, 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'debug rgw notification': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw d3n l1 datacache persistent path': '/tmp/rgw_datacache/', 'rgw d3n l1 datacache size': 10737418240, 'rgw d3n l1 local datacache enabled': True, 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-06-28T14:59:18.722 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:18.722 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-06-28T14:59:18.766 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.178 --addv c '[v2:172.21.15.178:3301,v1:172.21.15.178:6790]' --add b 172.21.15.195 --print /home/ubuntu/cephtest/ceph.monmap 2024-06-28T14:59:18.850 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T14:59:18.850 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T14:59:18.851 INFO:teuthology.orchestra.run.smithi178.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-06-28T14:59:18.851 INFO:teuthology.orchestra.run.smithi178.stdout:monmaptool: generated fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T14:59:18.851 INFO:teuthology.orchestra.run.smithi178.stdout:setting min_mon_release = quincy 2024-06-28T14:59:18.851 INFO:teuthology.orchestra.run.smithi178.stdout:epoch 0 2024-06-28T14:59:18.851 INFO:teuthology.orchestra.run.smithi178.stdout:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T14:59:18.851 INFO:teuthology.orchestra.run.smithi178.stdout:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T14:59:18.851 INFO:teuthology.orchestra.run.smithi178.stdout:created 2024-06-28T14:59:18.862098+0000 2024-06-28T14:59:18.851 INFO:teuthology.orchestra.run.smithi178.stdout:min_mon_release 17 (quincy) 2024-06-28T14:59:18.851 INFO:teuthology.orchestra.run.smithi178.stdout:election_strategy: 1 2024-06-28T14:59:18.851 INFO:teuthology.orchestra.run.smithi178.stdout:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T14:59:18.852 INFO:teuthology.orchestra.run.smithi178.stdout:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T14:59:18.852 INFO:teuthology.orchestra.run.smithi178.stdout:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T14:59:18.852 INFO:teuthology.orchestra.run.smithi178.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-06-28T14:59:18.853 DEBUG:teuthology.orchestra.run.smithi178:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-06-28T14:59:18.899 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID cc348236-88e8-4481-a265-c0b687a883fe... 2024-06-28T14:59:18.900 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-06-28T14:59:18.943 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-06-28T14:59:18.969 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-06-28T14:59:18.969 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2024-06-28T14:59:18.969 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-06-28T14:59:18.969 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2024-06-28T14:59:18.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.969 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2024-06-28T14:59:18.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.969 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2024-06-28T14:59:18.970 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.970 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2024-06-28T14:59:18.970 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2024-06-28T14:59:18.970 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.970 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2024-06-28T14:59:18.970 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2024-06-28T14:59:18.970 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.970 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2024-06-28T14:59:18.970 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2024-06-28T14:59:18.970 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.970 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2024-06-28T14:59:18.970 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2024-06-28T14:59:18.970 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.970 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2024-06-28T14:59:18.971 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.971 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2024-06-28T14:59:18.971 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2024-06-28T14:59:18.971 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2024-06-28T14:59:18.971 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2024-06-28T14:59:18.971 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2024-06-28T14:59:18.971 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2024-06-28T14:59:18.971 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2024-06-28T14:59:18.971 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-06-28T14:59:18.971 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2024-06-28T14:59:18.971 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2024-06-28T14:59:18.971 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.971 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-06-28T14:59:18.972 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.972 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2024-06-28T14:59:18.972 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.972 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2024-06-28T14:59:18.972 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.972 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2024-06-28T14:59:18.972 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2024-06-28T14:59:18.972 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2024-06-28T14:59:18.972 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790],172.21.15.195" 2024-06-28T14:59:18.972 INFO:teuthology.orchestra.run.smithi178.stdout: mon client directed command retry = 5 2024-06-28T14:59:18.972 INFO:teuthology.orchestra.run.smithi178.stdout: ms inject socket failures = 5000 2024-06-28T14:59:18.972 INFO:teuthology.orchestra.run.smithi178.stdout: osd heartbeat grace = 40 2024-06-28T14:59:18.972 INFO:teuthology.orchestra.run.smithi178.stdout: osd_max_pg_log_entries = 10 2024-06-28T14:59:18.972 INFO:teuthology.orchestra.run.smithi178.stdout: osd_min_pg_log_entries = 10 2024-06-28T14:59:18.973 INFO:teuthology.orchestra.run.smithi178.stdout: rgw override bucket index max shards = 1 2024-06-28T14:59:18.973 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T14:59:18.973 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.973 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2024-06-28T14:59:18.973 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2024-06-28T14:59:18.973 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.973 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2024-06-28T14:59:18.973 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 600 2024-06-28T14:59:18.973 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2024-06-28T14:59:18.973 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.973 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2024-06-28T14:59:18.973 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2024-06-28T14:59:18.973 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2024-06-28T14:59:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2024-06-28T14:59:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2024-06-28T14:59:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug trim objects = true 2024-06-28T14:59:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2024-06-28T14:59:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2024-06-28T14:59:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2024-06-28T14:59:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2024-06-28T14:59:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.974 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2024-06-28T14:59:18.975 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.975 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2024-06-28T14:59:18.975 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2024-06-28T14:59:18.975 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.975 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2024-06-28T14:59:18.975 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2024-06-28T14:59:18.975 INFO:teuthology.orchestra.run.smithi178.stdout: bdev async discard = True 2024-06-28T14:59:18.975 INFO:teuthology.orchestra.run.smithi178.stdout: bdev enable discard = True 2024-06-28T14:59:18.975 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore allocator = bitmap 2024-06-28T14:59:18.975 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore block size = 96636764160 2024-06-28T14:59:18.975 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore fsck on mount = True 2024-06-28T14:59:18.975 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluefs = 1/20 2024-06-28T14:59:18.975 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluestore = 1/20 2024-06-28T14:59:18.976 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-06-28T14:59:18.976 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2024-06-28T14:59:18.976 INFO:teuthology.orchestra.run.smithi178.stdout: debug rocksdb = 4/10 2024-06-28T14:59:18.976 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd backfillfull_ratio = 0.85 2024-06-28T14:59:18.976 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd full ratio = 0.9 2024-06-28T14:59:18.976 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd nearfull ratio = 0.8 2024-06-28T14:59:18.976 INFO:teuthology.orchestra.run.smithi178.stdout: osd failsafe full ratio = 0.95 2024-06-28T14:59:18.976 INFO:teuthology.orchestra.run.smithi178.stdout: osd fast shutdown = False 2024-06-28T14:59:18.976 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = bluestore 2024-06-28T14:59:18.976 INFO:teuthology.orchestra.run.smithi178.stdout: osd shutdown pgref assert = True 2024-06-28T14:59:18.976 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.976 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2024-06-28T14:59:18.976 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-06-28T14:59:18.976 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2024-06-28T14:59:18.977 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-06-28T14:59:18.977 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-06-28T14:59:18.977 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-06-28T14:59:18.977 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-06-28T14:59:18.977 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2024-06-28T14:59:18.977 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.977 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2024-06-28T14:59:18.977 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-06-28T14:59:18.977 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-06-28T14:59:18.977 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2024-06-28T14:59:18.977 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-06-28T14:59:18.977 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2024-06-28T14:59:18.977 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2024-06-28T14:59:18.977 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-06-28T14:59:18.978 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2024-06-28T14:59:18.978 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2024-06-28T14:59:18.978 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-06-28T14:59:18.978 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.978 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-06-28T14:59:18.978 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2024-06-28T14:59:18.978 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2024-06-28T14:59:18.978 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.978 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2024-06-28T14:59:18.978 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-06-28T14:59:18.978 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-06-28T14:59:18.978 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.978 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2024-06-28T14:59:18.979 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2024-06-28T14:59:18.979 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.979 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2024-06-28T14:59:18.979 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd crush smoke test = False 2024-06-28T14:59:18.979 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:18.979 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2024-06-28T14:59:18.979 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2024-06-28T14:59:18.979 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = True 2024-06-28T14:59:18.979 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2024-06-28T14:59:18.979 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-06-28T14:59:18.979 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-06-28T14:59:18.979 INFO:teuthology.orchestra.run.smithi178.stdout: debug rgw = 20 2024-06-28T14:59:18.979 INFO:teuthology.orchestra.run.smithi178.stdout: debug rgw notification = 20 2024-06-28T14:59:18.980 INFO:teuthology.orchestra.run.smithi178.stdout: rgw bucket counters cache = True 2024-06-28T14:59:18.980 INFO:teuthology.orchestra.run.smithi178.stdout: rgw crypt require ssl = False 2024-06-28T14:59:18.980 INFO:teuthology.orchestra.run.smithi178.stdout: rgw crypt s3 kms backend = testing 2024-06-28T14:59:18.980 INFO:teuthology.orchestra.run.smithi178.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-06-28T14:59:18.980 INFO:teuthology.orchestra.run.smithi178.stdout: rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2024-06-28T14:59:18.980 INFO:teuthology.orchestra.run.smithi178.stdout: rgw d3n l1 datacache size = 10737418240 2024-06-28T14:59:18.980 INFO:teuthology.orchestra.run.smithi178.stdout: rgw d3n l1 local datacache enabled = True 2024-06-28T14:59:18.980 INFO:teuthology.orchestra.run.smithi178.stdout: rgw max chunk size = 4194304 2024-06-28T14:59:18.980 INFO:teuthology.orchestra.run.smithi178.stdout: rgw obj stripe size = 6291456 2024-06-28T14:59:18.980 INFO:teuthology.orchestra.run.smithi178.stdout: rgw s3 auth use sts = True 2024-06-28T14:59:18.980 INFO:teuthology.orchestra.run.smithi178.stdout: rgw sts key = abcdefghijklmnop 2024-06-28T14:59:18.980 INFO:teuthology.orchestra.run.smithi178.stdout: rgw torrent flag = True 2024-06-28T14:59:18.980 INFO:teuthology.orchestra.run.smithi178.stdout: rgw user counters cache = True 2024-06-28T14:59:18.980 INFO:teuthology.orchestra.run.smithi178.stdout: setgroup = ceph 2024-06-28T14:59:18.981 INFO:teuthology.orchestra.run.smithi178.stdout: setuser = ceph 2024-06-28T14:59:18.981 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2024-06-28T14:59:18.981 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2024-06-28T14:59:18.981 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2024-06-28T14:59:18.982 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2024-06-28T14:59:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2024-06-28T14:59:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-06-28T14:59:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2024-06-28T14:59:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2024-06-28T14:59:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2024-06-28T14:59:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2024-06-28T14:59:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2024-06-28T14:59:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2024-06-28T14:59:18.983 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2024-06-28T14:59:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2024-06-28T14:59:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2024-06-28T14:59:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2024-06-28T14:59:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2024-06-28T14:59:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2024-06-28T14:59:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2024-06-28T14:59:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2024-06-28T14:59:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2024-06-28T14:59:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2024-06-28T14:59:18.984 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2024-06-28T14:59:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2024-06-28T14:59:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2024-06-28T14:59:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-06-28T14:59:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2024-06-28T14:59:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2024-06-28T14:59:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-06-28T14:59:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2024-06-28T14:59:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2024-06-28T14:59:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2024-06-28T14:59:18.985 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2024-06-28T14:59:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2024-06-28T14:59:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790],172.21.15.195" 2024-06-28T14:59:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: mon client directed command retry = 5 2024-06-28T14:59:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: ms inject socket failures = 5000 2024-06-28T14:59:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: osd heartbeat grace = 40 2024-06-28T14:59:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: osd_max_pg_log_entries = 10 2024-06-28T14:59:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: osd_min_pg_log_entries = 10 2024-06-28T14:59:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: rgw override bucket index max shards = 1 2024-06-28T14:59:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T14:59:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.986 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2024-06-28T14:59:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2024-06-28T14:59:18.986 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2024-06-28T14:59:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2024-06-28T14:59:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2024-06-28T14:59:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2024-06-28T14:59:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2024-06-28T14:59:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2024-06-28T14:59:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2024-06-28T14:59:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2024-06-28T14:59:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug trim objects = true 2024-06-28T14:59:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2024-06-28T14:59:18.987 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2024-06-28T14:59:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2024-06-28T14:59:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2024-06-28T14:59:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2024-06-28T14:59:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2024-06-28T14:59:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2024-06-28T14:59:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2024-06-28T14:59:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2024-06-28T14:59:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: bdev async discard = True 2024-06-28T14:59:18.988 INFO:teuthology.orchestra.run.smithi195.stdout: bdev enable discard = True 2024-06-28T14:59:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore allocator = bitmap 2024-06-28T14:59:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2024-06-28T14:59:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2024-06-28T14:59:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 1/20 2024-06-28T14:59:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 1/20 2024-06-28T14:59:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-06-28T14:59:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2024-06-28T14:59:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 4/10 2024-06-28T14:59:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2024-06-28T14:59:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2024-06-28T14:59:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2024-06-28T14:59:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2024-06-28T14:59:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: osd fast shutdown = False 2024-06-28T14:59:18.989 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2024-06-28T14:59:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: osd shutdown pgref assert = True 2024-06-28T14:59:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.990 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2024-06-28T14:59:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-06-28T14:59:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2024-06-28T14:59:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-06-28T14:59:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-06-28T14:59:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-06-28T14:59:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-06-28T14:59:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2024-06-28T14:59:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.990 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2024-06-28T14:59:18.990 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-06-28T14:59:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-06-28T14:59:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2024-06-28T14:59:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-06-28T14:59:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2024-06-28T14:59:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2024-06-28T14:59:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-06-28T14:59:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2024-06-28T14:59:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2024-06-28T14:59:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-06-28T14:59:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-06-28T14:59:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2024-06-28T14:59:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2024-06-28T14:59:18.991 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2024-06-28T14:59:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-06-28T14:59:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-06-28T14:59:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2024-06-28T14:59:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2024-06-28T14:59:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2024-06-28T14:59:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd crush smoke test = False 2024-06-28T14:59:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-06-28T14:59:18.992 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2024-06-28T14:59:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2024-06-28T14:59:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = True 2024-06-28T14:59:18.992 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2024-06-28T14:59:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-06-28T14:59:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-06-28T14:59:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: debug rgw = 20 2024-06-28T14:59:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: debug rgw notification = 20 2024-06-28T14:59:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: rgw bucket counters cache = True 2024-06-28T14:59:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: rgw crypt require ssl = False 2024-06-28T14:59:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: rgw crypt s3 kms backend = testing 2024-06-28T14:59:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-06-28T14:59:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: rgw d3n l1 datacache persistent path = /tmp/rgw_datacache/ 2024-06-28T14:59:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: rgw d3n l1 datacache size = 10737418240 2024-06-28T14:59:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: rgw d3n l1 local datacache enabled = True 2024-06-28T14:59:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: rgw max chunk size = 4194304 2024-06-28T14:59:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: rgw obj stripe size = 6291456 2024-06-28T14:59:18.993 INFO:teuthology.orchestra.run.smithi195.stdout: rgw s3 auth use sts = True 2024-06-28T14:59:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: rgw sts key = abcdefghijklmnop 2024-06-28T14:59:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: rgw torrent flag = True 2024-06-28T14:59:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: rgw user counters cache = True 2024-06-28T14:59:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: setgroup = ceph 2024-06-28T14:59:18.994 INFO:teuthology.orchestra.run.smithi195.stdout: setuser = ceph 2024-06-28T14:59:18.994 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2024-06-28T14:59:18.994 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2024-06-28T14:59:18.994 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2024-06-28T14:59:18.994 INFO:tasks.ceph:Creating admin key on mon.a... 2024-06-28T14:59:18.994 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-06-28T14:59:19.074 INFO:tasks.ceph:Copying monmap to all nodes... 2024-06-28T14:59:19.075 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:19.075 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-06-28T14:59:19.080 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:19.080 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-06-28T14:59:19.126 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi178.front.sepia.ceph.com 2024-06-28T14:59:19.126 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:19.126 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.keyring 2024-06-28T14:59:19.126 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-06-28T14:59:19.194 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:19.194 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-06-28T14:59:19.243 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi195.front.sepia.ceph.com 2024-06-28T14:59:19.243 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:19.243 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.keyring 2024-06-28T14:59:19.243 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-06-28T14:59:19.266 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:19.266 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-06-28T14:59:19.314 INFO:tasks.ceph:Setting up mon nodes... 2024-06-28T14:59:19.314 INFO:tasks.ceph:Setting up mgr nodes... 2024-06-28T14:59:19.315 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-06-28T14:59:19.358 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-06-28T14:59:19.361 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-06-28T14:59:19.406 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-06-28T14:59:19.418 INFO:tasks.ceph:Setting up mds nodes... 2024-06-28T14:59:19.419 INFO:tasks.ceph_client:Setting up client nodes... 2024-06-28T14:59:19.419 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-28T14:59:19.452 INFO:teuthology.orchestra.run.smithi178.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-06-28T14:59:19.464 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-06-28T14:59:19.498 INFO:teuthology.orchestra.run.smithi195.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-06-28T14:59:19.507 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-06-28T14:59:19.508 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-06-28T14:59:19.508 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-06-28T14:59:19.521 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-06-28T14:59:19.521 INFO:tasks.ceph:role: osd.0 2024-06-28T14:59:19.521 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com 2024-06-28T14:59:19.521 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-06-28T14:59:19.573 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-06-28T14:59:19.573 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-06-28T14:59:19.573 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-06-28T14:59:19.573 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-06-28T14:59:19.573 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-06-28T14:59:19.573 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-06-28T14:59:19.573 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-06-28T14:59:19.574 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-06-28T14:59:19.574 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-06-28T14:59:19.574 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-06-28T14:59:19.596 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-06-28T14:59:19.598 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-06-28T14:59:19.598 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-06-28T14:59:19.662 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-06-28T14:59:19.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T14:59:19.719 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: /sbin/restorecon: command not found 2024-06-28T14:59:19.720 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-06-28T14:59:19.773 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-06-28T14:59:19.773 INFO:tasks.ceph:role: osd.1 2024-06-28T14:59:19.773 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com 2024-06-28T14:59:19.773 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-06-28T14:59:19.829 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-06-28T14:59:19.829 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-06-28T14:59:19.829 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-06-28T14:59:19.829 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-06-28T14:59:19.830 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-06-28T14:59:19.830 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-06-28T14:59:19.830 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-06-28T14:59:19.830 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-06-28T14:59:19.830 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-06-28T14:59:19.830 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-06-28T14:59:19.853 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-06-28T14:59:19.854 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-06-28T14:59:19.854 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-06-28T14:59:19.915 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-06-28T14:59:19.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T14:59:19.967 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: /sbin/restorecon: command not found 2024-06-28T14:59:19.967 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-06-28T14:59:20.019 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-06-28T14:59:20.019 INFO:tasks.ceph:role: osd.2 2024-06-28T14:59:20.019 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com 2024-06-28T14:59:20.020 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-06-28T14:59:20.072 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-06-28T14:59:20.072 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-06-28T14:59:20.073 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-06-28T14:59:20.073 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-06-28T14:59:20.073 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-06-28T14:59:20.073 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-06-28T14:59:20.073 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-06-28T14:59:20.073 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-06-28T14:59:20.073 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-06-28T14:59:20.073 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-06-28T14:59:20.097 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-06-28T14:59:20.098 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-06-28T14:59:20.099 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-06-28T14:59:20.160 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-06-28T14:59:20.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T14:59:20.211 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: /sbin/restorecon: command not found 2024-06-28T14:59:20.211 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-06-28T14:59:20.268 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-06-28T14:59:20.268 INFO:tasks.ceph:role: osd.3 2024-06-28T14:59:20.268 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi178.front.sepia.ceph.com 2024-06-28T14:59:20.268 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-06-28T14:59:20.322 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-06-28T14:59:20.322 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-06-28T14:59:20.322 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-06-28T14:59:20.322 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-06-28T14:59:20.322 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-06-28T14:59:20.322 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-06-28T14:59:20.322 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-06-28T14:59:20.322 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-06-28T14:59:20.322 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-06-28T14:59:20.322 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-06-28T14:59:20.346 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-06-28T14:59:20.347 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-06-28T14:59:20.347 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-06-28T14:59:20.405 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-06-28T14:59:20.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T14:59:20.463 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: /sbin/restorecon: command not found 2024-06-28T14:59:20.464 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-06-28T14:59:20.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:20.595+0000 7f027c00e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-06-28T14:59:20.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:20.595+0000 7f027c00e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-06-28T14:59:20.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:20.595+0000 7f027c00e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-06-28T14:59:20.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:20.595+0000 7f027c00e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-06-28T14:59:20.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:20.595+0000 7f027c00e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-06-28T14:59:20.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:20.595+0000 7f027c00e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-06-28T14:59:23.595 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-28T14:59:23.609 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-06-28T14:59:23.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:23.699+0000 7fb9202907c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-06-28T14:59:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:23.699+0000 7fb9202907c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-06-28T14:59:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:23.699+0000 7fb9202907c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-06-28T14:59:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:23.699+0000 7fb9202907c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-06-28T14:59:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:23.699+0000 7fb9202907c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-06-28T14:59:23.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:23.699+0000 7fb9202907c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-06-28T14:59:26.698 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-28T14:59:26.759 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-06-28T14:59:26.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:26.847+0000 7f7f14b277c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-06-28T14:59:26.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:26.847+0000 7f7f14b277c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-06-28T14:59:26.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:26.847+0000 7f7f14b277c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-06-28T14:59:26.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:26.847+0000 7f7f14b277c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-06-28T14:59:26.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:26.847+0000 7f7f14b277c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-06-28T14:59:26.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:26.847+0000 7f7f14b277c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-06-28T14:59:29.801 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-28T14:59:29.853 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-06-28T14:59:29.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:29.943+0000 7f94d521a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-06-28T14:59:29.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:29.943+0000 7f94d521a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-06-28T14:59:29.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:29.943+0000 7f94d521a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-06-28T14:59:29.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:29.943+0000 7f94d521a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-06-28T14:59:29.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:29.943+0000 7f94d521a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-06-28T14:59:29.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T14:59:29.943+0000 7f94d521a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-06-28T14:59:32.921 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-28T14:59:32.978 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-06-28T14:59:32.993 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-06-28T14:59:32.994 INFO:tasks.ceph:role: osd.4 2024-06-28T14:59:32.994 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com 2024-06-28T14:59:32.994 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-06-28T14:59:33.049 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-06-28T14:59:33.049 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-06-28T14:59:33.049 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-06-28T14:59:33.049 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-06-28T14:59:33.049 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-06-28T14:59:33.049 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-06-28T14:59:33.049 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-06-28T14:59:33.050 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-06-28T14:59:33.050 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-06-28T14:59:33.050 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-06-28T14:59:33.073 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-06-28T14:59:33.075 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-06-28T14:59:33.075 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-06-28T14:59:33.135 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-06-28T14:59:33.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T14:59:33.187 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-06-28T14:59:33.187 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-06-28T14:59:33.244 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-06-28T14:59:33.244 INFO:tasks.ceph:role: osd.5 2024-06-28T14:59:33.244 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com 2024-06-28T14:59:33.244 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-06-28T14:59:33.297 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-06-28T14:59:33.297 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-06-28T14:59:33.297 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-06-28T14:59:33.297 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-06-28T14:59:33.297 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-06-28T14:59:33.297 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-06-28T14:59:33.297 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-06-28T14:59:33.297 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-06-28T14:59:33.297 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-06-28T14:59:33.297 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-06-28T14:59:33.320 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-06-28T14:59:33.322 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-06-28T14:59:33.322 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-06-28T14:59:33.382 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-06-28T14:59:33.433 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-06-28T14:59:33.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T14:59:33.435 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-06-28T14:59:33.488 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-06-28T14:59:33.488 INFO:tasks.ceph:role: osd.6 2024-06-28T14:59:33.488 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com 2024-06-28T14:59:33.488 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-06-28T14:59:33.541 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-06-28T14:59:33.541 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-06-28T14:59:33.541 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-06-28T14:59:33.541 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-06-28T14:59:33.541 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-06-28T14:59:33.542 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-06-28T14:59:33.542 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-06-28T14:59:33.542 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-06-28T14:59:33.542 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-06-28T14:59:33.542 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-06-28T14:59:33.565 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-06-28T14:59:33.566 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-06-28T14:59:33.566 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-06-28T14:59:33.625 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-06-28T14:59:33.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T14:59:33.679 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-06-28T14:59:33.679 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-06-28T14:59:33.732 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-06-28T14:59:33.732 INFO:tasks.ceph:role: osd.7 2024-06-28T14:59:33.732 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com 2024-06-28T14:59:33.732 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-06-28T14:59:33.789 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-06-28T14:59:33.789 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-06-28T14:59:33.789 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-06-28T14:59:33.789 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-06-28T14:59:33.789 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-06-28T14:59:33.789 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-06-28T14:59:33.789 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-06-28T14:59:33.789 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-06-28T14:59:33.789 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-06-28T14:59:33.789 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-06-28T14:59:33.812 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-06-28T14:59:33.815 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-06-28T14:59:33.815 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-06-28T14:59:33.872 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-06-28T14:59:33.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T14:59:33.927 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-06-28T14:59:33.928 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-06-28T14:59:34.040 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:34.044+0000 7f0b61e6e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-06-28T14:59:34.040 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:34.044+0000 7f0b61e6e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-06-28T14:59:34.040 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:34.044+0000 7f0b61e6e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-06-28T14:59:34.040 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:34.044+0000 7f0b61e6e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-06-28T14:59:34.040 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:34.044+0000 7f0b61e6e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-06-28T14:59:34.040 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:34.044+0000 7f0b61e6e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-06-28T14:59:37.026 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-28T14:59:37.039 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-06-28T14:59:37.120 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:37.128+0000 7f01b9b037c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-06-28T14:59:37.120 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:37.128+0000 7f01b9b037c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-06-28T14:59:37.120 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:37.128+0000 7f01b9b037c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-06-28T14:59:37.121 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:37.128+0000 7f01b9b037c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-06-28T14:59:37.121 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:37.128+0000 7f01b9b037c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-06-28T14:59:37.121 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:37.128+0000 7f01b9b037c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-06-28T14:59:40.102 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-28T14:59:40.156 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-06-28T14:59:40.239 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:40.248+0000 7f0ea36047c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-06-28T14:59:40.240 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:40.248+0000 7f0ea36047c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-06-28T14:59:40.240 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:40.248+0000 7f0ea36047c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-06-28T14:59:40.240 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:40.248+0000 7f0ea36047c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-06-28T14:59:40.240 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:40.248+0000 7f0ea36047c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-06-28T14:59:40.240 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:40.248+0000 7f0ea36047c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-06-28T14:59:43.244 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-06-28T14:59:43.296 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-06-28T14:59:43.380 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:43.388+0000 7f08d2b077c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-06-28T14:59:43.380 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:43.388+0000 7f08d2b077c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-06-28T14:59:43.380 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:43.388+0000 7f08d2b077c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-06-28T14:59:43.381 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:43.388+0000 7f08d2b077c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-06-28T14:59:43.381 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:43.388+0000 7f08d2b077c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-06-28T14:59:43.381 INFO:teuthology.orchestra.run.smithi195.stderr:2024-06-28T14:59:43.388+0000 7f08d2b077c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-06-28T14:59:46.352 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-06-28T14:59:46.410 INFO:tasks.ceph:Reading keys from all nodes... 2024-06-28T14:59:46.410 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:46.410 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-06-28T14:59:46.425 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:46.425 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-06-28T14:59:46.481 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:46.481 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-06-28T14:59:46.536 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:46.536 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-06-28T14:59:46.589 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:46.589 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-06-28T14:59:46.648 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:46.665 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-06-28T14:59:46.687 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:46.687 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-06-28T14:59:46.741 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:46.741 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-06-28T14:59:46.796 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:46.796 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-06-28T14:59:46.852 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:46.852 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-06-28T14:59:46.907 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:46.908 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-06-28T14:59:46.913 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:46.914 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-06-28T14:59:46.955 INFO:tasks.ceph:Adding keys to all mons... 2024-06-28T14:59:46.956 DEBUG:teuthology.orchestra.run.smithi178:> sudo tee -a /etc/ceph/ceph.keyring 2024-06-28T14:59:46.958 DEBUG:teuthology.orchestra.run.smithi195:> sudo tee -a /etc/ceph/ceph.keyring 2024-06-28T14:59:46.999 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr.y] 2024-06-28T14:59:46.999 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDHz35mztMHFhAAHqMicqye62S19ZhUMhRbtg== 2024-06-28T14:59:46.999 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.0] 2024-06-28T14:59:46.999 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDIz35mB2isIxAAiJGa7kPO46KzfAa1VT6V0A== 2024-06-28T14:59:46.999 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.1] 2024-06-28T14:59:46.999 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDLz35mB8nuKRAAOSky74q6czZ5jCYkP9ssQA== 2024-06-28T14:59:46.999 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.2] 2024-06-28T14:59:46.999 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDOz35m26C8MhAAx2IQKRrF0vw7Khbkcwq6ng== 2024-06-28T14:59:47.000 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.3] 2024-06-28T14:59:47.000 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDRz35m9SWBOBAAR3YB5HlPoesoMQPk3/ZzYg== 2024-06-28T14:59:47.000 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr.x] 2024-06-28T14:59:47.000 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDHz35mA7TeGBAAk5fx6VJvsOjonDc2FnZeJg== 2024-06-28T14:59:47.000 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.4] 2024-06-28T14:59:47.000 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDWz35mmPcNAxAAVmvWygvVJ7LcnI2s9L2iYQ== 2024-06-28T14:59:47.000 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.5] 2024-06-28T14:59:47.000 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDZz35mV+XVBxAA/cIvzJp8Jw62AgGx6AJDsg== 2024-06-28T14:59:47.000 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.6] 2024-06-28T14:59:47.000 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDcz35mn/n1DhAAZG96AohhSXp5sCMsUfxzJw== 2024-06-28T14:59:47.000 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.7] 2024-06-28T14:59:47.000 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDfz35m1pdXFxAAN9SB458FWxN8fqcVQikvdg== 2024-06-28T14:59:47.000 INFO:teuthology.orchestra.run.smithi178.stdout:[client.0] 2024-06-28T14:59:47.001 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDHz35mMAWoGxAA/dVFGygxJbM8Ef6rpeRA8Q== 2024-06-28T14:59:47.001 INFO:teuthology.orchestra.run.smithi178.stdout:[client.1] 2024-06-28T14:59:47.001 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDHz35m/bhcHhAAKf5FpI/4/3CI9I5g2a6BQg== 2024-06-28T14:59:47.007 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.y] 2024-06-28T14:59:47.007 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDHz35mztMHFhAAHqMicqye62S19ZhUMhRbtg== 2024-06-28T14:59:47.007 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.0] 2024-06-28T14:59:47.007 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDIz35mB2isIxAAiJGa7kPO46KzfAa1VT6V0A== 2024-06-28T14:59:47.007 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.1] 2024-06-28T14:59:47.008 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDLz35mB8nuKRAAOSky74q6czZ5jCYkP9ssQA== 2024-06-28T14:59:47.008 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.2] 2024-06-28T14:59:47.008 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDOz35m26C8MhAAx2IQKRrF0vw7Khbkcwq6ng== 2024-06-28T14:59:47.008 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.3] 2024-06-28T14:59:47.008 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDRz35m9SWBOBAAR3YB5HlPoesoMQPk3/ZzYg== 2024-06-28T14:59:47.008 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.x] 2024-06-28T14:59:47.008 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDHz35mA7TeGBAAk5fx6VJvsOjonDc2FnZeJg== 2024-06-28T14:59:47.008 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.4] 2024-06-28T14:59:47.008 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDWz35mmPcNAxAAVmvWygvVJ7LcnI2s9L2iYQ== 2024-06-28T14:59:47.008 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.5] 2024-06-28T14:59:47.008 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDZz35mV+XVBxAA/cIvzJp8Jw62AgGx6AJDsg== 2024-06-28T14:59:47.008 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.6] 2024-06-28T14:59:47.008 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDcz35mn/n1DhAAZG96AohhSXp5sCMsUfxzJw== 2024-06-28T14:59:47.009 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.7] 2024-06-28T14:59:47.009 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDfz35m1pdXFxAAN9SB458FWxN8fqcVQikvdg== 2024-06-28T14:59:47.009 INFO:teuthology.orchestra.run.smithi195.stdout:[client.0] 2024-06-28T14:59:47.009 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDHz35mMAWoGxAA/dVFGygxJbM8Ef6rpeRA8Q== 2024-06-28T14:59:47.009 INFO:teuthology.orchestra.run.smithi195.stdout:[client.1] 2024-06-28T14:59:47.009 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQDHz35m/bhcHhAAKf5FpI/4/3CI9I5g2a6BQg== 2024-06-28T14:59:47.010 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-06-28T14:59:47.047 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-06-28T14:59:47.097 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.135 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.179 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.218 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.263 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.302 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.345 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.390 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.431 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-06-28T14:59:47.475 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-06-28T14:59:47.514 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.554 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.595 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.639 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.678 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.723 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.764 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.807 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-06-28T14:59:47.849 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-06-28T14:59:47.895 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-06-28T14:59:47.944 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-06-28T14:59:47.979 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-06-28T14:59:48.025 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-06-28T14:59:48.025 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-06-28T14:59:48.037 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-06-28T14:59:48.130 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-06-28T14:59:48.140 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-06-28T14:59:48.191 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-06-28T14:59:48.274 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-06-28T14:59:48.327 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-06-28T14:59:48.341 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-06-28T14:59:48.436 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-06-28T14:59:48.447 DEBUG:teuthology.orchestra.run.smithi178:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-06-28T14:59:48.449 DEBUG:teuthology.orchestra.run.smithi195:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-06-28T14:59:48.493 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-06-28T14:59:48.494 INFO:tasks.ceph.mon.a:Restarting daemon 2024-06-28T14:59:48.494 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-06-28T14:59:48.498 INFO:tasks.ceph.mon.a:Started 2024-06-28T14:59:48.499 INFO:tasks.ceph.mon.c:Restarting daemon 2024-06-28T14:59:48.499 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-06-28T14:59:48.501 INFO:tasks.ceph.mon.c:Started 2024-06-28T14:59:48.501 INFO:tasks.ceph.mon.b:Restarting daemon 2024-06-28T14:59:48.501 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-06-28T14:59:48.538 INFO:tasks.ceph.mon.b:Started 2024-06-28T14:59:48.538 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-06-28T14:59:48.539 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-06-28T14:59:48.539 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-06-28T14:59:48.541 INFO:tasks.ceph.mgr.y:Started 2024-06-28T14:59:48.541 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-06-28T14:59:48.542 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-06-28T14:59:48.543 INFO:tasks.ceph.mgr.x:Started 2024-06-28T14:59:48.544 DEBUG:tasks.ceph:set 0 configs 2024-06-28T14:59:48.544 INFO:tasks.ceph:Setting crush tunables to default 2024-06-28T14:59:48.544 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd crush tunables default 2024-06-28T14:59:53.962 INFO:teuthology.orchestra.run.smithi178.stderr:adjusted tunables profile to default 2024-06-28T14:59:53.981 INFO:tasks.ceph:check_enable_crimson: False 2024-06-28T14:59:53.982 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-06-28T14:59:53.982 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:53.982 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-06-28T14:59:53.992 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:53.993 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-06-28T14:59:54.049 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:54.049 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-06-28T14:59:54.103 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T14:59:54.104 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-06-28T14:59:54.156 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:54.156 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-06-28T14:59:54.170 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:54.171 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-06-28T14:59:54.224 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:54.224 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-06-28T14:59:54.279 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-06-28T14:59:54.280 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-06-28T14:59:54.333 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new ff15d827-b738-4acc-84f1-577244382e19 0 2024-06-28T14:59:54.605 INFO:teuthology.orchestra.run.smithi195.stdout:0 2024-06-28T14:59:54.621 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 0999f2c0-81f9-4361-b3cb-e6bc5b0c25bc 1 2024-06-28T14:59:54.830 INFO:teuthology.orchestra.run.smithi195.stdout:1 2024-06-28T14:59:54.849 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 30cc4110-9499-411b-94da-b3de3c28627e 2 2024-06-28T14:59:55.053 INFO:teuthology.orchestra.run.smithi195.stdout:2 2024-06-28T14:59:55.069 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new ec1f89b8-fcca-46c9-ba09-4cda48a142e1 3 2024-06-28T14:59:55.264 INFO:teuthology.orchestra.run.smithi195.stdout:3 2024-06-28T14:59:55.281 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 4fc9b653-922a-4c3c-89e7-a347e7553393 4 2024-06-28T14:59:55.482 INFO:teuthology.orchestra.run.smithi195.stdout:4 2024-06-28T14:59:55.498 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 7a56dd0f-6d19-4c74-9108-d4cfdf51a36b 5 2024-06-28T14:59:55.689 INFO:teuthology.orchestra.run.smithi195.stdout:5 2024-06-28T14:59:55.706 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new dcb3e382-9ad2-4ae2-9e96-332a581ec20e 6 2024-06-28T14:59:55.902 INFO:teuthology.orchestra.run.smithi195.stdout:6 2024-06-28T14:59:55.918 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 18053f3f-16ae-42af-a795-89cec10f661e 7 2024-06-28T14:59:56.111 INFO:teuthology.orchestra.run.smithi195.stdout:7 2024-06-28T14:59:56.128 INFO:tasks.ceph.osd.0:Restarting daemon 2024-06-28T14:59:56.128 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-06-28T14:59:56.130 INFO:tasks.ceph.osd.0:Started 2024-06-28T14:59:56.130 INFO:tasks.ceph.osd.1:Restarting daemon 2024-06-28T14:59:56.130 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-06-28T14:59:56.132 INFO:tasks.ceph.osd.1:Started 2024-06-28T14:59:56.132 INFO:tasks.ceph.osd.2:Restarting daemon 2024-06-28T14:59:56.132 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-06-28T14:59:56.134 INFO:tasks.ceph.osd.2:Started 2024-06-28T14:59:56.134 INFO:tasks.ceph.osd.3:Restarting daemon 2024-06-28T14:59:56.134 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-06-28T14:59:56.136 INFO:tasks.ceph.osd.3:Started 2024-06-28T14:59:56.136 INFO:tasks.ceph.osd.4:Restarting daemon 2024-06-28T14:59:56.136 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-06-28T14:59:56.138 INFO:tasks.ceph.osd.4:Started 2024-06-28T14:59:56.139 INFO:tasks.ceph.osd.5:Restarting daemon 2024-06-28T14:59:56.139 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-06-28T14:59:56.141 INFO:tasks.ceph.osd.5:Started 2024-06-28T14:59:56.141 INFO:tasks.ceph.osd.6:Restarting daemon 2024-06-28T14:59:56.141 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-06-28T14:59:56.143 INFO:tasks.ceph.osd.6:Started 2024-06-28T14:59:56.144 INFO:tasks.ceph.osd.7:Restarting daemon 2024-06-28T14:59:56.144 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-06-28T14:59:56.146 INFO:tasks.ceph.osd.7:Started 2024-06-28T14:59:56.146 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-28T14:59:56.350 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T14:59:56.350 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":10,"fsid":"cc348236-88e8-4481-a265-c0b687a883fe","created":"2024-06-28T14:59:53.858969+0000","modified":"2024-06-28T14:59:56.118930+0000","last_up_change":"0.000000","last_in_change":"2024-06-28T14:59:56.118930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ff15d827-b738-4acc-84f1-577244382e19","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0999f2c0-81f9-4361-b3cb-e6bc5b0c25bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"30cc4110-9499-411b-94da-b3de3c28627e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ec1f89b8-fcca-46c9-ba09-4cda48a142e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4fc9b653-922a-4c3c-89e7-a347e7553393","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7a56dd0f-6d19-4c74-9108-d4cfdf51a36b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dcb3e382-9ad2-4ae2-9e96-332a581ec20e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"18053f3f-16ae-42af-a795-89cec10f661e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-28T14:59:56.366 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-06-28T14:59:56.366 INFO:tasks.ceph:Waiting for OSDs to come up 2024-06-28T14:59:56.753 INFO:tasks.ceph.mgr.y.smithi178.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-06-28T14:59:56.753 INFO:tasks.ceph.mgr.y.smithi178.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-06-28T14:59:56.753 INFO:tasks.ceph.mgr.y.smithi178.stderr: from numpy import show_config as show_numpy_config 2024-06-28T14:59:56.770 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-06-28T14:59:56.780+0000 7f139079d7c0 -1 Falling back to public interface 2024-06-28T14:59:56.770 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-06-28T14:59:56.780+0000 7f5e048507c0 -1 Falling back to public interface 2024-06-28T14:59:56.782 INFO:tasks.ceph.osd.0.smithi178.stderr:2024-06-28T14:59:56.790+0000 7f4772f237c0 -1 Falling back to public interface 2024-06-28T14:59:56.785 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-06-28T14:59:56.796+0000 7fd9e7a707c0 -1 Falling back to public interface 2024-06-28T14:59:56.798 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-06-28T14:59:56.806+0000 7f278770f7c0 -1 Falling back to public interface 2024-06-28T14:59:56.799 INFO:tasks.ceph.osd.1.smithi178.stderr:2024-06-28T14:59:56.806+0000 7fc11b1877c0 -1 Falling back to public interface 2024-06-28T14:59:56.802 INFO:tasks.ceph.osd.2.smithi178.stderr:2024-06-28T14:59:56.810+0000 7f92d57fd7c0 -1 Falling back to public interface 2024-06-28T14:59:56.805 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-06-28T14:59:56.816+0000 7fcde38fa7c0 -1 Falling back to public interface 2024-06-28T14:59:57.173 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-06-28T14:59:57.353 INFO:teuthology.misc.health.smithi178.stdout: 2024-06-28T14:59:57.353 INFO:teuthology.misc.health.smithi178.stdout:{"epoch":10,"fsid":"cc348236-88e8-4481-a265-c0b687a883fe","created":"2024-06-28T14:59:53.858969+0000","modified":"2024-06-28T14:59:56.118930+0000","last_up_change":"0.000000","last_in_change":"2024-06-28T14:59:56.118930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ff15d827-b738-4acc-84f1-577244382e19","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0999f2c0-81f9-4361-b3cb-e6bc5b0c25bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"30cc4110-9499-411b-94da-b3de3c28627e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ec1f89b8-fcca-46c9-ba09-4cda48a142e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4fc9b653-922a-4c3c-89e7-a347e7553393","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7a56dd0f-6d19-4c74-9108-d4cfdf51a36b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dcb3e382-9ad2-4ae2-9e96-332a581ec20e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"18053f3f-16ae-42af-a795-89cec10f661e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-28T14:59:57.368 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-06-28T14:59:57.472 INFO:tasks.ceph.mgr.x.smithi195.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-06-28T14:59:57.472 INFO:tasks.ceph.mgr.x.smithi195.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-06-28T14:59:57.472 INFO:tasks.ceph.mgr.x.smithi195.stderr: from numpy import show_config as show_numpy_config 2024-06-28T14:59:59.658 INFO:tasks.ceph.mgr.y.smithi178.stderr:2024-06-28T14:59:59.666+0000 7f7506930640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-06-28T14:59:59.659 INFO:tasks.ceph.mgr.y.smithi178.stderr:2024-06-28T14:59:59.666+0000 7f7506930640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-06-28T15:00:00.584 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-06-28T15:00:00.592+0000 7f5e048507c0 -1 osd.6 0 log_to_monitors true 2024-06-28T15:00:00.590 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-06-28T15:00:00.600+0000 7fd9e7a707c0 -1 osd.4 0 log_to_monitors true 2024-06-28T15:00:00.609 INFO:tasks.ceph.osd.0.smithi178.stderr:2024-06-28T15:00:00.614+0000 7f4772f237c0 -1 osd.0 0 log_to_monitors true 2024-06-28T15:00:00.624 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-06-28T15:00:00.632+0000 7f139079d7c0 -1 osd.5 0 log_to_monitors true 2024-06-28T15:00:00.640 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-06-28T15:00:00.648+0000 7fcde38fa7c0 -1 osd.7 0 log_to_monitors true 2024-06-28T15:00:00.708 INFO:tasks.ceph.osd.1.smithi178.stderr:2024-06-28T15:00:00.714+0000 7fc11b1877c0 -1 osd.1 0 log_to_monitors true 2024-06-28T15:00:00.727 INFO:tasks.ceph.osd.2.smithi178.stderr:2024-06-28T15:00:00.734+0000 7f92d57fd7c0 -1 osd.2 0 log_to_monitors true 2024-06-28T15:00:00.732 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-06-28T15:00:00.738+0000 7f278770f7c0 -1 osd.3 0 log_to_monitors true 2024-06-28T15:00:03.230 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-06-28T15:00:03.240+0000 7f5e007f9640 -1 osd.6 0 waiting for initial osdmap 2024-06-28T15:00:03.257 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-06-28T15:00:03.268+0000 7f5df862d640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-28T15:00:03.350 INFO:tasks.ceph.osd.0.smithi178.stderr:2024-06-28T15:00:03.358+0000 7f476eecc640 -1 osd.0 0 waiting for initial osdmap 2024-06-28T15:00:03.374 INFO:tasks.ceph.osd.0.smithi178.stderr:2024-06-28T15:00:03.382+0000 7f4766d00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-28T15:00:03.388 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-06-28T15:00:03.394+0000 7f27836b8640 -1 osd.3 0 waiting for initial osdmap 2024-06-28T15:00:03.391 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-06-28T15:00:03.400+0000 7f138cf58640 -1 osd.5 0 waiting for initial osdmap 2024-06-28T15:00:03.409 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-06-28T15:00:03.418+0000 7f277b4ec640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-28T15:00:03.418 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-06-28T15:00:03.428+0000 7fcde00b5640 -1 osd.7 0 waiting for initial osdmap 2024-06-28T15:00:03.421 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-06-28T15:00:03.432+0000 7f138457a640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-28T15:00:03.445 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-06-28T15:00:03.456+0000 7fcdd76d7640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-28T15:00:03.450 INFO:tasks.ceph.osd.2.smithi178.stderr:2024-06-28T15:00:03.458+0000 7f92d17a6640 -1 osd.2 0 waiting for initial osdmap 2024-06-28T15:00:03.456 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-06-28T15:00:03.464+0000 7fd9e422b640 -1 osd.4 0 waiting for initial osdmap 2024-06-28T15:00:03.489 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-06-28T15:00:03.500+0000 7fd9db84d640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-28T15:00:03.494 INFO:tasks.ceph.osd.2.smithi178.stderr:2024-06-28T15:00:03.502+0000 7f92c95da640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-28T15:00:03.535 INFO:tasks.ceph.osd.1.smithi178.stderr:2024-06-28T15:00:03.542+0000 7fc117130640 -1 osd.1 0 waiting for initial osdmap 2024-06-28T15:00:03.553 INFO:tasks.ceph.osd.1.smithi178.stderr:2024-06-28T15:00:03.562+0000 7fc10ef64640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-28T15:00:04.174 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-06-28T15:00:04.474 INFO:teuthology.misc.health.smithi178.stdout: 2024-06-28T15:00:04.474 INFO:teuthology.misc.health.smithi178.stdout:{"epoch":13,"fsid":"cc348236-88e8-4481-a265-c0b687a883fe","created":"2024-06-28T14:59:53.858969+0000","modified":"2024-06-28T15:00:04.244051+0000","last_up_change":"2024-06-28T15:00:04.244051+0000","last_in_change":"2024-06-28T14:59:56.118930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ff15d827-b738-4acc-84f1-577244382e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6801","nonce":2650530773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6803","nonce":2650530773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6807","nonce":2650530773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6805","nonce":2650530773}]},"public_addr":"172.21.15.178:6801/2650530773","cluster_addr":"172.21.15.178:6803/2650530773","heartbeat_back_addr":"172.21.15.178:6807/2650530773","heartbeat_front_addr":"172.21.15.178:6805/2650530773","state":["exists","up"]},{"osd":1,"uuid":"0999f2c0-81f9-4361-b3cb-e6bc5b0c25bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6812","nonce":1880801124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6816","nonce":1880801124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6824","nonce":1880801124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6820","nonce":1880801124}]},"public_addr":"172.21.15.178:6812/1880801124","cluster_addr":"172.21.15.178:6816/1880801124","heartbeat_back_addr":"172.21.15.178:6824/1880801124","heartbeat_front_addr":"172.21.15.178:6820/1880801124","state":["exists","up"]},{"osd":2,"uuid":"30cc4110-9499-411b-94da-b3de3c28627e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6825","nonce":1880098663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6827","nonce":1880098663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6831","nonce":1880098663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6829","nonce":1880098663}]},"public_addr":"172.21.15.178:6825/1880098663","cluster_addr":"172.21.15.178:6827/1880098663","heartbeat_back_addr":"172.21.15.178:6831/1880098663","heartbeat_front_addr":"172.21.15.178:6829/1880098663","state":["exists","up"]},{"osd":3,"uuid":"ec1f89b8-fcca-46c9-ba09-4cda48a142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6809","nonce":4178996824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6811","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6813","nonce":4178996824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6821","nonce":4178996824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6815","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6817","nonce":4178996824}]},"public_addr":"172.21.15.178:6809/4178996824","cluster_addr":"172.21.15.178:6813/4178996824","heartbeat_back_addr":"172.21.15.178:6821/4178996824","heartbeat_front_addr":"172.21.15.178:6817/4178996824","state":["exists","up"]},{"osd":4,"uuid":"4fc9b653-922a-4c3c-89e7-a347e7553393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6817","nonce":2928422746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6819","nonce":2928422746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6823","nonce":2928422746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6821","nonce":2928422746}]},"public_addr":"172.21.15.195:6817/2928422746","cluster_addr":"172.21.15.195:6819/2928422746","heartbeat_back_addr":"172.21.15.195:6823/2928422746","heartbeat_front_addr":"172.21.15.195:6821/2928422746","state":["exists","up"]},{"osd":5,"uuid":"7a56dd0f-6d19-4c74-9108-d4cfdf51a36b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6801","nonce":2644688681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6805","nonce":2644688681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6813","nonce":2644688681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6809","nonce":2644688681}]},"public_addr":"172.21.15.195:6801/2644688681","cluster_addr":"172.21.15.195:6805/2644688681","heartbeat_back_addr":"172.21.15.195:6813/2644688681","heartbeat_front_addr":"172.21.15.195:6809/2644688681","state":["exists","up"]},{"osd":6,"uuid":"dcb3e382-9ad2-4ae2-9e96-332a581ec20e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6804","nonce":1595562688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6808","nonce":1595562688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6815","nonce":1595562688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6812","nonce":1595562688}]},"public_addr":"172.21.15.195:6804/1595562688","cluster_addr":"172.21.15.195:6808/1595562688","heartbeat_back_addr":"172.21.15.195:6815/1595562688","heartbeat_front_addr":"172.21.15.195:6812/1595562688","state":["exists","up"]},{"osd":7,"uuid":"18053f3f-16ae-42af-a795-89cec10f661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6825","nonce":1278553753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6827","nonce":1278553753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6831","nonce":1278553753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6829","nonce":1278553753}]},"public_addr":"172.21.15.195:6825/1278553753","cluster_addr":"172.21.15.195:6827/1278553753","heartbeat_back_addr":"172.21.15.195:6831/1278553753","heartbeat_front_addr":"172.21.15.195:6829/1278553753","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-28T15:00:04.490 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-06-28T15:00:04.490 INFO:tasks.ceph:Creating RBD pool 2024-06-28T15:00:04.490 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd pool create rbd 8 2024-06-28T15:00:05.261 INFO:teuthology.orchestra.run.smithi178.stderr:pool 'rbd' created 2024-06-28T15:00:05.286 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-06-28T15:00:06.258 INFO:teuthology.orchestra.run.smithi178.stderr:enabled application 'rbd' on pool 'rbd' 2024-06-28T15:00:06.289 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-06-28T15:00:06.290 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-06-28T15:00:06.290 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-06-28T15:00:06.605 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:00:06.627 INFO:teuthology.orchestra.run.smithi178.stdout:[{"version":9,"timestamp":"2024-06-28T15:00:03.547681+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"3715.838053"}]}] 2024-06-28T15:00:06.627 INFO:tasks.ceph_manager:config epoch is 9 2024-06-28T15:00:06.628 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-28T15:00:06.628 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-06-28T15:00:06.628 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-06-28T15:00:06.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:00:06.985 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":4,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":886789604},{"type":"v1","addr":"172.21.15.178:6833","nonce":886789604}]},"active_addr":"172.21.15.178:6833/886789604","active_change":"2024-06-28T14:59:58.651290+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4114,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":69978548}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":1591090730}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":2857397725}]}]} 2024-06-28T15:00:06.989 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-06-28T15:00:06.989 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-06-28T15:00:06.989 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-28T15:00:07.276 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:00:07.276 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":15,"fsid":"cc348236-88e8-4481-a265-c0b687a883fe","created":"2024-06-28T14:59:53.858969+0000","modified":"2024-06-28T15:00:06.267502+0000","last_up_change":"2024-06-28T15:00:04.244051+0000","last_in_change":"2024-06-28T14:59:56.118930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-28T15:00:04.696005+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-06-28T15:00:04.788733+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff15d827-b738-4acc-84f1-577244382e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6801","nonce":2650530773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6803","nonce":2650530773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6807","nonce":2650530773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6805","nonce":2650530773}]},"public_addr":"172.21.15.178:6801/2650530773","cluster_addr":"172.21.15.178:6803/2650530773","heartbeat_back_addr":"172.21.15.178:6807/2650530773","heartbeat_front_addr":"172.21.15.178:6805/2650530773","state":["exists","up"]},{"osd":1,"uuid":"0999f2c0-81f9-4361-b3cb-e6bc5b0c25bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6812","nonce":1880801124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6816","nonce":1880801124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6824","nonce":1880801124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6820","nonce":1880801124}]},"public_addr":"172.21.15.178:6812/1880801124","cluster_addr":"172.21.15.178:6816/1880801124","heartbeat_back_addr":"172.21.15.178:6824/1880801124","heartbeat_front_addr":"172.21.15.178:6820/1880801124","state":["exists","up"]},{"osd":2,"uuid":"30cc4110-9499-411b-94da-b3de3c28627e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6825","nonce":1880098663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6827","nonce":1880098663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6831","nonce":1880098663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6829","nonce":1880098663}]},"public_addr":"172.21.15.178:6825/1880098663","cluster_addr":"172.21.15.178:6827/1880098663","heartbeat_back_addr":"172.21.15.178:6831/1880098663","heartbeat_front_addr":"172.21.15.178:6829/1880098663","state":["exists","up"]},{"osd":3,"uuid":"ec1f89b8-fcca-46c9-ba09-4cda48a142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6809","nonce":4178996824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6811","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6813","nonce":4178996824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6821","nonce":4178996824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6815","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6817","nonce":4178996824}]},"public_addr":"172.21.15.178:6809/4178996824","cluster_addr":"172.21.15.178:6813/4178996824","heartbeat_back_addr":"172.21.15.178:6821/4178996824","heartbeat_front_addr":"172.21.15.178:6817/4178996824","state":["exists","up"]},{"osd":4,"uuid":"4fc9b653-922a-4c3c-89e7-a347e7553393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6817","nonce":2928422746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6819","nonce":2928422746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6823","nonce":2928422746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6821","nonce":2928422746}]},"public_addr":"172.21.15.195:6817/2928422746","cluster_addr":"172.21.15.195:6819/2928422746","heartbeat_back_addr":"172.21.15.195:6823/2928422746","heartbeat_front_addr":"172.21.15.195:6821/2928422746","state":["exists","up"]},{"osd":5,"uuid":"7a56dd0f-6d19-4c74-9108-d4cfdf51a36b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6801","nonce":2644688681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6805","nonce":2644688681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6813","nonce":2644688681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6809","nonce":2644688681}]},"public_addr":"172.21.15.195:6801/2644688681","cluster_addr":"172.21.15.195:6805/2644688681","heartbeat_back_addr":"172.21.15.195:6813/2644688681","heartbeat_front_addr":"172.21.15.195:6809/2644688681","state":["exists","up"]},{"osd":6,"uuid":"dcb3e382-9ad2-4ae2-9e96-332a581ec20e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6804","nonce":1595562688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6808","nonce":1595562688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6815","nonce":1595562688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6812","nonce":1595562688}]},"public_addr":"172.21.15.195:6804/1595562688","cluster_addr":"172.21.15.195:6808/1595562688","heartbeat_back_addr":"172.21.15.195:6815/1595562688","heartbeat_front_addr":"172.21.15.195:6812/1595562688","state":["exists","up"]},{"osd":7,"uuid":"18053f3f-16ae-42af-a795-89cec10f661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6825","nonce":1278553753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6827","nonce":1278553753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6831","nonce":1278553753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6829","nonce":1278553753}]},"public_addr":"172.21.15.195:6825/1278553753","cluster_addr":"172.21.15.195:6827/1278553753","heartbeat_back_addr":"172.21.15.195:6831/1278553753","heartbeat_front_addr":"172.21.15.195:6829/1278553753","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.628549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.672292+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.714676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.718739+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.580848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.636756+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.636054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.668751+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-28T15:00:07.296 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-06-28T15:00:07.296 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-28T15:00:07.581 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:00:07.582 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":16,"fsid":"cc348236-88e8-4481-a265-c0b687a883fe","created":"2024-06-28T14:59:53.858969+0000","modified":"2024-06-28T15:00:07.278630+0000","last_up_change":"2024-06-28T15:00:04.244051+0000","last_in_change":"2024-06-28T14:59:56.118930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-28T15:00:04.696005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-06-28T15:00:04.788733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff15d827-b738-4acc-84f1-577244382e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6801","nonce":2650530773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6803","nonce":2650530773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6807","nonce":2650530773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6805","nonce":2650530773}]},"public_addr":"172.21.15.178:6801/2650530773","cluster_addr":"172.21.15.178:6803/2650530773","heartbeat_back_addr":"172.21.15.178:6807/2650530773","heartbeat_front_addr":"172.21.15.178:6805/2650530773","state":["exists","up"]},{"osd":1,"uuid":"0999f2c0-81f9-4361-b3cb-e6bc5b0c25bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6812","nonce":1880801124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6816","nonce":1880801124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6824","nonce":1880801124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6820","nonce":1880801124}]},"public_addr":"172.21.15.178:6812/1880801124","cluster_addr":"172.21.15.178:6816/1880801124","heartbeat_back_addr":"172.21.15.178:6824/1880801124","heartbeat_front_addr":"172.21.15.178:6820/1880801124","state":["exists","up"]},{"osd":2,"uuid":"30cc4110-9499-411b-94da-b3de3c28627e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6825","nonce":1880098663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6827","nonce":1880098663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6831","nonce":1880098663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6829","nonce":1880098663}]},"public_addr":"172.21.15.178:6825/1880098663","cluster_addr":"172.21.15.178:6827/1880098663","heartbeat_back_addr":"172.21.15.178:6831/1880098663","heartbeat_front_addr":"172.21.15.178:6829/1880098663","state":["exists","up"]},{"osd":3,"uuid":"ec1f89b8-fcca-46c9-ba09-4cda48a142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6809","nonce":4178996824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6811","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6813","nonce":4178996824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6821","nonce":4178996824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6815","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6817","nonce":4178996824}]},"public_addr":"172.21.15.178:6809/4178996824","cluster_addr":"172.21.15.178:6813/4178996824","heartbeat_back_addr":"172.21.15.178:6821/4178996824","heartbeat_front_addr":"172.21.15.178:6817/4178996824","state":["exists","up"]},{"osd":4,"uuid":"4fc9b653-922a-4c3c-89e7-a347e7553393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6817","nonce":2928422746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6819","nonce":2928422746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6823","nonce":2928422746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6821","nonce":2928422746}]},"public_addr":"172.21.15.195:6817/2928422746","cluster_addr":"172.21.15.195:6819/2928422746","heartbeat_back_addr":"172.21.15.195:6823/2928422746","heartbeat_front_addr":"172.21.15.195:6821/2928422746","state":["exists","up"]},{"osd":5,"uuid":"7a56dd0f-6d19-4c74-9108-d4cfdf51a36b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6801","nonce":2644688681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6805","nonce":2644688681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6813","nonce":2644688681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6809","nonce":2644688681}]},"public_addr":"172.21.15.195:6801/2644688681","cluster_addr":"172.21.15.195:6805/2644688681","heartbeat_back_addr":"172.21.15.195:6813/2644688681","heartbeat_front_addr":"172.21.15.195:6809/2644688681","state":["exists","up"]},{"osd":6,"uuid":"dcb3e382-9ad2-4ae2-9e96-332a581ec20e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6804","nonce":1595562688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6808","nonce":1595562688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6815","nonce":1595562688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6812","nonce":1595562688}]},"public_addr":"172.21.15.195:6804/1595562688","cluster_addr":"172.21.15.195:6808/1595562688","heartbeat_back_addr":"172.21.15.195:6815/1595562688","heartbeat_front_addr":"172.21.15.195:6812/1595562688","state":["exists","up"]},{"osd":7,"uuid":"18053f3f-16ae-42af-a795-89cec10f661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6825","nonce":1278553753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6827","nonce":1278553753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6831","nonce":1278553753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6829","nonce":1278553753}]},"public_addr":"172.21.15.195:6825/1278553753","cluster_addr":"172.21.15.195:6827/1278553753","heartbeat_back_addr":"172.21.15.195:6831/1278553753","heartbeat_front_addr":"172.21.15.195:6829/1278553753","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.628549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.672292+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.714676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.718739+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.580848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.636756+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.636054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.668751+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-28T15:00:07.599 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-06-28T15:00:07.600 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-06-28T15:00:07.600 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-06-28T15:00:07.601 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-06-28T15:00:07.601 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-06-28T15:00:07.602 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-06-28T15:00:07.602 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-06-28T15:00:07.602 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-06-28T15:00:07.852 INFO:teuthology.orchestra.run.smithi178.stdout:55834574851 2024-06-28T15:00:07.853 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-06-28T15:00:07.872 INFO:teuthology.orchestra.run.smithi178.stdout:55834574851 2024-06-28T15:00:07.872 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-06-28T15:00:07.873 INFO:teuthology.orchestra.run.smithi178.stdout:55834574850 2024-06-28T15:00:07.874 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-06-28T15:00:07.878 INFO:teuthology.orchestra.run.smithi178.stdout:55834574850 2024-06-28T15:00:07.878 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-06-28T15:00:07.880 INFO:teuthology.orchestra.run.smithi178.stdout:55834574851 2024-06-28T15:00:07.880 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-06-28T15:00:07.885 INFO:teuthology.orchestra.run.smithi178.stdout:55834574851 2024-06-28T15:00:07.886 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-06-28T15:00:07.887 INFO:teuthology.orchestra.run.smithi178.stdout:55834574852 2024-06-28T15:00:07.888 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-06-28T15:00:07.900 INFO:teuthology.orchestra.run.smithi178.stdout:55834574852 2024-06-28T15:00:07.900 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-06-28T15:00:08.359 INFO:teuthology.orchestra.run.smithi178.stdout:55834574850 2024-06-28T15:00:08.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2024-06-28T15:00:08.393 INFO:teuthology.orchestra.run.smithi178.stdout:55834574849 2024-06-28T15:00:08.413 INFO:teuthology.orchestra.run.smithi178.stdout:55834574850 2024-06-28T15:00:08.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.0 2024-06-28T15:00:08.420 INFO:teuthology.orchestra.run.smithi178.stdout:55834574850 2024-06-28T15:00:08.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.1 2024-06-28T15:00:08.441 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2024-06-28T15:00:08.441 INFO:teuthology.orchestra.run.smithi178.stdout:55834574849 2024-06-28T15:00:08.441 INFO:teuthology.orchestra.run.smithi178.stdout:55834574850 2024-06-28T15:00:08.468 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.6 2024-06-28T15:00:08.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2024-06-28T15:00:08.470 INFO:teuthology.orchestra.run.smithi178.stdout:55834574851 2024-06-28T15:00:08.487 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.4 2024-06-28T15:00:08.492 INFO:teuthology.orchestra.run.smithi178.stdout:55834574851 2024-06-28T15:00:08.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.2 2024-06-28T15:00:09.382 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-06-28T15:00:09.419 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-06-28T15:00:09.437 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-06-28T15:00:09.442 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-06-28T15:00:09.469 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-06-28T15:00:09.470 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-06-28T15:00:09.488 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-06-28T15:00:09.509 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-06-28T15:00:09.893 INFO:teuthology.orchestra.run.smithi178.stdout:55834574851 2024-06-28T15:00:09.920 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.3 2024-06-28T15:00:09.920 DEBUG:teuthology.parallel:result is None 2024-06-28T15:00:09.928 INFO:teuthology.orchestra.run.smithi178.stdout:55834574850 2024-06-28T15:00:09.954 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.0 2024-06-28T15:00:09.954 DEBUG:teuthology.parallel:result is None 2024-06-28T15:00:09.963 INFO:teuthology.orchestra.run.smithi178.stdout:55834574851 2024-06-28T15:00:09.966 INFO:teuthology.orchestra.run.smithi178.stdout:55834574850 2024-06-28T15:00:09.991 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2024-06-28T15:00:09.991 DEBUG:teuthology.parallel:result is None 2024-06-28T15:00:09.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.6 2024-06-28T15:00:09.993 DEBUG:teuthology.parallel:result is None 2024-06-28T15:00:10.015 INFO:teuthology.orchestra.run.smithi178.stdout:55834574852 2024-06-28T15:00:10.017 INFO:teuthology.orchestra.run.smithi178.stdout:55834574851 2024-06-28T15:00:10.018 INFO:teuthology.orchestra.run.smithi178.stdout:55834574852 2024-06-28T15:00:10.027 INFO:teuthology.orchestra.run.smithi178.stdout:55834574851 2024-06-28T15:00:10.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2024-06-28T15:00:10.040 DEBUG:teuthology.parallel:result is None 2024-06-28T15:00:10.043 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.1 2024-06-28T15:00:10.044 DEBUG:teuthology.parallel:result is None 2024-06-28T15:00:10.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2024-06-28T15:00:10.044 DEBUG:teuthology.parallel:result is None 2024-06-28T15:00:10.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2024-06-28T15:00:10.050 DEBUG:teuthology.parallel:result is None 2024-06-28T15:00:10.050 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-06-28T15:00:10.050 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-28T15:00:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:00:10.361 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-06-28T15:00:10.378 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-06-28T15:00:08.662181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":248184,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754726536,"statfs":{"total":773094113280,"available":772839972864,"internally_reserved":0,"allocated":2433024,"data_stored":1563816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":251055667},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.294175+0000","last_change":"2024-06-28T15:00:06.284169+0000","last_active":"2024-06-28T15:00:07.294175+0000","last_peered":"2024-06-28T15:00:07.294175+0000","last_clean":"2024-06-28T15:00:07.294175+0000","last_became_active":"2024-06-28T15:00:06.283380+0000","last_became_peered":"2024-06-28T15:00:06.283380+0000","last_unstale":"2024-06-28T15:00:07.294175+0000","last_undegraded":"2024-06-28T15:00:07.294175+0000","last_fullsized":"2024-06-28T15:00:07.294175+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:39:59.368666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.291939+0000","last_change":"2024-06-28T15:00:06.278832+0000","last_active":"2024-06-28T15:00:07.291939+0000","last_peered":"2024-06-28T15:00:07.291939+0000","last_clean":"2024-06-28T15:00:07.291939+0000","last_became_active":"2024-06-28T15:00:06.278443+0000","last_became_peered":"2024-06-28T15:00:06.278443+0000","last_unstale":"2024-06-28T15:00:07.291939+0000","last_undegraded":"2024-06-28T15:00:07.291939+0000","last_fullsized":"2024-06-28T15:00:07.291939+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:40:27.400018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.304590+0000","last_change":"2024-06-28T15:00:06.303984+0000","last_active":"2024-06-28T15:00:07.304590+0000","last_peered":"2024-06-28T15:00:07.304590+0000","last_clean":"2024-06-28T15:00:07.304590+0000","last_became_active":"2024-06-28T15:00:06.302962+0000","last_became_peered":"2024-06-28T15:00:06.302962+0000","last_unstale":"2024-06-28T15:00:07.304590+0000","last_undegraded":"2024-06-28T15:00:07.304590+0000","last_fullsized":"2024-06-28T15:00:07.304590+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:10:47.354122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.292034+0000","last_change":"2024-06-28T15:00:06.302842+0000","last_active":"2024-06-28T15:00:07.292034+0000","last_peered":"2024-06-28T15:00:07.292034+0000","last_clean":"2024-06-28T15:00:07.292034+0000","last_became_active":"2024-06-28T15:00:06.302449+0000","last_became_peered":"2024-06-28T15:00:06.302449+0000","last_unstale":"2024-06-28T15:00:07.292034+0000","last_undegraded":"2024-06-28T15:00:07.292034+0000","last_fullsized":"2024-06-28T15:00:07.292034+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:06:10.739681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.304855+0000","last_change":"2024-06-28T15:00:06.284111+0000","last_active":"2024-06-28T15:00:07.304855+0000","last_peered":"2024-06-28T15:00:07.304855+0000","last_clean":"2024-06-28T15:00:07.304855+0000","last_became_active":"2024-06-28T15:00:06.283106+0000","last_became_peered":"2024-06-28T15:00:06.283106+0000","last_unstale":"2024-06-28T15:00:07.304855+0000","last_undegraded":"2024-06-28T15:00:07.304855+0000","last_fullsized":"2024-06-28T15:00:07.304855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:07.069940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.303324+0000","last_change":"2024-06-28T15:00:06.280312+0000","last_active":"2024-06-28T15:00:07.303324+0000","last_peered":"2024-06-28T15:00:07.303324+0000","last_clean":"2024-06-28T15:00:07.303324+0000","last_became_active":"2024-06-28T15:00:06.279950+0000","last_became_peered":"2024-06-28T15:00:06.279950+0000","last_unstale":"2024-06-28T15:00:07.303324+0000","last_undegraded":"2024-06-28T15:00:07.303324+0000","last_fullsized":"2024-06-28T15:00:07.303324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:05:40.110987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.304483+0000","last_change":"2024-06-28T15:00:06.286231+0000","last_active":"2024-06-28T15:00:07.304483+0000","last_peered":"2024-06-28T15:00:07.304483+0000","last_clean":"2024-06-28T15:00:07.304483+0000","last_became_active":"2024-06-28T15:00:06.285489+0000","last_became_peered":"2024-06-28T15:00:06.285489+0000","last_unstale":"2024-06-28T15:00:07.304483+0000","last_undegraded":"2024-06-28T15:00:07.304483+0000","last_fullsized":"2024-06-28T15:00:07.304483+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:05:17.528909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.304797+0000","last_change":"2024-06-28T15:00:06.285125+0000","last_active":"2024-06-28T15:00:07.304797+0000","last_peered":"2024-06-28T15:00:07.304797+0000","last_clean":"2024-06-28T15:00:07.304797+0000","last_became_active":"2024-06-28T15:00:06.284218+0000","last_became_peered":"2024-06-28T15:00:06.284218+0000","last_unstale":"2024-06-28T15:00:07.304797+0000","last_undegraded":"2024-06-28T15:00:07.304797+0000","last_fullsized":"2024-06-28T15:00:07.304797+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:04:51.635552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":102,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.304588+0000","last_change":"2024-06-28T15:00:06.303487+0000","last_active":"2024-06-28T15:00:07.304588+0000","last_peered":"2024-06-28T15:00:07.304588+0000","last_clean":"2024-06-28T15:00:07.304588+0000","last_became_active":"2024-06-28T15:00:06.302725+0000","last_became_peered":"2024-06-28T15:00:06.302725+0000","last_unstale":"2024-06-28T15:00:07.304588+0000","last_undegraded":"2024-06-28T15:00:07.304588+0000","last_fullsized":"2024-06-28T15:00:07.304588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:45:32.083226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":749568,"data_stored":638253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574850,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":47885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":155648,"data_stored":47885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":47885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":47885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":47885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":47885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574850,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":749568,"data_stored":638253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-28T15:00:10.379 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-28T15:00:10.662 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:00:10.662 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-06-28T15:00:10.679 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-06-28T15:00:10.662774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":248184,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754726536,"statfs":{"total":773094113280,"available":772839972864,"internally_reserved":0,"allocated":2433024,"data_stored":1563816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":251055667},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000593"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.294175+0000","last_change":"2024-06-28T15:00:06.284169+0000","last_active":"2024-06-28T15:00:07.294175+0000","last_peered":"2024-06-28T15:00:07.294175+0000","last_clean":"2024-06-28T15:00:07.294175+0000","last_became_active":"2024-06-28T15:00:06.283380+0000","last_became_peered":"2024-06-28T15:00:06.283380+0000","last_unstale":"2024-06-28T15:00:07.294175+0000","last_undegraded":"2024-06-28T15:00:07.294175+0000","last_fullsized":"2024-06-28T15:00:07.294175+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:39:59.368666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.291939+0000","last_change":"2024-06-28T15:00:06.278832+0000","last_active":"2024-06-28T15:00:07.291939+0000","last_peered":"2024-06-28T15:00:07.291939+0000","last_clean":"2024-06-28T15:00:07.291939+0000","last_became_active":"2024-06-28T15:00:06.278443+0000","last_became_peered":"2024-06-28T15:00:06.278443+0000","last_unstale":"2024-06-28T15:00:07.291939+0000","last_undegraded":"2024-06-28T15:00:07.291939+0000","last_fullsized":"2024-06-28T15:00:07.291939+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:40:27.400018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.304590+0000","last_change":"2024-06-28T15:00:06.303984+0000","last_active":"2024-06-28T15:00:07.304590+0000","last_peered":"2024-06-28T15:00:07.304590+0000","last_clean":"2024-06-28T15:00:07.304590+0000","last_became_active":"2024-06-28T15:00:06.302962+0000","last_became_peered":"2024-06-28T15:00:06.302962+0000","last_unstale":"2024-06-28T15:00:07.304590+0000","last_undegraded":"2024-06-28T15:00:07.304590+0000","last_fullsized":"2024-06-28T15:00:07.304590+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:10:47.354122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.292034+0000","last_change":"2024-06-28T15:00:06.302842+0000","last_active":"2024-06-28T15:00:07.292034+0000","last_peered":"2024-06-28T15:00:07.292034+0000","last_clean":"2024-06-28T15:00:07.292034+0000","last_became_active":"2024-06-28T15:00:06.302449+0000","last_became_peered":"2024-06-28T15:00:06.302449+0000","last_unstale":"2024-06-28T15:00:07.292034+0000","last_undegraded":"2024-06-28T15:00:07.292034+0000","last_fullsized":"2024-06-28T15:00:07.292034+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:06:10.739681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.304855+0000","last_change":"2024-06-28T15:00:06.284111+0000","last_active":"2024-06-28T15:00:07.304855+0000","last_peered":"2024-06-28T15:00:07.304855+0000","last_clean":"2024-06-28T15:00:07.304855+0000","last_became_active":"2024-06-28T15:00:06.283106+0000","last_became_peered":"2024-06-28T15:00:06.283106+0000","last_unstale":"2024-06-28T15:00:07.304855+0000","last_undegraded":"2024-06-28T15:00:07.304855+0000","last_fullsized":"2024-06-28T15:00:07.304855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:07.069940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.303324+0000","last_change":"2024-06-28T15:00:06.280312+0000","last_active":"2024-06-28T15:00:07.303324+0000","last_peered":"2024-06-28T15:00:07.303324+0000","last_clean":"2024-06-28T15:00:07.303324+0000","last_became_active":"2024-06-28T15:00:06.279950+0000","last_became_peered":"2024-06-28T15:00:06.279950+0000","last_unstale":"2024-06-28T15:00:07.303324+0000","last_undegraded":"2024-06-28T15:00:07.303324+0000","last_fullsized":"2024-06-28T15:00:07.303324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:05:40.110987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.304483+0000","last_change":"2024-06-28T15:00:06.286231+0000","last_active":"2024-06-28T15:00:07.304483+0000","last_peered":"2024-06-28T15:00:07.304483+0000","last_clean":"2024-06-28T15:00:07.304483+0000","last_became_active":"2024-06-28T15:00:06.285489+0000","last_became_peered":"2024-06-28T15:00:06.285489+0000","last_unstale":"2024-06-28T15:00:07.304483+0000","last_undegraded":"2024-06-28T15:00:07.304483+0000","last_fullsized":"2024-06-28T15:00:07.304483+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:05:17.528909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.304797+0000","last_change":"2024-06-28T15:00:06.285125+0000","last_active":"2024-06-28T15:00:07.304797+0000","last_peered":"2024-06-28T15:00:07.304797+0000","last_clean":"2024-06-28T15:00:07.304797+0000","last_became_active":"2024-06-28T15:00:06.284218+0000","last_became_peered":"2024-06-28T15:00:06.284218+0000","last_unstale":"2024-06-28T15:00:07.304797+0000","last_undegraded":"2024-06-28T15:00:07.304797+0000","last_fullsized":"2024-06-28T15:00:07.304797+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:04:51.635552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":102,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-06-28T15:00:07.304588+0000","last_change":"2024-06-28T15:00:06.303487+0000","last_active":"2024-06-28T15:00:07.304588+0000","last_peered":"2024-06-28T15:00:07.304588+0000","last_clean":"2024-06-28T15:00:07.304588+0000","last_became_active":"2024-06-28T15:00:06.302725+0000","last_became_peered":"2024-06-28T15:00:06.302725+0000","last_unstale":"2024-06-28T15:00:07.304588+0000","last_undegraded":"2024-06-28T15:00:07.304588+0000","last_fullsized":"2024-06-28T15:00:07.304588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:45:32.083226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":749568,"data_stored":638253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":47885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":155648,"data_stored":47885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":47885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":47885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":47885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":47885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":749568,"data_stored":638253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-28T15:00:10.679 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-06-28T15:00:10.680 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-28T15:00:10.680 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-06-28T15:00:10.680 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-06-28T15:00:10.985 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:00:10.985 INFO:teuthology.orchestra.run.smithi178.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-28T15:00:11.003 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-06-28T15:00:11.003 INFO:teuthology.run_tasks:Running task openssl_keys... 2024-06-28T15:00:11.017 INFO:teuthology.run_tasks:Running task rgw... 2024-06-28T15:00:11.069 DEBUG:tasks.rgw:config is {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2024-06-28T15:00:11.069 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-06-28T15:00:11.069 INFO:tasks.rgw:Creating data pools 2024-06-28T15:00:11.069 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-06-28T15:00:11.069 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph osd erasure-code-profile set client.0 crush-failure-domain=osd k=3 m=1 name=testprofile --cluster ceph 2024-06-28T15:00:12.325 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2024-06-28T15:00:14.326 INFO:teuthology.orchestra.run.smithi178.stderr:pool 'default.rgw.buckets.data' created 2024-06-28T15:00:14.359 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-06-28T15:00:15.323 INFO:teuthology.orchestra.run.smithi178.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-06-28T15:00:15.370 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-06-28T15:00:16.333 INFO:teuthology.orchestra.run.smithi178.stderr:pool 'default.rgw.buckets.index' created 2024-06-28T15:00:16.370 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-06-28T15:00:17.353 INFO:teuthology.orchestra.run.smithi178.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-06-28T15:00:17.396 DEBUG:tasks.rgw:Pools created 2024-06-28T15:00:17.396 INFO:tasks.rgw:Configuring compression type = random 2024-06-28T15:00:17.396 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-06-28T15:00:17.396 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-06-28T15:00:17.397 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-06-28T15:00:17.447 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:00:17.448 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:00:19.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.446+0000 7fb453c5f640 0 -- 172.21.15.178:0/1399670524 >> [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] conn(0x7fb32806f690 msgr2=0x7fb328071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:00:19.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.454+0000 7fb45594d980 20 rados->read ofs=0 len=0 2024-06-28T15:00:19.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.454+0000 7fb45345e640 0 -- 172.21.15.178:0/2567412185 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x558636e2df00 msgr2=0x558636e303c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:00:19.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.658+0000 7fb45594d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:19.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.658+0000 7fb45594d980 20 realm 2024-06-28T15:00:19.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.658+0000 7fb45594d980 20 rados->read ofs=0 len=0 2024-06-28T15:00:19.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.658+0000 7fb45594d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:19.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.658+0000 7fb45594d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:00:19.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.658+0000 7fb45594d980 20 rados->read ofs=0 len=0 2024-06-28T15:00:19.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.658+0000 7fb45594d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:19.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.658+0000 7fb45594d980 20 rados->read ofs=0 len=0 2024-06-28T15:00:19.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.662+0000 7fb45594d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:00:19.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.662+0000 7fb45594d980 20 rados->read ofs=0 len=0 2024-06-28T15:00:19.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.662+0000 7fb45594d980 20 rados_obj.operate() r=0 bl.length=993 2024-06-28T15:00:19.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.662+0000 7fb45594d980 20 searching for the correct realm 2024-06-28T15:00:19.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.694+0000 7fb45594d980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:00:19.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.694+0000 7fb45594d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:00:19.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.694+0000 7fb45594d980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:00:19.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.694+0000 7fb45594d980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:00:19.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.694+0000 7fb45594d980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:00:19.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.694+0000 7fb45594d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:00:19.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.694+0000 7fb45594d980 20 rados->read ofs=0 len=0 2024-06-28T15:00:19.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.694+0000 7fb45594d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:19.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.694+0000 7fb45594d980 20 rados->read ofs=0 len=0 2024-06-28T15:00:19.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.698+0000 7fb45594d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:00:19.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.698+0000 7fb45594d980 20 rados->read ofs=0 len=0 2024-06-28T15:00:19.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.698+0000 7fb45594d980 20 rados_obj.operate() r=0 bl.length=436 2024-06-28T15:00:19.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.698+0000 7fb45594d980 20 zone default found 2024-06-28T15:00:19.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.698+0000 7fb45594d980 4 Realm: () 2024-06-28T15:00:19.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.698+0000 7fb45594d980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:00:19.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.698+0000 7fb45594d980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:00:19.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.698+0000 7fb45594d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:00:19.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.698+0000 7fb45594d980 20 zonegroup default 2024-06-28T15:00:19.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.698+0000 7fb45594d980 20 rados->read ofs=0 len=0 2024-06-28T15:00:19.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.698+0000 7fb45594d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:19.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:19.698+0000 7fb45594d980 20 rados->read ofs=0 len=0 2024-06-28T15:00:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:21.394+0000 7fb45594d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:21.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:21.394+0000 7fb45594d980 20 rados->read ofs=0 len=0 2024-06-28T15:00:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:21.398+0000 7fb45594d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:21.398+0000 7fb45594d980 20 started sync module instance, tier type = 2024-06-28T15:00:21.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:21.398+0000 7fb45594d980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:00:23.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.442+0000 7fb45594d980 20 add_watcher() i=0 2024-06-28T15:00:23.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.442+0000 7fb45594d980 20 add_watcher() i=1 2024-06-28T15:00:23.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.442+0000 7fb45594d980 20 add_watcher() i=2 2024-06-28T15:00:23.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.442+0000 7fb45594d980 20 add_watcher() i=3 2024-06-28T15:00:23.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.442+0000 7fb45594d980 20 add_watcher() i=4 2024-06-28T15:00:23.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.442+0000 7fb45594d980 20 add_watcher() i=5 2024-06-28T15:00:23.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.442+0000 7fb45594d980 20 add_watcher() i=6 2024-06-28T15:00:23.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 add_watcher() i=7 2024-06-28T15:00:23.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 2 all 8 watchers are set, enabling cache 2024-06-28T15:00:23.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 5 tl::expected, obj_version>, boost::system::error_code> logback_generations::read(const DoutPrefixProvider*, optional_yield):410: oid=data_loggenerations_metadata not found 2024-06-28T15:00:23.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.0 2024-06-28T15:00:23.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.0 does not exist 2024-06-28T15:00:23.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.1 2024-06-28T15:00:23.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.1 does not exist 2024-06-28T15:00:23.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.2 2024-06-28T15:00:23.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.2 does not exist 2024-06-28T15:00:23.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.3 2024-06-28T15:00:23.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.3 does not exist 2024-06-28T15:00:23.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.4 2024-06-28T15:00:23.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.446+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.4 does not exist 2024-06-28T15:00:23.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.5 2024-06-28T15:00:23.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.5 does not exist 2024-06-28T15:00:23.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.6 2024-06-28T15:00:23.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.6 does not exist 2024-06-28T15:00:23.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.7 2024-06-28T15:00:23.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.7 does not exist 2024-06-28T15:00:23.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.8 2024-06-28T15:00:23.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.8 does not exist 2024-06-28T15:00:23.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.9 2024-06-28T15:00:23.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.9 does not exist 2024-06-28T15:00:23.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.10 2024-06-28T15:00:23.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.10 does not exist 2024-06-28T15:00:23.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.11 2024-06-28T15:00:23.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.11 does not exist 2024-06-28T15:00:23.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.12 2024-06-28T15:00:23.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.450+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.12 does not exist 2024-06-28T15:00:23.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.13 2024-06-28T15:00:23.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.13 does not exist 2024-06-28T15:00:23.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.14 2024-06-28T15:00:23.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.14 does not exist 2024-06-28T15:00:23.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.15 2024-06-28T15:00:23.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.15 does not exist 2024-06-28T15:00:23.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.16 2024-06-28T15:00:23.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.16 does not exist 2024-06-28T15:00:23.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.17 2024-06-28T15:00:23.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.17 does not exist 2024-06-28T15:00:23.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.18 2024-06-28T15:00:23.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.18 does not exist 2024-06-28T15:00:23.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.19 2024-06-28T15:00:23.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.454+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.19 does not exist 2024-06-28T15:00:23.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.20 2024-06-28T15:00:23.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.20 does not exist 2024-06-28T15:00:23.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.21 2024-06-28T15:00:23.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.21 does not exist 2024-06-28T15:00:23.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.22 2024-06-28T15:00:23.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.22 does not exist 2024-06-28T15:00:23.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.23 2024-06-28T15:00:23.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.23 does not exist 2024-06-28T15:00:23.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.24 2024-06-28T15:00:23.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.24 does not exist 2024-06-28T15:00:23.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.25 2024-06-28T15:00:23.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.25 does not exist 2024-06-28T15:00:23.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.26 2024-06-28T15:00:23.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.26 does not exist 2024-06-28T15:00:23.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.27 2024-06-28T15:00:23.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.458+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.27 does not exist 2024-06-28T15:00:23.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.28 2024-06-28T15:00:23.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.28 does not exist 2024-06-28T15:00:23.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.29 2024-06-28T15:00:23.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.29 does not exist 2024-06-28T15:00:23.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.30 2024-06-28T15:00:23.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.30 does not exist 2024-06-28T15:00:23.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.31 2024-06-28T15:00:23.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.31 does not exist 2024-06-28T15:00:23.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.32 2024-06-28T15:00:23.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.32 does not exist 2024-06-28T15:00:23.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.33 2024-06-28T15:00:23.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.33 does not exist 2024-06-28T15:00:23.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.34 2024-06-28T15:00:23.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.462+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.34 does not exist 2024-06-28T15:00:23.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.35 2024-06-28T15:00:23.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.35 does not exist 2024-06-28T15:00:23.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.36 2024-06-28T15:00:23.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.36 does not exist 2024-06-28T15:00:23.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.37 2024-06-28T15:00:23.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.37 does not exist 2024-06-28T15:00:23.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.38 2024-06-28T15:00:23.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.38 does not exist 2024-06-28T15:00:23.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.39 2024-06-28T15:00:23.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.39 does not exist 2024-06-28T15:00:23.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.40 2024-06-28T15:00:23.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.40 does not exist 2024-06-28T15:00:23.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.41 2024-06-28T15:00:23.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.466+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.470+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.41 does not exist 2024-06-28T15:00:23.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.470+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.42 2024-06-28T15:00:23.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.470+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.470+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.42 does not exist 2024-06-28T15:00:23.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.470+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.43 2024-06-28T15:00:23.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.470+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.470+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.43 does not exist 2024-06-28T15:00:23.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.470+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.44 2024-06-28T15:00:23.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.470+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.470+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.44 does not exist 2024-06-28T15:00:23.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.470+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.45 2024-06-28T15:00:23.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.470+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.45 does not exist 2024-06-28T15:00:23.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.46 2024-06-28T15:00:23.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.46 does not exist 2024-06-28T15:00:23.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.47 2024-06-28T15:00:23.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.47 does not exist 2024-06-28T15:00:23.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.48 2024-06-28T15:00:23.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.48 does not exist 2024-06-28T15:00:23.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.49 2024-06-28T15:00:23.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.49 does not exist 2024-06-28T15:00:23.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.50 2024-06-28T15:00:23.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.50 does not exist 2024-06-28T15:00:23.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.51 2024-06-28T15:00:23.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.51 does not exist 2024-06-28T15:00:23.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.52 2024-06-28T15:00:23.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.52 does not exist 2024-06-28T15:00:23.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.53 2024-06-28T15:00:23.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.474+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.53 does not exist 2024-06-28T15:00:23.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.54 2024-06-28T15:00:23.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.54 does not exist 2024-06-28T15:00:23.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.55 2024-06-28T15:00:23.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.55 does not exist 2024-06-28T15:00:23.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.56 2024-06-28T15:00:23.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.56 does not exist 2024-06-28T15:00:23.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.57 2024-06-28T15:00:23.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.57 does not exist 2024-06-28T15:00:23.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.58 2024-06-28T15:00:23.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.478+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.58 does not exist 2024-06-28T15:00:23.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.59 2024-06-28T15:00:23.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.59 does not exist 2024-06-28T15:00:23.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.60 2024-06-28T15:00:23.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.60 does not exist 2024-06-28T15:00:23.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.61 2024-06-28T15:00:23.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.61 does not exist 2024-06-28T15:00:23.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.62 2024-06-28T15:00:23.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.62 does not exist 2024-06-28T15:00:23.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.63 2024-06-28T15:00:23.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.63 does not exist 2024-06-28T15:00:23.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.64 2024-06-28T15:00:23.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.64 does not exist 2024-06-28T15:00:23.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.65 2024-06-28T15:00:23.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.65 does not exist 2024-06-28T15:00:23.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.66 2024-06-28T15:00:23.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.482+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.66 does not exist 2024-06-28T15:00:23.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.67 2024-06-28T15:00:23.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.67 does not exist 2024-06-28T15:00:23.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.68 2024-06-28T15:00:23.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.68 does not exist 2024-06-28T15:00:23.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.69 2024-06-28T15:00:23.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.69 does not exist 2024-06-28T15:00:23.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.70 2024-06-28T15:00:23.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.70 does not exist 2024-06-28T15:00:23.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.71 2024-06-28T15:00:23.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.71 does not exist 2024-06-28T15:00:23.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.72 2024-06-28T15:00:23.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.72 does not exist 2024-06-28T15:00:23.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.73 2024-06-28T15:00:23.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.73 does not exist 2024-06-28T15:00:23.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.74 2024-06-28T15:00:23.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.486+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.74 does not exist 2024-06-28T15:00:23.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.75 2024-06-28T15:00:23.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.75 does not exist 2024-06-28T15:00:23.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.76 2024-06-28T15:00:23.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.76 does not exist 2024-06-28T15:00:23.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.77 2024-06-28T15:00:23.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.77 does not exist 2024-06-28T15:00:23.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.78 2024-06-28T15:00:23.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.78 does not exist 2024-06-28T15:00:23.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.79 2024-06-28T15:00:23.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.79 does not exist 2024-06-28T15:00:23.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.80 2024-06-28T15:00:23.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.80 does not exist 2024-06-28T15:00:23.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.81 2024-06-28T15:00:23.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.490+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.81 does not exist 2024-06-28T15:00:23.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.82 2024-06-28T15:00:23.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.82 does not exist 2024-06-28T15:00:23.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.83 2024-06-28T15:00:23.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.83 does not exist 2024-06-28T15:00:23.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.84 2024-06-28T15:00:23.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.84 does not exist 2024-06-28T15:00:23.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.85 2024-06-28T15:00:23.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.85 does not exist 2024-06-28T15:00:23.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.86 2024-06-28T15:00:23.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.86 does not exist 2024-06-28T15:00:23.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.87 2024-06-28T15:00:23.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.87 does not exist 2024-06-28T15:00:23.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.88 2024-06-28T15:00:23.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.494+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.88 does not exist 2024-06-28T15:00:23.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.89 2024-06-28T15:00:23.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.89 does not exist 2024-06-28T15:00:23.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.90 2024-06-28T15:00:23.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.90 does not exist 2024-06-28T15:00:23.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.91 2024-06-28T15:00:23.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.91 does not exist 2024-06-28T15:00:23.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.92 2024-06-28T15:00:23.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.92 does not exist 2024-06-28T15:00:23.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.93 2024-06-28T15:00:23.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.93 does not exist 2024-06-28T15:00:23.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.94 2024-06-28T15:00:23.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.94 does not exist 2024-06-28T15:00:23.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.95 2024-06-28T15:00:23.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.498+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.95 does not exist 2024-06-28T15:00:23.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.96 2024-06-28T15:00:23.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.96 does not exist 2024-06-28T15:00:23.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.97 2024-06-28T15:00:23.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.97 does not exist 2024-06-28T15:00:23.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.98 2024-06-28T15:00:23.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.98 does not exist 2024-06-28T15:00:23.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.99 2024-06-28T15:00:23.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.99 does not exist 2024-06-28T15:00:23.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.100 2024-06-28T15:00:23.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.100 does not exist 2024-06-28T15:00:23.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.101 2024-06-28T15:00:23.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.101 does not exist 2024-06-28T15:00:23.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.102 2024-06-28T15:00:23.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.502+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.102 does not exist 2024-06-28T15:00:23.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.103 2024-06-28T15:00:23.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.103 does not exist 2024-06-28T15:00:23.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.104 2024-06-28T15:00:23.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.104 does not exist 2024-06-28T15:00:23.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.105 2024-06-28T15:00:23.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.105 does not exist 2024-06-28T15:00:23.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.106 2024-06-28T15:00:23.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.106 does not exist 2024-06-28T15:00:23.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.107 2024-06-28T15:00:23.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.107 does not exist 2024-06-28T15:00:23.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.108 2024-06-28T15:00:23.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.108 does not exist 2024-06-28T15:00:23.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.109 2024-06-28T15:00:23.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.506+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.109 does not exist 2024-06-28T15:00:23.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.110 2024-06-28T15:00:23.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.110 does not exist 2024-06-28T15:00:23.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.111 2024-06-28T15:00:23.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.111 does not exist 2024-06-28T15:00:23.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.112 2024-06-28T15:00:23.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.112 does not exist 2024-06-28T15:00:23.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.113 2024-06-28T15:00:23.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.113 does not exist 2024-06-28T15:00:23.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.114 2024-06-28T15:00:23.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.114 does not exist 2024-06-28T15:00:23.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.115 2024-06-28T15:00:23.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.115 does not exist 2024-06-28T15:00:23.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.116 2024-06-28T15:00:23.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.510+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.116 does not exist 2024-06-28T15:00:23.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.117 2024-06-28T15:00:23.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.117 does not exist 2024-06-28T15:00:23.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.118 2024-06-28T15:00:23.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.118 does not exist 2024-06-28T15:00:23.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.119 2024-06-28T15:00:23.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.119 does not exist 2024-06-28T15:00:23.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.120 2024-06-28T15:00:23.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.120 does not exist 2024-06-28T15:00:23.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.121 2024-06-28T15:00:23.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.121 does not exist 2024-06-28T15:00:23.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.122 2024-06-28T15:00:23.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.122 does not exist 2024-06-28T15:00:23.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.123 2024-06-28T15:00:23.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.123 does not exist 2024-06-28T15:00:23.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.124 2024-06-28T15:00:23.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.514+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.518+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.124 does not exist 2024-06-28T15:00:23.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.518+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.125 2024-06-28T15:00:23.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.518+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.518+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.125 does not exist 2024-06-28T15:00:23.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.518+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.126 2024-06-28T15:00:23.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.518+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.518+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.126 does not exist 2024-06-28T15:00:23.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.518+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.127 2024-06-28T15:00:23.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.518+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.518+0000 7fb45594d980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.127 does not exist 2024-06-28T15:00:23.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.518+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::create(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, std::optional >, bool, uint64_t, uint64_t):1211 entering 2024-06-28T15:00:23.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.522+0000 7fb45594d980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-06-28T15:00:23.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.530+0000 7fb305fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:00:23.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.530+0000 7fb45594d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:00:23.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:23.530+0000 7fb45594d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:00:26.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.406+0000 7fb45594d980 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2024-06-28T15:00:26.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.406+0000 7fb45594d980 5 note: GC not initialized 2024-06-28T15:00:26.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.470+0000 7fb304fe1640 20 reqs_thread_entry: start 2024-06-28T15:00:26.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.470+0000 7fb45594d980 20 init_complete bucket index max shards: 1 2024-06-28T15:00:26.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.470+0000 7fb45594d980 20 Filter name: none 2024-06-28T15:00:26.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.470+0000 7fb26e7fc640 20 reqs_thread_entry: start 2024-06-28T15:00:26.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.494+0000 7fb305fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:00:26.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.502+0000 7fb45594d980 20 remove_watcher() i=0 2024-06-28T15:00:26.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.502+0000 7fb45594d980 2 removed watcher, disabling cache 2024-06-28T15:00:26.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.502+0000 7fb45594d980 20 remove_watcher() i=1 2024-06-28T15:00:26.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.506+0000 7fb45594d980 20 remove_watcher() i=2 2024-06-28T15:00:26.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.510+0000 7fb45594d980 20 remove_watcher() i=3 2024-06-28T15:00:26.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.514+0000 7fb45594d980 20 remove_watcher() i=4 2024-06-28T15:00:26.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.514+0000 7fb45594d980 20 remove_watcher() i=5 2024-06-28T15:00:26.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.514+0000 7fb45594d980 20 remove_watcher() i=6 2024-06-28T15:00:26.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.518+0000 7fb45594d980 20 remove_watcher() i=7 2024-06-28T15:00:26.528 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2024-06-28T15:00:26.528 DEBUG:tasks.util.rgw: json result: [] 2024-06-28T15:00:26.529 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-06-28T15:00:26.529 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-06-28T15:00:26.529 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2024-06-28T15:00:26.570 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:00:26.570 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:00:26.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.606+0000 7f7848185980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:00:26.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.606+0000 7f7848185980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:00:26.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.610+0000 7f7708fe1640 20 reqs_thread_entry: start 2024-06-28T15:00:26.624 INFO:teuthology.orchestra.run.smithi178.stdout:{"id":"843e24ce-085a-4043-8f55-38e05370e790","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-06-28T15:00:26.624 DEBUG:tasks.util.rgw: json result: {'id': '843e24ce-085a-4043-8f55-38e05370e790', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-06-28T15:00:26.624 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-06-28T15:00:26.624 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-06-28T15:00:26.625 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-06-28T15:00:26.701 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:00:26.701 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:00:26.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.750+0000 7f87969be980 20 rados->read ofs=0 len=0 2024-06-28T15:00:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.750+0000 7f87969be980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.750+0000 7f87969be980 20 realm 2024-06-28T15:00:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.750+0000 7f87969be980 20 rados->read ofs=0 len=0 2024-06-28T15:00:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.750+0000 7f87969be980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.750+0000 7f87969be980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:00:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.750+0000 7f87969be980 20 rados->read ofs=0 len=0 2024-06-28T15:00:26.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.750+0000 7f87969be980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:26.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.750+0000 7f87969be980 20 rados->read ofs=0 len=0 2024-06-28T15:00:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.754+0000 7f87969be980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:00:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.754+0000 7f87969be980 20 rados->read ofs=0 len=0 2024-06-28T15:00:26.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.754+0000 7f87969be980 20 rados_obj.operate() r=0 bl.length=1009 2024-06-28T15:00:26.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.754+0000 7f87969be980 20 searching for the correct realm 2024-06-28T15:00:26.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.762+0000 7f8794cd0640 0 -- 172.21.15.178:0/2047171413 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f877c1328e0 msgr2=0x7f877c1340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:00:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.978+0000 7f87944cf640 0 -- 172.21.15.178:0/2047171413 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55f5ef2c7bc0 msgr2=0x55f5ef2ca080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:00:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.986+0000 7f87969be980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:00:26.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.986+0000 7f87969be980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:00:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.986+0000 7f87969be980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:00:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.986+0000 7f87969be980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:00:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.986+0000 7f87969be980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:00:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.986+0000 7f87969be980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:00:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.986+0000 7f87969be980 20 rados->read ofs=0 len=0 2024-06-28T15:00:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.986+0000 7f87969be980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.986+0000 7f87969be980 20 rados->read ofs=0 len=0 2024-06-28T15:00:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.986+0000 7f87969be980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:00:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.986+0000 7f87969be980 20 rados->read ofs=0 len=0 2024-06-28T15:00:26.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 20 rados_obj.operate() r=0 bl.length=436 2024-06-28T15:00:26.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 20 zone default found 2024-06-28T15:00:26.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 4 Realm: () 2024-06-28T15:00:26.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:00:26.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:00:26.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:00:26.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 20 zonegroup default 2024-06-28T15:00:26.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 20 rados->read ofs=0 len=0 2024-06-28T15:00:26.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:26.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 20 rados->read ofs=0 len=0 2024-06-28T15:00:26.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:26.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 20 rados->read ofs=0 len=0 2024-06-28T15:00:26.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:26.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 20 started sync module instance, tier type = 2024-06-28T15:00:26.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:26.990+0000 7f87969be980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:00:27.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.014+0000 7f87969be980 20 add_watcher() i=0 2024-06-28T15:00:27.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.014+0000 7f87969be980 20 add_watcher() i=1 2024-06-28T15:00:27.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.014+0000 7f87969be980 20 add_watcher() i=2 2024-06-28T15:00:27.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.014+0000 7f87969be980 20 add_watcher() i=3 2024-06-28T15:00:27.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.014+0000 7f87969be980 20 add_watcher() i=4 2024-06-28T15:00:27.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.014+0000 7f87969be980 20 add_watcher() i=5 2024-06-28T15:00:27.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.014+0000 7f87969be980 20 add_watcher() i=6 2024-06-28T15:00:27.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.018+0000 7f87969be980 20 add_watcher() i=7 2024-06-28T15:00:27.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.018+0000 7f87969be980 2 all 8 watchers are set, enabling cache 2024-06-28T15:00:27.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.022+0000 7f864afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:00:27.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.022+0000 7f87969be980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:00:27.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.022+0000 7f87969be980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:00:27.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.022+0000 7f87969be980 5 note: GC not initialized 2024-06-28T15:00:27.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.022+0000 7f8649fdb640 20 reqs_thread_entry: start 2024-06-28T15:00:27.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.022+0000 7f87969be980 20 init_complete bucket index max shards: 1 2024-06-28T15:00:27.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.022+0000 7f87969be980 20 Filter name: none 2024-06-28T15:00:27.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.026+0000 7f85bb7fe640 20 reqs_thread_entry: start 2024-06-28T15:00:27.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.046+0000 7f864afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:00:27.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.050+0000 7f87969be980 20 remove_watcher() i=0 2024-06-28T15:00:27.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.050+0000 7f87969be980 2 removed watcher, disabling cache 2024-06-28T15:00:27.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.050+0000 7f87969be980 20 remove_watcher() i=1 2024-06-28T15:00:27.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.054+0000 7f87969be980 20 remove_watcher() i=2 2024-06-28T15:00:27.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.058+0000 7f87969be980 20 remove_watcher() i=3 2024-06-28T15:00:27.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.058+0000 7f87969be980 20 remove_watcher() i=4 2024-06-28T15:00:27.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.062+0000 7f87969be980 20 remove_watcher() i=5 2024-06-28T15:00:27.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.062+0000 7f87969be980 20 remove_watcher() i=6 2024-06-28T15:00:27.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.062+0000 7f87969be980 20 remove_watcher() i=7 2024-06-28T15:00:27.070 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2024-06-28T15:00:27.070 DEBUG:tasks.util.rgw: json result: [] 2024-06-28T15:00:27.070 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--placement-inline-data', 'false'] 2024-06-28T15:00:27.070 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--placement-inline-data', 'false'] 2024-06-28T15:00:27.070 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --placement-inline-data false 2024-06-28T15:00:27.152 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:00:27.153 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:00:27.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.190+0000 7fa17cdf1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:00:27.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.190+0000 7fa17cdf1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:00:27.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.194+0000 7fa03d7e2640 20 reqs_thread_entry: start 2024-06-28T15:00:27.209 INFO:teuthology.orchestra.run.smithi178.stdout:{"id":"843e24ce-085a-4043-8f55-38e05370e790","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-06-28T15:00:27.209 DEBUG:tasks.util.rgw: json result: {'id': '843e24ce-085a-4043-8f55-38e05370e790', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-06-28T15:00:27.209 INFO:tasks.rgw:Preparing directory for rgw datacache at /tmp/rgw_datacache 2024-06-28T15:00:27.209 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p /tmp/rgw_datacache 2024-06-28T15:00:27.258 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a+rwx /tmp/rgw_datacache 2024-06-28T15:00:27.313 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-06-28T15:00:27.313 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-06-28T15:00:27.313 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-06-28T15:00:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:00:27.390 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:00:27.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.438+0000 7f81062f8980 20 rados->read ofs=0 len=0 2024-06-28T15:00:27.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.438+0000 7f81062f8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:27.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.438+0000 7f81062f8980 20 realm 2024-06-28T15:00:27.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.438+0000 7f81062f8980 20 rados->read ofs=0 len=0 2024-06-28T15:00:27.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.438+0000 7f81062f8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:27.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.438+0000 7f81062f8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:00:27.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.438+0000 7f81062f8980 20 rados->read ofs=0 len=0 2024-06-28T15:00:27.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.438+0000 7f81062f8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:27.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.438+0000 7f81062f8980 20 rados->read ofs=0 len=0 2024-06-28T15:00:27.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.442+0000 7f81062f8980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:00:27.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.442+0000 7f81062f8980 20 rados->read ofs=0 len=0 2024-06-28T15:00:27.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.442+0000 7f81062f8980 20 rados_obj.operate() r=0 bl.length=1009 2024-06-28T15:00:27.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.442+0000 7f81062f8980 20 searching for the correct realm 2024-06-28T15:00:27.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.450+0000 7f8103608640 0 -- 172.21.15.178:0/1406683440 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f80f4074160 msgr2=0x7f80f408d1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:00:27.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.454+0000 7f8103608640 0 -- 172.21.15.178:0/1406683440 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f7fcc07cfc0 msgr2=0x7f7fcc07f3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:00:27.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:00:27.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:00:27.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:00:27.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:00:27.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:00:27.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:00:27.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 rados->read ofs=0 len=0 2024-06-28T15:00:27.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:27.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 rados->read ofs=0 len=0 2024-06-28T15:00:27.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:00:27.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 rados->read ofs=0 len=0 2024-06-28T15:00:27.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 rados_obj.operate() r=0 bl.length=436 2024-06-28T15:00:27.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 zone default found 2024-06-28T15:00:27.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 4 Realm: () 2024-06-28T15:00:27.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:00:27.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:00:27.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:00:27.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 zonegroup default 2024-06-28T15:00:27.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 rados->read ofs=0 len=0 2024-06-28T15:00:27.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:27.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 rados->read ofs=0 len=0 2024-06-28T15:00:27.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:27.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 rados->read ofs=0 len=0 2024-06-28T15:00:27.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:00:27.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 started sync module instance, tier type = 2024-06-28T15:00:27.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.462+0000 7f81062f8980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:00:27.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.478+0000 7f81062f8980 20 add_watcher() i=0 2024-06-28T15:00:27.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.478+0000 7f81062f8980 20 add_watcher() i=1 2024-06-28T15:00:27.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.478+0000 7f81062f8980 20 add_watcher() i=2 2024-06-28T15:00:27.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.478+0000 7f81062f8980 20 add_watcher() i=3 2024-06-28T15:00:27.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.478+0000 7f81062f8980 20 add_watcher() i=4 2024-06-28T15:00:27.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.478+0000 7f81062f8980 20 add_watcher() i=5 2024-06-28T15:00:27.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.478+0000 7f81062f8980 20 add_watcher() i=6 2024-06-28T15:00:27.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.482+0000 7f81062f8980 20 add_watcher() i=7 2024-06-28T15:00:27.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.482+0000 7f81062f8980 2 all 8 watchers are set, enabling cache 2024-06-28T15:00:27.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.482+0000 7f7fb9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:00:27.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.482+0000 7f81062f8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:00:27.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.482+0000 7f81062f8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:00:27.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.482+0000 7f81062f8980 5 note: GC not initialized 2024-06-28T15:00:27.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.482+0000 7f7fb8fe1640 20 reqs_thread_entry: start 2024-06-28T15:00:27.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.482+0000 7f81062f8980 20 init_complete bucket index max shards: 1 2024-06-28T15:00:27.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.482+0000 7f81062f8980 20 Filter name: none 2024-06-28T15:00:27.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.482+0000 7f7f327fc640 20 reqs_thread_entry: start 2024-06-28T15:00:27.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.502+0000 7f7fb9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:00:27.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.506+0000 7f81062f8980 20 remove_watcher() i=0 2024-06-28T15:00:27.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.506+0000 7f81062f8980 2 removed watcher, disabling cache 2024-06-28T15:00:27.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.506+0000 7f81062f8980 20 remove_watcher() i=1 2024-06-28T15:00:27.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.510+0000 7f81062f8980 20 remove_watcher() i=2 2024-06-28T15:00:27.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.510+0000 7f81062f8980 20 remove_watcher() i=3 2024-06-28T15:00:27.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.514+0000 7f81062f8980 20 remove_watcher() i=4 2024-06-28T15:00:27.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.514+0000 7f81062f8980 20 remove_watcher() i=5 2024-06-28T15:00:27.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.518+0000 7f81062f8980 20 remove_watcher() i=6 2024-06-28T15:00:27.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.518+0000 7f81062f8980 20 remove_watcher() i=7 2024-06-28T15:00:27.524 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2024-06-28T15:00:27.524 DEBUG:tasks.util.rgw: json result: [] 2024-06-28T15:00:27.524 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2024-06-28T15:00:27.524 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2024-06-28T15:00:27.524 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2024-06-28T15:00:27.525 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2024-06-28T15:00:27.609 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:00:27.609 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:00:27.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.646+0000 7f311d44b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:00:27.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.646+0000 7f311d44b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:00:27.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.650+0000 7f2fd8ff9640 20 reqs_thread_entry: start 2024-06-28T15:00:27.662 INFO:teuthology.orchestra.run.smithi178.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2024-06-28T15:00:27.662 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['LUKEWARM', 'STANDARD']}}] 2024-06-28T15:00:27.662 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2024-06-28T15:00:27.662 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2024-06-28T15:00:27.662 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2024-06-28T15:00:27.744 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:00:27.745 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:00:27.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.782+0000 7fc4ad820980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:00:27.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.782+0000 7fc4ad820980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:00:27.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.786+0000 7fc2f1ffb640 20 reqs_thread_entry: start 2024-06-28T15:00:27.795 INFO:teuthology.orchestra.run.smithi178.stdout:{"id":"843e24ce-085a-4043-8f55-38e05370e790","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-06-28T15:00:27.796 DEBUG:tasks.util.rgw: json result: {'id': '843e24ce-085a-4043-8f55-38e05370e790', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-06-28T15:00:27.796 INFO:tasks.rgw:Configuring storage class type = FROZEN 2024-06-28T15:00:27.796 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2024-06-28T15:00:27.796 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2024-06-28T15:00:27.796 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2024-06-28T15:00:27.876 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:00:27.876 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:00:27.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.910+0000 7f69e6036980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:00:27.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.910+0000 7f69e6036980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:00:27.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:27.914+0000 7f68a1ffb640 20 reqs_thread_entry: start 2024-06-28T15:00:27.926 INFO:teuthology.orchestra.run.smithi178.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2024-06-28T15:00:27.926 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['FROZEN', 'LUKEWARM', 'STANDARD']}}] 2024-06-28T15:00:27.926 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2024-06-28T15:00:27.926 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2024-06-28T15:00:27.926 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2024-06-28T15:00:28.005 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:00:28.005 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:00:28.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:28.042+0000 7f579f75f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:00:28.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:28.042+0000 7f579f75f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:00:28.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:00:28.046+0000 7f55db7fe640 20 reqs_thread_entry: start 2024-06-28T15:00:28.055 INFO:teuthology.orchestra.run.smithi178.stdout:{"id":"843e24ce-085a-4043-8f55-38e05370e790","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-06-28T15:00:28.056 DEBUG:tasks.util.rgw: json result: {'id': '843e24ce-085a-4043-8f55-38e05370e790', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'FROZEN': {'data_pool': 'default.rgw.buckets.data.frozen'}, 'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-06-28T15:00:28.056 INFO:tasks.rgw:Starting rgw... 2024-06-28T15:00:28.056 INFO:tasks.rgw:rgw client.0 config is {'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2024-06-28T15:00:28.056 INFO:tasks.rgw:Using beast as radosgw frontend 2024-06-28T15:00:28.056 DEBUG:teuthology.orchestra.run.smithi178:> sudo echo -n http://smithi178.front.sepia.ceph.com:80 | sudo tee /home/ubuntu/cephtest/url_file 2024-06-28T15:00:28.107 INFO:teuthology.orchestra.run.smithi178.stdout:http://smithi178.front.sepia.ceph.com:80 2024-06-28T15:00:28.107 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-06-28T15:00:28.159 DEBUG:tasks.ceph_manager:running ceph.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/ceph.client.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2024-06-28T15:00:28.159 INFO:tasks.rgw.client.0:Restarting daemon 2024-06-28T15:00:28.159 DEBUG:teuthology.orchestra.run.smithi178:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-06-28T15:00:28.201 INFO:tasks.rgw.client.0:Started 2024-06-28T15:00:28.202 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi178.front.sepia.ceph.com:80/ 2024-06-28T15:00:28.202 DEBUG:teuthology.orchestra.run.smithi178:> curl http://smithi178.front.sepia.ceph.com:80/ 2024-06-28T15:00:28.225 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-06-28T15:00:28.225 INFO:teuthology.orchestra.run.smithi178.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-06-28T15:00:28.225 INFO:teuthology.orchestra.run.smithi178.stderr: Dload Upload Total Spent Left Speed 2024-06-28T15:00:28.225 INFO:teuthology.orchestra.run.smithi178.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-06-28T15:00:28.226 INFO:teuthology.orchestra.run.smithi178.stderr:curl: (7) Failed to connect to smithi178.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-06-28T15:00:29.226 DEBUG:teuthology.orchestra.run.smithi178:> curl http://smithi178.front.sepia.ceph.com:80/ 2024-06-28T15:00:29.238 INFO:teuthology.orchestra.run.smithi178.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-06-28T15:00:29.238 INFO:teuthology.orchestra.run.smithi178.stderr: Dload Upload Total Spent Left Speed 2024-06-28T15:00:29.238 INFO:teuthology.orchestra.run.smithi178.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-06-28T15:00:29.238 INFO:teuthology.orchestra.run.smithi178.stderr:curl: (7) Failed to connect to smithi178.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-06-28T15:00:29.239 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-06-28T15:00:31.239 DEBUG:teuthology.orchestra.run.smithi178:> curl http://smithi178.front.sepia.ceph.com:80/ 2024-06-28T15:00:31.251 INFO:teuthology.orchestra.run.smithi178.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-06-28T15:00:31.251 INFO:teuthology.orchestra.run.smithi178.stderr: Dload Upload Total Spent Left Speed 2024-06-28T15:00:31.251 INFO:teuthology.orchestra.run.smithi178.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-06-28T15:00:31.251 INFO:teuthology.orchestra.run.smithi178.stderr:curl: (7) Failed to connect to smithi178.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-06-28T15:00:31.252 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-06-28T15:00:35.253 DEBUG:teuthology.orchestra.run.smithi178:> curl http://smithi178.front.sepia.ceph.com:80/ 2024-06-28T15:00:35.265 INFO:teuthology.orchestra.run.smithi178.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-06-28T15:00:35.265 INFO:teuthology.orchestra.run.smithi178.stderr: Dload Upload Total Spent Left Speed 2024-06-28T15:00:35.265 INFO:teuthology.orchestra.run.smithi178.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-06-28T15:00:35.265 INFO:teuthology.orchestra.run.smithi178.stderr:curl: (7) Failed to connect to smithi178.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-06-28T15:00:35.266 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-06-28T15:00:43.267 DEBUG:teuthology.orchestra.run.smithi178:> curl http://smithi178.front.sepia.ceph.com:80/ 2024-06-28T15:00:43.277 INFO:teuthology.orchestra.run.smithi178.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-06-28T15:00:43.277 INFO:teuthology.orchestra.run.smithi178.stderr: Dload Upload Total Spent Left Speed 2024-06-28T15:00:43.279 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-06-28T15:00:43.279 INFO:teuthology.orchestra.run.smithi178.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-06-28T15:00:43.279 INFO:teuthology.orchestra.run.smithi178.stderr:curl: (7) Failed to connect to smithi178.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-06-28T15:00:59.280 DEBUG:teuthology.orchestra.run.smithi178:> curl http://smithi178.front.sepia.ceph.com:80/ 2024-06-28T15:00:59.292 INFO:teuthology.orchestra.run.smithi178.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-06-28T15:00:59.292 INFO:teuthology.orchestra.run.smithi178.stderr: Dload Upload Total Spent Left Speed 2024-06-28T15:00:59.767 INFO:teuthology.orchestra.run.smithi178.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 393 0 --:--:-- --:--:-- --:--:-- 392 2024-06-28T15:00:59.768 INFO:teuthology.orchestra.run.smithi178.stdout:anonymous 2024-06-28T15:00:59.769 INFO:teuthology.run_tasks:Running task tox... 2024-06-28T15:00:59.781 INFO:tasks.tox:Deploying tox from pip... 2024-06-28T15:00:59.781 DEBUG:teuthology.orchestra.run.smithi178:> python3 -m venv /home/ubuntu/cephtest/tox-venv 2024-06-28T15:01:02.801 DEBUG:teuthology.orchestra.run.smithi178:> source /home/ubuntu/cephtest/tox-venv/bin/activate && pip install tox==3.15.0 2024-06-28T15:01:03.555 INFO:teuthology.orchestra.run.smithi178.stdout:Collecting tox==3.15.0 2024-06-28T15:01:03.705 INFO:teuthology.orchestra.run.smithi178.stdout: Downloading tox-3.15.0-py2.py3-none-any.whl (82 kB) 2024-06-28T15:01:03.804 INFO:teuthology.orchestra.run.smithi178.stdout: --------------------------------------- 82.1/82.1 KB 762.3 kB/s eta 0:00:00 2024-06-28T15:01:03.938 INFO:teuthology.orchestra.run.smithi178.stdout:Collecting toml>=0.9.4 2024-06-28T15:01:03.970 INFO:teuthology.orchestra.run.smithi178.stdout: Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) 2024-06-28T15:01:04.126 INFO:teuthology.orchestra.run.smithi178.stdout:Collecting py<2,>=1.4.17 2024-06-28T15:01:04.178 INFO:teuthology.orchestra.run.smithi178.stdout: Downloading py-1.11.0-py2.py3-none-any.whl (98 kB) 2024-06-28T15:01:04.221 INFO:teuthology.orchestra.run.smithi178.stdout: ---------------------------------------- 98.7/98.7 KB 2.2 MB/s eta 0:00:00 2024-06-28T15:01:04.378 INFO:teuthology.orchestra.run.smithi178.stdout:Collecting filelock<4,>=3.0.0 2024-06-28T15:01:04.420 INFO:teuthology.orchestra.run.smithi178.stdout: Downloading filelock-3.15.4-py3-none-any.whl (16 kB) 2024-06-28T15:01:04.756 INFO:teuthology.orchestra.run.smithi178.stdout:Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 2024-06-28T15:01:04.811 INFO:teuthology.orchestra.run.smithi178.stdout: Downloading virtualenv-20.26.3-py3-none-any.whl (5.7 MB) 2024-06-28T15:01:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: ---------------------------------------- 5.7/5.7 MB 907.9 kB/s eta 0:00:00 2024-06-28T15:01:11.262 INFO:teuthology.orchestra.run.smithi178.stdout:Collecting packaging>=14 2024-06-28T15:01:11.317 INFO:teuthology.orchestra.run.smithi178.stdout: Downloading packaging-24.1-py3-none-any.whl (53 kB) 2024-06-28T15:01:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: --------------------------------------- 54.0/54.0 KB 447.2 kB/s eta 0:00:00 2024-06-28T15:01:11.601 INFO:teuthology.orchestra.run.smithi178.stdout:Collecting pluggy<1,>=0.12.0 2024-06-28T15:01:11.657 INFO:teuthology.orchestra.run.smithi178.stdout: Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB) 2024-06-28T15:01:11.819 INFO:teuthology.orchestra.run.smithi178.stdout:Collecting six<2,>=1.14.0 2024-06-28T15:01:11.878 INFO:teuthology.orchestra.run.smithi178.stdout: Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-06-28T15:01:12.153 INFO:teuthology.orchestra.run.smithi178.stdout:Collecting platformdirs<5,>=3.9.1 2024-06-28T15:01:12.206 INFO:teuthology.orchestra.run.smithi178.stdout: Downloading platformdirs-4.2.2-py3-none-any.whl (18 kB) 2024-06-28T15:01:12.360 INFO:teuthology.orchestra.run.smithi178.stdout:Collecting distlib<1,>=0.3.7 2024-06-28T15:01:12.419 INFO:teuthology.orchestra.run.smithi178.stdout: Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB) 2024-06-28T15:01:13.085 INFO:teuthology.orchestra.run.smithi178.stdout: ------------------------------------- 468.9/468.9 KB 710.6 kB/s eta 0:00:00 2024-06-28T15:01:13.257 INFO:teuthology.orchestra.run.smithi178.stdout:Installing collected packages: distlib, toml, six, py, pluggy, platformdirs, packaging, filelock, virtualenv, tox 2024-06-28T15:01:14.193 INFO:teuthology.orchestra.run.smithi178.stdout:Successfully installed distlib-0.3.8 filelock-3.15.4 packaging-24.1 platformdirs-4.2.2 pluggy-0.13.1 py-1.11.0 six-1.16.0 toml-0.10.2 tox-3.15.0 virtualenv-20.26.3 2024-06-28T15:01:14.282 INFO:teuthology.run_tasks:Running task workunit... 2024-06-28T15:01:14.296 INFO:tasks.workunit:Pulling workunits from ref ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T15:01:14.297 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-28T15:01:14.297 DEBUG:teuthology.orchestra.run.smithi178:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:01:14.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T15:01:14.303 INFO:teuthology.orchestra.run.smithi178.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-06-28T15:01:14.304 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:01:14.354 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-06-28T15:01:14.354 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-06-28T15:01:14.402 INFO:tasks.workunit:timeout=3h 2024-06-28T15:01:14.402 INFO:tasks.workunit:cleanup=True 2024-06-28T15:01:14.402 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T15:01:14.452 INFO:tasks.workunit.client.0.smithi178.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-28T15:02:07.058 INFO:tasks.workunit.client.0.smithi178.stderr:Note: switching to 'ec2a38ba1b69b5357de8f0e8f96b00b4a577169d'. 2024-06-28T15:02:07.058 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:02:07.058 INFO:tasks.workunit.client.0.smithi178.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-28T15:02:07.058 INFO:tasks.workunit.client.0.smithi178.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-28T15:02:07.059 INFO:tasks.workunit.client.0.smithi178.stderr:state without impacting any branches by switching back to a branch. 2024-06-28T15:02:07.059 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:02:07.059 INFO:tasks.workunit.client.0.smithi178.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-28T15:02:07.059 INFO:tasks.workunit.client.0.smithi178.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-28T15:02:07.059 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:02:07.059 INFO:tasks.workunit.client.0.smithi178.stderr: git switch -c 2024-06-28T15:02:07.059 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:02:07.059 INFO:tasks.workunit.client.0.smithi178.stderr:Or undo this operation with: 2024-06-28T15:02:07.060 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:02:07.060 INFO:tasks.workunit.client.0.smithi178.stderr: git switch - 2024-06-28T15:02:07.060 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:02:07.060 INFO:tasks.workunit.client.0.smithi178.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-28T15:02:07.060 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:02:07.060 INFO:tasks.workunit.client.0.smithi178.stderr:HEAD is now at ec2a38ba1b6 Merge branches 'pr-56792', 'pr-57537', 'pr-57858', 'pr-57985', 'pr-58081' and 'pr-58288' into wip-cbodley-testing 2024-06-28T15:02:07.065 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-28T15:02:07.113 INFO:tasks.workunit.client.0.smithi178.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-28T15:02:07.116 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-28T15:02:07.116 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-28T15:02:07.820 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-28T15:02:07.900 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-28T15:02:07.952 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-28T15:02:07.953 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-28T15:02:07.953 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-28T15:02:08.005 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-28T15:02:08.009 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T15:02:08.009 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-28T15:02:08.059 INFO:tasks.workunit:Running workunits matching rgw/run-datacache.sh on client.0... 2024-06-28T15:02:08.060 INFO:tasks.workunit:Running workunit rgw/run-datacache.sh... 2024-06-28T15:02:08.060 DEBUG:teuthology.orchestra.run.smithi178:workunit test rgw/run-datacache.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RGW_DATACACHE_PATH=/tmp/rgw_datacache adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-datacache.sh 2024-06-28T15:02:08.113 INFO:tasks.workunit.client.0.smithi178.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-datacache.sh 2024-06-28T15:02:08.114 INFO:tasks.workunit.client.0.smithi178.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-06-28T15:02:08.114 INFO:tasks.workunit.client.0.smithi178.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-06-28T15:02:10.961 INFO:tasks.workunit.client.0.smithi178.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-06-28T15:02:10.961 INFO:tasks.workunit.client.0.smithi178.stderr:++ deactivate nondestructive 2024-06-28T15:02:10.961 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n '' ']' 2024-06-28T15:02:10.961 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n '' ']' 2024-06-28T15:02:10.962 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-06-28T15:02:10.962 INFO:tasks.workunit.client.0.smithi178.stderr:++ hash -r 2024-06-28T15:02:10.962 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n '' ']' 2024-06-28T15:02:10.962 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset VIRTUAL_ENV 2024-06-28T15:02:10.962 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-06-28T15:02:10.962 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-06-28T15:02:10.962 INFO:tasks.workunit.client.0.smithi178.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-06-28T15:02:10.962 INFO:tasks.workunit.client.0.smithi178.stderr:++ export VIRTUAL_ENV 2024-06-28T15:02:10.962 INFO:tasks.workunit.client.0.smithi178.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-06-28T15:02:10.962 INFO:tasks.workunit.client.0.smithi178.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-06-28T15:02:10.962 INFO:tasks.workunit.client.0.smithi178.stderr:++ export PATH 2024-06-28T15:02:10.962 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n '' ']' 2024-06-28T15:02:10.963 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-06-28T15:02:10.963 INFO:tasks.workunit.client.0.smithi178.stderr:++ _OLD_VIRTUAL_PS1= 2024-06-28T15:02:10.963 INFO:tasks.workunit.client.0.smithi178.stderr:++ PS1='(rgw) ' 2024-06-28T15:02:10.963 INFO:tasks.workunit.client.0.smithi178.stderr:++ export PS1 2024-06-28T15:02:10.963 INFO:tasks.workunit.client.0.smithi178.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-06-28T15:02:10.963 INFO:tasks.workunit.client.0.smithi178.stderr:++ export VIRTUAL_ENV_PROMPT 2024-06-28T15:02:10.963 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-06-28T15:02:10.963 INFO:tasks.workunit.client.0.smithi178.stderr:++ hash -r 2024-06-28T15:02:10.963 INFO:tasks.workunit.client.0.smithi178.stderr:+ pip install pip --upgrade 2024-06-28T15:02:11.323 INFO:tasks.workunit.client.0.smithi178.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-06-28T15:02:11.734 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting pip 2024-06-28T15:02:11.965 INFO:tasks.workunit.client.0.smithi178.stdout: Downloading pip-24.1.1-py3-none-any.whl (1.8 MB) 2024-06-28T15:02:12.326 INFO:tasks.workunit.client.0.smithi178.stdout: ---------------------------------------- 1.8/1.8 MB 5.1 MB/s eta 0:00:00 2024-06-28T15:02:12.401 INFO:tasks.workunit.client.0.smithi178.stdout:Installing collected packages: pip 2024-06-28T15:02:12.402 INFO:tasks.workunit.client.0.smithi178.stdout: Attempting uninstall: pip 2024-06-28T15:02:12.402 INFO:tasks.workunit.client.0.smithi178.stdout: Found existing installation: pip 22.0.2 2024-06-28T15:02:12.698 INFO:tasks.workunit.client.0.smithi178.stdout: Uninstalling pip-22.0.2: 2024-06-28T15:02:12.704 INFO:tasks.workunit.client.0.smithi178.stdout: Successfully uninstalled pip-22.0.2 2024-06-28T15:02:13.494 INFO:tasks.workunit.client.0.smithi178.stdout:Successfully installed pip-24.1.1 2024-06-28T15:02:13.557 INFO:tasks.workunit.client.0.smithi178.stderr:+ pip install configobj 2024-06-28T15:02:14.158 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting configobj 2024-06-28T15:02:14.287 INFO:tasks.workunit.client.0.smithi178.stdout: Downloading configobj-5.0.8-py2.py3-none-any.whl.metadata (3.4 kB) 2024-06-28T15:02:14.454 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting six (from configobj) 2024-06-28T15:02:14.491 INFO:tasks.workunit.client.0.smithi178.stdout: Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-06-28T15:02:14.595 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading configobj-5.0.8-py2.py3-none-any.whl (36 kB) 2024-06-28T15:02:14.708 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-06-28T15:02:14.791 INFO:tasks.workunit.client.0.smithi178.stdout:Installing collected packages: six, configobj 2024-06-28T15:02:14.954 INFO:tasks.workunit.client.0.smithi178.stdout:Successfully installed configobj-5.0.8 six-1.16.0 2024-06-28T15:02:15.365 INFO:tasks.workunit.client.0.smithi178.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_datacache.py 2024-06-28T15:02:15.403 INFO:tasks.workunit.client.0.smithi178.stderr:DEBUG:root:datacache dir from config is: /tmp/rgw_datacache 2024-06-28T15:02:15.404 INFO:tasks.workunit.client.0.smithi178.stderr:DEBUG:root:exec_cmd(pwd) 2024-06-28T15:02:15.405 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:command succeeded 2024-06-28T15:02:15.405 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:b'/home/ubuntu/cephtest/mnt.0/client.0/tmp\n' 2024-06-28T15:02:15.405 INFO:tasks.workunit.client.0.smithi178.stderr:DEBUG:root:pwd is: /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:02:15.405 INFO:tasks.workunit.client.0.smithi178.stderr:DEBUG:root:exec_cmd(sudo netstat -nltp | egrep "rados|valgr") 2024-06-28T15:02:15.447 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:command succeeded 2024-06-28T15:02:15.447 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:b'tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 37087/valgrind.bin \ntcp6 0 0 :::80 :::* LISTEN 37087/valgrind.bin \n' 2024-06-28T15:02:15.447 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:radosgw port: 80 2024-06-28T15:02:15.447 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:radosgw endpoint is: 127.0.0.1 2024-06-28T15:02:15.447 INFO:tasks.workunit.client.0.smithi178.stderr:DEBUG:root:exec_cmd(dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat bs=1M count=7) 2024-06-28T15:02:15.472 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:command succeeded 2024-06-28T15:02:15.473 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:b'' 2024-06-28T15:02:15.473 INFO:tasks.workunit.client.0.smithi178.stderr:DEBUG:root:exec_cmd(radosgw-admin user create --uid rgw_datacache_user --display-name DatacacheUser --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn) 2024-06-28T15:02:15.666 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:command succeeded 2024-06-28T15:02:15.666 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:b'{\n "user_id": "rgw_datacache_user",\n "display_name": "DatacacheUser",\n "email": "",\n "suspended": 0,\n "max_buckets": 1000,\n "subusers": [],\n "keys": [\n {\n "user": "rgw_datacache_user",\n "access_key": "NX5QOQKC6BH2IDN8HC7A",\n "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn",\n "active": true,\n "create_date": "2024-06-28T15:02:15.611064Z"\n }\n ],\n "swift_keys": [],\n "caps": [],\n "op_mask": "read, write, delete",\n "default_placement": "",\n "default_storage_class": "",\n "placement_tags": [],\n "bucket_quota": {\n "enabled": false,\n "check_on_raw": false,\n "max_size": -1,\n "max_size_kb": 0,\n "max_objects": -1\n },\n "user_quota": {\n "enabled": false,\n "check_on_raw": false,\n "max_size": -1,\n "max_size_kb": 0,\n "max_objects": -1\n },\n "temp_url_keys": [],\n "type": "rgw",\n "mfa_ids": [],\n "account_id": "",\n "path": "/",\n "create_date": "2024-06-28T15:02:15.611051Z",\n "tags": [],\n "group_ids": []\n}\n\n' 2024-06-28T15:02:15.666 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:Creating s3cmd config... 2024-06-28T15:02:15.666 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:proto for s3cmd config is http 2024-06-28T15:02:15.666 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:s3cmd config written 2024-06-28T15:02:15.666 INFO:tasks.workunit.client.0.smithi178.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 mb s3://datacachebucket) 2024-06-28T15:02:16.642 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:command succeeded 2024-06-28T15:02:16.642 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:b"Bucket 's3://datacachebucket/' created\n" 2024-06-28T15:02:16.642 INFO:tasks.workunit.client.0.smithi178.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 put /home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat s3://datacachebucket) 2024-06-28T15:02:19.010 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:command succeeded 2024-06-28T15:02:19.010 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:b"upload: '/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M.dat' -> 's3://datacachebucket/7M.dat' (7340032 bytes in 2.1 seconds, 3.36 MB/s) [1 of 1]\n" 2024-06-28T15:02:19.010 INFO:tasks.workunit.client.0.smithi178.stderr:DEBUG:root:exec_cmd(s3cmd --access_key=NX5QOQKC6BH2IDN8HC7A --secret_key=LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --config=/home/ubuntu/cephtest/mnt.0/client.0/tmp/s3cfg --no-check-hostname --host=127.0.0.1 get s3://datacachebucket/7M.dat /home/ubuntu/cephtest/mnt.0/client.0/tmp/7M-get.dat --force) 2024-06-28T15:02:19.898 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:command succeeded 2024-06-28T15:02:19.898 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:b"download: 's3://datacachebucket/7M.dat' -> '/home/ubuntu/cephtest/mnt.0/client.0/tmp/7M-get.dat' (7340032 bytes in 0.7 seconds, 10.20 MB/s)\n" 2024-06-28T15:02:19.898 INFO:tasks.workunit.client.0.smithi178.stderr:DEBUG:root:exec_cmd(radosgw-admin object stat --bucket=datacachebucket --object=7M.dat) 2024-06-28T15:02:20.289 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:command succeeded 2024-06-28T15:02:20.289 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:b'{\n "name": "7M.dat",\n "size": 7340376,\n "policy": {\n "acl": {\n "acl_user_map": [\n {\n "user": "rgw_datacache_user",\n "acl": 15\n }\n ],\n "acl_group_map": [],\n "grant_map": [\n {\n "id": "rgw_datacache_user",\n "grant": {\n "type": {\n "type": 0\n },\n "id": "rgw_datacache_user",\n "name": "DatacacheUser",\n "permission": {\n "flags": 15\n }\n }\n }\n ]\n },\n "owner": {\n "id": "rgw_datacache_user",\n "display_name": "DatacacheUser"\n }\n },\n "compression": {\n "compression_type": "snappy",\n "orig_size": 7340032,\n "blocks": [\n {\n "old_ofs": 0,\n "new_ofs": 0,\n "len": 4194500\n },\n {\n "old_ofs": 4194304,\n "new_ofs": 4194500,\n "len": 3145876\n }\n ]\n },\n "etag": "b2018f0392e2affb8652e437d5f0405b",\n "tag": "843e24ce-085a-4043-8f55-38e05370e790.4686.7532019485629968186",\n "manifest": {\n "objs": [],\n "obj_size": 7340376,\n "explicit_objs": false,\n "head_size": 0,\n "max_head_size": 0,\n "prefix": ".IOHAXk1uLCbv7J6R08_yXrLdGrGgXWw_",\n "rules": [\n {\n "key": 0,\n "val": {\n "start_part_num": 0,\n "start_ofs": 0,\n "part_size": 0,\n "stripe_max_size": 6291456,\n "override_prefix": ""\n }\n }\n ],\n "tail_instance": "",\n "tail_placement": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "843e24ce-085a-4043-8f55-38e05370e790.4686.1",\n "bucket_id": "843e24ce-085a-4043-8f55-38e05370e790.4686.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "placement_rule": "default-placement"\n },\n "tier_type": "",\n "begin_iter": {\n "part_ofs": 0,\n "stripe_ofs": 0,\n "ofs": 0,\n "stripe_size": 6291456,\n "cur_part_id": 0,\n "cur_stripe": 0,\n "cur_override_prefix": "",\n "location": {\n "placement_rule": "default-placement",\n "obj": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "843e24ce-085a-4043-8f55-38e05370e790.4686.1",\n "bucket_id": "843e24ce-085a-4043-8f55-38e05370e790.4686.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "key": {\n "name": ".IOHAXk1uLCbv7J6R08_yXrLdGrGgXWw_0",\n "instance": "",\n "ns": "shadow"\n }\n },\n "raw_obj": {\n "pool": "",\n "oid": "",\n "loc": ""\n },\n "is_raw": false\n }\n },\n "end_iter": {\n "part_ofs": 0,\n "stripe_ofs": 6291456,\n "ofs": 7340376,\n "stripe_size": 1048920,\n "cur_part_id": 0,\n "cur_stripe": 1,\n "cur_override_prefix": "",\n "location": {\n "placement_rule": "default-placement",\n "obj": {\n "bucket": {\n "name": "datacachebucket",\n "marker": "843e24ce-085a-4043-8f55-38e05370e790.4686.1",\n "bucket_id": "843e24ce-085a-4043-8f55-38e05370e790.4686.1",\n "tenant": "",\n "explicit_placement": {\n "data_pool": "",\n "data_extra_pool": "",\n "index_pool": ""\n }\n },\n "key": {\n "name": ".IOHAXk1uLCbv7J6R08_yXrLdGrGgXWw_1",\n "instance": "",\n "ns": "shadow"\n }\n },\n "raw_obj": {\n "pool": "",\n "oid": "",\n "loc": ""\n },\n "is_raw": false\n }\n }\n },\n "pg_ver": 0,\n "source_zone": 0,\n "torrent": "",\n "mtime": "2024-06-28T15:02:18.835433Z",\n "attrs": {\n "user.rgw.content_type": "application/octet-stream",\n "user.rgw.storage_class": "STANDARD",\n "user.rgw.tail_tag": "843e24ce-085a-4043-8f55-38e05370e790.4686.7532019485629968186",\n "user.rgw.x-amz-content-sha256": "cbcdca9ac38210e12be300bf076e3620804194e32ee9452705fa33f32e5c4b0a",\n "user.rgw.x-amz-date": "20240628T150216Z",\n "user.rgw.x-amz-meta-s3cmd-attrs": "atime:1719586935/ctime:1719586935/gid:1271/gname:ubuntu/md5:b2018f0392e2affb8652e437d5f0405b/mode:33204/mtime:1719586935/uid:1000/uname:ubuntu"\n }\n}\n' 2024-06-28T15:02:20.289 INFO:tasks.workunit.client.0.smithi178.stderr:DEBUG:root:Cached object name is: .IOHAXk1uLCbv7J6R08_yXrLdGrGgXWw_ 2024-06-28T15:02:20.289 INFO:tasks.workunit.client.0.smithi178.stderr:DEBUG:root:exec_cmd(find /tmp/rgw_datacache -type f | wc -l) 2024-06-28T15:02:20.292 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:command succeeded 2024-06-28T15:02:20.292 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:b'0\n' 2024-06-28T15:02:20.292 INFO:tasks.workunit.client.0.smithi178.stderr:DEBUG:root:Check cache dir content: 0 2024-06-28T15:02:20.292 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:NOTICE: datacache test object not found, inspect if datacache was bypassed or disabled during this check. 2024-06-28T15:02:20.293 INFO:tasks.workunit.client.0.smithi178.stderr:INFO:root:Completed Datacache tests 2024-06-28T15:02:20.299 INFO:tasks.workunit.client.0.smithi178.stderr:+ deactivate 2024-06-28T15:02:20.299 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-06-28T15:02:20.299 INFO:tasks.workunit.client.0.smithi178.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-06-28T15:02:20.300 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-06-28T15:02:20.300 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset _OLD_VIRTUAL_PATH 2024-06-28T15:02:20.300 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -n '' ']' 2024-06-28T15:02:20.300 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-06-28T15:02:20.300 INFO:tasks.workunit.client.0.smithi178.stderr:+ hash -r 2024-06-28T15:02:20.300 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -n '' ']' 2024-06-28T15:02:20.300 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset VIRTUAL_ENV 2024-06-28T15:02:20.300 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-06-28T15:02:20.300 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' '!' '' = nondestructive ']' 2024-06-28T15:02:20.300 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset -f deactivate 2024-06-28T15:02:20.300 INFO:tasks.workunit.client.0.smithi178.stdout:OK. 2024-06-28T15:02:20.301 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo OK. 2024-06-28T15:02:20.301 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:02:20.302 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:02:20.364 INFO:tasks.workunit:Stopping ['rgw/run-datacache.sh'] on client.0... 2024-06-28T15:02:20.365 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-28T15:02:20.683 DEBUG:teuthology.parallel:result is None 2024-06-28T15:02:20.684 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:02:20.697 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:02:20.697 DEBUG:teuthology.orchestra.run.smithi178:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:02:20.760 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:02:20.760 INFO:teuthology.run_tasks:Running task workunit... 2024-06-28T15:02:20.770 INFO:tasks.workunit:Pulling workunits from ref ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T15:02:20.771 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-28T15:02:20.771 DEBUG:teuthology.orchestra.run.smithi178:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:02:20.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T15:02:20.808 INFO:teuthology.orchestra.run.smithi178.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-06-28T15:02:20.808 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:02:20.854 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-06-28T15:02:20.855 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-06-28T15:02:20.902 INFO:tasks.workunit:timeout=3h 2024-06-28T15:02:20.902 INFO:tasks.workunit:cleanup=True 2024-06-28T15:02:20.903 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T15:02:20.953 INFO:tasks.workunit.client.0.smithi178.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-28T15:03:14.613 INFO:tasks.workunit.client.0.smithi178.stderr:Note: switching to 'ec2a38ba1b69b5357de8f0e8f96b00b4a577169d'. 2024-06-28T15:03:14.613 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:14.613 INFO:tasks.workunit.client.0.smithi178.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-28T15:03:14.613 INFO:tasks.workunit.client.0.smithi178.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-28T15:03:14.613 INFO:tasks.workunit.client.0.smithi178.stderr:state without impacting any branches by switching back to a branch. 2024-06-28T15:03:14.614 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:14.614 INFO:tasks.workunit.client.0.smithi178.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-28T15:03:14.614 INFO:tasks.workunit.client.0.smithi178.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-28T15:03:14.614 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:14.614 INFO:tasks.workunit.client.0.smithi178.stderr: git switch -c 2024-06-28T15:03:14.614 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:14.614 INFO:tasks.workunit.client.0.smithi178.stderr:Or undo this operation with: 2024-06-28T15:03:14.614 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:14.614 INFO:tasks.workunit.client.0.smithi178.stderr: git switch - 2024-06-28T15:03:14.614 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:14.614 INFO:tasks.workunit.client.0.smithi178.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-28T15:03:14.614 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:14.615 INFO:tasks.workunit.client.0.smithi178.stderr:HEAD is now at ec2a38ba1b6 Merge branches 'pr-56792', 'pr-57537', 'pr-57858', 'pr-57985', 'pr-58081' and 'pr-58288' into wip-cbodley-testing 2024-06-28T15:03:14.620 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-28T15:03:14.668 INFO:tasks.workunit.client.0.smithi178.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-28T15:03:14.671 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-28T15:03:14.671 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-28T15:03:14.730 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-28T15:03:14.787 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-28T15:03:14.831 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-28T15:03:14.833 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-28T15:03:14.833 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-28T15:03:14.882 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-28T15:03:14.886 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T15:03:14.886 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-28T15:03:14.933 INFO:tasks.workunit:Running workunits matching rgw/run-bucket-check.sh on client.0... 2024-06-28T15:03:14.935 INFO:tasks.workunit:Running workunit rgw/run-bucket-check.sh... 2024-06-28T15:03:14.935 DEBUG:teuthology.orchestra.run.smithi178:workunit test rgw/run-bucket-check.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2024-06-28T15:03:14.991 INFO:tasks.workunit.client.0.smithi178.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2024-06-28T15:03:14.992 INFO:tasks.workunit.client.0.smithi178.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-06-28T15:03:14.992 INFO:tasks.workunit.client.0.smithi178.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-06-28T15:03:17.858 INFO:tasks.workunit.client.0.smithi178.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-06-28T15:03:17.858 INFO:tasks.workunit.client.0.smithi178.stderr:++ deactivate nondestructive 2024-06-28T15:03:17.858 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n '' ']' 2024-06-28T15:03:17.858 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n '' ']' 2024-06-28T15:03:17.859 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-06-28T15:03:17.859 INFO:tasks.workunit.client.0.smithi178.stderr:++ hash -r 2024-06-28T15:03:17.859 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n '' ']' 2024-06-28T15:03:17.859 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset VIRTUAL_ENV 2024-06-28T15:03:17.859 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-06-28T15:03:17.859 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-06-28T15:03:17.859 INFO:tasks.workunit.client.0.smithi178.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-06-28T15:03:17.860 INFO:tasks.workunit.client.0.smithi178.stderr:++ export VIRTUAL_ENV 2024-06-28T15:03:17.860 INFO:tasks.workunit.client.0.smithi178.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-06-28T15:03:17.860 INFO:tasks.workunit.client.0.smithi178.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-06-28T15:03:17.860 INFO:tasks.workunit.client.0.smithi178.stderr:++ export PATH 2024-06-28T15:03:17.860 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n '' ']' 2024-06-28T15:03:17.860 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-06-28T15:03:17.861 INFO:tasks.workunit.client.0.smithi178.stderr:++ _OLD_VIRTUAL_PS1= 2024-06-28T15:03:17.861 INFO:tasks.workunit.client.0.smithi178.stderr:++ PS1='(rgw) ' 2024-06-28T15:03:17.861 INFO:tasks.workunit.client.0.smithi178.stderr:++ export PS1 2024-06-28T15:03:17.861 INFO:tasks.workunit.client.0.smithi178.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-06-28T15:03:17.861 INFO:tasks.workunit.client.0.smithi178.stderr:++ export VIRTUAL_ENV_PROMPT 2024-06-28T15:03:17.861 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-06-28T15:03:17.862 INFO:tasks.workunit.client.0.smithi178.stderr:++ hash -r 2024-06-28T15:03:17.862 INFO:tasks.workunit.client.0.smithi178.stderr:+ pip install pip --upgrade 2024-06-28T15:03:18.242 INFO:tasks.workunit.client.0.smithi178.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-06-28T15:03:18.619 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting pip 2024-06-28T15:03:18.625 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached pip-24.1.1-py3-none-any.whl (1.8 MB) 2024-06-28T15:03:18.664 INFO:tasks.workunit.client.0.smithi178.stdout:Installing collected packages: pip 2024-06-28T15:03:18.665 INFO:tasks.workunit.client.0.smithi178.stdout: Attempting uninstall: pip 2024-06-28T15:03:18.665 INFO:tasks.workunit.client.0.smithi178.stdout: Found existing installation: pip 22.0.2 2024-06-28T15:03:18.952 INFO:tasks.workunit.client.0.smithi178.stdout: Uninstalling pip-22.0.2: 2024-06-28T15:03:18.959 INFO:tasks.workunit.client.0.smithi178.stdout: Successfully uninstalled pip-22.0.2 2024-06-28T15:03:19.944 INFO:tasks.workunit.client.0.smithi178.stdout:Successfully installed pip-24.1.1 2024-06-28T15:03:20.013 INFO:tasks.workunit.client.0.smithi178.stderr:+ pip install boto3 2024-06-28T15:03:21.166 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting boto3 2024-06-28T15:03:21.262 INFO:tasks.workunit.client.0.smithi178.stdout: Downloading boto3-1.34.135-py3-none-any.whl.metadata (6.6 kB) 2024-06-28T15:03:21.985 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting botocore<1.35.0,>=1.34.135 (from boto3) 2024-06-28T15:03:22.032 INFO:tasks.workunit.client.0.smithi178.stdout: Downloading botocore-1.34.135-py3-none-any.whl.metadata (5.7 kB) 2024-06-28T15:03:22.243 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-06-28T15:03:22.292 INFO:tasks.workunit.client.0.smithi178.stdout: Downloading jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-06-28T15:03:22.526 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-06-28T15:03:22.578 INFO:tasks.workunit.client.0.smithi178.stdout: Downloading s3transfer-0.10.2-py3-none-any.whl.metadata (1.7 kB) 2024-06-28T15:03:22.803 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.135->boto3) 2024-06-28T15:03:22.860 INFO:tasks.workunit.client.0.smithi178.stdout: Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-06-28T15:03:23.136 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.35.0,>=1.34.135->boto3) 2024-06-28T15:03:23.193 INFO:tasks.workunit.client.0.smithi178.stdout: Downloading urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB) 2024-06-28T15:03:23.500 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.135->boto3) 2024-06-28T15:03:23.501 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-06-28T15:03:23.547 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading boto3-1.34.135-py3-none-any.whl (139 kB) 2024-06-28T15:03:23.708 INFO:tasks.workunit.client.0.smithi178.stdout: --------------------------------------- 139.2/139.2 kB 866.3 kB/s eta 0:00:00 2024-06-28T15:03:23.756 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading botocore-1.34.135-py3-none-any.whl (12.3 MB) 2024-06-28T15:03:28.946 INFO:tasks.workunit.client.0.smithi178.stdout: ---------------------------------------- 12.3/12.3 MB 2.4 MB/s eta 0:00:00 2024-06-28T15:03:29.000 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-06-28T15:03:29.141 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading s3transfer-0.10.2-py3-none-any.whl (82 kB) 2024-06-28T15:03:29.259 INFO:tasks.workunit.client.0.smithi178.stdout: ---------------------------------------- 82.7/82.7 kB 646.3 kB/s eta 0:00:00 2024-06-28T15:03:29.312 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-06-28T15:03:29.533 INFO:tasks.workunit.client.0.smithi178.stdout: ---------------------------------------- 229.9/229.9 kB 1.0 MB/s eta 0:00:00 2024-06-28T15:03:29.587 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading urllib3-2.2.2-py3-none-any.whl (121 kB) 2024-06-28T15:03:29.727 INFO:tasks.workunit.client.0.smithi178.stdout: --------------------------------------- 121.4/121.4 kB 838.3 kB/s eta 0:00:00 2024-06-28T15:03:29.731 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-06-28T15:03:29.856 INFO:tasks.workunit.client.0.smithi178.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-06-28T15:03:31.013 INFO:tasks.workunit.client.0.smithi178.stdout:Successfully installed boto3-1.34.135 botocore-1.34.135 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.2 six-1.16.0 urllib3-2.2.2 2024-06-28T15:03:31.150 INFO:tasks.workunit.client.0.smithi178.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_bucket_check.py 2024-06-28T15:03:31.297 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin user create --uid check-tester --display-name "Check Testing" --access-key OJODXSLNX4LUNHQG99PA --secret 3l6ffld34qaymfomuh832j94738aie2x4p2o8h6n 2024-06-28T15:03:31.335 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:31.335 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:31.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.388+0000 7f50a55eb980 20 rados->read ofs=0 len=0 2024-06-28T15:03:31.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.388+0000 7f50a55eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:31.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.388+0000 7f50a55eb980 20 realm 2024-06-28T15:03:31.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.388+0000 7f50a55eb980 20 rados->read ofs=0 len=0 2024-06-28T15:03:31.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.392+0000 7f50a55eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:31.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.392+0000 7f50a55eb980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:31.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.392+0000 7f50a55eb980 20 rados->read ofs=0 len=0 2024-06-28T15:03:31.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.392+0000 7f50a55eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:31.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.392+0000 7f50a55eb980 20 rados->read ofs=0 len=0 2024-06-28T15:03:31.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.392+0000 7f50a55eb980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:31.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.392+0000 7f50a55eb980 20 rados->read ofs=0 len=0 2024-06-28T15:03:31.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.392+0000 7f50a55eb980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:31.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.392+0000 7f50a55eb980 20 searching for the correct realm 2024-06-28T15:03:31.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.400+0000 7f50a28fb640 0 -- 172.21.15.178:0/2550623443 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f5094078050 msgr2=0x7f509406f360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:31.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.404+0000 7f50a28fb640 0 -- 172.21.15.178:0/2550623443 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f4f6c07d310 msgr2=0x7f4f6c07f740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:31.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:31.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:31.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:31.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:31.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:31.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:31.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 rados->read ofs=0 len=0 2024-06-28T15:03:31.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:31.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 rados->read ofs=0 len=0 2024-06-28T15:03:31.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:31.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 rados->read ofs=0 len=0 2024-06-28T15:03:31.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:31.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 zone default found 2024-06-28T15:03:31.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 4 Realm: () 2024-06-28T15:03:31.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:31.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:31.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:31.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 zonegroup default 2024-06-28T15:03:31.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 rados->read ofs=0 len=0 2024-06-28T15:03:31.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:31.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 rados->read ofs=0 len=0 2024-06-28T15:03:31.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:31.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 rados->read ofs=0 len=0 2024-06-28T15:03:31.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:31.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 started sync module instance, tier type = 2024-06-28T15:03:31.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.412+0000 7f50a55eb980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:31.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.428+0000 7f50a55eb980 20 add_watcher() i=0 2024-06-28T15:03:31.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.428+0000 7f50a55eb980 20 add_watcher() i=1 2024-06-28T15:03:31.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.428+0000 7f50a55eb980 20 add_watcher() i=2 2024-06-28T15:03:31.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.428+0000 7f50a55eb980 20 add_watcher() i=3 2024-06-28T15:03:31.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.428+0000 7f50a55eb980 20 add_watcher() i=4 2024-06-28T15:03:31.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.428+0000 7f50a55eb980 20 add_watcher() i=5 2024-06-28T15:03:31.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.428+0000 7f50a55eb980 20 add_watcher() i=6 2024-06-28T15:03:31.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.432+0000 7f50a55eb980 20 add_watcher() i=7 2024-06-28T15:03:31.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.432+0000 7f50a55eb980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:31.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.432+0000 7f4f59fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:31.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:31.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:31.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 5 note: GC not initialized 2024-06-28T15:03:31.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f4f58fe1640 20 reqs_thread_entry: start 2024-06-28T15:03:31.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:31.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 20 Filter name: none 2024-06-28T15:03:31.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f4ed27fc640 20 reqs_thread_entry: start 2024-06-28T15:03:31.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 10 cache get: name=default.rgw.meta+users.uid+check-tester : miss 2024-06-28T15:03:31.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 20 rados->read ofs=0 len=0 2024-06-28T15:03:31.427 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:31.427 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x0 2024-06-28T15:03:31.427 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 10 adding default.rgw.meta+users.uid+check-tester to cache LRU end 2024-06-28T15:03:31.427 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : miss 2024-06-28T15:03:31.427 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 20 rados->read ofs=0 len=0 2024-06-28T15:03:31.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:31.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x0 2024-06-28T15:03:31.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 10 adding default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-06-28T15:03:31.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2024-06-28T15:03:31.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.436+0000 7f50a55eb980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2024-06-28T15:03:31.430 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.440+0000 7f50a55eb980 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2024-06-28T15:03:31.430 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.440+0000 7f50a55eb980 10 moving default.rgw.meta+users.uid+check-tester to cache LRU end 2024-06-28T15:03:31.430 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.440+0000 7f50a55eb980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:check-tester, ofs0, ns] 2024-06-28T15:03:31.431 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.440+0000 7f4f86ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920769 cookie 94015874519792 notifier 4580 bl.length()=593 2024-06-28T15:03:31.431 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.440+0000 7f4f86ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2024-06-28T15:03:31.431 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.440+0000 7f4f86ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+check-tester to cache LRU end 2024-06-28T15:03:31.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.444+0000 7f50a55eb980 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2024-06-28T15:03:31.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.444+0000 7f50a55eb980 10 moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-06-28T15:03:31.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.444+0000 7f50a55eb980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.keys:OJODXSLNX4LUNHQG99PA, ofs0, ns] 2024-06-28T15:03:31.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.444+0000 7f4f867fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920769 cookie 94015874508320 notifier 4580 bl.length()=186 2024-06-28T15:03:31.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.444+0000 7f4f867fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2024-06-28T15:03:31.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.444+0000 7f4f867fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-06-28T15:03:31.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.448+0000 7f4f59fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:31.444 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.452+0000 7f50a55eb980 20 remove_watcher() i=0 2024-06-28T15:03:31.444 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.452+0000 7f50a55eb980 2 removed watcher, disabling cache 2024-06-28T15:03:31.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.456+0000 7f50a55eb980 20 remove_watcher() i=1 2024-06-28T15:03:31.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.456+0000 7f50a55eb980 20 remove_watcher() i=2 2024-06-28T15:03:31.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.460+0000 7f50a55eb980 20 remove_watcher() i=3 2024-06-28T15:03:31.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.460+0000 7f50a55eb980 20 remove_watcher() i=4 2024-06-28T15:03:31.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.464+0000 7f50a55eb980 20 remove_watcher() i=5 2024-06-28T15:03:31.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.464+0000 7f50a55eb980 20 remove_watcher() i=6 2024-06-28T15:03:31.456 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:31.464+0000 7f50a55eb980 20 remove_watcher() i=7 2024-06-28T15:03:33.691 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: recalculated bucket check stats are correct 2024-06-28T15:03:33.691 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:33.692 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2024-06-28T15:03:33.727 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:33.727 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:33.772 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:33.780+0000 7fa07f89b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:33.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:33.784+0000 7fa07f89b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:33.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:33.784+0000 7fa07f89b980 20 realm 2024-06-28T15:03:33.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:33.784+0000 7fa07f89b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:33.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:33.784+0000 7fa07f89b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:33.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:33.784+0000 7fa07f89b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:33.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:33.784+0000 7fa07f89b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:33.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:33.784+0000 7fa07f89b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:33.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:33.784+0000 7fa07f89b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.392+0000 7fa07f89b980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:34.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.392+0000 7fa07f89b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.396+0000 7fa07f89b980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:34.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.396+0000 7fa07f89b980 20 searching for the correct realm 2024-06-28T15:03:34.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.404+0000 7fa07dbad640 0 -- 172.21.15.178:0/2014880967 >> [v2:172.21.15.195:6824/1278553753,v1:172.21.15.195:6825/1278553753] conn(0x7fa074062040 msgr2=0x7fa074086b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:34.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.412+0000 7fa07d3ac640 0 -- 172.21.15.178:0/2014880967 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fa074008040 msgr2=0x7fa07406a9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:34.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.628+0000 7fa07f89b980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:34.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.628+0000 7fa07f89b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:34.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.628+0000 7fa07f89b980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:34.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.628+0000 7fa07f89b980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:34.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.628+0000 7fa07f89b980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:34.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.628+0000 7fa07f89b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:34.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.628+0000 7fa07f89b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.621 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.628+0000 7fa07f89b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:34.621 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.628+0000 7fa07f89b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.621 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:34.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:34.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 20 zone default found 2024-06-28T15:03:34.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 4 Realm: () 2024-06-28T15:03:34.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:34.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:34.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:34.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 20 zonegroup default 2024-06-28T15:03:34.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:34.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:34.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.625 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:34.625 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 20 started sync module instance, tier type = 2024-06-28T15:03:34.625 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.632+0000 7fa07f89b980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:34.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.660+0000 7fa07f89b980 20 add_watcher() i=0 2024-06-28T15:03:34.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.660+0000 7fa07f89b980 20 add_watcher() i=1 2024-06-28T15:03:34.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.660+0000 7fa07f89b980 20 add_watcher() i=2 2024-06-28T15:03:34.651 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.660+0000 7fa07f89b980 20 add_watcher() i=3 2024-06-28T15:03:34.651 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.660+0000 7fa07f89b980 20 add_watcher() i=4 2024-06-28T15:03:34.651 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.660+0000 7fa07f89b980 20 add_watcher() i=5 2024-06-28T15:03:34.651 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.660+0000 7fa07f89b980 20 add_watcher() i=6 2024-06-28T15:03:34.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.660+0000 7fa07f89b980 20 add_watcher() i=7 2024-06-28T15:03:34.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.660+0000 7fa07f89b980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:34.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.664+0000 7f9f33fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:34.657 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:34.657 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:34.657 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 5 note: GC not initialized 2024-06-28T15:03:34.658 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7f9f32fe5640 20 reqs_thread_entry: start 2024-06-28T15:03:34.658 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:34.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 20 Filter name: none 2024-06-28T15:03:34.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7f9f30fe1640 20 reqs_thread_entry: start 2024-06-28T15:03:34.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-06-28T15:03:34.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:34.660 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-06-28T15:03:34.660 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-06-28T15:03:34.660 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 : miss 2024-06-28T15:03:34.660 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.661 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:34.661 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 info.flags=0x17 2024-06-28T15:03:34.661 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 to cache LRU end 2024-06-28T15:03:34.661 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:03:34.661 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-06-28T15:03:34.661 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 2024-06-28T15:03:34.661 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.668+0000 7fa07f89b980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:34.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.676+0000 7f9f33fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:34.671 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.680+0000 7fa07f89b980 20 remove_watcher() i=0 2024-06-28T15:03:34.671 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.680+0000 7fa07f89b980 2 removed watcher, disabling cache 2024-06-28T15:03:34.673 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.684+0000 7fa07f89b980 20 remove_watcher() i=1 2024-06-28T15:03:34.676 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.684+0000 7fa07f89b980 20 remove_watcher() i=2 2024-06-28T15:03:34.678 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.688+0000 7fa07f89b980 20 remove_watcher() i=3 2024-06-28T15:03:34.680 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.688+0000 7fa07f89b980 20 remove_watcher() i=4 2024-06-28T15:03:34.682 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.692+0000 7fa07f89b980 20 remove_watcher() i=5 2024-06-28T15:03:34.683 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.692+0000 7fa07f89b980 20 remove_watcher() i=6 2024-06-28T15:03:34.685 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.696+0000 7fa07f89b980 20 remove_watcher() i=7 2024-06-28T15:03:34.705 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket check-bucket 2024-06-28T15:03:34.733 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:34.734 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:34.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.788+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.788+0000 7fbff8d03980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:34.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.788+0000 7fbff8d03980 20 realm 2024-06-28T15:03:34.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.788+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.788+0000 7fbff8d03980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:34.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.788+0000 7fbff8d03980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:34.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.788+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.788+0000 7fbff8d03980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:34.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.788+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.792+0000 7fbff8d03980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:34.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.792+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.792+0000 7fbff8d03980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:34.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.792+0000 7fbff8d03980 20 searching for the correct realm 2024-06-28T15:03:34.789 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.796+0000 7fbff6013640 0 -- 172.21.15.178:0/3783215779 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x559ce72255c0 msgr2=0x559ce7227a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:34.795 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.804+0000 7fbff6013640 0 -- 172.21.15.178:0/3783215779 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fbfe80fb960 msgr2=0x7fbfe80fa0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:34.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:34.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:34.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:34.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:34.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:34.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:34.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:34.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:34.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:34.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 zone default found 2024-06-28T15:03:34.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 4 Realm: () 2024-06-28T15:03:34.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:34.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:34.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:34.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 zonegroup default 2024-06-28T15:03:34.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:34.805 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.805 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:34.805 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.805 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:34.805 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 started sync module instance, tier type = 2024-06-28T15:03:34.805 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.812+0000 7fbff8d03980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:34.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.828+0000 7fbff8d03980 20 add_watcher() i=0 2024-06-28T15:03:34.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.828+0000 7fbff8d03980 20 add_watcher() i=1 2024-06-28T15:03:34.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.828+0000 7fbff8d03980 20 add_watcher() i=2 2024-06-28T15:03:34.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.828+0000 7fbff8d03980 20 add_watcher() i=3 2024-06-28T15:03:34.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.828+0000 7fbff8d03980 20 add_watcher() i=4 2024-06-28T15:03:34.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.828+0000 7fbff8d03980 20 add_watcher() i=5 2024-06-28T15:03:34.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.828+0000 7fbff8d03980 20 add_watcher() i=6 2024-06-28T15:03:34.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.832+0000 7fbff8d03980 20 add_watcher() i=7 2024-06-28T15:03:34.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.832+0000 7fbff8d03980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:34.824 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.832+0000 7fbead7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:34.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.832+0000 7fbff8d03980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:34.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.832+0000 7fbff8d03980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:34.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.832+0000 7fbff8d03980 5 note: GC not initialized 2024-06-28T15:03:34.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.832+0000 7fbe27fff640 20 reqs_thread_entry: start 2024-06-28T15:03:34.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.832+0000 7fbff8d03980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:34.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.836+0000 7fbff8d03980 20 Filter name: none 2024-06-28T15:03:34.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.836+0000 7fbe25ffb640 20 reqs_thread_entry: start 2024-06-28T15:03:34.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.836+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.836+0000 7fbff8d03980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:34.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.836+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.836+0000 7fbff8d03980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:34.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.836+0000 7fbff8d03980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:34.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.836+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.836+0000 7fbff8d03980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:34.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.836+0000 7fbff8d03980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.836+0000 7fbff8d03980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:34.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.836+0000 7fbff8d03980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:34.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.836+0000 7fbead7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:34.833 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.844+0000 7fbff8d03980 20 remove_watcher() i=0 2024-06-28T15:03:34.833 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.844+0000 7fbff8d03980 2 removed watcher, disabling cache 2024-06-28T15:03:34.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.844+0000 7fbff8d03980 20 remove_watcher() i=1 2024-06-28T15:03:34.837 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.848+0000 7fbff8d03980 20 remove_watcher() i=2 2024-06-28T15:03:34.839 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.848+0000 7fbff8d03980 20 remove_watcher() i=3 2024-06-28T15:03:34.841 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.852+0000 7fbff8d03980 20 remove_watcher() i=4 2024-06-28T15:03:34.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.852+0000 7fbff8d03980 20 remove_watcher() i=5 2024-06-28T15:03:34.844 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.852+0000 7fbff8d03980 20 remove_watcher() i=6 2024-06-28T15:03:34.845 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.856+0000 7fbff8d03980 20 remove_watcher() i=7 2024-06-28T15:03:34.859 INFO:tasks.workunit.client.0.smithi178.stderr:{'rgw.main': {'size': 54, 'size_actual': 24576, 'size_utilized': 88, 'size_kb': 1, 'size_kb_actual': 24, 'size_kb_utilized': 1, 'num_objects': 6}} 2024-06-28T15:03:34.859 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: bucket check unlinked does not report normal entries 2024-06-28T15:03:34.859 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:34.860 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-06-28T15:03:34.890 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:34.891 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:34.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.944+0000 7f57586ee980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.944+0000 7f57586ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:34.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.944+0000 7f57586ee980 20 realm 2024-06-28T15:03:34.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.944+0000 7f57586ee980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.944+0000 7f57586ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:34.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.944+0000 7f57586ee980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:34.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.944+0000 7f57586ee980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.944+0000 7f57586ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:34.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.944+0000 7f57586ee980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.948+0000 7f57586ee980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:34.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.948+0000 7f57586ee980 20 rados->read ofs=0 len=0 2024-06-28T15:03:34.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.948+0000 7f57586ee980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:34.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.948+0000 7f57586ee980 20 searching for the correct realm 2024-06-28T15:03:34.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:34.956+0000 7f5756a00640 0 -- 172.21.15.178:0/1576922391 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f574c0f7940 msgr2=0x7f574c0d3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:03:35.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.172+0000 7f57561ff640 0 -- 172.21.15.178:0/1576922391 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x560481371320 msgr2=0x560481373710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:35.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.180+0000 7f57586ee980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:35.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.180+0000 7f57586ee980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:35.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.180+0000 7f57586ee980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:35.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.180+0000 7f57586ee980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:35.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.180+0000 7f57586ee980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:35.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.180+0000 7f57586ee980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:35.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.180+0000 7f57586ee980 20 rados->read ofs=0 len=0 2024-06-28T15:03:35.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.180+0000 7f57586ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:35.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.180+0000 7f57586ee980 20 rados->read ofs=0 len=0 2024-06-28T15:03:35.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.180+0000 7f57586ee980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:35.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.180+0000 7f57586ee980 20 rados->read ofs=0 len=0 2024-06-28T15:03:35.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:35.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 20 zone default found 2024-06-28T15:03:35.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 4 Realm: () 2024-06-28T15:03:35.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:35.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:35.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:35.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 20 zonegroup default 2024-06-28T15:03:35.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 20 rados->read ofs=0 len=0 2024-06-28T15:03:35.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:35.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 20 rados->read ofs=0 len=0 2024-06-28T15:03:35.176 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:35.176 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 20 rados->read ofs=0 len=0 2024-06-28T15:03:35.177 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:35.177 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 20 started sync module instance, tier type = 2024-06-28T15:03:35.177 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.184+0000 7f57586ee980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:35.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.208+0000 7f57586ee980 20 add_watcher() i=0 2024-06-28T15:03:35.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.208+0000 7f57586ee980 20 add_watcher() i=1 2024-06-28T15:03:35.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.208+0000 7f57586ee980 20 add_watcher() i=2 2024-06-28T15:03:35.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.208+0000 7f57586ee980 20 add_watcher() i=3 2024-06-28T15:03:35.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.208+0000 7f57586ee980 20 add_watcher() i=4 2024-06-28T15:03:35.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.208+0000 7f57586ee980 20 add_watcher() i=5 2024-06-28T15:03:35.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.208+0000 7f57586ee980 20 add_watcher() i=6 2024-06-28T15:03:35.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.212+0000 7f57586ee980 20 add_watcher() i=7 2024-06-28T15:03:35.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.212+0000 7f57586ee980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:35.207 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.216+0000 7f560cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:35.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.216+0000 7f57586ee980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:35.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.216+0000 7f57586ee980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:35.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.216+0000 7f57586ee980 5 note: GC not initialized 2024-06-28T15:03:35.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.216+0000 7f55877fe640 20 reqs_thread_entry: start 2024-06-28T15:03:35.210 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:35.210 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 20 Filter name: none 2024-06-28T15:03:35.210 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f55857fa640 20 reqs_thread_entry: start 2024-06-28T15:03:35.210 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-06-28T15:03:35.210 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 20 rados->read ofs=0 len=0 2024-06-28T15:03:35.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:35.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-06-28T15:03:35.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-06-28T15:03:35.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 : miss 2024-06-28T15:03:35.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 20 rados->read ofs=0 len=0 2024-06-28T15:03:35.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:35.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 info.flags=0x17 2024-06-28T15:03:35.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 to cache LRU end 2024-06-28T15:03:35.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:03:35.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-06-28T15:03:35.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 2024-06-28T15:03:35.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:35.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 1 NOTICE: finished shard 0 (0 entries found) 2024-06-28T15:03:35.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.220+0000 7f57586ee980 1 NOTICE: finished all shards (0 entries found) 2024-06-28T15:03:35.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.224+0000 7f560cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:35.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.228+0000 7f57586ee980 20 remove_watcher() i=0 2024-06-28T15:03:35.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.228+0000 7f57586ee980 2 removed watcher, disabling cache 2024-06-28T15:03:35.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.232+0000 7f57586ee980 20 remove_watcher() i=1 2024-06-28T15:03:35.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.232+0000 7f57586ee980 20 remove_watcher() i=2 2024-06-28T15:03:35.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.236+0000 7f57586ee980 20 remove_watcher() i=3 2024-06-28T15:03:35.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.236+0000 7f57586ee980 20 remove_watcher() i=4 2024-06-28T15:03:35.230 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.240+0000 7f57586ee980 20 remove_watcher() i=5 2024-06-28T15:03:35.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.240+0000 7f57586ee980 20 remove_watcher() i=6 2024-06-28T15:03:35.233 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:35.244+0000 7f57586ee980 20 remove_watcher() i=7 2024-06-28T15:03:35.257 INFO:tasks.workunit.client.0.smithi178.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-06-28T15:03:35.686 INFO:tasks.workunit.client.0.smithi178.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-06-28T15:03:37.294 INFO:tasks.workunit.client.0.smithi178.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-06-28T15:03:37.295 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-06-28T15:03:37.329 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:37.329 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:37.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.384+0000 7fbb91e00980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.384+0000 7fbb91e00980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:37.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.384+0000 7fbb91e00980 20 realm 2024-06-28T15:03:37.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.384+0000 7fbb91e00980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.384+0000 7fbb91e00980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:37.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.384+0000 7fbb91e00980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:37.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.384+0000 7fbb91e00980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.384+0000 7fbb91e00980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:37.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.384+0000 7fbb91e00980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.388+0000 7fbb91e00980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:37.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.388+0000 7fbb91e00980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.388+0000 7fbb91e00980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:37.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.388+0000 7fbb91e00980 20 searching for the correct realm 2024-06-28T15:03:37.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.392+0000 7fbb8f110640 0 -- 172.21.15.178:0/1892163947 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fbb800c3ba0 msgr2=0x7fbb800c4ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:37.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.400+0000 7fbb90112640 0 -- 172.21.15.178:0/1892163947 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fbb84052940 msgr2=0x7fbb8403a440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:37.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:37.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:37.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:37.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:37.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:37.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:37.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:37.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:37.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:37.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 zone default found 2024-06-28T15:03:37.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 4 Realm: () 2024-06-28T15:03:37.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:37.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:37.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:37.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 zonegroup default 2024-06-28T15:03:37.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:37.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:37.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:37.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 started sync module instance, tier type = 2024-06-28T15:03:37.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.408+0000 7fbb91e00980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:37.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.424+0000 7fbb91e00980 20 add_watcher() i=0 2024-06-28T15:03:37.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.424+0000 7fbb91e00980 20 add_watcher() i=1 2024-06-28T15:03:37.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.424+0000 7fbb91e00980 20 add_watcher() i=2 2024-06-28T15:03:37.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.424+0000 7fbb91e00980 20 add_watcher() i=3 2024-06-28T15:03:37.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.424+0000 7fbb91e00980 20 add_watcher() i=4 2024-06-28T15:03:37.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.424+0000 7fbb91e00980 20 add_watcher() i=5 2024-06-28T15:03:37.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.424+0000 7fbb91e00980 20 add_watcher() i=6 2024-06-28T15:03:37.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.428+0000 7fbb91e00980 20 add_watcher() i=7 2024-06-28T15:03:37.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.428+0000 7fbb91e00980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:37.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.428+0000 7fba45fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:37.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.428+0000 7fbb91e00980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:37.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.428+0000 7fbb91e00980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:37.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.428+0000 7fbb91e00980 5 note: GC not initialized 2024-06-28T15:03:37.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.428+0000 7fba44fe1640 20 reqs_thread_entry: start 2024-06-28T15:03:37.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.432+0000 7fbb91e00980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:37.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.432+0000 7fbb91e00980 20 Filter name: none 2024-06-28T15:03:37.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.432+0000 7fb9be7fc640 20 reqs_thread_entry: start 2024-06-28T15:03:37.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.432+0000 7fbb91e00980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.432+0000 7fbb91e00980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:37.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.432+0000 7fbb91e00980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.432+0000 7fbb91e00980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:37.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.432+0000 7fbb91e00980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:37.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.432+0000 7fbb91e00980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-06-28T15:03:37.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.432+0000 7fbb91e00980 20 INFO: main: starting shard=0 2024-06-28T15:03:37.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.432+0000 7fbb91e00980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:37.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.432+0000 7fbb91e00980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-06-28T15:03:37.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.432+0000 7fbb91e00980 20 INFO: main: done 2024-06-28T15:03:37.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.432+0000 7fba45fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:37.429 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.436+0000 7fbb91e00980 20 remove_watcher() i=0 2024-06-28T15:03:37.429 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.436+0000 7fbb91e00980 2 removed watcher, disabling cache 2024-06-28T15:03:37.431 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.440+0000 7fbb91e00980 20 remove_watcher() i=1 2024-06-28T15:03:37.434 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.444+0000 7fbb91e00980 20 remove_watcher() i=2 2024-06-28T15:03:37.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.444+0000 7fbb91e00980 20 remove_watcher() i=3 2024-06-28T15:03:37.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.448+0000 7fbb91e00980 20 remove_watcher() i=4 2024-06-28T15:03:37.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.448+0000 7fbb91e00980 20 remove_watcher() i=5 2024-06-28T15:03:37.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.448+0000 7fbb91e00980 20 remove_watcher() i=6 2024-06-28T15:03:37.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.452+0000 7fbb91e00980 20 remove_watcher() i=7 2024-06-28T15:03:37.618 INFO:tasks.workunit.client.0.smithi178.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-06-28T15:03:37.618 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-06-28T15:03:37.652 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:37.653 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:37.698 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.708+0000 7fd360a05980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.708+0000 7fd360a05980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:37.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.708+0000 7fd360a05980 20 realm 2024-06-28T15:03:37.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.708+0000 7fd360a05980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.708+0000 7fd360a05980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:37.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.708+0000 7fd360a05980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:37.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.708+0000 7fd360a05980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.708+0000 7fd360a05980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:37.701 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.708+0000 7fd360a05980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.702 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.712+0000 7fd360a05980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:37.702 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.712+0000 7fd360a05980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.712+0000 7fd360a05980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:37.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.712+0000 7fd360a05980 20 searching for the correct realm 2024-06-28T15:03:37.708 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.716+0000 7fd35dd15640 0 -- 172.21.15.178:0/21913503 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x5640a4fedf60 msgr2=0x5640a4ff0350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:37.714 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.724+0000 7fd35e516640 0 -- 172.21.15.178:0/21913503 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fd3540c23e0 msgr2=0x7fd3540c2860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:37.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:37.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:37.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:37.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:37.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:37.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:37.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:37.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:37.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:37.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 zone default found 2024-06-28T15:03:37.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 4 Realm: () 2024-06-28T15:03:37.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:37.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:37.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:37.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 zonegroup default 2024-06-28T15:03:37.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:37.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:37.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.732+0000 7fd360a05980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.736+0000 7fd360a05980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:37.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.736+0000 7fd360a05980 20 started sync module instance, tier type = 2024-06-28T15:03:37.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.736+0000 7fd360a05980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:37.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.752+0000 7fd360a05980 20 add_watcher() i=0 2024-06-28T15:03:37.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.752+0000 7fd360a05980 20 add_watcher() i=1 2024-06-28T15:03:37.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.752+0000 7fd360a05980 20 add_watcher() i=2 2024-06-28T15:03:37.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.752+0000 7fd360a05980 20 add_watcher() i=3 2024-06-28T15:03:37.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.752+0000 7fd360a05980 20 add_watcher() i=4 2024-06-28T15:03:37.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.752+0000 7fd360a05980 20 add_watcher() i=5 2024-06-28T15:03:37.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.752+0000 7fd360a05980 20 add_watcher() i=6 2024-06-28T15:03:37.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.752+0000 7fd360a05980 20 add_watcher() i=7 2024-06-28T15:03:37.743 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.752+0000 7fd360a05980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:37.745 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd214fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:37.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:37.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:37.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 5 note: GC not initialized 2024-06-28T15:03:37.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd18f7fe640 20 reqs_thread_entry: start 2024-06-28T15:03:37.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:37.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 20 Filter name: none 2024-06-28T15:03:37.747 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd18d7fa640 20 reqs_thread_entry: start 2024-06-28T15:03:37.747 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.747 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:37.747 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 20 rados->read ofs=0 len=0 2024-06-28T15:03:37.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:37.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:37.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-06-28T15:03:37.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 20 INFO: main: starting shard=0 2024-06-28T15:03:37.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:37.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-06-28T15:03:37.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.756+0000 7fd360a05980 20 INFO: main: done 2024-06-28T15:03:37.749 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.760+0000 7fd214fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:37.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.764+0000 7fd360a05980 20 remove_watcher() i=0 2024-06-28T15:03:37.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.764+0000 7fd360a05980 2 removed watcher, disabling cache 2024-06-28T15:03:37.756 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.764+0000 7fd360a05980 20 remove_watcher() i=1 2024-06-28T15:03:37.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.768+0000 7fd360a05980 20 remove_watcher() i=2 2024-06-28T15:03:37.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.772+0000 7fd360a05980 20 remove_watcher() i=3 2024-06-28T15:03:37.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.772+0000 7fd360a05980 20 remove_watcher() i=4 2024-06-28T15:03:37.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.772+0000 7fd360a05980 20 remove_watcher() i=5 2024-06-28T15:03:37.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.776+0000 7fd360a05980 20 remove_watcher() i=6 2024-06-28T15:03:37.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:37.776+0000 7fd360a05980 20 remove_watcher() i=7 2024-06-28T15:03:37.926 INFO:tasks.workunit.client.0.smithi178.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-06-28T15:03:37.926 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket check-bucket --object e 2024-06-28T15:03:37.960 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:37.961 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:38.003 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.012+0000 7f25b0ff7980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.005 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.016+0000 7f25b0ff7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:38.005 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.016+0000 7f25b0ff7980 20 realm 2024-06-28T15:03:38.005 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.016+0000 7f25b0ff7980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.005 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.016+0000 7f25b0ff7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:38.005 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.016+0000 7f25b0ff7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:38.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.016+0000 7f25b0ff7980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.016+0000 7f25b0ff7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:38.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.016+0000 7f25b0ff7980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.016+0000 7f25b0ff7980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:38.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.016+0000 7f25b0ff7980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.008 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.016+0000 7f25b0ff7980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:38.008 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.016+0000 7f25b0ff7980 20 searching for the correct realm 2024-06-28T15:03:38.014 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.024+0000 7f25ae307640 0 -- 172.21.15.178:0/2911273659 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f25a0009c20 msgr2=0x7f25a006a180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:38.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.028+0000 7f25aeb08640 0 -- 172.21.15.178:0/2911273659 >> [v2:172.21.15.178:6823/1880098663,v1:172.21.15.178:6825/1880098663] conn(0x7f25a80c1030 msgr2=0x7f25a80f5980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:38.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:38.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:38.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:38.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:38.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:38.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:38.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:38.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:38.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:38.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 zone default found 2024-06-28T15:03:38.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 4 Realm: () 2024-06-28T15:03:38.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:38.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:38.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:38.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 zonegroup default 2024-06-28T15:03:38.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.036+0000 7f25b0ff7980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.040+0000 7f25b0ff7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:38.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.040+0000 7f25b0ff7980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.040+0000 7f25b0ff7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:38.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.040+0000 7f25b0ff7980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.040+0000 7f25b0ff7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:38.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.040+0000 7f25b0ff7980 20 started sync module instance, tier type = 2024-06-28T15:03:38.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.040+0000 7f25b0ff7980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:38.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.056+0000 7f25b0ff7980 20 add_watcher() i=0 2024-06-28T15:03:38.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.056+0000 7f25b0ff7980 20 add_watcher() i=1 2024-06-28T15:03:38.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.056+0000 7f25b0ff7980 20 add_watcher() i=2 2024-06-28T15:03:38.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.056+0000 7f25b0ff7980 20 add_watcher() i=3 2024-06-28T15:03:38.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.056+0000 7f25b0ff7980 20 add_watcher() i=4 2024-06-28T15:03:38.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.056+0000 7f25b0ff7980 20 add_watcher() i=5 2024-06-28T15:03:38.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.056+0000 7f25b0ff7980 20 add_watcher() i=6 2024-06-28T15:03:38.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.056+0000 7f25b0ff7980 20 add_watcher() i=7 2024-06-28T15:03:38.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.056+0000 7f25b0ff7980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:38.050 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f24657e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:38.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:38.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:38.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 5 note: GC not initialized 2024-06-28T15:03:38.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f23dffff640 20 reqs_thread_entry: start 2024-06-28T15:03:38.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:38.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 20 Filter name: none 2024-06-28T15:03:38.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f23ddffb640 20 reqs_thread_entry: start 2024-06-28T15:03:38.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:38.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:38.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:38.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-06-28T15:03:38.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 20 INFO: main: starting shard=0 2024-06-28T15:03:38.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:38.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-06-28T15:03:38.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.060+0000 7f25b0ff7980 20 INFO: main: done 2024-06-28T15:03:38.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.064+0000 7f24657e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:38.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.068+0000 7f25b0ff7980 20 remove_watcher() i=0 2024-06-28T15:03:38.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.068+0000 7f25b0ff7980 2 removed watcher, disabling cache 2024-06-28T15:03:38.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.068+0000 7f25b0ff7980 20 remove_watcher() i=1 2024-06-28T15:03:38.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.072+0000 7f25b0ff7980 20 remove_watcher() i=2 2024-06-28T15:03:38.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.076+0000 7f25b0ff7980 20 remove_watcher() i=3 2024-06-28T15:03:38.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.076+0000 7f25b0ff7980 20 remove_watcher() i=4 2024-06-28T15:03:38.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.076+0000 7f25b0ff7980 20 remove_watcher() i=5 2024-06-28T15:03:38.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.080+0000 7f25b0ff7980 20 remove_watcher() i=6 2024-06-28T15:03:38.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.080+0000 7f25b0ff7980 20 remove_watcher() i=7 2024-06-28T15:03:38.214 INFO:tasks.workunit.client.0.smithi178.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-06-28T15:03:38.214 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket check-bucket --object f 2024-06-28T15:03:38.250 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:38.250 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:38.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.304+0000 7fc37e22f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.304+0000 7fc37e22f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:38.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.304+0000 7fc37e22f980 20 realm 2024-06-28T15:03:38.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.304+0000 7fc37e22f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.304+0000 7fc37e22f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:38.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.304+0000 7fc37e22f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:38.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.304+0000 7fc37e22f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.308+0000 7fc37e22f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:38.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.308+0000 7fc37e22f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.308+0000 7fc37e22f980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:38.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.308+0000 7fc37e22f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.308+0000 7fc37e22f980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:38.300 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.308+0000 7fc37e22f980 20 searching for the correct realm 2024-06-28T15:03:38.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.312+0000 7fc37b53f640 0 -- 172.21.15.178:0/3563653884 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fc36c0ee060 msgr2=0x7fc36c137040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:38.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.528+0000 7fc37bd40640 0 -- 172.21.15.178:0/3563653884 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fc3700840b0 msgr2=0x7fc370090880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:03:38.736 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.744+0000 7fc37e22f980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:38.736 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.744+0000 7fc37e22f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:38.737 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.744+0000 7fc37e22f980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:38.737 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.744+0000 7fc37e22f980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:38.737 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.744+0000 7fc37e22f980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:38.737 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.744+0000 7fc37e22f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:38.737 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.744+0000 7fc37e22f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.737 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.748+0000 7fc37e22f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:38.737 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.748+0000 7fc37e22f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.738 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.748+0000 7fc37e22f980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:38.738 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.748+0000 7fc37e22f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.748+0000 7fc37e22f980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:38.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.748+0000 7fc37e22f980 20 zone default found 2024-06-28T15:03:38.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.748+0000 7fc37e22f980 4 Realm: () 2024-06-28T15:03:38.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.748+0000 7fc37e22f980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:38.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.748+0000 7fc37e22f980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:38.740 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.748+0000 7fc37e22f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:38.740 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.748+0000 7fc37e22f980 20 zonegroup default 2024-06-28T15:03:38.740 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.748+0000 7fc37e22f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.740 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.748+0000 7fc37e22f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:38.740 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.748+0000 7fc37e22f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.752+0000 7fc37e22f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:38.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.752+0000 7fc37e22f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.752+0000 7fc37e22f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:38.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.752+0000 7fc37e22f980 20 started sync module instance, tier type = 2024-06-28T15:03:38.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.752+0000 7fc37e22f980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:38.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.776+0000 7fc37e22f980 20 add_watcher() i=0 2024-06-28T15:03:38.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.776+0000 7fc37e22f980 20 add_watcher() i=1 2024-06-28T15:03:38.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.776+0000 7fc37e22f980 20 add_watcher() i=2 2024-06-28T15:03:38.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.776+0000 7fc37e22f980 20 add_watcher() i=3 2024-06-28T15:03:38.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.776+0000 7fc37e22f980 20 add_watcher() i=4 2024-06-28T15:03:38.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.776+0000 7fc37e22f980 20 add_watcher() i=5 2024-06-28T15:03:38.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.776+0000 7fc37e22f980 20 add_watcher() i=6 2024-06-28T15:03:38.769 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.776+0000 7fc37e22f980 20 add_watcher() i=7 2024-06-28T15:03:38.769 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.776+0000 7fc37e22f980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:38.773 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.784+0000 7fc22d7fa640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:38.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.784+0000 7fc37e22f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:38.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.784+0000 7fc37e22f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:38.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.784+0000 7fc37e22f980 5 note: GC not initialized 2024-06-28T15:03:38.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.784+0000 7fc1a7fff640 20 reqs_thread_entry: start 2024-06-28T15:03:38.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.784+0000 7fc37e22f980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:38.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.784+0000 7fc37e22f980 20 Filter name: none 2024-06-28T15:03:38.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.784+0000 7fc1a5ffb640 20 reqs_thread_entry: start 2024-06-28T15:03:38.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.784+0000 7fc37e22f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.784+0000 7fc37e22f980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:38.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.784+0000 7fc37e22f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:38.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.788+0000 7fc37e22f980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:38.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.788+0000 7fc37e22f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:38.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.788+0000 7fc37e22f980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-06-28T15:03:38.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.788+0000 7fc37e22f980 20 INFO: main: starting shard=0 2024-06-28T15:03:38.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.788+0000 7fc37e22f980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:38.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.788+0000 7fc37e22f980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-06-28T15:03:38.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.788+0000 7fc37e22f980 20 INFO: main: done 2024-06-28T15:03:38.779 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.788+0000 7fc22d7fa640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:38.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.792+0000 7fc37e22f980 20 remove_watcher() i=0 2024-06-28T15:03:38.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.792+0000 7fc37e22f980 2 removed watcher, disabling cache 2024-06-28T15:03:38.787 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.796+0000 7fc37e22f980 20 remove_watcher() i=1 2024-06-28T15:03:38.789 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.800+0000 7fc37e22f980 20 remove_watcher() i=2 2024-06-28T15:03:38.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.800+0000 7fc37e22f980 20 remove_watcher() i=3 2024-06-28T15:03:38.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.804+0000 7fc37e22f980 20 remove_watcher() i=4 2024-06-28T15:03:38.795 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.804+0000 7fc37e22f980 20 remove_watcher() i=5 2024-06-28T15:03:38.797 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.804+0000 7fc37e22f980 20 remove_watcher() i=6 2024-06-28T15:03:38.798 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:38.808+0000 7fc37e22f980 20 remove_watcher() i=7 2024-06-28T15:03:38.852 INFO:tasks.workunit.client.0.smithi178.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-06-28T15:03:39.281 INFO:tasks.workunit.client.0.smithi178.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-06-28T15:03:39.698 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: bucket check unlinked finds unlistable entries 2024-06-28T15:03:39.698 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:39.699 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-06-28T15:03:39.733 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:39.733 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:39.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:39.784+0000 7fb49378b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:39.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:39.788+0000 7fb49378b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:39.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:39.788+0000 7fb49378b980 20 realm 2024-06-28T15:03:39.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:39.788+0000 7fb49378b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:39.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:39.788+0000 7fb49378b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:39.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:39.788+0000 7fb49378b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:39.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:39.788+0000 7fb49378b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:39.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:39.788+0000 7fb49378b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:39.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:39.788+0000 7fb49378b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:39.779 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:39.788+0000 7fb49378b980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:39.779 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:39.788+0000 7fb49378b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:39.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:39.788+0000 7fb49378b980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:39.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:39.788+0000 7fb49378b980 20 searching for the correct realm 2024-06-28T15:03:39.786 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:39.796+0000 7fb491a9d640 0 -- 172.21.15.178:0/3462890713 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fb4881071f0 msgr2=0x7fb4880ff240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:40.003 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.012+0000 7fb49129c640 0 -- 172.21.15.178:0/3462890713 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x559c998c12d0 msgr2=0x559c998c36c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:40.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:40.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:40.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:40.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:40.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:40.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:40.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:40.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.012 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:40.012 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:40.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 zone default found 2024-06-28T15:03:40.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 4 Realm: () 2024-06-28T15:03:40.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:40.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:40.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:40.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 zonegroup default 2024-06-28T15:03:40.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.020+0000 7fb49378b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.024+0000 7fb49378b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:40.014 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.024+0000 7fb49378b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.014 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.024+0000 7fb49378b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:40.014 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.024+0000 7fb49378b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.014 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.024+0000 7fb49378b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:40.014 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.024+0000 7fb49378b980 20 started sync module instance, tier type = 2024-06-28T15:03:40.015 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.024+0000 7fb49378b980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:40.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.048+0000 7fb49378b980 20 add_watcher() i=0 2024-06-28T15:03:40.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.048+0000 7fb49378b980 20 add_watcher() i=1 2024-06-28T15:03:40.039 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.048+0000 7fb49378b980 20 add_watcher() i=2 2024-06-28T15:03:40.039 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.048+0000 7fb49378b980 20 add_watcher() i=3 2024-06-28T15:03:40.039 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.048+0000 7fb49378b980 20 add_watcher() i=4 2024-06-28T15:03:40.039 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.048+0000 7fb49378b980 20 add_watcher() i=5 2024-06-28T15:03:40.039 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.048+0000 7fb49378b980 20 add_watcher() i=6 2024-06-28T15:03:40.041 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.048+0000 7fb49378b980 20 add_watcher() i=7 2024-06-28T15:03:40.041 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.048+0000 7fb49378b980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:40.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.052+0000 7fb3477e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:40.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:40.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:40.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 5 note: GC not initialized 2024-06-28T15:03:40.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb3467e4640 20 reqs_thread_entry: start 2024-06-28T15:03:40.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:40.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 20 Filter name: none 2024-06-28T15:03:40.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb2b7fff640 20 reqs_thread_entry: start 2024-06-28T15:03:40.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-06-28T15:03:40.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:40.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-06-28T15:03:40.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-06-28T15:03:40.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 : miss 2024-06-28T15:03:40.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:40.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 info.flags=0x17 2024-06-28T15:03:40.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 to cache LRU end 2024-06-28T15:03:40.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:03:40.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-06-28T15:03:40.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 2024-06-28T15:03:40.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.056+0000 7fb49378b980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.060+0000 7fb49378b980 1 NOTICE: finished shard 0 (4 entries found) 2024-06-28T15:03:40.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.060+0000 7fb49378b980 1 NOTICE: finished all shards (4 entries found) 2024-06-28T15:03:40.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.060+0000 7fb3477e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:40.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.068+0000 7fb49378b980 20 remove_watcher() i=0 2024-06-28T15:03:40.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.068+0000 7fb49378b980 2 removed watcher, disabling cache 2024-06-28T15:03:40.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.068+0000 7fb49378b980 20 remove_watcher() i=1 2024-06-28T15:03:40.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.072+0000 7fb49378b980 20 remove_watcher() i=2 2024-06-28T15:03:40.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.072+0000 7fb49378b980 20 remove_watcher() i=3 2024-06-28T15:03:40.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.076+0000 7fb49378b980 20 remove_watcher() i=4 2024-06-28T15:03:40.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.076+0000 7fb49378b980 20 remove_watcher() i=5 2024-06-28T15:03:40.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.080+0000 7fb49378b980 20 remove_watcher() i=6 2024-06-28T15:03:40.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.080+0000 7fb49378b980 20 remove_watcher() i=7 2024-06-28T15:03:40.086 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: unlinked entries are not listable 2024-06-28T15:03:40.086 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:40.218 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: GET returns 404 for unlinked entry keys that have no other versions 2024-06-28T15:03:40.218 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:40.434 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: bucket check unlinked fixes unlistable entries 2024-06-28T15:03:40.434 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:40.434 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-06-28T15:03:40.474 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:40.474 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:40.517 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.528+0000 7fcdec453980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.528+0000 7fcdec453980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:40.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.528+0000 7fcdec453980 20 realm 2024-06-28T15:03:40.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.528+0000 7fcdec453980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.528+0000 7fcdec453980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:40.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.528+0000 7fcdec453980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:40.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.528+0000 7fcdec453980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.528+0000 7fcdec453980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:40.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.528+0000 7fcdec453980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.528+0000 7fcdec453980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:40.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.528+0000 7fcdec453980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.532+0000 7fcdec453980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:40.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.532+0000 7fcdec453980 20 searching for the correct realm 2024-06-28T15:03:40.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.536+0000 7fcde9763640 0 -- 172.21.15.178:0/1354014193 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fcddc0769c0 msgr2=0x7fcddc0638f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:40.533 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.544+0000 7fcdea765640 0 -- 172.21.15.178:0/1354014193 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fcde00aa8e0 msgr2=0x7fcde00b63c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:03:40.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.548+0000 7fcdec453980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:40.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.548+0000 7fcdec453980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:40.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.548+0000 7fcdec453980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:40.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.548+0000 7fcdec453980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:40.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.548+0000 7fcdec453980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:40.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.548+0000 7fcdec453980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:40.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.548+0000 7fcdec453980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.548+0000 7fcdec453980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:40.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.548+0000 7fcdec453980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.541 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.548+0000 7fcdec453980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:40.541 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.548+0000 7fcdec453980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:40.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 20 zone default found 2024-06-28T15:03:40.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 4 Realm: () 2024-06-28T15:03:40.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:40.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:40.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:40.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 20 zonegroup default 2024-06-28T15:03:40.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:40.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:40.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:40.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 20 started sync module instance, tier type = 2024-06-28T15:03:40.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.552+0000 7fcdec453980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:40.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.568+0000 7fcdec453980 20 add_watcher() i=0 2024-06-28T15:03:40.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.568+0000 7fcdec453980 20 add_watcher() i=1 2024-06-28T15:03:40.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.568+0000 7fcdec453980 20 add_watcher() i=2 2024-06-28T15:03:40.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.568+0000 7fcdec453980 20 add_watcher() i=3 2024-06-28T15:03:40.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.568+0000 7fcdec453980 20 add_watcher() i=4 2024-06-28T15:03:40.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.568+0000 7fcdec453980 20 add_watcher() i=5 2024-06-28T15:03:40.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.568+0000 7fcdec453980 20 add_watcher() i=6 2024-06-28T15:03:40.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.568+0000 7fcdec453980 20 add_watcher() i=7 2024-06-28T15:03:40.560 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.568+0000 7fcdec453980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:40.562 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcc9b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:40.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:40.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:40.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 5 note: GC not initialized 2024-06-28T15:03:40.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcc9a7fc640 20 reqs_thread_entry: start 2024-06-28T15:03:40.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:40.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 20 Filter name: none 2024-06-28T15:03:40.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcc0ffff640 20 reqs_thread_entry: start 2024-06-28T15:03:40.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-06-28T15:03:40.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:40.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-06-28T15:03:40.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-06-28T15:03:40.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 : miss 2024-06-28T15:03:40.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 20 rados->read ofs=0 len=0 2024-06-28T15:03:40.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:40.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 info.flags=0x17 2024-06-28T15:03:40.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 to cache LRU end 2024-06-28T15:03:40.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:03:40.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-06-28T15:03:40.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 2024-06-28T15:03:40.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.572+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.576+0000 7fcdec453980 20 get_obj_state: octx=0x5589dda12770 obj=check-bucket:c state=0x5589ddcc0718 s->prefetch_data=0 2024-06-28T15:03:40.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.576+0000 7fcdec453980 20 get_obj_state: setting s->obj_tag to c495fiqeh9kglkem6f8jm81newky8qwx 2024-06-28T15:03:40.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.576+0000 7fcdec453980 20 get_obj_state_impl: setting s->olh_tag to wwq33fjg97uhr9itkm6lgp6mk0cvpo8q 2024-06-28T15:03:40.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.576+0000 7fcdec453980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=wwq33fjg97uhr9itkm6lgp6mk0cvpo8q 2024-06-28T15:03:40.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.576+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.580+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.580+0000 7fcdec453980 20 olh_log_entry: epoch=3 op=3 key=c[p8Ymo8jK7NEHFkWKRpeyyZJ2B9HrR9V] 2024-06-28T15:03:40.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.580+0000 7fcdec453980 20 get_obj_state: octx=0x5589dda12770 obj=check-bucket:_:p8Ymo8jK7NEHFkWKRpeyyZJ2B9HrR9V_c state=0x5589ddcb9c98 s->prefetch_data=0 2024-06-28T15:03:40.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.580+0000 7fcdec453980 10 manifest: total_size = 11 2024-06-28T15:03:40.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.580+0000 7fcdec453980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.15292316766177518671 2024-06-28T15:03:40.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.580+0000 7fcdec453980 20 get_obj_state: octx=0x5589dda12770 obj=check-bucket:_:p8Ymo8jK7NEHFkWKRpeyyZJ2B9HrR9V_c state=0x5589ddcb9c98 s->prefetch_data=0 2024-06-28T15:03:40.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.580+0000 7fcdec453980 20 prepare_atomic_modification: state is not atomic. state=0x5589ddcb9c98 2024-06-28T15:03:40.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.580+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.584+0000 7fcdec453980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:03:40.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.584+0000 7fcdec453980 20 RGWObjManifest::operator++(): result: ofs=11 stripe_ofs=11 part_ofs=0 rule->part_size=0 2024-06-28T15:03:40.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.584+0000 7fcdec453980 0 deleting objects inline since gc isn't initialized 2024-06-28T15:03:40.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.584+0000 7fcdec453980 5 delete_objs_inline: removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.2__shadow:p8Ymo8jK7NEHFkWKRpeyyZJ2B9HrR9V_.FJsgKucMfjdNv8aW6HUt-tesRzxdn6W_0 2024-06-28T15:03:40.575 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.584+0000 7fcccd7fa640 20 handle_completion(): completion ok for obj=c[p8Ymo8jK7NEHFkWKRpeyyZJ2B9HrR9V] 2024-06-28T15:03:40.576 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.584+0000 7fcdec453980 5 delete_objs_inline: refcount put returned error -2 2024-06-28T15:03:40.578 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.588+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.578 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.588+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.588+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.588+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.588+0000 7fcdec453980 20 get_obj_state: octx=0x5589dda12770 obj=check-bucket:d state=0x5589ddcc0718 s->prefetch_data=0 2024-06-28T15:03:40.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.588+0000 7fcdec453980 20 get_obj_state: setting s->obj_tag to q8r9h2acp7p0d08n4rcr8zif7vj360d3 2024-06-28T15:03:40.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.588+0000 7fcdec453980 20 get_obj_state_impl: setting s->olh_tag to 1pqje13jsqexoqc81jptmr906f790eor 2024-06-28T15:03:40.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.588+0000 7fcdec453980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=1pqje13jsqexoqc81jptmr906f790eor 2024-06-28T15:03:40.583 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.592+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.592+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.592+0000 7fcdec453980 20 olh_log_entry: epoch=3 op=3 key=d[DxAXek4Q6osAIFRaPAG5QptB6ctGJ93] 2024-06-28T15:03:40.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.592+0000 7fcdec453980 20 get_obj_state: octx=0x5589dda12770 obj=check-bucket:_:DxAXek4Q6osAIFRaPAG5QptB6ctGJ93_d state=0x5589ddceb398 s->prefetch_data=0 2024-06-28T15:03:40.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.596+0000 7fcdec453980 10 manifest: total_size = 9 2024-06-28T15:03:40.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.596+0000 7fcdec453980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.3838615209483787876 2024-06-28T15:03:40.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.596+0000 7fcdec453980 20 get_obj_state: octx=0x5589dda12770 obj=check-bucket:_:DxAXek4Q6osAIFRaPAG5QptB6ctGJ93_d state=0x5589ddceb398 s->prefetch_data=0 2024-06-28T15:03:40.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.596+0000 7fcdec453980 20 prepare_atomic_modification: state is not atomic. state=0x5589ddceb398 2024-06-28T15:03:40.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.596+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.588 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.596+0000 7fcdec453980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:03:40.588 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.596+0000 7fcdec453980 20 RGWObjManifest::operator++(): result: ofs=9 stripe_ofs=9 part_ofs=0 rule->part_size=0 2024-06-28T15:03:40.588 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.596+0000 7fcdec453980 0 deleting objects inline since gc isn't initialized 2024-06-28T15:03:40.588 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.596+0000 7fcdec453980 5 delete_objs_inline: removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.2__shadow:DxAXek4Q6osAIFRaPAG5QptB6ctGJ93_.medctDS7dbPz191uT5MGn9kU42BBK51_0 2024-06-28T15:03:40.590 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.600+0000 7fccccff9640 20 handle_completion(): completion ok for obj=d[DxAXek4Q6osAIFRaPAG5QptB6ctGJ93] 2024-06-28T15:03:40.591 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.600+0000 7fcdec453980 5 delete_objs_inline: refcount put returned error -2 2024-06-28T15:03:40.593 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.600+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.593 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.600+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.604+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.604+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.595 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.604+0000 7fcdec453980 20 get_obj_state: octx=0x5589dda12770 obj=check-bucket:e state=0x5589ddcc0718 s->prefetch_data=0 2024-06-28T15:03:40.595 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.604+0000 7fcdec453980 20 get_obj_state: setting s->obj_tag to k6rr4opkehttf2iqn5cclgk6s5rac2yf 2024-06-28T15:03:40.595 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.604+0000 7fcdec453980 20 get_obj_state_impl: setting s->olh_tag to yh2jfh6k4dnbkff1qbw98yurgy462ms1 2024-06-28T15:03:40.595 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.604+0000 7fcdec453980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=yh2jfh6k4dnbkff1qbw98yurgy462ms1 2024-06-28T15:03:40.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.608+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.608+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.601 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.608+0000 7fcdec453980 20 olh_log_entry: epoch=2 op=2 key=e[] 2024-06-28T15:03:40.601 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.608+0000 7fcdec453980 20 olh_log_entry: epoch=2 op=3 key=e[Ui2JgryaW0v.IaODOYPv75mUKu05Nkk] 2024-06-28T15:03:40.601 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.608+0000 7fcdec453980 20 get_obj_state: octx=0x5589dda12770 obj=check-bucket:_:Ui2JgryaW0v.IaODOYPv75mUKu05Nkk_e state=0x5589ddceb398 s->prefetch_data=0 2024-06-28T15:03:40.601 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.612+0000 7fcdec453980 10 manifest: total_size = 22 2024-06-28T15:03:40.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.612+0000 7fcdec453980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.13979887240349429053 2024-06-28T15:03:40.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.612+0000 7fcdec453980 20 get_obj_state: octx=0x5589dda12770 obj=check-bucket:_:Ui2JgryaW0v.IaODOYPv75mUKu05Nkk_e state=0x5589ddceb398 s->prefetch_data=0 2024-06-28T15:03:40.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.612+0000 7fcdec453980 20 prepare_atomic_modification: state is not atomic. state=0x5589ddceb398 2024-06-28T15:03:40.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.612+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.616+0000 7fcdec453980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:03:40.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.616+0000 7fcdec453980 20 RGWObjManifest::operator++(): result: ofs=22 stripe_ofs=22 part_ofs=0 rule->part_size=0 2024-06-28T15:03:40.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.616+0000 7fcdec453980 0 deleting objects inline since gc isn't initialized 2024-06-28T15:03:40.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.616+0000 7fcdec453980 5 delete_objs_inline: removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.2__shadow:Ui2JgryaW0v.IaODOYPv75mUKu05Nkk_.B4k_eKE9qnlKdLewnGbgnXVGotUAXoa_0 2024-06-28T15:03:40.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.616+0000 7fcccd7fa640 20 handle_completion(): completion ok for obj=e[Ui2JgryaW0v.IaODOYPv75mUKu05Nkk] 2024-06-28T15:03:40.610 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.620+0000 7fcdec453980 5 delete_objs_inline: refcount put returned error -2 2024-06-28T15:03:40.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.624+0000 7fcdec453980 20 get_obj_state: octx=0x5589dda12770 obj=check-bucket:e state=0x5589ddcc0718 s->prefetch_data=0 2024-06-28T15:03:40.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.624+0000 7fcdec453980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=yh2jfh6k4dnbkff1qbw98yurgy462ms1 2024-06-28T15:03:40.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.632+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.632+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.632+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.632+0000 7fcdec453980 20 get_obj_state: octx=0x5589dda12770 obj=check-bucket:f state=0x5589ddcec2f8 s->prefetch_data=0 2024-06-28T15:03:40.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.632+0000 7fcdec453980 20 get_obj_state: setting s->obj_tag to e7w5rlsrpa2j13ejlwp6qe91oa7i8glj 2024-06-28T15:03:40.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.632+0000 7fcdec453980 20 get_obj_state_impl: setting s->olh_tag to ehd8pjcm0rk3qbqv41421xgh1n1ct3dw 2024-06-28T15:03:40.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.632+0000 7fcdec453980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=ehd8pjcm0rk3qbqv41421xgh1n1ct3dw 2024-06-28T15:03:40.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.636+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.636+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.636+0000 7fcdec453980 20 olh_log_entry: epoch=2 op=2 key=f[] 2024-06-28T15:03:40.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.636+0000 7fcdec453980 20 olh_log_entry: epoch=2 op=3 key=f[S01JltESt83JGmGLvSnLLe4Bd2Xcujq] 2024-06-28T15:03:40.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.636+0000 7fcdec453980 20 get_obj_state: octx=0x5589dda12770 obj=check-bucket:_:S01JltESt83JGmGLvSnLLe4Bd2Xcujq_f state=0x5589ddceb398 s->prefetch_data=0 2024-06-28T15:03:40.629 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.636+0000 7fcdec453980 10 manifest: total_size = 12 2024-06-28T15:03:40.629 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.636+0000 7fcdec453980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.586416847068908839 2024-06-28T15:03:40.629 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.636+0000 7fcdec453980 20 get_obj_state: octx=0x5589dda12770 obj=check-bucket:_:S01JltESt83JGmGLvSnLLe4Bd2Xcujq_f state=0x5589ddceb398 s->prefetch_data=0 2024-06-28T15:03:40.629 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.636+0000 7fcdec453980 20 prepare_atomic_modification: state is not atomic. state=0x5589ddceb398 2024-06-28T15:03:40.629 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.640+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.644+0000 7fcdec453980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:03:40.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.644+0000 7fcdec453980 20 RGWObjManifest::operator++(): result: ofs=12 stripe_ofs=12 part_ofs=0 rule->part_size=0 2024-06-28T15:03:40.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.644+0000 7fcdec453980 0 deleting objects inline since gc isn't initialized 2024-06-28T15:03:40.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.644+0000 7fcdec453980 5 delete_objs_inline: removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.2__shadow:S01JltESt83JGmGLvSnLLe4Bd2Xcujq_.OkTzmS4GIcWYG2_Xsa5l4-4eBdYGv6g_0 2024-06-28T15:03:40.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.644+0000 7fccccff9640 20 handle_completion(): completion ok for obj=f[S01JltESt83JGmGLvSnLLe4Bd2Xcujq] 2024-06-28T15:03:40.636 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.644+0000 7fcdec453980 5 delete_objs_inline: refcount put returned error -2 2024-06-28T15:03:40.639 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.648+0000 7fcdec453980 20 get_obj_state: octx=0x5589dda12770 obj=check-bucket:f state=0x5589ddcec2f8 s->prefetch_data=0 2024-06-28T15:03:40.639 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.648+0000 7fcdec453980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=ehd8pjcm0rk3qbqv41421xgh1n1ct3dw 2024-06-28T15:03:40.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.652+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.646 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.656+0000 7fcdec453980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:40.646 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.656+0000 7fcdec453980 1 NOTICE: finished shard 0 (4 entries removed) 2024-06-28T15:03:40.646 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.656+0000 7fcdec453980 1 NOTICE: finished all shards (4 entries removed) 2024-06-28T15:03:40.647 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.656+0000 7fcc9b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:40.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.660+0000 7fcdec453980 20 remove_watcher() i=0 2024-06-28T15:03:40.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.660+0000 7fcdec453980 2 removed watcher, disabling cache 2024-06-28T15:03:40.654 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.664+0000 7fcdec453980 20 remove_watcher() i=1 2024-06-28T15:03:40.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.664+0000 7fcdec453980 20 remove_watcher() i=2 2024-06-28T15:03:40.658 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.668+0000 7fcdec453980 20 remove_watcher() i=3 2024-06-28T15:03:40.660 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.668+0000 7fcdec453980 20 remove_watcher() i=4 2024-06-28T15:03:40.662 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.672+0000 7fcdec453980 20 remove_watcher() i=5 2024-06-28T15:03:40.663 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.672+0000 7fcdec453980 20 remove_watcher() i=6 2024-06-28T15:03:40.664 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:40.672+0000 7fcdec453980 20 remove_watcher() i=7 2024-06-28T15:03:40.869 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: bucket check unlinked does not affect normal entries 2024-06-28T15:03:40.869 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:41.258 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: bucket check unlinked does not find new unlistable entries after fix 2024-06-28T15:03:41.259 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:41.259 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-06-28T15:03:41.297 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:41.297 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:41.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.352+0000 7f9210fd1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:41.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.352+0000 7f9210fd1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:41.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.352+0000 7f9210fd1980 20 realm 2024-06-28T15:03:41.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.352+0000 7f9210fd1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:41.344 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.352+0000 7f9210fd1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:41.344 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.352+0000 7f9210fd1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:41.344 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.352+0000 7f9210fd1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:41.344 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.352+0000 7f9210fd1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:41.344 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.352+0000 7f9210fd1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:41.345 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.356+0000 7f9210fd1980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:41.345 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.356+0000 7f9210fd1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:41.346 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.356+0000 7f9210fd1980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:41.346 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.356+0000 7f9210fd1980 20 searching for the correct realm 2024-06-28T15:03:41.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.360+0000 7f920f2e3640 0 -- 172.21.15.178:0/896074445 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f91f8064310 msgr2=0x7f91f807d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:41.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.580+0000 7f920eae2640 0 -- 172.21.15.178:0/896074445 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x560558e5ad00 msgr2=0x560558e5d130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:41.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.588+0000 7f9210fd1980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:41.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.588+0000 7f9210fd1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:41.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.588+0000 7f9210fd1980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:41.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.588+0000 7f9210fd1980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:41.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.588+0000 7f9210fd1980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:41.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.588+0000 7f9210fd1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:41.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.588+0000 7f9210fd1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:41.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.588+0000 7f9210fd1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:41.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.588+0000 7f9210fd1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:41.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.588+0000 7f9210fd1980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:41.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.588+0000 7f9210fd1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:41.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:41.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 20 zone default found 2024-06-28T15:03:41.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 4 Realm: () 2024-06-28T15:03:41.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:41.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:41.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:41.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 20 zonegroup default 2024-06-28T15:03:41.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:41.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:41.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:41.583 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:41.583 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:41.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:41.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 20 started sync module instance, tier type = 2024-06-28T15:03:41.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.592+0000 7f9210fd1980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:41.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.620+0000 7f9210fd1980 20 add_watcher() i=0 2024-06-28T15:03:41.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.620+0000 7f9210fd1980 20 add_watcher() i=1 2024-06-28T15:03:41.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.620+0000 7f9210fd1980 20 add_watcher() i=2 2024-06-28T15:03:41.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.620+0000 7f9210fd1980 20 add_watcher() i=3 2024-06-28T15:03:41.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.620+0000 7f9210fd1980 20 add_watcher() i=4 2024-06-28T15:03:41.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.620+0000 7f9210fd1980 20 add_watcher() i=5 2024-06-28T15:03:41.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.620+0000 7f9210fd1980 20 add_watcher() i=6 2024-06-28T15:03:41.611 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.620+0000 7f9210fd1980 20 add_watcher() i=7 2024-06-28T15:03:41.611 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.620+0000 7f9210fd1980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:41.615 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.624+0000 7f90c57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:41.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.624+0000 7f9210fd1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:41.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.624+0000 7f9210fd1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:41.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.624+0000 7f9210fd1980 5 note: GC not initialized 2024-06-28T15:03:41.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.624+0000 7f903ffff640 20 reqs_thread_entry: start 2024-06-28T15:03:41.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:41.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 20 Filter name: none 2024-06-28T15:03:41.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f903dffb640 20 reqs_thread_entry: start 2024-06-28T15:03:41.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-06-28T15:03:41.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:41.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:41.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-06-28T15:03:41.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-06-28T15:03:41.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 : miss 2024-06-28T15:03:41.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:41.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:41.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 info.flags=0x17 2024-06-28T15:03:41.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 to cache LRU end 2024-06-28T15:03:41.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:03:41.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-06-28T15:03:41.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 2024-06-28T15:03:41.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:41.621 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 1 NOTICE: finished shard 0 (0 entries found) 2024-06-28T15:03:41.621 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.628+0000 7f9210fd1980 1 NOTICE: finished all shards (0 entries found) 2024-06-28T15:03:41.621 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.632+0000 7f90c57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:41.627 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.636+0000 7f9210fd1980 20 remove_watcher() i=0 2024-06-28T15:03:41.627 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.636+0000 7f9210fd1980 2 removed watcher, disabling cache 2024-06-28T15:03:41.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.640+0000 7f9210fd1980 20 remove_watcher() i=1 2024-06-28T15:03:41.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.640+0000 7f9210fd1980 20 remove_watcher() i=2 2024-06-28T15:03:41.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.644+0000 7f9210fd1980 20 remove_watcher() i=3 2024-06-28T15:03:41.636 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.644+0000 7f9210fd1980 20 remove_watcher() i=4 2024-06-28T15:03:41.637 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.648+0000 7f9210fd1980 20 remove_watcher() i=5 2024-06-28T15:03:41.639 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.648+0000 7f9210fd1980 20 remove_watcher() i=6 2024-06-28T15:03:41.640 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:41.648+0000 7f9210fd1980 20 remove_watcher() i=7 2024-06-28T15:03:42.104 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-06-28T15:03:42.138 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:42.233 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:42.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.192+0000 7fb13140f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.192+0000 7fb13140f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.192+0000 7fb13140f980 20 realm 2024-06-28T15:03:42.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.192+0000 7fb13140f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.192+0000 7fb13140f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.192+0000 7fb13140f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:42.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.192+0000 7fb13140f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.192+0000 7fb13140f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.192+0000 7fb13140f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.196+0000 7fb13140f980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:42.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.196+0000 7fb13140f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.196+0000 7fb13140f980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:42.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.196+0000 7fb13140f980 20 searching for the correct realm 2024-06-28T15:03:42.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.200+0000 7fb12e71f640 0 -- 172.21.15.178:0/3312578125 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fb120017070 msgr2=0x7fb120090ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:42.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.208+0000 7fb12e71f640 0 -- 172.21.15.178:0/3312578125 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7faff807d310 msgr2=0x7faff807f740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:42.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.212+0000 7fb13140f980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:42.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.212+0000 7fb13140f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:42.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.212+0000 7fb13140f980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:42.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.212+0000 7fb13140f980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:42.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.212+0000 7fb13140f980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:42.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.212+0000 7fb13140f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:42.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.212+0000 7fb13140f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:42.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:42.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 zone default found 2024-06-28T15:03:42.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 4 Realm: () 2024-06-28T15:03:42.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:42.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:42.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:42.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 zonegroup default 2024-06-28T15:03:42.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 started sync module instance, tier type = 2024-06-28T15:03:42.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.216+0000 7fb13140f980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:42.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.232+0000 7fb13140f980 20 add_watcher() i=0 2024-06-28T15:03:42.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.232+0000 7fb13140f980 20 add_watcher() i=1 2024-06-28T15:03:42.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.232+0000 7fb13140f980 20 add_watcher() i=2 2024-06-28T15:03:42.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.232+0000 7fb13140f980 20 add_watcher() i=3 2024-06-28T15:03:42.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.232+0000 7fb13140f980 20 add_watcher() i=4 2024-06-28T15:03:42.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.232+0000 7fb13140f980 20 add_watcher() i=5 2024-06-28T15:03:42.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.232+0000 7fb13140f980 20 add_watcher() i=6 2024-06-28T15:03:42.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.236+0000 7fb13140f980 20 add_watcher() i=7 2024-06-28T15:03:42.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.236+0000 7fb13140f980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:42.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.236+0000 7fafe57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:42.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.236+0000 7fb13140f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:42.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.236+0000 7fb13140f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:42.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.236+0000 7fb13140f980 5 note: GC not initialized 2024-06-28T15:03:42.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7faf5ffff640 20 reqs_thread_entry: start 2024-06-28T15:03:42.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7fb13140f980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:42.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7fb13140f980 20 Filter name: none 2024-06-28T15:03:42.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7faf5dffb640 20 reqs_thread_entry: start 2024-06-28T15:03:42.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7fb13140f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7fb13140f980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:42.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7fb13140f980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7fb13140f980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:42.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7fb13140f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:42.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7fb13140f980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-06-28T15:03:42.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7fb13140f980 20 INFO: main: starting shard=0 2024-06-28T15:03:42.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7fb13140f980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:42.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7fb13140f980 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, marker= 2024-06-28T15:03:42.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7fb13140f980 20 INFO: main: done 2024-06-28T15:03:42.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.240+0000 7fafe57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:42.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.248+0000 7fb13140f980 20 remove_watcher() i=0 2024-06-28T15:03:42.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.248+0000 7fb13140f980 2 removed watcher, disabling cache 2024-06-28T15:03:42.244 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.248+0000 7fb13140f980 20 remove_watcher() i=1 2024-06-28T15:03:42.244 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.252+0000 7fb13140f980 20 remove_watcher() i=2 2024-06-28T15:03:42.244 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.252+0000 7fb13140f980 20 remove_watcher() i=3 2024-06-28T15:03:42.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.256+0000 7fb13140f980 20 remove_watcher() i=4 2024-06-28T15:03:42.247 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.256+0000 7fb13140f980 20 remove_watcher() i=5 2024-06-28T15:03:42.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.260+0000 7fb13140f980 20 remove_watcher() i=6 2024-06-28T15:03:42.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.260+0000 7fb13140f980 20 remove_watcher() i=7 2024-06-28T15:03:42.264 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-06-28T15:03:42.293 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:42.293 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:42.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.344+0000 7f15295a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.348+0000 7f15295a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.348+0000 7f15295a0980 20 realm 2024-06-28T15:03:42.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.348+0000 7f15295a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.348+0000 7f15295a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.348+0000 7f15295a0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:42.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.348+0000 7f15295a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.348+0000 7f15295a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.348+0000 7f15295a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.348+0000 7f15295a0980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:42.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.348+0000 7f15295a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.348+0000 7f15295a0980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:42.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.348+0000 7f15295a0980 20 searching for the correct realm 2024-06-28T15:03:42.346 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.356+0000 7f15268b0640 0 -- 172.21.15.178:0/3110580646 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f15180f5d60 msgr2=0x7f15180fd5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:42.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.360+0000 7f15278b2640 0 -- 172.21.15.178:0/3110580646 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f151c084480 msgr2=0x7f151c063950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:03:42.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.368+0000 7f15295a0980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:42.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.368+0000 7f15295a0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:42.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.368+0000 7f15295a0980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:42.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.368+0000 7f15295a0980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:42.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.368+0000 7f15295a0980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:42.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.368+0000 7f15295a0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:42.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.368+0000 7f15295a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.361 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.368+0000 7f15295a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.361 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.368+0000 7f15295a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.361 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:42.361 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:42.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 20 zone default found 2024-06-28T15:03:42.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 4 Realm: () 2024-06-28T15:03:42.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:42.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:42.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:42.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 20 zonegroup default 2024-06-28T15:03:42.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.363 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.363 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.363 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.363 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 20 started sync module instance, tier type = 2024-06-28T15:03:42.363 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.372+0000 7f15295a0980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:42.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.388+0000 7f15295a0980 20 add_watcher() i=0 2024-06-28T15:03:42.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.388+0000 7f15295a0980 20 add_watcher() i=1 2024-06-28T15:03:42.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.388+0000 7f15295a0980 20 add_watcher() i=2 2024-06-28T15:03:42.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.388+0000 7f15295a0980 20 add_watcher() i=3 2024-06-28T15:03:42.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.388+0000 7f15295a0980 20 add_watcher() i=4 2024-06-28T15:03:42.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.388+0000 7f15295a0980 20 add_watcher() i=5 2024-06-28T15:03:42.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.388+0000 7f15295a0980 20 add_watcher() i=6 2024-06-28T15:03:42.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.388+0000 7f15295a0980 20 add_watcher() i=7 2024-06-28T15:03:42.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.388+0000 7f15295a0980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:42.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f13ddfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:42.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f15295a0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:42.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f15295a0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:42.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f15295a0980 5 note: GC not initialized 2024-06-28T15:03:42.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f13dcfe1640 20 reqs_thread_entry: start 2024-06-28T15:03:42.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f15295a0980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:42.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f15295a0980 20 Filter name: none 2024-06-28T15:03:42.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f13567fc640 20 reqs_thread_entry: start 2024-06-28T15:03:42.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f15295a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f15295a0980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:42.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f15295a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f15295a0980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:42.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f15295a0980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:42.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f15295a0980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-06-28T15:03:42.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f15295a0980 20 INFO: main: starting shard=0 2024-06-28T15:03:42.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.392+0000 7f15295a0980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:42.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.396+0000 7f15295a0980 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, marker= 2024-06-28T15:03:42.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.396+0000 7f15295a0980 20 INFO: main: done 2024-06-28T15:03:42.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.396+0000 7f13ddfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:42.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.400+0000 7f15295a0980 20 remove_watcher() i=0 2024-06-28T15:03:42.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.400+0000 7f15295a0980 2 removed watcher, disabling cache 2024-06-28T15:03:42.393 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.404+0000 7f15295a0980 20 remove_watcher() i=1 2024-06-28T15:03:42.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.404+0000 7f15295a0980 20 remove_watcher() i=2 2024-06-28T15:03:42.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.408+0000 7f15295a0980 20 remove_watcher() i=3 2024-06-28T15:03:42.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.408+0000 7f15295a0980 20 remove_watcher() i=4 2024-06-28T15:03:42.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.408+0000 7f15295a0980 20 remove_watcher() i=5 2024-06-28T15:03:42.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.412+0000 7f15295a0980 20 remove_watcher() i=6 2024-06-28T15:03:42.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.412+0000 7f15295a0980 20 remove_watcher() i=7 2024-06-28T15:03:42.416 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: bucket check olh finds leftover OLH entries 2024-06-28T15:03:42.416 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:42.417 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2024-06-28T15:03:42.445 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:42.445 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:42.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.496+0000 7f5c016a1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.500+0000 7f5c016a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.500+0000 7f5c016a1980 20 realm 2024-06-28T15:03:42.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.500+0000 7f5c016a1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.500+0000 7f5c016a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.500+0000 7f5c016a1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:42.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.500+0000 7f5c016a1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.500+0000 7f5c016a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.500+0000 7f5c016a1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.500+0000 7f5c016a1980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:42.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.500+0000 7f5c016a1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.500+0000 7f5c016a1980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:42.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.500+0000 7f5c016a1980 20 searching for the correct realm 2024-06-28T15:03:42.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.508+0000 7f5bff9b3640 0 -- 172.21.15.178:0/3439986081 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f5bf4008050 msgr2=0x7f5bf40859d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:03:42.714 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.724+0000 7f5bff1b2640 0 -- 172.21.15.178:0/3439986081 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55ad278262d0 msgr2=0x55ad278286c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:42.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:42.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:42.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:42.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:42.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:42.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:42.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:42.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:42.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 zone default found 2024-06-28T15:03:42.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 4 Realm: () 2024-06-28T15:03:42.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:42.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:42.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:42.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 zonegroup default 2024-06-28T15:03:42.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.732+0000 7f5c016a1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.736+0000 7f5c016a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.736+0000 7f5c016a1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.736+0000 7f5c016a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.736+0000 7f5c016a1980 20 started sync module instance, tier type = 2024-06-28T15:03:42.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.736+0000 7f5c016a1980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:42.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.760+0000 7f5c016a1980 20 add_watcher() i=0 2024-06-28T15:03:42.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.760+0000 7f5c016a1980 20 add_watcher() i=1 2024-06-28T15:03:42.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.760+0000 7f5c016a1980 20 add_watcher() i=2 2024-06-28T15:03:42.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.760+0000 7f5c016a1980 20 add_watcher() i=3 2024-06-28T15:03:42.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.760+0000 7f5c016a1980 20 add_watcher() i=4 2024-06-28T15:03:42.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.760+0000 7f5c016a1980 20 add_watcher() i=5 2024-06-28T15:03:42.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.760+0000 7f5c016a1980 20 add_watcher() i=6 2024-06-28T15:03:42.753 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.760+0000 7f5c016a1980 20 add_watcher() i=7 2024-06-28T15:03:42.753 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.760+0000 7f5c016a1980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:42.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.764+0000 7f5ab5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:42.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:42.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:42.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 5 note: GC not initialized 2024-06-28T15:03:42.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5ab4fe1640 20 reqs_thread_entry: start 2024-06-28T15:03:42.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:42.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 20 Filter name: none 2024-06-28T15:03:42.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5a2e7fc640 20 reqs_thread_entry: start 2024-06-28T15:03:42.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-06-28T15:03:42.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:42.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-06-28T15:03:42.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-06-28T15:03:42.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 : miss 2024-06-28T15:03:42.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:42.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 info.flags=0x17 2024-06-28T15:03:42.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 to cache LRU end 2024-06-28T15:03:42.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:03:42.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-06-28T15:03:42.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 2024-06-28T15:03:42.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.768+0000 7f5c016a1980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:42.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.772+0000 7f5c016a1980 1 NOTICE: finished shard 0 (2 entries found) 2024-06-28T15:03:42.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.772+0000 7f5c016a1980 1 NOTICE: finished all shards (2 entries found) 2024-06-28T15:03:42.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.772+0000 7f5ab5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:42.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.776+0000 7f5c016a1980 20 remove_watcher() i=0 2024-06-28T15:03:42.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.776+0000 7f5c016a1980 2 removed watcher, disabling cache 2024-06-28T15:03:42.769 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.780+0000 7f5c016a1980 20 remove_watcher() i=1 2024-06-28T15:03:42.772 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.780+0000 7f5c016a1980 20 remove_watcher() i=2 2024-06-28T15:03:42.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.784+0000 7f5c016a1980 20 remove_watcher() i=3 2024-06-28T15:03:42.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.784+0000 7f5c016a1980 20 remove_watcher() i=4 2024-06-28T15:03:42.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.788+0000 7f5c016a1980 20 remove_watcher() i=5 2024-06-28T15:03:42.779 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.788+0000 7f5c016a1980 20 remove_watcher() i=6 2024-06-28T15:03:42.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.788+0000 7f5c016a1980 20 remove_watcher() i=7 2024-06-28T15:03:42.794 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: bucket check olh fixes leftover OLH entries 2024-06-28T15:03:42.795 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:42.795 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket check olh --bucket check-bucket --fix --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-06-28T15:03:42.828 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:42.828 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:42.870 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.880+0000 7f4708d04980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.880+0000 7f4708d04980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.880+0000 7f4708d04980 20 realm 2024-06-28T15:03:42.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.880+0000 7f4708d04980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.880+0000 7f4708d04980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.880+0000 7f4708d04980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:42.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.880+0000 7f4708d04980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.880+0000 7f4708d04980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:42.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.880+0000 7f4708d04980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.873 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.884+0000 7f4708d04980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:42.873 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.884+0000 7f4708d04980 20 rados->read ofs=0 len=0 2024-06-28T15:03:42.874 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.884+0000 7f4708d04980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:42.874 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.884+0000 7f4708d04980 20 searching for the correct realm 2024-06-28T15:03:42.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:42.892+0000 7f4707016640 0 -- 172.21.15.178:0/2503670052 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f46f011a050 msgr2=0x7f46f012c840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:03:43.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.108+0000 7f4706815640 0 -- 172.21.15.178:0/2503670052 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x565274641a80 msgr2=0x565274643eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:43.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.116+0000 7f4708d04980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:43.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.116+0000 7f4708d04980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:43.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.116+0000 7f4708d04980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:43.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.116+0000 7f4708d04980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:43.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.116+0000 7f4708d04980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:43.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.116+0000 7f4708d04980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:43.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.116+0000 7f4708d04980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.116+0000 7f4708d04980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.116+0000 7f4708d04980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.116+0000 7f4708d04980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:43.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.116+0000 7f4708d04980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:43.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 20 zone default found 2024-06-28T15:03:43.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 4 Realm: () 2024-06-28T15:03:43.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:43.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:43.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:43.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 20 zonegroup default 2024-06-28T15:03:43.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.112 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 20 started sync module instance, tier type = 2024-06-28T15:03:43.112 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.120+0000 7f4708d04980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:43.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.144+0000 7f4708d04980 20 add_watcher() i=0 2024-06-28T15:03:43.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.144+0000 7f4708d04980 20 add_watcher() i=1 2024-06-28T15:03:43.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.144+0000 7f4708d04980 20 add_watcher() i=2 2024-06-28T15:03:43.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.144+0000 7f4708d04980 20 add_watcher() i=3 2024-06-28T15:03:43.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.144+0000 7f4708d04980 20 add_watcher() i=4 2024-06-28T15:03:43.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.144+0000 7f4708d04980 20 add_watcher() i=5 2024-06-28T15:03:43.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.144+0000 7f4708d04980 20 add_watcher() i=6 2024-06-28T15:03:43.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.148+0000 7f4708d04980 20 add_watcher() i=7 2024-06-28T15:03:43.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.148+0000 7f4708d04980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:43.143 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.152+0000 7f45bd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:43.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.152+0000 7f4708d04980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:43.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.152+0000 7f4708d04980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:43.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.152+0000 7f4708d04980 5 note: GC not initialized 2024-06-28T15:03:43.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.152+0000 7f4537fff640 20 reqs_thread_entry: start 2024-06-28T15:03:43.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.152+0000 7f4708d04980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:43.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 20 Filter name: none 2024-06-28T15:03:43.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4535ffb640 20 reqs_thread_entry: start 2024-06-28T15:03:43.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-06-28T15:03:43.146 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.146 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:43.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-06-28T15:03:43.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-06-28T15:03:43.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 : miss 2024-06-28T15:03:43.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:43.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 info.flags=0x17 2024-06-28T15:03:43.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 to cache LRU end 2024-06-28T15:03:43.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:03:43.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-06-28T15:03:43.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 2024-06-28T15:03:43.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:43.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 20 get_obj_state: octx=0x5652743d1760 obj=check-bucket:c state=0x56527467be38 s->prefetch_data=0 2024-06-28T15:03:43.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 20 get_obj_state: setting s->obj_tag to c495fiqeh9kglkem6f8jm81newky8qwx 2024-06-28T15:03:43.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 20 get_obj_state_impl: setting s->olh_tag to wwq33fjg97uhr9itkm6lgp6mk0cvpo8q 2024-06-28T15:03:43.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:43.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 20 olh_log_entry: epoch=4 op=2 key=c[] 2024-06-28T15:03:43.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.156+0000 7f4708d04980 20 olh_log_entry: epoch=4 op=3 key=c[qhgWYNqULanBadfe3HZlrYmBs13NoP5] 2024-06-28T15:03:43.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.160+0000 7f4708d04980 20 get_obj_state: octx=0x5652746a52a0 obj=check-bucket:_:qhgWYNqULanBadfe3HZlrYmBs13NoP5_c state=0x5652746766c8 s->prefetch_data=0 2024-06-28T15:03:43.152 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.160+0000 7f4708d04980 20 get_obj_state: octx=0x5652746a52a0 obj=check-bucket:c state=0x565274678538 s->prefetch_data=0 2024-06-28T15:03:43.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.160+0000 7f4708d04980 20 get_obj_state: setting s->obj_tag to c495fiqeh9kglkem6f8jm81newky8qwx 2024-06-28T15:03:43.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.160+0000 7f4708d04980 20 get_obj_state_impl: setting s->olh_tag to wwq33fjg97uhr9itkm6lgp6mk0cvpo8q 2024-06-28T15:03:43.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.160+0000 7f4708d04980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=wwq33fjg97uhr9itkm6lgp6mk0cvpo8q 2024-06-28T15:03:43.158 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.168+0000 7f4708d04980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:43.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.172+0000 7f4708d04980 20 get_obj_state: octx=0x5652743d1760 obj=check-bucket:d state=0x56527467be38 s->prefetch_data=0 2024-06-28T15:03:43.162 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.172+0000 7f4708d04980 20 get_obj_state: setting s->obj_tag to q8r9h2acp7p0d08n4rcr8zif7vj360d3 2024-06-28T15:03:43.162 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.172+0000 7f4708d04980 20 get_obj_state_impl: setting s->olh_tag to 1pqje13jsqexoqc81jptmr906f790eor 2024-06-28T15:03:43.162 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.172+0000 7f4708d04980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:43.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.172+0000 7f4708d04980 20 olh_log_entry: epoch=4 op=2 key=d[] 2024-06-28T15:03:43.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.172+0000 7f4708d04980 20 olh_log_entry: epoch=4 op=3 key=d[wO-kvZNy3NI57ENRnC-KUjszZB9m2BV] 2024-06-28T15:03:43.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.172+0000 7f4708d04980 20 get_obj_state: octx=0x5652746a52a0 obj=check-bucket:_:wO-kvZNy3NI57ENRnC-KUjszZB9m2BV_d state=0x5652746a5d58 s->prefetch_data=0 2024-06-28T15:03:43.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.176+0000 7f4708d04980 20 get_obj_state: octx=0x5652746a52a0 obj=check-bucket:d state=0x565274679098 s->prefetch_data=0 2024-06-28T15:03:43.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.176+0000 7f4708d04980 20 get_obj_state: setting s->obj_tag to q8r9h2acp7p0d08n4rcr8zif7vj360d3 2024-06-28T15:03:43.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.176+0000 7f4708d04980 20 get_obj_state_impl: setting s->olh_tag to 1pqje13jsqexoqc81jptmr906f790eor 2024-06-28T15:03:43.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.176+0000 7f4708d04980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=1pqje13jsqexoqc81jptmr906f790eor 2024-06-28T15:03:43.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.180+0000 7f4708d04980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:43.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.180+0000 7f4708d04980 1 NOTICE: finished shard 0 (2 entries removed) 2024-06-28T15:03:43.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.180+0000 7f4708d04980 1 NOTICE: finished all shards (2 entries removed) 2024-06-28T15:03:43.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.184+0000 7f45bd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:43.178 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.188+0000 7f4708d04980 20 remove_watcher() i=0 2024-06-28T15:03:43.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.188+0000 7f4708d04980 2 removed watcher, disabling cache 2024-06-28T15:03:43.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.188+0000 7f4708d04980 20 remove_watcher() i=1 2024-06-28T15:03:43.183 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.192+0000 7f4708d04980 20 remove_watcher() i=2 2024-06-28T15:03:43.185 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.196+0000 7f4708d04980 20 remove_watcher() i=3 2024-06-28T15:03:43.187 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.196+0000 7f4708d04980 20 remove_watcher() i=4 2024-06-28T15:03:43.189 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.196+0000 7f4708d04980 20 remove_watcher() i=5 2024-06-28T15:03:43.190 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.200+0000 7f4708d04980 20 remove_watcher() i=6 2024-06-28T15:03:43.191 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.200+0000 7f4708d04980 20 remove_watcher() i=7 2024-06-28T15:03:43.207 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-06-28T15:03:43.238 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:43.238 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:43.280 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.288+0000 7fd23b952980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.292+0000 7fd23b952980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.292+0000 7fd23b952980 20 realm 2024-06-28T15:03:43.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.292+0000 7fd23b952980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.292+0000 7fd23b952980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.292+0000 7fd23b952980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:43.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.292+0000 7fd23b952980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.292+0000 7fd23b952980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.292+0000 7fd23b952980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.292+0000 7fd23b952980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:43.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.292+0000 7fd23b952980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.292+0000 7fd23b952980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:43.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.292+0000 7fd23b952980 20 searching for the correct realm 2024-06-28T15:03:43.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.300+0000 7fd238c62640 0 -- 172.21.15.178:0/4228273356 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fd22c015070 msgr2=0x7fd22c06c050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:43.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.304+0000 7fd238c62640 0 -- 172.21.15.178:0/4228273356 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fd10407d2e0 msgr2=0x7fd10407f710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:43.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:43.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:43.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:43.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:43.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:43.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:43.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:43.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:43.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 zone default found 2024-06-28T15:03:43.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 4 Realm: () 2024-06-28T15:03:43.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:43.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:43.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:43.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 zonegroup default 2024-06-28T15:03:43.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.312+0000 7fd23b952980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.316+0000 7fd23b952980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.316+0000 7fd23b952980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.316+0000 7fd23b952980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.316+0000 7fd23b952980 20 started sync module instance, tier type = 2024-06-28T15:03:43.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.316+0000 7fd23b952980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:43.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.332+0000 7fd23b952980 20 add_watcher() i=0 2024-06-28T15:03:43.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.332+0000 7fd23b952980 20 add_watcher() i=1 2024-06-28T15:03:43.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.332+0000 7fd23b952980 20 add_watcher() i=2 2024-06-28T15:03:43.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.332+0000 7fd23b952980 20 add_watcher() i=3 2024-06-28T15:03:43.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.332+0000 7fd23b952980 20 add_watcher() i=4 2024-06-28T15:03:43.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.332+0000 7fd23b952980 20 add_watcher() i=5 2024-06-28T15:03:43.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.332+0000 7fd23b952980 20 add_watcher() i=6 2024-06-28T15:03:43.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.332+0000 7fd23b952980 20 add_watcher() i=7 2024-06-28T15:03:43.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.332+0000 7fd23b952980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:43.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd0ef7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:43.326 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:43.326 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:43.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 5 note: GC not initialized 2024-06-28T15:03:43.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd0ee7e4640 20 reqs_thread_entry: start 2024-06-28T15:03:43.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:43.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 20 Filter name: none 2024-06-28T15:03:43.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd05ffff640 20 reqs_thread_entry: start 2024-06-28T15:03:43.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:43.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.328 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:43.328 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:43.328 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-06-28T15:03:43.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 20 INFO: main: starting shard=0 2024-06-28T15:03:43.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:43.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-06-28T15:03:43.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd23b952980 20 INFO: main: done 2024-06-28T15:03:43.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.336+0000 7fd0ef7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:43.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.340+0000 7fd23b952980 20 remove_watcher() i=0 2024-06-28T15:03:43.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.340+0000 7fd23b952980 2 removed watcher, disabling cache 2024-06-28T15:03:43.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.344+0000 7fd23b952980 20 remove_watcher() i=1 2024-06-28T15:03:43.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.348+0000 7fd23b952980 20 remove_watcher() i=2 2024-06-28T15:03:43.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.348+0000 7fd23b952980 20 remove_watcher() i=3 2024-06-28T15:03:43.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.352+0000 7fd23b952980 20 remove_watcher() i=4 2024-06-28T15:03:43.342 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.352+0000 7fd23b952980 20 remove_watcher() i=5 2024-06-28T15:03:43.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.352+0000 7fd23b952980 20 remove_watcher() i=6 2024-06-28T15:03:43.344 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.352+0000 7fd23b952980 20 remove_watcher() i=7 2024-06-28T15:03:43.358 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-06-28T15:03:43.387 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:43.387 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:43.433 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.440+0000 7fc474412980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.444+0000 7fc474412980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.444+0000 7fc474412980 20 realm 2024-06-28T15:03:43.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.444+0000 7fc474412980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.444+0000 7fc474412980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.444+0000 7fc474412980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:43.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.444+0000 7fc474412980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.444+0000 7fc474412980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.444+0000 7fc474412980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.444+0000 7fc474412980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:43.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.444+0000 7fc474412980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.448+0000 7fc474412980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:43.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.448+0000 7fc474412980 20 searching for the correct realm 2024-06-28T15:03:43.444 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.452+0000 7fc472724640 0 -- 172.21.15.178:0/2047311062 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fc468008040 msgr2=0x7fc468060910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:43.658 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.668+0000 7fc471f23640 0 -- 172.21.15.178:0/2047311062 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55be67730c50 msgr2=0x55be67733110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:43.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:43.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:43.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:43.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:43.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:43.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:43.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.667 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:43.667 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:43.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 zone default found 2024-06-28T15:03:43.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 4 Realm: () 2024-06-28T15:03:43.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:43.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:43.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:43.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 zonegroup default 2024-06-28T15:03:43.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.669 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.669 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.676+0000 7fc474412980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.669 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.680+0000 7fc474412980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.669 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.680+0000 7fc474412980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.670 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.680+0000 7fc474412980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.670 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.680+0000 7fc474412980 20 started sync module instance, tier type = 2024-06-28T15:03:43.670 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.680+0000 7fc474412980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:43.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.704+0000 7fc474412980 20 add_watcher() i=0 2024-06-28T15:03:43.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.704+0000 7fc474412980 20 add_watcher() i=1 2024-06-28T15:03:43.694 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.704+0000 7fc474412980 20 add_watcher() i=2 2024-06-28T15:03:43.694 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.704+0000 7fc474412980 20 add_watcher() i=3 2024-06-28T15:03:43.694 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.704+0000 7fc474412980 20 add_watcher() i=4 2024-06-28T15:03:43.694 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.704+0000 7fc474412980 20 add_watcher() i=5 2024-06-28T15:03:43.694 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.704+0000 7fc474412980 20 add_watcher() i=6 2024-06-28T15:03:43.695 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.704+0000 7fc474412980 20 add_watcher() i=7 2024-06-28T15:03:43.696 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.704+0000 7fc474412980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:43.699 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.708+0000 7fc3237fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:43.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.708+0000 7fc474412980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:43.701 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.708+0000 7fc474412980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:43.701 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.708+0000 7fc474412980 5 note: GC not initialized 2024-06-28T15:03:43.701 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.712+0000 7fc3227fc640 20 reqs_thread_entry: start 2024-06-28T15:03:43.701 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.712+0000 7fc474412980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:43.701 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.712+0000 7fc474412980 20 Filter name: none 2024-06-28T15:03:43.702 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.712+0000 7fc293fff640 20 reqs_thread_entry: start 2024-06-28T15:03:43.702 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.712+0000 7fc474412980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.702 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.712+0000 7fc474412980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:43.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.712+0000 7fc474412980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.712+0000 7fc474412980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:43.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.712+0000 7fc474412980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:43.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.712+0000 7fc474412980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-06-28T15:03:43.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.712+0000 7fc474412980 20 INFO: main: starting shard=0 2024-06-28T15:03:43.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.712+0000 7fc474412980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:43.704 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.712+0000 7fc474412980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-06-28T15:03:43.704 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.712+0000 7fc474412980 20 INFO: main: done 2024-06-28T15:03:43.705 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.716+0000 7fc3237fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:43.710 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.720+0000 7fc474412980 20 remove_watcher() i=0 2024-06-28T15:03:43.710 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.720+0000 7fc474412980 2 removed watcher, disabling cache 2024-06-28T15:03:43.713 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.720+0000 7fc474412980 20 remove_watcher() i=1 2024-06-28T15:03:43.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.724+0000 7fc474412980 20 remove_watcher() i=2 2024-06-28T15:03:43.717 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.728+0000 7fc474412980 20 remove_watcher() i=3 2024-06-28T15:03:43.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.728+0000 7fc474412980 20 remove_watcher() i=4 2024-06-28T15:03:43.721 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.732+0000 7fc474412980 20 remove_watcher() i=5 2024-06-28T15:03:43.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.732+0000 7fc474412980 20 remove_watcher() i=6 2024-06-28T15:03:43.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.732+0000 7fc474412980 20 remove_watcher() i=7 2024-06-28T15:03:43.741 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: bucket check olh does not find new leftover OLH entries after fix 2024-06-28T15:03:43.741 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:43.742 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2024-06-28T15:03:43.774 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:43.774 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:43.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.828+0000 7fc2df3ba980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.828+0000 7fc2df3ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.828+0000 7fc2df3ba980 20 realm 2024-06-28T15:03:43.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.828+0000 7fc2df3ba980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.828+0000 7fc2df3ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.828+0000 7fc2df3ba980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:43.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.832+0000 7fc2df3ba980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.832+0000 7fc2df3ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:43.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.832+0000 7fc2df3ba980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.823 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.832+0000 7fc2df3ba980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:43.823 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.832+0000 7fc2df3ba980 20 rados->read ofs=0 len=0 2024-06-28T15:03:43.823 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.832+0000 7fc2df3ba980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:43.824 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.832+0000 7fc2df3ba980 20 searching for the correct realm 2024-06-28T15:03:43.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:43.836+0000 7fc2dc6ca640 0 -- 172.21.15.178:0/406687645 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fc2cc058170 msgr2=0x7fc2cc05b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:44.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.052+0000 7fc2dcecb640 0 -- 172.21.15.178:0/406687645 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fc2c407c2e0 msgr2=0x7fc2c4095970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:03:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.268+0000 7fc2df3ba980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.268+0000 7fc2df3ba980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.268+0000 7fc2df3ba980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.268+0000 7fc2df3ba980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.268+0000 7fc2df3ba980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.268+0000 7fc2df3ba980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.268+0000 7fc2df3ba980 20 rados->read ofs=0 len=0 2024-06-28T15:03:44.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 20 rados->read ofs=0 len=0 2024-06-28T15:03:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:44.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 20 rados->read ofs=0 len=0 2024-06-28T15:03:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 20 zone default found 2024-06-28T15:03:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 4 Realm: () 2024-06-28T15:03:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 20 zonegroup default 2024-06-28T15:03:44.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 20 rados->read ofs=0 len=0 2024-06-28T15:03:44.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:44.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 20 rados->read ofs=0 len=0 2024-06-28T15:03:44.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:44.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.272+0000 7fc2df3ba980 20 rados->read ofs=0 len=0 2024-06-28T15:03:44.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.276+0000 7fc2df3ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:44.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.276+0000 7fc2df3ba980 20 started sync module instance, tier type = 2024-06-28T15:03:44.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.276+0000 7fc2df3ba980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:44.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.296+0000 7fc2df3ba980 20 add_watcher() i=0 2024-06-28T15:03:44.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.296+0000 7fc2df3ba980 20 add_watcher() i=1 2024-06-28T15:03:44.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.296+0000 7fc2df3ba980 20 add_watcher() i=2 2024-06-28T15:03:44.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.296+0000 7fc2df3ba980 20 add_watcher() i=3 2024-06-28T15:03:44.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.296+0000 7fc2df3ba980 20 add_watcher() i=4 2024-06-28T15:03:44.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.296+0000 7fc2df3ba980 20 add_watcher() i=5 2024-06-28T15:03:44.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.296+0000 7fc2df3ba980 20 add_watcher() i=6 2024-06-28T15:03:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.300+0000 7fc2df3ba980 20 add_watcher() i=7 2024-06-28T15:03:44.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.300+0000 7fc2df3ba980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:44.294 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.304+0000 7fc192fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.304+0000 7fc2df3ba980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.304+0000 7fc2df3ba980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:44.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.304+0000 7fc2df3ba980 5 note: GC not initialized 2024-06-28T15:03:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.304+0000 7fc191fe3640 20 reqs_thread_entry: start 2024-06-28T15:03:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.304+0000 7fc2df3ba980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:44.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.304+0000 7fc2df3ba980 20 Filter name: none 2024-06-28T15:03:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.304+0000 7fc1037fe640 20 reqs_thread_entry: start 2024-06-28T15:03:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.304+0000 7fc2df3ba980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-06-28T15:03:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.304+0000 7fc2df3ba980 20 rados->read ofs=0 len=0 2024-06-28T15:03:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc2df3ba980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc2df3ba980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-06-28T15:03:44.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc2df3ba980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-06-28T15:03:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc2df3ba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 : miss 2024-06-28T15:03:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc2df3ba980 20 rados->read ofs=0 len=0 2024-06-28T15:03:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc2df3ba980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc2df3ba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 info.flags=0x17 2024-06-28T15:03:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc2df3ba980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 to cache LRU end 2024-06-28T15:03:44.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc2df3ba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:03:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc2df3ba980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-06-28T15:03:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc2df3ba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 2024-06-28T15:03:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc2df3ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc2df3ba980 1 NOTICE: finished shard 0 (0 entries found) 2024-06-28T15:03:44.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc2df3ba980 1 NOTICE: finished all shards (0 entries found) 2024-06-28T15:03:44.300 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.308+0000 7fc192fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.316+0000 7fc2df3ba980 20 remove_watcher() i=0 2024-06-28T15:03:44.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.316+0000 7fc2df3ba980 2 removed watcher, disabling cache 2024-06-28T15:03:44.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.316+0000 7fc2df3ba980 20 remove_watcher() i=1 2024-06-28T15:03:44.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.320+0000 7fc2df3ba980 20 remove_watcher() i=2 2024-06-28T15:03:44.312 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.320+0000 7fc2df3ba980 20 remove_watcher() i=3 2024-06-28T15:03:44.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.324+0000 7fc2df3ba980 20 remove_watcher() i=4 2024-06-28T15:03:44.317 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.324+0000 7fc2df3ba980 20 remove_watcher() i=5 2024-06-28T15:03:44.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.328+0000 7fc2df3ba980 20 remove_watcher() i=6 2024-06-28T15:03:44.320 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:44.328+0000 7fc2df3ba980 20 remove_watcher() i=7 2024-06-28T15:03:44.336 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: verify that bucket check fixes do not affect null version objects 2024-06-28T15:03:44.336 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:44.533 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: bucket check stats are correct in the presence of unlinked entries 2024-06-28T15:03:44.533 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:03:46.426 INFO:tasks.workunit.client.0.smithi178.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-06-28T15:03:46.858 INFO:tasks.workunit.client.0.smithi178.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-06-28T15:03:48.446 INFO:tasks.workunit.client.0.smithi178.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-06-28T15:03:48.446 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-06-28T15:03:48.480 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:48.480 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:48.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.532+0000 7ff606f65980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.536+0000 7ff606f65980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:48.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.536+0000 7ff606f65980 20 realm 2024-06-28T15:03:48.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.536+0000 7ff606f65980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.536+0000 7ff606f65980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:48.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.536+0000 7ff606f65980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:48.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.536+0000 7ff606f65980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.536+0000 7ff606f65980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:48.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.536+0000 7ff606f65980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.536+0000 7ff606f65980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:48.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.536+0000 7ff606f65980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.528 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.536+0000 7ff606f65980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:48.528 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.536+0000 7ff606f65980 20 searching for the correct realm 2024-06-28T15:03:48.534 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.544+0000 7ff604275640 0 -- 172.21.15.178:0/2442242298 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x5572f442c6d0 msgr2=0x5572f442eac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:48.539 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.548+0000 7ff604275640 0 -- 172.21.15.178:0/2442242298 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7ff5f41301a0 msgr2=0x7ff5f4130a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:03:48.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.556+0000 7ff606f65980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:48.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.556+0000 7ff606f65980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:48.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.556+0000 7ff606f65980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:48.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.556+0000 7ff606f65980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:48.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.556+0000 7ff606f65980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:48.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.556+0000 7ff606f65980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:48.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.556+0000 7ff606f65980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.556+0000 7ff606f65980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:48.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.556+0000 7ff606f65980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:48.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:48.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 20 zone default found 2024-06-28T15:03:48.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 4 Realm: () 2024-06-28T15:03:48.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:48.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:48.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:48.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 20 zonegroup default 2024-06-28T15:03:48.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.551 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:48.551 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.551 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:48.551 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.552 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:48.552 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 20 started sync module instance, tier type = 2024-06-28T15:03:48.552 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.560+0000 7ff606f65980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:48.567 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.576+0000 7ff606f65980 20 add_watcher() i=0 2024-06-28T15:03:48.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.576+0000 7ff606f65980 20 add_watcher() i=1 2024-06-28T15:03:48.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.576+0000 7ff606f65980 20 add_watcher() i=2 2024-06-28T15:03:48.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.576+0000 7ff606f65980 20 add_watcher() i=3 2024-06-28T15:03:48.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.576+0000 7ff606f65980 20 add_watcher() i=4 2024-06-28T15:03:48.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.576+0000 7ff606f65980 20 add_watcher() i=5 2024-06-28T15:03:48.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.576+0000 7ff606f65980 20 add_watcher() i=6 2024-06-28T15:03:48.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.576+0000 7ff606f65980 20 add_watcher() i=7 2024-06-28T15:03:48.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.576+0000 7ff606f65980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:48.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.580+0000 7ff4bafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:48.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.580+0000 7ff606f65980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:48.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.580+0000 7ff606f65980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:48.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.580+0000 7ff606f65980 5 note: GC not initialized 2024-06-28T15:03:48.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.580+0000 7ff4b9fe3640 20 reqs_thread_entry: start 2024-06-28T15:03:48.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.580+0000 7ff606f65980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:48.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.580+0000 7ff606f65980 20 Filter name: none 2024-06-28T15:03:48.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.580+0000 7ff42b7fe640 20 reqs_thread_entry: start 2024-06-28T15:03:48.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.580+0000 7ff606f65980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.580+0000 7ff606f65980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:48.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.580+0000 7ff606f65980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.584+0000 7ff606f65980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:48.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.584+0000 7ff606f65980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:48.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.584+0000 7ff606f65980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-06-28T15:03:48.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.584+0000 7ff606f65980 20 INFO: main: starting shard=0 2024-06-28T15:03:48.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.584+0000 7ff606f65980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:48.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.584+0000 7ff606f65980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-06-28T15:03:48.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.584+0000 7ff606f65980 20 INFO: main: done 2024-06-28T15:03:48.575 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.584+0000 7ff4bafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:48.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.588+0000 7ff606f65980 20 remove_watcher() i=0 2024-06-28T15:03:48.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.588+0000 7ff606f65980 2 removed watcher, disabling cache 2024-06-28T15:03:48.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.592+0000 7ff606f65980 20 remove_watcher() i=1 2024-06-28T15:03:48.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.592+0000 7ff606f65980 20 remove_watcher() i=2 2024-06-28T15:03:48.586 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.596+0000 7ff606f65980 20 remove_watcher() i=3 2024-06-28T15:03:48.588 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.596+0000 7ff606f65980 20 remove_watcher() i=4 2024-06-28T15:03:48.589 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.600+0000 7ff606f65980 20 remove_watcher() i=5 2024-06-28T15:03:48.591 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.600+0000 7ff606f65980 20 remove_watcher() i=6 2024-06-28T15:03:48.592 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.600+0000 7ff606f65980 20 remove_watcher() i=7 2024-06-28T15:03:48.810 INFO:tasks.workunit.client.0.smithi178.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-06-28T15:03:48.810 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-06-28T15:03:48.845 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:48.845 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:48.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.900+0000 7fad53788980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.900+0000 7fad53788980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:48.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.900+0000 7fad53788980 20 realm 2024-06-28T15:03:48.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.900+0000 7fad53788980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.900+0000 7fad53788980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:48.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.900+0000 7fad53788980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:48.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.900+0000 7fad53788980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.900+0000 7fad53788980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:48.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.900+0000 7fad53788980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.904+0000 7fad53788980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:48.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.904+0000 7fad53788980 20 rados->read ofs=0 len=0 2024-06-28T15:03:48.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.904+0000 7fad53788980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:48.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.904+0000 7fad53788980 20 searching for the correct realm 2024-06-28T15:03:48.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:48.912+0000 7fad51a9a640 0 -- 172.21.15.178:0/3242333119 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fad48070750 msgr2=0x7fad48068830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:03:49.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.128+0000 7fad51299640 0 -- 172.21.15.178:0/3242333119 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x5642fa651250 msgr2=0x5642fa653710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:49.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:49.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:49.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:49.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:49.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:49.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:49.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:49.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:49.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:49.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 zone default found 2024-06-28T15:03:49.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 4 Realm: () 2024-06-28T15:03:49.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:49.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:49.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:49.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 zonegroup default 2024-06-28T15:03:49.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:49.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.136+0000 7fad53788980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.140+0000 7fad53788980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:49.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.140+0000 7fad53788980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.140+0000 7fad53788980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:49.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.140+0000 7fad53788980 20 started sync module instance, tier type = 2024-06-28T15:03:49.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.140+0000 7fad53788980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:49.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.164+0000 7fad53788980 20 add_watcher() i=0 2024-06-28T15:03:49.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.164+0000 7fad53788980 20 add_watcher() i=1 2024-06-28T15:03:49.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.164+0000 7fad53788980 20 add_watcher() i=2 2024-06-28T15:03:49.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.164+0000 7fad53788980 20 add_watcher() i=3 2024-06-28T15:03:49.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.164+0000 7fad53788980 20 add_watcher() i=4 2024-06-28T15:03:49.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.164+0000 7fad53788980 20 add_watcher() i=5 2024-06-28T15:03:49.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.164+0000 7fad53788980 20 add_watcher() i=6 2024-06-28T15:03:49.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.164+0000 7fad53788980 20 add_watcher() i=7 2024-06-28T15:03:49.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.164+0000 7fad53788980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:49.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fac077e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:49.162 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fad53788980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:49.162 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fad53788980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:49.162 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fad53788980 5 note: GC not initialized 2024-06-28T15:03:49.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fac067e4640 20 reqs_thread_entry: start 2024-06-28T15:03:49.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fad53788980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:49.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fad53788980 20 Filter name: none 2024-06-28T15:03:49.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fab77fff640 20 reqs_thread_entry: start 2024-06-28T15:03:49.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fad53788980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.164 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fad53788980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:49.164 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fad53788980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fad53788980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:49.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fad53788980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:49.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fad53788980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-06-28T15:03:49.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fad53788980 20 INFO: main: starting shard=0 2024-06-28T15:03:49.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.172+0000 7fad53788980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:49.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.176+0000 7fad53788980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-06-28T15:03:49.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.176+0000 7fad53788980 20 INFO: main: done 2024-06-28T15:03:49.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.176+0000 7fac077e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:49.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.184+0000 7fad53788980 20 remove_watcher() i=0 2024-06-28T15:03:49.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.184+0000 7fad53788980 2 removed watcher, disabling cache 2024-06-28T15:03:49.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.184+0000 7fad53788980 20 remove_watcher() i=1 2024-06-28T15:03:49.177 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.188+0000 7fad53788980 20 remove_watcher() i=2 2024-06-28T15:03:49.180 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.188+0000 7fad53788980 20 remove_watcher() i=3 2024-06-28T15:03:49.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.192+0000 7fad53788980 20 remove_watcher() i=4 2024-06-28T15:03:49.183 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.192+0000 7fad53788980 20 remove_watcher() i=5 2024-06-28T15:03:49.185 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.192+0000 7fad53788980 20 remove_watcher() i=6 2024-06-28T15:03:49.186 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.196+0000 7fad53788980 20 remove_watcher() i=7 2024-06-28T15:03:49.418 INFO:tasks.workunit.client.0.smithi178.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-06-28T15:03:49.418 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket check-bucket --object e 2024-06-28T15:03:49.452 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:49.453 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:49.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.504+0000 7fa348484980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.508+0000 7fa348484980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:49.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.508+0000 7fa348484980 20 realm 2024-06-28T15:03:49.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.508+0000 7fa348484980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.508+0000 7fa348484980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:49.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.508+0000 7fa348484980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:49.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.508+0000 7fa348484980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.508+0000 7fa348484980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:49.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.508+0000 7fa348484980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.508+0000 7fa348484980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:49.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.508+0000 7fa348484980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.512+0000 7fa348484980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:49.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.512+0000 7fa348484980 20 searching for the correct realm 2024-06-28T15:03:49.507 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.516+0000 7fa346796640 0 -- 172.21.15.178:0/1433166684 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x55ad67110e00 msgr2=0x55ad671131f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:49.512 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.520+0000 7fa345f95640 0 -- 172.21.15.178:0/1433166684 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fa330038400 msgr2=0x7fa330038860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:49.520 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.528+0000 7fa348484980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:49.520 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.528+0000 7fa348484980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:49.520 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.528+0000 7fa348484980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:49.520 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.528+0000 7fa348484980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:49.520 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.528+0000 7fa348484980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:49.520 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.528+0000 7fa348484980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:49.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.528+0000 7fa348484980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.528+0000 7fa348484980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:49.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.528+0000 7fa348484980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:49.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:49.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 20 zone default found 2024-06-28T15:03:49.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 4 Realm: () 2024-06-28T15:03:49.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:49.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:49.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:49.523 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 20 zonegroup default 2024-06-28T15:03:49.523 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.523 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:49.523 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.523 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:49.523 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:49.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 20 started sync module instance, tier type = 2024-06-28T15:03:49.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.532+0000 7fa348484980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:49.539 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.548+0000 7fa348484980 20 add_watcher() i=0 2024-06-28T15:03:49.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.548+0000 7fa348484980 20 add_watcher() i=1 2024-06-28T15:03:49.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.548+0000 7fa348484980 20 add_watcher() i=2 2024-06-28T15:03:49.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.548+0000 7fa348484980 20 add_watcher() i=3 2024-06-28T15:03:49.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.548+0000 7fa348484980 20 add_watcher() i=4 2024-06-28T15:03:49.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.548+0000 7fa348484980 20 add_watcher() i=5 2024-06-28T15:03:49.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.548+0000 7fa348484980 20 add_watcher() i=6 2024-06-28T15:03:49.541 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.548+0000 7fa348484980 20 add_watcher() i=7 2024-06-28T15:03:49.541 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.548+0000 7fa348484980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:49.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.552+0000 7fa1f77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:49.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.552+0000 7fa348484980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:49.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.552+0000 7fa348484980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:49.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.552+0000 7fa348484980 5 note: GC not initialized 2024-06-28T15:03:49.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.552+0000 7fa1f67fc640 20 reqs_thread_entry: start 2024-06-28T15:03:49.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.552+0000 7fa348484980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:49.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.552+0000 7fa348484980 20 Filter name: none 2024-06-28T15:03:49.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.552+0000 7fa167fff640 20 reqs_thread_entry: start 2024-06-28T15:03:49.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.552+0000 7fa348484980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.545 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.552+0000 7fa348484980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:49.545 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.552+0000 7fa348484980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.545 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.556+0000 7fa348484980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:49.545 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.556+0000 7fa348484980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:49.545 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.556+0000 7fa348484980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-06-28T15:03:49.545 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.556+0000 7fa348484980 20 INFO: main: starting shard=0 2024-06-28T15:03:49.545 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.556+0000 7fa348484980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:49.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.556+0000 7fa348484980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-06-28T15:03:49.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.556+0000 7fa348484980 20 INFO: main: done 2024-06-28T15:03:49.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.556+0000 7fa1f77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:49.551 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.560+0000 7fa348484980 20 remove_watcher() i=0 2024-06-28T15:03:49.551 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.560+0000 7fa348484980 2 removed watcher, disabling cache 2024-06-28T15:03:49.554 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.564+0000 7fa348484980 20 remove_watcher() i=1 2024-06-28T15:03:49.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.564+0000 7fa348484980 20 remove_watcher() i=2 2024-06-28T15:03:49.558 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.568+0000 7fa348484980 20 remove_watcher() i=3 2024-06-28T15:03:49.560 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.568+0000 7fa348484980 20 remove_watcher() i=4 2024-06-28T15:03:49.561 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.572+0000 7fa348484980 20 remove_watcher() i=5 2024-06-28T15:03:49.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.572+0000 7fa348484980 20 remove_watcher() i=6 2024-06-28T15:03:49.564 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.572+0000 7fa348484980 20 remove_watcher() i=7 2024-06-28T15:03:49.706 INFO:tasks.workunit.client.0.smithi178.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-06-28T15:03:49.706 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket check-bucket --object f 2024-06-28T15:03:49.742 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:49.742 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:49.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.792+0000 7f44c7752980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.787 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.796+0000 7f44c7752980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:49.787 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.796+0000 7f44c7752980 20 realm 2024-06-28T15:03:49.787 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.796+0000 7f44c7752980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.787 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.796+0000 7f44c7752980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:49.787 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.796+0000 7f44c7752980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:49.787 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.796+0000 7f44c7752980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.787 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.796+0000 7f44c7752980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:49.788 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.796+0000 7f44c7752980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.789 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.800+0000 7f44c7752980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:49.789 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.800+0000 7f44c7752980 20 rados->read ofs=0 len=0 2024-06-28T15:03:49.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.800+0000 7f44c7752980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:49.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.800+0000 7f44c7752980 20 searching for the correct realm 2024-06-28T15:03:49.797 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:49.804+0000 7f44c5a64640 0 -- 172.21.15.178:0/420708464 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f44bc1360b0 msgr2=0x7f44bc117a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:50.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.020+0000 7f44c5263640 0 -- 172.21.15.178:0/420708464 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x562dd8d4d250 msgr2=0x562dd8d4f640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:50.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:50.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:50.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:50.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:50.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:50.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:50.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 rados->read ofs=0 len=0 2024-06-28T15:03:50.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:50.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 rados->read ofs=0 len=0 2024-06-28T15:03:50.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:50.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 rados->read ofs=0 len=0 2024-06-28T15:03:50.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:50.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 zone default found 2024-06-28T15:03:50.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 4 Realm: () 2024-06-28T15:03:50.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:50.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:50.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:50.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 zonegroup default 2024-06-28T15:03:50.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.028+0000 7f44c7752980 20 rados->read ofs=0 len=0 2024-06-28T15:03:50.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.032+0000 7f44c7752980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:50.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.032+0000 7f44c7752980 20 rados->read ofs=0 len=0 2024-06-28T15:03:50.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.032+0000 7f44c7752980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:50.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.032+0000 7f44c7752980 20 rados->read ofs=0 len=0 2024-06-28T15:03:50.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.032+0000 7f44c7752980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:50.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.032+0000 7f44c7752980 20 started sync module instance, tier type = 2024-06-28T15:03:50.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.032+0000 7f44c7752980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:50.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.052+0000 7f44c7752980 20 add_watcher() i=0 2024-06-28T15:03:50.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.052+0000 7f44c7752980 20 add_watcher() i=1 2024-06-28T15:03:50.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.052+0000 7f44c7752980 20 add_watcher() i=2 2024-06-28T15:03:50.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.052+0000 7f44c7752980 20 add_watcher() i=3 2024-06-28T15:03:50.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.052+0000 7f44c7752980 20 add_watcher() i=4 2024-06-28T15:03:50.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.052+0000 7f44c7752980 20 add_watcher() i=5 2024-06-28T15:03:50.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.052+0000 7f44c7752980 20 add_watcher() i=6 2024-06-28T15:03:50.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.056+0000 7f44c7752980 20 add_watcher() i=7 2024-06-28T15:03:50.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.056+0000 7f44c7752980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:50.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.060+0000 7f437b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:50.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.060+0000 7f44c7752980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:50.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.060+0000 7f44c7752980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:50.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.060+0000 7f44c7752980 5 note: GC not initialized 2024-06-28T15:03:50.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.060+0000 7f437a7e4640 20 reqs_thread_entry: start 2024-06-28T15:03:50.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.060+0000 7f44c7752980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:50.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.060+0000 7f44c7752980 20 Filter name: none 2024-06-28T15:03:50.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.064+0000 7f42ebfff640 20 reqs_thread_entry: start 2024-06-28T15:03:50.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.064+0000 7f44c7752980 20 rados->read ofs=0 len=0 2024-06-28T15:03:50.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.064+0000 7f44c7752980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:50.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.064+0000 7f44c7752980 20 rados->read ofs=0 len=0 2024-06-28T15:03:50.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.064+0000 7f44c7752980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:50.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.064+0000 7f44c7752980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:50.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.064+0000 7f44c7752980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=1 2024-06-28T15:03:50.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.064+0000 7f44c7752980 20 INFO: main: starting shard=0 2024-06-28T15:03:50.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.064+0000 7f44c7752980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:50.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.064+0000 7f44c7752980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-06-28T15:03:50.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.064+0000 7f44c7752980 20 INFO: main: done 2024-06-28T15:03:50.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.068+0000 7f437b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:50.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.072+0000 7f44c7752980 20 remove_watcher() i=0 2024-06-28T15:03:50.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.072+0000 7f44c7752980 2 removed watcher, disabling cache 2024-06-28T15:03:50.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.072+0000 7f44c7752980 20 remove_watcher() i=1 2024-06-28T15:03:50.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.076+0000 7f44c7752980 20 remove_watcher() i=2 2024-06-28T15:03:50.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.076+0000 7f44c7752980 20 remove_watcher() i=3 2024-06-28T15:03:50.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.080+0000 7f44c7752980 20 remove_watcher() i=4 2024-06-28T15:03:50.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.080+0000 7f44c7752980 20 remove_watcher() i=5 2024-06-28T15:03:50.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.084+0000 7f44c7752980 20 remove_watcher() i=6 2024-06-28T15:03:50.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:50.084+0000 7f44c7752980 20 remove_watcher() i=7 2024-06-28T15:03:50.122 INFO:tasks.workunit.client.0.smithi178.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-06-28T15:03:50.561 INFO:tasks.workunit.client.0.smithi178.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-06-28T15:03:50.987 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2024-06-28T15:03:51.019 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:51.019 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:51.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.068+0000 7f09574a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.072+0000 7f09574a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:51.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.072+0000 7f09574a0980 20 realm 2024-06-28T15:03:51.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.072+0000 7f09574a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.072+0000 7f09574a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:51.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.072+0000 7f09574a0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:51.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.072+0000 7f09574a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.072+0000 7f09574a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:51.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.072+0000 7f09574a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.072+0000 7f09574a0980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:51.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.072+0000 7f09574a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.076+0000 7f09574a0980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:51.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.076+0000 7f09574a0980 20 searching for the correct realm 2024-06-28T15:03:51.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.080+0000 7f09547b0640 0 -- 172.21.15.178:0/3373732477 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f094401a040 msgr2=0x7f0944019410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:51.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.088+0000 7f09557b2640 0 -- 172.21.15.178:0/3373732477 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f0948120670 msgr2=0x7f09480f7a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:51.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.292+0000 7f09574a0980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:51.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.292+0000 7f09574a0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:51.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.292+0000 7f09574a0980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:51.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.292+0000 7f09574a0980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:51.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.292+0000 7f09574a0980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:51.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.292+0000 7f09574a0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:51.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.292+0000 7f09574a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.292+0000 7f09574a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:51.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.292+0000 7f09574a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.292+0000 7f09574a0980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:51.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.292+0000 7f09574a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:51.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 20 zone default found 2024-06-28T15:03:51.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 4 Realm: () 2024-06-28T15:03:51.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:51.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:51.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:51.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 20 zonegroup default 2024-06-28T15:03:51.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:51.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:51.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:51.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 20 started sync module instance, tier type = 2024-06-28T15:03:51.289 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.296+0000 7f09574a0980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:51.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.320+0000 7f09574a0980 20 add_watcher() i=0 2024-06-28T15:03:51.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.320+0000 7f09574a0980 20 add_watcher() i=1 2024-06-28T15:03:51.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.320+0000 7f09574a0980 20 add_watcher() i=2 2024-06-28T15:03:51.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.320+0000 7f09574a0980 20 add_watcher() i=3 2024-06-28T15:03:51.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.320+0000 7f09574a0980 20 add_watcher() i=4 2024-06-28T15:03:51.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.320+0000 7f09574a0980 20 add_watcher() i=5 2024-06-28T15:03:51.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.320+0000 7f09574a0980 20 add_watcher() i=6 2024-06-28T15:03:51.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.324+0000 7f09574a0980 20 add_watcher() i=7 2024-06-28T15:03:51.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.324+0000 7f09574a0980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:51.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.328+0000 7f080b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:51.320 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.328+0000 7f09574a0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:51.320 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.328+0000 7f09574a0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:51.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.328+0000 7f09574a0980 5 note: GC not initialized 2024-06-28T15:03:51.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f080a7e4640 20 reqs_thread_entry: start 2024-06-28T15:03:51.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:51.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 20 Filter name: none 2024-06-28T15:03:51.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f08037fe640 20 reqs_thread_entry: start 2024-06-28T15:03:51.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-06-28T15:03:51.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:51.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-06-28T15:03:51.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-06-28T15:03:51.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 : miss 2024-06-28T15:03:51.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:51.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 info.flags=0x17 2024-06-28T15:03:51.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 to cache LRU end 2024-06-28T15:03:51.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:03:51.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-06-28T15:03:51.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 2024-06-28T15:03:51.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.332+0000 7f09574a0980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.328 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.336+0000 7f080b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:51.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.344+0000 7f09574a0980 20 remove_watcher() i=0 2024-06-28T15:03:51.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.344+0000 7f09574a0980 2 removed watcher, disabling cache 2024-06-28T15:03:51.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.344+0000 7f09574a0980 20 remove_watcher() i=1 2024-06-28T15:03:51.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.348+0000 7f09574a0980 20 remove_watcher() i=2 2024-06-28T15:03:51.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.348+0000 7f09574a0980 20 remove_watcher() i=3 2024-06-28T15:03:51.342 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.352+0000 7f09574a0980 20 remove_watcher() i=4 2024-06-28T15:03:51.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.352+0000 7f09574a0980 20 remove_watcher() i=5 2024-06-28T15:03:51.345 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.356+0000 7f09574a0980 20 remove_watcher() i=6 2024-06-28T15:03:51.346 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.356+0000 7f09574a0980 20 remove_watcher() i=7 2024-06-28T15:03:51.362 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-06-28T15:03:51.396 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:51.396 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:51.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.452+0000 7f3758370980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.444 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.452+0000 7f3758370980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:51.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.452+0000 7f3758370980 20 realm 2024-06-28T15:03:51.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.452+0000 7f3758370980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.452+0000 7f3758370980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:51.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.452+0000 7f3758370980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:51.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.452+0000 7f3758370980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.452+0000 7f3758370980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:51.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.452+0000 7f3758370980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.456+0000 7f3758370980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:51.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.456+0000 7f3758370980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.456+0000 7f3758370980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:51.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.456+0000 7f3758370980 20 searching for the correct realm 2024-06-28T15:03:51.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.464+0000 7f3756682640 0 -- 172.21.15.178:0/3547573419 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f374c045440 msgr2=0x7f374c0458a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:51.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.472+0000 7f3755e81640 0 -- 172.21.15.178:0/3547573419 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x5607c4c52ec0 msgr2=0x5607c4c552d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:03:51.468 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.476+0000 7f3758370980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:51.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.476+0000 7f3758370980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:51.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.476+0000 7f3758370980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:51.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.476+0000 7f3758370980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:51.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.476+0000 7f3758370980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:51.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.476+0000 7f3758370980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:51.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.476+0000 7f3758370980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.476+0000 7f3758370980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:51.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.476+0000 7f3758370980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:51.470 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.470 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:51.470 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 20 zone default found 2024-06-28T15:03:51.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 4 Realm: () 2024-06-28T15:03:51.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:51.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:51.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:51.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 20 zonegroup default 2024-06-28T15:03:51.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:51.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:51.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:51.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 20 started sync module instance, tier type = 2024-06-28T15:03:51.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.480+0000 7f3758370980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:51.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.496+0000 7f3758370980 20 add_watcher() i=0 2024-06-28T15:03:51.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.496+0000 7f3758370980 20 add_watcher() i=1 2024-06-28T15:03:51.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.496+0000 7f3758370980 20 add_watcher() i=2 2024-06-28T15:03:51.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.496+0000 7f3758370980 20 add_watcher() i=3 2024-06-28T15:03:51.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.496+0000 7f3758370980 20 add_watcher() i=4 2024-06-28T15:03:51.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.496+0000 7f3758370980 20 add_watcher() i=5 2024-06-28T15:03:51.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.496+0000 7f3758370980 20 add_watcher() i=6 2024-06-28T15:03:51.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.500+0000 7f3758370980 20 add_watcher() i=7 2024-06-28T15:03:51.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.500+0000 7f3758370980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:51.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.500+0000 7f360bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:51.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.500+0000 7f3758370980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:51.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.500+0000 7f3758370980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:51.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.500+0000 7f3758370980 5 note: GC not initialized 2024-06-28T15:03:51.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.500+0000 7f360afe5640 20 reqs_thread_entry: start 2024-06-28T15:03:51.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.500+0000 7f3758370980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:51.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.500+0000 7f3758370980 20 Filter name: none 2024-06-28T15:03:51.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.500+0000 7f3608fe1640 20 reqs_thread_entry: start 2024-06-28T15:03:51.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.500+0000 7f3758370980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-06-28T15:03:51.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.500+0000 7f3758370980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:51.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-06-28T15:03:51.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-06-28T15:03:51.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 : miss 2024-06-28T15:03:51.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 20 rados->read ofs=0 len=0 2024-06-28T15:03:51.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:51.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 info.flags=0x17 2024-06-28T15:03:51.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 to cache LRU end 2024-06-28T15:03:51.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:03:51.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-06-28T15:03:51.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.2 2024-06-28T15:03:51.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 20 get_obj_state: octx=0x5607c49e4770 obj=check-bucket:c state=0x5607c4c868f8 s->prefetch_data=0 2024-06-28T15:03:51.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 20 get_obj_state: setting s->obj_tag to nbsbo8rqdwlpupb4gz3vrknhp3pcn4pg 2024-06-28T15:03:51.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 20 get_obj_state_impl: setting s->olh_tag to cbgl15ehcdnk3h3moy9bvshd1frk82uc 2024-06-28T15:03:51.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.504+0000 7f3758370980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=cbgl15ehcdnk3h3moy9bvshd1frk82uc 2024-06-28T15:03:51.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.508+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.508+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.508+0000 7f3758370980 20 olh_log_entry: epoch=3 op=3 key=c[pa6l65Qcrc8ZOJp3zMlUxpRXWnJnPlc] 2024-06-28T15:03:51.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.508+0000 7f3758370980 20 get_obj_state: octx=0x5607c49e4770 obj=check-bucket:_:pa6l65Qcrc8ZOJp3zMlUxpRXWnJnPlc_c state=0x5607c4c8ea08 s->prefetch_data=0 2024-06-28T15:03:51.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.508+0000 7f3758370980 10 manifest: total_size = 11 2024-06-28T15:03:51.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.508+0000 7f3758370980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.12950746834536515258 2024-06-28T15:03:51.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.508+0000 7f3758370980 20 get_obj_state: octx=0x5607c49e4770 obj=check-bucket:_:pa6l65Qcrc8ZOJp3zMlUxpRXWnJnPlc_c state=0x5607c4c8ea08 s->prefetch_data=0 2024-06-28T15:03:51.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.508+0000 7f3758370980 20 prepare_atomic_modification: state is not atomic. state=0x5607c4c8ea08 2024-06-28T15:03:51.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.508+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.504 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.512+0000 7f3758370980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:03:51.504 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.512+0000 7f3758370980 20 RGWObjManifest::operator++(): result: ofs=11 stripe_ofs=11 part_ofs=0 rule->part_size=0 2024-06-28T15:03:51.504 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.512+0000 7f3758370980 0 deleting objects inline since gc isn't initialized 2024-06-28T15:03:51.504 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.512+0000 7f3758370980 5 delete_objs_inline: removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.2__shadow:pa6l65Qcrc8ZOJp3zMlUxpRXWnJnPlc_.WiZCyxe3k0tMuSkklwWN-M_jYpTc180_0 2024-06-28T15:03:51.506 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.516+0000 7f36397fa640 20 handle_completion(): completion ok for obj=c[pa6l65Qcrc8ZOJp3zMlUxpRXWnJnPlc] 2024-06-28T15:03:51.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.516+0000 7f3758370980 5 delete_objs_inline: refcount put returned error -2 2024-06-28T15:03:51.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.520+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.520+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.511 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.520+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.512 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.520+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.512 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.520+0000 7f3758370980 20 get_obj_state: octx=0x5607c49e4770 obj=check-bucket:d state=0x5607c4c8c9e8 s->prefetch_data=0 2024-06-28T15:03:51.513 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.524+0000 7f3758370980 20 get_obj_state: setting s->obj_tag to j68887frajnj6ypd1tzwz1z4885yyk4m 2024-06-28T15:03:51.513 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.524+0000 7f3758370980 20 get_obj_state_impl: setting s->olh_tag to o850o5mobw8j74qhk21fualbvc41i94l 2024-06-28T15:03:51.513 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.524+0000 7f3758370980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=o850o5mobw8j74qhk21fualbvc41i94l 2024-06-28T15:03:51.515 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.524+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.518 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.528+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.518 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.528+0000 7f3758370980 20 olh_log_entry: epoch=3 op=3 key=d[9vBqLM5XfSXvyz.n-x5rPahXRzXDMLp] 2024-06-28T15:03:51.518 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.528+0000 7f3758370980 20 get_obj_state: octx=0x5607c49e4770 obj=check-bucket:_:9vBqLM5XfSXvyz.n-x5rPahXRzXDMLp_d state=0x5607c4c888e8 s->prefetch_data=0 2024-06-28T15:03:51.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.528+0000 7f3758370980 10 manifest: total_size = 22 2024-06-28T15:03:51.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.528+0000 7f3758370980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.3215000691016025087 2024-06-28T15:03:51.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.528+0000 7f3758370980 20 get_obj_state: octx=0x5607c49e4770 obj=check-bucket:_:9vBqLM5XfSXvyz.n-x5rPahXRzXDMLp_d state=0x5607c4c888e8 s->prefetch_data=0 2024-06-28T15:03:51.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.528+0000 7f3758370980 20 prepare_atomic_modification: state is not atomic. state=0x5607c4c888e8 2024-06-28T15:03:51.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.528+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.532+0000 7f3758370980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:03:51.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.532+0000 7f3758370980 20 RGWObjManifest::operator++(): result: ofs=22 stripe_ofs=22 part_ofs=0 rule->part_size=0 2024-06-28T15:03:51.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.532+0000 7f3758370980 0 deleting objects inline since gc isn't initialized 2024-06-28T15:03:51.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.532+0000 7f3758370980 5 delete_objs_inline: removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.2__shadow:9vBqLM5XfSXvyz.n-x5rPahXRzXDMLp_.CQCg0nZalmvzh-1ikFOujOITFFAJ9UQ_0 2024-06-28T15:03:51.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.536+0000 7f3638ff9640 20 handle_completion(): completion ok for obj=d[9vBqLM5XfSXvyz.n-x5rPahXRzXDMLp] 2024-06-28T15:03:51.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.536+0000 7f3758370980 5 delete_objs_inline: refcount put returned error -2 2024-06-28T15:03:51.529 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.540+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.529 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.540+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.530 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.540+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.531 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.540+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.531 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.540+0000 7f3758370980 20 get_obj_state: octx=0x5607c49e4770 obj=check-bucket:e state=0x5607c4cbce68 s->prefetch_data=0 2024-06-28T15:03:51.532 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.540+0000 7f3758370980 20 get_obj_state: setting s->obj_tag to vzpwq2fetccp1tg60z6x53hhl2xww5d8 2024-06-28T15:03:51.532 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.540+0000 7f3758370980 20 get_obj_state_impl: setting s->olh_tag to k3mw5z7hvhzyxzvkxi9xd3444ji7evdx 2024-06-28T15:03:51.532 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.540+0000 7f3758370980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=k3mw5z7hvhzyxzvkxi9xd3444ji7evdx 2024-06-28T15:03:51.535 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.544+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.537 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.548+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.537 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.548+0000 7f3758370980 20 olh_log_entry: epoch=2 op=2 key=e[] 2024-06-28T15:03:51.537 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.548+0000 7f3758370980 20 olh_log_entry: epoch=2 op=3 key=e[nWPPxNbTlHzrUVq7-4cXqv63oIFAiPr] 2024-06-28T15:03:51.537 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.548+0000 7f3758370980 20 get_obj_state: octx=0x5607c49e4770 obj=check-bucket:_:nWPPxNbTlHzrUVq7-4cXqv63oIFAiPr_e state=0x5607c4c8ea08 s->prefetch_data=0 2024-06-28T15:03:51.538 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.548+0000 7f3758370980 10 manifest: total_size = 11 2024-06-28T15:03:51.538 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.548+0000 7f3758370980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.5953808296428352221 2024-06-28T15:03:51.538 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.548+0000 7f3758370980 20 get_obj_state: octx=0x5607c49e4770 obj=check-bucket:_:nWPPxNbTlHzrUVq7-4cXqv63oIFAiPr_e state=0x5607c4c8ea08 s->prefetch_data=0 2024-06-28T15:03:51.538 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.548+0000 7f3758370980 20 prepare_atomic_modification: state is not atomic. state=0x5607c4c8ea08 2024-06-28T15:03:51.538 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.548+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.552+0000 7f3758370980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:03:51.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.552+0000 7f3758370980 20 RGWObjManifest::operator++(): result: ofs=11 stripe_ofs=11 part_ofs=0 rule->part_size=0 2024-06-28T15:03:51.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.552+0000 7f3758370980 0 deleting objects inline since gc isn't initialized 2024-06-28T15:03:51.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.552+0000 7f3758370980 5 delete_objs_inline: removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.2__shadow:nWPPxNbTlHzrUVq7-4cXqv63oIFAiPr_.1GHqZJdVGuYwxHSsWWWzRzTdfEZMDGK_0 2024-06-28T15:03:51.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.552+0000 7f36397fa640 20 handle_completion(): completion ok for obj=e[nWPPxNbTlHzrUVq7-4cXqv63oIFAiPr] 2024-06-28T15:03:51.545 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.556+0000 7f3758370980 5 delete_objs_inline: refcount put returned error -2 2024-06-28T15:03:51.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.556+0000 7f3758370980 20 get_obj_state: octx=0x5607c49e4770 obj=check-bucket:e state=0x5607c4cbce68 s->prefetch_data=0 2024-06-28T15:03:51.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.556+0000 7f3758370980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=k3mw5z7hvhzyxzvkxi9xd3444ji7evdx 2024-06-28T15:03:51.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.564+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.557 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.568+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.558 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.568+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.558 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.568+0000 7f3758370980 20 get_obj_state: octx=0x5607c49e4770 obj=check-bucket:f state=0x5607c4cbce68 s->prefetch_data=0 2024-06-28T15:03:51.558 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.568+0000 7f3758370980 20 get_obj_state: setting s->obj_tag to sn787tqfvqbze4wp3bu623edcjcjphl8 2024-06-28T15:03:51.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.568+0000 7f3758370980 20 get_obj_state_impl: setting s->olh_tag to 0zj0br0gnfrgel8k129yp3w07sa3s09h 2024-06-28T15:03:51.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.568+0000 7f3758370980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=0zj0br0gnfrgel8k129yp3w07sa3s09h 2024-06-28T15:03:51.561 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.572+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.562 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.572+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.572+0000 7f3758370980 20 olh_log_entry: epoch=2 op=2 key=f[] 2024-06-28T15:03:51.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.572+0000 7f3758370980 20 olh_log_entry: epoch=2 op=3 key=f[cifvvCr6ki979NkG.EM8R8-Fzi6qKxu] 2024-06-28T15:03:51.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.572+0000 7f3758370980 20 get_obj_state: octx=0x5607c49e4770 obj=check-bucket:_:cifvvCr6ki979NkG.EM8R8-Fzi6qKxu_f state=0x5607c4c8ea08 s->prefetch_data=0 2024-06-28T15:03:51.564 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.572+0000 7f3758370980 10 manifest: total_size = 11 2024-06-28T15:03:51.564 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.572+0000 7f3758370980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.18149167714784962183 2024-06-28T15:03:51.564 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.572+0000 7f3758370980 20 get_obj_state: octx=0x5607c49e4770 obj=check-bucket:_:cifvvCr6ki979NkG.EM8R8-Fzi6qKxu_f state=0x5607c4c8ea08 s->prefetch_data=0 2024-06-28T15:03:51.564 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.572+0000 7f3758370980 20 prepare_atomic_modification: state is not atomic. state=0x5607c4c8ea08 2024-06-28T15:03:51.564 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.572+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.576+0000 7f3758370980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:03:51.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.576+0000 7f3758370980 20 RGWObjManifest::operator++(): result: ofs=11 stripe_ofs=11 part_ofs=0 rule->part_size=0 2024-06-28T15:03:51.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.576+0000 7f3758370980 0 deleting objects inline since gc isn't initialized 2024-06-28T15:03:51.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.576+0000 7f3758370980 5 delete_objs_inline: removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.2__shadow:cifvvCr6ki979NkG.EM8R8-Fzi6qKxu_.faBn2lpnpgQqiCN3ZyqZXrsd0Fjag9w_0 2024-06-28T15:03:51.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.580+0000 7f3638ff9640 20 handle_completion(): completion ok for obj=f[cifvvCr6ki979NkG.EM8R8-Fzi6qKxu] 2024-06-28T15:03:51.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.580+0000 7f3758370980 5 delete_objs_inline: refcount put returned error -2 2024-06-28T15:03:51.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.584+0000 7f3758370980 20 get_obj_state: octx=0x5607c49e4770 obj=check-bucket:f state=0x5607c4cbce68 s->prefetch_data=0 2024-06-28T15:03:51.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.584+0000 7f3758370980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=0zj0br0gnfrgel8k129yp3w07sa3s09h 2024-06-28T15:03:51.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.588+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.592+0000 7f3758370980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.2.0 2024-06-28T15:03:51.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.592+0000 7f3758370980 1 NOTICE: finished shard 0 (4 entries removed) 2024-06-28T15:03:51.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.592+0000 7f3758370980 1 NOTICE: finished all shards (4 entries removed) 2024-06-28T15:03:51.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.592+0000 7f360bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:51.587 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.596+0000 7f3758370980 20 remove_watcher() i=0 2024-06-28T15:03:51.587 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.596+0000 7f3758370980 2 removed watcher, disabling cache 2024-06-28T15:03:51.589 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.600+0000 7f3758370980 20 remove_watcher() i=1 2024-06-28T15:03:51.591 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.600+0000 7f3758370980 20 remove_watcher() i=2 2024-06-28T15:03:51.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.604+0000 7f3758370980 20 remove_watcher() i=3 2024-06-28T15:03:51.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.604+0000 7f3758370980 20 remove_watcher() i=4 2024-06-28T15:03:51.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.608+0000 7f3758370980 20 remove_watcher() i=5 2024-06-28T15:03:51.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.608+0000 7f3758370980 20 remove_watcher() i=6 2024-06-28T15:03:51.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:51.608+0000 7f3758370980 20 remove_watcher() i=7 2024-06-28T15:03:52.142 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket check-bucket 2024-06-28T15:03:52.174 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:03:52.175 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:03:52.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.228+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.232+0000 7f1f63aef980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:52.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.232+0000 7f1f63aef980 20 realm 2024-06-28T15:03:52.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.232+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.232+0000 7f1f63aef980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:52.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.232+0000 7f1f63aef980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:03:52.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.232+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.232+0000 7f1f63aef980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:52.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.232+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.232+0000 7f1f63aef980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:52.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.232+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.232+0000 7f1f63aef980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:03:52.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.232+0000 7f1f63aef980 20 searching for the correct realm 2024-06-28T15:03:52.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.240+0000 7f1f60dff640 0 -- 172.21.15.178:0/217723733 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x55a483509990 msgr2=0x55a48350bd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:03:52.236 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.244+0000 7f1f61600640 0 -- 172.21.15.178:0/217723733 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f1f580e0910 msgr2=0x7f1f580c83b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:03:52.244 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.252+0000 7f1f63aef980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:03:52.244 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.252+0000 7f1f63aef980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:03:52.245 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.252+0000 7f1f63aef980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:03:52.245 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.252+0000 7f1f63aef980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:03:52.245 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.252+0000 7f1f63aef980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:03:52.245 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.252+0000 7f1f63aef980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:03:52.245 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.252+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.252+0000 7f1f63aef980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:52.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.252+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.248 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:03:52.248 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.248 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:03:52.248 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 20 zone default found 2024-06-28T15:03:52.248 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 4 Realm: () 2024-06-28T15:03:52.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:03:52.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:03:52.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:03:52.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 20 zonegroup default 2024-06-28T15:03:52.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:52.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:52.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:03:52.251 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 20 started sync module instance, tier type = 2024-06-28T15:03:52.251 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.256+0000 7f1f63aef980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:03:52.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.272+0000 7f1f63aef980 20 add_watcher() i=0 2024-06-28T15:03:52.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.272+0000 7f1f63aef980 20 add_watcher() i=1 2024-06-28T15:03:52.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.272+0000 7f1f63aef980 20 add_watcher() i=2 2024-06-28T15:03:52.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.272+0000 7f1f63aef980 20 add_watcher() i=3 2024-06-28T15:03:52.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.272+0000 7f1f63aef980 20 add_watcher() i=4 2024-06-28T15:03:52.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.272+0000 7f1f63aef980 20 add_watcher() i=5 2024-06-28T15:03:52.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.272+0000 7f1f63aef980 20 add_watcher() i=6 2024-06-28T15:03:52.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.272+0000 7f1f63aef980 20 add_watcher() i=7 2024-06-28T15:03:52.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.272+0000 7f1f63aef980 2 all 8 watchers are set, enabling cache 2024-06-28T15:03:52.270 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.276+0000 7f1e177e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:52.270 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.276+0000 7f1f63aef980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:03:52.271 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.276+0000 7f1f63aef980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:03:52.271 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.276+0000 7f1f63aef980 5 note: GC not initialized 2024-06-28T15:03:52.271 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.276+0000 7f1e167e4640 20 reqs_thread_entry: start 2024-06-28T15:03:52.271 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.276+0000 7f1f63aef980 20 init_complete bucket index max shards: 1 2024-06-28T15:03:52.271 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.276+0000 7f1f63aef980 20 Filter name: none 2024-06-28T15:03:52.271 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.276+0000 7f1d87fff640 20 reqs_thread_entry: start 2024-06-28T15:03:52.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.276+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.276+0000 7f1f63aef980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:52.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.276+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.280+0000 7f1f63aef980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:52.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.280+0000 7f1f63aef980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:52.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.280+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.280+0000 7f1f63aef980 20 rados_obj.operate() r=0 bl.length=190 2024-06-28T15:03:52.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.280+0000 7f1f63aef980 20 rados->read ofs=0 len=0 2024-06-28T15:03:52.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.280+0000 7f1f63aef980 20 rados_obj.operate() r=0 bl.length=404 2024-06-28T15:03:52.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.280+0000 7f1f63aef980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:03:52.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.280+0000 7f1e177e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:03:52.276 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.284+0000 7f1f63aef980 20 remove_watcher() i=0 2024-06-28T15:03:52.276 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.284+0000 7f1f63aef980 2 removed watcher, disabling cache 2024-06-28T15:03:52.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.288+0000 7f1f63aef980 20 remove_watcher() i=1 2024-06-28T15:03:52.280 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.288+0000 7f1f63aef980 20 remove_watcher() i=2 2024-06-28T15:03:52.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.292+0000 7f1f63aef980 20 remove_watcher() i=3 2024-06-28T15:03:52.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.296+0000 7f1f63aef980 20 remove_watcher() i=4 2024-06-28T15:03:52.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.296+0000 7f1f63aef980 20 remove_watcher() i=5 2024-06-28T15:03:52.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.296+0000 7f1f63aef980 20 remove_watcher() i=6 2024-06-28T15:03:52.289 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:03:52.296+0000 7f1f63aef980 20 remove_watcher() i=7 2024-06-28T15:03:52.302 INFO:tasks.workunit.client.0.smithi178.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2024-06-28T15:03:52.302 INFO:tasks.workunit.client.0.smithi178.stderr:Deleting bucket check-bucket 2024-06-28T15:03:52.666 INFO:tasks.workunit.client.0.smithi178.stderr:Completed bucket check tests 2024-06-28T15:03:52.667 INFO:tasks.workunit.client.0.smithi178.stdout:connected to http://localhost:80 2024-06-28T15:03:52.763 INFO:tasks.workunit.client.0.smithi178.stderr:+ deactivate 2024-06-28T15:03:52.763 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-06-28T15:03:52.763 INFO:tasks.workunit.client.0.smithi178.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-06-28T15:03:52.763 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-06-28T15:03:52.763 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset _OLD_VIRTUAL_PATH 2024-06-28T15:03:52.764 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -n '' ']' 2024-06-28T15:03:52.764 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-06-28T15:03:52.764 INFO:tasks.workunit.client.0.smithi178.stderr:+ hash -r 2024-06-28T15:03:52.764 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -n '' ']' 2024-06-28T15:03:52.764 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset VIRTUAL_ENV 2024-06-28T15:03:52.764 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-06-28T15:03:52.764 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' '!' '' = nondestructive ']' 2024-06-28T15:03:52.764 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset -f deactivate 2024-06-28T15:03:52.764 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo OK. 2024-06-28T15:03:52.765 INFO:tasks.workunit.client.0.smithi178.stdout:OK. 2024-06-28T15:03:52.765 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:03:52.766 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:03:52.824 INFO:tasks.workunit:Stopping ['rgw/run-bucket-check.sh'] on client.0... 2024-06-28T15:03:52.825 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-28T15:03:53.244 DEBUG:teuthology.parallel:result is None 2024-06-28T15:03:53.244 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:03:53.258 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:03:53.259 DEBUG:teuthology.orchestra.run.smithi178:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:03:53.305 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:03:53.305 INFO:teuthology.run_tasks:Running task workunit... 2024-06-28T15:03:53.317 INFO:tasks.workunit:Pulling workunits from ref ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T15:03:53.317 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-28T15:03:53.317 DEBUG:teuthology.orchestra.run.smithi178:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:03:53.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T15:03:53.354 INFO:teuthology.orchestra.run.smithi178.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-06-28T15:03:53.354 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:03:53.401 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-06-28T15:03:53.402 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-06-28T15:03:53.450 INFO:tasks.workunit:timeout=3h 2024-06-28T15:03:53.451 INFO:tasks.workunit:cleanup=True 2024-06-28T15:03:53.451 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T15:03:53.503 INFO:tasks.workunit.client.0.smithi178.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-28T15:04:45.973 INFO:tasks.workunit.client.0.smithi178.stderr:Note: switching to 'ec2a38ba1b69b5357de8f0e8f96b00b4a577169d'. 2024-06-28T15:04:45.974 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:04:45.974 INFO:tasks.workunit.client.0.smithi178.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-28T15:04:45.974 INFO:tasks.workunit.client.0.smithi178.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-28T15:04:45.974 INFO:tasks.workunit.client.0.smithi178.stderr:state without impacting any branches by switching back to a branch. 2024-06-28T15:04:45.974 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:04:45.974 INFO:tasks.workunit.client.0.smithi178.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-28T15:04:45.974 INFO:tasks.workunit.client.0.smithi178.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-28T15:04:45.974 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:04:45.974 INFO:tasks.workunit.client.0.smithi178.stderr: git switch -c 2024-06-28T15:04:45.974 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:04:45.975 INFO:tasks.workunit.client.0.smithi178.stderr:Or undo this operation with: 2024-06-28T15:04:45.975 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:04:45.975 INFO:tasks.workunit.client.0.smithi178.stderr: git switch - 2024-06-28T15:04:45.975 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:04:45.975 INFO:tasks.workunit.client.0.smithi178.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-28T15:04:45.975 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:04:45.975 INFO:tasks.workunit.client.0.smithi178.stderr:HEAD is now at ec2a38ba1b6 Merge branches 'pr-56792', 'pr-57537', 'pr-57858', 'pr-57985', 'pr-58081' and 'pr-58288' into wip-cbodley-testing 2024-06-28T15:04:45.981 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-28T15:04:46.030 INFO:tasks.workunit.client.0.smithi178.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-28T15:04:46.032 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-28T15:04:46.033 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-28T15:04:46.086 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-28T15:04:46.136 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-28T15:04:46.180 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-28T15:04:46.181 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-28T15:04:46.181 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-28T15:04:46.225 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-28T15:04:46.228 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T15:04:46.228 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-28T15:04:46.276 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2024-06-28T15:04:46.278 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2024-06-28T15:04:46.278 DEBUG:teuthology.orchestra.run.smithi178:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2024-06-28T15:04:46.343 INFO:tasks.workunit.client.0.smithi178.stdout:Running main() from gmock_main.cc 2024-06-28T15:04:46.343 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 11 tests from 1 test suite. 2024-06-28T15:04:46.344 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:04:46.344 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 11 tests from ClsLock 2024-06-28T15:04:46.344 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsLock.TestMultiLocking 2024-06-28T15:04:47.993 INFO:tasks.workunit.client.0.smithi178.stdout:lock: mylock 2024-06-28T15:04:47.993 INFO:tasks.workunit.client.0.smithi178.stdout: lock_type: exclusive 2024-06-28T15:04:47.993 INFO:tasks.workunit.client.0.smithi178.stdout: tag: 2024-06-28T15:04:47.993 INFO:tasks.workunit.client.0.smithi178.stdout: lockers: 2024-06-28T15:04:47.994 INFO:tasks.workunit.client.0.smithi178.stdout: client.5039 expiration=2024-06-28T15:06:48.000413+0000 addr=v1:172.21.15.178:0/1692972790 cookie= 2024-06-28T15:04:48.000 INFO:tasks.workunit.client.0.smithi178.stdout:lock: mylock 2024-06-28T15:04:48.000 INFO:tasks.workunit.client.0.smithi178.stdout: lock_type: shared 2024-06-28T15:04:48.000 INFO:tasks.workunit.client.0.smithi178.stdout: tag: 2024-06-28T15:04:48.001 INFO:tasks.workunit.client.0.smithi178.stdout: lockers: 2024-06-28T15:04:48.001 INFO:tasks.workunit.client.0.smithi178.stdout: client.5039 expiration=2024-06-28T15:06:48.011928+0000 addr=v1:172.21.15.178:0/1692972790 cookie= 2024-06-28T15:04:48.001 INFO:tasks.workunit.client.0.smithi178.stdout: client.5045 expiration=0.000000 addr=v1:172.21.15.178:0/355292423 cookie= 2024-06-28T15:04:48.002 INFO:tasks.workunit.client.0.smithi178.stdout:lock: mylock 2024-06-28T15:04:48.003 INFO:tasks.workunit.client.0.smithi178.stdout: lock_type: shared 2024-06-28T15:04:48.003 INFO:tasks.workunit.client.0.smithi178.stdout: tag: 2024-06-28T15:04:48.003 INFO:tasks.workunit.client.0.smithi178.stdout: lockers: 2024-06-28T15:04:48.003 INFO:tasks.workunit.client.0.smithi178.stdout: client.5045 expiration=0.000000 addr=v1:172.21.15.178:0/355292423 cookie= 2024-06-28T15:04:48.954 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsLock.TestMultiLocking (2611 ms) 2024-06-28T15:04:48.955 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsLock.TestMeta 2024-06-28T15:04:51.016 INFO:tasks.workunit.client.0.smithi178.stdout:lock: mylock 2024-06-28T15:04:51.017 INFO:tasks.workunit.client.0.smithi178.stdout: lock_type: shared 2024-06-28T15:04:51.017 INFO:tasks.workunit.client.0.smithi178.stdout: tag: 2024-06-28T15:04:51.017 INFO:tasks.workunit.client.0.smithi178.stdout: lockers: 2024-06-28T15:04:51.017 INFO:tasks.workunit.client.0.smithi178.stdout: client.5250 expiration=0.000000 addr=v1:172.21.15.178:0/1302183818 cookie= 2024-06-28T15:04:51.020 INFO:tasks.workunit.client.0.smithi178.stdout:lock: mylock 2024-06-28T15:04:51.020 INFO:tasks.workunit.client.0.smithi178.stdout: lock_type: exclusive 2024-06-28T15:04:51.020 INFO:tasks.workunit.client.0.smithi178.stdout: tag: new_tag 2024-06-28T15:04:51.020 INFO:tasks.workunit.client.0.smithi178.stdout: lockers: 2024-06-28T15:04:51.020 INFO:tasks.workunit.client.0.smithi178.stdout: client.5244 expiration=0.000000 addr=v1:172.21.15.178:0/1885454171 cookie= 2024-06-28T15:04:51.976 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsLock.TestMeta (3022 ms) 2024-06-28T15:04:51.977 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsLock.TestCookie 2024-06-28T15:04:54.056 INFO:tasks.workunit.client.0.smithi178.stdout:lock: mylock 2024-06-28T15:04:54.056 INFO:tasks.workunit.client.0.smithi178.stdout: lock_type: exclusive 2024-06-28T15:04:54.057 INFO:tasks.workunit.client.0.smithi178.stdout: tag: 2024-06-28T15:04:54.057 INFO:tasks.workunit.client.0.smithi178.stdout: lockers: 2024-06-28T15:04:54.061 INFO:tasks.workunit.client.0.smithi178.stdout:lock: mylock 2024-06-28T15:04:54.061 INFO:tasks.workunit.client.0.smithi178.stdout: lock_type: shared 2024-06-28T15:04:54.061 INFO:tasks.workunit.client.0.smithi178.stdout: tag: 2024-06-28T15:04:54.061 INFO:tasks.workunit.client.0.smithi178.stdout: lockers: 2024-06-28T15:04:54.061 INFO:tasks.workunit.client.0.smithi178.stdout: client.5060 expiration=0.000000 addr=v1:172.21.15.178:0/1986703650 cookie= 2024-06-28T15:04:54.061 INFO:tasks.workunit.client.0.smithi178.stdout: client.5060 expiration=0.000000 addr=v1:172.21.15.178:0/1986703650 cookie=new cookie 2024-06-28T15:04:55.052 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsLock.TestCookie (3070 ms) 2024-06-28T15:04:55.052 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsLock.TestMultipleLocks 2024-06-28T15:04:58.082 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsLock.TestMultipleLocks (3036 ms) 2024-06-28T15:04:58.082 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsLock.TestLockDuration 2024-06-28T15:05:05.862 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsLock.TestLockDuration (7778 ms) 2024-06-28T15:05:05.862 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsLock.TestAssertLocked 2024-06-28T15:05:08.889 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsLock.TestAssertLocked (3029 ms) 2024-06-28T15:05:08.889 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsLock.TestSetCookie 2024-06-28T15:05:11.932 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsLock.TestSetCookie (3041 ms) 2024-06-28T15:05:11.932 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsLock.TestRenew 2024-06-28T15:05:32.904 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsLock.TestRenew (20974 ms) 2024-06-28T15:05:32.905 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2024-06-28T15:05:39.879 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (6976 ms) 2024-06-28T15:05:39.880 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2024-06-28T15:05:47.940 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (8060 ms) 2024-06-28T15:05:47.940 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2024-06-28T15:05:55.881 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (7941 ms) 2024-06-28T15:05:55.881 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 11 tests from ClsLock (69539 ms total) 2024-06-28T15:05:55.881 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:05:55.882 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:05:55.882 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 11 tests from 1 test suite ran. (69539 ms total) 2024-06-28T15:05:55.882 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 11 tests. 2024-06-28T15:05:55.887 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:05:55.887 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:05:55.944 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2024-06-28T15:05:55.945 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2024-06-28T15:05:55.945 DEBUG:teuthology.orchestra.run.smithi178:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2024-06-28T15:05:56.009 INFO:tasks.workunit.client.0.smithi178.stdout:Running main() from gmock_main.cc 2024-06-28T15:05:56.009 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 4 tests from 1 test suite. 2024-06-28T15:05:56.009 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:05:56.009 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 4 tests from cls_log 2024-06-28T15:05:56.009 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_log.test_log_add_same_time 2024-06-28T15:05:58.890 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_log.test_log_add_same_time (2881 ms) 2024-06-28T15:05:58.890 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_log.test_log_add_different_time 2024-06-28T15:06:01.941 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_log.test_log_add_different_time (3050 ms) 2024-06-28T15:06:01.941 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_log.trim_by_time 2024-06-28T15:06:04.959 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_log.trim_by_time (3020 ms) 2024-06-28T15:06:04.959 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_log.trim_by_marker 2024-06-28T15:06:08.041 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_log.trim_by_marker (3082 ms) 2024-06-28T15:06:08.042 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 4 tests from cls_log (12033 ms total) 2024-06-28T15:06:08.042 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:06:08.042 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:06:08.042 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 4 tests from 1 test suite ran. (12033 ms total) 2024-06-28T15:06:08.042 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 4 tests. 2024-06-28T15:06:08.048 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:06:08.048 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:06:08.104 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2024-06-28T15:06:08.105 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2024-06-28T15:06:08.105 DEBUG:teuthology.orchestra.run.smithi178:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2024-06-28T15:06:08.170 INFO:tasks.workunit.client.0.smithi178.stdout:Running main() from gmock_main.cc 2024-06-28T15:06:08.170 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 10 tests from 1 test suite. 2024-06-28T15:06:08.170 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:06:08.170 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 10 tests from cls_refcount 2024-06-28T15:06:08.170 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_refcount.test_implicit 2024-06-28T15:06:11.072 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_refcount.test_implicit (2902 ms) 2024-06-28T15:06:11.072 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_refcount.test_implicit_idempotent 2024-06-28T15:06:14.093 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_refcount.test_implicit_idempotent (3021 ms) 2024-06-28T15:06:14.094 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_refcount.test_put_snap 2024-06-28T15:06:23.209 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_refcount.test_put_snap (9116 ms) 2024-06-28T15:06:23.209 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_refcount.test_explicit 2024-06-28T15:06:26.230 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_refcount.test_explicit (3021 ms) 2024-06-28T15:06:26.230 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_refcount.set 2024-06-28T15:06:29.282 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_refcount.set (3052 ms) 2024-06-28T15:06:29.282 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_refcount.test_implicit_ec 2024-06-28T15:06:36.344 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_refcount.test_implicit_ec (7062 ms) 2024-06-28T15:06:36.345 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_refcount.test_implicit_idempotent_ec 2024-06-28T15:06:43.395 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_refcount.test_implicit_idempotent_ec (7046 ms) 2024-06-28T15:06:43.395 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_refcount.test_put_snap_ec 2024-06-28T15:06:55.558 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_refcount.test_put_snap_ec (12168 ms) 2024-06-28T15:06:55.559 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_refcount.test_explicit_ec 2024-06-28T15:07:02.638 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_refcount.test_explicit_ec (7076 ms) 2024-06-28T15:07:02.639 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_refcount.set_ec 2024-06-28T15:07:09.743 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_refcount.set_ec (7102 ms) 2024-06-28T15:07:09.743 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 10 tests from cls_refcount (61567 ms total) 2024-06-28T15:07:09.743 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:07:09.743 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:07:09.743 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 10 tests from 1 test suite ran. (61567 ms total) 2024-06-28T15:07:09.743 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 10 tests. 2024-06-28T15:07:09.743 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:07:09.744 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:07:09.799 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2024-06-28T15:07:09.800 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2024-06-28T15:07:09.801 DEBUG:teuthology.orchestra.run.smithi178:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2024-06-28T15:07:09.865 INFO:tasks.workunit.client.0.smithi178.stdout:Running main() from gmock_main.cc 2024-06-28T15:07:09.865 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 16 tests from 1 test suite. 2024-06-28T15:07:09.866 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:07:09.866 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 16 tests from cls_rgw 2024-06-28T15:07:11.743 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.index_basic 2024-06-28T15:07:11.824 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.index_basic (82 ms) 2024-06-28T15:07:11.825 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2024-06-28T15:07:11.897 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (72 ms) 2024-06-28T15:07:11.897 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.index_remove_object 2024-06-28T15:07:12.010 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.index_remove_object (113 ms) 2024-06-28T15:07:12.010 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.index_suggest 2024-06-28T15:07:14.533 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.index_suggest (2523 ms) 2024-06-28T15:07:14.533 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.index_suggest_complete 2024-06-28T15:07:14.547 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.index_suggest_complete (14 ms) 2024-06-28T15:07:14.547 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.index_list 2024-06-28T15:07:14.600 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.index_list (52 ms) 2024-06-28T15:07:14.600 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.index_list_delimited 2024-06-28T15:07:55.670 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.index_list_delimited (41070 ms) 2024-06-28T15:07:55.670 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.bi_list 2024-06-28T15:07:55.858 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.bi_list (189 ms) 2024-06-28T15:07:55.858 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.gc_set 2024-06-28T15:07:55.882 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.gc_set (24 ms) 2024-06-28T15:07:55.882 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.gc_list 2024-06-28T15:07:55.911 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.gc_list (29 ms) 2024-06-28T15:07:55.911 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.gc_defer 2024-06-28T15:08:02.710 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.gc_defer (6797 ms) 2024-06-28T15:08:02.710 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.usage_basic 2024-06-28T15:08:02.919 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.usage_basic (211 ms) 2024-06-28T15:08:02.919 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2024-06-28T15:08:02.922 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.usage_clear_no_obj (3 ms) 2024-06-28T15:08:02.922 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.usage_clear 2024-06-28T15:08:03.147 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.usage_clear (225 ms) 2024-06-28T15:08:03.147 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.bi_log_trim 2024-06-28T15:08:03.274 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.bi_log_trim (127 ms) 2024-06-28T15:08:03.275 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw.index_racing_removes 2024-06-28T15:08:03.306 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw.index_racing_removes (33 ms) 2024-06-28T15:08:03.720 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 16 tests from cls_rgw (51565 ms total) 2024-06-28T15:08:03.720 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:08:03.720 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:08:03.720 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 16 tests from 1 test suite ran. (53856 ms total) 2024-06-28T15:08:03.720 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 16 tests. 2024-06-28T15:08:03.730 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:08:03.730 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:08:03.786 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_gc.sh on client.0... 2024-06-28T15:08:03.787 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_gc.sh... 2024-06-28T15:08:03.787 DEBUG:teuthology.orchestra.run.smithi178:workunit test cls/test_cls_rgw_gc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_gc.sh 2024-06-28T15:08:03.850 INFO:tasks.workunit.client.0.smithi178.stdout:Running main() from gmock_main.cc 2024-06-28T15:08:03.851 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 9 tests from 1 test suite. 2024-06-28T15:08:03.851 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:08:03.851 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 9 tests from cls_rgw_gc 2024-06-28T15:08:03.851 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw_gc.init 2024-06-28T15:08:05.721 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw_gc.init (1870 ms) 2024-06-28T15:08:05.721 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops1 2024-06-28T15:08:05.763 INFO:tasks.workunit.client.0.smithi178.stderr:[ ] list info tag = chain-0 2024-06-28T15:08:05.764 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw_gc.gc_queue_ops1 (43 ms) 2024-06-28T15:08:05.764 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops2 2024-06-28T15:08:05.775 INFO:tasks.workunit.client.0.smithi178.stderr:[ ] next_marker is: = 0/4263 2024-06-28T15:08:05.776 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw_gc.gc_queue_ops2 (13 ms) 2024-06-28T15:08:05.776 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops5 2024-06-28T15:08:05.790 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw_gc.gc_queue_ops5 (15 ms) 2024-06-28T15:08:05.790 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops6 2024-06-28T15:08:05.982 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw_gc.gc_queue_ops6 (191 ms) 2024-06-28T15:08:05.982 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops7 2024-06-28T15:08:06.001 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw_gc.gc_queue_ops7 (19 ms) 2024-06-28T15:08:06.001 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops8 2024-06-28T15:08:06.019 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw_gc.gc_queue_ops8 (18 ms) 2024-06-28T15:08:06.019 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops9 2024-06-28T15:08:06.036 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw_gc.gc_queue_ops9 (17 ms) 2024-06-28T15:08:06.036 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw_gc.finalize 2024-06-28T15:08:06.735 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw_gc.finalize (699 ms) 2024-06-28T15:08:06.735 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 9 tests from cls_rgw_gc (2885 ms total) 2024-06-28T15:08:06.735 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:08:06.735 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:08:06.735 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 9 tests from 1 test suite ran. (2885 ms total) 2024-06-28T15:08:06.735 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 9 tests. 2024-06-28T15:08:06.742 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:08:06.742 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:08:06.799 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_stats.sh on client.0... 2024-06-28T15:08:06.800 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_stats.sh... 2024-06-28T15:08:06.800 DEBUG:teuthology.orchestra.run.smithi178:workunit test cls/test_cls_rgw_stats.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_stats.sh 2024-06-28T15:08:06.874 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:08:06.874 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:08:06.875 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 1 test from 1 test suite. 2024-06-28T15:08:06.875 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:08:08.768 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 1 test from cls_rgw_stats 2024-06-28T15:08:08.768 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] cls_rgw_stats.simulate 2024-06-28T15:08:08.794 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.808+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=H9OQR4SKEK7W type=del size=3489605 2024-06-28T15:08:08.797 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.812+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~ZCL3I124XQN2.0 size=1048576 2024-06-28T15:08:08.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.812+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~ZCL3I124XQN2.1 size=883714 2024-06-28T15:08:08.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.812+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=ZCL3I124XQN2 tag=FV7GURSHV39I type=write size=2980866 2024-06-28T15:08:08.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.816+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~LTSZX62AY4PE.0 size=1048576 2024-06-28T15:08:08.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.816+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~LTSZX62AY4PE.1 size=1048576 2024-06-28T15:08:08.805 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.816+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~LTSZX62AY4PE.2 size=837285 2024-06-28T15:08:08.805 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.816+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=LTSZX62AY4PE tag=3REM6Q72HXQO type=write size=3983013 2024-06-28T15:08:08.807 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.820+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=C85AWNMU62J1 type=del size=1112236 2024-06-28T15:08:08.809 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.820+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=R2W4ZX5L1CP2 type=del size=4143682 2024-06-28T15:08:08.813 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.824+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~QL80MO06TA8G.0 size=1048576 2024-06-28T15:08:08.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.828+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~QL80MO06TA8G.1 size=1048576 2024-06-28T15:08:08.816 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.828+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~QL80MO06TA8G.2 size=695647 2024-06-28T15:08:08.816 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.828+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=QL80MO06TA8G tag=7S98TA59ACBS type=write size=3841375 2024-06-28T15:08:08.819 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.832+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~2R3522JUS7EB.0 size=1048576 2024-06-28T15:08:08.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.832+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~2R3522JUS7EB.1 size=1048576 2024-06-28T15:08:08.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.836+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~2R3522JUS7EB.2 size=261788 2024-06-28T15:08:08.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.836+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=2R3522JUS7EB tag=1K08LO6QVIQB type=write size=3407516 2024-06-28T15:08:08.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.840+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~HTWCHLXR6ZVF.0 size=1048576 2024-06-28T15:08:08.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.840+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~HTWCHLXR6ZVF.1 size=459 2024-06-28T15:08:08.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.840+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=HTWCHLXR6ZVF tag=QTOA54B0BRHW type=write size=2097611 2024-06-28T15:08:08.829 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.840+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=KAU1ZZVH6GQA type=del size=1587634 2024-06-28T15:08:08.831 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.844+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=P2OIZPQONVOE type=del size=617187 2024-06-28T15:08:08.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.848+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~HMFZ9PW5KKO7.0 size=1048576 2024-06-28T15:08:08.836 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.848+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~HMFZ9PW5KKO7.1 size=5259 2024-06-28T15:08:08.836 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.848+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=HMFZ9PW5KKO7 tag=O4S1392VP9YG type=write size=2102411 2024-06-28T15:08:08.839 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.852+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~HN5T0D5CHFLQ.0 size=6554 2024-06-28T15:08:08.840 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.852+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=HN5T0D5CHFLQ tag=ZKMS32L6G7UY type=write size=1055130 2024-06-28T15:08:08.841 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.852+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=VGO3C64XHZT1 type=del size=1651276 2024-06-28T15:08:08.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.856+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=BALJI1CTCLA0 type=write size=670472 2024-06-28T15:08:08.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.860+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~SZCC2A9RGWEF.0 size=769888 2024-06-28T15:08:08.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.860+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=SZCC2A9RGWEF tag=MVUDWIGRPYEH type=write size=1818464 2024-06-28T15:08:08.850 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.864+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~YHCSPV251NLC.0 size=771677 2024-06-28T15:08:08.850 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.864+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=YHCSPV251NLC tag=YRNPOUWG25SW type=write size=1820253 2024-06-28T15:08:08.852 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.864+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=H9OQR4SKEK7W type=del 2024-06-28T15:08:08.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.868+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~8KTJ3EQCN7PB.0 size=1048576 2024-06-28T15:08:08.857 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.868+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~8KTJ3EQCN7PB.1 size=852968 2024-06-28T15:08:08.857 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.868+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=8KTJ3EQCN7PB tag=CGDKYQLCG61I type=write size=2950120 2024-06-28T15:08:08.858 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.872+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~ZCL3I124XQN2.0 size=1048576 2024-06-28T15:08:08.860 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.872+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~ZCL3I124XQN2.1 size=883714 2024-06-28T15:08:08.863 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.876+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=ZCL3I124XQN2 tag=FV7GURSHV39I type=write size=2980866 2024-06-28T15:08:08.865 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.876+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=4U41STQJ7SQ4 type=del size=2768767 2024-06-28T15:08:08.865 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.876+0000 7f65680598c0 1 < canceled part key=_multipart_RLJCNXKA2GYA.2~LTSZX62AY4PE.0 size=1048576 2024-06-28T15:08:08.866 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.880+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~LTSZX62AY4PE.1 size=1048576 2024-06-28T15:08:08.868 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.880+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~LTSZX62AY4PE.2 size=837285 2024-06-28T15:08:08.870 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.884+0000 7f65680598c0 1 < canceled multipart upload key=RLJCNXKA2GYA upload=LTSZX62AY4PE tag=3REM6Q72HXQO type=write size=3983013 2024-06-28T15:08:08.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.884+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=RXV866SFVZTF type=del size=2238850 2024-06-28T15:08:08.873 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.888+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=C85AWNMU62J1 type=del 2024-06-28T15:08:08.875 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.888+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=RY1V1XHU9ESE type=del size=4131021 2024-06-28T15:08:08.877 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.888+0000 7f65680598c0 1 < canceled operation key=60QVJVV16QFM tag=R2W4ZX5L1CP2 type=del size=4143682 2024-06-28T15:08:08.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.892+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~XYJKIWL1RNU6.0 size=137369 2024-06-28T15:08:08.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.892+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=XYJKIWL1RNU6 tag=CURTE6J8X1T6 type=write size=1185945 2024-06-28T15:08:08.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.896+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~QL80MO06TA8G.0 size=1048576 2024-06-28T15:08:08.883 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.896+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~QL80MO06TA8G.1 size=1048576 2024-06-28T15:08:08.885 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.896+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~QL80MO06TA8G.2 size=695647 2024-06-28T15:08:08.888 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.900+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=QL80MO06TA8G tag=7S98TA59ACBS type=write size=3841375 2024-06-28T15:08:08.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.904+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~FTGVE972VN7R.0 size=1048576 2024-06-28T15:08:08.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.904+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~FTGVE972VN7R.1 size=1048576 2024-06-28T15:08:08.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.908+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~FTGVE972VN7R.2 size=281258 2024-06-28T15:08:08.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.908+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=FTGVE972VN7R tag=R6E9ECCDSNW7 type=write size=3426986 2024-06-28T15:08:08.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.908+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~2R3522JUS7EB.0 size=1048576 2024-06-28T15:08:08.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.908+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~2R3522JUS7EB.1 size=1048576 2024-06-28T15:08:08.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.912+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~2R3522JUS7EB.2 size=261788 2024-06-28T15:08:08.901 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.912+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=2R3522JUS7EB tag=1K08LO6QVIQB type=write size=3407516 2024-06-28T15:08:08.903 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.916+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=Q5B3JOEGIX9Z type=del size=3028941 2024-06-28T15:08:08.905 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.916+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~HTWCHLXR6ZVF.0 size=1048576 2024-06-28T15:08:08.906 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.920+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~HTWCHLXR6ZVF.1 size=459 2024-06-28T15:08:08.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.924+0000 7f65680598c0 1 < completed multipart upload key=MQ3QWZBFN4YR upload=HTWCHLXR6ZVF tag=QTOA54B0BRHW type=write size=2097611 2024-06-28T15:08:08.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.924+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~YJZWCS522A5V.0 size=1048576 2024-06-28T15:08:08.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.928+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~YJZWCS522A5V.1 size=639180 2024-06-28T15:08:08.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.928+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=YJZWCS522A5V tag=Y7KUUGQWQFNH type=write size=2736332 2024-06-28T15:08:08.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.928+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=KAU1ZZVH6GQA type=del 2024-06-28T15:08:08.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.932+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~O6EYI03HS759.0 size=1048576 2024-06-28T15:08:08.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.932+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~O6EYI03HS759.1 size=811270 2024-06-28T15:08:08.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.932+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=O6EYI03HS759 tag=K4IXMYEB5TIV type=write size=2908422 2024-06-28T15:08:08.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.936+0000 7f65680598c0 1 < completed delete operation key=ZFNIA3O84EB7 tag=P2OIZPQONVOE type=del 2024-06-28T15:08:08.924 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.936+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=6LIBBMSEDM7H type=write size=132947 2024-06-28T15:08:08.926 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.940+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~HMFZ9PW5KKO7.0 size=1048576 2024-06-28T15:08:08.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.940+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~HMFZ9PW5KKO7.1 size=5259 2024-06-28T15:08:08.931 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.944+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=HMFZ9PW5KKO7 tag=O4S1392VP9YG type=write size=2102411 2024-06-28T15:08:08.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.948+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~VLT8W2AFANA7.0 size=1048576 2024-06-28T15:08:08.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.948+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~VLT8W2AFANA7.1 size=1048576 2024-06-28T15:08:08.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.948+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~VLT8W2AFANA7.2 size=590658 2024-06-28T15:08:08.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.948+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=VLT8W2AFANA7 tag=S5GXRW74OMBG type=write size=3736386 2024-06-28T15:08:08.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.952+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~HN5T0D5CHFLQ.0 size=6554 2024-06-28T15:08:08.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.956+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=HN5T0D5CHFLQ tag=ZKMS32L6G7UY type=write size=1055130 2024-06-28T15:08:08.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.956+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=9BRGHNU52PBS type=del size=3066766 2024-06-28T15:08:08.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.956+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=VGO3C64XHZT1 type=del 2024-06-28T15:08:08.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.960+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=ZJTOO8VIAR9E type=write size=283773 2024-06-28T15:08:08.949 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.960+0000 7f65680598c0 1 < completed write operation key=UP3XFF38IJXN tag=BALJI1CTCLA0 type=write size=670472 2024-06-28T15:08:08.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.964+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=FL2YGGX08JOU type=del size=1316843 2024-06-28T15:08:08.953 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.964+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~SZCC2A9RGWEF.0 size=769888 2024-06-28T15:08:08.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.968+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=SZCC2A9RGWEF tag=MVUDWIGRPYEH type=write size=1818464 2024-06-28T15:08:08.958 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.972+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=U2KGHG7R78GI type=del size=1727707 2024-06-28T15:08:08.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.972+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~YHCSPV251NLC.0 size=771677 2024-06-28T15:08:08.964 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.976+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=YHCSPV251NLC tag=YRNPOUWG25SW type=write size=1820253 2024-06-28T15:08:08.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.980+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=8CMLYAW5NW16 type=del size=427445 2024-06-28T15:08:08.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.980+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~8KTJ3EQCN7PB.0 size=1048576 2024-06-28T15:08:08.969 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.980+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~8KTJ3EQCN7PB.1 size=852968 2024-06-28T15:08:08.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.984+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=8KTJ3EQCN7PB tag=CGDKYQLCG61I type=write size=2950120 2024-06-28T15:08:08.975 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.988+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=FT50I2DZVG16 type=del size=3708709 2024-06-28T15:08:08.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.992+0000 7f65680598c0 1 < suggested update operation key=F6SEDD6BH1T2 tag=4U41STQJ7SQ4 type=del size=2768767 2024-06-28T15:08:08.981 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.992+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~SBPT23TFG2X9.0 size=1048576 2024-06-28T15:08:08.982 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.996+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~SBPT23TFG2X9.1 size=289653 2024-06-28T15:08:08.982 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.996+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=SBPT23TFG2X9 tag=89UCX0S306UY type=write size=2386805 2024-06-28T15:08:08.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:08.996+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=RXV866SFVZTF type=del 2024-06-28T15:08:08.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.000+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=YIOXZUBKH0L9 type=write size=934081 2024-06-28T15:08:08.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.000+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=RY1V1XHU9ESE type=del 2024-06-28T15:08:08.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.000+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=5AA7ZTTSD5N1 type=del size=3305422 2024-06-28T15:08:08.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.004+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~XYJKIWL1RNU6.0 size=137369 2024-06-28T15:08:08.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.004+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=XYJKIWL1RNU6 tag=CURTE6J8X1T6 type=write size=1185945 2024-06-28T15:08:08.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.008+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=L2R2S0WDJAHN type=del size=1633794 2024-06-28T15:08:08.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.008+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~FTGVE972VN7R.0 size=1048576 2024-06-28T15:08:08.998 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.012+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~FTGVE972VN7R.1 size=1048576 2024-06-28T15:08:08.999 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.012+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~FTGVE972VN7R.2 size=281258 2024-06-28T15:08:09.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.016+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=FTGVE972VN7R tag=R6E9ECCDSNW7 type=write size=3426986 2024-06-28T15:08:09.004 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.016+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=Z6DPRH8BHG04 type=del size=873323 2024-06-28T15:08:09.005 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.016+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=Q5B3JOEGIX9Z type=del 2024-06-28T15:08:09.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.020+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=IRMVT4OE5HXB type=del size=1540072 2024-06-28T15:08:09.008 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.020+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~YJZWCS522A5V.0 size=1048576 2024-06-28T15:08:09.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.024+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~YJZWCS522A5V.1 size=639180 2024-06-28T15:08:09.012 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.024+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=YJZWCS522A5V tag=Y7KUUGQWQFNH type=write size=2736332 2024-06-28T15:08:09.014 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.028+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=9ORKZ6ODXVTW type=del size=3514878 2024-06-28T15:08:09.016 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.028+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~O6EYI03HS759.0 size=1048576 2024-06-28T15:08:09.016 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.028+0000 7f65680598c0 1 < canceled part key=_multipart_C5C6SAHLPZV1.2~O6EYI03HS759.1 size=811270 2024-06-28T15:08:09.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.032+0000 7f65680598c0 1 < canceled multipart upload key=C5C6SAHLPZV1 upload=O6EYI03HS759 tag=K4IXMYEB5TIV type=write size=2908422 2024-06-28T15:08:09.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.036+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~I20M3O63VF9P.0 size=1048576 2024-06-28T15:08:09.023 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.036+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~I20M3O63VF9P.1 size=1048576 2024-06-28T15:08:09.024 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.036+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~I20M3O63VF9P.2 size=189931 2024-06-28T15:08:09.025 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.036+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=I20M3O63VF9P tag=EK3VXG12SQBP type=write size=3335659 2024-06-28T15:08:09.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.040+0000 7f65680598c0 1 < suggested remove operation key=60QVJVV16QFM tag=6LIBBMSEDM7H type=write 2024-06-28T15:08:09.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.044+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=5SOPKWE202D2 type=del size=851292 2024-06-28T15:08:09.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.044+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~VLT8W2AFANA7.0 size=1048576 2024-06-28T15:08:09.033 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.044+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~VLT8W2AFANA7.1 size=1048576 2024-06-28T15:08:09.035 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.048+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~VLT8W2AFANA7.2 size=590658 2024-06-28T15:08:09.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.052+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=VLT8W2AFANA7 tag=S5GXRW74OMBG type=write size=3736386 2024-06-28T15:08:09.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.056+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~BYKJVT784AG7.0 size=133920 2024-06-28T15:08:09.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.056+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=BYKJVT784AG7 tag=5KEMXKHTN2W9 type=write size=1182496 2024-06-28T15:08:09.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.060+0000 7f65680598c0 1 < suggested update operation key=BHW2VHUT0U1U tag=9BRGHNU52PBS type=del size=3066766 2024-06-28T15:08:09.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.064+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~2GVU7ATAMWFA.0 size=1048576 2024-06-28T15:08:09.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.064+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~2GVU7ATAMWFA.1 size=999176 2024-06-28T15:08:09.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.064+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=2GVU7ATAMWFA tag=E01RB8UM3EQ0 type=write size=3096328 2024-06-28T15:08:09.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.068+0000 7f65680598c0 1 < canceled operation key=RLJCNXKA2GYA tag=ZJTOO8VIAR9E type=write size=283773 2024-06-28T15:08:09.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.072+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=1FB987QJEC43 type=del size=2459919 2024-06-28T15:08:09.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.072+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=FL2YGGX08JOU type=del 2024-06-28T15:08:09.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.076+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~B2UXBFL0YDJE.0 size=76773 2024-06-28T15:08:09.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.076+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=B2UXBFL0YDJE tag=3Y12KPSBOLYL type=write size=1125349 2024-06-28T15:08:09.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.076+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=U2KGHG7R78GI type=del 2024-06-28T15:08:09.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.080+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=V7OMTR39WWJJ type=del size=3084968 2024-06-28T15:08:09.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.080+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=8CMLYAW5NW16 type=del 2024-06-28T15:08:09.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.084+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~5O2R74C7N8WU.0 size=1048576 2024-06-28T15:08:09.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.084+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~5O2R74C7N8WU.1 size=144399 2024-06-28T15:08:09.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.084+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=5O2R74C7N8WU tag=RA7P303D2007 type=write size=2241551 2024-06-28T15:08:09.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.088+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=FT50I2DZVG16 type=del 2024-06-28T15:08:09.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.088+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=AA5AP6FUB8N2 type=del size=1170916 2024-06-28T15:08:09.079 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.092+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~SBPT23TFG2X9.0 size=1048576 2024-06-28T15:08:09.081 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.092+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~SBPT23TFG2X9.1 size=289653 2024-06-28T15:08:09.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.096+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=SBPT23TFG2X9 tag=89UCX0S306UY type=write size=2386805 2024-06-28T15:08:09.086 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.100+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=QM8HRW6651SE type=del size=663417 2024-06-28T15:08:09.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.104+0000 7f65680598c0 1 < suggested update operation key=EN76QQWIQAXZ tag=YIOXZUBKH0L9 type=write size=934081 2024-06-28T15:08:09.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.104+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~40780M9P84O3.0 size=1048576 2024-06-28T15:08:09.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.108+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~40780M9P84O3.1 size=1048576 2024-06-28T15:08:09.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.108+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~40780M9P84O3.2 size=698263 2024-06-28T15:08:09.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.108+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=40780M9P84O3 tag=CHZETJE4XESX type=write size=3843991 2024-06-28T15:08:09.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.112+0000 7f65680598c0 1 < suggested update operation key=EN76QQWIQAXZ tag=5AA7ZTTSD5N1 type=del size=3305422 2024-06-28T15:08:09.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.116+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=KNMXTJPY2RWN type=del size=3084488 2024-06-28T15:08:09.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.116+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=L2R2S0WDJAHN type=del 2024-06-28T15:08:09.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.120+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~T1KABLICVJ77.0 size=1048576 2024-06-28T15:08:09.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.120+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~T1KABLICVJ77.1 size=1048576 2024-06-28T15:08:09.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.124+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~T1KABLICVJ77.2 size=969450 2024-06-28T15:08:09.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.124+0000 7f65680598c0 1 > prepared multipart upload key=2F968YIMJN9D upload=T1KABLICVJ77 tag=2JO0W99DDWWX type=write size=4115178 2024-06-28T15:08:09.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.124+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=Z6DPRH8BHG04 type=del 2024-06-28T15:08:09.115 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.128+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=F0NECA1XV9Q6 type=del size=167697 2024-06-28T15:08:09.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.132+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=IRMVT4OE5HXB type=del 2024-06-28T15:08:09.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.132+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=W9FNE39JAK4A type=write size=993373 2024-06-28T15:08:09.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.136+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=9ORKZ6ODXVTW type=del 2024-06-28T15:08:09.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.136+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=OVPN2PYSJOLY type=del size=3108688 2024-06-28T15:08:09.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.140+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~I20M3O63VF9P.0 size=1048576 2024-06-28T15:08:09.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.140+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~I20M3O63VF9P.1 size=1048576 2024-06-28T15:08:09.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.144+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~I20M3O63VF9P.2 size=189931 2024-06-28T15:08:09.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.144+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=I20M3O63VF9P tag=EK3VXG12SQBP type=write size=3335659 2024-06-28T15:08:09.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.148+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~T5O3DZIP0032.0 size=595421 2024-06-28T15:08:09.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.148+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=T5O3DZIP0032 tag=PZC8BM2RQCAK type=write size=1643997 2024-06-28T15:08:09.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.152+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=5SOPKWE202D2 type=del 2024-06-28T15:08:09.140 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.152+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=7336KSS551KS type=write size=979015 2024-06-28T15:08:09.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.156+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~BYKJVT784AG7.0 size=133920 2024-06-28T15:08:09.146 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.160+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=BYKJVT784AG7 tag=5KEMXKHTN2W9 type=write size=1182496 2024-06-28T15:08:09.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.160+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=M2D2ZIBN0KMM type=write size=734247 2024-06-28T15:08:09.152 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.164+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~2GVU7ATAMWFA.0 size=1048576 2024-06-28T15:08:09.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.164+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~2GVU7ATAMWFA.1 size=999176 2024-06-28T15:08:09.157 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.168+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=2GVU7ATAMWFA tag=E01RB8UM3EQ0 type=write size=3096328 2024-06-28T15:08:09.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.172+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=CH9NKQ0JVKF1 type=write size=408168 2024-06-28T15:08:09.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.172+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=1FB987QJEC43 type=del 2024-06-28T15:08:09.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.176+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=G5VZLBO0NAZ6 type=del size=1599354 2024-06-28T15:08:09.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.176+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~B2UXBFL0YDJE.0 size=76773 2024-06-28T15:08:09.168 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.180+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=B2UXBFL0YDJE tag=3Y12KPSBOLYL type=write size=1125349 2024-06-28T15:08:09.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.184+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~2NV6RVGI9XCV.0 size=1048576 2024-06-28T15:08:09.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.184+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~2NV6RVGI9XCV.1 size=717484 2024-06-28T15:08:09.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.184+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=2NV6RVGI9XCV tag=THZZ9HR3NZMT type=write size=2814636 2024-06-28T15:08:09.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.188+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=V7OMTR39WWJJ type=del 2024-06-28T15:08:09.177 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.188+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=0OXF7M6VJ9O2 type=del size=102114 2024-06-28T15:08:09.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.192+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~5O2R74C7N8WU.0 size=1048576 2024-06-28T15:08:09.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.192+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~5O2R74C7N8WU.1 size=144399 2024-06-28T15:08:09.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.196+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=5O2R74C7N8WU tag=RA7P303D2007 type=write size=2241551 2024-06-28T15:08:09.190 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.200+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~WTYJ4CJG0BB1.0 size=1048576 2024-06-28T15:08:09.190 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.200+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~WTYJ4CJG0BB1.1 size=265724 2024-06-28T15:08:09.190 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.200+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=WTYJ4CJG0BB1 tag=J7JVTR3WTFS1 type=write size=2362876 2024-06-28T15:08:09.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.204+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=AA5AP6FUB8N2 type=del 2024-06-28T15:08:09.195 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.208+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~9ZD5KYH6GQ1C.0 size=1048576 2024-06-28T15:08:09.196 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.208+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~9ZD5KYH6GQ1C.1 size=225292 2024-06-28T15:08:09.196 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.208+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=9ZD5KYH6GQ1C tag=JBK6HUA9V7X8 type=write size=2322444 2024-06-28T15:08:09.198 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.212+0000 7f65680598c0 1 < canceled operation key=LZX5UZCYLTAG tag=QM8HRW6651SE type=del size=663417 2024-06-28T15:08:09.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.212+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=XMKOACIM5X3A type=del size=54486 2024-06-28T15:08:09.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.216+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~40780M9P84O3.0 size=1048576 2024-06-28T15:08:09.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.216+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~40780M9P84O3.1 size=1048576 2024-06-28T15:08:09.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.216+0000 7f65680598c0 1 < canceled part key=_multipart_MQ3QWZBFN4YR.2~40780M9P84O3.2 size=698263 2024-06-28T15:08:09.207 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.220+0000 7f65680598c0 1 < canceled multipart upload key=MQ3QWZBFN4YR upload=40780M9P84O3 tag=CHZETJE4XESX type=write size=3843991 2024-06-28T15:08:09.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.220+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=EJSTG0ZI5PDK type=del size=2910335 2024-06-28T15:08:09.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.224+0000 7f65680598c0 1 < suggested remove operation key=AOBFAR0UTONH tag=KNMXTJPY2RWN type=del 2024-06-28T15:08:09.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.228+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=XFPFWQPX5YRT type=write size=1048423 2024-06-28T15:08:09.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.228+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~T1KABLICVJ77.0 size=1048576 2024-06-28T15:08:09.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.232+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~T1KABLICVJ77.1 size=1048576 2024-06-28T15:08:09.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.232+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~T1KABLICVJ77.2 size=969450 2024-06-28T15:08:09.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.236+0000 7f65680598c0 1 < completed multipart upload key=2F968YIMJN9D upload=T1KABLICVJ77 tag=2JO0W99DDWWX type=write size=4115178 2024-06-28T15:08:09.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.236+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=WZ8287LXO5YL type=del size=2456211 2024-06-28T15:08:09.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.240+0000 7f65680598c0 1 < canceled operation key=1VD7NDPEM3CD tag=F0NECA1XV9Q6 type=del size=167697 2024-06-28T15:08:09.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.240+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=NFMR6AX6Z7TM type=del size=558086 2024-06-28T15:08:09.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.244+0000 7f65680598c0 1 < completed write operation key=RLJCNXKA2GYA tag=W9FNE39JAK4A type=write size=993373 2024-06-28T15:08:09.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.244+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~W6NL73GYQI1M.0 size=1048576 2024-06-28T15:08:09.234 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.248+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~W6NL73GYQI1M.1 size=1048576 2024-06-28T15:08:09.235 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.248+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~W6NL73GYQI1M.2 size=981490 2024-06-28T15:08:09.235 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.248+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=W6NL73GYQI1M tag=JM3KDUPB5LF9 type=write size=4127218 2024-06-28T15:08:09.237 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.252+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=OVPN2PYSJOLY type=del 2024-06-28T15:08:09.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.252+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~M1HE4KIMF1VK.0 size=842353 2024-06-28T15:08:09.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.252+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=M1HE4KIMF1VK tag=4B0XSKWHXDMN type=write size=1890929 2024-06-28T15:08:09.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.256+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~T5O3DZIP0032.0 size=595421 2024-06-28T15:08:09.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.260+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=T5O3DZIP0032 tag=PZC8BM2RQCAK type=write size=1643997 2024-06-28T15:08:09.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.264+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~RX9NYH4S13OK.0 size=557927 2024-06-28T15:08:09.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.264+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=RX9NYH4S13OK tag=8RE5AHLJ1VPR type=write size=1606503 2024-06-28T15:08:09.252 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.264+0000 7f65680598c0 1 < completed write operation key=AXQ0YFWUH4FD tag=7336KSS551KS type=write size=979015 2024-06-28T15:08:09.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.268+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~X23V4MW1CZA9.0 size=371428 2024-06-28T15:08:09.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.268+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=X23V4MW1CZA9 tag=NLE5SXWA0KSI type=write size=1420004 2024-06-28T15:08:09.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.272+0000 7f65680598c0 1 < completed write operation key=O1CGONN4TEXF tag=M2D2ZIBN0KMM type=write size=734247 2024-06-28T15:08:09.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.272+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=LH7FN6G4VXBR type=del size=141595 2024-06-28T15:08:09.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.276+0000 7f65680598c0 1 < suggested remove operation key=7LB8MOJL39RM tag=CH9NKQ0JVKF1 type=write 2024-06-28T15:08:09.267 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.280+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~II2MACYMNCFL.0 size=1048576 2024-06-28T15:08:09.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.280+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~II2MACYMNCFL.1 size=370434 2024-06-28T15:08:09.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.280+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=II2MACYMNCFL tag=L2HF6AJR75YC type=write size=2467586 2024-06-28T15:08:09.271 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.284+0000 7f65680598c0 1 < completed delete operation key=1VD7NDPEM3CD tag=G5VZLBO0NAZ6 type=del 2024-06-28T15:08:09.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.284+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=291F7FHVKEQM type=del size=488790 2024-06-28T15:08:09.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.288+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~2NV6RVGI9XCV.0 size=1048576 2024-06-28T15:08:09.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.288+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~2NV6RVGI9XCV.1 size=717484 2024-06-28T15:08:09.280 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.292+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=2NV6RVGI9XCV tag=THZZ9HR3NZMT type=write size=2814636 2024-06-28T15:08:09.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.296+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=A3OPFYHGT3QR type=del size=914525 2024-06-28T15:08:09.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.296+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=0OXF7M6VJ9O2 type=del 2024-06-28T15:08:09.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.300+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=Y2L1W3L33LFE type=del size=1059250 2024-06-28T15:08:09.289 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.300+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~WTYJ4CJG0BB1.0 size=1048576 2024-06-28T15:08:09.291 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.304+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~WTYJ4CJG0BB1.1 size=265724 2024-06-28T15:08:09.294 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.308+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=WTYJ4CJG0BB1 tag=J7JVTR3WTFS1 type=write size=2362876 2024-06-28T15:08:09.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.308+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=PLKG118OKBOM type=write size=60496 2024-06-28T15:08:09.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.312+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~9ZD5KYH6GQ1C.0 size=1048576 2024-06-28T15:08:09.300 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.312+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~9ZD5KYH6GQ1C.1 size=225292 2024-06-28T15:08:09.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.316+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=9ZD5KYH6GQ1C tag=JBK6HUA9V7X8 type=write size=2322444 2024-06-28T15:08:09.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.320+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~W3IHFFIL8Q2D.0 size=786225 2024-06-28T15:08:09.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.320+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=W3IHFFIL8Q2D tag=B5CBA0QUMMLS type=write size=1834801 2024-06-28T15:08:09.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.324+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=XMKOACIM5X3A type=del 2024-06-28T15:08:09.312 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.324+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=IW1HU01M1P68 type=write size=99482 2024-06-28T15:08:09.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.328+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=EJSTG0ZI5PDK type=del 2024-06-28T15:08:09.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.332+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~MDMX32SJ4GDY.0 size=1048576 2024-06-28T15:08:09.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.332+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~MDMX32SJ4GDY.1 size=1024941 2024-06-28T15:08:09.320 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.332+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=MDMX32SJ4GDY tag=GWYWB54PJJW8 type=write size=3122093 2024-06-28T15:08:09.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.336+0000 7f65680598c0 1 < suggested update operation key=AXQ0YFWUH4FD tag=XFPFWQPX5YRT type=write size=1048423 2024-06-28T15:08:09.326 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.340+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=ZJSW975BT6AC type=del size=2360073 2024-06-28T15:08:09.328 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.340+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=WZ8287LXO5YL type=del 2024-06-28T15:08:09.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.344+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=02JZC5P9SR74 type=del size=2395089 2024-06-28T15:08:09.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.344+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=NFMR6AX6Z7TM type=del 2024-06-28T15:08:09.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.348+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=FDDOYJNY1129 type=write size=689804 2024-06-28T15:08:09.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.352+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~W6NL73GYQI1M.0 size=1048576 2024-06-28T15:08:09.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.356+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~W6NL73GYQI1M.1 size=1048576 2024-06-28T15:08:09.345 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.356+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~W6NL73GYQI1M.2 size=981490 2024-06-28T15:08:09.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.364+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=W6NL73GYQI1M tag=JM3KDUPB5LF9 type=write size=4127218 2024-06-28T15:08:09.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.364+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=G3F277FXVE3X type=del size=1957135 2024-06-28T15:08:09.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.368+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~M1HE4KIMF1VK.0 size=842353 2024-06-28T15:08:09.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.372+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=M1HE4KIMF1VK tag=4B0XSKWHXDMN type=write size=1890929 2024-06-28T15:08:09.363 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.376+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=LCQMB0KUP9JS type=del size=2353411 2024-06-28T15:08:09.365 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.376+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~RX9NYH4S13OK.0 size=557927 2024-06-28T15:08:09.368 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.380+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=RX9NYH4S13OK tag=8RE5AHLJ1VPR type=write size=1606503 2024-06-28T15:08:09.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.384+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=LNBGP3KESDMJ type=write size=263365 2024-06-28T15:08:09.373 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.388+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~X23V4MW1CZA9.0 size=371428 2024-06-28T15:08:09.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.392+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=X23V4MW1CZA9 tag=NLE5SXWA0KSI type=write size=1420004 2024-06-28T15:08:09.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.392+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=VXQSAWC4S6TK type=del size=1168566 2024-06-28T15:08:09.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.396+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=LH7FN6G4VXBR type=del 2024-06-28T15:08:09.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.400+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=5XA01XOIW22R type=del size=3726496 2024-06-28T15:08:09.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.400+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~II2MACYMNCFL.0 size=1048576 2024-06-28T15:08:09.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.404+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~II2MACYMNCFL.1 size=370434 2024-06-28T15:08:09.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.408+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=II2MACYMNCFL tag=L2HF6AJR75YC type=write size=2467586 2024-06-28T15:08:09.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.412+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~I6PIHPNGLAEL.0 size=1048576 2024-06-28T15:08:09.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.416+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~I6PIHPNGLAEL.1 size=1048576 2024-06-28T15:08:09.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.420+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~I6PIHPNGLAEL.2 size=933000 2024-06-28T15:08:09.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.420+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=I6PIHPNGLAEL tag=1D50R4CVI4S5 type=write size=4078728 2024-06-28T15:08:09.409 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.420+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=291F7FHVKEQM type=del 2024-06-28T15:08:09.411 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.424+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=CN2KHCBKDZRT type=del size=3817794 2024-06-28T15:08:09.414 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.428+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=A3OPFYHGT3QR type=del 2024-06-28T15:08:09.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.432+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=59W7621RFZCF type=del size=1502591 2024-06-28T15:08:09.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.436+0000 7f65680598c0 1 < suggested remove operation key=ZY973GBGBQ1Z tag=Y2L1W3L33LFE type=del 2024-06-28T15:08:09.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.440+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~KEQD7X1GBZ1C.0 size=1048576 2024-06-28T15:08:09.430 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.444+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~KEQD7X1GBZ1C.1 size=1048576 2024-06-28T15:08:09.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.444+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~KEQD7X1GBZ1C.2 size=894920 2024-06-28T15:08:09.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.444+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=KEQD7X1GBZ1C tag=QT5H1ZDN06IZ type=write size=4040648 2024-06-28T15:08:09.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.448+0000 7f65680598c0 1 < completed write operation key=EN76QQWIQAXZ tag=PLKG118OKBOM type=write size=60496 2024-06-28T15:08:09.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.452+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~IVEIW6BB1LG8.0 size=722633 2024-06-28T15:08:09.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.452+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=IVEIW6BB1LG8 tag=RJTF19IHRA15 type=write size=1771209 2024-06-28T15:08:09.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.456+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~W3IHFFIL8Q2D.0 size=786225 2024-06-28T15:08:09.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.456+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=W3IHFFIL8Q2D tag=B5CBA0QUMMLS type=write size=1834801 2024-06-28T15:08:09.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.460+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=2H39RYV072Y2 type=del size=2083970 2024-06-28T15:08:09.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.464+0000 7f65680598c0 -1 < failed to complete operation key=ZY973GBGBQ1Z tag=IW1HU01M1P68 type=write size=99482: (22) Invalid argument 2024-06-28T15:08:09.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.464+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=0XQV1I1NR76T type=del size=3203862 2024-06-28T15:08:09.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.468+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~MDMX32SJ4GDY.0 size=1048576 2024-06-28T15:08:09.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.468+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~MDMX32SJ4GDY.1 size=1024941 2024-06-28T15:08:09.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.472+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=MDMX32SJ4GDY tag=GWYWB54PJJW8 type=write size=3122093 2024-06-28T15:08:09.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.476+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=9Q6QJPU37GGA type=del size=4109185 2024-06-28T15:08:09.465 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.480+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=ZJSW975BT6AC type=del 2024-06-28T15:08:09.468 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.480+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=4MXRMR3E6KSG type=del size=964519 2024-06-28T15:08:09.470 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.484+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=02JZC5P9SR74 type=del 2024-06-28T15:08:09.473 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.484+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=0Z3JQRFQD584 type=del size=1575018 2024-06-28T15:08:09.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.488+0000 7f65680598c0 1 < completed write operation key=EN76QQWIQAXZ tag=FDDOYJNY1129 type=write size=689804 2024-06-28T15:08:09.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.492+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=TQ9Z1X1WN4YA type=del size=1195817 2024-06-28T15:08:09.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.492+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=G3F277FXVE3X type=del 2024-06-28T15:08:09.482 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.496+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=2D308AHMATSH type=write size=494876 2024-06-28T15:08:09.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.500+0000 7f65680598c0 1 < suggested update operation key=4CU86XS1R9G3 tag=LCQMB0KUP9JS type=del size=2353411 2024-06-28T15:08:09.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.504+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~667MBX5ZHPYB.0 size=1048576 2024-06-28T15:08:09.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.508+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~667MBX5ZHPYB.1 size=1048576 2024-06-28T15:08:09.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.508+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~667MBX5ZHPYB.2 size=826951 2024-06-28T15:08:09.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.508+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=667MBX5ZHPYB tag=3NBRH7QAEMC5 type=write size=3972679 2024-06-28T15:08:09.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.512+0000 7f65680598c0 1 < completed write operation key=I4RWCQSQVLCN tag=LNBGP3KESDMJ type=write size=263365 2024-06-28T15:08:09.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.512+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=MTW5H3JQN4YY type=del size=2377120 2024-06-28T15:08:09.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.516+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=VXQSAWC4S6TK type=del 2024-06-28T15:08:09.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.520+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~NPS7DHRIDKGS.0 size=1048576 2024-06-28T15:08:09.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.524+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~NPS7DHRIDKGS.1 size=1048576 2024-06-28T15:08:09.512 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.524+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~NPS7DHRIDKGS.2 size=574525 2024-06-28T15:08:09.512 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.524+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=NPS7DHRIDKGS tag=A74EVFHV4THL type=write size=3720253 2024-06-28T15:08:09.515 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.528+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=5XA01XOIW22R type=del 2024-06-28T15:08:09.517 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.532+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=0SZRG3XUMFIA type=del size=3682669 2024-06-28T15:08:09.518 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.532+0000 7f65680598c0 1 < canceled part key=_multipart_EN76QQWIQAXZ.2~I6PIHPNGLAEL.0 size=1048576 2024-06-28T15:08:09.520 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.532+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~I6PIHPNGLAEL.1 size=1048576 2024-06-28T15:08:09.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.536+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~I6PIHPNGLAEL.2 size=933000 2024-06-28T15:08:09.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.540+0000 7f65680598c0 1 < canceled multipart upload key=EN76QQWIQAXZ upload=I6PIHPNGLAEL tag=1D50R4CVI4S5 type=write size=4078728 2024-06-28T15:08:09.530 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.544+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=DH9LQO4ZWUJN type=del size=4102296 2024-06-28T15:08:09.536 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.548+0000 7f65680598c0 1 < suggested remove operation key=F6SEDD6BH1T2 tag=CN2KHCBKDZRT type=del 2024-06-28T15:08:09.539 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.552+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=ITZOHYSHALSH type=del size=3686935 2024-06-28T15:08:09.541 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.556+0000 7f65680598c0 1 < canceled operation key=7NL3CTFYY9E5 tag=59W7621RFZCF type=del size=1502591 2024-06-28T15:08:09.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.560+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~FI0EN2KNVCRY.0 size=1048576 2024-06-28T15:08:09.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.560+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~FI0EN2KNVCRY.1 size=1048576 2024-06-28T15:08:09.551 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.564+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~FI0EN2KNVCRY.2 size=244323 2024-06-28T15:08:09.551 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.564+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=FI0EN2KNVCRY tag=MD9KDZ3ST1N8 type=write size=3390051 2024-06-28T15:08:09.553 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.568+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~KEQD7X1GBZ1C.0 size=1048576 2024-06-28T15:08:09.554 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.568+0000 7f65680598c0 1 < canceled part key=_multipart_2FC9DNEZGWC4.2~KEQD7X1GBZ1C.1 size=1048576 2024-06-28T15:08:09.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.568+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~KEQD7X1GBZ1C.2 size=894920 2024-06-28T15:08:09.560 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.572+0000 7f65680598c0 1 < canceled multipart upload key=2FC9DNEZGWC4 upload=KEQD7X1GBZ1C tag=QT5H1ZDN06IZ type=write size=4040648 2024-06-28T15:08:09.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.576+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~BLYQCAI67267.0 size=507677 2024-06-28T15:08:09.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.576+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=BLYQCAI67267 tag=LOOE9X20DA3A type=write size=1556253 2024-06-28T15:08:09.567 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.580+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~IVEIW6BB1LG8.0 size=722633 2024-06-28T15:08:09.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.584+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=IVEIW6BB1LG8 tag=RJTF19IHRA15 type=write size=1771209 2024-06-28T15:08:09.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.588+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=WT7Q3JQ441FU type=del size=79316 2024-06-28T15:08:09.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.592+0000 7f65680598c0 1 < suggested remove operation key=ZFNIA3O84EB7 tag=2H39RYV072Y2 type=del 2024-06-28T15:08:09.587 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.600+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~L2V6FOMMTAL7.0 size=462920 2024-06-28T15:08:09.587 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.600+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=L2V6FOMMTAL7 tag=GIKU5MWXBD5Y type=write size=1511496 2024-06-28T15:08:09.590 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.604+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=0XQV1I1NR76T type=del 2024-06-28T15:08:09.593 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.604+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=ZN5IEANO0D58 type=del size=1180170 2024-06-28T15:08:09.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.608+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=9Q6QJPU37GGA type=del 2024-06-28T15:08:09.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.612+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~VDGI4N4W8LAD.0 size=1048576 2024-06-28T15:08:09.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.616+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~VDGI4N4W8LAD.1 size=29471 2024-06-28T15:08:09.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.616+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=VDGI4N4W8LAD tag=C25X066G4HWU type=write size=2126623 2024-06-28T15:08:09.605 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.616+0000 7f65680598c0 -1 < failed to complete operation key=ZFNIA3O84EB7 tag=4MXRMR3E6KSG type=del size=964519: (22) Invalid argument 2024-06-28T15:08:09.610 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.624+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~VFBGYOLI0K4T.0 size=24991 2024-06-28T15:08:09.610 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.624+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=VFBGYOLI0K4T tag=S61Q3WVMQ8M4 type=write size=1073567 2024-06-28T15:08:09.613 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.624+0000 7f65680598c0 -1 < failed to complete operation key=F6SEDD6BH1T2 tag=0Z3JQRFQD584 type=del size=1575018: (22) Invalid argument 2024-06-28T15:08:09.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.632+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~P5ND03CPSPYU.0 size=1048576 2024-06-28T15:08:09.621 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.632+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~P5ND03CPSPYU.1 size=1048576 2024-06-28T15:08:09.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.636+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~P5ND03CPSPYU.2 size=901294 2024-06-28T15:08:09.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.636+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=P5ND03CPSPYU tag=886V39ADM3XK type=write size=4047022 2024-06-28T15:08:09.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.640+0000 7f65680598c0 1 < suggested update operation key=R0IMFVF5NEOO tag=TQ9Z1X1WN4YA type=del size=1195817 2024-06-28T15:08:09.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.644+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=BAFZUT3SAFGA type=del size=3661203 2024-06-28T15:08:09.637 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.648+0000 7f65680598c0 1 < suggested update operation key=60QVJVV16QFM tag=2D308AHMATSH type=write size=494876 2024-06-28T15:08:09.643 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.656+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~DANCE2PZBHX6.0 size=1048576 2024-06-28T15:08:09.645 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.656+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~DANCE2PZBHX6.1 size=1048576 2024-06-28T15:08:09.647 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.660+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~DANCE2PZBHX6.2 size=1043005 2024-06-28T15:08:09.647 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.660+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=DANCE2PZBHX6 tag=RTM7N7AZA7KY type=write size=4188733 2024-06-28T15:08:09.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.664+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~667MBX5ZHPYB.0 size=1048576 2024-06-28T15:08:09.653 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.664+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~667MBX5ZHPYB.1 size=1048576 2024-06-28T15:08:09.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.668+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~667MBX5ZHPYB.2 size=826951 2024-06-28T15:08:09.660 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.672+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=667MBX5ZHPYB tag=3NBRH7QAEMC5 type=write size=3972679 2024-06-28T15:08:09.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.680+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~GP3ZB9AU4R6B.0 size=580464 2024-06-28T15:08:09.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.680+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=GP3ZB9AU4R6B tag=LSKALEF3O0Y4 type=write size=1629040 2024-06-28T15:08:09.669 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.680+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=MTW5H3JQN4YY type=del 2024-06-28T15:08:09.672 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.684+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=WEVW9NQ8X1NZ type=write size=143611 2024-06-28T15:08:09.675 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.688+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~NPS7DHRIDKGS.0 size=1048576 2024-06-28T15:08:09.677 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.688+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~NPS7DHRIDKGS.1 size=1048576 2024-06-28T15:08:09.679 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.692+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~NPS7DHRIDKGS.2 size=574525 2024-06-28T15:08:09.685 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.696+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=NPS7DHRIDKGS tag=A74EVFHV4THL type=write size=3720253 2024-06-28T15:08:09.690 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.704+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~UTQ7FMU9K62O.0 size=1048576 2024-06-28T15:08:09.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.704+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~UTQ7FMU9K62O.1 size=851695 2024-06-28T15:08:09.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.704+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=UTQ7FMU9K62O tag=BQNCGXRPWUOL type=write size=2948847 2024-06-28T15:08:09.699 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.712+0000 7f65680598c0 1 < suggested remove operation key=4CU86XS1R9G3 tag=0SZRG3XUMFIA type=del 2024-06-28T15:08:09.701 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.716+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=MIO2G272PL49 type=del size=3230732 2024-06-28T15:08:09.704 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.716+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=DH9LQO4ZWUJN type=del 2024-06-28T15:08:09.706 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.720+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=YF5NU9C05KF6 type=del size=1046247 2024-06-28T15:08:09.712 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.724+0000 7f65680598c0 1 < suggested remove operation key=O1CGONN4TEXF tag=ITZOHYSHALSH type=del 2024-06-28T15:08:09.714 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.728+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=EV8E6T89L98X type=del size=1107243 2024-06-28T15:08:09.717 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.728+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~FI0EN2KNVCRY.0 size=1048576 2024-06-28T15:08:09.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.732+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~FI0EN2KNVCRY.1 size=1048576 2024-06-28T15:08:09.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.736+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~FI0EN2KNVCRY.2 size=244323 2024-06-28T15:08:09.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.740+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=FI0EN2KNVCRY tag=MD9KDZ3ST1N8 type=write size=3390051 2024-06-28T15:08:09.732 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.744+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~Z336HIIJIRXG.0 size=596441 2024-06-28T15:08:09.732 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.744+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=Z336HIIJIRXG tag=0RB75DG34HSG type=write size=1645017 2024-06-28T15:08:09.734 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.748+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~BLYQCAI67267.0 size=507677 2024-06-28T15:08:09.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.752+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=BLYQCAI67267 tag=LOOE9X20DA3A type=write size=1556253 2024-06-28T15:08:09.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.756+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=7G6YZFNELJBN type=del size=3875616 2024-06-28T15:08:09.744 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.756+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=WT7Q3JQ441FU type=del 2024-06-28T15:08:09.747 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.760+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=N4P7UAXWUCE9 type=write size=416916 2024-06-28T15:08:09.749 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.760+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~L2V6FOMMTAL7.0 size=462920 2024-06-28T15:08:09.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.768+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=L2V6FOMMTAL7 tag=GIKU5MWXBD5Y type=write size=1511496 2024-06-28T15:08:09.756 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.768+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=AR8FVZ5U1829 type=del size=1824408 2024-06-28T15:08:09.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.772+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=ZN5IEANO0D58 type=del 2024-06-28T15:08:09.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.772+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=V81DNILJJEL2 type=del size=2716569 2024-06-28T15:08:09.764 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.776+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~VDGI4N4W8LAD.0 size=1048576 2024-06-28T15:08:09.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.780+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~VDGI4N4W8LAD.1 size=29471 2024-06-28T15:08:09.770 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.784+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=VDGI4N4W8LAD tag=C25X066G4HWU type=write size=2126623 2024-06-28T15:08:09.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.788+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~0EGQ64A027Z3.0 size=1048576 2024-06-28T15:08:09.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.792+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~0EGQ64A027Z3.1 size=1048576 2024-06-28T15:08:09.787 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.800+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~0EGQ64A027Z3.2 size=363618 2024-06-28T15:08:09.788 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.800+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=0EGQ64A027Z3 tag=8ET08LKR5OBS type=write size=3509346 2024-06-28T15:08:09.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.808+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~VFBGYOLI0K4T.0 size=24991 2024-06-28T15:08:09.796 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.808+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=VFBGYOLI0K4T tag=S61Q3WVMQ8M4 type=write size=1073567 2024-06-28T15:08:09.798 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.812+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=0F4RG5GERT6F type=del size=3457090 2024-06-28T15:08:09.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.812+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~P5ND03CPSPYU.0 size=1048576 2024-06-28T15:08:09.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.816+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~P5ND03CPSPYU.1 size=1048576 2024-06-28T15:08:09.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.816+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~P5ND03CPSPYU.2 size=901294 2024-06-28T15:08:09.809 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.820+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=P5ND03CPSPYU tag=886V39ADM3XK type=write size=4047022 2024-06-28T15:08:09.813 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.824+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~JNIA0AVE1AAC.0 size=1048576 2024-06-28T15:08:09.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.828+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~JNIA0AVE1AAC.1 size=821234 2024-06-28T15:08:09.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.828+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=JNIA0AVE1AAC tag=12FS9K4SRPV0 type=write size=2918386 2024-06-28T15:08:09.817 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.828+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=BAFZUT3SAFGA type=del 2024-06-28T15:08:09.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.832+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=G3H7SB1YOX1O type=del size=3721494 2024-06-28T15:08:09.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.836+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~DANCE2PZBHX6.0 size=1048576 2024-06-28T15:08:09.824 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.836+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~DANCE2PZBHX6.1 size=1048576 2024-06-28T15:08:09.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.840+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~DANCE2PZBHX6.2 size=1043005 2024-06-28T15:08:09.830 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.844+0000 7f65680598c0 1 < completed multipart upload key=MQ3QWZBFN4YR upload=DANCE2PZBHX6 tag=RTM7N7AZA7KY type=write size=4188733 2024-06-28T15:08:09.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.848+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~GB89R0IDF03N.0 size=439298 2024-06-28T15:08:09.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.848+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=GB89R0IDF03N tag=2EUTDI62D3K3 type=write size=1487874 2024-06-28T15:08:09.836 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.848+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~GP3ZB9AU4R6B.0 size=580464 2024-06-28T15:08:09.842 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.856+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=GP3ZB9AU4R6B tag=LSKALEF3O0Y4 type=write size=1629040 2024-06-28T15:08:09.844 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.856+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=ILIYA97E14YT type=write size=680635 2024-06-28T15:08:09.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.860+0000 7f65680598c0 1 < canceled operation key=I4RWCQSQVLCN tag=WEVW9NQ8X1NZ type=write size=143611 2024-06-28T15:08:09.849 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.860+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=XXLW8C0PGR3V type=write size=769552 2024-06-28T15:08:09.851 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.864+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~UTQ7FMU9K62O.0 size=1048576 2024-06-28T15:08:09.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.864+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~UTQ7FMU9K62O.1 size=851695 2024-06-28T15:08:09.857 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.868+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=UTQ7FMU9K62O tag=BQNCGXRPWUOL type=write size=2948847 2024-06-28T15:08:09.859 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.872+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=GDYQSU4R3TOK type=del size=3288144 2024-06-28T15:08:09.864 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.876+0000 7f65680598c0 1 < suggested update operation key=SFCIZMLY2Y2E tag=MIO2G272PL49 type=del size=3230732 2024-06-28T15:08:09.866 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.880+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=QMCR8D4IXYLC type=del size=2772235 2024-06-28T15:08:09.868 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.880+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=YF5NU9C05KF6 type=del 2024-06-28T15:08:09.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.884+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=EXUMJIQA21IF type=write size=752826 2024-06-28T15:08:09.873 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.884+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=EV8E6T89L98X type=del 2024-06-28T15:08:09.875 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.888+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=166KCSX4ESRK type=del size=1171932 2024-06-28T15:08:09.878 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.892+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~Z336HIIJIRXG.0 size=596441 2024-06-28T15:08:09.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.896+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=Z336HIIJIRXG tag=0RB75DG34HSG type=write size=1645017 2024-06-28T15:08:09.884 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.896+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=0K84WJWRSSWT type=write size=789234 2024-06-28T15:08:09.886 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.900+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=7G6YZFNELJBN type=del 2024-06-28T15:08:09.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.900+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=0V4GARYWM0OA type=del size=2412797 2024-06-28T15:08:09.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.904+0000 7f65680598c0 1 < completed write operation key=47TO7G9EGJKF tag=N4P7UAXWUCE9 type=write size=416916 2024-06-28T15:08:09.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.908+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~UJ3X1OF2CWQN.0 size=226821 2024-06-28T15:08:09.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.908+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=UJ3X1OF2CWQN tag=Y2P4KQ3DHBQ8 type=write size=1275397 2024-06-28T15:08:09.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.912+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=AR8FVZ5U1829 type=del 2024-06-28T15:08:09.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.912+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=WTBPFC665N1U type=del size=1092529 2024-06-28T15:08:09.903 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.916+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=V81DNILJJEL2 type=del 2024-06-28T15:08:09.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.920+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~BO40ZIP9XXHP.0 size=41605 2024-06-28T15:08:09.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.920+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=BO40ZIP9XXHP tag=OSLIIENA1LGW type=write size=1090181 2024-06-28T15:08:09.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.924+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~0EGQ64A027Z3.0 size=1048576 2024-06-28T15:08:09.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.924+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~0EGQ64A027Z3.1 size=1048576 2024-06-28T15:08:09.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.928+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~0EGQ64A027Z3.2 size=363618 2024-06-28T15:08:09.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.932+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=0EGQ64A027Z3 tag=8ET08LKR5OBS type=write size=3509346 2024-06-28T15:08:09.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.932+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=7A8OJ7M100UE type=write size=775430 2024-06-28T15:08:09.926 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.940+0000 7f65680598c0 1 < suggested update operation key=I4RWCQSQVLCN tag=0F4RG5GERT6F type=del size=3457090 2024-06-28T15:08:09.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.940+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=H1OG8BM80LNG type=del size=1848706 2024-06-28T15:08:09.931 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.944+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~JNIA0AVE1AAC.0 size=1048576 2024-06-28T15:08:09.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.944+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~JNIA0AVE1AAC.1 size=821234 2024-06-28T15:08:09.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.948+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=JNIA0AVE1AAC tag=12FS9K4SRPV0 type=write size=2918386 2024-06-28T15:08:09.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.956+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~O2PIP8SE4FE1.0 size=1048576 2024-06-28T15:08:09.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.956+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~O2PIP8SE4FE1.1 size=634714 2024-06-28T15:08:09.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.956+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=O2PIP8SE4FE1 tag=219479J65KJ5 type=write size=2731866 2024-06-28T15:08:09.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.960+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=G3H7SB1YOX1O type=del 2024-06-28T15:08:09.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.964+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~VOBOSPZWIYIW.0 size=1048576 2024-06-28T15:08:09.952 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.964+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~VOBOSPZWIYIW.1 size=748167 2024-06-28T15:08:09.952 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.964+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=VOBOSPZWIYIW tag=4O92WC1XVJNY type=write size=2845319 2024-06-28T15:08:09.955 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.968+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~GB89R0IDF03N.0 size=439298 2024-06-28T15:08:09.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.972+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=GB89R0IDF03N tag=2EUTDI62D3K3 type=write size=1487874 2024-06-28T15:08:09.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.972+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=LJYMIRJFRZDW type=del size=1039624 2024-06-28T15:08:09.964 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.976+0000 7f65680598c0 1 < completed write operation key=ZY973GBGBQ1Z tag=ILIYA97E14YT type=write size=680635 2024-06-28T15:08:09.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.980+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=DSQ3LX0S06Y1 type=del size=870348 2024-06-28T15:08:09.969 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.980+0000 7f65680598c0 1 < completed write operation key=ZY973GBGBQ1Z tag=XXLW8C0PGR3V type=write size=769552 2024-06-28T15:08:09.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.988+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~6PW0Z23X7UF3.0 size=1048576 2024-06-28T15:08:09.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.988+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~6PW0Z23X7UF3.1 size=429878 2024-06-28T15:08:09.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.988+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=6PW0Z23X7UF3 tag=37IHB8RGLVWO type=write size=2527030 2024-06-28T15:08:09.981 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.992+0000 7f65680598c0 1 < suggested remove operation key=MQ3QWZBFN4YR tag=GDYQSU4R3TOK type=del 2024-06-28T15:08:09.983 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:09.996+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=HWBJTHHUK5VU type=del size=15434 2024-06-28T15:08:09.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.000+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=QMCR8D4IXYLC type=del 2024-06-28T15:08:09.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.000+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=RO2ZP240U0QO type=del size=1179312 2024-06-28T15:08:09.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.004+0000 7f65680598c0 1 < completed write operation key=2FC9DNEZGWC4 tag=EXUMJIQA21IF type=write size=752826 2024-06-28T15:08:09.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.004+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=OQ5MZKNUWRJY type=del size=735812 2024-06-28T15:08:09.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.008+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=166KCSX4ESRK type=del 2024-06-28T15:08:09.998 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.012+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=0AJBQDZ0BHCD type=del size=631637 2024-06-28T15:08:10.000 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.012+0000 7f65680598c0 1 < canceled operation key=ZY973GBGBQ1Z tag=0K84WJWRSSWT type=write size=789234 2024-06-28T15:08:10.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.016+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=V5Q5SP7C37QF type=del size=1459 2024-06-28T15:08:10.004 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.016+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=0V4GARYWM0OA type=del 2024-06-28T15:08:10.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.020+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=NW0YJRQTCTRM type=del size=2162687 2024-06-28T15:08:10.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.024+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~UJ3X1OF2CWQN.0 size=226821 2024-06-28T15:08:10.014 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.028+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=UJ3X1OF2CWQN tag=Y2P4KQ3DHBQ8 type=write size=1275397 2024-06-28T15:08:10.016 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.028+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=L16BI1ZT1WPC type=del size=322008 2024-06-28T15:08:10.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.032+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=WTBPFC665N1U type=del 2024-06-28T15:08:10.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.032+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=7HSTJI6N92X5 type=del size=1194691 2024-06-28T15:08:10.023 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.036+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~BO40ZIP9XXHP.0 size=41605 2024-06-28T15:08:10.026 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.040+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=BO40ZIP9XXHP tag=OSLIIENA1LGW type=write size=1090181 2024-06-28T15:08:10.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.040+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=WQFFDZ5NCDXJ type=del size=1788108 2024-06-28T15:08:10.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.044+0000 7f65680598c0 1 < suggested update operation key=60QVJVV16QFM tag=7A8OJ7M100UE type=write size=775430 2024-06-28T15:08:10.036 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.048+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~RAY4BKS9Y949.0 size=1048576 2024-06-28T15:08:10.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.052+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~RAY4BKS9Y949.1 size=1048576 2024-06-28T15:08:10.039 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.052+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~RAY4BKS9Y949.2 size=971127 2024-06-28T15:08:10.039 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.052+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=RAY4BKS9Y949 tag=6AFUP1XXIOTK type=write size=4116855 2024-06-28T15:08:10.041 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.052+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=H1OG8BM80LNG type=del 2024-06-28T15:08:10.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.056+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=L9XA3L1NN8IT type=del size=4046742 2024-06-28T15:08:10.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.056+0000 7f65680598c0 1 < canceled part key=_multipart_1VD7NDPEM3CD.2~O2PIP8SE4FE1.0 size=1048576 2024-06-28T15:08:10.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.060+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~O2PIP8SE4FE1.1 size=634714 2024-06-28T15:08:10.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.060+0000 7f65680598c0 1 < canceled multipart upload key=1VD7NDPEM3CD upload=O2PIP8SE4FE1 tag=219479J65KJ5 type=write size=2731866 2024-06-28T15:08:10.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.064+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=SESVOAAF06CG type=del size=2154567 2024-06-28T15:08:10.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.068+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~VOBOSPZWIYIW.0 size=1048576 2024-06-28T15:08:10.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.068+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~VOBOSPZWIYIW.1 size=748167 2024-06-28T15:08:10.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.072+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=VOBOSPZWIYIW tag=4O92WC1XVJNY type=write size=2845319 2024-06-28T15:08:10.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.076+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~6UO2BSMUFWDD.0 size=1046019 2024-06-28T15:08:10.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.076+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=6UO2BSMUFWDD tag=I3QEZZXN0SW7 type=write size=2094595 2024-06-28T15:08:10.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.080+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=LJYMIRJFRZDW type=del 2024-06-28T15:08:10.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.084+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~05ESK6HCAHT0.0 size=1048576 2024-06-28T15:08:10.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.084+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~05ESK6HCAHT0.1 size=1048576 2024-06-28T15:08:10.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.084+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~05ESK6HCAHT0.2 size=1021404 2024-06-28T15:08:10.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.084+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=05ESK6HCAHT0 tag=4RTIIK7G7VJM type=write size=4167132 2024-06-28T15:08:10.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.088+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=DSQ3LX0S06Y1 type=del 2024-06-28T15:08:10.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.088+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=7G8122SKWC0D type=del size=1644883 2024-06-28T15:08:10.079 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.092+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~6PW0Z23X7UF3.0 size=1048576 2024-06-28T15:08:10.080 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.092+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~6PW0Z23X7UF3.1 size=429878 2024-06-28T15:08:10.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.096+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=6PW0Z23X7UF3 tag=37IHB8RGLVWO type=write size=2527030 2024-06-28T15:08:10.087 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.100+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~HUJTSUJOJY49.0 size=436095 2024-06-28T15:08:10.087 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.100+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=HUJTSUJOJY49 tag=W2B0HLD5A46J type=write size=1484671 2024-06-28T15:08:10.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.104+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=HWBJTHHUK5VU type=del 2024-06-28T15:08:10.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.104+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=PVY3ELI38JNB type=del size=1811912 2024-06-28T15:08:10.093 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.108+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=RO2ZP240U0QO type=del 2024-06-28T15:08:10.096 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.108+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=AD52VFSSFUGN type=del size=2256782 2024-06-28T15:08:10.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.112+0000 7f65680598c0 -1 < failed to complete operation key=60QVJVV16QFM tag=OQ5MZKNUWRJY type=del size=735812: (22) Invalid argument 2024-06-28T15:08:10.101 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.112+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=76T22YMGRMOV type=write size=1043263 2024-06-28T15:08:10.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.120+0000 7f65680598c0 1 < suggested remove operation key=ZFNIA3O84EB7 tag=0AJBQDZ0BHCD type=del 2024-06-28T15:08:10.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.120+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=MM5R0LII4YP4 type=del size=3283878 2024-06-28T15:08:10.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.124+0000 7f65680598c0 1 < suggested update operation key=2F968YIMJN9D tag=V5Q5SP7C37QF type=del size=1459 2024-06-28T15:08:10.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.132+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~UFAJPOA12Z1H.0 size=580915 2024-06-28T15:08:10.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.132+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=UFAJPOA12Z1H tag=QR3LZXTPZV2O type=write size=1629491 2024-06-28T15:08:10.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.132+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=NW0YJRQTCTRM type=del 2024-06-28T15:08:10.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.136+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~78BH857KC531.0 size=1048576 2024-06-28T15:08:10.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.140+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~78BH857KC531.1 size=1048576 2024-06-28T15:08:10.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.140+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~78BH857KC531.2 size=934399 2024-06-28T15:08:10.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.140+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=78BH857KC531 tag=JXQQ95CWJGB5 type=write size=4080127 2024-06-28T15:08:10.131 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.144+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=L16BI1ZT1WPC type=del 2024-06-28T15:08:10.136 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.148+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~1UC3CB1EE096.0 size=1048576 2024-06-28T15:08:10.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.152+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~1UC3CB1EE096.1 size=433263 2024-06-28T15:08:10.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.152+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=1UC3CB1EE096 tag=WYETSUT7BM0O type=write size=2530415 2024-06-28T15:08:10.141 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.152+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=7HSTJI6N92X5 type=del 2024-06-28T15:08:10.143 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.156+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=9857WXMKC9KZ type=del size=2698285 2024-06-28T15:08:10.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.160+0000 7f65680598c0 1 < suggested update operation key=1VD7NDPEM3CD tag=WQFFDZ5NCDXJ type=del size=1788108 2024-06-28T15:08:10.152 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.164+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~8Q6IZO6T2V0A.0 size=1048576 2024-06-28T15:08:10.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.168+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~8Q6IZO6T2V0A.1 size=402445 2024-06-28T15:08:10.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.168+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=8Q6IZO6T2V0A tag=X8GOE5E9SH7M type=write size=2499597 2024-06-28T15:08:10.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.168+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~RAY4BKS9Y949.0 size=1048576 2024-06-28T15:08:10.158 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.172+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~RAY4BKS9Y949.1 size=1048576 2024-06-28T15:08:10.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.172+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~RAY4BKS9Y949.2 size=971127 2024-06-28T15:08:10.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.176+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=RAY4BKS9Y949 tag=6AFUP1XXIOTK type=write size=4116855 2024-06-28T15:08:10.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.180+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=1KCOCXTF1U86 type=del size=4109272 2024-06-28T15:08:10.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.184+0000 7f65680598c0 1 < suggested update operation key=2F968YIMJN9D tag=L9XA3L1NN8IT type=del size=4046742 2024-06-28T15:08:10.176 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.188+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~G4QOU36ONJDZ.0 size=320519 2024-06-28T15:08:10.176 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.188+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=G4QOU36ONJDZ tag=0M7H6Y2256VJ type=write size=1369095 2024-06-28T15:08:10.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.192+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=SESVOAAF06CG type=del 2024-06-28T15:08:10.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.192+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=NR2KQLLROJ1G type=del size=585082 2024-06-28T15:08:10.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.196+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~6UO2BSMUFWDD.0 size=1046019 2024-06-28T15:08:10.188 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.200+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=6UO2BSMUFWDD tag=I3QEZZXN0SW7 type=write size=2094595 2024-06-28T15:08:10.190 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.204+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=8X2SLAXK22VD type=del size=989438 2024-06-28T15:08:10.191 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.204+0000 7f65680598c0 1 < canceled part key=_multipart_BHW2VHUT0U1U.2~05ESK6HCAHT0.0 size=1048576 2024-06-28T15:08:10.193 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.204+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~05ESK6HCAHT0.1 size=1048576 2024-06-28T15:08:10.194 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.208+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~05ESK6HCAHT0.2 size=1021404 2024-06-28T15:08:10.199 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.212+0000 7f65680598c0 1 < canceled multipart upload key=BHW2VHUT0U1U upload=05ESK6HCAHT0 tag=4RTIIK7G7VJM type=write size=4167132 2024-06-28T15:08:10.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.212+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=S4I6PMWFHY7G type=del size=2088306 2024-06-28T15:08:10.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.216+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=7G8122SKWC0D type=del 2024-06-28T15:08:10.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.220+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=0N7O5PAQ3RB5 type=write size=297074 2024-06-28T15:08:10.207 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.220+0000 7f65680598c0 1 < canceled part key=_multipart_GAZQ3PCPKN9Q.2~HUJTSUJOJY49.0 size=436095 2024-06-28T15:08:10.210 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.224+0000 7f65680598c0 1 < canceled multipart upload key=GAZQ3PCPKN9Q upload=HUJTSUJOJY49 tag=W2B0HLD5A46J type=write size=1484671 2024-06-28T15:08:10.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.224+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=ZC7XW26LVQVB type=del size=1817862 2024-06-28T15:08:10.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.228+0000 7f65680598c0 -1 < no bi entry to suggest for operation key=ZFNIA3O84EB7 tag=PVY3ELI38JNB type=del size=1811912: (2) No such file or directory 2024-06-28T15:08:10.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.228+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=HCQ7QRGGUITI type=del size=756258 2024-06-28T15:08:10.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.232+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=AD52VFSSFUGN type=del 2024-06-28T15:08:10.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.236+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~LHY4UTXCABMG.0 size=1004803 2024-06-28T15:08:10.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.236+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=LHY4UTXCABMG tag=FIGHF0RYVTDZ type=write size=2053379 2024-06-28T15:08:10.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.236+0000 7f65680598c0 1 < canceled operation key=D9D3OFIJP3D3 tag=76T22YMGRMOV type=write size=1043263 2024-06-28T15:08:10.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.240+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=KDQ2PFW986LV type=del size=912518 2024-06-28T15:08:10.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.240+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=MM5R0LII4YP4 type=del 2024-06-28T15:08:10.233 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.244+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~A431YEYOEF0C.0 size=778896 2024-06-28T15:08:10.233 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.244+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=A431YEYOEF0C tag=485SIN990J49 type=write size=1827472 2024-06-28T15:08:10.235 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.248+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~UFAJPOA12Z1H.0 size=580915 2024-06-28T15:08:10.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.252+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=UFAJPOA12Z1H tag=QR3LZXTPZV2O type=write size=1629491 2024-06-28T15:08:10.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.256+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=WDIC74S38915 type=write size=158519 2024-06-28T15:08:10.244 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.256+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~78BH857KC531.0 size=1048576 2024-06-28T15:08:10.244 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.256+0000 7f65680598c0 1 < canceled part key=_multipart_F6SEDD6BH1T2.2~78BH857KC531.1 size=1048576 2024-06-28T15:08:10.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.260+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~78BH857KC531.2 size=934399 2024-06-28T15:08:10.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.264+0000 7f65680598c0 1 < canceled multipart upload key=F6SEDD6BH1T2 upload=78BH857KC531 tag=JXQQ95CWJGB5 type=write size=4080127 2024-06-28T15:08:10.254 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.268+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~PKKAE4F4QEIJ.0 size=1048576 2024-06-28T15:08:10.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.268+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~PKKAE4F4QEIJ.1 size=100470 2024-06-28T15:08:10.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.268+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=PKKAE4F4QEIJ tag=9ZYRMNG12NBH type=write size=2197622 2024-06-28T15:08:10.258 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.272+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~1UC3CB1EE096.0 size=1048576 2024-06-28T15:08:10.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.272+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~1UC3CB1EE096.1 size=433263 2024-06-28T15:08:10.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.276+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=1UC3CB1EE096 tag=WYETSUT7BM0O type=write size=2530415 2024-06-28T15:08:10.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.280+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=MMAQJG002N33 type=del size=1191938 2024-06-28T15:08:10.268 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.280+0000 7f65680598c0 -1 < failed to complete operation key=2F968YIMJN9D tag=9857WXMKC9KZ type=del size=2698285: (22) Invalid argument 2024-06-28T15:08:10.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.284+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~MGYIDASUHPH8.0 size=338506 2024-06-28T15:08:10.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.284+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=MGYIDASUHPH8 tag=U3G99ZPWCNLN type=write size=1387082 2024-06-28T15:08:10.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.288+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~8Q6IZO6T2V0A.0 size=1048576 2024-06-28T15:08:10.276 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.288+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~8Q6IZO6T2V0A.1 size=402445 2024-06-28T15:08:10.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.292+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=8Q6IZO6T2V0A tag=X8GOE5E9SH7M type=write size=2499597 2024-06-28T15:08:10.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.296+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~PYG6A9BV8KKW.0 size=1048576 2024-06-28T15:08:10.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.300+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~PYG6A9BV8KKW.1 size=636442 2024-06-28T15:08:10.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.300+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=PYG6A9BV8KKW tag=PQDAK646TU02 type=write size=2733594 2024-06-28T15:08:10.289 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.300+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=1KCOCXTF1U86 type=del 2024-06-28T15:08:10.291 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.304+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=I212ANNEJWST type=del size=913613 2024-06-28T15:08:10.293 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.304+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~G4QOU36ONJDZ.0 size=320519 2024-06-28T15:08:10.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.308+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=G4QOU36ONJDZ tag=0M7H6Y2256VJ type=write size=1369095 2024-06-28T15:08:10.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.312+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=EGAW0JID1UK4 type=del size=238481 2024-06-28T15:08:10.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.316+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=NR2KQLLROJ1G type=del 2024-06-28T15:08:10.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.316+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=T3SII3P5UB86 type=del size=1323960 2024-06-28T15:08:10.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.320+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=8X2SLAXK22VD type=del 2024-06-28T15:08:10.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.320+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=F0GQK6JYPJBK type=del size=3476900 2024-06-28T15:08:10.311 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.324+0000 7f65680598c0 1 < canceled operation key=F6SEDD6BH1T2 tag=S4I6PMWFHY7G type=del size=2088306 2024-06-28T15:08:10.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.328+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~X2PI31IDLMA1.0 size=1048576 2024-06-28T15:08:10.316 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.328+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~X2PI31IDLMA1.1 size=1048576 2024-06-28T15:08:10.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.332+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~X2PI31IDLMA1.2 size=950339 2024-06-28T15:08:10.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.332+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=X2PI31IDLMA1 tag=UX0MA1BBHDCY type=write size=4096067 2024-06-28T15:08:10.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.336+0000 7f65680598c0 1 < suggested remove operation key=KFHGA64L1NLP tag=0N7O5PAQ3RB5 type=write 2024-06-28T15:08:10.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.336+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=62RN4E8BL94N type=del size=1247531 2024-06-28T15:08:10.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.340+0000 7f65680598c0 1 < suggested remove operation key=1VD7NDPEM3CD tag=ZC7XW26LVQVB type=del 2024-06-28T15:08:10.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.348+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~B7SYLV100A07.0 size=1048576 2024-06-28T15:08:10.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.348+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~B7SYLV100A07.1 size=679984 2024-06-28T15:08:10.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.348+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=B7SYLV100A07 tag=Y9Z4EWR37IEF type=write size=2777136 2024-06-28T15:08:10.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.352+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=HCQ7QRGGUITI type=del 2024-06-28T15:08:10.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.356+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~1OJQAPVVSJIB.0 size=493940 2024-06-28T15:08:10.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.356+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=1OJQAPVVSJIB tag=P3ZWIXCLQTGQ type=write size=1542516 2024-06-28T15:08:10.346 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.360+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~LHY4UTXCABMG.0 size=1004803 2024-06-28T15:08:10.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.364+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=LHY4UTXCABMG tag=FIGHF0RYVTDZ type=write size=2053379 2024-06-28T15:08:10.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.368+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~BEMMTMIARX26.0 size=1048576 2024-06-28T15:08:10.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.372+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~BEMMTMIARX26.1 size=1048576 2024-06-28T15:08:10.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.372+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~BEMMTMIARX26.2 size=659634 2024-06-28T15:08:10.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.372+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=BEMMTMIARX26 tag=V5NKHT6J35ZI type=write size=3805362 2024-06-28T15:08:10.363 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.376+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=KDQ2PFW986LV type=del 2024-06-28T15:08:10.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.380+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=I2MFYPIB9E66 type=del size=1943507 2024-06-28T15:08:10.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.380+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~A431YEYOEF0C.0 size=778896 2024-06-28T15:08:10.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.388+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=A431YEYOEF0C tag=485SIN990J49 type=write size=1827472 2024-06-28T15:08:10.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.388+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=KE7DNDBOCOAE type=write size=812554 2024-06-28T15:08:10.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.392+0000 7f65680598c0 1 < completed write operation key=4CU86XS1R9G3 tag=WDIC74S38915 type=write size=158519 2024-06-28T15:08:10.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.396+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=XSBRSYI6XQ5W type=del size=123813 2024-06-28T15:08:10.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.400+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~PKKAE4F4QEIJ.0 size=1048576 2024-06-28T15:08:10.388 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.400+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~PKKAE4F4QEIJ.1 size=100470 2024-06-28T15:08:10.393 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.404+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=PKKAE4F4QEIJ tag=9ZYRMNG12NBH type=write size=2197622 2024-06-28T15:08:10.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.408+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=TQWFVV3FKBSN type=del size=2555808 2024-06-28T15:08:10.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.412+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=MMAQJG002N33 type=del 2024-06-28T15:08:10.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.416+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=NXKQ6AS4RKM7 type=del size=507865 2024-06-28T15:08:10.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.416+0000 7f65680598c0 1 < canceled part key=_multipart_7NL3CTFYY9E5.2~MGYIDASUHPH8.0 size=338506 2024-06-28T15:08:10.408 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.420+0000 7f65680598c0 1 < canceled multipart upload key=7NL3CTFYY9E5 upload=MGYIDASUHPH8 tag=U3G99ZPWCNLN type=write size=1387082 2024-06-28T15:08:10.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.424+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=YO99ALSWGDTT type=write size=158387 2024-06-28T15:08:10.413 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.424+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~PYG6A9BV8KKW.0 size=1048576 2024-06-28T15:08:10.413 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.424+0000 7f65680598c0 1 < canceled part key=_multipart_P278ZM1H052S.2~PYG6A9BV8KKW.1 size=636442 2024-06-28T15:08:10.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.428+0000 7f65680598c0 1 < canceled multipart upload key=P278ZM1H052S upload=PYG6A9BV8KKW tag=PQDAK646TU02 type=write size=2733594 2024-06-28T15:08:10.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.432+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=RL8M55XQ1NG6 type=del size=1355287 2024-06-28T15:08:10.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.436+0000 7f65680598c0 1 < suggested update operation key=EN76QQWIQAXZ tag=I212ANNEJWST type=del size=913613 2024-06-28T15:08:10.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.440+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=7V63OBVMTA2P type=del size=2292605 2024-06-28T15:08:10.430 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.444+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=EGAW0JID1UK4 type=del 2024-06-28T15:08:10.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.448+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~49ZPMUL7N79M.0 size=1048576 2024-06-28T15:08:10.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.452+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~49ZPMUL7N79M.1 size=297885 2024-06-28T15:08:10.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.452+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=49ZPMUL7N79M tag=HYRK33JAX2SM type=write size=2395037 2024-06-28T15:08:10.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.452+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=T3SII3P5UB86 type=del 2024-06-28T15:08:10.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.456+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=D3ZGCSGFOJC8 type=write size=13189 2024-06-28T15:08:10.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.456+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=F0GQK6JYPJBK type=del 2024-06-28T15:08:10.451 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.464+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~KG1D07LQFVER.0 size=1048576 2024-06-28T15:08:10.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.464+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~KG1D07LQFVER.1 size=1048576 2024-06-28T15:08:10.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.468+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~KG1D07LQFVER.2 size=199707 2024-06-28T15:08:10.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.468+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=KG1D07LQFVER tag=KIYGGRRI2RP2 type=write size=3345435 2024-06-28T15:08:10.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.468+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~X2PI31IDLMA1.0 size=1048576 2024-06-28T15:08:10.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.472+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~X2PI31IDLMA1.1 size=1048576 2024-06-28T15:08:10.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.472+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~X2PI31IDLMA1.2 size=950339 2024-06-28T15:08:10.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.476+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=X2PI31IDLMA1 tag=UX0MA1BBHDCY type=write size=4096067 2024-06-28T15:08:10.466 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.480+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=AEGK8JUZOCDG type=del size=2817644 2024-06-28T15:08:10.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.480+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=62RN4E8BL94N type=del 2024-06-28T15:08:10.473 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.484+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~1RA59IQ1CY9M.0 size=1048576 2024-06-28T15:08:10.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.488+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~1RA59IQ1CY9M.1 size=743009 2024-06-28T15:08:10.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.488+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=1RA59IQ1CY9M tag=5NAHUYZ4CJST type=write size=2840161 2024-06-28T15:08:10.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.488+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~B7SYLV100A07.0 size=1048576 2024-06-28T15:08:10.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.492+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~B7SYLV100A07.1 size=679984 2024-06-28T15:08:10.483 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.496+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=B7SYLV100A07 tag=Y9Z4EWR37IEF type=write size=2777136 2024-06-28T15:08:10.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.500+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=BY1IAGS1TG0O type=write size=358283 2024-06-28T15:08:10.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.500+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~1OJQAPVVSJIB.0 size=493940 2024-06-28T15:08:10.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.504+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=1OJQAPVVSJIB tag=P3ZWIXCLQTGQ type=write size=1542516 2024-06-28T15:08:10.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.508+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=B4XFNHYWSAPO type=del size=2571971 2024-06-28T15:08:10.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.508+0000 7f65680598c0 1 < canceled part key=_multipart_7LB8MOJL39RM.2~BEMMTMIARX26.0 size=1048576 2024-06-28T15:08:10.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.508+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~BEMMTMIARX26.1 size=1048576 2024-06-28T15:08:10.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.512+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~BEMMTMIARX26.2 size=659634 2024-06-28T15:08:10.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.516+0000 7f65680598c0 1 < canceled multipart upload key=7LB8MOJL39RM upload=BEMMTMIARX26 tag=V5NKHT6J35ZI type=write size=3805362 2024-06-28T15:08:10.505 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.520+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=4MRJOG2PPPN2 type=del size=2293324 2024-06-28T15:08:10.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.520+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=I2MFYPIB9E66 type=del 2024-06-28T15:08:10.511 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.524+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=E4OMHTVCK28J type=del size=4058962 2024-06-28T15:08:10.514 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.528+0000 7f65680598c0 1 < completed write operation key=4CU86XS1R9G3 tag=KE7DNDBOCOAE type=write size=812554 2024-06-28T15:08:10.518 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.532+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=D4MDP0TRFHKB type=del size=772995 2024-06-28T15:08:10.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.536+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=XSBRSYI6XQ5W type=del 2024-06-28T15:08:10.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.540+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~5LLZ46SROLWV.0 size=1048576 2024-06-28T15:08:10.529 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.540+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~5LLZ46SROLWV.1 size=75802 2024-06-28T15:08:10.529 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.540+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=5LLZ46SROLWV tag=PA5YKFCB42YT type=write size=2172954 2024-06-28T15:08:10.534 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.548+0000 7f65680598c0 1 < suggested remove operation key=LZX5UZCYLTAG tag=TQWFVV3FKBSN type=del 2024-06-28T15:08:10.537 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.548+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=OIV4883U34QD type=del size=3986536 2024-06-28T15:08:10.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.556+0000 7f65680598c0 1 < suggested remove operation key=47TO7G9EGJKF tag=NXKQ6AS4RKM7 type=del 2024-06-28T15:08:10.545 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.556+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=29NXX6ID8TBQ type=del size=2327427 2024-06-28T15:08:10.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.560+0000 7f65680598c0 1 < canceled operation key=J311Q2712YM0 tag=YO99ALSWGDTT type=write size=158387 2024-06-28T15:08:10.554 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.568+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~HWKMY4PCBDT3.0 size=1048576 2024-06-28T15:08:10.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.568+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~HWKMY4PCBDT3.1 size=878058 2024-06-28T15:08:10.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.568+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=HWKMY4PCBDT3 tag=SRGHETFZDOF5 type=write size=2975210 2024-06-28T15:08:10.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.572+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=RL8M55XQ1NG6 type=del 2024-06-28T15:08:10.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.576+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=RPELZ6I4ZHVW type=del size=912989 2024-06-28T15:08:10.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.580+0000 7f65680598c0 1 < suggested remove operation key=R0IMFVF5NEOO tag=7V63OBVMTA2P type=del 2024-06-28T15:08:10.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.584+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=N57IBMV3L1F5 type=del size=112671 2024-06-28T15:08:10.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.588+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~49ZPMUL7N79M.0 size=1048576 2024-06-28T15:08:10.577 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.588+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~49ZPMUL7N79M.1 size=297885 2024-06-28T15:08:10.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.596+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=49ZPMUL7N79M tag=HYRK33JAX2SM type=write size=2395037 2024-06-28T15:08:10.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.596+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=QQHG45VAEW77 type=del size=3344792 2024-06-28T15:08:10.588 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.600+0000 7f65680598c0 1 < completed write operation key=SFCIZMLY2Y2E tag=D3ZGCSGFOJC8 type=write size=13189 2024-06-28T15:08:10.591 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.604+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=YOE0AMC7I8V0 type=del size=1230033 2024-06-28T15:08:10.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.608+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~KG1D07LQFVER.0 size=1048576 2024-06-28T15:08:10.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.608+0000 7f65680598c0 1 < canceled part key=_multipart_R0IMFVF5NEOO.2~KG1D07LQFVER.1 size=1048576 2024-06-28T15:08:10.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.608+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~KG1D07LQFVER.2 size=199707 2024-06-28T15:08:10.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.616+0000 7f65680598c0 1 < canceled multipart upload key=R0IMFVF5NEOO upload=KG1D07LQFVER tag=KIYGGRRI2RP2 type=write size=3345435 2024-06-28T15:08:10.605 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.616+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=539NSJB80AW4 type=write size=61333 2024-06-28T15:08:10.610 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.624+0000 7f65680598c0 1 < suggested remove operation key=7NL3CTFYY9E5 tag=AEGK8JUZOCDG type=del 2024-06-28T15:08:10.613 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.628+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=RL6KJMTOYKHE type=del size=325258 2024-06-28T15:08:10.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.628+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~1RA59IQ1CY9M.0 size=1048576 2024-06-28T15:08:10.621 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.632+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~1RA59IQ1CY9M.1 size=743009 2024-06-28T15:08:10.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.636+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=1RA59IQ1CY9M tag=5NAHUYZ4CJST type=write size=2840161 2024-06-28T15:08:10.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.644+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~8WT8OFVANVR0.0 size=1048576 2024-06-28T15:08:10.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.644+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~8WT8OFVANVR0.1 size=867239 2024-06-28T15:08:10.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.644+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=8WT8OFVANVR0 tag=BGUWK87V99E8 type=write size=2964391 2024-06-28T15:08:10.636 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.648+0000 7f65680598c0 -1 < failed to complete operation key=7NL3CTFYY9E5 tag=BY1IAGS1TG0O type=write size=358283: (22) Invalid argument 2024-06-28T15:08:10.641 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.652+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~XA5KX4AMY6TY.0 size=1048576 2024-06-28T15:08:10.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.656+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~XA5KX4AMY6TY.1 size=209790 2024-06-28T15:08:10.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.656+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=XA5KX4AMY6TY tag=LBSMU0V95ZH3 type=write size=2306942 2024-06-28T15:08:10.647 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.660+0000 7f65680598c0 1 < canceled operation key=60QVJVV16QFM tag=B4XFNHYWSAPO type=del size=2571971 2024-06-28T15:08:10.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.664+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=01DDYMPDV642 type=del size=3947183 2024-06-28T15:08:10.653 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.664+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=4MRJOG2PPPN2 type=del 2024-06-28T15:08:10.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.672+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~ZMOKI5YVDIUZ.0 size=999214 2024-06-28T15:08:10.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.672+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=ZMOKI5YVDIUZ tag=O4BK40K6PD84 type=write size=2047790 2024-06-28T15:08:10.662 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.676+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=E4OMHTVCK28J type=del 2024-06-28T15:08:10.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.676+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=78M4EQ08W2IQ type=write size=866000 2024-06-28T15:08:10.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.680+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=D4MDP0TRFHKB type=del 2024-06-28T15:08:10.673 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.688+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~F6MKEPHF8ES9.0 size=1048576 2024-06-28T15:08:10.676 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.688+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~F6MKEPHF8ES9.1 size=403969 2024-06-28T15:08:10.676 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.688+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=F6MKEPHF8ES9 tag=7F3947VC5ANJ type=write size=2501121 2024-06-28T15:08:10.679 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.692+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~5LLZ46SROLWV.0 size=1048576 2024-06-28T15:08:10.681 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.692+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~5LLZ46SROLWV.1 size=75802 2024-06-28T15:08:10.686 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.700+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=5LLZ46SROLWV tag=PA5YKFCB42YT type=write size=2172954 2024-06-28T15:08:10.689 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.700+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=5FYGHO6FO85J type=del size=1044972 2024-06-28T15:08:10.690 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.704+0000 7f65680598c0 -1 < no bi entry to suggest for operation key=7NL3CTFYY9E5 tag=OIV4883U34QD type=del size=3986536: (2) No such file or directory 2024-06-28T15:08:10.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.704+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=EAB2KBV6LLPL type=write size=923130 2024-06-28T15:08:10.696 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.708+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=29NXX6ID8TBQ type=del 2024-06-28T15:08:10.702 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.716+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~81FUV712BESQ.0 size=1048576 2024-06-28T15:08:10.704 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.716+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~81FUV712BESQ.1 size=1048576 2024-06-28T15:08:10.707 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.720+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~81FUV712BESQ.2 size=22127 2024-06-28T15:08:10.707 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.720+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=81FUV712BESQ tag=IO7JXHOB5991 type=write size=3167855 2024-06-28T15:08:10.710 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.724+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~HWKMY4PCBDT3.0 size=1048576 2024-06-28T15:08:10.713 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.724+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~HWKMY4PCBDT3.1 size=878058 2024-06-28T15:08:10.718 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.732+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=HWKMY4PCBDT3 tag=SRGHETFZDOF5 type=write size=2975210 2024-06-28T15:08:10.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.736+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=FT2UQDJDUZD1 type=del size=3575367 2024-06-28T15:08:10.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.740+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=RPELZ6I4ZHVW type=del 2024-06-28T15:08:10.732 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.744+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~189Z0W4FPW9E.0 size=1048576 2024-06-28T15:08:10.735 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.748+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~189Z0W4FPW9E.1 size=937423 2024-06-28T15:08:10.735 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.748+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=189Z0W4FPW9E tag=PGOHJTCVHOLH type=write size=3034575 2024-06-28T15:08:10.738 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.752+0000 7f65680598c0 1 < canceled operation key=GAZQ3PCPKN9Q tag=N57IBMV3L1F5 type=del size=112671 2024-06-28T15:08:10.744 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.756+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~QXLCLWD3DFKI.0 size=1048576 2024-06-28T15:08:10.747 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.760+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~QXLCLWD3DFKI.1 size=972869 2024-06-28T15:08:10.747 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.760+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=QXLCLWD3DFKI tag=DVCJCGVQ6XKA type=write size=3070021 2024-06-28T15:08:10.750 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.764+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=QQHG45VAEW77 type=del 2024-06-28T15:08:10.753 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.764+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=846OEDH98R3I type=del size=3088770 2024-06-28T15:08:10.756 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.768+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=YOE0AMC7I8V0 type=del 2024-06-28T15:08:10.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.776+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~JU51XM6X2806.0 size=410332 2024-06-28T15:08:10.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.776+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=JU51XM6X2806 tag=WRHXLVI2LI28 type=write size=1458908 2024-06-28T15:08:10.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.780+0000 7f65680598c0 1 < completed write operation key=V2L2Q1U62YW2 tag=539NSJB80AW4 type=write size=61333 2024-06-28T15:08:10.772 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.784+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~VVAFP26EZQT5.0 size=1048576 2024-06-28T15:08:10.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.788+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~VVAFP26EZQT5.1 size=1048576 2024-06-28T15:08:10.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.788+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~VVAFP26EZQT5.2 size=780996 2024-06-28T15:08:10.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.788+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=VVAFP26EZQT5 tag=LUE5YC44JVG2 type=write size=3926724 2024-06-28T15:08:10.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.792+0000 7f65680598c0 1 < canceled operation key=P278ZM1H052S tag=RL6KJMTOYKHE type=del size=325258 2024-06-28T15:08:10.787 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.800+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~BQU910ZGVB9M.0 size=1048576 2024-06-28T15:08:10.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.804+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~BQU910ZGVB9M.1 size=1048576 2024-06-28T15:08:10.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.804+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~BQU910ZGVB9M.2 size=919482 2024-06-28T15:08:10.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.804+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=BQU910ZGVB9M tag=NT03RXYRO2KM type=write size=4065210 2024-06-28T15:08:10.796 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.808+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~8WT8OFVANVR0.0 size=1048576 2024-06-28T15:08:10.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.812+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~8WT8OFVANVR0.1 size=867239 2024-06-28T15:08:10.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.816+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=8WT8OFVANVR0 tag=BGUWK87V99E8 type=write size=2964391 2024-06-28T15:08:10.808 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.820+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=HXPU7A3WYMSM type=write size=633437 2024-06-28T15:08:10.812 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.824+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~XA5KX4AMY6TY.0 size=1048576 2024-06-28T15:08:10.814 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.828+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~XA5KX4AMY6TY.1 size=209790 2024-06-28T15:08:10.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.832+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=XA5KX4AMY6TY tag=LBSMU0V95ZH3 type=write size=2306942 2024-06-28T15:08:10.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.836+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=KJNSMQBMQYI5 type=write size=196084 2024-06-28T15:08:10.829 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.840+0000 7f65680598c0 1 < suggested update operation key=1VD7NDPEM3CD tag=01DDYMPDV642 type=del size=3947183 2024-06-28T15:08:10.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.844+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=IZ6LAEEAPM7D type=write size=927588 2024-06-28T15:08:10.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.848+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~ZMOKI5YVDIUZ.0 size=999214 2024-06-28T15:08:10.840 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.852+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=ZMOKI5YVDIUZ tag=O4BK40K6PD84 type=write size=2047790 2024-06-28T15:08:10.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.856+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=PJ5Q2PML1U6K type=del size=1911600 2024-06-28T15:08:10.846 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.860+0000 7f65680598c0 1 < canceled operation key=MQ3QWZBFN4YR tag=78M4EQ08W2IQ type=write size=866000 2024-06-28T15:08:10.851 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.864+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~Q8TP53938Y3X.0 size=1048576 2024-06-28T15:08:10.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.868+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~Q8TP53938Y3X.1 size=1048576 2024-06-28T15:08:10.856 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.868+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~Q8TP53938Y3X.2 size=883104 2024-06-28T15:08:10.856 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.868+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=Q8TP53938Y3X tag=E03O9GMY8LT1 type=write size=4028832 2024-06-28T15:08:10.859 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.872+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~F6MKEPHF8ES9.0 size=1048576 2024-06-28T15:08:10.861 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.872+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~F6MKEPHF8ES9.1 size=403969 2024-06-28T15:08:10.866 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.880+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=F6MKEPHF8ES9 tag=7F3947VC5ANJ type=write size=2501121 2024-06-28T15:08:10.869 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.880+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=RMIGXOWP9WB9 type=del size=998971 2024-06-28T15:08:10.875 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.888+0000 7f65680598c0 1 < suggested update operation key=J311Q2712YM0 tag=5FYGHO6FO85J type=del size=1044972 2024-06-28T15:08:10.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.892+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~FSR65EF2XK7N.0 size=433736 2024-06-28T15:08:10.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.892+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=FSR65EF2XK7N tag=UME8ETS3VJRY type=write size=1482312 2024-06-28T15:08:10.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.900+0000 7f65680598c0 1 < suggested update operation key=C5C6SAHLPZV1 tag=EAB2KBV6LLPL type=write size=923130 2024-06-28T15:08:10.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.908+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~0Q6E071NVOY6.0 size=21414 2024-06-28T15:08:10.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.908+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=0Q6E071NVOY6 tag=P5PABJEENVND type=write size=1069990 2024-06-28T15:08:10.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.908+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~81FUV712BESQ.0 size=1048576 2024-06-28T15:08:10.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.912+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~81FUV712BESQ.1 size=1048576 2024-06-28T15:08:10.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.912+0000 7f65680598c0 1 < canceled part key=_multipart_7NL3CTFYY9E5.2~81FUV712BESQ.2 size=22127 2024-06-28T15:08:10.904 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.916+0000 7f65680598c0 1 < canceled multipart upload key=7NL3CTFYY9E5 upload=81FUV712BESQ tag=IO7JXHOB5991 type=write size=3167855 2024-06-28T15:08:10.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.920+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=L263358F9DB6 type=write size=801496 2024-06-28T15:08:10.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.924+0000 7f65680598c0 -1 < failed to complete operation key=1VD7NDPEM3CD tag=FT2UQDJDUZD1 type=del size=3575367: (22) Invalid argument 2024-06-28T15:08:10.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.928+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=YAY8C4HEXTFT type=del size=3003664 2024-06-28T15:08:10.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.928+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~189Z0W4FPW9E.0 size=1048576 2024-06-28T15:08:10.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.932+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~189Z0W4FPW9E.1 size=937423 2024-06-28T15:08:10.926 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.940+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=189Z0W4FPW9E tag=PGOHJTCVHOLH type=write size=3034575 2024-06-28T15:08:10.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.944+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~SQO6PMHNZXAO.0 size=1028799 2024-06-28T15:08:10.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.944+0000 7f65680598c0 1 > prepared multipart upload key=2F968YIMJN9D upload=SQO6PMHNZXAO tag=DD90FGCRGNXA type=write size=2077375 2024-06-28T15:08:10.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.948+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~QXLCLWD3DFKI.0 size=1048576 2024-06-28T15:08:10.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.952+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~QXLCLWD3DFKI.1 size=972869 2024-06-28T15:08:10.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.956+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=QXLCLWD3DFKI tag=DVCJCGVQ6XKA type=write size=3070021 2024-06-28T15:08:10.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.960+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=0FAJ2J4ZJIK5 type=write size=13055 2024-06-28T15:08:10.953 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.964+0000 7f65680598c0 1 < suggested remove operation key=4CU86XS1R9G3 tag=846OEDH98R3I type=del 2024-06-28T15:08:10.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.968+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=FWJA6T8DQLHL type=del size=4075191 2024-06-28T15:08:10.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.972+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~JU51XM6X2806.0 size=410332 2024-06-28T15:08:10.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.976+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=JU51XM6X2806 tag=WRHXLVI2LI28 type=write size=1458908 2024-06-28T15:08:10.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.980+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=5UNE97UAC1SB type=del size=1988672 2024-06-28T15:08:10.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.984+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~VVAFP26EZQT5.0 size=1048576 2024-06-28T15:08:10.975 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.988+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~VVAFP26EZQT5.1 size=1048576 2024-06-28T15:08:10.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.992+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~VVAFP26EZQT5.2 size=780996 2024-06-28T15:08:10.984 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:10.996+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=VVAFP26EZQT5 tag=LUE5YC44JVG2 type=write size=3926724 2024-06-28T15:08:10.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.000+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=LBC22P7BMMX2 type=del size=2141492 2024-06-28T15:08:10.991 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.004+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~BQU910ZGVB9M.0 size=1048576 2024-06-28T15:08:10.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.004+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~BQU910ZGVB9M.1 size=1048576 2024-06-28T15:08:10.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.008+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~BQU910ZGVB9M.2 size=919482 2024-06-28T15:08:11.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.012+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=BQU910ZGVB9M tag=NT03RXYRO2KM type=write size=4065210 2024-06-28T15:08:11.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.020+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~NQ3434CBMLO9.0 size=1048576 2024-06-28T15:08:11.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.020+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~NQ3434CBMLO9.1 size=520149 2024-06-28T15:08:11.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.020+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=NQ3434CBMLO9 tag=SANEBA3K1LWU type=write size=2617301 2024-06-28T15:08:11.012 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.024+0000 7f65680598c0 1 < completed write operation key=BHW2VHUT0U1U tag=HXPU7A3WYMSM type=write size=633437 2024-06-28T15:08:11.017 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.028+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~DME542B93GUD.0 size=1048576 2024-06-28T15:08:11.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.032+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~DME542B93GUD.1 size=1048576 2024-06-28T15:08:11.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.036+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~DME542B93GUD.2 size=891105 2024-06-28T15:08:11.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.036+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=DME542B93GUD tag=D8B9KCSOF627 type=write size=4036833 2024-06-28T15:08:11.024 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.036+0000 7f65680598c0 -1 < failed to complete operation key=4CU86XS1R9G3 tag=KJNSMQBMQYI5 type=write size=196084: (22) Invalid argument 2024-06-28T15:08:11.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.044+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~AY7QMZQJ2FE7.0 size=1048576 2024-06-28T15:08:11.033 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.044+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~AY7QMZQJ2FE7.1 size=1048576 2024-06-28T15:08:11.035 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.048+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~AY7QMZQJ2FE7.2 size=284686 2024-06-28T15:08:11.035 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.048+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=AY7QMZQJ2FE7 tag=WEZOKMFZ5V8I type=write size=3430414 2024-06-28T15:08:11.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.056+0000 7f65680598c0 1 < suggested update operation key=2F968YIMJN9D tag=IZ6LAEEAPM7D type=write size=927588 2024-06-28T15:08:11.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.056+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=ZFD65V08V4SC type=del size=643226 2024-06-28T15:08:11.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.060+0000 7f65680598c0 1 < completed delete operation key=1VD7NDPEM3CD tag=PJ5Q2PML1U6K type=del 2024-06-28T15:08:11.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.064+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=K2HXG0UNHRB9 type=del size=658510 2024-06-28T15:08:11.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.068+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~Q8TP53938Y3X.0 size=1048576 2024-06-28T15:08:11.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.072+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~Q8TP53938Y3X.1 size=1048576 2024-06-28T15:08:11.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.072+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~Q8TP53938Y3X.2 size=883104 2024-06-28T15:08:11.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.076+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=Q8TP53938Y3X tag=E03O9GMY8LT1 type=write size=4028832 2024-06-28T15:08:11.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.080+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=HN27USVEWD7E type=del size=4155728 2024-06-28T15:08:11.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.084+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=RMIGXOWP9WB9 type=del 2024-06-28T15:08:11.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.088+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=HIF7GMZ7I9LL type=del size=994665 2024-06-28T15:08:11.078 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.092+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~FSR65EF2XK7N.0 size=433736 2024-06-28T15:08:11.083 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.096+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=FSR65EF2XK7N tag=UME8ETS3VJRY type=write size=1482312 2024-06-28T15:08:11.088 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.100+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~891SSUKKY6FX.0 size=1048576 2024-06-28T15:08:11.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.104+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~891SSUKKY6FX.1 size=1007772 2024-06-28T15:08:11.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.104+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=891SSUKKY6FX tag=4UIZBELQXCB1 type=write size=3104924 2024-06-28T15:08:11.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.108+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~0Q6E071NVOY6.0 size=21414 2024-06-28T15:08:11.101 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.112+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=0Q6E071NVOY6 tag=P5PABJEENVND type=write size=1069990 2024-06-28T15:08:11.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.120+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~C7WHY3IQ47JC.0 size=263282 2024-06-28T15:08:11.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.120+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=C7WHY3IQ47JC tag=EEG1ZK359ERA type=write size=1311858 2024-06-28T15:08:11.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.124+0000 7f65680598c0 -1 < failed to complete operation key=2F968YIMJN9D tag=L263358F9DB6 type=write size=801496: (22) Invalid argument 2024-06-28T15:08:11.114 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.128+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=XAZW1DIV2SK1 type=del size=528997 2024-06-28T15:08:11.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.128+0000 7f65680598c0 1 < completed delete operation key=ZFNIA3O84EB7 tag=YAY8C4HEXTFT type=del 2024-06-28T15:08:11.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.136+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~3Q6SIRKA35MV.0 size=901335 2024-06-28T15:08:11.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.136+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=3Q6SIRKA35MV tag=M7ZHB5L3VFHK type=write size=1949911 2024-06-28T15:08:11.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.140+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~SQO6PMHNZXAO.0 size=1028799 2024-06-28T15:08:11.134 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.148+0000 7f65680598c0 1 < completed multipart upload key=2F968YIMJN9D upload=SQO6PMHNZXAO tag=DD90FGCRGNXA type=write size=2077375 2024-06-28T15:08:11.140 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.152+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~3LG1PN45ZE7O.0 size=299995 2024-06-28T15:08:11.140 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.152+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=3LG1PN45ZE7O tag=KI0L8L7HXT9H type=write size=1348571 2024-06-28T15:08:11.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.156+0000 7f65680598c0 1 < completed write operation key=60QVJVV16QFM tag=0FAJ2J4ZJIK5 type=write size=13055 2024-06-28T15:08:11.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.160+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~4XY5YWH121FS.0 size=1048576 2024-06-28T15:08:11.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.164+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~4XY5YWH121FS.1 size=968776 2024-06-28T15:08:11.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.164+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=4XY5YWH121FS tag=G8AP3WD02LAD type=write size=3065928 2024-06-28T15:08:11.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.164+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=FWJA6T8DQLHL type=del 2024-06-28T15:08:11.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.168+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=ODVAN3FPVASP type=write size=1009586 2024-06-28T15:08:11.158 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.172+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=5UNE97UAC1SB type=del 2024-06-28T15:08:11.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.172+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=N3BIRCK3NL4D type=del size=1262408 2024-06-28T15:08:11.164 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.176+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=LBC22P7BMMX2 type=del 2024-06-28T15:08:11.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.180+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=JWSCHZNX9WPM type=write size=101144 2024-06-28T15:08:11.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.184+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~NQ3434CBMLO9.0 size=1048576 2024-06-28T15:08:11.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.184+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~NQ3434CBMLO9.1 size=520149 2024-06-28T15:08:11.177 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.188+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=NQ3434CBMLO9 tag=SANEBA3K1LWU type=write size=2617301 2024-06-28T15:08:11.180 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.192+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=8NKHVDRW6W0V type=del size=1251138 2024-06-28T15:08:11.183 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.196+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~DME542B93GUD.0 size=1048576 2024-06-28T15:08:11.185 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.200+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~DME542B93GUD.1 size=1048576 2024-06-28T15:08:11.188 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.200+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~DME542B93GUD.2 size=891105 2024-06-28T15:08:11.193 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.204+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=DME542B93GUD tag=D8B9KCSOF627 type=write size=4036833 2024-06-28T15:08:11.196 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.208+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=LNGQC1ON72JZ type=del size=40904 2024-06-28T15:08:11.199 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.212+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~AY7QMZQJ2FE7.0 size=1048576 2024-06-28T15:08:11.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.216+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~AY7QMZQJ2FE7.1 size=1048576 2024-06-28T15:08:11.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.216+0000 7f65680598c0 1 < canceled part key=_multipart_9MQKWSA33R5G.2~AY7QMZQJ2FE7.2 size=284686 2024-06-28T15:08:11.207 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.220+0000 7f65680598c0 1 < canceled multipart upload key=9MQKWSA33R5G upload=AY7QMZQJ2FE7 tag=WEZOKMFZ5V8I type=write size=3430414 2024-06-28T15:08:11.210 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.224+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=I9Z21V5XUJB0 type=del size=2661976 2024-06-28T15:08:11.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.228+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=ZFD65V08V4SC type=del 2024-06-28T15:08:11.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.232+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~AQPLT804OJJ1.0 size=1048576 2024-06-28T15:08:11.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.236+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~AQPLT804OJJ1.1 size=1048576 2024-06-28T15:08:11.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.236+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~AQPLT804OJJ1.2 size=152018 2024-06-28T15:08:11.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.236+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=AQPLT804OJJ1 tag=5COHUFS1R5RM type=write size=3297746 2024-06-28T15:08:11.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.240+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=K2HXG0UNHRB9 type=del 2024-06-28T15:08:11.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.244+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=3KO6874G1F9D type=del size=1104862 2024-06-28T15:08:11.234 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.248+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=HN27USVEWD7E type=del 2024-06-28T15:08:11.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.252+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~0C3N2TNXUSZ9.0 size=58406 2024-06-28T15:08:11.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.252+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=0C3N2TNXUSZ9 tag=0P4VAZ97NLPG type=write size=1106982 2024-06-28T15:08:11.248 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.260+0000 7f65680598c0 1 < suggested update operation key=AXQ0YFWUH4FD tag=HIF7GMZ7I9LL type=del size=994665 2024-06-28T15:08:11.254 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.268+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~LOKXUESTRMBO.0 size=1048576 2024-06-28T15:08:11.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.268+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~LOKXUESTRMBO.1 size=1048576 2024-06-28T15:08:11.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.272+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~LOKXUESTRMBO.2 size=33243 2024-06-28T15:08:11.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.272+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=LOKXUESTRMBO tag=5YNJ7ZO2NBR7 type=write size=3178971 2024-06-28T15:08:11.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.276+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~891SSUKKY6FX.0 size=1048576 2024-06-28T15:08:11.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.280+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~891SSUKKY6FX.1 size=1007772 2024-06-28T15:08:11.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.284+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=891SSUKKY6FX tag=4UIZBELQXCB1 type=write size=3104924 2024-06-28T15:08:11.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.292+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~1X100W3K18XL.0 size=1048576 2024-06-28T15:08:11.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.292+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~1X100W3K18XL.1 size=293074 2024-06-28T15:08:11.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.292+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=1X100W3K18XL tag=XI3D8QOPZDOJ type=write size=2390226 2024-06-28T15:08:11.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.296+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~C7WHY3IQ47JC.0 size=263282 2024-06-28T15:08:11.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.304+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=C7WHY3IQ47JC tag=EEG1ZK359ERA type=write size=1311858 2024-06-28T15:08:11.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.308+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~98CAOVHYTH2I.0 size=1048576 2024-06-28T15:08:11.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.312+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~98CAOVHYTH2I.1 size=1048576 2024-06-28T15:08:11.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.316+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~98CAOVHYTH2I.2 size=704508 2024-06-28T15:08:11.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.316+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=98CAOVHYTH2I tag=KM5LHSLWVIVX type=write size=3850236 2024-06-28T15:08:11.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.320+0000 7f65680598c0 1 < suggested update operation key=47TO7G9EGJKF tag=XAZW1DIV2SK1 type=del size=528997 2024-06-28T15:08:11.311 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.324+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=6OEK8CIIMSDU type=del size=3462612 2024-06-28T15:08:11.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.328+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~3Q6SIRKA35MV.0 size=901335 2024-06-28T15:08:11.320 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.332+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=3Q6SIRKA35MV tag=M7ZHB5L3VFHK type=write size=1949911 2024-06-28T15:08:11.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.336+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=KGTM7ZJQKBBD type=write size=95096 2024-06-28T15:08:11.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.340+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~3LG1PN45ZE7O.0 size=299995 2024-06-28T15:08:11.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.344+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=3LG1PN45ZE7O tag=KI0L8L7HXT9H type=write size=1348571 2024-06-28T15:08:11.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.352+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~DLIFBJT4LEZP.0 size=1048576 2024-06-28T15:08:11.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.352+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~DLIFBJT4LEZP.1 size=1048576 2024-06-28T15:08:11.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.356+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~DLIFBJT4LEZP.2 size=418655 2024-06-28T15:08:11.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.356+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=DLIFBJT4LEZP tag=42QP1NVVVBVF type=write size=3564383 2024-06-28T15:08:11.347 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.360+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~4XY5YWH121FS.0 size=1048576 2024-06-28T15:08:11.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.360+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~4XY5YWH121FS.1 size=968776 2024-06-28T15:08:11.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.368+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=4XY5YWH121FS tag=G8AP3WD02LAD type=write size=3065928 2024-06-28T15:08:11.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.372+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~QUX37RXX0EBY.0 size=1048576 2024-06-28T15:08:11.363 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.376+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~QUX37RXX0EBY.1 size=933376 2024-06-28T15:08:11.363 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.376+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=QUX37RXX0EBY tag=VCEJXGH1X80Q type=write size=3030528 2024-06-28T15:08:11.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.380+0000 7f65680598c0 1 < completed write operation key=2F968YIMJN9D tag=ODVAN3FPVASP type=write size=1009586 2024-06-28T15:08:11.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.384+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~SYR2IVLICM91.0 size=160268 2024-06-28T15:08:11.373 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.384+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=SYR2IVLICM91 tag=LWEYG17IE4EX type=write size=1208844 2024-06-28T15:08:11.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.388+0000 7f65680598c0 -1 < failed to complete operation key=47TO7G9EGJKF tag=N3BIRCK3NL4D type=del size=1262408: (22) Invalid argument 2024-06-28T15:08:11.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.392+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=SFYDRWZM579B type=del size=1607476 2024-06-28T15:08:11.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.396+0000 7f65680598c0 1 < completed write operation key=2FC9DNEZGWC4 tag=JWSCHZNX9WPM type=write size=101144 2024-06-28T15:08:11.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.400+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=DXPI4TU24ZQ0 type=del size=299678 2024-06-28T15:08:11.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.400+0000 7f65680598c0 1 < completed delete operation key=ZFNIA3O84EB7 tag=8NKHVDRW6W0V type=del 2024-06-28T15:08:11.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.412+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~QCG0KPRPV5XI.0 size=362002 2024-06-28T15:08:11.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.412+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=QCG0KPRPV5XI tag=ID7T2VLWI2M6 type=write size=1410578 2024-06-28T15:08:11.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.420+0000 7f65680598c0 1 < suggested remove operation key=1VD7NDPEM3CD tag=LNGQC1ON72JZ type=del 2024-06-28T15:08:11.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.424+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=GE0PEDVX2UM3 type=del size=3139182 2024-06-28T15:08:11.413 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.424+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=I9Z21V5XUJB0 type=del 2024-06-28T15:08:11.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.428+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=9R9430HY8IMI type=del size=676051 2024-06-28T15:08:11.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.432+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~AQPLT804OJJ1.0 size=1048576 2024-06-28T15:08:11.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.432+0000 7f65680598c0 1 < canceled part key=_multipart_7LB8MOJL39RM.2~AQPLT804OJJ1.1 size=1048576 2024-06-28T15:08:11.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.436+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~AQPLT804OJJ1.2 size=152018 2024-06-28T15:08:11.430 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.444+0000 7f65680598c0 1 < canceled multipart upload key=7LB8MOJL39RM upload=AQPLT804OJJ1 tag=5COHUFS1R5RM type=write size=3297746 2024-06-28T15:08:11.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.448+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~F9JPT2FB6JSK.0 size=1048576 2024-06-28T15:08:11.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.452+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~F9JPT2FB6JSK.1 size=1048576 2024-06-28T15:08:11.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.456+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~F9JPT2FB6JSK.2 size=860507 2024-06-28T15:08:11.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.456+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=F9JPT2FB6JSK tag=3QXU6FODX01S type=write size=4006235 2024-06-28T15:08:11.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.456+0000 7f65680598c0 1 < canceled operation key=UP3XFF38IJXN tag=3KO6874G1F9D type=del size=1104862 2024-06-28T15:08:11.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.460+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=4ELATR6032DQ type=del size=3572925 2024-06-28T15:08:11.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.464+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~0C3N2TNXUSZ9.0 size=58406 2024-06-28T15:08:11.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.472+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=0C3N2TNXUSZ9 tag=0P4VAZ97NLPG type=write size=1106982 2024-06-28T15:08:11.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.472+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=V110MX8UXF80 type=del size=2448952 2024-06-28T15:08:11.462 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.476+0000 7f65680598c0 1 < canceled part key=_multipart_60QVJVV16QFM.2~LOKXUESTRMBO.0 size=1048576 2024-06-28T15:08:11.465 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.476+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~LOKXUESTRMBO.1 size=1048576 2024-06-28T15:08:11.468 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.480+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~LOKXUESTRMBO.2 size=33243 2024-06-28T15:08:11.474 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.488+0000 7f65680598c0 1 < canceled multipart upload key=60QVJVV16QFM upload=LOKXUESTRMBO tag=5YNJ7ZO2NBR7 type=write size=3178971 2024-06-28T15:08:11.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.488+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=WFQ6NIDEYIMZ type=del size=1624613 2024-06-28T15:08:11.481 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.492+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~1X100W3K18XL.0 size=1048576 2024-06-28T15:08:11.484 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.496+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~1X100W3K18XL.1 size=293074 2024-06-28T15:08:11.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.500+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=1X100W3K18XL tag=XI3D8QOPZDOJ type=write size=2390226 2024-06-28T15:08:11.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.504+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=8JJAT383TRZL type=write size=734181 2024-06-28T15:08:11.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.508+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~98CAOVHYTH2I.0 size=1048576 2024-06-28T15:08:11.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.512+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~98CAOVHYTH2I.1 size=1048576 2024-06-28T15:08:11.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.512+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~98CAOVHYTH2I.2 size=704508 2024-06-28T15:08:11.506 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.520+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=98CAOVHYTH2I tag=KM5LHSLWVIVX type=write size=3850236 2024-06-28T15:08:11.509 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.524+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=SUCEMK2JT96V type=del size=4137784 2024-06-28T15:08:11.516 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.528+0000 7f65680598c0 1 < suggested update operation key=2F968YIMJN9D tag=6OEK8CIIMSDU type=del size=3462612 2024-06-28T15:08:11.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.532+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=50RRI1404NFR type=del size=2565971 2024-06-28T15:08:11.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.536+0000 7f65680598c0 1 < canceled operation key=7NL3CTFYY9E5 tag=KGTM7ZJQKBBD type=write size=95096 2024-06-28T15:08:11.528 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.540+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~F8OPRCKV156J.0 size=1048576 2024-06-28T15:08:11.530 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.544+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~F8OPRCKV156J.1 size=26917 2024-06-28T15:08:11.530 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.544+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=F8OPRCKV156J tag=CNOBQ5E6BPML type=write size=2124069 2024-06-28T15:08:11.533 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.544+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~DLIFBJT4LEZP.0 size=1048576 2024-06-28T15:08:11.536 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.548+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~DLIFBJT4LEZP.1 size=1048576 2024-06-28T15:08:11.538 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.552+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~DLIFBJT4LEZP.2 size=418655 2024-06-28T15:08:11.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.556+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=DLIFBJT4LEZP tag=42QP1NVVVBVF type=write size=3564383 2024-06-28T15:08:11.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.560+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=MG232G2E0H90 type=del size=2452365 2024-06-28T15:08:11.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.564+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~QUX37RXX0EBY.0 size=1048576 2024-06-28T15:08:11.553 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.564+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~QUX37RXX0EBY.1 size=933376 2024-06-28T15:08:11.558 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.572+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=QUX37RXX0EBY tag=VCEJXGH1X80Q type=write size=3030528 2024-06-28T15:08:11.561 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.576+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=9TAZKITIMBRJ type=del size=457031 2024-06-28T15:08:11.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.576+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~SYR2IVLICM91.0 size=160268 2024-06-28T15:08:11.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.584+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=SYR2IVLICM91 tag=LWEYG17IE4EX type=write size=1208844 2024-06-28T15:08:11.576 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.588+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~KC02158MHZB8.0 size=1048576 2024-06-28T15:08:11.578 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.592+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~KC02158MHZB8.1 size=1048576 2024-06-28T15:08:11.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.592+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~KC02158MHZB8.2 size=1024103 2024-06-28T15:08:11.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.592+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=KC02158MHZB8 tag=Z6OUVFQZ0O9B type=write size=4169831 2024-06-28T15:08:11.587 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.600+0000 7f65680598c0 1 < suggested remove operation key=AXQ0YFWUH4FD tag=SFYDRWZM579B type=del 2024-06-28T15:08:11.593 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.604+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~0923ZQSJU1GG.0 size=1048576 2024-06-28T15:08:11.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.608+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~0923ZQSJU1GG.1 size=524694 2024-06-28T15:08:11.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.608+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=0923ZQSJU1GG tag=CLUP3Z4GGFWQ type=write size=2621846 2024-06-28T15:08:11.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.612+0000 7f65680598c0 -1 < failed to complete operation key=1VD7NDPEM3CD tag=DXPI4TU24ZQ0 type=del size=299678: (22) Invalid argument 2024-06-28T15:08:11.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.616+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=UOYHLFH9VPVX type=del size=2735856 2024-06-28T15:08:11.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.620+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~QCG0KPRPV5XI.0 size=362002 2024-06-28T15:08:11.611 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.624+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=QCG0KPRPV5XI tag=ID7T2VLWI2M6 type=write size=1410578 2024-06-28T15:08:11.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.628+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=IEWSM90W9ANS type=del size=832677 2024-06-28T15:08:11.621 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.632+0000 7f65680598c0 1 < suggested update operation key=D9D3OFIJP3D3 tag=GE0PEDVX2UM3 type=del size=3139182 2024-06-28T15:08:11.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.636+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=4W1YR2IGWWFV type=del size=745101 2024-06-28T15:08:11.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.640+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=9R9430HY8IMI type=del 2024-06-28T15:08:11.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.644+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=55G11S4KQOQX type=del size=1246446 2024-06-28T15:08:11.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.648+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~F9JPT2FB6JSK.0 size=1048576 2024-06-28T15:08:11.637 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.648+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~F9JPT2FB6JSK.1 size=1048576 2024-06-28T15:08:11.640 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.652+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~F9JPT2FB6JSK.2 size=860507 2024-06-28T15:08:11.645 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.656+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=F9JPT2FB6JSK tag=3QXU6FODX01S type=write size=4006235 2024-06-28T15:08:11.651 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.664+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~TS6WGD1JALUC.0 size=1048576 2024-06-28T15:08:11.653 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.664+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~TS6WGD1JALUC.1 size=1048576 2024-06-28T15:08:11.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.668+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~TS6WGD1JALUC.2 size=534561 2024-06-28T15:08:11.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.668+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=TS6WGD1JALUC tag=WOM4XIEG3AP6 type=write size=3680289 2024-06-28T15:08:11.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.672+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=4ELATR6032DQ type=del 2024-06-28T15:08:11.662 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.676+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=ZINJ5QDZKPZ0 type=write size=205769 2024-06-28T15:08:11.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.680+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=V110MX8UXF80 type=del 2024-06-28T15:08:11.670 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.684+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=CSV430YBW8H5 type=del size=2823214 2024-06-28T15:08:11.673 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.684+0000 7f65680598c0 -1 < failed to complete operation key=2F968YIMJN9D tag=WFQ6NIDEYIMZ type=del size=1624613: (22) Invalid argument 2024-06-28T15:08:11.679 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.692+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~MLZQ84IEVHF1.0 size=1048576 2024-06-28T15:08:11.682 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.696+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~MLZQ84IEVHF1.1 size=1048576 2024-06-28T15:08:11.685 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.696+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~MLZQ84IEVHF1.2 size=33059 2024-06-28T15:08:11.685 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.696+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=MLZQ84IEVHF1 tag=WGP6S7DGZ1UZ type=write size=3178787 2024-06-28T15:08:11.688 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.700+0000 7f65680598c0 1 < completed write operation key=1VD7NDPEM3CD tag=8JJAT383TRZL type=write size=734181 2024-06-28T15:08:11.691 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.704+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=KU3HM8I39UC0 type=del size=2392148 2024-06-28T15:08:11.695 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.708+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=SUCEMK2JT96V type=del 2024-06-28T15:08:11.698 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.712+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=S45MY56F5UT4 type=del size=1265124 2024-06-28T15:08:11.702 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.716+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=50RRI1404NFR type=del 2024-06-28T15:08:11.711 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.724+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~1Z53ZVOTJFJ9.0 size=1048576 2024-06-28T15:08:11.713 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.728+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~1Z53ZVOTJFJ9.1 size=1048576 2024-06-28T15:08:11.716 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.728+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~1Z53ZVOTJFJ9.2 size=205350 2024-06-28T15:08:11.716 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.728+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=1Z53ZVOTJFJ9 tag=FNBEJ6QHW3IF type=write size=3351078 2024-06-28T15:08:11.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.732+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~F8OPRCKV156J.0 size=1048576 2024-06-28T15:08:11.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.736+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~F8OPRCKV156J.1 size=26917 2024-06-28T15:08:11.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.740+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=F8OPRCKV156J tag=CNOBQ5E6BPML type=write size=2124069 2024-06-28T15:08:11.733 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.744+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~8EZNM1B0I752.0 size=1048576 2024-06-28T15:08:11.736 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.748+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~8EZNM1B0I752.1 size=677707 2024-06-28T15:08:11.736 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.748+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=8EZNM1B0I752 tag=KE0UE76YYGZH type=write size=2774859 2024-06-28T15:08:11.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.752+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=MG232G2E0H90 type=del 2024-06-28T15:08:11.745 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.756+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~U96367QWORZX.0 size=1048576 2024-06-28T15:08:11.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.760+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~U96367QWORZX.1 size=81373 2024-06-28T15:08:11.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.760+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=U96367QWORZX tag=CAKGE4MYXAAJ type=write size=2178525 2024-06-28T15:08:11.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.764+0000 7f65680598c0 -1 < failed to complete operation key=D9D3OFIJP3D3 tag=9TAZKITIMBRJ type=del size=457031: (22) Invalid argument 2024-06-28T15:08:11.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.768+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=NEZEIGZL1OB1 type=del size=3703798 2024-06-28T15:08:11.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.772+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~KC02158MHZB8.0 size=1048576 2024-06-28T15:08:11.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.772+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~KC02158MHZB8.1 size=1048576 2024-06-28T15:08:11.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.776+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~KC02158MHZB8.2 size=1024103 2024-06-28T15:08:11.768 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.780+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=KC02158MHZB8 tag=Z6OUVFQZ0O9B type=write size=4169831 2024-06-28T15:08:11.771 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.784+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=EQRPHD23AZ5E type=del size=295796 2024-06-28T15:08:11.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.788+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~0923ZQSJU1GG.0 size=1048576 2024-06-28T15:08:11.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.788+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~0923ZQSJU1GG.1 size=524694 2024-06-28T15:08:11.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.796+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=0923ZQSJU1GG tag=CLUP3Z4GGFWQ type=write size=2621846 2024-06-28T15:08:11.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.804+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~OF00KLLHPCHT.0 size=811244 2024-06-28T15:08:11.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.804+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=OF00KLLHPCHT tag=I4S83EB39YJT type=write size=1859820 2024-06-28T15:08:11.794 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.808+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=UOYHLFH9VPVX type=del 2024-06-28T15:08:11.797 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.808+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=VKDXM3MVOWDR type=del size=1976507 2024-06-28T15:08:11.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.812+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=IEWSM90W9ANS type=del 2024-06-28T15:08:11.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.816+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=IY8UP3616KZ6 type=del size=3026567 2024-06-28T15:08:11.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.816+0000 7f65680598c0 1 < canceled operation key=2FC9DNEZGWC4 tag=4W1YR2IGWWFV type=del size=745101 2024-06-28T15:08:11.807 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.820+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=LMX4XSZ62WKN type=del size=1388224 2024-06-28T15:08:11.810 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.824+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=55G11S4KQOQX type=del 2024-06-28T15:08:11.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.828+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~SASFWULRIBIG.0 size=46340 2024-06-28T15:08:11.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.828+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=SASFWULRIBIG tag=Z5QLRZT9YFDP type=write size=1094916 2024-06-28T15:08:11.817 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.832+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~TS6WGD1JALUC.0 size=1048576 2024-06-28T15:08:11.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.832+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~TS6WGD1JALUC.1 size=1048576 2024-06-28T15:08:11.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.836+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~TS6WGD1JALUC.2 size=534561 2024-06-28T15:08:11.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.840+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=TS6WGD1JALUC tag=WOM4XIEG3AP6 type=write size=3680289 2024-06-28T15:08:11.829 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.840+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=KLW4LPL4QR74 type=write size=111493 2024-06-28T15:08:11.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.844+0000 7f65680598c0 1 < completed write operation key=1VD7NDPEM3CD tag=ZINJ5QDZKPZ0 type=write size=205769 2024-06-28T15:08:11.836 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.848+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~98JJPT12LH5L.0 size=1048576 2024-06-28T15:08:11.838 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.852+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~98JJPT12LH5L.1 size=474780 2024-06-28T15:08:11.838 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.852+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=98JJPT12LH5L tag=D1A3OWD3MZG4 type=write size=2571932 2024-06-28T15:08:11.841 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.852+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=CSV430YBW8H5 type=del 2024-06-28T15:08:11.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.856+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=KPEO128TMLW3 type=del size=3268670 2024-06-28T15:08:11.846 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.860+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~MLZQ84IEVHF1.0 size=1048576 2024-06-28T15:08:11.848 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.860+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~MLZQ84IEVHF1.1 size=1048576 2024-06-28T15:08:11.851 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.864+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~MLZQ84IEVHF1.2 size=33059 2024-06-28T15:08:11.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.868+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=MLZQ84IEVHF1 tag=WGP6S7DGZ1UZ type=write size=3178787 2024-06-28T15:08:11.858 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.872+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=1N5VFWBKO91N type=del size=178083 2024-06-28T15:08:11.863 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.876+0000 7f65680598c0 1 < suggested remove operation key=UP3XFF38IJXN tag=KU3HM8I39UC0 type=del 2024-06-28T15:08:11.865 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.880+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=8AI25GQGLV8Z type=del size=1035773 2024-06-28T15:08:11.868 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.880+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=S45MY56F5UT4 type=del 2024-06-28T15:08:11.873 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.884+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~A6ROUNDT4619.0 size=830326 2024-06-28T15:08:11.873 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.884+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=A6ROUNDT4619 tag=KV86GDHMABN1 type=write size=1878902 2024-06-28T15:08:11.875 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.888+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~1Z53ZVOTJFJ9.0 size=1048576 2024-06-28T15:08:11.877 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.892+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~1Z53ZVOTJFJ9.1 size=1048576 2024-06-28T15:08:11.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.892+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~1Z53ZVOTJFJ9.2 size=205350 2024-06-28T15:08:11.884 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.896+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=1Z53ZVOTJFJ9 tag=FNBEJ6QHW3IF type=write size=3351078 2024-06-28T15:08:11.886 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.900+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=AFAK45JXHFPQ type=del size=912017 2024-06-28T15:08:11.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.900+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~8EZNM1B0I752.0 size=1048576 2024-06-28T15:08:11.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.904+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~8EZNM1B0I752.1 size=677707 2024-06-28T15:08:11.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.908+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=8EZNM1B0I752 tag=KE0UE76YYGZH type=write size=2774859 2024-06-28T15:08:11.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.912+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=UV48SWM6XHZQ type=del size=2961885 2024-06-28T15:08:11.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.912+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~U96367QWORZX.0 size=1048576 2024-06-28T15:08:11.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.916+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~U96367QWORZX.1 size=81373 2024-06-28T15:08:11.905 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.920+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=U96367QWORZX tag=CAKGE4MYXAAJ type=write size=2178525 2024-06-28T15:08:11.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.924+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~GWERTNAU0483.0 size=1048576 2024-06-28T15:08:11.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.924+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~GWERTNAU0483.1 size=437991 2024-06-28T15:08:11.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.924+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=GWERTNAU0483 tag=W3TPGV7MLVQH type=write size=2535143 2024-06-28T15:08:11.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.928+0000 7f65680598c0 1 < completed delete operation key=1VD7NDPEM3CD tag=NEZEIGZL1OB1 type=del 2024-06-28T15:08:11.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.928+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=WYLBNBL6DCWF type=del size=3757344 2024-06-28T15:08:11.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.932+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=EQRPHD23AZ5E type=del 2024-06-28T15:08:11.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.936+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~80MSPRJHMNX3.0 size=1011139 2024-06-28T15:08:11.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.936+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=80MSPRJHMNX3 tag=JMSMCEE5G8D5 type=write size=2059715 2024-06-28T15:08:11.926 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.940+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~OF00KLLHPCHT.0 size=811244 2024-06-28T15:08:11.930 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.944+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=OF00KLLHPCHT tag=I4S83EB39YJT type=write size=1859820 2024-06-28T15:08:11.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.944+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=IMAVT2Q79D7K type=write size=172777 2024-06-28T15:08:11.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.948+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=VKDXM3MVOWDR type=del 2024-06-28T15:08:11.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.952+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=ZYQL0B69HPY4 type=del size=1075668 2024-06-28T15:08:11.940 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.952+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=IY8UP3616KZ6 type=del 2024-06-28T15:08:11.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.956+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~3OMLNVXVGLTM.0 size=1048576 2024-06-28T15:08:11.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.960+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~3OMLNVXVGLTM.1 size=1048576 2024-06-28T15:08:11.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.960+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~3OMLNVXVGLTM.2 size=113379 2024-06-28T15:08:11.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.960+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=3OMLNVXVGLTM tag=I8HLQOJ8LY6C type=write size=3259107 2024-06-28T15:08:11.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.964+0000 7f65680598c0 1 < canceled operation key=47TO7G9EGJKF tag=LMX4XSZ62WKN type=del size=1388224 2024-06-28T15:08:11.954 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.968+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~E85KD6HTTMC9.0 size=1015883 2024-06-28T15:08:11.954 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.968+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=E85KD6HTTMC9 tag=3ZL81I46YNI0 type=write size=2064459 2024-06-28T15:08:11.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.968+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~SASFWULRIBIG.0 size=46340 2024-06-28T15:08:11.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.972+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=SASFWULRIBIG tag=Z5QLRZT9YFDP type=write size=1094916 2024-06-28T15:08:11.963 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.976+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=8B1VGWTM4KAB type=del size=1513420 2024-06-28T15:08:11.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.976+0000 7f65680598c0 1 < completed write operation key=D9D3OFIJP3D3 tag=KLW4LPL4QR74 type=write size=111493 2024-06-28T15:08:11.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.980+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=IW1W7RJ0Q6G5 type=del size=2242075 2024-06-28T15:08:11.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.980+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~98JJPT12LH5L.0 size=1048576 2024-06-28T15:08:11.970 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.984+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~98JJPT12LH5L.1 size=474780 2024-06-28T15:08:11.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.984+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=98JJPT12LH5L tag=D1A3OWD3MZG4 type=write size=2571932 2024-06-28T15:08:11.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.988+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~ZBVHP7OFQ5HQ.0 size=1048576 2024-06-28T15:08:11.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.992+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~ZBVHP7OFQ5HQ.1 size=1048576 2024-06-28T15:08:11.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.992+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~ZBVHP7OFQ5HQ.2 size=639581 2024-06-28T15:08:11.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.992+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=ZBVHP7OFQ5HQ tag=WNALIKR34B5Z type=write size=3785309 2024-06-28T15:08:11.981 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.996+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=KPEO128TMLW3 type=del 2024-06-28T15:08:11.983 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:11.996+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=TVKG125S7XEH type=del size=173010 2024-06-28T15:08:11.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.000+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=1N5VFWBKO91N type=del 2024-06-28T15:08:11.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.000+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=Y5VPYS5QUE9G type=del size=1759632 2024-06-28T15:08:11.992 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.004+0000 7f65680598c0 1 < suggested remove operation key=EN76QQWIQAXZ tag=8AI25GQGLV8Z type=del 2024-06-28T15:08:11.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.008+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=6BYIWSR3L85J type=write size=914137 2024-06-28T15:08:11.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.008+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~A6ROUNDT4619.0 size=830326 2024-06-28T15:08:11.999 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.012+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=A6ROUNDT4619 tag=KV86GDHMABN1 type=write size=1878902 2024-06-28T15:08:12.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.016+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=ZJ2O6E9A456S type=del size=3654498 2024-06-28T15:08:12.004 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.016+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=AFAK45JXHFPQ type=del 2024-06-28T15:08:12.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.020+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=8KH1YYQYWS2C type=del size=2891466 2024-06-28T15:08:12.008 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.020+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=UV48SWM6XHZQ type=del 2024-06-28T15:08:12.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.024+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=J2HL4HYCNT0I type=del size=3765833 2024-06-28T15:08:12.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.024+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~GWERTNAU0483.0 size=1048576 2024-06-28T15:08:12.015 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.028+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~GWERTNAU0483.1 size=437991 2024-06-28T15:08:12.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.032+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=GWERTNAU0483 tag=W3TPGV7MLVQH type=write size=2535143 2024-06-28T15:08:12.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.032+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=S8QVE0AEHQ8Q type=del size=611604 2024-06-28T15:08:12.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.036+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=WYLBNBL6DCWF type=del 2024-06-28T15:08:12.024 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.036+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=LI9X5XUKBVAP type=write size=153907 2024-06-28T15:08:12.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.040+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~80MSPRJHMNX3.0 size=1011139 2024-06-28T15:08:12.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.044+0000 7f65680598c0 1 < completed multipart upload key=MQ3QWZBFN4YR upload=80MSPRJHMNX3 tag=JMSMCEE5G8D5 type=write size=2059715 2024-06-28T15:08:12.034 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.048+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~FF8L07G05EY0.0 size=1048576 2024-06-28T15:08:12.035 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.048+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~FF8L07G05EY0.1 size=1048576 2024-06-28T15:08:12.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.048+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~FF8L07G05EY0.2 size=160205 2024-06-28T15:08:12.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.048+0000 7f65680598c0 1 > prepared multipart upload key=2F968YIMJN9D upload=FF8L07G05EY0 tag=I8I78NFIQT7E type=write size=3305933 2024-06-28T15:08:12.039 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.052+0000 7f65680598c0 1 < completed write operation key=I4RWCQSQVLCN tag=IMAVT2Q79D7K type=write size=172777 2024-06-28T15:08:12.040 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.052+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=PQ8A6IBH2VIY type=del size=4042631 2024-06-28T15:08:12.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.056+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=ZYQL0B69HPY4 type=del 2024-06-28T15:08:12.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.060+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~D9TU1BSQDPO1.0 size=1048576 2024-06-28T15:08:12.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.060+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~D9TU1BSQDPO1.1 size=830511 2024-06-28T15:08:12.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.060+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=D9TU1BSQDPO1 tag=U3DKQ1YO6ZQB type=write size=2927663 2024-06-28T15:08:12.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.064+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~3OMLNVXVGLTM.0 size=1048576 2024-06-28T15:08:12.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.064+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~3OMLNVXVGLTM.1 size=1048576 2024-06-28T15:08:12.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.064+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~3OMLNVXVGLTM.2 size=113379 2024-06-28T15:08:12.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.068+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=3OMLNVXVGLTM tag=I8HLQOJ8LY6C type=write size=3259107 2024-06-28T15:08:12.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.072+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=4IMKAXYR3SOD type=del size=3487651 2024-06-28T15:08:12.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.072+0000 7f65680598c0 1 < canceled part key=_multipart_7NL3CTFYY9E5.2~E85KD6HTTMC9.0 size=1015883 2024-06-28T15:08:12.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.076+0000 7f65680598c0 1 < canceled multipart upload key=7NL3CTFYY9E5 upload=E85KD6HTTMC9 tag=3ZL81I46YNI0 type=write size=2064459 2024-06-28T15:08:12.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.076+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=QA9736IYWEKF type=del size=1933665 2024-06-28T15:08:12.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.080+0000 7f65680598c0 1 < suggested remove operation key=D9D3OFIJP3D3 tag=8B1VGWTM4KAB type=del 2024-06-28T15:08:12.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.084+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=M9E0M0SVVZQ8 type=del size=2569744 2024-06-28T15:08:12.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.084+0000 7f65680598c0 1 < suggested update operation key=LZX5UZCYLTAG tag=IW1W7RJ0Q6G5 type=del size=2242075 2024-06-28T15:08:12.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.088+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~VBDTRGCOYRU8.0 size=1048576 2024-06-28T15:08:12.078 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.092+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~VBDTRGCOYRU8.1 size=985609 2024-06-28T15:08:12.078 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.092+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=VBDTRGCOYRU8 tag=0GIJPZDLVMLC type=write size=3082761 2024-06-28T15:08:12.080 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.092+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~ZBVHP7OFQ5HQ.0 size=1048576 2024-06-28T15:08:12.082 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.096+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~ZBVHP7OFQ5HQ.1 size=1048576 2024-06-28T15:08:12.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.096+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~ZBVHP7OFQ5HQ.2 size=639581 2024-06-28T15:08:12.087 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.100+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=ZBVHP7OFQ5HQ tag=WNALIKR34B5Z type=write size=3785309 2024-06-28T15:08:12.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.104+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~6TFCGDP4IWG6.0 size=1048576 2024-06-28T15:08:12.093 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.104+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~6TFCGDP4IWG6.1 size=1048576 2024-06-28T15:08:12.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.108+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~6TFCGDP4IWG6.2 size=67751 2024-06-28T15:08:12.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.108+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=6TFCGDP4IWG6 tag=15P3WD715JWC type=write size=3213479 2024-06-28T15:08:12.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.108+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=TVKG125S7XEH type=del 2024-06-28T15:08:12.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.112+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=3HJQ1E8O91Z2 type=del size=1050964 2024-06-28T15:08:12.101 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.112+0000 7f65680598c0 1 < canceled operation key=I4RWCQSQVLCN tag=Y5VPYS5QUE9G type=del size=1759632 2024-06-28T15:08:12.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.116+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=J9SUD78GA210 type=del size=499912 2024-06-28T15:08:12.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.120+0000 7f65680598c0 1 < suggested remove operation key=9MQKWSA33R5G tag=6BYIWSR3L85J type=write 2024-06-28T15:08:12.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.124+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=41WITA7X7RVY type=del size=2871987 2024-06-28T15:08:12.112 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.124+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=ZJ2O6E9A456S type=del 2024-06-28T15:08:12.114 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.128+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=VVE7MEIGDX49 type=del size=2474867 2024-06-28T15:08:12.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.128+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=8KH1YYQYWS2C type=del 2024-06-28T15:08:12.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.132+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~K376Q9B2NBF1.0 size=120291 2024-06-28T15:08:12.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.132+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=K376Q9B2NBF1 tag=JCF552E7RQ11 type=write size=1168867 2024-06-28T15:08:12.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.136+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=J2HL4HYCNT0I type=del 2024-06-28T15:08:12.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.140+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~EECZ18YTBIR8.0 size=147558 2024-06-28T15:08:12.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.140+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=EECZ18YTBIR8 tag=701BB21BR2BD type=write size=1196134 2024-06-28T15:08:12.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.140+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=S8QVE0AEHQ8Q type=del 2024-06-28T15:08:12.131 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.144+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=QAQTEBH2G5FC type=write size=738900 2024-06-28T15:08:12.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.144+0000 7f65680598c0 -1 < failed to complete operation key=D9D3OFIJP3D3 tag=LI9X5XUKBVAP type=write size=153907: (22) Invalid argument 2024-06-28T15:08:12.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.148+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~TWIH44VPDEAE.0 size=1048576 2024-06-28T15:08:12.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.152+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~TWIH44VPDEAE.1 size=76073 2024-06-28T15:08:12.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.152+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=TWIH44VPDEAE tag=2SN9VZV0REXS type=write size=2173225 2024-06-28T15:08:12.141 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.152+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~FF8L07G05EY0.0 size=1048576 2024-06-28T15:08:12.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.156+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~FF8L07G05EY0.1 size=1048576 2024-06-28T15:08:12.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.156+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~FF8L07G05EY0.2 size=160205 2024-06-28T15:08:12.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.160+0000 7f65680598c0 1 < completed multipart upload key=2F968YIMJN9D upload=FF8L07G05EY0 tag=I8I78NFIQT7E type=write size=3305933 2024-06-28T15:08:12.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.164+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~K0BSMEGZT04M.0 size=798658 2024-06-28T15:08:12.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.164+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=K0BSMEGZT04M tag=F3NJ98S1HN3Z type=write size=1847234 2024-06-28T15:08:12.155 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.168+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=PQ8A6IBH2VIY type=del 2024-06-28T15:08:12.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.172+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~K6ZR42HIWWQE.0 size=1048576 2024-06-28T15:08:12.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.172+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~K6ZR42HIWWQE.1 size=1048576 2024-06-28T15:08:12.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.176+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~K6ZR42HIWWQE.2 size=157891 2024-06-28T15:08:12.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.176+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=K6ZR42HIWWQE tag=FQT8B6IBHRLP type=write size=3303619 2024-06-28T15:08:12.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.176+0000 7f65680598c0 1 < canceled part key=_multipart_60QVJVV16QFM.2~D9TU1BSQDPO1.0 size=1048576 2024-06-28T15:08:12.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.176+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~D9TU1BSQDPO1.1 size=830511 2024-06-28T15:08:12.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.184+0000 7f65680598c0 1 < canceled multipart upload key=60QVJVV16QFM upload=D9TU1BSQDPO1 tag=U3DKQ1YO6ZQB type=write size=2927663 2024-06-28T15:08:12.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.184+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=0ML6CQM1L9NM type=del size=3385465 2024-06-28T15:08:12.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.188+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=4IMKAXYR3SOD type=del 2024-06-28T15:08:12.177 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.192+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~1ECN0LZJEKF0.0 size=1048576 2024-06-28T15:08:12.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.192+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~1ECN0LZJEKF0.1 size=661635 2024-06-28T15:08:12.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.192+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=1ECN0LZJEKF0 tag=KZYXY7I2O40N type=write size=2758787 2024-06-28T15:08:12.183 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.196+0000 7f65680598c0 1 < suggested update operation key=4CU86XS1R9G3 tag=QA9736IYWEKF type=del size=1933665 2024-06-28T15:08:12.187 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.200+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~MUNXEUQDDFYR.0 size=1048576 2024-06-28T15:08:12.188 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.200+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~MUNXEUQDDFYR.1 size=1048576 2024-06-28T15:08:12.190 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.204+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~MUNXEUQDDFYR.2 size=234324 2024-06-28T15:08:12.190 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.204+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=MUNXEUQDDFYR tag=F4ZYE23U2DVC type=write size=3380052 2024-06-28T15:08:12.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.204+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=M9E0M0SVVZQ8 type=del 2024-06-28T15:08:12.194 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.208+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=90IUOENOC5F8 type=write size=385128 2024-06-28T15:08:12.196 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.208+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~VBDTRGCOYRU8.0 size=1048576 2024-06-28T15:08:12.198 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.212+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~VBDTRGCOYRU8.1 size=985609 2024-06-28T15:08:12.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.216+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=VBDTRGCOYRU8 tag=0GIJPZDLVMLC type=write size=3082761 2024-06-28T15:08:12.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.216+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=Q0GNBX8E9ZEG type=del size=3743381 2024-06-28T15:08:12.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.220+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~6TFCGDP4IWG6.0 size=1048576 2024-06-28T15:08:12.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.220+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~6TFCGDP4IWG6.1 size=1048576 2024-06-28T15:08:12.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.224+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~6TFCGDP4IWG6.2 size=67751 2024-06-28T15:08:12.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.224+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=6TFCGDP4IWG6 tag=15P3WD715JWC type=write size=3213479 2024-06-28T15:08:12.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.228+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=9ZG62OKI54ZC type=del size=3499058 2024-06-28T15:08:12.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.232+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=3HJQ1E8O91Z2 type=del 2024-06-28T15:08:12.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.236+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~EW7I80KDQV3B.0 size=580150 2024-06-28T15:08:12.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.236+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=EW7I80KDQV3B tag=OA60X9VXU0RC type=write size=1628726 2024-06-28T15:08:12.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.236+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=J9SUD78GA210 type=del 2024-06-28T15:08:12.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.240+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=NTMSN7JKBNWZ type=write size=108986 2024-06-28T15:08:12.230 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.244+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=41WITA7X7RVY type=del 2024-06-28T15:08:12.234 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.248+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~KWQ1RP10KRJO.0 size=192862 2024-06-28T15:08:12.234 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.248+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=KWQ1RP10KRJO tag=9ZS798SNOX40 type=write size=1241438 2024-06-28T15:08:12.236 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.248+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=VVE7MEIGDX49 type=del 2024-06-28T15:08:12.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.252+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~1SURHM5WK2NB.0 size=442626 2024-06-28T15:08:12.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.252+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=1SURHM5WK2NB tag=V3ERC4ELFZTX type=write size=1491202 2024-06-28T15:08:12.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.256+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~K376Q9B2NBF1.0 size=120291 2024-06-28T15:08:12.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.260+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=K376Q9B2NBF1 tag=JCF552E7RQ11 type=write size=1168867 2024-06-28T15:08:12.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.260+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=IU3XPEOKSRZZ type=del size=1067637 2024-06-28T15:08:12.251 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.264+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~EECZ18YTBIR8.0 size=147558 2024-06-28T15:08:12.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.268+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=EECZ18YTBIR8 tag=701BB21BR2BD type=write size=1196134 2024-06-28T15:08:12.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.272+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~YXNRRLDYP3B3.0 size=1048576 2024-06-28T15:08:12.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.276+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~YXNRRLDYP3B3.1 size=1048576 2024-06-28T15:08:12.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.276+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~YXNRRLDYP3B3.2 size=310036 2024-06-28T15:08:12.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.276+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=YXNRRLDYP3B3 tag=73UHRZHYR81A type=write size=3455764 2024-06-28T15:08:12.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.280+0000 7f65680598c0 1 < completed write operation key=O1CGONN4TEXF tag=QAQTEBH2G5FC type=write size=738900 2024-06-28T15:08:12.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.280+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=NA4TTYIMKO7R type=del size=3597422 2024-06-28T15:08:12.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.280+0000 7f65680598c0 1 < canceled part key=_multipart_60QVJVV16QFM.2~TWIH44VPDEAE.0 size=1048576 2024-06-28T15:08:12.271 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.284+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~TWIH44VPDEAE.1 size=76073 2024-06-28T15:08:12.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.288+0000 7f65680598c0 1 < canceled multipart upload key=60QVJVV16QFM upload=TWIH44VPDEAE tag=2SN9VZV0REXS type=write size=2173225 2024-06-28T15:08:12.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.288+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=UALOXRGNFTR8 type=del size=1280671 2024-06-28T15:08:12.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.292+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~K0BSMEGZT04M.0 size=798658 2024-06-28T15:08:12.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.296+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=K0BSMEGZT04M tag=F3NJ98S1HN3Z type=write size=1847234 2024-06-28T15:08:12.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.296+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=MG97YHV93A7B type=del size=1000705 2024-06-28T15:08:12.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.300+0000 7f65680598c0 1 < canceled part key=_multipart_2FC9DNEZGWC4.2~K6ZR42HIWWQE.0 size=1048576 2024-06-28T15:08:12.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.300+0000 7f65680598c0 1 < canceled part key=_multipart_2FC9DNEZGWC4.2~K6ZR42HIWWQE.1 size=1048576 2024-06-28T15:08:12.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.300+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~K6ZR42HIWWQE.2 size=157891 2024-06-28T15:08:12.292 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.304+0000 7f65680598c0 1 < canceled multipart upload key=2FC9DNEZGWC4 upload=K6ZR42HIWWQE tag=FQT8B6IBHRLP type=write size=3303619 2024-06-28T15:08:12.294 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.308+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=XVJ1NPZLSLJ6 type=del size=1868047 2024-06-28T15:08:12.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.308+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=0ML6CQM1L9NM type=del 2024-06-28T15:08:12.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.312+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~OCF2QDLYD6YI.0 size=251383 2024-06-28T15:08:12.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.312+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=OCF2QDLYD6YI tag=XPWL9JLWA533 type=write size=1299959 2024-06-28T15:08:12.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.316+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~1ECN0LZJEKF0.0 size=1048576 2024-06-28T15:08:12.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.316+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~1ECN0LZJEKF0.1 size=661635 2024-06-28T15:08:12.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.320+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=1ECN0LZJEKF0 tag=KZYXY7I2O40N type=write size=2758787 2024-06-28T15:08:12.311 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.324+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=VNB17VS88TNV type=write size=849399 2024-06-28T15:08:12.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.328+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~MUNXEUQDDFYR.0 size=1048576 2024-06-28T15:08:12.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.328+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~MUNXEUQDDFYR.1 size=1048576 2024-06-28T15:08:12.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.332+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~MUNXEUQDDFYR.2 size=234324 2024-06-28T15:08:12.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.336+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=MUNXEUQDDFYR tag=F4ZYE23U2DVC type=write size=3380052 2024-06-28T15:08:12.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.340+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=F1CS2CP7IHYB type=write size=585241 2024-06-28T15:08:12.328 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.340+0000 7f65680598c0 1 < completed write operation key=R0IMFVF5NEOO tag=90IUOENOC5F8 type=write size=385128 2024-06-28T15:08:12.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.344+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=N4EB6V0BVI64 type=del size=859761 2024-06-28T15:08:12.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.344+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=Q0GNBX8E9ZEG type=del 2024-06-28T15:08:12.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.348+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=N9CR11Y94YDU type=del size=2855423 2024-06-28T15:08:12.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.352+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=9ZG62OKI54ZC type=del 2024-06-28T15:08:12.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.352+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=OLDGA3L1CMMK type=del size=1734446 2024-06-28T15:08:12.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.356+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~EW7I80KDQV3B.0 size=580150 2024-06-28T15:08:12.347 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.360+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=EW7I80KDQV3B tag=OA60X9VXU0RC type=write size=1628726 2024-06-28T15:08:12.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.364+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=7QDSIFJL074X type=write size=242411 2024-06-28T15:08:12.352 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.364+0000 7f65680598c0 1 < completed write operation key=EN76QQWIQAXZ tag=NTMSN7JKBNWZ type=write size=108986 2024-06-28T15:08:12.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.368+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~GJVF9XHPS81E.0 size=747028 2024-06-28T15:08:12.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.368+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=GJVF9XHPS81E tag=X00SKXREAGL7 type=write size=1795604 2024-06-28T15:08:12.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.372+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~KWQ1RP10KRJO.0 size=192862 2024-06-28T15:08:12.364 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.376+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=KWQ1RP10KRJO tag=9ZS798SNOX40 type=write size=1241438 2024-06-28T15:08:12.368 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.380+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=LEVNXEGE7NGV type=del size=1300606 2024-06-28T15:08:12.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.384+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~1SURHM5WK2NB.0 size=442626 2024-06-28T15:08:12.375 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.388+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=1SURHM5WK2NB tag=V3ERC4ELFZTX type=write size=1491202 2024-06-28T15:08:12.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.392+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~1KL03CIE0NZ4.0 size=1048576 2024-06-28T15:08:12.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.392+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~1KL03CIE0NZ4.1 size=1048576 2024-06-28T15:08:12.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.396+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~1KL03CIE0NZ4.2 size=938654 2024-06-28T15:08:12.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.396+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=1KL03CIE0NZ4 tag=IYYTZY7SZJS7 type=write size=4084382 2024-06-28T15:08:12.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.396+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=IU3XPEOKSRZZ type=del 2024-06-28T15:08:12.388 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.400+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=S1ZBK98GF2N2 type=del size=1195655 2024-06-28T15:08:12.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.404+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~YXNRRLDYP3B3.0 size=1048576 2024-06-28T15:08:12.392 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.404+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~YXNRRLDYP3B3.1 size=1048576 2024-06-28T15:08:12.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.408+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~YXNRRLDYP3B3.2 size=310036 2024-06-28T15:08:12.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.412+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=YXNRRLDYP3B3 tag=73UHRZHYR81A type=write size=3455764 2024-06-28T15:08:12.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.412+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=IR51H8RK4DTF type=del size=1583666 2024-06-28T15:08:12.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.416+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=NA4TTYIMKO7R type=del 2024-06-28T15:08:12.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.420+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=GO4H8Z66EUBZ type=del size=2208451 2024-06-28T15:08:12.411 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.424+0000 7f65680598c0 1 < suggested remove operation key=V2L2Q1U62YW2 tag=UALOXRGNFTR8 type=del 2024-06-28T15:08:12.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.428+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~4EFBH18DX26G.0 size=1048576 2024-06-28T15:08:12.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.428+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~4EFBH18DX26G.1 size=1048576 2024-06-28T15:08:12.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.432+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~4EFBH18DX26G.2 size=1024933 2024-06-28T15:08:12.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.432+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=4EFBH18DX26G tag=GB89JFOA5KX5 type=write size=4170661 2024-06-28T15:08:12.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.436+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=MG97YHV93A7B type=del 2024-06-28T15:08:12.427 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.440+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~YPBRIH5U4EN2.0 size=380989 2024-06-28T15:08:12.427 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.440+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=YPBRIH5U4EN2 tag=LC9FVHJ45KP7 type=write size=1429565 2024-06-28T15:08:12.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.444+0000 7f65680598c0 1 < suggested remove operation key=C5C6SAHLPZV1 tag=XVJ1NPZLSLJ6 type=del 2024-06-28T15:08:12.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.448+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=451XJ3T4GGDL type=del size=1445560 2024-06-28T15:08:12.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.452+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~OCF2QDLYD6YI.0 size=251383 2024-06-28T15:08:12.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.456+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=OCF2QDLYD6YI tag=XPWL9JLWA533 type=write size=1299959 2024-06-28T15:08:12.447 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.460+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~WDOQEEZL489F.0 size=571327 2024-06-28T15:08:12.447 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.460+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=WDOQEEZL489F tag=87439JOLFVH1 type=write size=1619903 2024-06-28T15:08:12.449 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.460+0000 7f65680598c0 1 < completed write operation key=O1CGONN4TEXF tag=VNB17VS88TNV type=write size=849399 2024-06-28T15:08:12.451 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.464+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=BXV00GWSCQFK type=del size=1309438 2024-06-28T15:08:12.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.468+0000 7f65680598c0 1 < completed write operation key=O1CGONN4TEXF tag=F1CS2CP7IHYB type=write size=585241 2024-06-28T15:08:12.456 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.468+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=3W6V0VLNGQRW type=del size=1466657 2024-06-28T15:08:12.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.472+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=N4EB6V0BVI64 type=del 2024-06-28T15:08:12.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.476+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~4GD19L4HZWJW.0 size=1048576 2024-06-28T15:08:12.466 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.480+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~4GD19L4HZWJW.1 size=1048576 2024-06-28T15:08:12.468 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.480+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~4GD19L4HZWJW.2 size=631630 2024-06-28T15:08:12.470 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.480+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=4GD19L4HZWJW tag=H5R1FJE65VYM type=write size=3777358 2024-06-28T15:08:12.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.484+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=N9CR11Y94YDU type=del 2024-06-28T15:08:12.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.488+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~SJK3B1KNBTTQ.0 size=936957 2024-06-28T15:08:12.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.488+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=SJK3B1KNBTTQ tag=87ARWMDI3095 type=write size=1985533 2024-06-28T15:08:12.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.492+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=OLDGA3L1CMMK type=del 2024-06-28T15:08:12.482 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.496+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=JRJAICO02L2C type=del size=1814657 2024-06-28T15:08:12.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.496+0000 7f65680598c0 -1 < failed to complete operation key=V2L2Q1U62YW2 tag=7QDSIFJL074X type=write size=242411: (22) Invalid argument 2024-06-28T15:08:12.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.500+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=AL6YVGEOUJLV type=del size=2255245 2024-06-28T15:08:12.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.504+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~GJVF9XHPS81E.0 size=747028 2024-06-28T15:08:12.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.508+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=GJVF9XHPS81E tag=X00SKXREAGL7 type=write size=1795604 2024-06-28T15:08:12.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.512+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=92C0WDBB5TDI type=write size=182717 2024-06-28T15:08:12.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.512+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=LEVNXEGE7NGV type=del 2024-06-28T15:08:12.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.516+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=Y9USKOHSZS6Q type=del size=3406890 2024-06-28T15:08:12.504 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.516+0000 7f65680598c0 1 < canceled part key=_multipart_AOBFAR0UTONH.2~1KL03CIE0NZ4.0 size=1048576 2024-06-28T15:08:12.506 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.520+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~1KL03CIE0NZ4.1 size=1048576 2024-06-28T15:08:12.509 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.520+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~1KL03CIE0NZ4.2 size=938654 2024-06-28T15:08:12.514 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.528+0000 7f65680598c0 1 < canceled multipart upload key=AOBFAR0UTONH upload=1KL03CIE0NZ4 tag=IYYTZY7SZJS7 type=write size=4084382 2024-06-28T15:08:12.520 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.532+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~2D42ZL2PWWRH.0 size=1048576 2024-06-28T15:08:12.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.536+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~2D42ZL2PWWRH.1 size=1048576 2024-06-28T15:08:12.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.536+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~2D42ZL2PWWRH.2 size=979743 2024-06-28T15:08:12.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.536+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=2D42ZL2PWWRH tag=TFCLQOCL8H66 type=write size=4125471 2024-06-28T15:08:12.528 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.540+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=S1ZBK98GF2N2 type=del 2024-06-28T15:08:12.531 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.544+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=SSSAOAB9BI2V type=del size=4154556 2024-06-28T15:08:12.537 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.548+0000 7f65680598c0 1 < suggested remove operation key=ZFNIA3O84EB7 tag=IR51H8RK4DTF type=del 2024-06-28T15:08:12.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.556+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~CCQLNRPF3QQC.0 size=164494 2024-06-28T15:08:12.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.556+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=CCQLNRPF3QQC tag=9BWZ9B5J5T9F type=write size=1213070 2024-06-28T15:08:12.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.560+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=GO4H8Z66EUBZ type=del 2024-06-28T15:08:12.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.560+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=ONZKT0VAKQ3H type=del size=174566 2024-06-28T15:08:12.552 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.564+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~4EFBH18DX26G.0 size=1048576 2024-06-28T15:08:12.554 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.568+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~4EFBH18DX26G.1 size=1048576 2024-06-28T15:08:12.557 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.568+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~4EFBH18DX26G.2 size=1024933 2024-06-28T15:08:12.562 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.576+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=4EFBH18DX26G tag=GB89JFOA5KX5 type=write size=4170661 2024-06-28T15:08:12.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.580+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~C8B63KM22QT7.0 size=1048576 2024-06-28T15:08:12.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.584+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~C8B63KM22QT7.1 size=1048576 2024-06-28T15:08:12.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.588+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~C8B63KM22QT7.2 size=810998 2024-06-28T15:08:12.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.588+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=C8B63KM22QT7 tag=Y8KGEFKMBJIH type=write size=3956726 2024-06-28T15:08:12.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.588+0000 7f65680598c0 1 < canceled part key=_multipart_2FC9DNEZGWC4.2~YPBRIH5U4EN2.0 size=380989 2024-06-28T15:08:12.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.592+0000 7f65680598c0 1 < canceled multipart upload key=2FC9DNEZGWC4 upload=YPBRIH5U4EN2 tag=LC9FVHJ45KP7 type=write size=1429565 2024-06-28T15:08:12.583 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.596+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=NTJKYKRDX972 type=del size=1598397 2024-06-28T15:08:12.586 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.600+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=451XJ3T4GGDL type=del 2024-06-28T15:08:12.592 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.604+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~29QEACNWJK88.0 size=1048576 2024-06-28T15:08:12.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.608+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~29QEACNWJK88.1 size=1048576 2024-06-28T15:08:12.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.608+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~29QEACNWJK88.2 size=495675 2024-06-28T15:08:12.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.608+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=29QEACNWJK88 tag=HETB3NEU37JJ type=write size=3641403 2024-06-28T15:08:12.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.612+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~WDOQEEZL489F.0 size=571327 2024-06-28T15:08:12.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.620+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=WDOQEEZL489F tag=87439JOLFVH1 type=write size=1619903 2024-06-28T15:08:12.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.624+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=R13ERRVX5381 type=del size=3509113 2024-06-28T15:08:12.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.624+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=BXV00GWSCQFK type=del 2024-06-28T15:08:12.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.628+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=4CU1SAR3TXTE type=del size=1544667 2024-06-28T15:08:12.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.632+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=3W6V0VLNGQRW type=del 2024-06-28T15:08:12.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.636+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=3EFY0Z1YC4A2 type=del size=2539846 2024-06-28T15:08:12.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.640+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~4GD19L4HZWJW.0 size=1048576 2024-06-28T15:08:12.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.640+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~4GD19L4HZWJW.1 size=1048576 2024-06-28T15:08:12.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.648+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~4GD19L4HZWJW.2 size=631630 2024-06-28T15:08:12.639 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.652+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=4GD19L4HZWJW tag=H5R1FJE65VYM type=write size=3777358 2024-06-28T15:08:12.643 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.656+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=H1PQINKWG0X4 type=del size=1008529 2024-06-28T15:08:12.646 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.660+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~SJK3B1KNBTTQ.0 size=936957 2024-06-28T15:08:12.651 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.664+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=SJK3B1KNBTTQ tag=87ARWMDI3095 type=write size=1985533 2024-06-28T15:08:12.658 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.672+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~2M9IU8BZQ5UT.0 size=1048576 2024-06-28T15:08:12.660 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.672+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~2M9IU8BZQ5UT.1 size=639490 2024-06-28T15:08:12.660 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.672+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=2M9IU8BZQ5UT tag=YKOCPFQVGMK5 type=write size=2736642 2024-06-28T15:08:12.664 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.676+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=JRJAICO02L2C type=del 2024-06-28T15:08:12.667 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.680+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=9C0U6JO6CF0F type=write size=668532 2024-06-28T15:08:12.670 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.684+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=AL6YVGEOUJLV type=del 2024-06-28T15:08:12.674 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.688+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=53941W746GPG type=del size=2493623 2024-06-28T15:08:12.678 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.688+0000 7f65680598c0 1 < completed write operation key=OUID7RX5NVV6 tag=92C0WDBB5TDI type=write size=182717 2024-06-28T15:08:12.681 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.692+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=XQ57RZ4RTAAD type=write size=97401 2024-06-28T15:08:12.683 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.696+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=Y9USKOHSZS6Q type=del 2024-06-28T15:08:12.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.700+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=VPW37DZ09XYZ type=del size=2067951 2024-06-28T15:08:12.690 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.704+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~2D42ZL2PWWRH.0 size=1048576 2024-06-28T15:08:12.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.704+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~2D42ZL2PWWRH.1 size=1048576 2024-06-28T15:08:12.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.704+0000 7f65680598c0 1 < canceled part key=_multipart_1VD7NDPEM3CD.2~2D42ZL2PWWRH.2 size=979743 2024-06-28T15:08:12.698 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.712+0000 7f65680598c0 1 < canceled multipart upload key=1VD7NDPEM3CD upload=2D42ZL2PWWRH tag=TFCLQOCL8H66 type=write size=4125471 2024-06-28T15:08:12.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.716+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~EUCHV7JB0MQ2.0 size=1048576 2024-06-28T15:08:12.706 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.720+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~EUCHV7JB0MQ2.1 size=1048576 2024-06-28T15:08:12.708 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.720+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~EUCHV7JB0MQ2.2 size=940721 2024-06-28T15:08:12.708 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.720+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=EUCHV7JB0MQ2 tag=LAJU8GYSCZF4 type=write size=4086449 2024-06-28T15:08:12.711 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.724+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=SSSAOAB9BI2V type=del 2024-06-28T15:08:12.714 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.728+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=WT7UM98CZAC5 type=del size=16638 2024-06-28T15:08:12.717 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.732+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~CCQLNRPF3QQC.0 size=164494 2024-06-28T15:08:12.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.736+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=CCQLNRPF3QQC tag=9BWZ9B5J5T9F type=write size=1213070 2024-06-28T15:08:12.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.740+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=6N89HLFDZ6T4 type=write size=411636 2024-06-28T15:08:12.731 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.744+0000 7f65680598c0 1 < suggested remove operation key=LZX5UZCYLTAG tag=ONZKT0VAKQ3H type=del 2024-06-28T15:08:12.734 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.748+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=3V4US643ZV2D type=del size=1082055 2024-06-28T15:08:12.737 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.748+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~C8B63KM22QT7.0 size=1048576 2024-06-28T15:08:12.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.752+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~C8B63KM22QT7.1 size=1048576 2024-06-28T15:08:12.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.756+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~C8B63KM22QT7.2 size=810998 2024-06-28T15:08:12.747 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.760+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=C8B63KM22QT7 tag=Y8KGEFKMBJIH type=write size=3956726 2024-06-28T15:08:12.752 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.764+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~7F1VBB8T1ICV.0 size=1048576 2024-06-28T15:08:12.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.768+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~7F1VBB8T1ICV.1 size=390405 2024-06-28T15:08:12.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.768+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=7F1VBB8T1ICV tag=BBVL7VIQ1O04 type=write size=2487557 2024-06-28T15:08:12.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.768+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=NTJKYKRDX972 type=del 2024-06-28T15:08:12.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.772+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=UI02H8FK63QU type=write size=847439 2024-06-28T15:08:12.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.776+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~29QEACNWJK88.0 size=1048576 2024-06-28T15:08:12.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.776+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~29QEACNWJK88.1 size=1048576 2024-06-28T15:08:12.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.780+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~29QEACNWJK88.2 size=495675 2024-06-28T15:08:12.772 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.784+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=29QEACNWJK88 tag=HETB3NEU37JJ type=write size=3641403 2024-06-28T15:08:12.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.788+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=ZYWPYPR9RCTH type=del size=437440 2024-06-28T15:08:12.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.792+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=R13ERRVX5381 type=del 2024-06-28T15:08:12.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.796+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~06H40PMGB7UW.0 size=1048576 2024-06-28T15:08:12.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.800+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~06H40PMGB7UW.1 size=1048576 2024-06-28T15:08:12.788 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.800+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~06H40PMGB7UW.2 size=937642 2024-06-28T15:08:12.788 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.800+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=06H40PMGB7UW tag=Y5TBOEV2O0NL type=write size=4083370 2024-06-28T15:08:12.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.804+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=4CU1SAR3TXTE type=del 2024-06-28T15:08:12.794 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.808+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=5OCB7T8C422L type=del size=3157784 2024-06-28T15:08:12.797 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.812+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=3EFY0Z1YC4A2 type=del 2024-06-28T15:08:12.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.812+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=MO83XAV2PIV0 type=del size=1003984 2024-06-28T15:08:12.808 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.820+0000 7f65680598c0 1 < suggested remove operation key=R0IMFVF5NEOO tag=H1PQINKWG0X4 type=del 2024-06-28T15:08:12.811 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.824+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=06XDAOVBWWIE type=del size=2014279 2024-06-28T15:08:12.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.828+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~2M9IU8BZQ5UT.0 size=1048576 2024-06-28T15:08:12.817 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.828+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~2M9IU8BZQ5UT.1 size=639490 2024-06-28T15:08:12.823 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.836+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=2M9IU8BZQ5UT tag=YKOCPFQVGMK5 type=write size=2736642 2024-06-28T15:08:12.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.840+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=EOXMFEOX4WIL type=write size=500850 2024-06-28T15:08:12.829 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.840+0000 7f65680598c0 1 < canceled operation key=OUID7RX5NVV6 tag=9C0U6JO6CF0F type=write size=668532 2024-06-28T15:08:12.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.844+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=1RS9C33IQBHI type=del size=2495126 2024-06-28T15:08:12.836 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.848+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=53941W746GPG type=del 2024-06-28T15:08:12.840 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.852+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=XOAA0FJU27VY type=write size=924216 2024-06-28T15:08:12.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.860+0000 7f65680598c0 1 < suggested update operation key=ZFNIA3O84EB7 tag=XQ57RZ4RTAAD type=write size=97401 2024-06-28T15:08:12.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.864+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~A2POXSRP4ZA7.0 size=1048576 2024-06-28T15:08:12.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.868+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~A2POXSRP4ZA7.1 size=643367 2024-06-28T15:08:12.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.868+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=A2POXSRP4ZA7 tag=TT4AC8E838E6 type=write size=2740519 2024-06-28T15:08:12.859 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.872+0000 7f65680598c0 1 < canceled operation key=P278ZM1H052S tag=VPW37DZ09XYZ type=del size=2067951 2024-06-28T15:08:12.862 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.876+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=2GNNAZOS1WU7 type=del size=929093 2024-06-28T15:08:12.866 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.880+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~EUCHV7JB0MQ2.0 size=1048576 2024-06-28T15:08:12.869 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.880+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~EUCHV7JB0MQ2.1 size=1048576 2024-06-28T15:08:12.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.884+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~EUCHV7JB0MQ2.2 size=940721 2024-06-28T15:08:12.877 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.892+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=EUCHV7JB0MQ2 tag=LAJU8GYSCZF4 type=write size=4086449 2024-06-28T15:08:12.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.892+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=JQECJVI73WW1 type=del size=3025026 2024-06-28T15:08:12.886 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.896+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=WT7UM98CZAC5 type=del 2024-06-28T15:08:12.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.900+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=4CWCKKKW026V type=write size=624050 2024-06-28T15:08:12.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.904+0000 7f65680598c0 1 < completed write operation key=P278ZM1H052S tag=6N89HLFDZ6T4 type=write size=411636 2024-06-28T15:08:12.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.904+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=UZNSXPGVNLVF type=del size=3516262 2024-06-28T15:08:12.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.908+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=3V4US643ZV2D type=del 2024-06-28T15:08:12.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.912+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=VUDI85G8BFR3 type=del size=1446329 2024-06-28T15:08:12.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.916+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~7F1VBB8T1ICV.0 size=1048576 2024-06-28T15:08:12.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.916+0000 7f65680598c0 1 < canceled part key=_multipart_ZY973GBGBQ1Z.2~7F1VBB8T1ICV.1 size=390405 2024-06-28T15:08:12.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.920+0000 7f65680598c0 1 < canceled multipart upload key=ZY973GBGBQ1Z upload=7F1VBB8T1ICV tag=BBVL7VIQ1O04 type=write size=2487557 2024-06-28T15:08:12.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.924+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=24PQMQ5I99CB type=del size=1835439 2024-06-28T15:08:12.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.924+0000 7f65680598c0 1 < completed write operation key=ZY973GBGBQ1Z tag=UI02H8FK63QU type=write size=847439 2024-06-28T15:08:12.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.932+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~462HC6D46W8K.0 size=1048576 2024-06-28T15:08:12.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.932+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~462HC6D46W8K.1 size=189749 2024-06-28T15:08:12.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.932+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=462HC6D46W8K tag=8EXTHTYC8MO5 type=write size=2286901 2024-06-28T15:08:12.925 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.936+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=ZYWPYPR9RCTH type=del 2024-06-28T15:08:12.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.948+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~I4QECNWRY9AA.0 size=83926 2024-06-28T15:08:12.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.948+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=I4QECNWRY9AA tag=8K0C11V4PVMT type=write size=1132502 2024-06-28T15:08:12.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.948+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~06H40PMGB7UW.0 size=1048576 2024-06-28T15:08:12.940 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.952+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~06H40PMGB7UW.1 size=1048576 2024-06-28T15:08:12.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.956+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~06H40PMGB7UW.2 size=937642 2024-06-28T15:08:12.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.960+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=06H40PMGB7UW tag=Y5TBOEV2O0NL type=write size=4083370 2024-06-28T15:08:12.952 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.964+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~GWWTH3NU4DPI.0 size=901590 2024-06-28T15:08:12.952 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.964+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=GWWTH3NU4DPI tag=U6AJOAXCZLZV type=write size=1950166 2024-06-28T15:08:12.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.968+0000 7f65680598c0 1 < suggested remove operation key=2F968YIMJN9D tag=5OCB7T8C422L type=del 2024-06-28T15:08:12.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.972+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=FT2D6X2C8UXS type=write size=871302 2024-06-28T15:08:12.963 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.976+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=MO83XAV2PIV0 type=del 2024-06-28T15:08:12.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.980+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=SR360SXIJDGC type=del size=1952467 2024-06-28T15:08:12.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.980+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=06XDAOVBWWIE type=del 2024-06-28T15:08:12.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.984+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=5TZ64F45B84S type=del size=2629150 2024-06-28T15:08:12.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.988+0000 7f65680598c0 1 < completed write operation key=O1CGONN4TEXF tag=EOXMFEOX4WIL type=write size=500850 2024-06-28T15:08:12.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.988+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=JD35L7Z8A4DJ type=del size=428970 2024-06-28T15:08:12.980 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:12.992+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=1RS9C33IQBHI type=del 2024-06-28T15:08:12.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.000+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~RCSZER9HDIZ4.0 size=1048576 2024-06-28T15:08:12.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.000+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~RCSZER9HDIZ4.1 size=1048576 2024-06-28T15:08:12.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.004+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~RCSZER9HDIZ4.2 size=599942 2024-06-28T15:08:12.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.004+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=RCSZER9HDIZ4 tag=MNZC768LX40T type=write size=3745670 2024-06-28T15:08:12.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.004+0000 7f65680598c0 1 < completed write operation key=KFHGA64L1NLP tag=XOAA0FJU27VY type=write size=924216 2024-06-28T15:08:12.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.008+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=XIRZ8SS75HTI type=del size=2937359 2024-06-28T15:08:12.999 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.012+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~A2POXSRP4ZA7.0 size=1048576 2024-06-28T15:08:13.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.016+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~A2POXSRP4ZA7.1 size=643367 2024-06-28T15:08:13.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.020+0000 7f65680598c0 1 < completed multipart upload key=MQ3QWZBFN4YR upload=A2POXSRP4ZA7 tag=TT4AC8E838E6 type=write size=2740519 2024-06-28T15:08:13.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.024+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=8N2GZ5VCAZLW type=write size=469693 2024-06-28T15:08:13.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.024+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=2GNNAZOS1WU7 type=del 2024-06-28T15:08:13.016 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.028+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=JD4EIESXQENK type=write size=98791 2024-06-28T15:08:13.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.032+0000 7f65680598c0 1 < canceled operation key=MQ3QWZBFN4YR tag=JQECJVI73WW1 type=del size=3025026 2024-06-28T15:08:13.023 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.036+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=FGU4K4Q3LVCP type=del size=2705155 2024-06-28T15:08:13.026 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.040+0000 7f65680598c0 1 < completed write operation key=D9D3OFIJP3D3 tag=4CWCKKKW026V type=write size=624050 2024-06-28T15:08:13.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.040+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=S7B2BZJE2NGS type=del size=1531242 2024-06-28T15:08:13.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.044+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=UZNSXPGVNLVF type=del 2024-06-28T15:08:13.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.048+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~WWI3UY61F2DJ.0 size=210807 2024-06-28T15:08:13.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.048+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=WWI3UY61F2DJ tag=2S3LQZGIPQJA type=write size=1259383 2024-06-28T15:08:13.040 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.052+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=VUDI85G8BFR3 type=del 2024-06-28T15:08:13.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.056+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~Z2NG4NIFZ5RK.0 size=519091 2024-06-28T15:08:13.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.056+0000 7f65680598c0 1 > prepared multipart upload key=2F968YIMJN9D upload=Z2NG4NIFZ5RK tag=9J82DSFGVKK4 type=write size=1567667 2024-06-28T15:08:13.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.060+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=24PQMQ5I99CB type=del 2024-06-28T15:08:13.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.064+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~S8A7HWHJDZP8.0 size=1048576 2024-06-28T15:08:13.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.068+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~S8A7HWHJDZP8.1 size=792326 2024-06-28T15:08:13.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.068+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=S8A7HWHJDZP8 tag=LAORABG9C004 type=write size=2889478 2024-06-28T15:08:13.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.072+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~462HC6D46W8K.0 size=1048576 2024-06-28T15:08:13.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.072+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~462HC6D46W8K.1 size=189749 2024-06-28T15:08:13.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.076+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=462HC6D46W8K tag=8EXTHTYC8MO5 type=write size=2286901 2024-06-28T15:08:13.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.084+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~WEKPHDTA46JQ.0 size=1048576 2024-06-28T15:08:13.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.084+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~WEKPHDTA46JQ.1 size=1048576 2024-06-28T15:08:13.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.088+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~WEKPHDTA46JQ.2 size=265885 2024-06-28T15:08:13.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.088+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=WEKPHDTA46JQ tag=EOIAK0JR3VLR type=write size=3411613 2024-06-28T15:08:13.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.092+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~I4QECNWRY9AA.0 size=83926 2024-06-28T15:08:13.082 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.096+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=I4QECNWRY9AA tag=8K0C11V4PVMT type=write size=1132502 2024-06-28T15:08:13.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.096+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=A6V95VZTAEW0 type=del size=2103781 2024-06-28T15:08:13.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.100+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~GWWTH3NU4DPI.0 size=901590 2024-06-28T15:08:13.093 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.104+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=GWWTH3NU4DPI tag=U6AJOAXCZLZV type=write size=1950166 2024-06-28T15:08:13.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.108+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=0K1C0KRXPW2T type=del size=3434605 2024-06-28T15:08:13.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.112+0000 7f65680598c0 1 < completed write operation key=2FC9DNEZGWC4 tag=FT2D6X2C8UXS type=write size=871302 2024-06-28T15:08:13.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.116+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=CMCX57REP91A type=del size=1979086 2024-06-28T15:08:13.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.116+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=SR360SXIJDGC type=del 2024-06-28T15:08:13.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.120+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=GVDQD2C2XLPE type=del size=3724 2024-06-28T15:08:13.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.124+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=5TZ64F45B84S type=del 2024-06-28T15:08:13.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.128+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~E5VZLSJVYWIS.0 size=1048576 2024-06-28T15:08:13.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.132+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~E5VZLSJVYWIS.1 size=104034 2024-06-28T15:08:13.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.132+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=E5VZLSJVYWIS tag=FB0T6U0W78KT type=write size=2201186 2024-06-28T15:08:13.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.136+0000 7f65680598c0 1 < suggested update operation key=ZY973GBGBQ1Z tag=JD35L7Z8A4DJ type=del size=428970 2024-06-28T15:08:13.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.144+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~90BLJ3X9L0I0.0 size=1048576 2024-06-28T15:08:13.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.144+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~90BLJ3X9L0I0.1 size=187007 2024-06-28T15:08:13.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.144+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=90BLJ3X9L0I0 tag=41P1GGRQFPIS type=write size=2284159 2024-06-28T15:08:13.136 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.148+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~RCSZER9HDIZ4.0 size=1048576 2024-06-28T15:08:13.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.152+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~RCSZER9HDIZ4.1 size=1048576 2024-06-28T15:08:13.141 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.152+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~RCSZER9HDIZ4.2 size=599942 2024-06-28T15:08:13.146 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.160+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=RCSZER9HDIZ4 tag=MNZC768LX40T type=write size=3745670 2024-06-28T15:08:13.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.160+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=00BSUW7T86NC type=del size=92185 2024-06-28T15:08:13.152 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.164+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=XIRZ8SS75HTI type=del 2024-06-28T15:08:13.155 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.168+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=70QTUCYRM81D type=del size=1589219 2024-06-28T15:08:13.158 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.172+0000 7f65680598c0 1 < completed write operation key=7NL3CTFYY9E5 tag=8N2GZ5VCAZLW type=write size=469693 2024-06-28T15:08:13.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.176+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=3WLFNGD37GIZ type=del size=647710 2024-06-28T15:08:13.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.176+0000 7f65680598c0 1 < completed write operation key=O1CGONN4TEXF tag=JD4EIESXQENK type=write size=98791 2024-06-28T15:08:13.168 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.180+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=PBYHN59Z7KY4 type=del size=2187400 2024-06-28T15:08:13.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.184+0000 7f65680598c0 1 < canceled operation key=BHW2VHUT0U1U tag=FGU4K4Q3LVCP type=del size=2705155 2024-06-28T15:08:13.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.188+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=ARU13DILU64Y type=write size=20089 2024-06-28T15:08:13.178 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.192+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=S7B2BZJE2NGS type=del 2024-06-28T15:08:13.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.196+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=ILXQ5T1ER6A8 type=del size=579699 2024-06-28T15:08:13.185 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.196+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~WWI3UY61F2DJ.0 size=210807 2024-06-28T15:08:13.190 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.204+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=WWI3UY61F2DJ tag=2S3LQZGIPQJA type=write size=1259383 2024-06-28T15:08:13.193 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.208+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=ISZFIB52XC6U type=del size=1458421 2024-06-28T15:08:13.197 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.208+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~Z2NG4NIFZ5RK.0 size=519091 2024-06-28T15:08:13.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.216+0000 7f65680598c0 1 < completed multipart upload key=2F968YIMJN9D upload=Z2NG4NIFZ5RK tag=9J82DSFGVKK4 type=write size=1567667 2024-06-28T15:08:13.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.216+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=O2PC277OB8OX type=write size=310433 2024-06-28T15:08:13.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.220+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~S8A7HWHJDZP8.0 size=1048576 2024-06-28T15:08:13.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.228+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~S8A7HWHJDZP8.1 size=792326 2024-06-28T15:08:13.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.232+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=S8A7HWHJDZP8 tag=LAORABG9C004 type=write size=2889478 2024-06-28T15:08:13.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.240+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~K11HQVWXJ2QK.0 size=1048576 2024-06-28T15:08:13.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.240+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~K11HQVWXJ2QK.1 size=629515 2024-06-28T15:08:13.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.240+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=K11HQVWXJ2QK tag=HEMF070GZXM5 type=write size=2726667 2024-06-28T15:08:13.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.244+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~WEKPHDTA46JQ.0 size=1048576 2024-06-28T15:08:13.233 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.244+0000 7f65680598c0 1 < canceled part key=_multipart_ZY973GBGBQ1Z.2~WEKPHDTA46JQ.1 size=1048576 2024-06-28T15:08:13.235 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.248+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~WEKPHDTA46JQ.2 size=265885 2024-06-28T15:08:13.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.252+0000 7f65680598c0 1 < canceled multipart upload key=ZY973GBGBQ1Z upload=WEKPHDTA46JQ tag=EOIAK0JR3VLR type=write size=3411613 2024-06-28T15:08:13.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.256+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=4HBPHW0FNRLM type=del size=2162232 2024-06-28T15:08:13.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.260+0000 7f65680598c0 -1 < failed to complete operation key=ZY973GBGBQ1Z tag=A6V95VZTAEW0 type=del size=2103781: (22) Invalid argument 2024-06-28T15:08:13.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.264+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=R521TVD0CXJ4 type=del size=1279683 2024-06-28T15:08:13.252 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.264+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=0K1C0KRXPW2T type=del 2024-06-28T15:08:13.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.268+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=9UK2752G4717 type=del size=2382638 2024-06-28T15:08:13.258 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.272+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=CMCX57REP91A type=del 2024-06-28T15:08:13.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.276+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=TS54D7OR4UE0 type=write size=601891 2024-06-28T15:08:13.268 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.280+0000 7f65680598c0 1 < suggested update operation key=P278ZM1H052S tag=GVDQD2C2XLPE type=del size=3724 2024-06-28T15:08:13.271 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.284+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=MQ8ZES4DLM56 type=del size=4146083 2024-06-28T15:08:13.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.288+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~E5VZLSJVYWIS.0 size=1048576 2024-06-28T15:08:13.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.288+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~E5VZLSJVYWIS.1 size=104034 2024-06-28T15:08:13.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.296+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=E5VZLSJVYWIS tag=FB0T6U0W78KT type=write size=2201186 2024-06-28T15:08:13.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.296+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=CP1V2UKCIS8T type=del size=3413340 2024-06-28T15:08:13.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.300+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~90BLJ3X9L0I0.0 size=1048576 2024-06-28T15:08:13.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.304+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~90BLJ3X9L0I0.1 size=187007 2024-06-28T15:08:13.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.308+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=90BLJ3X9L0I0 tag=41P1GGRQFPIS type=write size=2284159 2024-06-28T15:08:13.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.308+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=N4M8GP654UGK type=del size=4165048 2024-06-28T15:08:13.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.316+0000 7f65680598c0 1 < suggested update operation key=AOBFAR0UTONH tag=00BSUW7T86NC type=del size=92185 2024-06-28T15:08:13.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.320+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=WRD1J8MN3EF1 type=del size=2594432 2024-06-28T15:08:13.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.320+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=70QTUCYRM81D type=del 2024-06-28T15:08:13.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.328+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~OYF0K9PG32FN.0 size=1048576 2024-06-28T15:08:13.317 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.328+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~OYF0K9PG32FN.1 size=1048576 2024-06-28T15:08:13.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.332+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~OYF0K9PG32FN.2 size=105697 2024-06-28T15:08:13.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.332+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=OYF0K9PG32FN tag=BH3TT7CZWQQD type=write size=3251425 2024-06-28T15:08:13.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.336+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=3WLFNGD37GIZ type=del 2024-06-28T15:08:13.328 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.340+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~PYG95R92O919.0 size=533687 2024-06-28T15:08:13.328 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.340+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=PYG95R92O919 tag=NA3PC6FU90DB type=write size=1582263 2024-06-28T15:08:13.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.344+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=PBYHN59Z7KY4 type=del 2024-06-28T15:08:13.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.348+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~IIRRB5EL1318.0 size=1048576 2024-06-28T15:08:13.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.352+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~IIRRB5EL1318.1 size=482397 2024-06-28T15:08:13.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.352+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=IIRRB5EL1318 tag=LWI0442FR41M type=write size=2579549 2024-06-28T15:08:13.342 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.356+0000 7f65680598c0 1 < completed write operation key=UP3XFF38IJXN tag=ARU13DILU64Y type=write size=20089 2024-06-28T15:08:13.345 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.356+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=XPMAFDFOAVWD type=del size=1394107 2024-06-28T15:08:13.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.360+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=ILXQ5T1ER6A8 type=del 2024-06-28T15:08:13.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.364+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=7W8NFOVR5SMB type=del size=3164123 2024-06-28T15:08:13.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.368+0000 7f65680598c0 1 < suggested update operation key=MQ3QWZBFN4YR tag=ISZFIB52XC6U type=del size=1458421 2024-06-28T15:08:13.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.372+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=1P4LHO61IQGJ type=del size=2948245 2024-06-28T15:08:13.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.376+0000 7f65680598c0 1 < completed write operation key=J311Q2712YM0 tag=O2PC277OB8OX type=write size=310433 2024-06-28T15:08:13.368 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.380+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~KFYRTPBTS41Z.0 size=1048576 2024-06-28T15:08:13.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.384+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~KFYRTPBTS41Z.1 size=1048576 2024-06-28T15:08:13.373 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.384+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~KFYRTPBTS41Z.2 size=601564 2024-06-28T15:08:13.373 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.384+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=KFYRTPBTS41Z tag=OWL785MQ5B3Z type=write size=3747292 2024-06-28T15:08:13.375 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.388+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~K11HQVWXJ2QK.0 size=1048576 2024-06-28T15:08:13.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.392+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~K11HQVWXJ2QK.1 size=629515 2024-06-28T15:08:13.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.396+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=K11HQVWXJ2QK tag=HEMF070GZXM5 type=write size=2726667 2024-06-28T15:08:13.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.400+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~1XMSIZW10T35.0 size=826073 2024-06-28T15:08:13.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.400+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=1XMSIZW10T35 tag=TI31OZZNH6OG type=write size=1874649 2024-06-28T15:08:13.393 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.404+0000 7f65680598c0 1 < suggested remove operation key=2FC9DNEZGWC4 tag=4HBPHW0FNRLM type=del 2024-06-28T15:08:13.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.412+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~615SEPUYWA2T.0 size=1048576 2024-06-28T15:08:13.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.412+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~615SEPUYWA2T.1 size=251424 2024-06-28T15:08:13.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.412+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=615SEPUYWA2T tag=IFKUP2NXPB8O type=write size=2348576 2024-06-28T15:08:13.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.416+0000 7f65680598c0 -1 < failed to complete operation key=P278ZM1H052S tag=R521TVD0CXJ4 type=del size=1279683: (22) Invalid argument 2024-06-28T15:08:13.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.420+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=MWRMK191CI8Y type=del size=3032837 2024-06-28T15:08:13.411 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.424+0000 7f65680598c0 1 < suggested update operation key=P278ZM1H052S tag=9UK2752G4717 type=del size=2382638 2024-06-28T15:08:13.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.428+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=BERT08SZ9I44 type=del size=2841782 2024-06-28T15:08:13.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.432+0000 7f65680598c0 1 < canceled operation key=OUID7RX5NVV6 tag=TS54D7OR4UE0 type=write size=601891 2024-06-28T15:08:13.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.436+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=DXC7O33KHRMG type=del size=377022 2024-06-28T15:08:13.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.436+0000 7f65680598c0 -1 < failed to complete operation key=2FC9DNEZGWC4 tag=MQ8ZES4DLM56 type=del size=4146083: (22) Invalid argument 2024-06-28T15:08:13.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.440+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=P2CAOJP1DN8P type=del size=1436693 2024-06-28T15:08:13.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.444+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=CP1V2UKCIS8T type=del 2024-06-28T15:08:13.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.452+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~3YX4YQ0XYM1D.0 size=1048576 2024-06-28T15:08:13.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.452+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~3YX4YQ0XYM1D.1 size=1048576 2024-06-28T15:08:13.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.456+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~3YX4YQ0XYM1D.2 size=888583 2024-06-28T15:08:13.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.456+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=3YX4YQ0XYM1D tag=MPX3MH3OGW8Q type=write size=4034311 2024-06-28T15:08:13.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.460+0000 7f65680598c0 -1 < failed to complete operation key=MQ3QWZBFN4YR tag=N4M8GP654UGK type=del size=4165048: (22) Invalid argument 2024-06-28T15:08:13.451 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.464+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~GKYK8TVD2GK1.0 size=1048576 2024-06-28T15:08:13.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.468+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~GKYK8TVD2GK1.1 size=1048576 2024-06-28T15:08:13.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.468+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~GKYK8TVD2GK1.2 size=963356 2024-06-28T15:08:13.456 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.468+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=GKYK8TVD2GK1 tag=34UOANHVDA9K type=write size=4109084 2024-06-28T15:08:13.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.472+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=WRD1J8MN3EF1 type=del 2024-06-28T15:08:13.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.476+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=7X3CVYYL8KZ7 type=del size=4122986 2024-06-28T15:08:13.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.476+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~OYF0K9PG32FN.0 size=1048576 2024-06-28T15:08:13.466 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.480+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~OYF0K9PG32FN.1 size=1048576 2024-06-28T15:08:13.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.480+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~OYF0K9PG32FN.2 size=105697 2024-06-28T15:08:13.473 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.484+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=OYF0K9PG32FN tag=BH3TT7CZWQQD type=write size=3251425 2024-06-28T15:08:13.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.488+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=RW5GK3L6YO2F type=del size=4162369 2024-06-28T15:08:13.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.492+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~PYG95R92O919.0 size=533687 2024-06-28T15:08:13.483 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.496+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=PYG95R92O919 tag=NA3PC6FU90DB type=write size=1582263 2024-06-28T15:08:13.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.500+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~N444X4DTRMPF.0 size=688235 2024-06-28T15:08:13.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.500+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=N444X4DTRMPF tag=4A6SAVGIUVQZ type=write size=1736811 2024-06-28T15:08:13.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.508+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~IIRRB5EL1318.0 size=1048576 2024-06-28T15:08:13.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.508+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~IIRRB5EL1318.1 size=482397 2024-06-28T15:08:13.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.516+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=IIRRB5EL1318 tag=LWI0442FR41M type=write size=2579549 2024-06-28T15:08:13.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.520+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~W4ZLTS19HKBU.0 size=1048576 2024-06-28T15:08:13.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.524+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~W4ZLTS19HKBU.1 size=542540 2024-06-28T15:08:13.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.524+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=W4ZLTS19HKBU tag=D1K9FWLIZ105 type=write size=2639692 2024-06-28T15:08:13.513 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.524+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=XPMAFDFOAVWD type=del 2024-06-28T15:08:13.516 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.528+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=OJMF2I0I00I0 type=del size=861304 2024-06-28T15:08:13.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.532+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=7W8NFOVR5SMB type=del 2024-06-28T15:08:13.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.536+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=9UUBJGVI4L8H type=write size=543279 2024-06-28T15:08:13.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.540+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=1P4LHO61IQGJ type=del 2024-06-28T15:08:13.529 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.540+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=BWGVIH9FL60E type=write size=1004155 2024-06-28T15:08:13.532 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.544+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~KFYRTPBTS41Z.0 size=1048576 2024-06-28T15:08:13.535 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.548+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~KFYRTPBTS41Z.1 size=1048576 2024-06-28T15:08:13.538 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.552+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~KFYRTPBTS41Z.2 size=601564 2024-06-28T15:08:13.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.556+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=KFYRTPBTS41Z tag=OWL785MQ5B3Z type=write size=3747292 2024-06-28T15:08:13.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.560+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=IV8AUUB8HMQH type=del size=3260740 2024-06-28T15:08:13.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.564+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~1XMSIZW10T35.0 size=826073 2024-06-28T15:08:13.555 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.568+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=1XMSIZW10T35 tag=TI31OZZNH6OG type=write size=1874649 2024-06-28T15:08:13.558 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.572+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=MQGMGP3MBNRA type=del size=3871742 2024-06-28T15:08:13.561 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.576+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~615SEPUYWA2T.0 size=1048576 2024-06-28T15:08:13.564 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.576+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~615SEPUYWA2T.1 size=251424 2024-06-28T15:08:13.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.584+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=615SEPUYWA2T tag=IFKUP2NXPB8O type=write size=2348576 2024-06-28T15:08:13.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.588+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=2PCR76MC70QK type=del size=949286 2024-06-28T15:08:13.577 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.588+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=MWRMK191CI8Y type=del 2024-06-28T15:08:13.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.596+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~Q8HRWB4XYND1.0 size=1048576 2024-06-28T15:08:13.586 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.600+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~Q8HRWB4XYND1.1 size=442691 2024-06-28T15:08:13.586 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.600+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=Q8HRWB4XYND1 tag=KL769YS69865 type=write size=2539843 2024-06-28T15:08:13.590 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.604+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=BERT08SZ9I44 type=del 2024-06-28T15:08:13.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.608+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~JUQB21960CSJ.0 size=397714 2024-06-28T15:08:13.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.608+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=JUQB21960CSJ tag=LDS2R1L0PXIH type=write size=1446290 2024-06-28T15:08:13.603 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.616+0000 7f65680598c0 1 < suggested update operation key=LZX5UZCYLTAG tag=DXC7O33KHRMG type=del size=377022 2024-06-28T15:08:13.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.620+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=4RX9U5X9R8FJ type=del size=1632548 2024-06-28T15:08:13.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.620+0000 7f65680598c0 1 < canceled operation key=OUID7RX5NVV6 tag=P2CAOJP1DN8P type=del size=1436693 2024-06-28T15:08:13.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.624+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=T7QWMYZ4A6SY type=del size=4191388 2024-06-28T15:08:13.615 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.628+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~3YX4YQ0XYM1D.0 size=1048576 2024-06-28T15:08:13.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.632+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~3YX4YQ0XYM1D.1 size=1048576 2024-06-28T15:08:13.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.632+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~3YX4YQ0XYM1D.2 size=888583 2024-06-28T15:08:13.625 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.636+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=3YX4YQ0XYM1D tag=MPX3MH3OGW8Q type=write size=4034311 2024-06-28T15:08:13.629 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.640+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=BVU5D3WRHE12 type=del size=3907700 2024-06-28T15:08:13.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.644+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~GKYK8TVD2GK1.0 size=1048576 2024-06-28T15:08:13.635 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.648+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~GKYK8TVD2GK1.1 size=1048576 2024-06-28T15:08:13.638 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.652+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~GKYK8TVD2GK1.2 size=963356 2024-06-28T15:08:13.643 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.656+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=GKYK8TVD2GK1 tag=34UOANHVDA9K type=write size=4109084 2024-06-28T15:08:13.646 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.660+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=AQ3JL5EXHQ0B type=del size=1803570 2024-06-28T15:08:13.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.664+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=7X3CVYYL8KZ7 type=del 2024-06-28T15:08:13.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.668+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~P0MVUDN9HGVT.0 size=672665 2024-06-28T15:08:13.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.668+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=P0MVUDN9HGVT tag=XH02KCRE3YOM type=write size=1721241 2024-06-28T15:08:13.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.672+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=RW5GK3L6YO2F type=del 2024-06-28T15:08:13.662 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.676+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=TZ1JC1K93Y5C type=del size=658143 2024-06-28T15:08:13.663 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.676+0000 7f65680598c0 1 < canceled part key=_multipart_ZFNIA3O84EB7.2~N444X4DTRMPF.0 size=688235 2024-06-28T15:08:13.667 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.680+0000 7f65680598c0 1 < canceled multipart upload key=ZFNIA3O84EB7 upload=N444X4DTRMPF tag=4A6SAVGIUVQZ type=write size=1736811 2024-06-28T15:08:13.671 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.684+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=WXZ5E0I7Y6WO type=del size=246507 2024-06-28T15:08:13.673 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.688+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~W4ZLTS19HKBU.0 size=1048576 2024-06-28T15:08:13.676 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.688+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~W4ZLTS19HKBU.1 size=542540 2024-06-28T15:08:13.680 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.692+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=W4ZLTS19HKBU tag=D1K9FWLIZ105 type=write size=2639692 2024-06-28T15:08:13.686 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.700+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~VWLSPSMX0NE7.0 size=1048576 2024-06-28T15:08:13.689 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.700+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~VWLSPSMX0NE7.1 size=1048576 2024-06-28T15:08:13.691 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.704+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~VWLSPSMX0NE7.2 size=485329 2024-06-28T15:08:13.691 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.704+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=VWLSPSMX0NE7 tag=08QXMBPB7N51 type=write size=3631057 2024-06-28T15:08:13.694 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.708+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=OJMF2I0I00I0 type=del 2024-06-28T15:08:13.697 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.708+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=7XZW4W6N08H3 type=del size=2880530 2024-06-28T15:08:13.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.712+0000 7f65680598c0 1 < canceled operation key=4CU86XS1R9G3 tag=9UUBJGVI4L8H type=write size=543279 2024-06-28T15:08:13.706 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.720+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~F8M9PPN90975.0 size=1048576 2024-06-28T15:08:13.709 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.720+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~F8M9PPN90975.1 size=1048576 2024-06-28T15:08:13.712 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.724+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~F8M9PPN90975.2 size=372834 2024-06-28T15:08:13.712 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.724+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=F8M9PPN90975 tag=XYPJ7YA0RZGK type=write size=3518562 2024-06-28T15:08:13.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.728+0000 7f65680598c0 1 < completed write operation key=J311Q2712YM0 tag=BWGVIH9FL60E type=write size=1004155 2024-06-28T15:08:13.718 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.732+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=UJCDKZGFLE8C type=del size=581601 2024-06-28T15:08:13.721 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.732+0000 7f65680598c0 1 < completed delete operation key=1VD7NDPEM3CD tag=IV8AUUB8HMQH type=del 2024-06-28T15:08:13.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.740+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~2CX3F4604U17.0 size=76276 2024-06-28T15:08:13.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.740+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=2CX3F4604U17 tag=THG41OW3LV3I type=write size=1124852 2024-06-28T15:08:13.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.744+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=MQGMGP3MBNRA type=del 2024-06-28T15:08:13.736 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.748+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~R6GFHTXD1VQX.0 size=152969 2024-06-28T15:08:13.736 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.748+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=R6GFHTXD1VQX tag=894JO35L2KMN type=write size=1201545 2024-06-28T15:08:13.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.752+0000 7f65680598c0 1 < completed delete operation key=ZFNIA3O84EB7 tag=2PCR76MC70QK type=del 2024-06-28T15:08:13.745 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.756+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~7VD5S08ZGAHV.0 size=1009553 2024-06-28T15:08:13.745 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.756+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=7VD5S08ZGAHV tag=ZG8UG8AN5WBW type=write size=2058129 2024-06-28T15:08:13.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.760+0000 7f65680598c0 1 < canceled part key=_multipart_ZY973GBGBQ1Z.2~Q8HRWB4XYND1.0 size=1048576 2024-06-28T15:08:13.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.760+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~Q8HRWB4XYND1.1 size=442691 2024-06-28T15:08:13.753 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.764+0000 7f65680598c0 1 < canceled multipart upload key=ZY973GBGBQ1Z upload=Q8HRWB4XYND1 tag=KL769YS69865 type=write size=2539843 2024-06-28T15:08:13.756 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.768+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=F19XYH0ZKADR type=del size=774935 2024-06-28T15:08:13.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.772+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~JUQB21960CSJ.0 size=397714 2024-06-28T15:08:13.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.776+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=JUQB21960CSJ tag=LDS2R1L0PXIH type=write size=1446290 2024-06-28T15:08:13.771 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.784+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~KCLHUB0H66RB.0 size=314752 2024-06-28T15:08:13.771 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.784+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=KCLHUB0H66RB tag=U9IM36YLAZOV type=write size=1363328 2024-06-28T15:08:13.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.788+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=4RX9U5X9R8FJ type=del 2024-06-28T15:08:13.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.788+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=IETV3JGV60FB type=del size=3814454 2024-06-28T15:08:13.779 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.792+0000 7f65680598c0 1 < canceled operation key=2FC9DNEZGWC4 tag=T7QWMYZ4A6SY type=del size=4191388 2024-06-28T15:08:13.786 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.800+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~5LCG1K3BJRVZ.0 size=1048576 2024-06-28T15:08:13.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.804+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~5LCG1K3BJRVZ.1 size=1048576 2024-06-28T15:08:13.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.808+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~5LCG1K3BJRVZ.2 size=256258 2024-06-28T15:08:13.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.808+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=5LCG1K3BJRVZ tag=V7SHRMH0EEPA type=write size=3401986 2024-06-28T15:08:13.796 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.808+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=BVU5D3WRHE12 type=del 2024-06-28T15:08:13.798 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.812+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=W6UPXE8FCCC0 type=write size=396596 2024-06-28T15:08:13.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.812+0000 7f65680598c0 1 < canceled operation key=RLJCNXKA2GYA tag=AQ3JL5EXHQ0B type=del size=1803570 2024-06-28T15:08:13.807 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.820+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~M3X2IS618OVE.0 size=1008038 2024-06-28T15:08:13.807 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.820+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=M3X2IS618OVE tag=V6HDP95O4B59 type=write size=2056614 2024-06-28T15:08:13.810 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.824+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~P0MVUDN9HGVT.0 size=672665 2024-06-28T15:08:13.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.828+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=P0MVUDN9HGVT tag=XH02KCRE3YOM type=write size=1721241 2024-06-28T15:08:13.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.832+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~CDUCNT06BD02.0 size=1048576 2024-06-28T15:08:13.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.836+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~CDUCNT06BD02.1 size=1048576 2024-06-28T15:08:13.824 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.836+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~CDUCNT06BD02.2 size=5111 2024-06-28T15:08:13.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.836+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=CDUCNT06BD02 tag=9GW9AITWQB31 type=write size=3150839 2024-06-28T15:08:13.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.840+0000 7f65680598c0 1 < canceled operation key=KFHGA64L1NLP tag=TZ1JC1K93Y5C type=del size=658143 2024-06-28T15:08:13.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.848+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~36L5NJP2CS0J.0 size=671512 2024-06-28T15:08:13.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.848+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=36L5NJP2CS0J tag=XOCHJN3BY58N type=write size=1720088 2024-06-28T15:08:13.840 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.852+0000 7f65680598c0 1 < suggested update operation key=D9D3OFIJP3D3 tag=WXZ5E0I7Y6WO type=del size=246507 2024-06-28T15:08:13.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.856+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=BEW4HNLIEHXO type=del size=672436 2024-06-28T15:08:13.846 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.860+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~VWLSPSMX0NE7.0 size=1048576 2024-06-28T15:08:13.848 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.860+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~VWLSPSMX0NE7.1 size=1048576 2024-06-28T15:08:13.848 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.860+0000 7f65680598c0 1 < canceled part key=_multipart_UP3XFF38IJXN.2~VWLSPSMX0NE7.2 size=485329 2024-06-28T15:08:13.854 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.868+0000 7f65680598c0 1 < canceled multipart upload key=UP3XFF38IJXN upload=VWLSPSMX0NE7 tag=08QXMBPB7N51 type=write size=3631057 2024-06-28T15:08:13.860 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.872+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~EX4CSQ84Z8H4.0 size=509285 2024-06-28T15:08:13.860 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.872+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=EX4CSQ84Z8H4 tag=W11Y8SUGK4X0 type=write size=1557861 2024-06-28T15:08:13.863 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.876+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=7XZW4W6N08H3 type=del 2024-06-28T15:08:13.869 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.884+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~58X8YQGOHXH6.0 size=1048576 2024-06-28T15:08:13.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.884+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~58X8YQGOHXH6.1 size=228080 2024-06-28T15:08:13.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.884+0000 7f65680598c0 1 > prepared multipart upload key=2F968YIMJN9D upload=58X8YQGOHXH6 tag=NJZTLFR3G0CY type=write size=2325232 2024-06-28T15:08:13.876 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.888+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~F8M9PPN90975.0 size=1048576 2024-06-28T15:08:13.878 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.892+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~F8M9PPN90975.1 size=1048576 2024-06-28T15:08:13.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.896+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~F8M9PPN90975.2 size=372834 2024-06-28T15:08:13.886 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.900+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=F8M9PPN90975 tag=XYPJ7YA0RZGK type=write size=3518562 2024-06-28T15:08:13.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.904+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~XNMJQESA2EU3.0 size=1048576 2024-06-28T15:08:13.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.908+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~XNMJQESA2EU3.1 size=1048576 2024-06-28T15:08:13.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.908+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~XNMJQESA2EU3.2 size=154095 2024-06-28T15:08:13.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.908+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=XNMJQESA2EU3 tag=F9DEE4GX2TLX type=write size=3299823 2024-06-28T15:08:13.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.912+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=UJCDKZGFLE8C type=del 2024-06-28T15:08:13.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.916+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=44ST88M9BGAU type=del size=3603786 2024-06-28T15:08:13.905 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.916+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~2CX3F4604U17.0 size=76276 2024-06-28T15:08:13.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.924+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=2CX3F4604U17 tag=THG41OW3LV3I type=write size=1124852 2024-06-28T15:08:13.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.924+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=9DE7CQ9FMVCC type=del size=2359544 2024-06-28T15:08:13.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.928+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~R6GFHTXD1VQX.0 size=152969 2024-06-28T15:08:13.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.936+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=R6GFHTXD1VQX tag=894JO35L2KMN type=write size=1201545 2024-06-28T15:08:13.925 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.936+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=D4NEG2OHIHYQ type=del size=3459847 2024-06-28T15:08:13.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.940+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~7VD5S08ZGAHV.0 size=1009553 2024-06-28T15:08:13.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.944+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=7VD5S08ZGAHV tag=ZG8UG8AN5WBW type=write size=2058129 2024-06-28T15:08:13.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.948+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=0WE0Q5O6CGNW type=del size=2729641 2024-06-28T15:08:13.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.952+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=F19XYH0ZKADR type=del 2024-06-28T15:08:13.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.956+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~2E40SJ7KAAXE.0 size=1048576 2024-06-28T15:08:13.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.956+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~2E40SJ7KAAXE.1 size=69404 2024-06-28T15:08:13.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.956+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=2E40SJ7KAAXE tag=60WL43I1BGUD type=write size=2166556 2024-06-28T15:08:13.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.960+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~KCLHUB0H66RB.0 size=314752 2024-06-28T15:08:13.951 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.964+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=KCLHUB0H66RB tag=U9IM36YLAZOV type=write size=1363328 2024-06-28T15:08:13.954 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.968+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=CY4GSQF8BPYZ type=del size=3433770 2024-06-28T15:08:13.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.968+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=IETV3JGV60FB type=del 2024-06-28T15:08:13.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.972+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=R4PQ1QWP4GYI type=write size=859005 2024-06-28T15:08:13.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.976+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~5LCG1K3BJRVZ.0 size=1048576 2024-06-28T15:08:13.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.976+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~5LCG1K3BJRVZ.1 size=1048576 2024-06-28T15:08:13.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.980+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~5LCG1K3BJRVZ.2 size=256258 2024-06-28T15:08:13.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.984+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=5LCG1K3BJRVZ tag=V7SHRMH0EEPA type=write size=3401986 2024-06-28T15:08:13.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.988+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~1LWN9QG8BOZT.0 size=1048576 2024-06-28T15:08:13.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.992+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~1LWN9QG8BOZT.1 size=1004666 2024-06-28T15:08:13.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.992+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=1LWN9QG8BOZT tag=IT7744J73C2E type=write size=3101818 2024-06-28T15:08:13.981 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.996+0000 7f65680598c0 1 < canceled operation key=60QVJVV16QFM tag=W6UPXE8FCCC0 type=write size=396596 2024-06-28T15:08:13.984 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:13.996+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=YCQRN4W04C5P type=del size=1090751 2024-06-28T15:08:13.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.000+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~M3X2IS618OVE.0 size=1008038 2024-06-28T15:08:13.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.004+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=M3X2IS618OVE tag=V6HDP95O4B59 type=write size=2056614 2024-06-28T15:08:13.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.004+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=MWEMFJF55PEY type=del size=1789778 2024-06-28T15:08:13.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.008+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~CDUCNT06BD02.0 size=1048576 2024-06-28T15:08:13.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.008+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~CDUCNT06BD02.1 size=1048576 2024-06-28T15:08:13.999 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.012+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~CDUCNT06BD02.2 size=5111 2024-06-28T15:08:14.003 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.016+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=CDUCNT06BD02 tag=9GW9AITWQB31 type=write size=3150839 2024-06-28T15:08:14.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.020+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~F1ND95A73YRO.0 size=1048576 2024-06-28T15:08:14.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.024+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~F1ND95A73YRO.1 size=1048576 2024-06-28T15:08:14.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.024+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~F1ND95A73YRO.2 size=280503 2024-06-28T15:08:14.012 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.024+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=F1ND95A73YRO tag=3EUW98OKDT3Q type=write size=3426231 2024-06-28T15:08:14.014 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.028+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~36L5NJP2CS0J.0 size=671512 2024-06-28T15:08:14.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.032+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=36L5NJP2CS0J tag=XOCHJN3BY58N type=write size=1720088 2024-06-28T15:08:14.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.032+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=CZBEM9HW73S8 type=del size=3061515 2024-06-28T15:08:14.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.036+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=BEW4HNLIEHXO type=del 2024-06-28T15:08:14.026 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.040+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~4ZB5Z1ZL3DYK.0 size=1048576 2024-06-28T15:08:14.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.040+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~4ZB5Z1ZL3DYK.1 size=1048576 2024-06-28T15:08:14.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.044+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~4ZB5Z1ZL3DYK.2 size=538557 2024-06-28T15:08:14.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.044+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=4ZB5Z1ZL3DYK tag=N5AHIDYM8Y6M type=write size=3684285 2024-06-28T15:08:14.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.044+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~EX4CSQ84Z8H4.0 size=509285 2024-06-28T15:08:14.036 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.048+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=EX4CSQ84Z8H4 tag=W11Y8SUGK4X0 type=write size=1557861 2024-06-28T15:08:14.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.052+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=C7BUBI1BWBO3 type=del size=383573 2024-06-28T15:08:14.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.052+0000 7f65680598c0 1 < canceled part key=_multipart_2F968YIMJN9D.2~58X8YQGOHXH6.0 size=1048576 2024-06-28T15:08:14.040 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.052+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~58X8YQGOHXH6.1 size=228080 2024-06-28T15:08:14.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.060+0000 7f65680598c0 1 < canceled multipart upload key=2F968YIMJN9D upload=58X8YQGOHXH6 tag=NJZTLFR3G0CY type=write size=2325232 2024-06-28T15:08:14.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.064+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~CPKRP4TRKJUA.0 size=1048576 2024-06-28T15:08:14.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.064+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~CPKRP4TRKJUA.1 size=673082 2024-06-28T15:08:14.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.064+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=CPKRP4TRKJUA tag=H2G4OMHF8AGC type=write size=2770234 2024-06-28T15:08:14.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.068+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~XNMJQESA2EU3.0 size=1048576 2024-06-28T15:08:14.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.068+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~XNMJQESA2EU3.1 size=1048576 2024-06-28T15:08:14.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.072+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~XNMJQESA2EU3.2 size=154095 2024-06-28T15:08:14.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.076+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=XNMJQESA2EU3 tag=F9DEE4GX2TLX type=write size=3299823 2024-06-28T15:08:14.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.076+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=58SYDUYNYTYD type=del size=3056409 2024-06-28T15:08:14.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.080+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=44ST88M9BGAU type=del 2024-06-28T15:08:14.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.084+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=M2OFR0XBEKWF type=del size=1382162 2024-06-28T15:08:14.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.084+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=9DE7CQ9FMVCC type=del 2024-06-28T15:08:14.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.088+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~VSQGIPD8OC7V.0 size=1048576 2024-06-28T15:08:14.078 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.092+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~VSQGIPD8OC7V.1 size=414358 2024-06-28T15:08:14.078 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.092+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=VSQGIPD8OC7V tag=9ZSSOG2CJVOB type=write size=2511510 2024-06-28T15:08:14.083 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.096+0000 7f65680598c0 1 < suggested remove operation key=2F968YIMJN9D tag=D4NEG2OHIHYQ type=del 2024-06-28T15:08:14.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.100+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~5CP0L7MZ0H9T.0 size=1048576 2024-06-28T15:08:14.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.104+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~5CP0L7MZ0H9T.1 size=225242 2024-06-28T15:08:14.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.104+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=5CP0L7MZ0H9T tag=LVW2FJWT7PZ9 type=write size=2322394 2024-06-28T15:08:14.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.104+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=0WE0Q5O6CGNW type=del 2024-06-28T15:08:14.096 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.108+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~2BXE9MDON6WE.0 size=1048576 2024-06-28T15:08:14.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.112+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~2BXE9MDON6WE.1 size=296548 2024-06-28T15:08:14.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.112+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=2BXE9MDON6WE tag=GOT1M9FG9HW0 type=write size=2393700 2024-06-28T15:08:14.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.112+0000 7f65680598c0 1 < canceled part key=_multipart_1VD7NDPEM3CD.2~2E40SJ7KAAXE.0 size=1048576 2024-06-28T15:08:14.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.112+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~2E40SJ7KAAXE.1 size=69404 2024-06-28T15:08:14.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.116+0000 7f65680598c0 1 < canceled multipart upload key=1VD7NDPEM3CD upload=2E40SJ7KAAXE tag=60WL43I1BGUD type=write size=2166556 2024-06-28T15:08:14.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.120+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=LEFNU3DN8281 type=del size=2648619 2024-06-28T15:08:14.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.120+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=CY4GSQF8BPYZ type=del 2024-06-28T15:08:14.112 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.124+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~7N9TXXQT63GL.0 size=1048576 2024-06-28T15:08:14.114 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.128+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~7N9TXXQT63GL.1 size=1048576 2024-06-28T15:08:14.115 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.128+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~7N9TXXQT63GL.2 size=83409 2024-06-28T15:08:14.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.128+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=7N9TXXQT63GL tag=AWJ6M7RVPP49 type=write size=3229137 2024-06-28T15:08:14.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.132+0000 7f65680598c0 1 < suggested remove operation key=P278ZM1H052S tag=R4PQ1QWP4GYI type=write 2024-06-28T15:08:14.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.136+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=DNP54BFVX584 type=del size=444594 2024-06-28T15:08:14.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.136+0000 7f65680598c0 1 < canceled part key=_multipart_BHW2VHUT0U1U.2~1LWN9QG8BOZT.0 size=1048576 2024-06-28T15:08:14.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.136+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~1LWN9QG8BOZT.1 size=1004666 2024-06-28T15:08:14.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.140+0000 7f65680598c0 1 < canceled multipart upload key=BHW2VHUT0U1U upload=1LWN9QG8BOZT tag=IT7744J73C2E type=write size=3101818 2024-06-28T15:08:14.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.144+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~TFL5XK5C2CHF.0 size=1048576 2024-06-28T15:08:14.132 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.144+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~TFL5XK5C2CHF.1 size=22689 2024-06-28T15:08:14.132 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.144+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=TFL5XK5C2CHF tag=VR4QN6LHAP6P type=write size=2119841 2024-06-28T15:08:14.134 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.148+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=YCQRN4W04C5P type=del 2024-06-28T15:08:14.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.152+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~8QZIHTT1V3N1.0 size=580915 2024-06-28T15:08:14.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.152+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=8QZIHTT1V3N1 tag=RZ60FLNIYH9T type=write size=1629491 2024-06-28T15:08:14.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.152+0000 7f65680598c0 1 < completed delete operation key=ZFNIA3O84EB7 tag=MWEMFJF55PEY type=del 2024-06-28T15:08:14.143 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.156+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~B0EPELME0BH2.0 size=1048576 2024-06-28T15:08:14.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.156+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~B0EPELME0BH2.1 size=1048576 2024-06-28T15:08:14.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.160+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~B0EPELME0BH2.2 size=1028291 2024-06-28T15:08:14.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.160+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=B0EPELME0BH2 tag=8UYLDGB1VLXI type=write size=4174019 2024-06-28T15:08:14.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.160+0000 7f65680598c0 1 < canceled part key=_multipart_2FC9DNEZGWC4.2~F1ND95A73YRO.0 size=1048576 2024-06-28T15:08:14.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.160+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~F1ND95A73YRO.1 size=1048576 2024-06-28T15:08:14.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.164+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~F1ND95A73YRO.2 size=280503 2024-06-28T15:08:14.155 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.168+0000 7f65680598c0 1 < canceled multipart upload key=2FC9DNEZGWC4 upload=F1ND95A73YRO tag=3EUW98OKDT3Q type=write size=3426231 2024-06-28T15:08:14.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.172+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~GJG07COPAD84.0 size=1048576 2024-06-28T15:08:14.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.172+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~GJG07COPAD84.1 size=365869 2024-06-28T15:08:14.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.172+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=GJG07COPAD84 tag=QGCR6VAV9GQI type=write size=2463021 2024-06-28T15:08:14.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.180+0000 7f65680598c0 1 < suggested remove operation key=SFCIZMLY2Y2E tag=CZBEM9HW73S8 type=del 2024-06-28T15:08:14.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.184+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~2CCX7DD7GG5T.0 size=1048576 2024-06-28T15:08:14.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.184+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~2CCX7DD7GG5T.1 size=709805 2024-06-28T15:08:14.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.184+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=2CCX7DD7GG5T tag=AFLNH6G6WTND type=write size=2806957 2024-06-28T15:08:14.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.188+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~4ZB5Z1ZL3DYK.0 size=1048576 2024-06-28T15:08:14.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.188+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~4ZB5Z1ZL3DYK.1 size=1048576 2024-06-28T15:08:14.177 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.188+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~4ZB5Z1ZL3DYK.2 size=538557 2024-06-28T15:08:14.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.192+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=4ZB5Z1ZL3DYK tag=N5AHIDYM8Y6M type=write size=3684285 2024-06-28T15:08:14.183 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.196+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=7EPBC3Z8WCZX type=write size=321941 2024-06-28T15:08:14.185 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.196+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=C7BUBI1BWBO3 type=del 2024-06-28T15:08:14.189 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.200+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~1RQ21ALFXHDX.0 size=1048576 2024-06-28T15:08:14.191 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.204+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~1RQ21ALFXHDX.1 size=108600 2024-06-28T15:08:14.191 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.204+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=1RQ21ALFXHDX tag=KY314UJ2E3H7 type=write size=2205752 2024-06-28T15:08:14.193 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.204+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~CPKRP4TRKJUA.0 size=1048576 2024-06-28T15:08:14.194 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.208+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~CPKRP4TRKJUA.1 size=673082 2024-06-28T15:08:14.198 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.212+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=CPKRP4TRKJUA tag=H2G4OMHF8AGC type=write size=2770234 2024-06-28T15:08:14.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.212+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=PFNL7YEZIKBB type=del size=2241999 2024-06-28T15:08:14.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.216+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=58SYDUYNYTYD type=del 2024-06-28T15:08:14.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.220+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~ELHTN1YDJXQE.0 size=1048576 2024-06-28T15:08:14.207 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.220+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~ELHTN1YDJXQE.1 size=1048576 2024-06-28T15:08:14.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.220+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~ELHTN1YDJXQE.2 size=268227 2024-06-28T15:08:14.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.220+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=ELHTN1YDJXQE tag=GN1ELOPTM97L type=write size=3413955 2024-06-28T15:08:14.211 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.224+0000 7f65680598c0 1 < completed delete operation key=1VD7NDPEM3CD tag=M2OFR0XBEKWF type=del 2024-06-28T15:08:14.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.228+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~XK5UD8T4R26F.0 size=87510 2024-06-28T15:08:14.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.228+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=XK5UD8T4R26F tag=WF0EAT1PDY17 type=write size=1136086 2024-06-28T15:08:14.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.232+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~VSQGIPD8OC7V.0 size=1048576 2024-06-28T15:08:14.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.232+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~VSQGIPD8OC7V.1 size=414358 2024-06-28T15:08:14.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.236+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=VSQGIPD8OC7V tag=9ZSSOG2CJVOB type=write size=2511510 2024-06-28T15:08:14.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.236+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=RO2BN76GKZBB type=del size=932722 2024-06-28T15:08:14.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.240+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~5CP0L7MZ0H9T.0 size=1048576 2024-06-28T15:08:14.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.240+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~5CP0L7MZ0H9T.1 size=225242 2024-06-28T15:08:14.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.244+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=5CP0L7MZ0H9T tag=LVW2FJWT7PZ9 type=write size=2322394 2024-06-28T15:08:14.234 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.248+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=NJOL9GTGU034 type=del size=1122570 2024-06-28T15:08:14.236 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.248+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~2BXE9MDON6WE.0 size=1048576 2024-06-28T15:08:14.237 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.248+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~2BXE9MDON6WE.1 size=296548 2024-06-28T15:08:14.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.256+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=2BXE9MDON6WE tag=GOT1M9FG9HW0 type=write size=2393700 2024-06-28T15:08:14.244 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.256+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=SXVY659SNENZ type=del size=161024 2024-06-28T15:08:14.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.260+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=LEFNU3DN8281 type=del 2024-06-28T15:08:14.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.264+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~CXE19IS2Q75D.0 size=345264 2024-06-28T15:08:14.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.264+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=CXE19IS2Q75D tag=LJ4JBRSGIDBY type=write size=1393840 2024-06-28T15:08:14.252 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.264+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~7N9TXXQT63GL.0 size=1048576 2024-06-28T15:08:14.253 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.264+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~7N9TXXQT63GL.1 size=1048576 2024-06-28T15:08:14.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.268+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~7N9TXXQT63GL.2 size=83409 2024-06-28T15:08:14.258 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.272+0000 7f65680598c0 1 < completed multipart upload key=MQ3QWZBFN4YR upload=7N9TXXQT63GL tag=AWJ6M7RVPP49 type=write size=3229137 2024-06-28T15:08:14.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.272+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=61I1JBYPCPPH type=write size=297969 2024-06-28T15:08:14.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.276+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=DNP54BFVX584 type=del 2024-06-28T15:08:14.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.280+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~DX6G7HXJLLSL.0 size=1048576 2024-06-28T15:08:14.267 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.280+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~DX6G7HXJLLSL.1 size=1048576 2024-06-28T15:08:14.268 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.280+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~DX6G7HXJLLSL.2 size=529741 2024-06-28T15:08:14.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.280+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=DX6G7HXJLLSL tag=MD7CE8VQ33FF type=write size=3675469 2024-06-28T15:08:14.270 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.284+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~TFL5XK5C2CHF.0 size=1048576 2024-06-28T15:08:14.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.284+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~TFL5XK5C2CHF.1 size=22689 2024-06-28T15:08:14.276 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.288+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=TFL5XK5C2CHF tag=VR4QN6LHAP6P type=write size=2119841 2024-06-28T15:08:14.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.292+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=55FGOY4FRMQW type=del size=832012 2024-06-28T15:08:14.280 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.292+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~8QZIHTT1V3N1.0 size=580915 2024-06-28T15:08:14.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.296+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=8QZIHTT1V3N1 tag=RZ60FLNIYH9T type=write size=1629491 2024-06-28T15:08:14.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.296+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=PCBLZTT5FX3Y type=del size=2430004 2024-06-28T15:08:14.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.300+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~B0EPELME0BH2.0 size=1048576 2024-06-28T15:08:14.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.300+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~B0EPELME0BH2.1 size=1048576 2024-06-28T15:08:14.289 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.300+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~B0EPELME0BH2.2 size=1028291 2024-06-28T15:08:14.292 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.304+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=B0EPELME0BH2 tag=8UYLDGB1VLXI type=write size=4174019 2024-06-28T15:08:14.294 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.308+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=Z6Z0K3GETIA6 type=del size=99664 2024-06-28T15:08:14.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.308+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~GJG07COPAD84.0 size=1048576 2024-06-28T15:08:14.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.312+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~GJG07COPAD84.1 size=365869 2024-06-28T15:08:14.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.312+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=GJG07COPAD84 tag=QGCR6VAV9GQI type=write size=2463021 2024-06-28T15:08:14.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.316+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=B6NMU79DNC95 type=del size=2388819 2024-06-28T15:08:14.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.316+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~2CCX7DD7GG5T.0 size=1048576 2024-06-28T15:08:14.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.320+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~2CCX7DD7GG5T.1 size=709805 2024-06-28T15:08:14.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.324+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=2CCX7DD7GG5T tag=AFLNH6G6WTND type=write size=2806957 2024-06-28T15:08:14.311 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.324+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=VLO0IK0AA3NM type=del size=1853470 2024-06-28T15:08:14.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.328+0000 7f65680598c0 1 < completed write operation key=7LB8MOJL39RM tag=7EPBC3Z8WCZX type=write size=321941 2024-06-28T15:08:14.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.328+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=CLHVQZ53217O type=del size=1935214 2024-06-28T15:08:14.317 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.328+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~1RQ21ALFXHDX.0 size=1048576 2024-06-28T15:08:14.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.332+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~1RQ21ALFXHDX.1 size=108600 2024-06-28T15:08:14.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.336+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=1RQ21ALFXHDX tag=KY314UJ2E3H7 type=write size=2205752 2024-06-28T15:08:14.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.336+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=3TD2OMAY3BCE type=del size=4101520 2024-06-28T15:08:14.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.340+0000 7f65680598c0 1 < suggested update operation key=EN76QQWIQAXZ tag=PFNL7YEZIKBB type=del size=2241999 2024-06-28T15:08:14.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.344+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~6SLQWSNBTUMP.0 size=1048576 2024-06-28T15:08:14.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.344+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~6SLQWSNBTUMP.1 size=1048576 2024-06-28T15:08:14.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.348+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~6SLQWSNBTUMP.2 size=843921 2024-06-28T15:08:14.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.348+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=6SLQWSNBTUMP tag=NT5JV2LH3AEG type=write size=3989649 2024-06-28T15:08:14.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.348+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~ELHTN1YDJXQE.0 size=1048576 2024-06-28T15:08:14.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.348+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~ELHTN1YDJXQE.1 size=1048576 2024-06-28T15:08:14.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.352+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~ELHTN1YDJXQE.2 size=268227 2024-06-28T15:08:14.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.352+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=ELHTN1YDJXQE tag=GN1ELOPTM97L type=write size=3413955 2024-06-28T15:08:14.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.356+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=5Y9VF93XFFYQ type=del size=2645881 2024-06-28T15:08:14.345 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.356+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~XK5UD8T4R26F.0 size=87510 2024-06-28T15:08:14.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.360+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=XK5UD8T4R26F tag=WF0EAT1PDY17 type=write size=1136086 2024-06-28T15:08:14.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.364+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~QOAXZ3KTUTXH.0 size=1048576 2024-06-28T15:08:14.352 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.364+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~QOAXZ3KTUTXH.1 size=962907 2024-06-28T15:08:14.352 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.364+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=QOAXZ3KTUTXH tag=NBJM128N2AN4 type=write size=3060059 2024-06-28T15:08:14.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.368+0000 7f65680598c0 1 < canceled operation key=J311Q2712YM0 tag=RO2BN76GKZBB type=del size=932722 2024-06-28T15:08:14.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.368+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=VOIHAGRR7U3I type=del size=271011 2024-06-28T15:08:14.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.372+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=NJOL9GTGU034 type=del 2024-06-28T15:08:14.361 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.372+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~VWD9G7S01RAC.0 size=424927 2024-06-28T15:08:14.361 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.372+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=VWD9G7S01RAC tag=VJDBFKXOAILD type=write size=1473503 2024-06-28T15:08:14.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.376+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=SXVY659SNENZ type=del 2024-06-28T15:08:14.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.380+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~O53LA5I918NN.0 size=377257 2024-06-28T15:08:14.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.380+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=O53LA5I918NN tag=DNOKSQMK9Y07 type=write size=1425833 2024-06-28T15:08:14.367 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.380+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~CXE19IS2Q75D.0 size=345264 2024-06-28T15:08:14.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.384+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=CXE19IS2Q75D tag=LJ4JBRSGIDBY type=write size=1393840 2024-06-28T15:08:14.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.384+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=K0MXFQX3M7AA type=del size=2684988 2024-06-28T15:08:14.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.388+0000 7f65680598c0 1 < canceled operation key=7NL3CTFYY9E5 tag=61I1JBYPCPPH type=write size=297969 2024-06-28T15:08:14.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.392+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~TKTUCYYMAUC8.0 size=1048576 2024-06-28T15:08:14.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.392+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~TKTUCYYMAUC8.1 size=975505 2024-06-28T15:08:14.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.392+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=TKTUCYYMAUC8 tag=90XANMAO32RB type=write size=3072657 2024-06-28T15:08:14.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.392+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~DX6G7HXJLLSL.0 size=1048576 2024-06-28T15:08:14.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.396+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~DX6G7HXJLLSL.1 size=1048576 2024-06-28T15:08:14.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.396+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~DX6G7HXJLLSL.2 size=529741 2024-06-28T15:08:14.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.400+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=DX6G7HXJLLSL tag=MD7CE8VQ33FF type=write size=3675469 2024-06-28T15:08:14.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.400+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=T1XWJ143I91U type=del size=1692460 2024-06-28T15:08:14.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.404+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=55FGOY4FRMQW type=del 2024-06-28T15:08:14.393 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.404+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=9SG6MW0176NX type=del size=4031016 2024-06-28T15:08:14.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.408+0000 7f65680598c0 1 < suggested update operation key=7NL3CTFYY9E5 tag=PCBLZTT5FX3Y type=del size=2430004 2024-06-28T15:08:14.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.412+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=WJPFR3KHYB7R type=del size=3248504 2024-06-28T15:08:14.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.412+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=Z6Z0K3GETIA6 type=del 2024-06-28T15:08:14.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.416+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=3N5B1OV7J0H9 type=del size=195575 2024-06-28T15:08:14.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.416+0000 7f65680598c0 1 < suggested update operation key=AXQ0YFWUH4FD tag=B6NMU79DNC95 type=del size=2388819 2024-06-28T15:08:14.407 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.420+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=BNCFU0QV2WLL type=del size=1858034 2024-06-28T15:08:14.408 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.420+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=VLO0IK0AA3NM type=del 2024-06-28T15:08:14.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.424+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=6MS7IT5VTS21 type=del size=3563940 2024-06-28T15:08:14.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.428+0000 7f65680598c0 1 < suggested update operation key=ZFNIA3O84EB7 tag=CLHVQZ53217O type=del size=1935214 2024-06-28T15:08:14.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.428+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=JH545AO9Y720 type=del size=216132 2024-06-28T15:08:14.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.432+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=3TD2OMAY3BCE type=del 2024-06-28T15:08:14.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.432+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=8XTUKYMNSSGO type=del size=705248 2024-06-28T15:08:14.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.436+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~6SLQWSNBTUMP.0 size=1048576 2024-06-28T15:08:14.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.436+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~6SLQWSNBTUMP.1 size=1048576 2024-06-28T15:08:14.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.440+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~6SLQWSNBTUMP.2 size=843921 2024-06-28T15:08:14.429 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.440+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=6SLQWSNBTUMP tag=NT5JV2LH3AEG type=write size=3989649 2024-06-28T15:08:14.431 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.444+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=COR0HYOVY7IH type=del size=2641334 2024-06-28T15:08:14.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.444+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=5Y9VF93XFFYQ type=del 2024-06-28T15:08:14.434 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.448+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=2ZPM3ZACSNNU type=write size=775958 2024-06-28T15:08:14.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.448+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~QOAXZ3KTUTXH.0 size=1048576 2024-06-28T15:08:14.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.452+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~QOAXZ3KTUTXH.1 size=962907 2024-06-28T15:08:14.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.452+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=QOAXZ3KTUTXH tag=NBJM128N2AN4 type=write size=3060059 2024-06-28T15:08:14.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.456+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=ZCMPYMMOXTUW type=write size=674780 2024-06-28T15:08:14.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.456+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=VOIHAGRR7U3I type=del 2024-06-28T15:08:14.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.456+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=NQ2UVC9ER9O7 type=del size=2833385 2024-06-28T15:08:14.447 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.460+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~VWD9G7S01RAC.0 size=424927 2024-06-28T15:08:14.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.464+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=VWD9G7S01RAC tag=VJDBFKXOAILD type=write size=1473503 2024-06-28T15:08:14.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.464+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=IZJWFSOLPX1H type=del size=1000936 2024-06-28T15:08:14.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.464+0000 7f65680598c0 1 < canceled part key=_multipart_AOBFAR0UTONH.2~O53LA5I918NN.0 size=377257 2024-06-28T15:08:14.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.468+0000 7f65680598c0 1 < canceled multipart upload key=AOBFAR0UTONH upload=O53LA5I918NN tag=DNOKSQMK9Y07 type=write size=1425833 2024-06-28T15:08:14.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.468+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=TWBCYDTNBCO3 type=del size=1845944 2024-06-28T15:08:14.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.472+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=K0MXFQX3M7AA type=del 2024-06-28T15:08:14.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.472+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=MVI5779N2DAI type=write size=700526 2024-06-28T15:08:14.462 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.476+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~TKTUCYYMAUC8.0 size=1048576 2024-06-28T15:08:14.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.476+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~TKTUCYYMAUC8.1 size=975505 2024-06-28T15:08:14.466 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.480+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=TKTUCYYMAUC8 tag=90XANMAO32RB type=write size=3072657 2024-06-28T15:08:14.468 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.480+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=LMOQCQT0SER8 type=del size=838003 2024-06-28T15:08:14.470 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.484+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=T1XWJ143I91U type=del 2024-06-28T15:08:14.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.484+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=X33NPY3I0VAH type=del size=3602956 2024-06-28T15:08:14.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.488+0000 7f65680598c0 1 < suggested remove operation key=7LB8MOJL39RM tag=9SG6MW0176NX type=del 2024-06-28T15:08:14.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.488+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=GC0RSP1W1TMR type=del size=3270148 2024-06-28T15:08:14.481 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.492+0000 7f65680598c0 1 < suggested remove operation key=KFHGA64L1NLP tag=WJPFR3KHYB7R type=del 2024-06-28T15:08:14.482 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.496+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=6R02IB7FPH3D type=del size=1523635 2024-06-28T15:08:14.484 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.496+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=3N5B1OV7J0H9 type=del 2024-06-28T15:08:14.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.500+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~NDGOO35827FW.0 size=1048576 2024-06-28T15:08:14.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.500+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~NDGOO35827FW.1 size=68820 2024-06-28T15:08:14.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.500+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=NDGOO35827FW tag=BZP6KXR8UCVR type=write size=2165972 2024-06-28T15:08:14.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.504+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=BNCFU0QV2WLL type=del 2024-06-28T15:08:14.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.504+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=PM9CG2MVJKQ5 type=del size=1329626 2024-06-28T15:08:14.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.508+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=6MS7IT5VTS21 type=del 2024-06-28T15:08:14.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.508+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~8SKW5OKUIL7J.0 size=1048576 2024-06-28T15:08:14.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.512+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~8SKW5OKUIL7J.1 size=1048576 2024-06-28T15:08:14.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.512+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~8SKW5OKUIL7J.2 size=1043358 2024-06-28T15:08:14.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.512+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=8SKW5OKUIL7J tag=TWLWEB5UEQDY type=write size=4189086 2024-06-28T15:08:14.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.516+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=JH545AO9Y720 type=del 2024-06-28T15:08:14.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.516+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=GSJKSC3HZ5D2 type=del size=2121275 2024-06-28T15:08:14.505 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.516+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=8XTUKYMNSSGO type=del 2024-06-28T15:08:14.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.520+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~3E5WAY949FTU.0 size=1048576 2024-06-28T15:08:14.509 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.520+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~3E5WAY949FTU.1 size=1048576 2024-06-28T15:08:14.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.524+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~3E5WAY949FTU.2 size=811263 2024-06-28T15:08:14.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.524+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=3E5WAY949FTU tag=3UJ3YI1SEUMJ type=write size=3956991 2024-06-28T15:08:14.512 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.524+0000 7f65680598c0 1 < canceled operation key=LZX5UZCYLTAG tag=COR0HYOVY7IH type=del size=2641334 2024-06-28T15:08:14.514 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.528+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=MGJ69WSJOHLG type=del size=3506704 2024-06-28T15:08:14.517 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.528+0000 7f65680598c0 1 < suggested remove operation key=ZFNIA3O84EB7 tag=2ZPM3ZACSNNU type=write 2024-06-28T15:08:14.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.532+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=X8LKKXD0U3R1 type=del size=3259944 2024-06-28T15:08:14.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.532+0000 7f65680598c0 1 < completed write operation key=1VD7NDPEM3CD tag=ZCMPYMMOXTUW type=write size=674780 2024-06-28T15:08:14.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.536+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=75BTRDVJ88GZ type=del size=1941811 2024-06-28T15:08:14.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.536+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=NQ2UVC9ER9O7 type=del 2024-06-28T15:08:14.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.540+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=FPOO9VI8Q2UO type=del size=1501860 2024-06-28T15:08:14.528 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.540+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=IZJWFSOLPX1H type=del 2024-06-28T15:08:14.531 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.544+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~CM5M51SY57HA.0 size=1048576 2024-06-28T15:08:14.533 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.544+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~CM5M51SY57HA.1 size=464970 2024-06-28T15:08:14.533 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.544+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=CM5M51SY57HA tag=Q3LI4PLS2OFE type=write size=2562122 2024-06-28T15:08:14.534 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.548+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=TWBCYDTNBCO3 type=del 2024-06-28T15:08:14.537 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.552+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~VSKERVI69R6O.0 size=1048576 2024-06-28T15:08:14.539 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.552+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~VSKERVI69R6O.1 size=1048576 2024-06-28T15:08:14.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.552+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~VSKERVI69R6O.2 size=414065 2024-06-28T15:08:14.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.552+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=VSKERVI69R6O tag=D31Y1JC70D9L type=write size=3559793 2024-06-28T15:08:14.541 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.556+0000 7f65680598c0 1 < completed write operation key=UP3XFF38IJXN tag=MVI5779N2DAI type=write size=700526 2024-06-28T15:08:14.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.556+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=YJTWFKVV23LG type=del size=2879515 2024-06-28T15:08:14.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.556+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=LMOQCQT0SER8 type=del 2024-06-28T15:08:14.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.560+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~9GMSD7278MNQ.0 size=401669 2024-06-28T15:08:14.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.560+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=9GMSD7278MNQ tag=3YCMPWVWNTH9 type=write size=1450245 2024-06-28T15:08:14.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.560+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=X33NPY3I0VAH type=del 2024-06-28T15:08:14.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.564+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=IVM5OI8XGNFU type=del size=3647123 2024-06-28T15:08:14.552 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.564+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=GC0RSP1W1TMR type=del 2024-06-28T15:08:14.555 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.568+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~TGD8XMK92VDK.0 size=1048576 2024-06-28T15:08:14.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.568+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~TGD8XMK92VDK.1 size=1048576 2024-06-28T15:08:14.558 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.572+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~TGD8XMK92VDK.2 size=731160 2024-06-28T15:08:14.558 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.572+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=TGD8XMK92VDK tag=USNACTXJTWIJ type=write size=3876888 2024-06-28T15:08:14.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.572+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=6R02IB7FPH3D type=del 2024-06-28T15:08:14.561 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.572+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=9GL1ETC6CGX6 type=del size=34261 2024-06-28T15:08:14.561 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.572+0000 7f65680598c0 1 < canceled part key=_multipart_1VD7NDPEM3CD.2~NDGOO35827FW.0 size=1048576 2024-06-28T15:08:14.562 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.576+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~NDGOO35827FW.1 size=68820 2024-06-28T15:08:14.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.580+0000 7f65680598c0 1 < canceled multipart upload key=1VD7NDPEM3CD upload=NDGOO35827FW tag=BZP6KXR8UCVR type=write size=2165972 2024-06-28T15:08:14.567 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.580+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=6B0GNL4WBYHN type=del size=978626 2024-06-28T15:08:14.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.584+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=PM9CG2MVJKQ5 type=del 2024-06-28T15:08:14.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.584+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=YTHRJS5IGNXZ type=del size=794916 2024-06-28T15:08:14.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.584+0000 7f65680598c0 1 < canceled part key=_multipart_2FC9DNEZGWC4.2~8SKW5OKUIL7J.0 size=1048576 2024-06-28T15:08:14.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.588+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~8SKW5OKUIL7J.1 size=1048576 2024-06-28T15:08:14.575 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.588+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~8SKW5OKUIL7J.2 size=1043358 2024-06-28T15:08:14.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.592+0000 7f65680598c0 1 < canceled multipart upload key=2FC9DNEZGWC4 upload=8SKW5OKUIL7J tag=TWLWEB5UEQDY type=write size=4189086 2024-06-28T15:08:14.583 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.596+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~Y2K3Q2PP6N2W.0 size=1048576 2024-06-28T15:08:14.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.596+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~Y2K3Q2PP6N2W.1 size=1048576 2024-06-28T15:08:14.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.596+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~Y2K3Q2PP6N2W.2 size=389637 2024-06-28T15:08:14.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.596+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=Y2K3Q2PP6N2W tag=HYBSSJV9JLD5 type=write size=3535365 2024-06-28T15:08:14.587 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.600+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=GSJKSC3HZ5D2 type=del 2024-06-28T15:08:14.590 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.604+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~UR1UGST0IH3J.0 size=1048576 2024-06-28T15:08:14.591 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.604+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~UR1UGST0IH3J.1 size=299607 2024-06-28T15:08:14.592 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.604+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=UR1UGST0IH3J tag=FU114Z41NT3D type=write size=2396759 2024-06-28T15:08:14.593 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.608+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~3E5WAY949FTU.0 size=1048576 2024-06-28T15:08:14.595 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.608+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~3E5WAY949FTU.1 size=1048576 2024-06-28T15:08:14.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.608+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~3E5WAY949FTU.2 size=811263 2024-06-28T15:08:14.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.612+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=3E5WAY949FTU tag=3UJ3YI1SEUMJ type=write size=3956991 2024-06-28T15:08:14.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.616+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=886IHRLC7U02 type=del size=3800210 2024-06-28T15:08:14.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.616+0000 7f65680598c0 1 < canceled operation key=C5C6SAHLPZV1 tag=MGJ69WSJOHLG type=del size=3506704 2024-06-28T15:08:14.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.620+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=LFK8S3RFKZXL type=del size=1027407 2024-06-28T15:08:14.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.620+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=X8LKKXD0U3R1 type=del 2024-06-28T15:08:14.610 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.624+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=9ON6M6POE2S1 type=write size=751143 2024-06-28T15:08:14.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.624+0000 7f65680598c0 1 < canceled operation key=KFHGA64L1NLP tag=75BTRDVJ88GZ type=del size=1941811 2024-06-28T15:08:14.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.628+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=VBMJDLVZ090N type=del size=2237228 2024-06-28T15:08:14.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.632+0000 7f65680598c0 1 < suggested remove operation key=2F968YIMJN9D tag=FPOO9VI8Q2UO type=del 2024-06-28T15:08:14.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.636+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~7NTI4V233ULY.0 size=666522 2024-06-28T15:08:14.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.636+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=7NTI4V233ULY tag=6280B0230M4S type=write size=1715098 2024-06-28T15:08:14.625 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.636+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~CM5M51SY57HA.0 size=1048576 2024-06-28T15:08:14.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.640+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~CM5M51SY57HA.1 size=464970 2024-06-28T15:08:14.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.644+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=CM5M51SY57HA tag=Q3LI4PLS2OFE type=write size=2562122 2024-06-28T15:08:14.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.644+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=OZ1C1I2SYQY1 type=write size=970493 2024-06-28T15:08:14.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.648+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~VSKERVI69R6O.0 size=1048576 2024-06-28T15:08:14.636 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.648+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~VSKERVI69R6O.1 size=1048576 2024-06-28T15:08:14.638 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.652+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~VSKERVI69R6O.2 size=414065 2024-06-28T15:08:14.642 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.656+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=VSKERVI69R6O tag=D31Y1JC70D9L type=write size=3559793 2024-06-28T15:08:14.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.656+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=5451OKDEWNCZ type=del size=1794791 2024-06-28T15:08:14.646 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.660+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=YJTWFKVV23LG type=del 2024-06-28T15:08:14.648 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.660+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=SK9VGH9OF5IK type=del size=3131841 2024-06-28T15:08:14.648 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.660+0000 7f65680598c0 1 < canceled part key=_multipart_F6SEDD6BH1T2.2~9GMSD7278MNQ.0 size=401669 2024-06-28T15:08:14.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.664+0000 7f65680598c0 1 < canceled multipart upload key=F6SEDD6BH1T2 upload=9GMSD7278MNQ tag=3YCMPWVWNTH9 type=write size=1450245 2024-06-28T15:08:14.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.668+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~W6V76YP1C2M1.0 size=1048576 2024-06-28T15:08:14.657 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.668+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~W6V76YP1C2M1.1 size=1048213 2024-06-28T15:08:14.657 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.668+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=W6V76YP1C2M1 tag=K8Z604RWY5O2 type=write size=3145365 2024-06-28T15:08:14.661 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.672+0000 7f65680598c0 1 < suggested update operation key=EN76QQWIQAXZ tag=IVM5OI8XGNFU type=del size=3647123 2024-06-28T15:08:14.663 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.676+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=D9J0OD8MM2PW type=del size=3814243 2024-06-28T15:08:14.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.676+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~TGD8XMK92VDK.0 size=1048576 2024-06-28T15:08:14.667 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.680+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~TGD8XMK92VDK.1 size=1048576 2024-06-28T15:08:14.667 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.680+0000 7f65680598c0 1 < canceled part key=_multipart_EN76QQWIQAXZ.2~TGD8XMK92VDK.2 size=731160 2024-06-28T15:08:14.670 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.684+0000 7f65680598c0 1 < canceled multipart upload key=EN76QQWIQAXZ upload=TGD8XMK92VDK tag=USNACTXJTWIJ type=write size=3876888 2024-06-28T15:08:14.672 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.684+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=FMUVP5TISCEE type=del size=1675183 2024-06-28T15:08:14.674 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.688+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=9GL1ETC6CGX6 type=del 2024-06-28T15:08:14.677 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.688+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=LHG4I31QRD55 type=del size=1036745 2024-06-28T15:08:14.681 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.692+0000 7f65680598c0 1 < suggested remove operation key=ZFNIA3O84EB7 tag=6B0GNL4WBYHN type=del 2024-06-28T15:08:14.683 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.696+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=CHW0P9KZ4XGI type=del size=3117673 2024-06-28T15:08:14.685 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.696+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=YTHRJS5IGNXZ type=del 2024-06-28T15:08:14.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.700+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=9CNBFFTQR5C3 type=write size=873101 2024-06-28T15:08:14.689 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.700+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~Y2K3Q2PP6N2W.0 size=1048576 2024-06-28T15:08:14.690 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.704+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~Y2K3Q2PP6N2W.1 size=1048576 2024-06-28T15:08:14.692 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.704+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~Y2K3Q2PP6N2W.2 size=389637 2024-06-28T15:08:14.695 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.708+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=Y2K3Q2PP6N2W tag=HYBSSJV9JLD5 type=write size=3535365 2024-06-28T15:08:14.699 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.712+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~KN2ADD0O3O1H.0 size=1048576 2024-06-28T15:08:14.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.712+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~KN2ADD0O3O1H.1 size=1048576 2024-06-28T15:08:14.702 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.716+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~KN2ADD0O3O1H.2 size=333193 2024-06-28T15:08:14.702 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.716+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=KN2ADD0O3O1H tag=PXLJS8L29UD6 type=write size=3478921 2024-06-28T15:08:14.704 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.716+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~UR1UGST0IH3J.0 size=1048576 2024-06-28T15:08:14.705 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.720+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~UR1UGST0IH3J.1 size=299607 2024-06-28T15:08:14.708 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.720+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=UR1UGST0IH3J tag=FU114Z41NT3D type=write size=2396759 2024-06-28T15:08:14.712 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.724+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~3IM1A5V3KAA8.0 size=1048576 2024-06-28T15:08:14.713 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.728+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~3IM1A5V3KAA8.1 size=1048576 2024-06-28T15:08:14.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.728+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~3IM1A5V3KAA8.2 size=906874 2024-06-28T15:08:14.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.728+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=3IM1A5V3KAA8 tag=DAZQNFF5TDQ7 type=write size=4052602 2024-06-28T15:08:14.717 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.732+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=886IHRLC7U02 type=del 2024-06-28T15:08:14.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.732+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=C7L9Z5ITGTC6 type=write size=283434 2024-06-28T15:08:14.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.736+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=LFK8S3RFKZXL type=del 2024-06-28T15:08:14.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.740+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~WVCUKFPC3BLJ.0 size=268682 2024-06-28T15:08:14.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.740+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=WVCUKFPC3BLJ tag=NKU7QKNMYW5L type=write size=1317258 2024-06-28T15:08:14.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.740+0000 7f65680598c0 1 < canceled operation key=7NL3CTFYY9E5 tag=9ON6M6POE2S1 type=write size=751143 2024-06-28T15:08:14.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.744+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=BOXNEE2116PW type=del size=2107354 2024-06-28T15:08:14.733 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.744+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=VBMJDLVZ090N type=del 2024-06-28T15:08:14.735 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.748+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=MT8PTYVI2ZPN type=del size=1496058 2024-06-28T15:08:14.737 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.752+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~7NTI4V233ULY.0 size=666522 2024-06-28T15:08:14.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.752+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=7NTI4V233ULY tag=6280B0230M4S type=write size=1715098 2024-06-28T15:08:14.745 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.760+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~X240DP4L1FWL.0 size=1048576 2024-06-28T15:08:14.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.760+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~X240DP4L1FWL.1 size=1048576 2024-06-28T15:08:14.749 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.764+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~X240DP4L1FWL.2 size=135467 2024-06-28T15:08:14.750 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.764+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=X240DP4L1FWL tag=69L23F3341NV type=write size=3281195 2024-06-28T15:08:14.752 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.764+0000 7f65680598c0 1 < completed write operation key=D9D3OFIJP3D3 tag=OZ1C1I2SYQY1 type=write size=970493 2024-06-28T15:08:14.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.768+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=2WPBFA7W79X6 type=write size=368792 2024-06-28T15:08:14.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.768+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=5451OKDEWNCZ type=del 2024-06-28T15:08:14.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.772+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=S0YE33WZ3Q1C type=del size=3799372 2024-06-28T15:08:14.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.776+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=SK9VGH9OF5IK type=del 2024-06-28T15:08:14.764 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.776+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=THSB7BNK3CYU type=del size=2951692 2024-06-28T15:08:14.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.780+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~W6V76YP1C2M1.0 size=1048576 2024-06-28T15:08:14.769 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.780+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~W6V76YP1C2M1.1 size=1048213 2024-06-28T15:08:14.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.788+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=W6V76YP1C2M1 tag=K8Z604RWY5O2 type=write size=3145365 2024-06-28T15:08:14.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.788+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=YLHJ2T2EJOHA type=del size=288134 2024-06-28T15:08:14.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.792+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=D9J0OD8MM2PW type=del 2024-06-28T15:08:14.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.796+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~NM46EBBPR8R0.0 size=1048576 2024-06-28T15:08:14.786 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.800+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~NM46EBBPR8R0.1 size=1048576 2024-06-28T15:08:14.788 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.800+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~NM46EBBPR8R0.2 size=670083 2024-06-28T15:08:14.788 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.800+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=NM46EBBPR8R0 tag=PCIYDABZJ0UP type=write size=3815811 2024-06-28T15:08:14.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.804+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=FMUVP5TISCEE type=del 2024-06-28T15:08:14.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.804+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=UQKHENIINSXU type=del size=4043306 2024-06-28T15:08:14.796 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.808+0000 7f65680598c0 1 < suggested remove operation key=1VD7NDPEM3CD tag=LHG4I31QRD55 type=del 2024-06-28T15:08:14.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.812+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=2DZ3HYW3739V type=write size=698025 2024-06-28T15:08:14.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.812+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=CHW0P9KZ4XGI type=del 2024-06-28T15:08:14.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.816+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=77DOKOGP0U77 type=del size=4021270 2024-06-28T15:08:14.809 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.820+0000 7f65680598c0 1 < suggested update operation key=7LB8MOJL39RM tag=9CNBFFTQR5C3 type=write size=873101 2024-06-28T15:08:14.811 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.824+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=U4MLJJYCOEVK type=del size=923087 2024-06-28T15:08:14.814 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.828+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~KN2ADD0O3O1H.0 size=1048576 2024-06-28T15:08:14.816 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.828+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~KN2ADD0O3O1H.1 size=1048576 2024-06-28T15:08:14.818 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.832+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~KN2ADD0O3O1H.2 size=333193 2024-06-28T15:08:14.823 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.836+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=KN2ADD0O3O1H tag=PXLJS8L29UD6 type=write size=3478921 2024-06-28T15:08:14.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.840+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~8OKLQV6IYEO9.0 size=1048576 2024-06-28T15:08:14.830 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.844+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~8OKLQV6IYEO9.1 size=791664 2024-06-28T15:08:14.830 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.844+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=8OKLQV6IYEO9 tag=6GF49HZ4B9I9 type=write size=2888816 2024-06-28T15:08:14.833 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.844+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~3IM1A5V3KAA8.0 size=1048576 2024-06-28T15:08:14.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.848+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~3IM1A5V3KAA8.1 size=1048576 2024-06-28T15:08:14.838 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.852+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~3IM1A5V3KAA8.2 size=906874 2024-06-28T15:08:14.842 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.856+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=3IM1A5V3KAA8 tag=DAZQNFF5TDQ7 type=write size=4052602 2024-06-28T15:08:14.845 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.856+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=WK733K5UA37D type=write size=688168 2024-06-28T15:08:14.850 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.864+0000 7f65680598c0 1 < suggested update operation key=GAZQ3PCPKN9Q tag=C7L9Z5ITGTC6 type=write size=283434 2024-06-28T15:08:14.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.864+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=A13V1QFSJYNE type=del size=3477466 2024-06-28T15:08:14.856 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.868+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~WVCUKFPC3BLJ.0 size=268682 2024-06-28T15:08:14.861 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.872+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=WVCUKFPC3BLJ tag=NKU7QKNMYW5L type=write size=1317258 2024-06-28T15:08:14.863 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.876+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=55MKX5EB5UHN type=del size=2363538 2024-06-28T15:08:14.866 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.880+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=BOXNEE2116PW type=del 2024-06-28T15:08:14.869 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.880+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=18N30GPDHM24 type=del size=3370243 2024-06-28T15:08:14.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.884+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=MT8PTYVI2ZPN type=del 2024-06-28T15:08:14.875 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.888+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=55IBSO04LHXA type=write size=48992 2024-06-28T15:08:14.878 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.892+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~X240DP4L1FWL.0 size=1048576 2024-06-28T15:08:14.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.892+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~X240DP4L1FWL.1 size=1048576 2024-06-28T15:08:14.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.896+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~X240DP4L1FWL.2 size=135467 2024-06-28T15:08:14.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.900+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=X240DP4L1FWL tag=69L23F3341NV type=write size=3281195 2024-06-28T15:08:14.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.904+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=6KIE1QIYY11G type=del size=2844797 2024-06-28T15:08:14.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.908+0000 7f65680598c0 1 < suggested update operation key=47TO7G9EGJKF tag=2WPBFA7W79X6 type=write size=368792 2024-06-28T15:08:14.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.912+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=JWTDHKEVKI87 type=del size=1582290 2024-06-28T15:08:14.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.912+0000 7f65680598c0 1 < canceled operation key=EN76QQWIQAXZ tag=S0YE33WZ3Q1C type=del size=3799372 2024-06-28T15:08:14.903 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.916+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=0SRBY9YZB43E type=del size=950305 2024-06-28T15:08:14.905 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.916+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=THSB7BNK3CYU type=del 2024-06-28T15:08:14.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.920+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=TRBYQDMZWW61 type=del size=1256059 2024-06-28T15:08:14.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.924+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=YLHJ2T2EJOHA type=del 2024-06-28T15:08:14.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.924+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=T6ZUAGKZ86J5 type=write size=488986 2024-06-28T15:08:14.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.928+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~NM46EBBPR8R0.0 size=1048576 2024-06-28T15:08:14.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.932+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~NM46EBBPR8R0.1 size=1048576 2024-06-28T15:08:14.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.932+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~NM46EBBPR8R0.2 size=670083 2024-06-28T15:08:14.924 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.936+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=NM46EBBPR8R0 tag=PCIYDABZJ0UP type=write size=3815811 2024-06-28T15:08:14.930 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.940+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=RY9428XG0SYM type=del size=3645674 2024-06-28T15:08:14.930 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.944+0000 7f65680598c0 1 < canceled operation key=F6SEDD6BH1T2 tag=UQKHENIINSXU type=del size=4043306 2024-06-28T15:08:14.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.948+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~CWQ3OB90CLCC.0 size=704408 2024-06-28T15:08:14.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.948+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=CWQ3OB90CLCC tag=1QQREYTG1S2Q type=write size=1752984 2024-06-28T15:08:14.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.948+0000 7f65680598c0 1 < completed write operation key=RLJCNXKA2GYA tag=2DZ3HYW3739V type=write size=698025 2024-06-28T15:08:14.940 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.952+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=ASYV0A39QN95 type=write size=180880 2024-06-28T15:08:14.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.956+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=77DOKOGP0U77 type=del 2024-06-28T15:08:14.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.956+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=ZQT46LJ3PW83 type=del size=4167770 2024-06-28T15:08:14.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.960+0000 7f65680598c0 1 < canceled operation key=J311Q2712YM0 tag=U4MLJJYCOEVK type=del size=923087 2024-06-28T15:08:14.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.964+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=1AQW1TOLFGHI type=del size=3081162 2024-06-28T15:08:14.952 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.964+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~8OKLQV6IYEO9.0 size=1048576 2024-06-28T15:08:14.954 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.968+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~8OKLQV6IYEO9.1 size=791664 2024-06-28T15:08:14.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.972+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=8OKLQV6IYEO9 tag=6GF49HZ4B9I9 type=write size=2888816 2024-06-28T15:08:14.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.976+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=ETTN0DA0EZV5 type=del size=3903356 2024-06-28T15:08:14.964 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.976+0000 7f65680598c0 1 < completed write operation key=O1CGONN4TEXF tag=WK733K5UA37D type=write size=688168 2024-06-28T15:08:14.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.980+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=7MPQCFCRD8U5 type=del size=1917956 2024-06-28T15:08:14.970 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.984+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=A13V1QFSJYNE type=del 2024-06-28T15:08:14.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.984+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=BXJUPUQ9YV2F type=del size=2405606 2024-06-28T15:08:14.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.988+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=55MKX5EB5UHN type=del 2024-06-28T15:08:14.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.992+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=3RHI70VBGEYO type=del size=4149429 2024-06-28T15:08:14.982 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.996+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=18N30GPDHM24 type=del 2024-06-28T15:08:14.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:14.996+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=8T3CBFEJS7G8 type=del size=2732603 2024-06-28T15:08:14.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.000+0000 7f65680598c0 1 < completed write operation key=7NL3CTFYY9E5 tag=55IBSO04LHXA type=write size=48992 2024-06-28T15:08:14.991 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.004+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=NCXJ79SMBJP9 type=del size=1588664 2024-06-28T15:08:14.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.008+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=6KIE1QIYY11G type=del 2024-06-28T15:08:15.000 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.012+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~MTX7LK1P0S0P.0 size=341561 2024-06-28T15:08:15.000 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.012+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=MTX7LK1P0S0P tag=7YT8V3ANOETV type=write size=1390137 2024-06-28T15:08:15.004 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.016+0000 7f65680598c0 1 < completed delete operation key=1VD7NDPEM3CD tag=JWTDHKEVKI87 type=del 2024-06-28T15:08:15.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.020+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=60CZR21KI5VV type=write size=937764 2024-06-28T15:08:15.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.024+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=0SRBY9YZB43E type=del 2024-06-28T15:08:15.015 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.028+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~ENIS70BDBHYN.0 size=1048576 2024-06-28T15:08:15.017 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.032+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~ENIS70BDBHYN.1 size=1048576 2024-06-28T15:08:15.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.032+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~ENIS70BDBHYN.2 size=412484 2024-06-28T15:08:15.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.032+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=ENIS70BDBHYN tag=YTCVOWT62FG3 type=write size=3558212 2024-06-28T15:08:15.026 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.040+0000 7f65680598c0 1 < suggested remove operation key=EN76QQWIQAXZ tag=TRBYQDMZWW61 type=del 2024-06-28T15:08:15.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.044+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=KRVOGX9F9ZJH type=del size=2160298 2024-06-28T15:08:15.035 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.048+0000 7f65680598c0 1 < completed write operation key=ZY973GBGBQ1Z tag=T6ZUAGKZ86J5 type=write size=488986 2024-06-28T15:08:15.039 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.052+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=9AECEK7X97RT type=del size=237005 2024-06-28T15:08:15.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.056+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=RY9428XG0SYM type=del 2024-06-28T15:08:15.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.060+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~OMIPP57R1P8T.0 size=1048576 2024-06-28T15:08:15.050 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.064+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~OMIPP57R1P8T.1 size=340108 2024-06-28T15:08:15.050 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.064+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=OMIPP57R1P8T tag=IPXUJZBL26TT type=write size=2437260 2024-06-28T15:08:15.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.064+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~CWQ3OB90CLCC.0 size=704408 2024-06-28T15:08:15.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.068+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=CWQ3OB90CLCC tag=1QQREYTG1S2Q type=write size=1752984 2024-06-28T15:08:15.061 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.072+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~TO9MQ3YQOEW8.0 size=1048576 2024-06-28T15:08:15.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.076+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~TO9MQ3YQOEW8.1 size=619387 2024-06-28T15:08:15.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.076+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=TO9MQ3YQOEW8 tag=1WF7WGWJ23T5 type=write size=2716539 2024-06-28T15:08:15.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.080+0000 7f65680598c0 1 < suggested update operation key=J311Q2712YM0 tag=ASYV0A39QN95 type=write size=180880 2024-06-28T15:08:15.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.084+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~925DM6B3DP6J.0 size=1048576 2024-06-28T15:08:15.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.088+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~925DM6B3DP6J.1 size=785902 2024-06-28T15:08:15.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.088+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=925DM6B3DP6J tag=WC33FWCR0CJG type=write size=2883054 2024-06-28T15:08:15.080 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.092+0000 7f65680598c0 1 < suggested update operation key=2F968YIMJN9D tag=ZQT46LJ3PW83 type=del size=4167770 2024-06-28T15:08:15.082 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.096+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=YUZ2IVZMXH90 type=del size=3332932 2024-06-28T15:08:15.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.096+0000 7f65680598c0 -1 < failed to complete operation key=J311Q2712YM0 tag=1AQW1TOLFGHI type=del size=3081162: (22) Invalid argument 2024-06-28T15:08:15.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.104+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~LP0Q92VSPPC3.0 size=1048576 2024-06-28T15:08:15.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.104+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~LP0Q92VSPPC3.1 size=948510 2024-06-28T15:08:15.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.104+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=LP0Q92VSPPC3 tag=2RIMG00IEAYC type=write size=3045662 2024-06-28T15:08:15.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.108+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=ETTN0DA0EZV5 type=del 2024-06-28T15:08:15.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.112+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~7WAMZ7TDF0IJ.0 size=1048576 2024-06-28T15:08:15.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.112+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~7WAMZ7TDF0IJ.1 size=82769 2024-06-28T15:08:15.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.112+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=7WAMZ7TDF0IJ tag=B5S9UCB4HYR5 type=write size=2179921 2024-06-28T15:08:15.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.116+0000 7f65680598c0 1 < canceled operation key=R0IMFVF5NEOO tag=7MPQCFCRD8U5 type=del size=1917956 2024-06-28T15:08:15.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.116+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=SU1RAT2ID9DP type=del size=1979425 2024-06-28T15:08:15.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.120+0000 7f65680598c0 -1 < no bi entry to suggest for operation key=EN76QQWIQAXZ tag=BXJUPUQ9YV2F type=del size=2405606: (2) No such file or directory 2024-06-28T15:08:15.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.124+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~4FNE44KFBPYC.0 size=1048576 2024-06-28T15:08:15.112 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.124+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~4FNE44KFBPYC.1 size=870386 2024-06-28T15:08:15.112 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.124+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=4FNE44KFBPYC tag=LBESWNY42873 type=write size=2967538 2024-06-28T15:08:15.115 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.128+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=3RHI70VBGEYO type=del 2024-06-28T15:08:15.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.128+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=NZ0BBL62X88A type=write size=723231 2024-06-28T15:08:15.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.132+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=8T3CBFEJS7G8 type=del 2024-06-28T15:08:15.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.136+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~2Q2RPYEGYJ5T.0 size=821661 2024-06-28T15:08:15.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.136+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=2Q2RPYEGYJ5T tag=I90BZ35XP1UV type=write size=1870237 2024-06-28T15:08:15.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.140+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=NCXJ79SMBJP9 type=del 2024-06-28T15:08:15.134 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.144+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~F71JUBBIVXM1.0 size=1048576 2024-06-28T15:08:15.134 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.144+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~F71JUBBIVXM1.1 size=978477 2024-06-28T15:08:15.134 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.144+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=F71JUBBIVXM1 tag=ZDSE8HV36DMM type=write size=3075629 2024-06-28T15:08:15.135 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.148+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~MTX7LK1P0S0P.0 size=341561 2024-06-28T15:08:15.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.152+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=MTX7LK1P0S0P tag=7YT8V3ANOETV type=write size=1390137 2024-06-28T15:08:15.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.156+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=GXPOH30Y131W type=del size=1155596 2024-06-28T15:08:15.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.156+0000 7f65680598c0 1 < completed write operation key=OUID7RX5NVV6 tag=60CZR21KI5VV type=write size=937764 2024-06-28T15:08:15.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.160+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=XO2GPT310YCF type=del size=1138848 2024-06-28T15:08:15.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.160+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~ENIS70BDBHYN.0 size=1048576 2024-06-28T15:08:15.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.164+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~ENIS70BDBHYN.1 size=1048576 2024-06-28T15:08:15.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.164+0000 7f65680598c0 1 < canceled part key=_multipart_7LB8MOJL39RM.2~ENIS70BDBHYN.2 size=412484 2024-06-28T15:08:15.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.168+0000 7f65680598c0 1 < canceled multipart upload key=7LB8MOJL39RM upload=ENIS70BDBHYN tag=YTCVOWT62FG3 type=write size=3558212 2024-06-28T15:08:15.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.172+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~934VT5L44IG2.0 size=509279 2024-06-28T15:08:15.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.172+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=934VT5L44IG2 tag=VR7HE5YXMZS9 type=write size=1557855 2024-06-28T15:08:15.168 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.180+0000 7f65680598c0 1 < suggested remove operation key=1VD7NDPEM3CD tag=KRVOGX9F9ZJH type=del 2024-06-28T15:08:15.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.188+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~979ZMCCR3IX9.0 size=1009762 2024-06-28T15:08:15.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.188+0000 7f65680598c0 1 > prepared multipart upload key=2F968YIMJN9D upload=979ZMCCR3IX9 tag=RAKS1TFHCYSV type=write size=2058338 2024-06-28T15:08:15.177 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.192+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=9AECEK7X97RT type=del 2024-06-28T15:08:15.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.192+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=TVNN4NO6DWWU type=del size=2955416 2024-06-28T15:08:15.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.196+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~OMIPP57R1P8T.0 size=1048576 2024-06-28T15:08:15.187 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.200+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~OMIPP57R1P8T.1 size=340108 2024-06-28T15:08:15.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.204+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=OMIPP57R1P8T tag=IPXUJZBL26TT type=write size=2437260 2024-06-28T15:08:15.195 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.208+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=FP1WEY54ME9H type=del size=3090284 2024-06-28T15:08:15.198 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.212+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~TO9MQ3YQOEW8.0 size=1048576 2024-06-28T15:08:15.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.212+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~TO9MQ3YQOEW8.1 size=619387 2024-06-28T15:08:15.207 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.220+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=TO9MQ3YQOEW8 tag=1WF7WGWJ23T5 type=write size=2716539 2024-06-28T15:08:15.212 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.224+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~HI95QD06OSO8.0 size=1048576 2024-06-28T15:08:15.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.228+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~HI95QD06OSO8.1 size=1026459 2024-06-28T15:08:15.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.228+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=HI95QD06OSO8 tag=WOXLEYJK9OZU type=write size=3123611 2024-06-28T15:08:15.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.232+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~925DM6B3DP6J.0 size=1048576 2024-06-28T15:08:15.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.232+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~925DM6B3DP6J.1 size=785902 2024-06-28T15:08:15.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.236+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=925DM6B3DP6J tag=WC33FWCR0CJG type=write size=2883054 2024-06-28T15:08:15.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.240+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=3OU2K7Q0GGZO type=del size=2449288 2024-06-28T15:08:15.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.244+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=YUZ2IVZMXH90 type=del 2024-06-28T15:08:15.234 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.248+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=6VOQ5VMZEG4G type=write size=180792 2024-06-28T15:08:15.235 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.248+0000 7f65680598c0 1 < canceled part key=_multipart_O1CGONN4TEXF.2~LP0Q92VSPPC3.0 size=1048576 2024-06-28T15:08:15.237 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.252+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~LP0Q92VSPPC3.1 size=948510 2024-06-28T15:08:15.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.256+0000 7f65680598c0 1 < canceled multipart upload key=O1CGONN4TEXF upload=LP0Q92VSPPC3 tag=2RIMG00IEAYC type=write size=3045662 2024-06-28T15:08:15.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.260+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~AI5RZUMMAQV5.0 size=1048576 2024-06-28T15:08:15.251 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.264+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~AI5RZUMMAQV5.1 size=1048576 2024-06-28T15:08:15.253 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.268+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~AI5RZUMMAQV5.2 size=699710 2024-06-28T15:08:15.254 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.268+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=AI5RZUMMAQV5 tag=AWE0ZFBEMWIL type=write size=3845438 2024-06-28T15:08:15.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.268+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~7WAMZ7TDF0IJ.0 size=1048576 2024-06-28T15:08:15.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.272+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~7WAMZ7TDF0IJ.1 size=82769 2024-06-28T15:08:15.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.276+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=7WAMZ7TDF0IJ tag=B5S9UCB4HYR5 type=write size=2179921 2024-06-28T15:08:15.268 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.280+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~31LBAYSTB9PF.0 size=1048576 2024-06-28T15:08:15.270 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.284+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~31LBAYSTB9PF.1 size=631811 2024-06-28T15:08:15.271 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.284+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=31LBAYSTB9PF tag=I0Q7ZZS7S1BH type=write size=2728963 2024-06-28T15:08:15.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.288+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=SU1RAT2ID9DP type=del 2024-06-28T15:08:15.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.288+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=BJ84URVW5STM type=del size=3271965 2024-06-28T15:08:15.280 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.292+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~4FNE44KFBPYC.0 size=1048576 2024-06-28T15:08:15.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.296+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~4FNE44KFBPYC.1 size=870386 2024-06-28T15:08:15.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.300+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=4FNE44KFBPYC tag=LBESWNY42873 type=write size=2967538 2024-06-28T15:08:15.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.308+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~H0O9A1R6A4IX.0 size=634859 2024-06-28T15:08:15.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.308+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=H0O9A1R6A4IX tag=7HO8NDC8E6B2 type=write size=1683435 2024-06-28T15:08:15.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.312+0000 7f65680598c0 1 < completed write operation key=47TO7G9EGJKF tag=NZ0BBL62X88A type=write size=723231 2024-06-28T15:08:15.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.312+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=H0YCCX389KTW type=write size=544895 2024-06-28T15:08:15.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.316+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~2Q2RPYEGYJ5T.0 size=821661 2024-06-28T15:08:15.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.320+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=2Q2RPYEGYJ5T tag=I90BZ35XP1UV type=write size=1870237 2024-06-28T15:08:15.312 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.324+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=DZR0APZV5CSB type=write size=818180 2024-06-28T15:08:15.316 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.328+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~F71JUBBIVXM1.0 size=1048576 2024-06-28T15:08:15.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.332+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~F71JUBBIVXM1.1 size=978477 2024-06-28T15:08:15.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.336+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=F71JUBBIVXM1 tag=ZDSE8HV36DMM type=write size=3075629 2024-06-28T15:08:15.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.344+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~DU7R9199F548.0 size=402958 2024-06-28T15:08:15.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.344+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=DU7R9199F548 tag=4E5XJ3OQPSXX type=write size=1451534 2024-06-28T15:08:15.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.348+0000 7f65680598c0 1 < canceled operation key=LZX5UZCYLTAG tag=GXPOH30Y131W type=del size=1155596 2024-06-28T15:08:15.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.348+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=UP3J2MXKQ41J type=del size=1812248 2024-06-28T15:08:15.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.352+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=XO2GPT310YCF type=del 2024-06-28T15:08:15.344 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.356+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=RWKGTHNMLXPJ type=del size=57547 2024-06-28T15:08:15.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.360+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~934VT5L44IG2.0 size=509279 2024-06-28T15:08:15.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.368+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=934VT5L44IG2 tag=VR7HE5YXMZS9 type=write size=1557855 2024-06-28T15:08:15.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.368+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=0C0LDDO2YNUF type=del size=3466905 2024-06-28T15:08:15.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.372+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~979ZMCCR3IX9.0 size=1009762 2024-06-28T15:08:15.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.380+0000 7f65680598c0 1 < completed multipart upload key=2F968YIMJN9D upload=979ZMCCR3IX9 tag=RAKS1TFHCYSV type=write size=2058338 2024-06-28T15:08:15.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.384+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=MWY9VYZD2F6Y type=del size=2608706 2024-06-28T15:08:15.373 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.384+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=TVNN4NO6DWWU type=del 2024-06-28T15:08:15.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.392+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~0LX6SA7XWATB.0 size=1048576 2024-06-28T15:08:15.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.396+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~0LX6SA7XWATB.1 size=562988 2024-06-28T15:08:15.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.396+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=0LX6SA7XWATB tag=ZUM4HSIPVHJF type=write size=2660140 2024-06-28T15:08:15.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.400+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=FP1WEY54ME9H type=del 2024-06-28T15:08:15.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.400+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=SR91X6JJ9INW type=del size=680982 2024-06-28T15:08:15.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.408+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~HI95QD06OSO8.0 size=1048576 2024-06-28T15:08:15.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.408+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~HI95QD06OSO8.1 size=1026459 2024-06-28T15:08:15.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.416+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=HI95QD06OSO8 tag=WOXLEYJK9OZU type=write size=3123611 2024-06-28T15:08:15.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.424+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~07VN287E8B0I.0 size=1048576 2024-06-28T15:08:15.412 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.424+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~07VN287E8B0I.1 size=772688 2024-06-28T15:08:15.412 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.424+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=07VN287E8B0I tag=CYIY9D0IZISB type=write size=2869840 2024-06-28T15:08:15.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.432+0000 7f65680598c0 1 < suggested remove operation key=D9D3OFIJP3D3 tag=3OU2K7Q0GGZO type=del 2024-06-28T15:08:15.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.432+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=PK613357SZMN type=del size=1808456 2024-06-28T15:08:15.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.436+0000 7f65680598c0 1 < completed write operation key=O1CGONN4TEXF tag=6VOQ5VMZEG4G type=write size=180792 2024-06-28T15:08:15.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.440+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~NWL80HMXV6FL.0 size=1048576 2024-06-28T15:08:15.429 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.444+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~NWL80HMXV6FL.1 size=1048576 2024-06-28T15:08:15.431 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.444+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~NWL80HMXV6FL.2 size=1020926 2024-06-28T15:08:15.431 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.444+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=NWL80HMXV6FL tag=XVZ08UMPN8XJ type=write size=4166654 2024-06-28T15:08:15.434 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.448+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~AI5RZUMMAQV5.0 size=1048576 2024-06-28T15:08:15.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.448+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~AI5RZUMMAQV5.1 size=1048576 2024-06-28T15:08:15.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.452+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~AI5RZUMMAQV5.2 size=699710 2024-06-28T15:08:15.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.456+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=AI5RZUMMAQV5 tag=AWE0ZFBEMWIL type=write size=3845438 2024-06-28T15:08:15.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.460+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~SPVNHTDDKTG0.0 size=1048576 2024-06-28T15:08:15.447 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.460+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~SPVNHTDDKTG0.1 size=1048576 2024-06-28T15:08:15.449 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.460+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~SPVNHTDDKTG0.2 size=610934 2024-06-28T15:08:15.449 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.460+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=SPVNHTDDKTG0 tag=6SMZ3XZLN356 type=write size=3756662 2024-06-28T15:08:15.451 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.464+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~31LBAYSTB9PF.0 size=1048576 2024-06-28T15:08:15.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.468+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~31LBAYSTB9PF.1 size=631811 2024-06-28T15:08:15.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.472+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=31LBAYSTB9PF tag=I0Q7ZZS7S1BH type=write size=2728963 2024-06-28T15:08:15.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.476+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~X70H9DNQ5YSW.0 size=1048576 2024-06-28T15:08:15.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.476+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~X70H9DNQ5YSW.1 size=256786 2024-06-28T15:08:15.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.476+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=X70H9DNQ5YSW tag=MBK4YOV18GG1 type=write size=2353938 2024-06-28T15:08:15.466 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.480+0000 7f65680598c0 1 < canceled operation key=60QVJVV16QFM tag=BJ84URVW5STM type=del size=3271965 2024-06-28T15:08:15.468 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.480+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=6A3OQYZS9OOG type=del size=700885 2024-06-28T15:08:15.468 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.480+0000 7f65680598c0 1 < canceled part key=_multipart_C5C6SAHLPZV1.2~H0O9A1R6A4IX.0 size=634859 2024-06-28T15:08:15.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.484+0000 7f65680598c0 1 < canceled multipart upload key=C5C6SAHLPZV1 upload=H0O9A1R6A4IX tag=7HO8NDC8E6B2 type=write size=1683435 2024-06-28T15:08:15.474 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.488+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=QC82B79W15S8 type=write size=927557 2024-06-28T15:08:15.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.488+0000 7f65680598c0 1 < completed write operation key=P278ZM1H052S tag=H0YCCX389KTW type=write size=544895 2024-06-28T15:08:15.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.492+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=KQ086IVQQT5C type=del size=315617 2024-06-28T15:08:15.483 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.496+0000 7f65680598c0 1 < suggested remove operation key=AXQ0YFWUH4FD tag=DZR0APZV5CSB type=write 2024-06-28T15:08:15.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.500+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=HREKGK2R0FV4 type=del size=2994099 2024-06-28T15:08:15.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.500+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~DU7R9199F548.0 size=402958 2024-06-28T15:08:15.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.504+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=DU7R9199F548 tag=4E5XJ3OQPSXX type=write size=1451534 2024-06-28T15:08:15.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.508+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~5NU90C52U428.0 size=368207 2024-06-28T15:08:15.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.508+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=5NU90C52U428 tag=AWQKNB7DGA6J type=write size=1416783 2024-06-28T15:08:15.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.512+0000 7f65680598c0 1 < suggested update operation key=UP3XFF38IJXN tag=UP3J2MXKQ41J type=del size=1812248 2024-06-28T15:08:15.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.516+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=5P96GQAVG19Q type=del size=2557470 2024-06-28T15:08:15.504 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.516+0000 7f65680598c0 1 < canceled operation key=BHW2VHUT0U1U tag=RWKGTHNMLXPJ type=del size=57547 2024-06-28T15:08:15.506 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.520+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=U3OD19DMC4AJ type=del size=2405440 2024-06-28T15:08:15.509 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.520+0000 7f65680598c0 1 < canceled operation key=60QVJVV16QFM tag=0C0LDDO2YNUF type=del size=3466905 2024-06-28T15:08:15.511 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.524+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=SZBASNRTM1H6 type=del size=1349825 2024-06-28T15:08:15.513 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.524+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=MWY9VYZD2F6Y type=del 2024-06-28T15:08:15.517 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.532+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~DYSV6IDQ5N3Z.0 size=1048576 2024-06-28T15:08:15.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.532+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~DYSV6IDQ5N3Z.1 size=1048576 2024-06-28T15:08:15.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.532+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~DYSV6IDQ5N3Z.2 size=252895 2024-06-28T15:08:15.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.532+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=DYSV6IDQ5N3Z tag=7MYYB1WYI29M type=write size=3398623 2024-06-28T15:08:15.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.536+0000 7f65680598c0 1 < canceled part key=_multipart_UP3XFF38IJXN.2~0LX6SA7XWATB.0 size=1048576 2024-06-28T15:08:15.523 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.536+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~0LX6SA7XWATB.1 size=562988 2024-06-28T15:08:15.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.540+0000 7f65680598c0 1 < canceled multipart upload key=UP3XFF38IJXN upload=0LX6SA7XWATB tag=ZUM4HSIPVHJF type=write size=2660140 2024-06-28T15:08:15.530 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.540+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=AWL880BO87CH type=del size=3097134 2024-06-28T15:08:15.533 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.548+0000 7f65680598c0 1 < suggested remove operation key=SFCIZMLY2Y2E tag=SR91X6JJ9INW type=del 2024-06-28T15:08:15.536 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.548+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=ZRAC744LBYKY type=del size=773241 2024-06-28T15:08:15.538 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.552+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~07VN287E8B0I.0 size=1048576 2024-06-28T15:08:15.539 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.552+0000 7f65680598c0 1 < canceled part key=_multipart_R0IMFVF5NEOO.2~07VN287E8B0I.1 size=772688 2024-06-28T15:08:15.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.556+0000 7f65680598c0 1 < canceled multipart upload key=R0IMFVF5NEOO upload=07VN287E8B0I tag=CYIY9D0IZISB type=write size=2869840 2024-06-28T15:08:15.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.560+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~JIFX7AVG5G5Z.0 size=922826 2024-06-28T15:08:15.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.560+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=JIFX7AVG5G5Z tag=OQZHMQD4CBLS type=write size=1971402 2024-06-28T15:08:15.554 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.568+0000 7f65680598c0 1 < suggested update operation key=P278ZM1H052S tag=PK613357SZMN type=del size=1808456 2024-06-28T15:08:15.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.568+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=5ZLN1SJB4XQ2 type=del size=1679087 2024-06-28T15:08:15.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.572+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~NWL80HMXV6FL.0 size=1048576 2024-06-28T15:08:15.562 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.576+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~NWL80HMXV6FL.1 size=1048576 2024-06-28T15:08:15.564 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.576+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~NWL80HMXV6FL.2 size=1020926 2024-06-28T15:08:15.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.580+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=NWL80HMXV6FL tag=XVZ08UMPN8XJ type=write size=4166654 2024-06-28T15:08:15.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.584+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~F6I0N684N1SF.0 size=1048576 2024-06-28T15:08:15.575 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.588+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~F6I0N684N1SF.1 size=1048576 2024-06-28T15:08:15.576 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.588+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~F6I0N684N1SF.2 size=562714 2024-06-28T15:08:15.576 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.588+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=F6I0N684N1SF tag=S8S7F1U2GK8H type=write size=3708442 2024-06-28T15:08:15.577 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.588+0000 7f65680598c0 1 < canceled part key=_multipart_7NL3CTFYY9E5.2~SPVNHTDDKTG0.0 size=1048576 2024-06-28T15:08:15.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.592+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~SPVNHTDDKTG0.1 size=1048576 2024-06-28T15:08:15.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.592+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~SPVNHTDDKTG0.2 size=610934 2024-06-28T15:08:15.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.596+0000 7f65680598c0 1 < canceled multipart upload key=7NL3CTFYY9E5 upload=SPVNHTDDKTG0 tag=6SMZ3XZLN356 type=write size=3756662 2024-06-28T15:08:15.588 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.600+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=U89IY696JXTT type=del size=2881862 2024-06-28T15:08:15.590 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.604+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~X70H9DNQ5YSW.0 size=1048576 2024-06-28T15:08:15.592 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.604+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~X70H9DNQ5YSW.1 size=256786 2024-06-28T15:08:15.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.608+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=X70H9DNQ5YSW tag=MBK4YOV18GG1 type=write size=2353938 2024-06-28T15:08:15.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.612+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=7EY9058JDJXL type=write size=333423 2024-06-28T15:08:15.601 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.612+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=6A3OQYZS9OOG type=del 2024-06-28T15:08:15.603 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.616+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=YQ0165ZNRQZI type=del size=1006708 2024-06-28T15:08:15.605 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.616+0000 7f65680598c0 1 < canceled operation key=ZY973GBGBQ1Z tag=QC82B79W15S8 type=write size=927557 2024-06-28T15:08:15.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.620+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=77O2ELLPW6GY type=del size=960886 2024-06-28T15:08:15.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.624+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=KQ086IVQQT5C type=del 2024-06-28T15:08:15.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.624+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=U2HF2WZN3PIB type=del size=2931872 2024-06-28T15:08:15.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.628+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=HREKGK2R0FV4 type=del 2024-06-28T15:08:15.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.628+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=M7ILGGNXYFSQ type=write size=180701 2024-06-28T15:08:15.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.632+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~5NU90C52U428.0 size=368207 2024-06-28T15:08:15.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.636+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=5NU90C52U428 tag=AWQKNB7DGA6J type=write size=1416783 2024-06-28T15:08:15.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.640+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=P0R57ZR9TT6T type=write size=578759 2024-06-28T15:08:15.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.640+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=5P96GQAVG19Q type=del 2024-06-28T15:08:15.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.644+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=31ZF7D02TWHT type=del size=3087834 2024-06-28T15:08:15.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.648+0000 7f65680598c0 1 < canceled operation key=7NL3CTFYY9E5 tag=U3OD19DMC4AJ type=del size=2405440 2024-06-28T15:08:15.636 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.648+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=LICOS68OI3TE type=del size=1663703 2024-06-28T15:08:15.638 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.652+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=SZBASNRTM1H6 type=del 2024-06-28T15:08:15.641 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.652+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=6VFP41HEZRK6 type=del size=1836126 2024-06-28T15:08:15.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.656+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~DYSV6IDQ5N3Z.0 size=1048576 2024-06-28T15:08:15.646 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.660+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~DYSV6IDQ5N3Z.1 size=1048576 2024-06-28T15:08:15.648 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.660+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~DYSV6IDQ5N3Z.2 size=252895 2024-06-28T15:08:15.653 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.664+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=DYSV6IDQ5N3Z tag=7MYYB1WYI29M type=write size=3398623 2024-06-28T15:08:15.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.668+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=MVV1O21NMCME type=del size=1750864 2024-06-28T15:08:15.662 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.676+0000 7f65680598c0 1 < suggested update operation key=GAZQ3PCPKN9Q tag=AWL880BO87CH type=del size=3097134 2024-06-28T15:08:15.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.676+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=FR3P67XHDJ0O type=write size=130266 2024-06-28T15:08:15.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.680+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=ZRAC744LBYKY type=del 2024-06-28T15:08:15.673 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.684+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~A3HMJ6XFZUFL.0 size=1048576 2024-06-28T15:08:15.675 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.688+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~A3HMJ6XFZUFL.1 size=1048576 2024-06-28T15:08:15.677 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.688+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~A3HMJ6XFZUFL.2 size=1013103 2024-06-28T15:08:15.677 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.688+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=A3HMJ6XFZUFL tag=2EBEDB48ZHSL type=write size=4158831 2024-06-28T15:08:15.680 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.692+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~JIFX7AVG5G5Z.0 size=922826 2024-06-28T15:08:15.685 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.696+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=JIFX7AVG5G5Z tag=OQZHMQD4CBLS type=write size=1971402 2024-06-28T15:08:15.691 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.704+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~CFXHMY1F3URR.0 size=774807 2024-06-28T15:08:15.691 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.704+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=CFXHMY1F3URR tag=Z66A16V7X0Q2 type=write size=1823383 2024-06-28T15:08:15.696 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.708+0000 7f65680598c0 1 < suggested remove operation key=UP3XFF38IJXN tag=5ZLN1SJB4XQ2 type=del 2024-06-28T15:08:15.699 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.712+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=AHYH40JSCJJP type=del size=2814699 2024-06-28T15:08:15.702 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.716+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~F6I0N684N1SF.0 size=1048576 2024-06-28T15:08:15.705 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.716+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~F6I0N684N1SF.1 size=1048576 2024-06-28T15:08:15.707 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.720+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~F6I0N684N1SF.2 size=562714 2024-06-28T15:08:15.713 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.724+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=F6I0N684N1SF tag=S8S7F1U2GK8H type=write size=3708442 2024-06-28T15:08:15.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.732+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~2Q2NRMUC2T34.0 size=1048576 2024-06-28T15:08:15.721 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.732+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~2Q2NRMUC2T34.1 size=1048576 2024-06-28T15:08:15.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.736+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~2Q2NRMUC2T34.2 size=415802 2024-06-28T15:08:15.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.736+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=2Q2NRMUC2T34 tag=5C3NPI15ADQZ type=write size=3561530 2024-06-28T15:08:15.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.740+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=U89IY696JXTT type=del 2024-06-28T15:08:15.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.744+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=13Q6LRZEMS40 type=del size=1803600 2024-06-28T15:08:15.733 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.744+0000 7f65680598c0 1 < canceled operation key=P278ZM1H052S tag=7EY9058JDJXL type=write size=333423 2024-06-28T15:08:15.736 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.748+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=OK71Q94A5WSU type=del size=3591042 2024-06-28T15:08:15.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.752+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=YQ0165ZNRQZI type=del 2024-06-28T15:08:15.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.756+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=1PUDT6O5E0BZ type=del size=61156 2024-06-28T15:08:15.745 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.756+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=77O2ELLPW6GY type=del 2024-06-28T15:08:15.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.760+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=DZTMTJIYN7RG type=del size=93728 2024-06-28T15:08:15.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.764+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=U2HF2WZN3PIB type=del 2024-06-28T15:08:15.755 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.768+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~D9GWYDRPSJ1A.0 size=1048576 2024-06-28T15:08:15.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.772+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~D9GWYDRPSJ1A.1 size=1048576 2024-06-28T15:08:15.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.772+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~D9GWYDRPSJ1A.2 size=615535 2024-06-28T15:08:15.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.772+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=D9GWYDRPSJ1A tag=080EP86KM2LE type=write size=3761263 2024-06-28T15:08:15.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.776+0000 7f65680598c0 1 < completed write operation key=2FC9DNEZGWC4 tag=M7ILGGNXYFSQ type=write size=180701 2024-06-28T15:08:15.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.776+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=QX6DMKX1T51E type=del size=3143927 2024-06-28T15:08:15.769 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.780+0000 7f65680598c0 1 < completed write operation key=60QVJVV16QFM tag=P0R57ZR9TT6T type=write size=578759 2024-06-28T15:08:15.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.788+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~TM9RH10KWKJI.0 size=1048576 2024-06-28T15:08:15.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.788+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~TM9RH10KWKJI.1 size=716387 2024-06-28T15:08:15.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.788+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=TM9RH10KWKJI tag=SZ54YN3E0G6G type=write size=2813539 2024-06-28T15:08:15.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.796+0000 7f65680598c0 1 < suggested update operation key=OUID7RX5NVV6 tag=31ZF7D02TWHT type=del size=3087834 2024-06-28T15:08:15.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.796+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=P3K7DWKNWE6O type=del size=3007562 2024-06-28T15:08:15.787 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.800+0000 7f65680598c0 1 < canceled operation key=EN76QQWIQAXZ tag=LICOS68OI3TE type=del size=1663703 2024-06-28T15:08:15.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.804+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=9OBQKSXSZR1T type=del size=3106032 2024-06-28T15:08:15.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.804+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=6VFP41HEZRK6 type=del 2024-06-28T15:08:15.798 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.812+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~QVVR4N07XY8F.0 size=1048576 2024-06-28T15:08:15.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.816+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~QVVR4N07XY8F.1 size=248713 2024-06-28T15:08:15.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.816+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=QVVR4N07XY8F tag=TSUMB9XMC98N type=write size=2345865 2024-06-28T15:08:15.806 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.820+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=MVV1O21NMCME type=del 2024-06-28T15:08:15.810 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.824+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~1QFBP3L2DBWA.0 size=1048576 2024-06-28T15:08:15.812 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.824+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~1QFBP3L2DBWA.1 size=633231 2024-06-28T15:08:15.812 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.824+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=1QFBP3L2DBWA tag=PXH1UGD0BIO3 type=write size=2730383 2024-06-28T15:08:15.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.828+0000 7f65680598c0 1 < completed write operation key=ZY973GBGBQ1Z tag=FR3P67XHDJ0O type=write size=130266 2024-06-28T15:08:15.817 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.832+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=K6SU7G9WI926 type=del size=2077871 2024-06-28T15:08:15.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.832+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~A3HMJ6XFZUFL.0 size=1048576 2024-06-28T15:08:15.823 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.836+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~A3HMJ6XFZUFL.1 size=1048576 2024-06-28T15:08:15.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.836+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~A3HMJ6XFZUFL.2 size=1013103 2024-06-28T15:08:15.830 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.844+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=A3HMJ6XFZUFL tag=2EBEDB48ZHSL type=write size=4158831 2024-06-28T15:08:15.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.848+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~0ZM4MT1TK053.0 size=1026789 2024-06-28T15:08:15.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.848+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=0ZM4MT1TK053 tag=MY5DNLNVCZPL type=write size=2075365 2024-06-28T15:08:15.839 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.852+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~CFXHMY1F3URR.0 size=774807 2024-06-28T15:08:15.842 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.856+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=CFXHMY1F3URR tag=Z66A16V7X0Q2 type=write size=1823383 2024-06-28T15:08:15.845 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.856+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=LK0WJNUYIDS3 type=del size=2639311 2024-06-28T15:08:15.849 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.860+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=AHYH40JSCJJP type=del 2024-06-28T15:08:15.851 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.864+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=I7N90I0BRHZI type=write size=1020731 2024-06-28T15:08:15.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.868+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~2Q2NRMUC2T34.0 size=1048576 2024-06-28T15:08:15.858 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.868+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~2Q2NRMUC2T34.1 size=1048576 2024-06-28T15:08:15.861 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.872+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~2Q2NRMUC2T34.2 size=415802 2024-06-28T15:08:15.866 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.876+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=2Q2NRMUC2T34 tag=5C3NPI15ADQZ type=write size=3561530 2024-06-28T15:08:15.869 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.880+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~ZY9J7805FLIE.0 size=1048576 2024-06-28T15:08:15.870 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.884+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~ZY9J7805FLIE.1 size=1048576 2024-06-28T15:08:15.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.884+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~ZY9J7805FLIE.2 size=182406 2024-06-28T15:08:15.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.884+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=ZY9J7805FLIE tag=QO0JBPC9BAI1 type=write size=3328134 2024-06-28T15:08:15.877 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.892+0000 7f65680598c0 1 < suggested update operation key=GAZQ3PCPKN9Q tag=13Q6LRZEMS40 type=del size=1803600 2024-06-28T15:08:15.883 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.896+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~4T3TM67I326N.0 size=1048576 2024-06-28T15:08:15.884 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.896+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~4T3TM67I326N.1 size=1048576 2024-06-28T15:08:15.886 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.900+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~4T3TM67I326N.2 size=587355 2024-06-28T15:08:15.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.900+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=4T3TM67I326N tag=FDOA6HQQ5IGK type=write size=3733083 2024-06-28T15:08:15.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.904+0000 7f65680598c0 1 < completed delete operation key=1VD7NDPEM3CD tag=OK71Q94A5WSU type=del 2024-06-28T15:08:15.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.904+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=TW7SANQWFAP6 type=del size=847015 2024-06-28T15:08:15.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.908+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=1PUDT6O5E0BZ type=del 2024-06-28T15:08:15.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.912+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=PA9UL30UT351 type=del size=3802940 2024-06-28T15:08:15.901 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.912+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=DZTMTJIYN7RG type=del 2024-06-28T15:08:15.903 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.916+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=0Y1YOW8YQSSJ type=write size=496527 2024-06-28T15:08:15.906 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.920+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~D9GWYDRPSJ1A.0 size=1048576 2024-06-28T15:08:15.908 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.920+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~D9GWYDRPSJ1A.1 size=1048576 2024-06-28T15:08:15.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.924+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~D9GWYDRPSJ1A.2 size=615535 2024-06-28T15:08:15.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.928+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=D9GWYDRPSJ1A tag=080EP86KM2LE type=write size=3761263 2024-06-28T15:08:15.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.932+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~Q0YX6TN0TD1Z.0 size=1048576 2024-06-28T15:08:15.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.936+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~Q0YX6TN0TD1Z.1 size=296875 2024-06-28T15:08:15.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.936+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=Q0YX6TN0TD1Z tag=CUZW43AGRL28 type=write size=2394027 2024-06-28T15:08:15.925 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.936+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=QX6DMKX1T51E type=del 2024-06-28T15:08:15.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.940+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=O9FSU9NY9W0K type=del size=3097581 2024-06-28T15:08:15.929 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.940+0000 7f65680598c0 1 < canceled part key=_multipart_V2L2Q1U62YW2.2~TM9RH10KWKJI.0 size=1048576 2024-06-28T15:08:15.931 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.944+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~TM9RH10KWKJI.1 size=716387 2024-06-28T15:08:15.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.948+0000 7f65680598c0 1 < canceled multipart upload key=V2L2Q1U62YW2 upload=TM9RH10KWKJI tag=SZ54YN3E0G6G type=write size=2813539 2024-06-28T15:08:15.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.952+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=GTYM71Z0RLHG type=write size=524175 2024-06-28T15:08:15.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.952+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=P3K7DWKNWE6O type=del 2024-06-28T15:08:15.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.956+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=TNMTBSY6WEX3 type=del size=1977278 2024-06-28T15:08:15.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.960+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=9OBQKSXSZR1T type=del 2024-06-28T15:08:15.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.960+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=B6W7LZBKY2I3 type=write size=99827 2024-06-28T15:08:15.951 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.964+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~QVVR4N07XY8F.0 size=1048576 2024-06-28T15:08:15.953 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.968+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~QVVR4N07XY8F.1 size=248713 2024-06-28T15:08:15.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.972+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=QVVR4N07XY8F tag=TSUMB9XMC98N type=write size=2345865 2024-06-28T15:08:15.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.972+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=SCC7Y1FSCN7Z type=del size=1623409 2024-06-28T15:08:15.963 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.976+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~1QFBP3L2DBWA.0 size=1048576 2024-06-28T15:08:15.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.980+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~1QFBP3L2DBWA.1 size=633231 2024-06-28T15:08:15.970 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.984+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=1QFBP3L2DBWA tag=PXH1UGD0BIO3 type=write size=2730383 2024-06-28T15:08:15.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.984+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=HNXUPXSLRA77 type=del size=2951479 2024-06-28T15:08:15.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.988+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=K6SU7G9WI926 type=del 2024-06-28T15:08:15.981 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.996+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~BLEGUYLEG8GY.0 size=1048576 2024-06-28T15:08:15.983 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:15.996+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~BLEGUYLEG8GY.1 size=1048576 2024-06-28T15:08:15.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.000+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~BLEGUYLEG8GY.2 size=145545 2024-06-28T15:08:15.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.000+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=BLEGUYLEG8GY tag=5LZBANH38BUD type=write size=3291273 2024-06-28T15:08:15.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.000+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~0ZM4MT1TK053.0 size=1026789 2024-06-28T15:08:15.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.004+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=0ZM4MT1TK053 tag=MY5DNLNVCZPL type=write size=2075365 2024-06-28T15:08:15.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.008+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=0U7UKPZ1K6C5 type=del size=1558292 2024-06-28T15:08:15.999 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.012+0000 7f65680598c0 1 < canceled operation key=O1CGONN4TEXF tag=LK0WJNUYIDS3 type=del size=2639311 2024-06-28T15:08:16.004 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.016+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~7P2W7KYTL0FZ.0 size=1048576 2024-06-28T15:08:16.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.020+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~7P2W7KYTL0FZ.1 size=258122 2024-06-28T15:08:16.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.020+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=7P2W7KYTL0FZ tag=GZLZCMIUJAWX type=write size=2355274 2024-06-28T15:08:16.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.024+0000 7f65680598c0 1 < completed write operation key=SFCIZMLY2Y2E tag=I7N90I0BRHZI type=write size=1020731 2024-06-28T15:08:16.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.024+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=R3B1EP6Z9P3D type=write size=202350 2024-06-28T15:08:16.023 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.028+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~ZY9J7805FLIE.0 size=1048576 2024-06-28T15:08:16.024 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.032+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~ZY9J7805FLIE.1 size=1048576 2024-06-28T15:08:16.024 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.032+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~ZY9J7805FLIE.2 size=182406 2024-06-28T15:08:16.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.040+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=ZY9J7805FLIE tag=QO0JBPC9BAI1 type=write size=3328134 2024-06-28T15:08:16.033 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.044+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~O50L006VSFV7.0 size=753616 2024-06-28T15:08:16.033 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.044+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=O50L006VSFV7 tag=EGF4IMFEQ6QU type=write size=1802192 2024-06-28T15:08:16.036 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.048+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~4T3TM67I326N.0 size=1048576 2024-06-28T15:08:16.039 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.052+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~4T3TM67I326N.1 size=1048576 2024-06-28T15:08:16.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.056+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~4T3TM67I326N.2 size=587355 2024-06-28T15:08:16.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.060+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=4T3TM67I326N tag=FDOA6HQQ5IGK type=write size=3733083 2024-06-28T15:08:16.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.064+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~NXSSA4VP7832.0 size=1048576 2024-06-28T15:08:16.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.068+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~NXSSA4VP7832.1 size=1048576 2024-06-28T15:08:16.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.068+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~NXSSA4VP7832.2 size=535070 2024-06-28T15:08:16.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.068+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=NXSSA4VP7832 tag=AEYRB5GNPIYM type=write size=3680798 2024-06-28T15:08:16.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.076+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=TW7SANQWFAP6 type=del 2024-06-28T15:08:16.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.080+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=M64K93FBV0S0 type=del size=2844682 2024-06-28T15:08:16.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.084+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=PA9UL30UT351 type=del 2024-06-28T15:08:16.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.084+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=242PLMI8XGZL type=del size=1221127 2024-06-28T15:08:16.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.088+0000 7f65680598c0 1 < completed write operation key=O1CGONN4TEXF tag=0Y1YOW8YQSSJ type=write size=496527 2024-06-28T15:08:16.079 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.092+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=3U7CEJKTP66Z type=del size=2483342 2024-06-28T15:08:16.083 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.096+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~Q0YX6TN0TD1Z.0 size=1048576 2024-06-28T15:08:16.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.100+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~Q0YX6TN0TD1Z.1 size=296875 2024-06-28T15:08:16.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.104+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=Q0YX6TN0TD1Z tag=CUZW43AGRL28 type=write size=2394027 2024-06-28T15:08:16.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.108+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~BXUPY9QL97ZG.0 size=1679 2024-06-28T15:08:16.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.108+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=BXUPY9QL97ZG tag=46SF45QS0KFU type=write size=1050255 2024-06-28T15:08:16.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.116+0000 7f65680598c0 1 < suggested remove operation key=O1CGONN4TEXF tag=O9FSU9NY9W0K type=del 2024-06-28T15:08:16.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.116+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=XRRD5AVDE461 type=del size=2509999 2024-06-28T15:08:16.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.124+0000 7f65680598c0 1 < suggested update operation key=I4RWCQSQVLCN tag=GTYM71Z0RLHG type=write size=524175 2024-06-28T15:08:16.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.128+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~MEAZTA7937VK.0 size=1048576 2024-06-28T15:08:16.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.132+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~MEAZTA7937VK.1 size=6551 2024-06-28T15:08:16.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.132+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=MEAZTA7937VK tag=TGLQVBL7J6NE type=write size=2103703 2024-06-28T15:08:16.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.136+0000 7f65680598c0 1 < suggested remove operation key=ZY973GBGBQ1Z tag=TNMTBSY6WEX3 type=del 2024-06-28T15:08:16.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.144+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~2RFTJFUBDFUW.0 size=250550 2024-06-28T15:08:16.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.144+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=2RFTJFUBDFUW tag=5ROE99JPRVQ8 type=write size=1299126 2024-06-28T15:08:16.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.144+0000 7f65680598c0 1 < completed write operation key=J311Q2712YM0 tag=B6W7LZBKY2I3 type=write size=99827 2024-06-28T15:08:16.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.152+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~A1509A8YNL36.0 size=270562 2024-06-28T15:08:16.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.152+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=A1509A8YNL36 tag=26T8IYZAYO7F type=write size=1319138 2024-06-28T15:08:16.140 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.152+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=SCC7Y1FSCN7Z type=del 2024-06-28T15:08:16.143 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.156+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=53JYT0B8QPVH type=del size=1167298 2024-06-28T15:08:16.146 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.160+0000 7f65680598c0 -1 < failed to complete operation key=ZY973GBGBQ1Z tag=HNXUPXSLRA77 type=del size=2951479: (22) Invalid argument 2024-06-28T15:08:16.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.160+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=KVP5VE88JPX4 type=write size=251707 2024-06-28T15:08:16.152 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.164+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~BLEGUYLEG8GY.0 size=1048576 2024-06-28T15:08:16.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.168+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~BLEGUYLEG8GY.1 size=1048576 2024-06-28T15:08:16.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.168+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~BLEGUYLEG8GY.2 size=145545 2024-06-28T15:08:16.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.172+0000 7f65680598c0 1 < completed multipart upload key=MQ3QWZBFN4YR upload=BLEGUYLEG8GY tag=5LZBANH38BUD type=write size=3291273 2024-06-28T15:08:16.164 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.176+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=FZ9TE6VPHQRW type=del size=3733605 2024-06-28T15:08:16.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.180+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=0U7UKPZ1K6C5 type=del 2024-06-28T15:08:16.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.184+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~WCTJAEU991PM.0 size=1048576 2024-06-28T15:08:16.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.188+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~WCTJAEU991PM.1 size=889749 2024-06-28T15:08:16.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.188+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=WCTJAEU991PM tag=BRDI81EVV1PL type=write size=2986901 2024-06-28T15:08:16.177 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.188+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~7P2W7KYTL0FZ.0 size=1048576 2024-06-28T15:08:16.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.192+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~7P2W7KYTL0FZ.1 size=258122 2024-06-28T15:08:16.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.196+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=7P2W7KYTL0FZ tag=GZLZCMIUJAWX type=write size=2355274 2024-06-28T15:08:16.187 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.200+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=UW0GO399D71Z type=del size=454189 2024-06-28T15:08:16.190 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.204+0000 7f65680598c0 1 < completed write operation key=J311Q2712YM0 tag=R3B1EP6Z9P3D type=write size=202350 2024-06-28T15:08:16.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.204+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=BNF7851WZ3N5 type=del size=250578 2024-06-28T15:08:16.195 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.208+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~O50L006VSFV7.0 size=753616 2024-06-28T15:08:16.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.212+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=O50L006VSFV7 tag=EGF4IMFEQ6QU type=write size=1802192 2024-06-28T15:08:16.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.216+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=W6BFUYUTT9O9 type=write size=1012719 2024-06-28T15:08:16.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.220+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~NXSSA4VP7832.0 size=1048576 2024-06-28T15:08:16.212 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.224+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~NXSSA4VP7832.1 size=1048576 2024-06-28T15:08:16.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.228+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~NXSSA4VP7832.2 size=535070 2024-06-28T15:08:16.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.232+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=NXSSA4VP7832 tag=AEYRB5GNPIYM type=write size=3680798 2024-06-28T15:08:16.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.240+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~918SEF56YQF4.0 size=989335 2024-06-28T15:08:16.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.240+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=918SEF56YQF4 tag=FC5QEHR060I7 type=write size=2037911 2024-06-28T15:08:16.233 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.244+0000 7f65680598c0 1 < suggested update operation key=GAZQ3PCPKN9Q tag=M64K93FBV0S0 type=del size=2844682 2024-06-28T15:08:16.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.252+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~73I0BSX81HFX.0 size=1048576 2024-06-28T15:08:16.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.256+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~73I0BSX81HFX.1 size=756301 2024-06-28T15:08:16.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.256+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=73I0BSX81HFX tag=IT52V7AY9GSS type=write size=2853453 2024-06-28T15:08:16.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.260+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=242PLMI8XGZL type=del 2024-06-28T15:08:16.253 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.264+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~11VLT17VJT5B.0 size=1048576 2024-06-28T15:08:16.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.268+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~11VLT17VJT5B.1 size=656504 2024-06-28T15:08:16.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.268+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=11VLT17VJT5B tag=RMZ7WZ633LMV type=write size=2753656 2024-06-28T15:08:16.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.272+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=3U7CEJKTP66Z type=del 2024-06-28T15:08:16.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.276+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=GZPJC42KVJKS type=del size=955173 2024-06-28T15:08:16.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.280+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~BXUPY9QL97ZG.0 size=1679 2024-06-28T15:08:16.271 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.284+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=BXUPY9QL97ZG tag=46SF45QS0KFU type=write size=1050255 2024-06-28T15:08:16.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.288+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~2RO27Y1R4G1R.0 size=1048576 2024-06-28T15:08:16.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.292+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~2RO27Y1R4G1R.1 size=961436 2024-06-28T15:08:16.280 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.292+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=2RO27Y1R4G1R tag=3QCL0PNUOU55 type=write size=3058588 2024-06-28T15:08:16.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.300+0000 7f65680598c0 1 < suggested update operation key=UP3XFF38IJXN tag=XRRD5AVDE461 type=del size=2509999 2024-06-28T15:08:16.292 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.304+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~JP9W13GBL9L2.0 size=788448 2024-06-28T15:08:16.292 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.304+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=JP9W13GBL9L2 tag=VC83U8VTFOXJ type=write size=1837024 2024-06-28T15:08:16.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.308+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~MEAZTA7937VK.0 size=1048576 2024-06-28T15:08:16.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.308+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~MEAZTA7937VK.1 size=6551 2024-06-28T15:08:16.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.316+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=MEAZTA7937VK tag=TGLQVBL7J6NE type=write size=2103703 2024-06-28T15:08:16.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.320+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~LAVHLELU1MMD.0 size=1048576 2024-06-28T15:08:16.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.324+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~LAVHLELU1MMD.1 size=295256 2024-06-28T15:08:16.311 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.324+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=LAVHLELU1MMD tag=V9J7UJMKG239 type=write size=2392408 2024-06-28T15:08:16.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.328+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~2RFTJFUBDFUW.0 size=250550 2024-06-28T15:08:16.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.332+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=2RFTJFUBDFUW tag=5ROE99JPRVQ8 type=write size=1299126 2024-06-28T15:08:16.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.336+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=K0HV1C5E17ZC type=del size=984949 2024-06-28T15:08:16.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.336+0000 7f65680598c0 1 < canceled part key=_multipart_1VD7NDPEM3CD.2~A1509A8YNL36.0 size=270562 2024-06-28T15:08:16.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.344+0000 7f65680598c0 1 < canceled multipart upload key=1VD7NDPEM3CD upload=A1509A8YNL36 tag=26T8IYZAYO7F type=write size=1319138 2024-06-28T15:08:16.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.344+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=MCYUA4F71SJ4 type=del size=2048106 2024-06-28T15:08:16.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.348+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=53JYT0B8QPVH type=del 2024-06-28T15:08:16.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.352+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=E1VHLS7M1RZ2 type=del size=2308496 2024-06-28T15:08:16.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.356+0000 7f65680598c0 1 < completed write operation key=2F968YIMJN9D tag=KVP5VE88JPX4 type=write size=251707 2024-06-28T15:08:16.347 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.360+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=XNCER5MIWIA1 type=del size=4116473 2024-06-28T15:08:16.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.364+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=FZ9TE6VPHQRW type=del 2024-06-28T15:08:16.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.372+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~4AVUVYPGTF6K.0 size=489310 2024-06-28T15:08:16.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.372+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=4AVUVYPGTF6K tag=HU2KJM998LV5 type=write size=1537886 2024-06-28T15:08:16.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.376+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~WCTJAEU991PM.0 size=1048576 2024-06-28T15:08:16.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.380+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~WCTJAEU991PM.1 size=889749 2024-06-28T15:08:16.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.384+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=WCTJAEU991PM tag=BRDI81EVV1PL type=write size=2986901 2024-06-28T15:08:16.375 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.388+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=OZX84K0E5UTB type=write size=911531 2024-06-28T15:08:16.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.392+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=UW0GO399D71Z type=del 2024-06-28T15:08:16.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.396+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=ZY56ZHPV22LN type=del size=2056640 2024-06-28T15:08:16.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.400+0000 7f65680598c0 1 < canceled operation key=SFCIZMLY2Y2E tag=BNF7851WZ3N5 type=del size=250578 2024-06-28T15:08:16.392 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.404+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~ROCGAW9W8Z3O.0 size=1048576 2024-06-28T15:08:16.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.408+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~ROCGAW9W8Z3O.1 size=1048576 2024-06-28T15:08:16.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.412+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~ROCGAW9W8Z3O.2 size=357430 2024-06-28T15:08:16.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.412+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=ROCGAW9W8Z3O tag=QECXM7BRC5C3 type=write size=3503158 2024-06-28T15:08:16.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.412+0000 7f65680598c0 1 < canceled operation key=7LB8MOJL39RM tag=W6BFUYUTT9O9 type=write size=1012719 2024-06-28T15:08:16.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.416+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=JPG7IK3I7J06 type=write size=822792 2024-06-28T15:08:16.409 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.420+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~918SEF56YQF4.0 size=989335 2024-06-28T15:08:16.414 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.428+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=918SEF56YQF4 tag=FC5QEHR060I7 type=write size=2037911 2024-06-28T15:08:16.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.432+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~F48NIRK0NL7D.0 size=1048576 2024-06-28T15:08:16.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.436+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~F48NIRK0NL7D.1 size=268721 2024-06-28T15:08:16.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.436+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=F48NIRK0NL7D tag=PK0G47XTG681 type=write size=2365873 2024-06-28T15:08:16.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.440+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~73I0BSX81HFX.0 size=1048576 2024-06-28T15:08:16.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.440+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~73I0BSX81HFX.1 size=756301 2024-06-28T15:08:16.434 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.448+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=73I0BSX81HFX tag=IT52V7AY9GSS type=write size=2853453 2024-06-28T15:08:16.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.452+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~W1JK83ULVFPR.0 size=1010956 2024-06-28T15:08:16.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.452+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=W1JK83ULVFPR tag=8G1QLVPSUJ99 type=write size=2059532 2024-06-28T15:08:16.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.456+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~11VLT17VJT5B.0 size=1048576 2024-06-28T15:08:16.444 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.456+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~11VLT17VJT5B.1 size=656504 2024-06-28T15:08:16.449 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.460+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=11VLT17VJT5B tag=RMZ7WZ633LMV type=write size=2753656 2024-06-28T15:08:16.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.464+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=QPDHWJZT46LX type=del size=1519435 2024-06-28T15:08:16.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.468+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=GZPJC42KVJKS type=del 2024-06-28T15:08:16.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.472+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=HXITML0E3GHO type=del size=1394495 2024-06-28T15:08:16.462 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.476+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~2RO27Y1R4G1R.0 size=1048576 2024-06-28T15:08:16.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.476+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~2RO27Y1R4G1R.1 size=961436 2024-06-28T15:08:16.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.480+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=2RO27Y1R4G1R tag=3QCL0PNUOU55 type=write size=3058588 2024-06-28T15:08:16.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.484+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=75NK0GQN1JYZ type=write size=430031 2024-06-28T15:08:16.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.488+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~JP9W13GBL9L2.0 size=788448 2024-06-28T15:08:16.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.492+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=JP9W13GBL9L2 tag=VC83U8VTFOXJ type=write size=1837024 2024-06-28T15:08:16.483 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.496+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=E1WHIWK4LH8O type=write size=636905 2024-06-28T15:08:16.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.500+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~LAVHLELU1MMD.0 size=1048576 2024-06-28T15:08:16.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.504+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~LAVHLELU1MMD.1 size=295256 2024-06-28T15:08:16.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.508+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=LAVHLELU1MMD tag=V9J7UJMKG239 type=write size=2392408 2024-06-28T15:08:16.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.512+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=IRKU1BDJ02A7 type=del size=2051479 2024-06-28T15:08:16.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.516+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=K0HV1C5E17ZC type=del 2024-06-28T15:08:16.505 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.516+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=QFRNJV0NXOG7 type=del size=1227888 2024-06-28T15:08:16.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.520+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=MCYUA4F71SJ4 type=del 2024-06-28T15:08:16.511 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.524+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=LFEIANAI050K type=del size=4005447 2024-06-28T15:08:16.514 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.528+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=E1VHLS7M1RZ2 type=del 2024-06-28T15:08:16.520 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.532+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~R0KZKIN8WO12.0 size=1048576 2024-06-28T15:08:16.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.536+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~R0KZKIN8WO12.1 size=1048576 2024-06-28T15:08:16.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.536+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~R0KZKIN8WO12.2 size=639527 2024-06-28T15:08:16.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.536+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=R0KZKIN8WO12 tag=7M13BEWZUSQD type=write size=3785255 2024-06-28T15:08:16.531 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.544+0000 7f65680598c0 1 < suggested remove operation key=EN76QQWIQAXZ tag=XNCER5MIWIA1 type=del 2024-06-28T15:08:16.534 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.548+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=YBY9A7HMPW2E type=del size=1849597 2024-06-28T15:08:16.535 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.548+0000 7f65680598c0 1 < canceled part key=_multipart_ZY973GBGBQ1Z.2~4AVUVYPGTF6K.0 size=489310 2024-06-28T15:08:16.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.552+0000 7f65680598c0 1 < canceled multipart upload key=ZY973GBGBQ1Z upload=4AVUVYPGTF6K tag=HU2KJM998LV5 type=write size=1537886 2024-06-28T15:08:16.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.556+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=9DSB40410COI type=del size=667382 2024-06-28T15:08:16.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.560+0000 7f65680598c0 1 < canceled operation key=1VD7NDPEM3CD tag=OZX84K0E5UTB type=write size=911531 2024-06-28T15:08:16.553 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.564+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~ZO63B93NH07F.0 size=1048576 2024-06-28T15:08:16.555 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.568+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~ZO63B93NH07F.1 size=1048576 2024-06-28T15:08:16.558 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.572+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~ZO63B93NH07F.2 size=426441 2024-06-28T15:08:16.558 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.572+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=ZO63B93NH07F tag=JIECSTZRHJCB type=write size=3572169 2024-06-28T15:08:16.561 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.572+0000 7f65680598c0 1 < canceled operation key=SFCIZMLY2Y2E tag=ZY56ZHPV22LN type=del size=2056640 2024-06-28T15:08:16.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.580+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~EC3TN3QQD3V3.0 size=1048576 2024-06-28T15:08:16.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.580+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~EC3TN3QQD3V3.1 size=1048576 2024-06-28T15:08:16.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.584+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~EC3TN3QQD3V3.2 size=582100 2024-06-28T15:08:16.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.584+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=EC3TN3QQD3V3 tag=EMIA8LDZZZAL type=write size=3727828 2024-06-28T15:08:16.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.588+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~ROCGAW9W8Z3O.0 size=1048576 2024-06-28T15:08:16.577 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.588+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~ROCGAW9W8Z3O.1 size=1048576 2024-06-28T15:08:16.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.592+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~ROCGAW9W8Z3O.2 size=357430 2024-06-28T15:08:16.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.596+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=ROCGAW9W8Z3O tag=QECXM7BRC5C3 type=write size=3503158 2024-06-28T15:08:16.589 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.604+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~6M6120D30JH8.0 size=796601 2024-06-28T15:08:16.589 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.604+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=6M6120D30JH8 tag=DB7SPHC1CD7O type=write size=1845177 2024-06-28T15:08:16.593 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.604+0000 7f65680598c0 1 < completed write operation key=LZX5UZCYLTAG tag=JPG7IK3I7J06 type=write size=822792 2024-06-28T15:08:16.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.612+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~EHTTSUHX4HH6.0 size=87339 2024-06-28T15:08:16.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.612+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=EHTTSUHX4HH6 tag=2KU51TM5H0ZA type=write size=1135915 2024-06-28T15:08:16.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.616+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~F48NIRK0NL7D.0 size=1048576 2024-06-28T15:08:16.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.616+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~F48NIRK0NL7D.1 size=268721 2024-06-28T15:08:16.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.620+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=F48NIRK0NL7D tag=PK0G47XTG681 type=write size=2365873 2024-06-28T15:08:16.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.624+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=Y3MDTL96V6YU type=del size=3708627 2024-06-28T15:08:16.615 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.628+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~W1JK83ULVFPR.0 size=1010956 2024-06-28T15:08:16.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.632+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=W1JK83ULVFPR tag=8G1QLVPSUJ99 type=write size=2059532 2024-06-28T15:08:16.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.636+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=W98I85E8VJDE type=del size=3443665 2024-06-28T15:08:16.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.640+0000 7f65680598c0 1 < suggested update operation key=R0IMFVF5NEOO tag=QPDHWJZT46LX type=del size=1519435 2024-06-28T15:08:16.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.644+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=VQD9QU65NJKV type=del size=419443 2024-06-28T15:08:16.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.644+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=HXITML0E3GHO type=del 2024-06-28T15:08:16.636 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.648+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=3MB2Y2XOR256 type=del size=3336185 2024-06-28T15:08:16.639 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.652+0000 7f65680598c0 1 < completed write operation key=I4RWCQSQVLCN tag=75NK0GQN1JYZ type=write size=430031 2024-06-28T15:08:16.641 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.656+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=QRHPSEXBOJAT type=del size=1372879 2024-06-28T15:08:16.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.656+0000 7f65680598c0 -1 < failed to complete operation key=R0IMFVF5NEOO tag=E1WHIWK4LH8O type=write size=636905: (22) Invalid argument 2024-06-28T15:08:16.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.664+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~V36PG9MGD7Y3.0 size=298011 2024-06-28T15:08:16.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.664+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=V36PG9MGD7Y3 tag=PUETHQWUW72H type=write size=1346587 2024-06-28T15:08:16.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.668+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=IRKU1BDJ02A7 type=del 2024-06-28T15:08:16.657 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.672+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=907AFB7AOIPV type=del size=1859839 2024-06-28T15:08:16.661 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.672+0000 7f65680598c0 1 < canceled operation key=2F968YIMJN9D tag=QFRNJV0NXOG7 type=del size=1227888 2024-06-28T15:08:16.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.676+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~6TKNBCKXEZ4E.0 size=498188 2024-06-28T15:08:16.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.676+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=6TKNBCKXEZ4E tag=0ND5QIVSBWY3 type=write size=1546764 2024-06-28T15:08:16.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.680+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=LFEIANAI050K type=del 2024-06-28T15:08:16.673 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.684+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~GQ67FHNGY3WE.0 size=1048576 2024-06-28T15:08:16.675 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.688+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~GQ67FHNGY3WE.1 size=1048576 2024-06-28T15:08:16.677 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.692+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~GQ67FHNGY3WE.2 size=372473 2024-06-28T15:08:16.677 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.692+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=GQ67FHNGY3WE tag=PHAW3VDW57DW type=write size=3518201 2024-06-28T15:08:16.681 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.692+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~R0KZKIN8WO12.0 size=1048576 2024-06-28T15:08:16.684 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.696+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~R0KZKIN8WO12.1 size=1048576 2024-06-28T15:08:16.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.700+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~R0KZKIN8WO12.2 size=639527 2024-06-28T15:08:16.691 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.704+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=R0KZKIN8WO12 tag=7M13BEWZUSQD type=write size=3785255 2024-06-28T15:08:16.697 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.712+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~EWG9BFCA8W1S.0 size=805887 2024-06-28T15:08:16.697 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.712+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=EWG9BFCA8W1S tag=S723HZ7BI88P type=write size=1854463 2024-06-28T15:08:16.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.712+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=YBY9A7HMPW2E type=del 2024-06-28T15:08:16.706 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.720+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~I09H3324Y3KH.0 size=1048576 2024-06-28T15:08:16.708 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.720+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~I09H3324Y3KH.1 size=988228 2024-06-28T15:08:16.708 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.720+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=I09H3324Y3KH tag=RV8MWR52GBZS type=write size=3085380 2024-06-28T15:08:16.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.728+0000 7f65680598c0 1 < suggested remove operation key=GAZQ3PCPKN9Q tag=9DSB40410COI type=del 2024-06-28T15:08:16.718 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.732+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=2037G7HMQVKU type=del size=2343966 2024-06-28T15:08:16.721 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.736+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~ZO63B93NH07F.0 size=1048576 2024-06-28T15:08:16.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.736+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~ZO63B93NH07F.1 size=1048576 2024-06-28T15:08:16.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.740+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~ZO63B93NH07F.2 size=426441 2024-06-28T15:08:16.733 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.744+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=ZO63B93NH07F tag=JIECSTZRHJCB type=write size=3572169 2024-06-28T15:08:16.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.752+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~COD4PSVVW3BQ.0 size=1048576 2024-06-28T15:08:16.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.756+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~COD4PSVVW3BQ.1 size=1048576 2024-06-28T15:08:16.745 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.756+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~COD4PSVVW3BQ.2 size=910309 2024-06-28T15:08:16.745 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.756+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=COD4PSVVW3BQ tag=W70T73CEBLWR type=write size=4056037 2024-06-28T15:08:16.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.760+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~EC3TN3QQD3V3.0 size=1048576 2024-06-28T15:08:16.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.764+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~EC3TN3QQD3V3.1 size=1048576 2024-06-28T15:08:16.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.768+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~EC3TN3QQD3V3.2 size=582100 2024-06-28T15:08:16.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.772+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=EC3TN3QQD3V3 tag=EMIA8LDZZZAL type=write size=3727828 2024-06-28T15:08:16.764 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.776+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=N0LV7TOR16BO type=del size=1184589 2024-06-28T15:08:16.768 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.780+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~6M6120D30JH8.0 size=796601 2024-06-28T15:08:16.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.784+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=6M6120D30JH8 tag=DB7SPHC1CD7O type=write size=1845177 2024-06-28T15:08:16.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.792+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~BQNZ3SUUMWSJ.0 size=1048576 2024-06-28T15:08:16.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.796+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~BQNZ3SUUMWSJ.1 size=368122 2024-06-28T15:08:16.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.796+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=BQNZ3SUUMWSJ tag=X3V9WS5OXJP5 type=write size=2465274 2024-06-28T15:08:16.786 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.800+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~EHTTSUHX4HH6.0 size=87339 2024-06-28T15:08:16.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.804+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=EHTTSUHX4HH6 tag=2KU51TM5H0ZA type=write size=1135915 2024-06-28T15:08:16.795 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.808+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=1DAVH2ERAUS9 type=del size=1800430 2024-06-28T15:08:16.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.812+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=Y3MDTL96V6YU type=del 2024-06-28T15:08:16.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.816+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=P65H67MJW75B type=write size=387881 2024-06-28T15:08:16.809 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.824+0000 7f65680598c0 1 < suggested remove operation key=ZFNIA3O84EB7 tag=W98I85E8VJDE type=del 2024-06-28T15:08:16.813 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.824+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=GM0ZED5L4P21 type=del size=3565177 2024-06-28T15:08:16.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.832+0000 7f65680598c0 1 < suggested update operation key=UP3XFF38IJXN tag=VQD9QU65NJKV type=del size=419443 2024-06-28T15:08:16.824 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.836+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=S2SVEPABTYAA type=del size=2670082 2024-06-28T15:08:16.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.840+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=3MB2Y2XOR256 type=del 2024-06-28T15:08:16.833 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.844+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~UWW7ASCEQCBA.0 size=1048576 2024-06-28T15:08:16.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.848+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~UWW7ASCEQCBA.1 size=1048576 2024-06-28T15:08:16.838 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.852+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~UWW7ASCEQCBA.2 size=421456 2024-06-28T15:08:16.838 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.852+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=UWW7ASCEQCBA tag=WEB23H3GLM1T type=write size=3567184 2024-06-28T15:08:16.844 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.856+0000 7f65680598c0 1 < suggested update operation key=1VD7NDPEM3CD tag=QRHPSEXBOJAT type=del size=1372879 2024-06-28T15:08:16.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.860+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=P93U5VBLNK30 type=del size=2840783 2024-06-28T15:08:16.848 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.860+0000 7f65680598c0 1 < canceled part key=_multipart_I4RWCQSQVLCN.2~V36PG9MGD7Y3.0 size=298011 2024-06-28T15:08:16.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.864+0000 7f65680598c0 1 < canceled multipart upload key=I4RWCQSQVLCN upload=V36PG9MGD7Y3 tag=PUETHQWUW72H type=write size=1346587 2024-06-28T15:08:16.856 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.868+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=5YN8UFAZKD7P type=del size=3765641 2024-06-28T15:08:16.860 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.872+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=907AFB7AOIPV type=del 2024-06-28T15:08:16.863 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.876+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=R32EKWX7WX0T type=del size=3899174 2024-06-28T15:08:16.866 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.880+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~6TKNBCKXEZ4E.0 size=498188 2024-06-28T15:08:16.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.884+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=6TKNBCKXEZ4E tag=0ND5QIVSBWY3 type=write size=1546764 2024-06-28T15:08:16.878 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.892+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~1UXXHXAES2BU.0 size=1048576 2024-06-28T15:08:16.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.892+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~1UXXHXAES2BU.1 size=1048576 2024-06-28T15:08:16.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.896+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~1UXXHXAES2BU.2 size=33822 2024-06-28T15:08:16.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.896+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=1UXXHXAES2BU tag=G002QB6QAVI4 type=write size=3179550 2024-06-28T15:08:16.885 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.896+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~GQ67FHNGY3WE.0 size=1048576 2024-06-28T15:08:16.888 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.900+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~GQ67FHNGY3WE.1 size=1048576 2024-06-28T15:08:16.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.904+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~GQ67FHNGY3WE.2 size=372473 2024-06-28T15:08:16.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.908+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=GQ67FHNGY3WE tag=PHAW3VDW57DW type=write size=3518201 2024-06-28T15:08:16.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.912+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=PR9G8E5JBVRE type=del size=1682261 2024-06-28T15:08:16.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.916+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~EWG9BFCA8W1S.0 size=805887 2024-06-28T15:08:16.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.920+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=EWG9BFCA8W1S tag=S723HZ7BI88P type=write size=1854463 2024-06-28T15:08:16.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.924+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=OKH85XO7AN9W type=del size=3595788 2024-06-28T15:08:16.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.924+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~I09H3324Y3KH.0 size=1048576 2024-06-28T15:08:16.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.928+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~I09H3324Y3KH.1 size=988228 2024-06-28T15:08:16.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.932+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=I09H3324Y3KH tag=RV8MWR52GBZS type=write size=3085380 2024-06-28T15:08:16.925 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.936+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~MZA6PD2JT1WJ.0 size=960991 2024-06-28T15:08:16.925 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.936+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=MZA6PD2JT1WJ tag=3M2AD6YHBQKS type=write size=2009567 2024-06-28T15:08:16.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.940+0000 7f65680598c0 -1 < failed to complete operation key=UP3XFF38IJXN tag=2037G7HMQVKU type=del size=2343966: (22) Invalid argument 2024-06-28T15:08:16.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.948+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~YSY7C2AJZZLR.0 size=1048576 2024-06-28T15:08:16.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.948+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~YSY7C2AJZZLR.1 size=952609 2024-06-28T15:08:16.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.948+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=YSY7C2AJZZLR tag=G5YH62JGC0Q9 type=write size=3049761 2024-06-28T15:08:16.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.952+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~COD4PSVVW3BQ.0 size=1048576 2024-06-28T15:08:16.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.956+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~COD4PSVVW3BQ.1 size=1048576 2024-06-28T15:08:16.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.960+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~COD4PSVVW3BQ.2 size=910309 2024-06-28T15:08:16.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.964+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=COD4PSVVW3BQ tag=W70T73CEBLWR type=write size=4056037 2024-06-28T15:08:16.954 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.968+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~7ZVTSGLICWY5.0 size=1048576 2024-06-28T15:08:16.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.968+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~7ZVTSGLICWY5.1 size=1048576 2024-06-28T15:08:16.958 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.972+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~7ZVTSGLICWY5.2 size=266341 2024-06-28T15:08:16.958 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.972+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=7ZVTSGLICWY5 tag=JWX64O17G7H4 type=write size=3412069 2024-06-28T15:08:16.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.972+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=N0LV7TOR16BO type=del 2024-06-28T15:08:16.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.976+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~853SP04V9INL.0 size=1048576 2024-06-28T15:08:16.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.980+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~853SP04V9INL.1 size=574811 2024-06-28T15:08:16.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.980+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=853SP04V9INL tag=VGKZMZFOD800 type=write size=2671963 2024-06-28T15:08:16.970 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.984+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~BQNZ3SUUMWSJ.0 size=1048576 2024-06-28T15:08:16.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.984+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~BQNZ3SUUMWSJ.1 size=368122 2024-06-28T15:08:16.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.988+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=BQNZ3SUUMWSJ tag=X3V9WS5OXJP5 type=write size=2465274 2024-06-28T15:08:16.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.992+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=LQVP8IS50847 type=write size=686473 2024-06-28T15:08:16.981 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.996+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=1DAVH2ERAUS9 type=del 2024-06-28T15:08:16.984 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:16.996+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=YH04Z1YKEM4H type=del size=477463 2024-06-28T15:08:16.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.000+0000 7f65680598c0 1 < completed write operation key=7NL3CTFYY9E5 tag=P65H67MJW75B type=write size=387881 2024-06-28T15:08:16.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.000+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=NR47Y216Y6KY type=write size=486114 2024-06-28T15:08:16.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.008+0000 7f65680598c0 1 < suggested update operation key=ZY973GBGBQ1Z tag=GM0ZED5L4P21 type=del size=3565177 2024-06-28T15:08:16.998 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.012+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=99TK31TJOWNI type=del size=859185 2024-06-28T15:08:17.003 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.016+0000 7f65680598c0 1 < suggested update operation key=GAZQ3PCPKN9Q tag=S2SVEPABTYAA type=del size=2670082 2024-06-28T15:08:17.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.020+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=ZMY94MI7HO4W type=del size=2334115 2024-06-28T15:08:17.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.020+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~UWW7ASCEQCBA.0 size=1048576 2024-06-28T15:08:17.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.024+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~UWW7ASCEQCBA.1 size=1048576 2024-06-28T15:08:17.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.024+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~UWW7ASCEQCBA.2 size=421456 2024-06-28T15:08:17.017 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.032+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=UWW7ASCEQCBA tag=WEB23H3GLM1T type=write size=3567184 2024-06-28T15:08:17.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.032+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=Y21PYPTYN4ZB type=del size=351588 2024-06-28T15:08:17.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.036+0000 7f65680598c0 1 < canceled operation key=7NL3CTFYY9E5 tag=P93U5VBLNK30 type=del size=2840783 2024-06-28T15:08:17.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.040+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~S6DB84Y3PAWC.0 size=21081 2024-06-28T15:08:17.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.040+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=S6DB84Y3PAWC tag=C1EI59ZQE944 type=write size=1069657 2024-06-28T15:08:17.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.044+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=5YN8UFAZKD7P type=del 2024-06-28T15:08:17.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.044+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=QWHWIQ8VVKGD type=write size=1040193 2024-06-28T15:08:17.035 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.048+0000 7f65680598c0 1 < canceled operation key=RLJCNXKA2GYA tag=R32EKWX7WX0T type=del size=3899174 2024-06-28T15:08:17.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.052+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=1Q313AKGBZOD type=write size=393787 2024-06-28T15:08:17.040 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.052+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~1UXXHXAES2BU.0 size=1048576 2024-06-28T15:08:17.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.056+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~1UXXHXAES2BU.1 size=1048576 2024-06-28T15:08:17.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.056+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~1UXXHXAES2BU.2 size=33822 2024-06-28T15:08:17.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.060+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=1UXXHXAES2BU tag=G002QB6QAVI4 type=write size=3179550 2024-06-28T15:08:17.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.064+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~EP85HHWYM3MP.0 size=1048576 2024-06-28T15:08:17.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.068+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~EP85HHWYM3MP.1 size=115395 2024-06-28T15:08:17.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.068+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=EP85HHWYM3MP tag=A2L98RLJJ40M type=write size=2212547 2024-06-28T15:08:17.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.072+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=PR9G8E5JBVRE type=del 2024-06-28T15:08:17.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.072+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=YLBYVAPJISEA type=del size=1535581 2024-06-28T15:08:17.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.076+0000 7f65680598c0 1 < suggested remove operation key=J311Q2712YM0 tag=OKH85XO7AN9W type=del 2024-06-28T15:08:17.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.080+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=UVYL48OSQO8X type=del size=143894 2024-06-28T15:08:17.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.084+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~MZA6PD2JT1WJ.0 size=960991 2024-06-28T15:08:17.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.088+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=MZA6PD2JT1WJ tag=3M2AD6YHBQKS type=write size=2009567 2024-06-28T15:08:17.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.088+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=AN0BT0CRM2KG type=del size=2733495 2024-06-28T15:08:17.080 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.092+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~YSY7C2AJZZLR.0 size=1048576 2024-06-28T15:08:17.082 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.096+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~YSY7C2AJZZLR.1 size=952609 2024-06-28T15:08:17.086 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.100+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=YSY7C2AJZZLR tag=G5YH62JGC0Q9 type=write size=3049761 2024-06-28T15:08:17.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.104+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~2XXC9Z7PGBS8.0 size=651361 2024-06-28T15:08:17.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.104+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=2XXC9Z7PGBS8 tag=6QC0FASDQUX7 type=write size=1699937 2024-06-28T15:08:17.093 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.104+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~7ZVTSGLICWY5.0 size=1048576 2024-06-28T15:08:17.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.108+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~7ZVTSGLICWY5.1 size=1048576 2024-06-28T15:08:17.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.112+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~7ZVTSGLICWY5.2 size=266341 2024-06-28T15:08:17.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.116+0000 7f65680598c0 1 < completed multipart upload key=MQ3QWZBFN4YR upload=7ZVTSGLICWY5 tag=JWX64O17G7H4 type=write size=3412069 2024-06-28T15:08:17.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.120+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~3KM1OA62FOL4.0 size=1048576 2024-06-28T15:08:17.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.124+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~3KM1OA62FOL4.1 size=316819 2024-06-28T15:08:17.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.124+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=3KM1OA62FOL4 tag=HIWE8AIFX0HO type=write size=2413971 2024-06-28T15:08:17.114 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.128+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~853SP04V9INL.0 size=1048576 2024-06-28T15:08:17.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.128+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~853SP04V9INL.1 size=574811 2024-06-28T15:08:17.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.132+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=853SP04V9INL tag=VGKZMZFOD800 type=write size=2671963 2024-06-28T15:08:17.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.136+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=TW2845D9PEUR type=del size=1711823 2024-06-28T15:08:17.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.140+0000 7f65680598c0 1 < canceled operation key=7NL3CTFYY9E5 tag=LQVP8IS50847 type=write size=686473 2024-06-28T15:08:17.131 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.144+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~PB3PJ0J5R772.0 size=1048576 2024-06-28T15:08:17.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.144+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~PB3PJ0J5R772.1 size=261255 2024-06-28T15:08:17.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.144+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=PB3PJ0J5R772 tag=25HBV2CMF7BK type=write size=2358407 2024-06-28T15:08:17.136 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.148+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=YH04Z1YKEM4H type=del 2024-06-28T15:08:17.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.152+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=HDO404OF7GJP type=del size=1324862 2024-06-28T15:08:17.141 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.152+0000 7f65680598c0 1 < completed write operation key=KFHGA64L1NLP tag=NR47Y216Y6KY type=write size=486114 2024-06-28T15:08:17.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.156+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=BQJNBG4M2WLU type=del size=2449388 2024-06-28T15:08:17.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.160+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=99TK31TJOWNI type=del 2024-06-28T15:08:17.152 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.164+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~LBUF9UV24T32.0 size=1048576 2024-06-28T15:08:17.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.168+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~LBUF9UV24T32.1 size=1048576 2024-06-28T15:08:17.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.168+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~LBUF9UV24T32.2 size=164389 2024-06-28T15:08:17.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.168+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=LBUF9UV24T32 tag=8Z04I4HBQGKO type=write size=3310117 2024-06-28T15:08:17.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.172+0000 7f65680598c0 1 < canceled operation key=BHW2VHUT0U1U tag=ZMY94MI7HO4W type=del size=2334115 2024-06-28T15:08:17.162 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.176+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=9SORVAH68WRD type=del size=3679634 2024-06-28T15:08:17.164 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.176+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=Y21PYPTYN4ZB type=del 2024-06-28T15:08:17.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.180+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~BPLGSMP13B2Q.0 size=1048576 2024-06-28T15:08:17.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.184+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~BPLGSMP13B2Q.1 size=1048576 2024-06-28T15:08:17.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.184+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~BPLGSMP13B2Q.2 size=327781 2024-06-28T15:08:17.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.184+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=BPLGSMP13B2Q tag=1H5K9U521ZU3 type=write size=3473509 2024-06-28T15:08:17.176 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.188+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~S6DB84Y3PAWC.0 size=21081 2024-06-28T15:08:17.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.192+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=S6DB84Y3PAWC tag=C1EI59ZQE944 type=write size=1069657 2024-06-28T15:08:17.189 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.204+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~CON4DJFBJU1P.0 size=984554 2024-06-28T15:08:17.189 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.204+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=CON4DJFBJU1P tag=CGO1D8LMOWKZ type=write size=2033130 2024-06-28T15:08:17.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.204+0000 7f65680598c0 1 < canceled operation key=ZY973GBGBQ1Z tag=QWHWIQ8VVKGD type=write size=1040193 2024-06-28T15:08:17.195 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.208+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=UY1EU1J6K60A type=del size=2590068 2024-06-28T15:08:17.198 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.212+0000 7f65680598c0 1 < completed write operation key=4CU86XS1R9G3 tag=1Q313AKGBZOD type=write size=393787 2024-06-28T15:08:17.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.216+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~BWDIQ0MZEUI0.0 size=321315 2024-06-28T15:08:17.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.216+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=BWDIQ0MZEUI0 tag=0U32CMFEJ3PN type=write size=1369891 2024-06-28T15:08:17.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.220+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~EP85HHWYM3MP.0 size=1048576 2024-06-28T15:08:17.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.220+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~EP85HHWYM3MP.1 size=115395 2024-06-28T15:08:17.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.228+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=EP85HHWYM3MP tag=A2L98RLJJ40M type=write size=2212547 2024-06-28T15:08:17.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.228+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=S99IO8E5FFBX type=del size=1826993 2024-06-28T15:08:17.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.232+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=YLBYVAPJISEA type=del 2024-06-28T15:08:17.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.236+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=ZJ2UBL4RZCSL type=del size=1500745 2024-06-28T15:08:17.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.236+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=UVYL48OSQO8X type=del 2024-06-28T15:08:17.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.240+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=HU74XWDP9EF6 type=del size=1662849 2024-06-28T15:08:17.233 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.248+0000 7f65680598c0 1 < suggested update operation key=ZY973GBGBQ1Z tag=AN0BT0CRM2KG type=del size=2733495 2024-06-28T15:08:17.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.252+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~ICLRWD7TI5QT.0 size=1048576 2024-06-28T15:08:17.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.252+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~ICLRWD7TI5QT.1 size=838485 2024-06-28T15:08:17.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.252+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=ICLRWD7TI5QT tag=JXWIQFH7X02D type=write size=2935637 2024-06-28T15:08:17.244 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.256+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~2XXC9Z7PGBS8.0 size=651361 2024-06-28T15:08:17.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.264+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=2XXC9Z7PGBS8 tag=6QC0FASDQUX7 type=write size=1699937 2024-06-28T15:08:17.253 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.264+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=CGMQ9GY1R7EH type=write size=259510 2024-06-28T15:08:17.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.268+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~3KM1OA62FOL4.0 size=1048576 2024-06-28T15:08:17.258 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.272+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~3KM1OA62FOL4.1 size=316819 2024-06-28T15:08:17.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.276+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=3KM1OA62FOL4 tag=HIWE8AIFX0HO type=write size=2413971 2024-06-28T15:08:17.267 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.280+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=0OTC9EZPZ6R1 type=del size=3518511 2024-06-28T15:08:17.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.284+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=TW2845D9PEUR type=del 2024-06-28T15:08:17.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.284+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=U2KEGU81U8OV type=del size=2435634 2024-06-28T15:08:17.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.288+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~PB3PJ0J5R772.0 size=1048576 2024-06-28T15:08:17.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.292+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~PB3PJ0J5R772.1 size=261255 2024-06-28T15:08:17.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.296+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=PB3PJ0J5R772 tag=25HBV2CMF7BK type=write size=2358407 2024-06-28T15:08:17.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.300+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=0ZXQZQJI205S type=write size=75193 2024-06-28T15:08:17.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.304+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=HDO404OF7GJP type=del 2024-06-28T15:08:17.293 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.304+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=2P492N2S82QH type=del size=3746586 2024-06-28T15:08:17.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.308+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=BQJNBG4M2WLU type=del 2024-06-28T15:08:17.300 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.312+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=1F3G7PXWU6H0 type=write size=821408 2024-06-28T15:08:17.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.312+0000 7f65680598c0 1 < canceled part key=_multipart_P278ZM1H052S.2~LBUF9UV24T32.0 size=1048576 2024-06-28T15:08:17.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.312+0000 7f65680598c0 1 < canceled part key=_multipart_P278ZM1H052S.2~LBUF9UV24T32.1 size=1048576 2024-06-28T15:08:17.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.316+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~LBUF9UV24T32.2 size=164389 2024-06-28T15:08:17.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.320+0000 7f65680598c0 1 < canceled multipart upload key=P278ZM1H052S upload=LBUF9UV24T32 tag=8Z04I4HBQGKO type=write size=3310117 2024-06-28T15:08:17.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.324+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=1OAP1YEXOTXW type=del size=2058132 2024-06-28T15:08:17.317 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.328+0000 7f65680598c0 1 < canceled operation key=OUID7RX5NVV6 tag=9SORVAH68WRD type=del size=3679634 2024-06-28T15:08:17.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.336+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~LVRU6AMBYW2N.0 size=478475 2024-06-28T15:08:17.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.336+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=LVRU6AMBYW2N tag=O30G2UDW3CBB type=write size=1527051 2024-06-28T15:08:17.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.336+0000 7f65680598c0 1 < canceled part key=_multipart_MQ3QWZBFN4YR.2~BPLGSMP13B2Q.0 size=1048576 2024-06-28T15:08:17.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.340+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~BPLGSMP13B2Q.1 size=1048576 2024-06-28T15:08:17.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.340+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~BPLGSMP13B2Q.2 size=327781 2024-06-28T15:08:17.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.348+0000 7f65680598c0 1 < canceled multipart upload key=MQ3QWZBFN4YR upload=BPLGSMP13B2Q tag=1H5K9U521ZU3 type=write size=3473509 2024-06-28T15:08:17.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.352+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=GRXGH715ISQM type=del size=3408458 2024-06-28T15:08:17.342 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.356+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~CON4DJFBJU1P.0 size=984554 2024-06-28T15:08:17.347 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.360+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=CON4DJFBJU1P tag=CGO1D8LMOWKZ type=write size=2033130 2024-06-28T15:08:17.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.368+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~YK1K2EHP2QQT.0 size=524678 2024-06-28T15:08:17.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.368+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=YK1K2EHP2QQT tag=V7WJAJVBLU17 type=write size=1573254 2024-06-28T15:08:17.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.372+0000 7f65680598c0 -1 < failed to complete operation key=ZY973GBGBQ1Z tag=UY1EU1J6K60A type=del size=2590068: (22) Invalid argument 2024-06-28T15:08:17.363 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.376+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~XN2ILVL02FZ4.0 size=1048576 2024-06-28T15:08:17.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.380+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~XN2ILVL02FZ4.1 size=1048576 2024-06-28T15:08:17.368 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.380+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~XN2ILVL02FZ4.2 size=180105 2024-06-28T15:08:17.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.380+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=XN2ILVL02FZ4 tag=W76S72REM99C type=write size=3325833 2024-06-28T15:08:17.373 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.384+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~BWDIQ0MZEUI0.0 size=321315 2024-06-28T15:08:17.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.392+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=BWDIQ0MZEUI0 tag=0U32CMFEJ3PN type=write size=1369891 2024-06-28T15:08:17.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.396+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=25P234L2DQF3 type=del size=1561819 2024-06-28T15:08:17.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.400+0000 7f65680598c0 1 < suggested update operation key=I4RWCQSQVLCN tag=S99IO8E5FFBX type=del size=1826993 2024-06-28T15:08:17.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.404+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=62FLIA0SDZWU type=write size=198720 2024-06-28T15:08:17.393 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.408+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=ZJ2UBL4RZCSL type=del 2024-06-28T15:08:17.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.412+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~9J5S2I9SEWCW.0 size=1048576 2024-06-28T15:08:17.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.412+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~9J5S2I9SEWCW.1 size=692101 2024-06-28T15:08:17.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.412+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=9J5S2I9SEWCW tag=MHPQ7ZJMMWA1 type=write size=2789253 2024-06-28T15:08:17.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.416+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=HU74XWDP9EF6 type=del 2024-06-28T15:08:17.408 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.420+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~H7U5417CAJWL.0 size=1048576 2024-06-28T15:08:17.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.424+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~H7U5417CAJWL.1 size=865353 2024-06-28T15:08:17.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.424+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=H7U5417CAJWL tag=0IE5LR45D32R type=write size=2962505 2024-06-28T15:08:17.413 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.428+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~ICLRWD7TI5QT.0 size=1048576 2024-06-28T15:08:17.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.428+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~ICLRWD7TI5QT.1 size=838485 2024-06-28T15:08:17.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.432+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=ICLRWD7TI5QT tag=JXWIQFH7X02D type=write size=2935637 2024-06-28T15:08:17.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.436+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=8E8HO7IU4VLR type=del size=2308322 2024-06-28T15:08:17.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.436+0000 7f65680598c0 1 < completed write operation key=EN76QQWIQAXZ tag=CGMQ9GY1R7EH type=write size=259510 2024-06-28T15:08:17.427 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.440+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=1BH06BDQO36R type=del size=3091513 2024-06-28T15:08:17.430 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.444+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=0OTC9EZPZ6R1 type=del 2024-06-28T15:08:17.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.444+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=ASJNSSLDPXCB type=del size=3966406 2024-06-28T15:08:17.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.448+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=U2KEGU81U8OV type=del 2024-06-28T15:08:17.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.452+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=DK26XXM8EMLD type=write size=898018 2024-06-28T15:08:17.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.452+0000 7f65680598c0 1 < completed write operation key=P278ZM1H052S tag=0ZXQZQJI205S type=write size=75193 2024-06-28T15:08:17.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.456+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~0LYXVXLR0KNW.0 size=207868 2024-06-28T15:08:17.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.456+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=0LYXVXLR0KNW tag=WFBBEW7T7LLZ type=write size=1256444 2024-06-28T15:08:17.447 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.460+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=2P492N2S82QH type=del 2024-06-28T15:08:17.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.464+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~ATYB323UMHM8.0 size=508929 2024-06-28T15:08:17.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.464+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=ATYB323UMHM8 tag=1236839PWV1G type=write size=1557505 2024-06-28T15:08:17.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.468+0000 7f65680598c0 -1 < failed to complete operation key=I4RWCQSQVLCN tag=1F3G7PXWU6H0 type=write size=821408: (22) Invalid argument 2024-06-28T15:08:17.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.468+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=ZG9R666P4ZC5 type=del size=1880371 2024-06-28T15:08:17.462 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.476+0000 7f65680598c0 1 < suggested update operation key=F6SEDD6BH1T2 tag=1OAP1YEXOTXW type=del size=2058132 2024-06-28T15:08:17.465 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.476+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=J64NRO7PLJ8B type=del size=3265436 2024-06-28T15:08:17.467 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.480+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~LVRU6AMBYW2N.0 size=478475 2024-06-28T15:08:17.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.488+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=LVRU6AMBYW2N tag=O30G2UDW3CBB type=write size=1527051 2024-06-28T15:08:17.478 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.492+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=OOMRP99FBRH7 type=write size=866110 2024-06-28T15:08:17.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.492+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=GRXGH715ISQM type=del 2024-06-28T15:08:17.482 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.496+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=9VOR1EF9EKAN type=write size=72601 2024-06-28T15:08:17.484 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.496+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~YK1K2EHP2QQT.0 size=524678 2024-06-28T15:08:17.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.500+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=YK1K2EHP2QQT tag=V7WJAJVBLU17 type=write size=1573254 2024-06-28T15:08:17.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.504+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=P58FH23Q12VW type=del size=1685583 2024-06-28T15:08:17.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.508+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~XN2ILVL02FZ4.0 size=1048576 2024-06-28T15:08:17.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.508+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~XN2ILVL02FZ4.1 size=1048576 2024-06-28T15:08:17.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.512+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~XN2ILVL02FZ4.2 size=180105 2024-06-28T15:08:17.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.516+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=XN2ILVL02FZ4 tag=W76S72REM99C type=write size=3325833 2024-06-28T15:08:17.505 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.516+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=UTO4QCKTHIWK type=del size=3978928 2024-06-28T15:08:17.507 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.520+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=25P234L2DQF3 type=del 2024-06-28T15:08:17.512 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.524+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~VLLRUW0UZ6VS.0 size=1048576 2024-06-28T15:08:17.514 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.528+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~VLLRUW0UZ6VS.1 size=274149 2024-06-28T15:08:17.514 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.528+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=VLLRUW0UZ6VS tag=ZKI5BD7778PZ type=write size=2371301 2024-06-28T15:08:17.517 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.528+0000 7f65680598c0 1 < completed write operation key=UP3XFF38IJXN tag=62FLIA0SDZWU type=write size=198720 2024-06-28T15:08:17.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.532+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=0V1HPM61EGV2 type=del size=3139196 2024-06-28T15:08:17.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.536+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~9J5S2I9SEWCW.0 size=1048576 2024-06-28T15:08:17.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.536+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~9J5S2I9SEWCW.1 size=692101 2024-06-28T15:08:17.528 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.540+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=9J5S2I9SEWCW tag=MHPQ7ZJMMWA1 type=write size=2789253 2024-06-28T15:08:17.530 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.544+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=48D575LJ41V1 type=del size=1320482 2024-06-28T15:08:17.532 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.544+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~H7U5417CAJWL.0 size=1048576 2024-06-28T15:08:17.534 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.548+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~H7U5417CAJWL.1 size=865353 2024-06-28T15:08:17.538 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.552+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=H7U5417CAJWL tag=0IE5LR45D32R type=write size=2962505 2024-06-28T15:08:17.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.556+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~6KS8QF5GTUWM.0 size=338885 2024-06-28T15:08:17.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.556+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=6KS8QF5GTUWM tag=K2PI3XD8KIC8 type=write size=1387461 2024-06-28T15:08:17.545 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.560+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=8E8HO7IU4VLR type=del 2024-06-28T15:08:17.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.560+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=PGXD61T741J5 type=del size=4152366 2024-06-28T15:08:17.551 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.564+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=1BH06BDQO36R type=del 2024-06-28T15:08:17.554 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.568+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=SJALZ1081N8J type=write size=228350 2024-06-28T15:08:17.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.572+0000 7f65680598c0 1 < suggested remove operation key=AOBFAR0UTONH tag=ASJNSSLDPXCB type=del 2024-06-28T15:08:17.561 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.576+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=7DWQOCV9G5NS type=del size=3932210 2024-06-28T15:08:17.564 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.576+0000 7f65680598c0 1 < completed write operation key=60QVJVV16QFM tag=DK26XXM8EMLD type=write size=898018 2024-06-28T15:08:17.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.580+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=JSXFLAX7BTYN type=del size=2670499 2024-06-28T15:08:17.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.580+0000 7f65680598c0 1 < canceled part key=_multipart_ZFNIA3O84EB7.2~0LYXVXLR0KNW.0 size=207868 2024-06-28T15:08:17.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.588+0000 7f65680598c0 1 < canceled multipart upload key=ZFNIA3O84EB7 upload=0LYXVXLR0KNW tag=WFBBEW7T7LLZ type=write size=1256444 2024-06-28T15:08:17.577 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.592+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=50PCWZ72AE0P type=del size=2769973 2024-06-28T15:08:17.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.592+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~ATYB323UMHM8.0 size=508929 2024-06-28T15:08:17.586 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.600+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=ATYB323UMHM8 tag=1236839PWV1G type=write size=1557505 2024-06-28T15:08:17.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.604+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~YC4DX14R2K1Z.0 size=198603 2024-06-28T15:08:17.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.604+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=YC4DX14R2K1Z tag=EEW4W3RJ2FVA type=write size=1247179 2024-06-28T15:08:17.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.608+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=ZG9R666P4ZC5 type=del 2024-06-28T15:08:17.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.612+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=63YSO2TFAZ6X type=del size=2837104 2024-06-28T15:08:17.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.616+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=J64NRO7PLJ8B type=del 2024-06-28T15:08:17.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.620+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=H2YAIEWU925U type=write size=763915 2024-06-28T15:08:17.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.620+0000 7f65680598c0 -1 < no bi entry to suggest for operation key=AOBFAR0UTONH tag=OOMRP99FBRH7 type=write size=866110: (2) No such file or directory 2024-06-28T15:08:17.611 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.624+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=0C45JKHRL89R type=del size=1212381 2024-06-28T15:08:17.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.628+0000 7f65680598c0 1 < completed write operation key=9MQKWSA33R5G tag=9VOR1EF9EKAN type=write size=72601 2024-06-28T15:08:17.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.628+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=7DOSW6029BSJ type=del size=4167573 2024-06-28T15:08:17.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.636+0000 7f65680598c0 1 < suggested remove operation key=O1CGONN4TEXF tag=P58FH23Q12VW type=del 2024-06-28T15:08:17.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.644+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~Y5BXXP3TNED2.0 size=1048576 2024-06-28T15:08:17.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.644+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~Y5BXXP3TNED2.1 size=473376 2024-06-28T15:08:17.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.644+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=Y5BXXP3TNED2 tag=NO2RO20KBATR type=write size=2570528 2024-06-28T15:08:17.638 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.652+0000 7f65680598c0 1 < suggested remove operation key=ZY973GBGBQ1Z tag=UTO4QCKTHIWK type=del 2024-06-28T15:08:17.642 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.656+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=UYWA1E5U523B type=del size=474001 2024-06-28T15:08:17.645 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.656+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~VLLRUW0UZ6VS.0 size=1048576 2024-06-28T15:08:17.648 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.660+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~VLLRUW0UZ6VS.1 size=274149 2024-06-28T15:08:17.653 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.664+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=VLLRUW0UZ6VS tag=ZKI5BD7778PZ type=write size=2371301 2024-06-28T15:08:17.660 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.672+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~62V3RIJXGA7F.0 size=1048576 2024-06-28T15:08:17.662 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.676+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~62V3RIJXGA7F.1 size=608642 2024-06-28T15:08:17.662 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.676+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=62V3RIJXGA7F tag=0Q6CAQ82Z37V type=write size=2705794 2024-06-28T15:08:17.663 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.676+0000 7f65680598c0 -1 < no bi entry to suggest for operation key=O1CGONN4TEXF tag=0V1HPM61EGV2 type=del size=3139196: (2) No such file or directory 2024-06-28T15:08:17.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.680+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=9ZCEZ0JC96D7 type=del size=320852 2024-06-28T15:08:17.672 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.684+0000 7f65680598c0 1 < suggested update operation key=OUID7RX5NVV6 tag=48D575LJ41V1 type=del size=1320482 2024-06-28T15:08:17.676 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.688+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=474E1PDA3XOV type=write size=50838 2024-06-28T15:08:17.679 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.692+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~6KS8QF5GTUWM.0 size=338885 2024-06-28T15:08:17.684 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.696+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=6KS8QF5GTUWM tag=K2PI3XD8KIC8 type=write size=1387461 2024-06-28T15:08:17.688 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.700+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=8G6NMTNIMOQ8 type=del size=2052167 2024-06-28T15:08:17.695 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.708+0000 7f65680598c0 1 < suggested update operation key=47TO7G9EGJKF tag=PGXD61T741J5 type=del size=4152366 2024-06-28T15:08:17.701 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.712+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~Y905E1D7UJZ5.0 size=442897 2024-06-28T15:08:17.701 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.712+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=Y905E1D7UJZ5 tag=I9K88FF8CU8O type=write size=1491473 2024-06-28T15:08:17.705 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.716+0000 7f65680598c0 1 < completed write operation key=GAZQ3PCPKN9Q tag=SJALZ1081N8J type=write size=228350 2024-06-28T15:08:17.709 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.720+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=25BZLPKK6S0G type=del size=933891 2024-06-28T15:08:17.713 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.724+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=7DWQOCV9G5NS type=del 2024-06-28T15:08:17.716 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.728+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=JD8G31SLWURF type=write size=992121 2024-06-28T15:08:17.720 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.732+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=JSXFLAX7BTYN type=del 2024-06-28T15:08:17.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.736+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=50ACWL222UW3 type=del size=2220499 2024-06-28T15:08:17.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.740+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=50PCWZ72AE0P type=del 2024-06-28T15:08:17.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.744+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=24BM96JBF7I3 type=write size=581842 2024-06-28T15:08:17.733 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.744+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~YC4DX14R2K1Z.0 size=198603 2024-06-28T15:08:17.738 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.752+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=YC4DX14R2K1Z tag=EEW4W3RJ2FVA type=write size=1247179 2024-06-28T15:08:17.744 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.756+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~JL6YFLQM5RZ1.0 size=411664 2024-06-28T15:08:17.744 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.756+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=JL6YFLQM5RZ1 tag=GMOV76HPWC1H type=write size=1460240 2024-06-28T15:08:17.747 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.760+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=63YSO2TFAZ6X type=del 2024-06-28T15:08:17.750 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.764+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=T5YY0RMMKMY4 type=del size=3145062 2024-06-28T15:08:17.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.768+0000 7f65680598c0 1 < completed write operation key=F6SEDD6BH1T2 tag=H2YAIEWU925U type=write size=763915 2024-06-28T15:08:17.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.768+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=QIQUNTZ16HX9 type=del size=2567813 2024-06-28T15:08:17.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.776+0000 7f65680598c0 -1 < failed to complete operation key=ZY973GBGBQ1Z tag=0C45JKHRL89R type=del size=1212381: (22) Invalid argument 2024-06-28T15:08:17.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.780+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=OC8Z72N3JGA6 type=write size=579583 2024-06-28T15:08:17.769 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.780+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=7DOSW6029BSJ type=del 2024-06-28T15:08:17.772 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.784+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=CI0LO39ONKFI type=write size=741852 2024-06-28T15:08:17.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.788+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~Y5BXXP3TNED2.0 size=1048576 2024-06-28T15:08:17.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.792+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~Y5BXXP3TNED2.1 size=473376 2024-06-28T15:08:17.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.796+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=Y5BXXP3TNED2 tag=NO2RO20KBATR type=write size=2570528 2024-06-28T15:08:17.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.796+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=B5QG8JWXHYFO type=del size=2164618 2024-06-28T15:08:17.788 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.800+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=UYWA1E5U523B type=del 2024-06-28T15:08:17.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.804+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=R1YERFD14STB type=del size=681896 2024-06-28T15:08:17.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.804+0000 7f65680598c0 1 < canceled part key=_multipart_O1CGONN4TEXF.2~62V3RIJXGA7F.0 size=1048576 2024-06-28T15:08:17.794 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.808+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~62V3RIJXGA7F.1 size=608642 2024-06-28T15:08:17.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.812+0000 7f65680598c0 1 < canceled multipart upload key=O1CGONN4TEXF upload=62V3RIJXGA7F tag=0Q6CAQ82Z37V type=write size=2705794 2024-06-28T15:08:17.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.816+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=OUAWPMYDY86V type=write size=24524 2024-06-28T15:08:17.805 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.816+0000 7f65680598c0 1 < canceled operation key=KFHGA64L1NLP tag=9ZCEZ0JC96D7 type=del size=320852 2024-06-28T15:08:17.808 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.820+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=2GVMGJPCBOU4 type=del size=662637 2024-06-28T15:08:17.811 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.824+0000 7f65680598c0 1 < canceled operation key=9MQKWSA33R5G tag=474E1PDA3XOV type=write size=50838 2024-06-28T15:08:17.814 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.828+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=3T30UGYZVAHZ type=del size=3330212 2024-06-28T15:08:17.817 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.828+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=8G6NMTNIMOQ8 type=del 2024-06-28T15:08:17.823 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.836+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~21QGMQIO73L7.0 size=1048576 2024-06-28T15:08:17.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.840+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~21QGMQIO73L7.1 size=1048576 2024-06-28T15:08:17.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.840+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~21QGMQIO73L7.2 size=913001 2024-06-28T15:08:17.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.840+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=21QGMQIO73L7 tag=MPMYJHR81I8M type=write size=4058729 2024-06-28T15:08:17.830 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.844+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~Y905E1D7UJZ5.0 size=442897 2024-06-28T15:08:17.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.848+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=Y905E1D7UJZ5 tag=I9K88FF8CU8O type=write size=1491473 2024-06-28T15:08:17.840 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.852+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~D3AFJ0EX52G8.0 size=1048576 2024-06-28T15:08:17.842 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.856+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~D3AFJ0EX52G8.1 size=1048576 2024-06-28T15:08:17.844 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.856+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~D3AFJ0EX52G8.2 size=736329 2024-06-28T15:08:17.844 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.856+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=D3AFJ0EX52G8 tag=5DMA4N8MV328 type=write size=3882057 2024-06-28T15:08:17.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.860+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=25BZLPKK6S0G type=del 2024-06-28T15:08:17.850 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.864+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=UW560K1JRRPW type=write size=572733 2024-06-28T15:08:17.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.868+0000 7f65680598c0 1 < completed write operation key=AXQ0YFWUH4FD tag=JD8G31SLWURF type=write size=992121 2024-06-28T15:08:17.856 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.868+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=P77NFMOK3A7M type=del size=3191714 2024-06-28T15:08:17.859 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.872+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=50ACWL222UW3 type=del 2024-06-28T15:08:17.865 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.876+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~Y794DV2FOAKV.0 size=253466 2024-06-28T15:08:17.865 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.876+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=Y794DV2FOAKV tag=LT0J430NR2K9 type=write size=1302042 2024-06-28T15:08:17.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.884+0000 7f65680598c0 1 < suggested remove operation key=I4RWCQSQVLCN tag=24BM96JBF7I3 type=write 2024-06-28T15:08:17.874 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.888+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=RDPOYU33XS4M type=del size=675141 2024-06-28T15:08:17.874 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.888+0000 7f65680598c0 1 < canceled part key=_multipart_60QVJVV16QFM.2~JL6YFLQM5RZ1.0 size=411664 2024-06-28T15:08:17.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.892+0000 7f65680598c0 1 < canceled multipart upload key=60QVJVV16QFM upload=JL6YFLQM5RZ1 tag=GMOV76HPWC1H type=write size=1460240 2024-06-28T15:08:17.886 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.900+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~EKOR5NHICN3U.0 size=653149 2024-06-28T15:08:17.886 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.900+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=EKOR5NHICN3U tag=R110O33VANP7 type=write size=1701725 2024-06-28T15:08:17.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.904+0000 7f65680598c0 -1 < failed to complete operation key=I4RWCQSQVLCN tag=T5YY0RMMKMY4 type=del size=3145062: (22) Invalid argument 2024-06-28T15:08:17.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.904+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=RRNPU1W36ZIX type=del size=1868539 2024-06-28T15:08:17.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.908+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=QIQUNTZ16HX9 type=del 2024-06-28T15:08:17.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.912+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=8X6FSILBE4OY type=del size=2419004 2024-06-28T15:08:17.905 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.916+0000 7f65680598c0 1 < suggested remove operation key=KFHGA64L1NLP tag=OC8Z72N3JGA6 type=write 2024-06-28T15:08:17.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.924+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~KRJMKL7F3492.0 size=1048576 2024-06-28T15:08:17.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.924+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~KRJMKL7F3492.1 size=1048576 2024-06-28T15:08:17.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.928+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~KRJMKL7F3492.2 size=979167 2024-06-28T15:08:17.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.928+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=KRJMKL7F3492 tag=RPEGONWDWZII type=write size=4124895 2024-06-28T15:08:17.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.932+0000 7f65680598c0 1 < completed write operation key=9MQKWSA33R5G tag=CI0LO39ONKFI type=write size=741852 2024-06-28T15:08:17.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.936+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~U05FV6L535XS.0 size=627202 2024-06-28T15:08:17.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.936+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=U05FV6L535XS tag=J7I1PTLLKFEQ type=write size=1675778 2024-06-28T15:08:17.925 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.936+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=B5QG8JWXHYFO type=del 2024-06-28T15:08:17.931 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.944+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~TSZUZXY9Y52T.0 size=1048576 2024-06-28T15:08:17.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.948+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~TSZUZXY9Y52T.1 size=742833 2024-06-28T15:08:17.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.948+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=TSZUZXY9Y52T tag=57AB239FOTJR type=write size=2839985 2024-06-28T15:08:17.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.952+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=R1YERFD14STB type=del 2024-06-28T15:08:17.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.952+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=WIDNVLUU1KM9 type=del size=2415297 2024-06-28T15:08:17.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.956+0000 7f65680598c0 1 < completed write operation key=47TO7G9EGJKF tag=OUAWPMYDY86V type=write size=24524 2024-06-28T15:08:17.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.960+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=HUYVU8SFXIBW type=del size=279776 2024-06-28T15:08:17.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.964+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=2GVMGJPCBOU4 type=del 2024-06-28T15:08:17.953 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.964+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=Y1NKG8J9762C type=del size=3562041 2024-06-28T15:08:17.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.968+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=3T30UGYZVAHZ type=del 2024-06-28T15:08:17.958 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.972+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=Y6K7IV6F032B type=del size=99888 2024-06-28T15:08:17.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.976+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~21QGMQIO73L7.0 size=1048576 2024-06-28T15:08:17.963 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.976+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~21QGMQIO73L7.1 size=1048576 2024-06-28T15:08:17.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.980+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~21QGMQIO73L7.2 size=913001 2024-06-28T15:08:17.970 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.984+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=21QGMQIO73L7 tag=MPMYJHR81I8M type=write size=4058729 2024-06-28T15:08:17.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.988+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~XW5EGESOEKTT.0 size=1048576 2024-06-28T15:08:17.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.992+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~XW5EGESOEKTT.1 size=1048576 2024-06-28T15:08:17.981 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.992+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~XW5EGESOEKTT.2 size=752609 2024-06-28T15:08:17.981 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.992+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=XW5EGESOEKTT tag=VPT8PLUVQALT type=write size=3898337 2024-06-28T15:08:17.984 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:17.996+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~D3AFJ0EX52G8.0 size=1048576 2024-06-28T15:08:17.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.000+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~D3AFJ0EX52G8.1 size=1048576 2024-06-28T15:08:17.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.000+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~D3AFJ0EX52G8.2 size=736329 2024-06-28T15:08:17.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.008+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=D3AFJ0EX52G8 tag=5DMA4N8MV328 type=write size=3882057 2024-06-28T15:08:18.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.012+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~XGQB4Z34TLIE.0 size=1048576 2024-06-28T15:08:18.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.016+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~XGQB4Z34TLIE.1 size=296115 2024-06-28T15:08:18.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.016+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=XGQB4Z34TLIE tag=Z6Y83H8BUCWH type=write size=2393267 2024-06-28T15:08:18.005 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.016+0000 7f65680598c0 1 < completed write operation key=7NL3CTFYY9E5 tag=UW560K1JRRPW type=write size=572733 2024-06-28T15:08:18.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.020+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=E34A0NA3CXW9 type=write size=581982 2024-06-28T15:08:18.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.024+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=P77NFMOK3A7M type=del 2024-06-28T15:08:18.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.024+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=2WQ7X5NTZPVU type=del size=3999453 2024-06-28T15:08:18.016 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.028+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~Y794DV2FOAKV.0 size=253466 2024-06-28T15:08:18.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.032+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=Y794DV2FOAKV tag=LT0J430NR2K9 type=write size=1302042 2024-06-28T15:08:18.023 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.036+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=6VIPOCYG6CDP type=write size=435610 2024-06-28T15:08:18.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.040+0000 7f65680598c0 1 < suggested update operation key=O1CGONN4TEXF tag=RDPOYU33XS4M type=del size=675141 2024-06-28T15:08:18.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.044+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=JXWW3IZJ88I1 type=del size=33614 2024-06-28T15:08:18.034 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.048+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~EKOR5NHICN3U.0 size=653149 2024-06-28T15:08:18.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.056+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=EKOR5NHICN3U tag=R110O33VANP7 type=write size=1701725 2024-06-28T15:08:18.050 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.064+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~WWY7BI10BRIZ.0 size=237848 2024-06-28T15:08:18.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.064+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=WWY7BI10BRIZ tag=YZ7VM0ZOSYNN type=write size=1286424 2024-06-28T15:08:18.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.068+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=RRNPU1W36ZIX type=del 2024-06-28T15:08:18.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.068+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=RK56DJWGLQ5E type=del size=2117466 2024-06-28T15:08:18.061 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.072+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=8X6FSILBE4OY type=del 2024-06-28T15:08:18.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.076+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=03KP56F6BBHX type=del size=609915 2024-06-28T15:08:18.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.080+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~KRJMKL7F3492.0 size=1048576 2024-06-28T15:08:18.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.084+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~KRJMKL7F3492.1 size=1048576 2024-06-28T15:08:18.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.084+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~KRJMKL7F3492.2 size=979167 2024-06-28T15:08:18.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.092+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=KRJMKL7F3492 tag=RPEGONWDWZII type=write size=4124895 2024-06-28T15:08:18.080 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.092+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=KMZMSSTJT68S type=del size=1851201 2024-06-28T15:08:18.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.096+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~U05FV6L535XS.0 size=627202 2024-06-28T15:08:18.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.100+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=U05FV6L535XS tag=J7I1PTLLKFEQ type=write size=1675778 2024-06-28T15:08:18.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.104+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=OQKKBAI2UCW7 type=del size=2915777 2024-06-28T15:08:18.096 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.108+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~TSZUZXY9Y52T.0 size=1048576 2024-06-28T15:08:18.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.112+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~TSZUZXY9Y52T.1 size=742833 2024-06-28T15:08:18.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.116+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=TSZUZXY9Y52T tag=57AB239FOTJR type=write size=2839985 2024-06-28T15:08:18.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.120+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=GWXOSSCHNSDL type=del size=2373297 2024-06-28T15:08:18.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.124+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=WIDNVLUU1KM9 type=del 2024-06-28T15:08:18.114 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.128+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=HAN016CPA1Z2 type=del size=2917673 2024-06-28T15:08:18.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.132+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=HUYVU8SFXIBW type=del 2024-06-28T15:08:18.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.136+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~08JPNCSOY78O.0 size=510995 2024-06-28T15:08:18.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.136+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=08JPNCSOY78O tag=CU6TASRZ98MI type=write size=1559571 2024-06-28T15:08:18.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.144+0000 7f65680598c0 1 < suggested update operation key=C5C6SAHLPZV1 tag=Y1NKG8J9762C type=del size=3562041 2024-06-28T15:08:18.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.144+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=L14BXSJBJ4EG type=del size=1456422 2024-06-28T15:08:18.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.148+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=Y6K7IV6F032B type=del 2024-06-28T15:08:18.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.156+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~H3543QEE6JZ9.0 size=1048576 2024-06-28T15:08:18.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.156+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~H3543QEE6JZ9.1 size=1048576 2024-06-28T15:08:18.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.160+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~H3543QEE6JZ9.2 size=178904 2024-06-28T15:08:18.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.160+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=H3543QEE6JZ9 tag=O0AC638HCLRY type=write size=3324632 2024-06-28T15:08:18.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.164+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~XW5EGESOEKTT.0 size=1048576 2024-06-28T15:08:18.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.168+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~XW5EGESOEKTT.1 size=1048576 2024-06-28T15:08:18.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.168+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~XW5EGESOEKTT.2 size=752609 2024-06-28T15:08:18.162 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.176+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=XW5EGESOEKTT tag=VPT8PLUVQALT type=write size=3898337 2024-06-28T15:08:18.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.180+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=2XRNVTBPT2Z9 type=del size=848661 2024-06-28T15:08:18.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.180+0000 7f65680598c0 1 < canceled part key=_multipart_UP3XFF38IJXN.2~XGQB4Z34TLIE.0 size=1048576 2024-06-28T15:08:18.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.184+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~XGQB4Z34TLIE.1 size=296115 2024-06-28T15:08:18.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.188+0000 7f65680598c0 1 < canceled multipart upload key=UP3XFF38IJXN upload=XGQB4Z34TLIE tag=Z6Y83H8BUCWH type=write size=2393267 2024-06-28T15:08:18.178 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.192+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=16CT8TIEX9NZ type=del size=1618298 2024-06-28T15:08:18.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.196+0000 7f65680598c0 1 < canceled operation key=AXQ0YFWUH4FD tag=E34A0NA3CXW9 type=write size=581982 2024-06-28T15:08:18.185 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.196+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=HRDT40ZF6JUD type=del size=2306948 2024-06-28T15:08:18.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.204+0000 7f65680598c0 1 < suggested update operation key=AOBFAR0UTONH tag=2WQ7X5NTZPVU type=del size=3999453 2024-06-28T15:08:18.195 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.208+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=68UTC3WZOOFP type=del size=616696 2024-06-28T15:08:18.198 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.212+0000 7f65680598c0 1 < completed write operation key=60QVJVV16QFM tag=6VIPOCYG6CDP type=write size=435610 2024-06-28T15:08:18.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.216+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=B4HO69WF4TP9 type=del size=706085 2024-06-28T15:08:18.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.216+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=JXWW3IZJ88I1 type=del 2024-06-28T15:08:18.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.220+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=26YA9MCRX7QR type=del size=217735 2024-06-28T15:08:18.211 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.224+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~WWY7BI10BRIZ.0 size=237848 2024-06-28T15:08:18.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.228+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=WWY7BI10BRIZ tag=YZ7VM0ZOSYNN type=write size=1286424 2024-06-28T15:08:18.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.236+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~UM88U203UDLP.0 size=1048576 2024-06-28T15:08:18.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.236+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~UM88U203UDLP.1 size=343287 2024-06-28T15:08:18.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.236+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=UM88U203UDLP tag=ML6FM5QNV5V1 type=write size=2440439 2024-06-28T15:08:18.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.240+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=RK56DJWGLQ5E type=del 2024-06-28T15:08:18.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.244+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=YYSTE2HIXK02 type=write size=337761 2024-06-28T15:08:18.235 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.248+0000 7f65680598c0 -1 < failed to complete operation key=C5C6SAHLPZV1 tag=03KP56F6BBHX type=del size=609915: (22) Invalid argument 2024-06-28T15:08:18.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.252+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~G3FVA3PUZ7PR.0 size=1048576 2024-06-28T15:08:18.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.256+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~G3FVA3PUZ7PR.1 size=760260 2024-06-28T15:08:18.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.256+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=G3FVA3PUZ7PR tag=U7BYTCYYL8Q5 type=write size=2857412 2024-06-28T15:08:18.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.260+0000 7f65680598c0 1 < completed delete operation key=1VD7NDPEM3CD tag=KMZMSSTJT68S type=del 2024-06-28T15:08:18.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.260+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=6QPGULWOJZO2 type=del size=2242963 2024-06-28T15:08:18.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.268+0000 7f65680598c0 1 < suggested remove operation key=GAZQ3PCPKN9Q tag=OQKKBAI2UCW7 type=del 2024-06-28T15:08:18.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.276+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~DWA8MB65Q17F.0 size=1048576 2024-06-28T15:08:18.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.276+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~DWA8MB65Q17F.1 size=792811 2024-06-28T15:08:18.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.276+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=DWA8MB65Q17F tag=EXVT1NCIE6T2 type=write size=2889963 2024-06-28T15:08:18.268 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.280+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=GWXOSSCHNSDL type=del 2024-06-28T15:08:18.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.288+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~MOQALUJ02DZX.0 size=1048576 2024-06-28T15:08:18.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.288+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~MOQALUJ02DZX.1 size=1003593 2024-06-28T15:08:18.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.288+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=MOQALUJ02DZX tag=X0CQSNT5B2T5 type=write size=3100745 2024-06-28T15:08:18.280 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.292+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=HAN016CPA1Z2 type=del 2024-06-28T15:08:18.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.300+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~5P0V1CJDJGXJ.0 size=855326 2024-06-28T15:08:18.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.300+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=5P0V1CJDJGXJ tag=BEF7BTJ7DXK8 type=write size=1903902 2024-06-28T15:08:18.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.304+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~08JPNCSOY78O.0 size=510995 2024-06-28T15:08:18.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.308+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=08JPNCSOY78O tag=CU6TASRZ98MI type=write size=1559571 2024-06-28T15:08:18.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.312+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=L0LKGQEQI9TS type=write size=1023940 2024-06-28T15:08:18.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.316+0000 7f65680598c0 1 < canceled operation key=9MQKWSA33R5G tag=L14BXSJBJ4EG type=del size=1456422 2024-06-28T15:08:18.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.320+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~7YB3KYSKA6ZI.0 size=1048576 2024-06-28T15:08:18.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.324+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~7YB3KYSKA6ZI.1 size=1048576 2024-06-28T15:08:18.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.324+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~7YB3KYSKA6ZI.2 size=281723 2024-06-28T15:08:18.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.324+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=7YB3KYSKA6ZI tag=ZLPXNJUX8MK7 type=write size=3427451 2024-06-28T15:08:18.316 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.328+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~H3543QEE6JZ9.0 size=1048576 2024-06-28T15:08:18.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.332+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~H3543QEE6JZ9.1 size=1048576 2024-06-28T15:08:18.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.336+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~H3543QEE6JZ9.2 size=178904 2024-06-28T15:08:18.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.340+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=H3543QEE6JZ9 tag=O0AC638HCLRY type=write size=3324632 2024-06-28T15:08:18.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.344+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=LYRXM914CY7G type=del size=1223136 2024-06-28T15:08:18.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.348+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=2XRNVTBPT2Z9 type=del 2024-06-28T15:08:18.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.348+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=70ZWPBSA1CMN type=del size=1087862 2024-06-28T15:08:18.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.352+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=16CT8TIEX9NZ type=del 2024-06-28T15:08:18.344 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.356+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=E69CN8JFB1KY type=del size=894493 2024-06-28T15:08:18.347 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.360+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=HRDT40ZF6JUD type=del 2024-06-28T15:08:18.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.372+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~SY682PORQRJX.0 size=1048576 2024-06-28T15:08:18.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.372+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~SY682PORQRJX.1 size=1048576 2024-06-28T15:08:18.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.376+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~SY682PORQRJX.2 size=786368 2024-06-28T15:08:18.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.376+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=SY682PORQRJX tag=YVC0412HYER5 type=write size=3932096 2024-06-28T15:08:18.365 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.376+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=68UTC3WZOOFP type=del 2024-06-28T15:08:18.368 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.380+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=KWF9I4C5PX6T type=del size=2501051 2024-06-28T15:08:18.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.384+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=B4HO69WF4TP9 type=del 2024-06-28T15:08:18.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.388+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~3OT5N0B94OK4.0 size=1048576 2024-06-28T15:08:18.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.392+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~3OT5N0B94OK4.1 size=1048576 2024-06-28T15:08:18.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.396+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~3OT5N0B94OK4.2 size=134052 2024-06-28T15:08:18.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.396+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=3OT5N0B94OK4 tag=YS6E1M52MDAP type=write size=3279780 2024-06-28T15:08:18.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.396+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=26YA9MCRX7QR type=del 2024-06-28T15:08:18.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.404+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~TSQNY1YXSJOK.0 size=1048576 2024-06-28T15:08:18.393 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.404+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~TSQNY1YXSJOK.1 size=1048576 2024-06-28T15:08:18.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.408+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~TSQNY1YXSJOK.2 size=943362 2024-06-28T15:08:18.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.408+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=TSQNY1YXSJOK tag=Y8RFATP8RUAW type=write size=4089090 2024-06-28T15:08:18.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.412+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~UM88U203UDLP.0 size=1048576 2024-06-28T15:08:18.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.412+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~UM88U203UDLP.1 size=343287 2024-06-28T15:08:18.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.420+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=UM88U203UDLP tag=ML6FM5QNV5V1 type=write size=2440439 2024-06-28T15:08:18.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.420+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=CU3P5T1R49I9 type=del size=40792 2024-06-28T15:08:18.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.428+0000 7f65680598c0 1 < suggested update operation key=D9D3OFIJP3D3 tag=YYSTE2HIXK02 type=write size=337761 2024-06-28T15:08:18.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.432+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=QXK3FFLWJG9O type=write size=653083 2024-06-28T15:08:18.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.432+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~G3FVA3PUZ7PR.0 size=1048576 2024-06-28T15:08:18.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.436+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~G3FVA3PUZ7PR.1 size=760260 2024-06-28T15:08:18.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.440+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=G3FVA3PUZ7PR tag=U7BYTCYYL8Q5 type=write size=2857412 2024-06-28T15:08:18.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.444+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=4A4ZR26G5P6T type=del size=3476549 2024-06-28T15:08:18.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.448+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=6QPGULWOJZO2 type=del 2024-06-28T15:08:18.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.456+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~WWMA67KDA2SY.0 size=1048576 2024-06-28T15:08:18.444 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.456+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~WWMA67KDA2SY.1 size=1048576 2024-06-28T15:08:18.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.460+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~WWMA67KDA2SY.2 size=135289 2024-06-28T15:08:18.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.460+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=WWMA67KDA2SY tag=9W7S6JDJI9QF type=write size=3281017 2024-06-28T15:08:18.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.464+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~DWA8MB65Q17F.0 size=1048576 2024-06-28T15:08:18.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.464+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~DWA8MB65Q17F.1 size=792811 2024-06-28T15:08:18.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.472+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=DWA8MB65Q17F tag=EXVT1NCIE6T2 type=write size=2889963 2024-06-28T15:08:18.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.472+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=9BEVV8I50ZQM type=del size=1499156 2024-06-28T15:08:18.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.476+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~MOQALUJ02DZX.0 size=1048576 2024-06-28T15:08:18.467 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.480+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~MOQALUJ02DZX.1 size=1003593 2024-06-28T15:08:18.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.484+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=MOQALUJ02DZX tag=X0CQSNT5B2T5 type=write size=3100745 2024-06-28T15:08:18.478 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.492+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~C8Q3Q381E6KZ.0 size=1048576 2024-06-28T15:08:18.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.492+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~C8Q3Q381E6KZ.1 size=1048576 2024-06-28T15:08:18.483 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.496+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~C8Q3Q381E6KZ.2 size=886739 2024-06-28T15:08:18.483 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.496+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=C8Q3Q381E6KZ tag=H9KSD0AANX93 type=write size=4032467 2024-06-28T15:08:18.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.500+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~5P0V1CJDJGXJ.0 size=855326 2024-06-28T15:08:18.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.504+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=5P0V1CJDJGXJ tag=BEF7BTJ7DXK8 type=write size=1903902 2024-06-28T15:08:18.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.508+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~TBKI1ROCLIMR.0 size=700392 2024-06-28T15:08:18.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.508+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=TBKI1ROCLIMR tag=FNSARE1H9NZZ type=write size=1748968 2024-06-28T15:08:18.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.512+0000 7f65680598c0 1 < completed write operation key=7LB8MOJL39RM tag=L0LKGQEQI9TS type=write size=1023940 2024-06-28T15:08:18.506 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.520+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~2X2KLCDJJPLS.0 size=1048576 2024-06-28T15:08:18.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.520+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~2X2KLCDJJPLS.1 size=1048576 2024-06-28T15:08:18.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.524+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~2X2KLCDJJPLS.2 size=465161 2024-06-28T15:08:18.511 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.524+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=2X2KLCDJJPLS tag=DKNXYN8BK7RO type=write size=3610889 2024-06-28T15:08:18.514 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.528+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~7YB3KYSKA6ZI.0 size=1048576 2024-06-28T15:08:18.516 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.528+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~7YB3KYSKA6ZI.1 size=1048576 2024-06-28T15:08:18.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.532+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~7YB3KYSKA6ZI.2 size=281723 2024-06-28T15:08:18.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.536+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=7YB3KYSKA6ZI tag=ZLPXNJUX8MK7 type=write size=3427451 2024-06-28T15:08:18.528 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.540+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=LZKAH16S0PTK type=del size=2223500 2024-06-28T15:08:18.531 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.544+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=LYRXM914CY7G type=del 2024-06-28T15:08:18.537 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.548+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~5J3V4HGT6T3M.0 size=516410 2024-06-28T15:08:18.537 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.548+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=5J3V4HGT6T3M tag=CD3M78ZNQ13Q type=write size=1564986 2024-06-28T15:08:18.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.552+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=70ZWPBSA1CMN type=del 2024-06-28T15:08:18.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.556+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=3OU1JKVOGMWL type=write size=791609 2024-06-28T15:08:18.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.560+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=E69CN8JFB1KY type=del 2024-06-28T15:08:18.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.564+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=UO613DRPJY67 type=del size=1131285 2024-06-28T15:08:18.553 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.568+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~SY682PORQRJX.0 size=1048576 2024-06-28T15:08:18.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.568+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~SY682PORQRJX.1 size=1048576 2024-06-28T15:08:18.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.572+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~SY682PORQRJX.2 size=786368 2024-06-28T15:08:18.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.576+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=SY682PORQRJX tag=YVC0412HYER5 type=write size=3932096 2024-06-28T15:08:18.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.580+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=YW27PP9P64JR type=del size=2197950 2024-06-28T15:08:18.575 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.588+0000 7f65680598c0 1 < suggested remove operation key=2FC9DNEZGWC4 tag=KWF9I4C5PX6T type=del 2024-06-28T15:08:18.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.592+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~MHFMU7737ECM.0 size=1048576 2024-06-28T15:08:18.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.596+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~MHFMU7737ECM.1 size=1048576 2024-06-28T15:08:18.587 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.600+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~MHFMU7737ECM.2 size=205989 2024-06-28T15:08:18.587 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.600+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=MHFMU7737ECM tag=MZSY73XXT3FW type=write size=3351717 2024-06-28T15:08:18.590 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.604+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~3OT5N0B94OK4.0 size=1048576 2024-06-28T15:08:18.592 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.604+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~3OT5N0B94OK4.1 size=1048576 2024-06-28T15:08:18.595 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.608+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~3OT5N0B94OK4.2 size=134052 2024-06-28T15:08:18.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.612+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=3OT5N0B94OK4 tag=YS6E1M52MDAP type=write size=3279780 2024-06-28T15:08:18.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.616+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=I2SER3OCMO47 type=del size=344731 2024-06-28T15:08:18.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.620+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~TSQNY1YXSJOK.0 size=1048576 2024-06-28T15:08:18.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.624+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~TSQNY1YXSJOK.1 size=1048576 2024-06-28T15:08:18.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.628+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~TSQNY1YXSJOK.2 size=943362 2024-06-28T15:08:18.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.632+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=TSQNY1YXSJOK tag=Y8RFATP8RUAW type=write size=4089090 2024-06-28T15:08:18.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.640+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~XY956PH0MHCQ.0 size=636196 2024-06-28T15:08:18.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.640+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=XY956PH0MHCQ tag=DJF2588A6DI4 type=write size=1684772 2024-06-28T15:08:18.629 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.640+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=CU3P5T1R49I9 type=del 2024-06-28T15:08:18.635 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.648+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~M2P3013CB9BE.0 size=239998 2024-06-28T15:08:18.635 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.648+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=M2P3013CB9BE tag=WISJXEXE16XB type=write size=1288574 2024-06-28T15:08:18.638 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.652+0000 7f65680598c0 1 < completed write operation key=GAZQ3PCPKN9Q tag=QXK3FFLWJG9O type=write size=653083 2024-06-28T15:08:18.642 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.656+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=83XUWRWZ9O8S type=del size=276807 2024-06-28T15:08:18.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.660+0000 7f65680598c0 1 < canceled operation key=OUID7RX5NVV6 tag=4A4ZR26G5P6T type=del size=3476549 2024-06-28T15:08:18.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.664+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=WAFY49CZKGOY type=write size=994844 2024-06-28T15:08:18.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.668+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~WWMA67KDA2SY.0 size=1048576 2024-06-28T15:08:18.658 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.672+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~WWMA67KDA2SY.1 size=1048576 2024-06-28T15:08:18.661 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.672+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~WWMA67KDA2SY.2 size=135289 2024-06-28T15:08:18.667 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.680+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=WWMA67KDA2SY tag=9W7S6JDJI9QF type=write size=3281017 2024-06-28T15:08:18.674 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.688+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~F8V468K1GJCR.0 size=1048576 2024-06-28T15:08:18.676 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.688+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~F8V468K1GJCR.1 size=1048576 2024-06-28T15:08:18.678 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.692+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~F8V468K1GJCR.2 size=482184 2024-06-28T15:08:18.679 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.692+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=F8V468K1GJCR tag=X2L1ADW2IDFW type=write size=3627912 2024-06-28T15:08:18.682 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.696+0000 7f65680598c0 1 < canceled operation key=SFCIZMLY2Y2E tag=9BEVV8I50ZQM type=del size=1499156 2024-06-28T15:08:18.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.700+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~XPDO2TYHQDCN.0 size=1048576 2024-06-28T15:08:18.690 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.704+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~XPDO2TYHQDCN.1 size=389456 2024-06-28T15:08:18.690 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.704+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=XPDO2TYHQDCN tag=OFX5M5KV2WMN type=write size=2486608 2024-06-28T15:08:18.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.708+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~C8Q3Q381E6KZ.0 size=1048576 2024-06-28T15:08:18.696 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.708+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~C8Q3Q381E6KZ.1 size=1048576 2024-06-28T15:08:18.699 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.712+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~C8Q3Q381E6KZ.2 size=886739 2024-06-28T15:08:18.704 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.716+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=C8Q3Q381E6KZ tag=H9KSD0AANX93 type=write size=4032467 2024-06-28T15:08:18.710 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.724+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~UUP1PF7YYD3M.0 size=1048576 2024-06-28T15:08:18.712 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.724+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~UUP1PF7YYD3M.1 size=1048576 2024-06-28T15:08:18.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.728+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~UUP1PF7YYD3M.2 size=649434 2024-06-28T15:08:18.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.728+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=UUP1PF7YYD3M tag=1BJ6HA6MID0D type=write size=3795162 2024-06-28T15:08:18.718 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.732+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~TBKI1ROCLIMR.0 size=700392 2024-06-28T15:08:18.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.736+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=TBKI1ROCLIMR tag=FNSARE1H9NZZ type=write size=1748968 2024-06-28T15:08:18.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.740+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~5PM9GGO9LBIA.0 size=1048576 2024-06-28T15:08:18.731 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.744+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~5PM9GGO9LBIA.1 size=1048576 2024-06-28T15:08:18.733 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.744+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~5PM9GGO9LBIA.2 size=408019 2024-06-28T15:08:18.733 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.744+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=5PM9GGO9LBIA tag=YF8V5SVQHHX8 type=write size=3553747 2024-06-28T15:08:18.736 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.748+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~2X2KLCDJJPLS.0 size=1048576 2024-06-28T15:08:18.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.752+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~2X2KLCDJJPLS.1 size=1048576 2024-06-28T15:08:18.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.756+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~2X2KLCDJJPLS.2 size=465161 2024-06-28T15:08:18.747 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.760+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=2X2KLCDJJPLS tag=DKNXYN8BK7RO type=write size=3610889 2024-06-28T15:08:18.750 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.764+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=I0Z09WL7AMTK type=del size=2374090 2024-06-28T15:08:18.753 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.768+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=LZKAH16S0PTK type=del 2024-06-28T15:08:18.756 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.768+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=L4HD2NB1PDBP type=del size=4160884 2024-06-28T15:08:18.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.772+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~5J3V4HGT6T3M.0 size=516410 2024-06-28T15:08:18.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.776+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=5J3V4HGT6T3M tag=CD3M78ZNQ13Q type=write size=1564986 2024-06-28T15:08:18.770 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.784+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~0WF3V9TNZPIG.0 size=164617 2024-06-28T15:08:18.770 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.784+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=0WF3V9TNZPIG tag=5L550MDTHX9B type=write size=1213193 2024-06-28T15:08:18.773 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.788+0000 7f65680598c0 1 < completed write operation key=1VD7NDPEM3CD tag=3OU1JKVOGMWL type=write size=791609 2024-06-28T15:08:18.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.788+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=Y4MKV9PO5X79 type=del size=2148553 2024-06-28T15:08:18.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.792+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=UO613DRPJY67 type=del 2024-06-28T15:08:18.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.796+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=1F87EGC2P8MP type=del size=2759378 2024-06-28T15:08:18.787 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.800+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=YW27PP9P64JR type=del 2024-06-28T15:08:18.794 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.808+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~9CDEIX4WPHRA.0 size=1048576 2024-06-28T15:08:18.797 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.808+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~9CDEIX4WPHRA.1 size=1048576 2024-06-28T15:08:18.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.812+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~9CDEIX4WPHRA.2 size=462073 2024-06-28T15:08:18.800 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.812+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=9CDEIX4WPHRA tag=A808JY26RZZL type=write size=3607801 2024-06-28T15:08:18.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.816+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~MHFMU7737ECM.0 size=1048576 2024-06-28T15:08:18.805 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.820+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~MHFMU7737ECM.1 size=1048576 2024-06-28T15:08:18.808 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.820+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~MHFMU7737ECM.2 size=205989 2024-06-28T15:08:18.814 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.828+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=MHFMU7737ECM tag=MZSY73XXT3FW type=write size=3351717 2024-06-28T15:08:18.818 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.828+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=GPB17O7H0B37 type=del size=543589 2024-06-28T15:08:18.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.832+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=I2SER3OCMO47 type=del 2024-06-28T15:08:18.824 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.836+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=YE931NY0RMP5 type=write size=1007343 2024-06-28T15:08:18.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.840+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~XY956PH0MHCQ.0 size=636196 2024-06-28T15:08:18.833 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.844+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=XY956PH0MHCQ tag=DJF2588A6DI4 type=write size=1684772 2024-06-28T15:08:18.836 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.848+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=HA9RGD3Y0W1I type=del size=3572059 2024-06-28T15:08:18.839 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.852+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~M2P3013CB9BE.0 size=239998 2024-06-28T15:08:18.845 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.856+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=M2P3013CB9BE tag=WISJXEXE16XB type=write size=1288574 2024-06-28T15:08:18.851 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.864+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~QHBMYGJ8KIHO.0 size=1048576 2024-06-28T15:08:18.854 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.868+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~QHBMYGJ8KIHO.1 size=1048576 2024-06-28T15:08:18.857 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.868+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~QHBMYGJ8KIHO.2 size=446055 2024-06-28T15:08:18.857 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.868+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=QHBMYGJ8KIHO tag=V52UTI471XWH type=write size=3591783 2024-06-28T15:08:18.861 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.872+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=83XUWRWZ9O8S type=del 2024-06-28T15:08:18.866 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.880+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~GDESHTQ46U1F.0 size=1048576 2024-06-28T15:08:18.869 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.880+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~GDESHTQ46U1F.1 size=1048576 2024-06-28T15:08:18.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.884+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~GDESHTQ46U1F.2 size=202422 2024-06-28T15:08:18.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.884+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=GDESHTQ46U1F tag=VX35U84GEKSK type=write size=3348150 2024-06-28T15:08:18.878 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.892+0000 7f65680598c0 1 < suggested update operation key=I4RWCQSQVLCN tag=WAFY49CZKGOY type=write size=994844 2024-06-28T15:08:18.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.896+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=SD834YE47VNK type=write size=892283 2024-06-28T15:08:18.885 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.896+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~F8V468K1GJCR.0 size=1048576 2024-06-28T15:08:18.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.900+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~F8V468K1GJCR.1 size=1048576 2024-06-28T15:08:18.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.904+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~F8V468K1GJCR.2 size=482184 2024-06-28T15:08:18.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.908+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=F8V468K1GJCR tag=X2L1ADW2IDFW type=write size=3627912 2024-06-28T15:08:18.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.912+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~2D7AK07LUBVJ.0 size=1048576 2024-06-28T15:08:18.903 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.916+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~2D7AK07LUBVJ.1 size=834072 2024-06-28T15:08:18.903 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.916+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=2D7AK07LUBVJ tag=96JPDDM6JJUO type=write size=2931224 2024-06-28T15:08:18.906 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.920+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~XPDO2TYHQDCN.0 size=1048576 2024-06-28T15:08:18.906 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.920+0000 7f65680598c0 1 < canceled part key=_multipart_AOBFAR0UTONH.2~XPDO2TYHQDCN.1 size=389456 2024-06-28T15:08:18.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.924+0000 7f65680598c0 1 < canceled multipart upload key=AOBFAR0UTONH upload=XPDO2TYHQDCN tag=OFX5M5KV2WMN type=write size=2486608 2024-06-28T15:08:18.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.928+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=5S4AXG81MWXV type=del size=952176 2024-06-28T15:08:18.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.928+0000 7f65680598c0 1 < canceled part key=_multipart_2FC9DNEZGWC4.2~UUP1PF7YYD3M.0 size=1048576 2024-06-28T15:08:18.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.932+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~UUP1PF7YYD3M.1 size=1048576 2024-06-28T15:08:18.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.936+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~UUP1PF7YYD3M.2 size=649434 2024-06-28T15:08:18.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.940+0000 7f65680598c0 1 < canceled multipart upload key=2FC9DNEZGWC4 upload=UUP1PF7YYD3M tag=1BJ6HA6MID0D type=write size=3795162 2024-06-28T15:08:18.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.948+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~OAA9ZI3VYMTY.0 size=1048576 2024-06-28T15:08:18.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.948+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~OAA9ZI3VYMTY.1 size=180371 2024-06-28T15:08:18.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.948+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=OAA9ZI3VYMTY tag=9ES6JBMG42G8 type=write size=2277523 2024-06-28T15:08:18.940 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.952+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~5PM9GGO9LBIA.0 size=1048576 2024-06-28T15:08:18.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.960+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~5PM9GGO9LBIA.1 size=1048576 2024-06-28T15:08:18.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.964+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~5PM9GGO9LBIA.2 size=408019 2024-06-28T15:08:18.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.968+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=5PM9GGO9LBIA tag=YF8V5SVQHHX8 type=write size=3553747 2024-06-28T15:08:18.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.972+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=7E0UJ1R0PHO3 type=write size=637135 2024-06-28T15:08:18.963 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.976+0000 7f65680598c0 1 < canceled operation key=RLJCNXKA2GYA tag=I0Z09WL7AMTK type=del size=2374090 2024-06-28T15:08:18.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.980+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=YI003WAYJXV3 type=del size=2454275 2024-06-28T15:08:18.970 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.984+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=L4HD2NB1PDBP type=del 2024-06-28T15:08:18.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.988+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~1JYEUM3H89J6.0 size=634216 2024-06-28T15:08:18.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.988+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=1JYEUM3H89J6 tag=ZFGPEPDAAE96 type=write size=1682792 2024-06-28T15:08:18.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.992+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~0WF3V9TNZPIG.0 size=164617 2024-06-28T15:08:18.984 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:18.996+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=0WF3V9TNZPIG tag=5L550MDTHX9B type=write size=1213193 2024-06-28T15:08:18.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.000+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=KBW7AL01KHZW type=del size=2958200 2024-06-28T15:08:18.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.004+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=Y4MKV9PO5X79 type=del 2024-06-28T15:08:18.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.008+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~BNAV4A8L3919.0 size=1048576 2024-06-28T15:08:18.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.008+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~BNAV4A8L3919.1 size=178418 2024-06-28T15:08:18.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.008+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=BNAV4A8L3919 tag=G9BKQ9UODUDC type=write size=2275570 2024-06-28T15:08:19.000 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.012+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=1F87EGC2P8MP type=del 2024-06-28T15:08:19.005 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.016+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~ST7Y3ZDGDEBV.0 size=1048576 2024-06-28T15:08:19.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.020+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~ST7Y3ZDGDEBV.1 size=898661 2024-06-28T15:08:19.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.020+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=ST7Y3ZDGDEBV tag=B8FHDP3L2MTN type=write size=2995813 2024-06-28T15:08:19.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.020+0000 7f65680598c0 1 < canceled part key=_multipart_V2L2Q1U62YW2.2~9CDEIX4WPHRA.0 size=1048576 2024-06-28T15:08:19.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.024+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~9CDEIX4WPHRA.1 size=1048576 2024-06-28T15:08:19.012 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.024+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~9CDEIX4WPHRA.2 size=462073 2024-06-28T15:08:19.016 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.028+0000 7f65680598c0 1 < canceled multipart upload key=V2L2Q1U62YW2 upload=9CDEIX4WPHRA tag=A808JY26RZZL type=write size=3607801 2024-06-28T15:08:19.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.032+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=ZQZ17F55X88F type=del size=997008 2024-06-28T15:08:19.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.032+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=GPB17O7H0B37 type=del 2024-06-28T15:08:19.024 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.036+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=0M8DFJRC83IN type=write size=473224 2024-06-28T15:08:19.026 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.040+0000 7f65680598c0 1 < completed write operation key=7NL3CTFYY9E5 tag=YE931NY0RMP5 type=write size=1007343 2024-06-28T15:08:19.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.040+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=5DOKNEOKXPR3 type=del size=3999953 2024-06-28T15:08:19.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.044+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=HA9RGD3Y0W1I type=del 2024-06-28T15:08:19.036 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.048+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~YWGZVYGYC4NX.0 size=1048576 2024-06-28T15:08:19.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.052+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~YWGZVYGYC4NX.1 size=121580 2024-06-28T15:08:19.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.052+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=YWGZVYGYC4NX tag=N0FF0IJBLLQ1 type=write size=2218732 2024-06-28T15:08:19.040 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.052+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~QHBMYGJ8KIHO.0 size=1048576 2024-06-28T15:08:19.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.056+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~QHBMYGJ8KIHO.1 size=1048576 2024-06-28T15:08:19.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.056+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~QHBMYGJ8KIHO.2 size=446055 2024-06-28T15:08:19.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.060+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=QHBMYGJ8KIHO tag=V52UTI471XWH type=write size=3591783 2024-06-28T15:08:19.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.064+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=LR5HR14S15HO type=del size=2729573 2024-06-28T15:08:19.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.068+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~GDESHTQ46U1F.0 size=1048576 2024-06-28T15:08:19.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.068+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~GDESHTQ46U1F.1 size=1048576 2024-06-28T15:08:19.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.072+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~GDESHTQ46U1F.2 size=202422 2024-06-28T15:08:19.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.076+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=GDESHTQ46U1F tag=VX35U84GEKSK type=write size=3348150 2024-06-28T15:08:19.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.080+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=D6HOISG62CF8 type=write size=1008786 2024-06-28T15:08:19.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.080+0000 7f65680598c0 1 < completed write operation key=R0IMFVF5NEOO tag=SD834YE47VNK type=write size=892283 2024-06-28T15:08:19.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.084+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~OIAD7Q1YU4R3.0 size=1048576 2024-06-28T15:08:19.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.088+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~OIAD7Q1YU4R3.1 size=397676 2024-06-28T15:08:19.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.088+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=OIAD7Q1YU4R3 tag=EEMGDOCU3KD3 type=write size=2494828 2024-06-28T15:08:19.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.092+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~2D7AK07LUBVJ.0 size=1048576 2024-06-28T15:08:19.079 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.092+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~2D7AK07LUBVJ.1 size=834072 2024-06-28T15:08:19.083 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.096+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=2D7AK07LUBVJ tag=96JPDDM6JJUO type=write size=2931224 2024-06-28T15:08:19.086 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.100+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=WKCYJEX4H96O type=del size=3376922 2024-06-28T15:08:19.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.104+0000 7f65680598c0 1 < suggested remove operation key=LZX5UZCYLTAG tag=5S4AXG81MWXV type=del 2024-06-28T15:08:19.093 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.104+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=LEJAGI67XQ8G type=del size=1996769 2024-06-28T15:08:19.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.108+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~OAA9ZI3VYMTY.0 size=1048576 2024-06-28T15:08:19.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.112+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~OAA9ZI3VYMTY.1 size=180371 2024-06-28T15:08:19.101 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.112+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=OAA9ZI3VYMTY tag=9ES6JBMG42G8 type=write size=2277523 2024-06-28T15:08:19.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.116+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=93I1D0Q0D0KV type=del size=352631 2024-06-28T15:08:19.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.120+0000 7f65680598c0 1 < suggested remove operation key=UP3XFF38IJXN tag=7E0UJ1R0PHO3 type=write 2024-06-28T15:08:19.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.124+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=29YCPT7PIMK4 type=write size=1011246 2024-06-28T15:08:19.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.128+0000 7f65680598c0 1 < suggested remove operation key=9MQKWSA33R5G tag=YI003WAYJXV3 type=del 2024-06-28T15:08:19.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.132+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=JHQIL4SQI7Z1 type=del size=3463787 2024-06-28T15:08:19.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.136+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~1JYEUM3H89J6.0 size=634216 2024-06-28T15:08:19.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.140+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=1JYEUM3H89J6 tag=ZFGPEPDAAE96 type=write size=1682792 2024-06-28T15:08:19.131 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.144+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~1ENB37E60044.0 size=1048576 2024-06-28T15:08:19.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.144+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~1ENB37E60044.1 size=1048576 2024-06-28T15:08:19.135 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.148+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~1ENB37E60044.2 size=733387 2024-06-28T15:08:19.135 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.148+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=1ENB37E60044 tag=MVQE1F3Y9148 type=write size=3879115 2024-06-28T15:08:19.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.152+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=KBW7AL01KHZW type=del 2024-06-28T15:08:19.140 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.152+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=G09YJK13ROH2 type=del size=4052306 2024-06-28T15:08:19.143 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.156+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~BNAV4A8L3919.0 size=1048576 2024-06-28T15:08:19.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.160+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~BNAV4A8L3919.1 size=178418 2024-06-28T15:08:19.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.164+0000 7f65680598c0 1 < completed multipart upload key=MQ3QWZBFN4YR upload=BNAV4A8L3919 tag=G9BKQ9UODUDC type=write size=2275570 2024-06-28T15:08:19.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.164+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=TXDCN2ZTUKBJ type=del size=1848598 2024-06-28T15:08:19.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.168+0000 7f65680598c0 1 < canceled part key=_multipart_1VD7NDPEM3CD.2~ST7Y3ZDGDEBV.0 size=1048576 2024-06-28T15:08:19.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.168+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~ST7Y3ZDGDEBV.1 size=898661 2024-06-28T15:08:19.160 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.172+0000 7f65680598c0 1 < canceled multipart upload key=1VD7NDPEM3CD upload=ST7Y3ZDGDEBV tag=B8FHDP3L2MTN type=write size=2995813 2024-06-28T15:08:19.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.176+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=DYSCS20OVPP4 type=write size=690832 2024-06-28T15:08:19.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.180+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=ZQZ17F55X88F type=del 2024-06-28T15:08:19.168 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.180+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=52SSDYNL7HJA type=del size=2137809 2024-06-28T15:08:19.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.184+0000 7f65680598c0 1 < completed write operation key=EN76QQWIQAXZ tag=0M8DFJRC83IN type=write size=473224 2024-06-28T15:08:19.176 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.188+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~YCR9E4FMSGLE.0 size=1048576 2024-06-28T15:08:19.178 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.192+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~YCR9E4FMSGLE.1 size=1048576 2024-06-28T15:08:19.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.192+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~YCR9E4FMSGLE.2 size=632596 2024-06-28T15:08:19.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.192+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=YCR9E4FMSGLE tag=LSMG3LIBP1F1 type=write size=3778324 2024-06-28T15:08:19.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.196+0000 7f65680598c0 -1 < failed to complete operation key=9MQKWSA33R5G tag=5DOKNEOKXPR3 type=del size=3999953: (22) Invalid argument 2024-06-28T15:08:19.186 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.200+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=4ZUS50UZNC23 type=del size=3460789 2024-06-28T15:08:19.189 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.204+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~YWGZVYGYC4NX.0 size=1048576 2024-06-28T15:08:19.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.204+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~YWGZVYGYC4NX.1 size=121580 2024-06-28T15:08:19.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.212+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=YWGZVYGYC4NX tag=N0FF0IJBLLQ1 type=write size=2218732 2024-06-28T15:08:19.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.216+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=APMXSIHIVQYM type=write size=820826 2024-06-28T15:08:19.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.220+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=LR5HR14S15HO type=del 2024-06-28T15:08:19.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.220+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=BXMM1MCTME4E type=write size=574208 2024-06-28T15:08:19.212 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.224+0000 7f65680598c0 1 < completed write operation key=AXQ0YFWUH4FD tag=D6HOISG62CF8 type=write size=1008786 2024-06-28T15:08:19.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.228+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=UZFXF9AJ6Y0Z type=del size=3580692 2024-06-28T15:08:19.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.228+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~OIAD7Q1YU4R3.0 size=1048576 2024-06-28T15:08:19.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.232+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~OIAD7Q1YU4R3.1 size=397676 2024-06-28T15:08:19.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.236+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=OIAD7Q1YU4R3 tag=EEMGDOCU3KD3 type=write size=2494828 2024-06-28T15:08:19.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.240+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=85CY2HLV2P1A type=del size=506293 2024-06-28T15:08:19.230 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.244+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=WKCYJEX4H96O type=del 2024-06-28T15:08:19.233 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.244+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=WZ5095X9CHF6 type=write size=930080 2024-06-28T15:08:19.235 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.248+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=LEJAGI67XQ8G type=del 2024-06-28T15:08:19.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.252+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=A56N2LSL7VNV type=del size=2233465 2024-06-28T15:08:19.244 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.256+0000 7f65680598c0 1 < suggested remove operation key=7NL3CTFYY9E5 tag=93I1D0Q0D0KV type=del 2024-06-28T15:08:19.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.260+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~GSVYZPZMYMVX.0 size=725348 2024-06-28T15:08:19.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.260+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=GSVYZPZMYMVX tag=1LJP4A2QIS9K type=write size=1773924 2024-06-28T15:08:19.252 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.264+0000 7f65680598c0 1 < completed write operation key=D9D3OFIJP3D3 tag=29YCPT7PIMK4 type=write size=1011246 2024-06-28T15:08:19.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.268+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=BV4CD0Q5OOPH type=write size=1027675 2024-06-28T15:08:19.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.272+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=JHQIL4SQI7Z1 type=del 2024-06-28T15:08:19.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.272+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=INTQ4KLHTUO4 type=write size=797638 2024-06-28T15:08:19.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.276+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~1ENB37E60044.0 size=1048576 2024-06-28T15:08:19.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.276+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~1ENB37E60044.1 size=1048576 2024-06-28T15:08:19.267 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.280+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~1ENB37E60044.2 size=733387 2024-06-28T15:08:19.270 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.284+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=1ENB37E60044 tag=MVQE1F3Y9148 type=write size=3879115 2024-06-28T15:08:19.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.284+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=O4AQ1ATBU3BU type=del size=1430672 2024-06-28T15:08:19.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.288+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=G09YJK13ROH2 type=del 2024-06-28T15:08:19.280 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.292+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~53GQ060JQO61.0 size=1048576 2024-06-28T15:08:19.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.296+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~53GQ060JQO61.1 size=376521 2024-06-28T15:08:19.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.296+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=53GQ060JQO61 tag=WN96Z7CWVN3X type=write size=2473673 2024-06-28T15:08:19.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.296+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=TXDCN2ZTUKBJ type=del 2024-06-28T15:08:19.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.300+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=95BM9OIN6EAF type=del size=177618 2024-06-28T15:08:19.289 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.300+0000 7f65680598c0 1 < completed write operation key=I4RWCQSQVLCN tag=DYSCS20OVPP4 type=write size=690832 2024-06-28T15:08:19.292 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.304+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=1K1SFS0XW81N type=del size=398821 2024-06-28T15:08:19.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.308+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=52SSDYNL7HJA type=del 2024-06-28T15:08:19.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.312+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~QV8022DCKFVM.0 size=1048576 2024-06-28T15:08:19.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.316+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~QV8022DCKFVM.1 size=921867 2024-06-28T15:08:19.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.316+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=QV8022DCKFVM tag=E3CWJGL6GSSX type=write size=3019019 2024-06-28T15:08:19.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.316+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~YCR9E4FMSGLE.0 size=1048576 2024-06-28T15:08:19.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.320+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~YCR9E4FMSGLE.1 size=1048576 2024-06-28T15:08:19.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.320+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~YCR9E4FMSGLE.2 size=632596 2024-06-28T15:08:19.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.324+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=YCR9E4FMSGLE tag=LSMG3LIBP1F1 type=write size=3778324 2024-06-28T15:08:19.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.332+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~MAK1OS8CNHDT.0 size=1048576 2024-06-28T15:08:19.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.332+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~MAK1OS8CNHDT.1 size=1048576 2024-06-28T15:08:19.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.336+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~MAK1OS8CNHDT.2 size=77109 2024-06-28T15:08:19.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.336+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=MAK1OS8CNHDT tag=T9MCOC8L0BM3 type=write size=3222837 2024-06-28T15:08:19.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.336+0000 7f65680598c0 1 < canceled operation key=UP3XFF38IJXN tag=4ZUS50UZNC23 type=del size=3460789 2024-06-28T15:08:19.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.340+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=WLDISY22LQ61 type=del size=2242507 2024-06-28T15:08:19.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.344+0000 7f65680598c0 1 < completed write operation key=2F968YIMJN9D tag=APMXSIHIVQYM type=write size=820826 2024-06-28T15:08:19.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.344+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=CPMV8ECA6G26 type=del size=270419 2024-06-28T15:08:19.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.348+0000 7f65680598c0 1 < completed write operation key=4CU86XS1R9G3 tag=BXMM1MCTME4E type=write size=574208 2024-06-28T15:08:19.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.352+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~QU3VY1A24KF6.0 size=1048576 2024-06-28T15:08:19.342 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.356+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~QU3VY1A24KF6.1 size=1048576 2024-06-28T15:08:19.344 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.356+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~QU3VY1A24KF6.2 size=436996 2024-06-28T15:08:19.344 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.356+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=QU3VY1A24KF6 tag=XCK1LQQ52FB5 type=write size=3582724 2024-06-28T15:08:19.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.360+0000 7f65680598c0 1 < suggested remove operation key=RLJCNXKA2GYA tag=UZFXF9AJ6Y0Z type=del 2024-06-28T15:08:19.352 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.364+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=1FTMAYI2U8J3 type=del size=3841060 2024-06-28T15:08:19.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.368+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=85CY2HLV2P1A type=del 2024-06-28T15:08:19.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.368+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=B0I9KCGC1N41 type=del size=1495326 2024-06-28T15:08:19.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.376+0000 7f65680598c0 1 < suggested remove operation key=OUID7RX5NVV6 tag=WZ5095X9CHF6 type=write 2024-06-28T15:08:19.367 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.380+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~A2Z0XSZ5BQS6.0 size=1048576 2024-06-28T15:08:19.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.380+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~A2Z0XSZ5BQS6.1 size=652300 2024-06-28T15:08:19.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.380+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=A2Z0XSZ5BQS6 tag=TL0AHIF2EOAK type=write size=2749452 2024-06-28T15:08:19.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.384+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=A56N2LSL7VNV type=del 2024-06-28T15:08:19.375 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.388+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=T4LTEM1Y7OK6 type=del size=2361417 2024-06-28T15:08:19.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.392+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~GSVYZPZMYMVX.0 size=725348 2024-06-28T15:08:19.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.396+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=GSVYZPZMYMVX tag=1LJP4A2QIS9K type=write size=1773924 2024-06-28T15:08:19.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.400+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=ANHTB05MEXYT type=del size=2432822 2024-06-28T15:08:19.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.400+0000 7f65680598c0 1 < completed write operation key=7LB8MOJL39RM tag=BV4CD0Q5OOPH type=write size=1027675 2024-06-28T15:08:19.393 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.408+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~33UTD20QJ8LL.0 size=1048576 2024-06-28T15:08:19.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.408+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~33UTD20QJ8LL.1 size=1048576 2024-06-28T15:08:19.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.412+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~33UTD20QJ8LL.2 size=924280 2024-06-28T15:08:19.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.412+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=33UTD20QJ8LL tag=ZFB0PZAOW3QH type=write size=4070008 2024-06-28T15:08:19.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.412+0000 7f65680598c0 1 < canceled operation key=BHW2VHUT0U1U tag=INTQ4KLHTUO4 type=write size=797638 2024-06-28T15:08:19.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.420+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~XT5DQ8LDSBYD.0 size=1048576 2024-06-28T15:08:19.408 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.420+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~XT5DQ8LDSBYD.1 size=1048576 2024-06-28T15:08:19.411 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.424+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~XT5DQ8LDSBYD.2 size=191721 2024-06-28T15:08:19.411 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.424+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=XT5DQ8LDSBYD tag=YLMAG2R10084 type=write size=3337449 2024-06-28T15:08:19.413 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.428+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=O4AQ1ATBU3BU type=del 2024-06-28T15:08:19.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.432+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~70MT9Q8ANE69.0 size=1048576 2024-06-28T15:08:19.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.432+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~70MT9Q8ANE69.1 size=1048576 2024-06-28T15:08:19.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.436+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~70MT9Q8ANE69.2 size=789590 2024-06-28T15:08:19.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.436+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=70MT9Q8ANE69 tag=23E7AM69SOK7 type=write size=3935318 2024-06-28T15:08:19.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.440+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~53GQ060JQO61.0 size=1048576 2024-06-28T15:08:19.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.440+0000 7f65680598c0 1 < canceled part key=_multipart_UP3XFF38IJXN.2~53GQ060JQO61.1 size=376521 2024-06-28T15:08:19.430 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.444+0000 7f65680598c0 1 < canceled multipart upload key=UP3XFF38IJXN upload=53GQ060JQO61 tag=WN96Z7CWVN3X type=write size=2473673 2024-06-28T15:08:19.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.444+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=GZUI7A2JCWCH type=write size=649742 2024-06-28T15:08:19.434 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.448+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=95BM9OIN6EAF type=del 2024-06-28T15:08:19.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.448+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=BJQ1XTK2BC5F type=del size=2702561 2024-06-28T15:08:19.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.456+0000 7f65680598c0 1 < suggested remove operation key=D9D3OFIJP3D3 tag=1K1SFS0XW81N type=del 2024-06-28T15:08:19.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.456+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=LG7KZ1BG366R type=del size=2868046 2024-06-28T15:08:19.447 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.460+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~QV8022DCKFVM.0 size=1048576 2024-06-28T15:08:19.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.464+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~QV8022DCKFVM.1 size=921867 2024-06-28T15:08:19.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.472+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=QV8022DCKFVM tag=E3CWJGL6GSSX type=write size=3019019 2024-06-28T15:08:19.462 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.476+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~P3T4XVQ6ROSB.0 size=1048576 2024-06-28T15:08:19.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.476+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~P3T4XVQ6ROSB.1 size=482258 2024-06-28T15:08:19.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.476+0000 7f65680598c0 1 > prepared multipart upload key=2F968YIMJN9D upload=P3T4XVQ6ROSB tag=QIT8KXDEFYAF type=write size=2579410 2024-06-28T15:08:19.467 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.480+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~MAK1OS8CNHDT.0 size=1048576 2024-06-28T15:08:19.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.484+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~MAK1OS8CNHDT.1 size=1048576 2024-06-28T15:08:19.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.484+0000 7f65680598c0 1 < canceled part key=_multipart_V2L2Q1U62YW2.2~MAK1OS8CNHDT.2 size=77109 2024-06-28T15:08:19.474 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.488+0000 7f65680598c0 1 < canceled multipart upload key=V2L2Q1U62YW2 upload=MAK1OS8CNHDT tag=T9MCOC8L0BM3 type=write size=3222837 2024-06-28T15:08:19.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.488+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=3QYILPO1GFL4 type=del size=1674566 2024-06-28T15:08:19.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.492+0000 7f65680598c0 1 < canceled operation key=SFCIZMLY2Y2E tag=WLDISY22LQ61 type=del size=2242507 2024-06-28T15:08:19.481 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.496+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=TDTWAM0JR45P type=del size=3832357 2024-06-28T15:08:19.484 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.496+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=CPMV8ECA6G26 type=del 2024-06-28T15:08:19.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.500+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=7E80JEKVCF7B type=del size=3693509 2024-06-28T15:08:19.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.500+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~QU3VY1A24KF6.0 size=1048576 2024-06-28T15:08:19.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.504+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~QU3VY1A24KF6.1 size=1048576 2024-06-28T15:08:19.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.504+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~QU3VY1A24KF6.2 size=436996 2024-06-28T15:08:19.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.512+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=QU3VY1A24KF6 tag=XCK1LQQ52FB5 type=write size=3582724 2024-06-28T15:08:19.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.512+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=8EW4F642FPDU type=write size=830899 2024-06-28T15:08:19.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.516+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=1FTMAYI2U8J3 type=del 2024-06-28T15:08:19.505 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.516+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=V9A2OFKP3GKJ type=del size=3410028 2024-06-28T15:08:19.507 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.520+0000 7f65680598c0 -1 < failed to complete operation key=OUID7RX5NVV6 tag=B0I9KCGC1N41 type=del size=1495326: (22) Invalid argument 2024-06-28T15:08:19.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.524+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=7CVKESZ55AZ7 type=write size=370461 2024-06-28T15:08:19.513 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.524+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~A2Z0XSZ5BQS6.0 size=1048576 2024-06-28T15:08:19.515 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.528+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~A2Z0XSZ5BQS6.1 size=652300 2024-06-28T15:08:19.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.532+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=A2Z0XSZ5BQS6 tag=TL0AHIF2EOAK type=write size=2749452 2024-06-28T15:08:19.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.536+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=YBRA0ODGPBGV type=del size=1233430 2024-06-28T15:08:19.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.540+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=T4LTEM1Y7OK6 type=del 2024-06-28T15:08:19.530 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.544+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=HZBZ6RLA2WH2 type=del size=3662405 2024-06-28T15:08:19.534 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.548+0000 7f65680598c0 -1 < failed to complete operation key=D9D3OFIJP3D3 tag=ANHTB05MEXYT type=del size=2432822: (22) Invalid argument 2024-06-28T15:08:19.539 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.552+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~EUTQVIOGAERG.0 size=1048576 2024-06-28T15:08:19.541 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.556+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~EUTQVIOGAERG.1 size=1048576 2024-06-28T15:08:19.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.556+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~EUTQVIOGAERG.2 size=419372 2024-06-28T15:08:19.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.556+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=EUTQVIOGAERG tag=QK9R577I5ZBQ type=write size=3565100 2024-06-28T15:08:19.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.560+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~33UTD20QJ8LL.0 size=1048576 2024-06-28T15:08:19.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.564+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~33UTD20QJ8LL.1 size=1048576 2024-06-28T15:08:19.552 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.564+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~33UTD20QJ8LL.2 size=924280 2024-06-28T15:08:19.557 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.568+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=33UTD20QJ8LL tag=ZFB0PZAOW3QH type=write size=4070008 2024-06-28T15:08:19.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.572+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=X1U3VTXQX8UV type=write size=1032772 2024-06-28T15:08:19.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.576+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~XT5DQ8LDSBYD.0 size=1048576 2024-06-28T15:08:19.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.576+0000 7f65680598c0 1 < canceled part key=_multipart_MQ3QWZBFN4YR.2~XT5DQ8LDSBYD.1 size=1048576 2024-06-28T15:08:19.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.576+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~XT5DQ8LDSBYD.2 size=191721 2024-06-28T15:08:19.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.584+0000 7f65680598c0 1 < canceled multipart upload key=MQ3QWZBFN4YR upload=XT5DQ8LDSBYD tag=YLMAG2R10084 type=write size=3337449 2024-06-28T15:08:19.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.584+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=969D2C2IMOB3 type=del size=3166668 2024-06-28T15:08:19.576 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.588+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~70MT9Q8ANE69.0 size=1048576 2024-06-28T15:08:19.578 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.592+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~70MT9Q8ANE69.1 size=1048576 2024-06-28T15:08:19.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.592+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~70MT9Q8ANE69.2 size=789590 2024-06-28T15:08:19.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.600+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=70MT9Q8ANE69 tag=23E7AM69SOK7 type=write size=3935318 2024-06-28T15:08:19.588 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.600+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=9R7ULU732QEQ type=del size=1748882 2024-06-28T15:08:19.593 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.604+0000 7f65680598c0 1 < suggested remove operation key=OUID7RX5NVV6 tag=GZUI7A2JCWCH type=write 2024-06-28T15:08:19.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.608+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~E22456L4P8GS.0 size=1048576 2024-06-28T15:08:19.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.612+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~E22456L4P8GS.1 size=1048576 2024-06-28T15:08:19.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.612+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~E22456L4P8GS.2 size=382073 2024-06-28T15:08:19.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.612+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=E22456L4P8GS tag=AANI2H6T4AKC type=write size=3527801 2024-06-28T15:08:19.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.616+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=BJQ1XTK2BC5F type=del 2024-06-28T15:08:19.605 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.616+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=D7WMDASWHL2U type=write size=973022 2024-06-28T15:08:19.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.620+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=LG7KZ1BG366R type=del 2024-06-28T15:08:19.611 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.624+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~KMKCHD3QX30D.0 size=228838 2024-06-28T15:08:19.611 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.624+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=KMKCHD3QX30D tag=PPUM5B57HVI6 type=write size=1277414 2024-06-28T15:08:19.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.628+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~P3T4XVQ6ROSB.0 size=1048576 2024-06-28T15:08:19.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.628+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~P3T4XVQ6ROSB.1 size=482258 2024-06-28T15:08:19.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.632+0000 7f65680598c0 1 < completed multipart upload key=2F968YIMJN9D upload=P3T4XVQ6ROSB tag=QIT8KXDEFYAF type=write size=2579410 2024-06-28T15:08:19.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.636+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=MA9BDX1VOS33 type=write size=758018 2024-06-28T15:08:19.625 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.640+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=3QYILPO1GFL4 type=del 2024-06-28T15:08:19.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.640+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=7E0BLE40VFWE type=del size=1090397 2024-06-28T15:08:19.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.644+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=TDTWAM0JR45P type=del 2024-06-28T15:08:19.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.648+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=B9LUVD4ZNVUW type=del size=637520 2024-06-28T15:08:19.636 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.648+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=7E80JEKVCF7B type=del 2024-06-28T15:08:19.639 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.652+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=2AOAPVFJ8TG4 type=del size=267064 2024-06-28T15:08:19.642 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.656+0000 7f65680598c0 1 < canceled operation key=V2L2Q1U62YW2 tag=8EW4F642FPDU type=write size=830899 2024-06-28T15:08:19.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.656+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=GBDA17XZ6XTH type=write size=938048 2024-06-28T15:08:19.647 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.660+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=V9A2OFKP3GKJ type=del 2024-06-28T15:08:19.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.664+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=M1RN6UHWLP98 type=del size=2761747 2024-06-28T15:08:19.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.664+0000 7f65680598c0 1 < completed write operation key=ZY973GBGBQ1Z tag=7CVKESZ55AZ7 type=write size=370461 2024-06-28T15:08:19.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.668+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=4VXTW9NQQ1TY type=del size=920898 2024-06-28T15:08:19.657 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.672+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=YBRA0ODGPBGV type=del 2024-06-28T15:08:19.662 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.676+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~Q95Q6V8OMPBY.0 size=1048576 2024-06-28T15:08:19.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.676+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~Q95Q6V8OMPBY.1 size=1048576 2024-06-28T15:08:19.667 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.680+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~Q95Q6V8OMPBY.2 size=682464 2024-06-28T15:08:19.667 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.680+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=Q95Q6V8OMPBY tag=WHRYEBE4DWOF type=write size=3828192 2024-06-28T15:08:19.670 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.684+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=HZBZ6RLA2WH2 type=del 2024-06-28T15:08:19.674 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.688+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~727W8S4LJCFX.0 size=1048576 2024-06-28T15:08:19.676 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.688+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~727W8S4LJCFX.1 size=333104 2024-06-28T15:08:19.676 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.688+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=727W8S4LJCFX tag=DB0HRSAYAW9O type=write size=2430256 2024-06-28T15:08:19.679 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.692+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~EUTQVIOGAERG.0 size=1048576 2024-06-28T15:08:19.681 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.696+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~EUTQVIOGAERG.1 size=1048576 2024-06-28T15:08:19.684 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.696+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~EUTQVIOGAERG.2 size=419372 2024-06-28T15:08:19.688 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.700+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=EUTQVIOGAERG tag=QK9R577I5ZBQ type=write size=3565100 2024-06-28T15:08:19.694 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.708+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~1XH5A98L3W36.0 size=1048576 2024-06-28T15:08:19.696 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.708+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~1XH5A98L3W36.1 size=119610 2024-06-28T15:08:19.696 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.708+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=1XH5A98L3W36 tag=JYSE543IKZTQ type=write size=2216762 2024-06-28T15:08:19.699 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.712+0000 7f65680598c0 1 < completed write operation key=AXQ0YFWUH4FD tag=X1U3VTXQX8UV type=write size=1032772 2024-06-28T15:08:19.707 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.720+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~2RW6C769345H.0 size=142020 2024-06-28T15:08:19.708 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.720+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=2RW6C769345H tag=YRWPXLLZM9OV type=write size=1190596 2024-06-28T15:08:19.710 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.724+0000 7f65680598c0 1 < completed delete operation key=ZFNIA3O84EB7 tag=969D2C2IMOB3 type=del 2024-06-28T15:08:19.713 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.724+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=AFZA143G14BW type=del size=2785242 2024-06-28T15:08:19.716 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.728+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=9R7ULU732QEQ type=del 2024-06-28T15:08:19.718 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.732+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=JYTMZ9RJQ5LV type=del size=2229299 2024-06-28T15:08:19.721 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.732+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~E22456L4P8GS.0 size=1048576 2024-06-28T15:08:19.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.736+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~E22456L4P8GS.1 size=1048576 2024-06-28T15:08:19.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.736+0000 7f65680598c0 1 < canceled part key=_multipart_SFCIZMLY2Y2E.2~E22456L4P8GS.2 size=382073 2024-06-28T15:08:19.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.740+0000 7f65680598c0 1 < canceled multipart upload key=SFCIZMLY2Y2E upload=E22456L4P8GS tag=AANI2H6T4AKC type=write size=3527801 2024-06-28T15:08:19.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.744+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=4BQ6TVWNCHLO type=write size=894826 2024-06-28T15:08:19.734 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.748+0000 7f65680598c0 1 < completed write operation key=AOBFAR0UTONH tag=D7WMDASWHL2U type=write size=973022 2024-06-28T15:08:19.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.752+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~0RSPB2S8O46T.0 size=1048576 2024-06-28T15:08:19.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.752+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~0RSPB2S8O46T.1 size=209401 2024-06-28T15:08:19.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.752+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=0RSPB2S8O46T tag=Q91KXYILCZZF type=write size=2306553 2024-06-28T15:08:19.744 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.756+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~KMKCHD3QX30D.0 size=228838 2024-06-28T15:08:19.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.760+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=KMKCHD3QX30D tag=PPUM5B57HVI6 type=write size=1277414 2024-06-28T15:08:19.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.764+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=SZ32117FDCWC type=del size=3450393 2024-06-28T15:08:19.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.768+0000 7f65680598c0 1 < canceled operation key=D9D3OFIJP3D3 tag=MA9BDX1VOS33 type=write size=758018 2024-06-28T15:08:19.756 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.768+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=J3B6ABTGUCQO type=write size=307718 2024-06-28T15:08:19.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.772+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=7E0BLE40VFWE type=del 2024-06-28T15:08:19.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.776+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=73GYY8NBCCT2 type=write size=459788 2024-06-28T15:08:19.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.776+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=B9LUVD4ZNVUW type=del 2024-06-28T15:08:19.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.780+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=8D1VYCY3T6ZA type=write size=176116 2024-06-28T15:08:19.770 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.784+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=2AOAPVFJ8TG4 type=del 2024-06-28T15:08:19.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.788+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~H8XCHLGL68S1.0 size=1048576 2024-06-28T15:08:19.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.788+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~H8XCHLGL68S1.1 size=964074 2024-06-28T15:08:19.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.788+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=H8XCHLGL68S1 tag=2HT7Y1N4M77I type=write size=3061226 2024-06-28T15:08:19.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.792+0000 7f65680598c0 1 < completed write operation key=60QVJVV16QFM tag=GBDA17XZ6XTH type=write size=938048 2024-06-28T15:08:19.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.796+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=ROVH2CAXPYOI type=del size=17239 2024-06-28T15:08:19.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.796+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=M1RN6UHWLP98 type=del 2024-06-28T15:08:19.789 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.800+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~NJ7GAN3R9TK5.0 size=1048576 2024-06-28T15:08:19.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.804+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~NJ7GAN3R9TK5.1 size=1048576 2024-06-28T15:08:19.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.804+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~NJ7GAN3R9TK5.2 size=408669 2024-06-28T15:08:19.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.804+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=NJ7GAN3R9TK5 tag=PCQ1YM5YKLIH type=write size=3554397 2024-06-28T15:08:19.796 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.808+0000 7f65680598c0 1 < canceled operation key=60QVJVV16QFM tag=4VXTW9NQQ1TY type=del size=920898 2024-06-28T15:08:19.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.812+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=EDHMBPMPHNI0 type=del size=1800093 2024-06-28T15:08:19.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.812+0000 7f65680598c0 1 < canceled part key=_multipart_OUID7RX5NVV6.2~Q95Q6V8OMPBY.0 size=1048576 2024-06-28T15:08:19.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.816+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~Q95Q6V8OMPBY.1 size=1048576 2024-06-28T15:08:19.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.816+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~Q95Q6V8OMPBY.2 size=682464 2024-06-28T15:08:19.808 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.820+0000 7f65680598c0 1 < canceled multipart upload key=OUID7RX5NVV6 upload=Q95Q6V8OMPBY tag=WHRYEBE4DWOF type=write size=3828192 2024-06-28T15:08:19.813 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.824+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~F3J3LEJ83ZPA.0 size=1048576 2024-06-28T15:08:19.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.828+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~F3J3LEJ83ZPA.1 size=692238 2024-06-28T15:08:19.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.828+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=F3J3LEJ83ZPA tag=VXMWWD3X71LT type=write size=2789390 2024-06-28T15:08:19.819 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.832+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~727W8S4LJCFX.0 size=1048576 2024-06-28T15:08:19.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.836+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~727W8S4LJCFX.1 size=333104 2024-06-28T15:08:19.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.840+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=727W8S4LJCFX tag=DB0HRSAYAW9O type=write size=2430256 2024-06-28T15:08:19.829 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.840+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=H8O3UQY5WSR6 type=write size=52950 2024-06-28T15:08:19.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.844+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~1XH5A98L3W36.0 size=1048576 2024-06-28T15:08:19.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.848+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~1XH5A98L3W36.1 size=119610 2024-06-28T15:08:19.838 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.852+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=1XH5A98L3W36 tag=JYSE543IKZTQ type=write size=2216762 2024-06-28T15:08:19.840 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.852+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=Z2HCQW2KJ8A3 type=del size=1293096 2024-06-28T15:08:19.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.856+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~2RW6C769345H.0 size=142020 2024-06-28T15:08:19.848 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.860+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=2RW6C769345H tag=YRWPXLLZM9OV type=write size=1190596 2024-06-28T15:08:19.851 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.864+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=WE3XJDGYO1A2 type=del size=3412955 2024-06-28T15:08:19.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.864+0000 7f65680598c0 1 < completed delete operation key=ZFNIA3O84EB7 tag=AFZA143G14BW type=del 2024-06-28T15:08:19.856 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.868+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=E0L9F87K63DS type=del size=3512052 2024-06-28T15:08:19.859 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.872+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=JYTMZ9RJQ5LV type=del 2024-06-28T15:08:19.863 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.876+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~43IR2QQ54ESY.0 size=214752 2024-06-28T15:08:19.863 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.876+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=43IR2QQ54ESY tag=ZT2P98SYE6O9 type=write size=1263328 2024-06-28T15:08:19.866 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.880+0000 7f65680598c0 1 < completed write operation key=OUID7RX5NVV6 tag=4BQ6TVWNCHLO type=write size=894826 2024-06-28T15:08:19.869 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.880+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=SH7EV0GVKZER type=del size=2654687 2024-06-28T15:08:19.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.884+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~0RSPB2S8O46T.0 size=1048576 2024-06-28T15:08:19.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.884+0000 7f65680598c0 1 < canceled part key=_multipart_V2L2Q1U62YW2.2~0RSPB2S8O46T.1 size=209401 2024-06-28T15:08:19.876 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.888+0000 7f65680598c0 1 < canceled multipart upload key=V2L2Q1U62YW2 upload=0RSPB2S8O46T tag=Q91KXYILCZZF type=write size=2306553 2024-06-28T15:08:19.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.892+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~CCBSQ0FXFWVE.0 size=746475 2024-06-28T15:08:19.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.892+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=CCBSQ0FXFWVE tag=HX1MNHQBFUXM type=write size=1795051 2024-06-28T15:08:19.883 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.896+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=SZ32117FDCWC type=del 2024-06-28T15:08:19.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.900+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~2SSL5728GXR8.0 size=1048576 2024-06-28T15:08:19.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.904+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~2SSL5728GXR8.1 size=622775 2024-06-28T15:08:19.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.904+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=2SSL5728GXR8 tag=40QUJVYG3S2R type=write size=2719927 2024-06-28T15:08:19.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.904+0000 7f65680598c0 1 < completed write operation key=7NL3CTFYY9E5 tag=J3B6ABTGUCQO type=write size=307718 2024-06-28T15:08:19.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.908+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=6FOHWNHH39UX type=del size=2897108 2024-06-28T15:08:19.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.912+0000 7f65680598c0 1 < completed write operation key=2FC9DNEZGWC4 tag=73GYY8NBCCT2 type=write size=459788 2024-06-28T15:08:19.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.912+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=EMNJPT2QXJ7R type=write size=669883 2024-06-28T15:08:19.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.916+0000 7f65680598c0 1 < canceled operation key=AOBFAR0UTONH tag=8D1VYCY3T6ZA type=write size=176116 2024-06-28T15:08:19.905 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.920+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=KL9FLPLAHFZ4 type=write size=299554 2024-06-28T15:08:19.908 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.920+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~H8XCHLGL68S1.0 size=1048576 2024-06-28T15:08:19.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.924+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~H8XCHLGL68S1.1 size=964074 2024-06-28T15:08:19.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.928+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=H8XCHLGL68S1 tag=2HT7Y1N4M77I type=write size=3061226 2024-06-28T15:08:19.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.932+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=DG5G14SZ84GJ type=del size=824209 2024-06-28T15:08:19.924 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.936+0000 7f65680598c0 1 < suggested remove operation key=OUID7RX5NVV6 tag=ROVH2CAXPYOI type=del 2024-06-28T15:08:19.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.940+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=S61IYQ5WHV42 type=del size=1182118 2024-06-28T15:08:19.930 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.944+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~NJ7GAN3R9TK5.0 size=1048576 2024-06-28T15:08:19.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.944+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~NJ7GAN3R9TK5.1 size=1048576 2024-06-28T15:08:19.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.948+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~NJ7GAN3R9TK5.2 size=408669 2024-06-28T15:08:19.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.952+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=NJ7GAN3R9TK5 tag=PCQ1YM5YKLIH type=write size=3554397 2024-06-28T15:08:19.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.956+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=BEYKQJS91QUR type=del size=2467667 2024-06-28T15:08:19.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.956+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=EDHMBPMPHNI0 type=del 2024-06-28T15:08:19.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.960+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=3WL14A9MJX3G type=del size=246973 2024-06-28T15:08:19.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.960+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~F3J3LEJ83ZPA.0 size=1048576 2024-06-28T15:08:19.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.964+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~F3J3LEJ83ZPA.1 size=692238 2024-06-28T15:08:19.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.972+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=F3J3LEJ83ZPA tag=VXMWWD3X71LT type=write size=2789390 2024-06-28T15:08:19.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.976+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~784UW861GUJ2.0 size=1048576 2024-06-28T15:08:19.964 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.976+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~784UW861GUJ2.1 size=16239 2024-06-28T15:08:19.964 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.976+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=784UW861GUJ2 tag=YB2IMAYC5RCK type=write size=2113391 2024-06-28T15:08:19.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.980+0000 7f65680598c0 1 < canceled operation key=2FC9DNEZGWC4 tag=H8O3UQY5WSR6 type=write size=52950 2024-06-28T15:08:19.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.984+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~TCA1K04WZHTT.0 size=1048576 2024-06-28T15:08:19.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.988+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~TCA1K04WZHTT.1 size=416260 2024-06-28T15:08:19.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.988+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=TCA1K04WZHTT tag=2PGXLYS5LAGZ type=write size=2513412 2024-06-28T15:08:19.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.988+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=Z2HCQW2KJ8A3 type=del 2024-06-28T15:08:19.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.992+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=4UG9O4UCRFLA type=del size=1487412 2024-06-28T15:08:19.981 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.992+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=WE3XJDGYO1A2 type=del 2024-06-28T15:08:19.984 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:19.996+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=64ZOTQKWA5HW type=del size=2280351 2024-06-28T15:08:19.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.000+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=E0L9F87K63DS type=del 2024-06-28T15:08:19.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.004+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~XGH6K49XU9HN.0 size=1048576 2024-06-28T15:08:19.992 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.004+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~XGH6K49XU9HN.1 size=1048576 2024-06-28T15:08:19.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.008+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~XGH6K49XU9HN.2 size=834620 2024-06-28T15:08:19.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.008+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=XGH6K49XU9HN tag=VHMOIWFM7BFQ type=write size=3980348 2024-06-28T15:08:19.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.008+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~43IR2QQ54ESY.0 size=214752 2024-06-28T15:08:20.000 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.012+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=43IR2QQ54ESY tag=ZT2P98SYE6O9 type=write size=1263328 2024-06-28T15:08:20.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.016+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=KTLPI4QPCJC7 type=write size=14617 2024-06-28T15:08:20.005 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.016+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=SH7EV0GVKZER type=del 2024-06-28T15:08:20.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.020+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=U3WPBYQ9OUU2 type=del size=2625668 2024-06-28T15:08:20.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.024+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~CCBSQ0FXFWVE.0 size=746475 2024-06-28T15:08:20.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.028+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=CCBSQ0FXFWVE tag=HX1MNHQBFUXM type=write size=1795051 2024-06-28T15:08:20.016 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.028+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=2LAUW42CXRT6 type=del size=3438649 2024-06-28T15:08:20.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.032+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~2SSL5728GXR8.0 size=1048576 2024-06-28T15:08:20.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.032+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~2SSL5728GXR8.1 size=622775 2024-06-28T15:08:20.023 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.036+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=2SSL5728GXR8 tag=40QUJVYG3S2R type=write size=2719927 2024-06-28T15:08:20.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.040+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~O3T4SGAQP22B.0 size=436407 2024-06-28T15:08:20.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.040+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=O3T4SGAQP22B tag=TKBQB19FFW4O type=write size=1484983 2024-06-28T15:08:20.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.044+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=6FOHWNHH39UX type=del 2024-06-28T15:08:20.034 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.048+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=LO25FDE2A6Q5 type=write size=216965 2024-06-28T15:08:20.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.048+0000 7f65680598c0 1 < completed write operation key=2FC9DNEZGWC4 tag=EMNJPT2QXJ7R type=write size=669883 2024-06-28T15:08:20.041 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.052+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~9FM13K8R7LCJ.0 size=1048576 2024-06-28T15:08:20.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.056+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~9FM13K8R7LCJ.1 size=1048576 2024-06-28T15:08:20.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.056+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~9FM13K8R7LCJ.2 size=303793 2024-06-28T15:08:20.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.056+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=9FM13K8R7LCJ tag=7UWHI57TZOMU type=write size=3449521 2024-06-28T15:08:20.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.060+0000 7f65680598c0 1 < completed write operation key=I4RWCQSQVLCN tag=KL9FLPLAHFZ4 type=write size=299554 2024-06-28T15:08:20.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.060+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=JR0WPW7GLGHH type=del size=2446032 2024-06-28T15:08:20.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.064+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=DG5G14SZ84GJ type=del 2024-06-28T15:08:20.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.068+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~4YE9LV7FIQ6A.0 size=146302 2024-06-28T15:08:20.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.068+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=4YE9LV7FIQ6A tag=A8ZRGJXB56Z6 type=write size=1194878 2024-06-28T15:08:20.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.072+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=S61IYQ5WHV42 type=del 2024-06-28T15:08:20.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.072+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=UM8O0RZTHLLK type=del size=2405967 2024-06-28T15:08:20.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.076+0000 7f65680598c0 1 < suggested update operation key=OUID7RX5NVV6 tag=BEYKQJS91QUR type=del size=2467667 2024-06-28T15:08:20.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.080+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=JAN9CLWWVZIE type=write size=147433 2024-06-28T15:08:20.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.080+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=3WL14A9MJX3G type=del 2024-06-28T15:08:20.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.084+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=BW20SB32YLV4 type=del size=2972342 2024-06-28T15:08:20.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.088+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~784UW861GUJ2.0 size=1048576 2024-06-28T15:08:20.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.088+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~784UW861GUJ2.1 size=16239 2024-06-28T15:08:20.079 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.092+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=784UW861GUJ2 tag=YB2IMAYC5RCK type=write size=2113391 2024-06-28T15:08:20.081 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.096+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=VAGSQ4GVMUE9 type=del size=1474840 2024-06-28T15:08:20.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.096+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~TCA1K04WZHTT.0 size=1048576 2024-06-28T15:08:20.086 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.100+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~TCA1K04WZHTT.1 size=416260 2024-06-28T15:08:20.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.104+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=TCA1K04WZHTT tag=2PGXLYS5LAGZ type=write size=2513412 2024-06-28T15:08:20.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.104+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=IKNLS1HJ2UPS type=del size=548402 2024-06-28T15:08:20.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.112+0000 7f65680598c0 1 < suggested remove operation key=MQ3QWZBFN4YR tag=4UG9O4UCRFLA type=del 2024-06-28T15:08:20.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.116+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~I9JCBABNJGC4.0 size=1048576 2024-06-28T15:08:20.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.116+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~I9JCBABNJGC4.1 size=489846 2024-06-28T15:08:20.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.116+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=I9JCBABNJGC4 tag=R2ZIDBDDZRDZ type=write size=2586998 2024-06-28T15:08:20.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.120+0000 7f65680598c0 1 < suggested update operation key=F6SEDD6BH1T2 tag=64ZOTQKWA5HW type=del size=2280351 2024-06-28T15:08:20.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.124+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=FR5251FQRW0G type=del size=3978607 2024-06-28T15:08:20.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.124+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~XGH6K49XU9HN.0 size=1048576 2024-06-28T15:08:20.115 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.128+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~XGH6K49XU9HN.1 size=1048576 2024-06-28T15:08:20.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.128+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~XGH6K49XU9HN.2 size=834620 2024-06-28T15:08:20.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.136+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=XGH6K49XU9HN tag=VHMOIWFM7BFQ type=write size=3980348 2024-06-28T15:08:20.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.136+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=BD45V9ZUBB0D type=del size=118373 2024-06-28T15:08:20.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.140+0000 7f65680598c0 1 < completed write operation key=7NL3CTFYY9E5 tag=KTLPI4QPCJC7 type=write size=14617 2024-06-28T15:08:20.131 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.144+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~LATAP4G61A3K.0 size=1048576 2024-06-28T15:08:20.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.148+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~LATAP4G61A3K.1 size=208348 2024-06-28T15:08:20.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.148+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=LATAP4G61A3K tag=6PHW7XQLK9JR type=write size=2305500 2024-06-28T15:08:20.136 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.148+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=U3WPBYQ9OUU2 type=del 2024-06-28T15:08:20.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.152+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=RK33YE6L48OX type=write size=112540 2024-06-28T15:08:20.140 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.152+0000 7f65680598c0 1 < canceled operation key=AOBFAR0UTONH tag=2LAUW42CXRT6 type=del size=3438649 2024-06-28T15:08:20.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.156+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=IX3GR521ERQX type=del size=2892506 2024-06-28T15:08:20.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.156+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~O3T4SGAQP22B.0 size=436407 2024-06-28T15:08:20.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.160+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=O3T4SGAQP22B tag=TKBQB19FFW4O type=write size=1484983 2024-06-28T15:08:20.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.168+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~GMDM6WRC5SLH.0 size=1048576 2024-06-28T15:08:20.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.168+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~GMDM6WRC5SLH.1 size=512100 2024-06-28T15:08:20.157 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.168+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=GMDM6WRC5SLH tag=K43NDW0QP0NO type=write size=2609252 2024-06-28T15:08:20.162 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.176+0000 7f65680598c0 1 < suggested update operation key=MQ3QWZBFN4YR tag=LO25FDE2A6Q5 type=write size=216965 2024-06-28T15:08:20.164 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.176+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=U9L7XWZZ24X8 type=del size=3149167 2024-06-28T15:08:20.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.180+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~9FM13K8R7LCJ.0 size=1048576 2024-06-28T15:08:20.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.184+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~9FM13K8R7LCJ.1 size=1048576 2024-06-28T15:08:20.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.184+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~9FM13K8R7LCJ.2 size=303793 2024-06-28T15:08:20.176 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.188+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=9FM13K8R7LCJ tag=7UWHI57TZOMU type=write size=3449521 2024-06-28T15:08:20.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.192+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=25XEO2ENI032 type=del size=3843851 2024-06-28T15:08:20.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.196+0000 7f65680598c0 -1 < failed to complete operation key=OUID7RX5NVV6 tag=JR0WPW7GLGHH type=del size=2446032: (22) Invalid argument 2024-06-28T15:08:20.187 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.200+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=2QCMTRUIZWCW type=del size=882998 2024-06-28T15:08:20.191 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.204+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~4YE9LV7FIQ6A.0 size=146302 2024-06-28T15:08:20.196 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.208+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=4YE9LV7FIQ6A tag=A8ZRGJXB56Z6 type=write size=1194878 2024-06-28T15:08:20.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.212+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~GE3MCOG5MSO1.0 size=1048576 2024-06-28T15:08:20.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.216+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~GE3MCOG5MSO1.1 size=1048576 2024-06-28T15:08:20.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.216+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~GE3MCOG5MSO1.2 size=305267 2024-06-28T15:08:20.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.216+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=GE3MCOG5MSO1 tag=JCHJNLRWZNQZ type=write size=3450995 2024-06-28T15:08:20.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.220+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=UM8O0RZTHLLK type=del 2024-06-28T15:08:20.211 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.224+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=00ODCBHO9Y8A type=del size=807740 2024-06-28T15:08:20.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.228+0000 7f65680598c0 1 < completed write operation key=SFCIZMLY2Y2E tag=JAN9CLWWVZIE type=write size=147433 2024-06-28T15:08:20.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.232+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=DAW3CKVDXMOG type=del size=3186143 2024-06-28T15:08:20.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.236+0000 7f65680598c0 1 < suggested remove operation key=RLJCNXKA2GYA tag=BW20SB32YLV4 type=del 2024-06-28T15:08:20.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.240+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=8FBOT058AO2R type=del size=3482275 2024-06-28T15:08:20.230 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.244+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=VAGSQ4GVMUE9 type=del 2024-06-28T15:08:20.236 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.248+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~JG9JYV7CP9N1.0 size=1048576 2024-06-28T15:08:20.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.252+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~JG9JYV7CP9N1.1 size=1048576 2024-06-28T15:08:20.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.252+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~JG9JYV7CP9N1.2 size=985765 2024-06-28T15:08:20.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.252+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=JG9JYV7CP9N1 tag=TOXCWP2GLDIT type=write size=4131493 2024-06-28T15:08:20.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.256+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=IKNLS1HJ2UPS type=del 2024-06-28T15:08:20.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.260+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=2BLFYAKV833N type=del size=3134395 2024-06-28T15:08:20.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.260+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~I9JCBABNJGC4.0 size=1048576 2024-06-28T15:08:20.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.264+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~I9JCBABNJGC4.1 size=489846 2024-06-28T15:08:20.254 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.268+0000 7f65680598c0 1 < completed multipart upload key=MQ3QWZBFN4YR upload=I9JCBABNJGC4 tag=R2ZIDBDDZRDZ type=write size=2586998 2024-06-28T15:08:20.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.268+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=434SL9JFFK4S type=del size=2032076 2024-06-28T15:08:20.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.272+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=FR5251FQRW0G type=del 2024-06-28T15:08:20.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.276+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=D92D5XCOQHHT type=write size=510662 2024-06-28T15:08:20.267 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.280+0000 7f65680598c0 1 < suggested remove operation key=GAZQ3PCPKN9Q tag=BD45V9ZUBB0D type=del 2024-06-28T15:08:20.271 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.284+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~HMW2XI031BQR.0 size=1048576 2024-06-28T15:08:20.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.288+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~HMW2XI031BQR.1 size=1048576 2024-06-28T15:08:20.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.288+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~HMW2XI031BQR.2 size=52729 2024-06-28T15:08:20.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.288+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=HMW2XI031BQR tag=68EB24L08IJT type=write size=3198457 2024-06-28T15:08:20.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.292+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~LATAP4G61A3K.0 size=1048576 2024-06-28T15:08:20.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.292+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~LATAP4G61A3K.1 size=208348 2024-06-28T15:08:20.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.296+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=LATAP4G61A3K tag=6PHW7XQLK9JR type=write size=2305500 2024-06-28T15:08:20.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.300+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=YVFDFEL6A40B type=del size=1149201 2024-06-28T15:08:20.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.300+0000 7f65680598c0 1 < completed write operation key=9MQKWSA33R5G tag=RK33YE6L48OX type=write size=112540 2024-06-28T15:08:20.289 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.300+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=NVOEG275RB28 type=del size=326019 2024-06-28T15:08:20.291 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.304+0000 7f65680598c0 -1 < failed to complete operation key=MQ3QWZBFN4YR tag=IX3GR521ERQX type=del size=2892506: (22) Invalid argument 2024-06-28T15:08:20.293 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.304+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=2XS8XUEAM48A type=del size=3727570 2024-06-28T15:08:20.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.308+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~GMDM6WRC5SLH.0 size=1048576 2024-06-28T15:08:20.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.308+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~GMDM6WRC5SLH.1 size=512100 2024-06-28T15:08:20.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.312+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=GMDM6WRC5SLH tag=K43NDW0QP0NO type=write size=2609252 2024-06-28T15:08:20.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.316+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~LSFDBAXMED4H.0 size=1048576 2024-06-28T15:08:20.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.316+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~LSFDBAXMED4H.1 size=1048576 2024-06-28T15:08:20.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.316+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~LSFDBAXMED4H.2 size=3286 2024-06-28T15:08:20.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.316+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=LSFDBAXMED4H tag=O593NITA2HVB type=write size=3149014 2024-06-28T15:08:20.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.320+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=U9L7XWZZ24X8 type=del 2024-06-28T15:08:20.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.320+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=MWLCQB6RHEOB type=write size=519990 2024-06-28T15:08:20.311 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.324+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=25XEO2ENI032 type=del 2024-06-28T15:08:20.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.328+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~3Y5JPG9HVROU.0 size=1048576 2024-06-28T15:08:20.316 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.328+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~3Y5JPG9HVROU.1 size=578881 2024-06-28T15:08:20.316 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.328+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=3Y5JPG9HVROU tag=F18SM1OJXXG6 type=write size=2676033 2024-06-28T15:08:20.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.332+0000 7f65680598c0 1 < suggested update operation key=OUID7RX5NVV6 tag=2QCMTRUIZWCW type=del size=882998 2024-06-28T15:08:20.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.336+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~EPLVRAPJS820.0 size=1048576 2024-06-28T15:08:20.326 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.340+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~EPLVRAPJS820.1 size=1048576 2024-06-28T15:08:20.328 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.340+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~EPLVRAPJS820.2 size=225086 2024-06-28T15:08:20.328 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.340+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=EPLVRAPJS820 tag=CKV6W6QXDA7Z type=write size=3370814 2024-06-28T15:08:20.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.344+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~GE3MCOG5MSO1.0 size=1048576 2024-06-28T15:08:20.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.344+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~GE3MCOG5MSO1.1 size=1048576 2024-06-28T15:08:20.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.344+0000 7f65680598c0 1 < canceled part key=_multipart_ZFNIA3O84EB7.2~GE3MCOG5MSO1.2 size=305267 2024-06-28T15:08:20.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.348+0000 7f65680598c0 1 < canceled multipart upload key=ZFNIA3O84EB7 upload=GE3MCOG5MSO1 tag=JCHJNLRWZNQZ type=write size=3450995 2024-06-28T15:08:20.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.352+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=A4ZCDBKVTPLT type=write size=398105 2024-06-28T15:08:20.342 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.356+0000 7f65680598c0 1 < suggested update operation key=AOBFAR0UTONH tag=00ODCBHO9Y8A type=del size=807740 2024-06-28T15:08:20.346 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.360+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~P33S2QALW85O.0 size=987343 2024-06-28T15:08:20.346 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.360+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=P33S2QALW85O tag=GY0IBLN68KLJ type=write size=2035919 2024-06-28T15:08:20.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.364+0000 7f65680598c0 1 < suggested update operation key=OUID7RX5NVV6 tag=DAW3CKVDXMOG type=del size=3186143 2024-06-28T15:08:20.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.368+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~Y22098Q5679E.0 size=1048576 2024-06-28T15:08:20.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.368+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~Y22098Q5679E.1 size=629762 2024-06-28T15:08:20.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.368+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=Y22098Q5679E tag=0ZPOQGJQ2SDH type=write size=2726914 2024-06-28T15:08:20.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.372+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=8FBOT058AO2R type=del 2024-06-28T15:08:20.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.372+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=3SHV2D9J530E type=del size=3857548 2024-06-28T15:08:20.361 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.376+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~JG9JYV7CP9N1.0 size=1048576 2024-06-28T15:08:20.363 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.376+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~JG9JYV7CP9N1.1 size=1048576 2024-06-28T15:08:20.365 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.376+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~JG9JYV7CP9N1.2 size=985765 2024-06-28T15:08:20.368 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.380+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=JG9JYV7CP9N1 tag=TOXCWP2GLDIT type=write size=4131493 2024-06-28T15:08:20.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.384+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=DUQC52AXM8WD type=del size=2834424 2024-06-28T15:08:20.373 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.384+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=2BLFYAKV833N type=del 2024-06-28T15:08:20.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.388+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=JCSE15UCSJS4 type=write size=371826 2024-06-28T15:08:20.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.388+0000 7f65680598c0 1 < canceled operation key=I4RWCQSQVLCN tag=434SL9JFFK4S type=del size=2032076 2024-06-28T15:08:20.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.392+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=YPE1XW0S69KJ type=del size=4173270 2024-06-28T15:08:20.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.392+0000 7f65680598c0 1 < canceled operation key=MQ3QWZBFN4YR tag=D92D5XCOQHHT type=write size=510662 2024-06-28T15:08:20.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.396+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=HRL8SVP9VZ3Z type=del size=3869974 2024-06-28T15:08:20.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.396+0000 7f65680598c0 1 < canceled part key=_multipart_MQ3QWZBFN4YR.2~HMW2XI031BQR.0 size=1048576 2024-06-28T15:08:20.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.396+0000 7f65680598c0 1 < canceled part key=_multipart_MQ3QWZBFN4YR.2~HMW2XI031BQR.1 size=1048576 2024-06-28T15:08:20.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.400+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~HMW2XI031BQR.2 size=52729 2024-06-28T15:08:20.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.404+0000 7f65680598c0 1 < canceled multipart upload key=MQ3QWZBFN4YR upload=HMW2XI031BQR tag=68EB24L08IJT type=write size=3198457 2024-06-28T15:08:20.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.404+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=6UQFEK4EAQ0O type=del size=955472 2024-06-28T15:08:20.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.408+0000 7f65680598c0 1 < suggested update operation key=AXQ0YFWUH4FD tag=YVFDFEL6A40B type=del size=1149201 2024-06-28T15:08:20.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.412+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~3FU9LGWARLYS.0 size=1048576 2024-06-28T15:08:20.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.416+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~3FU9LGWARLYS.1 size=1048576 2024-06-28T15:08:20.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.420+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~3FU9LGWARLYS.2 size=722295 2024-06-28T15:08:20.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.420+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=3FU9LGWARLYS tag=5SD3X1HPZ0M8 type=write size=3868023 2024-06-28T15:08:20.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.424+0000 7f65680598c0 1 < suggested remove operation key=EN76QQWIQAXZ tag=NVOEG275RB28 type=del 2024-06-28T15:08:20.412 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.424+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=A0OUX2A5JTBZ type=del size=920334 2024-06-28T15:08:20.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.428+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=2XS8XUEAM48A type=del 2024-06-28T15:08:20.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.428+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=OYY5B4DKAC3A type=del size=2474578 2024-06-28T15:08:20.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.432+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~LSFDBAXMED4H.0 size=1048576 2024-06-28T15:08:20.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.432+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~LSFDBAXMED4H.1 size=1048576 2024-06-28T15:08:20.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.436+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~LSFDBAXMED4H.2 size=3286 2024-06-28T15:08:20.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.440+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=LSFDBAXMED4H tag=O593NITA2HVB type=write size=3149014 2024-06-28T15:08:20.431 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.444+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~NSCT24UBG4HI.0 size=27684 2024-06-28T15:08:20.431 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.444+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=NSCT24UBG4HI tag=0D30U60IURW1 type=write size=1076260 2024-06-28T15:08:20.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.448+0000 7f65680598c0 1 < suggested remove operation key=UP3XFF38IJXN tag=MWLCQB6RHEOB type=write 2024-06-28T15:08:20.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.452+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=FY51IF2V366E type=write size=145093 2024-06-28T15:08:20.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.452+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~3Y5JPG9HVROU.0 size=1048576 2024-06-28T15:08:20.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.456+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~3Y5JPG9HVROU.1 size=578881 2024-06-28T15:08:20.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.460+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=3Y5JPG9HVROU tag=F18SM1OJXXG6 type=write size=2676033 2024-06-28T15:08:20.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.464+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~KEBT970O1E29.0 size=1048576 2024-06-28T15:08:20.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.464+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~KEBT970O1E29.1 size=398873 2024-06-28T15:08:20.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.464+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=KEBT970O1E29 tag=EHQAYVK1N38W type=write size=2496025 2024-06-28T15:08:20.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.468+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~EPLVRAPJS820.0 size=1048576 2024-06-28T15:08:20.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.468+0000 7f65680598c0 1 < canceled part key=_multipart_MQ3QWZBFN4YR.2~EPLVRAPJS820.1 size=1048576 2024-06-28T15:08:20.456 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.468+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~EPLVRAPJS820.2 size=225086 2024-06-28T15:08:20.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.472+0000 7f65680598c0 1 < canceled multipart upload key=MQ3QWZBFN4YR upload=EPLVRAPJS820 tag=CKV6W6QXDA7Z type=write size=3370814 2024-06-28T15:08:20.462 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.476+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=KN2UQBEG149K type=del size=3638123 2024-06-28T15:08:20.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.476+0000 7f65680598c0 1 < completed write operation key=GAZQ3PCPKN9Q tag=A4ZCDBKVTPLT type=write size=398105 2024-06-28T15:08:20.466 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.480+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=JZT6ZVH1WPQV type=del size=391609 2024-06-28T15:08:20.468 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.480+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~P33S2QALW85O.0 size=987343 2024-06-28T15:08:20.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.484+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=P33S2QALW85O tag=GY0IBLN68KLJ type=write size=2035919 2024-06-28T15:08:20.474 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.488+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=GEYR6O6MPS0P type=del size=4185913 2024-06-28T15:08:20.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.488+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~Y22098Q5679E.0 size=1048576 2024-06-28T15:08:20.478 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.492+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~Y22098Q5679E.1 size=629762 2024-06-28T15:08:20.482 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.496+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=Y22098Q5679E tag=0ZPOQGJQ2SDH type=write size=2726914 2024-06-28T15:08:20.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.500+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~LBM2M1QTI1JP.0 size=1048576 2024-06-28T15:08:20.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.500+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~LBM2M1QTI1JP.1 size=283122 2024-06-28T15:08:20.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.500+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=LBM2M1QTI1JP tag=8O11L9VW9AZL type=write size=2380274 2024-06-28T15:08:20.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.504+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=3SHV2D9J530E type=del 2024-06-28T15:08:20.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.504+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=NT9IMFY20U3R type=del size=3989267 2024-06-28T15:08:20.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.508+0000 7f65680598c0 1 < suggested update operation key=MQ3QWZBFN4YR tag=DUQC52AXM8WD type=del size=2834424 2024-06-28T15:08:20.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.512+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=T551SYYTHBO9 type=del size=2912915 2024-06-28T15:08:20.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.516+0000 7f65680598c0 1 < completed write operation key=J311Q2712YM0 tag=JCSE15UCSJS4 type=write size=371826 2024-06-28T15:08:20.506 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.520+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~K29RUW3GHVKE.0 size=1048576 2024-06-28T15:08:20.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.520+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~K29RUW3GHVKE.1 size=1048576 2024-06-28T15:08:20.509 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.524+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~K29RUW3GHVKE.2 size=604290 2024-06-28T15:08:20.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.524+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=K29RUW3GHVKE tag=F1N2ZVXO9164 type=write size=3750018 2024-06-28T15:08:20.512 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.524+0000 7f65680598c0 -1 < failed to complete operation key=UP3XFF38IJXN tag=YPE1XW0S69KJ type=del size=4173270: (22) Invalid argument 2024-06-28T15:08:20.514 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.528+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=BVFKRMBOK2ZO type=del size=2681591 2024-06-28T15:08:20.516 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.528+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=HRL8SVP9VZ3Z type=del 2024-06-28T15:08:20.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.532+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=1GVHE3D05DN0 type=del size=177507 2024-06-28T15:08:20.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.536+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=6UQFEK4EAQ0O type=del 2024-06-28T15:08:20.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.536+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=XL1Y937I43MV type=del size=99084 2024-06-28T15:08:20.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.540+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~3FU9LGWARLYS.0 size=1048576 2024-06-28T15:08:20.528 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.540+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~3FU9LGWARLYS.1 size=1048576 2024-06-28T15:08:20.531 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.544+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~3FU9LGWARLYS.2 size=722295 2024-06-28T15:08:20.535 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.548+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=3FU9LGWARLYS tag=5SD3X1HPZ0M8 type=write size=3868023 2024-06-28T15:08:20.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.552+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~SR20VLTEYSAC.0 size=353603 2024-06-28T15:08:20.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.552+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=SR20VLTEYSAC tag=A9ZZG58FNCW3 type=write size=1402179 2024-06-28T15:08:20.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.556+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=A0OUX2A5JTBZ type=del 2024-06-28T15:08:20.545 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.556+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=4930ANDQIDLG type=del size=1224588 2024-06-28T15:08:20.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.560+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=OYY5B4DKAC3A type=del 2024-06-28T15:08:20.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.564+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=E2MDANVMGHV8 type=del size=1845157 2024-06-28T15:08:20.553 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.564+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~NSCT24UBG4HI.0 size=27684 2024-06-28T15:08:20.557 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.568+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=NSCT24UBG4HI tag=0D30U60IURW1 type=write size=1076260 2024-06-28T15:08:20.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.572+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=P3GRYY1W9KNK type=del size=323445 2024-06-28T15:08:20.561 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.576+0000 7f65680598c0 -1 < failed to complete operation key=MQ3QWZBFN4YR tag=FY51IF2V366E type=write size=145093: (22) Invalid argument 2024-06-28T15:08:20.564 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.576+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=C6SLE4XFBZVZ type=del size=3631803 2024-06-28T15:08:20.567 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.580+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~KEBT970O1E29.0 size=1048576 2024-06-28T15:08:20.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.580+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~KEBT970O1E29.1 size=398873 2024-06-28T15:08:20.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.584+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=KEBT970O1E29 tag=EHQAYVK1N38W type=write size=2496025 2024-06-28T15:08:20.577 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.592+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~5PQPDU268ESX.0 size=1048576 2024-06-28T15:08:20.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.592+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~5PQPDU268ESX.1 size=797292 2024-06-28T15:08:20.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.592+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=5PQPDU268ESX tag=CL18CLG6Z21J type=write size=2894444 2024-06-28T15:08:20.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.596+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=KN2UQBEG149K type=del 2024-06-28T15:08:20.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.596+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=FZRUB0SQDVCO type=del size=3841164 2024-06-28T15:08:20.588 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.600+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=JZT6ZVH1WPQV type=del 2024-06-28T15:08:20.590 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.604+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=OGX52IC9RLKN type=del size=2249320 2024-06-28T15:08:20.593 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.604+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=GEYR6O6MPS0P type=del 2024-06-28T15:08:20.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.612+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~Y1CJCXKTTL3T.0 size=1048576 2024-06-28T15:08:20.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.612+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~Y1CJCXKTTL3T.1 size=1048576 2024-06-28T15:08:20.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.616+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~Y1CJCXKTTL3T.2 size=907277 2024-06-28T15:08:20.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.616+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=Y1CJCXKTTL3T tag=YIKTG6GXXFQT type=write size=4053005 2024-06-28T15:08:20.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.616+0000 7f65680598c0 1 < canceled part key=_multipart_9MQKWSA33R5G.2~LBM2M1QTI1JP.0 size=1048576 2024-06-28T15:08:20.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.616+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~LBM2M1QTI1JP.1 size=283122 2024-06-28T15:08:20.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.620+0000 7f65680598c0 1 < canceled multipart upload key=9MQKWSA33R5G upload=LBM2M1QTI1JP tag=8O11L9VW9AZL type=write size=2380274 2024-06-28T15:08:20.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.624+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~B7LNCF9IHMPG.0 size=492539 2024-06-28T15:08:20.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.624+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=B7LNCF9IHMPG tag=O0A40KK42DEU type=write size=1541115 2024-06-28T15:08:20.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.628+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=NT9IMFY20U3R type=del 2024-06-28T15:08:20.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.628+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=5CUN55M41FGK type=del size=2360553 2024-06-28T15:08:20.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.632+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=T551SYYTHBO9 type=del 2024-06-28T15:08:20.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.640+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~ETUFQ2GM62NY.0 size=267767 2024-06-28T15:08:20.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.640+0000 7f65680598c0 1 > prepared multipart upload key=2F968YIMJN9D upload=ETUFQ2GM62NY tag=ESOLEEU86GGY type=write size=1316343 2024-06-28T15:08:20.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.644+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~K29RUW3GHVKE.0 size=1048576 2024-06-28T15:08:20.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.644+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~K29RUW3GHVKE.1 size=1048576 2024-06-28T15:08:20.635 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.648+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~K29RUW3GHVKE.2 size=604290 2024-06-28T15:08:20.639 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.652+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=K29RUW3GHVKE tag=F1N2ZVXO9164 type=write size=3750018 2024-06-28T15:08:20.642 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.656+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=QWV5YAC5NOP9 type=del size=1943536 2024-06-28T15:08:20.645 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.656+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=BVFKRMBOK2ZO type=del 2024-06-28T15:08:20.648 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.660+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=41L4OQU16Z2J type=del size=2764457 2024-06-28T15:08:20.651 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.664+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=1GVHE3D05DN0 type=del 2024-06-28T15:08:20.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.668+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~WYTN5MYIWCOX.0 size=1048576 2024-06-28T15:08:20.657 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.672+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~WYTN5MYIWCOX.1 size=1023696 2024-06-28T15:08:20.657 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.672+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=WYTN5MYIWCOX tag=1F4DN1SOAWFB type=write size=3120848 2024-06-28T15:08:20.663 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.676+0000 7f65680598c0 1 < suggested update operation key=OUID7RX5NVV6 tag=XL1Y937I43MV type=del size=99084 2024-06-28T15:08:20.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.680+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~721TSYGK8W3R.0 size=1048576 2024-06-28T15:08:20.669 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.684+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~721TSYGK8W3R.1 size=1048576 2024-06-28T15:08:20.671 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.684+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~721TSYGK8W3R.2 size=81058 2024-06-28T15:08:20.672 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.684+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=721TSYGK8W3R tag=2CUAEALN3WFG type=write size=3226786 2024-06-28T15:08:20.674 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.688+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~SR20VLTEYSAC.0 size=353603 2024-06-28T15:08:20.679 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.692+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=SR20VLTEYSAC tag=A9ZZG58FNCW3 type=write size=1402179 2024-06-28T15:08:20.683 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.696+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~M3BOHNGMA1BW.0 size=1048576 2024-06-28T15:08:20.685 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.696+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~M3BOHNGMA1BW.1 size=1048576 2024-06-28T15:08:20.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.700+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~M3BOHNGMA1BW.2 size=61370 2024-06-28T15:08:20.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.700+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=M3BOHNGMA1BW tag=SKP861H46P7H type=write size=3207098 2024-06-28T15:08:20.689 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.704+0000 7f65680598c0 1 < canceled operation key=R0IMFVF5NEOO tag=4930ANDQIDLG type=del size=1224588 2024-06-28T15:08:20.692 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.704+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=ECBZ5K8EZIPI type=del size=2417728 2024-06-28T15:08:20.697 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.708+0000 7f65680598c0 1 < suggested update operation key=SFCIZMLY2Y2E tag=E2MDANVMGHV8 type=del size=1845157 2024-06-28T15:08:20.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.712+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=VSET3UC14ZIL type=del size=3811007 2024-06-28T15:08:20.705 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.720+0000 7f65680598c0 1 < suggested remove operation key=V2L2Q1U62YW2 tag=P3GRYY1W9KNK type=del 2024-06-28T15:08:20.708 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.720+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=CB3UE5309FF1 type=write size=636891 2024-06-28T15:08:20.710 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.724+0000 7f65680598c0 1 < canceled operation key=UP3XFF38IJXN tag=C6SLE4XFBZVZ type=del size=3631803 2024-06-28T15:08:20.713 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.724+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=7F632PGECLRH type=del size=3355721 2024-06-28T15:08:20.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.728+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~5PQPDU268ESX.0 size=1048576 2024-06-28T15:08:20.717 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.728+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~5PQPDU268ESX.1 size=797292 2024-06-28T15:08:20.720 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.732+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=5PQPDU268ESX tag=CL18CLG6Z21J type=write size=2894444 2024-06-28T15:08:20.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.736+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=GE7PQVI5NP3Y type=del size=572135 2024-06-28T15:08:20.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.740+0000 7f65680598c0 1 < suggested update operation key=P278ZM1H052S tag=FZRUB0SQDVCO type=del size=3841164 2024-06-28T15:08:20.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.744+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=ER1O3EB30N7V type=write size=536098 2024-06-28T15:08:20.732 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.744+0000 7f65680598c0 1 < canceled operation key=2FC9DNEZGWC4 tag=OGX52IC9RLKN type=del size=2249320 2024-06-28T15:08:20.736 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.748+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~TLXDQDV8PJTY.0 size=1048576 2024-06-28T15:08:20.737 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.752+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~TLXDQDV8PJTY.1 size=787002 2024-06-28T15:08:20.737 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.752+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=TLXDQDV8PJTY tag=ZLV22WRDMEN9 type=write size=2884154 2024-06-28T15:08:20.740 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.752+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~Y1CJCXKTTL3T.0 size=1048576 2024-06-28T15:08:20.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.756+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~Y1CJCXKTTL3T.1 size=1048576 2024-06-28T15:08:20.743 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.756+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~Y1CJCXKTTL3T.2 size=907277 2024-06-28T15:08:20.747 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.760+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=Y1CJCXKTTL3T tag=YIKTG6GXXFQT type=write size=4053005 2024-06-28T15:08:20.749 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.760+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=OQBH8ENSDRHY type=del size=1055086 2024-06-28T15:08:20.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.764+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~B7LNCF9IHMPG.0 size=492539 2024-06-28T15:08:20.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.768+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=B7LNCF9IHMPG tag=O0A40KK42DEU type=write size=1541115 2024-06-28T15:08:20.756 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.768+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=JWHMIBYDS2I0 type=del size=747799 2024-06-28T15:08:20.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.772+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=5CUN55M41FGK type=del 2024-06-28T15:08:20.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.776+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~U4YI4RQJFM5E.0 size=1048576 2024-06-28T15:08:20.764 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.776+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~U4YI4RQJFM5E.1 size=1048576 2024-06-28T15:08:20.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.780+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~U4YI4RQJFM5E.2 size=330234 2024-06-28T15:08:20.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.780+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=U4YI4RQJFM5E tag=870Y1Q0FJFSV type=write size=3475962 2024-06-28T15:08:20.768 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.780+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~ETUFQ2GM62NY.0 size=267767 2024-06-28T15:08:20.772 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.784+0000 7f65680598c0 1 < completed multipart upload key=2F968YIMJN9D upload=ETUFQ2GM62NY tag=ESOLEEU86GGY type=write size=1316343 2024-06-28T15:08:20.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.788+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~MDDJYE38MLOQ.0 size=1048576 2024-06-28T15:08:20.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.792+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~MDDJYE38MLOQ.1 size=32912 2024-06-28T15:08:20.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.792+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=MDDJYE38MLOQ tag=EQTDT8GJRED5 type=write size=2130064 2024-06-28T15:08:20.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.792+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=QWV5YAC5NOP9 type=del 2024-06-28T15:08:20.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.796+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=2BYRNUVDTKAE type=del size=761408 2024-06-28T15:08:20.786 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.800+0000 7f65680598c0 1 < canceled operation key=2F968YIMJN9D tag=41L4OQU16Z2J type=del size=2764457 2024-06-28T15:08:20.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.804+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~YR33XV9DMPOY.0 size=1048576 2024-06-28T15:08:20.794 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.808+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~YR33XV9DMPOY.1 size=884631 2024-06-28T15:08:20.794 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.808+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=YR33XV9DMPOY tag=H1GOKZ0JGMTD type=write size=2981783 2024-06-28T15:08:20.797 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.808+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~WYTN5MYIWCOX.0 size=1048576 2024-06-28T15:08:20.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.812+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~WYTN5MYIWCOX.1 size=1023696 2024-06-28T15:08:20.805 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.816+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=WYTN5MYIWCOX tag=1F4DN1SOAWFB type=write size=3120848 2024-06-28T15:08:20.808 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.820+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=UELN3I7RL41E type=del size=3552721 2024-06-28T15:08:20.811 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.824+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~721TSYGK8W3R.0 size=1048576 2024-06-28T15:08:20.814 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.828+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~721TSYGK8W3R.1 size=1048576 2024-06-28T15:08:20.816 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.828+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~721TSYGK8W3R.2 size=81058 2024-06-28T15:08:20.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.836+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=721TSYGK8W3R tag=2CUAEALN3WFG type=write size=3226786 2024-06-28T15:08:20.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.840+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~997JM8GHZ4PU.0 size=1048576 2024-06-28T15:08:20.830 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.844+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~997JM8GHZ4PU.1 size=1030372 2024-06-28T15:08:20.830 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.844+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=997JM8GHZ4PU tag=YMGI3CZ19UNM type=write size=3127524 2024-06-28T15:08:20.833 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.848+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~M3BOHNGMA1BW.0 size=1048576 2024-06-28T15:08:20.836 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.848+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~M3BOHNGMA1BW.1 size=1048576 2024-06-28T15:08:20.838 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.852+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~M3BOHNGMA1BW.2 size=61370 2024-06-28T15:08:20.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.856+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=M3BOHNGMA1BW tag=SKP861H46P7H type=write size=3207098 2024-06-28T15:08:20.846 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.860+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=YJD97V65UO0D type=write size=253652 2024-06-28T15:08:20.849 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.860+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=ECBZ5K8EZIPI type=del 2024-06-28T15:08:20.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.868+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~940AH5WO2PZD.0 size=482832 2024-06-28T15:08:20.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.868+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=940AH5WO2PZD tag=DTCRRR093SRM type=write size=1531408 2024-06-28T15:08:20.858 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.872+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=VSET3UC14ZIL type=del 2024-06-28T15:08:20.861 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.872+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=YKMF55EVPYT8 type=del size=117989 2024-06-28T15:08:20.864 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.876+0000 7f65680598c0 1 < completed write operation key=I4RWCQSQVLCN tag=CB3UE5309FF1 type=write size=636891 2024-06-28T15:08:20.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.884+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=6AKKOWLO1IF2 type=del size=1780173 2024-06-28T15:08:20.874 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.888+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=7F632PGECLRH type=del 2024-06-28T15:08:20.877 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.888+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=E0O12BYE0LM1 type=del size=963846 2024-06-28T15:08:20.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.892+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=GE7PQVI5NP3Y type=del 2024-06-28T15:08:20.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.896+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=E3ZF1I9RNYRW type=write size=598563 2024-06-28T15:08:20.885 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.896+0000 7f65680598c0 1 < completed write operation key=UP3XFF38IJXN tag=ER1O3EB30N7V type=write size=536098 2024-06-28T15:08:20.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.900+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~WNUK4710XOPS.0 size=1048576 2024-06-28T15:08:20.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.904+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~WNUK4710XOPS.1 size=1030740 2024-06-28T15:08:20.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.904+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=WNUK4710XOPS tag=08ZOK3UQRWMV type=write size=3127892 2024-06-28T15:08:20.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.908+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~TLXDQDV8PJTY.0 size=1048576 2024-06-28T15:08:20.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.908+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~TLXDQDV8PJTY.1 size=787002 2024-06-28T15:08:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.912+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=TLXDQDV8PJTY tag=ZLV22WRDMEN9 type=write size=2884154 2024-06-28T15:08:20.901 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.916+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=KNDU77SQ7YKG type=del size=998234 2024-06-28T15:08:20.906 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.920+0000 7f65680598c0 1 < suggested remove operation key=1VD7NDPEM3CD tag=OQBH8ENSDRHY type=del 2024-06-28T15:08:20.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.924+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~43KKID7HOW48.0 size=935323 2024-06-28T15:08:20.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.924+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=43KKID7HOW48 tag=L0QW4GEQJMGN type=write size=1983899 2024-06-28T15:08:20.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.928+0000 7f65680598c0 1 < suggested update operation key=V2L2Q1U62YW2 tag=JWHMIBYDS2I0 type=del size=747799 2024-06-28T15:08:20.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.932+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~XNRYCYA9L3CQ.0 size=1048576 2024-06-28T15:08:20.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.936+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~XNRYCYA9L3CQ.1 size=1048576 2024-06-28T15:08:20.924 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.936+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~XNRYCYA9L3CQ.2 size=548989 2024-06-28T15:08:20.924 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.936+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=XNRYCYA9L3CQ tag=FE7ZGA1PWMDZ type=write size=3694717 2024-06-28T15:08:20.926 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.940+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~U4YI4RQJFM5E.0 size=1048576 2024-06-28T15:08:20.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.940+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~U4YI4RQJFM5E.1 size=1048576 2024-06-28T15:08:20.931 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.944+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~U4YI4RQJFM5E.2 size=330234 2024-06-28T15:08:20.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.948+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=U4YI4RQJFM5E tag=870Y1Q0FJFSV type=write size=3475962 2024-06-28T15:08:20.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.956+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~LL68CREYOED0.0 size=1048576 2024-06-28T15:08:20.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.956+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~LL68CREYOED0.1 size=1048576 2024-06-28T15:08:20.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.956+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~LL68CREYOED0.2 size=102672 2024-06-28T15:08:20.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.956+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=LL68CREYOED0 tag=5S3KFJBCD6IJ type=write size=3248400 2024-06-28T15:08:20.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.960+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~MDDJYE38MLOQ.0 size=1048576 2024-06-28T15:08:20.949 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.960+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~MDDJYE38MLOQ.1 size=32912 2024-06-28T15:08:20.953 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.964+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=MDDJYE38MLOQ tag=EQTDT8GJRED5 type=write size=2130064 2024-06-28T15:08:20.955 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.968+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=I6TMYOQ7IUPA type=del size=2723825 2024-06-28T15:08:20.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.972+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=2BYRNUVDTKAE type=del 2024-06-28T15:08:20.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.972+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=MJRXGLMZHWSJ type=del size=949866 2024-06-28T15:08:20.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.976+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~YR33XV9DMPOY.0 size=1048576 2024-06-28T15:08:20.964 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.976+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~YR33XV9DMPOY.1 size=884631 2024-06-28T15:08:20.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.980+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=YR33XV9DMPOY tag=H1GOKZ0JGMTD type=write size=2981783 2024-06-28T15:08:20.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.984+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=OAA4PY9MAWH9 type=del size=3228587 2024-06-28T15:08:20.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.988+0000 7f65680598c0 1 < canceled operation key=4CU86XS1R9G3 tag=UELN3I7RL41E type=del size=3552721 2024-06-28T15:08:20.975 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.988+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=RL37WOQZG34L type=del size=2864264 2024-06-28T15:08:20.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.992+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~997JM8GHZ4PU.0 size=1048576 2024-06-28T15:08:20.980 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.992+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~997JM8GHZ4PU.1 size=1030372 2024-06-28T15:08:20.984 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:20.996+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=997JM8GHZ4PU tag=YMGI3CZ19UNM type=write size=3127524 2024-06-28T15:08:20.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.000+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=HOMW2JZ3N533 type=del size=1674320 2024-06-28T15:08:20.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.000+0000 7f65680598c0 1 < canceled operation key=9MQKWSA33R5G tag=YJD97V65UO0D type=write size=253652 2024-06-28T15:08:20.991 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.004+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=41YK734PV8MK type=del size=2133355 2024-06-28T15:08:20.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.008+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~940AH5WO2PZD.0 size=482832 2024-06-28T15:08:20.998 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.012+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=940AH5WO2PZD tag=DTCRRR093SRM type=write size=1531408 2024-06-28T15:08:21.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.016+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~P8B6QFSIF5XW.0 size=1048576 2024-06-28T15:08:21.004 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.016+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~P8B6QFSIF5XW.1 size=1048576 2024-06-28T15:08:21.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.020+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~P8B6QFSIF5XW.2 size=811779 2024-06-28T15:08:21.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.020+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=P8B6QFSIF5XW tag=9LSO53IDUWCB type=write size=3957507 2024-06-28T15:08:21.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.024+0000 7f65680598c0 1 < suggested update operation key=9MQKWSA33R5G tag=YKMF55EVPYT8 type=del size=117989 2024-06-28T15:08:21.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.024+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=2OEWRUBXYW53 type=del size=2695179 2024-06-28T15:08:21.015 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.028+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=6AKKOWLO1IF2 type=del 2024-06-28T15:08:21.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.032+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=HGO0JF6469QE type=del size=253185 2024-06-28T15:08:21.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.032+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=E0O12BYE0LM1 type=del 2024-06-28T15:08:21.025 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.036+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~P3XS1U3WK2TO.0 size=1048576 2024-06-28T15:08:21.026 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.040+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~P3XS1U3WK2TO.1 size=457379 2024-06-28T15:08:21.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.040+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=P3XS1U3WK2TO tag=Z0IK1G4HZDTX type=write size=2554531 2024-06-28T15:08:21.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.044+0000 7f65680598c0 1 < suggested remove operation key=C5C6SAHLPZV1 tag=E3ZF1I9RNYRW type=write 2024-06-28T15:08:21.036 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.048+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~XXS1TRIZKXME.0 size=1048576 2024-06-28T15:08:21.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.052+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~XXS1TRIZKXME.1 size=1048576 2024-06-28T15:08:21.040 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.052+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~XXS1TRIZKXME.2 size=645631 2024-06-28T15:08:21.040 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.052+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=XXS1TRIZKXME tag=OFB1RTTI7B1C type=write size=3791359 2024-06-28T15:08:21.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.056+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~WNUK4710XOPS.0 size=1048576 2024-06-28T15:08:21.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.056+0000 7f65680598c0 1 < canceled part key=_multipart_MQ3QWZBFN4YR.2~WNUK4710XOPS.1 size=1030740 2024-06-28T15:08:21.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.060+0000 7f65680598c0 1 < canceled multipart upload key=MQ3QWZBFN4YR upload=WNUK4710XOPS tag=08ZOK3UQRWMV type=write size=3127892 2024-06-28T15:08:21.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.060+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=47VNYB011F71 type=del size=943045 2024-06-28T15:08:21.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.064+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=KNDU77SQ7YKG type=del 2024-06-28T15:08:21.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.068+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~QA8FOZPS2VCD.0 size=1014807 2024-06-28T15:08:21.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.068+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=QA8FOZPS2VCD tag=CMGVQ18W72LQ type=write size=2063383 2024-06-28T15:08:21.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.068+0000 7f65680598c0 1 < canceled part key=_multipart_9MQKWSA33R5G.2~43KKID7HOW48.0 size=935323 2024-06-28T15:08:21.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.072+0000 7f65680598c0 1 < canceled multipart upload key=9MQKWSA33R5G upload=43KKID7HOW48 tag=L0QW4GEQJMGN type=write size=1983899 2024-06-28T15:08:21.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.076+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~S5PYC69I8NCJ.0 size=904905 2024-06-28T15:08:21.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.076+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=S5PYC69I8NCJ tag=3JZAVYWHGFF2 type=write size=1953481 2024-06-28T15:08:21.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.080+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~XNRYCYA9L3CQ.0 size=1048576 2024-06-28T15:08:21.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.084+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~XNRYCYA9L3CQ.1 size=1048576 2024-06-28T15:08:21.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.084+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~XNRYCYA9L3CQ.2 size=548989 2024-06-28T15:08:21.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.088+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=XNRYCYA9L3CQ tag=FE7ZGA1PWMDZ type=write size=3694717 2024-06-28T15:08:21.078 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.092+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=BFYM8EEJAH6A type=del size=230282 2024-06-28T15:08:21.080 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.092+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~LL68CREYOED0.0 size=1048576 2024-06-28T15:08:21.082 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.096+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~LL68CREYOED0.1 size=1048576 2024-06-28T15:08:21.082 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.096+0000 7f65680598c0 1 < canceled part key=_multipart_UP3XFF38IJXN.2~LL68CREYOED0.2 size=102672 2024-06-28T15:08:21.086 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.100+0000 7f65680598c0 1 < canceled multipart upload key=UP3XFF38IJXN upload=LL68CREYOED0 tag=5S3KFJBCD6IJ type=write size=3248400 2024-06-28T15:08:21.088 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.100+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=1WPKLXH4CVY5 type=write size=973832 2024-06-28T15:08:21.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.104+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=I6TMYOQ7IUPA type=del 2024-06-28T15:08:21.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.108+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=KJ7BLNRHHELD type=del size=2712247 2024-06-28T15:08:21.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.112+0000 7f65680598c0 1 < canceled operation key=ZFNIA3O84EB7 tag=MJRXGLMZHWSJ type=del size=949866 2024-06-28T15:08:21.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.116+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=BXAA8US4EGRI type=del size=2426702 2024-06-28T15:08:21.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.116+0000 7f65680598c0 1 < canceled operation key=BHW2VHUT0U1U tag=OAA4PY9MAWH9 type=del size=3228587 2024-06-28T15:08:21.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.120+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=QU2XLF1W0PU4 type=write size=618328 2024-06-28T15:08:21.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.120+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=RL37WOQZG34L type=del 2024-06-28T15:08:21.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.124+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~CRJHOZOL5L73.0 size=480163 2024-06-28T15:08:21.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.124+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=CRJHOZOL5L73 tag=RTPDM5XDFEPI type=write size=1528739 2024-06-28T15:08:21.115 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.128+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=HOMW2JZ3N533 type=del 2024-06-28T15:08:21.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.128+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=7JW10V4KO3SO type=del size=916934 2024-06-28T15:08:21.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.132+0000 7f65680598c0 1 < canceled operation key=P278ZM1H052S tag=41YK734PV8MK type=del size=2133355 2024-06-28T15:08:21.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.136+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=6ON3PV4T8Q02 type=del size=284861 2024-06-28T15:08:21.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.136+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~P8B6QFSIF5XW.0 size=1048576 2024-06-28T15:08:21.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.140+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~P8B6QFSIF5XW.1 size=1048576 2024-06-28T15:08:21.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.140+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~P8B6QFSIF5XW.2 size=811779 2024-06-28T15:08:21.132 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.144+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=P8B6QFSIF5XW tag=9LSO53IDUWCB type=write size=3957507 2024-06-28T15:08:21.134 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.148+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=6MRILZQAO3FS type=del size=3337515 2024-06-28T15:08:21.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.148+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=2OEWRUBXYW53 type=del 2024-06-28T15:08:21.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.152+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=UFN6Y4T5KQ4B type=del size=2191317 2024-06-28T15:08:21.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.156+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=HGO0JF6469QE type=del 2024-06-28T15:08:21.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.156+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=2HYGIKZ0EZ8X type=del size=2350959 2024-06-28T15:08:21.146 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.160+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~P3XS1U3WK2TO.0 size=1048576 2024-06-28T15:08:21.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.160+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~P3XS1U3WK2TO.1 size=457379 2024-06-28T15:08:21.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.168+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=P3XS1U3WK2TO tag=Z0IK1G4HZDTX type=write size=2554531 2024-06-28T15:08:21.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.168+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=O9Q6SVD3B8LV type=del size=372083 2024-06-28T15:08:21.157 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.168+0000 7f65680598c0 1 < canceled part key=_multipart_60QVJVV16QFM.2~XXS1TRIZKXME.0 size=1048576 2024-06-28T15:08:21.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.172+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~XXS1TRIZKXME.1 size=1048576 2024-06-28T15:08:21.162 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.176+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~XXS1TRIZKXME.2 size=645631 2024-06-28T15:08:21.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.180+0000 7f65680598c0 1 < canceled multipart upload key=60QVJVV16QFM upload=XXS1TRIZKXME tag=OFB1RTTI7B1C type=write size=3791359 2024-06-28T15:08:21.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.184+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~ZGHEAZ0WFBGK.0 size=1048576 2024-06-28T15:08:21.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.188+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~ZGHEAZ0WFBGK.1 size=872224 2024-06-28T15:08:21.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.188+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=ZGHEAZ0WFBGK tag=CB2ESN74A3O1 type=write size=2969376 2024-06-28T15:08:21.177 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.188+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=47VNYB011F71 type=del 2024-06-28T15:08:21.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.196+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~M3RBO34RJF67.0 size=215963 2024-06-28T15:08:21.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.196+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=M3RBO34RJF67 tag=266QDYN9J0X3 type=write size=1264539 2024-06-28T15:08:21.185 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.196+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~QA8FOZPS2VCD.0 size=1014807 2024-06-28T15:08:21.190 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.204+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=QA8FOZPS2VCD tag=CMGVQ18W72LQ type=write size=2063383 2024-06-28T15:08:21.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.204+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=Y3RC9ZL48H04 type=del size=3519998 2024-06-28T15:08:21.196 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.208+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~S5PYC69I8NCJ.0 size=904905 2024-06-28T15:08:21.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.212+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=S5PYC69I8NCJ tag=3JZAVYWHGFF2 type=write size=1953481 2024-06-28T15:08:21.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.216+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=BSLWKV3VJ9WK type=write size=392207 2024-06-28T15:08:21.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.220+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=BFYM8EEJAH6A type=del 2024-06-28T15:08:21.210 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.224+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~B9Y72Z1GHHDW.0 size=921405 2024-06-28T15:08:21.210 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.224+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=B9Y72Z1GHHDW tag=CB5YFPAQ3NF3 type=write size=1969981 2024-06-28T15:08:21.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.228+0000 7f65680598c0 1 < completed write operation key=ZY973GBGBQ1Z tag=1WPKLXH4CVY5 type=write size=973832 2024-06-28T15:08:21.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.228+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=KM04VD2GHGH5 type=write size=1001743 2024-06-28T15:08:21.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.236+0000 7f65680598c0 1 < suggested remove operation key=KFHGA64L1NLP tag=KJ7BLNRHHELD type=del 2024-06-28T15:08:21.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.240+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~WUSJ3UH35F33.0 size=1048576 2024-06-28T15:08:21.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.244+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~WUSJ3UH35F33.1 size=1048576 2024-06-28T15:08:21.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.244+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~WUSJ3UH35F33.2 size=940267 2024-06-28T15:08:21.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.244+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=WUSJ3UH35F33 tag=AHIGOE8SZAHP type=write size=4085995 2024-06-28T15:08:21.235 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.248+0000 7f65680598c0 1 < canceled operation key=MQ3QWZBFN4YR tag=BXAA8US4EGRI type=del size=2426702 2024-06-28T15:08:21.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.252+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~63PYEP3RSKTJ.0 size=253120 2024-06-28T15:08:21.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.252+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=63PYEP3RSKTJ tag=5KCRNMFAXR4B type=write size=1301696 2024-06-28T15:08:21.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.256+0000 7f65680598c0 1 < completed write operation key=P278ZM1H052S tag=QU2XLF1W0PU4 type=write size=618328 2024-06-28T15:08:21.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.260+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=YUALDLYW1177 type=del size=1950751 2024-06-28T15:08:21.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.264+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~CRJHOZOL5L73.0 size=480163 2024-06-28T15:08:21.254 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.268+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=CRJHOZOL5L73 tag=RTPDM5XDFEPI type=write size=1528739 2024-06-28T15:08:21.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.272+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=MWPIOXE7T1AQ type=del size=4053969 2024-06-28T15:08:21.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.272+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=7JW10V4KO3SO type=del 2024-06-28T15:08:21.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.276+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=CROLXN8LJ6BC type=del size=1974224 2024-06-28T15:08:21.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.280+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=6ON3PV4T8Q02 type=del 2024-06-28T15:08:21.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.280+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=K6UHKSXHZE0S type=del size=549440 2024-06-28T15:08:21.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.284+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=6MRILZQAO3FS type=del 2024-06-28T15:08:21.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.288+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=F25OGI9CAI5P type=del size=3723644 2024-06-28T15:08:21.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.292+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=UFN6Y4T5KQ4B type=del 2024-06-28T15:08:21.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.292+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=OPWBP6ULCUR2 type=del size=1792819 2024-06-28T15:08:21.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.296+0000 7f65680598c0 1 < completed delete operation key=ZFNIA3O84EB7 tag=2HYGIKZ0EZ8X type=del 2024-06-28T15:08:21.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.300+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=5ZUY5ZIC1G37 type=del size=3881678 2024-06-28T15:08:21.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.304+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=O9Q6SVD3B8LV type=del 2024-06-28T15:08:21.293 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.304+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=OEW51DNKPY3H type=del size=2323886 2024-06-28T15:08:21.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.308+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~ZGHEAZ0WFBGK.0 size=1048576 2024-06-28T15:08:21.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.312+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~ZGHEAZ0WFBGK.1 size=872224 2024-06-28T15:08:21.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.316+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=ZGHEAZ0WFBGK tag=CB2ESN74A3O1 type=write size=2969376 2024-06-28T15:08:21.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.320+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=MCZU4W63TQ49 type=del size=213707 2024-06-28T15:08:21.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.320+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~M3RBO34RJF67.0 size=215963 2024-06-28T15:08:21.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.328+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=M3RBO34RJF67 tag=266QDYN9J0X3 type=write size=1264539 2024-06-28T15:08:21.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.332+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=7R09UZ5KMA3T type=write size=626840 2024-06-28T15:08:21.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.332+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=Y3RC9ZL48H04 type=del 2024-06-28T15:08:21.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.336+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=OPBL2BKXJQ1C type=del size=3980491 2024-06-28T15:08:21.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.340+0000 7f65680598c0 1 < canceled operation key=O1CGONN4TEXF tag=BSLWKV3VJ9WK type=write size=392207 2024-06-28T15:08:21.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.344+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=8TPFIMRNV2FN type=del size=82641 2024-06-28T15:08:21.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.348+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~B9Y72Z1GHHDW.0 size=921405 2024-06-28T15:08:21.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.352+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=B9Y72Z1GHHDW tag=CB5YFPAQ3NF3 type=write size=1969981 2024-06-28T15:08:21.345 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.360+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~3PGGU6M9L39N.0 size=1048576 2024-06-28T15:08:21.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.360+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~3PGGU6M9L39N.1 size=658681 2024-06-28T15:08:21.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.360+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=3PGGU6M9L39N tag=JFOAQS2X3RG2 type=write size=2755833 2024-06-28T15:08:21.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.364+0000 7f65680598c0 1 < completed write operation key=I4RWCQSQVLCN tag=KM04VD2GHGH5 type=write size=1001743 2024-06-28T15:08:21.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.368+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=EG0KQFWTNORD type=del size=2630634 2024-06-28T15:08:21.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.372+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~WUSJ3UH35F33.0 size=1048576 2024-06-28T15:08:21.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.372+0000 7f65680598c0 1 < canceled part key=_multipart_2FC9DNEZGWC4.2~WUSJ3UH35F33.1 size=1048576 2024-06-28T15:08:21.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.372+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~WUSJ3UH35F33.2 size=940267 2024-06-28T15:08:21.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.384+0000 7f65680598c0 1 < canceled multipart upload key=2FC9DNEZGWC4 upload=WUSJ3UH35F33 tag=AHIGOE8SZAHP type=write size=4085995 2024-06-28T15:08:21.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.384+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=LOUOHTO36WQE type=del size=760374 2024-06-28T15:08:21.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.388+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~63PYEP3RSKTJ.0 size=253120 2024-06-28T15:08:21.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.392+0000 7f65680598c0 1 < completed multipart upload key=MQ3QWZBFN4YR upload=63PYEP3RSKTJ tag=5KCRNMFAXR4B type=write size=1301696 2024-06-28T15:08:21.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.400+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~FRWBI861OXNJ.0 size=570415 2024-06-28T15:08:21.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.400+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=FRWBI861OXNJ tag=M1QB738MC4PW type=write size=1618991 2024-06-28T15:08:21.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.404+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=YUALDLYW1177 type=del 2024-06-28T15:08:21.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.408+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=3ZPSDF1A32DL type=del size=620409 2024-06-28T15:08:21.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.412+0000 7f65680598c0 1 < suggested update operation key=2F968YIMJN9D tag=MWPIOXE7T1AQ type=del size=4053969 2024-06-28T15:08:21.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.416+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=2VDWYSOPSPK2 type=write size=927251 2024-06-28T15:08:21.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.420+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=CROLXN8LJ6BC type=del 2024-06-28T15:08:21.409 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.420+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=T0PNOFXLONGA type=del size=3285967 2024-06-28T15:08:21.412 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.424+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=K6UHKSXHZE0S type=del 2024-06-28T15:08:21.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.428+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=YECO51JPM299 type=del size=2678674 2024-06-28T15:08:21.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.432+0000 7f65680598c0 1 < completed delete operation key=1VD7NDPEM3CD tag=F25OGI9CAI5P type=del 2024-06-28T15:08:21.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.436+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~GYJFN0DWP385.0 size=1048576 2024-06-28T15:08:21.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.440+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~GYJFN0DWP385.1 size=1048576 2024-06-28T15:08:21.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.440+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~GYJFN0DWP385.2 size=953633 2024-06-28T15:08:21.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.440+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=GYJFN0DWP385 tag=0YCMXPS0AUS1 type=write size=4099361 2024-06-28T15:08:21.431 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.444+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=OPWBP6ULCUR2 type=del 2024-06-28T15:08:21.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.448+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~K9W8HE5XV5BC.0 size=1048576 2024-06-28T15:08:21.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.452+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~K9W8HE5XV5BC.1 size=826003 2024-06-28T15:08:21.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.452+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=K9W8HE5XV5BC tag=DC8456H9O9II type=write size=2923155 2024-06-28T15:08:21.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.456+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=5ZUY5ZIC1G37 type=del 2024-06-28T15:08:21.447 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.460+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~G174RVEV10PX.0 size=1048576 2024-06-28T15:08:21.449 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.460+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~G174RVEV10PX.1 size=1048576 2024-06-28T15:08:21.451 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.464+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~G174RVEV10PX.2 size=615062 2024-06-28T15:08:21.451 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.464+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=G174RVEV10PX tag=MGUOCVJ9A343 type=write size=3760790 2024-06-28T15:08:21.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.468+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=OEW51DNKPY3H type=del 2024-06-28T15:08:21.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.472+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=J0B0HT9I36V3 type=del size=3588491 2024-06-28T15:08:21.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.472+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=MCZU4W63TQ49 type=del 2024-06-28T15:08:21.466 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.480+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~W1S0J8UOILV6.0 size=483837 2024-06-28T15:08:21.466 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.480+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=W1S0J8UOILV6 tag=7RZROYY28GFW type=write size=1532413 2024-06-28T15:08:21.468 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.480+0000 7f65680598c0 -1 < failed to complete operation key=2F968YIMJN9D tag=7R09UZ5KMA3T type=write size=626840: (22) Invalid argument 2024-06-28T15:08:21.474 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.488+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~IQY81WFM8FRF.0 size=651614 2024-06-28T15:08:21.474 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.488+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=IQY81WFM8FRF tag=W3JTKIFYWWHJ type=write size=1700190 2024-06-28T15:08:21.478 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.488+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=OPBL2BKXJQ1C type=del 2024-06-28T15:08:21.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.492+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=ROD9Q2DARV7S type=del size=2034399 2024-06-28T15:08:21.484 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.496+0000 7f65680598c0 1 < suggested update operation key=P278ZM1H052S tag=8TPFIMRNV2FN type=del size=82641 2024-06-28T15:08:21.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.500+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~TAB34ZM91FI3.0 size=1048576 2024-06-28T15:08:21.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.504+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~TAB34ZM91FI3.1 size=218422 2024-06-28T15:08:21.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.504+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=TAB34ZM91FI3 tag=G1PDHQ2UNB88 type=write size=2315574 2024-06-28T15:08:21.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.508+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~3PGGU6M9L39N.0 size=1048576 2024-06-28T15:08:21.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.508+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~3PGGU6M9L39N.1 size=658681 2024-06-28T15:08:21.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.516+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=3PGGU6M9L39N tag=JFOAQS2X3RG2 type=write size=2755833 2024-06-28T15:08:21.505 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.520+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=X12YEBNQUCGW type=del size=1025034 2024-06-28T15:08:21.509 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.520+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=EG0KQFWTNORD type=del 2024-06-28T15:08:21.512 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.524+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=DN5N8QLSKLK9 type=del size=3814129 2024-06-28T15:08:21.516 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.528+0000 7f65680598c0 1 < completed delete operation key=1VD7NDPEM3CD tag=LOUOHTO36WQE type=del 2024-06-28T15:08:21.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.536+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~2JCCF9FXJP04.0 size=1048576 2024-06-28T15:08:21.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.536+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~2JCCF9FXJP04.1 size=1048576 2024-06-28T15:08:21.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.540+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~2JCCF9FXJP04.2 size=139109 2024-06-28T15:08:21.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.540+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=2JCCF9FXJP04 tag=DCMS9P743QVA type=write size=3284837 2024-06-28T15:08:21.530 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.544+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~FRWBI861OXNJ.0 size=570415 2024-06-28T15:08:21.535 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.548+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=FRWBI861OXNJ tag=M1QB738MC4PW type=write size=1618991 2024-06-28T15:08:21.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.552+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~ESRL3CG96C6V.0 size=1048576 2024-06-28T15:08:21.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.556+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~ESRL3CG96C6V.1 size=1020413 2024-06-28T15:08:21.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.556+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=ESRL3CG96C6V tag=O1IB7V19M9EG type=write size=3117565 2024-06-28T15:08:21.545 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.556+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=3ZPSDF1A32DL type=del 2024-06-28T15:08:21.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.564+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=9C7FFYX9RUKG type=del size=3908717 2024-06-28T15:08:21.553 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.564+0000 7f65680598c0 1 < canceled operation key=BHW2VHUT0U1U tag=2VDWYSOPSPK2 type=write size=927251 2024-06-28T15:08:21.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.568+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=R63ICID69BB0 type=del size=4154261 2024-06-28T15:08:21.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.572+0000 7f65680598c0 1 < canceled operation key=OUID7RX5NVV6 tag=T0PNOFXLONGA type=del size=3285967 2024-06-28T15:08:21.561 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.576+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=P8W0PSMB0XK5 type=del size=121443 2024-06-28T15:08:21.564 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.576+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=YECO51JPM299 type=del 2024-06-28T15:08:21.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.584+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~AO5AOAULAF7B.0 size=1048576 2024-06-28T15:08:21.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.584+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~AO5AOAULAF7B.1 size=1048576 2024-06-28T15:08:21.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.588+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~AO5AOAULAF7B.2 size=314095 2024-06-28T15:08:21.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.588+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=AO5AOAULAF7B tag=OUWPZC022YJ4 type=write size=3459823 2024-06-28T15:08:21.576 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.588+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~GYJFN0DWP385.0 size=1048576 2024-06-28T15:08:21.578 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.592+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~GYJFN0DWP385.1 size=1048576 2024-06-28T15:08:21.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.592+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~GYJFN0DWP385.2 size=953633 2024-06-28T15:08:21.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.600+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=GYJFN0DWP385 tag=0YCMXPS0AUS1 type=write size=4099361 2024-06-28T15:08:21.588 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.600+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=2JWD2WA49UJ3 type=del size=1640362 2024-06-28T15:08:21.591 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.604+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~K9W8HE5XV5BC.0 size=1048576 2024-06-28T15:08:21.593 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.608+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~K9W8HE5XV5BC.1 size=826003 2024-06-28T15:08:21.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.608+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=K9W8HE5XV5BC tag=DC8456H9O9II type=write size=2923155 2024-06-28T15:08:21.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.612+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=58TZAM9V25QB type=del size=926967 2024-06-28T15:08:21.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.616+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~G174RVEV10PX.0 size=1048576 2024-06-28T15:08:21.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.616+0000 7f65680598c0 1 < canceled part key=_multipart_F6SEDD6BH1T2.2~G174RVEV10PX.1 size=1048576 2024-06-28T15:08:21.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.616+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~G174RVEV10PX.2 size=615062 2024-06-28T15:08:21.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.620+0000 7f65680598c0 1 < canceled multipart upload key=F6SEDD6BH1T2 upload=G174RVEV10PX tag=MGUOCVJ9A343 type=write size=3760790 2024-06-28T15:08:21.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.624+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~CQ008AB7Y18G.0 size=177667 2024-06-28T15:08:21.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.624+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=CQ008AB7Y18G tag=BLFBJV7FGY6W type=write size=1226243 2024-06-28T15:08:21.615 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.628+0000 7f65680598c0 1 < canceled operation key=V2L2Q1U62YW2 tag=J0B0HT9I36V3 type=del size=3588491 2024-06-28T15:08:21.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.632+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=L9YWOC0B3MOG type=del size=781497 2024-06-28T15:08:21.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.632+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~W1S0J8UOILV6.0 size=483837 2024-06-28T15:08:21.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.636+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=W1S0J8UOILV6 tag=7RZROYY28GFW type=write size=1532413 2024-06-28T15:08:21.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.640+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=LFVDJZ3W6XL2 type=del size=1042622 2024-06-28T15:08:21.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.640+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~IQY81WFM8FRF.0 size=651614 2024-06-28T15:08:21.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.644+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=IQY81WFM8FRF tag=W3JTKIFYWWHJ type=write size=1700190 2024-06-28T15:08:21.638 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.652+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=Y2X4YYUYT1KJ type=write size=219991 2024-06-28T15:08:21.640 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.652+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=ROD9Q2DARV7S type=del 2024-06-28T15:08:21.642 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.656+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=H4BY23G1XD65 type=del size=3924235 2024-06-28T15:08:21.643 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.656+0000 7f65680598c0 1 < canceled part key=_multipart_MQ3QWZBFN4YR.2~TAB34ZM91FI3.0 size=1048576 2024-06-28T15:08:21.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.656+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~TAB34ZM91FI3.1 size=218422 2024-06-28T15:08:21.648 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.660+0000 7f65680598c0 1 < canceled multipart upload key=MQ3QWZBFN4YR upload=TAB34ZM91FI3 tag=G1PDHQ2UNB88 type=write size=2315574 2024-06-28T15:08:21.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.664+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=5TC3EM1L3UTN type=del size=3938144 2024-06-28T15:08:21.653 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.664+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=X12YEBNQUCGW type=del 2024-06-28T15:08:21.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.668+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=TLVGBB6UAZEK type=del size=2374435 2024-06-28T15:08:21.657 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.672+0000 7f65680598c0 1 < canceled operation key=9MQKWSA33R5G tag=DN5N8QLSKLK9 type=del size=3814129 2024-06-28T15:08:21.662 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.676+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~ALVA2JROC7VX.0 size=1048576 2024-06-28T15:08:21.663 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.676+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~ALVA2JROC7VX.1 size=1048576 2024-06-28T15:08:21.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.676+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~ALVA2JROC7VX.2 size=908031 2024-06-28T15:08:21.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.676+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=ALVA2JROC7VX tag=2HO53VTFIK3M type=write size=4053759 2024-06-28T15:08:21.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.680+0000 7f65680598c0 1 < canceled part key=_multipart_1VD7NDPEM3CD.2~2JCCF9FXJP04.0 size=1048576 2024-06-28T15:08:21.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.680+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~2JCCF9FXJP04.1 size=1048576 2024-06-28T15:08:21.669 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.684+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~2JCCF9FXJP04.2 size=139109 2024-06-28T15:08:21.673 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.684+0000 7f65680598c0 1 < canceled multipart upload key=1VD7NDPEM3CD upload=2JCCF9FXJP04 tag=DCMS9P743QVA type=write size=3284837 2024-06-28T15:08:21.675 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.688+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=9KRHMLZ7UUOE type=write size=5277 2024-06-28T15:08:21.677 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.688+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~ESRL3CG96C6V.0 size=1048576 2024-06-28T15:08:21.682 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.692+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~ESRL3CG96C6V.1 size=1020413 2024-06-28T15:08:21.683 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.696+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=ESRL3CG96C6V tag=O1IB7V19M9EG type=write size=3117565 2024-06-28T15:08:21.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.700+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~KTGMXICU7LBX.0 size=1048576 2024-06-28T15:08:21.689 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.704+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~KTGMXICU7LBX.1 size=1048576 2024-06-28T15:08:21.691 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.704+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~KTGMXICU7LBX.2 size=1046864 2024-06-28T15:08:21.691 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.704+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=KTGMXICU7LBX tag=6SMWUGQUHAGQ type=write size=4192592 2024-06-28T15:08:21.694 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.708+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=9C7FFYX9RUKG type=del 2024-06-28T15:08:21.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.712+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~EXR27B7YCLDX.0 size=682508 2024-06-28T15:08:21.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.712+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=EXR27B7YCLDX tag=ZJKIREH2QUN6 type=write size=1731084 2024-06-28T15:08:21.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.716+0000 7f65680598c0 1 < canceled operation key=7NL3CTFYY9E5 tag=R63ICID69BB0 type=del size=4154261 2024-06-28T15:08:21.709 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.720+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~XCHEWJK2KM7F.0 size=1048576 2024-06-28T15:08:21.711 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.724+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~XCHEWJK2KM7F.1 size=161283 2024-06-28T15:08:21.711 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.724+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=XCHEWJK2KM7F tag=5OJKEQF63FAE type=write size=2258435 2024-06-28T15:08:21.714 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.728+0000 7f65680598c0 1 < canceled operation key=47TO7G9EGJKF tag=P8W0PSMB0XK5 type=del size=121443 2024-06-28T15:08:21.716 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.728+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=54WIY68QAIEJ type=write size=215573 2024-06-28T15:08:21.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.732+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~AO5AOAULAF7B.0 size=1048576 2024-06-28T15:08:21.721 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.736+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~AO5AOAULAF7B.1 size=1048576 2024-06-28T15:08:21.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.736+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~AO5AOAULAF7B.2 size=314095 2024-06-28T15:08:21.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.740+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=AO5AOAULAF7B tag=OUWPZC022YJ4 type=write size=3459823 2024-06-28T15:08:21.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.744+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=XJ8GR7PO2LIP type=write size=771455 2024-06-28T15:08:21.737 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.748+0000 7f65680598c0 1 < suggested remove operation key=7NL3CTFYY9E5 tag=2JWD2WA49UJ3 type=del 2024-06-28T15:08:21.740 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.752+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=AA1ZWNVARB14 type=write size=405952 2024-06-28T15:08:21.743 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.756+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=58TZAM9V25QB type=del 2024-06-28T15:08:21.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.760+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=RWS8N6BKJ932 type=write size=196447 2024-06-28T15:08:21.749 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.764+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~CQ008AB7Y18G.0 size=177667 2024-06-28T15:08:21.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.768+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=CQ008AB7Y18G tag=BLFBJV7FGY6W type=write size=1226243 2024-06-28T15:08:21.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.768+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=GBJQAMD3TOFI type=del size=3794422 2024-06-28T15:08:21.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.772+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=L9YWOC0B3MOG type=del 2024-06-28T15:08:21.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.776+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~GFCR9HQAE2O2.0 size=1048576 2024-06-28T15:08:21.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.780+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~GFCR9HQAE2O2.1 size=13015 2024-06-28T15:08:21.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.780+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=GFCR9HQAE2O2 tag=ZD1ZKBBE8V7V type=write size=2110167 2024-06-28T15:08:21.773 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.784+0000 7f65680598c0 1 < suggested update operation key=60QVJVV16QFM tag=LFVDJZ3W6XL2 type=del size=1042622 2024-06-28T15:08:21.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.792+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~BAS8QW3HPHR3.0 size=1048576 2024-06-28T15:08:21.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.792+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~BAS8QW3HPHR3.1 size=663302 2024-06-28T15:08:21.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.792+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=BAS8QW3HPHR3 tag=VCDE3ESSD0UP type=write size=2760454 2024-06-28T15:08:21.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.796+0000 7f65680598c0 1 < completed write operation key=UP3XFF38IJXN tag=Y2X4YYUYT1KJ type=write size=219991 2024-06-28T15:08:21.787 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.800+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=IGSV0Y3Q0MLT type=del size=41785 2024-06-28T15:08:21.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.804+0000 7f65680598c0 1 < suggested update operation key=C5C6SAHLPZV1 tag=H4BY23G1XD65 type=del size=3924235 2024-06-28T15:08:21.795 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.808+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=7VZNPH0IS80I type=write size=944463 2024-06-28T15:08:21.800 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.812+0000 7f65680598c0 1 < suggested update operation key=ZY973GBGBQ1Z tag=5TC3EM1L3UTN type=del size=3938144 2024-06-28T15:08:21.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.816+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=R0CIWCDFASG8 type=del size=985309 2024-06-28T15:08:21.806 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.820+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=TLVGBB6UAZEK type=del 2024-06-28T15:08:21.808 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.820+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=VUAVY82LQ5AE type=del size=3031714 2024-06-28T15:08:21.811 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.824+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~ALVA2JROC7VX.0 size=1048576 2024-06-28T15:08:21.813 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.828+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~ALVA2JROC7VX.1 size=1048576 2024-06-28T15:08:21.816 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.828+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~ALVA2JROC7VX.2 size=908031 2024-06-28T15:08:21.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.832+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=ALVA2JROC7VX tag=2HO53VTFIK3M type=write size=4053759 2024-06-28T15:08:21.823 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.836+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=FJL57EC9IDLU type=del size=973535 2024-06-28T15:08:21.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.840+0000 7f65680598c0 -1 < failed to complete operation key=ZY973GBGBQ1Z tag=9KRHMLZ7UUOE type=write size=5277: (22) Invalid argument 2024-06-28T15:08:21.829 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.840+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=ALSNOGRZT33P type=write size=1039582 2024-06-28T15:08:21.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.844+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~KTGMXICU7LBX.0 size=1048576 2024-06-28T15:08:21.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.844+0000 7f65680598c0 1 < canceled part key=_multipart_1VD7NDPEM3CD.2~KTGMXICU7LBX.1 size=1048576 2024-06-28T15:08:21.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.848+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~KTGMXICU7LBX.2 size=1046864 2024-06-28T15:08:21.839 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.852+0000 7f65680598c0 1 < canceled multipart upload key=1VD7NDPEM3CD upload=KTGMXICU7LBX tag=6SMWUGQUHAGQ type=write size=4192592 2024-06-28T15:08:21.845 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.856+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~EARTZZFRIN2U.0 size=593321 2024-06-28T15:08:21.845 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.856+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=EARTZZFRIN2U tag=CIPBEV0DNIYC type=write size=1641897 2024-06-28T15:08:21.850 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.864+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~EXR27B7YCLDX.0 size=682508 2024-06-28T15:08:21.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.868+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=EXR27B7YCLDX tag=ZJKIREH2QUN6 type=write size=1731084 2024-06-28T15:08:21.858 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.872+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=TZZGHQNKNAX7 type=del size=3893987 2024-06-28T15:08:21.861 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.876+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~XCHEWJK2KM7F.0 size=1048576 2024-06-28T15:08:21.864 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.876+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~XCHEWJK2KM7F.1 size=161283 2024-06-28T15:08:21.869 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.880+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=XCHEWJK2KM7F tag=5OJKEQF63FAE type=write size=2258435 2024-06-28T15:08:21.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.884+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=Q90VJ05RMZGP type=write size=651847 2024-06-28T15:08:21.875 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.888+0000 7f65680598c0 1 < completed write operation key=J311Q2712YM0 tag=54WIY68QAIEJ type=write size=215573 2024-06-28T15:08:21.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.892+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~4KINX7NUHQ4K.0 size=1048576 2024-06-28T15:08:21.886 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.896+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~4KINX7NUHQ4K.1 size=1048576 2024-06-28T15:08:21.886 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.900+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~4KINX7NUHQ4K.2 size=947575 2024-06-28T15:08:21.886 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.900+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=4KINX7NUHQ4K tag=TL6WK92EYFWF type=write size=4093303 2024-06-28T15:08:21.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.904+0000 7f65680598c0 1 < suggested remove operation key=ZFNIA3O84EB7 tag=XJ8GR7PO2LIP type=write 2024-06-28T15:08:21.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.912+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=SBRPR7I1G14V type=del size=322542 2024-06-28T15:08:21.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.916+0000 7f65680598c0 1 < completed write operation key=BHW2VHUT0U1U tag=AA1ZWNVARB14 type=write size=405952 2024-06-28T15:08:21.908 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.920+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~JHW9AAWF1LEL.0 size=1048576 2024-06-28T15:08:21.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.924+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~JHW9AAWF1LEL.1 size=1048576 2024-06-28T15:08:21.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.924+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~JHW9AAWF1LEL.2 size=53600 2024-06-28T15:08:21.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.924+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=JHW9AAWF1LEL tag=1KZ2Q59T89AC type=write size=3199328 2024-06-28T15:08:21.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.928+0000 7f65680598c0 1 < completed write operation key=I4RWCQSQVLCN tag=RWS8N6BKJ932 type=write size=196447 2024-06-28T15:08:21.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.932+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~84HPCECENTJZ.0 size=352606 2024-06-28T15:08:21.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.932+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=84HPCECENTJZ tag=EBQ119GP12RE type=write size=1401182 2024-06-28T15:08:21.924 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.936+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=GBJQAMD3TOFI type=del 2024-06-28T15:08:21.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.940+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=MI4UBAOHE0F1 type=del size=3933832 2024-06-28T15:08:21.930 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.944+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~GFCR9HQAE2O2.0 size=1048576 2024-06-28T15:08:21.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.948+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~GFCR9HQAE2O2.1 size=13015 2024-06-28T15:08:21.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.952+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=GFCR9HQAE2O2 tag=ZD1ZKBBE8V7V type=write size=2110167 2024-06-28T15:08:21.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.960+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~DAWXS4OZLI9J.0 size=1048576 2024-06-28T15:08:21.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.960+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~DAWXS4OZLI9J.1 size=1048576 2024-06-28T15:08:21.951 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.964+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~DAWXS4OZLI9J.2 size=698805 2024-06-28T15:08:21.951 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.964+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=DAWXS4OZLI9J tag=IH9D7Y5T970R type=write size=3844533 2024-06-28T15:08:21.954 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.968+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~BAS8QW3HPHR3.0 size=1048576 2024-06-28T15:08:21.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.968+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~BAS8QW3HPHR3.1 size=663302 2024-06-28T15:08:21.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.976+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=BAS8QW3HPHR3 tag=VCDE3ESSD0UP type=write size=2760454 2024-06-28T15:08:21.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.980+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=45ON1ZAFWADQ type=del size=632117 2024-06-28T15:08:21.969 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.980+0000 7f65680598c0 1 < canceled operation key=KFHGA64L1NLP tag=IGSV0Y3Q0MLT type=del size=41785 2024-06-28T15:08:21.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.984+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=3YY8V6GPHP93 type=del size=688468 2024-06-28T15:08:21.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.988+0000 7f65680598c0 1 < completed write operation key=60QVJVV16QFM tag=7VZNPH0IS80I type=write size=944463 2024-06-28T15:08:21.981 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.996+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~06ITMY8T4ZOT.0 size=717501 2024-06-28T15:08:21.982 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.996+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=06ITMY8T4ZOT tag=VHKCIWRSEELF type=write size=1766077 2024-06-28T15:08:21.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:21.996+0000 7f65680598c0 1 < canceled operation key=60QVJVV16QFM tag=R0CIWCDFASG8 type=del size=985309 2024-06-28T15:08:21.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.000+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=M9G59T81TO1D type=write size=530178 2024-06-28T15:08:21.991 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.004+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=VUAVY82LQ5AE type=del 2024-06-28T15:08:21.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.008+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=1TEII15IMQ1A type=del size=535935 2024-06-28T15:08:21.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.008+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=FJL57EC9IDLU type=del 2024-06-28T15:08:22.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.016+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~JVTCT34MH45T.0 size=827273 2024-06-28T15:08:22.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.016+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=JVTCT34MH45T tag=2WM032N4AQ1W type=write size=1875849 2024-06-28T15:08:22.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.020+0000 7f65680598c0 1 < canceled operation key=1VD7NDPEM3CD tag=ALSNOGRZT33P type=write size=1039582 2024-06-28T15:08:22.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.020+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=5IQJ3VL4RPKZ type=write size=510752 2024-06-28T15:08:22.012 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.024+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~EARTZZFRIN2U.0 size=593321 2024-06-28T15:08:22.017 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.028+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=EARTZZFRIN2U tag=CIPBEV0DNIYC type=write size=1641897 2024-06-28T15:08:22.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.036+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~XC31MBS396OH.0 size=1048576 2024-06-28T15:08:22.025 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.036+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~XC31MBS396OH.1 size=1048576 2024-06-28T15:08:22.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.040+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~XC31MBS396OH.2 size=347180 2024-06-28T15:08:22.027 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.040+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=XC31MBS396OH tag=4EP6EMBWBR8J type=write size=3492908 2024-06-28T15:08:22.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.044+0000 7f65680598c0 1 < canceled operation key=RLJCNXKA2GYA tag=TZZGHQNKNAX7 type=del size=3893987 2024-06-28T15:08:22.034 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.048+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=0ZQPRRERK6LH type=write size=98306 2024-06-28T15:08:22.039 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.052+0000 7f65680598c0 1 < suggested update operation key=GAZQ3PCPKN9Q tag=Q90VJ05RMZGP type=write size=651847 2024-06-28T15:08:22.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.056+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~N33NBNUO54UI.0 size=1048576 2024-06-28T15:08:22.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.060+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~N33NBNUO54UI.1 size=45388 2024-06-28T15:08:22.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.060+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=N33NBNUO54UI tag=UU49LPZXF5GI type=write size=2142540 2024-06-28T15:08:22.050 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.064+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~4KINX7NUHQ4K.0 size=1048576 2024-06-28T15:08:22.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.064+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~4KINX7NUHQ4K.1 size=1048576 2024-06-28T15:08:22.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.068+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~4KINX7NUHQ4K.2 size=947575 2024-06-28T15:08:22.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.072+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=4KINX7NUHQ4K tag=TL6WK92EYFWF type=write size=4093303 2024-06-28T15:08:22.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.080+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~7LK4RN7AMLHV.0 size=1048576 2024-06-28T15:08:22.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.080+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~7LK4RN7AMLHV.1 size=371917 2024-06-28T15:08:22.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.080+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=7LK4RN7AMLHV tag=6FMUZXG9H66H type=write size=2469069 2024-06-28T15:08:22.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.084+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=SBRPR7I1G14V type=del 2024-06-28T15:08:22.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.088+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~5VX0IKOK6JTH.0 size=1048576 2024-06-28T15:08:22.079 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.092+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~5VX0IKOK6JTH.1 size=107833 2024-06-28T15:08:22.079 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.092+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=5VX0IKOK6JTH tag=4HW0GJCYNZJO type=write size=2204985 2024-06-28T15:08:22.082 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.096+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~JHW9AAWF1LEL.0 size=1048576 2024-06-28T15:08:22.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.096+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~JHW9AAWF1LEL.1 size=1048576 2024-06-28T15:08:22.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.100+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~JHW9AAWF1LEL.2 size=53600 2024-06-28T15:08:22.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.104+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=JHW9AAWF1LEL tag=1KZ2Q59T89AC type=write size=3199328 2024-06-28T15:08:22.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.108+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=W5PA9052QATJ type=del size=2804897 2024-06-28T15:08:22.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.112+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~84HPCECENTJZ.0 size=352606 2024-06-28T15:08:22.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.116+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=84HPCECENTJZ tag=EBQ119GP12RE type=write size=1401182 2024-06-28T15:08:22.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.120+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~AZ978MH473LC.0 size=1048576 2024-06-28T15:08:22.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.124+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~AZ978MH473LC.1 size=124858 2024-06-28T15:08:22.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.124+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=AZ978MH473LC tag=50Z7P6WL6Q9H type=write size=2222010 2024-06-28T15:08:22.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.128+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=MI4UBAOHE0F1 type=del 2024-06-28T15:08:22.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.128+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=JZUB6T5BZUX8 type=del size=3458991 2024-06-28T15:08:22.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.132+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~DAWXS4OZLI9J.0 size=1048576 2024-06-28T15:08:22.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.136+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~DAWXS4OZLI9J.1 size=1048576 2024-06-28T15:08:22.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.136+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~DAWXS4OZLI9J.2 size=698805 2024-06-28T15:08:22.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.144+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=DAWXS4OZLI9J tag=IH9D7Y5T970R type=write size=3844533 2024-06-28T15:08:22.135 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.148+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~OGHFCAI9MK9C.0 size=1048576 2024-06-28T15:08:22.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.152+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~OGHFCAI9MK9C.1 size=154349 2024-06-28T15:08:22.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.152+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=OGHFCAI9MK9C tag=8SCY4QK6OQFZ type=write size=2251501 2024-06-28T15:08:22.140 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.152+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=45ON1ZAFWADQ type=del 2024-06-28T15:08:22.143 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.156+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=ND5FKQYZ9GPO type=write size=452812 2024-06-28T15:08:22.146 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.160+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=3YY8V6GPHP93 type=del 2024-06-28T15:08:22.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.164+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~KUC3671F8KFA.0 size=1048576 2024-06-28T15:08:22.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.168+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~KUC3671F8KFA.1 size=1048576 2024-06-28T15:08:22.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.168+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~KUC3671F8KFA.2 size=960116 2024-06-28T15:08:22.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.168+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=KUC3671F8KFA tag=3F14JSG0I5MF type=write size=4105844 2024-06-28T15:08:22.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.172+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~06ITMY8T4ZOT.0 size=717501 2024-06-28T15:08:22.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.176+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=06ITMY8T4ZOT tag=VHKCIWRSEELF type=write size=1766077 2024-06-28T15:08:22.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.180+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~DR2GL68S6Q3G.0 size=1048576 2024-06-28T15:08:22.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.184+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~DR2GL68S6Q3G.1 size=950617 2024-06-28T15:08:22.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.184+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=DR2GL68S6Q3G tag=EB54OQVMBY44 type=write size=3047769 2024-06-28T15:08:22.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.184+0000 7f65680598c0 1 < completed write operation key=F6SEDD6BH1T2 tag=M9G59T81TO1D type=write size=530178 2024-06-28T15:08:22.178 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.192+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=1MYBW13GVM6E type=del size=454532 2024-06-28T15:08:22.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.196+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=1TEII15IMQ1A type=del 2024-06-28T15:08:22.185 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.196+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=IER0A2OSBM0W type=del size=3441474 2024-06-28T15:08:22.188 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.200+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~JVTCT34MH45T.0 size=827273 2024-06-28T15:08:22.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.204+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=JVTCT34MH45T tag=2WM032N4AQ1W type=write size=1875849 2024-06-28T15:08:22.195 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.208+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=KWPZUQMCU8P3 type=del size=3563507 2024-06-28T15:08:22.198 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.212+0000 7f65680598c0 1 < completed write operation key=9MQKWSA33R5G tag=5IQJ3VL4RPKZ type=write size=510752 2024-06-28T15:08:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.216+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~3RLE5MPLH5F7.0 size=1048576 2024-06-28T15:08:22.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.220+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~3RLE5MPLH5F7.1 size=665807 2024-06-28T15:08:22.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.220+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=3RLE5MPLH5F7 tag=8V0GV4O05PJ9 type=write size=2762959 2024-06-28T15:08:22.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.220+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~XC31MBS396OH.0 size=1048576 2024-06-28T15:08:22.211 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.224+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~XC31MBS396OH.1 size=1048576 2024-06-28T15:08:22.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.224+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~XC31MBS396OH.2 size=347180 2024-06-28T15:08:22.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.232+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=XC31MBS396OH tag=4EP6EMBWBR8J type=write size=3492908 2024-06-28T15:08:22.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.232+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=CFBG02DVCGJS type=del size=2368694 2024-06-28T15:08:22.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.236+0000 7f65680598c0 1 < canceled operation key=P278ZM1H052S tag=0ZQPRRERK6LH type=write size=98306 2024-06-28T15:08:22.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.240+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=2FZ5529RYGTJ type=del size=3938282 2024-06-28T15:08:22.230 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.244+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~N33NBNUO54UI.0 size=1048576 2024-06-28T15:08:22.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.244+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~N33NBNUO54UI.1 size=45388 2024-06-28T15:08:22.237 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.248+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=N33NBNUO54UI tag=UU49LPZXF5GI type=write size=2142540 2024-06-28T15:08:22.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.252+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=ZWVSPMKCIK7X type=del size=550002 2024-06-28T15:08:22.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.256+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~7LK4RN7AMLHV.0 size=1048576 2024-06-28T15:08:22.245 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.256+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~7LK4RN7AMLHV.1 size=371917 2024-06-28T15:08:22.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.264+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=7LK4RN7AMLHV tag=6FMUZXG9H66H type=write size=2469069 2024-06-28T15:08:22.253 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.264+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=1ALYEWAR7Y05 type=del size=3104522 2024-06-28T15:08:22.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.268+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~5VX0IKOK6JTH.0 size=1048576 2024-06-28T15:08:22.258 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.272+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~5VX0IKOK6JTH.1 size=107833 2024-06-28T15:08:22.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.276+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=5VX0IKOK6JTH tag=4HW0GJCYNZJO type=write size=2204985 2024-06-28T15:08:22.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.280+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=G2Z1Z4ZNTTNH type=del size=1111903 2024-06-28T15:08:22.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.280+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=W5PA9052QATJ type=del 2024-06-28T15:08:22.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.288+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~020U6M3V78QA.0 size=313134 2024-06-28T15:08:22.274 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.288+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=020U6M3V78QA tag=6RLFTXEQWPIG type=write size=1361710 2024-06-28T15:08:22.276 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.288+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~AZ978MH473LC.0 size=1048576 2024-06-28T15:08:22.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.292+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~AZ978MH473LC.1 size=124858 2024-06-28T15:08:22.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.296+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=AZ978MH473LC tag=50Z7P6WL6Q9H type=write size=2222010 2024-06-28T15:08:22.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.300+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=LDUKJPZI3ZA8 type=del size=2085445 2024-06-28T15:08:22.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.304+0000 7f65680598c0 1 < suggested remove operation key=GAZQ3PCPKN9Q tag=JZUB6T5BZUX8 type=del 2024-06-28T15:08:22.294 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.308+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~GJO9XE0LOV1D.0 size=1048576 2024-06-28T15:08:22.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.308+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~GJO9XE0LOV1D.1 size=998204 2024-06-28T15:08:22.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.308+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=GJO9XE0LOV1D tag=9VX3GAVWCQ23 type=write size=3095356 2024-06-28T15:08:22.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.312+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~OGHFCAI9MK9C.0 size=1048576 2024-06-28T15:08:22.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.312+0000 7f65680598c0 1 < canceled part key=_multipart_SFCIZMLY2Y2E.2~OGHFCAI9MK9C.1 size=154349 2024-06-28T15:08:22.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.316+0000 7f65680598c0 1 < canceled multipart upload key=SFCIZMLY2Y2E upload=OGHFCAI9MK9C tag=8SCY4QK6OQFZ type=write size=2251501 2024-06-28T15:08:22.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.320+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=D2EB64DIGNV2 type=del size=1903372 2024-06-28T15:08:22.312 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.324+0000 7f65680598c0 1 < suggested update operation key=1VD7NDPEM3CD tag=ND5FKQYZ9GPO type=write size=452812 2024-06-28T15:08:22.316 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.328+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~6QVD471KI9ZH.0 size=1048576 2024-06-28T15:08:22.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.332+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~6QVD471KI9ZH.1 size=1048576 2024-06-28T15:08:22.320 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.332+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~6QVD471KI9ZH.2 size=702831 2024-06-28T15:08:22.320 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.332+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=6QVD471KI9ZH tag=U4CW1H14JDFC type=write size=3848559 2024-06-28T15:08:22.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.336+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~KUC3671F8KFA.0 size=1048576 2024-06-28T15:08:22.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.336+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~KUC3671F8KFA.1 size=1048576 2024-06-28T15:08:22.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.340+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~KUC3671F8KFA.2 size=960116 2024-06-28T15:08:22.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.344+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=KUC3671F8KFA tag=3F14JSG0I5MF type=write size=4105844 2024-06-28T15:08:22.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.348+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=0170N066902N type=del size=592301 2024-06-28T15:08:22.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.352+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~DR2GL68S6Q3G.0 size=1048576 2024-06-28T15:08:22.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.352+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~DR2GL68S6Q3G.1 size=950617 2024-06-28T15:08:22.344 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.356+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=DR2GL68S6Q3G tag=EB54OQVMBY44 type=write size=3047769 2024-06-28T15:08:22.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.360+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~OR7HB7J3GMM0.0 size=1048576 2024-06-28T15:08:22.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.364+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~OR7HB7J3GMM0.1 size=943138 2024-06-28T15:08:22.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.364+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=OR7HB7J3GMM0 tag=DAATEUH1DGE7 type=write size=3040290 2024-06-28T15:08:22.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.368+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=1MYBW13GVM6E type=del 2024-06-28T15:08:22.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.368+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=KH9NAI3BXXOF type=del size=620090 2024-06-28T15:08:22.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.372+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=IER0A2OSBM0W type=del 2024-06-28T15:08:22.363 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.376+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~70KFP9YZGMZ4.0 size=1048576 2024-06-28T15:08:22.365 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.380+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~70KFP9YZGMZ4.1 size=1039066 2024-06-28T15:08:22.365 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.380+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=70KFP9YZGMZ4 tag=UDZVJKMNNR7W type=write size=3136218 2024-06-28T15:08:22.368 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.380+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=KWPZUQMCU8P3 type=del 2024-06-28T15:08:22.373 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.384+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~B3F9121ALRR5.0 size=63804 2024-06-28T15:08:22.373 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.384+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=B3F9121ALRR5 tag=2COME2NIOVXF type=write size=1112380 2024-06-28T15:08:22.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.388+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~3RLE5MPLH5F7.0 size=1048576 2024-06-28T15:08:22.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.392+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~3RLE5MPLH5F7.1 size=665807 2024-06-28T15:08:22.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.396+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=3RLE5MPLH5F7 tag=8V0GV4O05PJ9 type=write size=2762959 2024-06-28T15:08:22.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.400+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=ZIYCU9IRMNY8 type=del size=1954495 2024-06-28T15:08:22.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.400+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=CFBG02DVCGJS type=del 2024-06-28T15:08:22.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.408+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~UCYTQOR0S1W9.0 size=1048576 2024-06-28T15:08:22.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.408+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~UCYTQOR0S1W9.1 size=390427 2024-06-28T15:08:22.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.408+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=UCYTQOR0S1W9 tag=VYY4OV6SJ2IL type=write size=2487579 2024-06-28T15:08:22.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.412+0000 7f65680598c0 1 < completed delete operation key=ZFNIA3O84EB7 tag=2FZ5529RYGTJ type=del 2024-06-28T15:08:22.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.416+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~7M5J07HRX7MW.0 size=311458 2024-06-28T15:08:22.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.416+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=7M5J07HRX7MW tag=8LTMLZJU1HT4 type=write size=1360034 2024-06-28T15:08:22.411 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.424+0000 7f65680598c0 1 < suggested update operation key=2FC9DNEZGWC4 tag=ZWVSPMKCIK7X type=del size=550002 2024-06-28T15:08:22.414 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.428+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=0WHHUXLTKO4U type=write size=652787 2024-06-28T15:08:22.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.428+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=1ALYEWAR7Y05 type=del 2024-06-28T15:08:22.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.432+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=GH4EX20SLVPA type=write size=538372 2024-06-28T15:08:22.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.436+0000 7f65680598c0 1 < canceled operation key=SFCIZMLY2Y2E tag=G2Z1Z4ZNTTNH type=del size=1111903 2024-06-28T15:08:22.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.440+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=L9SH4M49Q9GQ type=del size=1533403 2024-06-28T15:08:22.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.440+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~020U6M3V78QA.0 size=313134 2024-06-28T15:08:22.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.448+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=020U6M3V78QA tag=6RLFTXEQWPIG type=write size=1361710 2024-06-28T15:08:22.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.452+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=0N5A7RS8WD7U type=del size=543888 2024-06-28T15:08:22.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.456+0000 7f65680598c0 1 < suggested update operation key=EN76QQWIQAXZ tag=LDUKJPZI3ZA8 type=del size=2085445 2024-06-28T15:08:22.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.460+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~22Q2EEUEAITK.0 size=1048576 2024-06-28T15:08:22.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.464+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~22Q2EEUEAITK.1 size=129112 2024-06-28T15:08:22.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.464+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=22Q2EEUEAITK tag=IJQHCR6NZVYQ type=write size=2226264 2024-06-28T15:08:22.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.464+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~GJO9XE0LOV1D.0 size=1048576 2024-06-28T15:08:22.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.468+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~GJO9XE0LOV1D.1 size=998204 2024-06-28T15:08:22.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.472+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=GJO9XE0LOV1D tag=9VX3GAVWCQ23 type=write size=3095356 2024-06-28T15:08:22.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.472+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=QMZVZA9B17Y5 type=del size=1897474 2024-06-28T15:08:22.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.476+0000 7f65680598c0 -1 < failed to complete operation key=EN76QQWIQAXZ tag=D2EB64DIGNV2 type=del size=1903372: (22) Invalid argument 2024-06-28T15:08:22.466 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.480+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=O5I6GFFLTPWE type=del size=748594 2024-06-28T15:08:22.468 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.480+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~6QVD471KI9ZH.0 size=1048576 2024-06-28T15:08:22.470 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.484+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~6QVD471KI9ZH.1 size=1048576 2024-06-28T15:08:22.470 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.484+0000 7f65680598c0 1 < canceled part key=_multipart_C5C6SAHLPZV1.2~6QVD471KI9ZH.2 size=702831 2024-06-28T15:08:22.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.488+0000 7f65680598c0 1 < canceled multipart upload key=C5C6SAHLPZV1 upload=6QVD471KI9ZH tag=U4CW1H14JDFC type=write size=3848559 2024-06-28T15:08:22.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.492+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=HNIJLFQ6M433 type=write size=451838 2024-06-28T15:08:22.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.492+0000 7f65680598c0 1 < canceled operation key=LZX5UZCYLTAG tag=0170N066902N type=del size=592301 2024-06-28T15:08:22.484 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.496+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~7M0XH9XFRPH1.0 size=1048576 2024-06-28T15:08:22.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.500+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~7M0XH9XFRPH1.1 size=1048576 2024-06-28T15:08:22.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.500+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~7M0XH9XFRPH1.2 size=85126 2024-06-28T15:08:22.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.500+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=7M0XH9XFRPH1 tag=6VF7XTXWGUGX type=write size=3230854 2024-06-28T15:08:22.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.500+0000 7f65680598c0 1 < canceled part key=_multipart_MQ3QWZBFN4YR.2~OR7HB7J3GMM0.0 size=1048576 2024-06-28T15:08:22.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.504+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~OR7HB7J3GMM0.1 size=943138 2024-06-28T15:08:22.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.508+0000 7f65680598c0 1 < canceled multipart upload key=MQ3QWZBFN4YR upload=OR7HB7J3GMM0 tag=DAATEUH1DGE7 type=write size=3040290 2024-06-28T15:08:22.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.512+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=LMM0KMHQFAZY type=del size=3668491 2024-06-28T15:08:22.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.516+0000 7f65680598c0 1 < suggested remove operation key=ZY973GBGBQ1Z tag=KH9NAI3BXXOF type=del 2024-06-28T15:08:22.506 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.520+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=OHY135ESTMCW type=write size=621528 2024-06-28T15:08:22.509 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.520+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~70KFP9YZGMZ4.0 size=1048576 2024-06-28T15:08:22.511 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.524+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~70KFP9YZGMZ4.1 size=1039066 2024-06-28T15:08:22.515 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.528+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=70KFP9YZGMZ4 tag=UDZVJKMNNR7W type=write size=3136218 2024-06-28T15:08:22.520 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.532+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~HFHQR0O7R8KS.0 size=1048576 2024-06-28T15:08:22.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.536+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~HFHQR0O7R8KS.1 size=204407 2024-06-28T15:08:22.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.536+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=HFHQR0O7R8KS tag=HVNADRHGD2Y6 type=write size=2301559 2024-06-28T15:08:22.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.536+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~B3F9121ALRR5.0 size=63804 2024-06-28T15:08:22.529 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.544+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=B3F9121ALRR5 tag=2COME2NIOVXF type=write size=1112380 2024-06-28T15:08:22.534 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.548+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~KIT0K7BQ7HNK.0 size=1048576 2024-06-28T15:08:22.536 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.548+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~KIT0K7BQ7HNK.1 size=1048576 2024-06-28T15:08:22.537 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.552+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~KIT0K7BQ7HNK.2 size=265645 2024-06-28T15:08:22.537 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.552+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=KIT0K7BQ7HNK tag=AVPD86UG0CYO type=write size=3411373 2024-06-28T15:08:22.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.552+0000 7f65680598c0 1 < canceled operation key=BHW2VHUT0U1U tag=ZIYCU9IRMNY8 type=del size=1954495 2024-06-28T15:08:22.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.556+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=WBJC2R045FK4 type=del size=573851 2024-06-28T15:08:22.545 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.556+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~UCYTQOR0S1W9.0 size=1048576 2024-06-28T15:08:22.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.560+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~UCYTQOR0S1W9.1 size=390427 2024-06-28T15:08:22.551 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.564+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=UCYTQOR0S1W9 tag=VYY4OV6SJ2IL type=write size=2487579 2024-06-28T15:08:22.553 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.568+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=MI3ACHETT8E6 type=write size=898993 2024-06-28T15:08:22.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.568+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~7M5J07HRX7MW.0 size=311458 2024-06-28T15:08:22.560 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.572+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=7M5J07HRX7MW tag=8LTMLZJU1HT4 type=write size=1360034 2024-06-28T15:08:22.564 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.576+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~O4VLZTA57EU7.0 size=1048576 2024-06-28T15:08:22.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.580+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~O4VLZTA57EU7.1 size=1048576 2024-06-28T15:08:22.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.580+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~O4VLZTA57EU7.2 size=573502 2024-06-28T15:08:22.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.580+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=O4VLZTA57EU7 tag=PUN85CHM7RYI type=write size=3719230 2024-06-28T15:08:22.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.588+0000 7f65680598c0 1 < suggested remove operation key=O1CGONN4TEXF tag=0WHHUXLTKO4U type=write 2024-06-28T15:08:22.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.592+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~9JLGVG4SMSDY.0 size=1048576 2024-06-28T15:08:22.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.592+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~9JLGVG4SMSDY.1 size=876463 2024-06-28T15:08:22.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.592+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=9JLGVG4SMSDY tag=TCODFO93SB9A type=write size=2973615 2024-06-28T15:08:22.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.596+0000 7f65680598c0 1 < completed write operation key=2F968YIMJN9D tag=GH4EX20SLVPA type=write size=538372 2024-06-28T15:08:22.587 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.600+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=N9NL2WYMOINE type=del size=467169 2024-06-28T15:08:22.590 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.604+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=L9SH4M49Q9GQ type=del 2024-06-28T15:08:22.593 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.604+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=AFUMRSXTLMPS type=del size=280491 2024-06-28T15:08:22.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.608+0000 7f65680598c0 -1 < failed to complete operation key=O1CGONN4TEXF tag=0N5A7RS8WD7U type=del size=543888: (22) Invalid argument 2024-06-28T15:08:22.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.612+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=KAAK6KUD8CFR type=del size=2014492 2024-06-28T15:08:22.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.616+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~22Q2EEUEAITK.0 size=1048576 2024-06-28T15:08:22.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.616+0000 7f65680598c0 1 < canceled part key=_multipart_BHW2VHUT0U1U.2~22Q2EEUEAITK.1 size=129112 2024-06-28T15:08:22.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.620+0000 7f65680598c0 1 < canceled multipart upload key=BHW2VHUT0U1U upload=22Q2EEUEAITK tag=IJQHCR6NZVYQ type=write size=2226264 2024-06-28T15:08:22.611 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.624+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=BK94O142QPT6 type=del size=1297296 2024-06-28T15:08:22.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.632+0000 7f65680598c0 1 < suggested remove operation key=UP3XFF38IJXN tag=QMZVZA9B17Y5 type=del 2024-06-28T15:08:22.621 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.632+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=D0S3IKC0V1RP type=write size=583121 2024-06-28T15:08:22.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.636+0000 7f65680598c0 1 < canceled operation key=ZFNIA3O84EB7 tag=O5I6GFFLTPWE type=del size=748594 2024-06-28T15:08:22.629 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.640+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~KG3DDQ2H68BJ.0 size=1048576 2024-06-28T15:08:22.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.644+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~KG3DDQ2H68BJ.1 size=14184 2024-06-28T15:08:22.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.644+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=KG3DDQ2H68BJ tag=34HCTOVOM977 type=write size=2111336 2024-06-28T15:08:22.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.648+0000 7f65680598c0 1 < completed write operation key=EN76QQWIQAXZ tag=HNIJLFQ6M433 type=write size=451838 2024-06-28T15:08:22.637 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.652+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=BJGLZ985L1BN type=del size=2202877 2024-06-28T15:08:22.641 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.652+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~7M0XH9XFRPH1.0 size=1048576 2024-06-28T15:08:22.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.656+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~7M0XH9XFRPH1.1 size=1048576 2024-06-28T15:08:22.646 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.660+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~7M0XH9XFRPH1.2 size=85126 2024-06-28T15:08:22.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.664+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=7M0XH9XFRPH1 tag=6VF7XTXWGUGX type=write size=3230854 2024-06-28T15:08:22.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.668+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=QM370E3VSQGL type=del size=3418113 2024-06-28T15:08:22.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.672+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=LMM0KMHQFAZY type=del 2024-06-28T15:08:22.662 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.676+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=VXF3UZYEC71N type=del size=2633859 2024-06-28T15:08:22.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.676+0000 7f65680598c0 1 < completed write operation key=ZY973GBGBQ1Z tag=OHY135ESTMCW type=write size=621528 2024-06-28T15:08:22.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.680+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=953DMQXN3B16 type=del size=24206 2024-06-28T15:08:22.672 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.684+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~HFHQR0O7R8KS.0 size=1048576 2024-06-28T15:08:22.675 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.688+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~HFHQR0O7R8KS.1 size=204407 2024-06-28T15:08:22.680 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.692+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=HFHQR0O7R8KS tag=HVNADRHGD2Y6 type=write size=2301559 2024-06-28T15:08:22.683 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.696+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=P4SX2B0DJDLP type=del size=2262939 2024-06-28T15:08:22.686 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.700+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~KIT0K7BQ7HNK.0 size=1048576 2024-06-28T15:08:22.689 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.700+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~KIT0K7BQ7HNK.1 size=1048576 2024-06-28T15:08:22.694 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.708+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~KIT0K7BQ7HNK.2 size=265645 2024-06-28T15:08:22.699 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.712+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=KIT0K7BQ7HNK tag=AVPD86UG0CYO type=write size=3411373 2024-06-28T15:08:22.706 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.720+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~B5WV6S8P726G.0 size=1048576 2024-06-28T15:08:22.708 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.720+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~B5WV6S8P726G.1 size=589490 2024-06-28T15:08:22.708 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.720+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=B5WV6S8P726G tag=54FZGWNBN20W type=write size=2686642 2024-06-28T15:08:22.711 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.724+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=WBJC2R045FK4 type=del 2024-06-28T15:08:22.714 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.728+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=3UGZM3EL48TY type=del size=1052390 2024-06-28T15:08:22.717 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.732+0000 7f65680598c0 1 < completed write operation key=V2L2Q1U62YW2 tag=MI3ACHETT8E6 type=write size=898993 2024-06-28T15:08:22.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.736+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~GOFQ3A4HRTLI.0 size=1048576 2024-06-28T15:08:22.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.740+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~GOFQ3A4HRTLI.1 size=481565 2024-06-28T15:08:22.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.740+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=GOFQ3A4HRTLI tag=16BQSIGY9LRJ type=write size=2578717 2024-06-28T15:08:22.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.744+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~O4VLZTA57EU7.0 size=1048576 2024-06-28T15:08:22.732 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.744+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~O4VLZTA57EU7.1 size=1048576 2024-06-28T15:08:22.735 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.748+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~O4VLZTA57EU7.2 size=573502 2024-06-28T15:08:22.740 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.752+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=O4VLZTA57EU7 tag=PUN85CHM7RYI type=write size=3719230 2024-06-28T15:08:22.747 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.760+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~17NMJG7VE2CC.0 size=735679 2024-06-28T15:08:22.747 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.760+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=17NMJG7VE2CC tag=WF53K19B31SU type=write size=1784255 2024-06-28T15:08:22.750 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.764+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~9JLGVG4SMSDY.0 size=1048576 2024-06-28T15:08:22.752 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.764+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~9JLGVG4SMSDY.1 size=876463 2024-06-28T15:08:22.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.768+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=9JLGVG4SMSDY tag=TCODFO93SB9A type=write size=2973615 2024-06-28T15:08:22.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.772+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=MCKSB7V33W0S type=del size=618726 2024-06-28T15:08:22.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.776+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=N9NL2WYMOINE type=del 2024-06-28T15:08:22.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.780+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=LOSJHW3C7EGC type=del size=3895938 2024-06-28T15:08:22.768 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.780+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=AFUMRSXTLMPS type=del 2024-06-28T15:08:22.771 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.784+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=QH037OM5I8XU type=del size=2287673 2024-06-28T15:08:22.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.788+0000 7f65680598c0 1 < canceled operation key=GAZQ3PCPKN9Q tag=KAAK6KUD8CFR type=del size=2014492 2024-06-28T15:08:22.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.792+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~Q57HBOU9QDDX.0 size=1048576 2024-06-28T15:08:22.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.792+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~Q57HBOU9QDDX.1 size=1048576 2024-06-28T15:08:22.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.796+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~Q57HBOU9QDDX.2 size=155248 2024-06-28T15:08:22.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.796+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=Q57HBOU9QDDX tag=1O0N000O9BM0 type=write size=3300976 2024-06-28T15:08:22.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.800+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=BK94O142QPT6 type=del 2024-06-28T15:08:22.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.804+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~C7D406X0VB9A.0 size=1048576 2024-06-28T15:08:22.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.804+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~C7D406X0VB9A.1 size=1048576 2024-06-28T15:08:22.794 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.808+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~C7D406X0VB9A.2 size=857181 2024-06-28T15:08:22.794 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.808+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=C7D406X0VB9A tag=FBVQPB1F2KL7 type=write size=4002909 2024-06-28T15:08:22.797 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.812+0000 7f65680598c0 1 < completed write operation key=AOBFAR0UTONH tag=D0S3IKC0V1RP type=write size=583121 2024-06-28T15:08:22.800 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.812+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=L23KW8AO1S9Y type=del size=1345923 2024-06-28T15:08:22.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.816+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~KG3DDQ2H68BJ.0 size=1048576 2024-06-28T15:08:22.805 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.820+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~KG3DDQ2H68BJ.1 size=14184 2024-06-28T15:08:22.811 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.824+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=KG3DDQ2H68BJ tag=34HCTOVOM977 type=write size=2111336 2024-06-28T15:08:22.814 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.828+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=3GKPMEY55C4N type=write size=904007 2024-06-28T15:08:22.817 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.832+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=BJGLZ985L1BN type=del 2024-06-28T15:08:22.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.832+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=SAA1869N1ARW type=del size=3935603 2024-06-28T15:08:22.824 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.836+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=QM370E3VSQGL type=del 2024-06-28T15:08:22.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.840+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=8A14TY6O70MS type=del size=1359722 2024-06-28T15:08:22.831 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.844+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=VXF3UZYEC71N type=del 2024-06-28T15:08:22.837 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.852+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~B7PMCIU9PTKI.0 size=1048576 2024-06-28T15:08:22.840 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.852+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~B7PMCIU9PTKI.1 size=685913 2024-06-28T15:08:22.840 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.852+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=B7PMCIU9PTKI tag=ZHXXLDV0Y2PR type=write size=2783065 2024-06-28T15:08:22.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.860+0000 7f65680598c0 1 < suggested remove operation key=9MQKWSA33R5G tag=953DMQXN3B16 type=del 2024-06-28T15:08:22.850 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.864+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=XT0YCQTKKOTN type=del size=2794950 2024-06-28T15:08:22.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.868+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=P4SX2B0DJDLP type=del 2024-06-28T15:08:22.856 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.868+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=8CD7KR4AZ18V type=del size=2620075 2024-06-28T15:08:22.860 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.872+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~B5WV6S8P726G.0 size=1048576 2024-06-28T15:08:22.862 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.876+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~B5WV6S8P726G.1 size=589490 2024-06-28T15:08:22.868 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.880+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=B5WV6S8P726G tag=54FZGWNBN20W type=write size=2686642 2024-06-28T15:08:22.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.884+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=CR2SHRIJAI3C type=del size=1767196 2024-06-28T15:08:22.875 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.888+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=3UGZM3EL48TY type=del 2024-06-28T15:08:22.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.892+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~5S1J7TPIZS2G.0 size=342514 2024-06-28T15:08:22.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.892+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=5S1J7TPIZS2G tag=E71D70EKV2N6 type=write size=1391090 2024-06-28T15:08:22.883 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.896+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~GOFQ3A4HRTLI.0 size=1048576 2024-06-28T15:08:22.886 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.900+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~GOFQ3A4HRTLI.1 size=481565 2024-06-28T15:08:22.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.904+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=GOFQ3A4HRTLI tag=16BQSIGY9LRJ type=write size=2578717 2024-06-28T15:08:22.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.912+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~QZ0OYSQ28L7A.0 size=721605 2024-06-28T15:08:22.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.912+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=QZ0OYSQ28L7A tag=1YGZDEXK9BVD type=write size=1770181 2024-06-28T15:08:22.901 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.912+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~17NMJG7VE2CC.0 size=735679 2024-06-28T15:08:22.906 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.920+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=17NMJG7VE2CC tag=WF53K19B31SU type=write size=1784255 2024-06-28T15:08:22.909 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.924+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=G1P96HS3IMPS type=write size=43253 2024-06-28T15:08:22.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.924+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=MCKSB7V33W0S type=del 2024-06-28T15:08:22.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.932+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~20H35BE49NMR.0 size=1048576 2024-06-28T15:08:22.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.936+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~20H35BE49NMR.1 size=1048576 2024-06-28T15:08:22.925 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.936+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~20H35BE49NMR.2 size=735110 2024-06-28T15:08:22.925 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.936+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=20H35BE49NMR tag=JVIPZN64HEK2 type=write size=3880838 2024-06-28T15:08:22.929 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.940+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=LOSJHW3C7EGC type=del 2024-06-28T15:08:22.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.944+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=N8ZV3YVLX6OY type=write size=655234 2024-06-28T15:08:22.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.948+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=QH037OM5I8XU type=del 2024-06-28T15:08:22.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.952+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=G3BXLMAKNUWY type=write size=937661 2024-06-28T15:08:22.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.956+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~Q57HBOU9QDDX.0 size=1048576 2024-06-28T15:08:22.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.956+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~Q57HBOU9QDDX.1 size=1048576 2024-06-28T15:08:22.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.956+0000 7f65680598c0 1 < canceled part key=_multipart_AOBFAR0UTONH.2~Q57HBOU9QDDX.2 size=155248 2024-06-28T15:08:22.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.964+0000 7f65680598c0 1 < canceled multipart upload key=AOBFAR0UTONH upload=Q57HBOU9QDDX tag=1O0N000O9BM0 type=write size=3300976 2024-06-28T15:08:22.953 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.964+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=Z9C6Q1YKBRCH type=del size=1348441 2024-06-28T15:08:22.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.968+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~C7D406X0VB9A.0 size=1048576 2024-06-28T15:08:22.958 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.972+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~C7D406X0VB9A.1 size=1048576 2024-06-28T15:08:22.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.972+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~C7D406X0VB9A.2 size=857181 2024-06-28T15:08:22.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.980+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=C7D406X0VB9A tag=FBVQPB1F2KL7 type=write size=4002909 2024-06-28T15:08:22.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.984+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~0AVLYH71EVBJ.0 size=3731 2024-06-28T15:08:22.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.984+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=0AVLYH71EVBJ tag=857GVC051Q18 type=write size=1052307 2024-06-28T15:08:22.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.988+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=L23KW8AO1S9Y type=del 2024-06-28T15:08:22.980 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.992+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~2GT5P1HYE9IB.0 size=1048576 2024-06-28T15:08:22.984 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.996+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~2GT5P1HYE9IB.1 size=330129 2024-06-28T15:08:22.984 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:22.996+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=2GT5P1HYE9IB tag=RQ8E93H8AJKU type=write size=2427281 2024-06-28T15:08:22.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.000+0000 7f65680598c0 1 < completed write operation key=R0IMFVF5NEOO tag=3GKPMEY55C4N type=write size=904007 2024-06-28T15:08:22.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.004+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=I4TBDJFET4AP type=del size=28647 2024-06-28T15:08:22.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.004+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=SAA1869N1ARW type=del 2024-06-28T15:08:22.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.008+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=3K76NKLXKEQ2 type=del size=2248595 2024-06-28T15:08:22.999 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.012+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=8A14TY6O70MS type=del 2024-06-28T15:08:23.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.016+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=E4S26WJLHURC type=del size=2747480 2024-06-28T15:08:23.005 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.016+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~B7PMCIU9PTKI.0 size=1048576 2024-06-28T15:08:23.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.020+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~B7PMCIU9PTKI.1 size=685913 2024-06-28T15:08:23.012 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.024+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=B7PMCIU9PTKI tag=ZHXXLDV0Y2PR type=write size=2783065 2024-06-28T15:08:23.017 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.032+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~B0ZUSPRKTH0T.0 size=1048576 2024-06-28T15:08:23.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.032+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~B0ZUSPRKTH0T.1 size=363634 2024-06-28T15:08:23.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.032+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=B0ZUSPRKTH0T tag=KB1J9H2BJX0A type=write size=2460786 2024-06-28T15:08:23.023 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.036+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=XT0YCQTKKOTN type=del 2024-06-28T15:08:23.026 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.040+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=SNMJ259IEPGQ type=del size=3625606 2024-06-28T15:08:23.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.040+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=8CD7KR4AZ18V type=del 2024-06-28T15:08:23.034 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.048+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~1DD0GQ4YKPNU.0 size=1048576 2024-06-28T15:08:23.036 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.048+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~1DD0GQ4YKPNU.1 size=391984 2024-06-28T15:08:23.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.048+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=1DD0GQ4YKPNU tag=KA5DOW87KZRV type=write size=2489136 2024-06-28T15:08:23.039 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.052+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=CR2SHRIJAI3C type=del 2024-06-28T15:08:23.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.056+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=L2LMFYGYSPY9 type=del size=758021 2024-06-28T15:08:23.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.060+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~5S1J7TPIZS2G.0 size=342514 2024-06-28T15:08:23.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.064+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=5S1J7TPIZS2G tag=E71D70EKV2N6 type=write size=1391090 2024-06-28T15:08:23.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.068+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=Q5VCZ3QSUC78 type=del size=3199256 2024-06-28T15:08:23.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.072+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~QZ0OYSQ28L7A.0 size=721605 2024-06-28T15:08:23.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.076+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=QZ0OYSQ28L7A tag=1YGZDEXK9BVD type=write size=1770181 2024-06-28T15:08:23.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.080+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~HGVMTHMDMFEP.0 size=940940 2024-06-28T15:08:23.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.080+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=HGVMTHMDMFEP tag=PAG5LB0FXBQZ type=write size=1989516 2024-06-28T15:08:23.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.084+0000 7f65680598c0 1 < completed write operation key=2F968YIMJN9D tag=G1P96HS3IMPS type=write size=43253 2024-06-28T15:08:23.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.088+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~X6J4QKMTETMH.0 size=1048576 2024-06-28T15:08:23.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.092+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~X6J4QKMTETMH.1 size=1048576 2024-06-28T15:08:23.080 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.092+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~X6J4QKMTETMH.2 size=296474 2024-06-28T15:08:23.080 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.092+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=X6J4QKMTETMH tag=L2KJ3Z6RS3CJ type=write size=3442202 2024-06-28T15:08:23.082 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.096+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~20H35BE49NMR.0 size=1048576 2024-06-28T15:08:23.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.096+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~20H35BE49NMR.1 size=1048576 2024-06-28T15:08:23.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.096+0000 7f65680598c0 1 < canceled part key=_multipart_7NL3CTFYY9E5.2~20H35BE49NMR.2 size=735110 2024-06-28T15:08:23.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.100+0000 7f65680598c0 1 < canceled multipart upload key=7NL3CTFYY9E5 upload=20H35BE49NMR tag=JVIPZN64HEK2 type=write size=3880838 2024-06-28T15:08:23.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.104+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=UAGA2Z54NYNO type=del size=670333 2024-06-28T15:08:23.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.108+0000 7f65680598c0 1 < completed write operation key=2FC9DNEZGWC4 tag=N8ZV3YVLX6OY type=write size=655234 2024-06-28T15:08:23.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.112+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~4UKMCBP0I7QC.0 size=1048576 2024-06-28T15:08:23.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.116+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~4UKMCBP0I7QC.1 size=1048576 2024-06-28T15:08:23.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.116+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~4UKMCBP0I7QC.2 size=964497 2024-06-28T15:08:23.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.116+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=4UKMCBP0I7QC tag=LUVHQ0SEWNP9 type=write size=4110225 2024-06-28T15:08:23.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.120+0000 7f65680598c0 1 < completed write operation key=AXQ0YFWUH4FD tag=G3BXLMAKNUWY type=write size=937661 2024-06-28T15:08:23.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.124+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=C841LM85LEI5 type=write size=315246 2024-06-28T15:08:23.112 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.124+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=Z9C6Q1YKBRCH type=del 2024-06-28T15:08:23.115 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.128+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=3ZZZLEYFQM18 type=del size=1554363 2024-06-28T15:08:23.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.132+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~0AVLYH71EVBJ.0 size=3731 2024-06-28T15:08:23.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.136+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=0AVLYH71EVBJ tag=857GVC051Q18 type=write size=1052307 2024-06-28T15:08:23.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.136+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=QU2TRHW148EX type=del size=281567 2024-06-28T15:08:23.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.136+0000 7f65680598c0 1 < canceled part key=_multipart_LZX5UZCYLTAG.2~2GT5P1HYE9IB.0 size=1048576 2024-06-28T15:08:23.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.140+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~2GT5P1HYE9IB.1 size=330129 2024-06-28T15:08:23.131 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.144+0000 7f65680598c0 1 < canceled multipart upload key=LZX5UZCYLTAG upload=2GT5P1HYE9IB tag=RQ8E93H8AJKU type=write size=2427281 2024-06-28T15:08:23.134 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.148+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=HGP22PBP0KPQ type=del size=1020121 2024-06-28T15:08:23.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.148+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=I4TBDJFET4AP type=del 2024-06-28T15:08:23.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.152+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=70LHZ9ED1VPY type=del size=888588 2024-06-28T15:08:23.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.156+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=3K76NKLXKEQ2 type=del 2024-06-28T15:08:23.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.156+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=U2RP6VDMBG9E type=del size=1915611 2024-06-28T15:08:23.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.160+0000 7f65680598c0 1 < canceled operation key=P278ZM1H052S tag=E4S26WJLHURC type=del size=2747480 2024-06-28T15:08:23.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.164+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=UW1YZ21X1XSP type=del size=2968259 2024-06-28T15:08:23.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.164+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~B0ZUSPRKTH0T.0 size=1048576 2024-06-28T15:08:23.155 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.168+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~B0ZUSPRKTH0T.1 size=363634 2024-06-28T15:08:23.160 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.172+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=B0ZUSPRKTH0T tag=KB1J9H2BJX0A type=write size=2460786 2024-06-28T15:08:23.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.180+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~DGK6ORAG4NS2.0 size=1048576 2024-06-28T15:08:23.168 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.180+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~DGK6ORAG4NS2.1 size=1048576 2024-06-28T15:08:23.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.184+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~DGK6ORAG4NS2.2 size=890078 2024-06-28T15:08:23.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.184+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=DGK6ORAG4NS2 tag=DV3W1KHSY3XC type=write size=4035806 2024-06-28T15:08:23.176 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.188+0000 7f65680598c0 1 < suggested remove operation key=ZFNIA3O84EB7 tag=SNMJ259IEPGQ type=del 2024-06-28T15:08:23.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.196+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~BPWVBB01CF9L.0 size=980170 2024-06-28T15:08:23.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.196+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=BPWVBB01CF9L tag=O5VZW0K79I88 type=write size=2028746 2024-06-28T15:08:23.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.196+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~1DD0GQ4YKPNU.0 size=1048576 2024-06-28T15:08:23.187 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.200+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~1DD0GQ4YKPNU.1 size=391984 2024-06-28T15:08:23.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.204+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=1DD0GQ4YKPNU tag=KA5DOW87KZRV type=write size=2489136 2024-06-28T15:08:23.198 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.212+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~H4W90F62H61Z.0 size=1048576 2024-06-28T15:08:23.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.212+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~H4W90F62H61Z.1 size=1048576 2024-06-28T15:08:23.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.216+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~H4W90F62H61Z.2 size=676706 2024-06-28T15:08:23.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.216+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=H4W90F62H61Z tag=AGR8PZ053LBA type=write size=3822434 2024-06-28T15:08:23.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.220+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=L2LMFYGYSPY9 type=del 2024-06-28T15:08:23.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.220+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=T3PSXY47JSXS type=del size=3800915 2024-06-28T15:08:23.212 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.224+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=Q5VCZ3QSUC78 type=del 2024-06-28T15:08:23.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.228+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=43BX1W2OS2QT type=del size=3002575 2024-06-28T15:08:23.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.232+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~HGVMTHMDMFEP.0 size=940940 2024-06-28T15:08:23.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.236+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=HGVMTHMDMFEP tag=PAG5LB0FXBQZ type=write size=1989516 2024-06-28T15:08:23.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.236+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=TLZ4YJIL75PX type=del size=3563466 2024-06-28T15:08:23.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.240+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~X6J4QKMTETMH.0 size=1048576 2024-06-28T15:08:23.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.244+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~X6J4QKMTETMH.1 size=1048576 2024-06-28T15:08:23.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.244+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~X6J4QKMTETMH.2 size=296474 2024-06-28T15:08:23.236 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.248+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=X6J4QKMTETMH tag=L2KJ3Z6RS3CJ type=write size=3442202 2024-06-28T15:08:23.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.256+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~381I4NFOU7WY.0 size=1048576 2024-06-28T15:08:23.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.256+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~381I4NFOU7WY.1 size=1048576 2024-06-28T15:08:23.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.260+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~381I4NFOU7WY.2 size=723603 2024-06-28T15:08:23.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.260+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=381I4NFOU7WY tag=QR4YO9GW29ZT type=write size=3869331 2024-06-28T15:08:23.252 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.264+0000 7f65680598c0 1 < completed delete operation key=1VD7NDPEM3CD tag=UAGA2Z54NYNO type=del 2024-06-28T15:08:23.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.268+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~I3KTE96SDT2D.0 size=1028526 2024-06-28T15:08:23.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.268+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=I3KTE96SDT2D tag=E1CL0YGM2RFW type=write size=2077102 2024-06-28T15:08:23.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.272+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~4UKMCBP0I7QC.0 size=1048576 2024-06-28T15:08:23.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.276+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~4UKMCBP0I7QC.1 size=1048576 2024-06-28T15:08:23.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.276+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~4UKMCBP0I7QC.2 size=964497 2024-06-28T15:08:23.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.280+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=4UKMCBP0I7QC tag=LUVHQ0SEWNP9 type=write size=4110225 2024-06-28T15:08:23.271 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.284+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=PYZ8XY92Y7C4 type=del size=300519 2024-06-28T15:08:23.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.288+0000 7f65680598c0 1 < completed write operation key=4CU86XS1R9G3 tag=C841LM85LEI5 type=write size=315246 2024-06-28T15:08:23.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.288+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=V6M5SN0EXMFZ type=del size=3180981 2024-06-28T15:08:23.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.292+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=3ZZZLEYFQM18 type=del 2024-06-28T15:08:23.280 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.292+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=X79T8WMRBSUL type=write size=725754 2024-06-28T15:08:23.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.296+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=QU2TRHW148EX type=del 2024-06-28T15:08:23.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.296+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=OZO2XKZ0JYUS type=write size=252644 2024-06-28T15:08:23.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.300+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=HGP22PBP0KPQ type=del 2024-06-28T15:08:23.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.304+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=T64WA2XCX6US type=del size=47544 2024-06-28T15:08:23.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.308+0000 7f65680598c0 1 < suggested update operation key=9MQKWSA33R5G tag=70LHZ9ED1VPY type=del size=888588 2024-06-28T15:08:23.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.312+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~G5IJI60XOMQJ.0 size=1048576 2024-06-28T15:08:23.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.312+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~G5IJI60XOMQJ.1 size=1048576 2024-06-28T15:08:23.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.316+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~G5IJI60XOMQJ.2 size=230925 2024-06-28T15:08:23.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.316+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=G5IJI60XOMQJ tag=VQH6W5Y3F80D type=write size=3376653 2024-06-28T15:08:23.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.320+0000 7f65680598c0 1 < suggested update operation key=9MQKWSA33R5G tag=U2RP6VDMBG9E type=del size=1915611 2024-06-28T15:08:23.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.324+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=4UANUP3NPO5F type=del size=3068923 2024-06-28T15:08:23.312 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.324+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=UW1YZ21X1XSP type=del 2024-06-28T15:08:23.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.328+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=LSFOWYM1L9R9 type=del size=2193333 2024-06-28T15:08:23.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.332+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~DGK6ORAG4NS2.0 size=1048576 2024-06-28T15:08:23.320 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.332+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~DGK6ORAG4NS2.1 size=1048576 2024-06-28T15:08:23.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.336+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~DGK6ORAG4NS2.2 size=890078 2024-06-28T15:08:23.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.340+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=DGK6ORAG4NS2 tag=DV3W1KHSY3XC type=write size=4035806 2024-06-28T15:08:23.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.344+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~Y831LZH1ZEHU.0 size=1048576 2024-06-28T15:08:23.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.348+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~Y831LZH1ZEHU.1 size=1048576 2024-06-28T15:08:23.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.352+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~Y831LZH1ZEHU.2 size=59651 2024-06-28T15:08:23.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.352+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=Y831LZH1ZEHU tag=TA70BO1FL49S type=write size=3205379 2024-06-28T15:08:23.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.352+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~BPWVBB01CF9L.0 size=980170 2024-06-28T15:08:23.346 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.360+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=BPWVBB01CF9L tag=O5VZW0K79I88 type=write size=2028746 2024-06-28T15:08:23.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.364+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=JB75OZTQ7H38 type=del size=3095853 2024-06-28T15:08:23.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.364+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~H4W90F62H61Z.0 size=1048576 2024-06-28T15:08:23.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.368+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~H4W90F62H61Z.1 size=1048576 2024-06-28T15:08:23.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.372+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~H4W90F62H61Z.2 size=676706 2024-06-28T15:08:23.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.376+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=H4W90F62H61Z tag=AGR8PZ053LBA type=write size=3822434 2024-06-28T15:08:23.367 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.380+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~2HHUZLICGS25.0 size=124643 2024-06-28T15:08:23.367 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.380+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=2HHUZLICGS25 tag=T3JYXT7DJPRG type=write size=1173219 2024-06-28T15:08:23.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.380+0000 7f65680598c0 1 < canceled operation key=R0IMFVF5NEOO tag=T3PSXY47JSXS type=del size=3800915 2024-06-28T15:08:23.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.384+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=ZC1UQWW15L2E type=del size=3746282 2024-06-28T15:08:23.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.388+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=43BX1W2OS2QT type=del 2024-06-28T15:08:23.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.392+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=1UJ8ZAH2UXS5 type=del size=2633098 2024-06-28T15:08:23.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.392+0000 7f65680598c0 1 < completed delete operation key=1VD7NDPEM3CD tag=TLZ4YJIL75PX type=del 2024-06-28T15:08:23.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.396+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=S5AH84X6G8D4 type=write size=978179 2024-06-28T15:08:23.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.396+0000 7f65680598c0 1 < canceled part key=_multipart_I4RWCQSQVLCN.2~381I4NFOU7WY.0 size=1048576 2024-06-28T15:08:23.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.400+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~381I4NFOU7WY.1 size=1048576 2024-06-28T15:08:23.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.400+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~381I4NFOU7WY.2 size=723603 2024-06-28T15:08:23.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.408+0000 7f65680598c0 1 < canceled multipart upload key=I4RWCQSQVLCN upload=381I4NFOU7WY tag=QR4YO9GW29ZT type=write size=3869331 2024-06-28T15:08:23.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.408+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=2YY93GWHY5F0 type=del size=1278557 2024-06-28T15:08:23.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.412+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~I3KTE96SDT2D.0 size=1028526 2024-06-28T15:08:23.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.416+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=I3KTE96SDT2D tag=E1CL0YGM2RFW type=write size=2077102 2024-06-28T15:08:23.409 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.424+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~1HKSP1UT35V2.0 size=1048576 2024-06-28T15:08:23.411 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.424+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~1HKSP1UT35V2.1 size=699769 2024-06-28T15:08:23.411 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.424+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=1HKSP1UT35V2 tag=V7O3KOMTM5S3 type=write size=2796921 2024-06-28T15:08:23.414 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.428+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=PYZ8XY92Y7C4 type=del 2024-06-28T15:08:23.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.432+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~SYAIHBVC4F60.0 size=180649 2024-06-28T15:08:23.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.432+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=SYAIHBVC4F60 tag=OXKGPBPCJDZP type=write size=1229225 2024-06-28T15:08:23.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.436+0000 7f65680598c0 1 < canceled operation key=AOBFAR0UTONH tag=V6M5SN0EXMFZ type=del size=3180981 2024-06-28T15:08:23.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.436+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=N6L4T5VFZRY1 type=del size=4181842 2024-06-28T15:08:23.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.440+0000 7f65680598c0 1 < completed write operation key=60QVJVV16QFM tag=X79T8WMRBSUL type=write size=725754 2024-06-28T15:08:23.429 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.440+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=QQDZ2PMKUGHL type=del size=2766778 2024-06-28T15:08:23.433 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.448+0000 7f65680598c0 1 < suggested update operation key=I4RWCQSQVLCN tag=OZO2XKZ0JYUS type=write size=252644 2024-06-28T15:08:23.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.448+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=R8TY7PZ9AOUT type=del size=637729 2024-06-28T15:08:23.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.452+0000 7f65680598c0 -1 < failed to complete operation key=I4RWCQSQVLCN tag=T64WA2XCX6US type=del size=47544: (22) Invalid argument 2024-06-28T15:08:23.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.452+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=FSHO34IGI272 type=del size=804177 2024-06-28T15:08:23.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.456+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~G5IJI60XOMQJ.0 size=1048576 2024-06-28T15:08:23.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.456+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~G5IJI60XOMQJ.1 size=1048576 2024-06-28T15:08:23.447 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.460+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~G5IJI60XOMQJ.2 size=230925 2024-06-28T15:08:23.451 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.464+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=G5IJI60XOMQJ tag=VQH6W5Y3F80D type=write size=3376653 2024-06-28T15:08:23.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.468+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=JC95Z3NM57W0 type=write size=649233 2024-06-28T15:08:23.456 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.468+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=4UANUP3NPO5F type=del 2024-06-28T15:08:23.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.472+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=1FR195KZBX2L type=del size=850669 2024-06-28T15:08:23.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.472+0000 7f65680598c0 1 < canceled operation key=MQ3QWZBFN4YR tag=LSFOWYM1L9R9 type=del size=2193333 2024-06-28T15:08:23.465 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.476+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~0NFHAPC5530X.0 size=1048576 2024-06-28T15:08:23.467 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.480+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~0NFHAPC5530X.1 size=628039 2024-06-28T15:08:23.467 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.480+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=0NFHAPC5530X tag=K9CLM6SW5LV3 type=write size=2725191 2024-06-28T15:08:23.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.484+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~Y831LZH1ZEHU.0 size=1048576 2024-06-28T15:08:23.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.484+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~Y831LZH1ZEHU.1 size=1048576 2024-06-28T15:08:23.474 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.488+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~Y831LZH1ZEHU.2 size=59651 2024-06-28T15:08:23.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.492+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=Y831LZH1ZEHU tag=TA70BO1FL49S type=write size=3205379 2024-06-28T15:08:23.483 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.496+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~J6MWF53WGCHD.0 size=1048576 2024-06-28T15:08:23.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.500+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~J6MWF53WGCHD.1 size=408216 2024-06-28T15:08:23.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.500+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=J6MWF53WGCHD tag=V2I7A9JYLF98 type=write size=2505368 2024-06-28T15:08:23.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.500+0000 7f65680598c0 -1 < failed to complete operation key=I4RWCQSQVLCN tag=JB75OZTQ7H38 type=del size=3095853: (22) Invalid argument 2024-06-28T15:08:23.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.504+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=VD1NRLULU5PD type=del size=2059217 2024-06-28T15:08:23.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.508+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~2HHUZLICGS25.0 size=124643 2024-06-28T15:08:23.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.516+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=2HHUZLICGS25 tag=T3JYXT7DJPRG type=write size=1173219 2024-06-28T15:08:23.507 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.520+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~L0A9NNAM8MDF.0 size=1048576 2024-06-28T15:08:23.509 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.520+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~L0A9NNAM8MDF.1 size=1048576 2024-06-28T15:08:23.511 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.524+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~L0A9NNAM8MDF.2 size=978189 2024-06-28T15:08:23.511 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.524+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=L0A9NNAM8MDF tag=87ZRPNURIQZB type=write size=4123917 2024-06-28T15:08:23.513 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.524+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=ZC1UQWW15L2E type=del 2024-06-28T15:08:23.516 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.528+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=3ALHF2B2UG7G type=del size=421803 2024-06-28T15:08:23.518 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.532+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=1UJ8ZAH2UXS5 type=del 2024-06-28T15:08:23.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.536+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=DP4SMD51UELX type=del size=1113878 2024-06-28T15:08:23.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.536+0000 7f65680598c0 1 < completed write operation key=RLJCNXKA2GYA tag=S5AH84X6G8D4 type=write size=978179 2024-06-28T15:08:23.529 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.540+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~EPJ75G3KLVND.0 size=1048576 2024-06-28T15:08:23.531 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.544+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~EPJ75G3KLVND.1 size=86283 2024-06-28T15:08:23.531 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.544+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=EPJ75G3KLVND tag=9GGG3QCDG10J type=write size=2183435 2024-06-28T15:08:23.536 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.548+0000 7f65680598c0 1 < suggested update operation key=2F968YIMJN9D tag=2YY93GWHY5F0 type=del size=1278557 2024-06-28T15:08:23.539 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.552+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=I1KWUHQ7XW3L type=write size=100196 2024-06-28T15:08:23.541 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.556+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~1HKSP1UT35V2.0 size=1048576 2024-06-28T15:08:23.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.556+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~1HKSP1UT35V2.1 size=699769 2024-06-28T15:08:23.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.560+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=1HKSP1UT35V2 tag=V7O3KOMTM5S3 type=write size=2796921 2024-06-28T15:08:23.552 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.564+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~REJ85QU5D8JQ.0 size=1048576 2024-06-28T15:08:23.554 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.568+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~REJ85QU5D8JQ.1 size=681120 2024-06-28T15:08:23.555 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.568+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=REJ85QU5D8JQ tag=IEHX3XDUYYEA type=write size=2778272 2024-06-28T15:08:23.555 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.568+0000 7f65680598c0 1 < canceled part key=_multipart_SFCIZMLY2Y2E.2~SYAIHBVC4F60.0 size=180649 2024-06-28T15:08:23.560 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.572+0000 7f65680598c0 1 < canceled multipart upload key=SFCIZMLY2Y2E upload=SYAIHBVC4F60 tag=OXKGPBPCJDZP type=write size=1229225 2024-06-28T15:08:23.562 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.576+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=3M4M7O05W7AH type=del size=1110970 2024-06-28T15:08:23.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.576+0000 7f65680598c0 1 < canceled operation key=AOBFAR0UTONH tag=N6L4T5VFZRY1 type=del size=4181842 2024-06-28T15:08:23.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.580+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=6JXP25S9FYPS type=del size=1841773 2024-06-28T15:08:23.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.584+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=QQDZ2PMKUGHL type=del 2024-06-28T15:08:23.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.588+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=0D80F7ELY100 type=del size=1475396 2024-06-28T15:08:23.576 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.588+0000 7f65680598c0 -1 < failed to complete operation key=2F968YIMJN9D tag=R8TY7PZ9AOUT type=del size=637729: (22) Invalid argument 2024-06-28T15:08:23.578 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.592+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=ZTEPRMLHREQE type=del size=1466898 2024-06-28T15:08:23.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.592+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=FSHO34IGI272 type=del 2024-06-28T15:08:23.586 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.600+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~AA84L6BB7QHF.0 size=28178 2024-06-28T15:08:23.586 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.600+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=AA84L6BB7QHF tag=0ZC4F2JIM8XJ type=write size=1076754 2024-06-28T15:08:23.591 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.604+0000 7f65680598c0 1 < suggested remove operation key=F6SEDD6BH1T2 tag=JC95Z3NM57W0 type=write 2024-06-28T15:08:23.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.608+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=TQOLYDE9Y7WA type=del size=1911045 2024-06-28T15:08:23.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.608+0000 7f65680598c0 1 < completed delete operation key=4CU86XS1R9G3 tag=1FR195KZBX2L type=del 2024-06-28T15:08:23.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.612+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=V6TCNV7P8ZGK type=del size=570711 2024-06-28T15:08:23.603 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.616+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~0NFHAPC5530X.0 size=1048576 2024-06-28T15:08:23.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.620+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~0NFHAPC5530X.1 size=628039 2024-06-28T15:08:23.611 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.624+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=0NFHAPC5530X tag=K9CLM6SW5LV3 type=write size=2725191 2024-06-28T15:08:23.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.628+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=YZR8ZOCWQ9IN type=del size=3970646 2024-06-28T15:08:23.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.628+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~J6MWF53WGCHD.0 size=1048576 2024-06-28T15:08:23.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.632+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~J6MWF53WGCHD.1 size=408216 2024-06-28T15:08:23.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.636+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=J6MWF53WGCHD tag=V2I7A9JYLF98 type=write size=2505368 2024-06-28T15:08:23.627 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.640+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=V42EA8CPI09S type=del size=3908626 2024-06-28T15:08:23.629 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.644+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=VD1NRLULU5PD type=del 2024-06-28T15:08:23.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.644+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=9K6OGOIVOSLO type=del size=2172228 2024-06-28T15:08:23.636 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.648+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~L0A9NNAM8MDF.0 size=1048576 2024-06-28T15:08:23.638 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.652+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~L0A9NNAM8MDF.1 size=1048576 2024-06-28T15:08:23.640 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.652+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~L0A9NNAM8MDF.2 size=978189 2024-06-28T15:08:23.646 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.660+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=L0A9NNAM8MDF tag=87ZRPNURIQZB type=write size=4123917 2024-06-28T15:08:23.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.664+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=M92P4GJ9AIZ0 type=del size=2753923 2024-06-28T15:08:23.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.664+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=3ALHF2B2UG7G type=del 2024-06-28T15:08:23.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.668+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=6BEIBNAKSS9P type=del size=3570362 2024-06-28T15:08:23.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.672+0000 7f65680598c0 -1 < failed to complete operation key=F6SEDD6BH1T2 tag=DP4SMD51UELX type=del size=1113878: (22) Invalid argument 2024-06-28T15:08:23.661 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.672+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=T69RE8MZCMWH type=del size=2755233 2024-06-28T15:08:23.664 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.676+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~EPJ75G3KLVND.0 size=1048576 2024-06-28T15:08:23.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.680+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~EPJ75G3KLVND.1 size=86283 2024-06-28T15:08:23.671 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.684+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=EPJ75G3KLVND tag=9GGG3QCDG10J type=write size=2183435 2024-06-28T15:08:23.675 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.688+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~J7FKUJLXLVW0.0 size=951740 2024-06-28T15:08:23.675 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.688+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=J7FKUJLXLVW0 tag=14VF3SXCZOS3 type=write size=2000316 2024-06-28T15:08:23.678 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.692+0000 7f65680598c0 1 < completed write operation key=RLJCNXKA2GYA tag=I1KWUHQ7XW3L type=write size=100196 2024-06-28T15:08:23.682 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.696+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~OS2EP5D56WMT.0 size=63323 2024-06-28T15:08:23.682 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.696+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=OS2EP5D56WMT tag=BRFBW9FYAH0S type=write size=1111899 2024-06-28T15:08:23.685 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.696+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~REJ85QU5D8JQ.0 size=1048576 2024-06-28T15:08:23.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.700+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~REJ85QU5D8JQ.1 size=681120 2024-06-28T15:08:23.691 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.704+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=REJ85QU5D8JQ tag=IEHX3XDUYYEA type=write size=2778272 2024-06-28T15:08:23.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.708+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=P4428KDNX1VQ type=del size=2163392 2024-06-28T15:08:23.696 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.708+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=3M4M7O05W7AH type=del 2024-06-28T15:08:23.701 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.712+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~MP9QZCVNZ1QD.0 size=1048576 2024-06-28T15:08:23.702 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.716+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~MP9QZCVNZ1QD.1 size=164142 2024-06-28T15:08:23.702 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.716+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=MP9QZCVNZ1QD tag=IYMSSXIZM0NA type=write size=2261294 2024-06-28T15:08:23.705 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.716+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=6JXP25S9FYPS type=del 2024-06-28T15:08:23.707 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.720+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=KRJ9EEPIG861 type=write size=347602 2024-06-28T15:08:23.712 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.724+0000 7f65680598c0 1 < suggested update operation key=AXQ0YFWUH4FD tag=0D80F7ELY100 type=del size=1475396 2024-06-28T15:08:23.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.728+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=APETNN71C154 type=del size=872625 2024-06-28T15:08:23.717 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.732+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=ZTEPRMLHREQE type=del 2024-06-28T15:08:23.720 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.732+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=OF2XX8BOGYL9 type=del size=850345 2024-06-28T15:08:23.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.736+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~AA84L6BB7QHF.0 size=28178 2024-06-28T15:08:23.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.740+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=AA84L6BB7QHF tag=0ZC4F2JIM8XJ type=write size=1076754 2024-06-28T15:08:23.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.744+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=496KHT0B81K9 type=del size=1639130 2024-06-28T15:08:23.733 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.744+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=TQOLYDE9Y7WA type=del 2024-06-28T15:08:23.735 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.748+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=11LOC16AADV7 type=del size=2250326 2024-06-28T15:08:23.738 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.752+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=V6TCNV7P8ZGK type=del 2024-06-28T15:08:23.743 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.756+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~JF0XDNIIEQPJ.0 size=1027096 2024-06-28T15:08:23.743 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.756+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=JF0XDNIIEQPJ tag=ZAHMUAIHQFN4 type=write size=2075672 2024-06-28T15:08:23.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.760+0000 7f65680598c0 1 < suggested update operation key=SFCIZMLY2Y2E tag=YZR8ZOCWQ9IN type=del size=3970646 2024-06-28T15:08:23.755 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.768+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~3OWDVWG1QH41.0 size=1048576 2024-06-28T15:08:23.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.772+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~3OWDVWG1QH41.1 size=174221 2024-06-28T15:08:23.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.772+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=3OWDVWG1QH41 tag=6KHU9U939G58 type=write size=2271373 2024-06-28T15:08:23.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.772+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=V42EA8CPI09S type=del 2024-06-28T15:08:23.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.776+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=H21FYC1EJ21D type=del size=3036899 2024-06-28T15:08:23.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.780+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=9K6OGOIVOSLO type=del 2024-06-28T15:08:23.768 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.780+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=LSDKACUUY8G3 type=del size=1405360 2024-06-28T15:08:23.771 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.784+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=M92P4GJ9AIZ0 type=del 2024-06-28T15:08:23.773 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.788+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=57ZDP6UGC5X0 type=del size=1544080 2024-06-28T15:08:23.779 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.792+0000 7f65680598c0 1 < suggested update operation key=ZY973GBGBQ1Z tag=6BEIBNAKSS9P type=del size=3570362 2024-06-28T15:08:23.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.796+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=LB4K7V8GM7FC type=del size=3008641 2024-06-28T15:08:23.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.796+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=T69RE8MZCMWH type=del 2024-06-28T15:08:23.786 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.800+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=QX7K3OW9DKVG type=del size=2881263 2024-06-28T15:08:23.789 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.800+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~J7FKUJLXLVW0.0 size=951740 2024-06-28T15:08:23.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.808+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=J7FKUJLXLVW0 tag=14VF3SXCZOS3 type=write size=2000316 2024-06-28T15:08:23.796 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.808+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=AT10ZQNNR4OP type=del size=1531446 2024-06-28T15:08:23.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.812+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~OS2EP5D56WMT.0 size=63323 2024-06-28T15:08:23.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.816+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=OS2EP5D56WMT tag=BRFBW9FYAH0S type=write size=1111899 2024-06-28T15:08:23.807 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.820+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=YUYUXGQ04UTM type=del size=3089376 2024-06-28T15:08:23.812 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.824+0000 7f65680598c0 1 < suggested remove operation key=EN76QQWIQAXZ tag=P4428KDNX1VQ type=del 2024-06-28T15:08:23.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.828+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=SCA24TZP243M type=del size=3692219 2024-06-28T15:08:23.818 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.832+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~MP9QZCVNZ1QD.0 size=1048576 2024-06-28T15:08:23.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.832+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~MP9QZCVNZ1QD.1 size=164142 2024-06-28T15:08:23.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.836+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=MP9QZCVNZ1QD tag=IYMSSXIZM0NA type=write size=2261294 2024-06-28T15:08:23.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.840+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=PWIQU944S43R type=del size=1015473 2024-06-28T15:08:23.833 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.844+0000 7f65680598c0 1 < suggested update operation key=AXQ0YFWUH4FD tag=KRJ9EEPIG861 type=write size=347602 2024-06-28T15:08:23.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.848+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=N00108DSKWHG type=write size=552693 2024-06-28T15:08:23.838 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.852+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=APETNN71C154 type=del 2024-06-28T15:08:23.840 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.852+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=6K18L4UCSTLK type=del size=1263841 2024-06-28T15:08:23.844 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.856+0000 7f65680598c0 1 < completed delete operation key=ZFNIA3O84EB7 tag=OF2XX8BOGYL9 type=del 2024-06-28T15:08:23.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.860+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=2NCD3WW0M4V2 type=del size=2704426 2024-06-28T15:08:23.850 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.864+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=496KHT0B81K9 type=del 2024-06-28T15:08:23.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.868+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=EDR3V8Q3FQYW type=del size=3108356 2024-06-28T15:08:23.860 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.872+0000 7f65680598c0 1 < suggested update operation key=AOBFAR0UTONH tag=11LOC16AADV7 type=del size=2250326 2024-06-28T15:08:23.863 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.876+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=JNNQN4WEB42N type=del size=401493 2024-06-28T15:08:23.866 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.880+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~JF0XDNIIEQPJ.0 size=1027096 2024-06-28T15:08:23.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.884+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=JF0XDNIIEQPJ tag=ZAHMUAIHQFN4 type=write size=2075672 2024-06-28T15:08:23.877 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.892+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~EKH6JWIKBRL1.0 size=1048576 2024-06-28T15:08:23.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.892+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~EKH6JWIKBRL1.1 size=1048576 2024-06-28T15:08:23.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.896+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~EKH6JWIKBRL1.2 size=230527 2024-06-28T15:08:23.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.896+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=EKH6JWIKBRL1 tag=HN8FTZ30VI2K type=write size=3376255 2024-06-28T15:08:23.885 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.900+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~3OWDVWG1QH41.0 size=1048576 2024-06-28T15:08:23.888 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.900+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~3OWDVWG1QH41.1 size=174221 2024-06-28T15:08:23.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.904+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=3OWDVWG1QH41 tag=6KHU9U939G58 type=write size=2271373 2024-06-28T15:08:23.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.908+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=QFEKDF145C4W type=del size=1554485 2024-06-28T15:08:23.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.912+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=H21FYC1EJ21D type=del 2024-06-28T15:08:23.904 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.916+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~EVDLSJQ2CT0T.0 size=691810 2024-06-28T15:08:23.904 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.916+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=EVDLSJQ2CT0T tag=N2LN2DZD7IOD type=write size=1740386 2024-06-28T15:08:23.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.924+0000 7f65680598c0 1 < suggested remove operation key=J311Q2712YM0 tag=LSDKACUUY8G3 type=del 2024-06-28T15:08:23.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.928+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~F2H04E537I1R.0 size=363138 2024-06-28T15:08:23.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.928+0000 7f65680598c0 1 > prepared multipart upload key=2F968YIMJN9D upload=F2H04E537I1R tag=5Z9ITVS6QKPN type=write size=1411714 2024-06-28T15:08:23.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.932+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=57ZDP6UGC5X0 type=del 2024-06-28T15:08:23.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.936+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=B72XZ4WYZCZ4 type=del size=4003930 2024-06-28T15:08:23.925 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.936+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=LB4K7V8GM7FC type=del 2024-06-28T15:08:23.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.944+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~H22GFQSM4JI0.0 size=1048576 2024-06-28T15:08:23.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.944+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~H22GFQSM4JI0.1 size=733057 2024-06-28T15:08:23.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.944+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=H22GFQSM4JI0 tag=LA8V465NRDJS type=write size=2830209 2024-06-28T15:08:23.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.948+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=QX7K3OW9DKVG type=del 2024-06-28T15:08:23.940 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.952+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=U0NRMBD5KOFO type=del size=3471126 2024-06-28T15:08:23.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.956+0000 7f65680598c0 1 < completed delete operation key=ZFNIA3O84EB7 tag=AT10ZQNNR4OP type=del 2024-06-28T15:08:23.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.960+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=LOMYBXJ682DA type=del size=323420 2024-06-28T15:08:23.949 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.960+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=YUYUXGQ04UTM type=del 2024-06-28T15:08:23.951 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.964+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=G477HNZGB7MR type=del size=2148846 2024-06-28T15:08:23.954 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.968+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=SCA24TZP243M type=del 2024-06-28T15:08:23.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.968+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=R8T790B7NYBI type=del size=3078253 2024-06-28T15:08:23.958 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.972+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=PWIQU944S43R type=del 2024-06-28T15:08:23.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.972+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=52BDL3TRMTP9 type=write size=771797 2024-06-28T15:08:23.963 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.976+0000 7f65680598c0 1 < completed write operation key=O1CGONN4TEXF tag=N00108DSKWHG type=write size=552693 2024-06-28T15:08:23.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.976+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=43JQE9IZ9417 type=del size=3868700 2024-06-28T15:08:23.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.980+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=6K18L4UCSTLK type=del 2024-06-28T15:08:23.970 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.984+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=6NR4QXQ2BDMW type=del size=4103470 2024-06-28T15:08:23.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.984+0000 7f65680598c0 1 < completed delete operation key=ZFNIA3O84EB7 tag=2NCD3WW0M4V2 type=del 2024-06-28T15:08:23.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.992+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~2034F44HAWMZ.0 size=1048576 2024-06-28T15:08:23.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.992+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~2034F44HAWMZ.1 size=500290 2024-06-28T15:08:23.980 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.992+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=2034F44HAWMZ tag=BXWZMPG9HGXS type=write size=2597442 2024-06-28T15:08:23.982 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:23.996+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=EDR3V8Q3FQYW type=del 2024-06-28T15:08:23.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.000+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~WCG65HRUT28B.0 size=1048576 2024-06-28T15:08:23.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.000+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~WCG65HRUT28B.1 size=1048576 2024-06-28T15:08:23.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.004+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~WCG65HRUT28B.2 size=111042 2024-06-28T15:08:23.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.004+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=WCG65HRUT28B tag=1VSP7IECSQZP type=write size=3256770 2024-06-28T15:08:23.992 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.004+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=JNNQN4WEB42N type=del 2024-06-28T15:08:23.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.008+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~833F5VX6A26F.0 size=1048576 2024-06-28T15:08:23.999 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.012+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~833F5VX6A26F.1 size=1048576 2024-06-28T15:08:24.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.012+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~833F5VX6A26F.2 size=333245 2024-06-28T15:08:24.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.012+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=833F5VX6A26F tag=FE5OG7PD5E9N type=write size=3478973 2024-06-28T15:08:24.003 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.016+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~EKH6JWIKBRL1.0 size=1048576 2024-06-28T15:08:24.003 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.016+0000 7f65680598c0 1 < canceled part key=_multipart_AXQ0YFWUH4FD.2~EKH6JWIKBRL1.1 size=1048576 2024-06-28T15:08:24.003 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.016+0000 7f65680598c0 1 < canceled part key=_multipart_AXQ0YFWUH4FD.2~EKH6JWIKBRL1.2 size=230527 2024-06-28T15:08:24.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.020+0000 7f65680598c0 1 < canceled multipart upload key=AXQ0YFWUH4FD upload=EKH6JWIKBRL1 tag=HN8FTZ30VI2K type=write size=3376255 2024-06-28T15:08:24.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.024+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=DCGCDI46TRQ8 type=del size=3393231 2024-06-28T15:08:24.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.024+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=QFEKDF145C4W type=del 2024-06-28T15:08:24.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.032+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~L127O332Z6WV.0 size=834786 2024-06-28T15:08:24.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.032+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=L127O332Z6WV tag=T9CNO577M96F type=write size=1883362 2024-06-28T15:08:24.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.032+0000 7f65680598c0 1 < canceled part key=_multipart_4CU86XS1R9G3.2~EVDLSJQ2CT0T.0 size=691810 2024-06-28T15:08:24.025 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.036+0000 7f65680598c0 1 < canceled multipart upload key=4CU86XS1R9G3 upload=EVDLSJQ2CT0T tag=N2LN2DZD7IOD type=write size=1740386 2024-06-28T15:08:24.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.040+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~JZKIGNK2ZK4G.0 size=151287 2024-06-28T15:08:24.028 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.040+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=JZKIGNK2ZK4G tag=XH25KG2E4IOP type=write size=1199863 2024-06-28T15:08:24.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.044+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~F2H04E537I1R.0 size=363138 2024-06-28T15:08:24.033 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.048+0000 7f65680598c0 1 < completed multipart upload key=2F968YIMJN9D upload=F2H04E537I1R tag=5Z9ITVS6QKPN type=write size=1411714 2024-06-28T15:08:24.035 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.048+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=PZMDGL4QHBN4 type=del size=3702499 2024-06-28T15:08:24.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.052+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=B72XZ4WYZCZ4 type=del 2024-06-28T15:08:24.041 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.052+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~WMHI4V7D49D7.0 size=1048576 2024-06-28T15:08:24.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.056+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~WMHI4V7D49D7.1 size=1048576 2024-06-28T15:08:24.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.056+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~WMHI4V7D49D7.2 size=384693 2024-06-28T15:08:24.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.056+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=WMHI4V7D49D7 tag=UDQ72B8G0RK6 type=write size=3530421 2024-06-28T15:08:24.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.060+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~H22GFQSM4JI0.0 size=1048576 2024-06-28T15:08:24.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.060+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~H22GFQSM4JI0.1 size=733057 2024-06-28T15:08:24.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.064+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=H22GFQSM4JI0 tag=LA8V465NRDJS type=write size=2830209 2024-06-28T15:08:24.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.064+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=0ZDJYUFJNS3P type=del size=567833 2024-06-28T15:08:24.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.068+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=U0NRMBD5KOFO type=del 2024-06-28T15:08:24.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.072+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=V7NKAXMLW91R type=del size=70950 2024-06-28T15:08:24.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.072+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=LOMYBXJ682DA type=del 2024-06-28T15:08:24.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.076+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=XT1ILSG5YKE8 type=del size=3919178 2024-06-28T15:08:24.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.076+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=G477HNZGB7MR type=del 2024-06-28T15:08:24.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.080+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=BITOJ13TMD2T type=del size=3374321 2024-06-28T15:08:24.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.080+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=R8T790B7NYBI type=del 2024-06-28T15:08:24.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.084+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=K6NCZEU48RL7 type=del size=2901022 2024-06-28T15:08:24.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.084+0000 7f65680598c0 1 < completed write operation key=RLJCNXKA2GYA tag=52BDL3TRMTP9 type=write size=771797 2024-06-28T15:08:24.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.088+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=JQ8DIIMKMD2L type=del size=576300 2024-06-28T15:08:24.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.088+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=43JQE9IZ9417 type=del 2024-06-28T15:08:24.079 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.092+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=22DB2JNRCHJZ type=write size=480929 2024-06-28T15:08:24.081 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.092+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=6NR4QXQ2BDMW type=del 2024-06-28T15:08:24.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.096+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~7G7NP4IZYKC0.0 size=612265 2024-06-28T15:08:24.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.096+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=7G7NP4IZYKC0 tag=YRQMBA2OEQI2 type=write size=1660841 2024-06-28T15:08:24.087 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.100+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~2034F44HAWMZ.0 size=1048576 2024-06-28T15:08:24.088 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.100+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~2034F44HAWMZ.1 size=500290 2024-06-28T15:08:24.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.104+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=2034F44HAWMZ tag=BXWZMPG9HGXS type=write size=2597442 2024-06-28T15:08:24.093 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.108+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=AXU2IICSQXJF type=del size=2262688 2024-06-28T15:08:24.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.108+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~WCG65HRUT28B.0 size=1048576 2024-06-28T15:08:24.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.108+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~WCG65HRUT28B.1 size=1048576 2024-06-28T15:08:24.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.112+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~WCG65HRUT28B.2 size=111042 2024-06-28T15:08:24.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.116+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=WCG65HRUT28B tag=1VSP7IECSQZP type=write size=3256770 2024-06-28T15:08:24.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.120+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~RRQ524JHMX1L.0 size=1048576 2024-06-28T15:08:24.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.120+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~RRQ524JHMX1L.1 size=968854 2024-06-28T15:08:24.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.120+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=RRQ524JHMX1L tag=D8GWP5AK4LHF type=write size=3066006 2024-06-28T15:08:24.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.124+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~833F5VX6A26F.0 size=1048576 2024-06-28T15:08:24.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.124+0000 7f65680598c0 1 < canceled part key=_multipart_C5C6SAHLPZV1.2~833F5VX6A26F.1 size=1048576 2024-06-28T15:08:24.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.124+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~833F5VX6A26F.2 size=333245 2024-06-28T15:08:24.115 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.128+0000 7f65680598c0 1 < canceled multipart upload key=C5C6SAHLPZV1 upload=833F5VX6A26F tag=FE5OG7PD5E9N type=write size=3478973 2024-06-28T15:08:24.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.132+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=4L4CLG9WWO97 type=write size=158448 2024-06-28T15:08:24.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.132+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=DCGCDI46TRQ8 type=del 2024-06-28T15:08:24.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.136+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=50Q4GO9SVUXT type=del size=401441 2024-06-28T15:08:24.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.140+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~L127O332Z6WV.0 size=834786 2024-06-28T15:08:24.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.144+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=L127O332Z6WV tag=T9CNO577M96F type=write size=1883362 2024-06-28T15:08:24.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.144+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=EX2MINQJOYC0 type=del size=3526472 2024-06-28T15:08:24.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.148+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~JZKIGNK2ZK4G.0 size=151287 2024-06-28T15:08:24.140 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.152+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=JZKIGNK2ZK4G tag=XH25KG2E4IOP type=write size=1199863 2024-06-28T15:08:24.143 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.156+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=RXPP0MMEU7K7 type=write size=731693 2024-06-28T15:08:24.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.160+0000 7f65680598c0 1 < suggested remove operation key=P278ZM1H052S tag=PZMDGL4QHBN4 type=del 2024-06-28T15:08:24.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.168+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~BYIWZ48DSVVH.0 size=1048576 2024-06-28T15:08:24.155 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.168+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~BYIWZ48DSVVH.1 size=523266 2024-06-28T15:08:24.155 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.168+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=BYIWZ48DSVVH tag=3XKCEIG1MXB8 type=write size=2620418 2024-06-28T15:08:24.158 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.172+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~WMHI4V7D49D7.0 size=1048576 2024-06-28T15:08:24.160 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.172+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~WMHI4V7D49D7.1 size=1048576 2024-06-28T15:08:24.162 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.176+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~WMHI4V7D49D7.2 size=384693 2024-06-28T15:08:24.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.180+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=WMHI4V7D49D7 tag=UDQ72B8G0RK6 type=write size=3530421 2024-06-28T15:08:24.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.180+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=PL9FAW5ANBS2 type=del size=829669 2024-06-28T15:08:24.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.184+0000 7f65680598c0 1 < canceled operation key=60QVJVV16QFM tag=0ZDJYUFJNS3P type=del size=567833 2024-06-28T15:08:24.176 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.188+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~T0V3TMXHYCDA.0 size=1048576 2024-06-28T15:08:24.178 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.192+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~T0V3TMXHYCDA.1 size=1048576 2024-06-28T15:08:24.180 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.192+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~T0V3TMXHYCDA.2 size=487942 2024-06-28T15:08:24.180 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.192+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=T0V3TMXHYCDA tag=1ZQQSKJ4WQ39 type=write size=3633670 2024-06-28T15:08:24.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.196+0000 7f65680598c0 1 < canceled operation key=AXQ0YFWUH4FD tag=V7NKAXMLW91R type=del size=70950 2024-06-28T15:08:24.185 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.200+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=F6X4294ZFIV3 type=del size=1944745 2024-06-28T15:08:24.188 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.200+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=XT1ILSG5YKE8 type=del 2024-06-28T15:08:24.191 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.204+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=G2VUNKXD3AL8 type=del size=1690497 2024-06-28T15:08:24.194 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.208+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=BITOJ13TMD2T type=del 2024-06-28T15:08:24.196 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.208+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=480DOQDPOAPO type=del size=1058563 2024-06-28T15:08:24.199 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.212+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=K6NCZEU48RL7 type=del 2024-06-28T15:08:24.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.216+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~DH93B8QIUSML.0 size=1048576 2024-06-28T15:08:24.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.220+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~DH93B8QIUSML.1 size=1048576 2024-06-28T15:08:24.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.220+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~DH93B8QIUSML.2 size=722901 2024-06-28T15:08:24.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.220+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=DH93B8QIUSML tag=MG1VL05UWF6A type=write size=3868629 2024-06-28T15:08:24.211 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.224+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=JQ8DIIMKMD2L type=del 2024-06-28T15:08:24.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.228+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~VEIJGRXRMUFQ.0 size=1048576 2024-06-28T15:08:24.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.232+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~VEIJGRXRMUFQ.1 size=1048576 2024-06-28T15:08:24.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.232+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~VEIJGRXRMUFQ.2 size=295489 2024-06-28T15:08:24.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.232+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=VEIJGRXRMUFQ tag=M3OY7H2FBX3S type=write size=3441217 2024-06-28T15:08:24.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.236+0000 7f65680598c0 1 < completed write operation key=LZX5UZCYLTAG tag=22DB2JNRCHJZ type=write size=480929 2024-06-28T15:08:24.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.236+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=8G9CD0J4UN3J type=del size=3517749 2024-06-28T15:08:24.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.240+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~7G7NP4IZYKC0.0 size=612265 2024-06-28T15:08:24.232 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.244+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=7G7NP4IZYKC0 tag=YRQMBA2OEQI2 type=write size=1660841 2024-06-28T15:08:24.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.252+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~75R52UXB361K.0 size=1048576 2024-06-28T15:08:24.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.256+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~75R52UXB361K.1 size=22218 2024-06-28T15:08:24.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.256+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=75R52UXB361K tag=0HU9BBCZ8JT3 type=write size=2119370 2024-06-28T15:08:24.244 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.256+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=AXU2IICSQXJF type=del 2024-06-28T15:08:24.248 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.260+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~7X3NHNGA8GX2.0 size=1048576 2024-06-28T15:08:24.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.264+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~7X3NHNGA8GX2.1 size=182371 2024-06-28T15:08:24.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.264+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=7X3NHNGA8GX2 tag=GYZRL4MOV56M type=write size=2279523 2024-06-28T15:08:24.251 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.264+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~RRQ524JHMX1L.0 size=1048576 2024-06-28T15:08:24.253 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.268+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~RRQ524JHMX1L.1 size=968854 2024-06-28T15:08:24.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.268+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=RRQ524JHMX1L tag=D8GWP5AK4LHF type=write size=3066006 2024-06-28T15:08:24.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.272+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~N0HZWB6O2LFK.0 size=1048576 2024-06-28T15:08:24.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.276+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~N0HZWB6O2LFK.1 size=910941 2024-06-28T15:08:24.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.276+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=N0HZWB6O2LFK tag=DA9ZJABYC5GZ type=write size=3008093 2024-06-28T15:08:24.267 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.280+0000 7f65680598c0 1 < suggested update operation key=LZX5UZCYLTAG tag=4L4CLG9WWO97 type=write size=158448 2024-06-28T15:08:24.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.280+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=GDI11MTMD0BY type=write size=572914 2024-06-28T15:08:24.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.288+0000 7f65680598c0 1 < suggested remove operation key=1VD7NDPEM3CD tag=50Q4GO9SVUXT type=del 2024-06-28T15:08:24.276 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.288+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=1BOLJFAXC7W0 type=write size=952056 2024-06-28T15:08:24.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.292+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=EX2MINQJOYC0 type=del 2024-06-28T15:08:24.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.296+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~T1EWDRXIS6UN.0 size=158396 2024-06-28T15:08:24.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.296+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=T1EWDRXIS6UN tag=SUOQ2W2DATEC type=write size=1206972 2024-06-28T15:08:24.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.296+0000 7f65680598c0 1 < canceled operation key=ZY973GBGBQ1Z tag=RXPP0MMEU7K7 type=write size=731693 2024-06-28T15:08:24.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.300+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=7MHZG7EPO5BQ type=del size=3593073 2024-06-28T15:08:24.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.300+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~BYIWZ48DSVVH.0 size=1048576 2024-06-28T15:08:24.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.304+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~BYIWZ48DSVVH.1 size=523266 2024-06-28T15:08:24.293 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.308+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=BYIWZ48DSVVH tag=3XKCEIG1MXB8 type=write size=2620418 2024-06-28T15:08:24.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.308+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=UJ4J1MD7T3EJ type=del size=369915 2024-06-28T15:08:24.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.312+0000 7f65680598c0 -1 < failed to complete operation key=LZX5UZCYLTAG tag=PL9FAW5ANBS2 type=del size=829669: (22) Invalid argument 2024-06-28T15:08:24.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.316+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~F835Q8V3TME0.0 size=1048576 2024-06-28T15:08:24.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.316+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~F835Q8V3TME0.1 size=1048576 2024-06-28T15:08:24.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.320+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~F835Q8V3TME0.2 size=467472 2024-06-28T15:08:24.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.320+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=F835Q8V3TME0 tag=BBNKV3O1WKX6 type=write size=3613200 2024-06-28T15:08:24.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.320+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~T0V3TMXHYCDA.0 size=1048576 2024-06-28T15:08:24.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.324+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~T0V3TMXHYCDA.1 size=1048576 2024-06-28T15:08:24.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.324+0000 7f65680598c0 1 < canceled part key=_multipart_J311Q2712YM0.2~T0V3TMXHYCDA.2 size=487942 2024-06-28T15:08:24.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.328+0000 7f65680598c0 1 < canceled multipart upload key=J311Q2712YM0 upload=T0V3TMXHYCDA tag=1ZQQSKJ4WQ39 type=write size=3633670 2024-06-28T15:08:24.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.332+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~95DD1KK27WB4.0 size=1048576 2024-06-28T15:08:24.320 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.332+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~95DD1KK27WB4.1 size=617669 2024-06-28T15:08:24.320 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.332+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=95DD1KK27WB4 tag=JYUNRKCLJDSA type=write size=2714821 2024-06-28T15:08:24.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.336+0000 7f65680598c0 1 < completed delete operation key=2FC9DNEZGWC4 tag=F6X4294ZFIV3 type=del 2024-06-28T15:08:24.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.340+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~IQTAYNMPFVO2.0 size=17837 2024-06-28T15:08:24.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.340+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=IQTAYNMPFVO2 tag=E3JFYPBVZRIT type=write size=1066413 2024-06-28T15:08:24.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.344+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=G2VUNKXD3AL8 type=del 2024-06-28T15:08:24.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.344+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=KDH0AAECX7R5 type=del size=3786538 2024-06-28T15:08:24.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.348+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=480DOQDPOAPO type=del 2024-06-28T15:08:24.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.352+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=KKJMLBAYN8VT type=del size=732458 2024-06-28T15:08:24.342 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.352+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~DH93B8QIUSML.0 size=1048576 2024-06-28T15:08:24.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.356+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~DH93B8QIUSML.1 size=1048576 2024-06-28T15:08:24.345 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.356+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~DH93B8QIUSML.2 size=722901 2024-06-28T15:08:24.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.360+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=DH93B8QIUSML tag=MG1VL05UWF6A type=write size=3868629 2024-06-28T15:08:24.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.368+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~50E3TYRJG4ZY.0 size=1048576 2024-06-28T15:08:24.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.368+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~50E3TYRJG4ZY.1 size=1048576 2024-06-28T15:08:24.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.372+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~50E3TYRJG4ZY.2 size=425015 2024-06-28T15:08:24.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.372+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=50E3TYRJG4ZY tag=A9B3YV6QIS34 type=write size=3570743 2024-06-28T15:08:24.361 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.376+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~VEIJGRXRMUFQ.0 size=1048576 2024-06-28T15:08:24.363 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.376+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~VEIJGRXRMUFQ.1 size=1048576 2024-06-28T15:08:24.365 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.380+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~VEIJGRXRMUFQ.2 size=295489 2024-06-28T15:08:24.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.384+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=VEIJGRXRMUFQ tag=M3OY7H2FBX3S type=write size=3441217 2024-06-28T15:08:24.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.388+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~0XO9OBOL490X.0 size=1048576 2024-06-28T15:08:24.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.388+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~0XO9OBOL490X.1 size=363623 2024-06-28T15:08:24.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.388+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=0XO9OBOL490X tag=DPBY7V2J9TDS type=write size=2460775 2024-06-28T15:08:24.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.392+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=8G9CD0J4UN3J type=del 2024-06-28T15:08:24.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.396+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~0JRTL29A3PPM.0 size=1048576 2024-06-28T15:08:24.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.396+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~0JRTL29A3PPM.1 size=1048576 2024-06-28T15:08:24.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.400+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~0JRTL29A3PPM.2 size=968567 2024-06-28T15:08:24.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.400+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=0JRTL29A3PPM tag=RFWSQVR7SCFZ type=write size=4114295 2024-06-28T15:08:24.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.404+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~75R52UXB361K.0 size=1048576 2024-06-28T15:08:24.392 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.404+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~75R52UXB361K.1 size=22218 2024-06-28T15:08:24.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.408+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=75R52UXB361K tag=0HU9BBCZ8JT3 type=write size=2119370 2024-06-28T15:08:24.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.412+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~VCMH6DQ5WXU5.0 size=1048576 2024-06-28T15:08:24.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.416+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~VCMH6DQ5WXU5.1 size=1048576 2024-06-28T15:08:24.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.420+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~VCMH6DQ5WXU5.2 size=840572 2024-06-28T15:08:24.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.420+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=VCMH6DQ5WXU5 tag=U8ACU1MR3WRC type=write size=3986300 2024-06-28T15:08:24.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.424+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~7X3NHNGA8GX2.0 size=1048576 2024-06-28T15:08:24.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.424+0000 7f65680598c0 1 < canceled part key=_multipart_C5C6SAHLPZV1.2~7X3NHNGA8GX2.1 size=182371 2024-06-28T15:08:24.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.428+0000 7f65680598c0 1 < canceled multipart upload key=C5C6SAHLPZV1 upload=7X3NHNGA8GX2 tag=GYZRL4MOV56M type=write size=2279523 2024-06-28T15:08:24.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.432+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=BFFR2P6F1R0X type=del size=1617997 2024-06-28T15:08:24.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.432+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~N0HZWB6O2LFK.0 size=1048576 2024-06-28T15:08:24.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.436+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~N0HZWB6O2LFK.1 size=910941 2024-06-28T15:08:24.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.440+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=N0HZWB6O2LFK tag=DA9ZJABYC5GZ type=write size=3008093 2024-06-28T15:08:24.431 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.444+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=DMDND8JHKVTV type=write size=925966 2024-06-28T15:08:24.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.452+0000 7f65680598c0 1 < suggested update operation key=7LB8MOJL39RM tag=GDI11MTMD0BY type=write size=572914 2024-06-28T15:08:24.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.456+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~JLD5NRKAQTXY.0 size=1048576 2024-06-28T15:08:24.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.460+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~JLD5NRKAQTXY.1 size=1048576 2024-06-28T15:08:24.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.460+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~JLD5NRKAQTXY.2 size=512649 2024-06-28T15:08:24.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.460+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=JLD5NRKAQTXY tag=Y4OLNCXQBNTE type=write size=3658377 2024-06-28T15:08:24.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.464+0000 7f65680598c0 1 < completed write operation key=47TO7G9EGJKF tag=1BOLJFAXC7W0 type=write size=952056 2024-06-28T15:08:24.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.468+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=VMQ16C1Y7LWJ type=del size=2382689 2024-06-28T15:08:24.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.468+0000 7f65680598c0 1 < canceled part key=_multipart_60QVJVV16QFM.2~T1EWDRXIS6UN.0 size=158396 2024-06-28T15:08:24.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.472+0000 7f65680598c0 1 < canceled multipart upload key=60QVJVV16QFM upload=T1EWDRXIS6UN tag=SUOQ2W2DATEC type=write size=1206972 2024-06-28T15:08:24.462 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.476+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=5DFUCXKF130O type=del size=3801295 2024-06-28T15:08:24.465 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.476+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=7MHZG7EPO5BQ type=del 2024-06-28T15:08:24.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.484+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=PZGUTKKE6SRR type=del size=1479019 2024-06-28T15:08:24.474 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.488+0000 7f65680598c0 1 < canceled operation key=ZY973GBGBQ1Z tag=UJ4J1MD7T3EJ type=del size=369915 2024-06-28T15:08:24.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.492+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~YHTCC02PTPYS.0 size=343073 2024-06-28T15:08:24.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.492+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=YHTCC02PTPYS tag=7JH4IU0USX08 type=write size=1391649 2024-06-28T15:08:24.482 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.496+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~F835Q8V3TME0.0 size=1048576 2024-06-28T15:08:24.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.496+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~F835Q8V3TME0.1 size=1048576 2024-06-28T15:08:24.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.500+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~F835Q8V3TME0.2 size=467472 2024-06-28T15:08:24.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.504+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=F835Q8V3TME0 tag=BBNKV3O1WKX6 type=write size=3613200 2024-06-28T15:08:24.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.508+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=O2XYRJJZD1DU type=del size=3163852 2024-06-28T15:08:24.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.508+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~95DD1KK27WB4.0 size=1048576 2024-06-28T15:08:24.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.512+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~95DD1KK27WB4.1 size=617669 2024-06-28T15:08:24.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.516+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=95DD1KK27WB4 tag=JYUNRKCLJDSA type=write size=2714821 2024-06-28T15:08:24.507 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.520+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~ZW6I2DZ5XN41.0 size=346138 2024-06-28T15:08:24.507 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.520+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=ZW6I2DZ5XN41 tag=PKC6KJNNFPDM type=write size=1394714 2024-06-28T15:08:24.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.524+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~IQTAYNMPFVO2.0 size=17837 2024-06-28T15:08:24.514 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.528+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=IQTAYNMPFVO2 tag=E3JFYPBVZRIT type=write size=1066413 2024-06-28T15:08:24.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.532+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~57XJ0LKH3MVF.0 size=1048576 2024-06-28T15:08:24.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.536+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~57XJ0LKH3MVF.1 size=625591 2024-06-28T15:08:24.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.536+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=57XJ0LKH3MVF tag=HHN9MRZOJXRK type=write size=2722743 2024-06-28T15:08:24.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.536+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=KDH0AAECX7R5 type=del 2024-06-28T15:08:24.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.540+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=7KAH44SPAS9L type=del size=2101769 2024-06-28T15:08:24.530 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.544+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=KKJMLBAYN8VT type=del 2024-06-28T15:08:24.533 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.544+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=0MGZ2AHRQF2D type=del size=1227324 2024-06-28T15:08:24.536 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.548+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~50E3TYRJG4ZY.0 size=1048576 2024-06-28T15:08:24.538 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.552+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~50E3TYRJG4ZY.1 size=1048576 2024-06-28T15:08:24.538 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.552+0000 7f65680598c0 1 < canceled part key=_multipart_GAZQ3PCPKN9Q.2~50E3TYRJG4ZY.2 size=425015 2024-06-28T15:08:24.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.556+0000 7f65680598c0 1 < canceled multipart upload key=GAZQ3PCPKN9Q upload=50E3TYRJG4ZY tag=A9B3YV6QIS34 type=write size=3570743 2024-06-28T15:08:24.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.560+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=EYARPQO9ML2B type=del size=790247 2024-06-28T15:08:24.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.560+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~0XO9OBOL490X.0 size=1048576 2024-06-28T15:08:24.552 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.564+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~0XO9OBOL490X.1 size=363623 2024-06-28T15:08:24.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.568+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=0XO9OBOL490X tag=DPBY7V2J9TDS type=write size=2460775 2024-06-28T15:08:24.561 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.572+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~1EROK7GEEQ6Q.0 size=1048576 2024-06-28T15:08:24.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.576+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~1EROK7GEEQ6Q.1 size=953240 2024-06-28T15:08:24.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.576+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=1EROK7GEEQ6Q tag=QICJGQOZ8MGH type=write size=3050392 2024-06-28T15:08:24.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.580+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~0JRTL29A3PPM.0 size=1048576 2024-06-28T15:08:24.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.580+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~0JRTL29A3PPM.1 size=1048576 2024-06-28T15:08:24.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.584+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~0JRTL29A3PPM.2 size=968567 2024-06-28T15:08:24.576 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.588+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=0JRTL29A3PPM tag=RFWSQVR7SCFZ type=write size=4114295 2024-06-28T15:08:24.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.592+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=J1U22L3SW2OZ type=del size=1801649 2024-06-28T15:08:24.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.596+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~VCMH6DQ5WXU5.0 size=1048576 2024-06-28T15:08:24.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.596+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~VCMH6DQ5WXU5.1 size=1048576 2024-06-28T15:08:24.586 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.600+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~VCMH6DQ5WXU5.2 size=840572 2024-06-28T15:08:24.590 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.604+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=VCMH6DQ5WXU5 tag=U8ACU1MR3WRC type=write size=3986300 2024-06-28T15:08:24.595 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.608+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~XB81MMV6NOU6.0 size=1048576 2024-06-28T15:08:24.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.612+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~XB81MMV6NOU6.1 size=1048576 2024-06-28T15:08:24.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.612+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~XB81MMV6NOU6.2 size=279392 2024-06-28T15:08:24.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.612+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=XB81MMV6NOU6 tag=3S3GFKBZP5W9 type=write size=3425120 2024-06-28T15:08:24.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.616+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=BFFR2P6F1R0X type=del 2024-06-28T15:08:24.605 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.616+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=LUIRQZJ5AALU type=del size=3304154 2024-06-28T15:08:24.610 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.624+0000 7f65680598c0 1 < suggested remove operation key=2FC9DNEZGWC4 tag=DMDND8JHKVTV type=write 2024-06-28T15:08:24.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.628+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~9I83JJ9XU6MJ.0 size=811526 2024-06-28T15:08:24.615 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.628+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=9I83JJ9XU6MJ tag=Q2FF8PW1F97J type=write size=1860102 2024-06-28T15:08:24.615 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.628+0000 7f65680598c0 1 < canceled part key=_multipart_C5C6SAHLPZV1.2~JLD5NRKAQTXY.0 size=1048576 2024-06-28T15:08:24.615 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.628+0000 7f65680598c0 1 < canceled part key=_multipart_C5C6SAHLPZV1.2~JLD5NRKAQTXY.1 size=1048576 2024-06-28T15:08:24.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.632+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~JLD5NRKAQTXY.2 size=512649 2024-06-28T15:08:24.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.636+0000 7f65680598c0 1 < canceled multipart upload key=C5C6SAHLPZV1 upload=JLD5NRKAQTXY tag=Y4OLNCXQBNTE type=write size=3658377 2024-06-28T15:08:24.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.640+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~UV6CFKW3N6T5.0 size=1048576 2024-06-28T15:08:24.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.640+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~UV6CFKW3N6T5.1 size=1048576 2024-06-28T15:08:24.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.644+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~UV6CFKW3N6T5.2 size=193743 2024-06-28T15:08:24.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.644+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=UV6CFKW3N6T5 tag=FV47T8SJH1UX type=write size=3339471 2024-06-28T15:08:24.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.644+0000 7f65680598c0 -1 < failed to complete operation key=2FC9DNEZGWC4 tag=VMQ16C1Y7LWJ type=del size=2382689: (22) Invalid argument 2024-06-28T15:08:24.638 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.652+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~7P8VNG61CI80.0 size=582747 2024-06-28T15:08:24.638 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.652+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=7P8VNG61CI80 tag=5U8URUR2J13C type=write size=1631323 2024-06-28T15:08:24.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.656+0000 7f65680598c0 1 < suggested update operation key=UP3XFF38IJXN tag=5DFUCXKF130O type=del size=3801295 2024-06-28T15:08:24.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.660+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~F7S0C4IRQ7LK.0 size=1048576 2024-06-28T15:08:24.651 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.664+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~F7S0C4IRQ7LK.1 size=457716 2024-06-28T15:08:24.651 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.664+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=F7S0C4IRQ7LK tag=R13W53X93NK2 type=write size=2554868 2024-06-28T15:08:24.654 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.668+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=PZGUTKKE6SRR type=del 2024-06-28T15:08:24.657 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.672+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=70DEZ3EYQD7P type=del size=333676 2024-06-28T15:08:24.660 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.672+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~YHTCC02PTPYS.0 size=343073 2024-06-28T15:08:24.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.680+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=YHTCC02PTPYS tag=7JH4IU0USX08 type=write size=1391649 2024-06-28T15:08:24.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.680+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=ADIC6HQYDU9B type=del size=1150803 2024-06-28T15:08:24.671 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.684+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=O2XYRJJZD1DU type=del 2024-06-28T15:08:24.674 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.688+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=YI2Q42HCXY4N type=write size=914854 2024-06-28T15:08:24.676 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.688+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~ZW6I2DZ5XN41.0 size=346138 2024-06-28T15:08:24.681 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.692+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=ZW6I2DZ5XN41 tag=PKC6KJNNFPDM type=write size=1394714 2024-06-28T15:08:24.684 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.696+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=VNQN12ECB2YL type=del size=2779914 2024-06-28T15:08:24.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.700+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~57XJ0LKH3MVF.0 size=1048576 2024-06-28T15:08:24.690 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.704+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~57XJ0LKH3MVF.1 size=625591 2024-06-28T15:08:24.695 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.708+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=57XJ0LKH3MVF tag=HHN9MRZOJXRK type=write size=2722743 2024-06-28T15:08:24.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.712+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~EW5WSUS96AM9.0 size=253729 2024-06-28T15:08:24.701 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.712+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=EW5WSUS96AM9 tag=MU8MSN5POJDV type=write size=1302305 2024-06-28T15:08:24.706 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.720+0000 7f65680598c0 1 < suggested remove operation key=V2L2Q1U62YW2 tag=7KAH44SPAS9L type=del 2024-06-28T15:08:24.712 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.724+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~M45M07C1X3BN.0 size=1048576 2024-06-28T15:08:24.714 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.728+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~M45M07C1X3BN.1 size=1009838 2024-06-28T15:08:24.714 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.728+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=M45M07C1X3BN tag=L160HH33DW94 type=write size=3106990 2024-06-28T15:08:24.717 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.728+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=0MGZ2AHRQF2D type=del 2024-06-28T15:08:24.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.736+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~BWDV438D2IB6.0 size=636179 2024-06-28T15:08:24.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.736+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=BWDV438D2IB6 tag=4HLLOXQOWONB type=write size=1684755 2024-06-28T15:08:24.729 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.740+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=EYARPQO9ML2B type=del 2024-06-28T15:08:24.732 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.744+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=B9ZQL8GWQCLC type=write size=713524 2024-06-28T15:08:24.736 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.748+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~1EROK7GEEQ6Q.0 size=1048576 2024-06-28T15:08:24.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.752+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~1EROK7GEEQ6Q.1 size=953240 2024-06-28T15:08:24.744 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.756+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=1EROK7GEEQ6Q tag=QICJGQOZ8MGH type=write size=3050392 2024-06-28T15:08:24.750 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.764+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~DWTH5IUB1R9F.0 size=62243 2024-06-28T15:08:24.750 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.764+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=DWTH5IUB1R9F tag=D5FO7975NQE0 type=write size=1110819 2024-06-28T15:08:24.753 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.764+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=J1U22L3SW2OZ type=del 2024-06-28T15:08:24.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.772+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~GEBBDB6L80KA.0 size=1048576 2024-06-28T15:08:24.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.772+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~GEBBDB6L80KA.1 size=1048576 2024-06-28T15:08:24.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.776+0000 7f65680598c0 1 > prepared part key=_multipart_60QVJVV16QFM.2~GEBBDB6L80KA.2 size=23166 2024-06-28T15:08:24.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.776+0000 7f65680598c0 1 > prepared multipart upload key=60QVJVV16QFM upload=GEBBDB6L80KA tag=FWCDI88SNKHL type=write size=3168894 2024-06-28T15:08:24.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.776+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~XB81MMV6NOU6.0 size=1048576 2024-06-28T15:08:24.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.780+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~XB81MMV6NOU6.1 size=1048576 2024-06-28T15:08:24.769 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.780+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~XB81MMV6NOU6.2 size=279392 2024-06-28T15:08:24.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.788+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=XB81MMV6NOU6 tag=3S3GFKBZP5W9 type=write size=3425120 2024-06-28T15:08:24.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.788+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=SE44M16W3UL8 type=del size=2341257 2024-06-28T15:08:24.779 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.792+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=LUIRQZJ5AALU type=del 2024-06-28T15:08:24.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.796+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=12KKLZ2T9NLY type=del size=106454 2024-06-28T15:08:24.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.796+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~9I83JJ9XU6MJ.0 size=811526 2024-06-28T15:08:24.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.804+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=9I83JJ9XU6MJ tag=Q2FF8PW1F97J type=write size=1860102 2024-06-28T15:08:24.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.804+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=MNOAFJJB6KSX type=del size=738899 2024-06-28T15:08:24.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.804+0000 7f65680598c0 1 < canceled part key=_multipart_EN76QQWIQAXZ.2~UV6CFKW3N6T5.0 size=1048576 2024-06-28T15:08:24.795 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.808+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~UV6CFKW3N6T5.1 size=1048576 2024-06-28T15:08:24.797 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.808+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~UV6CFKW3N6T5.2 size=193743 2024-06-28T15:08:24.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.816+0000 7f65680598c0 1 < canceled multipart upload key=EN76QQWIQAXZ upload=UV6CFKW3N6T5 tag=FV47T8SJH1UX type=write size=3339471 2024-06-28T15:08:24.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.816+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=KIW9D7IGBR4J type=del size=1332271 2024-06-28T15:08:24.807 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.820+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~7P8VNG61CI80.0 size=582747 2024-06-28T15:08:24.810 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.824+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=7P8VNG61CI80 tag=5U8URUR2J13C type=write size=1631323 2024-06-28T15:08:24.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.828+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~22308H1WD34H.0 size=1048576 2024-06-28T15:08:24.817 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.828+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~22308H1WD34H.1 size=1048576 2024-06-28T15:08:24.819 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.832+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~22308H1WD34H.2 size=539001 2024-06-28T15:08:24.819 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.832+0000 7f65680598c0 1 > prepared multipart upload key=2F968YIMJN9D upload=22308H1WD34H tag=BORW8DYQDZ00 type=write size=3684729 2024-06-28T15:08:24.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.836+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~F7S0C4IRQ7LK.0 size=1048576 2024-06-28T15:08:24.823 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.836+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~F7S0C4IRQ7LK.1 size=457716 2024-06-28T15:08:24.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.840+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=F7S0C4IRQ7LK tag=R13W53X93NK2 type=write size=2554868 2024-06-28T15:08:24.831 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.844+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~9K56OD39AIKJ.0 size=1048576 2024-06-28T15:08:24.833 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.844+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~9K56OD39AIKJ.1 size=869713 2024-06-28T15:08:24.833 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.844+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=9K56OD39AIKJ tag=IUBPIW46Q4D4 type=write size=2966865 2024-06-28T15:08:24.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.848+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=70DEZ3EYQD7P type=del 2024-06-28T15:08:24.838 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.852+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=5F9113BUKMWO type=del size=2087674 2024-06-28T15:08:24.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.856+0000 7f65680598c0 1 < suggested update operation key=AXQ0YFWUH4FD tag=ADIC6HQYDU9B type=del size=1150803 2024-06-28T15:08:24.845 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.860+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=4VVELKJFT44Y type=del size=422017 2024-06-28T15:08:24.848 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.860+0000 7f65680598c0 1 < completed write operation key=7LB8MOJL39RM tag=YI2Q42HCXY4N type=write size=914854 2024-06-28T15:08:24.851 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.864+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=WZ5JTN9EO9RV type=del size=3018535 2024-06-28T15:08:24.854 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.868+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=VNQN12ECB2YL type=del 2024-06-28T15:08:24.856 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.868+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=7RQIF9OIF51C type=write size=31003 2024-06-28T15:08:24.859 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.872+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~EW5WSUS96AM9.0 size=253729 2024-06-28T15:08:24.863 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.876+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=EW5WSUS96AM9 tag=MU8MSN5POJDV type=write size=1302305 2024-06-28T15:08:24.865 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.880+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=CFXP58TQ28EE type=del size=60720 2024-06-28T15:08:24.868 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.880+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~M45M07C1X3BN.0 size=1048576 2024-06-28T15:08:24.870 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.884+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~M45M07C1X3BN.1 size=1009838 2024-06-28T15:08:24.874 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.888+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=M45M07C1X3BN tag=L160HH33DW94 type=write size=3106990 2024-06-28T15:08:24.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.892+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~BKRA0S44VSTG.0 size=1048576 2024-06-28T15:08:24.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.892+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~BKRA0S44VSTG.1 size=1048576 2024-06-28T15:08:24.883 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.896+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~BKRA0S44VSTG.2 size=929853 2024-06-28T15:08:24.883 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.896+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=BKRA0S44VSTG tag=JDPEYFKCXTR1 type=write size=4075581 2024-06-28T15:08:24.886 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.900+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~BWDV438D2IB6.0 size=636179 2024-06-28T15:08:24.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.904+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=BWDV438D2IB6 tag=4HLLOXQOWONB type=write size=1684755 2024-06-28T15:08:24.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.904+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=SRF7EDXB6AH2 type=del size=162247 2024-06-28T15:08:24.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.908+0000 7f65680598c0 1 < completed write operation key=EN76QQWIQAXZ tag=B9ZQL8GWQCLC type=write size=713524 2024-06-28T15:08:24.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.912+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~TWPD57D6KH5G.0 size=1048576 2024-06-28T15:08:24.901 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.912+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~TWPD57D6KH5G.1 size=1048576 2024-06-28T15:08:24.903 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.916+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~TWPD57D6KH5G.2 size=182543 2024-06-28T15:08:24.903 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.916+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=TWPD57D6KH5G tag=857WQ4GMVPUG type=write size=3328271 2024-06-28T15:08:24.905 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.920+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~DWTH5IUB1R9F.0 size=62243 2024-06-28T15:08:24.909 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.924+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=DWTH5IUB1R9F tag=D5FO7975NQE0 type=write size=1110819 2024-06-28T15:08:24.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.924+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=6JMB3WAFQ6E1 type=write size=338459 2024-06-28T15:08:24.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.928+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~GEBBDB6L80KA.0 size=1048576 2024-06-28T15:08:24.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.928+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~GEBBDB6L80KA.1 size=1048576 2024-06-28T15:08:24.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.932+0000 7f65680598c0 1 < completed part key=_multipart_60QVJVV16QFM.2~GEBBDB6L80KA.2 size=23166 2024-06-28T15:08:24.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.936+0000 7f65680598c0 1 < completed multipart upload key=60QVJVV16QFM upload=GEBBDB6L80KA tag=FWCDI88SNKHL type=write size=3168894 2024-06-28T15:08:24.929 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.940+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~R5VZ0LWB6LVG.0 size=1048576 2024-06-28T15:08:24.931 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.944+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~R5VZ0LWB6LVG.1 size=1048576 2024-06-28T15:08:24.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.944+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~R5VZ0LWB6LVG.2 size=26400 2024-06-28T15:08:24.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.944+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=R5VZ0LWB6LVG tag=LHYGHM0NOE01 type=write size=3172128 2024-06-28T15:08:24.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.952+0000 7f65680598c0 1 < suggested remove operation key=9MQKWSA33R5G tag=SE44M16W3UL8 type=del 2024-06-28T15:08:24.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.956+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=UWWCPO3KG3QJ type=del size=3943323 2024-06-28T15:08:24.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.956+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=12KKLZ2T9NLY type=del 2024-06-28T15:08:24.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.960+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=QSKESBIT8901 type=del size=3808511 2024-06-28T15:08:24.954 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.964+0000 7f65680598c0 1 < suggested remove operation key=D9D3OFIJP3D3 tag=MNOAFJJB6KSX type=del 2024-06-28T15:08:24.955 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.968+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=YJQ3KMSQ9KUK type=write size=405503 2024-06-28T15:08:24.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.972+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=KIW9D7IGBR4J type=del 2024-06-28T15:08:24.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.972+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=EXF79M7FZ20R type=write size=62780 2024-06-28T15:08:24.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.972+0000 7f65680598c0 1 < canceled part key=_multipart_2F968YIMJN9D.2~22308H1WD34H.0 size=1048576 2024-06-28T15:08:24.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.976+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~22308H1WD34H.1 size=1048576 2024-06-28T15:08:24.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.976+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~22308H1WD34H.2 size=539001 2024-06-28T15:08:24.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.984+0000 7f65680598c0 1 < canceled multipart upload key=2F968YIMJN9D upload=22308H1WD34H tag=BORW8DYQDZ00 type=write size=3684729 2024-06-28T15:08:24.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.988+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~Z2BS33UIN1GQ.0 size=1048576 2024-06-28T15:08:24.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.992+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~Z2BS33UIN1GQ.1 size=1036972 2024-06-28T15:08:24.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.992+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=Z2BS33UIN1GQ tag=LUP6H7UJB2JC type=write size=3134124 2024-06-28T15:08:24.981 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.992+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~9K56OD39AIKJ.0 size=1048576 2024-06-28T15:08:24.982 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:24.996+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~9K56OD39AIKJ.1 size=869713 2024-06-28T15:08:24.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.000+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=9K56OD39AIKJ tag=IUBPIW46Q4D4 type=write size=2966865 2024-06-28T15:08:24.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.004+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~K2RCOOH704PK.0 size=1048576 2024-06-28T15:08:24.992 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.004+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~K2RCOOH704PK.1 size=1048576 2024-06-28T15:08:24.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.008+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~K2RCOOH704PK.2 size=644836 2024-06-28T15:08:24.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.008+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=K2RCOOH704PK tag=ECIKQWO3URVN type=write size=3790564 2024-06-28T15:08:24.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.008+0000 7f65680598c0 1 < completed delete operation key=MQ3QWZBFN4YR tag=5F9113BUKMWO type=del 2024-06-28T15:08:24.999 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.012+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=TICLP3L7CMCB type=del size=2466434 2024-06-28T15:08:25.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.012+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=4VVELKJFT44Y type=del 2024-06-28T15:08:25.003 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.016+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=J16YWDLOZX75 type=del size=1204070 2024-06-28T15:08:25.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.020+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=WZ5JTN9EO9RV type=del 2024-06-28T15:08:25.008 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.020+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=XXKXLPYK7SMF type=del size=3616127 2024-06-28T15:08:25.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.024+0000 7f65680598c0 1 < completed write operation key=R0IMFVF5NEOO tag=7RQIF9OIF51C type=write size=31003 2024-06-28T15:08:25.015 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.028+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~Q9KLPUHUUPTS.0 size=787479 2024-06-28T15:08:25.016 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.028+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=Q9KLPUHUUPTS tag=V9O1QENAM75U type=write size=1836055 2024-06-28T15:08:25.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.032+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=CFXP58TQ28EE type=del 2024-06-28T15:08:25.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.032+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=8ES1XM4A092V type=del size=2992813 2024-06-28T15:08:25.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.036+0000 7f65680598c0 1 < canceled part key=_multipart_D9D3OFIJP3D3.2~BKRA0S44VSTG.0 size=1048576 2024-06-28T15:08:25.023 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.036+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~BKRA0S44VSTG.1 size=1048576 2024-06-28T15:08:25.025 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.036+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~BKRA0S44VSTG.2 size=929853 2024-06-28T15:08:25.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.040+0000 7f65680598c0 1 < canceled multipart upload key=D9D3OFIJP3D3 upload=BKRA0S44VSTG tag=JDPEYFKCXTR1 type=write size=4075581 2024-06-28T15:08:25.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.044+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=T47ZR2J1KBFI type=del size=2337746 2024-06-28T15:08:25.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.052+0000 7f65680598c0 1 < suggested remove operation key=EN76QQWIQAXZ tag=SRF7EDXB6AH2 type=del 2024-06-28T15:08:25.041 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.056+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~J7936L0BXKUQ.0 size=551198 2024-06-28T15:08:25.041 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.056+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=J7936L0BXKUQ tag=XNYGA3K5D0CP type=write size=1599774 2024-06-28T15:08:25.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.056+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~TWPD57D6KH5G.0 size=1048576 2024-06-28T15:08:25.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.060+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~TWPD57D6KH5G.1 size=1048576 2024-06-28T15:08:25.047 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.060+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~TWPD57D6KH5G.2 size=182543 2024-06-28T15:08:25.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.064+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=TWPD57D6KH5G tag=857WQ4GMVPUG type=write size=3328271 2024-06-28T15:08:25.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.068+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=7PDP323U3BNX type=del size=2198294 2024-06-28T15:08:25.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.068+0000 7f65680598c0 1 < completed write operation key=MQ3QWZBFN4YR tag=6JMB3WAFQ6E1 type=write size=338459 2024-06-28T15:08:25.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.072+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~520JXQKGVWFM.0 size=1048576 2024-06-28T15:08:25.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.076+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~520JXQKGVWFM.1 size=913949 2024-06-28T15:08:25.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.076+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=520JXQKGVWFM tag=VQ4RHMQJST1M type=write size=3011101 2024-06-28T15:08:25.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.076+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~R5VZ0LWB6LVG.0 size=1048576 2024-06-28T15:08:25.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.080+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~R5VZ0LWB6LVG.1 size=1048576 2024-06-28T15:08:25.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.080+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~R5VZ0LWB6LVG.2 size=26400 2024-06-28T15:08:25.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.088+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=R5VZ0LWB6LVG tag=LHYGHM0NOE01 type=write size=3172128 2024-06-28T15:08:25.078 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.092+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~387Q888QTKCC.0 size=1048576 2024-06-28T15:08:25.080 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.092+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~387Q888QTKCC.1 size=9338 2024-06-28T15:08:25.080 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.092+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=387Q888QTKCC tag=4KQJ29MKYCSZ type=write size=2106490 2024-06-28T15:08:25.082 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.096+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=UWWCPO3KG3QJ type=del 2024-06-28T15:08:25.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.096+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=2Y5LKKY1J9QL type=del size=2808475 2024-06-28T15:08:25.087 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.100+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=QSKESBIT8901 type=del 2024-06-28T15:08:25.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.104+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=2KKU74D1YHDU type=del size=3672382 2024-06-28T15:08:25.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.104+0000 7f65680598c0 1 < completed write operation key=OUID7RX5NVV6 tag=YJQ3KMSQ9KUK type=write size=405503 2024-06-28T15:08:25.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.108+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~30D5TNFXUYRS.0 size=858375 2024-06-28T15:08:25.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.108+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=30D5TNFXUYRS tag=4N22XDMK8M1M type=write size=1906951 2024-06-28T15:08:25.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.112+0000 7f65680598c0 1 < completed write operation key=60QVJVV16QFM tag=EXF79M7FZ20R type=write size=62780 2024-06-28T15:08:25.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.116+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=Y4MIO9ISTE2C type=del size=27721 2024-06-28T15:08:25.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.116+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~Z2BS33UIN1GQ.0 size=1048576 2024-06-28T15:08:25.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.120+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~Z2BS33UIN1GQ.1 size=1036972 2024-06-28T15:08:25.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.124+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=Z2BS33UIN1GQ tag=LUP6H7UJB2JC type=write size=3134124 2024-06-28T15:08:25.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.124+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=2O8V3QM5MUSF type=del size=3118349 2024-06-28T15:08:25.115 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.128+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~K2RCOOH704PK.0 size=1048576 2024-06-28T15:08:25.115 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.128+0000 7f65680598c0 1 < canceled part key=_multipart_O1CGONN4TEXF.2~K2RCOOH704PK.1 size=1048576 2024-06-28T15:08:25.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.128+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~K2RCOOH704PK.2 size=644836 2024-06-28T15:08:25.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.132+0000 7f65680598c0 1 < canceled multipart upload key=O1CGONN4TEXF upload=K2RCOOH704PK tag=ECIKQWO3URVN type=write size=3790564 2024-06-28T15:08:25.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.136+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~7ONFND2AHAF7.0 size=1048576 2024-06-28T15:08:25.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.140+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~7ONFND2AHAF7.1 size=1048576 2024-06-28T15:08:25.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.140+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~7ONFND2AHAF7.2 size=819531 2024-06-28T15:08:25.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.140+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=7ONFND2AHAF7 tag=81GXLS42TVCA type=write size=3965259 2024-06-28T15:08:25.134 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.148+0000 7f65680598c0 1 < suggested update operation key=47TO7G9EGJKF tag=TICLP3L7CMCB type=del size=2466434 2024-06-28T15:08:25.136 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.148+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=VX7GXOP4FGA5 type=del size=1590657 2024-06-28T15:08:25.139 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.152+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=J16YWDLOZX75 type=del 2024-06-28T15:08:25.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.156+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~HYVSXWHN7VKL.0 size=1048576 2024-06-28T15:08:25.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.160+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~HYVSXWHN7VKL.1 size=811675 2024-06-28T15:08:25.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.160+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=HYVSXWHN7VKL tag=2FVM1Z3W3P3A type=write size=2908827 2024-06-28T15:08:25.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.164+0000 7f65680598c0 1 < suggested update operation key=AXQ0YFWUH4FD tag=XXKXLPYK7SMF type=del size=3616127 2024-06-28T15:08:25.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.168+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~XPMV22EWFH1T.0 size=1048576 2024-06-28T15:08:25.158 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.172+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~XPMV22EWFH1T.1 size=1048576 2024-06-28T15:08:25.160 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.172+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~XPMV22EWFH1T.2 size=294277 2024-06-28T15:08:25.160 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.172+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=XPMV22EWFH1T tag=H224XTKV8ZQC type=write size=3440005 2024-06-28T15:08:25.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.176+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~Q9KLPUHUUPTS.0 size=787479 2024-06-28T15:08:25.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.180+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=Q9KLPUHUUPTS tag=V9O1QENAM75U type=write size=1836055 2024-06-28T15:08:25.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.184+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=P73WFU5MZUYJ type=del size=1793653 2024-06-28T15:08:25.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.184+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=8ES1XM4A092V type=del 2024-06-28T15:08:25.177 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.188+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~0M5RMN9YEZCE.0 size=26454 2024-06-28T15:08:25.177 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.188+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=0M5RMN9YEZCE tag=ETPYU2WZJE9L type=write size=1075030 2024-06-28T15:08:25.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.192+0000 7f65680598c0 1 < completed delete operation key=P278ZM1H052S tag=T47ZR2J1KBFI type=del 2024-06-28T15:08:25.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.196+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=0GO4705HOMYS type=del size=4030022 2024-06-28T15:08:25.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.196+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~J7936L0BXKUQ.0 size=551198 2024-06-28T15:08:25.189 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.200+0000 7f65680598c0 1 < completed multipart upload key=MQ3QWZBFN4YR upload=J7936L0BXKUQ tag=XNYGA3K5D0CP type=write size=1599774 2024-06-28T15:08:25.191 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.204+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=ZSZ6N6PN1YBQ type=del size=3514583 2024-06-28T15:08:25.194 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.208+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=7PDP323U3BNX type=del 2024-06-28T15:08:25.196 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.208+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=S9VA47MZRI17 type=write size=211989 2024-06-28T15:08:25.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.212+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~520JXQKGVWFM.0 size=1048576 2024-06-28T15:08:25.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.216+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~520JXQKGVWFM.1 size=913949 2024-06-28T15:08:25.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.220+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=520JXQKGVWFM tag=VQ4RHMQJST1M type=write size=3011101 2024-06-28T15:08:25.210 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.224+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=S2J0X3Y24MPK type=del size=944500 2024-06-28T15:08:25.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.224+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~387Q888QTKCC.0 size=1048576 2024-06-28T15:08:25.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.228+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~387Q888QTKCC.1 size=9338 2024-06-28T15:08:25.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.232+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=387Q888QTKCC tag=4KQJ29MKYCSZ type=write size=2106490 2024-06-28T15:08:25.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.232+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=ANLGEBL95E6B type=del size=3124680 2024-06-28T15:08:25.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.236+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=2Y5LKKY1J9QL type=del 2024-06-28T15:08:25.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.240+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=Q3N5LWEGWOVH type=del size=2640116 2024-06-28T15:08:25.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.240+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=2KKU74D1YHDU type=del 2024-06-28T15:08:25.234 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.248+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~TA74EVMY0YYI.0 size=1048576 2024-06-28T15:08:25.237 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.248+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~TA74EVMY0YYI.1 size=638639 2024-06-28T15:08:25.237 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.248+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=TA74EVMY0YYI tag=GG4KB8MHDNTI type=write size=2735791 2024-06-28T15:08:25.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.252+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~30D5TNFXUYRS.0 size=858375 2024-06-28T15:08:25.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.256+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=30D5TNFXUYRS tag=4N22XDMK8M1M type=write size=1906951 2024-06-28T15:08:25.248 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.260+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~5TATPC955I44.0 size=1048576 2024-06-28T15:08:25.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.264+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~5TATPC955I44.1 size=1048576 2024-06-28T15:08:25.252 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.264+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~5TATPC955I44.2 size=799008 2024-06-28T15:08:25.252 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.264+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=5TATPC955I44 tag=QIFOY1VZ8JC9 type=write size=3944736 2024-06-28T15:08:25.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.268+0000 7f65680598c0 1 < canceled operation key=P278ZM1H052S tag=Y4MIO9ISTE2C type=del size=27721 2024-06-28T15:08:25.258 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.272+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=9RKLRX1W2G78 type=del size=2181867 2024-06-28T15:08:25.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.272+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=2O8V3QM5MUSF type=del 2024-06-28T15:08:25.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.276+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=96GCMM8HM2NC type=del size=747494 2024-06-28T15:08:25.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.280+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~7ONFND2AHAF7.0 size=1048576 2024-06-28T15:08:25.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.280+0000 7f65680598c0 1 < canceled part key=_multipart_47TO7G9EGJKF.2~7ONFND2AHAF7.1 size=1048576 2024-06-28T15:08:25.268 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.280+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~7ONFND2AHAF7.2 size=819531 2024-06-28T15:08:25.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.284+0000 7f65680598c0 1 < canceled multipart upload key=47TO7G9EGJKF upload=7ONFND2AHAF7 tag=81GXLS42TVCA type=write size=3965259 2024-06-28T15:08:25.276 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.288+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~KZK4WIDUCNLR.0 size=1036316 2024-06-28T15:08:25.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.288+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=KZK4WIDUCNLR tag=S36MTFHIUDH6 type=write size=2084892 2024-06-28T15:08:25.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.292+0000 7f65680598c0 1 < canceled operation key=7NL3CTFYY9E5 tag=VX7GXOP4FGA5 type=del size=1590657 2024-06-28T15:08:25.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.296+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=Z5H2Q4ML4OSR type=del size=214343 2024-06-28T15:08:25.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.296+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~HYVSXWHN7VKL.0 size=1048576 2024-06-28T15:08:25.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.300+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~HYVSXWHN7VKL.1 size=811675 2024-06-28T15:08:25.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.304+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=HYVSXWHN7VKL tag=2FVM1Z3W3P3A type=write size=2908827 2024-06-28T15:08:25.294 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.308+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~6MD3VTVFYS3J.0 size=1048576 2024-06-28T15:08:25.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.308+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~6MD3VTVFYS3J.1 size=1048576 2024-06-28T15:08:25.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.312+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~6MD3VTVFYS3J.2 size=899194 2024-06-28T15:08:25.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.312+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=6MD3VTVFYS3J tag=WFL3UPWNC4ID type=write size=4044922 2024-06-28T15:08:25.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.316+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~XPMV22EWFH1T.0 size=1048576 2024-06-28T15:08:25.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.316+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~XPMV22EWFH1T.1 size=1048576 2024-06-28T15:08:25.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.320+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~XPMV22EWFH1T.2 size=294277 2024-06-28T15:08:25.311 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.324+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=XPMV22EWFH1T tag=H224XTKV8ZQC type=write size=3440005 2024-06-28T15:08:25.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.324+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=BR1K68C3352Z type=del size=227287 2024-06-28T15:08:25.316 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.328+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=P73WFU5MZUYJ type=del 2024-06-28T15:08:25.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.332+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=YZC4J08ERRZD type=del size=2603007 2024-06-28T15:08:25.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.336+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~0M5RMN9YEZCE.0 size=26454 2024-06-28T15:08:25.326 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.340+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=0M5RMN9YEZCE tag=ETPYU2WZJE9L type=write size=1075030 2024-06-28T15:08:25.328 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.340+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=2F3DP9G3ZYDY type=del size=3500054 2024-06-28T15:08:25.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.344+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=0GO4705HOMYS type=del 2024-06-28T15:08:25.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.348+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~LS4YJCP4P1LL.0 size=1048576 2024-06-28T15:08:25.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.352+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~LS4YJCP4P1LL.1 size=949148 2024-06-28T15:08:25.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.352+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=LS4YJCP4P1LL tag=MDS76LU85VG3 type=write size=3046300 2024-06-28T15:08:25.342 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.356+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=ZSZ6N6PN1YBQ type=del 2024-06-28T15:08:25.345 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.360+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=UCT6URJ1YKXM type=write size=390459 2024-06-28T15:08:25.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.360+0000 7f65680598c0 1 < completed write operation key=BHW2VHUT0U1U tag=S9VA47MZRI17 type=write size=211989 2024-06-28T15:08:25.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.368+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~58IRPSUU781P.0 size=362882 2024-06-28T15:08:25.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.368+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=58IRPSUU781P tag=SVCRJOUSPU7J type=write size=1411458 2024-06-28T15:08:25.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.372+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=S2J0X3Y24MPK type=del 2024-06-28T15:08:25.361 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.372+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=X00U15D3R58I type=del size=1129613 2024-06-28T15:08:25.364 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.376+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=ANLGEBL95E6B type=del 2024-06-28T15:08:25.368 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.380+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=TNMHKPFOAR3Z type=del size=456815 2024-06-28T15:08:25.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.384+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=Q3N5LWEGWOVH type=del 2024-06-28T15:08:25.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.388+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=87X5ATKWNOOR type=del size=2219468 2024-06-28T15:08:25.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.392+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~TA74EVMY0YYI.0 size=1048576 2024-06-28T15:08:25.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.392+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~TA74EVMY0YYI.1 size=638639 2024-06-28T15:08:25.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.400+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=TA74EVMY0YYI tag=GG4KB8MHDNTI type=write size=2735791 2024-06-28T15:08:25.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.404+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=RC2BC0OT3A2B type=del size=3859984 2024-06-28T15:08:25.393 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.404+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~5TATPC955I44.0 size=1048576 2024-06-28T15:08:25.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.408+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~5TATPC955I44.1 size=1048576 2024-06-28T15:08:25.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.412+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~5TATPC955I44.2 size=799008 2024-06-28T15:08:25.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.416+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=5TATPC955I44 tag=QIFOY1VZ8JC9 type=write size=3944736 2024-06-28T15:08:25.407 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.420+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=BL35DSTOMMJ4 type=write size=736250 2024-06-28T15:08:25.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.424+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=9RKLRX1W2G78 type=del 2024-06-28T15:08:25.416 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.428+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~UNN9YD11NQ5T.0 size=431048 2024-06-28T15:08:25.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.428+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=UNN9YD11NQ5T tag=INC9XYQWFH38 type=write size=1479624 2024-06-28T15:08:25.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.432+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=96GCMM8HM2NC type=del 2024-06-28T15:08:25.427 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.440+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~6CNZJ3E9I687.0 size=960566 2024-06-28T15:08:25.427 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.440+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=6CNZJ3E9I687 tag=KVYP7TN9L1J3 type=write size=2009142 2024-06-28T15:08:25.430 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.444+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~KZK4WIDUCNLR.0 size=1036316 2024-06-28T15:08:25.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.448+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=KZK4WIDUCNLR tag=S36MTFHIUDH6 type=write size=2084892 2024-06-28T15:08:25.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.452+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=0YD6GFBZLYFE type=del size=877878 2024-06-28T15:08:25.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.456+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=Z5H2Q4ML4OSR type=del 2024-06-28T15:08:25.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.460+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~ABHIMU3C23HQ.0 size=1048576 2024-06-28T15:08:25.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.464+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~ABHIMU3C23HQ.1 size=809196 2024-06-28T15:08:25.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.464+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=ABHIMU3C23HQ tag=RA4YF7YL1AUL type=write size=2906348 2024-06-28T15:08:25.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.468+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~6MD3VTVFYS3J.0 size=1048576 2024-06-28T15:08:25.456 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.468+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~6MD3VTVFYS3J.1 size=1048576 2024-06-28T15:08:25.456 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.468+0000 7f65680598c0 1 < canceled part key=_multipart_4CU86XS1R9G3.2~6MD3VTVFYS3J.2 size=899194 2024-06-28T15:08:25.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.472+0000 7f65680598c0 1 < canceled multipart upload key=4CU86XS1R9G3 upload=6MD3VTVFYS3J tag=WFL3UPWNC4ID type=write size=4044922 2024-06-28T15:08:25.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.476+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=1LEMTUDSZ2E4 type=del size=4000721 2024-06-28T15:08:25.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.484+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=BR1K68C3352Z type=del 2024-06-28T15:08:25.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.488+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~UTNQLY0XWY77.0 size=428997 2024-06-28T15:08:25.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.488+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=UTNQLY0XWY77 tag=JJ1ACK7MJPYD type=write size=1477573 2024-06-28T15:08:25.483 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.496+0000 7f65680598c0 1 < suggested remove operation key=2FC9DNEZGWC4 tag=YZC4J08ERRZD type=del 2024-06-28T15:08:25.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.500+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=S474WSCSTDKW type=write size=713462 2024-06-28T15:08:25.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.504+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=2F3DP9G3ZYDY type=del 2024-06-28T15:08:25.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.508+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~J4WVCP6SWQC4.0 size=1048576 2024-06-28T15:08:25.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.512+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~J4WVCP6SWQC4.1 size=1048576 2024-06-28T15:08:25.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.512+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~J4WVCP6SWQC4.2 size=500460 2024-06-28T15:08:25.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.512+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=J4WVCP6SWQC4 tag=HKCP1B09UHMF type=write size=3646188 2024-06-28T15:08:25.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.516+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~LS4YJCP4P1LL.0 size=1048576 2024-06-28T15:08:25.505 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.520+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~LS4YJCP4P1LL.1 size=949148 2024-06-28T15:08:25.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.524+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=LS4YJCP4P1LL tag=MDS76LU85VG3 type=write size=3046300 2024-06-28T15:08:25.516 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.528+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~AVJ1677QPG6U.0 size=1048576 2024-06-28T15:08:25.518 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.532+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~AVJ1677QPG6U.1 size=1048576 2024-06-28T15:08:25.520 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.532+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~AVJ1677QPG6U.2 size=441772 2024-06-28T15:08:25.520 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.532+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=AVJ1677QPG6U tag=6GGKT0O3LY8S type=write size=3587500 2024-06-28T15:08:25.523 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.536+0000 7f65680598c0 1 < completed write operation key=9MQKWSA33R5G tag=UCT6URJ1YKXM type=write size=390459 2024-06-28T15:08:25.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.540+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=04CE1XG1JGI1 type=del size=2327415 2024-06-28T15:08:25.529 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.544+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~58IRPSUU781P.0 size=362882 2024-06-28T15:08:25.534 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.548+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=58IRPSUU781P tag=SVCRJOUSPU7J type=write size=1411458 2024-06-28T15:08:25.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.552+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~5EKEIBHYFI72.0 size=1048576 2024-06-28T15:08:25.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.556+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~5EKEIBHYFI72.1 size=817662 2024-06-28T15:08:25.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.556+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=5EKEIBHYFI72 tag=9KPRWTL6VRGS type=write size=2914814 2024-06-28T15:08:25.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.560+0000 7f65680598c0 1 < suggested update operation key=2F968YIMJN9D tag=X00U15D3R58I type=del size=1129613 2024-06-28T15:08:25.554 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.568+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~NEVL0VPT0DFZ.0 size=1048576 2024-06-28T15:08:25.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.568+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~NEVL0VPT0DFZ.1 size=1048576 2024-06-28T15:08:25.558 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.572+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~NEVL0VPT0DFZ.2 size=748557 2024-06-28T15:08:25.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.572+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=NEVL0VPT0DFZ tag=TX3QB8TUF1NP type=write size=3894285 2024-06-28T15:08:25.562 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.576+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=TNMHKPFOAR3Z type=del 2024-06-28T15:08:25.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.576+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=V3JE1JFMQULM type=del size=2610019 2024-06-28T15:08:25.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.584+0000 7f65680598c0 1 < suggested update operation key=GAZQ3PCPKN9Q tag=87X5ATKWNOOR type=del size=2219468 2024-06-28T15:08:25.575 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.588+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=867OJPIF5H4C type=del size=1451401 2024-06-28T15:08:25.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.592+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=RC2BC0OT3A2B type=del 2024-06-28T15:08:25.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.596+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=SOKLLTAKU94T type=del size=2737649 2024-06-28T15:08:25.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.596+0000 7f65680598c0 1 < completed write operation key=O1CGONN4TEXF tag=BL35DSTOMMJ4 type=write size=736250 2024-06-28T15:08:25.588 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.600+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=HE3KR2V05FF7 type=del size=1255765 2024-06-28T15:08:25.591 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.604+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~UNN9YD11NQ5T.0 size=431048 2024-06-28T15:08:25.595 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.608+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=UNN9YD11NQ5T tag=INC9XYQWFH38 type=write size=1479624 2024-06-28T15:08:25.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.612+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~MRJC32VSEW5V.0 size=346066 2024-06-28T15:08:25.601 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.612+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=MRJC32VSEW5V tag=WDVACSULV023 type=write size=1394642 2024-06-28T15:08:25.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.616+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~6CNZJ3E9I687.0 size=960566 2024-06-28T15:08:25.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.620+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=6CNZJ3E9I687 tag=KVYP7TN9L1J3 type=write size=2009142 2024-06-28T15:08:25.611 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.624+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=WL351F52AP5Y type=del size=2464082 2024-06-28T15:08:25.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.628+0000 7f65680598c0 1 < completed delete operation key=OUID7RX5NVV6 tag=0YD6GFBZLYFE type=del 2024-06-28T15:08:25.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.628+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=JPIK4EZ57E0X type=del size=3323596 2024-06-28T15:08:25.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.632+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~ABHIMU3C23HQ.0 size=1048576 2024-06-28T15:08:25.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.636+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~ABHIMU3C23HQ.1 size=809196 2024-06-28T15:08:25.627 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.640+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=ABHIMU3C23HQ tag=RA4YF7YL1AUL type=write size=2906348 2024-06-28T15:08:25.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.644+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=K0P8ZHA40QLL type=del size=3276575 2024-06-28T15:08:25.636 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.648+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=1LEMTUDSZ2E4 type=del 2024-06-28T15:08:25.638 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.652+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=ZXWZ2H7FQPEM type=del size=3624712 2024-06-28T15:08:25.642 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.656+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~UTNQLY0XWY77.0 size=428997 2024-06-28T15:08:25.648 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.660+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=UTNQLY0XWY77 tag=JJ1ACK7MJPYD type=write size=1477573 2024-06-28T15:08:25.653 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.664+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~G9ZZ5BMDUJOP.0 size=1048576 2024-06-28T15:08:25.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.668+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~G9ZZ5BMDUJOP.1 size=30244 2024-06-28T15:08:25.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.668+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=G9ZZ5BMDUJOP tag=0FN3JUMO2NTA type=write size=2127396 2024-06-28T15:08:25.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.672+0000 7f65680598c0 1 < canceled operation key=D9D3OFIJP3D3 tag=S474WSCSTDKW type=write size=713462 2024-06-28T15:08:25.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.680+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~6LLP59CWXETL.0 size=1048576 2024-06-28T15:08:25.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.680+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~6LLP59CWXETL.1 size=1048576 2024-06-28T15:08:25.670 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.684+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~6LLP59CWXETL.2 size=6350 2024-06-28T15:08:25.670 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.684+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=6LLP59CWXETL tag=X0PSWBA91K0O type=write size=3152078 2024-06-28T15:08:25.671 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.684+0000 7f65680598c0 1 < canceled part key=_multipart_7LB8MOJL39RM.2~J4WVCP6SWQC4.0 size=1048576 2024-06-28T15:08:25.673 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.688+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~J4WVCP6SWQC4.1 size=1048576 2024-06-28T15:08:25.676 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.688+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~J4WVCP6SWQC4.2 size=500460 2024-06-28T15:08:25.682 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.696+0000 7f65680598c0 1 < canceled multipart upload key=7LB8MOJL39RM upload=J4WVCP6SWQC4 tag=HKCP1B09UHMF type=write size=3646188 2024-06-28T15:08:25.685 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.696+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=B76GLJWUW9QS type=del size=2360065 2024-06-28T15:08:25.688 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.700+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~AVJ1677QPG6U.0 size=1048576 2024-06-28T15:08:25.690 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.704+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~AVJ1677QPG6U.1 size=1048576 2024-06-28T15:08:25.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.704+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~AVJ1677QPG6U.2 size=441772 2024-06-28T15:08:25.698 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.712+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=AVJ1677QPG6U tag=6GGKT0O3LY8S type=write size=3587500 2024-06-28T15:08:25.701 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.716+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=CJBIQZTQ85AN type=del size=1677768 2024-06-28T15:08:25.704 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.716+0000 7f65680598c0 1 < completed delete operation key=1VD7NDPEM3CD tag=04CE1XG1JGI1 type=del 2024-06-28T15:08:25.709 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.724+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=WMFFT0K23C6N type=write size=108491 2024-06-28T15:08:25.712 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.724+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~5EKEIBHYFI72.0 size=1048576 2024-06-28T15:08:25.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.728+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~5EKEIBHYFI72.1 size=817662 2024-06-28T15:08:25.720 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.732+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=5EKEIBHYFI72 tag=9KPRWTL6VRGS type=write size=2914814 2024-06-28T15:08:25.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.736+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=GL92ZPIQ095M type=del size=2141423 2024-06-28T15:08:25.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.740+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~NEVL0VPT0DFZ.0 size=1048576 2024-06-28T15:08:25.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.740+0000 7f65680598c0 1 < canceled part key=_multipart_UP3XFF38IJXN.2~NEVL0VPT0DFZ.1 size=1048576 2024-06-28T15:08:25.729 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.740+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~NEVL0VPT0DFZ.2 size=748557 2024-06-28T15:08:25.735 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.748+0000 7f65680598c0 1 < canceled multipart upload key=UP3XFF38IJXN upload=NEVL0VPT0DFZ tag=TX3QB8TUF1NP type=write size=3894285 2024-06-28T15:08:25.738 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.752+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=MQ7BY2ULAV2U type=write size=113840 2024-06-28T15:08:25.744 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.756+0000 7f65680598c0 1 < suggested remove operation key=O1CGONN4TEXF tag=V3JE1JFMQULM type=del 2024-06-28T15:08:25.747 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.760+0000 7f65680598c0 1 > prepared operation key=2FC9DNEZGWC4 tag=ODFI0BP4DVY4 type=write size=65563 2024-06-28T15:08:25.753 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.764+0000 7f65680598c0 1 < suggested remove operation key=J311Q2712YM0 tag=867OJPIF5H4C type=del 2024-06-28T15:08:25.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.776+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~J5MLLPNY7TF2.0 size=1048576 2024-06-28T15:08:25.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.780+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~J5MLLPNY7TF2.1 size=553789 2024-06-28T15:08:25.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.780+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=J5MLLPNY7TF2 tag=9N17B2GGD70S type=write size=2650941 2024-06-28T15:08:25.768 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.780+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=SOKLLTAKU94T type=del 2024-06-28T15:08:25.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.788+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~1GDYKW7YNR9Z.0 size=1048576 2024-06-28T15:08:25.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.788+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~1GDYKW7YNR9Z.1 size=745935 2024-06-28T15:08:25.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.788+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=1GDYKW7YNR9Z tag=MPSROHTVBRRR type=write size=2843087 2024-06-28T15:08:25.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.796+0000 7f65680598c0 1 < suggested remove operation key=47TO7G9EGJKF tag=HE3KR2V05FF7 type=del 2024-06-28T15:08:25.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.796+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=H84UCJWFOP8K type=write size=491664 2024-06-28T15:08:25.789 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.800+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~MRJC32VSEW5V.0 size=346066 2024-06-28T15:08:25.794 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.808+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=MRJC32VSEW5V tag=WDVACSULV023 type=write size=1394642 2024-06-28T15:08:25.798 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.812+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=FSM93UG0SLVY type=del size=2412610 2024-06-28T15:08:25.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.812+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=WL351F52AP5Y type=del 2024-06-28T15:08:25.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.816+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=N7571PNFCWC1 type=del size=3533768 2024-06-28T15:08:25.807 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.820+0000 7f65680598c0 -1 < failed to complete operation key=O1CGONN4TEXF tag=JPIK4EZ57E0X type=del size=3323596: (22) Invalid argument 2024-06-28T15:08:25.810 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.824+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=LFS8C0OQI0EC type=del size=928462 2024-06-28T15:08:25.816 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.828+0000 7f65680598c0 1 < suggested update operation key=R0IMFVF5NEOO tag=K0P8ZHA40QLL type=del size=3276575 2024-06-28T15:08:25.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.836+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~2QSP1Z5JLPSP.0 size=1048576 2024-06-28T15:08:25.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.840+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~2QSP1Z5JLPSP.1 size=434379 2024-06-28T15:08:25.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.840+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=2QSP1Z5JLPSP tag=2OW2PU1XCYF7 type=write size=2531531 2024-06-28T15:08:25.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.844+0000 7f65680598c0 1 < suggested remove operation key=I4RWCQSQVLCN tag=ZXWZ2H7FQPEM type=del 2024-06-28T15:08:25.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.848+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=KOANEPHT439Z type=del size=311842 2024-06-28T15:08:25.839 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.852+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~G9ZZ5BMDUJOP.0 size=1048576 2024-06-28T15:08:25.839 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.852+0000 7f65680598c0 1 < canceled part key=_multipart_R0IMFVF5NEOO.2~G9ZZ5BMDUJOP.1 size=30244 2024-06-28T15:08:25.844 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.856+0000 7f65680598c0 1 < canceled multipart upload key=R0IMFVF5NEOO upload=G9ZZ5BMDUJOP tag=0FN3JUMO2NTA type=write size=2127396 2024-06-28T15:08:25.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.860+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=Y4CQT204TRQA type=del size=3387682 2024-06-28T15:08:25.850 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.864+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~6LLP59CWXETL.0 size=1048576 2024-06-28T15:08:25.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.864+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~6LLP59CWXETL.1 size=1048576 2024-06-28T15:08:25.856 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.868+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~6LLP59CWXETL.2 size=6350 2024-06-28T15:08:25.861 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.872+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=6LLP59CWXETL tag=X0PSWBA91K0O type=write size=3152078 2024-06-28T15:08:25.865 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.876+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=CP5BV6QN9XMH type=write size=797445 2024-06-28T15:08:25.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.884+0000 7f65680598c0 1 < suggested update operation key=7LB8MOJL39RM tag=B76GLJWUW9QS type=del size=2360065 2024-06-28T15:08:25.874 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.888+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=RKZ0Q51W3MDK type=del size=1900034 2024-06-28T15:08:25.877 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.888+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=CJBIQZTQ85AN type=del 2024-06-28T15:08:25.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.892+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=UKRQW3IE1OCO type=write size=71187 2024-06-28T15:08:25.883 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.896+0000 7f65680598c0 1 < completed write operation key=V2L2Q1U62YW2 tag=WMFFT0K23C6N type=write size=108491 2024-06-28T15:08:25.885 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.900+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=SH5PJX0K5VQE type=del size=3276345 2024-06-28T15:08:25.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.900+0000 7f65680598c0 -1 < failed to complete operation key=J311Q2712YM0 tag=GL92ZPIQ095M type=del size=2141423: (22) Invalid argument 2024-06-28T15:08:25.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.908+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~93TTX38SXQ3Z.0 size=1048576 2024-06-28T15:08:25.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.908+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~93TTX38SXQ3Z.1 size=393784 2024-06-28T15:08:25.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.908+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=93TTX38SXQ3Z tag=SGXG7ROFJQ68 type=write size=2490936 2024-06-28T15:08:25.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.912+0000 7f65680598c0 1 < completed write operation key=C5C6SAHLPZV1 tag=MQ7BY2ULAV2U type=write size=113840 2024-06-28T15:08:25.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.916+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=YVBW94OK9UYL type=write size=33724 2024-06-28T15:08:25.905 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.916+0000 7f65680598c0 1 < completed write operation key=2FC9DNEZGWC4 tag=ODFI0BP4DVY4 type=write size=65563 2024-06-28T15:08:25.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.924+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~V4LMVGQOFH2A.0 size=1048576 2024-06-28T15:08:25.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.924+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~V4LMVGQOFH2A.1 size=497732 2024-06-28T15:08:25.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.924+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=V4LMVGQOFH2A tag=E2OX1RCELL6W type=write size=2594884 2024-06-28T15:08:25.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.928+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~J5MLLPNY7TF2.0 size=1048576 2024-06-28T15:08:25.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.932+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~J5MLLPNY7TF2.1 size=553789 2024-06-28T15:08:25.925 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.940+0000 7f65680598c0 1 < completed multipart upload key=MQ3QWZBFN4YR upload=J5MLLPNY7TF2 tag=9N17B2GGD70S type=write size=2650941 2024-06-28T15:08:25.929 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.940+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=VEZUH99WHK1H type=del size=271533 2024-06-28T15:08:25.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.944+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~1GDYKW7YNR9Z.0 size=1048576 2024-06-28T15:08:25.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.948+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~1GDYKW7YNR9Z.1 size=745935 2024-06-28T15:08:25.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.952+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=1GDYKW7YNR9Z tag=MPSROHTVBRRR type=write size=2843087 2024-06-28T15:08:25.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.956+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=VIN85KAW3BW7 type=write size=683250 2024-06-28T15:08:25.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.960+0000 7f65680598c0 1 < completed write operation key=EN76QQWIQAXZ tag=H84UCJWFOP8K type=write size=491664 2024-06-28T15:08:25.951 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.964+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~0HX8GH93JEGY.0 size=129157 2024-06-28T15:08:25.951 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.964+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=0HX8GH93JEGY tag=IJJ4NP8WL8XY type=write size=1177733 2024-06-28T15:08:25.954 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.968+0000 7f65680598c0 1 < canceled operation key=UP3XFF38IJXN tag=FSM93UG0SLVY type=del size=2412610 2024-06-28T15:08:25.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.972+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~8W0B1RUTEEC0.0 size=1048576 2024-06-28T15:08:25.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.976+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~8W0B1RUTEEC0.1 size=1001141 2024-06-28T15:08:25.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.976+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=8W0B1RUTEEC0 tag=K158PF1TYL0J type=write size=3098293 2024-06-28T15:08:25.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.976+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=N7571PNFCWC1 type=del 2024-06-28T15:08:25.970 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.984+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~SAJ066OO0JJQ.0 size=1048576 2024-06-28T15:08:25.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.984+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~SAJ066OO0JJQ.1 size=666423 2024-06-28T15:08:25.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.984+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=SAJ066OO0JJQ tag=JGRS9HQEGHSQ type=write size=2763575 2024-06-28T15:08:25.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.992+0000 7f65680598c0 1 < suggested remove operation key=AXQ0YFWUH4FD tag=LFS8C0OQI0EC type=del 2024-06-28T15:08:25.980 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.992+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=Z64ASH2F1IKF type=write size=724401 2024-06-28T15:08:25.981 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.992+0000 7f65680598c0 1 < canceled part key=_multipart_ZY973GBGBQ1Z.2~2QSP1Z5JLPSP.0 size=1048576 2024-06-28T15:08:25.981 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:25.992+0000 7f65680598c0 1 < canceled part key=_multipart_ZY973GBGBQ1Z.2~2QSP1Z5JLPSP.1 size=434379 2024-06-28T15:08:25.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.000+0000 7f65680598c0 1 < canceled multipart upload key=ZY973GBGBQ1Z upload=2QSP1Z5JLPSP tag=2OW2PU1XCYF7 type=write size=2531531 2024-06-28T15:08:25.991 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.004+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~CG5K0Y06TAVV.0 size=1048576 2024-06-28T15:08:25.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.008+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~CG5K0Y06TAVV.1 size=812823 2024-06-28T15:08:25.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.008+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=CG5K0Y06TAVV tag=L0T8JJ7O1LZV type=write size=2909975 2024-06-28T15:08:25.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.008+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=KOANEPHT439Z type=del 2024-06-28T15:08:26.000 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.012+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=W10SIWLL331E type=write size=354352 2024-06-28T15:08:26.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.020+0000 7f65680598c0 1 < suggested update operation key=9MQKWSA33R5G tag=Y4CQT204TRQA type=del size=3387682 2024-06-28T15:08:26.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.020+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=9ED44KX0HM4O type=del size=4145796 2024-06-28T15:08:26.015 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.028+0000 7f65680598c0 1 < suggested update operation key=UP3XFF38IJXN tag=CP5BV6QN9XMH type=write size=797445 2024-06-28T15:08:26.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.032+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~UMI3VQNWJHPG.0 size=1048576 2024-06-28T15:08:26.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.036+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~UMI3VQNWJHPG.1 size=1048576 2024-06-28T15:08:26.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.036+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~UMI3VQNWJHPG.2 size=532277 2024-06-28T15:08:26.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.036+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=UMI3VQNWJHPG tag=GJPUBSHF3TP6 type=write size=3678005 2024-06-28T15:08:26.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.040+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=RKZ0Q51W3MDK type=del 2024-06-28T15:08:26.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.044+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~NDAYN30DBN7K.0 size=1048576 2024-06-28T15:08:26.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.048+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~NDAYN30DBN7K.1 size=1048576 2024-06-28T15:08:26.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.048+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~NDAYN30DBN7K.2 size=673097 2024-06-28T15:08:26.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.048+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=NDAYN30DBN7K tag=Y4EAZHZWCW31 type=write size=3818825 2024-06-28T15:08:26.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.052+0000 7f65680598c0 1 < completed write operation key=7LB8MOJL39RM tag=UKRQW3IE1OCO type=write size=71187 2024-06-28T15:08:26.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.052+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=KSSR09YXP7FM type=del size=1923045 2024-06-28T15:08:26.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.056+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=SH5PJX0K5VQE type=del 2024-06-28T15:08:26.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.060+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=NZ0053B5HO7R type=del size=2716434 2024-06-28T15:08:26.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.064+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~93TTX38SXQ3Z.0 size=1048576 2024-06-28T15:08:26.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.064+0000 7f65680598c0 1 < canceled part key=_multipart_9MQKWSA33R5G.2~93TTX38SXQ3Z.1 size=393784 2024-06-28T15:08:26.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.068+0000 7f65680598c0 1 < canceled multipart upload key=9MQKWSA33R5G upload=93TTX38SXQ3Z tag=SGXG7ROFJQ68 type=write size=2490936 2024-06-28T15:08:26.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.068+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=VWH3RX1M8375 type=del size=3830002 2024-06-28T15:08:26.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.072+0000 7f65680598c0 1 < suggested remove operation key=I4RWCQSQVLCN tag=YVBW94OK9UYL type=write 2024-06-28T15:08:26.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.076+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~UW928KD7XVUX.0 size=1048576 2024-06-28T15:08:26.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.080+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~UW928KD7XVUX.1 size=250755 2024-06-28T15:08:26.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.080+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=UW928KD7XVUX tag=76261C97USL6 type=write size=2347907 2024-06-28T15:08:26.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.080+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~V4LMVGQOFH2A.0 size=1048576 2024-06-28T15:08:26.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.084+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~V4LMVGQOFH2A.1 size=497732 2024-06-28T15:08:26.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.088+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=V4LMVGQOFH2A tag=E2OX1RCELL6W type=write size=2594884 2024-06-28T15:08:26.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.088+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=1SWJJHE4GBO9 type=del size=1751436 2024-06-28T15:08:26.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.092+0000 7f65680598c0 -1 < failed to complete operation key=AXQ0YFWUH4FD tag=VEZUH99WHK1H type=del size=271533: (22) Invalid argument 2024-06-28T15:08:26.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.096+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=IL1MMWZY12OQ type=del size=2207932 2024-06-28T15:08:26.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.100+0000 7f65680598c0 1 < suggested update operation key=UP3XFF38IJXN tag=VIN85KAW3BW7 type=write size=683250 2024-06-28T15:08:26.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.100+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=HFLPTKONZ9EN type=del size=2052390 2024-06-28T15:08:26.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.104+0000 7f65680598c0 1 < canceled part key=_multipart_UP3XFF38IJXN.2~0HX8GH93JEGY.0 size=129157 2024-06-28T15:08:26.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.104+0000 7f65680598c0 1 < canceled multipart upload key=UP3XFF38IJXN upload=0HX8GH93JEGY tag=IJJ4NP8WL8XY type=write size=1177733 2024-06-28T15:08:26.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.112+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~RXB03GGIXPLI.0 size=1048576 2024-06-28T15:08:26.101 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.112+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~RXB03GGIXPLI.1 size=287262 2024-06-28T15:08:26.101 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.112+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=RXB03GGIXPLI tag=I7CU8IXW355K type=write size=2384414 2024-06-28T15:08:26.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.116+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~8W0B1RUTEEC0.0 size=1048576 2024-06-28T15:08:26.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.116+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~8W0B1RUTEEC0.1 size=1001141 2024-06-28T15:08:26.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.120+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=8W0B1RUTEEC0 tag=K158PF1TYL0J type=write size=3098293 2024-06-28T15:08:26.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.124+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=J3X9BSK60JRT type=del size=2427941 2024-06-28T15:08:26.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.124+0000 7f65680598c0 1 < canceled part key=_multipart_V2L2Q1U62YW2.2~SAJ066OO0JJQ.0 size=1048576 2024-06-28T15:08:26.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.124+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~SAJ066OO0JJQ.1 size=666423 2024-06-28T15:08:26.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.128+0000 7f65680598c0 1 < canceled multipart upload key=V2L2Q1U62YW2 upload=SAJ066OO0JJQ tag=JGRS9HQEGHSQ type=write size=2763575 2024-06-28T15:08:26.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.132+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=W78LI6GX9VWJ type=del size=1561906 2024-06-28T15:08:26.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.136+0000 7f65680598c0 1 < completed write operation key=O1CGONN4TEXF tag=Z64ASH2F1IKF type=write size=724401 2024-06-28T15:08:26.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.140+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~B1BORD30ABGK.0 size=1048576 2024-06-28T15:08:26.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.140+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~B1BORD30ABGK.1 size=1048576 2024-06-28T15:08:26.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.144+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~B1BORD30ABGK.2 size=557465 2024-06-28T15:08:26.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.144+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=B1BORD30ABGK tag=S8CFM3RQHHTN type=write size=3703193 2024-06-28T15:08:26.131 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.144+0000 7f65680598c0 1 < canceled part key=_multipart_KFHGA64L1NLP.2~CG5K0Y06TAVV.0 size=1048576 2024-06-28T15:08:26.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.144+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~CG5K0Y06TAVV.1 size=812823 2024-06-28T15:08:26.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.152+0000 7f65680598c0 1 < canceled multipart upload key=KFHGA64L1NLP upload=CG5K0Y06TAVV tag=L0T8JJ7O1LZV type=write size=2909975 2024-06-28T15:08:26.140 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.152+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=N51K16QMM72H type=write size=509883 2024-06-28T15:08:26.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.160+0000 7f65680598c0 1 < suggested update operation key=4CU86XS1R9G3 tag=W10SIWLL331E type=write size=354352 2024-06-28T15:08:26.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.164+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~4G957BWEZMRU.0 size=1048576 2024-06-28T15:08:26.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.164+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~4G957BWEZMRU.1 size=589490 2024-06-28T15:08:26.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.164+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=4G957BWEZMRU tag=VB2Q1C9P1ODX type=write size=2686642 2024-06-28T15:08:26.155 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.168+0000 7f65680598c0 -1 < failed to complete operation key=I4RWCQSQVLCN tag=9ED44KX0HM4O type=del size=4145796: (22) Invalid argument 2024-06-28T15:08:26.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.172+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~QP9FUPC5EKHV.0 size=421160 2024-06-28T15:08:26.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.172+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=QP9FUPC5EKHV tag=0EWZ3H6W43JH type=write size=1469736 2024-06-28T15:08:26.164 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.176+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~UMI3VQNWJHPG.0 size=1048576 2024-06-28T15:08:26.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.180+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~UMI3VQNWJHPG.1 size=1048576 2024-06-28T15:08:26.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.180+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~UMI3VQNWJHPG.2 size=532277 2024-06-28T15:08:26.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.188+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=UMI3VQNWJHPG tag=GJPUBSHF3TP6 type=write size=3678005 2024-06-28T15:08:26.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.192+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~8O75B0425ES8.0 size=1048576 2024-06-28T15:08:26.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.196+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~8O75B0425ES8.1 size=704803 2024-06-28T15:08:26.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.196+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=8O75B0425ES8 tag=JA6I5A4YSWEW type=write size=2801955 2024-06-28T15:08:26.185 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.196+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~NDAYN30DBN7K.0 size=1048576 2024-06-28T15:08:26.187 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.200+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~NDAYN30DBN7K.1 size=1048576 2024-06-28T15:08:26.187 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.200+0000 7f65680598c0 1 < canceled part key=_multipart_BHW2VHUT0U1U.2~NDAYN30DBN7K.2 size=673097 2024-06-28T15:08:26.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.204+0000 7f65680598c0 1 < canceled multipart upload key=BHW2VHUT0U1U upload=NDAYN30DBN7K tag=Y4EAZHZWCW31 type=write size=3818825 2024-06-28T15:08:26.195 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.208+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=S45LPFUGVRC0 type=del size=2962454 2024-06-28T15:08:26.198 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.212+0000 7f65680598c0 1 < completed delete operation key=ZY973GBGBQ1Z tag=KSSR09YXP7FM type=del 2024-06-28T15:08:26.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.212+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=JUNS3Q3W5MR0 type=del size=827269 2024-06-28T15:08:26.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.216+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=NZ0053B5HO7R type=del 2024-06-28T15:08:26.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.220+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~VAPM569HFPYL.0 size=1048576 2024-06-28T15:08:26.210 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.224+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~VAPM569HFPYL.1 size=1048576 2024-06-28T15:08:26.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.224+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~VAPM569HFPYL.2 size=919651 2024-06-28T15:08:26.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.224+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=VAPM569HFPYL tag=B379DOMMNAXA type=write size=4065379 2024-06-28T15:08:26.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.228+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=VWH3RX1M8375 type=del 2024-06-28T15:08:26.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.232+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=1J7B17YYMY3J type=write size=110962 2024-06-28T15:08:26.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.236+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~UW928KD7XVUX.0 size=1048576 2024-06-28T15:08:26.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.240+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~UW928KD7XVUX.1 size=250755 2024-06-28T15:08:26.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.244+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=UW928KD7XVUX tag=76261C97USL6 type=write size=2347907 2024-06-28T15:08:26.234 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.248+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=FA4N67O70L1G type=del size=3717764 2024-06-28T15:08:26.237 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.248+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=1SWJJHE4GBO9 type=del 2024-06-28T15:08:26.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.252+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=4XGD6YX2HIO3 type=del size=534210 2024-06-28T15:08:26.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.256+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=IL1MMWZY12OQ type=del 2024-06-28T15:08:26.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.260+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=S55YBJ9VPM3I type=write size=765154 2024-06-28T15:08:26.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.264+0000 7f65680598c0 1 < canceled operation key=47TO7G9EGJKF tag=HFLPTKONZ9EN type=del size=2052390 2024-06-28T15:08:26.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.268+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~2NMZ092KMUJ3.0 size=196717 2024-06-28T15:08:26.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.268+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=2NMZ092KMUJ3 tag=VMA31JKT1CAD type=write size=1245293 2024-06-28T15:08:26.258 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.272+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~RXB03GGIXPLI.0 size=1048576 2024-06-28T15:08:26.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.272+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~RXB03GGIXPLI.1 size=287262 2024-06-28T15:08:26.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.280+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=RXB03GGIXPLI tag=I7CU8IXW355K type=write size=2384414 2024-06-28T15:08:26.270 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.284+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=X0XLW0OSIAEO type=del size=2199643 2024-06-28T15:08:26.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.284+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=J3X9BSK60JRT type=del 2024-06-28T15:08:26.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.288+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=I1EIEBMXGJX8 type=del size=2297181 2024-06-28T15:08:26.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.292+0000 7f65680598c0 1 < canceled operation key=7NL3CTFYY9E5 tag=W78LI6GX9VWJ type=del size=1561906 2024-06-28T15:08:26.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.296+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=2A608AUYAYWU type=del size=1547616 2024-06-28T15:08:26.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.296+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~B1BORD30ABGK.0 size=1048576 2024-06-28T15:08:26.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.300+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~B1BORD30ABGK.1 size=1048576 2024-06-28T15:08:26.289 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.300+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~B1BORD30ABGK.2 size=557465 2024-06-28T15:08:26.294 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.308+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=B1BORD30ABGK tag=S8CFM3RQHHTN type=write size=3703193 2024-06-28T15:08:26.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.316+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~A7B5K3Y3YERH.0 size=1048576 2024-06-28T15:08:26.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.316+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~A7B5K3Y3YERH.1 size=395414 2024-06-28T15:08:26.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.316+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=A7B5K3Y3YERH tag=9EXY70ZZEBZR type=write size=2492566 2024-06-28T15:08:26.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.320+0000 7f65680598c0 -1 < failed to complete operation key=4CU86XS1R9G3 tag=N51K16QMM72H type=write size=509883: (22) Invalid argument 2024-06-28T15:08:26.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.324+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=4R0590TFDMRS type=del size=1344038 2024-06-28T15:08:26.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.324+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~4G957BWEZMRU.0 size=1048576 2024-06-28T15:08:26.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.328+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~4G957BWEZMRU.1 size=589490 2024-06-28T15:08:26.320 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.332+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=4G957BWEZMRU tag=VB2Q1C9P1ODX type=write size=2686642 2024-06-28T15:08:26.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.336+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=W3OO0Z0VK5KO type=del size=2657732 2024-06-28T15:08:26.326 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.340+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~QP9FUPC5EKHV.0 size=421160 2024-06-28T15:08:26.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.344+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=QP9FUPC5EKHV tag=0EWZ3H6W43JH type=write size=1469736 2024-06-28T15:08:26.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.348+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=HLM0JLW6MYQB type=del size=772068 2024-06-28T15:08:26.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.352+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~8O75B0425ES8.0 size=1048576 2024-06-28T15:08:26.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.352+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~8O75B0425ES8.1 size=704803 2024-06-28T15:08:26.345 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.356+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=8O75B0425ES8 tag=JA6I5A4YSWEW type=write size=2801955 2024-06-28T15:08:26.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.360+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=RJVDTZBFNERN type=del size=3353119 2024-06-28T15:08:26.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.364+0000 7f65680598c0 1 < canceled operation key=9MQKWSA33R5G tag=S45LPFUGVRC0 type=del size=2962454 2024-06-28T15:08:26.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.368+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=YMZ04NW2MBBL type=write size=295342 2024-06-28T15:08:26.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.372+0000 7f65680598c0 1 < suggested update operation key=I4RWCQSQVLCN tag=JUNS3Q3W5MR0 type=del size=827269 2024-06-28T15:08:26.365 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.376+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~XKGCDXTOZPDT.0 size=1048576 2024-06-28T15:08:26.367 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.380+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~XKGCDXTOZPDT.1 size=300591 2024-06-28T15:08:26.367 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.380+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=XKGCDXTOZPDT tag=GRNJ2ODAZDFO type=write size=2397743 2024-06-28T15:08:26.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.384+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~VAPM569HFPYL.0 size=1048576 2024-06-28T15:08:26.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.384+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~VAPM569HFPYL.1 size=1048576 2024-06-28T15:08:26.375 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.388+0000 7f65680598c0 1 < completed part key=_multipart_LZX5UZCYLTAG.2~VAPM569HFPYL.2 size=919651 2024-06-28T15:08:26.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.392+0000 7f65680598c0 1 < completed multipart upload key=LZX5UZCYLTAG upload=VAPM569HFPYL tag=B379DOMMNAXA type=write size=4065379 2024-06-28T15:08:26.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.396+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~CA8XN3Y9A2LF.0 size=94479 2024-06-28T15:08:26.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.396+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=CA8XN3Y9A2LF tag=P0HRN2S599WU type=write size=1143055 2024-06-28T15:08:26.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.400+0000 7f65680598c0 1 < completed write operation key=P278ZM1H052S tag=1J7B17YYMY3J type=write size=110962 2024-06-28T15:08:26.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.404+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=YBB0WC0RHY8N type=del size=720602 2024-06-28T15:08:26.393 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.408+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=FA4N67O70L1G type=del 2024-06-28T15:08:26.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.408+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=MI4PVJ3FTUGV type=del size=387043 2024-06-28T15:08:26.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.412+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=4XGD6YX2HIO3 type=del 2024-06-28T15:08:26.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.416+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=3A4R2U5L0X14 type=del size=334273 2024-06-28T15:08:26.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.416+0000 7f65680598c0 1 < completed write operation key=AXQ0YFWUH4FD tag=S55YBJ9VPM3I type=write size=765154 2024-06-28T15:08:26.411 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.424+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~OMRM2ONFY0ET.0 size=1048576 2024-06-28T15:08:26.413 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.424+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~OMRM2ONFY0ET.1 size=85589 2024-06-28T15:08:26.413 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.424+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=OMRM2ONFY0ET tag=XA41Z2H2UAXN type=write size=2182741 2024-06-28T15:08:26.416 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.428+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~2NMZ092KMUJ3.0 size=196717 2024-06-28T15:08:26.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.436+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=2NMZ092KMUJ3 tag=VMA31JKT1CAD type=write size=1245293 2024-06-28T15:08:26.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.436+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=DRC5D94MVFKF type=del size=3268850 2024-06-28T15:08:26.427 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.440+0000 7f65680598c0 1 < canceled operation key=AOBFAR0UTONH tag=X0XLW0OSIAEO type=del size=2199643 2024-06-28T15:08:26.430 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.444+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=ZCPXFR8Z7EHA type=del size=99880 2024-06-28T15:08:26.433 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.448+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=I1EIEBMXGJX8 type=del 2024-06-28T15:08:26.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.448+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=DGVIY4VNW70G type=write size=975115 2024-06-28T15:08:26.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.452+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=2A608AUYAYWU type=del 2024-06-28T15:08:26.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.456+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=CGPCF5L241X2 type=del size=488563 2024-06-28T15:08:26.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.460+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~A7B5K3Y3YERH.0 size=1048576 2024-06-28T15:08:26.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.460+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~A7B5K3Y3YERH.1 size=395414 2024-06-28T15:08:26.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.468+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=A7B5K3Y3YERH tag=9EXY70ZZEBZR type=write size=2492566 2024-06-28T15:08:26.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.472+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=04GTHYX3JLQY type=del size=2247747 2024-06-28T15:08:26.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.472+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=4R0590TFDMRS type=del 2024-06-28T15:08:26.466 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.480+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~OFI5SZ8VXH6G.0 size=1048576 2024-06-28T15:08:26.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.480+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~OFI5SZ8VXH6G.1 size=951965 2024-06-28T15:08:26.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.480+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=OFI5SZ8VXH6G tag=7N9MHFW6UN9M type=write size=3049117 2024-06-28T15:08:26.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.488+0000 7f65680598c0 1 < suggested remove operation key=MQ3QWZBFN4YR tag=W3OO0Z0VK5KO type=del 2024-06-28T15:08:26.478 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.492+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=SFHYIV057O4N type=del size=1530461 2024-06-28T15:08:26.481 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.492+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=HLM0JLW6MYQB type=del 2024-06-28T15:08:26.484 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.496+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=O2D20O3GLHAN type=del size=1275040 2024-06-28T15:08:26.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.504+0000 7f65680598c0 1 < suggested remove operation key=AXQ0YFWUH4FD tag=RJVDTZBFNERN type=del 2024-06-28T15:08:26.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.504+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=EQ9AY3EAADWB type=del size=2347554 2024-06-28T15:08:26.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.508+0000 7f65680598c0 1 < completed write operation key=UP3XFF38IJXN tag=YMZ04NW2MBBL type=write size=295342 2024-06-28T15:08:26.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.512+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=QFHYG9WTZCD2 type=del size=4107222 2024-06-28T15:08:26.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.516+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~XKGCDXTOZPDT.0 size=1048576 2024-06-28T15:08:26.504 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.516+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~XKGCDXTOZPDT.1 size=300591 2024-06-28T15:08:26.509 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.520+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=XKGCDXTOZPDT tag=GRNJ2ODAZDFO type=write size=2397743 2024-06-28T15:08:26.512 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.524+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=2UQCISEXR8HR type=del size=2737287 2024-06-28T15:08:26.515 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.528+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~CA8XN3Y9A2LF.0 size=94479 2024-06-28T15:08:26.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.532+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=CA8XN3Y9A2LF tag=P0HRN2S599WU type=write size=1143055 2024-06-28T15:08:26.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.536+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~4Q6QU4AVG3OW.0 size=1048576 2024-06-28T15:08:26.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.540+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~4Q6QU4AVG3OW.1 size=1048576 2024-06-28T15:08:26.530 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.544+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~4Q6QU4AVG3OW.2 size=753798 2024-06-28T15:08:26.530 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.544+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=4Q6QU4AVG3OW tag=UL63UKY5UJGT type=write size=3899526 2024-06-28T15:08:26.535 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.548+0000 7f65680598c0 1 < suggested remove operation key=ZFNIA3O84EB7 tag=YBB0WC0RHY8N type=del 2024-06-28T15:08:26.538 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.552+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=BXXO2BC7S56V type=del size=2142922 2024-06-28T15:08:26.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.556+0000 7f65680598c0 1 < suggested update operation key=9MQKWSA33R5G tag=MI4PVJ3FTUGV type=del size=387043 2024-06-28T15:08:26.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.560+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=RRVN8WX64GH2 type=del size=1171987 2024-06-28T15:08:26.551 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.564+0000 7f65680598c0 1 < canceled operation key=AOBFAR0UTONH tag=3A4R2U5L0X14 type=del size=334273 2024-06-28T15:08:26.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.568+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~AIU27VTDZD8Q.0 size=3342 2024-06-28T15:08:26.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.568+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=AIU27VTDZD8Q tag=CMGT9EEJ48CA type=write size=1051918 2024-06-28T15:08:26.559 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.572+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~OMRM2ONFY0ET.0 size=1048576 2024-06-28T15:08:26.562 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.576+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~OMRM2ONFY0ET.1 size=85589 2024-06-28T15:08:26.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.580+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=OMRM2ONFY0ET tag=XA41Z2H2UAXN type=write size=2182741 2024-06-28T15:08:26.575 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.588+0000 7f65680598c0 1 > prepared part key=_multipart_LZX5UZCYLTAG.2~J86QBU2WER49.0 size=629024 2024-06-28T15:08:26.575 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.588+0000 7f65680598c0 1 > prepared multipart upload key=LZX5UZCYLTAG upload=J86QBU2WER49 tag=SUD7B4PG0Y33 type=write size=1677600 2024-06-28T15:08:26.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.596+0000 7f65680598c0 1 < suggested update operation key=ZFNIA3O84EB7 tag=DRC5D94MVFKF type=del size=3268850 2024-06-28T15:08:26.590 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.604+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=IOQY7W0E0PDA type=del size=1681500 2024-06-28T15:08:26.593 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.604+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=ZCPXFR8Z7EHA type=del 2024-06-28T15:08:26.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.612+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~MOHJP0E302TZ.0 size=217432 2024-06-28T15:08:26.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.612+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=MOHJP0E302TZ tag=Y5LGGS9JDMR3 type=write size=1266008 2024-06-28T15:08:26.605 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.616+0000 7f65680598c0 1 < suggested update operation key=C5C6SAHLPZV1 tag=DGVIY4VNW70G type=write size=975115 2024-06-28T15:08:26.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.620+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=DQ2OWZ79QOWU type=del size=3550154 2024-06-28T15:08:26.611 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.624+0000 7f65680598c0 1 < canceled operation key=KFHGA64L1NLP tag=CGPCF5L241X2 type=del size=488563 2024-06-28T15:08:26.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.632+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~Z297QKBRAQQR.0 size=1048576 2024-06-28T15:08:26.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.632+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~Z297QKBRAQQR.1 size=1048576 2024-06-28T15:08:26.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.636+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~Z297QKBRAQQR.2 size=1034151 2024-06-28T15:08:26.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.636+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=Z297QKBRAQQR tag=X021E63S3XIX type=write size=4179879 2024-06-28T15:08:26.625 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.636+0000 7f65680598c0 -1 < failed to complete operation key=MQ3QWZBFN4YR tag=04GTHYX3JLQY type=del size=2247747: (22) Invalid argument 2024-06-28T15:08:26.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.640+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=442LIV32UO8Y type=del size=2099342 2024-06-28T15:08:26.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.644+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~OFI5SZ8VXH6G.0 size=1048576 2024-06-28T15:08:26.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.648+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~OFI5SZ8VXH6G.1 size=951965 2024-06-28T15:08:26.641 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.652+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=OFI5SZ8VXH6G tag=7N9MHFW6UN9M type=write size=3049117 2024-06-28T15:08:26.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.656+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=T0Q76QJB0C7J type=del size=3270489 2024-06-28T15:08:26.647 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.660+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=SFHYIV057O4N type=del 2024-06-28T15:08:26.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.664+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=Q1G449QB0YJQ type=del size=3909826 2024-06-28T15:08:26.653 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.668+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=O2D20O3GLHAN type=del 2024-06-28T15:08:26.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.668+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=E61D9Q116C0P type=del size=3935715 2024-06-28T15:08:26.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.672+0000 7f65680598c0 1 < canceled operation key=MQ3QWZBFN4YR tag=EQ9AY3EAADWB type=del size=2347554 2024-06-28T15:08:26.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.676+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~Z5IL38S7DIHT.0 size=1048576 2024-06-28T15:08:26.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.680+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~Z5IL38S7DIHT.1 size=423057 2024-06-28T15:08:26.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.680+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=Z5IL38S7DIHT tag=XZZ6REKA3BWX type=write size=2520209 2024-06-28T15:08:26.671 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.684+0000 7f65680598c0 1 < canceled operation key=AXQ0YFWUH4FD tag=QFHYG9WTZCD2 type=del size=4107222 2024-06-28T15:08:26.677 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.688+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~M1D7EPJ9I5IN.0 size=148777 2024-06-28T15:08:26.677 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.688+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=M1D7EPJ9I5IN tag=HT4L1TL4EM1S type=write size=1197353 2024-06-28T15:08:26.680 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.692+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=2UQCISEXR8HR type=del 2024-06-28T15:08:26.686 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.700+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~KDUTILOY6CO9.0 size=1048576 2024-06-28T15:08:26.688 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.700+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~KDUTILOY6CO9.1 size=351061 2024-06-28T15:08:26.688 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.700+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=KDUTILOY6CO9 tag=E7GOY8IH0TC1 type=write size=2448213 2024-06-28T15:08:26.689 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.700+0000 7f65680598c0 1 < canceled part key=_multipart_AOBFAR0UTONH.2~4Q6QU4AVG3OW.0 size=1048576 2024-06-28T15:08:26.689 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.700+0000 7f65680598c0 1 < canceled part key=_multipart_AOBFAR0UTONH.2~4Q6QU4AVG3OW.1 size=1048576 2024-06-28T15:08:26.691 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.704+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~4Q6QU4AVG3OW.2 size=753798 2024-06-28T15:08:26.697 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.708+0000 7f65680598c0 1 < canceled multipart upload key=AOBFAR0UTONH upload=4Q6QU4AVG3OW tag=UL63UKY5UJGT type=write size=3899526 2024-06-28T15:08:26.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.712+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=A8ZAJXGA41Z4 type=del size=4034014 2024-06-28T15:08:26.706 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.720+0000 7f65680598c0 1 < suggested update operation key=47TO7G9EGJKF tag=BXXO2BC7S56V type=del size=2142922 2024-06-28T15:08:26.708 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.720+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=54HJCQ1YIPM0 type=del size=730639 2024-06-28T15:08:26.711 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.724+0000 7f65680598c0 1 < canceled operation key=4CU86XS1R9G3 tag=RRVN8WX64GH2 type=del size=1171987 2024-06-28T15:08:26.714 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.728+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=3QLBEXLZMSOJ type=del size=2973839 2024-06-28T15:08:26.718 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.732+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~AIU27VTDZD8Q.0 size=3342 2024-06-28T15:08:26.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.736+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=AIU27VTDZD8Q tag=CMGT9EEJ48CA type=write size=1051918 2024-06-28T15:08:26.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.740+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=OQ17MK2G9F7Z type=del size=1178219 2024-06-28T15:08:26.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.740+0000 7f65680598c0 1 < canceled part key=_multipart_LZX5UZCYLTAG.2~J86QBU2WER49.0 size=629024 2024-06-28T15:08:26.731 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.744+0000 7f65680598c0 1 < canceled multipart upload key=LZX5UZCYLTAG upload=J86QBU2WER49 tag=SUD7B4PG0Y33 type=write size=1677600 2024-06-28T15:08:26.737 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.748+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~1RHY3UOLPBCX.0 size=93473 2024-06-28T15:08:26.737 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.748+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=1RHY3UOLPBCX tag=9MK3F97RW5MF type=write size=1142049 2024-06-28T15:08:26.740 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.752+0000 7f65680598c0 1 < completed delete operation key=BHW2VHUT0U1U tag=IOQY7W0E0PDA type=del 2024-06-28T15:08:26.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.760+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~IAPXNUMNZRPY.0 size=516757 2024-06-28T15:08:26.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.760+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=IAPXNUMNZRPY tag=Q9LBCYSC07SY type=write size=1565333 2024-06-28T15:08:26.749 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.764+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~MOHJP0E302TZ.0 size=217432 2024-06-28T15:08:26.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.768+0000 7f65680598c0 1 < completed multipart upload key=V2L2Q1U62YW2 upload=MOHJP0E302TZ tag=Y5LGGS9JDMR3 type=write size=1266008 2024-06-28T15:08:26.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.772+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~BK5FA4RMNK5W.0 size=1048576 2024-06-28T15:08:26.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.776+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~BK5FA4RMNK5W.1 size=678506 2024-06-28T15:08:26.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.776+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=BK5FA4RMNK5W tag=7QUBUWNGWE04 type=write size=2775658 2024-06-28T15:08:26.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.776+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=DQ2OWZ79QOWU type=del 2024-06-28T15:08:26.768 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.780+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=A4JQSUZ6EJWX type=write size=1004115 2024-06-28T15:08:26.770 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.784+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~Z297QKBRAQQR.0 size=1048576 2024-06-28T15:08:26.773 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.784+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~Z297QKBRAQQR.1 size=1048576 2024-06-28T15:08:26.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.788+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~Z297QKBRAQQR.2 size=1034151 2024-06-28T15:08:26.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.792+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=Z297QKBRAQQR tag=X021E63S3XIX type=write size=4179879 2024-06-28T15:08:26.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.796+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=ARMAQJP1Q4QL type=del size=1594770 2024-06-28T15:08:26.786 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.800+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=442LIV32UO8Y type=del 2024-06-28T15:08:26.789 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.800+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=AM7HK3094VXC type=del size=4008668 2024-06-28T15:08:26.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.804+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=T0Q76QJB0C7J type=del 2024-06-28T15:08:26.797 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.812+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~57BAYA2E0VFG.0 size=925934 2024-06-28T15:08:26.797 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.812+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=57BAYA2E0VFG tag=7LCVCQQHOZ97 type=write size=1974510 2024-06-28T15:08:26.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.812+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=Q1G449QB0YJQ type=del 2024-06-28T15:08:26.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.816+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=881DJS687SH2 type=del size=3060305 2024-06-28T15:08:26.807 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.820+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=E61D9Q116C0P type=del 2024-06-28T15:08:26.812 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.824+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~ASVRAJHMYP14.0 size=1048576 2024-06-28T15:08:26.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.828+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~ASVRAJHMYP14.1 size=951383 2024-06-28T15:08:26.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.828+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=ASVRAJHMYP14 tag=6AJHJ7XX19G9 type=write size=3048535 2024-06-28T15:08:26.817 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.832+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~Z5IL38S7DIHT.0 size=1048576 2024-06-28T15:08:26.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.832+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~Z5IL38S7DIHT.1 size=423057 2024-06-28T15:08:26.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.836+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=Z5IL38S7DIHT tag=XZZ6REKA3BWX type=write size=2520209 2024-06-28T15:08:26.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.840+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=C1TQX13T3TIL type=write size=251575 2024-06-28T15:08:26.831 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.844+0000 7f65680598c0 1 < completed part key=_multipart_9MQKWSA33R5G.2~M1D7EPJ9I5IN.0 size=148777 2024-06-28T15:08:26.836 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.848+0000 7f65680598c0 1 < completed multipart upload key=9MQKWSA33R5G upload=M1D7EPJ9I5IN tag=HT4L1TL4EM1S type=write size=1197353 2024-06-28T15:08:26.842 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.856+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~BT5GC83RR4XE.0 size=1048576 2024-06-28T15:08:26.844 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.856+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~BT5GC83RR4XE.1 size=1048576 2024-06-28T15:08:26.846 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.860+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~BT5GC83RR4XE.2 size=932335 2024-06-28T15:08:26.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.860+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=BT5GC83RR4XE tag=2U8QSEZCVZGM type=write size=4078063 2024-06-28T15:08:26.850 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.864+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~KDUTILOY6CO9.0 size=1048576 2024-06-28T15:08:26.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.864+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~KDUTILOY6CO9.1 size=351061 2024-06-28T15:08:26.859 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.872+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=KDUTILOY6CO9 tag=E7GOY8IH0TC1 type=write size=2448213 2024-06-28T15:08:26.863 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.876+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=L8RK40BUJEO1 type=del size=1053423 2024-06-28T15:08:26.866 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.880+0000 7f65680598c0 1 < completed delete operation key=ZFNIA3O84EB7 tag=A8ZAJXGA41Z4 type=del 2024-06-28T15:08:26.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.884+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~2T7WP118HCKF.0 size=540245 2024-06-28T15:08:26.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.884+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=2T7WP118HCKF tag=5F0QJ1FC57B2 type=write size=1588821 2024-06-28T15:08:26.883 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.896+0000 7f65680598c0 1 < suggested update operation key=D9D3OFIJP3D3 tag=54HJCQ1YIPM0 type=del size=730639 2024-06-28T15:08:26.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.904+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~Z8SH7C431R79.0 size=1048576 2024-06-28T15:08:26.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.904+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~Z8SH7C431R79.1 size=1048576 2024-06-28T15:08:26.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.908+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~Z8SH7C431R79.2 size=821375 2024-06-28T15:08:26.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.908+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=Z8SH7C431R79 tag=GZVJFIDWW04J type=write size=3967103 2024-06-28T15:08:26.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.912+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=3QLBEXLZMSOJ type=del 2024-06-28T15:08:26.906 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.920+0000 7f65680598c0 1 > prepared part key=_multipart_7LB8MOJL39RM.2~K981W18YNYPO.0 size=433488 2024-06-28T15:08:26.906 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.920+0000 7f65680598c0 1 > prepared multipart upload key=7LB8MOJL39RM upload=K981W18YNYPO tag=0L6HAWB45D0H type=write size=1482064 2024-06-28T15:08:26.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.924+0000 7f65680598c0 1 < suggested update operation key=EN76QQWIQAXZ tag=OQ17MK2G9F7Z type=del size=1178219 2024-06-28T15:08:26.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.928+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=V005IQBE0FL1 type=del size=1880023 2024-06-28T15:08:26.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.932+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~1RHY3UOLPBCX.0 size=93473 2024-06-28T15:08:26.925 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.936+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=1RHY3UOLPBCX tag=9MK3F97RW5MF type=write size=1142049 2024-06-28T15:08:26.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.940+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=AOY4WC9HFSP8 type=del size=640373 2024-06-28T15:08:26.929 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.944+0000 7f65680598c0 1 < canceled part key=_multipart_47TO7G9EGJKF.2~IAPXNUMNZRPY.0 size=516757 2024-06-28T15:08:26.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.948+0000 7f65680598c0 1 < canceled multipart upload key=47TO7G9EGJKF upload=IAPXNUMNZRPY tag=Q9LBCYSC07SY type=write size=1565333 2024-06-28T15:08:26.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.952+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=W7U4ZS5S9TW2 type=write size=387818 2024-06-28T15:08:26.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.952+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~BK5FA4RMNK5W.0 size=1048576 2024-06-28T15:08:26.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.952+0000 7f65680598c0 1 < canceled part key=_multipart_RLJCNXKA2GYA.2~BK5FA4RMNK5W.1 size=678506 2024-06-28T15:08:26.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.960+0000 7f65680598c0 1 < canceled multipart upload key=RLJCNXKA2GYA upload=BK5FA4RMNK5W tag=7QUBUWNGWE04 type=write size=2775658 2024-06-28T15:08:26.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.964+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=1PF2XAMRQXRF type=del size=3268513 2024-06-28T15:08:26.954 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.968+0000 7f65680598c0 1 < completed write operation key=KFHGA64L1NLP tag=A4JQSUZ6EJWX type=write size=1004115 2024-06-28T15:08:26.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.972+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~RNCY2C3TXSR6.0 size=1048576 2024-06-28T15:08:26.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.976+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~RNCY2C3TXSR6.1 size=1048576 2024-06-28T15:08:26.963 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.976+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~RNCY2C3TXSR6.2 size=701371 2024-06-28T15:08:26.963 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.976+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=RNCY2C3TXSR6 tag=93ZAU47I6H4T type=write size=3847099 2024-06-28T15:08:26.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.980+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=ARMAQJP1Q4QL type=del 2024-06-28T15:08:26.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.984+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~1N3LFBOLF0P3.0 size=628898 2024-06-28T15:08:26.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.984+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=1N3LFBOLF0P3 tag=O5XVF14XPO22 type=write size=1677474 2024-06-28T15:08:26.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.988+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=AM7HK3094VXC type=del 2024-06-28T15:08:26.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.988+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=0AHRTI6J2F2Z type=del size=1551443 2024-06-28T15:08:26.980 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.992+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~57BAYA2E0VFG.0 size=925934 2024-06-28T15:08:26.984 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:26.996+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=57BAYA2E0VFG tag=7LCVCQQHOZ97 type=write size=1974510 2024-06-28T15:08:26.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.000+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~FSDK0YL95M1Y.0 size=1048576 2024-06-28T15:08:26.991 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.004+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~FSDK0YL95M1Y.1 size=374945 2024-06-28T15:08:26.991 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.004+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=FSDK0YL95M1Y tag=1ZOHU8C0D763 type=write size=2472097 2024-06-28T15:08:26.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.008+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=881DJS687SH2 type=del 2024-06-28T15:08:27.000 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.012+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~M20WQIE8K62K.0 size=1048576 2024-06-28T15:08:27.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.016+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~M20WQIE8K62K.1 size=752133 2024-06-28T15:08:27.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.016+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=M20WQIE8K62K tag=ZTV09VFFB9TE type=write size=2849285 2024-06-28T15:08:27.004 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.016+0000 7f65680598c0 1 < canceled part key=_multipart_47TO7G9EGJKF.2~ASVRAJHMYP14.0 size=1048576 2024-06-28T15:08:27.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.020+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~ASVRAJHMYP14.1 size=951383 2024-06-28T15:08:27.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.024+0000 7f65680598c0 1 < canceled multipart upload key=47TO7G9EGJKF upload=ASVRAJHMYP14 tag=6AJHJ7XX19G9 type=write size=3048535 2024-06-28T15:08:27.017 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.028+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~U82IR35R5AWV.0 size=1048576 2024-06-28T15:08:27.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.032+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~U82IR35R5AWV.1 size=996658 2024-06-28T15:08:27.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.032+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=U82IR35R5AWV tag=3DOH2C8RBX23 type=write size=3093810 2024-06-28T15:08:27.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.036+0000 7f65680598c0 1 < completed write operation key=KFHGA64L1NLP tag=C1TQX13T3TIL type=write size=251575 2024-06-28T15:08:27.026 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.040+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=SMWU5RZPNBCP type=del size=1336437 2024-06-28T15:08:27.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.040+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~BT5GC83RR4XE.0 size=1048576 2024-06-28T15:08:27.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.044+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~BT5GC83RR4XE.1 size=1048576 2024-06-28T15:08:27.034 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.048+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~BT5GC83RR4XE.2 size=932335 2024-06-28T15:08:27.039 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.052+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=BT5GC83RR4XE tag=2U8QSEZCVZGM type=write size=4078063 2024-06-28T15:08:27.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.056+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=8VV8SA9ZQ23A type=del size=899650 2024-06-28T15:08:27.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.056+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=L8RK40BUJEO1 type=del 2024-06-28T15:08:27.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.060+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=5BLYY3Q8KT8Y type=del size=3083891 2024-06-28T15:08:27.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.064+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~2T7WP118HCKF.0 size=540245 2024-06-28T15:08:27.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.068+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=2T7WP118HCKF tag=5F0QJ1FC57B2 type=write size=1588821 2024-06-28T15:08:27.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.072+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=TVFI7UDUGYKP type=del size=1277672 2024-06-28T15:08:27.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.076+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~Z8SH7C431R79.0 size=1048576 2024-06-28T15:08:27.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.080+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~Z8SH7C431R79.1 size=1048576 2024-06-28T15:08:27.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.080+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~Z8SH7C431R79.2 size=821375 2024-06-28T15:08:27.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.084+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=Z8SH7C431R79 tag=GZVJFIDWW04J type=write size=3967103 2024-06-28T15:08:27.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.088+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=31T69YC1OS9V type=del size=3232469 2024-06-28T15:08:27.079 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.092+0000 7f65680598c0 1 < completed part key=_multipart_7LB8MOJL39RM.2~K981W18YNYPO.0 size=433488 2024-06-28T15:08:27.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.096+0000 7f65680598c0 1 < completed multipart upload key=7LB8MOJL39RM upload=K981W18YNYPO tag=0L6HAWB45D0H type=write size=1482064 2024-06-28T15:08:27.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.104+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~BTQSKZCZCSZW.0 size=281955 2024-06-28T15:08:27.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.104+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=BTQSKZCZCSZW tag=SJ61UIA31WBO type=write size=1330531 2024-06-28T15:08:27.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.112+0000 7f65680598c0 1 < suggested update operation key=ZFNIA3O84EB7 tag=V005IQBE0FL1 type=del size=1880023 2024-06-28T15:08:27.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.116+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~S7M1PKWT8FO8.0 size=372786 2024-06-28T15:08:27.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.116+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=S7M1PKWT8FO8 tag=PGCHCWLYLUI5 type=write size=1421362 2024-06-28T15:08:27.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.120+0000 7f65680598c0 1 < completed delete operation key=V2L2Q1U62YW2 tag=AOY4WC9HFSP8 type=del 2024-06-28T15:08:27.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.120+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=L6NJLW2GGP26 type=del size=625117 2024-06-28T15:08:27.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.124+0000 7f65680598c0 1 < canceled operation key=V2L2Q1U62YW2 tag=W7U4ZS5S9TW2 type=write size=387818 2024-06-28T15:08:27.114 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.128+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=69P0DEJ2ATXJ type=del size=2034590 2024-06-28T15:08:27.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.132+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=1PF2XAMRQXRF type=del 2024-06-28T15:08:27.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.136+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~AEY9TBJS4JYN.0 size=364381 2024-06-28T15:08:27.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.136+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=AEY9TBJS4JYN tag=RKT9JIHDD1X8 type=write size=1412957 2024-06-28T15:08:27.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.140+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~RNCY2C3TXSR6.0 size=1048576 2024-06-28T15:08:27.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.140+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~RNCY2C3TXSR6.1 size=1048576 2024-06-28T15:08:27.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.144+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~RNCY2C3TXSR6.2 size=701371 2024-06-28T15:08:27.135 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.148+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=RNCY2C3TXSR6 tag=93ZAU47I6H4T type=write size=3847099 2024-06-28T15:08:27.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.152+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=UI8BODGAYJ61 type=del size=443828 2024-06-28T15:08:27.141 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.152+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~1N3LFBOLF0P3.0 size=628898 2024-06-28T15:08:27.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.160+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=1N3LFBOLF0P3 tag=O5XVF14XPO22 type=write size=1677474 2024-06-28T15:08:27.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.164+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=DFLLA55AXERE type=del size=1371611 2024-06-28T15:08:27.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.164+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=0AHRTI6J2F2Z type=del 2024-06-28T15:08:27.156 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.168+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=T4XWZ52W5LX2 type=write size=174849 2024-06-28T15:08:27.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.172+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~FSDK0YL95M1Y.0 size=1048576 2024-06-28T15:08:27.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.176+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~FSDK0YL95M1Y.1 size=374945 2024-06-28T15:08:27.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.180+0000 7f65680598c0 1 < completed multipart upload key=OUID7RX5NVV6 upload=FSDK0YL95M1Y tag=1ZOHU8C0D763 type=write size=2472097 2024-06-28T15:08:27.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.184+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=0MBKCOSEU68W type=del size=2283168 2024-06-28T15:08:27.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.188+0000 7f65680598c0 1 < canceled part key=_multipart_OUID7RX5NVV6.2~M20WQIE8K62K.0 size=1048576 2024-06-28T15:08:27.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.188+0000 7f65680598c0 1 < canceled part key=_multipart_OUID7RX5NVV6.2~M20WQIE8K62K.1 size=752133 2024-06-28T15:08:27.178 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.192+0000 7f65680598c0 1 < canceled multipart upload key=OUID7RX5NVV6 upload=M20WQIE8K62K tag=ZTV09VFFB9TE type=write size=2849285 2024-06-28T15:08:27.183 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.196+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~8JJYTUO0ZO2P.0 size=1048576 2024-06-28T15:08:27.185 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.200+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~8JJYTUO0ZO2P.1 size=1048576 2024-06-28T15:08:27.187 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.200+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~8JJYTUO0ZO2P.2 size=960052 2024-06-28T15:08:27.188 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.200+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=8JJYTUO0ZO2P tag=ZSQCK9F0OGCC type=write size=4105780 2024-06-28T15:08:27.191 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.204+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~U82IR35R5AWV.0 size=1048576 2024-06-28T15:08:27.193 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.208+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~U82IR35R5AWV.1 size=996658 2024-06-28T15:08:27.198 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.212+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=U82IR35R5AWV tag=3DOH2C8RBX23 type=write size=3093810 2024-06-28T15:08:27.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.216+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=UZP5W8WZ6LMZ type=del size=676646 2024-06-28T15:08:27.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.216+0000 7f65680598c0 1 < canceled operation key=GAZQ3PCPKN9Q tag=SMWU5RZPNBCP type=del size=1336437 2024-06-28T15:08:27.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.220+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=RCSTECIAV6HL type=del size=457015 2024-06-28T15:08:27.211 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.224+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=8VV8SA9ZQ23A type=del 2024-06-28T15:08:27.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.228+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~PYN2CM3RSI95.0 size=1048576 2024-06-28T15:08:27.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.232+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~PYN2CM3RSI95.1 size=1048576 2024-06-28T15:08:27.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.236+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~PYN2CM3RSI95.2 size=107402 2024-06-28T15:08:27.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.236+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=PYN2CM3RSI95 tag=IIUA41ACNGDK type=write size=3253130 2024-06-28T15:08:27.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.240+0000 7f65680598c0 1 < suggested update operation key=KFHGA64L1NLP tag=5BLYY3Q8KT8Y type=del size=3083891 2024-06-28T15:08:27.235 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.248+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~Z4AFW7ZGGBU9.0 size=1048576 2024-06-28T15:08:27.237 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.252+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~Z4AFW7ZGGBU9.1 size=1048576 2024-06-28T15:08:27.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.252+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~Z4AFW7ZGGBU9.2 size=908037 2024-06-28T15:08:27.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.252+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=Z4AFW7ZGGBU9 tag=0F9K1IEX3NP3 type=write size=4053765 2024-06-28T15:08:27.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.260+0000 7f65680598c0 1 < suggested update operation key=ZY973GBGBQ1Z tag=TVFI7UDUGYKP type=del size=1277672 2024-06-28T15:08:27.250 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.264+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=T1KDBCSHM2T7 type=del size=245984 2024-06-28T15:08:27.253 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.264+0000 7f65680598c0 1 < completed delete operation key=7LB8MOJL39RM tag=31T69YC1OS9V type=del 2024-06-28T15:08:27.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.268+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=DMX63KFZ88B6 type=del size=1556943 2024-06-28T15:08:27.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.272+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~BTQSKZCZCSZW.0 size=281955 2024-06-28T15:08:27.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.276+0000 7f65680598c0 1 < completed multipart upload key=P278ZM1H052S upload=BTQSKZCZCSZW tag=SJ61UIA31WBO type=write size=1330531 2024-06-28T15:08:27.268 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.280+0000 7f65680598c0 1 > prepared operation key=4CU86XS1R9G3 tag=UZOR1XXATPQV type=del size=2819470 2024-06-28T15:08:27.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.284+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~S7M1PKWT8FO8.0 size=372786 2024-06-28T15:08:27.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.292+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=S7M1PKWT8FO8 tag=PGCHCWLYLUI5 type=write size=1421362 2024-06-28T15:08:27.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.292+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=IE6WMZX3GPSQ type=del size=636557 2024-06-28T15:08:27.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.296+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=L6NJLW2GGP26 type=del 2024-06-28T15:08:27.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.304+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~PU4Y65VHEH5O.0 size=231589 2024-06-28T15:08:27.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.304+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=PU4Y65VHEH5O tag=4OQ9615JV8WV type=write size=1280165 2024-06-28T15:08:27.294 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.308+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=69P0DEJ2ATXJ type=del 2024-06-28T15:08:27.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.312+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=B2RPBEQXAW01 type=del size=2488095 2024-06-28T15:08:27.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.312+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~AEY9TBJS4JYN.0 size=364381 2024-06-28T15:08:27.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.320+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=AEY9TBJS4JYN tag=RKT9JIHDD1X8 type=write size=1412957 2024-06-28T15:08:27.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.324+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=8CTJ0CIXG19N type=write size=1023866 2024-06-28T15:08:27.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.328+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=UI8BODGAYJ61 type=del 2024-06-28T15:08:27.317 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.328+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=WKQ7WLVWWDT4 type=del size=1063861 2024-06-28T15:08:27.320 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.332+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=DFLLA55AXERE type=del 2024-06-28T15:08:27.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.336+0000 7f65680598c0 1 > prepared operation key=BHW2VHUT0U1U tag=LFWEBBX7QFNZ type=write size=416236 2024-06-28T15:08:27.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.340+0000 7f65680598c0 1 < completed write operation key=AOBFAR0UTONH tag=T4XWZ52W5LX2 type=write size=174849 2024-06-28T15:08:27.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.348+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~NPX1HIG82DVV.0 size=1048576 2024-06-28T15:08:27.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.348+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~NPX1HIG82DVV.1 size=850863 2024-06-28T15:08:27.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.348+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=NPX1HIG82DVV tag=AHWIUCEFUV06 type=write size=2948015 2024-06-28T15:08:27.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.352+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=0MBKCOSEU68W type=del 2024-06-28T15:08:27.346 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.360+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~J42GWGEPW43N.0 size=296667 2024-06-28T15:08:27.346 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.360+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=J42GWGEPW43N tag=4IY0RXL4MD1F type=write size=1345243 2024-06-28T15:08:27.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.364+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~8JJYTUO0ZO2P.0 size=1048576 2024-06-28T15:08:27.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.364+0000 7f65680598c0 1 < canceled part key=_multipart_C5C6SAHLPZV1.2~8JJYTUO0ZO2P.1 size=1048576 2024-06-28T15:08:27.352 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.364+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~8JJYTUO0ZO2P.2 size=960052 2024-06-28T15:08:27.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.372+0000 7f65680598c0 1 < canceled multipart upload key=C5C6SAHLPZV1 upload=8JJYTUO0ZO2P tag=ZSQCK9F0OGCC type=write size=4105780 2024-06-28T15:08:27.361 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.376+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=ZFHGTBLE8CBT type=del size=1587952 2024-06-28T15:08:27.364 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.376+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=UZP5W8WZ6LMZ type=del 2024-06-28T15:08:27.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.384+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~A7TTE403SCW3.0 size=23052 2024-06-28T15:08:27.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.384+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=A7TTE403SCW3 tag=LC4H3HLY3U8U type=write size=1071628 2024-06-28T15:08:27.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.388+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=RCSTECIAV6HL type=del 2024-06-28T15:08:27.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.388+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=Y2QHNNCCXRR5 type=del size=1039529 2024-06-28T15:08:27.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.392+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~PYN2CM3RSI95.0 size=1048576 2024-06-28T15:08:27.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.392+0000 7f65680598c0 1 < canceled part key=_multipart_SFCIZMLY2Y2E.2~PYN2CM3RSI95.1 size=1048576 2024-06-28T15:08:27.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.396+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~PYN2CM3RSI95.2 size=107402 2024-06-28T15:08:27.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.400+0000 7f65680598c0 1 < canceled multipart upload key=SFCIZMLY2Y2E upload=PYN2CM3RSI95 tag=IIUA41ACNGDK type=write size=3253130 2024-06-28T15:08:27.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.408+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~N203ZX7R1OAL.0 size=1048576 2024-06-28T15:08:27.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.412+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~N203ZX7R1OAL.1 size=1048576 2024-06-28T15:08:27.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.412+0000 7f65680598c0 1 > prepared part key=_multipart_47TO7G9EGJKF.2~N203ZX7R1OAL.2 size=641636 2024-06-28T15:08:27.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.412+0000 7f65680598c0 1 > prepared multipart upload key=47TO7G9EGJKF upload=N203ZX7R1OAL tag=BGJ74P8PWLH7 type=write size=3787364 2024-06-28T15:08:27.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.416+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~Z4AFW7ZGGBU9.0 size=1048576 2024-06-28T15:08:27.408 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.420+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~Z4AFW7ZGGBU9.1 size=1048576 2024-06-28T15:08:27.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.424+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~Z4AFW7ZGGBU9.2 size=908037 2024-06-28T15:08:27.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.428+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=Z4AFW7ZGGBU9 tag=0F9K1IEX3NP3 type=write size=4053765 2024-06-28T15:08:27.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.432+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=7CT5BF61QYIO type=del size=3034085 2024-06-28T15:08:27.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.440+0000 7f65680598c0 1 < suggested update operation key=ZY973GBGBQ1Z tag=T1KDBCSHM2T7 type=del size=245984 2024-06-28T15:08:27.434 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.448+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~2VRTJNBCGY1B.0 size=1048576 2024-06-28T15:08:27.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.448+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~2VRTJNBCGY1B.1 size=1048576 2024-06-28T15:08:27.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.452+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~2VRTJNBCGY1B.2 size=365021 2024-06-28T15:08:27.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.452+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=2VRTJNBCGY1B tag=KB27241A8U5K type=write size=3510749 2024-06-28T15:08:27.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.456+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=DMX63KFZ88B6 type=del 2024-06-28T15:08:27.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.460+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=3GMMTLBKENOI type=del size=998748 2024-06-28T15:08:27.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.468+0000 7f65680598c0 1 < suggested remove operation key=4CU86XS1R9G3 tag=UZOR1XXATPQV type=del 2024-06-28T15:08:27.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.472+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~UOYJO5YOAKYF.0 size=551483 2024-06-28T15:08:27.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.472+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=UOYJO5YOAKYF tag=30ISCBLBYLCA type=write size=1600059 2024-06-28T15:08:27.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.476+0000 7f65680598c0 1 < canceled operation key=AOBFAR0UTONH tag=IE6WMZX3GPSQ type=del size=636557 2024-06-28T15:08:27.467 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.480+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=6WPJ2PX5JWV0 type=del size=1564062 2024-06-28T15:08:27.470 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.484+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~PU4Y65VHEH5O.0 size=231589 2024-06-28T15:08:27.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.488+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=PU4Y65VHEH5O tag=4OQ9615JV8WV type=write size=1280165 2024-06-28T15:08:27.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.492+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=ZW0P2OQVUS4S type=del size=3134221 2024-06-28T15:08:27.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.500+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=B2RPBEQXAW01 type=del 2024-06-28T15:08:27.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.504+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=SHF1O3Q99450 type=del size=3193551 2024-06-28T15:08:27.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.508+0000 7f65680598c0 1 < completed write operation key=9MQKWSA33R5G tag=8CTJ0CIXG19N type=write size=1023866 2024-06-28T15:08:27.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.512+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=CMEN7BJL2CRV type=del size=3317605 2024-06-28T15:08:27.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.516+0000 7f65680598c0 1 < canceled operation key=F6SEDD6BH1T2 tag=WKQ7WLVWWDT4 type=del size=1063861 2024-06-28T15:08:27.505 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.516+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=XWCNKJT9AFCJ type=del size=1120032 2024-06-28T15:08:27.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.520+0000 7f65680598c0 1 < completed write operation key=BHW2VHUT0U1U tag=LFWEBBX7QFNZ type=write size=416236 2024-06-28T15:08:27.511 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.524+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=95JYJWDVUQ29 type=del size=2163391 2024-06-28T15:08:27.515 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.528+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~NPX1HIG82DVV.0 size=1048576 2024-06-28T15:08:27.515 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.528+0000 7f65680598c0 1 < canceled part key=_multipart_D9D3OFIJP3D3.2~NPX1HIG82DVV.1 size=850863 2024-06-28T15:08:27.520 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.532+0000 7f65680598c0 1 < canceled multipart upload key=D9D3OFIJP3D3 upload=NPX1HIG82DVV tag=AHWIUCEFUV06 type=write size=2948015 2024-06-28T15:08:27.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.536+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=6ULPNKMI6STJ type=del size=529738 2024-06-28T15:08:27.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.540+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~J42GWGEPW43N.0 size=296667 2024-06-28T15:08:27.532 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.544+0000 7f65680598c0 1 < completed multipart upload key=UP3XFF38IJXN upload=J42GWGEPW43N tag=4IY0RXL4MD1F type=write size=1345243 2024-06-28T15:08:27.537 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.552+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~GGR58VCPNG38.0 size=1048576 2024-06-28T15:08:27.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.552+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~GGR58VCPNG38.1 size=39856 2024-06-28T15:08:27.540 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.552+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=GGR58VCPNG38 tag=Y3QY0EYRQVZL type=write size=2137008 2024-06-28T15:08:27.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.556+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=ZFHGTBLE8CBT type=del 2024-06-28T15:08:27.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.560+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~N0Z1O4GOLO9I.0 size=822459 2024-06-28T15:08:27.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.560+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=N0Z1O4GOLO9I tag=AHGQOL455NQK type=write size=1871035 2024-06-28T15:08:27.552 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.564+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~A7TTE403SCW3.0 size=23052 2024-06-28T15:08:27.556 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.568+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=A7TTE403SCW3 tag=LC4H3HLY3U8U type=write size=1071628 2024-06-28T15:08:27.562 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.576+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~RRIPQXVHWOOV.0 size=1048576 2024-06-28T15:08:27.564 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.576+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~RRIPQXVHWOOV.1 size=1048576 2024-06-28T15:08:27.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.580+0000 7f65680598c0 1 > prepared part key=_multipart_AXQ0YFWUH4FD.2~RRIPQXVHWOOV.2 size=898780 2024-06-28T15:08:27.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.580+0000 7f65680598c0 1 > prepared multipart upload key=AXQ0YFWUH4FD upload=RRIPQXVHWOOV tag=4VC3L2L7YKGX type=write size=4044508 2024-06-28T15:08:27.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.584+0000 7f65680598c0 1 < canceled operation key=J311Q2712YM0 tag=Y2QHNNCCXRR5 type=del size=1039529 2024-06-28T15:08:27.575 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.588+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~P60TRH6GAD4G.0 size=1048576 2024-06-28T15:08:27.577 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.592+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~P60TRH6GAD4G.1 size=1048576 2024-06-28T15:08:27.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.592+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~P60TRH6GAD4G.2 size=687773 2024-06-28T15:08:27.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.592+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=P60TRH6GAD4G tag=5ACGI20F3B5K type=write size=3833501 2024-06-28T15:08:27.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.596+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~N203ZX7R1OAL.0 size=1048576 2024-06-28T15:08:27.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.596+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~N203ZX7R1OAL.1 size=1048576 2024-06-28T15:08:27.587 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.600+0000 7f65680598c0 1 < completed part key=_multipart_47TO7G9EGJKF.2~N203ZX7R1OAL.2 size=641636 2024-06-28T15:08:27.593 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.604+0000 7f65680598c0 1 < completed multipart upload key=47TO7G9EGJKF upload=N203ZX7R1OAL tag=BGJ74P8PWLH7 type=write size=3787364 2024-06-28T15:08:27.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.612+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~CHOVUL6VPYCR.0 size=1048576 2024-06-28T15:08:27.601 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.612+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~CHOVUL6VPYCR.1 size=553701 2024-06-28T15:08:27.601 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.612+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=CHOVUL6VPYCR tag=Q5166AWCI0W2 type=write size=2650853 2024-06-28T15:08:27.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.616+0000 7f65680598c0 1 < completed delete operation key=7NL3CTFYY9E5 tag=7CT5BF61QYIO type=del 2024-06-28T15:08:27.611 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.624+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~N91NZP73I77F.0 size=1048576 2024-06-28T15:08:27.613 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.624+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~N91NZP73I77F.1 size=645164 2024-06-28T15:08:27.613 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.628+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=N91NZP73I77F tag=9SA1YMTORQAV type=write size=2742316 2024-06-28T15:08:27.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.628+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~2VRTJNBCGY1B.0 size=1048576 2024-06-28T15:08:27.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.632+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~2VRTJNBCGY1B.1 size=1048576 2024-06-28T15:08:27.621 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.636+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~2VRTJNBCGY1B.2 size=365021 2024-06-28T15:08:27.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.640+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=2VRTJNBCGY1B tag=KB27241A8U5K type=write size=3510749 2024-06-28T15:08:27.629 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.644+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=JECF6BQNT43Q type=del size=2515341 2024-06-28T15:08:27.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.644+0000 7f65680598c0 1 < completed delete operation key=47TO7G9EGJKF tag=3GMMTLBKENOI type=del 2024-06-28T15:08:27.639 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.652+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~JQ03JUY5BHN1.0 size=1048576 2024-06-28T15:08:27.641 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.656+0000 7f65680598c0 1 > prepared part key=_multipart_2F968YIMJN9D.2~JQ03JUY5BHN1.1 size=964853 2024-06-28T15:08:27.641 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.656+0000 7f65680598c0 1 > prepared multipart upload key=2F968YIMJN9D upload=JQ03JUY5BHN1 tag=8ZRNI91GPM2M type=write size=3062005 2024-06-28T15:08:27.645 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.656+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~UOYJO5YOAKYF.0 size=551483 2024-06-28T15:08:27.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.664+0000 7f65680598c0 1 < completed multipart upload key=F6SEDD6BH1T2 upload=UOYJO5YOAKYF tag=30ISCBLBYLCA type=write size=1600059 2024-06-28T15:08:27.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.668+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~YYE8TY56TC6R.0 size=804795 2024-06-28T15:08:27.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.668+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=YYE8TY56TC6R tag=QK5K02Z47ZEC type=write size=1853371 2024-06-28T15:08:27.663 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.676+0000 7f65680598c0 1 < suggested update operation key=UP3XFF38IJXN tag=6WPJ2PX5JWV0 type=del size=1564062 2024-06-28T15:08:27.667 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.680+0000 7f65680598c0 1 > prepared operation key=D9D3OFIJP3D3 tag=DZAWX5OCOAJA type=del size=82020 2024-06-28T15:08:27.670 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.684+0000 7f65680598c0 1 < canceled operation key=EN76QQWIQAXZ tag=ZW0P2OQVUS4S type=del size=3134221 2024-06-28T15:08:27.673 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.688+0000 7f65680598c0 1 > prepared operation key=V2L2Q1U62YW2 tag=LHP3FLT2Q7OL type=write size=973878 2024-06-28T15:08:27.677 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.692+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=SHF1O3Q99450 type=del 2024-06-28T15:08:27.681 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.692+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=WOAYBXZJ3W4Q type=write size=313572 2024-06-28T15:08:27.684 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.696+0000 7f65680598c0 1 < canceled operation key=ZY973GBGBQ1Z tag=CMEN7BJL2CRV type=del size=3317605 2024-06-28T15:08:27.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.700+0000 7f65680598c0 1 > prepared operation key=P278ZM1H052S tag=9ARN0XQRUWHU type=write size=736854 2024-06-28T15:08:27.690 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.704+0000 7f65680598c0 1 < completed delete operation key=KFHGA64L1NLP tag=XWCNKJT9AFCJ type=del 2024-06-28T15:08:27.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.708+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=8I0H7UCJO4E1 type=del size=2466792 2024-06-28T15:08:27.696 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.708+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=95JYJWDVUQ29 type=del 2024-06-28T15:08:27.702 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.716+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~Q02VGSEETULL.0 size=1048576 2024-06-28T15:08:27.704 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.716+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~Q02VGSEETULL.1 size=735983 2024-06-28T15:08:27.704 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.716+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=Q02VGSEETULL tag=096RJUY3DEZC type=write size=2833135 2024-06-28T15:08:27.708 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.720+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=6ULPNKMI6STJ type=del 2024-06-28T15:08:27.713 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.724+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~RRNDXAU7AOMS.0 size=275247 2024-06-28T15:08:27.713 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.724+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=RRNDXAU7AOMS tag=Q4EMDBJV9R8J type=write size=1323823 2024-06-28T15:08:27.716 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.728+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~GGR58VCPNG38.0 size=1048576 2024-06-28T15:08:27.718 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.732+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~GGR58VCPNG38.1 size=39856 2024-06-28T15:08:27.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.736+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=GGR58VCPNG38 tag=Y3QY0EYRQVZL type=write size=2137008 2024-06-28T15:08:27.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.740+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=QBZFQXNUPWV4 type=write size=992980 2024-06-28T15:08:27.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.744+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~N0Z1O4GOLO9I.0 size=822459 2024-06-28T15:08:27.735 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.748+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=N0Z1O4GOLO9I tag=AHGQOL455NQK type=write size=1871035 2024-06-28T15:08:27.738 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.752+0000 7f65680598c0 1 > prepared operation key=C5C6SAHLPZV1 tag=O9W9663A5Z31 type=del size=2957994 2024-06-28T15:08:27.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.752+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~RRIPQXVHWOOV.0 size=1048576 2024-06-28T15:08:27.744 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.756+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~RRIPQXVHWOOV.1 size=1048576 2024-06-28T15:08:27.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.760+0000 7f65680598c0 1 < completed part key=_multipart_AXQ0YFWUH4FD.2~RRIPQXVHWOOV.2 size=898780 2024-06-28T15:08:27.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.764+0000 7f65680598c0 1 < completed multipart upload key=AXQ0YFWUH4FD upload=RRIPQXVHWOOV tag=4VC3L2L7YKGX type=write size=4044508 2024-06-28T15:08:27.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.768+0000 7f65680598c0 1 > prepared operation key=47TO7G9EGJKF tag=JASORJIA37MQ type=write size=515882 2024-06-28T15:08:27.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.768+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~P60TRH6GAD4G.0 size=1048576 2024-06-28T15:08:27.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.772+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~P60TRH6GAD4G.1 size=1048576 2024-06-28T15:08:27.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.772+0000 7f65680598c0 1 < canceled part key=_multipart_R0IMFVF5NEOO.2~P60TRH6GAD4G.2 size=687773 2024-06-28T15:08:27.764 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.776+0000 7f65680598c0 1 < canceled multipart upload key=R0IMFVF5NEOO upload=P60TRH6GAD4G tag=5ACGI20F3B5K type=write size=3833501 2024-06-28T15:08:27.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.780+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=7POUSM1LW6Z0 type=write size=786656 2024-06-28T15:08:27.768 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.780+0000 7f65680598c0 1 < canceled part key=_multipart_F6SEDD6BH1T2.2~CHOVUL6VPYCR.0 size=1048576 2024-06-28T15:08:27.770 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.784+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~CHOVUL6VPYCR.1 size=553701 2024-06-28T15:08:27.779 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.792+0000 7f65680598c0 1 < canceled multipart upload key=F6SEDD6BH1T2 upload=CHOVUL6VPYCR tag=Q5166AWCI0W2 type=write size=2650853 2024-06-28T15:08:27.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.796+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=S57CRBO26P1T type=del size=3368065 2024-06-28T15:08:27.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.796+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~N91NZP73I77F.0 size=1048576 2024-06-28T15:08:27.787 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.800+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~N91NZP73I77F.1 size=645164 2024-06-28T15:08:27.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.804+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=N91NZP73I77F tag=9SA1YMTORQAV type=write size=2742316 2024-06-28T15:08:27.795 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.808+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=5IW1I0ZV26R1 type=write size=288870 2024-06-28T15:08:27.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.812+0000 7f65680598c0 1 < suggested remove operation key=I4RWCQSQVLCN tag=JECF6BQNT43Q type=del 2024-06-28T15:08:27.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.816+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=5IILF8D68R8N type=write size=240385 2024-06-28T15:08:27.807 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.820+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~JQ03JUY5BHN1.0 size=1048576 2024-06-28T15:08:27.809 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.820+0000 7f65680598c0 1 < completed part key=_multipart_2F968YIMJN9D.2~JQ03JUY5BHN1.1 size=964853 2024-06-28T15:08:27.814 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.828+0000 7f65680598c0 1 < completed multipart upload key=2F968YIMJN9D upload=JQ03JUY5BHN1 tag=8ZRNI91GPM2M type=write size=3062005 2024-06-28T15:08:27.816 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.828+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=ZPRDTFU7R5FA type=del size=1621554 2024-06-28T15:08:27.817 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.828+0000 7f65680598c0 1 < canceled part key=_multipart_R0IMFVF5NEOO.2~YYE8TY56TC6R.0 size=804795 2024-06-28T15:08:27.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.836+0000 7f65680598c0 1 < canceled multipart upload key=R0IMFVF5NEOO upload=YYE8TY56TC6R tag=QK5K02Z47ZEC type=write size=1853371 2024-06-28T15:08:27.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.836+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=QMZN0O4B0QIB type=del size=4135175 2024-06-28T15:08:27.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.840+0000 7f65680598c0 1 < completed delete operation key=D9D3OFIJP3D3 tag=DZAWX5OCOAJA type=del 2024-06-28T15:08:27.831 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.844+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=W3V57HTREW4O type=write size=538978 2024-06-28T15:08:27.833 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.848+0000 7f65680598c0 1 < completed write operation key=V2L2Q1U62YW2 tag=LHP3FLT2Q7OL type=write size=973878 2024-06-28T15:08:27.836 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.848+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=GY7G7Y95M1JK type=del size=3455050 2024-06-28T15:08:27.839 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.852+0000 7f65680598c0 1 < completed write operation key=SFCIZMLY2Y2E tag=WOAYBXZJ3W4Q type=write size=313572 2024-06-28T15:08:27.844 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.856+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~9FY74TCI30VG.0 size=1048576 2024-06-28T15:08:27.846 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.860+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~9FY74TCI30VG.1 size=1048576 2024-06-28T15:08:27.848 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.860+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~9FY74TCI30VG.2 size=936440 2024-06-28T15:08:27.849 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.860+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=9FY74TCI30VG tag=CO6JWUI6HGRD type=write size=4082168 2024-06-28T15:08:27.852 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.864+0000 7f65680598c0 1 < completed write operation key=P278ZM1H052S tag=9ARN0XQRUWHU type=write size=736854 2024-06-28T15:08:27.857 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.868+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~E2ZOPDIHV2PY.0 size=965121 2024-06-28T15:08:27.857 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.868+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=E2ZOPDIHV2PY tag=ESNLPIR7K8NK type=write size=2013697 2024-06-28T15:08:27.860 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.872+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=8I0H7UCJO4E1 type=del 2024-06-28T15:08:27.863 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.876+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=IAV3BZKHS52H type=del size=3318747 2024-06-28T15:08:27.866 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.880+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~Q02VGSEETULL.0 size=1048576 2024-06-28T15:08:27.868 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.880+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~Q02VGSEETULL.1 size=735983 2024-06-28T15:08:27.873 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.888+0000 7f65680598c0 1 < completed multipart upload key=GAZQ3PCPKN9Q upload=Q02VGSEETULL tag=096RJUY3DEZC type=write size=2833135 2024-06-28T15:08:27.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.892+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~ZRDBJPNA3Y7W.0 size=1048576 2024-06-28T15:08:27.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.892+0000 7f65680598c0 1 > prepared part key=_multipart_GAZQ3PCPKN9Q.2~ZRDBJPNA3Y7W.1 size=482832 2024-06-28T15:08:27.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.892+0000 7f65680598c0 1 > prepared multipart upload key=GAZQ3PCPKN9Q upload=ZRDBJPNA3Y7W tag=8I1KPXB5MTGB type=write size=2579984 2024-06-28T15:08:27.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.896+0000 7f65680598c0 1 < canceled part key=_multipart_F6SEDD6BH1T2.2~RRNDXAU7AOMS.0 size=275247 2024-06-28T15:08:27.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.900+0000 7f65680598c0 1 < canceled multipart upload key=F6SEDD6BH1T2 upload=RRNDXAU7AOMS tag=Q4EMDBJV9R8J type=write size=1323823 2024-06-28T15:08:27.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.904+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=Z23T4WFLX0ML type=del size=446865 2024-06-28T15:08:27.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.908+0000 7f65680598c0 1 < suggested update operation key=7NL3CTFYY9E5 tag=QBZFQXNUPWV4 type=write size=992980 2024-06-28T15:08:27.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.916+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~6PTYO81Z9NGI.0 size=1048576 2024-06-28T15:08:27.904 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.916+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~6PTYO81Z9NGI.1 size=1048576 2024-06-28T15:08:27.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.920+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~6PTYO81Z9NGI.2 size=928376 2024-06-28T15:08:27.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.920+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=6PTYO81Z9NGI tag=5N1YB0YLBIE7 type=write size=4074104 2024-06-28T15:08:27.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.924+0000 7f65680598c0 1 < completed delete operation key=C5C6SAHLPZV1 tag=O9W9663A5Z31 type=del 2024-06-28T15:08:27.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.928+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=CDU5IUUJGEB2 type=write size=223082 2024-06-28T15:08:27.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.932+0000 7f65680598c0 1 < suggested update operation key=47TO7G9EGJKF tag=JASORJIA37MQ type=write size=515882 2024-06-28T15:08:27.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.936+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=KB5B8EYXAVAG type=del size=1005212 2024-06-28T15:08:27.930 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.944+0000 7f65680598c0 1 < suggested update operation key=MQ3QWZBFN4YR tag=7POUSM1LW6Z0 type=write size=786656 2024-06-28T15:08:27.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.948+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=YGJ6KSRG4A2G type=del size=2086582 2024-06-28T15:08:27.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.952+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=S57CRBO26P1T type=del 2024-06-28T15:08:27.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.956+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~0KEDLLE3USH5.0 size=232907 2024-06-28T15:08:27.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.956+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=0KEDLLE3USH5 tag=JJ1EG7G8MO02 type=write size=1281483 2024-06-28T15:08:27.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.960+0000 7f65680598c0 1 < completed write operation key=EN76QQWIQAXZ tag=5IW1I0ZV26R1 type=write size=288870 2024-06-28T15:08:27.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.964+0000 7f65680598c0 1 > prepared operation key=LZX5UZCYLTAG tag=1DA6XNJBPJAR type=del size=3807526 2024-06-28T15:08:27.953 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.968+0000 7f65680598c0 1 < completed write operation key=UP3XFF38IJXN tag=5IILF8D68R8N type=write size=240385 2024-06-28T15:08:27.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.972+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~KSXM7W7Y31PX.0 size=794391 2024-06-28T15:08:27.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.972+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=KSXM7W7Y31PX tag=JDWO82GKC7PT type=write size=1842967 2024-06-28T15:08:27.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.976+0000 7f65680598c0 1 < completed delete operation key=SFCIZMLY2Y2E tag=ZPRDTFU7R5FA type=del 2024-06-28T15:08:27.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.976+0000 7f65680598c0 1 > prepared operation key=J311Q2712YM0 tag=PQOB9WNPF74M type=del size=287473 2024-06-28T15:08:27.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.980+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=QMZN0O4B0QIB type=del 2024-06-28T15:08:27.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.984+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=J9L17DNHIZ5L type=write size=403021 2024-06-28T15:08:27.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.988+0000 7f65680598c0 1 < completed write operation key=F6SEDD6BH1T2 tag=W3V57HTREW4O type=write size=538978 2024-06-28T15:08:27.980 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.992+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~FIVC8TXP11OJ.0 size=1048576 2024-06-28T15:08:27.982 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.996+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~FIVC8TXP11OJ.1 size=554847 2024-06-28T15:08:27.982 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.996+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=FIVC8TXP11OJ tag=BE1GIAVDEKG0 type=write size=2651999 2024-06-28T15:08:27.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:27.996+0000 7f65680598c0 1 < canceled operation key=60QVJVV16QFM tag=GY7G7Y95M1JK type=del size=3455050 2024-06-28T15:08:27.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.000+0000 7f65680598c0 1 > prepared operation key=R0IMFVF5NEOO tag=C5GFDV3OKHGT type=del size=1694271 2024-06-28T15:08:27.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.004+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~9FY74TCI30VG.0 size=1048576 2024-06-28T15:08:27.992 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.004+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~9FY74TCI30VG.1 size=1048576 2024-06-28T15:08:27.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.008+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~9FY74TCI30VG.2 size=936440 2024-06-28T15:08:28.000 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.012+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=9FY74TCI30VG tag=CO6JWUI6HGRD type=write size=4082168 2024-06-28T15:08:28.005 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.020+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~C9LVJMEPJ34U.0 size=1048576 2024-06-28T15:08:28.008 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.020+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~C9LVJMEPJ34U.1 size=420718 2024-06-28T15:08:28.008 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.020+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=C9LVJMEPJ34U tag=6M8VCF5YTV31 type=write size=2517870 2024-06-28T15:08:28.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.024+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~E2ZOPDIHV2PY.0 size=965121 2024-06-28T15:08:28.016 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.028+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=E2ZOPDIHV2PY tag=ESNLPIR7K8NK type=write size=2013697 2024-06-28T15:08:28.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.036+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~JMB3VUF8Q6AQ.0 size=1048576 2024-06-28T15:08:28.026 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.036+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~JMB3VUF8Q6AQ.1 size=1048576 2024-06-28T15:08:28.026 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.040+0000 7f65680598c0 1 > prepared part key=_multipart_UP3XFF38IJXN.2~JMB3VUF8Q6AQ.2 size=584988 2024-06-28T15:08:28.026 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.040+0000 7f65680598c0 1 > prepared multipart upload key=UP3XFF38IJXN upload=JMB3VUF8Q6AQ tag=ZV2Q0U4IKC1Q type=write size=3730716 2024-06-28T15:08:28.033 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.048+0000 7f65680598c0 1 < suggested update operation key=7NL3CTFYY9E5 tag=IAV3BZKHS52H type=del size=3318747 2024-06-28T15:08:28.036 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.048+0000 7f65680598c0 1 > prepared operation key=7LB8MOJL39RM tag=3CDL3S7GNASX type=write size=449740 2024-06-28T15:08:28.040 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.052+0000 7f65680598c0 1 < completed part key=_multipart_GAZQ3PCPKN9Q.2~ZRDBJPNA3Y7W.0 size=1048576 2024-06-28T15:08:28.040 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.052+0000 7f65680598c0 1 < canceled part key=_multipart_GAZQ3PCPKN9Q.2~ZRDBJPNA3Y7W.1 size=482832 2024-06-28T15:08:28.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.056+0000 7f65680598c0 1 < canceled multipart upload key=GAZQ3PCPKN9Q upload=ZRDBJPNA3Y7W tag=8I1KPXB5MTGB type=write size=2579984 2024-06-28T15:08:28.050 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.064+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~GQUPDMNFHZC4.0 size=88965 2024-06-28T15:08:28.050 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.064+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=GQUPDMNFHZC4 tag=TKZR72E21QM9 type=write size=1137541 2024-06-28T15:08:28.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.064+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=Z23T4WFLX0ML type=del 2024-06-28T15:08:28.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.072+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~1HQLLN8G3ZS0.0 size=1048576 2024-06-28T15:08:28.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.076+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~1HQLLN8G3ZS0.1 size=888408 2024-06-28T15:08:28.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.076+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=1HQLLN8G3ZS0 tag=Q8J63XCLKGL6 type=write size=2985560 2024-06-28T15:08:28.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.076+0000 7f65680598c0 1 < canceled part key=_multipart_I4RWCQSQVLCN.2~6PTYO81Z9NGI.0 size=1048576 2024-06-28T15:08:28.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.076+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~6PTYO81Z9NGI.1 size=1048576 2024-06-28T15:08:28.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.080+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~6PTYO81Z9NGI.2 size=928376 2024-06-28T15:08:28.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.084+0000 7f65680598c0 1 < canceled multipart upload key=I4RWCQSQVLCN upload=6PTYO81Z9NGI tag=5N1YB0YLBIE7 type=write size=4074104 2024-06-28T15:08:28.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.088+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~34OWY0HG5E18.0 size=1048576 2024-06-28T15:08:28.078 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.092+0000 7f65680598c0 1 > prepared part key=_multipart_C5C6SAHLPZV1.2~34OWY0HG5E18.1 size=606375 2024-06-28T15:08:28.078 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.092+0000 7f65680598c0 1 > prepared multipart upload key=C5C6SAHLPZV1 upload=34OWY0HG5E18 tag=2CUFYKDA1OJ0 type=write size=2703527 2024-06-28T15:08:28.083 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.096+0000 7f65680598c0 1 < suggested update operation key=UP3XFF38IJXN tag=CDU5IUUJGEB2 type=write size=223082 2024-06-28T15:08:28.088 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.100+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~JM0QNPDZ68QN.0 size=1048576 2024-06-28T15:08:28.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.104+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~JM0QNPDZ68QN.1 size=1048576 2024-06-28T15:08:28.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.104+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~JM0QNPDZ68QN.2 size=312731 2024-06-28T15:08:28.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.104+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=JM0QNPDZ68QN tag=P5VYYBN7JZEZ type=write size=3458459 2024-06-28T15:08:28.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.108+0000 7f65680598c0 1 < completed delete operation key=AOBFAR0UTONH tag=KB5B8EYXAVAG type=del 2024-06-28T15:08:28.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.112+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~QXY8KGKHXK04.0 size=1048576 2024-06-28T15:08:28.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.116+0000 7f65680598c0 1 > prepared part key=_multipart_KFHGA64L1NLP.2~QXY8KGKHXK04.1 size=420491 2024-06-28T15:08:28.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.116+0000 7f65680598c0 1 > prepared multipart upload key=KFHGA64L1NLP upload=QXY8KGKHXK04 tag=Y2G03Y7TR2S8 type=write size=2517643 2024-06-28T15:08:28.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.116+0000 7f65680598c0 1 < completed delete operation key=RLJCNXKA2GYA tag=YGJ6KSRG4A2G type=del 2024-06-28T15:08:28.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.120+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=FG00JBO6AKLE type=write size=183761 2024-06-28T15:08:28.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.124+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~0KEDLLE3USH5.0 size=232907 2024-06-28T15:08:28.115 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.128+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=0KEDLLE3USH5 tag=JJ1EG7G8MO02 type=write size=1281483 2024-06-28T15:08:28.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.132+0000 7f65680598c0 1 > prepared operation key=7NL3CTFYY9E5 tag=VP1IL3LMLDGD type=write size=12611 2024-06-28T15:08:28.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.136+0000 7f65680598c0 1 < completed delete operation key=LZX5UZCYLTAG tag=1DA6XNJBPJAR type=del 2024-06-28T15:08:28.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.136+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=NSGYDII2NXA0 type=del size=1420186 2024-06-28T15:08:28.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.140+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~KSXM7W7Y31PX.0 size=794391 2024-06-28T15:08:28.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.148+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=KSXM7W7Y31PX tag=JDWO82GKC7PT type=write size=1842967 2024-06-28T15:08:28.136 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.148+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=04NJI0O0MJYU type=del size=970 2024-06-28T15:08:28.140 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.152+0000 7f65680598c0 1 < completed delete operation key=J311Q2712YM0 tag=PQOB9WNPF74M type=del 2024-06-28T15:08:28.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.156+0000 7f65680598c0 1 > prepared part key=_multipart_9MQKWSA33R5G.2~SIG0JZM0KLYL.0 size=805665 2024-06-28T15:08:28.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.156+0000 7f65680598c0 1 > prepared multipart upload key=9MQKWSA33R5G upload=SIG0JZM0KLYL tag=80CQ65XPZCD9 type=write size=1854241 2024-06-28T15:08:28.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.160+0000 7f65680598c0 1 < completed write operation key=9MQKWSA33R5G tag=J9L17DNHIZ5L type=write size=403021 2024-06-28T15:08:28.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.164+0000 7f65680598c0 1 > prepared operation key=AOBFAR0UTONH tag=GMJKCD5BKY9Z type=del size=648385 2024-06-28T15:08:28.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.164+0000 7f65680598c0 1 < canceled part key=_multipart_OUID7RX5NVV6.2~FIVC8TXP11OJ.0 size=1048576 2024-06-28T15:08:28.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.168+0000 7f65680598c0 1 < completed part key=_multipart_OUID7RX5NVV6.2~FIVC8TXP11OJ.1 size=554847 2024-06-28T15:08:28.158 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.172+0000 7f65680598c0 1 < canceled multipart upload key=OUID7RX5NVV6 upload=FIVC8TXP11OJ tag=BE1GIAVDEKG0 type=write size=2651999 2024-06-28T15:08:28.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.176+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=MIX54NNTXQNH type=del size=1794447 2024-06-28T15:08:28.164 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.176+0000 7f65680598c0 1 < completed delete operation key=R0IMFVF5NEOO tag=C5GFDV3OKHGT type=del 2024-06-28T15:08:28.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.180+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=7EVY18EA30TY type=del size=4131668 2024-06-28T15:08:28.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.184+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~C9LVJMEPJ34U.0 size=1048576 2024-06-28T15:08:28.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.184+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~C9LVJMEPJ34U.1 size=420718 2024-06-28T15:08:28.178 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.192+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=C9LVJMEPJ34U tag=6M8VCF5YTV31 type=write size=2517870 2024-06-28T15:08:28.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.196+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~OMD0FL6Z6DE1.0 size=1048576 2024-06-28T15:08:28.186 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.200+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~OMD0FL6Z6DE1.1 size=1048576 2024-06-28T15:08:28.189 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.200+0000 7f65680598c0 1 > prepared part key=_multipart_1VD7NDPEM3CD.2~OMD0FL6Z6DE1.2 size=196880 2024-06-28T15:08:28.189 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.200+0000 7f65680598c0 1 > prepared multipart upload key=1VD7NDPEM3CD upload=OMD0FL6Z6DE1 tag=5E2TMP9SU1RB type=write size=3342608 2024-06-28T15:08:28.190 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.204+0000 7f65680598c0 1 < canceled part key=_multipart_UP3XFF38IJXN.2~JMB3VUF8Q6AQ.0 size=1048576 2024-06-28T15:08:28.193 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.204+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~JMB3VUF8Q6AQ.1 size=1048576 2024-06-28T15:08:28.196 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.208+0000 7f65680598c0 1 < completed part key=_multipart_UP3XFF38IJXN.2~JMB3VUF8Q6AQ.2 size=584988 2024-06-28T15:08:28.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.212+0000 7f65680598c0 1 < canceled multipart upload key=UP3XFF38IJXN upload=JMB3VUF8Q6AQ tag=ZV2Q0U4IKC1Q type=write size=3730716 2024-06-28T15:08:28.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.220+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~WWUNQMPKONZY.0 size=1048576 2024-06-28T15:08:28.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.220+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~WWUNQMPKONZY.1 size=1048576 2024-06-28T15:08:28.212 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.224+0000 7f65680598c0 1 > prepared part key=_multipart_BHW2VHUT0U1U.2~WWUNQMPKONZY.2 size=256076 2024-06-28T15:08:28.212 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.224+0000 7f65680598c0 1 > prepared multipart upload key=BHW2VHUT0U1U upload=WWUNQMPKONZY tag=L8OR9676M3E5 type=write size=3401804 2024-06-28T15:08:28.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.228+0000 7f65680598c0 1 < completed write operation key=7LB8MOJL39RM tag=3CDL3S7GNASX type=write size=449740 2024-06-28T15:08:28.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.232+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=PJVH336EMO07 type=write size=976695 2024-06-28T15:08:28.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.232+0000 7f65680598c0 1 < canceled part key=_multipart_BHW2VHUT0U1U.2~GQUPDMNFHZC4.0 size=88965 2024-06-28T15:08:28.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.236+0000 7f65680598c0 1 < canceled multipart upload key=BHW2VHUT0U1U upload=GQUPDMNFHZC4 tag=TKZR72E21QM9 type=write size=1137541 2024-06-28T15:08:28.230 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.240+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=3KGS2G5V32TD type=del size=2428380 2024-06-28T15:08:28.230 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.244+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~1HQLLN8G3ZS0.0 size=1048576 2024-06-28T15:08:28.233 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.244+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~1HQLLN8G3ZS0.1 size=888408 2024-06-28T15:08:28.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.252+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=1HQLLN8G3ZS0 tag=Q8J63XCLKGL6 type=write size=2985560 2024-06-28T15:08:28.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.256+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=Q44HSMD8I471 type=del size=3916501 2024-06-28T15:08:28.245 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.260+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~34OWY0HG5E18.0 size=1048576 2024-06-28T15:08:28.248 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.260+0000 7f65680598c0 1 < completed part key=_multipart_C5C6SAHLPZV1.2~34OWY0HG5E18.1 size=606375 2024-06-28T15:08:28.253 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.264+0000 7f65680598c0 1 < completed multipart upload key=C5C6SAHLPZV1 upload=34OWY0HG5E18 tag=2CUFYKDA1OJ0 type=write size=2703527 2024-06-28T15:08:28.258 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.272+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~4L6KHT3YQEZP.0 size=1048576 2024-06-28T15:08:28.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.272+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~4L6KHT3YQEZP.1 size=1048576 2024-06-28T15:08:28.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.276+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~4L6KHT3YQEZP.2 size=900836 2024-06-28T15:08:28.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.276+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=4L6KHT3YQEZP tag=026CYXMMGXYJ type=write size=4046564 2024-06-28T15:08:28.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.280+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~JM0QNPDZ68QN.0 size=1048576 2024-06-28T15:08:28.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.280+0000 7f65680598c0 1 < canceled part key=_multipart_MQ3QWZBFN4YR.2~JM0QNPDZ68QN.1 size=1048576 2024-06-28T15:08:28.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.280+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~JM0QNPDZ68QN.2 size=312731 2024-06-28T15:08:28.273 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.288+0000 7f65680598c0 1 < canceled multipart upload key=MQ3QWZBFN4YR upload=JM0QNPDZ68QN tag=P5VYYBN7JZEZ type=write size=3458459 2024-06-28T15:08:28.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.292+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~J7TJ7WWICHW1.0 size=1048576 2024-06-28T15:08:28.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.292+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~J7TJ7WWICHW1.1 size=1048576 2024-06-28T15:08:28.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.296+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~J7TJ7WWICHW1.2 size=820524 2024-06-28T15:08:28.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.296+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=J7TJ7WWICHW1 tag=553DXU1ES8GM type=write size=3966252 2024-06-28T15:08:28.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.300+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~QXY8KGKHXK04.0 size=1048576 2024-06-28T15:08:28.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.300+0000 7f65680598c0 1 < completed part key=_multipart_KFHGA64L1NLP.2~QXY8KGKHXK04.1 size=420491 2024-06-28T15:08:28.293 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.304+0000 7f65680598c0 1 < completed multipart upload key=KFHGA64L1NLP upload=QXY8KGKHXK04 tag=Y2G03Y7TR2S8 type=write size=2517643 2024-06-28T15:08:28.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.312+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~KTMLIOCD9QDQ.0 size=936215 2024-06-28T15:08:28.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.312+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=KTMLIOCD9QDQ tag=FAR76ED3INXL type=write size=1984791 2024-06-28T15:08:28.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.312+0000 7f65680598c0 1 < completed write operation key=7NL3CTFYY9E5 tag=FG00JBO6AKLE type=write size=183761 2024-06-28T15:08:28.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.320+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~QXFV6S3UPDXA.0 size=1048576 2024-06-28T15:08:28.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.320+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~QXFV6S3UPDXA.1 size=1048576 2024-06-28T15:08:28.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.324+0000 7f65680598c0 1 > prepared part key=_multipart_P278ZM1H052S.2~QXFV6S3UPDXA.2 size=393596 2024-06-28T15:08:28.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.324+0000 7f65680598c0 1 > prepared multipart upload key=P278ZM1H052S upload=QXFV6S3UPDXA tag=KSP220H02B3I type=write size=3539324 2024-06-28T15:08:28.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.324+0000 7f65680598c0 1 < completed write operation key=7NL3CTFYY9E5 tag=VP1IL3LMLDGD type=write size=12611 2024-06-28T15:08:28.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.332+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~3E2AV5VOHO6Z.0 size=1048576 2024-06-28T15:08:28.320 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.332+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~3E2AV5VOHO6Z.1 size=1048576 2024-06-28T15:08:28.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.336+0000 7f65680598c0 1 > prepared part key=_multipart_AOBFAR0UTONH.2~3E2AV5VOHO6Z.2 size=128559 2024-06-28T15:08:28.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.336+0000 7f65680598c0 1 > prepared multipart upload key=AOBFAR0UTONH upload=3E2AV5VOHO6Z tag=L73DDS1UGROX type=write size=3274287 2024-06-28T15:08:28.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.340+0000 7f65680598c0 1 < completed delete operation key=60QVJVV16QFM tag=NSGYDII2NXA0 type=del 2024-06-28T15:08:28.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.344+0000 7f65680598c0 1 > prepared operation key=AXQ0YFWUH4FD tag=RNA5LPONORFJ type=del size=2819963 2024-06-28T15:08:28.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.348+0000 7f65680598c0 1 < completed delete operation key=F6SEDD6BH1T2 tag=04NJI0O0MJYU type=del 2024-06-28T15:08:28.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.352+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=5Y5QNIO0FEM9 type=del size=283313 2024-06-28T15:08:28.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.352+0000 7f65680598c0 1 < canceled part key=_multipart_9MQKWSA33R5G.2~SIG0JZM0KLYL.0 size=805665 2024-06-28T15:08:28.345 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.356+0000 7f65680598c0 1 < canceled multipart upload key=9MQKWSA33R5G upload=SIG0JZM0KLYL tag=80CQ65XPZCD9 type=write size=1854241 2024-06-28T15:08:28.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.364+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~R71WB1YKHZ72.0 size=1048576 2024-06-28T15:08:28.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.364+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~R71WB1YKHZ72.1 size=1048576 2024-06-28T15:08:28.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.368+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~R71WB1YKHZ72.2 size=449691 2024-06-28T15:08:28.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.368+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=R71WB1YKHZ72 tag=5DR4ZQ3ABPGK type=write size=3595419 2024-06-28T15:08:28.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.372+0000 7f65680598c0 1 < canceled operation key=AOBFAR0UTONH tag=GMJKCD5BKY9Z type=del size=648385 2024-06-28T15:08:28.361 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.376+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=CJIYDUQ1BPSK type=del size=2012910 2024-06-28T15:08:28.368 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.380+0000 7f65680598c0 1 < suggested update operation key=ZFNIA3O84EB7 tag=MIX54NNTXQNH type=del size=1794447 2024-06-28T15:08:28.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.384+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=OAICYIJNX5D7 type=del size=2659988 2024-06-28T15:08:28.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.388+0000 7f65680598c0 1 < canceled operation key=9MQKWSA33R5G tag=7EVY18EA30TY type=del size=4131668 2024-06-28T15:08:28.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.388+0000 7f65680598c0 1 > prepared operation key=2F968YIMJN9D tag=USTDRHDQA33P type=del size=1526852 2024-06-28T15:08:28.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.392+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~OMD0FL6Z6DE1.0 size=1048576 2024-06-28T15:08:28.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.396+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~OMD0FL6Z6DE1.1 size=1048576 2024-06-28T15:08:28.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.396+0000 7f65680598c0 1 < completed part key=_multipart_1VD7NDPEM3CD.2~OMD0FL6Z6DE1.2 size=196880 2024-06-28T15:08:28.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.404+0000 7f65680598c0 1 < completed multipart upload key=1VD7NDPEM3CD upload=OMD0FL6Z6DE1 tag=5E2TMP9SU1RB type=write size=3342608 2024-06-28T15:08:28.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.408+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~LIPIR5W6UW8T.0 size=1048576 2024-06-28T15:08:28.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.412+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~LIPIR5W6UW8T.1 size=1048576 2024-06-28T15:08:28.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.412+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~LIPIR5W6UW8T.2 size=730825 2024-06-28T15:08:28.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.412+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=LIPIR5W6UW8T tag=6K0V86J3HIP8 type=write size=3876553 2024-06-28T15:08:28.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.416+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~WWUNQMPKONZY.0 size=1048576 2024-06-28T15:08:28.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.420+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~WWUNQMPKONZY.1 size=1048576 2024-06-28T15:08:28.408 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.420+0000 7f65680598c0 1 < completed part key=_multipart_BHW2VHUT0U1U.2~WWUNQMPKONZY.2 size=256076 2024-06-28T15:08:28.412 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.424+0000 7f65680598c0 1 < completed multipart upload key=BHW2VHUT0U1U upload=WWUNQMPKONZY tag=L8OR9676M3E5 type=write size=3401804 2024-06-28T15:08:28.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.432+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~F6QITGH124R9.0 size=1048576 2024-06-28T15:08:28.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.432+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~F6QITGH124R9.1 size=851790 2024-06-28T15:08:28.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.432+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=F6QITGH124R9 tag=7MJ2T4484Y70 type=write size=2948942 2024-06-28T15:08:28.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.436+0000 7f65680598c0 1 < completed write operation key=1VD7NDPEM3CD tag=PJVH336EMO07 type=write size=976695 2024-06-28T15:08:28.427 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.440+0000 7f65680598c0 1 > prepared operation key=GAZQ3PCPKN9Q tag=IYWLSYSB919T type=del size=2969173 2024-06-28T15:08:28.430 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.444+0000 7f65680598c0 1 < completed delete operation key=GAZQ3PCPKN9Q tag=3KGS2G5V32TD type=del 2024-06-28T15:08:28.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.448+0000 7f65680598c0 1 > prepared part key=_multipart_RLJCNXKA2GYA.2~Y5LU14D56TF3.0 size=612814 2024-06-28T15:08:28.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.448+0000 7f65680598c0 1 > prepared multipart upload key=RLJCNXKA2GYA upload=Y5LU14D56TF3 tag=YGEPDUUH4J2O type=write size=1661390 2024-06-28T15:08:28.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.452+0000 7f65680598c0 1 < completed delete operation key=EN76QQWIQAXZ tag=Q44HSMD8I471 type=del 2024-06-28T15:08:28.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.456+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=QU770B06O9PY type=del size=1595853 2024-06-28T15:08:28.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.460+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~4L6KHT3YQEZP.0 size=1048576 2024-06-28T15:08:28.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.460+0000 7f65680598c0 1 < completed part key=_multipart_F6SEDD6BH1T2.2~4L6KHT3YQEZP.1 size=1048576 2024-06-28T15:08:28.449 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.460+0000 7f65680598c0 1 < canceled part key=_multipart_F6SEDD6BH1T2.2~4L6KHT3YQEZP.2 size=900836 2024-06-28T15:08:28.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.468+0000 7f65680598c0 1 < canceled multipart upload key=F6SEDD6BH1T2 upload=4L6KHT3YQEZP tag=026CYXMMGXYJ type=write size=4046564 2024-06-28T15:08:28.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.472+0000 7f65680598c0 1 > prepared operation key=SFCIZMLY2Y2E tag=AB2C11D0BX6L type=del size=2166766 2024-06-28T15:08:28.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.476+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~J7TJ7WWICHW1.0 size=1048576 2024-06-28T15:08:28.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.476+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~J7TJ7WWICHW1.1 size=1048576 2024-06-28T15:08:28.467 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.480+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~J7TJ7WWICHW1.2 size=820524 2024-06-28T15:08:28.474 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.488+0000 7f65680598c0 1 < completed multipart upload key=7NL3CTFYY9E5 upload=J7TJ7WWICHW1 tag=553DXU1ES8GM type=write size=3966252 2024-06-28T15:08:28.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.492+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~1HJJUFPREFCU.0 size=1048576 2024-06-28T15:08:28.483 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.496+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~1HJJUFPREFCU.1 size=1048576 2024-06-28T15:08:28.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.500+0000 7f65680598c0 1 > prepared part key=_multipart_ZFNIA3O84EB7.2~1HJJUFPREFCU.2 size=979069 2024-06-28T15:08:28.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.500+0000 7f65680598c0 1 > prepared multipart upload key=ZFNIA3O84EB7 upload=1HJJUFPREFCU tag=XA37XHU12W7P type=write size=4124797 2024-06-28T15:08:28.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.500+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~KTMLIOCD9QDQ.0 size=936215 2024-06-28T15:08:28.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.508+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=KTMLIOCD9QDQ tag=FAR76ED3INXL type=write size=1984791 2024-06-28T15:08:28.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.508+0000 7f65680598c0 1 > prepared operation key=RLJCNXKA2GYA tag=82AUMRDU23CA type=del size=709391 2024-06-28T15:08:28.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.512+0000 7f65680598c0 1 < canceled part key=_multipart_P278ZM1H052S.2~QXFV6S3UPDXA.0 size=1048576 2024-06-28T15:08:28.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.512+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~QXFV6S3UPDXA.1 size=1048576 2024-06-28T15:08:28.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.516+0000 7f65680598c0 1 < completed part key=_multipart_P278ZM1H052S.2~QXFV6S3UPDXA.2 size=393596 2024-06-28T15:08:28.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.520+0000 7f65680598c0 1 < canceled multipart upload key=P278ZM1H052S upload=QXFV6S3UPDXA tag=KSP220H02B3I type=write size=3539324 2024-06-28T15:08:28.511 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.524+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=0AE0JYI37B94 type=del size=1573073 2024-06-28T15:08:28.515 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.528+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~3E2AV5VOHO6Z.0 size=1048576 2024-06-28T15:08:28.517 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.532+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~3E2AV5VOHO6Z.1 size=1048576 2024-06-28T15:08:28.520 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.532+0000 7f65680598c0 1 < completed part key=_multipart_AOBFAR0UTONH.2~3E2AV5VOHO6Z.2 size=128559 2024-06-28T15:08:28.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.540+0000 7f65680598c0 1 < completed multipart upload key=AOBFAR0UTONH upload=3E2AV5VOHO6Z tag=L73DDS1UGROX type=write size=3274287 2024-06-28T15:08:28.531 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.544+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~EX9LEBMG1P70.0 size=1048576 2024-06-28T15:08:28.534 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.548+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~EX9LEBMG1P70.1 size=1048576 2024-06-28T15:08:28.536 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.548+0000 7f65680598c0 1 > prepared part key=_multipart_4CU86XS1R9G3.2~EX9LEBMG1P70.2 size=760176 2024-06-28T15:08:28.536 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.548+0000 7f65680598c0 1 > prepared multipart upload key=4CU86XS1R9G3 upload=EX9LEBMG1P70 tag=8CU5R7BZVSME type=write size=3905904 2024-06-28T15:08:28.539 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.552+0000 7f65680598c0 1 < completed delete operation key=AXQ0YFWUH4FD tag=RNA5LPONORFJ type=del 2024-06-28T15:08:28.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.556+0000 7f65680598c0 1 > prepared operation key=ZY973GBGBQ1Z tag=1JEBY41IRPIS type=write size=804584 2024-06-28T15:08:28.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.560+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=5Y5QNIO0FEM9 type=del 2024-06-28T15:08:28.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.560+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=I7XIP2OJK10M type=del size=3976003 2024-06-28T15:08:28.552 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.564+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~R71WB1YKHZ72.0 size=1048576 2024-06-28T15:08:28.555 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.568+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~R71WB1YKHZ72.1 size=1048576 2024-06-28T15:08:28.557 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.572+0000 7f65680598c0 1 < completed part key=_multipart_J311Q2712YM0.2~R71WB1YKHZ72.2 size=449691 2024-06-28T15:08:28.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.576+0000 7f65680598c0 1 < completed multipart upload key=J311Q2712YM0 upload=R71WB1YKHZ72 tag=5DR4ZQ3ABPGK type=write size=3595419 2024-06-28T15:08:28.568 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.580+0000 7f65680598c0 1 > prepared part key=_multipart_D9D3OFIJP3D3.2~GV6ZHR05QV7K.0 size=933563 2024-06-28T15:08:28.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.580+0000 7f65680598c0 1 > prepared multipart upload key=D9D3OFIJP3D3 upload=GV6ZHR05QV7K tag=PEYHRQNYA1Z5 type=write size=1982139 2024-06-28T15:08:28.575 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.588+0000 7f65680598c0 1 < suggested update operation key=ZFNIA3O84EB7 tag=CJIYDUQ1BPSK type=del size=2012910 2024-06-28T15:08:28.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.596+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~PLOK9WW95Y1O.0 size=1048576 2024-06-28T15:08:28.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.596+0000 7f65680598c0 1 > prepared part key=_multipart_ZY973GBGBQ1Z.2~PLOK9WW95Y1O.1 size=790862 2024-06-28T15:08:28.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.596+0000 7f65680598c0 1 > prepared multipart upload key=ZY973GBGBQ1Z upload=PLOK9WW95Y1O tag=A75OC1SU76MO type=write size=2888014 2024-06-28T15:08:28.587 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.600+0000 7f65680598c0 1 < completed delete operation key=UP3XFF38IJXN tag=OAICYIJNX5D7 type=del 2024-06-28T15:08:28.591 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.604+0000 7f65680598c0 1 > prepared operation key=EN76QQWIQAXZ tag=3MZVYA5PARJL type=del size=1938211 2024-06-28T15:08:28.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.608+0000 7f65680598c0 1 < completed delete operation key=2F968YIMJN9D tag=USTDRHDQA33P type=del 2024-06-28T15:08:28.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.612+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~R6KPZCYR64ZP.0 size=1048576 2024-06-28T15:08:28.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.616+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~R6KPZCYR64ZP.1 size=1048576 2024-06-28T15:08:28.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.616+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~R6KPZCYR64ZP.2 size=370045 2024-06-28T15:08:28.605 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.616+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=R6KPZCYR64ZP tag=356J76KI4MB1 type=write size=3515773 2024-06-28T15:08:28.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.620+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~LIPIR5W6UW8T.0 size=1048576 2024-06-28T15:08:28.611 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.624+0000 7f65680598c0 1 < completed part key=_multipart_V2L2Q1U62YW2.2~LIPIR5W6UW8T.1 size=1048576 2024-06-28T15:08:28.611 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.624+0000 7f65680598c0 1 < canceled part key=_multipart_V2L2Q1U62YW2.2~LIPIR5W6UW8T.2 size=730825 2024-06-28T15:08:28.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.628+0000 7f65680598c0 1 < canceled multipart upload key=V2L2Q1U62YW2 upload=LIPIR5W6UW8T tag=6K0V86J3HIP8 type=write size=3876553 2024-06-28T15:08:28.627 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.640+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~B9PKZR0INBWA.0 size=1048576 2024-06-28T15:08:28.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.644+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~B9PKZR0INBWA.1 size=872659 2024-06-28T15:08:28.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.644+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=B9PKZR0INBWA tag=7UG5THJBKVF9 type=write size=2969811 2024-06-28T15:08:28.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.644+0000 7f65680598c0 1 < canceled part key=_multipart_7NL3CTFYY9E5.2~F6QITGH124R9.0 size=1048576 2024-06-28T15:08:28.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.644+0000 7f65680598c0 1 < canceled part key=_multipart_7NL3CTFYY9E5.2~F6QITGH124R9.1 size=851790 2024-06-28T15:08:28.638 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.648+0000 7f65680598c0 1 < canceled multipart upload key=7NL3CTFYY9E5 upload=F6QITGH124R9 tag=7MJ2T4484Y70 type=write size=2948942 2024-06-28T15:08:28.642 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.656+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~8AK8FH1HZFOO.0 size=1048576 2024-06-28T15:08:28.645 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.656+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~8AK8FH1HZFOO.1 size=467090 2024-06-28T15:08:28.645 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.656+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=8AK8FH1HZFOO tag=HH8DIE3YYO8D type=write size=2564242 2024-06-28T15:08:28.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.664+0000 7f65680598c0 1 < suggested update operation key=GAZQ3PCPKN9Q tag=IYWLSYSB919T type=del size=2969173 2024-06-28T15:08:28.658 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.672+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~CG7OD0HQ8ZDS.0 size=1048576 2024-06-28T15:08:28.661 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.672+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~CG7OD0HQ8ZDS.1 size=1048576 2024-06-28T15:08:28.664 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.676+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~CG7OD0HQ8ZDS.2 size=428796 2024-06-28T15:08:28.664 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.676+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=CG7OD0HQ8ZDS tag=M335OTTM7WVM type=write size=3574524 2024-06-28T15:08:28.668 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.680+0000 7f65680598c0 1 < completed part key=_multipart_RLJCNXKA2GYA.2~Y5LU14D56TF3.0 size=612814 2024-06-28T15:08:28.674 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.688+0000 7f65680598c0 1 < completed multipart upload key=RLJCNXKA2GYA upload=Y5LU14D56TF3 tag=YGEPDUUH4J2O type=write size=1661390 2024-06-28T15:08:28.676 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.688+0000 7f65680598c0 1 > prepared operation key=F6SEDD6BH1T2 tag=6RGTC0AX7JJ7 type=del size=3431554 2024-06-28T15:08:28.682 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.692+0000 7f65680598c0 1 < completed delete operation key=I4RWCQSQVLCN tag=QU770B06O9PY type=del 2024-06-28T15:08:28.682 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.696+0000 7f65680598c0 1 > prepared operation key=60QVJVV16QFM tag=2KGPB1U60Y7L type=write size=57298 2024-06-28T15:08:28.688 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.700+0000 7f65680598c0 1 < suggested update operation key=SFCIZMLY2Y2E tag=AB2C11D0BX6L type=del size=2166766 2024-06-28T15:08:28.691 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.704+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=MJR1JLPG9M1T type=del size=1782171 2024-06-28T15:08:28.694 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.708+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~1HJJUFPREFCU.0 size=1048576 2024-06-28T15:08:28.696 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.708+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~1HJJUFPREFCU.1 size=1048576 2024-06-28T15:08:28.698 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.712+0000 7f65680598c0 1 < completed part key=_multipart_ZFNIA3O84EB7.2~1HJJUFPREFCU.2 size=979069 2024-06-28T15:08:28.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.716+0000 7f65680598c0 1 < completed multipart upload key=ZFNIA3O84EB7 upload=1HJJUFPREFCU tag=XA37XHU12W7P type=write size=4124797 2024-06-28T15:08:28.707 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.720+0000 7f65680598c0 1 > prepared operation key=ZFNIA3O84EB7 tag=DZWHZAGDG1SP type=del size=1981115 2024-06-28T15:08:28.710 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.724+0000 7f65680598c0 1 < canceled operation key=RLJCNXKA2GYA tag=82AUMRDU23CA type=del size=709391 2024-06-28T15:08:28.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.728+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~K89AYNI9A8YA.0 size=53826 2024-06-28T15:08:28.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.728+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=K89AYNI9A8YA tag=E9093915I9GK type=write size=1102402 2024-06-28T15:08:28.718 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.732+0000 7f65680598c0 -1 < failed to complete operation key=ZFNIA3O84EB7 tag=0AE0JYI37B94 type=del size=1573073: (22) Invalid argument 2024-06-28T15:08:28.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.736+0000 7f65680598c0 1 > prepared part key=_multipart_MQ3QWZBFN4YR.2~JVF2AI6MOWFK.0 size=194028 2024-06-28T15:08:28.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.736+0000 7f65680598c0 1 > prepared multipart upload key=MQ3QWZBFN4YR upload=JVF2AI6MOWFK tag=7SYMX6MT68LL type=write size=1242604 2024-06-28T15:08:28.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.740+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~EX9LEBMG1P70.0 size=1048576 2024-06-28T15:08:28.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.740+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~EX9LEBMG1P70.1 size=1048576 2024-06-28T15:08:28.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.744+0000 7f65680598c0 1 < completed part key=_multipart_4CU86XS1R9G3.2~EX9LEBMG1P70.2 size=760176 2024-06-28T15:08:28.735 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.748+0000 7f65680598c0 1 < completed multipart upload key=4CU86XS1R9G3 upload=EX9LEBMG1P70 tag=8CU5R7BZVSME type=write size=3905904 2024-06-28T15:08:28.738 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.752+0000 7f65680598c0 1 > prepared operation key=O1CGONN4TEXF tag=OUJ0AKGI8GLK type=del size=581204 2024-06-28T15:08:28.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.752+0000 7f65680598c0 1 < completed write operation key=ZY973GBGBQ1Z tag=1JEBY41IRPIS type=write size=804584 2024-06-28T15:08:28.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.760+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~M3SSNX29FCOT.0 size=1048576 2024-06-28T15:08:28.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.760+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~M3SSNX29FCOT.1 size=1048576 2024-06-28T15:08:28.750 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.764+0000 7f65680598c0 1 > prepared part key=_multipart_7NL3CTFYY9E5.2~M3SSNX29FCOT.2 size=287829 2024-06-28T15:08:28.750 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.764+0000 7f65680598c0 1 > prepared multipart upload key=7NL3CTFYY9E5 upload=M3SSNX29FCOT tag=1K9WEMXL22AI type=write size=3433557 2024-06-28T15:08:28.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.768+0000 7f65680598c0 1 < completed delete operation key=9MQKWSA33R5G tag=I7XIP2OJK10M type=del 2024-06-28T15:08:28.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.772+0000 7f65680598c0 1 > prepared part key=_multipart_R0IMFVF5NEOO.2~9FQ6S9243QZW.0 size=985506 2024-06-28T15:08:28.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.772+0000 7f65680598c0 1 > prepared multipart upload key=R0IMFVF5NEOO upload=9FQ6S9243QZW tag=4YB1WL0IMYSF type=write size=2034082 2024-06-28T15:08:28.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.776+0000 7f65680598c0 1 < completed part key=_multipart_D9D3OFIJP3D3.2~GV6ZHR05QV7K.0 size=933563 2024-06-28T15:08:28.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.780+0000 7f65680598c0 1 < completed multipart upload key=D9D3OFIJP3D3 upload=GV6ZHR05QV7K tag=PEYHRQNYA1Z5 type=write size=1982139 2024-06-28T15:08:28.773 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.784+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~IQH3VLRYXJKM.0 size=1048576 2024-06-28T15:08:28.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.788+0000 7f65680598c0 1 > prepared part key=_multipart_SFCIZMLY2Y2E.2~IQH3VLRYXJKM.1 size=66326 2024-06-28T15:08:28.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.788+0000 7f65680598c0 1 > prepared multipart upload key=SFCIZMLY2Y2E upload=IQH3VLRYXJKM tag=BTYF0H0SUNW4 type=write size=2163478 2024-06-28T15:08:28.779 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.792+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~PLOK9WW95Y1O.0 size=1048576 2024-06-28T15:08:28.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.792+0000 7f65680598c0 1 < completed part key=_multipart_ZY973GBGBQ1Z.2~PLOK9WW95Y1O.1 size=790862 2024-06-28T15:08:28.786 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.800+0000 7f65680598c0 1 < completed multipart upload key=ZY973GBGBQ1Z upload=PLOK9WW95Y1O tag=A75OC1SU76MO type=write size=2888014 2024-06-28T15:08:28.789 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.804+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=IAYCF3XIBITV type=del size=4089509 2024-06-28T15:08:28.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.804+0000 7f65680598c0 1 < canceled operation key=EN76QQWIQAXZ tag=3MZVYA5PARJL type=del size=1938211 2024-06-28T15:08:28.795 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.808+0000 7f65680598c0 1 > prepared operation key=KFHGA64L1NLP tag=4486TTUEN1AX type=write size=902617 2024-06-28T15:08:28.798 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.812+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~R6KPZCYR64ZP.0 size=1048576 2024-06-28T15:08:28.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.812+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~R6KPZCYR64ZP.1 size=1048576 2024-06-28T15:08:28.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.816+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~R6KPZCYR64ZP.2 size=370045 2024-06-28T15:08:28.808 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.820+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=R6KPZCYR64ZP tag=356J76KI4MB1 type=write size=3515773 2024-06-28T15:08:28.811 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.824+0000 7f65680598c0 1 > prepared operation key=MQ3QWZBFN4YR tag=HNDBDKV1AGYE type=del size=3591094 2024-06-28T15:08:28.814 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.828+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~B9PKZR0INBWA.0 size=1048576 2024-06-28T15:08:28.816 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.828+0000 7f65680598c0 1 < completed part key=_multipart_O1CGONN4TEXF.2~B9PKZR0INBWA.1 size=872659 2024-06-28T15:08:28.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.836+0000 7f65680598c0 1 < completed multipart upload key=O1CGONN4TEXF upload=B9PKZR0INBWA tag=7UG5THJBKVF9 type=write size=2969811 2024-06-28T15:08:28.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.840+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~Z5NCDLS33VIP.0 size=1048576 2024-06-28T15:08:28.829 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.840+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~Z5NCDLS33VIP.1 size=1048576 2024-06-28T15:08:28.831 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.844+0000 7f65680598c0 1 > prepared part key=_multipart_OUID7RX5NVV6.2~Z5NCDLS33VIP.2 size=121434 2024-06-28T15:08:28.831 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.844+0000 7f65680598c0 1 > prepared multipart upload key=OUID7RX5NVV6 upload=Z5NCDLS33VIP tag=U91RR9PTT1DD type=write size=3267162 2024-06-28T15:08:28.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.848+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~8AK8FH1HZFOO.0 size=1048576 2024-06-28T15:08:28.836 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.848+0000 7f65680598c0 1 < completed part key=_multipart_2FC9DNEZGWC4.2~8AK8FH1HZFOO.1 size=467090 2024-06-28T15:08:28.841 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.852+0000 7f65680598c0 1 < completed multipart upload key=2FC9DNEZGWC4 upload=8AK8FH1HZFOO tag=HH8DIE3YYO8D type=write size=2564242 2024-06-28T15:08:28.846 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.860+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~1GVQGH7RVO63.0 size=1048576 2024-06-28T15:08:28.848 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.860+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~1GVQGH7RVO63.1 size=1048576 2024-06-28T15:08:28.850 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.864+0000 7f65680598c0 1 > prepared part key=_multipart_V2L2Q1U62YW2.2~1GVQGH7RVO63.2 size=471821 2024-06-28T15:08:28.850 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.864+0000 7f65680598c0 1 > prepared multipart upload key=V2L2Q1U62YW2 upload=1GVQGH7RVO63 tag=GKAG8L5SLYCQ type=write size=3617549 2024-06-28T15:08:28.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.864+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~CG7OD0HQ8ZDS.0 size=1048576 2024-06-28T15:08:28.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.868+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~CG7OD0HQ8ZDS.1 size=1048576 2024-06-28T15:08:28.858 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.872+0000 7f65680598c0 1 < completed part key=_multipart_I4RWCQSQVLCN.2~CG7OD0HQ8ZDS.2 size=428796 2024-06-28T15:08:28.862 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.876+0000 7f65680598c0 1 < completed multipart upload key=I4RWCQSQVLCN upload=CG7OD0HQ8ZDS tag=M335OTTM7WVM type=write size=3574524 2024-06-28T15:08:28.867 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.880+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~WDCTVXA9P574.0 size=1048576 2024-06-28T15:08:28.870 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.884+0000 7f65680598c0 1 > prepared part key=_multipart_EN76QQWIQAXZ.2~WDCTVXA9P574.1 size=779170 2024-06-28T15:08:28.870 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.884+0000 7f65680598c0 1 > prepared multipart upload key=EN76QQWIQAXZ upload=WDCTVXA9P574 tag=BK2NMAAWN999 type=write size=2876322 2024-06-28T15:08:28.875 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.888+0000 7f65680598c0 1 < suggested update operation key=F6SEDD6BH1T2 tag=6RGTC0AX7JJ7 type=del size=3431554 2024-06-28T15:08:28.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.892+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~C9CB54Q2SEX2.0 size=1048576 2024-06-28T15:08:28.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.896+0000 7f65680598c0 1 > prepared part key=_multipart_J311Q2712YM0.2~C9CB54Q2SEX2.1 size=934604 2024-06-28T15:08:28.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.896+0000 7f65680598c0 1 > prepared multipart upload key=J311Q2712YM0 upload=C9CB54Q2SEX2 tag=1EMY4NBCDW4Q type=write size=3031756 2024-06-28T15:08:28.885 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.896+0000 7f65680598c0 1 < completed write operation key=60QVJVV16QFM tag=2KGPB1U60Y7L type=write size=57298 2024-06-28T15:08:28.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.904+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~NJ8F01FRL6OR.0 size=1048576 2024-06-28T15:08:28.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.904+0000 7f65680598c0 1 > prepared part key=_multipart_F6SEDD6BH1T2.2~NJ8F01FRL6OR.1 size=260552 2024-06-28T15:08:28.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.904+0000 7f65680598c0 1 > prepared multipart upload key=F6SEDD6BH1T2 upload=NJ8F01FRL6OR tag=O4NDYMSIP0HJ type=write size=2357704 2024-06-28T15:08:28.901 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.912+0000 7f65680598c0 1 < suggested update operation key=9MQKWSA33R5G tag=MJR1JLPG9M1T type=del size=1782171 2024-06-28T15:08:28.904 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.916+0000 7f65680598c0 1 > prepared operation key=I4RWCQSQVLCN tag=CIHMNXH8O9NU type=del size=4101292 2024-06-28T15:08:28.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.924+0000 7f65680598c0 1 < suggested update operation key=ZFNIA3O84EB7 tag=DZWHZAGDG1SP type=del size=1981115 2024-06-28T15:08:28.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.928+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~PO65DFER3U2Z.0 size=1048576 2024-06-28T15:08:28.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.932+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~PO65DFER3U2Z.1 size=1048576 2024-06-28T15:08:28.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.932+0000 7f65680598c0 1 > prepared part key=_multipart_O1CGONN4TEXF.2~PO65DFER3U2Z.2 size=37528 2024-06-28T15:08:28.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.932+0000 7f65680598c0 1 > prepared multipart upload key=O1CGONN4TEXF upload=PO65DFER3U2Z tag=JZ6947BG0Q3G type=write size=3183256 2024-06-28T15:08:28.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.936+0000 7f65680598c0 1 < completed part key=_multipart_EN76QQWIQAXZ.2~K89AYNI9A8YA.0 size=53826 2024-06-28T15:08:28.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.940+0000 7f65680598c0 1 < completed multipart upload key=EN76QQWIQAXZ upload=K89AYNI9A8YA tag=E9093915I9GK type=write size=1102402 2024-06-28T15:08:28.931 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.944+0000 7f65680598c0 1 > prepared operation key=9MQKWSA33R5G tag=AI8YC7QGE1Z2 type=del size=2285150 2024-06-28T15:08:28.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.948+0000 7f65680598c0 1 < completed part key=_multipart_MQ3QWZBFN4YR.2~JVF2AI6MOWFK.0 size=194028 2024-06-28T15:08:28.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.952+0000 7f65680598c0 1 < completed multipart upload key=MQ3QWZBFN4YR upload=JVF2AI6MOWFK tag=7SYMX6MT68LL type=write size=1242604 2024-06-28T15:08:28.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.956+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~TZ4TWBQW6D6B.0 size=1048576 2024-06-28T15:08:28.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.960+0000 7f65680598c0 1 > prepared part key=_multipart_2FC9DNEZGWC4.2~TZ4TWBQW6D6B.1 size=37115 2024-06-28T15:08:28.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.960+0000 7f65680598c0 1 > prepared multipart upload key=2FC9DNEZGWC4 upload=TZ4TWBQW6D6B tag=NWL1XBGHFZ85 type=write size=2134267 2024-06-28T15:08:28.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.960+0000 7f65680598c0 1 < completed delete operation key=O1CGONN4TEXF tag=OUJ0AKGI8GLK type=del 2024-06-28T15:08:28.951 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.964+0000 7f65680598c0 1 > prepared operation key=1VD7NDPEM3CD tag=47Z68UW3SHYN type=del size=4021243 2024-06-28T15:08:28.952 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.964+0000 7f65680598c0 1 < canceled part key=_multipart_7NL3CTFYY9E5.2~M3SSNX29FCOT.0 size=1048576 2024-06-28T15:08:28.954 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.968+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~M3SSNX29FCOT.1 size=1048576 2024-06-28T15:08:28.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.968+0000 7f65680598c0 1 < completed part key=_multipart_7NL3CTFYY9E5.2~M3SSNX29FCOT.2 size=287829 2024-06-28T15:08:28.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.976+0000 7f65680598c0 1 < canceled multipart upload key=7NL3CTFYY9E5 upload=M3SSNX29FCOT tag=1K9WEMXL22AI type=write size=3433557 2024-06-28T15:08:28.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.980+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~8KQP9LWB9G40.0 size=1048576 2024-06-28T15:08:28.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.980+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~8KQP9LWB9G40.1 size=1048576 2024-06-28T15:08:28.970 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.984+0000 7f65680598c0 1 > prepared part key=_multipart_I4RWCQSQVLCN.2~8KQP9LWB9G40.2 size=151714 2024-06-28T15:08:28.970 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.984+0000 7f65680598c0 1 > prepared multipart upload key=I4RWCQSQVLCN upload=8KQP9LWB9G40 tag=MPVGQWJON5D6 type=write size=3297442 2024-06-28T15:08:28.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.984+0000 7f65680598c0 1 < completed part key=_multipart_R0IMFVF5NEOO.2~9FQ6S9243QZW.0 size=985506 2024-06-28T15:08:28.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.988+0000 7f65680598c0 1 < completed multipart upload key=R0IMFVF5NEOO upload=9FQ6S9243QZW tag=4YB1WL0IMYSF type=write size=2034082 2024-06-28T15:08:28.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.992+0000 7f65680598c0 1 > prepared operation key=OUID7RX5NVV6 tag=5P4DS55CDP2N type=del size=733451 2024-06-28T15:08:28.982 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.996+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~IQH3VLRYXJKM.0 size=1048576 2024-06-28T15:08:28.984 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:28.996+0000 7f65680598c0 1 < completed part key=_multipart_SFCIZMLY2Y2E.2~IQH3VLRYXJKM.1 size=66326 2024-06-28T15:08:28.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:29.000+0000 7f65680598c0 1 < completed multipart upload key=SFCIZMLY2Y2E upload=IQH3VLRYXJKM tag=BTYF0H0SUNW4 type=write size=2163478 2024-06-28T15:08:28.991 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:08:29.004+0000 7f65680598c0 1 > prepared operation key=UP3XFF38IJXN tag=8O2IP1A6X3Z1 type=del size=2061500 2024-06-28T15:08:28.991 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] cls_rgw_stats.simulate (20218 ms) 2024-06-28T15:08:28.992 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 1 test from cls_rgw_stats (20224 ms total) 2024-06-28T15:08:28.992 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:08:28.992 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:08:29.980 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 1 test from 1 test suite ran. (23105 ms total) 2024-06-28T15:08:29.980 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 1 test. 2024-06-28T15:08:29.986 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:08:29.986 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:08:30.042 INFO:tasks.workunit:Running workunits matching cls/test_cls_cmpomap.sh on client.0... 2024-06-28T15:08:30.043 INFO:tasks.workunit:Running workunit cls/test_cls_cmpomap.sh... 2024-06-28T15:08:30.043 DEBUG:teuthology.orchestra.run.smithi178:workunit test cls/test_cls_cmpomap.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_cmpomap.sh 2024-06-28T15:08:30.109 INFO:tasks.workunit.client.0.smithi178.stdout:Running main() from gmock_main.cc 2024-06-28T15:08:30.109 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 31 tests from 1 test suite. 2024-06-28T15:08:30.109 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:08:31.988 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 31 tests from CmpOmap 2024-06-28T15:08:31.988 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str 2024-06-28T15:08:32.016 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str (28 ms) 2024-06-28T15:08:32.016 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str_default 2024-06-28T15:08:32.046 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str_default (31 ms) 2024-06-28T15:08:32.046 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64 2024-06-28T15:08:32.052 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64 (5 ms) 2024-06-28T15:08:32.052 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64_default 2024-06-28T15:08:32.058 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64_default (7 ms) 2024-06-28T15:08:32.058 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_vals_str 2024-06-28T15:08:32.073 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_vals_str (15 ms) 2024-06-28T15:08:32.073 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_vals_u64 2024-06-28T15:08:32.086 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_vals_u64 (13 ms) 2024-06-28T15:08:32.086 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_input 2024-06-28T15:08:32.093 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_input (6 ms) 2024-06-28T15:08:32.093 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_default 2024-06-28T15:08:32.102 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_default (9 ms) 2024-06-28T15:08:32.102 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_vals_u64_empty_default 2024-06-28T15:08:32.112 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_vals_u64_empty_default (10 ms) 2024-06-28T15:08:32.112 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_value 2024-06-28T15:08:32.123 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_value (10 ms) 2024-06-28T15:08:32.123 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_vals_at_max_keys 2024-06-28T15:08:32.125 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_vals_at_max_keys (2 ms) 2024-06-28T15:08:32.125 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_vals_over_max_keys 2024-06-28T15:08:32.127 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_vals_over_max_keys (2 ms) 2024-06-28T15:08:32.127 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str 2024-06-28T15:08:32.143 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str (16 ms) 2024-06-28T15:08:32.143 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str_default 2024-06-28T15:08:32.158 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str_default (16 ms) 2024-06-28T15:08:32.159 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64 2024-06-28T15:08:32.175 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64 (16 ms) 2024-06-28T15:08:32.175 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64_default 2024-06-28T15:08:32.192 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64_default (17 ms) 2024-06-28T15:08:32.192 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_set_vals_str 2024-06-28T15:08:32.210 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_set_vals_str (19 ms) 2024-06-28T15:08:32.211 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64 2024-06-28T15:08:32.231 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_set_vals_u64 (21 ms) 2024-06-28T15:08:32.231 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_einval 2024-06-28T15:08:32.238 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_einval (6 ms) 2024-06-28T15:08:32.238 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_eio 2024-06-28T15:08:32.244 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_eio (6 ms) 2024-06-28T15:08:32.244 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_set_vals_at_max_keys 2024-06-28T15:08:32.247 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_set_vals_at_max_keys (3 ms) 2024-06-28T15:08:32.247 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_set_vals_over_max_keys 2024-06-28T15:08:32.249 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_set_vals_over_max_keys (2 ms) 2024-06-28T15:08:32.249 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_str 2024-06-28T15:08:32.265 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_str (17 ms) 2024-06-28T15:08:32.265 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_u64 2024-06-28T15:08:32.282 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_u64 (16 ms) 2024-06-28T15:08:32.282 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_rm_keys_str 2024-06-28T15:08:32.301 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_rm_keys_str (20 ms) 2024-06-28T15:08:32.301 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64 2024-06-28T15:08:32.322 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64 (20 ms) 2024-06-28T15:08:32.322 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_einval 2024-06-28T15:08:32.328 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_einval (6 ms) 2024-06-28T15:08:32.328 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_eio 2024-06-28T15:08:32.334 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_eio (7 ms) 2024-06-28T15:08:32.334 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_rm_keys_at_max_keys 2024-06-28T15:08:32.337 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_rm_keys_at_max_keys (3 ms) 2024-06-28T15:08:32.337 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_rm_keys_over_max_keys 2024-06-28T15:08:32.339 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_rm_keys_over_max_keys (2 ms) 2024-06-28T15:08:32.339 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_empty 2024-06-28T15:08:32.359 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_empty (19 ms) 2024-06-28T15:08:32.359 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 31 tests from CmpOmap (371 ms total) 2024-06-28T15:08:32.359 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:08:32.359 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:08:32.996 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 31 tests from 1 test suite ran. (2887 ms total) 2024-06-28T15:08:32.996 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 31 tests. 2024-06-28T15:08:33.001 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:08:33.001 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:08:33.058 INFO:tasks.workunit:Running workunits matching cls/test_cls_2pc_queue.sh on client.0... 2024-06-28T15:08:33.060 INFO:tasks.workunit:Running workunit cls/test_cls_2pc_queue.sh... 2024-06-28T15:08:33.060 DEBUG:teuthology.orchestra.run.smithi178:workunit test cls/test_cls_2pc_queue.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh 2024-06-28T15:08:33.124 INFO:tasks.workunit.client.0.smithi178.stdout:Running main() from gmock_main.cc 2024-06-28T15:08:33.124 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 24 tests from 1 test suite. 2024-06-28T15:08:33.125 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:08:33.125 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 24 tests from TestCls2PCQueue 2024-06-28T15:08:33.125 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.GetCapacity 2024-06-28T15:08:36.035 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.GetCapacity (2910 ms) 2024-06-28T15:08:36.035 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.AsyncGetCapacity 2024-06-28T15:08:39.077 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.AsyncGetCapacity (3042 ms) 2024-06-28T15:08:39.077 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.Reserve 2024-06-28T15:08:42.105 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.Reserve (3026 ms) 2024-06-28T15:08:42.105 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.AsyncReserve 2024-06-28T15:08:45.150 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.AsyncReserve (3047 ms) 2024-06-28T15:08:45.150 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.Commit 2024-06-28T15:08:49.234 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.Commit (4084 ms) 2024-06-28T15:08:49.234 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.Stats 2024-06-28T15:08:53.279 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.Stats (4045 ms) 2024-06-28T15:08:53.280 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.UpgradeFromReef 2024-06-28T15:08:57.342 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.UpgradeFromReef (4058 ms) 2024-06-28T15:08:57.342 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.Abort 2024-06-28T15:09:00.391 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.Abort (3053 ms) 2024-06-28T15:09:00.391 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.ReserveError 2024-06-28T15:09:03.418 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.ReserveError (3028 ms) 2024-06-28T15:09:03.418 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.CommitError 2024-06-28T15:09:06.483 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.CommitError (3065 ms) 2024-06-28T15:09:06.483 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.AbortError 2024-06-28T15:09:09.525 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.AbortError (3041 ms) 2024-06-28T15:09:09.525 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.MultiReserve 2024-06-28T15:09:12.535 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.MultiReserve (3010 ms) 2024-06-28T15:09:12.535 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.MultiCommit 2024-06-28T15:09:15.601 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.MultiCommit (3062 ms) 2024-06-28T15:09:15.601 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.MultiAbort 2024-06-28T15:09:18.640 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.MultiAbort (3044 ms) 2024-06-28T15:09:18.641 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.ReserveCommit 2024-06-28T15:09:21.697 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.ReserveCommit (3047 ms) 2024-06-28T15:09:21.697 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.ReserveAbort 2024-06-28T15:09:24.731 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.ReserveAbort (3043 ms) 2024-06-28T15:09:24.731 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.ManualCleanup 2024-06-28T15:09:27.779 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.ManualCleanup (3045 ms) 2024-06-28T15:09:27.779 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.Cleanup 2024-06-28T15:09:30.793 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.Cleanup (3015 ms) 2024-06-28T15:09:30.793 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.MultiProducer 2024-06-28T15:09:42.987 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.MultiProducer (12196 ms) 2024-06-28T15:09:42.987 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.AsyncConsumer 2024-06-28T15:09:48.058 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.AsyncConsumer (5071 ms) 2024-06-28T15:09:48.058 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.MultiProducerConsumer 2024-06-28T15:10:01.004 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.MultiProducerConsumer (12946 ms) 2024-06-28T15:10:01.005 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.ReserveSpillover 2024-06-28T15:10:07.056 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.ReserveSpillover (6052 ms) 2024-06-28T15:10:07.056 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.CommitSpillover 2024-06-28T15:10:17.010 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.CommitSpillover (9952 ms) 2024-06-28T15:10:17.010 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestCls2PCQueue.AbortSpillover 2024-06-28T15:10:27.005 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestCls2PCQueue.AbortSpillover (9998 ms) 2024-06-28T15:10:27.006 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 24 tests from TestCls2PCQueue (113882 ms total) 2024-06-28T15:10:27.007 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:10:27.007 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:10:27.007 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 24 tests from 1 test suite ran. (113883 ms total) 2024-06-28T15:10:27.007 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 24 tests. 2024-06-28T15:10:27.018 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:10:27.018 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:10:27.073 INFO:tasks.workunit:Running workunits matching cls/test_cls_user.sh on client.0... 2024-06-28T15:10:27.074 INFO:tasks.workunit:Running workunit cls/test_cls_user.sh... 2024-06-28T15:10:27.074 DEBUG:teuthology.orchestra.run.smithi178:workunit test cls/test_cls_user.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh 2024-06-28T15:10:27.149 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:10:27.149 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:10:27.150 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 4 tests from 1 test suite. 2024-06-28T15:10:27.150 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:10:29.010 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 4 tests from ClsAccount 2024-06-28T15:10:29.010 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsAccount.add 2024-06-28T15:10:29.059 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsAccount.add (49 ms) 2024-06-28T15:10:29.059 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsAccount.get 2024-06-28T15:10:29.066 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsAccount.get (8 ms) 2024-06-28T15:10:29.066 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsAccount.rm 2024-06-28T15:10:29.078 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsAccount.rm (11 ms) 2024-06-28T15:10:29.078 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] ClsAccount.list 2024-06-28T15:10:29.118 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] ClsAccount.list (40 ms) 2024-06-28T15:10:29.118 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 4 tests from ClsAccount (109 ms total) 2024-06-28T15:10:29.118 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:10:29.118 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:10:30.044 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 4 tests from 1 test suite ran. (2895 ms total) 2024-06-28T15:10:30.044 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 4 tests. 2024-06-28T15:10:30.051 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:10:30.051 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:10:30.109 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2024-06-28T15:10:30.111 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2024-06-28T15:10:30.111 DEBUG:teuthology.orchestra.run.smithi178:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2024-06-28T15:10:30.185 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:10:30.186 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:10:30.186 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 6 tests from 1 test suite. 2024-06-28T15:10:30.186 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:10:32.071 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 6 tests from rgw_gc_log 2024-06-28T15:10:32.072 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2024-06-28T15:10:32.285 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] rgw_gc_log.init_existing_queue (214 ms) 2024-06-28T15:10:32.285 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2024-06-28T15:10:32.292 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] rgw_gc_log.init_existing_omap (7 ms) 2024-06-28T15:10:32.292 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2024-06-28T15:10:32.300 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (8 ms) 2024-06-28T15:10:32.300 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2024-06-28T15:10:32.304 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (4 ms) 2024-06-28T15:10:32.304 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2024-06-28T15:10:32.310 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] rgw_gc_log.defer1_after_init (6 ms) 2024-06-28T15:10:32.310 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2024-06-28T15:10:32.314 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] rgw_gc_log.defer2_before_init (4 ms) 2024-06-28T15:10:32.314 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 6 tests from rgw_gc_log (243 ms total) 2024-06-28T15:10:32.315 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:10:32.315 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:10:33.103 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 6 tests from 1 test suite ran. (2916 ms total) 2024-06-28T15:10:33.103 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 6 tests. 2024-06-28T15:10:33.110 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:10:33.110 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:10:33.165 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_obj.sh on client.0... 2024-06-28T15:10:33.166 INFO:tasks.workunit:Running workunit rgw/test_rgw_obj.sh... 2024-06-28T15:10:33.166 DEBUG:teuthology.orchestra.run.smithi178:workunit test rgw/test_rgw_obj.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_obj.sh 2024-06-28T15:10:33.241 INFO:tasks.workunit.client.0.smithi178.stdout:Running main() from gmock_main.cc 2024-06-28T15:10:33.241 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 4 tests from 1 test suite. 2024-06-28T15:10:33.242 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:10:33.242 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 4 tests from TestRGWObj 2024-06-28T15:10:33.242 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestRGWObj.underscore 2024-06-28T15:10:33.242 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj 2024-06-28T15:10:33.242 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj 2024-06-28T15:10:33.242 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: _obj ns= i= 2024-06-28T15:10:33.242 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj 2024-06-28T15:10:33.242 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj 2024-06-28T15:10:33.242 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: _obj ns= i= 2024-06-28T15:10:33.242 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj 2024-06-28T15:10:33.242 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: _obj ns= i= 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout: "obj1": { 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.243 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.244 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.244 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.244 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.244 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.244 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.244 INFO:tasks.workunit.client.0.smithi178.stdout: "obj2": { 2024-06-28T15:10:33.244 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.244 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.244 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.244 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.244 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.244 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.244 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.244 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.245 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.245 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.245 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.245 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.245 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.245 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.245 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.245 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.245 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.245 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3": { 2024-06-28T15:10:33.245 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.245 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.245 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.245 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.246 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.246 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.246 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.246 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.246 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.246 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.246 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.246 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.246 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.246 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.246 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.246 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.247 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.247 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3-2": { 2024-06-28T15:10:33.247 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.247 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.247 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.247 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.247 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.247 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.247 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.247 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.247 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.247 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.247 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.248 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.248 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.248 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.248 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.248 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.248 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.248 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.248 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj 2024-06-28T15:10:33.248 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj 2024-06-28T15:10:33.248 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: _obj ns=ns i= 2024-06-28T15:10:33.248 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj 2024-06-28T15:10:33.248 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj 2024-06-28T15:10:33.248 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: _obj ns=ns i= 2024-06-28T15:10:33.249 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj 2024-06-28T15:10:33.249 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj 2024-06-28T15:10:33.249 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: _obj ns=ns i= 2024-06-28T15:10:33.249 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.249 INFO:tasks.workunit.client.0.smithi178.stdout: "obj1": { 2024-06-28T15:10:33.249 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.249 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.249 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.249 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.249 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.249 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.249 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.249 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.249 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.250 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.250 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.250 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.250 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.250 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.250 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.250 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.250 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.250 INFO:tasks.workunit.client.0.smithi178.stdout: "obj2": { 2024-06-28T15:10:33.250 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.250 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.250 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.250 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.250 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.251 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.251 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.251 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.251 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.251 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.251 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.251 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.251 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.251 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.251 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.251 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.251 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.251 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3": { 2024-06-28T15:10:33.251 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.252 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.252 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.252 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.252 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.252 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.252 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.252 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.252 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.252 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.252 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.252 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.252 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.252 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3-2": { 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.253 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.254 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.254 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.254 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.254 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.254 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.254 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.254 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.254 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj[v1] 2024-06-28T15:10:33.254 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj[v1] 2024-06-28T15:10:33.254 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: _obj ns= i=v1 2024-06-28T15:10:33.254 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj[v1] 2024-06-28T15:10:33.254 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj[v1] 2024-06-28T15:10:33.254 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: _obj ns= i=v1 2024-06-28T15:10:33.254 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj[v1] 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj[v1] 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: _obj ns= i=v1 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout: "obj1": { 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.255 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.256 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.256 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.256 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "v1", 2024-06-28T15:10:33.256 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.256 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.256 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.256 INFO:tasks.workunit.client.0.smithi178.stdout: "obj2": { 2024-06-28T15:10:33.256 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.256 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.256 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.256 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.256 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.256 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "v1", 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3": { 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.257 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.258 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.258 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.258 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.258 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.258 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.258 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.258 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.258 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.258 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.258 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.258 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.258 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.258 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.259 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.259 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3-2": { 2024-06-28T15:10:33.259 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.259 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.259 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.259 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.259 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.259 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.259 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.259 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.259 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.259 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.259 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.259 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.260 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.260 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "v1", 2024-06-28T15:10:33.260 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.260 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.260 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.260 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.260 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj[v1] 2024-06-28T15:10:33.260 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj[v1] 2024-06-28T15:10:33.260 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: _obj ns=ns i=v1 2024-06-28T15:10:33.260 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj[v1] 2024-06-28T15:10:33.260 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj[v1] 2024-06-28T15:10:33.260 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: _obj ns=ns i=v1 2024-06-28T15:10:33.260 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj[v1] 2024-06-28T15:10:33.260 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: _obj[v1] 2024-06-28T15:10:33.261 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: _obj ns=ns i=v1 2024-06-28T15:10:33.261 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.261 INFO:tasks.workunit.client.0.smithi178.stdout: "obj1": { 2024-06-28T15:10:33.261 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.261 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.261 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.261 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.261 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.261 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.261 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.261 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.261 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.261 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.261 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.262 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.262 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.262 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "v1", 2024-06-28T15:10:33.262 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.262 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.262 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.262 INFO:tasks.workunit.client.0.smithi178.stdout: "obj2": { 2024-06-28T15:10:33.262 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.262 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.262 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.262 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.262 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.263 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.263 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.263 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.263 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.263 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.263 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.263 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.263 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.263 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "v1", 2024-06-28T15:10:33.263 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.263 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.263 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.263 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3": { 2024-06-28T15:10:33.263 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.264 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.264 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.264 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.264 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.264 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.264 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.264 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.264 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.264 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.264 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.264 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.264 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.264 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.264 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3-2": { 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.265 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.266 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_obj", 2024-06-28T15:10:33.266 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "v1", 2024-06-28T15:10:33.266 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.266 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.266 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.266 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.266 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestRGWObj.underscore (0 ms) 2024-06-28T15:10:33.266 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestRGWObj.no_underscore 2024-06-28T15:10:33.266 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj 2024-06-28T15:10:33.266 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj 2024-06-28T15:10:33.266 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: obj ns= i= 2024-06-28T15:10:33.266 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj 2024-06-28T15:10:33.267 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj 2024-06-28T15:10:33.267 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: obj ns= i= 2024-06-28T15:10:33.267 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj 2024-06-28T15:10:33.267 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj 2024-06-28T15:10:33.267 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: obj ns= i= 2024-06-28T15:10:33.267 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.267 INFO:tasks.workunit.client.0.smithi178.stdout: "obj1": { 2024-06-28T15:10:33.267 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.267 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.267 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.267 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.267 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.267 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.267 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.268 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.268 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.268 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.268 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.268 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.268 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.268 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.268 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.268 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.268 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.268 INFO:tasks.workunit.client.0.smithi178.stdout: "obj2": { 2024-06-28T15:10:33.268 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.268 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.268 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.269 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.269 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.269 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.269 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.269 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.269 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.269 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.269 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.270 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.270 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.270 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.270 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.270 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.270 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.270 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3": { 2024-06-28T15:10:33.270 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.270 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.270 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.270 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.270 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.270 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.271 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.271 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.271 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.271 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.271 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.271 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.271 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.271 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.271 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.271 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.271 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.271 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3-2": { 2024-06-28T15:10:33.271 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.271 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.272 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.272 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.272 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.272 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.272 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.272 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.272 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.272 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.272 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.272 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.272 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.272 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.272 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.272 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.273 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.273 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.273 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj 2024-06-28T15:10:33.273 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj 2024-06-28T15:10:33.273 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: obj ns=ns i= 2024-06-28T15:10:33.273 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj 2024-06-28T15:10:33.273 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj 2024-06-28T15:10:33.273 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: obj ns=ns i= 2024-06-28T15:10:33.273 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj 2024-06-28T15:10:33.273 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj 2024-06-28T15:10:33.273 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: obj ns=ns i= 2024-06-28T15:10:33.273 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.273 INFO:tasks.workunit.client.0.smithi178.stdout: "obj1": { 2024-06-28T15:10:33.273 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.274 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.274 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.274 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.274 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.274 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.274 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.274 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.274 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.274 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.274 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.274 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.274 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.274 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.275 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.275 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.275 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.275 INFO:tasks.workunit.client.0.smithi178.stdout: "obj2": { 2024-06-28T15:10:33.275 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.275 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.275 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.275 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.275 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.275 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.275 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.275 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.275 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.275 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3": { 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.276 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.277 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.277 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.277 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.277 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.277 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.277 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.277 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.277 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.277 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.277 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.277 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3-2": { 2024-06-28T15:10:33.277 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.277 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.278 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.278 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.278 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.278 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.278 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.278 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.278 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.278 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.278 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.278 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.278 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.278 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.278 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.278 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.279 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.279 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.279 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj[v1] 2024-06-28T15:10:33.279 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj[v1] 2024-06-28T15:10:33.279 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: obj ns= i=v1 2024-06-28T15:10:33.279 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj[v1] 2024-06-28T15:10:33.279 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj[v1] 2024-06-28T15:10:33.279 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: obj ns= i=v1 2024-06-28T15:10:33.279 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj[v1] 2024-06-28T15:10:33.279 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj[v1] 2024-06-28T15:10:33.279 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: obj ns= i=v1 2024-06-28T15:10:33.279 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.279 INFO:tasks.workunit.client.0.smithi178.stdout: "obj1": { 2024-06-28T15:10:33.279 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.280 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.280 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.280 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.280 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.280 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.280 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.280 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.280 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.280 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.280 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.280 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.280 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.280 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "v1", 2024-06-28T15:10:33.280 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.281 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.281 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.281 INFO:tasks.workunit.client.0.smithi178.stdout: "obj2": { 2024-06-28T15:10:33.281 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.281 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.281 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.281 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.281 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.281 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.281 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.281 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.281 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.281 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.282 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.282 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.282 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.282 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "v1", 2024-06-28T15:10:33.282 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.282 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.282 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.282 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3": { 2024-06-28T15:10:33.282 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.282 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.282 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.282 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.282 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.283 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.283 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.283 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.283 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.283 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.283 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.283 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.283 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.283 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.283 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.283 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.283 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.283 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3-2": { 2024-06-28T15:10:33.283 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.284 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.284 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.284 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.284 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.284 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.284 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.284 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.284 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.284 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.284 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.284 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.284 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.284 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "v1", 2024-06-28T15:10:33.284 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.285 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.285 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.285 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.285 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj[v1] 2024-06-28T15:10:33.285 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj[v1] 2024-06-28T15:10:33.285 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: obj ns=ns i=v1 2024-06-28T15:10:33.285 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj[v1] 2024-06-28T15:10:33.285 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj[v1] 2024-06-28T15:10:33.285 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: obj ns=ns i=v1 2024-06-28T15:10:33.285 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj[v1] 2024-06-28T15:10:33.285 INFO:tasks.workunit.client.0.smithi178.stdout:parsed: obj[v1] 2024-06-28T15:10:33.285 INFO:tasks.workunit.client.0.smithi178.stdout:stripped: obj ns=ns i=v1 2024-06-28T15:10:33.285 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.286 INFO:tasks.workunit.client.0.smithi178.stdout: "obj1": { 2024-06-28T15:10:33.286 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.286 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.286 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.286 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.286 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.286 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.286 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.286 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.286 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.286 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.286 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.286 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.286 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.287 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "v1", 2024-06-28T15:10:33.287 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.287 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.287 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.287 INFO:tasks.workunit.client.0.smithi178.stdout: "obj2": { 2024-06-28T15:10:33.287 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.287 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.287 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.287 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.287 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.287 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.287 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.287 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.287 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.288 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.288 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.288 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.288 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.288 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "v1", 2024-06-28T15:10:33.288 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.288 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.288 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.288 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3": { 2024-06-28T15:10:33.288 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.288 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.288 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.288 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.288 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.289 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.289 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.289 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.289 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.289 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.289 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.289 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.289 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.289 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.289 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.289 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.289 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.289 INFO:tasks.workunit.client.0.smithi178.stdout: "obj3-2": { 2024-06-28T15:10:33.289 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.290 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.290 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.290 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.290 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.290 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.290 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.290 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.290 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.290 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.290 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.290 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.290 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "obj", 2024-06-28T15:10:33.290 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "v1", 2024-06-28T15:10:33.290 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.291 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.291 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.291 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.291 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestRGWObj.no_underscore (0 ms) 2024-06-28T15:10:33.291 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestRGWObj.obj_to_raw 2024-06-28T15:10:33.291 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.291 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.291 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.291 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.291 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.291 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.291 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.291 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.292 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.292 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.292 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.292 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.292 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.292 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.292 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.292 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.292 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.292 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.292 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.292 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.292 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.292 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.293 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.293 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.293 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.293 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.293 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.293 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.293 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.293 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.293 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.293 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.293 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.293 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.293 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.294 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": "default-placement.data", 2024-06-28T15:10:33.294 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker_myobj", 2024-06-28T15:10:33.294 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.294 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.294 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.294 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.294 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.294 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.294 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.294 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.294 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.294 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.294 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.295 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.295 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.295 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.295 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.295 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.295 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.295 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.295 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.295 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.295 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.295 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.295 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.295 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.295 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.296 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.296 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.296 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.296 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.296 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.296 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.296 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.296 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.296 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.296 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.296 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.296 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.296 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.296 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.297 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.298 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.298 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.298 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.298 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.298 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.298 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker_myobj", 2024-06-28T15:10:33.298 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.298 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.298 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.298 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.298 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.298 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.298 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.298 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.299 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.299 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.299 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.299 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.299 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.299 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.299 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.299 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.299 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.299 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.299 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.299 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.299 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.299 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.300 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.300 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.300 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.300 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.300 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.300 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.300 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.300 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.300 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.300 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.300 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.300 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.300 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.300 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.301 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.301 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.301 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.301 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.301 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.301 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.301 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.301 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.301 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.301 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.301 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.301 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.301 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.301 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.302 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.302 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.302 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.302 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.302 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.302 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.302 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.302 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.302 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": "default-placement.data", 2024-06-28T15:10:33.302 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns_myobj", 2024-06-28T15:10:33.302 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.302 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.302 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.302 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.303 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.303 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.303 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.303 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.303 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.303 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.303 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.303 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.303 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.303 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.303 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.303 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.303 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.303 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.304 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.304 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.304 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.304 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.304 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.304 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.304 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.304 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.304 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.304 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.304 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.304 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.304 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.304 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.305 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.305 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.305 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.305 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.305 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.305 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.305 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.305 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.305 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.305 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.305 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.305 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.306 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.306 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.306 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.306 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.306 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.306 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.306 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.306 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.306 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.307 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.307 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.307 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.307 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.307 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.307 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.307 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns_myobj", 2024-06-28T15:10:33.307 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.307 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.307 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.307 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.307 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.307 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.307 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.308 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.308 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.308 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.308 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.308 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.308 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.308 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.308 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.308 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.308 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.308 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.308 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.308 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.309 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.310 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.310 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.310 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.310 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.310 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.310 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.310 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.310 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.310 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.310 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.310 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.310 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.310 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.311 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.311 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.311 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.311 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.311 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.311 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.311 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.311 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.311 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.311 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.311 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": "default-placement.data", 2024-06-28T15:10:33.311 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__:inst_myobj", 2024-06-28T15:10:33.311 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.312 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.312 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.312 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.312 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.312 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.312 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.312 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.312 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.312 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.312 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.312 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.312 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.312 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.312 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.313 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.314 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.314 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.314 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.314 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.314 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.314 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.314 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.314 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.314 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.314 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.314 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.314 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.314 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.314 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.315 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.315 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.315 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.315 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.315 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.315 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.315 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.315 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.315 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.315 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.315 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.315 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.315 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.316 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.316 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.316 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__:inst_myobj", 2024-06-28T15:10:33.316 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.316 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.316 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.316 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.316 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.316 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.316 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.316 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.316 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.316 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.316 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.317 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.317 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.317 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.317 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.317 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.317 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.317 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.317 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.317 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.317 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.317 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.317 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.317 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.318 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.318 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.318 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.318 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.318 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.318 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.318 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.318 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.318 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.318 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.318 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.318 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.318 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.318 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.319 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.319 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.319 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.319 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.319 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.319 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.319 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.319 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.319 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.319 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.319 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.319 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.319 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.320 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.320 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.320 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.320 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.320 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.320 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.320 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.320 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": "default-placement.data", 2024-06-28T15:10:33.320 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns:inst_myobj", 2024-06-28T15:10:33.320 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.320 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.320 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.320 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.320 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.321 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.321 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.321 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.321 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.321 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.321 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.321 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.321 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.321 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.321 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.321 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.321 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.321 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.322 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.322 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.322 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.322 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.322 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.322 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.322 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.322 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.322 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.322 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.322 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.322 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.322 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.323 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.323 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.323 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.323 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.323 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.323 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.323 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.323 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.323 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.323 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.323 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.323 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.323 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.323 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.324 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.324 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.324 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.324 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.324 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.324 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.324 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.324 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.324 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.324 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.324 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.324 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.324 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.324 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.325 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns:inst_myobj", 2024-06-28T15:10:33.325 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.325 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.325 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.325 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.325 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.325 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.325 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.325 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.325 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.325 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.325 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.325 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.325 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.326 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.326 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.326 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.326 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.326 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.326 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.326 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.326 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.326 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.326 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.326 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.326 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.327 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.327 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.327 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.327 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.327 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.327 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.327 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.327 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.327 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.327 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.327 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.327 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.327 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.327 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.328 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.328 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.328 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.328 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.328 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.328 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.328 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.328 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.328 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.328 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.328 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.328 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.328 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.329 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.329 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.329 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.329 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.329 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.329 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.329 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": "default-placement.data", 2024-06-28T15:10:33.329 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker___myobj", 2024-06-28T15:10:33.329 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj" 2024-06-28T15:10:33.329 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.329 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.329 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.329 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.330 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.330 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.330 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.330 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.330 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.330 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.330 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.330 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.330 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.330 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.330 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.330 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.330 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.331 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.332 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.332 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.332 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.332 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.332 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.332 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.332 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.332 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.332 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.332 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.332 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.332 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.332 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.332 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.333 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.333 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.333 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.333 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.333 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.333 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.333 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.333 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.333 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.333 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.333 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.333 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.333 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker___myobj", 2024-06-28T15:10:33.333 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj" 2024-06-28T15:10:33.334 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.334 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.334 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.334 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.334 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.334 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.334 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.334 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.334 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.334 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.334 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.334 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.334 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.335 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.335 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.335 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.335 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.335 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.335 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.335 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.335 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.335 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.335 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.335 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.335 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.335 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.335 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.336 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.336 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.336 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.336 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.336 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.336 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.336 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.336 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.336 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.336 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.336 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.336 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.336 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.336 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.337 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.337 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.337 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.337 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.337 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.337 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.337 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.337 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.337 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.337 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.337 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.337 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.337 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.337 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.338 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.338 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.338 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": "default-placement.data", 2024-06-28T15:10:33.338 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns__myobj", 2024-06-28T15:10:33.338 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.338 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.338 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.338 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.338 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.338 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.338 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.338 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.338 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.339 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.339 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.339 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.339 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.339 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.339 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.339 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.339 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.339 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.339 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.339 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.339 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.339 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.339 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.340 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.340 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.340 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.340 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.340 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.340 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.340 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.340 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.340 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.340 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.340 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.340 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.340 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.340 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.341 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.341 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.341 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.341 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.341 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.341 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.341 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.341 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.341 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.341 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.341 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.341 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.341 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.341 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.342 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.342 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.342 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.342 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.342 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.342 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.342 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.342 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.342 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns__myobj", 2024-06-28T15:10:33.342 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.342 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.342 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.342 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.343 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.343 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.343 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.343 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.343 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.343 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.343 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.343 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.343 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.343 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.343 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.343 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.343 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.343 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.344 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.344 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.344 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.344 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.344 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.344 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.344 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.344 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.344 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.344 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.344 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.344 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.344 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.344 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.345 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.345 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.345 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.345 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.345 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.345 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.345 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.345 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.345 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.345 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.345 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.345 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.345 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.345 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.346 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.346 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.346 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.346 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.346 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.346 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.346 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.346 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.346 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.346 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.346 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.346 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.346 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": "default-placement.data", 2024-06-28T15:10:33.347 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__:inst__myobj", 2024-06-28T15:10:33.347 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj" 2024-06-28T15:10:33.347 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.347 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.347 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.347 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.347 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.347 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.347 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.347 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.347 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.347 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.347 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.347 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.348 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.348 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.348 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.348 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.348 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.348 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.348 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.348 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.348 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.348 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.348 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.348 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.348 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.348 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.349 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.349 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.349 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.349 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.349 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.349 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.349 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.349 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.349 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.349 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.349 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.349 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.349 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.349 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.350 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.350 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.350 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.350 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.350 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.350 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.350 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.350 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.350 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.350 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.350 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.350 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.350 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.351 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.351 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.351 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.351 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.351 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.351 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__:inst__myobj", 2024-06-28T15:10:33.351 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj" 2024-06-28T15:10:33.351 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.351 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.351 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.351 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.351 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.351 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.352 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.352 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.352 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.352 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.352 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.352 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.352 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.352 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.352 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.352 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.352 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.352 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.352 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.352 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.353 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.354 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.355 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.355 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.355 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.355 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.355 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.355 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.355 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.355 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": "default-placement.data", 2024-06-28T15:10:33.355 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns:inst__myobj", 2024-06-28T15:10:33.355 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.355 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.355 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.355 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.356 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.357 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.357 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.357 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.357 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.357 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.357 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.357 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.357 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.357 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.357 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.357 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.357 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.357 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.357 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.358 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.358 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.358 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.358 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.358 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.358 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.358 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.358 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.358 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.358 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.358 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.358 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.358 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.359 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.359 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.359 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.359 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.359 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.359 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.359 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.359 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.359 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.359 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.359 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.359 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.359 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.359 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns:inst__myobj", 2024-06-28T15:10:33.360 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.360 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.360 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.360 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.360 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.360 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.360 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.360 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.360 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.360 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.360 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.360 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.360 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.360 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.361 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.361 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.361 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.361 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.361 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.361 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.361 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.361 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.361 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.361 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.361 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.361 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.361 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.361 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.362 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.362 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.362 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.362 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.362 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.362 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.362 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.362 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.362 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.362 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.362 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.362 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.362 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.362 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.363 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.363 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.363 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.363 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.363 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.363 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.363 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.363 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.363 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.363 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.363 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.363 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.363 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.363 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": "default-placement.data", 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker___myobj_", 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj_" 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.364 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.365 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.366 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.366 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.366 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.366 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.366 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.366 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.366 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.366 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.366 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.366 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.366 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.366 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.366 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.367 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.367 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.367 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.367 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.367 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.367 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.367 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.367 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.367 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.367 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.367 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.367 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.367 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.368 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.368 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.368 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.368 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.368 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.368 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.368 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.368 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker___myobj_", 2024-06-28T15:10:33.368 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj_" 2024-06-28T15:10:33.368 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.368 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.368 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.368 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.368 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.369 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.369 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.369 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.369 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.369 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.369 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.369 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.369 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.369 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.370 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.370 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.370 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.370 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.370 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.370 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.370 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.370 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.370 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.370 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.370 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.370 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.370 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.371 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.371 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.371 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.371 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.371 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.371 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.371 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.371 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.371 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.371 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.371 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.371 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.371 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.371 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.372 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.372 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.372 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.372 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.372 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.372 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.372 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.372 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.372 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.372 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.372 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.372 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.372 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.373 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.373 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.373 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.373 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": "default-placement.data", 2024-06-28T15:10:33.373 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns__myobj_", 2024-06-28T15:10:33.373 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.373 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.373 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.373 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.373 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.373 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.373 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.373 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.373 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.374 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.374 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.374 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.374 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.374 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.374 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.374 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.374 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.374 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.374 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.374 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.374 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.374 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.375 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.375 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.375 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.375 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.375 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.375 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.375 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.375 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.375 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.375 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.375 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.375 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.375 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.375 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.376 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.376 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.376 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.376 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.376 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.376 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.376 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.376 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.376 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.376 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.376 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.376 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.376 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.377 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.377 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.377 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.377 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.377 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.377 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.377 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.377 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.377 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.377 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.377 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns__myobj_", 2024-06-28T15:10:33.377 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.377 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.377 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.378 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.378 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.378 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.378 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.378 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.378 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.378 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.378 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.378 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.378 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.378 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.378 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.379 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.380 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.381 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.381 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.381 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.381 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.381 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.381 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.381 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.381 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.381 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.381 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.381 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.381 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.381 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.381 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": "default-placement.data", 2024-06-28T15:10:33.382 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__:inst__myobj_", 2024-06-28T15:10:33.382 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj_" 2024-06-28T15:10:33.382 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.382 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.382 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.382 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.382 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.382 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.382 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.382 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.382 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.382 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.382 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.383 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.383 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.383 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.383 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.383 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.383 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.383 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.383 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.383 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.383 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.383 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.383 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.383 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.383 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.384 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.384 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.384 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.384 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.384 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.384 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.384 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.384 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.384 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.384 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.384 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.384 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.384 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.385 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.385 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.385 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.385 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.385 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.385 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.385 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.385 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.385 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.385 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.385 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.385 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.385 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.386 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.386 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.386 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.386 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.386 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.386 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.386 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.386 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__:inst__myobj_", 2024-06-28T15:10:33.386 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj_" 2024-06-28T15:10:33.386 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.386 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.386 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.386 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.386 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.387 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.387 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.387 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.387 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.387 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.387 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.387 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.387 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.387 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.387 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.387 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.387 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.387 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.387 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.388 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.388 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.388 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.388 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.388 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.388 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.388 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.388 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.388 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.388 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.388 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.388 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.388 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.389 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.390 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.390 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.390 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.390 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.390 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.390 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.390 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.390 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.390 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.390 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.390 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": "default-placement.data", 2024-06-28T15:10:33.390 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns:inst__myobj_", 2024-06-28T15:10:33.390 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.391 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.391 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.391 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.391 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.391 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.391 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "test", 2024-06-28T15:10:33.391 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.391 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.391 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.391 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.391 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": "", 2024-06-28T15:10:33.391 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.391 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": "" 2024-06-28T15:10:33.391 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.392 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.392 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.392 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.392 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.392 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.392 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.392 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.392 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.392 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.392 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.392 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.392 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.392 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.392 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.393 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.393 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.393 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.393 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.393 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.393 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.393 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.393 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.393 INFO:tasks.workunit.client.0.smithi178.stdout: "obj": { 2024-06-28T15:10:33.393 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.393 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.393 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.393 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.393 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.394 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.394 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.394 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.394 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.394 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.394 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.394 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.394 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.394 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.394 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.394 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.394 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.394 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.395 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.395 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.395 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.395 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns:inst__myobj_", 2024-06-28T15:10:33.395 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.395 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.395 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.395 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.395 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.395 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.395 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.395 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.395 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.395 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.396 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.396 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.396 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.396 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.396 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.396 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.396 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.396 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.396 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.396 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.396 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.396 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.396 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.396 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestRGWObj.obj_to_raw (1 ms) 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] TestRGWObj.old_to_raw 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker_myobj", 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.397 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.398 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.398 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.398 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.398 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.398 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.398 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.398 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.398 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.398 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.398 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.398 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.398 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.398 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout:raw=.data-pool:marker_myobj 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj2": { 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker_myobj", 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj2": { 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.399 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout:raw2=.data-pool:marker_myobj 2024-06-28T15:10:33.400 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.401 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.401 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.401 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns_myobj", 2024-06-28T15:10:33.401 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.401 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.401 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.401 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.401 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.401 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.401 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.401 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.401 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.401 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.401 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.402 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.402 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.402 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.402 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.402 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.402 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.402 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.402 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.402 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.402 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.402 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.402 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.402 INFO:tasks.workunit.client.0.smithi178.stdout:raw=.data-pool:marker__ns_myobj 2024-06-28T15:10:33.402 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.403 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj2": { 2024-06-28T15:10:33.403 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.403 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns_myobj", 2024-06-28T15:10:33.403 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.403 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.403 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.403 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.403 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj2": { 2024-06-28T15:10:33.403 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.403 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.403 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.403 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.403 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.404 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.404 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.404 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.404 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.404 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.404 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.404 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.404 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.404 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.404 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.404 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.404 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.404 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.404 INFO:tasks.workunit.client.0.smithi178.stdout:raw2=.data-pool:marker__ns_myobj 2024-06-28T15:10:33.405 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.405 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.405 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.405 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__:inst_myobj", 2024-06-28T15:10:33.405 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.405 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.405 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.405 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.405 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.405 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.405 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.405 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.405 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.406 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.406 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.406 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.406 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.406 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.406 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.406 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.406 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.406 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.406 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.406 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.407 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.407 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.407 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.407 INFO:tasks.workunit.client.0.smithi178.stdout:raw=.data-pool:marker__:inst_myobj 2024-06-28T15:10:33.407 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.407 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj2": { 2024-06-28T15:10:33.407 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.407 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__:inst_myobj", 2024-06-28T15:10:33.407 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.407 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.407 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.407 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.407 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj2": { 2024-06-28T15:10:33.408 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.408 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.408 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.408 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.408 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.408 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.408 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.408 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.408 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.408 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.408 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.408 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.408 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.409 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.409 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.409 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.409 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.409 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.409 INFO:tasks.workunit.client.0.smithi178.stdout:raw2=.data-pool:marker__:inst_myobj 2024-06-28T15:10:33.409 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.409 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.409 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.409 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns:inst_myobj", 2024-06-28T15:10:33.409 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.409 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.409 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.409 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.410 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.410 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.410 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.410 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.410 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.410 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.410 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.410 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.410 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.410 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.410 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.410 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.410 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.410 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.411 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.411 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.411 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.411 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.411 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.411 INFO:tasks.workunit.client.0.smithi178.stdout:raw=.data-pool:marker__ns:inst_myobj 2024-06-28T15:10:33.411 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.411 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj2": { 2024-06-28T15:10:33.411 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.411 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns:inst_myobj", 2024-06-28T15:10:33.411 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.411 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.411 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.411 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.412 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj2": { 2024-06-28T15:10:33.412 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.412 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.412 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.412 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.412 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.412 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.412 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.412 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.412 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.412 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.412 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.412 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.412 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "myobj", 2024-06-28T15:10:33.413 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.413 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.413 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.413 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.413 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.413 INFO:tasks.workunit.client.0.smithi178.stdout:raw2=.data-pool:marker__ns:inst_myobj 2024-06-28T15:10:33.413 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.413 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.413 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.413 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker___myobj", 2024-06-28T15:10:33.413 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj" 2024-06-28T15:10:33.413 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.413 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.414 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.414 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.414 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.414 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.414 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.414 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.414 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.414 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.414 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.414 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.414 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.414 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.414 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.415 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.415 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.415 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.415 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.415 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.415 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.415 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.415 INFO:tasks.workunit.client.0.smithi178.stdout:raw=.data-pool:marker___myobj 2024-06-28T15:10:33.415 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.415 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj2": { 2024-06-28T15:10:33.415 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.415 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker___myobj", 2024-06-28T15:10:33.416 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj" 2024-06-28T15:10:33.416 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.416 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.416 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.416 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj2": { 2024-06-28T15:10:33.416 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.416 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.416 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.416 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.416 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.416 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.416 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.416 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.416 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.417 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.417 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.417 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.417 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.417 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.417 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.417 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.417 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.417 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.417 INFO:tasks.workunit.client.0.smithi178.stdout:raw2=.data-pool:marker___myobj 2024-06-28T15:10:33.417 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.417 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.417 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.418 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns__myobj", 2024-06-28T15:10:33.418 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.418 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.418 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.418 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.418 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.418 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.418 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.418 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.418 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.418 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.418 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.418 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.419 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.419 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.419 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.419 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.419 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.419 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.419 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.419 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.419 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.419 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.419 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.419 INFO:tasks.workunit.client.0.smithi178.stdout:raw=.data-pool:marker__ns__myobj 2024-06-28T15:10:33.419 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.420 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj2": { 2024-06-28T15:10:33.420 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.420 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns__myobj", 2024-06-28T15:10:33.420 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.420 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.420 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.420 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.420 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj2": { 2024-06-28T15:10:33.420 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.420 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.420 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.420 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.420 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.420 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.421 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.421 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.421 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.421 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.421 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.421 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.421 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.421 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.421 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.421 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.421 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.421 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.421 INFO:tasks.workunit.client.0.smithi178.stdout:raw2=.data-pool:marker__ns__myobj 2024-06-28T15:10:33.421 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.422 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.422 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.422 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__:inst__myobj", 2024-06-28T15:10:33.422 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj" 2024-06-28T15:10:33.422 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.422 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.422 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.422 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.422 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.422 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.422 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.422 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.422 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.423 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.423 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.423 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.423 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.423 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.423 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.423 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.423 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.423 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.423 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.423 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.423 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.423 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.423 INFO:tasks.workunit.client.0.smithi178.stdout:raw=.data-pool:marker__:inst__myobj 2024-06-28T15:10:33.424 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.424 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj2": { 2024-06-28T15:10:33.424 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.424 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__:inst__myobj", 2024-06-28T15:10:33.424 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj" 2024-06-28T15:10:33.424 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.424 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.424 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.424 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj2": { 2024-06-28T15:10:33.424 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.424 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.424 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.424 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.425 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.425 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.425 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.425 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.425 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.425 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.425 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.425 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.425 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.425 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.425 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.425 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.425 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.425 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.426 INFO:tasks.workunit.client.0.smithi178.stdout:raw2=.data-pool:marker__:inst__myobj 2024-06-28T15:10:33.426 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.426 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.426 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.426 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns:inst__myobj", 2024-06-28T15:10:33.426 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.426 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.426 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.426 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.426 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.426 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.426 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.426 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.426 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.427 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.427 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.427 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.427 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.427 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.427 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.427 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.427 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.427 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.427 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.427 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.427 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.427 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.427 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.428 INFO:tasks.workunit.client.0.smithi178.stdout:raw=.data-pool:marker__ns:inst__myobj 2024-06-28T15:10:33.428 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.428 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj2": { 2024-06-28T15:10:33.428 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.428 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns:inst__myobj", 2024-06-28T15:10:33.428 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.428 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.428 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.428 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.428 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj2": { 2024-06-28T15:10:33.428 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.428 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.428 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.428 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj", 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.429 INFO:tasks.workunit.client.0.smithi178.stdout:raw2=.data-pool:marker__ns:inst__myobj 2024-06-28T15:10:33.430 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.430 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.430 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.430 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker___myobj_", 2024-06-28T15:10:33.430 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj_" 2024-06-28T15:10:33.430 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.430 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.430 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.430 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.430 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.430 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.430 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.430 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.431 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.431 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.431 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.431 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.431 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.431 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.431 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.431 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.431 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.431 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.431 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.431 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.431 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.432 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.432 INFO:tasks.workunit.client.0.smithi178.stdout:raw=.data-pool:marker___myobj_ 2024-06-28T15:10:33.432 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.432 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj2": { 2024-06-28T15:10:33.432 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.432 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker___myobj_", 2024-06-28T15:10:33.432 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj_" 2024-06-28T15:10:33.432 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.432 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.432 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.432 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj2": { 2024-06-28T15:10:33.432 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.432 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.432 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.433 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.434 INFO:tasks.workunit.client.0.smithi178.stdout:raw2=.data-pool:marker___myobj_ 2024-06-28T15:10:33.434 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.434 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.434 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.434 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns__myobj_", 2024-06-28T15:10:33.434 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.434 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.434 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.434 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.434 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.434 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.434 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.434 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.435 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.435 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.435 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.435 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.435 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.435 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.435 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.435 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.435 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.435 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.435 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.435 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.435 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.435 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout:raw=.data-pool:marker__ns__myobj_ 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj2": { 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns__myobj_", 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj2": { 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.436 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "", 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.437 INFO:tasks.workunit.client.0.smithi178.stdout:raw2=.data-pool:marker__ns__myobj_ 2024-06-28T15:10:33.438 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.438 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.438 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.438 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__:inst__myobj_", 2024-06-28T15:10:33.438 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj_" 2024-06-28T15:10:33.438 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.438 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.438 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.438 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.438 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.438 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.438 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.438 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.439 INFO:tasks.workunit.client.0.smithi178.stdout:raw=.data-pool:marker__:inst__myobj_ 2024-06-28T15:10:33.440 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.440 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj2": { 2024-06-28T15:10:33.440 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.440 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__:inst__myobj_", 2024-06-28T15:10:33.440 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "marker__myobj_" 2024-06-28T15:10:33.440 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.440 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.440 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.440 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj2": { 2024-06-28T15:10:33.440 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.440 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.440 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.440 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.440 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.441 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.441 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.441 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.441 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.441 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.441 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.441 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.441 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.441 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.441 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "" 2024-06-28T15:10:33.441 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.441 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.441 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.441 INFO:tasks.workunit.client.0.smithi178.stdout:raw2=.data-pool:marker__:inst__myobj_ 2024-06-28T15:10:33.442 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.442 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj": { 2024-06-28T15:10:33.442 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.442 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns:inst__myobj_", 2024-06-28T15:10:33.442 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.442 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.442 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.442 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.442 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj": { 2024-06-28T15:10:33.442 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.442 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.442 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.442 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.443 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.443 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.443 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.443 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.443 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.443 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.443 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.443 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.443 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.443 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.443 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.443 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.443 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.443 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.444 INFO:tasks.workunit.client.0.smithi178.stdout:raw=.data-pool:marker__ns:inst__myobj_ 2024-06-28T15:10:33.444 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.444 INFO:tasks.workunit.client.0.smithi178.stdout: "raw_obj2": { 2024-06-28T15:10:33.444 INFO:tasks.workunit.client.0.smithi178.stdout: "pool": ".data-pool", 2024-06-28T15:10:33.444 INFO:tasks.workunit.client.0.smithi178.stdout: "oid": "marker__ns:inst__myobj_", 2024-06-28T15:10:33.444 INFO:tasks.workunit.client.0.smithi178.stdout: "loc": "" 2024-06-28T15:10:33.444 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.444 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.444 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.444 INFO:tasks.workunit.client.0.smithi178.stdout: "new_obj2": { 2024-06-28T15:10:33.444 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket": { 2024-06-28T15:10:33.444 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "ebtest", 2024-06-28T15:10:33.444 INFO:tasks.workunit.client.0.smithi178.stdout: "marker": "marker", 2024-06-28T15:10:33.444 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_id": "bucket-id", 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout: "tenant": "", 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout: "explicit_placement": { 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout: "data_pool": ".data-pool", 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout: "data_extra_pool": "", 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout: "index_pool": ".index-pool" 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout: "key": { 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout: "name": "_myobj_", 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout: "instance": "inst", 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout: "ns": "ns" 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.445 INFO:tasks.workunit.client.0.smithi178.stdout:raw2=.data-pool:marker__ns:inst__myobj_ 2024-06-28T15:10:33.446 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] TestRGWObj.old_to_raw (0 ms) 2024-06-28T15:10:33.446 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 4 tests from TestRGWObj (1 ms total) 2024-06-28T15:10:33.446 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:10:33.446 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:10:33.446 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 4 tests from 1 test suite ran. (1 ms total) 2024-06-28T15:10:33.446 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 4 tests. 2024-06-28T15:10:33.447 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:10:33.447 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:10:33.502 INFO:tasks.workunit:Running workunits matching rgw/test_librgw_file.sh on client.0... 2024-06-28T15:10:33.503 INFO:tasks.workunit:Running workunit rgw/test_librgw_file.sh... 2024-06-28T15:10:33.504 DEBUG:teuthology.orchestra.run.smithi178:workunit test rgw/test_librgw_file.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_librgw_file.sh 2024-06-28T15:10:33.696 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:10:33.696 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:10:33.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.751+0000 7f2dada83980 20 rados->read ofs=0 len=0 2024-06-28T15:10:33.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.755+0000 7f2dada83980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:10:33.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.755+0000 7f2dada83980 20 realm 2024-06-28T15:10:33.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.755+0000 7f2dada83980 20 rados->read ofs=0 len=0 2024-06-28T15:10:33.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.755+0000 7f2dada83980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:10:33.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.755+0000 7f2dada83980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:10:33.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.755+0000 7f2dada83980 20 rados->read ofs=0 len=0 2024-06-28T15:10:33.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.755+0000 7f2dada83980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:10:33.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.755+0000 7f2dada83980 20 rados->read ofs=0 len=0 2024-06-28T15:10:33.743 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.755+0000 7f2dada83980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:10:33.743 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.755+0000 7f2dada83980 20 rados->read ofs=0 len=0 2024-06-28T15:10:33.744 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.759+0000 7f2dada83980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:10:33.744 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.759+0000 7f2dada83980 20 searching for the correct realm 2024-06-28T15:10:33.750 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.763+0000 7f2daad93640 0 -- 172.21.15.178:0/956022874 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f2d9c008040 msgr2=0x7f2d9c03d940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:10:33.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.771+0000 7f2daad93640 0 -- 172.21.15.178:0/956022874 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f2c7407d410 msgr2=0x7f2c7407f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:10:33.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:10:33.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:10:33.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:10:33.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:10:33.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:10:33.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:10:33.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 rados->read ofs=0 len=0 2024-06-28T15:10:33.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:10:33.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 rados->read ofs=0 len=0 2024-06-28T15:10:33.768 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:10:33.768 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 rados->read ofs=0 len=0 2024-06-28T15:10:33.768 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:10:33.769 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 zone default found 2024-06-28T15:10:33.769 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 4 Realm: () 2024-06-28T15:10:33.769 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:10:33.770 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:10:33.770 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:10:33.770 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 zonegroup default 2024-06-28T15:10:33.770 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.779+0000 7f2dada83980 20 rados->read ofs=0 len=0 2024-06-28T15:10:33.770 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.783+0000 7f2dada83980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:10:33.770 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.783+0000 7f2dada83980 20 rados->read ofs=0 len=0 2024-06-28T15:10:33.770 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.783+0000 7f2dada83980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:10:33.771 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.783+0000 7f2dada83980 20 rados->read ofs=0 len=0 2024-06-28T15:10:33.771 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.783+0000 7f2dada83980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:10:33.771 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.783+0000 7f2dada83980 20 started sync module instance, tier type = 2024-06-28T15:10:33.771 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.783+0000 7f2dada83980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:10:33.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.799+0000 7f2dada83980 20 add_watcher() i=0 2024-06-28T15:10:33.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.799+0000 7f2dada83980 20 add_watcher() i=1 2024-06-28T15:10:33.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.799+0000 7f2dada83980 20 add_watcher() i=2 2024-06-28T15:10:33.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.799+0000 7f2dada83980 20 add_watcher() i=3 2024-06-28T15:10:33.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.799+0000 7f2dada83980 20 add_watcher() i=4 2024-06-28T15:10:33.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.799+0000 7f2dada83980 20 add_watcher() i=5 2024-06-28T15:10:33.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.799+0000 7f2dada83980 20 add_watcher() i=6 2024-06-28T15:10:33.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.799+0000 7f2dada83980 20 add_watcher() i=7 2024-06-28T15:10:33.786 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.799+0000 7f2dada83980 2 all 8 watchers are set, enabling cache 2024-06-28T15:10:33.788 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2c61fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:10:33.789 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:10:33.789 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:10:33.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 5 note: GC not initialized 2024-06-28T15:10:33.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2c60fe1640 20 reqs_thread_entry: start 2024-06-28T15:10:33.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 20 init_complete bucket index max shards: 1 2024-06-28T15:10:33.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 20 Filter name: none 2024-06-28T15:10:33.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2bda7fc640 20 reqs_thread_entry: start 2024-06-28T15:10:33.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 10 cache get: name=default.rgw.meta+users.uid+ceph-test-librgw-file : miss 2024-06-28T15:10:33.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 20 rados->read ofs=0 len=0 2024-06-28T15:10:33.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:10:33.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x0 2024-06-28T15:10:33.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 10 adding default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-06-28T15:10:33.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 10 cache get: name=default.rgw.meta+users.email+librgw@example.com : miss 2024-06-28T15:10:33.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 20 rados->read ofs=0 len=0 2024-06-28T15:10:33.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:10:33.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x0 2024-06-28T15:10:33.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 10 adding default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-06-28T15:10:33.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 10 cache get: name=default.rgw.meta+users.keys+VUkX+/sJDEzVGJ8jzj69qgyAMvg= : miss 2024-06-28T15:10:33.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 20 rados->read ofs=0 len=0 2024-06-28T15:10:33.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:10:33.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 10 cache put: name=default.rgw.meta+users.keys+VUkX+/sJDEzVGJ8jzj69qgyAMvg= info.flags=0x0 2024-06-28T15:10:33.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 10 adding default.rgw.meta+users.keys+VUkX+/sJDEzVGJ8jzj69qgyAMvg= to cache LRU end 2024-06-28T15:10:33.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 10 cache get: name=default.rgw.meta+users.keys+VUkX+/sJDEzVGJ8jzj69qgyAMvg= : hit (negative entry) 2024-06-28T15:10:33.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.803+0000 7f2dada83980 10 cache get: name=default.rgw.meta+users.keys+VUkX+/sJDEzVGJ8jzj69qgyAMvg= : hit (negative entry) 2024-06-28T15:10:33.795 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.807+0000 7f2dada83980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2024-06-28T15:10:33.795 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.807+0000 7f2dada83980 10 moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-06-28T15:10:33.795 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.807+0000 7f2dada83980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-librgw-file, ofs0, ns] 2024-06-28T15:10:33.795 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.807+0000 7f2c8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1056561954818 cookie 94248997964624 notifier 5339 bl.length()=697 2024-06-28T15:10:33.796 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.807+0000 7f2c8effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2024-06-28T15:10:33.796 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.807+0000 7f2c8effd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-06-28T15:10:33.800 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.815+0000 7f2dada83980 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2024-06-28T15:10:33.800 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.815+0000 7f2dada83980 10 moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-06-28T15:10:33.800 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.815+0000 7f2dada83980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:librgw@example.com, ofs0, ns] 2024-06-28T15:10:33.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.815+0000 7f2c8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1056561954816 cookie 94248997962352 notifier 5339 bl.length()=194 2024-06-28T15:10:33.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.815+0000 7f2c8effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2024-06-28T15:10:33.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.815+0000 7f2c8effd640 10 rgw watcher librados: moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-06-28T15:10:33.806 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.819+0000 7f2dada83980 10 cache put: name=default.rgw.meta+users.keys+VUkX+/sJDEzVGJ8jzj69qgyAMvg= info.flags=0x7 2024-06-28T15:10:33.806 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.819+0000 7f2dada83980 10 moving default.rgw.meta+users.keys+VUkX+/sJDEzVGJ8jzj69qgyAMvg= to cache LRU end 2024-06-28T15:10:33.806 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.819+0000 7f2dada83980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:VUkX+/sJDEzVGJ8jzj69qgyAMvg=, ofs0, ns] 2024-06-28T15:10:33.807 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.819+0000 7f2c8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1056561954818 cookie 94248997973664 notifier 5339 bl.length()=203 2024-06-28T15:10:33.807 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.819+0000 7f2c8effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+VUkX+/sJDEzVGJ8jzj69qgyAMvg= info.flags=0x7 2024-06-28T15:10:33.807 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.819+0000 7f2c8effd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+VUkX+/sJDEzVGJ8jzj69qgyAMvg= to cache LRU end 2024-06-28T15:10:33.808 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:10:33.808 INFO:tasks.workunit.client.0.smithi178.stdout: "user_id": "ceph-test-librgw-file", 2024-06-28T15:10:33.808 INFO:tasks.workunit.client.0.smithi178.stdout: "display_name": "librgw test user", 2024-06-28T15:10:33.809 INFO:tasks.workunit.client.0.smithi178.stdout: "email": "librgw@example.com", 2024-06-28T15:10:33.809 INFO:tasks.workunit.client.0.smithi178.stdout: "suspended": 0, 2024-06-28T15:10:33.809 INFO:tasks.workunit.client.0.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:10:33.809 INFO:tasks.workunit.client.0.smithi178.stdout: "subusers": [], 2024-06-28T15:10:33.809 INFO:tasks.workunit.client.0.smithi178.stdout: "keys": [ 2024-06-28T15:10:33.809 INFO:tasks.workunit.client.0.smithi178.stdout: { 2024-06-28T15:10:33.809 INFO:tasks.workunit.client.0.smithi178.stdout: "user": "ceph-test-librgw-file", 2024-06-28T15:10:33.809 INFO:tasks.workunit.client.0.smithi178.stdout: "access_key": "VUkX+/sJDEzVGJ8jzj69qgyAMvg=", 2024-06-28T15:10:33.809 INFO:tasks.workunit.client.0.smithi178.stdout: "secret_key": "HrjdIhHcIUtph5/gO+DWY2zs7itHMoH5d8P0noHdZnojoDwXs60pzA==", 2024-06-28T15:10:33.809 INFO:tasks.workunit.client.0.smithi178.stdout: "active": true, 2024-06-28T15:10:33.809 INFO:tasks.workunit.client.0.smithi178.stdout: "create_date": "2024-06-28T15:10:33.808496Z" 2024-06-28T15:10:33.809 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:10:33.809 INFO:tasks.workunit.client.0.smithi178.stdout: ], 2024-06-28T15:10:33.809 INFO:tasks.workunit.client.0.smithi178.stdout: "swift_keys": [], 2024-06-28T15:10:33.810 INFO:tasks.workunit.client.0.smithi178.stdout: "caps": [], 2024-06-28T15:10:33.810 INFO:tasks.workunit.client.0.smithi178.stdout: "op_mask": "read, write, delete", 2024-06-28T15:10:33.810 INFO:tasks.workunit.client.0.smithi178.stdout: "default_placement": "", 2024-06-28T15:10:33.810 INFO:tasks.workunit.client.0.smithi178.stdout: "default_storage_class": "", 2024-06-28T15:10:33.810 INFO:tasks.workunit.client.0.smithi178.stdout: "placement_tags": [], 2024-06-28T15:10:33.810 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_quota": { 2024-06-28T15:10:33.810 INFO:tasks.workunit.client.0.smithi178.stdout: "enabled": false, 2024-06-28T15:10:33.810 INFO:tasks.workunit.client.0.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:10:33.810 INFO:tasks.workunit.client.0.smithi178.stdout: "max_size": -1, 2024-06-28T15:10:33.810 INFO:tasks.workunit.client.0.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:10:33.810 INFO:tasks.workunit.client.0.smithi178.stdout: "max_objects": -1 2024-06-28T15:10:33.810 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.810 INFO:tasks.workunit.client.0.smithi178.stdout: "user_quota": { 2024-06-28T15:10:33.811 INFO:tasks.workunit.client.0.smithi178.stdout: "enabled": false, 2024-06-28T15:10:33.811 INFO:tasks.workunit.client.0.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:10:33.811 INFO:tasks.workunit.client.0.smithi178.stdout: "max_size": -1, 2024-06-28T15:10:33.811 INFO:tasks.workunit.client.0.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:10:33.811 INFO:tasks.workunit.client.0.smithi178.stdout: "max_objects": -1 2024-06-28T15:10:33.811 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:10:33.811 INFO:tasks.workunit.client.0.smithi178.stdout: "temp_url_keys": [], 2024-06-28T15:10:33.811 INFO:tasks.workunit.client.0.smithi178.stdout: "type": "rgw", 2024-06-28T15:10:33.811 INFO:tasks.workunit.client.0.smithi178.stdout: "mfa_ids": [], 2024-06-28T15:10:33.811 INFO:tasks.workunit.client.0.smithi178.stdout: "account_id": "", 2024-06-28T15:10:33.811 INFO:tasks.workunit.client.0.smithi178.stdout: "path": "/", 2024-06-28T15:10:33.811 INFO:tasks.workunit.client.0.smithi178.stdout: "create_date": "2024-06-28T15:10:33.808479Z", 2024-06-28T15:10:33.811 INFO:tasks.workunit.client.0.smithi178.stdout: "tags": [], 2024-06-28T15:10:33.811 INFO:tasks.workunit.client.0.smithi178.stdout: "group_ids": [] 2024-06-28T15:10:33.812 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:10:33.812 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:10:33.812 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.823+0000 7f2c61fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:10:33.814 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.827+0000 7f2dada83980 20 remove_watcher() i=0 2024-06-28T15:10:33.814 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.827+0000 7f2dada83980 2 removed watcher, disabling cache 2024-06-28T15:10:33.816 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.827+0000 7f2dada83980 20 remove_watcher() i=1 2024-06-28T15:10:33.818 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.831+0000 7f2dada83980 20 remove_watcher() i=2 2024-06-28T15:10:33.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.835+0000 7f2dada83980 20 remove_watcher() i=3 2024-06-28T15:10:33.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.835+0000 7f2dada83980 20 remove_watcher() i=4 2024-06-28T15:10:33.823 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.835+0000 7f2dada83980 20 remove_watcher() i=5 2024-06-28T15:10:33.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.839+0000 7f2dada83980 20 remove_watcher() i=6 2024-06-28T15:10:33.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:33.839+0000 7f2dada83980 20 remove_watcher() i=7 2024-06-28T15:10:33.839 INFO:tasks.workunit.client.0.smithi178.stdout:phase 1.1 2024-06-28T15:10:33.856 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 24 tests from 1 test suite. 2024-06-28T15:10:33.856 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:10:33.856 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 24 tests from LibRGW 2024-06-28T15:10:33.857 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.INIT 2024-06-28T15:10:33.868 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:10:33.868 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:10:34.403 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.INIT (547 ms) 2024-06-28T15:10:34.403 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-06-28T15:10:34.406 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT_NOROOT (3 ms) 2024-06-28T15:10:34.406 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT 2024-06-28T15:10:34.406 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-06-28T15:10:34.406 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-06-28T15:10:34.423 INFO:tasks.workunit.client.0.smithi178.stdout:creating: nfsroot:foo/bar/baz/quux 2024-06-28T15:10:34.432 INFO:tasks.workunit.client.0.smithi178.stdout:creating: nfsroot:foo/f1 2024-06-28T15:10:34.438 INFO:tasks.workunit.client.0.smithi178.stdout:creating: nfsroot:foo/f2 2024-06-28T15:10:34.444 INFO:tasks.workunit.client.0.smithi178.stdout:creating: nfsroot:foo/bar/f1 2024-06-28T15:10:34.449 INFO:tasks.workunit.client.0.smithi178.stdout:creating: nfsroot:foo/bar/d1/ 2024-06-28T15:10:34.454 INFO:tasks.workunit.client.0.smithi178.stdout:creating: nfsroot:foo/bar/baz/hungry 2024-06-28T15:10:34.459 INFO:tasks.workunit.client.0.smithi178.stdout:creating: nfsroot:foo/bar/baz/hungry/ 2024-06-28T15:10:34.464 INFO:tasks.workunit.client.0.smithi178.stdout:creating: nfsroot:foo/bar/baz/momma 2024-06-28T15:10:34.469 INFO:tasks.workunit.client.0.smithi178.stdout:creating: nfsroot:foo/bar/baz/bear/ 2024-06-28T15:10:34.475 INFO:tasks.workunit.client.0.smithi178.stdout:creating: nfsroot:foo/bar/baz/sasquatch 2024-06-28T15:10:34.481 INFO:tasks.workunit.client.0.smithi178.stdout:creating: nfsroot:foo/bar/baz/sasquatch/ 2024-06-28T15:10:34.487 INFO:tasks.workunit.client.0.smithi178.stdout:creating: nfsroot:foo/bar/baz/frobozz 2024-06-28T15:10:34.493 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SETUP_HIER1 (87 ms) 2024-06-28T15:10:34.493 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-06-28T15:10:34.736 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SETUP_DIRS1 (242 ms) 2024-06-28T15:10:34.736 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SETATTR 2024-06-28T15:10:34.736 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-06-28T15:10:34.736 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-06-28T15:10:34.777 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (42 ms) 2024-06-28T15:10:34.777 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-06-28T15:10:34.884 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (107 ms) 2024-06-28T15:10:34.884 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-06-28T15:10:34.884 INFO:tasks.workunit.client.0.smithi178.stdout:rename file rfile_0 to rfile_r0 (bucket brename0) 2024-06-28T15:10:34.901 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (17 ms) 2024-06-28T15:10:34.901 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-06-28T15:10:34.901 INFO:tasks.workunit.client.0.smithi178.stdout:rename file rfile_1 (bucket brename0) to rfile_rhilldog (bucket brename1) 2024-06-28T15:10:34.920 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (18 ms) 2024-06-28T15:10:34.920 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-06-28T15:10:34.920 INFO:tasks.workunit.client.0.smithi178.stdout:scanning objects in /bdirs1/dir_0 2024-06-28T15:10:34.920 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-06-28T15:10:34.920 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-06-28T15:10:34.920 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-06-28T15:10:34.920 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-06-28T15:10:34.921 INFO:tasks.workunit.client.0.smithi178.stdout:scanning objects in /bdirs1/dir_1 2024-06-28T15:10:34.921 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-06-28T15:10:34.921 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-06-28T15:10:34.921 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-06-28T15:10:34.921 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-06-28T15:10:34.921 INFO:tasks.workunit.client.0.smithi178.stdout:scanning objects in /bdirs1/dir_2 2024-06-28T15:10:34.921 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-06-28T15:10:34.921 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-06-28T15:10:34.921 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-06-28T15:10:34.922 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-06-28T15:10:34.922 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-06-28T15:10:34.922 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-06-28T15:10:34.922 INFO:tasks.workunit.client.0.smithi178.stdout:read back objects in /bdirs1/dir_0 2024-06-28T15:10:34.922 INFO:tasks.workunit.client.0.smithi178.stdout:reading 0,256 dir_0/sfile_0 2024-06-28T15:10:34.924 INFO:tasks.workunit.client.0.smithi178.stdout: read back from sfile_0 : "data for sfile_0" 2024-06-28T15:10:34.925 INFO:tasks.workunit.client.0.smithi178.stdout:reading 0,256 dir_0/sfile_1 2024-06-28T15:10:34.928 INFO:tasks.workunit.client.0.smithi178.stdout: read back from sfile_1 : "data for sfile_1" 2024-06-28T15:10:34.928 INFO:tasks.workunit.client.0.smithi178.stdout:read back objects in /bdirs1/dir_1 2024-06-28T15:10:34.928 INFO:tasks.workunit.client.0.smithi178.stdout:reading 0,256 dir_1/sfile_0 2024-06-28T15:10:34.934 INFO:tasks.workunit.client.0.smithi178.stdout: read back from sfile_0 : "data for sfile_0" 2024-06-28T15:10:34.934 INFO:tasks.workunit.client.0.smithi178.stdout:reading 0,256 dir_1/sfile_1 2024-06-28T15:10:34.935 INFO:tasks.workunit.client.0.smithi178.stdout: read back from sfile_1 : "data for sfile_1" 2024-06-28T15:10:34.935 INFO:tasks.workunit.client.0.smithi178.stdout:read back objects in /bdirs1/dir_2 2024-06-28T15:10:34.935 INFO:tasks.workunit.client.0.smithi178.stdout:reading 0,256 dir_2/sfile_0 2024-06-28T15:10:34.941 INFO:tasks.workunit.client.0.smithi178.stdout: read back from sfile_0 : "data for sfile_0" 2024-06-28T15:10:34.941 INFO:tasks.workunit.client.0.smithi178.stdout:reading 0,256 dir_2/sfile_1 2024-06-28T15:10:34.944 INFO:tasks.workunit.client.0.smithi178.stdout: read back from sfile_1 : "data for sfile_1" 2024-06-28T15:10:34.945 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.READ_DIRS1 (25 ms) 2024-06-28T15:10:34.945 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-06-28T15:10:35.177 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.READF_SETUP1 (233 ms) 2024-06-28T15:10:35.177 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-06-28T15:10:35.177 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-06-28T15:10:35.177 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-06-28T15:10:35.178 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-06-28T15:10:35.178 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-06-28T15:10:35.178 INFO:tasks.workunit.client.0.smithi178.stdout:release sdir_0 type: DIR refs: 2 2024-06-28T15:10:35.178 INFO:tasks.workunit.client.0.smithi178.stdout:release sfile_0 type: FILE refs: 2 2024-06-28T15:10:35.178 INFO:tasks.workunit.client.0.smithi178.stdout:release sdir_1 type: DIR refs: 2 2024-06-28T15:10:35.178 INFO:tasks.workunit.client.0.smithi178.stdout:release sfile_1 type: FILE refs: 2 2024-06-28T15:10:35.178 INFO:tasks.workunit.client.0.smithi178.stdout:release sdir_0 type: DIR refs: 2 2024-06-28T15:10:35.178 INFO:tasks.workunit.client.0.smithi178.stdout:release sfile_0 type: FILE refs: 2 2024-06-28T15:10:35.178 INFO:tasks.workunit.client.0.smithi178.stdout:release sdir_1 type: DIR refs: 2 2024-06-28T15:10:35.178 INFO:tasks.workunit.client.0.smithi178.stdout:release sfile_1 type: FILE refs: 2 2024-06-28T15:10:35.178 INFO:tasks.workunit.client.0.smithi178.stdout:release sdir_0 type: DIR refs: 2 2024-06-28T15:10:35.178 INFO:tasks.workunit.client.0.smithi178.stdout:release sfile_0 type: FILE refs: 2 2024-06-28T15:10:35.178 INFO:tasks.workunit.client.0.smithi178.stdout:release sdir_1 type: DIR refs: 2 2024-06-28T15:10:35.179 INFO:tasks.workunit.client.0.smithi178.stdout:release sfile_1 type: FILE refs: 2 2024-06-28T15:10:35.179 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-06-28T15:10:35.179 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.HIER1 2024-06-28T15:10:35.192 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/f2 (f2): FILE 2024-06-28T15:10:35.193 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/f1 (f1): FILE 2024-06-28T15:10:35.199 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2024-06-28T15:10:35.201 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2024-06-28T15:10:35.207 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-06-28T15:10:35.208 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-06-28T15:10:35.210 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2024-06-28T15:10:35.211 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2024-06-28T15:10:35.212 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-06-28T15:10:35.214 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-06-28T15:10:35.215 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2024-06-28T15:10:35.218 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2024-06-28T15:10:35.219 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz (baz): DIR 2024-06-28T15:10:35.219 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar (bar): DIR 2024-06-28T15:10:35.219 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo (foo): DIR 2024-06-28T15:10:35.219 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot (nfsroot): DIR 2024-06-28T15:10:35.219 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.HIER1 (41 ms) 2024-06-28T15:10:35.219 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-06-28T15:10:35.219 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-06-28T15:10:35.219 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-06-28T15:10:35.219 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-06-28T15:10:35.219 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-06-28T15:10:35.219 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-06-28T15:10:35.219 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-06-28T15:10:35.219 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-06-28T15:10:35.219 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CLEANUP 2024-06-28T15:10:35.220 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-06-28T15:10:35.220 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.UMOUNT 2024-06-28T15:10:35.220 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-06-28T15:10:35.220 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-06-28T15:10:35.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:35.231+0000 7fd80ec8ca80 -1 shutting down 2024-06-28T15:10:35.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:10:35.235+0000 7fd6e89de640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 55913, si_uid : 1000, si_addr0x3e80000da69, si_status0 2024-06-28T15:11:04.425 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SHUTDOWN (29207 ms) 2024-06-28T15:11:04.426 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 24 tests from LibRGW (30569 ms total) 2024-06-28T15:11:04.426 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:11:04.426 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:11:04.426 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 24 tests from 1 test suite ran. (30569 ms total) 2024-06-28T15:11:04.426 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 24 tests. 2024-06-28T15:11:04.439 INFO:tasks.workunit.client.0.smithi178.stdout:phase 1.2 2024-06-28T15:11:04.461 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 24 tests from 1 test suite. 2024-06-28T15:11:04.461 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:11:04.461 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 24 tests from LibRGW 2024-06-28T15:11:04.461 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.INIT 2024-06-28T15:11:04.471 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:11:04.471 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:11:05.000 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.INIT (539 ms) 2024-06-28T15:11:05.036 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-06-28T15:11:05.037 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT_NOROOT (2 ms) 2024-06-28T15:11:05.037 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT 2024-06-28T15:11:05.037 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-06-28T15:11:05.037 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-06-28T15:11:05.037 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SETUP_HIER1 (0 ms) 2024-06-28T15:11:05.038 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-06-28T15:11:05.038 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SETUP_DIRS1 (0 ms) 2024-06-28T15:11:05.038 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SETATTR 2024-06-28T15:11:05.038 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-06-28T15:11:05.038 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-06-28T15:11:05.038 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2024-06-28T15:11:05.038 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-06-28T15:11:05.038 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2024-06-28T15:11:05.038 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-06-28T15:11:05.038 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2024-06-28T15:11:05.038 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-06-28T15:11:05.038 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2024-06-28T15:11:05.038 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-06-28T15:11:05.039 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-06-28T15:11:05.039 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-06-28T15:11:05.039 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.READ_DIRS1 (0 ms) 2024-06-28T15:11:05.039 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-06-28T15:11:05.039 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2024-06-28T15:11:05.039 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-06-28T15:11:05.039 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-06-28T15:11:05.039 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-06-28T15:11:05.039 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-06-28T15:11:05.039 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-06-28T15:11:05.039 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-06-28T15:11:05.039 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.HIER1 2024-06-28T15:11:05.039 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.HIER1 (0 ms) 2024-06-28T15:11:05.040 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-06-28T15:11:05.040 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-06-28T15:11:05.040 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-06-28T15:11:05.040 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-06-28T15:11:05.040 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-06-28T15:11:05.040 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-06-28T15:11:05.040 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-06-28T15:11:05.040 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-06-28T15:11:05.040 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CLEANUP 2024-06-28T15:11:05.040 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-06-28T15:11:05.040 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.UMOUNT 2024-06-28T15:11:05.040 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-06-28T15:11:05.040 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-06-28T15:11:05.041 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:11:05.014+0000 7f0ac459da80 -1 shutting down 2024-06-28T15:11:05.041 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:11:05.018+0000 7f099e2ef640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 56559, si_uid : 1000, si_addr0x3e80000dcef, si_status0 2024-06-28T15:11:35.161 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SHUTDOWN (30158 ms) 2024-06-28T15:11:35.162 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 24 tests from LibRGW (30700 ms total) 2024-06-28T15:11:35.162 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:11:35.162 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:11:35.162 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 24 tests from 1 test suite ran. (30700 ms total) 2024-06-28T15:11:35.162 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 24 tests. 2024-06-28T15:11:35.176 INFO:tasks.workunit.client.0.smithi178.stdout:phase 1.3 2024-06-28T15:11:35.199 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 24 tests from 1 test suite. 2024-06-28T15:11:35.199 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:11:35.199 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 24 tests from LibRGW 2024-06-28T15:11:35.199 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.INIT 2024-06-28T15:11:35.211 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:11:35.211 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:11:35.514 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.INIT (316 ms) 2024-06-28T15:11:35.514 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-06-28T15:11:35.516 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT_NOROOT (2 ms) 2024-06-28T15:11:35.516 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT 2024-06-28T15:11:35.516 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-06-28T15:11:35.516 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-06-28T15:11:35.526 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SETUP_HIER1 (8 ms) 2024-06-28T15:11:35.526 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-06-28T15:11:35.644 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SETUP_DIRS1 (119 ms) 2024-06-28T15:11:35.644 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SETATTR 2024-06-28T15:11:35.644 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-06-28T15:11:35.645 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-06-28T15:11:35.645 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2024-06-28T15:11:35.645 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-06-28T15:11:35.645 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2024-06-28T15:11:35.645 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-06-28T15:11:35.645 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2024-06-28T15:11:35.645 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-06-28T15:11:35.645 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2024-06-28T15:11:35.645 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-06-28T15:11:35.645 INFO:tasks.workunit.client.0.smithi178.stdout:scanning objects in /bdirs1/dir_0 2024-06-28T15:11:35.645 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-06-28T15:11:35.645 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-06-28T15:11:35.645 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-06-28T15:11:35.645 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-06-28T15:11:35.646 INFO:tasks.workunit.client.0.smithi178.stdout:scanning objects in /bdirs1/dir_1 2024-06-28T15:11:35.646 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-06-28T15:11:35.646 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-06-28T15:11:35.646 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-06-28T15:11:35.646 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-06-28T15:11:35.646 INFO:tasks.workunit.client.0.smithi178.stdout:scanning objects in /bdirs1/dir_2 2024-06-28T15:11:35.646 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-06-28T15:11:35.646 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-06-28T15:11:35.646 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-06-28T15:11:35.647 INFO:tasks.workunit.client.0.smithi178.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-06-28T15:11:35.647 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-06-28T15:11:35.647 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-06-28T15:11:35.648 INFO:tasks.workunit.client.0.smithi178.stdout:read back objects in /bdirs1/dir_0 2024-06-28T15:11:35.648 INFO:tasks.workunit.client.0.smithi178.stdout:reading 0,256 dir_0/sfile_0 2024-06-28T15:11:35.648 INFO:tasks.workunit.client.0.smithi178.stdout: read back from sfile_0 : "data for sfile_0" 2024-06-28T15:11:35.649 INFO:tasks.workunit.client.0.smithi178.stdout:reading 0,256 dir_0/sfile_1 2024-06-28T15:11:35.650 INFO:tasks.workunit.client.0.smithi178.stdout: read back from sfile_1 : "data for sfile_1" 2024-06-28T15:11:35.650 INFO:tasks.workunit.client.0.smithi178.stdout:read back objects in /bdirs1/dir_1 2024-06-28T15:11:35.650 INFO:tasks.workunit.client.0.smithi178.stdout:reading 0,256 dir_1/sfile_0 2024-06-28T15:11:35.653 INFO:tasks.workunit.client.0.smithi178.stdout: read back from sfile_0 : "data for sfile_0" 2024-06-28T15:11:35.653 INFO:tasks.workunit.client.0.smithi178.stdout:reading 0,256 dir_1/sfile_1 2024-06-28T15:11:35.655 INFO:tasks.workunit.client.0.smithi178.stdout: read back from sfile_1 : "data for sfile_1" 2024-06-28T15:11:35.655 INFO:tasks.workunit.client.0.smithi178.stdout:read back objects in /bdirs1/dir_2 2024-06-28T15:11:35.656 INFO:tasks.workunit.client.0.smithi178.stdout:reading 0,256 dir_2/sfile_0 2024-06-28T15:11:35.657 INFO:tasks.workunit.client.0.smithi178.stdout: read back from sfile_0 : "data for sfile_0" 2024-06-28T15:11:35.657 INFO:tasks.workunit.client.0.smithi178.stdout:reading 0,256 dir_2/sfile_1 2024-06-28T15:11:35.661 INFO:tasks.workunit.client.0.smithi178.stdout: read back from sfile_1 : "data for sfile_1" 2024-06-28T15:11:35.661 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.READ_DIRS1 (17 ms) 2024-06-28T15:11:35.661 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-06-28T15:11:35.661 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2024-06-28T15:11:35.661 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-06-28T15:11:35.661 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-06-28T15:11:35.661 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-06-28T15:11:35.661 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-06-28T15:11:35.661 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-06-28T15:11:35.661 INFO:tasks.workunit.client.0.smithi178.stdout:release sdir_0 type: DIR refs: 2 2024-06-28T15:11:35.661 INFO:tasks.workunit.client.0.smithi178.stdout:release sfile_0 type: FILE refs: 2 2024-06-28T15:11:35.662 INFO:tasks.workunit.client.0.smithi178.stdout:release sdir_1 type: DIR refs: 2 2024-06-28T15:11:35.662 INFO:tasks.workunit.client.0.smithi178.stdout:release sfile_1 type: FILE refs: 2 2024-06-28T15:11:35.662 INFO:tasks.workunit.client.0.smithi178.stdout:release sdir_0 type: DIR refs: 2 2024-06-28T15:11:35.662 INFO:tasks.workunit.client.0.smithi178.stdout:release sfile_0 type: FILE refs: 2 2024-06-28T15:11:35.662 INFO:tasks.workunit.client.0.smithi178.stdout:release sdir_1 type: DIR refs: 2 2024-06-28T15:11:35.662 INFO:tasks.workunit.client.0.smithi178.stdout:release sfile_1 type: FILE refs: 2 2024-06-28T15:11:35.662 INFO:tasks.workunit.client.0.smithi178.stdout:release sdir_0 type: DIR refs: 2 2024-06-28T15:11:35.662 INFO:tasks.workunit.client.0.smithi178.stdout:release sfile_0 type: FILE refs: 2 2024-06-28T15:11:35.662 INFO:tasks.workunit.client.0.smithi178.stdout:release sdir_1 type: DIR refs: 2 2024-06-28T15:11:35.662 INFO:tasks.workunit.client.0.smithi178.stdout:release sfile_1 type: FILE refs: 2 2024-06-28T15:11:35.662 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-06-28T15:11:35.662 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.HIER1 2024-06-28T15:11:35.671 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/f2 (f2): FILE 2024-06-28T15:11:35.672 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/f1 (f1): FILE 2024-06-28T15:11:35.676 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2024-06-28T15:11:35.678 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2024-06-28T15:11:35.684 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-06-28T15:11:35.685 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-06-28T15:11:35.686 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2024-06-28T15:11:35.687 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2024-06-28T15:11:35.688 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-06-28T15:11:35.689 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-06-28T15:11:35.690 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2024-06-28T15:11:35.694 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2024-06-28T15:11:35.694 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar/baz (baz): DIR 2024-06-28T15:11:35.694 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo/bar (bar): DIR 2024-06-28T15:11:35.694 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot/foo (foo): DIR 2024-06-28T15:11:35.694 INFO:tasks.workunit.client.0.smithi178.stdout:/nfsroot (nfsroot): DIR 2024-06-28T15:11:35.694 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.HIER1 (33 ms) 2024-06-28T15:11:35.694 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-06-28T15:11:35.694 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-06-28T15:11:35.694 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-06-28T15:11:35.694 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-06-28T15:11:35.694 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-06-28T15:11:35.694 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-06-28T15:11:35.695 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-06-28T15:11:35.695 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-06-28T15:11:35.695 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CLEANUP 2024-06-28T15:11:35.695 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-06-28T15:11:35.695 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.UMOUNT 2024-06-28T15:11:35.695 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-06-28T15:11:35.695 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-06-28T15:11:35.695 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:11:35.706+0000 7f8b6d051a80 -1 shutting down 2024-06-28T15:11:35.696 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:11:35.710+0000 7f8a46da3640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 57206, si_uid : 1000, si_addr0x3e80000df76, si_status0 2024-06-28T15:12:05.509 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SHUTDOWN (29816 ms) 2024-06-28T15:12:05.509 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 24 tests from LibRGW (30311 ms total) 2024-06-28T15:12:05.509 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:12:05.509 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:12:05.510 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 24 tests from 1 test suite ran. (30311 ms total) 2024-06-28T15:12:05.510 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 24 tests. 2024-06-28T15:12:05.518 INFO:tasks.workunit.client.0.smithi178.stdout:phase 2.1 2024-06-28T15:12:05.539 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 9 tests from 1 test suite. 2024-06-28T15:12:05.540 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:12:05.540 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 9 tests from LibRGW 2024-06-28T15:12:05.540 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.INIT 2024-06-28T15:12:05.550 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:12:05.550 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:12:05.855 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.INIT (316 ms) 2024-06-28T15:12:05.855 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT 2024-06-28T15:12:05.857 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-06-28T15:12:05.857 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-06-28T15:12:05.873 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CREATE_BUCKET (16 ms) 2024-06-28T15:12:05.873 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-06-28T15:12:05.873 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-06-28T15:12:05.873 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2024-06-28T15:12:05.888 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave0 2024-06-28T15:12:05.907 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave1 2024-06-28T15:12:05.926 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave2 2024-06-28T15:12:05.949 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave3 2024-06-28T15:12:05.971 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave4 2024-06-28T15:12:05.994 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave5 2024-06-28T15:12:06.018 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave6 2024-06-28T15:12:06.041 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave7 2024-06-28T15:12:06.061 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave8 2024-06-28T15:12:06.081 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave9 2024-06-28T15:12:06.081 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (208 ms) 2024-06-28T15:12:06.081 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2024-06-28T15:12:06.562 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (481 ms) 2024-06-28T15:12:06.563 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CLEANUP 2024-06-28T15:12:06.563 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-06-28T15:12:06.563 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.UMOUNT 2024-06-28T15:12:06.563 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-06-28T15:12:06.563 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-06-28T15:12:06.563 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:12:06.573+0000 7f799dc1fa80 -1 shutting down 2024-06-28T15:12:06.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:12:06.577+0000 7f7877971640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 57851, si_uid : 1000, si_addr0x3e80000e1fb, si_status0 2024-06-28T15:12:35.954 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SHUTDOWN (29392 ms) 2024-06-28T15:12:35.955 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 9 tests from LibRGW (30415 ms total) 2024-06-28T15:12:35.955 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:12:35.955 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:12:35.955 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 9 tests from 1 test suite ran. (30415 ms total) 2024-06-28T15:12:35.955 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 9 tests. 2024-06-28T15:12:35.968 INFO:tasks.workunit.client.0.smithi178.stdout:phase 2.2 2024-06-28T15:12:35.991 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 9 tests from 1 test suite. 2024-06-28T15:12:35.991 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:12:35.991 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 9 tests from LibRGW 2024-06-28T15:12:35.991 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.INIT 2024-06-28T15:12:36.002 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:12:36.002 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:12:36.309 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.INIT (318 ms) 2024-06-28T15:12:36.309 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT 2024-06-28T15:12:36.311 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-06-28T15:12:36.311 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-06-28T15:12:36.311 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-06-28T15:12:36.311 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-06-28T15:12:36.341 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.DELETE_BUCKET (31 ms) 2024-06-28T15:12:36.341 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2024-06-28T15:12:36.357 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave0 2024-06-28T15:12:36.373 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave1 2024-06-28T15:12:36.390 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave2 2024-06-28T15:12:36.406 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave3 2024-06-28T15:12:36.424 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave4 2024-06-28T15:12:36.440 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave5 2024-06-28T15:12:36.460 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave6 2024-06-28T15:12:36.479 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave7 2024-06-28T15:12:36.497 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave8 2024-06-28T15:12:36.520 INFO:tasks.workunit.client.0.smithi178.stdout:created: sorrydave9 2024-06-28T15:12:36.520 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (178 ms) 2024-06-28T15:12:36.520 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2024-06-28T15:12:37.002 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (482 ms) 2024-06-28T15:12:37.002 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CLEANUP 2024-06-28T15:12:37.003 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-06-28T15:12:37.003 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.UMOUNT 2024-06-28T15:12:37.003 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-06-28T15:12:37.003 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-06-28T15:12:37.003 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:12:37.013+0000 7f1124c36a80 -1 shutting down 2024-06-28T15:12:37.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:12:37.017+0000 7f0ffe988640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 58496, si_uid : 1000, si_addr0x3e80000e480, si_status0 2024-06-28T15:13:06.703 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SHUTDOWN (29701 ms) 2024-06-28T15:13:06.703 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 9 tests from LibRGW (30712 ms total) 2024-06-28T15:13:06.703 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:13:06.703 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:13:06.703 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 9 tests from 1 test suite ran. (30712 ms total) 2024-06-28T15:13:06.703 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 9 tests. 2024-06-28T15:13:06.716 INFO:tasks.workunit.client.0.smithi178.stdout:phase 3.1 2024-06-28T15:13:07.050 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 13 tests from 1 test suite. 2024-06-28T15:13:07.050 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:13:07.050 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 13 tests from LibRGW 2024-06-28T15:13:07.050 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.INIT 2024-06-28T15:13:07.060 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:13:07.060 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:13:07.334 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.INIT (283 ms) 2024-06-28T15:13:07.334 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT 2024-06-28T15:13:07.336 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT (3 ms) 2024-06-28T15:13:07.336 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-06-28T15:13:07.350 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CREATE_BUCKET (14 ms) 2024-06-28T15:13:07.350 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-06-28T15:13:07.357 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (7 ms) 2024-06-28T15:13:07.357 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-06-28T15:13:07.360 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (2 ms) 2024-06-28T15:13:07.360 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LARGE1 2024-06-28T15:13:07.432 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LARGE1 (73 ms) 2024-06-28T15:13:07.432 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LARGE2 2024-06-28T15:13:07.515 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LARGE2 (83 ms) 2024-06-28T15:13:07.515 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-06-28T15:13:07.515 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-06-28T15:13:07.515 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-06-28T15:13:07.515 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2024-06-28T15:13:07.515 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-06-28T15:13:07.516 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-06-28T15:13:07.516 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CLEANUP 2024-06-28T15:13:07.516 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-06-28T15:13:07.516 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.UMOUNT 2024-06-28T15:13:07.516 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-06-28T15:13:07.516 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-06-28T15:13:07.516 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:13:07.529+0000 7fbee9c31a80 -1 shutting down 2024-06-28T15:13:07.518 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:13:07.533+0000 7fbdc3983640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 59141, si_uid : 1000, si_addr0x3e80000e705, si_status0 2024-06-28T15:13:37.405 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SHUTDOWN (29889 ms) 2024-06-28T15:13:37.405 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 13 tests from LibRGW (30354 ms total) 2024-06-28T15:13:37.405 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:13:37.405 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:13:37.405 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 13 tests from 1 test suite ran. (30354 ms total) 2024-06-28T15:13:37.405 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 13 tests. 2024-06-28T15:13:37.423 INFO:tasks.workunit.client.0.smithi178.stdout:phase 3.2 2024-06-28T15:13:37.755 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 13 tests from 1 test suite. 2024-06-28T15:13:37.756 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:13:37.756 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 13 tests from LibRGW 2024-06-28T15:13:37.756 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.INIT 2024-06-28T15:13:37.765 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:13:37.765 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:13:38.520 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.INIT (765 ms) 2024-06-28T15:13:38.520 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT 2024-06-28T15:13:38.522 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-06-28T15:13:38.522 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-06-28T15:13:38.522 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-06-28T15:13:38.522 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-06-28T15:13:38.529 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (8 ms) 2024-06-28T15:13:38.529 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-06-28T15:13:38.536 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (6 ms) 2024-06-28T15:13:38.536 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LARGE1 2024-06-28T15:13:38.623 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LARGE1 (87 ms) 2024-06-28T15:13:38.623 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LARGE2 2024-06-28T15:13:38.623 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LARGE2 (0 ms) 2024-06-28T15:13:38.623 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-06-28T15:13:38.624 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-06-28T15:13:38.624 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-06-28T15:13:38.635 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.DELETE_OBJECT (12 ms) 2024-06-28T15:13:38.635 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-06-28T15:13:38.680 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.DELETE_BUCKET (45 ms) 2024-06-28T15:13:38.680 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CLEANUP 2024-06-28T15:13:38.680 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-06-28T15:13:38.680 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.UMOUNT 2024-06-28T15:13:38.680 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-06-28T15:13:38.680 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-06-28T15:13:38.681 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:13:38.696+0000 7f5c171b5a80 -1 shutting down 2024-06-28T15:13:38.683 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:13:38.700+0000 7f5af0f07640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 59787, si_uid : 1000, si_addr0x3e80000e98b, si_status0 2024-06-28T15:14:08.654 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SHUTDOWN (29973 ms) 2024-06-28T15:14:08.654 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 13 tests from LibRGW (30898 ms total) 2024-06-28T15:14:08.654 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:14:08.654 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:14:08.654 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 13 tests from 1 test suite ran. (30898 ms total) 2024-06-28T15:14:08.654 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 13 tests. 2024-06-28T15:14:08.668 INFO:tasks.workunit.client.0.smithi178.stdout:phase 4.1 2024-06-28T15:14:08.690 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 11 tests from 1 test suite. 2024-06-28T15:14:08.690 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:14:08.690 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 11 tests from LibRGW 2024-06-28T15:14:08.690 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.TVAR 2024-06-28T15:14:08.690 INFO:tasks.workunit.client.0.smithi178.stdout:read i1: 64001 2024-06-28T15:14:08.690 INFO:tasks.workunit.client.0.smithi178.stdout:read s1: blunderbuss 2024-06-28T15:14:08.691 INFO:tasks.workunit.client.0.smithi178.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2024-06-28T15:14:08.691 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.INIT 2024-06-28T15:14:08.703 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:14:08.703 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:14:08.990 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.INIT (300 ms) 2024-06-28T15:14:08.990 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT 2024-06-28T15:14:08.992 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT (3 ms) 2024-06-28T15:14:08.992 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-06-28T15:14:09.019 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (27 ms) 2024-06-28T15:14:09.020 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-06-28T15:14:10.173 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_0 2024-06-28T15:14:10.173 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_1 2024-06-28T15:14:10.173 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_2 2024-06-28T15:14:10.173 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_3 2024-06-28T15:14:10.173 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_4 2024-06-28T15:14:10.173 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_5 2024-06-28T15:14:10.173 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_6 2024-06-28T15:14:10.174 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_7 2024-06-28T15:14:10.174 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_8 2024-06-28T15:14:10.174 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_9 2024-06-28T15:14:10.174 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_10 2024-06-28T15:14:10.174 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_11 2024-06-28T15:14:10.174 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_12 2024-06-28T15:14:10.174 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_13 2024-06-28T15:14:10.174 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_14 2024-06-28T15:14:10.174 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_15 2024-06-28T15:14:10.174 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_16 2024-06-28T15:14:10.174 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_17 2024-06-28T15:14:10.174 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_18 2024-06-28T15:14:10.175 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_19 2024-06-28T15:14:10.175 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_20 2024-06-28T15:14:10.175 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_21 2024-06-28T15:14:10.175 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_22 2024-06-28T15:14:10.175 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_23 2024-06-28T15:14:10.175 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_24 2024-06-28T15:14:10.175 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_25 2024-06-28T15:14:10.175 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_26 2024-06-28T15:14:10.175 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_27 2024-06-28T15:14:10.175 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_28 2024-06-28T15:14:10.175 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_29 2024-06-28T15:14:10.175 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_30 2024-06-28T15:14:10.175 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_31 2024-06-28T15:14:10.175 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_32 2024-06-28T15:14:10.176 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_33 2024-06-28T15:14:10.176 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_34 2024-06-28T15:14:10.176 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_35 2024-06-28T15:14:10.176 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_36 2024-06-28T15:14:10.176 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_37 2024-06-28T15:14:10.176 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_38 2024-06-28T15:14:10.176 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_39 2024-06-28T15:14:10.176 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_40 2024-06-28T15:14:10.176 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_41 2024-06-28T15:14:10.176 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_42 2024-06-28T15:14:10.176 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_43 2024-06-28T15:14:10.176 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_44 2024-06-28T15:14:10.176 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_45 2024-06-28T15:14:10.177 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_46 2024-06-28T15:14:10.177 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_47 2024-06-28T15:14:10.177 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_48 2024-06-28T15:14:10.177 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_49 2024-06-28T15:14:10.177 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_50 2024-06-28T15:14:10.177 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_51 2024-06-28T15:14:10.177 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_52 2024-06-28T15:14:10.177 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_53 2024-06-28T15:14:10.177 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_54 2024-06-28T15:14:10.177 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_55 2024-06-28T15:14:10.177 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_56 2024-06-28T15:14:10.177 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_57 2024-06-28T15:14:10.177 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_58 2024-06-28T15:14:10.177 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_59 2024-06-28T15:14:10.178 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_60 2024-06-28T15:14:10.178 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_61 2024-06-28T15:14:10.178 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_62 2024-06-28T15:14:10.178 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_63 2024-06-28T15:14:10.178 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_64 2024-06-28T15:14:10.178 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_65 2024-06-28T15:14:10.178 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_66 2024-06-28T15:14:10.178 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_67 2024-06-28T15:14:10.178 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_68 2024-06-28T15:14:10.178 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_69 2024-06-28T15:14:10.178 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_70 2024-06-28T15:14:10.178 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_71 2024-06-28T15:14:10.179 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_72 2024-06-28T15:14:10.179 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_73 2024-06-28T15:14:10.179 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_74 2024-06-28T15:14:10.179 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_75 2024-06-28T15:14:10.179 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_76 2024-06-28T15:14:10.179 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_77 2024-06-28T15:14:10.179 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_78 2024-06-28T15:14:10.179 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_79 2024-06-28T15:14:10.179 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_80 2024-06-28T15:14:10.179 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_81 2024-06-28T15:14:10.179 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_82 2024-06-28T15:14:10.179 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_83 2024-06-28T15:14:10.179 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_84 2024-06-28T15:14:10.180 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_85 2024-06-28T15:14:10.180 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_86 2024-06-28T15:14:10.180 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_87 2024-06-28T15:14:10.180 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_88 2024-06-28T15:14:10.180 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_89 2024-06-28T15:14:10.180 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_90 2024-06-28T15:14:10.180 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_91 2024-06-28T15:14:10.180 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_92 2024-06-28T15:14:10.180 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_93 2024-06-28T15:14:10.180 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_94 2024-06-28T15:14:10.180 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_95 2024-06-28T15:14:10.180 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_96 2024-06-28T15:14:10.180 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_97 2024-06-28T15:14:10.180 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_98 2024-06-28T15:14:10.181 INFO:tasks.workunit.client.0.smithi178.stdout:created: dmarker:f_99 2024-06-28T15:14:10.181 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (1153 ms) 2024-06-28T15:14:10.181 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-06-28T15:14:10.181 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=. flags=4 2024-06-28T15:14:10.182 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=.. flags=4 2024-06-28T15:14:10.182 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-06-28T15:14:10.182 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-06-28T15:14:10.182 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-06-28T15:14:10.182 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-06-28T15:14:10.182 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-06-28T15:14:10.182 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-06-28T15:14:10.182 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-06-28T15:14:10.182 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-06-28T15:14:10.182 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-06-28T15:14:10.182 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-06-28T15:14:10.182 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-06-28T15:14:10.183 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-06-28T15:14:10.183 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-06-28T15:14:10.183 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-06-28T15:14:10.183 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-06-28T15:14:10.183 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-06-28T15:14:10.183 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-06-28T15:14:10.183 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-06-28T15:14:10.183 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-06-28T15:14:10.183 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-06-28T15:14:10.183 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-06-28T15:14:10.183 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-06-28T15:14:10.183 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-06-28T15:14:10.184 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-06-28T15:14:10.184 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-06-28T15:14:10.184 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-06-28T15:14:10.184 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-06-28T15:14:10.184 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-06-28T15:14:10.184 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-06-28T15:14:10.184 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-06-28T15:14:10.184 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-06-28T15:14:10.184 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-06-28T15:14:10.184 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-06-28T15:14:10.184 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-06-28T15:14:10.184 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-06-28T15:14:10.184 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-06-28T15:14:10.185 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-06-28T15:14:10.185 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-06-28T15:14:10.185 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-06-28T15:14:10.185 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-06-28T15:14:10.185 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-06-28T15:14:10.185 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-06-28T15:14:10.185 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-06-28T15:14:10.185 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-06-28T15:14:10.185 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-06-28T15:14:10.185 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-06-28T15:14:10.185 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-06-28T15:14:10.185 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-06-28T15:14:10.186 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-06-28T15:14:10.186 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-06-28T15:14:10.186 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-06-28T15:14:10.186 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-06-28T15:14:10.186 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-06-28T15:14:10.186 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-06-28T15:14:10.186 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-06-28T15:14:10.186 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-06-28T15:14:10.186 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-06-28T15:14:10.186 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-06-28T15:14:10.186 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-06-28T15:14:10.186 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-06-28T15:14:10.186 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-06-28T15:14:10.187 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-06-28T15:14:10.187 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-06-28T15:14:10.187 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-06-28T15:14:10.187 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-06-28T15:14:10.187 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-06-28T15:14:10.187 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-06-28T15:14:10.187 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-06-28T15:14:10.187 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-06-28T15:14:10.187 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-06-28T15:14:10.187 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-06-28T15:14:10.187 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-06-28T15:14:10.187 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-06-28T15:14:10.187 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-06-28T15:14:10.187 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-06-28T15:14:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-06-28T15:14:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-06-28T15:14:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-06-28T15:14:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-06-28T15:14:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-06-28T15:14:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-06-28T15:14:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-06-28T15:14:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-06-28T15:14:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-06-28T15:14:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-06-28T15:14:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-06-28T15:14:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-06-28T15:14:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-06-28T15:14:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-06-28T15:14:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-06-28T15:14:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-06-28T15:14:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-06-28T15:14:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-06-28T15:14:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-06-28T15:14:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-06-28T15:14:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-06-28T15:14:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-06-28T15:14:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-06-28T15:14:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-06-28T15:14:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-06-28T15:14:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_READDIR (6 ms) 2024-06-28T15:14:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2024-06-28T15:14:10.190 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-06-28T15:14:10.190 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-06-28T15:14:10.190 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-06-28T15:14:10.191 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-06-28T15:14:10.191 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-06-28T15:14:10.191 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-06-28T15:14:10.191 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-06-28T15:14:10.191 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-06-28T15:14:10.191 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-06-28T15:14:10.191 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-06-28T15:14:10.191 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-06-28T15:14:10.191 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-06-28T15:14:10.191 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-06-28T15:14:10.191 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-06-28T15:14:10.191 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-06-28T15:14:10.191 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-06-28T15:14:10.192 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-06-28T15:14:10.192 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-06-28T15:14:10.192 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-06-28T15:14:10.192 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-06-28T15:14:10.192 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-06-28T15:14:10.192 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-06-28T15:14:10.192 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-06-28T15:14:10.192 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-06-28T15:14:10.192 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-06-28T15:14:10.192 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-06-28T15:14:10.192 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-06-28T15:14:10.192 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-06-28T15:14:10.192 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-06-28T15:14:10.193 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-06-28T15:14:10.193 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-06-28T15:14:10.193 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-06-28T15:14:10.193 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-06-28T15:14:10.193 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-06-28T15:14:10.193 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-06-28T15:14:10.193 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-06-28T15:14:10.193 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-06-28T15:14:10.193 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-06-28T15:14:10.193 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-06-28T15:14:10.193 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-06-28T15:14:10.193 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-06-28T15:14:10.193 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-06-28T15:14:10.193 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-06-28T15:14:10.194 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-06-28T15:14:10.194 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-06-28T15:14:10.194 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-06-28T15:14:10.194 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-06-28T15:14:10.194 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-06-28T15:14:10.194 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-06-28T15:14:10.194 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-06-28T15:14:10.194 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-06-28T15:14:10.194 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-06-28T15:14:10.194 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-06-28T15:14:10.194 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-06-28T15:14:10.194 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-06-28T15:14:10.194 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-06-28T15:14:10.195 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-06-28T15:14:10.195 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-06-28T15:14:10.195 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-06-28T15:14:10.195 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-06-28T15:14:10.195 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-06-28T15:14:10.195 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-06-28T15:14:10.195 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-06-28T15:14:10.195 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-06-28T15:14:10.195 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-06-28T15:14:10.195 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-06-28T15:14:10.195 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-06-28T15:14:10.195 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-06-28T15:14:10.195 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-06-28T15:14:10.195 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-06-28T15:14:10.196 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-06-28T15:14:10.196 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-06-28T15:14:10.196 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-06-28T15:14:10.196 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-06-28T15:14:10.196 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-06-28T15:14:10.196 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-06-28T15:14:10.196 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-06-28T15:14:10.196 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-06-28T15:14:10.196 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-06-28T15:14:10.196 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-06-28T15:14:10.196 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-06-28T15:14:10.196 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-06-28T15:14:10.196 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-06-28T15:14:10.196 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-06-28T15:14:10.197 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-06-28T15:14:10.197 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-06-28T15:14:10.197 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-06-28T15:14:10.197 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-06-28T15:14:10.197 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-06-28T15:14:10.197 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-06-28T15:14:10.197 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-06-28T15:14:10.197 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-06-28T15:14:10.197 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-06-28T15:14:10.197 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-06-28T15:14:10.197 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-06-28T15:14:10.197 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-06-28T15:14:10.197 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-06-28T15:14:10.197 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-06-28T15:14:10.198 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-06-28T15:14:10.198 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-06-28T15:14:10.198 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER2_READDIR (4 ms) 2024-06-28T15:14:10.198 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-06-28T15:14:10.198 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-06-28T15:14:10.198 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CLEANUP 2024-06-28T15:14:10.198 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-06-28T15:14:10.198 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.UMOUNT 2024-06-28T15:14:10.198 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.UMOUNT (1 ms) 2024-06-28T15:14:10.198 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-06-28T15:14:10.199 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:14:10.200+0000 7ff945edba80 -1 shutting down 2024-06-28T15:14:10.199 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:14:10.204+0000 7ff81fc2d640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 60432, si_uid : 1000, si_addr0x3e80000ec10, si_status0 2024-06-28T15:14:39.408 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SHUTDOWN (29224 ms) 2024-06-28T15:14:39.408 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 11 tests from LibRGW (30718 ms total) 2024-06-28T15:14:39.409 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:14:39.409 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:14:39.409 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 11 tests from 1 test suite ran. (30718 ms total) 2024-06-28T15:14:39.409 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 11 tests. 2024-06-28T15:14:39.425 INFO:tasks.workunit.client.0.smithi178.stdout:phase 4.2 2024-06-28T15:14:39.447 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 11 tests from 1 test suite. 2024-06-28T15:14:39.447 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:14:39.447 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 11 tests from LibRGW 2024-06-28T15:14:39.448 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.TVAR 2024-06-28T15:14:39.448 INFO:tasks.workunit.client.0.smithi178.stdout:read i1: 64001 2024-06-28T15:14:39.448 INFO:tasks.workunit.client.0.smithi178.stdout:read s1: blunderbuss 2024-06-28T15:14:39.448 INFO:tasks.workunit.client.0.smithi178.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2024-06-28T15:14:39.448 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.INIT 2024-06-28T15:14:39.457 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:14:39.457 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:14:39.734 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.INIT (286 ms) 2024-06-28T15:14:39.734 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT 2024-06-28T15:14:39.735 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-06-28T15:14:39.735 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-06-28T15:14:39.752 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (16 ms) 2024-06-28T15:14:39.752 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-06-28T15:14:39.752 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-06-28T15:14:39.752 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-06-28T15:14:39.752 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-06-28T15:14:39.752 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2024-06-28T15:14:39.755 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-06-28T15:14:39.755 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-06-28T15:14:39.755 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-06-28T15:14:39.755 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-06-28T15:14:39.755 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-06-28T15:14:39.756 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-06-28T15:14:39.756 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-06-28T15:14:39.756 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-06-28T15:14:39.756 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-06-28T15:14:39.756 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-06-28T15:14:39.756 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-06-28T15:14:39.756 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-06-28T15:14:39.756 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-06-28T15:14:39.756 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-06-28T15:14:39.756 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-06-28T15:14:39.756 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-06-28T15:14:39.756 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-06-28T15:14:39.756 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-06-28T15:14:39.757 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-06-28T15:14:39.757 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-06-28T15:14:39.757 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-06-28T15:14:39.757 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-06-28T15:14:39.757 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-06-28T15:14:39.757 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-06-28T15:14:39.757 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-06-28T15:14:39.757 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-06-28T15:14:39.757 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-06-28T15:14:39.757 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-06-28T15:14:39.757 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-06-28T15:14:39.757 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-06-28T15:14:39.757 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-06-28T15:14:39.758 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-06-28T15:14:39.758 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-06-28T15:14:39.758 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-06-28T15:14:39.758 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-06-28T15:14:39.758 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-06-28T15:14:39.758 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-06-28T15:14:39.758 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-06-28T15:14:39.758 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-06-28T15:14:39.758 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-06-28T15:14:39.758 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-06-28T15:14:39.758 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-06-28T15:14:39.758 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-06-28T15:14:39.758 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-06-28T15:14:39.759 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-06-28T15:14:39.759 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-06-28T15:14:39.759 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-06-28T15:14:39.759 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-06-28T15:14:39.759 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-06-28T15:14:39.759 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-06-28T15:14:39.759 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-06-28T15:14:39.759 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-06-28T15:14:39.759 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-06-28T15:14:39.759 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-06-28T15:14:39.759 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-06-28T15:14:39.759 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-06-28T15:14:39.759 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-06-28T15:14:39.760 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-06-28T15:14:39.760 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-06-28T15:14:39.760 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-06-28T15:14:39.760 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-06-28T15:14:39.760 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-06-28T15:14:39.760 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-06-28T15:14:39.760 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-06-28T15:14:39.760 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-06-28T15:14:39.760 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-06-28T15:14:39.760 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-06-28T15:14:39.760 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-06-28T15:14:39.760 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-06-28T15:14:39.760 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-06-28T15:14:39.761 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-06-28T15:14:39.761 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-06-28T15:14:39.761 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-06-28T15:14:39.761 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-06-28T15:14:39.761 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-06-28T15:14:39.761 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-06-28T15:14:39.761 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-06-28T15:14:39.761 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-06-28T15:14:39.761 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-06-28T15:14:39.761 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-06-28T15:14:39.761 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-06-28T15:14:39.761 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-06-28T15:14:39.761 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-06-28T15:14:39.762 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-06-28T15:14:39.762 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-06-28T15:14:39.762 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-06-28T15:14:39.762 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-06-28T15:14:39.762 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-06-28T15:14:39.762 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-06-28T15:14:39.762 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-06-28T15:14:39.762 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-06-28T15:14:39.762 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-06-28T15:14:39.762 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-06-28T15:14:39.762 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-06-28T15:14:39.762 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-06-28T15:14:39.762 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-06-28T15:14:39.763 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-06-28T15:14:39.763 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-06-28T15:14:39.763 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-06-28T15:14:39.763 INFO:tasks.workunit.client.0.smithi178.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-06-28T15:14:39.763 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER2_READDIR (4 ms) 2024-06-28T15:14:39.763 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-06-28T15:14:39.763 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-06-28T15:14:39.763 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CLEANUP 2024-06-28T15:14:39.763 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-06-28T15:14:39.763 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.UMOUNT 2024-06-28T15:14:39.763 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-06-28T15:14:39.763 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-06-28T15:14:39.764 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:14:39.772+0000 7fe4d3e0aa80 -1 shutting down 2024-06-28T15:14:39.764 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:14:39.776+0000 7fe3adb5c640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 61077, si_uid : 1000, si_addr0x3e80000ee95, si_status0 2024-06-28T15:15:09.817 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SHUTDOWN (30061 ms) 2024-06-28T15:15:09.818 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 11 tests from LibRGW (30369 ms total) 2024-06-28T15:15:09.818 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:15:09.818 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:15:09.818 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 11 tests from 1 test suite ran. (30369 ms total) 2024-06-28T15:15:09.818 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 11 tests. 2024-06-28T15:15:09.830 INFO:tasks.workunit.client.0.smithi178.stdout:phase 5.1 2024-06-28T15:15:09.864 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 19 tests from 1 test suite. 2024-06-28T15:15:09.864 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:15:09.864 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 19 tests from LibRGW 2024-06-28T15:15:09.865 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.INIT 2024-06-28T15:15:09.875 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:15:09.875 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:15:10.146 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.INIT (282 ms) 2024-06-28T15:15:10.147 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT 2024-06-28T15:15:10.148 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-06-28T15:15:10.149 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-06-28T15:15:10.164 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CREATE_BUCKET (16 ms) 2024-06-28T15:15:10.164 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-06-28T15:15:10.169 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (5 ms) 2024-06-28T15:15:10.169 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2024-06-28T15:15:10.169 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2024-06-28T15:15:10.169 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-06-28T15:15:10.171 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (2 ms) 2024-06-28T15:15:10.171 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.OBJ_OPEN 2024-06-28T15:15:10.171 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2024-06-28T15:15:10.172 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.PUT_OBJECT 2024-06-28T15:15:10.182 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.PUT_OBJECT (11 ms) 2024-06-28T15:15:10.183 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.GET_OBJECT 2024-06-28T15:15:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.GET_OBJECT (6 ms) 2024-06-28T15:15:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-06-28T15:15:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-06-28T15:15:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2024-06-28T15:15:10.188 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.WRITEV 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.READV 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.READV (0 ms) 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CLEANUP 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.UMOUNT 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-06-28T15:15:10.189 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-06-28T15:15:10.190 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:15:10.203+0000 7eff981aca80 -1 shutting down 2024-06-28T15:15:10.191 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:15:10.207+0000 7efe71efe640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 61725, si_uid : 1000, si_addr0x3e80000f11d, si_status0 2024-06-28T15:15:40.334 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SHUTDOWN (30145 ms) 2024-06-28T15:15:40.334 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 19 tests from LibRGW (30469 ms total) 2024-06-28T15:15:40.334 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:15:40.334 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:15:40.334 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 19 tests from 1 test suite ran. (30469 ms total) 2024-06-28T15:15:40.334 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 19 tests. 2024-06-28T15:15:40.346 INFO:tasks.workunit.client.0.smithi178.stdout:phase 5.2 2024-06-28T15:15:40.377 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 19 tests from 1 test suite. 2024-06-28T15:15:40.377 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:15:40.377 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 19 tests from LibRGW 2024-06-28T15:15:40.377 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.INIT 2024-06-28T15:15:40.385 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:15:40.385 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:15:40.930 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.INIT (552 ms) 2024-06-28T15:15:40.930 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT 2024-06-28T15:15:40.932 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT (3 ms) 2024-06-28T15:15:40.932 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-06-28T15:15:40.932 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-06-28T15:15:40.932 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-06-28T15:15:40.942 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (10 ms) 2024-06-28T15:15:40.942 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2024-06-28T15:15:40.943 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2024-06-28T15:15:40.943 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-06-28T15:15:40.943 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (0 ms) 2024-06-28T15:15:40.943 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.OBJ_OPEN 2024-06-28T15:15:40.943 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2024-06-28T15:15:40.943 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.PUT_OBJECT 2024-06-28T15:15:40.943 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.PUT_OBJECT (0 ms) 2024-06-28T15:15:40.943 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.GET_OBJECT 2024-06-28T15:15:40.943 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.GET_OBJECT (0 ms) 2024-06-28T15:15:40.943 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-06-28T15:15:40.943 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-06-28T15:15:40.943 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2024-06-28T15:15:40.944 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2024-06-28T15:15:40.944 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.WRITEV 2024-06-28T15:15:40.944 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2024-06-28T15:15:40.944 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.READV 2024-06-28T15:15:40.944 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.READV (0 ms) 2024-06-28T15:15:40.944 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2024-06-28T15:15:40.944 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2024-06-28T15:15:40.945 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-06-28T15:15:40.953 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.DELETE_OBJECT (11 ms) 2024-06-28T15:15:40.953 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-06-28T15:15:40.981 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.DELETE_BUCKET (28 ms) 2024-06-28T15:15:40.981 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CLEANUP 2024-06-28T15:15:40.981 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-06-28T15:15:40.981 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.UMOUNT 2024-06-28T15:15:40.981 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-06-28T15:15:40.981 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-06-28T15:15:40.982 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:15:40.999+0000 7f0540a6aa80 -1 shutting down 2024-06-28T15:15:40.984 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:15:40.999+0000 7f041a7bc640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 62370, si_uid : 1000, si_addr0x3e80000f3a2, si_status0 2024-06-28T15:16:11.235 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SHUTDOWN (30253 ms) 2024-06-28T15:16:11.235 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 19 tests from LibRGW (30857 ms total) 2024-06-28T15:16:11.235 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:16:11.235 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:16:11.235 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 19 tests from 1 test suite ran. (30857 ms total) 2024-06-28T15:16:11.235 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 19 tests. 2024-06-28T15:16:11.248 INFO:tasks.workunit.client.0.smithi178.stdout:phase 6.1 2024-06-28T15:16:11.270 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] Running 10 tests from 1 test suite. 2024-06-28T15:16:11.270 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment set-up. 2024-06-28T15:16:11.270 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 10 tests from LibRGW 2024-06-28T15:16:11.270 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.INIT 2024-06-28T15:16:11.279 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:16:11.279 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:16:11.564 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.INIT (294 ms) 2024-06-28T15:16:11.564 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.MOUNT 2024-06-28T15:16:11.565 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-06-28T15:16:11.566 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CREATE_BUCKETS 2024-06-28T15:16:11.600 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CREATE_BUCKETS (34 ms) 2024-06-28T15:16:11.600 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.LOOKUP_BUCKETS 2024-06-28T15:16:11.612 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.LOOKUP_BUCKETS (12 ms) 2024-06-28T15:16:11.612 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.TOPDIR_RENAME 2024-06-28T15:16:11.652 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.TOPDIR_RENAME (40 ms) 2024-06-28T15:16:11.653 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SUBDIR_RENAME 2024-06-28T15:16:11.705 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SUBDIR_RENAME (53 ms) 2024-06-28T15:16:11.705 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CROSS_BUCKET_RENAME 2024-06-28T15:16:11.757 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CROSS_BUCKET_RENAME (52 ms) 2024-06-28T15:16:11.757 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.CLEANUP 2024-06-28T15:16:11.757 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-06-28T15:16:11.757 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.UMOUNT 2024-06-28T15:16:11.757 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-06-28T15:16:11.757 INFO:tasks.workunit.client.0.smithi178.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-06-28T15:16:11.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:11.774+0000 7fe7204a7a80 -1 shutting down 2024-06-28T15:16:11.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:11.778+0000 7fe5fa1f9640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 63015, si_uid : 1000, si_addr0x3e80000f627, si_status0 2024-06-28T15:16:41.700 INFO:tasks.workunit.client.0.smithi178.stdout:[ OK ] LibRGW.SHUTDOWN (29943 ms) 2024-06-28T15:16:41.701 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] 10 tests from LibRGW (30430 ms total) 2024-06-28T15:16:41.701 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:16:41.701 INFO:tasks.workunit.client.0.smithi178.stdout:[----------] Global test environment tear-down 2024-06-28T15:16:41.701 INFO:tasks.workunit.client.0.smithi178.stdout:[==========] 10 tests from 1 test suite ran. (30430 ms total) 2024-06-28T15:16:41.701 INFO:tasks.workunit.client.0.smithi178.stdout:[ PASSED ] 10 tests. 2024-06-28T15:16:41.715 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:16:41.715 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:16:41.731 INFO:tasks.workunit:Running workunits matching rgw/test_awssdkv4_sig.sh on client.0... 2024-06-28T15:16:41.732 INFO:tasks.workunit:Running workunit rgw/test_awssdkv4_sig.sh... 2024-06-28T15:16:41.733 DEBUG:teuthology.orchestra.run.smithi178:workunit test rgw/test_awssdkv4_sig.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_awssdkv4_sig.sh 2024-06-28T15:16:41.781 INFO:tasks.workunit.client.0.smithi178.stderr:+ [ -z ] 2024-06-28T15:16:41.781 INFO:tasks.workunit.client.0.smithi178.stderr:+ export AWS_ACCESS_KEY_ID=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-06-28T15:16:41.781 INFO:tasks.workunit.client.0.smithi178.stderr:+ export AWS_SECRET_ACCESS_KEY=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-06-28T15:16:41.782 INFO:tasks.workunit.client.0.smithi178.stderr:+ radosgw-admin user create --uid ceph-test-maven --access-key lNCnR47C2g+ZidCWBAUuwfSAA7Q= --secret tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== --display-name maven test user --email sigv4@example.com 2024-06-28T15:16:41.822 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:16:41.822 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:16:41.869 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.885+0000 7fc91e9e9980 20 rados->read ofs=0 len=0 2024-06-28T15:16:41.870 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.885+0000 7fc91e9e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:16:41.870 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.885+0000 7fc91e9e9980 20 realm 2024-06-28T15:16:41.870 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.885+0000 7fc91e9e9980 20 rados->read ofs=0 len=0 2024-06-28T15:16:41.870 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.885+0000 7fc91e9e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:16:41.870 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.885+0000 7fc91e9e9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:16:41.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.885+0000 7fc91e9e9980 20 rados->read ofs=0 len=0 2024-06-28T15:16:41.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.885+0000 7fc91e9e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:16:41.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.885+0000 7fc91e9e9980 20 rados->read ofs=0 len=0 2024-06-28T15:16:41.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.885+0000 7fc91e9e9980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:16:41.873 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.885+0000 7fc91e9e9980 20 rados->read ofs=0 len=0 2024-06-28T15:16:41.873 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.889+0000 7fc91e9e9980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:16:41.873 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.889+0000 7fc91e9e9980 20 searching for the correct realm 2024-06-28T15:16:41.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.893+0000 7fc91bcf9640 0 -- 172.21.15.178:0/1866346183 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fc90c0759f0 msgr2=0x7fc90c059b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:16:41.886 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.901+0000 7fc91ccfb640 0 -- 172.21.15.178:0/1866346183 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fc904050b80 msgr2=0x7fc904049960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:16:41.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.905+0000 7fc91e9e9980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:16:41.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.905+0000 7fc91e9e9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:16:41.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.905+0000 7fc91e9e9980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:16:41.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.905+0000 7fc91e9e9980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:16:41.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.905+0000 7fc91e9e9980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:16:41.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.905+0000 7fc91e9e9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:16:41.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.905+0000 7fc91e9e9980 20 rados->read ofs=0 len=0 2024-06-28T15:16:41.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.905+0000 7fc91e9e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:16:41.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.905+0000 7fc91e9e9980 20 rados->read ofs=0 len=0 2024-06-28T15:16:41.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:16:41.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 20 rados->read ofs=0 len=0 2024-06-28T15:16:41.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:16:41.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 20 zone default found 2024-06-28T15:16:41.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 4 Realm: () 2024-06-28T15:16:41.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:16:41.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:16:41.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:16:41.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 20 zonegroup default 2024-06-28T15:16:41.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 20 rados->read ofs=0 len=0 2024-06-28T15:16:41.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:16:41.895 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 20 rados->read ofs=0 len=0 2024-06-28T15:16:41.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:16:41.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 20 rados->read ofs=0 len=0 2024-06-28T15:16:41.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:16:41.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 20 started sync module instance, tier type = 2024-06-28T15:16:41.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.909+0000 7fc91e9e9980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:16:41.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.925+0000 7fc91e9e9980 20 add_watcher() i=0 2024-06-28T15:16:41.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.925+0000 7fc91e9e9980 20 add_watcher() i=1 2024-06-28T15:16:41.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.925+0000 7fc91e9e9980 20 add_watcher() i=2 2024-06-28T15:16:41.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.925+0000 7fc91e9e9980 20 add_watcher() i=3 2024-06-28T15:16:41.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.925+0000 7fc91e9e9980 20 add_watcher() i=4 2024-06-28T15:16:41.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.925+0000 7fc91e9e9980 20 add_watcher() i=5 2024-06-28T15:16:41.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.925+0000 7fc91e9e9980 20 add_watcher() i=6 2024-06-28T15:16:41.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.925+0000 7fc91e9e9980 20 add_watcher() i=7 2024-06-28T15:16:41.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.925+0000 7fc91e9e9980 2 all 8 watchers are set, enabling cache 2024-06-28T15:16:41.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc7d2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:16:41.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:16:41.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:16:41.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 5 note: GC not initialized 2024-06-28T15:16:41.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc7d1fdb640 20 reqs_thread_entry: start 2024-06-28T15:16:41.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 20 init_complete bucket index max shards: 1 2024-06-28T15:16:41.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 20 Filter name: none 2024-06-28T15:16:41.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc7437fe640 20 reqs_thread_entry: start 2024-06-28T15:16:41.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 10 cache get: name=default.rgw.meta+users.uid+ceph-test-maven : miss 2024-06-28T15:16:41.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 20 rados->read ofs=0 len=0 2024-06-28T15:16:41.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:16:41.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x0 2024-06-28T15:16:41.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 10 adding default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-06-28T15:16:41.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 10 cache get: name=default.rgw.meta+users.email+sigv4@example.com : miss 2024-06-28T15:16:41.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 20 rados->read ofs=0 len=0 2024-06-28T15:16:41.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:16:41.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x0 2024-06-28T15:16:41.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 10 adding default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-06-28T15:16:41.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : miss 2024-06-28T15:16:41.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.929+0000 7fc91e9e9980 20 rados->read ofs=0 len=0 2024-06-28T15:16:41.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.933+0000 7fc91e9e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:16:41.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.933+0000 7fc91e9e9980 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x0 2024-06-28T15:16:41.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.933+0000 7fc91e9e9980 10 adding default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-06-28T15:16:41.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.933+0000 7fc91e9e9980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2024-06-28T15:16:41.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.933+0000 7fc91e9e9980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2024-06-28T15:16:41.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.933+0000 7fc91e9e9980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2024-06-28T15:16:41.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.933+0000 7fc91e9e9980 10 moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-06-28T15:16:41.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.933+0000 7fc91e9e9980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-maven, ofs0, ns] 2024-06-28T15:16:41.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.933+0000 7fc7f37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1056561954818 cookie 94181083083872 notifier 5558 bl.length()=677 2024-06-28T15:16:41.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.933+0000 7fc7f37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2024-06-28T15:16:41.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.933+0000 7fc7f37fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-06-28T15:16:41.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.937+0000 7fc91e9e9980 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2024-06-28T15:16:41.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.937+0000 7fc91e9e9980 10 moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-06-28T15:16:41.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.937+0000 7fc91e9e9980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:sigv4@example.com, ofs0, ns] 2024-06-28T15:16:41.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.937+0000 7fc7f37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1056561954828 cookie 94181083070512 notifier 5558 bl.length()=187 2024-06-28T15:16:41.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.937+0000 7fc7f37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2024-06-28T15:16:41.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.937+0000 7fc7f37fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-06-28T15:16:41.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.941+0000 7fc91e9e9980 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2024-06-28T15:16:41.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.941+0000 7fc91e9e9980 10 moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-06-28T15:16:41.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.941+0000 7fc91e9e9980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:lNCnR47C2g+ZidCWBAUuwfSAA7Q=, ofs0, ns] 2024-06-28T15:16:41.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.941+0000 7fc7f2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1056561954829 cookie 94181083070512 notifier 5558 bl.length()=197 2024-06-28T15:16:41.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.941+0000 7fc7f2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2024-06-28T15:16:41.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.941+0000 7fc7f2ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-06-28T15:16:41.930 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:16:41.930 INFO:tasks.workunit.client.0.smithi178.stdout: "user_id": "ceph-test-maven", 2024-06-28T15:16:41.931 INFO:tasks.workunit.client.0.smithi178.stdout: "display_name": "maven test user", 2024-06-28T15:16:41.931 INFO:tasks.workunit.client.0.smithi178.stdout: "email": "sigv4@example.com", 2024-06-28T15:16:41.931 INFO:tasks.workunit.client.0.smithi178.stdout: "suspended": 0, 2024-06-28T15:16:41.931 INFO:tasks.workunit.client.0.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:16:41.931 INFO:tasks.workunit.client.0.smithi178.stdout: "subusers": [], 2024-06-28T15:16:41.931 INFO:tasks.workunit.client.0.smithi178.stdout: "keys": [ 2024-06-28T15:16:41.931 INFO:tasks.workunit.client.0.smithi178.stdout: { 2024-06-28T15:16:41.931 INFO:tasks.workunit.client.0.smithi178.stdout: "user": "ceph-test-maven", 2024-06-28T15:16:41.931 INFO:tasks.workunit.client.0.smithi178.stdout: "access_key": "lNCnR47C2g+ZidCWBAUuwfSAA7Q=", 2024-06-28T15:16:41.931 INFO:tasks.workunit.client.0.smithi178.stdout: "secret_key": "tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g==", 2024-06-28T15:16:41.931 INFO:tasks.workunit.client.0.smithi178.stdout: "active": true, 2024-06-28T15:16:41.931 INFO:tasks.workunit.client.0.smithi178.stdout: "create_date": "2024-06-28T15:16:41.937169Z" 2024-06-28T15:16:41.932 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:16:41.932 INFO:tasks.workunit.client.0.smithi178.stdout: ], 2024-06-28T15:16:41.932 INFO:tasks.workunit.client.0.smithi178.stdout: "swift_keys": [], 2024-06-28T15:16:41.932 INFO:tasks.workunit.client.0.smithi178.stdout: "caps": [], 2024-06-28T15:16:41.932 INFO:tasks.workunit.client.0.smithi178.stdout: "op_mask": "read, write, delete", 2024-06-28T15:16:41.932 INFO:tasks.workunit.client.0.smithi178.stdout: "default_placement": "", 2024-06-28T15:16:41.932 INFO:tasks.workunit.client.0.smithi178.stdout: "default_storage_class": "", 2024-06-28T15:16:41.932 INFO:tasks.workunit.client.0.smithi178.stdout: "placement_tags": [], 2024-06-28T15:16:41.932 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_quota": { 2024-06-28T15:16:41.932 INFO:tasks.workunit.client.0.smithi178.stdout: "enabled": false, 2024-06-28T15:16:41.932 INFO:tasks.workunit.client.0.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:16:41.933 INFO:tasks.workunit.client.0.smithi178.stdout: "max_size": -1, 2024-06-28T15:16:41.933 INFO:tasks.workunit.client.0.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:16:41.933 INFO:tasks.workunit.client.0.smithi178.stdout: "max_objects": -1 2024-06-28T15:16:41.933 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:16:41.933 INFO:tasks.workunit.client.0.smithi178.stdout: "user_quota": { 2024-06-28T15:16:41.933 INFO:tasks.workunit.client.0.smithi178.stdout: "enabled": false, 2024-06-28T15:16:41.933 INFO:tasks.workunit.client.0.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:16:41.933 INFO:tasks.workunit.client.0.smithi178.stdout: "max_size": -1, 2024-06-28T15:16:41.933 INFO:tasks.workunit.client.0.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:16:41.933 INFO:tasks.workunit.client.0.smithi178.stdout: "max_objects": -1 2024-06-28T15:16:41.933 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:16:41.933 INFO:tasks.workunit.client.0.smithi178.stdout: "temp_url_keys": [], 2024-06-28T15:16:41.933 INFO:tasks.workunit.client.0.smithi178.stdout: "type": "rgw", 2024-06-28T15:16:41.934 INFO:tasks.workunit.client.0.smithi178.stdout: "mfa_ids": [], 2024-06-28T15:16:41.934 INFO:tasks.workunit.client.0.smithi178.stdout: "account_id": "", 2024-06-28T15:16:41.934 INFO:tasks.workunit.client.0.smithi178.stdout: "path": "/", 2024-06-28T15:16:41.934 INFO:tasks.workunit.client.0.smithi178.stdout: "create_date": "2024-06-28T15:16:41.937148Z", 2024-06-28T15:16:41.934 INFO:tasks.workunit.client.0.smithi178.stdout: "tags": [], 2024-06-28T15:16:41.934 INFO:tasks.workunit.client.0.smithi178.stdout: "group_ids": [] 2024-06-28T15:16:41.934 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:16:41.934 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:16:41.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.945+0000 7fc7d2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:16:41.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.949+0000 7fc91e9e9980 20 remove_watcher() i=0 2024-06-28T15:16:41.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.949+0000 7fc91e9e9980 2 removed watcher, disabling cache 2024-06-28T15:16:41.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.953+0000 7fc91e9e9980 20 remove_watcher() i=1 2024-06-28T15:16:41.940 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.953+0000 7fc91e9e9980 20 remove_watcher() i=2 2024-06-28T15:16:41.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.957+0000 7fc91e9e9980 20 remove_watcher() i=3 2024-06-28T15:16:41.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.957+0000 7fc91e9e9980 20 remove_watcher() i=4 2024-06-28T15:16:41.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.961+0000 7fc91e9e9980 20 remove_watcher() i=5 2024-06-28T15:16:41.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.961+0000 7fc91e9e9980 20 remove_watcher() i=6 2024-06-28T15:16:41.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:16:41.961+0000 7fc91e9e9980 20 remove_watcher() i=7 2024-06-28T15:16:41.961 INFO:tasks.workunit.client.0.smithi178.stderr:+ [ -z ] 2024-06-28T15:16:41.961 INFO:tasks.workunit.client.0.smithi178.stderr:+ [ -z /home/ubuntu/cephtest ] 2024-06-28T15:16:41.961 INFO:tasks.workunit.client.0.smithi178.stderr:+ cat /home/ubuntu/cephtest/url_file 2024-06-28T15:16:41.962 INFO:tasks.workunit.client.0.smithi178.stderr:+ export RGW_HTTP_ENDPOINT_URL=http://smithi178.front.sepia.ceph.com:80 2024-06-28T15:16:41.962 INFO:tasks.workunit.client.0.smithi178.stderr:+ [ -z /home/ubuntu/cephtest/clone.client.0 ] 2024-06-28T15:16:41.962 INFO:tasks.workunit.client.0.smithi178.stderr:+ cd /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum 2024-06-28T15:16:41.962 INFO:tasks.workunit.client.0.smithi178.stderr:+ ./mvnw clean package 2024-06-28T15:16:41.968 INFO:tasks.workunit.client.0.smithi178.stdout:Warning: JAVA_HOME environment variable is not set. 2024-06-28T15:16:46.793 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Scanning for projects... 2024-06-28T15:16:46.925 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom 2024-06-28T15:16:47.166 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.4/5.6 kB Progress (1): 2.8/5.6 kB Progress (1): 4.1/5.6 kB Progress (1): 5.5/5.6 kB Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom (5.6 kB at 23 kB/s) 2024-06-28T15:16:47.276 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom 2024-06-28T15:16:47.479 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.4/83 kB Progress (1): 2.8/83 kB Progress (1): 4.1/83 kB Progress (1): 5.5/83 kB Progress (1): 6.9/83 kB Progress (1): 8.3/83 kB Progress (1): 9.7/83 kB Progress (1): 11/83 kB Progress (1): 12/83 kB Progress (1): 14/83 kB Progress (1): 15/83 kB Progress (1): 16/83 kB Progress (1): 33/83 kB Progress (1): 49/83 kB Progress (1): 65/83 kB Progress (1): 82/83 kB Progress (1): 83 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom (83 kB at 409 kB/s) 2024-06-28T15:16:47.512 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom 2024-06-28T15:16:47.674 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/77 kB Progress (1): 33/77 kB Progress (1): 49/77 kB Progress (1): 64/77 kB Progress (1): 77 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom (77 kB at 471 kB/s) 2024-06-28T15:16:47.746 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:16:47.746 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2024-06-28T15:16:47.746 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Building jcksum 1.0 2024-06-28T15:16:47.746 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] from pom.xml 2024-06-28T15:16:47.746 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2024-06-28T15:16:47.753 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom 2024-06-28T15:16:47.818 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom (5.3 kB at 80 kB/s) 2024-06-28T15:16:47.834 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom 2024-06-28T15:16:47.902 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom (9.9 kB at 146 kB/s) 2024-06-28T15:16:47.910 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom 2024-06-28T15:16:48.022 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/45 kB Progress (1): 32/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom (45 kB at 404 kB/s) 2024-06-28T15:16:48.035 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom 2024-06-28T15:16:48.115 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom (21 kB at 257 kB/s) 2024-06-28T15:16:48.143 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar 2024-06-28T15:16:48.238 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/36 kB Progress (1): 33/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar (36 kB at 376 kB/s) 2024-06-28T15:16:48.256 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom 2024-06-28T15:16:48.320 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom (8.2 kB at 127 kB/s) 2024-06-28T15:16:48.327 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom 2024-06-28T15:16:48.394 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom (8.1 kB at 121 kB/s) 2024-06-28T15:16:48.400 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom 2024-06-28T15:16:48.524 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/48 kB Progress (1): 33/48 kB Progress (1): 48 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom (48 kB at 387 kB/s) 2024-06-28T15:16:48.533 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom 2024-06-28T15:16:48.608 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom (21 kB at 276 kB/s) 2024-06-28T15:16:48.621 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar 2024-06-28T15:16:48.705 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar (31 kB at 364 kB/s) 2024-06-28T15:16:48.719 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom 2024-06-28T15:16:48.781 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 9.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom (9.8 kB at 156 kB/s) 2024-06-28T15:16:48.796 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar 2024-06-28T15:16:48.922 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/66 kB Progress (1): 33/66 kB Progress (1): 49/66 kB Progress (1): 66/66 kB Progress (1): 66 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar (66 kB at 517 kB/s) 2024-06-28T15:16:48.938 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom 2024-06-28T15:16:49.000 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom (5.3 kB at 86 kB/s) 2024-06-28T15:16:49.006 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom 2024-06-28T15:16:49.077 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom (22 kB at 309 kB/s) 2024-06-28T15:16:49.086 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom 2024-06-28T15:16:49.194 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/50 kB Progress (1): 32/50 kB Progress (1): 49/50 kB Progress (1): 50 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom (50 kB at 462 kB/s) 2024-06-28T15:16:49.205 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom 2024-06-28T15:16:49.283 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/24 kB Progress (1): 24 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom (24 kB at 302 kB/s) 2024-06-28T15:16:49.297 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom 2024-06-28T15:16:49.363 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom (5.6 kB at 85 kB/s) 2024-06-28T15:16:49.371 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar 2024-06-28T15:16:49.491 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/45 kB Progress (1): 33/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar (45 kB at 371 kB/s) 2024-06-28T15:16:49.510 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom 2024-06-28T15:16:49.573 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom (6.8 kB at 106 kB/s) 2024-06-28T15:16:49.579 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom 2024-06-28T15:16:49.643 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom (9.9 kB at 157 kB/s) 2024-06-28T15:16:49.648 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom 2024-06-28T15:16:49.749 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/46 kB Progress (1): 33/46 kB Progress (1): 46 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom (46 kB at 452 kB/s) 2024-06-28T15:16:49.757 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom 2024-06-28T15:16:49.828 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom (20 kB at 287 kB/s) 2024-06-28T15:16:49.839 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar 2024-06-28T15:16:49.918 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar (27 kB at 342 kB/s) 2024-06-28T15:16:49.956 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom 2024-06-28T15:16:50.017 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom (6.1 kB at 100 kB/s) 2024-06-28T15:16:50.022 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom 2024-06-28T15:16:50.099 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom (21 kB at 277 kB/s) 2024-06-28T15:16:50.111 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom 2024-06-28T15:16:50.185 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom (20 kB at 270 kB/s) 2024-06-28T15:16:50.203 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom 2024-06-28T15:16:50.258 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom (4.0 kB at 72 kB/s) 2024-06-28T15:16:50.263 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom 2024-06-28T15:16:50.323 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom (1.6 kB at 26 kB/s) 2024-06-28T15:16:50.327 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom 2024-06-28T15:16:50.387 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom (2.2 kB at 38 kB/s) 2024-06-28T15:16:50.398 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom 2024-06-28T15:16:50.457 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom (3.6 kB at 62 kB/s) 2024-06-28T15:16:50.468 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom 2024-06-28T15:16:50.533 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom (3.2 kB at 50 kB/s) 2024-06-28T15:16:50.542 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom 2024-06-28T15:16:50.613 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom (11 kB at 157 kB/s) 2024-06-28T15:16:50.623 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom 2024-06-28T15:16:50.674 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom (1.7 kB at 32 kB/s) 2024-06-28T15:16:50.684 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom 2024-06-28T15:16:50.739 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom (3.8 kB at 70 kB/s) 2024-06-28T15:16:50.745 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom 2024-06-28T15:16:50.800 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom (4.9 kB at 89 kB/s) 2024-06-28T15:16:50.805 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom 2024-06-28T15:16:50.855 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom (1.2 kB at 23 kB/s) 2024-06-28T15:16:50.860 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom 2024-06-28T15:16:50.911 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom (3.8 kB at 74 kB/s) 2024-06-28T15:16:50.916 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom 2024-06-28T15:16:50.983 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom (14 kB at 206 kB/s) 2024-06-28T15:16:50.991 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom 2024-06-28T15:16:51.047 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom (3.7 kB at 66 kB/s) 2024-06-28T15:16:51.061 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom 2024-06-28T15:16:51.117 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom (2.5 kB at 44 kB/s) 2024-06-28T15:16:51.123 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom 2024-06-28T15:16:51.186 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom (2.9 kB at 46 kB/s) 2024-06-28T15:16:51.199 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom 2024-06-28T15:16:51.261 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 7.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom (7.4 kB at 120 kB/s) 2024-06-28T15:16:51.270 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom 2024-06-28T15:16:51.325 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 5.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom (5.5 kB at 99 kB/s) 2024-06-28T15:16:51.332 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom 2024-06-28T15:16:51.386 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom (3.5 kB at 63 kB/s) 2024-06-28T15:16:51.398 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom 2024-06-28T15:16:51.452 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom (4.4 kB at 78 kB/s) 2024-06-28T15:16:51.457 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom 2024-06-28T15:16:51.507 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom (2.4 kB at 47 kB/s) 2024-06-28T15:16:51.516 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom 2024-06-28T15:16:51.568 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom (6.4 kB at 124 kB/s) 2024-06-28T15:16:51.578 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom 2024-06-28T15:16:51.628 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom (6.1 kB at 121 kB/s) 2024-06-28T15:16:51.638 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom 2024-06-28T15:16:51.690 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom (3.0 kB at 56 kB/s) 2024-06-28T15:16:51.699 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom 2024-06-28T15:16:51.799 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom (3.5 kB at 35 kB/s) 2024-06-28T15:16:51.810 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom 2024-06-28T15:16:51.866 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom (4.1 kB at 72 kB/s) 2024-06-28T15:16:51.873 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom 2024-06-28T15:16:51.927 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom (1.8 kB at 32 kB/s) 2024-06-28T15:16:51.938 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom 2024-06-28T15:16:52.001 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom (6.6 kB at 103 kB/s) 2024-06-28T15:16:52.007 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom 2024-06-28T15:16:52.074 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom (16 kB at 241 kB/s) 2024-06-28T15:16:52.080 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom 2024-06-28T15:16:52.171 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/35 kB Progress (1): 32/35 kB Progress (1): 35 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom (35 kB at 381 kB/s) 2024-06-28T15:16:52.179 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom 2024-06-28T15:16:52.249 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom (17 kB at 244 kB/s) 2024-06-28T15:16:52.259 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom 2024-06-28T15:16:52.308 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom (5.0 kB at 99 kB/s) 2024-06-28T15:16:52.313 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom 2024-06-28T15:16:52.378 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom (13 kB at 206 kB/s) 2024-06-28T15:16:52.388 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom 2024-06-28T15:16:52.464 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom (14 kB at 184 kB/s) 2024-06-28T15:16:52.469 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom 2024-06-28T15:16:52.569 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/68 kB Progress (1): 32/68 kB Progress (1): 49/68 kB Progress (1): 65/68 kB Progress (1): 68 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom (68 kB at 680 kB/s) 2024-06-28T15:16:52.591 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2024-06-28T15:16:52.657 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom (16 kB at 234 kB/s) 2024-06-28T15:16:52.671 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom 2024-06-28T15:16:52.736 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom (15 kB at 238 kB/s) 2024-06-28T15:16:52.741 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom 2024-06-28T15:16:52.867 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/79 kB Progress (1): 33/79 kB Progress (1): 49/79 kB Progress (1): 66/79 kB Progress (1): 79 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom (79 kB at 634 kB/s) 2024-06-28T15:16:52.882 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom 2024-06-28T15:16:52.935 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/18 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom (18 kB at 348 kB/s) 2024-06-28T15:16:52.946 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom 2024-06-28T15:16:52.993 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom (20 kB at 416 kB/s) 2024-06-28T15:16:52.998 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom 2024-06-28T15:16:53.148 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/78 kB Progress (1): 33/78 kB Progress (1): 49/78 kB Progress (1): 66/78 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom (78 kB at 523 kB/s) 2024-06-28T15:16:53.167 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom 2024-06-28T15:16:53.220 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom (3.8 kB at 72 kB/s) 2024-06-28T15:16:53.224 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom 2024-06-28T15:16:53.297 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom (14 kB at 192 kB/s) 2024-06-28T15:16:53.305 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom 2024-06-28T15:16:53.363 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom (3.2 kB at 54 kB/s) 2024-06-28T15:16:53.370 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom 2024-06-28T15:16:53.410 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom (3.2 kB at 80 kB/s) 2024-06-28T15:16:53.415 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom 2024-06-28T15:16:53.465 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom (2.0 kB at 41 kB/s) 2024-06-28T15:16:53.470 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom 2024-06-28T15:16:53.527 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom (2.8 kB at 50 kB/s) 2024-06-28T15:16:53.533 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom 2024-06-28T15:16:53.596 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom (1.5 kB at 24 kB/s) 2024-06-28T15:16:53.602 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom 2024-06-28T15:16:53.651 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom (3.0 kB at 60 kB/s) 2024-06-28T15:16:53.658 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom 2024-06-28T15:16:53.702 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom (3.2 kB at 73 kB/s) 2024-06-28T15:16:53.709 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom 2024-06-28T15:16:53.771 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom (3.2 kB at 49 kB/s) 2024-06-28T15:16:53.779 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom 2024-06-28T15:16:53.831 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom (1.0 kB at 20 kB/s) 2024-06-28T15:16:53.840 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom 2024-06-28T15:16:53.869 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 964 B Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom (964 B at 33 kB/s) 2024-06-28T15:16:53.876 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom 2024-06-28T15:16:53.936 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom (3.1 kB at 51 kB/s) 2024-06-28T15:16:53.942 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom 2024-06-28T15:16:53.988 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom (3.0 kB at 66 kB/s) 2024-06-28T15:16:53.994 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom 2024-06-28T15:16:54.035 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom (3.4 kB at 82 kB/s) 2024-06-28T15:16:54.041 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom 2024-06-28T15:16:54.082 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom (3.4 kB at 82 kB/s) 2024-06-28T15:16:54.088 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom 2024-06-28T15:16:54.132 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom (3.0 kB at 69 kB/s) 2024-06-28T15:16:54.182 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar 2024-06-28T15:16:56.956 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3 MB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar (3.3 MB at 1.2 MB/s) 2024-06-28T15:16:56.959 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar 2024-06-28T15:16:56.960 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar 2024-06-28T15:16:56.960 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar 2024-06-28T15:16:56.960 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar 2024-06-28T15:16:56.961 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar 2024-06-28T15:16:57.093 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/98 kB Progress (1): 33/98 kB Progress (1): 49/98 kB Progress (1): 64/98 kB Progress (1): 80/98 kB Progress (1): 97/98 kB Progress (1): 98 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar (98 kB at 721 kB/s) 2024-06-28T15:16:57.093 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar 2024-06-28T15:16:57.139 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.4/71 kB Progress (1): 2.8/71 kB Progress (1): 4.1/71 kB Progress (2): 4.1/71 kB | 1.4/35 kB Progress (2): 5.5/71 kB | 1.4/35 kB Progress (2): 5.5/71 kB | 2.8/35 kB Progress (2): 6.9/71 kB | 2.8/35 kB Progress (2): 6.9/71 kB | 4.1/35 kB Progress (2): 8.3/71 kB | 4.1/35 kB Progress (2): 8.3/71 kB | 5.5/35 kB Progress (2): 9.7/71 kB | 5.5/35 kB Progress (2): 9.7/71 kB | 6.9/35 kB Progress (2): 9.7/71 kB | 8.3/35 kB Progress (2): 11/71 kB | 8.3/35 kB Progress (2): 11/71 kB | 9.7/35 kB Progress (2): 12/71 kB | 9.7/35 kB Progress (3): 12/71 kB | 9.7/35 kB | 1.4/16 kB Progress (3): 12/71 kB | 9.7/35 kB | 2.8/16 kB Progress (3): 12/71 kB | 9.7/35 kB | 4.1/16 kB Progress (3): 12/71 kB | 9.7/35 kB | 5.5/16 kB Progress (4): 12/71 kB | 9.7/35 kB | 5.5/16 kB | 1.4/49 kB Progress (4): 14/71 kB | 9.7/35 kB | 5.5/16 kB | 1.4/49 kB Progress (4): 14/71 kB | 9.7/35 kB | 6.9/16 kB | 1.4/49 kB Progress (4): 14/71 kB | 9.7/35 kB | 6.9/16 kB | 2.7/49 kB Progress (4): 15/71 kB | 9.7/35 kB | 6.9/16 kB | 2.7/49 kB Progress (4): 15/71 kB | 9.7/35 kB | 8.2/16 kB | 2.7/49 kB Progress (4): 16/71 kB | 9.7/35 kB | 8.2/16 kB | 2.7/49 kB Progress (4): 16/71 kB | 9.7/35 kB | 8.2/16 kB | 4.1/49 kB Progress (4): 16/71 kB | 9.7/35 kB | 9.6/16 kB | 4.1/49 kB Progress (4): 16/71 kB | 9.7/35 kB | 11/16 kB | 4.1/49 kB Progress (4): 16/71 kB | 9.7/35 kB | 12/16 kB | 4.1/49 kB Progress (4): 16/71 kB | 9.7/35 kB | 14/16 kB | 4.1/49 kB Progress (4): 16/71 kB | 9.7/35 kB | 15/16 kB | 4.1/49 kB Progress (4): 16/71 kB | 9.7/35 kB | 16 kB | 4.1/49 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar (16 kB at 88 kB/s) 2024-06-28T15:16:57.139 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar 2024-06-28T15:16:57.164 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (4): 16/71 kB | 9.7/35 kB | 4.1/49 kB | 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar (16 kB at 79 kB/s) 2024-06-28T15:16:57.164 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar 2024-06-28T15:16:57.206 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 16/71 kB | 11/35 kB | 4.1/49 kB Progress (3): 16/71 kB | 12/35 kB | 4.1/49 kB Progress (3): 16/71 kB | 12/35 kB | 5.5/49 kB Progress (3): 16/71 kB | 14/35 kB | 5.5/49 kB Progress (3): 16/71 kB | 15/35 kB | 5.5/49 kB Progress (3): 16/71 kB | 16/35 kB | 5.5/49 kB Progress (3): 16/71 kB | 16/35 kB | 6.9/49 kB Progress (3): 16/71 kB | 18/35 kB | 6.9/49 kB Progress (3): 16/71 kB | 19/35 kB | 6.9/49 kB Progress (3): 16/71 kB | 21/35 kB | 6.9/49 kB Progress (3): 16/71 kB | 22/35 kB | 6.9/49 kB Progress (3): 16/71 kB | 22/35 kB | 8.2/49 kB Progress (3): 16/71 kB | 23/35 kB | 8.2/49 kB Progress (3): 16/71 kB | 25/35 kB | 8.2/49 kB Progress (3): 16/71 kB | 25/35 kB | 9.6/49 kB Progress (3): 16/71 kB | 25/35 kB | 11/49 kB Progress (3): 16/71 kB | 25/35 kB | 12/49 kB Progress (4): 16/71 kB | 25/35 kB | 12/49 kB | 1.4/782 kB Progress (4): 16/71 kB | 25/35 kB | 14/49 kB | 1.4/782 kB Progress (4): 16/71 kB | 25/35 kB | 15/49 kB | 1.4/782 kB Progress (4): 16/71 kB | 26/35 kB | 15/49 kB | 1.4/782 kB Progress (4): 16/71 kB | 26/35 kB | 15/49 kB | 2.8/782 kB Progress (4): 16/71 kB | 26/35 kB | 16/49 kB | 2.8/782 kB Progress (4): 16/71 kB | 26/35 kB | 16/49 kB | 4.1/782 kB Progress (4): 16/71 kB | 26/35 kB | 18/49 kB | 4.1/782 kB Progress (4): 16/71 kB | 27/35 kB | 18/49 kB | 4.1/782 kB Progress (4): 16/71 kB | 27/35 kB | 19/49 kB | 4.1/782 kB Progress (4): 16/71 kB | 29/35 kB | 19/49 kB | 4.1/782 kB Progress (4): 16/71 kB | 29/35 kB | 19/49 kB | 5.5/782 kB Progress (4): 16/71 kB | 30/35 kB | 19/49 kB | 5.5/782 kB Progress (4): 16/71 kB | 30/35 kB | 21/49 kB | 5.5/782 kB Progress (4): 16/71 kB | 30/35 kB | 21/49 kB | 6.9/782 kB Progress (4): 33/71 kB | 30/35 kB | 21/49 kB | 6.9/782 kB Progress (4): 33/71 kB | 32/35 kB | 21/49 kB | 6.9/782 kB Progress (4): 33/71 kB | 32/35 kB | 22/49 kB | 6.9/782 kB Progress (4): 33/71 kB | 33/35 kB | 22/49 kB | 6.9/782 kB Progress (4): 33/71 kB | 33/35 kB | 22/49 kB | 8.3/782 kB Progress (4): 33/71 kB | 33/35 kB | 23/49 kB | 8.3/782 kB Progress (4): 33/71 kB | 33/35 kB | 23/49 kB | 9.7/782 kB Progress (4): 33/71 kB | 34/35 kB | 23/49 kB | 9.7/782 kB Progress (4): 33/71 kB | 34/35 kB | 23/49 kB | 11/782 kB Progress (4): 33/71 kB | 34/35 kB | 25/49 kB | 11/782 kB Progress (4): 33/71 kB | 34/35 kB | 25/49 kB | 12/782 kB Progress (4): 33/71 kB | 34/35 kB | 26/49 kB | 12/782 kB Progress (4): 33/71 kB | 35 kB | 26/49 kB | 12/782 kB Progress (4): 33/71 kB | 35 kB | 27/49 kB | 12/782 kB Progress (4): 33/71 kB | 35 kB | 27/49 kB | 14/782 kB Progress (4): 33/71 kB | 35 kB | 29/49 kB | 14/782 kB Progress (4): 33/71 kB | 35 kB | 29/49 kB | 15/782 kB Progress (4): 33/71 kB | 35 kB | 30/49 kB | 15/782 kB Progress (4): 33/71 kB | 35 kB | 31/49 kB | 15/782 kB Progress (4): 33/71 kB | 35 kB | 33/49 kB | 15/782 kB Progress (4): 33/71 kB | 35 kB | 34/49 kB | 15/782 kB Progress (4): 33/71 kB | 35 kB | 34/49 kB | 16/782 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar (35 kB at 143 kB/s) 2024-06-28T15:16:57.207 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 33/71 kB | 34/49 kB | 18/782 kB Progress (3): 33/71 kB | 36/49 kB | 18/782 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar 2024-06-28T15:16:57.212 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 33/71 kB | 37/49 kB | 18/782 kB Progress (3): 33/71 kB | 38/49 kB | 18/782 kB Progress (3): 33/71 kB | 40/49 kB | 18/782 kB Progress (3): 33/71 kB | 40/49 kB | 19/782 kB Progress (3): 33/71 kB | 41/49 kB | 19/782 kB Progress (3): 33/71 kB | 41/49 kB | 21/782 kB Progress (3): 33/71 kB | 42/49 kB | 21/782 kB Progress (3): 33/71 kB | 44/49 kB | 21/782 kB Progress (3): 33/71 kB | 44/49 kB | 22/782 kB Progress (3): 33/71 kB | 45/49 kB | 22/782 kB Progress (3): 33/71 kB | 45/49 kB | 23/782 kB Progress (3): 33/71 kB | 47/49 kB | 23/782 kB Progress (3): 33/71 kB | 48/49 kB | 23/782 kB Progress (3): 33/71 kB | 48/49 kB | 25/782 kB Progress (3): 33/71 kB | 49 kB | 25/782 kB Progress (3): 33/71 kB | 49 kB | 26/782 kB Progress (3): 33/71 kB | 49 kB | 27/782 kB Progress (3): 33/71 kB | 49 kB | 29/782 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar (49 kB at 197 kB/s) 2024-06-28T15:16:57.212 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 33/71 kB | 30/782 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar 2024-06-28T15:16:57.223 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 33/71 kB | 32/782 kB Progress (2): 33/71 kB | 33/782 kB Progress (2): 33/71 kB | 34/782 kB Progress (2): 33/71 kB | 36/782 kB Progress (2): 33/71 kB | 37/782 kB Progress (2): 33/71 kB | 38/782 kB Progress (3): 33/71 kB | 38/782 kB | 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar (11 kB at 44 kB/s) 2024-06-28T15:16:57.223 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar 2024-06-28T15:16:57.233 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 49/71 kB | 38/782 kB Progress (2): 66/71 kB | 38/782 kB Progress (2): 71 kB | 38/782 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar (71 kB at 257 kB/s) 2024-06-28T15:16:57.233 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar 2024-06-28T15:16:57.282 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 40/782 kB Progress (1): 41/782 kB Progress (1): 43/782 kB Progress (1): 44/782 kB Progress (1): 45/782 kB Progress (1): 47/782 kB Progress (1): 48/782 kB Progress (1): 64/782 kB Progress (2): 64/782 kB | 1.4/205 kB Progress (2): 64/782 kB | 2.8/205 kB Progress (2): 64/782 kB | 4.1/205 kB Progress (2): 64/782 kB | 5.5/205 kB Progress (2): 81/782 kB | 5.5/205 kB Progress (2): 81/782 kB | 6.9/205 kB Progress (2): 81/782 kB | 8.3/205 kB Progress (2): 81/782 kB | 9.7/205 kB Progress (2): 95/782 kB | 9.7/205 kB Progress (2): 95/782 kB | 11/205 kB Progress (2): 95/782 kB | 12/205 kB Progress (2): 95/782 kB | 14/205 kB Progress (2): 95/782 kB | 15/205 kB Progress (2): 95/782 kB | 16/205 kB Progress (3): 95/782 kB | 16/205 kB | 16/30 kB Progress (4): 95/782 kB | 16/205 kB | 16/30 kB | 16/81 kB Progress (4): 111/782 kB | 16/205 kB | 16/30 kB | 16/81 kB Progress (4): 128/782 kB | 16/205 kB | 16/30 kB | 16/81 kB Progress (4): 144/782 kB | 16/205 kB | 16/30 kB | 16/81 kB Progress (4): 160/782 kB | 16/205 kB | 16/30 kB | 16/81 kB Progress (4): 160/782 kB | 16/205 kB | 16/30 kB | 33/81 kB Progress (4): 160/782 kB | 33/205 kB | 16/30 kB | 33/81 kB Progress (4): 177/782 kB | 33/205 kB | 16/30 kB | 33/81 kB Progress (5): 177/782 kB | 33/205 kB | 16/30 kB | 33/81 kB | 16/798 kB Progress (5): 193/782 kB | 33/205 kB | 16/30 kB | 33/81 kB | 16/798 kB Progress (5): 210/782 kB | 33/205 kB | 16/30 kB | 33/81 kB | 16/798 kB Progress (5): 210/782 kB | 33/205 kB | 30 kB | 33/81 kB | 16/798 kB Progress (5): 226/782 kB | 33/205 kB | 30 kB | 33/81 kB | 16/798 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar (30 kB at 95 kB/s) 2024-06-28T15:16:57.282 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar 2024-06-28T15:16:57.307 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (4): 226/782 kB | 33/205 kB | 49/81 kB | 16/798 kB Progress (4): 242/782 kB | 33/205 kB | 49/81 kB | 16/798 kB Progress (4): 259/782 kB | 33/205 kB | 49/81 kB | 16/798 kB Progress (4): 259/782 kB | 49/205 kB | 49/81 kB | 16/798 kB Progress (4): 275/782 kB | 49/205 kB | 49/81 kB | 16/798 kB Progress (4): 275/782 kB | 49/205 kB | 63/81 kB | 16/798 kB Progress (4): 291/782 kB | 49/205 kB | 63/81 kB | 16/798 kB Progress (4): 291/782 kB | 49/205 kB | 63/81 kB | 33/798 kB Progress (4): 308/782 kB | 49/205 kB | 63/81 kB | 33/798 kB Progress (4): 324/782 kB | 49/205 kB | 63/81 kB | 33/798 kB Progress (4): 341/782 kB | 49/205 kB | 63/81 kB | 33/798 kB Progress (4): 357/782 kB | 49/205 kB | 63/81 kB | 33/798 kB Progress (4): 373/782 kB | 49/205 kB | 63/81 kB | 33/798 kB Progress (4): 390/782 kB | 49/205 kB | 63/81 kB | 33/798 kB Progress (4): 390/782 kB | 49/205 kB | 79/81 kB | 33/798 kB Progress (4): 390/782 kB | 49/205 kB | 81 kB | 33/798 kB Progress (4): 406/782 kB | 49/205 kB | 81 kB | 33/798 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar (81 kB at 236 kB/s) 2024-06-28T15:16:57.307 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 422/782 kB | 49/205 kB | 33/798 kB Progress (3): 422/782 kB | 63/205 kB | 33/798 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar 2024-06-28T15:16:57.323 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 439/782 kB | 63/205 kB | 33/798 kB Progress (3): 455/782 kB | 63/205 kB | 33/798 kB Progress (3): 472/782 kB | 63/205 kB | 33/798 kB Progress (3): 488/782 kB | 63/205 kB | 33/798 kB Progress (3): 488/782 kB | 63/205 kB | 49/798 kB Progress (3): 504/782 kB | 63/205 kB | 49/798 kB Progress (3): 504/782 kB | 79/205 kB | 49/798 kB Progress (3): 521/782 kB | 79/205 kB | 49/798 kB Progress (3): 537/782 kB | 79/205 kB | 49/798 kB Progress (3): 554/782 kB | 79/205 kB | 49/798 kB Progress (3): 570/782 kB | 79/205 kB | 49/798 kB Progress (4): 570/782 kB | 79/205 kB | 49/798 kB | 13 kB Progress (4): 586/782 kB | 79/205 kB | 49/798 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar (13 kB at 37 kB/s) 2024-06-28T15:16:57.323 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar 2024-06-28T15:16:57.333 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 603/782 kB | 79/205 kB | 49/798 kB Progress (3): 603/782 kB | 79/205 kB | 66/798 kB Progress (3): 619/782 kB | 79/205 kB | 66/798 kB Progress (3): 635/782 kB | 79/205 kB | 66/798 kB Progress (3): 652/782 kB | 79/205 kB | 66/798 kB Progress (3): 668/782 kB | 79/205 kB | 66/798 kB Progress (3): 685/782 kB | 79/205 kB | 66/798 kB Progress (3): 701/782 kB | 79/205 kB | 66/798 kB Progress (3): 717/782 kB | 79/205 kB | 66/798 kB Progress (3): 717/782 kB | 95/205 kB | 66/798 kB Progress (3): 734/782 kB | 95/205 kB | 66/798 kB Progress (3): 750/782 kB | 95/205 kB | 66/798 kB Progress (3): 767/782 kB | 95/205 kB | 66/798 kB Progress (3): 782 kB | 95/205 kB | 66/798 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar (782 kB at 2.1 MB/s) 2024-06-28T15:16:57.334 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar 2024-06-28T15:16:57.364 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 95/205 kB | 82/798 kB Progress (2): 95/205 kB | 94/798 kB Progress (2): 112/205 kB | 94/798 kB Progress (3): 112/205 kB | 94/798 kB | 16/182 kB Progress (3): 112/205 kB | 94/798 kB | 33/182 kB Progress (3): 112/205 kB | 111/798 kB | 33/182 kB Progress (3): 128/205 kB | 111/798 kB | 33/182 kB Progress (3): 128/205 kB | 111/798 kB | 49/182 kB Progress (3): 128/205 kB | 127/798 kB | 49/182 kB Progress (4): 128/205 kB | 127/798 kB | 49/182 kB | 16/151 kB Progress (5): 128/205 kB | 127/798 kB | 49/182 kB | 16/151 kB | 16/27 kB Progress (5): 128/205 kB | 127/798 kB | 63/182 kB | 16/151 kB | 16/27 kB Progress (5): 128/205 kB | 127/798 kB | 63/182 kB | 16/151 kB | 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar (27 kB at 68 kB/s) 2024-06-28T15:16:57.364 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar 2024-06-28T15:16:57.411 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (4): 144/205 kB | 127/798 kB | 63/182 kB | 16/151 kB Progress (4): 144/205 kB | 144/798 kB | 63/182 kB | 16/151 kB Progress (4): 144/205 kB | 144/798 kB | 80/182 kB | 16/151 kB Progress (4): 161/205 kB | 144/798 kB | 80/182 kB | 16/151 kB Progress (4): 161/205 kB | 144/798 kB | 80/182 kB | 33/151 kB Progress (4): 161/205 kB | 144/798 kB | 96/182 kB | 33/151 kB Progress (4): 161/205 kB | 160/798 kB | 96/182 kB | 33/151 kB Progress (4): 161/205 kB | 160/798 kB | 112/182 kB | 33/151 kB Progress (4): 177/205 kB | 160/798 kB | 112/182 kB | 33/151 kB Progress (4): 177/205 kB | 160/798 kB | 112/182 kB | 49/151 kB Progress (4): 177/205 kB | 176/798 kB | 112/182 kB | 49/151 kB Progress (4): 177/205 kB | 176/798 kB | 129/182 kB | 49/151 kB Progress (5): 177/205 kB | 176/798 kB | 129/182 kB | 49/151 kB | 15/30 kB Progress (5): 177/205 kB | 176/798 kB | 129/182 kB | 49/151 kB | 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar (30 kB at 67 kB/s) 2024-06-28T15:16:57.411 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar 2024-06-28T15:16:57.423 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (4): 194/205 kB | 176/798 kB | 129/182 kB | 49/151 kB Progress (4): 194/205 kB | 193/798 kB | 129/182 kB | 49/151 kB Progress (4): 194/205 kB | 193/798 kB | 145/182 kB | 49/151 kB Progress (4): 194/205 kB | 193/798 kB | 145/182 kB | 63/151 kB Progress (4): 205 kB | 193/798 kB | 145/182 kB | 63/151 kB Progress (4): 205 kB | 193/798 kB | 162/182 kB | 63/151 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar (205 kB at 445 kB/s) 2024-06-28T15:16:57.423 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar 2024-06-28T15:16:57.444 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 209/798 kB | 162/182 kB | 63/151 kB Progress (3): 209/798 kB | 178/182 kB | 63/151 kB Progress (3): 209/798 kB | 178/182 kB | 79/151 kB Progress (3): 209/798 kB | 182 kB | 79/151 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar (182 kB at 378 kB/s) 2024-06-28T15:16:57.444 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar 2024-06-28T15:16:57.470 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 226/798 kB | 79/151 kB Progress (3): 226/798 kB | 79/151 kB | 16/385 kB Progress (3): 226/798 kB | 79/151 kB | 33/385 kB Progress (3): 226/798 kB | 79/151 kB | 49/385 kB Progress (3): 226/798 kB | 79/151 kB | 61/385 kB Progress (3): 242/798 kB | 79/151 kB | 61/385 kB Progress (3): 242/798 kB | 79/151 kB | 78/385 kB Progress (3): 242/798 kB | 79/151 kB | 94/385 kB Progress (3): 242/798 kB | 79/151 kB | 111/385 kB Progress (3): 242/798 kB | 96/151 kB | 111/385 kB Progress (3): 242/798 kB | 96/151 kB | 127/385 kB Progress (3): 242/798 kB | 96/151 kB | 143/385 kB Progress (3): 242/798 kB | 96/151 kB | 160/385 kB Progress (3): 242/798 kB | 96/151 kB | 176/385 kB Progress (3): 242/798 kB | 96/151 kB | 193/385 kB Progress (3): 242/798 kB | 96/151 kB | 209/385 kB Progress (3): 242/798 kB | 96/151 kB | 225/385 kB Progress (3): 242/798 kB | 96/151 kB | 242/385 kB Progress (3): 242/798 kB | 96/151 kB | 258/385 kB Progress (3): 258/798 kB | 96/151 kB | 258/385 kB Progress (3): 258/798 kB | 96/151 kB | 274/385 kB Progress (3): 258/798 kB | 96/151 kB | 291/385 kB Progress (3): 258/798 kB | 96/151 kB | 307/385 kB Progress (3): 258/798 kB | 96/151 kB | 324/385 kB Progress (3): 258/798 kB | 96/151 kB | 328/385 kB Progress (3): 258/798 kB | 96/151 kB | 344/385 kB Progress (4): 258/798 kB | 96/151 kB | 344/385 kB | 13 kB Progress (4): 258/798 kB | 96/151 kB | 361/385 kB | 13 kB Progress (4): 258/798 kB | 112/151 kB | 361/385 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar (13 kB at 26 kB/s) 2024-06-28T15:16:57.470 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 258/798 kB | 112/151 kB | 377/385 kB Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar 2024-06-28T15:16:57.472 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 258/798 kB | 112/151 kB | 385 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar (385 kB at 756 kB/s) 2024-06-28T15:16:57.472 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar 2024-06-28T15:16:57.501 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 275/798 kB | 112/151 kB Progress (2): 275/798 kB | 129/151 kB Progress (3): 275/798 kB | 129/151 kB | 16/75 kB Progress (3): 291/798 kB | 129/151 kB | 16/75 kB Progress (3): 291/798 kB | 129/151 kB | 33/75 kB Progress (3): 291/798 kB | 145/151 kB | 33/75 kB Progress (3): 307/798 kB | 145/151 kB | 33/75 kB Progress (3): 307/798 kB | 145/151 kB | 49/75 kB Progress (3): 307/798 kB | 151 kB | 49/75 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar (151 kB at 280 kB/s) 2024-06-28T15:16:57.501 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar 2024-06-28T15:16:57.515 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 307/798 kB | 49/75 kB | 16/780 kB Progress (3): 307/798 kB | 49/75 kB | 32/780 kB Progress (3): 307/798 kB | 49/75 kB | 49/780 kB Progress (3): 307/798 kB | 49/75 kB | 65/780 kB Progress (3): 307/798 kB | 49/75 kB | 81/780 kB Progress (3): 307/798 kB | 49/75 kB | 98/780 kB Progress (3): 307/798 kB | 49/75 kB | 114/780 kB Progress (3): 307/798 kB | 49/75 kB | 130/780 kB Progress (3): 307/798 kB | 49/75 kB | 147/780 kB Progress (3): 307/798 kB | 49/75 kB | 163/780 kB Progress (3): 307/798 kB | 49/75 kB | 180/780 kB Progress (3): 307/798 kB | 66/75 kB | 180/780 kB Progress (3): 307/798 kB | 66/75 kB | 196/780 kB Progress (3): 307/798 kB | 66/75 kB | 212/780 kB Progress (3): 324/798 kB | 66/75 kB | 212/780 kB Progress (3): 324/798 kB | 66/75 kB | 229/780 kB Progress (3): 324/798 kB | 66/75 kB | 245/780 kB Progress (3): 324/798 kB | 66/75 kB | 262/780 kB Progress (3): 324/798 kB | 66/75 kB | 278/780 kB Progress (3): 324/798 kB | 75 kB | 278/780 kB Progress (3): 324/798 kB | 75 kB | 294/780 kB Progress (3): 324/798 kB | 75 kB | 311/780 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar (75 kB at 135 kB/s) 2024-06-28T15:16:57.516 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar 2024-06-28T15:16:57.538 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 324/798 kB | 311/780 kB | 16/329 kB Progress (3): 324/798 kB | 327/780 kB | 16/329 kB Progress (3): 324/798 kB | 343/780 kB | 16/329 kB Progress (3): 324/798 kB | 360/780 kB | 16/329 kB Progress (3): 324/798 kB | 376/780 kB | 16/329 kB Progress (3): 324/798 kB | 393/780 kB | 16/329 kB Progress (3): 324/798 kB | 402/780 kB | 16/329 kB Progress (3): 324/798 kB | 418/780 kB | 16/329 kB Progress (3): 324/798 kB | 435/780 kB | 16/329 kB Progress (3): 340/798 kB | 435/780 kB | 16/329 kB Progress (3): 340/798 kB | 451/780 kB | 16/329 kB Progress (3): 340/798 kB | 468/780 kB | 16/329 kB Progress (3): 340/798 kB | 484/780 kB | 16/329 kB Progress (3): 340/798 kB | 500/780 kB | 16/329 kB Progress (3): 340/798 kB | 517/780 kB | 16/329 kB Progress (3): 340/798 kB | 533/780 kB | 16/329 kB Progress (3): 340/798 kB | 549/780 kB | 16/329 kB Progress (3): 340/798 kB | 566/780 kB | 16/329 kB Progress (3): 340/798 kB | 582/780 kB | 16/329 kB Progress (3): 340/798 kB | 599/780 kB | 16/329 kB Progress (3): 340/798 kB | 615/780 kB | 16/329 kB Progress (3): 340/798 kB | 631/780 kB | 16/329 kB Progress (3): 340/798 kB | 648/780 kB | 16/329 kB Progress (3): 340/798 kB | 664/780 kB | 16/329 kB Progress (3): 340/798 kB | 664/780 kB | 33/329 kB Progress (3): 357/798 kB | 664/780 kB | 33/329 kB Progress (3): 357/798 kB | 681/780 kB | 33/329 kB Progress (3): 357/798 kB | 697/780 kB | 33/329 kB Progress (3): 357/798 kB | 713/780 kB | 33/329 kB Progress (3): 357/798 kB | 730/780 kB | 33/329 kB Progress (3): 357/798 kB | 746/780 kB | 33/329 kB Progress (3): 357/798 kB | 762/780 kB | 33/329 kB Progress (3): 357/798 kB | 779/780 kB | 33/329 kB Progress (3): 357/798 kB | 780 kB | 33/329 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar (780 kB at 1.4 MB/s) 2024-06-28T15:16:57.538 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar 2024-06-28T15:16:57.571 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 357/798 kB | 49/329 kB Progress (3): 357/798 kB | 49/329 kB | 16/501 kB Progress (3): 373/798 kB | 49/329 kB | 16/501 kB Progress (3): 373/798 kB | 49/329 kB | 33/501 kB Progress (3): 373/798 kB | 66/329 kB | 33/501 kB Progress (3): 389/798 kB | 66/329 kB | 33/501 kB Progress (4): 389/798 kB | 66/329 kB | 33/501 kB | 16/354 kB Progress (4): 406/798 kB | 66/329 kB | 33/501 kB | 16/354 kB Progress (4): 406/798 kB | 66/329 kB | 33/501 kB | 33/354 kB Progress (4): 406/798 kB | 82/329 kB | 33/501 kB | 33/354 kB Progress (5): 406/798 kB | 82/329 kB | 33/501 kB | 33/354 kB | 16/41 kB Progress (5): 406/798 kB | 82/329 kB | 33/501 kB | 49/354 kB | 16/41 kB Progress (5): 406/798 kB | 82/329 kB | 33/501 kB | 49/354 kB | 33/41 kB Progress (5): 406/798 kB | 82/329 kB | 33/501 kB | 49/354 kB | 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar (41 kB at 68 kB/s) 2024-06-28T15:16:57.571 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar 2024-06-28T15:16:57.603 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (4): 422/798 kB | 82/329 kB | 33/501 kB | 49/354 kB Progress (4): 422/798 kB | 82/329 kB | 49/501 kB | 49/354 kB Progress (4): 422/798 kB | 82/329 kB | 49/501 kB | 66/354 kB Progress (4): 422/798 kB | 82/329 kB | 65/501 kB | 66/354 kB Progress (4): 422/798 kB | 98/329 kB | 65/501 kB | 66/354 kB Progress (4): 422/798 kB | 98/329 kB | 65/501 kB | 82/354 kB Progress (4): 439/798 kB | 98/329 kB | 65/501 kB | 82/354 kB Progress (4): 439/798 kB | 98/329 kB | 82/501 kB | 82/354 kB Progress (4): 439/798 kB | 98/329 kB | 82/501 kB | 98/354 kB Progress (5): 439/798 kB | 98/329 kB | 82/501 kB | 98/354 kB | 6.4 kB Progress (5): 439/798 kB | 115/329 kB | 82/501 kB | 98/354 kB | 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar (6.4 kB at 10.0 kB/s) 2024-06-28T15:16:57.603 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar 2024-06-28T15:16:57.660 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (4): 455/798 kB | 115/329 kB | 82/501 kB | 98/354 kB Progress (4): 455/798 kB | 115/329 kB | 82/501 kB | 115/354 kB Progress (4): 455/798 kB | 115/329 kB | 98/501 kB | 115/354 kB Progress (4): 455/798 kB | 131/329 kB | 98/501 kB | 115/354 kB Progress (4): 471/798 kB | 131/329 kB | 98/501 kB | 115/354 kB Progress (4): 471/798 kB | 131/329 kB | 98/501 kB | 131/354 kB Progress (4): 471/798 kB | 131/329 kB | 98/501 kB | 147/354 kB Progress (4): 471/798 kB | 131/329 kB | 114/501 kB | 147/354 kB Progress (4): 488/798 kB | 131/329 kB | 114/501 kB | 147/354 kB Progress (4): 488/798 kB | 147/329 kB | 114/501 kB | 147/354 kB Progress (4): 488/798 kB | 147/329 kB | 114/501 kB | 164/354 kB Progress (4): 488/798 kB | 147/329 kB | 131/501 kB | 164/354 kB Progress (4): 504/798 kB | 147/329 kB | 131/501 kB | 164/354 kB Progress (5): 504/798 kB | 147/329 kB | 131/501 kB | 164/354 kB | 16/586 kB Progress (5): 504/798 kB | 147/329 kB | 131/501 kB | 164/354 kB | 33/586 kB Progress (5): 504/798 kB | 147/329 kB | 131/501 kB | 164/354 kB | 49/586 kB Progress (5): 504/798 kB | 147/329 kB | 131/501 kB | 180/354 kB | 49/586 kB Progress (5): 504/798 kB | 147/329 kB | 131/501 kB | 180/354 kB | 64/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 64/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 81/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 97/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 114/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 130/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 146/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 163/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 179/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 196/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 212/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 228/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 245/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 261/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 277/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 294/586 kB Progress (5): 504/798 kB | 164/329 kB | 131/501 kB | 180/354 kB | 310/586 kB Progress (5): 504/798 kB | 164/329 kB | 147/501 kB | 180/354 kB | 310/586 kB Progress (5): 504/798 kB | 164/329 kB | 147/501 kB | 180/354 kB | 327/586 kB Progress (5): 520/798 kB | 164/329 kB | 147/501 kB | 180/354 kB | 327/586 kB Progress (5): 520/798 kB | 164/329 kB | 147/501 kB | 197/354 kB | 327/586 kB Progress (5): 520/798 kB | 164/329 kB | 147/501 kB | 197/354 kB | 343/586 kB Progress (5): 520/798 kB | 164/329 kB | 147/501 kB | 197/354 kB | 359/586 kB Progress (5): 520/798 kB | 164/329 kB | 147/501 kB | 197/354 kB | 376/586 kB Progress (5): 520/798 kB | 164/329 kB | 147/501 kB | 197/354 kB | 392/586 kB Progress (5): 520/798 kB | 164/329 kB | 147/501 kB | 197/354 kB | 409/586 kB Progress (5): 520/798 kB | 164/329 kB | 147/501 kB | 197/354 kB | 425/586 kB Progress (5): 520/798 kB | 164/329 kB | 147/501 kB | 197/354 kB | 441/586 kB Progress (5): 520/798 kB | 164/329 kB | 147/501 kB | 197/354 kB | 458/586 kB Progress (5): 520/798 kB | 164/329 kB | 147/501 kB | 197/354 kB | 474/586 kB Progress (5): 520/798 kB | 180/329 kB | 147/501 kB | 197/354 kB | 474/586 kB Progress (5): 520/798 kB | 180/329 kB | 147/501 kB | 197/354 kB | 490/586 kB Progress (5): 520/798 kB | 180/329 kB | 147/501 kB | 197/354 kB | 507/586 kB Progress (5): 520/798 kB | 180/329 kB | 147/501 kB | 197/354 kB | 523/586 kB Progress (5): 520/798 kB | 180/329 kB | 147/501 kB | 213/354 kB | 523/586 kB Progress (5): 520/798 kB | 180/329 kB | 147/501 kB | 213/354 kB | 540/586 kB Progress (5): 520/798 kB | 180/329 kB | 147/501 kB | 213/354 kB | 556/586 kB Progress (5): 520/798 kB | 180/329 kB | 147/501 kB | 213/354 kB | 572/586 kB Progress (5): 520/798 kB | 180/329 kB | 147/501 kB | 213/354 kB | 586 kB Progress (5): 537/798 kB | 180/329 kB | 147/501 kB | 213/354 kB | 586 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar (586 kB at 841 kB/s) 2024-06-28T15:16:57.661 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar 2024-06-28T15:16:57.706 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (4): 537/798 kB | 180/329 kB | 147/501 kB | 229/354 kB Progress (4): 537/798 kB | 197/329 kB | 147/501 kB | 229/354 kB Progress (4): 537/798 kB | 197/329 kB | 163/501 kB | 229/354 kB Progress (4): 553/798 kB | 197/329 kB | 163/501 kB | 229/354 kB Progress (4): 553/798 kB | 197/329 kB | 163/501 kB | 246/354 kB Progress (4): 553/798 kB | 197/329 kB | 180/501 kB | 246/354 kB Progress (4): 553/798 kB | 213/329 kB | 180/501 kB | 246/354 kB Progress (4): 570/798 kB | 213/329 kB | 180/501 kB | 246/354 kB Progress (4): 570/798 kB | 213/329 kB | 180/501 kB | 262/354 kB Progress (4): 570/798 kB | 213/329 kB | 180/501 kB | 279/354 kB Progress (4): 586/798 kB | 213/329 kB | 180/501 kB | 279/354 kB Progress (4): 586/798 kB | 229/329 kB | 180/501 kB | 279/354 kB Progress (4): 586/798 kB | 229/329 kB | 196/501 kB | 279/354 kB Progress (5): 586/798 kB | 229/329 kB | 196/501 kB | 279/354 kB | 15/211 kB Progress (5): 586/798 kB | 229/329 kB | 196/501 kB | 295/354 kB | 15/211 kB Progress (5): 586/798 kB | 229/329 kB | 196/501 kB | 295/354 kB | 31/211 kB Progress (5): 586/798 kB | 229/329 kB | 196/501 kB | 295/354 kB | 48/211 kB Progress (5): 586/798 kB | 229/329 kB | 196/501 kB | 295/354 kB | 64/211 kB Progress (5): 586/798 kB | 229/329 kB | 196/501 kB | 295/354 kB | 79/211 kB Progress (5): 586/798 kB | 229/329 kB | 196/501 kB | 295/354 kB | 95/211 kB Progress (5): 602/798 kB | 229/329 kB | 196/501 kB | 295/354 kB | 95/211 kB Progress (5): 602/798 kB | 229/329 kB | 196/501 kB | 295/354 kB | 112/211 kB Progress (5): 602/798 kB | 229/329 kB | 196/501 kB | 295/354 kB | 128/211 kB Progress (5): 602/798 kB | 229/329 kB | 196/501 kB | 295/354 kB | 144/211 kB Progress (5): 602/798 kB | 229/329 kB | 196/501 kB | 295/354 kB | 161/211 kB Progress (5): 602/798 kB | 229/329 kB | 196/501 kB | 295/354 kB | 177/211 kB Progress (5): 602/798 kB | 246/329 kB | 196/501 kB | 295/354 kB | 177/211 kB Progress (5): 602/798 kB | 246/329 kB | 196/501 kB | 295/354 kB | 194/211 kB Progress (5): 602/798 kB | 246/329 kB | 196/501 kB | 295/354 kB | 210/211 kB Progress (5): 602/798 kB | 246/329 kB | 196/501 kB | 295/354 kB | 211 kB Progress (5): 602/798 kB | 246/329 kB | 196/501 kB | 311/354 kB | 211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar (211 kB at 284 kB/s) 2024-06-28T15:16:57.706 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar 2024-06-28T15:16:57.732 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (4): 602/798 kB | 246/329 kB | 213/501 kB | 311/354 kB Progress (4): 602/798 kB | 256/329 kB | 213/501 kB | 311/354 kB Progress (4): 619/798 kB | 256/329 kB | 213/501 kB | 311/354 kB Progress (4): 619/798 kB | 256/329 kB | 213/501 kB | 328/354 kB Progress (4): 635/798 kB | 256/329 kB | 213/501 kB | 328/354 kB Progress (4): 635/798 kB | 256/329 kB | 213/501 kB | 344/354 kB Progress (4): 635/798 kB | 256/329 kB | 229/501 kB | 344/354 kB Progress (4): 635/798 kB | 272/329 kB | 229/501 kB | 344/354 kB Progress (4): 635/798 kB | 272/329 kB | 229/501 kB | 354 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar (354 kB at 460 kB/s) 2024-06-28T15:16:57.732 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar 2024-06-28T15:16:57.750 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 652/798 kB | 272/329 kB | 229/501 kB Progress (3): 652/798 kB | 289/329 kB | 229/501 kB Progress (3): 652/798 kB | 289/329 kB | 245/501 kB Progress (4): 652/798 kB | 289/329 kB | 245/501 kB | 14 kB Progress (4): 668/798 kB | 289/329 kB | 245/501 kB | 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar (14 kB at 18 kB/s) 2024-06-28T15:16:57.750 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar 2024-06-28T15:16:57.778 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 668/798 kB | 289/329 kB | 262/501 kB Progress (3): 668/798 kB | 305/329 kB | 262/501 kB Progress (3): 684/798 kB | 305/329 kB | 262/501 kB Progress (3): 684/798 kB | 321/329 kB | 262/501 kB Progress (3): 684/798 kB | 321/329 kB | 278/501 kB Progress (3): 701/798 kB | 321/329 kB | 278/501 kB Progress (3): 701/798 kB | 329 kB | 278/501 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar (329 kB at 403 kB/s) 2024-06-28T15:16:57.778 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar 2024-06-28T15:16:57.787 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 701/798 kB | 278/501 kB | 16/106 kB Progress (3): 701/798 kB | 278/501 kB | 32/106 kB Progress (3): 717/798 kB | 278/501 kB | 32/106 kB Progress (3): 717/798 kB | 295/501 kB | 32/106 kB Progress (4): 717/798 kB | 295/501 kB | 32/106 kB | 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar (6.8 kB at 8.3 kB/s) 2024-06-28T15:16:57.787 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar 2024-06-28T15:16:57.830 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 717/798 kB | 295/501 kB | 49/106 kB Progress (3): 733/798 kB | 295/501 kB | 49/106 kB Progress (3): 733/798 kB | 295/501 kB | 65/106 kB Progress (3): 733/798 kB | 311/501 kB | 65/106 kB Progress (3): 750/798 kB | 311/501 kB | 65/106 kB Progress (3): 750/798 kB | 311/501 kB | 81/106 kB Progress (3): 766/798 kB | 311/501 kB | 81/106 kB Progress (3): 766/798 kB | 327/501 kB | 81/106 kB Progress (3): 766/798 kB | 327/501 kB | 98/106 kB Progress (4): 766/798 kB | 327/501 kB | 98/106 kB | 16/205 kB Progress (4): 766/798 kB | 327/501 kB | 98/106 kB | 33/205 kB Progress (5): 766/798 kB | 327/501 kB | 98/106 kB | 33/205 kB | 16/245 kB Progress (5): 766/798 kB | 327/501 kB | 98/106 kB | 49/205 kB | 16/245 kB Progress (5): 766/798 kB | 327/501 kB | 98/106 kB | 63/205 kB | 16/245 kB Progress (5): 766/798 kB | 327/501 kB | 106 kB | 63/205 kB | 16/245 kB Progress (5): 766/798 kB | 327/501 kB | 106 kB | 79/205 kB | 16/245 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar (106 kB at 123 kB/s) 2024-06-28T15:16:57.830 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (4): 766/798 kB | 327/501 kB | 95/205 kB | 16/245 kB Progress (4): 783/798 kB | 327/501 kB | 95/205 kB | 16/245 kB Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar 2024-06-28T15:16:57.837 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (4): 783/798 kB | 327/501 kB | 112/205 kB | 16/245 kB Progress (4): 783/798 kB | 344/501 kB | 112/205 kB | 16/245 kB Progress (4): 783/798 kB | 344/501 kB | 128/205 kB | 16/245 kB Progress (4): 783/798 kB | 344/501 kB | 145/205 kB | 16/245 kB Progress (4): 783/798 kB | 344/501 kB | 161/205 kB | 16/245 kB Progress (4): 783/798 kB | 344/501 kB | 177/205 kB | 16/245 kB Progress (4): 783/798 kB | 344/501 kB | 194/205 kB | 16/245 kB Progress (4): 783/798 kB | 344/501 kB | 205 kB | 16/245 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar (205 kB at 235 kB/s) 2024-06-28T15:16:57.837 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar 2024-06-28T15:16:57.845 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 798 kB | 344/501 kB | 16/245 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar (798 kB at 905 kB/s) 2024-06-28T15:16:57.845 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar 2024-06-28T15:16:57.886 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 344/501 kB | 33/245 kB Progress (2): 360/501 kB | 33/245 kB Progress (2): 360/501 kB | 49/245 kB Progress (2): 376/501 kB | 49/245 kB Progress (2): 376/501 kB | 66/245 kB Progress (3): 376/501 kB | 66/245 kB | 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar (15 kB at 17 kB/s) 2024-06-28T15:16:57.886 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar 2024-06-28T15:16:57.890 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 376/501 kB | 66/245 kB | 16/16 kB Progress (3): 376/501 kB | 66/245 kB | 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar (16 kB at 18 kB/s) 2024-06-28T15:16:57.891 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar 2024-06-28T15:16:57.901 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 393/501 kB | 66/245 kB Progress (3): 393/501 kB | 66/245 kB | 6.4 kB Progress (3): 393/501 kB | 82/245 kB | 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar (6.4 kB at 6.8 kB/s) 2024-06-28T15:16:57.901 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar 2024-06-28T15:16:57.942 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 409/501 kB | 82/245 kB Progress (2): 409/501 kB | 94/245 kB Progress (2): 426/501 kB | 94/245 kB Progress (2): 426/501 kB | 111/245 kB Progress (3): 426/501 kB | 111/245 kB | 2.7/24 kB Progress (4): 426/501 kB | 111/245 kB | 2.7/24 kB | 16/23 kB Progress (4): 426/501 kB | 111/245 kB | 9.6/24 kB | 16/23 kB Progress (4): 426/501 kB | 111/245 kB | 15/24 kB | 16/23 kB Progress (4): 426/501 kB | 127/245 kB | 15/24 kB | 16/23 kB Progress (4): 426/501 kB | 127/245 kB | 21/24 kB | 16/23 kB Progress (4): 442/501 kB | 127/245 kB | 21/24 kB | 16/23 kB Progress (4): 442/501 kB | 127/245 kB | 24 kB | 16/23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar (24 kB at 25 kB/s) 2024-06-28T15:16:57.942 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar 2024-06-28T15:16:57.944 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 442/501 kB | 127/245 kB | 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar (23 kB at 23 kB/s) 2024-06-28T15:16:57.959 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 442/501 kB | 143/245 kB Progress (3): 442/501 kB | 143/245 kB | 16/17 kB Progress (3): 442/501 kB | 143/245 kB | 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar (17 kB at 17 kB/s) 2024-06-28T15:16:58.007 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 458/501 kB | 143/245 kB Progress (2): 458/501 kB | 160/245 kB Progress (2): 475/501 kB | 160/245 kB Progress (2): 475/501 kB | 176/245 kB Progress (2): 491/501 kB | 176/245 kB Progress (3): 491/501 kB | 176/245 kB | 16/184 kB Progress (3): 491/501 kB | 193/245 kB | 16/184 kB Progress (3): 501 kB | 193/245 kB | 16/184 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar (501 kB at 481 kB/s) 2024-06-28T15:16:58.056 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 193/245 kB | 33/184 kB Progress (2): 193/245 kB | 49/184 kB Progress (2): 209/245 kB | 49/184 kB Progress (2): 209/245 kB | 66/184 kB Progress (2): 225/245 kB | 66/184 kB Progress (2): 225/245 kB | 82/184 kB Progress (2): 242/245 kB | 82/184 kB Progress (2): 242/245 kB | 98/184 kB Progress (2): 245 kB | 98/184 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar (245 kB at 224 kB/s) 2024-06-28T15:16:58.110 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 115/184 kB Progress (1): 131/184 kB Progress (1): 147/184 kB Progress (1): 164/184 kB Progress (1): 180/184 kB Progress (1): 184 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar (184 kB at 161 kB/s) 2024-06-28T15:16:58.156 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:16:58.156 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] --- clean:3.2.0:clean (default-clean) @ jcksum --- 2024-06-28T15:16:58.170 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom 2024-06-28T15:16:58.213 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom (5.8 kB at 136 kB/s) 2024-06-28T15:16:58.217 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom 2024-06-28T15:16:58.255 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom (5.1 kB at 134 kB/s) 2024-06-28T15:16:58.258 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom 2024-06-28T15:16:58.314 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/43 kB Progress (1): 33/43 kB Progress (1): 43 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom (43 kB at 765 kB/s) 2024-06-28T15:16:58.322 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom 2024-06-28T15:16:58.366 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom (14 kB at 324 kB/s) 2024-06-28T15:16:58.377 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar 2024-06-28T15:16:58.490 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/153 kB Progress (1): 33/153 kB Progress (1): 49/153 kB Progress (1): 66/153 kB Progress (1): 82/153 kB Progress (1): 98/153 kB Progress (1): 115/153 kB Progress (1): 131/153 kB Progress (1): 147/153 kB Progress (1): 153 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar (153 kB at 1.4 MB/s) 2024-06-28T15:16:58.491 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar 2024-06-28T15:16:58.655 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/215 kB Progress (1): 33/215 kB Progress (1): 49/215 kB Progress (1): 62/215 kB Progress (1): 78/215 kB Progress (1): 95/215 kB Progress (1): 111/215 kB Progress (1): 128/215 kB Progress (1): 144/215 kB Progress (1): 160/215 kB Progress (1): 177/215 kB Progress (1): 193/215 kB Progress (1): 209/215 kB Progress (1): 215 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar (215 kB at 1.3 MB/s) 2024-06-28T15:16:58.824 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:16:58.824 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2024-06-28T15:16:58.830 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom 2024-06-28T15:16:58.871 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom (2.7 kB at 65 kB/s) 2024-06-28T15:16:58.874 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom 2024-06-28T15:16:58.934 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom (23 kB at 369 kB/s) 2024-06-28T15:16:58.941 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom 2024-06-28T15:16:59.006 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 8.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom (8.8 kB at 135 kB/s) 2024-06-28T15:16:59.011 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom 2024-06-28T15:16:59.085 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom (25 kB at 343 kB/s) 2024-06-28T15:16:59.092 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom 2024-06-28T15:16:59.134 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom (6.0 kB at 144 kB/s) 2024-06-28T15:16:59.138 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom 2024-06-28T15:16:59.188 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom (3.2 kB at 64 kB/s) 2024-06-28T15:16:59.195 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom 2024-06-28T15:16:59.256 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 612 B Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom (612 B at 10 kB/s) 2024-06-28T15:16:59.260 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom 2024-06-28T15:16:59.318 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom (2.7 kB at 48 kB/s) 2024-06-28T15:16:59.322 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom 2024-06-28T15:16:59.388 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom (14 kB at 214 kB/s) 2024-06-28T15:16:59.394 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom 2024-06-28T15:16:59.451 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom (3.2 kB at 56 kB/s) 2024-06-28T15:16:59.454 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom 2024-06-28T15:16:59.501 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom (8.4 kB at 182 kB/s) 2024-06-28T15:16:59.505 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom 2024-06-28T15:16:59.556 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom (8.4 kB at 164 kB/s) 2024-06-28T15:16:59.562 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom 2024-06-28T15:16:59.614 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 8.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom (8.0 kB at 154 kB/s) 2024-06-28T15:16:59.620 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom 2024-06-28T15:16:59.668 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom (20 kB at 420 kB/s) 2024-06-28T15:16:59.682 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom 2024-06-28T15:16:59.722 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom (5.1 kB at 127 kB/s) 2024-06-28T15:16:59.728 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom 2024-06-28T15:16:59.787 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom (31 kB at 529 kB/s) 2024-06-28T15:16:59.801 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom 2024-06-28T15:16:59.856 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom (5.1 kB at 91 kB/s) 2024-06-28T15:16:59.868 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar 2024-06-28T15:16:59.977 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 15/85 kB Progress (1): 31/85 kB Progress (1): 48/85 kB Progress (1): 64/85 kB Progress (1): 81/85 kB Progress (1): 85 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar (85 kB at 783 kB/s) 2024-06-28T15:16:59.978 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar 2024-06-28T15:16:59.978 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar 2024-06-28T15:16:59.979 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar 2024-06-28T15:16:59.979 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar 2024-06-28T15:16:59.979 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar 2024-06-28T15:17:00.016 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar (2.5 kB at 66 kB/s) 2024-06-28T15:17:00.016 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar 2024-06-28T15:17:00.018 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 8.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar (8.5 kB at 212 kB/s) 2024-06-28T15:17:00.018 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar 2024-06-28T15:17:00.046 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/55 kB Progress (2): 16/55 kB | 16/269 kB Progress (3): 16/55 kB | 16/269 kB | 16/41 kB Progress (3): 33/55 kB | 16/269 kB | 16/41 kB Progress (3): 33/55 kB | 16/269 kB | 33/41 kB Progress (3): 33/55 kB | 33/269 kB | 33/41 kB Progress (3): 49/55 kB | 33/269 kB | 33/41 kB Progress (3): 55 kB | 33/269 kB | 33/41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar (55 kB at 809 kB/s) 2024-06-28T15:17:00.047 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 33/269 kB | 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar (41 kB at 596 kB/s) 2024-06-28T15:17:00.085 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 49/269 kB Progress (2): 49/269 kB | 16/587 kB Progress (2): 49/269 kB | 33/587 kB Progress (2): 49/269 kB | 49/587 kB Progress (2): 49/269 kB | 66/587 kB Progress (2): 49/269 kB | 82/587 kB Progress (2): 49/269 kB | 98/587 kB Progress (2): 49/269 kB | 115/587 kB Progress (2): 49/269 kB | 131/587 kB Progress (2): 49/269 kB | 147/587 kB Progress (2): 49/269 kB | 164/587 kB Progress (2): 49/269 kB | 180/587 kB Progress (2): 49/269 kB | 197/587 kB Progress (3): 49/269 kB | 197/587 kB | 16/327 kB Progress (3): 49/269 kB | 213/587 kB | 16/327 kB Progress (3): 49/269 kB | 229/587 kB | 16/327 kB Progress (3): 49/269 kB | 246/587 kB | 16/327 kB Progress (3): 49/269 kB | 262/587 kB | 16/327 kB Progress (3): 49/269 kB | 279/587 kB | 16/327 kB Progress (3): 66/269 kB | 279/587 kB | 16/327 kB Progress (3): 66/269 kB | 295/587 kB | 16/327 kB Progress (3): 66/269 kB | 311/587 kB | 16/327 kB Progress (3): 66/269 kB | 328/587 kB | 16/327 kB Progress (3): 66/269 kB | 344/587 kB | 16/327 kB Progress (3): 66/269 kB | 360/587 kB | 16/327 kB Progress (3): 66/269 kB | 377/587 kB | 16/327 kB Progress (3): 66/269 kB | 393/587 kB | 16/327 kB Progress (3): 66/269 kB | 410/587 kB | 16/327 kB Progress (3): 66/269 kB | 426/587 kB | 16/327 kB Progress (3): 66/269 kB | 442/587 kB | 16/327 kB Progress (3): 66/269 kB | 459/587 kB | 16/327 kB Progress (3): 66/269 kB | 475/587 kB | 16/327 kB Progress (3): 66/269 kB | 492/587 kB | 16/327 kB Progress (3): 66/269 kB | 508/587 kB | 16/327 kB Progress (3): 66/269 kB | 524/587 kB | 16/327 kB Progress (3): 66/269 kB | 541/587 kB | 16/327 kB Progress (3): 66/269 kB | 541/587 kB | 32/327 kB Progress (3): 66/269 kB | 557/587 kB | 32/327 kB Progress (3): 82/269 kB | 557/587 kB | 32/327 kB Progress (3): 82/269 kB | 573/587 kB | 32/327 kB Progress (3): 82/269 kB | 585/587 kB | 32/327 kB Progress (3): 82/269 kB | 587 kB | 32/327 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar (587 kB at 5.6 MB/s) 2024-06-28T15:17:00.228 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 98/269 kB | 32/327 kB Progress (2): 98/269 kB | 49/327 kB Progress (2): 115/269 kB | 49/327 kB Progress (2): 115/269 kB | 65/327 kB Progress (2): 131/269 kB | 65/327 kB Progress (2): 131/269 kB | 81/327 kB Progress (2): 147/269 kB | 81/327 kB Progress (2): 147/269 kB | 98/327 kB Progress (2): 164/269 kB | 98/327 kB Progress (2): 164/269 kB | 114/327 kB Progress (2): 180/269 kB | 114/327 kB Progress (2): 180/269 kB | 130/327 kB Progress (2): 197/269 kB | 130/327 kB Progress (2): 197/269 kB | 147/327 kB Progress (2): 213/269 kB | 147/327 kB Progress (2): 229/269 kB | 147/327 kB Progress (2): 229/269 kB | 163/327 kB Progress (2): 246/269 kB | 163/327 kB Progress (2): 246/269 kB | 180/327 kB Progress (2): 246/269 kB | 196/327 kB Progress (2): 262/269 kB | 196/327 kB Progress (2): 269 kB | 196/327 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar (269 kB at 1.1 MB/s) 2024-06-28T15:17:00.362 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 212/327 kB Progress (1): 229/327 kB Progress (1): 245/327 kB Progress (1): 262/327 kB Progress (1): 278/327 kB Progress (1): 294/327 kB Progress (1): 311/327 kB Progress (1): 327/327 kB Progress (1): 327 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar (327 kB at 856 kB/s) 2024-06-28T15:17:00.440 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2024-06-28T15:17:00.441 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:17:00.441 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2024-06-28T15:17:00.449 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom 2024-06-28T15:17:00.492 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom (4.7 kB at 113 kB/s) 2024-06-28T15:17:00.495 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom 2024-06-28T15:17:00.559 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom (6.4 kB at 99 kB/s) 2024-06-28T15:17:00.563 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom 2024-06-28T15:17:00.640 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/33 kB Progress (1): 33 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom (33 kB at 423 kB/s) 2024-06-28T15:17:00.645 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2024-06-28T15:17:00.703 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom (14 kB at 241 kB/s) 2024-06-28T15:17:00.709 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom 2024-06-28T15:17:00.766 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 815 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom (815 B at 14 kB/s) 2024-06-28T15:17:00.769 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom 2024-06-28T15:17:00.813 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom (4.2 kB at 94 kB/s) 2024-06-28T15:17:00.817 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom 2024-06-28T15:17:00.861 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom (17 kB at 384 kB/s) 2024-06-28T15:17:00.867 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom 2024-06-28T15:17:00.903 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom (5.0 kB at 142 kB/s) 2024-06-28T15:17:00.907 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom 2024-06-28T15:17:00.962 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom (4.1 kB at 77 kB/s) 2024-06-28T15:17:00.968 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom 2024-06-28T15:17:01.026 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom (2.4 kB at 41 kB/s) 2024-06-28T15:17:01.029 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom 2024-06-28T15:17:01.090 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom (11 kB at 182 kB/s) 2024-06-28T15:17:01.095 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom 2024-06-28T15:17:01.166 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom (17 kB at 246 kB/s) 2024-06-28T15:17:01.171 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2024-06-28T15:17:01.234 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom (6.6 kB at 104 kB/s) 2024-06-28T15:17:01.239 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom 2024-06-28T15:17:01.300 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom (1.1 kB at 17 kB/s) 2024-06-28T15:17:01.304 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom 2024-06-28T15:17:01.365 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom (8.4 kB at 137 kB/s) 2024-06-28T15:17:01.368 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom 2024-06-28T15:17:01.401 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom (2.7 kB at 84 kB/s) 2024-06-28T15:17:01.405 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom 2024-06-28T15:17:01.446 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom (5.6 kB at 137 kB/s) 2024-06-28T15:17:01.450 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom 2024-06-28T15:17:01.496 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom (1.1 kB at 25 kB/s) 2024-06-28T15:17:01.501 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom 2024-06-28T15:17:01.555 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 770 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom (770 B at 14 kB/s) 2024-06-28T15:17:01.557 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom 2024-06-28T15:17:01.620 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom (6.0 kB at 97 kB/s) 2024-06-28T15:17:01.623 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom 2024-06-28T15:17:01.701 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/26 kB Progress (1): 26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom (26 kB at 330 kB/s) 2024-06-28T15:17:01.707 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom 2024-06-28T15:17:01.768 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom (1.2 kB at 20 kB/s) 2024-06-28T15:17:01.772 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom 2024-06-28T15:17:01.829 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom (1.3 kB at 23 kB/s) 2024-06-28T15:17:01.842 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar 2024-06-28T15:17:01.911 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar (14 kB at 196 kB/s) 2024-06-28T15:17:01.912 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar 2024-06-28T15:17:01.912 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar 2024-06-28T15:17:01.913 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar 2024-06-28T15:17:01.913 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar 2024-06-28T15:17:01.913 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar 2024-06-28T15:17:01.971 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/122 kB Progress (1): 33/122 kB Progress (2): 33/122 kB | 4.2 kB Progress (2): 49/122 kB | 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar (4.2 kB at 70 kB/s) 2024-06-28T15:17:01.971 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar 2024-06-28T15:17:01.975 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 65/122 kB Progress (1): 81/122 kB Progress (1): 98/122 kB Progress (1): 114/122 kB Progress (1): 122 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar (122 kB at 1.9 MB/s) 2024-06-28T15:17:01.975 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/27 kB Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar 2024-06-28T15:17:01.980 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 16/27 kB | 16/55 kB Progress (3): 16/27 kB | 16/55 kB | 16/334 kB Progress (3): 27 kB | 16/55 kB | 16/334 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar (27 kB at 403 kB/s) 2024-06-28T15:17:01.980 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar 2024-06-28T15:17:02.025 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 16/55 kB | 33/334 kB Progress (2): 33/55 kB | 33/334 kB Progress (2): 33/55 kB | 49/334 kB Progress (2): 49/55 kB | 49/334 kB Progress (2): 55 kB | 49/334 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar (55 kB at 483 kB/s) 2024-06-28T15:17:02.032 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 65/334 kB Progress (2): 65/334 kB | 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar (4.7 kB at 39 kB/s) 2024-06-28T15:17:02.051 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 65/334 kB | 16/267 kB Progress (2): 81/334 kB | 16/267 kB Progress (3): 81/334 kB | 16/267 kB | 16/23 kB Progress (3): 81/334 kB | 16/267 kB | 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar (23 kB at 166 kB/s) 2024-06-28T15:17:02.297 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 81/334 kB | 33/267 kB Progress (2): 97/334 kB | 33/267 kB Progress (2): 97/334 kB | 49/267 kB Progress (2): 114/334 kB | 49/267 kB Progress (2): 114/334 kB | 66/267 kB Progress (2): 130/334 kB | 66/267 kB Progress (2): 147/334 kB | 66/267 kB Progress (2): 147/334 kB | 82/267 kB Progress (2): 147/334 kB | 98/267 kB Progress (2): 163/334 kB | 98/267 kB Progress (2): 179/334 kB | 98/267 kB Progress (2): 179/334 kB | 115/267 kB Progress (2): 196/334 kB | 115/267 kB Progress (2): 196/334 kB | 131/267 kB Progress (2): 212/334 kB | 131/267 kB Progress (2): 212/334 kB | 147/267 kB Progress (2): 228/334 kB | 147/267 kB Progress (2): 228/334 kB | 164/267 kB Progress (2): 245/334 kB | 164/267 kB Progress (2): 245/334 kB | 180/267 kB Progress (2): 261/334 kB | 180/267 kB Progress (2): 278/334 kB | 180/267 kB Progress (2): 294/334 kB | 180/267 kB Progress (2): 310/334 kB | 180/267 kB Progress (2): 310/334 kB | 197/267 kB Progress (2): 310/334 kB | 213/267 kB Progress (2): 310/334 kB | 229/267 kB Progress (2): 327/334 kB | 229/267 kB Progress (2): 327/334 kB | 246/267 kB Progress (2): 334 kB | 246/267 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar (334 kB at 870 kB/s) 2024-06-28T15:17:02.319 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 262/267 kB Progress (1): 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar (267 kB at 658 kB/s) 2024-06-28T15:17:02.483 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Changes detected - recompiling the module! :source 2024-06-28T15:17:02.485 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/classes 2024-06-28T15:17:03.646 INFO:tasks.workunit.client.0.smithi178.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2024-06-28T15:17:03.647 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:17:03.647 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2024-06-28T15:17:03.650 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2024-06-28T15:17:03.650 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:17:03.650 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2024-06-28T15:17:03.655 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Changes detected - recompiling the module! :dependency 2024-06-28T15:17:03.655 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/test-classes 2024-06-28T15:17:03.831 INFO:tasks.workunit.client.0.smithi178.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2024-06-28T15:17:03.831 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:17:03.831 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2024-06-28T15:17:03.835 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom 2024-06-28T15:17:03.886 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom (6.2 kB at 125 kB/s) 2024-06-28T15:17:03.889 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom 2024-06-28T15:17:03.936 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom (3.5 kB at 77 kB/s) 2024-06-28T15:17:03.940 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom 2024-06-28T15:17:03.990 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom (3.3 kB at 64 kB/s) 2024-06-28T15:17:03.996 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom 2024-06-28T15:17:04.044 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom (4.1 kB at 85 kB/s) 2024-06-28T15:17:04.051 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom 2024-06-28T15:17:04.094 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom (3.3 kB at 75 kB/s) 2024-06-28T15:17:04.102 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom 2024-06-28T15:17:04.133 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom (4.5 kB at 148 kB/s) 2024-06-28T15:17:04.140 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom 2024-06-28T15:17:04.168 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom (1.8 kB at 63 kB/s) 2024-06-28T15:17:04.176 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom 2024-06-28T15:17:04.205 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom (2.2 kB at 76 kB/s) 2024-06-28T15:17:04.208 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom 2024-06-28T15:17:04.257 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/30 kB Progress (1): 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom (30 kB at 615 kB/s) 2024-06-28T15:17:04.263 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom 2024-06-28T15:17:04.293 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom (1.9 kB at 65 kB/s) 2024-06-28T15:17:04.298 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom 2024-06-28T15:17:04.333 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom (5.8 kB at 176 kB/s) 2024-06-28T15:17:04.338 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom 2024-06-28T15:17:04.364 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom (2.3 kB at 87 kB/s) 2024-06-28T15:17:04.368 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom 2024-06-28T15:17:04.409 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom (22 kB at 546 kB/s) 2024-06-28T15:17:04.413 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom 2024-06-28T15:17:04.473 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/37 kB Progress (1): 33/37 kB Progress (1): 37 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom (37 kB at 635 kB/s) 2024-06-28T15:17:04.478 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2024-06-28T15:17:04.512 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom (15 kB at 448 kB/s) 2024-06-28T15:17:04.518 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom 2024-06-28T15:17:04.554 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 8.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom (8.7 kB at 240 kB/s) 2024-06-28T15:17:04.558 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom 2024-06-28T15:17:04.606 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom (27 kB at 570 kB/s) 2024-06-28T15:17:04.614 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom 2024-06-28T15:17:04.652 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom (8.1 kB at 212 kB/s) 2024-06-28T15:17:04.660 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom 2024-06-28T15:17:04.693 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom (2.2 kB at 64 kB/s) 2024-06-28T15:17:04.700 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom 2024-06-28T15:17:04.731 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom (2.6 kB at 81 kB/s) 2024-06-28T15:17:04.740 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom 2024-06-28T15:17:04.775 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom (1.5 kB at 44 kB/s) 2024-06-28T15:17:04.778 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom 2024-06-28T15:17:04.812 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom (3.1 kB at 93 kB/s) 2024-06-28T15:17:04.816 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom 2024-06-28T15:17:04.866 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom (20 kB at 409 kB/s) 2024-06-28T15:17:04.870 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom 2024-06-28T15:17:04.906 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom (6.8 kB at 188 kB/s) 2024-06-28T15:17:04.910 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom 2024-06-28T15:17:04.954 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom (14 kB at 308 kB/s) 2024-06-28T15:17:04.960 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom 2024-06-28T15:17:04.999 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom (3.0 kB at 76 kB/s) 2024-06-28T15:17:05.002 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom 2024-06-28T15:17:05.036 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom (6.8 kB at 206 kB/s) 2024-06-28T15:17:05.039 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom 2024-06-28T15:17:05.075 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom (8.4 kB at 233 kB/s) 2024-06-28T15:17:05.080 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom 2024-06-28T15:17:05.114 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom (2.1 kB at 59 kB/s) 2024-06-28T15:17:05.119 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom 2024-06-28T15:17:05.154 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom (2.2 kB at 68 kB/s) 2024-06-28T15:17:05.161 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom 2024-06-28T15:17:05.198 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom (3.0 kB at 82 kB/s) 2024-06-28T15:17:05.205 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom 2024-06-28T15:17:05.252 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom (15 kB at 312 kB/s) 2024-06-28T15:17:05.257 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom 2024-06-28T15:17:05.305 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom (15 kB at 296 kB/s) 2024-06-28T15:17:05.311 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom 2024-06-28T15:17:05.358 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom (13 kB at 286 kB/s) 2024-06-28T15:17:05.362 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom 2024-06-28T15:17:05.403 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom (4.8 kB at 117 kB/s) 2024-06-28T15:17:05.408 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom 2024-06-28T15:17:05.451 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom (6.3 kB at 143 kB/s) 2024-06-28T15:17:05.456 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom 2024-06-28T15:17:05.519 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/32 kB Progress (1): 32 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom (32 kB at 513 kB/s) 2024-06-28T15:17:05.526 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom 2024-06-28T15:17:05.578 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom (17 kB at 333 kB/s) 2024-06-28T15:17:05.582 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom 2024-06-28T15:17:05.634 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom (15 kB at 290 kB/s) 2024-06-28T15:17:05.640 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom 2024-06-28T15:17:05.681 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 750 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom (750 B at 18 kB/s) 2024-06-28T15:17:05.684 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom 2024-06-28T15:17:05.725 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom (4.8 kB at 117 kB/s) 2024-06-28T15:17:05.730 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom 2024-06-28T15:17:05.775 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 7.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom (7.9 kB at 176 kB/s) 2024-06-28T15:17:05.780 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom 2024-06-28T15:17:05.824 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom (3.0 kB at 68 kB/s) 2024-06-28T15:17:05.830 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom 2024-06-28T15:17:05.876 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom (4.2 kB at 92 kB/s) 2024-06-28T15:17:05.883 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom 2024-06-28T15:17:05.926 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom (2.1 kB at 48 kB/s) 2024-06-28T15:17:05.932 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom 2024-06-28T15:17:05.978 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom (3.5 kB at 76 kB/s) 2024-06-28T15:17:05.984 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom 2024-06-28T15:17:06.036 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom (11 kB at 210 kB/s) 2024-06-28T15:17:06.040 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom 2024-06-28T15:17:06.096 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom (13 kB at 241 kB/s) 2024-06-28T15:17:06.099 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom 2024-06-28T15:17:06.158 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/19 kB Progress (1): 19 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom (19 kB at 323 kB/s) 2024-06-28T15:17:06.164 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom 2024-06-28T15:17:06.210 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 363 B Downloaded from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom (363 B at 7.7 kB/s) 2024-06-28T15:17:06.306 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom 2024-06-28T15:17:06.306 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom (6.1 kB at 105 kB/s) 2024-06-28T15:17:06.306 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom 2024-06-28T15:17:06.334 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom (7.3 kB at 126 kB/s) 2024-06-28T15:17:06.337 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2024-06-28T15:17:06.391 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom (4.8 kB at 88 kB/s) 2024-06-28T15:17:06.396 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom 2024-06-28T15:17:06.449 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom (7.3 kB at 140 kB/s) 2024-06-28T15:17:06.455 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom 2024-06-28T15:17:06.522 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom (22 kB at 323 kB/s) 2024-06-28T15:17:06.526 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom 2024-06-28T15:17:06.657 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/78 kB Progress (1): 26/78 kB Progress (1): 42/78 kB Progress (1): 59/78 kB Progress (1): 75/78 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom (78 kB at 595 kB/s) 2024-06-28T15:17:06.666 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom 2024-06-28T15:17:06.733 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom (23 kB at 347 kB/s) 2024-06-28T15:17:06.739 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom 2024-06-28T15:17:06.792 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom (5.6 kB at 109 kB/s) 2024-06-28T15:17:06.797 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom 2024-06-28T15:17:06.848 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom (4.3 kB at 84 kB/s) 2024-06-28T15:17:06.852 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom 2024-06-28T15:17:06.902 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom (3.2 kB at 63 kB/s) 2024-06-28T15:17:06.906 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom 2024-06-28T15:17:06.981 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/28 kB Progress (1): 28 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom (28 kB at 371 kB/s) 2024-06-28T15:17:06.990 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom 2024-06-28T15:17:07.037 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom (2.4 kB at 47 kB/s) 2024-06-28T15:17:07.049 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar 2024-06-28T15:17:07.438 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/308 kB Progress (1): 32/308 kB Progress (1): 49/308 kB Progress (1): 65/308 kB Progress (1): 81/308 kB Progress (1): 97/308 kB Progress (1): 114/308 kB Progress (1): 130/308 kB Progress (1): 147/308 kB Progress (1): 163/308 kB Progress (1): 179/308 kB Progress (1): 196/308 kB Progress (1): 212/308 kB Progress (1): 228/308 kB Progress (1): 245/308 kB Progress (1): 261/308 kB Progress (1): 278/308 kB Progress (1): 294/308 kB Progress (1): 308 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar (308 kB at 794 kB/s) 2024-06-28T15:17:07.439 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar 2024-06-28T15:17:07.439 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar 2024-06-28T15:17:07.439 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar 2024-06-28T15:17:07.440 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar 2024-06-28T15:17:07.440 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar 2024-06-28T15:17:07.500 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/118 kB Progress (1): 33/118 kB Progress (1): 49/118 kB Progress (1): 63/118 kB Progress (1): 80/118 kB Progress (1): 96/118 kB Progress (1): 113/118 kB Progress (1): 118 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar (118 kB at 2.0 MB/s) 2024-06-28T15:17:07.500 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar 2024-06-28T15:17:07.502 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar (8.2 kB at 130 kB/s) 2024-06-28T15:17:07.503 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar 2024-06-28T15:17:07.504 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/26 kB Progress (2): 16/26 kB | 14 kB Progress (3): 16/26 kB | 14 kB | 16/171 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar (14 kB at 209 kB/s) 2024-06-28T15:17:07.504 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar 2024-06-28T15:17:07.512 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 26 kB | 16/171 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar (26 kB at 354 kB/s) 2024-06-28T15:17:07.512 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar 2024-06-28T15:17:07.561 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 33/171 kB Progress (1): 49/171 kB Progress (1): 65/171 kB Progress (2): 65/171 kB | 16/146 kB Progress (2): 65/171 kB | 33/146 kB Progress (2): 65/171 kB | 49/146 kB Progress (2): 65/171 kB | 66/146 kB Progress (2): 65/171 kB | 82/146 kB Progress (2): 65/171 kB | 98/146 kB Progress (2): 65/171 kB | 113/146 kB Progress (2): 65/171 kB | 129/146 kB Progress (2): 65/171 kB | 145/146 kB Progress (2): 65/171 kB | 146 kB Progress (2): 81/171 kB | 146 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar (146 kB at 1.2 MB/s) 2024-06-28T15:17:07.561 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar 2024-06-28T15:17:07.620 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 81/171 kB | 16/136 kB Progress (3): 81/171 kB | 16/136 kB | 16/61 kB Progress (4): 81/171 kB | 16/136 kB | 16/61 kB | 16/501 kB Progress (4): 98/171 kB | 16/136 kB | 16/61 kB | 16/501 kB Progress (4): 98/171 kB | 33/136 kB | 16/61 kB | 16/501 kB Progress (4): 98/171 kB | 33/136 kB | 16/61 kB | 33/501 kB Progress (4): 98/171 kB | 33/136 kB | 33/61 kB | 33/501 kB Progress (4): 98/171 kB | 33/136 kB | 33/61 kB | 49/501 kB Progress (4): 114/171 kB | 33/136 kB | 33/61 kB | 49/501 kB Progress (4): 114/171 kB | 49/136 kB | 33/61 kB | 49/501 kB Progress (4): 114/171 kB | 49/136 kB | 33/61 kB | 66/501 kB Progress (4): 130/171 kB | 49/136 kB | 33/61 kB | 66/501 kB Progress (4): 130/171 kB | 49/136 kB | 49/61 kB | 66/501 kB Progress (4): 130/171 kB | 64/136 kB | 49/61 kB | 66/501 kB Progress (5): 130/171 kB | 64/136 kB | 49/61 kB | 66/501 kB | 15/58 kB Progress (5): 130/171 kB | 64/136 kB | 49/61 kB | 66/501 kB | 31/58 kB Progress (5): 130/171 kB | 64/136 kB | 49/61 kB | 66/501 kB | 48/58 kB Progress (5): 130/171 kB | 64/136 kB | 49/61 kB | 66/501 kB | 58 kB Progress (5): 130/171 kB | 64/136 kB | 49/61 kB | 78/501 kB | 58 kB Progress (5): 147/171 kB | 64/136 kB | 49/61 kB | 78/501 kB | 58 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar (58 kB at 320 kB/s) 2024-06-28T15:17:07.620 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar 2024-06-28T15:17:07.624 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (4): 147/171 kB | 64/136 kB | 61 kB | 78/501 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar (61 kB at 331 kB/s) 2024-06-28T15:17:07.624 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar 2024-06-28T15:17:07.639 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 147/171 kB | 80/136 kB | 78/501 kB Progress (3): 147/171 kB | 80/136 kB | 94/501 kB Progress (3): 163/171 kB | 80/136 kB | 94/501 kB Progress (3): 163/171 kB | 80/136 kB | 111/501 kB Progress (3): 171 kB | 80/136 kB | 111/501 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar (171 kB at 848 kB/s) 2024-06-28T15:17:07.680 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 96/136 kB | 111/501 kB Progress (2): 96/136 kB | 127/501 kB Progress (2): 96/136 kB | 143/501 kB Progress (2): 113/136 kB | 143/501 kB Progress (3): 113/136 kB | 143/501 kB | 16/124 kB Progress (3): 113/136 kB | 143/501 kB | 33/124 kB Progress (3): 113/136 kB | 143/501 kB | 37/124 kB Progress (3): 113/136 kB | 160/501 kB | 37/124 kB Progress (3): 113/136 kB | 160/501 kB | 53/124 kB Progress (3): 113/136 kB | 160/501 kB | 69/124 kB Progress (3): 113/136 kB | 160/501 kB | 86/124 kB Progress (3): 129/136 kB | 160/501 kB | 86/124 kB Progress (3): 129/136 kB | 160/501 kB | 102/124 kB Progress (3): 129/136 kB | 160/501 kB | 118/124 kB Progress (3): 129/136 kB | 160/501 kB | 124 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar (124 kB at 515 kB/s) 2024-06-28T15:17:07.682 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 136 kB | 160/501 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar (136 kB at 563 kB/s) 2024-06-28T15:17:07.921 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 176/501 kB Progress (2): 176/501 kB | 0/2.4 MB Progress (2): 193/501 kB | 0/2.4 MB Progress (2): 209/501 kB | 0/2.4 MB Progress (2): 209/501 kB | 0/2.4 MB Progress (2): 225/501 kB | 0/2.4 MB Progress (2): 242/501 kB | 0/2.4 MB Progress (2): 242/501 kB | 0/2.4 MB Progress (2): 258/501 kB | 0/2.4 MB Progress (2): 258/501 kB | 0.1/2.4 MB Progress (2): 275/501 kB | 0.1/2.4 MB Progress (2): 275/501 kB | 0.1/2.4 MB Progress (2): 291/501 kB | 0.1/2.4 MB Progress (2): 307/501 kB | 0.1/2.4 MB Progress (2): 307/501 kB | 0.1/2.4 MB Progress (2): 324/501 kB | 0.1/2.4 MB Progress (2): 340/501 kB | 0.1/2.4 MB Progress (2): 340/501 kB | 0.1/2.4 MB Progress (2): 356/501 kB | 0.1/2.4 MB Progress (2): 373/501 kB | 0.1/2.4 MB Progress (2): 373/501 kB | 0.1/2.4 MB Progress (2): 389/501 kB | 0.1/2.4 MB Progress (2): 389/501 kB | 0.1/2.4 MB Progress (2): 406/501 kB | 0.1/2.4 MB Progress (2): 422/501 kB | 0.1/2.4 MB Progress (2): 438/501 kB | 0.1/2.4 MB Progress (2): 455/501 kB | 0.1/2.4 MB Progress (2): 471/501 kB | 0.1/2.4 MB Progress (2): 488/501 kB | 0.1/2.4 MB Progress (2): 488/501 kB | 0.2/2.4 MB Progress (2): 488/501 kB | 0.2/2.4 MB Progress (2): 488/501 kB | 0.2/2.4 MB Progress (2): 501 kB | 0.2/2.4 MB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar (501 kB at 1.0 MB/s) 2024-06-28T15:17:10.152 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar (2.4 MB at 899 kB/s) 2024-06-28T15:17:10.332 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:17:10.333 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] --- jar:3.3.0:jar (default-jar) @ jcksum --- 2024-06-28T15:17:10.336 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom 2024-06-28T15:17:10.400 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom (4.5 kB at 71 kB/s) 2024-06-28T15:17:10.403 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom 2024-06-28T15:17:10.461 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom (4.9 kB at 84 kB/s) 2024-06-28T15:17:10.464 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom 2024-06-28T15:17:10.545 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/45 kB Progress (1): 32/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom (45 kB at 561 kB/s) 2024-06-28T15:17:10.550 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom 2024-06-28T15:17:10.615 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom (21 kB at 316 kB/s) 2024-06-28T15:17:10.622 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom 2024-06-28T15:17:10.682 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 5.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom (5.2 kB at 85 kB/s) 2024-06-28T15:17:10.687 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom 2024-06-28T15:17:10.745 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 3.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom (3.9 kB at 66 kB/s) 2024-06-28T15:17:10.751 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom 2024-06-28T15:17:10.806 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom (6.0 kB at 109 kB/s) 2024-06-28T15:17:10.811 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom 2024-06-28T15:17:10.867 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom (6.3 kB at 112 kB/s) 2024-06-28T15:17:10.872 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom 2024-06-28T15:17:10.938 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom (20 kB at 298 kB/s) 2024-06-28T15:17:10.947 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom 2024-06-28T15:17:11.010 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom (15 kB at 224 kB/s) 2024-06-28T15:17:11.015 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom 2024-06-28T15:17:11.071 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom (2.0 kB at 37 kB/s) 2024-06-28T15:17:11.075 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom 2024-06-28T15:17:11.131 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom (8.2 kB at 147 kB/s) 2024-06-28T15:17:11.134 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom 2024-06-28T15:17:11.205 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom (25 kB at 363 kB/s) 2024-06-28T15:17:11.215 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar 2024-06-28T15:17:11.293 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/36 kB Progress (1): 33/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar (36 kB at 466 kB/s) 2024-06-28T15:17:11.294 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar 2024-06-28T15:17:11.294 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar 2024-06-28T15:17:11.294 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar 2024-06-28T15:17:11.295 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar 2024-06-28T15:17:11.295 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar 2024-06-28T15:17:11.374 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 0/1.0 MB Progress (1): 0/1.0 MB Progress (1): 0/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (2): 0.1/1.0 MB | 16/79 kB Progress (2): 0.1/1.0 MB | 16/79 kB Progress (2): 0.1/1.0 MB | 16/79 kB Progress (3): 0.1/1.0 MB | 16/79 kB | 16/58 kB Progress (3): 0.2/1.0 MB | 16/79 kB | 16/58 kB Progress (4): 0.2/1.0 MB | 16/79 kB | 16/58 kB | 16/211 kB Progress (4): 0.2/1.0 MB | 16/79 kB | 16/58 kB | 16/211 kB Progress (5): 0.2/1.0 MB | 16/79 kB | 16/58 kB | 16/211 kB | 16/26 kB Progress (5): 0.2/1.0 MB | 16/79 kB | 16/58 kB | 16/211 kB | 16/26 kB Progress (5): 0.2/1.0 MB | 16/79 kB | 16/58 kB | 16/211 kB | 16/26 kB Progress (5): 0.2/1.0 MB | 16/79 kB | 16/58 kB | 16/211 kB | 16/26 kB Progress (5): 0.2/1.0 MB | 16/79 kB | 16/58 kB | 16/211 kB | 16/26 kB Progress (5): 0.3/1.0 MB | 16/79 kB | 16/58 kB | 16/211 kB | 16/26 kB Progress (5): 0.3/1.0 MB | 16/79 kB | 16/58 kB | 16/211 kB | 16/26 kB Progress (5): 0.3/1.0 MB | 16/79 kB | 16/58 kB | 16/211 kB | 16/26 kB Progress (5): 0.3/1.0 MB | 16/79 kB | 16/58 kB | 16/211 kB | 16/26 kB Progress (5): 0.3/1.0 MB | 33/79 kB | 16/58 kB | 16/211 kB | 16/26 kB Progress (5): 0.3/1.0 MB | 33/79 kB | 16/58 kB | 16/211 kB | 16/26 kB Progress (5): 0.3/1.0 MB | 33/79 kB | 16/58 kB | 16/211 kB | 16/26 kB Progress (5): 0.4/1.0 MB | 33/79 kB | 16/58 kB | 16/211 kB | 16/26 kB Progress (5): 0.4/1.0 MB | 33/79 kB | 16/58 kB | 16/211 kB | 26 kB Progress (5): 0.4/1.0 MB | 33/79 kB | 16/58 kB | 16/211 kB | 26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar (26 kB at 324 kB/s) 2024-06-28T15:17:11.375 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar 2024-06-28T15:17:11.399 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (4): 0.4/1.0 MB | 33/79 kB | 33/58 kB | 16/211 kB Progress (4): 0.4/1.0 MB | 33/79 kB | 33/58 kB | 16/211 kB Progress (4): 0.4/1.0 MB | 33/79 kB | 33/58 kB | 16/211 kB Progress (4): 0.4/1.0 MB | 33/79 kB | 33/58 kB | 33/211 kB Progress (4): 0.4/1.0 MB | 33/79 kB | 33/58 kB | 33/211 kB Progress (4): 0.4/1.0 MB | 33/79 kB | 33/58 kB | 33/211 kB Progress (4): 0.5/1.0 MB | 33/79 kB | 33/58 kB | 33/211 kB Progress (4): 0.5/1.0 MB | 33/79 kB | 33/58 kB | 33/211 kB Progress (4): 0.5/1.0 MB | 33/79 kB | 33/58 kB | 33/211 kB Progress (4): 0.5/1.0 MB | 33/79 kB | 33/58 kB | 33/211 kB Progress (4): 0.5/1.0 MB | 49/79 kB | 33/58 kB | 33/211 kB Progress (4): 0.5/1.0 MB | 49/79 kB | 33/58 kB | 33/211 kB Progress (4): 0.5/1.0 MB | 49/79 kB | 33/58 kB | 33/211 kB Progress (4): 0.5/1.0 MB | 49/79 kB | 49/58 kB | 33/211 kB Progress (4): 0.5/1.0 MB | 49/79 kB | 58 kB | 33/211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar (58 kB at 557 kB/s) 2024-06-28T15:17:11.399 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar 2024-06-28T15:17:11.459 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (4): 0.5/1.0 MB | 49/79 kB | 33/211 kB | 16/116 kB Progress (4): 0.6/1.0 MB | 49/79 kB | 33/211 kB | 16/116 kB Progress (4): 0.6/1.0 MB | 49/79 kB | 33/211 kB | 16/116 kB Progress (4): 0.6/1.0 MB | 49/79 kB | 33/211 kB | 16/116 kB Progress (4): 0.6/1.0 MB | 49/79 kB | 33/211 kB | 16/116 kB Progress (4): 0.6/1.0 MB | 49/79 kB | 33/211 kB | 16/116 kB Progress (4): 0.6/1.0 MB | 49/79 kB | 33/211 kB | 16/116 kB Progress (4): 0.6/1.0 MB | 49/79 kB | 33/211 kB | 32/116 kB Progress (4): 0.6/1.0 MB | 49/79 kB | 33/211 kB | 32/116 kB Progress (4): 0.7/1.0 MB | 49/79 kB | 33/211 kB | 32/116 kB Progress (4): 0.7/1.0 MB | 49/79 kB | 33/211 kB | 32/116 kB Progress (4): 0.7/1.0 MB | 49/79 kB | 33/211 kB | 32/116 kB Progress (4): 0.7/1.0 MB | 49/79 kB | 33/211 kB | 32/116 kB Progress (4): 0.7/1.0 MB | 49/79 kB | 33/211 kB | 32/116 kB Progress (4): 0.7/1.0 MB | 66/79 kB | 33/211 kB | 32/116 kB Progress (4): 0.7/1.0 MB | 79 kB | 33/211 kB | 32/116 kB Progress (4): 0.7/1.0 MB | 79 kB | 49/211 kB | 32/116 kB Progress (4): 0.7/1.0 MB | 79 kB | 62/211 kB | 32/116 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar (79 kB at 481 kB/s) 2024-06-28T15:17:11.472 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 0.7/1.0 MB | 62/211 kB | 32/116 kB Progress (3): 0.7/1.0 MB | 78/211 kB | 32/116 kB Progress (3): 0.8/1.0 MB | 78/211 kB | 32/116 kB Progress (4): 0.8/1.0 MB | 78/211 kB | 32/116 kB | 16/267 kB Progress (4): 0.8/1.0 MB | 95/211 kB | 32/116 kB | 16/267 kB Progress (4): 0.8/1.0 MB | 95/211 kB | 32/116 kB | 16/267 kB Progress (4): 0.8/1.0 MB | 95/211 kB | 49/116 kB | 16/267 kB Progress (4): 0.8/1.0 MB | 95/211 kB | 49/116 kB | 16/267 kB Progress (4): 0.8/1.0 MB | 95/211 kB | 49/116 kB | 16/267 kB Progress (4): 0.8/1.0 MB | 95/211 kB | 49/116 kB | 16/267 kB Progress (4): 0.8/1.0 MB | 95/211 kB | 49/116 kB | 16/267 kB Progress (4): 0.9/1.0 MB | 95/211 kB | 49/116 kB | 16/267 kB Progress (4): 0.9/1.0 MB | 95/211 kB | 49/116 kB | 16/267 kB Progress (4): 0.9/1.0 MB | 95/211 kB | 49/116 kB | 16/267 kB Progress (4): 0.9/1.0 MB | 95/211 kB | 49/116 kB | 16/267 kB Progress (4): 0.9/1.0 MB | 95/211 kB | 49/116 kB | 16/267 kB Progress (4): 0.9/1.0 MB | 95/211 kB | 49/116 kB | 16/267 kB Progress (4): 1.0/1.0 MB | 95/211 kB | 49/116 kB | 16/267 kB Progress (4): 1.0/1.0 MB | 111/211 kB | 49/116 kB | 16/267 kB Progress (4): 1.0/1.0 MB | 111/211 kB | 49/116 kB | 16/267 kB Progress (4): 1.0/1.0 MB | 111/211 kB | 49/116 kB | 16/267 kB Progress (4): 1.0/1.0 MB | 111/211 kB | 49/116 kB | 16/267 kB Progress (4): 1.0 MB | 111/211 kB | 49/116 kB | 16/267 kB Progress (4): 1.0 MB | 111/211 kB | 49/116 kB | 33/267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar (1.0 MB at 5.7 MB/s) 2024-06-28T15:17:11.527 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (3): 111/211 kB | 49/116 kB | 49/267 kB Progress (3): 111/211 kB | 65/116 kB | 49/267 kB Progress (3): 128/211 kB | 65/116 kB | 49/267 kB Progress (3): 128/211 kB | 65/116 kB | 62/267 kB Progress (3): 128/211 kB | 81/116 kB | 62/267 kB Progress (3): 144/211 kB | 81/116 kB | 62/267 kB Progress (3): 144/211 kB | 81/116 kB | 78/267 kB Progress (3): 144/211 kB | 98/116 kB | 78/267 kB Progress (3): 144/211 kB | 98/116 kB | 94/267 kB Progress (3): 144/211 kB | 114/116 kB | 94/267 kB Progress (3): 144/211 kB | 116 kB | 94/267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar (116 kB at 500 kB/s) 2024-06-28T15:17:11.587 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 160/211 kB | 94/267 kB Progress (2): 160/211 kB | 111/267 kB Progress (2): 160/211 kB | 127/267 kB Progress (2): 177/211 kB | 127/267 kB Progress (2): 177/211 kB | 144/267 kB Progress (2): 177/211 kB | 160/267 kB Progress (2): 193/211 kB | 160/267 kB Progress (2): 193/211 kB | 176/267 kB Progress (2): 209/211 kB | 176/267 kB Progress (2): 211 kB | 176/267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar (211 kB at 717 kB/s) 2024-06-28T15:17:11.650 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 193/267 kB Progress (1): 209/267 kB Progress (1): 225/267 kB Progress (1): 242/267 kB Progress (1): 258/267 kB Progress (1): 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar (267 kB at 755 kB/s) 2024-06-28T15:17:11.875 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Building jar: /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/target/jcksum-1.0.jar 2024-06-28T15:17:11.905 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] ------------------------------------------------------------------------ 2024-06-28T15:17:11.905 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] BUILD SUCCESS 2024-06-28T15:17:11.905 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] ------------------------------------------------------------------------ 2024-06-28T15:17:11.906 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Total time: 25.139 s 2024-06-28T15:17:11.906 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Finished at: 2024-06-28T15:17:11Z 2024-06-28T15:17:11.906 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] ------------------------------------------------------------------------ 2024-06-28T15:17:11.936 INFO:tasks.workunit.client.0.smithi178.stderr:+ ./mvnw test -Dtest=PutObjects 2024-06-28T15:17:11.942 INFO:tasks.workunit.client.0.smithi178.stdout:Warning: JAVA_HOME environment variable is not set. 2024-06-28T15:17:13.197 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Scanning for projects... 2024-06-28T15:17:13.324 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:17:13.324 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2024-06-28T15:17:13.324 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Building jcksum 1.0 2024-06-28T15:17:13.325 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] from pom.xml 2024-06-28T15:17:13.325 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2024-06-28T15:17:13.727 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:17:13.727 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2024-06-28T15:17:13.864 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2024-06-28T15:17:13.865 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:17:13.865 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2024-06-28T15:17:13.965 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Nothing to compile - all classes are up to date 2024-06-28T15:17:13.965 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:17:13.965 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2024-06-28T15:17:13.967 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2024-06-28T15:17:13.967 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:17:13.967 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2024-06-28T15:17:13.971 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Nothing to compile - all classes are up to date 2024-06-28T15:17:13.971 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:17:13.971 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2024-06-28T15:17:14.185 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider 2024-06-28T15:17:14.435 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom 2024-06-28T15:17:14.781 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.4/4.7 kB Progress (1): 2.8/4.7 kB Progress (1): 4.1/4.7 kB Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom (4.7 kB at 13 kB/s) 2024-06-28T15:17:14.794 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom 2024-06-28T15:17:14.850 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.4/2.6 kB Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom (2.6 kB at 47 kB/s) 2024-06-28T15:17:14.862 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom 2024-06-28T15:17:14.924 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.4/2.8 kB Progress (1): 2.8/2.8 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom (2.8 kB at 46 kB/s) 2024-06-28T15:17:14.935 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom 2024-06-28T15:17:14.991 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.4/3.0 kB Progress (1): 2.8/3.0 kB Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom (3.0 kB at 53 kB/s) 2024-06-28T15:17:14.997 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom 2024-06-28T15:17:15.051 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.4/3.2 kB Progress (1): 2.8/3.2 kB Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom (3.2 kB at 59 kB/s) 2024-06-28T15:17:15.057 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom 2024-06-28T15:17:15.110 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.4/1.7 kB Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom (1.7 kB at 32 kB/s) 2024-06-28T15:17:15.115 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom 2024-06-28T15:17:15.169 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.4/2.8 kB Progress (1): 2.8/2.8 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom (2.8 kB at 52 kB/s) 2024-06-28T15:17:15.181 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar 2024-06-28T15:17:15.251 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.4/27 kB Progress (1): 2.8/27 kB Progress (1): 4.1/27 kB Progress (1): 5.5/27 kB Progress (1): 6.9/27 kB Progress (1): 8.3/27 kB Progress (1): 9.7/27 kB Progress (1): 11/27 kB Progress (1): 12/27 kB Progress (1): 14/27 kB Progress (1): 15/27 kB Progress (1): 17/27 kB Progress (1): 18/27 kB Progress (1): 19/27 kB Progress (1): 21/27 kB Progress (1): 22/27 kB Progress (1): 23/27 kB Progress (1): 25/27 kB Progress (1): 26/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar (27 kB at 382 kB/s) 2024-06-28T15:17:15.255 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar 2024-06-28T15:17:15.255 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar 2024-06-28T15:17:15.256 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar 2024-06-28T15:17:15.257 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar 2024-06-28T15:17:15.257 INFO:tasks.workunit.client.0.smithi178.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar 2024-06-28T15:17:15.319 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 16/18 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar (18 kB at 258 kB/s) 2024-06-28T15:17:15.431 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 1.4/7.7 kB Progress (2): 1.4/7.7 kB | 1.4/169 kB Progress (2): 2.8/7.7 kB | 1.4/169 kB Progress (3): 2.8/7.7 kB | 1.4/169 kB | 1.4/189 kB Progress (3): 4.1/7.7 kB | 1.4/169 kB | 1.4/189 kB Progress (3): 4.1/7.7 kB | 2.8/169 kB | 1.4/189 kB Progress (3): 5.5/7.7 kB | 2.8/169 kB | 1.4/189 kB Progress (3): 5.5/7.7 kB | 2.8/169 kB | 2.8/189 kB Progress (3): 6.9/7.7 kB | 2.8/169 kB | 2.8/189 kB Progress (3): 6.9/7.7 kB | 4.1/169 kB | 2.8/189 kB Progress (3): 7.7 kB | 4.1/169 kB | 2.8/189 kB Progress (3): 7.7 kB | 4.1/169 kB | 4.1/189 kB Progress (3): 7.7 kB | 5.5/169 kB | 4.1/189 kB Progress (3): 7.7 kB | 5.5/169 kB | 5.5/189 kB Progress (3): 7.7 kB | 6.9/169 kB | 5.5/189 kB Progress (3): 7.7 kB | 6.9/169 kB | 6.9/189 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar (7.7 kB at 43 kB/s) 2024-06-28T15:17:15.544 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 8.3/169 kB | 6.9/189 kB Progress (2): 8.3/169 kB | 8.3/189 kB Progress (2): 9.7/169 kB | 8.3/189 kB Progress (2): 9.7/169 kB | 9.7/189 kB Progress (2): 11/169 kB | 9.7/189 kB Progress (2): 11/169 kB | 11/189 kB Progress (2): 12/169 kB | 11/189 kB Progress (2): 12/169 kB | 12/189 kB Progress (2): 14/169 kB | 12/189 kB Progress (2): 14/169 kB | 14/189 kB Progress (2): 15/169 kB | 14/189 kB Progress (2): 15/169 kB | 15/189 kB Progress (2): 16/169 kB | 15/189 kB Progress (2): 16/169 kB | 16/189 kB Progress (2): 16/169 kB | 18/189 kB Progress (2): 16/169 kB | 19/189 kB Progress (2): 33/169 kB | 19/189 kB Progress (2): 33/169 kB | 21/189 kB Progress (2): 33/169 kB | 22/189 kB Progress (2): 33/169 kB | 23/189 kB Progress (2): 33/169 kB | 25/189 kB Progress (2): 33/169 kB | 26/189 kB Progress (2): 33/169 kB | 27/189 kB Progress (2): 33/169 kB | 29/189 kB Progress (2): 33/169 kB | 30/189 kB Progress (2): 33/169 kB | 32/189 kB Progress (2): 33/169 kB | 33/189 kB Progress (2): 33/169 kB | 34/189 kB Progress (2): 33/169 kB | 36/189 kB Progress (2): 33/169 kB | 37/189 kB Progress (2): 33/169 kB | 38/189 kB Progress (2): 33/169 kB | 40/189 kB Progress (2): 33/169 kB | 41/189 kB Progress (2): 33/169 kB | 43/189 kB Progress (2): 33/169 kB | 44/189 kB Progress (2): 33/169 kB | 45/189 kB Progress (2): 49/169 kB | 45/189 kB Progress (2): 49/169 kB | 47/189 kB Progress (2): 49/169 kB | 48/189 kB Progress (2): 49/169 kB | 49/189 kB Progress (2): 49/169 kB | 51/189 kB Progress (2): 49/169 kB | 52/189 kB Progress (2): 49/169 kB | 54/189 kB Progress (2): 49/169 kB | 55/189 kB Progress (2): 66/169 kB | 55/189 kB Progress (2): 66/169 kB | 56/189 kB Progress (2): 66/169 kB | 58/189 kB Progress (2): 66/169 kB | 59/189 kB Progress (2): 66/169 kB | 61/189 kB Progress (2): 66/169 kB | 61/189 kB Progress (3): 66/169 kB | 61/189 kB | 16/103 kB Progress (3): 82/169 kB | 61/189 kB | 16/103 kB Progress (3): 82/169 kB | 61/189 kB | 33/103 kB Progress (3): 98/169 kB | 61/189 kB | 33/103 kB Progress (3): 98/169 kB | 78/189 kB | 33/103 kB Progress (3): 115/169 kB | 78/189 kB | 33/103 kB Progress (3): 115/169 kB | 94/189 kB | 33/103 kB Progress (3): 131/169 kB | 94/189 kB | 33/103 kB Progress (3): 131/169 kB | 111/189 kB | 33/103 kB Progress (3): 147/169 kB | 111/189 kB | 33/103 kB Progress (3): 147/169 kB | 127/189 kB | 33/103 kB Progress (3): 164/169 kB | 127/189 kB | 33/103 kB Progress (3): 169 kB | 127/189 kB | 33/103 kB Progress (3): 169 kB | 143/189 kB | 33/103 kB Progress (3): 169 kB | 143/189 kB | 49/103 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar (169 kB at 582 kB/s) 2024-06-28T15:17:15.553 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (2): 160/189 kB | 49/103 kB Progress (2): 176/189 kB | 49/103 kB Progress (2): 176/189 kB | 66/103 kB Progress (2): 189 kB | 66/103 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar (189 kB at 633 kB/s) 2024-06-28T15:17:15.592 INFO:tasks.workunit.client.0.smithi178.stdout:Progress (1): 82/103 kB Progress (1): 98/103 kB Progress (1): 103 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar (103 kB at 307 kB/s) 2024-06-28T15:17:15.629 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:17:15.629 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] ------------------------------------------------------- 2024-06-28T15:17:15.630 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] T E S T S 2024-06-28T15:17:15.630 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] ------------------------------------------------------- 2024-06-28T15:17:16.407 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Running io.ceph.jcksum.PutObjects 2024-06-28T15:17:16.482 INFO:tasks.workunit.client.0.smithi178.stdout:PutObjects.java: starting test run: 2024-06-28T15:17:16.482 INFO:tasks.workunit.client.0.smithi178.stdout: AccessKey=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-06-28T15:17:16.482 INFO:tasks.workunit.client.0.smithi178.stdout: SecretKey=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-06-28T15:17:16.482 INFO:tasks.workunit.client.0.smithi178.stdout: EndpointUrl=http://smithi178.front.sepia.ceph.com:80 2024-06-28T15:17:16.483 INFO:tasks.workunit.client.0.smithi178.stderr:[main] WARN software.amazon.awssdk.http.apache.ApacheHttpClient - SSL Certificate verification is disabled. This is not a safe setting and should only be used for testing. 2024-06-28T15:17:16.679 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: Environment Variables Read 2024-06-28T15:17:17.082 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: S3 Client Initialized 2024-06-28T15:17:17.082 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: Generating File 2024-06-28T15:17:17.082 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: File Generated 2024-06-28T15:17:17.082 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: Generating File 2024-06-28T15:17:17.082 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: File Generated 2024-06-28T15:17:17.082 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: Generating File 2024-06-28T15:17:17.082 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: File Generated 2024-06-28T15:17:17.082 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: Generating File 2024-06-28T15:17:17.082 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: File Generated 2024-06-28T15:17:17.082 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: Generating File 2024-06-28T15:17:17.083 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: File Generated 2024-06-28T15:17:17.083 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: Generating File 2024-06-28T15:17:17.083 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: File Generated 2024-06-28T15:17:17.083 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: Generated Big Files 2024-06-28T15:17:17.498 INFO:tasks.workunit.client.0.smithi178.stdout:HeadBucketResponse() 2024-06-28T15:17:17.498 INFO:tasks.workunit.client.0.smithi178.stdout:sheik is ready 2024-06-28T15:17:17.498 INFO:tasks.workunit.client.0.smithi178.stdout:DEBUG: Test Bucket Created 2024-06-28T15:17:17.598 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileNoCksum called with file-0b 2024-06-28T15:17:17.999 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileNoCksum called with file-8b 2024-06-28T15:17:18.400 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileNoCksum called with file-200b 2024-06-28T15:17:18.600 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileNoCksum called with file-21983b 2024-06-28T15:17:19.102 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileNoCksum called with file-5519b 2024-06-28T15:17:19.503 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileNoCksum called with file-204329b 2024-06-28T15:17:19.904 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileNoCksum called with file-256k 2024-06-28T15:17:20.405 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileNoCksum called with file-1m 2024-06-28T15:17:21.006 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileNoCksum called with file-1038757b 2024-06-28T15:17:21.909 INFO:tasks.workunit.client.0.smithi178.stdout:mpuObjectFromFileNoCksum called with file-5m 2024-06-28T15:17:26.920 INFO:tasks.workunit.client.0.smithi178.stdout:mpuObjectFromFileNoCksum called with file-10m 2024-06-28T15:17:30.728 INFO:tasks.workunit.client.0.smithi178.stdout:mpuObjectFromFileNoCksum called with file-100m 2024-06-28T15:18:07.247 INFO:tasks.workunit.client.0.smithi178.stdout:arg: file-0b 2024-06-28T15:18:07.248 INFO:tasks.workunit.client.0.smithi178.stdout:arg: file-8b 2024-06-28T15:18:07.249 INFO:tasks.workunit.client.0.smithi178.stdout:arg: file-200b 2024-06-28T15:18:07.250 INFO:tasks.workunit.client.0.smithi178.stdout:arg: file-21983b 2024-06-28T15:18:07.251 INFO:tasks.workunit.client.0.smithi178.stdout:arg: file-5519b 2024-06-28T15:18:07.253 INFO:tasks.workunit.client.0.smithi178.stdout:arg: file-204329b 2024-06-28T15:18:07.254 INFO:tasks.workunit.client.0.smithi178.stdout:arg: file-256k 2024-06-28T15:18:07.255 INFO:tasks.workunit.client.0.smithi178.stdout:arg: file-1m 2024-06-28T15:18:07.256 INFO:tasks.workunit.client.0.smithi178.stdout:arg: file-1038757b 2024-06-28T15:18:07.291 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileCksum called with file-0b 2024-06-28T15:18:07.491 INFO:tasks.workunit.client.0.smithi178.stdout:PutObjectResponse 2024-06-28T15:18:07.591 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileCksum called with file-8b 2024-06-28T15:18:07.691 INFO:tasks.workunit.client.0.smithi178.stdout:PutObjectResponse 2024-06-28T15:18:07.792 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileCksum called with file-200b 2024-06-28T15:18:07.892 INFO:tasks.workunit.client.0.smithi178.stdout:PutObjectResponse 2024-06-28T15:18:08.092 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileCksum called with file-21983b 2024-06-28T15:18:08.192 INFO:tasks.workunit.client.0.smithi178.stdout:PutObjectResponse 2024-06-28T15:18:08.393 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileCksum called with file-5519b 2024-06-28T15:18:08.493 INFO:tasks.workunit.client.0.smithi178.stdout:PutObjectResponse 2024-06-28T15:18:08.593 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileCksum called with file-204329b 2024-06-28T15:18:08.793 INFO:tasks.workunit.client.0.smithi178.stdout:PutObjectResponse 2024-06-28T15:18:08.994 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileCksum called with file-256k 2024-06-28T15:18:09.194 INFO:tasks.workunit.client.0.smithi178.stdout:PutObjectResponse 2024-06-28T15:18:09.294 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileCksum called with file-1m 2024-06-28T15:18:09.695 INFO:tasks.workunit.client.0.smithi178.stdout:PutObjectResponse 2024-06-28T15:18:09.895 INFO:tasks.workunit.client.0.smithi178.stdout:putObjectFromFileCksum called with file-1038757b 2024-06-28T15:18:10.596 INFO:tasks.workunit.client.0.smithi178.stdout:PutObjectResponse 2024-06-28T15:18:10.797 INFO:tasks.workunit.client.0.smithi178.stdout:mpuObjectFromFileCksum called with file-5m 2024-06-28T15:18:14.100 INFO:tasks.workunit.client.0.smithi178.stdout:mpuObjectFromFileCksum called with file-10m 2024-06-28T15:18:17.905 INFO:tasks.workunit.client.0.smithi178.stdout:mpuObjectFromFileCksum called with file-100m 2024-06-28T15:19:14.327 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 117.9 s -- in io.ceph.jcksum.PutObjects 2024-06-28T15:19:14.353 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:19:14.353 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Results: 2024-06-28T15:19:14.353 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:19:14.353 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0 2024-06-28T15:19:14.353 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] 2024-06-28T15:19:14.356 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] ------------------------------------------------------------------------ 2024-06-28T15:19:14.356 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] BUILD SUCCESS 2024-06-28T15:19:14.356 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] ------------------------------------------------------------------------ 2024-06-28T15:19:14.357 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Total time: 02:01 min 2024-06-28T15:19:14.357 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] Finished at: 2024-06-28T15:19:14Z 2024-06-28T15:19:14.357 INFO:tasks.workunit.client.0.smithi178.stdout:[INFO] ------------------------------------------------------------------------ 2024-06-28T15:19:14.376 INFO:tasks.workunit.client.0.smithi178.stderr:+ exit 2024-06-28T15:19:14.377 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:19:14.377 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:19:14.429 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'cls/test_cls_user.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh'] on client.0... 2024-06-28T15:19:14.430 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-28T15:19:14.799 DEBUG:teuthology.parallel:result is None 2024-06-28T15:19:14.799 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:19:14.811 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:19:14.811 DEBUG:teuthology.orchestra.run.smithi178:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:19:14.856 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:19:14.856 INFO:teuthology.run_tasks:Running task workunit... 2024-06-28T15:19:14.866 INFO:tasks.workunit:Pulling workunits from ref ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T15:19:14.866 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-28T15:19:14.866 DEBUG:teuthology.orchestra.run.smithi178:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:19:14.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T15:19:14.904 INFO:teuthology.orchestra.run.smithi178.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-06-28T15:19:14.905 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:19:14.952 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-06-28T15:19:14.953 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-06-28T15:19:15.000 INFO:tasks.workunit:timeout=3h 2024-06-28T15:19:15.001 INFO:tasks.workunit:cleanup=True 2024-06-28T15:19:15.001 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T15:19:15.050 INFO:tasks.workunit.client.0.smithi178.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-28T15:20:08.359 INFO:tasks.workunit.client.0.smithi178.stderr:Note: switching to 'ec2a38ba1b69b5357de8f0e8f96b00b4a577169d'. 2024-06-28T15:20:08.360 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:20:08.360 INFO:tasks.workunit.client.0.smithi178.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-28T15:20:08.360 INFO:tasks.workunit.client.0.smithi178.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-28T15:20:08.360 INFO:tasks.workunit.client.0.smithi178.stderr:state without impacting any branches by switching back to a branch. 2024-06-28T15:20:08.360 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:20:08.360 INFO:tasks.workunit.client.0.smithi178.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-28T15:20:08.360 INFO:tasks.workunit.client.0.smithi178.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-28T15:20:08.360 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:20:08.360 INFO:tasks.workunit.client.0.smithi178.stderr: git switch -c 2024-06-28T15:20:08.360 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:20:08.360 INFO:tasks.workunit.client.0.smithi178.stderr:Or undo this operation with: 2024-06-28T15:20:08.360 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:20:08.361 INFO:tasks.workunit.client.0.smithi178.stderr: git switch - 2024-06-28T15:20:08.361 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:20:08.361 INFO:tasks.workunit.client.0.smithi178.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-28T15:20:08.361 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:20:08.361 INFO:tasks.workunit.client.0.smithi178.stderr:HEAD is now at ec2a38ba1b6 Merge branches 'pr-56792', 'pr-57537', 'pr-57858', 'pr-57985', 'pr-58081' and 'pr-58288' into wip-cbodley-testing 2024-06-28T15:20:08.367 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-28T15:20:08.413 INFO:tasks.workunit.client.0.smithi178.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-28T15:20:08.416 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-28T15:20:08.416 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-28T15:20:08.473 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-28T15:20:08.529 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-28T15:20:08.574 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-28T15:20:08.575 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-28T15:20:08.576 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-28T15:20:08.615 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-28T15:20:08.618 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T15:20:08.618 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-28T15:20:08.664 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_s3_mp_reupload.sh on client.0... 2024-06-28T15:20:08.665 INFO:tasks.workunit:Running workunit rgw/test_rgw_s3_mp_reupload.sh... 2024-06-28T15:20:08.666 DEBUG:teuthology.orchestra.run.smithi178:workunit test rgw/test_rgw_s3_mp_reupload.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh 2024-06-28T15:20:08.718 INFO:tasks.workunit.client.0.smithi178.stdout:INFO: fully qualified domain name: smithi178.front.sepia.ceph.com 2024-06-28T15:20:08.719 INFO:tasks.workunit.client.0.smithi178.stdout:== TESTING *COMPLETING* MULTIPART UPLOAD WITH RE-UPLOADS == 2024-06-28T15:20:08.719 INFO:tasks.workunit.client.0.smithi178.stdout:== TESTING NON-VERSIONED BUCKET == 2024-06-28T15:20:11.969 INFO:tasks.workunit.client.0.smithi178.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-06-28T15:20:12.326 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting pip 2024-06-28T15:20:12.332 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached pip-24.1.1-py3-none-any.whl (1.8 MB) 2024-06-28T15:20:12.368 INFO:tasks.workunit.client.0.smithi178.stdout:Installing collected packages: pip 2024-06-28T15:20:12.368 INFO:tasks.workunit.client.0.smithi178.stdout: Attempting uninstall: pip 2024-06-28T15:20:12.368 INFO:tasks.workunit.client.0.smithi178.stdout: Found existing installation: pip 22.0.2 2024-06-28T15:20:12.633 INFO:tasks.workunit.client.0.smithi178.stdout: Uninstalling pip-22.0.2: 2024-06-28T15:20:12.639 INFO:tasks.workunit.client.0.smithi178.stdout: Successfully uninstalled pip-22.0.2 2024-06-28T15:20:13.635 INFO:tasks.workunit.client.0.smithi178.stdout:Successfully installed pip-24.1.1 2024-06-28T15:20:14.794 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting boto3 2024-06-28T15:20:14.796 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached boto3-1.34.135-py3-none-any.whl.metadata (6.6 kB) 2024-06-28T15:20:15.401 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting botocore<1.35.0,>=1.34.135 (from boto3) 2024-06-28T15:20:15.402 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached botocore-1.34.135-py3-none-any.whl.metadata (5.7 kB) 2024-06-28T15:20:15.482 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-06-28T15:20:15.483 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-06-28T15:20:15.586 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-06-28T15:20:15.587 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached s3transfer-0.10.2-py3-none-any.whl.metadata (1.7 kB) 2024-06-28T15:20:15.718 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.135->boto3) 2024-06-28T15:20:15.721 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-06-28T15:20:15.894 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.35.0,>=1.34.135->boto3) 2024-06-28T15:20:15.896 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB) 2024-06-28T15:20:16.212 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.135->boto3) 2024-06-28T15:20:16.213 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-06-28T15:20:16.217 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached boto3-1.34.135-py3-none-any.whl (139 kB) 2024-06-28T15:20:16.219 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached botocore-1.34.135-py3-none-any.whl (12.3 MB) 2024-06-28T15:20:16.244 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-06-28T15:20:16.245 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached s3transfer-0.10.2-py3-none-any.whl (82 kB) 2024-06-28T15:20:16.247 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-06-28T15:20:16.248 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached urllib3-2.2.2-py3-none-any.whl (121 kB) 2024-06-28T15:20:16.250 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-06-28T15:20:16.325 INFO:tasks.workunit.client.0.smithi178.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-06-28T15:20:17.703 INFO:tasks.workunit.client.0.smithi178.stdout:Successfully installed boto3-1.34.135 botocore-1.34.135 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.2 six-1.16.0 urllib3-2.2.2 2024-06-28T15:20:18.225 INFO:tasks.workunit.client.0.smithi178.stdout:INFO: creating user 2024-06-28T15:20:18.367 INFO:tasks.workunit.client.0.smithi178.stdout:{ 2024-06-28T15:20:18.367 INFO:tasks.workunit.client.0.smithi178.stdout: "user_id": "testid", 2024-06-28T15:20:18.367 INFO:tasks.workunit.client.0.smithi178.stdout: "display_name": "M. Tester", 2024-06-28T15:20:18.367 INFO:tasks.workunit.client.0.smithi178.stdout: "email": "tester@ceph.com", 2024-06-28T15:20:18.368 INFO:tasks.workunit.client.0.smithi178.stdout: "suspended": 0, 2024-06-28T15:20:18.368 INFO:tasks.workunit.client.0.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:20:18.368 INFO:tasks.workunit.client.0.smithi178.stdout: "subusers": [], 2024-06-28T15:20:18.368 INFO:tasks.workunit.client.0.smithi178.stdout: "keys": [ 2024-06-28T15:20:18.368 INFO:tasks.workunit.client.0.smithi178.stdout: { 2024-06-28T15:20:18.368 INFO:tasks.workunit.client.0.smithi178.stdout: "user": "testid", 2024-06-28T15:20:18.368 INFO:tasks.workunit.client.0.smithi178.stdout: "access_key": "0555b35654ad1656d804", 2024-06-28T15:20:18.368 INFO:tasks.workunit.client.0.smithi178.stdout: "secret_key": "h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==", 2024-06-28T15:20:18.368 INFO:tasks.workunit.client.0.smithi178.stdout: "active": true, 2024-06-28T15:20:18.368 INFO:tasks.workunit.client.0.smithi178.stdout: "create_date": "2024-06-28T15:20:18.374878Z" 2024-06-28T15:20:18.368 INFO:tasks.workunit.client.0.smithi178.stdout: } 2024-06-28T15:20:18.368 INFO:tasks.workunit.client.0.smithi178.stdout: ], 2024-06-28T15:20:18.368 INFO:tasks.workunit.client.0.smithi178.stdout: "swift_keys": [], 2024-06-28T15:20:18.369 INFO:tasks.workunit.client.0.smithi178.stdout: "caps": [], 2024-06-28T15:20:18.369 INFO:tasks.workunit.client.0.smithi178.stdout: "op_mask": "read, write, delete", 2024-06-28T15:20:18.369 INFO:tasks.workunit.client.0.smithi178.stdout: "default_placement": "", 2024-06-28T15:20:18.369 INFO:tasks.workunit.client.0.smithi178.stdout: "default_storage_class": "", 2024-06-28T15:20:18.369 INFO:tasks.workunit.client.0.smithi178.stdout: "placement_tags": [], 2024-06-28T15:20:18.369 INFO:tasks.workunit.client.0.smithi178.stdout: "bucket_quota": { 2024-06-28T15:20:18.369 INFO:tasks.workunit.client.0.smithi178.stdout: "enabled": false, 2024-06-28T15:20:18.369 INFO:tasks.workunit.client.0.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:20:18.369 INFO:tasks.workunit.client.0.smithi178.stdout: "max_size": -1, 2024-06-28T15:20:18.369 INFO:tasks.workunit.client.0.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:20:18.369 INFO:tasks.workunit.client.0.smithi178.stdout: "max_objects": -1 2024-06-28T15:20:18.369 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:20:18.369 INFO:tasks.workunit.client.0.smithi178.stdout: "user_quota": { 2024-06-28T15:20:18.369 INFO:tasks.workunit.client.0.smithi178.stdout: "enabled": false, 2024-06-28T15:20:18.370 INFO:tasks.workunit.client.0.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:20:18.370 INFO:tasks.workunit.client.0.smithi178.stdout: "max_size": -1, 2024-06-28T15:20:18.370 INFO:tasks.workunit.client.0.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:20:18.370 INFO:tasks.workunit.client.0.smithi178.stdout: "max_objects": -1 2024-06-28T15:20:18.370 INFO:tasks.workunit.client.0.smithi178.stdout: }, 2024-06-28T15:20:18.370 INFO:tasks.workunit.client.0.smithi178.stdout: "temp_url_keys": [], 2024-06-28T15:20:18.370 INFO:tasks.workunit.client.0.smithi178.stdout: "type": "rgw", 2024-06-28T15:20:18.370 INFO:tasks.workunit.client.0.smithi178.stdout: "mfa_ids": [], 2024-06-28T15:20:18.370 INFO:tasks.workunit.client.0.smithi178.stdout: "account_id": "", 2024-06-28T15:20:18.370 INFO:tasks.workunit.client.0.smithi178.stdout: "path": "/", 2024-06-28T15:20:18.370 INFO:tasks.workunit.client.0.smithi178.stdout: "create_date": "2024-06-28T15:20:18.374861Z", 2024-06-28T15:20:18.370 INFO:tasks.workunit.client.0.smithi178.stdout: "tags": [], 2024-06-28T15:20:18.370 INFO:tasks.workunit.client.0.smithi178.stdout: "group_ids": [] 2024-06-28T15:20:18.370 INFO:tasks.workunit.client.0.smithi178.stdout:} 2024-06-28T15:20:18.371 INFO:tasks.workunit.client.0.smithi178.stdout: 2024-06-28T15:20:18.741 INFO:tasks.workunit.client.0.smithi178.stderr:5+0 records in 2024-06-28T15:20:18.741 INFO:tasks.workunit.client.0.smithi178.stderr:5+0 records out 2024-06-28T15:20:18.742 INFO:tasks.workunit.client.0.smithi178.stderr:5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.0234899 s, 223 MB/s 2024-06-28T15:20:32.119 INFO:tasks.workunit.client.0.smithi178.stdout:bucket nams is reupload-bkt-111820 2024-06-28T15:20:32.120 INFO:tasks.workunit.client.0.smithi178.stdout:endpoint is http://smithi178.front.sepia.ceph.com:80 2024-06-28T15:20:32.120 INFO:tasks.workunit.client.0.smithi178.stdout:start UploadId=2~8P83fT6XhS1-S-a8p8BOKq7BGYkY8H5 2024-06-28T15:20:32.120 INFO:tasks.workunit.client.0.smithi178.stdout:phase 1 uploaded part {'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 1} 2024-06-28T15:20:32.120 INFO:tasks.workunit.client.0.smithi178.stdout:phase 1 uploaded part {'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 2} 2024-06-28T15:20:32.120 INFO:tasks.workunit.client.0.smithi178.stdout:phase 2 uploaded part {'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 1} 2024-06-28T15:20:32.120 INFO:tasks.workunit.client.0.smithi178.stdout:phase 2 uploaded part {'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 1} 2024-06-28T15:20:32.120 INFO:tasks.workunit.client.0.smithi178.stdout:phase 2 uploaded part {'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 1} 2024-06-28T15:20:32.120 INFO:tasks.workunit.client.0.smithi178.stdout:phase 2 uploaded part {'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 1} 2024-06-28T15:20:32.120 INFO:tasks.workunit.client.0.smithi178.stdout:phase 2 uploaded part {'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 1} 2024-06-28T15:20:32.120 INFO:tasks.workunit.client.0.smithi178.stdout:phase 2 uploaded part {'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 1} 2024-06-28T15:20:32.120 INFO:tasks.workunit.client.0.smithi178.stdout:phase 2 uploaded part {'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 1} 2024-06-28T15:20:32.120 INFO:tasks.workunit.client.0.smithi178.stdout:phase 2 uploaded part {'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 1} 2024-06-28T15:20:32.121 INFO:tasks.workunit.client.0.smithi178.stdout:phase 2 uploaded part {'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 1} 2024-06-28T15:20:32.121 INFO:tasks.workunit.client.0.smithi178.stdout:phase 2 uploaded part {'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 1} 2024-06-28T15:20:32.121 INFO:tasks.workunit.client.0.smithi178.stdout:phase 2 uploaded part {'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 1} 2024-06-28T15:20:32.121 INFO:tasks.workunit.client.0.smithi178.stdout:completing multipart upload, parts=[{'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 1}, {'ETag': '"94fcd9fffdf32c34228870a638fe1c0a"', 'PartNumber': 2}] 2024-06-28T15:20:33.425 INFO:tasks.workunit.client.0.smithi178.stdout:radosgw-admin bucket radoslist: 2024-06-28T15:20:33.425 INFO:tasks.workunit.client.0.smithi178.stdout:843e24ce-085a-4043-8f55-38e05370e790.4686.4_mpu_test4 2024-06-28T15:20:33.425 INFO:tasks.workunit.client.0.smithi178.stdout:843e24ce-085a-4043-8f55-38e05370e790.4686.4__multipart_mpu_test4.2~8P83fT6XhS1-S-a8p8BOKq7BGYkY8H5.2 2024-06-28T15:20:33.425 INFO:tasks.workunit.client.0.smithi178.stdout:843e24ce-085a-4043-8f55-38e05370e790.4686.4__multipart_mpu_test4.-8IL4E_L-laRV1FU9DNVI-8tvutbTNcw.1 2024-06-28T15:20:33.515 INFO:tasks.workunit.client.0.smithi178.stdout:rados ls: 2024-06-28T15:20:33.515 INFO:tasks.workunit.client.0.smithi178.stdout:843e24ce-085a-4043-8f55-38e05370e790.4686.4_mpu_test4 2024-06-28T15:20:33.515 INFO:tasks.workunit.client.0.smithi178.stdout:843e24ce-085a-4043-8f55-38e05370e790.4686.4__multipart_mpu_test4.2~8P83fT6XhS1-S-a8p8BOKq7BGYkY8H5.2 2024-06-28T15:20:33.515 INFO:tasks.workunit.client.0.smithi178.stdout:843e24ce-085a-4043-8f55-38e05370e790.4686.4__multipart_mpu_test4.-8IL4E_L-laRV1FU9DNVI-8tvutbTNcw.1 2024-06-28T15:20:33.535 INFO:tasks.workunit.client.0.smithi178.stdout:== PASSED == 2024-06-28T15:20:33.536 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:20:33.536 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:20:33.592 INFO:tasks.workunit:Stopping ['rgw/test_rgw_s3_mp_reupload.sh'] on client.0... 2024-06-28T15:20:33.592 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-28T15:20:33.985 DEBUG:teuthology.parallel:result is None 2024-06-28T15:20:33.986 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:20:33.997 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:20:33.997 DEBUG:teuthology.orchestra.run.smithi178:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:20:34.048 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:20:34.048 INFO:teuthology.run_tasks:Running task ragweed... 2024-06-28T15:20:34.064 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'default-branch': 'ceph-master', 'rgw_server': 'client.0', 'stages': 'prepare,check'}} 2024-06-28T15:20:34.064 INFO:tasks.ragweed:Downloading ragweed... 2024-06-28T15:20:34.065 INFO:tasks.ragweed:Using branch 'wip-cbodley-testing' for ragweed 2024-06-28T15:20:34.065 DEBUG:teuthology.orchestra.run.smithi178:> git clone -b wip-cbodley-testing https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2024-06-28T15:20:34.096 INFO:teuthology.orchestra.run.smithi178.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2024-06-28T15:20:34.153 INFO:teuthology.orchestra.run.smithi178.stderr:fatal: Remote branch wip-cbodley-testing not found in upstream origin 2024-06-28T15:20:34.154 DEBUG:teuthology.orchestra.run:got remote process result: 128 2024-06-28T15:20:34.155 INFO:tasks.ragweed:Using branch 'ceph-master' for ragweed 2024-06-28T15:20:34.155 DEBUG:teuthology.orchestra.run.smithi178:> git clone -b ceph-master https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2024-06-28T15:20:34.201 INFO:teuthology.orchestra.run.smithi178.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2024-06-28T15:20:34.378 INFO:tasks.ragweed:Creating rgw users... 2024-06-28T15:20:34.378 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2024-06-28T15:20:34.378 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key DCOBIBMLAMCBAZSEXBWO --secret wHfI2xJOzfYzgc+P5oSgzIXWSiQk5t9DpTJpAWgHmxNJTiSgh6/iMA== --email ragweed.client.0+test@test.test --system false 2024-06-28T15:20:34.467 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:20:34.467 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:20:34.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.521+0000 7f9400271980 20 rados->read ofs=0 len=0 2024-06-28T15:20:34.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.525+0000 7f9400271980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:34.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.525+0000 7f9400271980 20 realm 2024-06-28T15:20:34.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.525+0000 7f9400271980 20 rados->read ofs=0 len=0 2024-06-28T15:20:34.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.525+0000 7f9400271980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:34.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.525+0000 7f9400271980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:20:34.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.525+0000 7f9400271980 20 rados->read ofs=0 len=0 2024-06-28T15:20:34.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.525+0000 7f9400271980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:34.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.525+0000 7f9400271980 20 rados->read ofs=0 len=0 2024-06-28T15:20:34.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.525+0000 7f9400271980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:20:34.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.525+0000 7f9400271980 20 rados->read ofs=0 len=0 2024-06-28T15:20:34.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.529+0000 7f9400271980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:20:34.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.529+0000 7f9400271980 20 searching for the correct realm 2024-06-28T15:20:34.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.737+0000 7f93fd581640 0 -- 172.21.15.178:0/1726939454 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f93f800a9c0 msgr2=0x7f93f800f3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:20:34.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.745+0000 7f93fdd82640 0 -- 172.21.15.178:0/1726939454 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f93f406fbf0 msgr2=0x7f93f4098430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:20:34.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.961+0000 7f9400271980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:20:34.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.961+0000 7f9400271980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:20:34.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.961+0000 7f9400271980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:20:34.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.961+0000 7f9400271980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:20:34.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.961+0000 7f9400271980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:20:34.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.961+0000 7f9400271980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:20:34.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.961+0000 7f9400271980 20 rados->read ofs=0 len=0 2024-06-28T15:20:34.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.965+0000 7f9400271980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:34.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.965+0000 7f9400271980 20 rados->read ofs=0 len=0 2024-06-28T15:20:34.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.965+0000 7f9400271980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:20:34.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.965+0000 7f9400271980 20 rados->read ofs=0 len=0 2024-06-28T15:20:34.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.965+0000 7f9400271980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:20:34.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.965+0000 7f9400271980 20 zone default found 2024-06-28T15:20:34.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.965+0000 7f9400271980 4 Realm: () 2024-06-28T15:20:34.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.965+0000 7f9400271980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:20:34.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.965+0000 7f9400271980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:20:34.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.965+0000 7f9400271980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:20:34.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.965+0000 7f9400271980 20 zonegroup default 2024-06-28T15:20:34.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.965+0000 7f9400271980 20 rados->read ofs=0 len=0 2024-06-28T15:20:34.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.965+0000 7f9400271980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:34.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.965+0000 7f9400271980 20 rados->read ofs=0 len=0 2024-06-28T15:20:34.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.969+0000 7f9400271980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:34.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.969+0000 7f9400271980 20 rados->read ofs=0 len=0 2024-06-28T15:20:34.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.969+0000 7f9400271980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:34.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.969+0000 7f9400271980 20 started sync module instance, tier type = 2024-06-28T15:20:34.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.969+0000 7f9400271980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:20:34.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.989+0000 7f9400271980 20 add_watcher() i=0 2024-06-28T15:20:34.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.989+0000 7f9400271980 20 add_watcher() i=1 2024-06-28T15:20:34.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.989+0000 7f9400271980 20 add_watcher() i=2 2024-06-28T15:20:34.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.989+0000 7f9400271980 20 add_watcher() i=3 2024-06-28T15:20:34.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.989+0000 7f9400271980 20 add_watcher() i=4 2024-06-28T15:20:34.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.989+0000 7f9400271980 20 add_watcher() i=5 2024-06-28T15:20:34.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.989+0000 7f9400271980 20 add_watcher() i=6 2024-06-28T15:20:34.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.989+0000 7f9400271980 20 add_watcher() i=7 2024-06-28T15:20:34.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.989+0000 7f9400271980 2 all 8 watchers are set, enabling cache 2024-06-28T15:20:34.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.993+0000 7f92af7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:20:34.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.993+0000 7f9400271980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:20:34.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.993+0000 7f9400271980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:20:34.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.993+0000 7f9400271980 5 note: GC not initialized 2024-06-28T15:20:34.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.993+0000 7f92ae7fc640 20 reqs_thread_entry: start 2024-06-28T15:20:34.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.993+0000 7f9400271980 20 init_complete bucket index max shards: 1 2024-06-28T15:20:34.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.993+0000 7f9400271980 20 Filter name: none 2024-06-28T15:20:34.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.993+0000 7f921ffff640 20 reqs_thread_entry: start 2024-06-28T15:20:34.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.993+0000 7f9400271980 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2024-06-28T15:20:34.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.993+0000 7f9400271980 20 rados->read ofs=0 len=0 2024-06-28T15:20:34.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:34.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2024-06-28T15:20:34.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-06-28T15:20:34.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2024-06-28T15:20:34.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 20 rados->read ofs=0 len=0 2024-06-28T15:20:34.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:34.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2024-06-28T15:20:34.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-06-28T15:20:34.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 10 cache get: name=default.rgw.meta+users.keys+DCOBIBMLAMCBAZSEXBWO : miss 2024-06-28T15:20:34.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 20 rados->read ofs=0 len=0 2024-06-28T15:20:34.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:34.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 10 cache put: name=default.rgw.meta+users.keys+DCOBIBMLAMCBAZSEXBWO info.flags=0x0 2024-06-28T15:20:34.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 10 adding default.rgw.meta+users.keys+DCOBIBMLAMCBAZSEXBWO to cache LRU end 2024-06-28T15:20:34.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 10 cache get: name=default.rgw.meta+users.keys+DCOBIBMLAMCBAZSEXBWO : hit (negative entry) 2024-06-28T15:20:34.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:34.997+0000 7f9400271980 10 cache get: name=default.rgw.meta+users.keys+DCOBIBMLAMCBAZSEXBWO : hit (negative entry) 2024-06-28T15:20:34.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.001+0000 7f9400271980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2024-06-28T15:20:34.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.001+0000 7f9400271980 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-06-28T15:20:34.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.001+0000 7f9400271980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2024-06-28T15:20:34.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.001+0000 7f92e17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1069446856727 cookie 94059647813840 notifier 5985 bl.length()=675 2024-06-28T15:20:34.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.001+0000 7f92e17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2024-06-28T15:20:34.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.001+0000 7f92e17fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-06-28T15:20:34.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.005+0000 7f9400271980 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2024-06-28T15:20:34.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.005+0000 7f9400271980 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-06-28T15:20:34.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.005+0000 7f9400271980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2024-06-28T15:20:34.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.009+0000 7f92e17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1069446856719 cookie 94059647807488 notifier 5985 bl.length()=202 2024-06-28T15:20:34.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.009+0000 7f92e17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2024-06-28T15:20:34.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.009+0000 7f92e17fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-06-28T15:20:34.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.013+0000 7f9400271980 10 cache put: name=default.rgw.meta+users.keys+DCOBIBMLAMCBAZSEXBWO info.flags=0x7 2024-06-28T15:20:34.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.013+0000 7f9400271980 10 moving default.rgw.meta+users.keys+DCOBIBMLAMCBAZSEXBWO to cache LRU end 2024-06-28T15:20:34.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.013+0000 7f9400271980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:DCOBIBMLAMCBAZSEXBWO, ofs0, ns] 2024-06-28T15:20:34.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.013+0000 7f92e0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1069446856718 cookie 94059647798016 notifier 5985 bl.length()=190 2024-06-28T15:20:34.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.013+0000 7f92e0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+DCOBIBMLAMCBAZSEXBWO info.flags=0x7 2024-06-28T15:20:34.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.013+0000 7f92e0ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+DCOBIBMLAMCBAZSEXBWO to cache LRU end 2024-06-28T15:20:34.999 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T15:20:34.999 INFO:teuthology.orchestra.run.smithi178.stdout: "user_id": "ragweed.client.0", 2024-06-28T15:20:34.999 INFO:teuthology.orchestra.run.smithi178.stdout: "display_name": "Mr. ragweed.client.0", 2024-06-28T15:20:34.999 INFO:teuthology.orchestra.run.smithi178.stdout: "email": "ragweed.client.0+test@test.test", 2024-06-28T15:20:34.999 INFO:teuthology.orchestra.run.smithi178.stdout: "suspended": 0, 2024-06-28T15:20:34.999 INFO:teuthology.orchestra.run.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:20:34.999 INFO:teuthology.orchestra.run.smithi178.stdout: "subusers": [], 2024-06-28T15:20:34.999 INFO:teuthology.orchestra.run.smithi178.stdout: "keys": [ 2024-06-28T15:20:34.999 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T15:20:34.999 INFO:teuthology.orchestra.run.smithi178.stdout: "user": "ragweed.client.0", 2024-06-28T15:20:35.000 INFO:teuthology.orchestra.run.smithi178.stdout: "access_key": "DCOBIBMLAMCBAZSEXBWO", 2024-06-28T15:20:35.000 INFO:teuthology.orchestra.run.smithi178.stdout: "secret_key": "wHfI2xJOzfYzgc+P5oSgzIXWSiQk5t9DpTJpAWgHmxNJTiSgh6/iMA==", 2024-06-28T15:20:35.000 INFO:teuthology.orchestra.run.smithi178.stdout: "active": true, 2024-06-28T15:20:35.000 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:20:35.002746Z" 2024-06-28T15:20:35.000 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T15:20:35.000 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-06-28T15:20:35.000 INFO:teuthology.orchestra.run.smithi178.stdout: "swift_keys": [], 2024-06-28T15:20:35.000 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": [], 2024-06-28T15:20:35.000 INFO:teuthology.orchestra.run.smithi178.stdout: "op_mask": "read, write, delete", 2024-06-28T15:20:35.000 INFO:teuthology.orchestra.run.smithi178.stdout: "default_placement": "", 2024-06-28T15:20:35.000 INFO:teuthology.orchestra.run.smithi178.stdout: "default_storage_class": "", 2024-06-28T15:20:35.000 INFO:teuthology.orchestra.run.smithi178.stdout: "placement_tags": [], 2024-06-28T15:20:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: "bucket_quota": { 2024-06-28T15:20:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:20:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:20:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:20:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:20:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:20:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:20:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: "user_quota": { 2024-06-28T15:20:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:20:35.002 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:20:35.002 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:20:35.002 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:20:35.002 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:20:35.002 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:20:35.002 INFO:teuthology.orchestra.run.smithi178.stdout: "temp_url_keys": [], 2024-06-28T15:20:35.002 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "rgw", 2024-06-28T15:20:35.002 INFO:teuthology.orchestra.run.smithi178.stdout: "mfa_ids": [], 2024-06-28T15:20:35.002 INFO:teuthology.orchestra.run.smithi178.stdout: "account_id": "", 2024-06-28T15:20:35.002 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2024-06-28T15:20:35.002 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:20:35.002730Z", 2024-06-28T15:20:35.002 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2024-06-28T15:20:35.002 INFO:teuthology.orchestra.run.smithi178.stdout: "group_ids": [] 2024-06-28T15:20:35.003 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T15:20:35.003 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:20:35.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.017+0000 7f92af7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:20:35.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.021+0000 7f9400271980 20 remove_watcher() i=0 2024-06-28T15:20:35.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.021+0000 7f9400271980 2 removed watcher, disabling cache 2024-06-28T15:20:35.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.021+0000 7f9400271980 20 remove_watcher() i=1 2024-06-28T15:20:35.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.025+0000 7f9400271980 20 remove_watcher() i=2 2024-06-28T15:20:35.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.025+0000 7f9400271980 20 remove_watcher() i=3 2024-06-28T15:20:35.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.029+0000 7f9400271980 20 remove_watcher() i=4 2024-06-28T15:20:35.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.033+0000 7f9400271980 20 remove_watcher() i=5 2024-06-28T15:20:35.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.033+0000 7f9400271980 20 remove_watcher() i=6 2024-06-28T15:20:35.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.033+0000 7f9400271980 20 remove_watcher() i=7 2024-06-28T15:20:35.035 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2024-06-28T15:20:35.036 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key YECNJEQCOQGRZLQMVUYL --secret ssFIsdUzQ5nmn65OULjMK4wKIuDa908KCC2O1GFNaH/MwJCcvD7gmQ== --email sysuser.client.0+test@test.test --system true 2024-06-28T15:20:35.077 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:20:35.077 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:20:35.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.133+0000 7fee77aa1980 20 rados->read ofs=0 len=0 2024-06-28T15:20:35.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.137+0000 7fee77aa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:35.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.137+0000 7fee77aa1980 20 realm 2024-06-28T15:20:35.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.137+0000 7fee77aa1980 20 rados->read ofs=0 len=0 2024-06-28T15:20:35.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.137+0000 7fee77aa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:35.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.137+0000 7fee77aa1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:20:35.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.137+0000 7fee77aa1980 20 rados->read ofs=0 len=0 2024-06-28T15:20:35.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.137+0000 7fee77aa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:35.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.137+0000 7fee77aa1980 20 rados->read ofs=0 len=0 2024-06-28T15:20:35.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.137+0000 7fee77aa1980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:20:35.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.137+0000 7fee77aa1980 20 rados->read ofs=0 len=0 2024-06-28T15:20:35.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.141+0000 7fee77aa1980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:20:35.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.141+0000 7fee77aa1980 20 searching for the correct realm 2024-06-28T15:20:35.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.145+0000 7fee75db3640 0 -- 172.21.15.178:0/1509219489 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x55de02f60000 msgr2=0x55de02f62490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:20:35.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.149+0000 7fee755b2640 0 -- 172.21.15.178:0/1509219489 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fee6c10ae60 msgr2=0x7fee6c11a560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:20:35.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.157+0000 7fee77aa1980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:20:35.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.157+0000 7fee77aa1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:20:35.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.157+0000 7fee77aa1980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:20:35.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.157+0000 7fee77aa1980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:20:35.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.157+0000 7fee77aa1980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:20:35.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.157+0000 7fee77aa1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:20:35.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.157+0000 7fee77aa1980 20 rados->read ofs=0 len=0 2024-06-28T15:20:35.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.157+0000 7fee77aa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:35.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.157+0000 7fee77aa1980 20 rados->read ofs=0 len=0 2024-06-28T15:20:35.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.157+0000 7fee77aa1980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:20:35.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.157+0000 7fee77aa1980 20 rados->read ofs=0 len=0 2024-06-28T15:20:35.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:20:35.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 20 zone default found 2024-06-28T15:20:35.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 4 Realm: () 2024-06-28T15:20:35.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:20:35.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:20:35.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:20:35.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 20 zonegroup default 2024-06-28T15:20:35.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 20 rados->read ofs=0 len=0 2024-06-28T15:20:35.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:35.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 20 rados->read ofs=0 len=0 2024-06-28T15:20:35.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:35.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 20 rados->read ofs=0 len=0 2024-06-28T15:20:35.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:35.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 20 started sync module instance, tier type = 2024-06-28T15:20:35.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.161+0000 7fee77aa1980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:20:35.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.177+0000 7fee77aa1980 20 add_watcher() i=0 2024-06-28T15:20:35.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.177+0000 7fee77aa1980 20 add_watcher() i=1 2024-06-28T15:20:35.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.177+0000 7fee77aa1980 20 add_watcher() i=2 2024-06-28T15:20:35.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.177+0000 7fee77aa1980 20 add_watcher() i=3 2024-06-28T15:20:35.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.177+0000 7fee77aa1980 20 add_watcher() i=4 2024-06-28T15:20:35.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.177+0000 7fee77aa1980 20 add_watcher() i=5 2024-06-28T15:20:35.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.177+0000 7fee77aa1980 20 add_watcher() i=6 2024-06-28T15:20:35.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.177+0000 7fee77aa1980 20 add_watcher() i=7 2024-06-28T15:20:35.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.177+0000 7fee77aa1980 2 all 8 watchers are set, enabling cache 2024-06-28T15:20:35.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.181+0000 7fed2bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:20:35.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.181+0000 7fee77aa1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:20:35.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.181+0000 7fee77aa1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:20:35.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.181+0000 7fee77aa1980 5 note: GC not initialized 2024-06-28T15:20:35.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.181+0000 7fed2afe5640 20 reqs_thread_entry: start 2024-06-28T15:20:35.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.181+0000 7fee77aa1980 20 init_complete bucket index max shards: 1 2024-06-28T15:20:35.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.181+0000 7fee77aa1980 20 Filter name: none 2024-06-28T15:20:35.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.181+0000 7fed28fe1640 20 reqs_thread_entry: start 2024-06-28T15:20:35.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.181+0000 7fee77aa1980 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2024-06-28T15:20:35.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.181+0000 7fee77aa1980 20 rados->read ofs=0 len=0 2024-06-28T15:20:35.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:35.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2024-06-28T15:20:35.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-06-28T15:20:35.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2024-06-28T15:20:35.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 20 rados->read ofs=0 len=0 2024-06-28T15:20:35.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:35.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2024-06-28T15:20:35.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-06-28T15:20:35.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 10 cache get: name=default.rgw.meta+users.keys+YECNJEQCOQGRZLQMVUYL : miss 2024-06-28T15:20:35.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 20 rados->read ofs=0 len=0 2024-06-28T15:20:35.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:20:35.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 10 cache put: name=default.rgw.meta+users.keys+YECNJEQCOQGRZLQMVUYL info.flags=0x0 2024-06-28T15:20:35.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 10 adding default.rgw.meta+users.keys+YECNJEQCOQGRZLQMVUYL to cache LRU end 2024-06-28T15:20:35.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 10 cache get: name=default.rgw.meta+users.keys+YECNJEQCOQGRZLQMVUYL : hit (negative entry) 2024-06-28T15:20:35.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.185+0000 7fee77aa1980 10 cache get: name=default.rgw.meta+users.keys+YECNJEQCOQGRZLQMVUYL : hit (negative entry) 2024-06-28T15:20:35.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.189+0000 7fee77aa1980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2024-06-28T15:20:35.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.189+0000 7fee77aa1980 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-06-28T15:20:35.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.189+0000 7fee77aa1980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2024-06-28T15:20:35.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.189+0000 7fed47fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1069446856720 cookie 94412020835360 notifier 6000 bl.length()=675 2024-06-28T15:20:35.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.189+0000 7fed47fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2024-06-28T15:20:35.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.189+0000 7fed47fff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-06-28T15:20:35.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.193+0000 7fee77aa1980 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2024-06-28T15:20:35.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.193+0000 7fee77aa1980 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-06-28T15:20:35.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.193+0000 7fee77aa1980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2024-06-28T15:20:35.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.193+0000 7fed47fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1069446856728 cookie 94412020837664 notifier 6000 bl.length()=202 2024-06-28T15:20:35.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.193+0000 7fed47fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2024-06-28T15:20:35.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.193+0000 7fed47fff640 10 rgw watcher librados: moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-06-28T15:20:35.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.197+0000 7fee77aa1980 10 cache put: name=default.rgw.meta+users.keys+YECNJEQCOQGRZLQMVUYL info.flags=0x7 2024-06-28T15:20:35.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.197+0000 7fee77aa1980 10 moving default.rgw.meta+users.keys+YECNJEQCOQGRZLQMVUYL to cache LRU end 2024-06-28T15:20:35.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.197+0000 7fee77aa1980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:YECNJEQCOQGRZLQMVUYL, ofs0, ns] 2024-06-28T15:20:35.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.197+0000 7fed47fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1069446856729 cookie 94412020837664 notifier 6000 bl.length()=190 2024-06-28T15:20:35.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.197+0000 7fed47fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+YECNJEQCOQGRZLQMVUYL info.flags=0x7 2024-06-28T15:20:35.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.197+0000 7fed47fff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+YECNJEQCOQGRZLQMVUYL to cache LRU end 2024-06-28T15:20:35.185 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T15:20:35.185 INFO:teuthology.orchestra.run.smithi178.stdout: "user_id": "sysuser.client.0", 2024-06-28T15:20:35.186 INFO:teuthology.orchestra.run.smithi178.stdout: "display_name": "Mr. sysuser.client.0", 2024-06-28T15:20:35.186 INFO:teuthology.orchestra.run.smithi178.stdout: "email": "sysuser.client.0+test@test.test", 2024-06-28T15:20:35.186 INFO:teuthology.orchestra.run.smithi178.stdout: "suspended": 0, 2024-06-28T15:20:35.186 INFO:teuthology.orchestra.run.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:20:35.186 INFO:teuthology.orchestra.run.smithi178.stdout: "subusers": [], 2024-06-28T15:20:35.186 INFO:teuthology.orchestra.run.smithi178.stdout: "keys": [ 2024-06-28T15:20:35.186 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T15:20:35.186 INFO:teuthology.orchestra.run.smithi178.stdout: "user": "sysuser.client.0", 2024-06-28T15:20:35.186 INFO:teuthology.orchestra.run.smithi178.stdout: "access_key": "YECNJEQCOQGRZLQMVUYL", 2024-06-28T15:20:35.186 INFO:teuthology.orchestra.run.smithi178.stdout: "secret_key": "ssFIsdUzQ5nmn65OULjMK4wKIuDa908KCC2O1GFNaH/MwJCcvD7gmQ==", 2024-06-28T15:20:35.186 INFO:teuthology.orchestra.run.smithi178.stdout: "active": true, 2024-06-28T15:20:35.186 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:20:35.190544Z" 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: "swift_keys": [], 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": [], 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: "op_mask": "read, write, delete", 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: "system": true, 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: "default_placement": "", 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: "default_storage_class": "", 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: "placement_tags": [], 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: "bucket_quota": { 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:20:35.187 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: "user_quota": { 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: "temp_url_keys": [], 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "rgw", 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: "mfa_ids": [], 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: "account_id": "", 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:20:35.190520Z", 2024-06-28T15:20:35.188 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2024-06-28T15:20:35.189 INFO:teuthology.orchestra.run.smithi178.stdout: "group_ids": [] 2024-06-28T15:20:35.189 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T15:20:35.189 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:20:35.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.201+0000 7fed2bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:20:35.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.205+0000 7fee77aa1980 20 remove_watcher() i=0 2024-06-28T15:20:35.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.205+0000 7fee77aa1980 2 removed watcher, disabling cache 2024-06-28T15:20:35.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.209+0000 7fee77aa1980 20 remove_watcher() i=1 2024-06-28T15:20:35.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.209+0000 7fee77aa1980 20 remove_watcher() i=2 2024-06-28T15:20:35.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.213+0000 7fee77aa1980 20 remove_watcher() i=3 2024-06-28T15:20:35.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.217+0000 7fee77aa1980 20 remove_watcher() i=4 2024-06-28T15:20:35.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.217+0000 7fee77aa1980 20 remove_watcher() i=5 2024-06-28T15:20:35.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.217+0000 7fee77aa1980 20 remove_watcher() i=6 2024-06-28T15:20:35.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:20:35.221+0000 7fee77aa1980 20 remove_watcher() i=7 2024-06-28T15:20:35.219 INFO:tasks.ragweed:Configuring ragweed... 2024-06-28T15:20:35.219 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T15:20:35.219 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/archive/ragweed.client.0.conf 2024-06-28T15:20:35.264 INFO:tasks.ragweed:Configuring boto... 2024-06-28T15:20:35.265 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T15:20:35.265 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/boto.cfg 2024-06-28T15:20:35.316 DEBUG:teuthology.orchestra.run.smithi178:ragweed tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw' 2024-06-28T15:20:35.798 INFO:teuthology.orchestra.run.smithi178.stdout:GLOB sdist-make: /home/ubuntu/cephtest/ragweed.client.0/setup.py 2024-06-28T15:20:35.987 INFO:teuthology.orchestra.run.smithi178.stdout:py create: /home/ubuntu/cephtest/ragweed.client.0/.tox/py 2024-06-28T15:20:36.558 INFO:teuthology.orchestra.run.smithi178.stdout:py installdeps: -rrequirements.txt 2024-06-28T15:20:40.124 INFO:teuthology.orchestra.run.smithi178.stdout:py inst: /home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.zip 2024-06-28T15:20:42.642 INFO:teuthology.orchestra.run.smithi178.stdout:py installed: appdirs==1.4.4,asyncssh==2.5.0,attrs==21.2.0,Babel==2.8.0,bcrypt==3.2.0,beautifulsoup4==4.10.0,beniget==0.4.1,blinker==1.4,boto==2.49.0,Brotli==1.0.9,cachetools==5.0.0,ceph==1.0.0,ceph-volume==1.0.0,cephfs==2.0.0,certifi==2020.6.20,chardet==4.0.0,cheroot==8.5.2+ds1,CherryPy==18.6.1,command-not-found==0.3,configobj==5.0.6,cryptography==3.4.8,cycler==0.11.0,dbus-python==1.2.18,decorator==4.4.2,distlib==0.3.4,distro==1.7.0,distro-info===1.1build1,docopt==0.6.2,execnet==1.9.0,filelock==3.6.0,fonttools==4.29.1,fs==2.4.12,gast==0.5.2,gevent==21.8.0,google-auth==1.5.1,greenlet==1.1.2,html5lib==1.1,httplib2==0.20.2,idna==3.3,importlib-metadata==4.6.4,iniconfig==1.1.1,isodate==0.6.1,jaraco.classes==3.2.1,jaraco.collections==3.4.0,jaraco.functools==3.4.0,jaraco.text==3.6.0,jeepney==0.7.1,Jinja2==3.0.3,joblib==0.17.0,keyring==23.5.0,kiwisolver==1.3.2,kubernetes==12.0.1,language-selector==0.1,launchpadlib==1.10.16,lazr.restfulclient==0.14.4,lazr.uri==1.0.6,logutils==0.3.3,lxml==4.8.0,lz4==3.1.3+dfsg,Mako==1.1.3,MarkupSafe==2.0.1,matplotlib==3.5.1,more-itertools==8.10.0,mpmath==0.0.0,munch==4.0.0,natsort==8.0.2,netifaces==0.11.0,nose==1.3.7,numpy==1.21.5,oauthlib==3.2.0,olefile==0.46,packaging==21.3,Paste==3.5.0,PasteDeploy==2.1.1,PasteScript==2.0.2,pecan==1.3.3,Pillow==9.0.1,platformdirs==2.5.1,pluggy==0.13.0,ply==3.11,portend==3.0.0,prettytable==2.5.0,psutil==5.9.0,py==1.10.0,pyasn1==0.4.8,pyasn1-modules==0.2.1,Pygments==2.11.2,PyGObject==3.42.1,pyinotify==0.9.6,PyJWT==2.3.0,pymacaroons==0.13.0,PyNaCl==1.5.0,pyOpenSSL==21.0.0,pyparsing==2.4.7,pytest==6.2.5,python-apt==2.3.0+ubuntu2.1,python-dateutil==2.8.1,python-magic==0.4.24,pythran==0.10.0,pytz==2022.1,PyYAML==5.4.1,rados==2.0.0,ragweed @ file:///home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.zip#sha256=0c371793779aa6409419635b0983c4194117084c56449c6ba32a92b16f362e81,rbd==2.0.0,registries-conf-ctl @ git+https://github.com/sebastian-philipp/registries-conf-ctl@0e9bf2dc202aa9be9b0bf8e8265aac3f2308f9db,remoto==1.2.1,repoze.lru==0.7,requests==2.25.1,requests-oauthlib==1.3.0,rgw==2.0.0,Routes==2.5.1,rsa==4.8,s3cmd==2.2.0,scikit-learn==0.23.2,scipy==1.8.0,SecretStorage==3.3.1,simplegeneric==0.8.1,simplejson==3.17.6,six==1.16.0,soupsieve==2.3.1,ssh-import-id==5.11,sympy==1.9,Tempita==0.5.2,tempora==4.1.2,threadpoolctl==3.1.0,toml==0.10.2,ubuntu-advantage-tools==27.12,ufoLib2==0.13.1,ufw==0.36.1,unattended-upgrades==0.1,unicodedata2==14.0.0,urllib3==1.26.5,virtualenv==20.13.0+ds,wadllib==1.3.6,waitress==1.4.4,wcwidth==0.2.5,webencodings==0.5.1,WebOb==1.8.6,websocket-client==1.2.3,WebTest==2.0.35,Werkzeug==2.0.2,zc.lockfile==2.0,zipp==1.0.0,zope.event==4.4,zope.interface==5.4.0 2024-06-28T15:20:42.643 INFO:teuthology.orchestra.run.smithi178.stdout:py run-test-pre: PYTHONHASHSEED='2366231580' 2024-06-28T15:20:42.643 INFO:teuthology.orchestra.run.smithi178.stdout:py run-test: commands[0] | python -m pytest -v -m 'not fails_on_rgw' 2024-06-28T15:20:42.819 INFO:teuthology.orchestra.run.smithi178.stdout:============================= test session starts ============================== 2024-06-28T15:20:42.819 INFO:teuthology.orchestra.run.smithi178.stdout:platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0 -- /home/ubuntu/cephtest/ragweed.client.0/.tox/py/bin/python 2024-06-28T15:20:42.819 INFO:teuthology.orchestra.run.smithi178.stdout:cachedir: .tox/py/.pytest_cache 2024-06-28T15:20:42.819 INFO:teuthology.orchestra.run.smithi178.stdout:rootdir: /home/ubuntu/cephtest/ragweed.client.0, configfile: pytest.ini 2024-06-28T15:20:42.978 INFO:teuthology.orchestra.run.smithi178.stdout:collecting ... collected 9 items 2024-06-28T15:20:42.978 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:20:47.780 INFO:teuthology.orchestra.run.smithi178.stdout:ragweed/tests/tests.py::r_test_small_obj_data::test PASSED [ 11%] 2024-06-28T15:20:51.568 INFO:teuthology.orchestra.run.smithi178.stdout:ragweed/tests/tests.py::r_test_multipart_simple::test PASSED [ 22%] 2024-06-28T15:20:55.176 INFO:teuthology.orchestra.run.smithi178.stdout:ragweed/tests/tests.py::r_test_multipart_index_versioning::test PASSED [ 33%] 2024-06-28T15:20:59.436 INFO:teuthology.orchestra.run.smithi178.stdout:ragweed/tests/tests.py::r_test_multipart_defer_complete::test PASSED [ 44%] 2024-06-28T15:21:04.863 INFO:teuthology.orchestra.run.smithi178.stdout:ragweed/tests/tests.py::r_test_multipart_defer_update_complete::test PASSED [ 55%] 2024-06-28T15:21:16.303 INFO:teuthology.orchestra.run.smithi178.stdout:ragweed/tests/tests.py::r_test_obj_storage_class::test PASSED [ 66%] 2024-06-28T15:21:25.589 INFO:teuthology.orchestra.run.smithi178.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart::test PASSED [ 77%] 2024-06-28T15:21:35.238 INFO:teuthology.orchestra.run.smithi178.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_copy::test PASSED [ 88%] 2024-06-28T15:21:53.418 INFO:teuthology.orchestra.run.smithi178.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart_copy::test PASSED [100%] 2024-06-28T15:21:53.419 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:21:53.419 INFO:teuthology.orchestra.run.smithi178.stdout:=============================== warnings summary =============================== 2024-06-28T15:21:53.419 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/boto/plugin.py:40 2024-06-28T15:21:53.419 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/ragweed.client.0/.tox/py/lib/python3.10/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses 2024-06-28T15:21:53.419 INFO:teuthology.orchestra.run.smithi178.stdout: import imp 2024-06-28T15:21:53.419 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:21:53.419 INFO:teuthology.orchestra.run.smithi178.stdout:-- Docs: https://docs.pytest.org/en/stable/warnings.html 2024-06-28T15:21:53.419 INFO:teuthology.orchestra.run.smithi178.stdout:=================== 9 passed, 1 warning in 70.60s (0:01:10) ==================== 2024-06-28T15:21:53.467 INFO:teuthology.orchestra.run.smithi178.stdout:___________________________________ summary ____________________________________ 2024-06-28T15:21:53.467 INFO:teuthology.orchestra.run.smithi178.stdout: py: commands succeeded 2024-06-28T15:21:53.467 INFO:teuthology.orchestra.run.smithi178.stdout: congratulations :) 2024-06-28T15:21:53.487 INFO:tasks.ragweed:Cleaning up boto... 2024-06-28T15:21:53.488 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /home/ubuntu/cephtest/boto.cfg 2024-06-28T15:21:53.493 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2024-06-28T15:21:53.569 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:21:53.569 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:21:53.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.836+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:53.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.836+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:53.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.836+0000 7f3fb21ba980 20 realm 2024-06-28T15:21:53.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.836+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:53.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.836+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:53.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.836+0000 7f3fb21ba980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:21:53.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.836+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:53.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.836+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:53.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.836+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:53.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.840+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:21:53.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.840+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:53.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.840+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:21:53.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.840+0000 7f3fb21ba980 20 searching for the correct realm 2024-06-28T15:21:53.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.844+0000 7f3fb04cc640 0 -- 172.21.15.178:0/4154209051 >> [v2:172.21.15.195:6824/1278553753,v1:172.21.15.195:6825/1278553753] conn(0x7f3f98036060 msgr2=0x7f3f980615a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:21:53.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:53.852+0000 7f3fafccb640 0 -- 172.21.15.178:0/4154209051 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f3f9802d200 msgr2=0x7f3f98023030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:21:54.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:21:54.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:21:54.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:21:54.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:21:54.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:21:54.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:21:54.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:54.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:21:54.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:21:54.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 zone default found 2024-06-28T15:21:54.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 4 Realm: () 2024-06-28T15:21:54.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:21:54.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:21:54.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:21:54.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 zonegroup default 2024-06-28T15:21:54.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.072+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:54.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.076+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.076+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:54.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.076+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.076+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:54.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.076+0000 7f3fb21ba980 20 started sync module instance, tier type = 2024-06-28T15:21:54.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.076+0000 7f3fb21ba980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:21:54.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.100+0000 7f3fb21ba980 20 add_watcher() i=0 2024-06-28T15:21:54.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.100+0000 7f3fb21ba980 20 add_watcher() i=1 2024-06-28T15:21:54.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.100+0000 7f3fb21ba980 20 add_watcher() i=2 2024-06-28T15:21:54.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.100+0000 7f3fb21ba980 20 add_watcher() i=3 2024-06-28T15:21:54.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.100+0000 7f3fb21ba980 20 add_watcher() i=4 2024-06-28T15:21:54.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.100+0000 7f3fb21ba980 20 add_watcher() i=5 2024-06-28T15:21:54.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.100+0000 7f3fb21ba980 20 add_watcher() i=6 2024-06-28T15:21:54.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.100+0000 7f3fb21ba980 20 add_watcher() i=7 2024-06-28T15:21:54.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.100+0000 7f3fb21ba980 2 all 8 watchers are set, enabling cache 2024-06-28T15:21:54.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.104+0000 7f3e65fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:21:54.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.108+0000 7f3fb21ba980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:21:54.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.108+0000 7f3fb21ba980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:21:54.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.200+0000 7f3e64fe1640 20 reqs_thread_entry: start 2024-06-28T15:21:54.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.200+0000 7f3fb21ba980 20 init_complete bucket index max shards: 1 2024-06-28T15:21:54.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.200+0000 7f3fb21ba980 20 Filter name: none 2024-06-28T15:21:54.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.200+0000 7f3dde7fc640 20 reqs_thread_entry: start 2024-06-28T15:21:54.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.200+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2024-06-28T15:21:54.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.200+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=486 2024-06-28T15:21:54.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x13 2024-06-28T15:21:54.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-06-28T15:21:54.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:21:54.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:843e24ce-085a-4043-8f55-38e05370e790.4686.5 : miss 2024-06-28T15:21:54.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=418 2024-06-28T15:21:54.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:843e24ce-085a-4043-8f55-38e05370e790.4686.5 info.flags=0x17 2024-06-28T15:21:54.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:843e24ce-085a-4043-8f55-38e05370e790.4686.5 to cache LRU end 2024-06-28T15:21:54.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:21:54.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-06-28T15:21:54.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:843e24ce-085a-4043-8f55-38e05370e790.4686.5 2024-06-28T15:21:54.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete 2024-06-28T15:21:54.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete 2024-06-28T15:21:54.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning 2024-06-28T15:21:54.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got tests/r_test_multipart_simple 2024-06-28T15:21:54.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class 2024-06-28T15:21:54.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy 2024-06-28T15:21:54.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart 2024-06-28T15:21:54.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy 2024-06-28T15:21:54.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got tests/r_test_small_obj_data 2024-06-28T15:21:54.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.204+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x56149d79f2a8 s->prefetch_data=0 2024-06-28T15:21:54.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.208+0000 7f3fb21ba980 10 manifest: total_size = 4970 2024-06-28T15:21:54.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.208+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.3043720328400159613 2024-06-28T15:21:54.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.208+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x56149d79f2a8 s->prefetch_data=0 2024-06-28T15:21:54.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.208+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d79f2a8 2024-06-28T15:21:54.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.208+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.5.0 2024-06-28T15:21:54.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=4970 stripe_ofs=4970 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.3043720328400159613 2024-06-28T15:21:54.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.lP5DhIrL25P6OG_3BLMGtP_hqdIBdeI_0 2024-06-28T15:21:54.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-06-28T15:21:54.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.3043720328400159613 2024-06-28T15:21:54.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_complete 2024-06-28T15:21:54.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3fb21ba980 10 manifest: total_size = 5248 2024-06-28T15:21:54.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.12560414832874831080 2024-06-28T15:21:54.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d727ab8 2024-06-28T15:21:54.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.212+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.5.0 2024-06-28T15:21:54.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.216+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.216+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5248 stripe_ofs=5248 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.216+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.12560414832874831080 2024-06-28T15:21:54.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.216+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.216+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.kMtjaixVfqqQVLy5VOIi9lWcG6uo-8i_0 2024-06-28T15:21:54.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.216+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-06-28T15:21:54.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.216+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.216+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.12560414832874831080 2024-06-28T15:21:54.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.220+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_update_complete 2024-06-28T15:21:54.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.220+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.224+0000 7f3fb21ba980 10 manifest: total_size = 12811 2024-06-28T15:21:54.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.224+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.12149661540729176768 2024-06-28T15:21:54.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.224+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.224+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d727ab8 2024-06-28T15:21:54.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.224+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.5.0 2024-06-28T15:21:54.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.228+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.228+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=12811 stripe_ofs=12811 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.228+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.12149661540729176768 2024-06-28T15:21:54.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.228+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.228+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.OilZA6weLTPfLV0jLZG8MbHosg_dIhk_0 2024-06-28T15:21:54.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.228+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-06-28T15:21:54.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.228+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.228+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.12149661540729176768 2024-06-28T15:21:54.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.232+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_index_versioning 2024-06-28T15:21:54.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.232+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.232+0000 7f3fb21ba980 10 manifest: total_size = 5140 2024-06-28T15:21:54.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.232+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.4187850353238860256 2024-06-28T15:21:54.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.232+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.232+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d727ab8 2024-06-28T15:21:54.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.232+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.5.0 2024-06-28T15:21:54.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.236+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.236+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5140 stripe_ofs=5140 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.236+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.4187850353238860256 2024-06-28T15:21:54.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.236+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.236+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.xLxxFeFKU69d8QatRUyjVrtraYDc29Y_0 2024-06-28T15:21:54.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.236+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-06-28T15:21:54.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.236+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.236+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.4187850353238860256 2024-06-28T15:21:54.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.240+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_simple 2024-06-28T15:21:54.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.240+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.240+0000 7f3fb21ba980 10 manifest: total_size = 7638 2024-06-28T15:21:54.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.240+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.3611825967390322888 2024-06-28T15:21:54.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.240+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.240+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d727ab8 2024-06-28T15:21:54.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.240+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.5.0 2024-06-28T15:21:54.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.244+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.244+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=7638 stripe_ofs=7638 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.244+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.3611825967390322888 2024-06-28T15:21:54.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.244+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.244+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.ZBcakGelu7-Kf8eg6D68hQOzlkoMwP7_0 2024-06-28T15:21:54.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.244+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-06-28T15:21:54.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.244+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.244+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.3611825967390322888 2024-06-28T15:21:54.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.248+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class 2024-06-28T15:21:54.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.248+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.248+0000 7f3fb21ba980 10 manifest: total_size = 15906 2024-06-28T15:21:54.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.248+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.5764869312466433341 2024-06-28T15:21:54.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.248+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.248+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d727ab8 2024-06-28T15:21:54.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.248+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.5.0 2024-06-28T15:21:54.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.252+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.252+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=15906 stripe_ofs=15906 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.252+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.5764869312466433341 2024-06-28T15:21:54.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.252+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.252+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.M7EjadUhwoSAtTf53ukLTRQF3iozu5A_0 2024-06-28T15:21:54.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.252+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-06-28T15:21:54.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.252+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.252+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.5764869312466433341 2024-06-28T15:21:54.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.256+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_copy 2024-06-28T15:21:54.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.256+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.256+0000 7f3fb21ba980 10 manifest: total_size = 5139 2024-06-28T15:21:54.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.256+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.10061548790983698222 2024-06-28T15:21:54.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.256+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.256+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d727ab8 2024-06-28T15:21:54.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.256+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.5.0 2024-06-28T15:21:54.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.260+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.260+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5139 stripe_ofs=5139 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.260+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.10061548790983698222 2024-06-28T15:21:54.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.260+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.260+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.j5xlxUvls8Q_iPS9jMuhRV65Wtrg-lm_0 2024-06-28T15:21:54.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.260+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-06-28T15:21:54.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.260+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.260+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.10061548790983698222 2024-06-28T15:21:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.264+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart 2024-06-28T15:21:54.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.264+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.264+0000 7f3fb21ba980 10 manifest: total_size = 5051 2024-06-28T15:21:54.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.264+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.9725363603752228366 2024-06-28T15:21:54.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.264+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.264+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d727ab8 2024-06-28T15:21:54.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.264+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.5.0 2024-06-28T15:21:54.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5051 stripe_ofs=5051 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.9725363603752228366 2024-06-28T15:21:54.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.T4mH8dfZRqRVP1yOxCm9zYyoNNRWRua_0 2024-06-28T15:21:54.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-06-28T15:21:54.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.9725363603752228366 2024-06-28T15:21:54.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart_copy 2024-06-28T15:21:54.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3fb21ba980 10 manifest: total_size = 5374 2024-06-28T15:21:54.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.12695288585948761914 2024-06-28T15:21:54.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x56149d727ab8 s->prefetch_data=0 2024-06-28T15:21:54.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d727ab8 2024-06-28T15:21:54.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.272+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.5.0 2024-06-28T15:21:54.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.276+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.276+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5374 stripe_ofs=5374 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.276+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.12695288585948761914 2024-06-28T15:21:54.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.276+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.276+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.DCdAK61jqs8FF8IgYo6F8mlU4Wn11Un_0 2024-06-28T15:21:54.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.276+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-06-28T15:21:54.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.276+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.276+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.12695288585948761914 2024-06-28T15:21:54.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.280+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=tests/r_test_small_obj_data 2024-06-28T15:21:54.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.280+0000 7f3fb21ba980 20 list_objects_ordered: starting attempt 1 2024-06-28T15:21:54.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.280+0000 7f3fb21ba980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-06-28T15:21:54.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.280+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-06-28T15:21:54.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.280+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-06-28T15:21:54.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.280+0000 7f3fb21ba980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-06-28T15:21:54.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.280+0000 7f3fb21ba980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-06-28T15:21:54.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.280+0000 7f3fb21ba980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[843e24ce-085a-4043-8f55-38e05370e790.4686.5])"; uploads.size()=0; is_truncated=0 2024-06-28T15:21:54.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.284+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2024-06-28T15:21:54.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.284+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.284+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=204 2024-06-28T15:21:54.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.284+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x11 2024-06-28T15:21:54.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.284+0000 7f3fb21ba980 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2024-06-28T15:21:54.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.288+0000 7f3fb21ba980 10 removing default.rgw.meta+root+test-client.0-conf from cache 2024-06-28T15:21:54.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.288+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-conf, ofs0, ns] 2024-06-28T15:21:54.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.288+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660513 cookie 94646540629632 notifier 6036 bl.length()=162 2024-06-28T15:21:54.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.292+0000 7f3fb21ba980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:843e24ce-085a-4043-8f55-38e05370e790.4686.5 from cache 2024-06-28T15:21:54.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.296+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-conf:843e24ce-085a-4043-8f55-38e05370e790.4686.5, ofs0, ns] 2024-06-28T15:21:54.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.296+0000 7f3e8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660514 cookie 94646540629632 notifier 6036 bl.length()=219 2024-06-28T15:21:54.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.300+0000 7f3fb21ba980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-conf/843e24ce-085a-4043-8f55-38e05370e790.4686.5 : miss 2024-06-28T15:21:54.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.300+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.300+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:54.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.300+0000 7f3fb21ba980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-conf/843e24ce-085a-4043-8f55-38e05370e790.4686.5 info.flags=0x0 2024-06-28T15:21:54.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.300+0000 7f3fb21ba980 10 adding default.rgw.log++pubsub..bucket.test-client.0-conf/843e24ce-085a-4043-8f55-38e05370e790.4686.5 to cache LRU end 2024-06-28T15:21:54.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.304+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:843e24ce-085a-4043-8f55-38e05370e790.4686.12 : miss 2024-06-28T15:21:54.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.304+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.304+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=449 2024-06-28T15:21:54.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.304+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:843e24ce-085a-4043-8f55-38e05370e790.4686.12 info.flags=0x17 2024-06-28T15:21:54.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.304+0000 7f3fb21ba980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:843e24ce-085a-4043-8f55-38e05370e790.4686.12 to cache LRU end 2024-06-28T15:21:54.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.304+0000 7f3fb21ba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:21:54.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.304+0000 7f3fb21ba980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-06-28T15:21:54.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.304+0000 7f3fb21ba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:843e24ce-085a-4043-8f55-38e05370e790.4686.12 2024-06-28T15:21:54.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.308+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo 2024-06-28T15:21:54.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.308+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x56149d722568 s->prefetch_data=0 2024-06-28T15:21:54.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.308+0000 7f3fb21ba980 10 manifest: total_size = 1667 2024-06-28T15:21:54.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.308+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.16075280366673605200 2024-06-28T15:21:54.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.308+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9802ddf0 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x56149d722568 s->prefetch_data=0 2024-06-28T15:21:54.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.308+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d722568 2024-06-28T15:21:54.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.308+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.12.0 2024-06-28T15:21:54.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-06-28T15:21:54.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=516 2024-06-28T15:21:54.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-06-28T15:21:54.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-06-28T15:21:54.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291972 part_ofs=516 rule->part_size=516 2024-06-28T15:21:54.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=516 2024-06-28T15:21:54.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-06-28T15:21:54.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6292488 part_ofs=1032 rule->part_size=516 2024-06-28T15:21:54.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=1548 stripe_ofs=1548 part_ofs=1548 rule->part_size=119 2024-06-28T15:21:54.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=119 rules.size()=2 2024-06-28T15:21:54.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6293004 part_ofs=1548 rule->part_size=119 2024-06-28T15:21:54.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=1667 stripe_ofs=1667 part_ofs=1667 rule->part_size=119 2024-06-28T15:21:54.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.16075280366673605200 2024-06-28T15:21:54.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.12__multipart_foo.2~xTHSyet_mXJEGmeACs0Br1TaRSbwvDn.1 2024-06-28T15:21:54.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 347 2024-06-28T15:21:54.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.12__multipart_foo.2~xTHSyet_mXJEGmeACs0Br1TaRSbwvDn.2 2024-06-28T15:21:54.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 593 2024-06-28T15:21:54.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.12__multipart_foo.2~xTHSyet_mXJEGmeACs0Br1TaRSbwvDn.3 2024-06-28T15:21:54.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 839 2024-06-28T15:21:54.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.12__multipart_foo.2~xTHSyet_mXJEGmeACs0Br1TaRSbwvDn.4 2024-06-28T15:21:54.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1085 2024-06-28T15:21:54.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.312+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.16075280366673605200 2024-06-28T15:21:54.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.316+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo 2024-06-28T15:21:54.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.316+0000 7f3fb21ba980 20 list_objects_ordered: starting attempt 1 2024-06-28T15:21:54.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.316+0000 7f3fb21ba980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-06-28T15:21:54.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.316+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-06-28T15:21:54.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.316+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-06-28T15:21:54.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.316+0000 7f3fb21ba980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-06-28T15:21:54.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.316+0000 7f3fb21ba980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-06-28T15:21:54.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.316+0000 7f3fb21ba980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[843e24ce-085a-4043-8f55-38e05370e790.4686.12])"; uploads.size()=0; is_truncated=0 2024-06-28T15:21:54.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.320+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2024-06-28T15:21:54.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.320+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.320+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=235 2024-06-28T15:21:54.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.320+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x11 2024-06-28T15:21:54.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.320+0000 7f3fb21ba980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2024-06-28T15:21:54.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.324+0000 7f3fb21ba980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2024-06-28T15:21:54.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.324+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1, ofs0, ns] 2024-06-28T15:21:54.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.324+0000 7f3e8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660578 cookie 94646540639104 notifier 6036 bl.length()=191 2024-06-28T15:21:54.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.328+0000 7f3fb21ba980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:843e24ce-085a-4043-8f55-38e05370e790.4686.12 from cache 2024-06-28T15:21:54.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.328+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:843e24ce-085a-4043-8f55-38e05370e790.4686.12, ofs0, ns] 2024-06-28T15:21:54.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.332+0000 7f3e8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660502 cookie 94646540640160 notifier 6036 bl.length()=249 2024-06-28T15:21:54.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.336+0000 7f3fb21ba980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/843e24ce-085a-4043-8f55-38e05370e790.4686.12 : miss 2024-06-28T15:21:54.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.336+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.340+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:54.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.340+0000 7f3fb21ba980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/843e24ce-085a-4043-8f55-38e05370e790.4686.12 info.flags=0x0 2024-06-28T15:21:54.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.340+0000 7f3fb21ba980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/843e24ce-085a-4043-8f55-38e05370e790.4686.12 to cache LRU end 2024-06-28T15:21:54.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:843e24ce-085a-4043-8f55-38e05370e790.4686.13 : miss 2024-06-28T15:21:54.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=456 2024-06-28T15:21:54.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:843e24ce-085a-4043-8f55-38e05370e790.4686.13 info.flags=0x17 2024-06-28T15:21:54.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:843e24ce-085a-4043-8f55-38e05370e790.4686.13 to cache LRU end 2024-06-28T15:21:54.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:21:54.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-06-28T15:21:54.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:843e24ce-085a-4043-8f55-38e05370e790.4686.13 2024-06-28T15:21:54.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo 2024-06-28T15:21:54.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d79fe50 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x56149d72e8d8 s->prefetch_data=0 2024-06-28T15:21:54.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 10 manifest: total_size = 26079 2024-06-28T15:21:54.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.14468871253119589124 2024-06-28T15:21:54.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d79fe50 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x56149d72e8d8 s->prefetch_data=0 2024-06-28T15:21:54.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d72e8d8 2024-06-28T15:21:54.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.344+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.13.0 2024-06-28T15:21:54.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=7671 rules.size()=3 2024-06-28T15:21:54.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=7671 2024-06-28T15:21:54.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=7671 stripe_ofs=7671 part_ofs=7671 rule->part_size=7671 2024-06-28T15:21:54.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=7671 rules.size()=3 2024-06-28T15:21:54.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6299127 part_ofs=7671 rule->part_size=7671 2024-06-28T15:21:54.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=15342 stripe_ofs=15342 part_ofs=15342 rule->part_size=7670 2024-06-28T15:21:54.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=7670 rules.size()=3 2024-06-28T15:21:54.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6306798 part_ofs=15342 rule->part_size=7670 2024-06-28T15:21:54.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=23012 stripe_ofs=23012 part_ofs=23012 rule->part_size=3067 2024-06-28T15:21:54.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=3067 rules.size()=3 2024-06-28T15:21:54.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6314468 part_ofs=23012 rule->part_size=3067 2024-06-28T15:21:54.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=26079 stripe_ofs=26079 part_ofs=26079 rule->part_size=3067 2024-06-28T15:21:54.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.14468871253119589124 2024-06-28T15:21:54.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.13__multipart_foo.2~rKM6r06SyAbjSQW5B-r_P28bJ4olQt5.1 2024-06-28T15:21:54.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 347 2024-06-28T15:21:54.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.13__multipart_foo.2~rKM6r06SyAbjSQW5B-r_P28bJ4olQt5.2 2024-06-28T15:21:54.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 593 2024-06-28T15:21:54.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.13__multipart_foo.2~rKM6r06SyAbjSQW5B-r_P28bJ4olQt5.3 2024-06-28T15:21:54.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 839 2024-06-28T15:21:54.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.13__multipart_foo.2~rKM6r06SyAbjSQW5B-r_P28bJ4olQt5.4 2024-06-28T15:21:54.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1085 2024-06-28T15:21:54.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.352+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.14468871253119589124 2024-06-28T15:21:54.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.356+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo 2024-06-28T15:21:54.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.356+0000 7f3fb21ba980 20 list_objects_ordered: starting attempt 1 2024-06-28T15:21:54.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.356+0000 7f3fb21ba980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-06-28T15:21:54.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.356+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-06-28T15:21:54.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.356+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-06-28T15:21:54.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.356+0000 7f3fb21ba980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-06-28T15:21:54.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.356+0000 7f3fb21ba980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-06-28T15:21:54.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.356+0000 7f3fb21ba980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[843e24ce-085a-4043-8f55-38e05370e790.4686.13])"; uploads.size()=0; is_truncated=0 2024-06-28T15:21:54.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.360+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2024-06-28T15:21:54.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.360+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.360+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=242 2024-06-28T15:21:54.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.360+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x11 2024-06-28T15:21:54.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.360+0000 7f3fb21ba980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2024-06-28T15:21:54.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.364+0000 7f3fb21ba980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2024-06-28T15:21:54.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.364+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1, ofs0, ns] 2024-06-28T15:21:54.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.364+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660503 cookie 94646540640160 notifier 6036 bl.length()=198 2024-06-28T15:21:54.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.368+0000 7f3fb21ba980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:843e24ce-085a-4043-8f55-38e05370e790.4686.13 from cache 2024-06-28T15:21:54.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.368+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:843e24ce-085a-4043-8f55-38e05370e790.4686.13, ofs0, ns] 2024-06-28T15:21:54.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.372+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660486 cookie 94646540654496 notifier 6036 bl.length()=256 2024-06-28T15:21:54.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.376+0000 7f3fb21ba980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/843e24ce-085a-4043-8f55-38e05370e790.4686.13 : miss 2024-06-28T15:21:54.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.376+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.376+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:54.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.376+0000 7f3fb21ba980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/843e24ce-085a-4043-8f55-38e05370e790.4686.13 info.flags=0x0 2024-06-28T15:21:54.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.376+0000 7f3fb21ba980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/843e24ce-085a-4043-8f55-38e05370e790.4686.13 to cache LRU end 2024-06-28T15:21:54.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.380+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:843e24ce-085a-4043-8f55-38e05370e790.4686.11 : miss 2024-06-28T15:21:54.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.380+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.384+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=451 2024-06-28T15:21:54.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.384+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:843e24ce-085a-4043-8f55-38e05370e790.4686.11 info.flags=0x17 2024-06-28T15:21:54.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.384+0000 7f3fb21ba980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:843e24ce-085a-4043-8f55-38e05370e790.4686.11 to cache LRU end 2024-06-28T15:21:54.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.384+0000 7f3fb21ba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:21:54.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.384+0000 7f3fb21ba980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-06-28T15:21:54.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.384+0000 7f3fb21ba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:843e24ce-085a-4043-8f55-38e05370e790.4686.11 2024-06-28T15:21:54.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.384+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo[uCjzIbwa7v3vlbjla0PiKh0JGE4yX23] 2024-06-28T15:21:54.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.384+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.11.0 2024-06-28T15:21:54.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.384+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa8100240 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x56149d726e08 s->prefetch_data=0 2024-06-28T15:21:54.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.384+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 7c9b2uqiurf4imsnpc4c9hwnrw3z5n0f 2024-06-28T15:21:54.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.384+0000 7f3fb21ba980 20 get_obj_state_impl: setting s->olh_tag to 60jkifax33lafjpb1704pbli2y85cfcb 2024-06-28T15:21:54.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.384+0000 7f3fb21ba980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=60jkifax33lafjpb1704pbli2y85cfcb 2024-06-28T15:21:54.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.388+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.11.0 2024-06-28T15:21:54.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.388+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.11.0 2024-06-28T15:21:54.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.392+0000 7f3fb21ba980 20 olh_log_entry: epoch=3 op=2 key=foo[] 2024-06-28T15:21:54.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.392+0000 7f3fb21ba980 20 olh_log_entry: epoch=3 op=3 key=foo[uCjzIbwa7v3vlbjla0PiKh0JGE4yX23] 2024-06-28T15:21:54.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.392+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa8100240 obj=test-client.0-r-test-multipart-index-versioning.1:_:uCjzIbwa7v3vlbjla0PiKh0JGE4yX23_foo state=0x56149d736528 s->prefetch_data=0 2024-06-28T15:21:54.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.392+0000 7f3fb21ba980 10 manifest: total_size = 74198 2024-06-28T15:21:54.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.392+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.17870332517152078614 2024-06-28T15:21:54.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.392+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa8100240 obj=test-client.0-r-test-multipart-index-versioning.1:_:uCjzIbwa7v3vlbjla0PiKh0JGE4yX23_foo state=0x56149d736528 s->prefetch_data=0 2024-06-28T15:21:54.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.392+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d736528 2024-06-28T15:21:54.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.392+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.11.0 2024-06-28T15:21:54.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-06-28T15:21:54.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=20613 2024-06-28T15:21:54.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2024-06-28T15:21:54.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-06-28T15:21:54.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6312069 part_ofs=20613 rule->part_size=20613 2024-06-28T15:21:54.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=20613 2024-06-28T15:21:54.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2024-06-28T15:21:54.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6332682 part_ofs=41226 rule->part_size=20613 2024-06-28T15:21:54.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=61839 stripe_ofs=61839 part_ofs=61839 rule->part_size=12359 2024-06-28T15:21:54.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=12359 rules.size()=2 2024-06-28T15:21:54.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6353295 part_ofs=61839 rule->part_size=12359 2024-06-28T15:21:54.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=74198 stripe_ofs=74198 part_ofs=74198 rule->part_size=12359 2024-06-28T15:21:54.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.17870332517152078614 2024-06-28T15:21:54.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.11__multipart_foo.2~Rlj2fxm8bYt9LC8GIdaJSFx-t7whYsg.1 2024-06-28T15:21:54.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 347 2024-06-28T15:21:54.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.11__multipart_foo.2~Rlj2fxm8bYt9LC8GIdaJSFx-t7whYsg.2 2024-06-28T15:21:54.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 593 2024-06-28T15:21:54.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.11__multipart_foo.2~Rlj2fxm8bYt9LC8GIdaJSFx-t7whYsg.3 2024-06-28T15:21:54.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 839 2024-06-28T15:21:54.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.11__multipart_foo.2~Rlj2fxm8bYt9LC8GIdaJSFx-t7whYsg.4 2024-06-28T15:21:54.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1085 2024-06-28T15:21:54.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.396+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.17870332517152078614 2024-06-28T15:21:54.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.400+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo[uCjzIbwa7v3vlbjla0PiKh0JGE4yX23] 2024-06-28T15:21:54.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.404+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa8100240 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x56149d726e08 s->prefetch_data=0 2024-06-28T15:21:54.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.408+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.11.0 2024-06-28T15:21:54.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.408+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.11.0 2024-06-28T15:21:54.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.408+0000 7f3fb21ba980 20 list_objects_ordered: starting attempt 1 2024-06-28T15:21:54.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.408+0000 7f3fb21ba980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-06-28T15:21:54.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.412+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-06-28T15:21:54.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.412+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-06-28T15:21:54.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.412+0000 7f3fb21ba980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-06-28T15:21:54.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.412+0000 7f3fb21ba980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-06-28T15:21:54.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.412+0000 7f3fb21ba980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[843e24ce-085a-4043-8f55-38e05370e790.4686.11])"; uploads.size()=0; is_truncated=0 2024-06-28T15:21:54.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.412+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2024-06-28T15:21:54.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.412+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.412+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=237 2024-06-28T15:21:54.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.412+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x11 2024-06-28T15:21:54.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.412+0000 7f3fb21ba980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2024-06-28T15:21:54.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.416+0000 7f3fb21ba980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2024-06-28T15:21:54.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.416+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1, ofs0, ns] 2024-06-28T15:21:54.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.416+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660504 cookie 94646540640160 notifier 6036 bl.length()=193 2024-06-28T15:21:54.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.420+0000 7f3fb21ba980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:843e24ce-085a-4043-8f55-38e05370e790.4686.11 from cache 2024-06-28T15:21:54.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.420+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:843e24ce-085a-4043-8f55-38e05370e790.4686.11, ofs0, ns] 2024-06-28T15:21:54.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.424+0000 7f3e8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660499 cookie 94646540649792 notifier 6036 bl.length()=251 2024-06-28T15:21:54.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.428+0000 7f3fb21ba980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/843e24ce-085a-4043-8f55-38e05370e790.4686.11 : miss 2024-06-28T15:21:54.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.428+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.428+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:54.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.428+0000 7f3fb21ba980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/843e24ce-085a-4043-8f55-38e05370e790.4686.11 info.flags=0x0 2024-06-28T15:21:54.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.428+0000 7f3fb21ba980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/843e24ce-085a-4043-8f55-38e05370e790.4686.11 to cache LRU end 2024-06-28T15:21:54.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.432+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:843e24ce-085a-4043-8f55-38e05370e790.4686.10 : miss 2024-06-28T15:21:54.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.432+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.432+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=441 2024-06-28T15:21:54.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.432+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:843e24ce-085a-4043-8f55-38e05370e790.4686.10 info.flags=0x17 2024-06-28T15:21:54.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.432+0000 7f3fb21ba980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:843e24ce-085a-4043-8f55-38e05370e790.4686.10 to cache LRU end 2024-06-28T15:21:54.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.432+0000 7f3fb21ba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:21:54.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.432+0000 7f3fb21ba980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-06-28T15:21:54.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.432+0000 7f3fb21ba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:843e24ce-085a-4043-8f55-38e05370e790.4686.10 2024-06-28T15:21:54.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.436+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo 2024-06-28T15:21:54.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.436+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d7292b0 obj=test-client.0-r-test-multipart-simple.1:foo state=0x56149d726b88 s->prefetch_data=0 2024-06-28T15:21:54.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.436+0000 7f3fb21ba980 10 manifest: total_size = 1667 2024-06-28T15:21:54.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.436+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.13889709452861551557 2024-06-28T15:21:54.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.436+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d7292b0 obj=test-client.0-r-test-multipart-simple.1:foo state=0x56149d726b88 s->prefetch_data=0 2024-06-28T15:21:54.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.436+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d726b88 2024-06-28T15:21:54.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.436+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.10.0 2024-06-28T15:21:54.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-06-28T15:21:54.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=516 2024-06-28T15:21:54.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-06-28T15:21:54.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-06-28T15:21:54.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291972 part_ofs=516 rule->part_size=516 2024-06-28T15:21:54.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=516 2024-06-28T15:21:54.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-06-28T15:21:54.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6292488 part_ofs=1032 rule->part_size=516 2024-06-28T15:21:54.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=1548 stripe_ofs=1548 part_ofs=1548 rule->part_size=119 2024-06-28T15:21:54.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=119 rules.size()=2 2024-06-28T15:21:54.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6293004 part_ofs=1548 rule->part_size=119 2024-06-28T15:21:54.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=1667 stripe_ofs=1667 part_ofs=1667 rule->part_size=119 2024-06-28T15:21:54.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.13889709452861551557 2024-06-28T15:21:54.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.10__multipart_foo.2~ciuZ3mMrUjxgn1vBVm8kyCvgCYx1c42.1 2024-06-28T15:21:54.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 347 2024-06-28T15:21:54.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.10__multipart_foo.2~ciuZ3mMrUjxgn1vBVm8kyCvgCYx1c42.2 2024-06-28T15:21:54.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 593 2024-06-28T15:21:54.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.10__multipart_foo.2~ciuZ3mMrUjxgn1vBVm8kyCvgCYx1c42.3 2024-06-28T15:21:54.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 839 2024-06-28T15:21:54.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.10__multipart_foo.2~ciuZ3mMrUjxgn1vBVm8kyCvgCYx1c42.4 2024-06-28T15:21:54.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1085 2024-06-28T15:21:54.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.440+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.13889709452861551557 2024-06-28T15:21:54.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.444+0000 7f3fb21ba980 20 list_objects_ordered: starting attempt 1 2024-06-28T15:21:54.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.444+0000 7f3fb21ba980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-06-28T15:21:54.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.444+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo 2024-06-28T15:21:54.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.444+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-06-28T15:21:54.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.444+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-06-28T15:21:54.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.444+0000 7f3fb21ba980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-06-28T15:21:54.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.444+0000 7f3fb21ba980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-06-28T15:21:54.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.444+0000 7f3fb21ba980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[843e24ce-085a-4043-8f55-38e05370e790.4686.10])"; uploads.size()=0; is_truncated=0 2024-06-28T15:21:54.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.448+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2024-06-28T15:21:54.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.448+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.448+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=227 2024-06-28T15:21:54.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.448+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x11 2024-06-28T15:21:54.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.448+0000 7f3fb21ba980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2024-06-28T15:21:54.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.452+0000 7f3fb21ba980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2024-06-28T15:21:54.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.452+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-simple.1, ofs0, ns] 2024-06-28T15:21:54.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.452+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660515 cookie 94646540646048 notifier 6036 bl.length()=183 2024-06-28T15:21:54.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.456+0000 7f3fb21ba980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:843e24ce-085a-4043-8f55-38e05370e790.4686.10 from cache 2024-06-28T15:21:54.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.456+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:843e24ce-085a-4043-8f55-38e05370e790.4686.10, ofs0, ns] 2024-06-28T15:21:54.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.460+0000 7f3e8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660501 cookie 94646540644992 notifier 6036 bl.length()=241 2024-06-28T15:21:54.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.464+0000 7f3fb21ba980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/843e24ce-085a-4043-8f55-38e05370e790.4686.10 : miss 2024-06-28T15:21:54.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.464+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.464+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:54.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.464+0000 7f3fb21ba980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/843e24ce-085a-4043-8f55-38e05370e790.4686.10 info.flags=0x0 2024-06-28T15:21:54.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.464+0000 7f3fb21ba980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/843e24ce-085a-4043-8f55-38e05370e790.4686.10 to cache LRU end 2024-06-28T15:21:54.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.468+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:843e24ce-085a-4043-8f55-38e05370e790.4686.16 : miss 2024-06-28T15:21:54.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.468+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.468+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=447 2024-06-28T15:21:54.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.468+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:843e24ce-085a-4043-8f55-38e05370e790.4686.16 info.flags=0x17 2024-06-28T15:21:54.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.468+0000 7f3fb21ba980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:843e24ce-085a-4043-8f55-38e05370e790.4686.16 to cache LRU end 2024-06-28T15:21:54.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.468+0000 7f3fb21ba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:21:54.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.468+0000 7f3fb21ba980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-06-28T15:21:54.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.468+0000 7f3fb21ba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:843e24ce-085a-4043-8f55-38e05370e790.4686.16 2024-06-28T15:21:54.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-06-28T15:21:54.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2024-06-28T15:21:54.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2024-06-28T15:21:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2024-06-28T15:21:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-06-28T15:21:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2024-06-28T15:21:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2024-06-28T15:21:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2024-06-28T15:21:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-06-28T15:21:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2024-06-28T15:21:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2024-06-28T15:21:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2024-06-28T15:21:54.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x56149d7269a8 s->prefetch_data=0 2024-06-28T15:21:54.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 10 manifest: total_size = 5242880 2024-06-28T15:21:54.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.11204990728759701358 2024-06-28T15:21:54.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x56149d7269a8 s->prefetch_data=0 2024-06-28T15:21:54.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d7269a8 2024-06-28T15:21:54.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.472+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.16.0 2024-06-28T15:21:54.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.476+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.476+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.476+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.11204990728759701358 2024-06-28T15:21:54.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.476+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.476+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.3h8nBofYqfTXfnIUkfD5984-BBtf2Gz_0 2024-06-28T15:21:54.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.476+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 338 2024-06-28T15:21:54.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.476+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.476+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.11204990728759701358 2024-06-28T15:21:54.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.480+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-06-28T15:21:54.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.480+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x56149d7269a8 s->prefetch_data=0 2024-06-28T15:21:54.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.480+0000 7f3fb21ba980 10 manifest: total_size = 5242880 2024-06-28T15:21:54.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.480+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.480+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.480+0000 7f3fb21ba980 10 generate_fake_tag new tag=843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.3h8nBofYqfTXfnIUkfD5984-BBtf2Gz_1_d827886f1ba02dcee0a9d6d5a18d293b 2024-06-28T15:21:54.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.480+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.3h8nBofYqfTXfnIUkfD5984-BBtf2Gz_1_d827886f1ba02dcee0a9d6d5a18d293b 2024-06-28T15:21:54.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.480+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x56149d7269a8 s->prefetch_data=0 2024-06-28T15:21:54.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.480+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d7269a8 2024-06-28T15:21:54.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.480+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.16.0 2024-06-28T15:21:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.488+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.488+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.488+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.3h8nBofYqfTXfnIUkfD5984-BBtf2Gz_1_d827886f1ba02dcee0a9d6d5a18d293b 2024-06-28T15:21:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.488+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.488+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.3h8nBofYqfTXfnIUkfD5984-BBtf2Gz_0 2024-06-28T15:21:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.488+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 396 2024-06-28T15:21:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.488+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.488+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.3h8nBofYqfTXfnIUkfD5984-BBtf2Gz_1_d827886f1ba02dcee0a9d6d5a18d293b 2024-06-28T15:21:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.488+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2024-06-28T15:21:54.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.492+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x56149d726498 s->prefetch_data=0 2024-06-28T15:21:54.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.492+0000 7f3fb21ba980 10 manifest: total_size = 5242880 2024-06-28T15:21:54.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.492+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to _vlahhlCU5vSdYyZi3QWYG_byviuBrE9 2024-06-28T15:21:54.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.492+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x56149d726498 s->prefetch_data=0 2024-06-28T15:21:54.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.492+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d726498 2024-06-28T15:21:54.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.492+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.16.0 2024-06-28T15:21:54.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.496+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.496+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.496+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: _vlahhlCU5vSdYyZi3QWYG_byviuBrE9 2024-06-28T15:21:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.496+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.496+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.M57Kjtb2oYS-KjgBnBasqr2QasddNuR_0 2024-06-28T15:21:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.496+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-06-28T15:21:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.496+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.496+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.2tag is: _vlahhlCU5vSdYyZi3QWYG_byviuBrE9 2024-06-28T15:21:54.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.500+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2024-06-28T15:21:54.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.500+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d72c060 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x56149d724d48 s->prefetch_data=0 2024-06-28T15:21:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.500+0000 7f3fb21ba980 10 manifest: total_size = 5242880 2024-06-28T15:21:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.500+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to _XdGhjbaiCXQ-Jjz24bkSGLEwTMQoFQ4 2024-06-28T15:21:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.500+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d72c060 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x56149d724d48 s->prefetch_data=0 2024-06-28T15:21:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.500+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d724d48 2024-06-28T15:21:54.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.500+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.16.0 2024-06-28T15:21:54.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.508+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.508+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.508+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: _XdGhjbaiCXQ-Jjz24bkSGLEwTMQoFQ4 2024-06-28T15:21:54.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.508+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.508+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.8ycHKyODPBp9Ax5XUWBD8C5AqbYFdtg_0 2024-06-28T15:21:54.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.508+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-06-28T15:21:54.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.508+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.508+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: _XdGhjbaiCXQ-Jjz24bkSGLEwTMQoFQ4 2024-06-28T15:21:54.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.508+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2024-06-28T15:21:54.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.512+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x56149d724d48 s->prefetch_data=0 2024-06-28T15:21:54.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.512+0000 7f3fb21ba980 10 manifest: total_size = 5242880 2024-06-28T15:21:54.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.512+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.1881584416359652293 2024-06-28T15:21:54.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.512+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x56149d724d48 s->prefetch_data=0 2024-06-28T15:21:54.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.512+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d724d48 2024-06-28T15:21:54.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.512+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.16.0 2024-06-28T15:21:54.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.516+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.516+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.516+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.1881584416359652293 2024-06-28T15:21:54.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.516+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.516+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.05oqeZkcZBXD7L5ZKW3Yg7ODJxNZ8DC_0 2024-06-28T15:21:54.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.516+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 339 2024-06-28T15:21:54.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.516+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.516+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.1881584416359652293 2024-06-28T15:21:54.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.520+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa80f6c90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x56149d724d48 s->prefetch_data=0 2024-06-28T15:21:54.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.524+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-06-28T15:21:54.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.524+0000 7f3fb21ba980 10 manifest: total_size = 5242880 2024-06-28T15:21:54.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.524+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to _KQFYKaTh-9doPnysAYflcKpbO6SbcgV 2024-06-28T15:21:54.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.524+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa80f6c90 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x56149d724d48 s->prefetch_data=0 2024-06-28T15:21:54.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.524+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d724d48 2024-06-28T15:21:54.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.524+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.16.0 2024-06-28T15:21:54.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.528+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.528+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.528+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: _KQFYKaTh-9doPnysAYflcKpbO6SbcgV 2024-06-28T15:21:54.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.528+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.528+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.NIcKd001J0p3V_9x5kkKjpF57U6PSFl_0 2024-06-28T15:21:54.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.528+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-06-28T15:21:54.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.528+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.528+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: _KQFYKaTh-9doPnysAYflcKpbO6SbcgV 2024-06-28T15:21:54.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.532+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2024-06-28T15:21:54.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.532+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x56149d724d48 s->prefetch_data=0 2024-06-28T15:21:54.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.532+0000 7f3fb21ba980 10 manifest: total_size = 5242880 2024-06-28T15:21:54.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.532+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.532+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.532+0000 7f3fb21ba980 10 generate_fake_tag new tag=843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.05oqeZkcZBXD7L5ZKW3Yg7ODJxNZ8DC_1_042f62947b5bb2063ce9c3578e63f830 2024-06-28T15:21:54.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.532+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.05oqeZkcZBXD7L5ZKW3Yg7ODJxNZ8DC_1_042f62947b5bb2063ce9c3578e63f830 2024-06-28T15:21:54.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.532+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x56149d724d48 s->prefetch_data=0 2024-06-28T15:21:54.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.532+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d724d48 2024-06-28T15:21:54.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.532+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.16.0 2024-06-28T15:21:54.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.536+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.536+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.536+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.05oqeZkcZBXD7L5ZKW3Yg7ODJxNZ8DC_1_042f62947b5bb2063ce9c3578e63f830 2024-06-28T15:21:54.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.536+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.536+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.05oqeZkcZBXD7L5ZKW3Yg7ODJxNZ8DC_0 2024-06-28T15:21:54.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.536+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 398 2024-06-28T15:21:54.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.536+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.536+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.05oqeZkcZBXD7L5ZKW3Yg7ODJxNZ8DC_1_042f62947b5bb2063ce9c3578e63f830 2024-06-28T15:21:54.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.540+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2024-06-28T15:21:54.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.540+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x56149d733768 s->prefetch_data=0 2024-06-28T15:21:54.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.544+0000 7f3fb21ba980 10 manifest: total_size = 5242880 2024-06-28T15:21:54.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.544+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to _4CudkS5lgEZ8zloP_ToCgshC19S6yZY 2024-06-28T15:21:54.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.544+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x56149d733768 s->prefetch_data=0 2024-06-28T15:21:54.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.544+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d733768 2024-06-28T15:21:54.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.544+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.16.0 2024-06-28T15:21:54.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.548+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.548+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.548+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: _4CudkS5lgEZ8zloP_ToCgshC19S6yZY 2024-06-28T15:21:54.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.548+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.548+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.xxk4_Dc-vtTqwKoHhv8M6X0Wl-DWaYh_0 2024-06-28T15:21:54.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.548+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-06-28T15:21:54.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.548+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.548+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: _4CudkS5lgEZ8zloP_ToCgshC19S6yZY 2024-06-28T15:21:54.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.552+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2024-06-28T15:21:54.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.552+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa80fcc80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x56149d733768 s->prefetch_data=0 2024-06-28T15:21:54.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.552+0000 7f3fb21ba980 10 manifest: total_size = 324647 2024-06-28T15:21:54.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.552+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.2114521308000306919 2024-06-28T15:21:54.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.552+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa80fcc80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x56149d733768 s->prefetch_data=0 2024-06-28T15:21:54.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.552+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d733768 2024-06-28T15:21:54.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.552+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.16.0 2024-06-28T15:21:54.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.560+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.560+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=324647 stripe_ofs=324647 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.560+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.2114521308000306919 2024-06-28T15:21:54.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.560+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.560+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.0_BMyaWYD_CRK7ZjgmtPTdbTcrC3iWs_0 2024-06-28T15:21:54.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.560+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 330 2024-06-28T15:21:54.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.560+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.560+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.2114521308000306919 2024-06-28T15:21:54.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.560+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d72d680 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x56149d733768 s->prefetch_data=0 2024-06-28T15:21:54.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.560+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-06-28T15:21:54.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.560+0000 7f3fb21ba980 10 manifest: total_size = 324647 2024-06-28T15:21:54.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.564+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to _-qSAQ1IQaDM8KYlGdteAQi16c6bS2_g 2024-06-28T15:21:54.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.564+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d72d680 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x56149d733768 s->prefetch_data=0 2024-06-28T15:21:54.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.564+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d733768 2024-06-28T15:21:54.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.564+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.16.0 2024-06-28T15:21:54.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.568+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.568+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=324647 stripe_ofs=324647 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.568+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: _-qSAQ1IQaDM8KYlGdteAQi16c6bS2_g 2024-06-28T15:21:54.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.568+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.568+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.0I-_usiTRLpI1DB-RKeIDUyxHE4rJsB_0 2024-06-28T15:21:54.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.568+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-06-28T15:21:54.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.568+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.568+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: _-qSAQ1IQaDM8KYlGdteAQi16c6bS2_g 2024-06-28T15:21:54.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.572+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x56149d733768 s->prefetch_data=0 2024-06-28T15:21:54.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.572+0000 7f3fb21ba980 10 manifest: total_size = 324647 2024-06-28T15:21:54.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.572+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to _vwMLj2567_mZeHpWnpIRgpi7ztcAeld 2024-06-28T15:21:54.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.572+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x56149d733768 s->prefetch_data=0 2024-06-28T15:21:54.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.572+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d733768 2024-06-28T15:21:54.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.572+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2024-06-28T15:21:54.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.572+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.16.0 2024-06-28T15:21:54.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.580+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.580+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=324647 stripe_ofs=324647 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.580+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: _vwMLj2567_mZeHpWnpIRgpi7ztcAeld 2024-06-28T15:21:54.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.580+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.580+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.AsKF9D4X18jdffSCO2aq_O7jMedmfRM_0 2024-06-28T15:21:54.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.580+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-06-28T15:21:54.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.580+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.580+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.27tag is: _vwMLj2567_mZeHpWnpIRgpi7ztcAeld 2024-06-28T15:21:54.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.580+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2024-06-28T15:21:54.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.584+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x56149d724d48 s->prefetch_data=0 2024-06-28T15:21:54.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.584+0000 7f3fb21ba980 10 manifest: total_size = 324647 2024-06-28T15:21:54.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.584+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.584+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=324647 stripe_ofs=324647 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.584+0000 7f3fb21ba980 10 generate_fake_tag new tag=843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.0_BMyaWYD_CRK7ZjgmtPTdbTcrC3iWs_1_73aac645e1e440535c1c09f0714b79ce 2024-06-28T15:21:54.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.584+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.0_BMyaWYD_CRK7ZjgmtPTdbTcrC3iWs_1_73aac645e1e440535c1c09f0714b79ce 2024-06-28T15:21:54.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.584+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x56149d724d48 s->prefetch_data=0 2024-06-28T15:21:54.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.584+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d724d48 2024-06-28T15:21:54.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.584+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.16.0 2024-06-28T15:21:54.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.588+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.588+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=324647 stripe_ofs=324647 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.588+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.0_BMyaWYD_CRK7ZjgmtPTdbTcrC3iWs_1_73aac645e1e440535c1c09f0714b79ce 2024-06-28T15:21:54.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.588+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.588+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.0_BMyaWYD_CRK7ZjgmtPTdbTcrC3iWs_0 2024-06-28T15:21:54.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.588+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 389 2024-06-28T15:21:54.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.588+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.588+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.20tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.0_BMyaWYD_CRK7ZjgmtPTdbTcrC3iWs_1_73aac645e1e440535c1c09f0714b79ce 2024-06-28T15:21:54.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.592+0000 7f3fb21ba980 20 list_objects_ordered: starting attempt 1 2024-06-28T15:21:54.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.592+0000 7f3fb21ba980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-06-28T15:21:54.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.592+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2024-06-28T15:21:54.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.592+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-06-28T15:21:54.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.592+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-06-28T15:21:54.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.592+0000 7f3fb21ba980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-06-28T15:21:54.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.592+0000 7f3fb21ba980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-06-28T15:21:54.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.592+0000 7f3fb21ba980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[843e24ce-085a-4043-8f55-38e05370e790.4686.16])"; uploads.size()=0; is_truncated=0 2024-06-28T15:21:54.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.596+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2024-06-28T15:21:54.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.596+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.596+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=233 2024-06-28T15:21:54.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.596+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x11 2024-06-28T15:21:54.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.596+0000 7f3fb21ba980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2024-06-28T15:21:54.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.600+0000 7f3fb21ba980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2024-06-28T15:21:54.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.600+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1, ofs0, ns] 2024-06-28T15:21:54.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.600+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660505 cookie 94646540640160 notifier 6036 bl.length()=189 2024-06-28T15:21:54.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.608+0000 7f3fb21ba980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:843e24ce-085a-4043-8f55-38e05370e790.4686.16 from cache 2024-06-28T15:21:54.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.608+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:843e24ce-085a-4043-8f55-38e05370e790.4686.16, ofs0, ns] 2024-06-28T15:21:54.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.608+0000 7f3e8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660500 cookie 94646540649792 notifier 6036 bl.length()=247 2024-06-28T15:21:54.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.616+0000 7f3fb21ba980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/843e24ce-085a-4043-8f55-38e05370e790.4686.16 : miss 2024-06-28T15:21:54.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.616+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.616+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:54.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.616+0000 7f3fb21ba980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/843e24ce-085a-4043-8f55-38e05370e790.4686.16 info.flags=0x0 2024-06-28T15:21:54.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.616+0000 7f3fb21ba980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/843e24ce-085a-4043-8f55-38e05370e790.4686.16 to cache LRU end 2024-06-28T15:21:54.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.620+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:843e24ce-085a-4043-8f55-38e05370e790.4686.17 : miss 2024-06-28T15:21:54.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.620+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=457 2024-06-28T15:21:54.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:843e24ce-085a-4043-8f55-38e05370e790.4686.17 info.flags=0x17 2024-06-28T15:21:54.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:843e24ce-085a-4043-8f55-38e05370e790.4686.17 to cache LRU end 2024-06-28T15:21:54.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:21:54.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-06-28T15:21:54.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:843e24ce-085a-4043-8f55-38e05370e790.4686.17 2024-06-28T15:21:54.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-06-28T15:21:54.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2024-06-28T15:21:54.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2024-06-28T15:21:54.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2024-06-28T15:21:54.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-06-28T15:21:54.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2024-06-28T15:21:54.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2024-06-28T15:21:54.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2024-06-28T15:21:54.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-06-28T15:21:54.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2024-06-28T15:21:54.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2024-06-28T15:21:54.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2024-06-28T15:21:54.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa80f3960 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 10 manifest: total_size = 11534336 2024-06-28T15:21:54.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.2743304086958580136 2024-06-28T15:21:54.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa80f3960 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d72a6e8 2024-06-28T15:21:54.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.624+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.17.0 2024-06-28T15:21:54.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-06-28T15:21:54.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-06-28T15:21:54.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-06-28T15:21:54.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.2743304086958580136 2024-06-28T15:21:54.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.1 2024-06-28T15:21:54.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 367 2024-06-28T15:21:54.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.2 2024-06-28T15:21:54.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 634 2024-06-28T15:21:54.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.3 2024-06-28T15:21:54.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 901 2024-06-28T15:21:54.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.632+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.2743304086958580136 2024-06-28T15:21:54.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.636+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa80f3960 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.636+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-06-28T15:21:54.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.636+0000 7f3fb21ba980 10 manifest: total_size = 11534336 2024-06-28T15:21:54.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.636+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.636+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-06-28T15:21:54.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.636+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.636+0000 7f3fb21ba980 10 generate_fake_tag new tag=843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.2_724698bb6f712eb05c7aa6a9df77043f 2024-06-28T15:21:54.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.636+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.2_724698bb6f712eb05c7aa6a9df77043f 2024-06-28T15:21:54.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.636+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa80f3960 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.636+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d72a6e8 2024-06-28T15:21:54.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.636+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.17.0 2024-06-28T15:21:54.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-06-28T15:21:54.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-06-28T15:21:54.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-06-28T15:21:54.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.2_724698bb6f712eb05c7aa6a9df77043f 2024-06-28T15:21:54.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.1 2024-06-28T15:21:54.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 441 2024-06-28T15:21:54.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.2 2024-06-28T15:21:54.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 708 2024-06-28T15:21:54.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.3 2024-06-28T15:21:54.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 975 2024-06-28T15:21:54.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.640+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.2_724698bb6f712eb05c7aa6a9df77043f 2024-06-28T15:21:54.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.644+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2024-06-28T15:21:54.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.644+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d72c0f0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.648+0000 7f3fb21ba980 10 manifest: total_size = 11534336 2024-06-28T15:21:54.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.648+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to _T11QvOyYfRM1_IJbyrkT5fpPOifhl8U 2024-06-28T15:21:54.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.648+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d72c0f0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.648+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d72a6e8 2024-06-28T15:21:54.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.648+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.17.0 2024-06-28T15:21:54.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.652+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.652+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.652+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.652+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.652+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: _T11QvOyYfRM1_IJbyrkT5fpPOifhl8U 2024-06-28T15:21:54.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.652+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.652+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.DfNtUAZMT2iMHx7mS9oU5rVOV2zY7BZ_0 2024-06-28T15:21:54.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.652+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-06-28T15:21:54.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.652+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.DfNtUAZMT2iMHx7mS9oU5rVOV2zY7BZ_1 2024-06-28T15:21:54.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.652+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 549 2024-06-28T15:21:54.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.652+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.652+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.21tag is: _T11QvOyYfRM1_IJbyrkT5fpPOifhl8U 2024-06-28T15:21:54.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.656+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2024-06-28T15:21:54.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.656+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa80f3960 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.656+0000 7f3fb21ba980 10 manifest: total_size = 11534336 2024-06-28T15:21:54.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.656+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to _2pL3HWtQ7O3f6zEI9QjB7PvdTs8Wcq6 2024-06-28T15:21:54.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.656+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa80f3960 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.656+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d72a6e8 2024-06-28T15:21:54.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.656+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.17.0 2024-06-28T15:21:54.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.664+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.664+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.664+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.664+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.664+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: _2pL3HWtQ7O3f6zEI9QjB7PvdTs8Wcq6 2024-06-28T15:21:54.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.664+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.664+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.8Yroa_OsiFNVbq-v8nKUsVhPyWEjPpP_0 2024-06-28T15:21:54.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.664+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-06-28T15:21:54.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.664+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.8Yroa_OsiFNVbq-v8nKUsVhPyWEjPpP_1 2024-06-28T15:21:54.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.664+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-06-28T15:21:54.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.664+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.664+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: _2pL3HWtQ7O3f6zEI9QjB7PvdTs8Wcq6 2024-06-28T15:21:54.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.664+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2024-06-28T15:21:54.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.668+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d73bef0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.668+0000 7f3fb21ba980 10 manifest: total_size = 11534336 2024-06-28T15:21:54.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.668+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.8666825984778865762 2024-06-28T15:21:54.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.668+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d73bef0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.668+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d72a6e8 2024-06-28T15:21:54.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.668+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.17.0 2024-06-28T15:21:54.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-06-28T15:21:54.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-06-28T15:21:54.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-06-28T15:21:54.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.8666825984778865762 2024-06-28T15:21:54.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.1 2024-06-28T15:21:54.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 373 2024-06-28T15:21:54.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.2 2024-06-28T15:21:54.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 646 2024-06-28T15:21:54.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.3 2024-06-28T15:21:54.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 919 2024-06-28T15:21:54.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.676+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.8666825984778865762 2024-06-28T15:21:54.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.680+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-06-28T15:21:54.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.680+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d726b20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.680+0000 7f3fb21ba980 10 manifest: total_size = 11534336 2024-06-28T15:21:54.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.680+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to _XLf-MakSHGTjxFKDyWrzLBCYGLStajN 2024-06-28T15:21:54.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.680+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d726b20 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.680+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d72a6e8 2024-06-28T15:21:54.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.680+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.17.0 2024-06-28T15:21:54.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.688+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.688+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.688+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.688+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.688+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: _XLf-MakSHGTjxFKDyWrzLBCYGLStajN 2024-06-28T15:21:54.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.688+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.688+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.IWQLiw9c7uiRkdJmU5Eqz_8U3xfa2BA_0 2024-06-28T15:21:54.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.688+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-06-28T15:21:54.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.688+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.IWQLiw9c7uiRkdJmU5Eqz_8U3xfa2BA_1 2024-06-28T15:21:54.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.688+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 545 2024-06-28T15:21:54.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.688+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.688+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.8tag is: _XLf-MakSHGTjxFKDyWrzLBCYGLStajN 2024-06-28T15:21:54.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.692+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2024-06-28T15:21:54.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.692+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f98037530 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.692+0000 7f3fb21ba980 10 manifest: total_size = 11534336 2024-06-28T15:21:54.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.692+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.692+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-06-28T15:21:54.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.692+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.692+0000 7f3fb21ba980 10 generate_fake_tag new tag=843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.2_59e5b182d51295a5faf47e285dd68abe 2024-06-28T15:21:54.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.692+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.2_59e5b182d51295a5faf47e285dd68abe 2024-06-28T15:21:54.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.692+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f98037530 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.692+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d72a6e8 2024-06-28T15:21:54.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.692+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.17.0 2024-06-28T15:21:54.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-06-28T15:21:54.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-06-28T15:21:54.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-06-28T15:21:54.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.2_59e5b182d51295a5faf47e285dd68abe 2024-06-28T15:21:54.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.1 2024-06-28T15:21:54.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 449 2024-06-28T15:21:54.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.2 2024-06-28T15:21:54.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 722 2024-06-28T15:21:54.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.3 2024-06-28T15:21:54.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 995 2024-06-28T15:21:54.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.696+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.2_59e5b182d51295a5faf47e285dd68abe 2024-06-28T15:21:54.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.700+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2024-06-28T15:21:54.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.700+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.700+0000 7f3fb21ba980 10 manifest: total_size = 11534336 2024-06-28T15:21:54.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.700+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to _GkAN5wu5sM0-Wse6E7A9jzdb6QWLzPh 2024-06-28T15:21:54.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.700+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.700+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d72a6e8 2024-06-28T15:21:54.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.700+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.17.0 2024-06-28T15:21:54.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.708+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.708+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.708+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.708+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.708+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: _GkAN5wu5sM0-Wse6E7A9jzdb6QWLzPh 2024-06-28T15:21:54.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.708+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.708+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.UwlAV0X9NBMdkaVOr0E601-Niw-N0LA_0 2024-06-28T15:21:54.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.708+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-06-28T15:21:54.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.708+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.UwlAV0X9NBMdkaVOr0E601-Niw-N0LA_1 2024-06-28T15:21:54.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.708+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-06-28T15:21:54.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.708+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.708+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.10tag is: _GkAN5wu5sM0-Wse6E7A9jzdb6QWLzPh 2024-06-28T15:21:54.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.712+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f98037530 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.712+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2024-06-28T15:21:54.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.712+0000 7f3fb21ba980 10 manifest: total_size = 11534336 2024-06-28T15:21:54.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.712+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.3382539424937402029 2024-06-28T15:21:54.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.712+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f98037530 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.712+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d72a6e8 2024-06-28T15:21:54.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.712+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.17.0 2024-06-28T15:21:54.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-06-28T15:21:54.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-06-28T15:21:54.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-06-28T15:21:54.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.3382539424937402029 2024-06-28T15:21:54.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.1 2024-06-28T15:21:54.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 364 2024-06-28T15:21:54.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.2 2024-06-28T15:21:54.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 628 2024-06-28T15:21:54.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.3 2024-06-28T15:21:54.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 892 2024-06-28T15:21:54.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.720+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.10tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.3382539424937402029 2024-06-28T15:21:54.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.724+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-06-28T15:21:54.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.724+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa80f3960 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.724+0000 7f3fb21ba980 10 manifest: total_size = 11534336 2024-06-28T15:21:54.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.724+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to _dnTwCM1lBSW-j3HTwp92fx5BeGNM0-X 2024-06-28T15:21:54.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.724+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa80f3960 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.724+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d72a6e8 2024-06-28T15:21:54.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.724+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.17.0 2024-06-28T15:21:54.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.732+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.732+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.732+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.732+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.732+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: _dnTwCM1lBSW-j3HTwp92fx5BeGNM0-X 2024-06-28T15:21:54.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.732+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.732+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.rjPary_F-RFuLp1X7o4lYwIO8gv0gPg_0 2024-06-28T15:21:54.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.732+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-06-28T15:21:54.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.732+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.rjPary_F-RFuLp1X7o4lYwIO8gv0gPg_1 2024-06-28T15:21:54.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.732+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 545 2024-06-28T15:21:54.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.732+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.732+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: _dnTwCM1lBSW-j3HTwp92fx5BeGNM0-X 2024-06-28T15:21:54.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.732+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2024-06-28T15:21:54.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.732+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.736+0000 7f3fb21ba980 10 manifest: total_size = 11534336 2024-06-28T15:21:54.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.736+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to __HbLmKkyqGHBakVveXe42oUiQiLzywV 2024-06-28T15:21:54.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.736+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.736+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d72a6e8 2024-06-28T15:21:54.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.736+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.17.0 2024-06-28T15:21:54.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.740+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.740+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.740+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.740+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.740+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: __HbLmKkyqGHBakVveXe42oUiQiLzywV 2024-06-28T15:21:54.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.740+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.740+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.H0s5D5UYzconMghB09AAKEVcheSCumD_0 2024-06-28T15:21:54.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.740+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-06-28T15:21:54.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.740+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.H0s5D5UYzconMghB09AAKEVcheSCumD_1 2024-06-28T15:21:54.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.740+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 549 2024-06-28T15:21:54.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.740+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.740+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.21tag is: __HbLmKkyqGHBakVveXe42oUiQiLzywV 2024-06-28T15:21:54.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.744+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2024-06-28T15:21:54.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.744+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f98037530 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.744+0000 7f3fb21ba980 10 manifest: total_size = 11534336 2024-06-28T15:21:54.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.744+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.744+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-06-28T15:21:54.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.744+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.744+0000 7f3fb21ba980 10 generate_fake_tag new tag=843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.2_16a036810e9433481d99fdfb9e1f2f8d 2024-06-28T15:21:54.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.744+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.2_16a036810e9433481d99fdfb9e1f2f8d 2024-06-28T15:21:54.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.744+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f98037530 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.744+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d72a6e8 2024-06-28T15:21:54.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.744+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.17.0 2024-06-28T15:21:54.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-06-28T15:21:54.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-06-28T15:21:54.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-06-28T15:21:54.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.2_16a036810e9433481d99fdfb9e1f2f8d 2024-06-28T15:21:54.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.1 2024-06-28T15:21:54.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 440 2024-06-28T15:21:54.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.2 2024-06-28T15:21:54.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 704 2024-06-28T15:21:54.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.3 2024-06-28T15:21:54.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 968 2024-06-28T15:21:54.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.2_16a036810e9433481d99fdfb9e1f2f8d 2024-06-28T15:21:54.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2024-06-28T15:21:54.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 20 list_objects_ordered: starting attempt 1 2024-06-28T15:21:54.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.752+0000 7f3fb21ba980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-06-28T15:21:54.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.756+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-06-28T15:21:54.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.756+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-06-28T15:21:54.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.756+0000 7f3fb21ba980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-06-28T15:21:54.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.756+0000 7f3fb21ba980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-06-28T15:21:54.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.756+0000 7f3fb21ba980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[843e24ce-085a-4043-8f55-38e05370e790.4686.17])"; uploads.size()=0; is_truncated=0 2024-06-28T15:21:54.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.760+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2024-06-28T15:21:54.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.760+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.760+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=243 2024-06-28T15:21:54.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.760+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x11 2024-06-28T15:21:54.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.760+0000 7f3fb21ba980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2024-06-28T15:21:54.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.764+0000 7f3fb21ba980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2024-06-28T15:21:54.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.764+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1, ofs0, ns] 2024-06-28T15:21:54.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.764+0000 7f3e8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660502 cookie 94646540644992 notifier 6036 bl.length()=199 2024-06-28T15:21:54.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.768+0000 7f3fb21ba980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:843e24ce-085a-4043-8f55-38e05370e790.4686.17 from cache 2024-06-28T15:21:54.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.768+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:843e24ce-085a-4043-8f55-38e05370e790.4686.17, ofs0, ns] 2024-06-28T15:21:54.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.772+0000 7f3e8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660516 cookie 94646540629632 notifier 6036 bl.length()=257 2024-06-28T15:21:54.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.776+0000 7f3fb21ba980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/843e24ce-085a-4043-8f55-38e05370e790.4686.17 : miss 2024-06-28T15:21:54.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.776+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.776+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:54.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.776+0000 7f3fb21ba980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/843e24ce-085a-4043-8f55-38e05370e790.4686.17 info.flags=0x0 2024-06-28T15:21:54.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.776+0000 7f3fb21ba980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/843e24ce-085a-4043-8f55-38e05370e790.4686.17 to cache LRU end 2024-06-28T15:21:54.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.780+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:843e24ce-085a-4043-8f55-38e05370e790.4686.15 : miss 2024-06-28T15:21:54.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.780+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.780+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=452 2024-06-28T15:21:54.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.780+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:843e24ce-085a-4043-8f55-38e05370e790.4686.15 info.flags=0x17 2024-06-28T15:21:54.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.780+0000 7f3fb21ba980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:843e24ce-085a-4043-8f55-38e05370e790.4686.15 to cache LRU end 2024-06-28T15:21:54.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.780+0000 7f3fb21ba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:21:54.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.780+0000 7f3fb21ba980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-06-28T15:21:54.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.780+0000 7f3fb21ba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:843e24ce-085a-4043-8f55-38e05370e790.4686.15 2024-06-28T15:21:54.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.784+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-06-28T15:21:54.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.784+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-06-28T15:21:54.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.784+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-06-28T15:21:54.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.784+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d7339d0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x56149d73cf08 s->prefetch_data=0 2024-06-28T15:21:54.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.784+0000 7f3fb21ba980 10 manifest: total_size = 11534336 2024-06-28T15:21:54.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.784+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.14317980836071081796 2024-06-28T15:21:54.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.784+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d7339d0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x56149d73cf08 s->prefetch_data=0 2024-06-28T15:21:54.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.784+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d73cf08 2024-06-28T15:21:54.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.784+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.15.0 2024-06-28T15:21:54.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-06-28T15:21:54.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-06-28T15:21:54.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-06-28T15:21:54.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.14317980836071081796 2024-06-28T15:21:54.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-FROZEN.2~rT2zkyXXyEwXPN-6nXcB-u7-as7cHJB.1 2024-06-28T15:21:54.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 368 2024-06-28T15:21:54.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-FROZEN.2~rT2zkyXXyEwXPN-6nXcB-u7-as7cHJB.2 2024-06-28T15:21:54.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 635 2024-06-28T15:21:54.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-FROZEN.2~rT2zkyXXyEwXPN-6nXcB-u7-as7cHJB.3 2024-06-28T15:21:54.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 902 2024-06-28T15:21:54.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.788+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.14317980836071081796 2024-06-28T15:21:54.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.792+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-06-28T15:21:54.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.792+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9805ef20 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x56149d724ca8 s->prefetch_data=0 2024-06-28T15:21:54.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.792+0000 7f3fb21ba980 10 manifest: total_size = 11534336 2024-06-28T15:21:54.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.792+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.3749436943869302853 2024-06-28T15:21:54.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.792+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3f9805ef20 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x56149d724ca8 s->prefetch_data=0 2024-06-28T15:21:54.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.792+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d724ca8 2024-06-28T15:21:54.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.792+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.15.0 2024-06-28T15:21:54.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-06-28T15:21:54.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-06-28T15:21:54.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-06-28T15:21:54.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-06-28T15:21:54.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-06-28T15:21:54.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-06-28T15:21:54.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.3749436943869302853 2024-06-28T15:21:54.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-LUKEWARM.2~0yRDU9th93HT14K4ALzXZT0n3impj1A.1 2024-06-28T15:21:54.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 373 2024-06-28T15:21:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-LUKEWARM.2~0yRDU9th93HT14K4ALzXZT0n3impj1A.2 2024-06-28T15:21:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 646 2024-06-28T15:21:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-LUKEWARM.2~0yRDU9th93HT14K4ALzXZT0n3impj1A.3 2024-06-28T15:21:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 919 2024-06-28T15:21:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.800+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.20tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.3749436943869302853 2024-06-28T15:21:54.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.804+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-06-28T15:21:54.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.804+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d733950 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x56149d73c9f8 s->prefetch_data=0 2024-06-28T15:21:54.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.804+0000 7f3fb21ba980 10 manifest: total_size = 16377 2024-06-28T15:21:54.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.804+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.1332505300669175676 2024-06-28T15:21:54.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.804+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d733950 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x56149d73c9f8 s->prefetch_data=0 2024-06-28T15:21:54.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.804+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d73c9f8 2024-06-28T15:21:54.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.804+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.15.0 2024-06-28T15:21:54.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=7672 rules.size()=3 2024-06-28T15:21:54.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=7672 2024-06-28T15:21:54.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=7672 stripe_ofs=7672 part_ofs=7672 rule->part_size=7670 2024-06-28T15:21:54.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=7670 rules.size()=3 2024-06-28T15:21:54.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6299128 part_ofs=7672 rule->part_size=7670 2024-06-28T15:21:54.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=15342 stripe_ofs=15342 part_ofs=15342 rule->part_size=1035 2024-06-28T15:21:54.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=1035 rules.size()=3 2024-06-28T15:21:54.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): stripe_ofs=6306798 part_ofs=15342 rule->part_size=1035 2024-06-28T15:21:54.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=16377 stripe_ofs=16377 part_ofs=16377 rule->part_size=1035 2024-06-28T15:21:54.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.1332505300669175676 2024-06-28T15:21:54.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-STANDARD.2~HW--NWmyhMyiauJwopSuQ8FEdWTm3fm.1 2024-06-28T15:21:54.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 364 2024-06-28T15:21:54.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-STANDARD.2~HW--NWmyhMyiauJwopSuQ8FEdWTm3fm.2 2024-06-28T15:21:54.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 628 2024-06-28T15:21:54.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-STANDARD.2~HW--NWmyhMyiauJwopSuQ8FEdWTm3fm.3 2024-06-28T15:21:54.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 892 2024-06-28T15:21:54.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.808+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.1332505300669175676 2024-06-28T15:21:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.812+0000 7f3fb21ba980 20 list_objects_ordered: starting attempt 1 2024-06-28T15:21:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.812+0000 7f3fb21ba980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-06-28T15:21:54.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.812+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-06-28T15:21:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.816+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-06-28T15:21:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.816+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-06-28T15:21:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.816+0000 7f3fb21ba980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-06-28T15:21:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.816+0000 7f3fb21ba980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-06-28T15:21:54.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.816+0000 7f3fb21ba980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[843e24ce-085a-4043-8f55-38e05370e790.4686.15])"; uploads.size()=0; is_truncated=0 2024-06-28T15:21:54.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.816+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2024-06-28T15:21:54.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.816+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.820+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=238 2024-06-28T15:21:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.820+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x11 2024-06-28T15:21:54.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.820+0000 7f3fb21ba980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2024-06-28T15:21:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.820+0000 7f3fb21ba980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2024-06-28T15:21:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.820+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1, ofs0, ns] 2024-06-28T15:21:54.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.820+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660501 cookie 94646540649792 notifier 6036 bl.length()=194 2024-06-28T15:21:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.828+0000 7f3fb21ba980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:843e24ce-085a-4043-8f55-38e05370e790.4686.15 from cache 2024-06-28T15:21:54.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.828+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:843e24ce-085a-4043-8f55-38e05370e790.4686.15, ofs0, ns] 2024-06-28T15:21:54.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.828+0000 7f3e8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660517 cookie 94646540629632 notifier 6036 bl.length()=252 2024-06-28T15:21:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.836+0000 7f3fb21ba980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/843e24ce-085a-4043-8f55-38e05370e790.4686.15 : miss 2024-06-28T15:21:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.836+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.836+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.836+0000 7f3fb21ba980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/843e24ce-085a-4043-8f55-38e05370e790.4686.15 info.flags=0x0 2024-06-28T15:21:54.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.836+0000 7f3fb21ba980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/843e24ce-085a-4043-8f55-38e05370e790.4686.15 to cache LRU end 2024-06-28T15:21:54.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.840+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:843e24ce-085a-4043-8f55-38e05370e790.4686.14 : miss 2024-06-28T15:21:54.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.840+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.840+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=442 2024-06-28T15:21:54.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.840+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:843e24ce-085a-4043-8f55-38e05370e790.4686.14 info.flags=0x17 2024-06-28T15:21:54.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.840+0000 7f3fb21ba980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:843e24ce-085a-4043-8f55-38e05370e790.4686.14 to cache LRU end 2024-06-28T15:21:54.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.840+0000 7f3fb21ba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:21:54.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.840+0000 7f3fb21ba980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-06-28T15:21:54.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.840+0000 7f3fb21ba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:843e24ce-085a-4043-8f55-38e05370e790.4686.14 2024-06-28T15:21:54.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.844+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-06-28T15:21:54.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.844+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-06-28T15:21:54.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.844+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-06-28T15:21:54.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.844+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d727c70 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x56149d73c9f8 s->prefetch_data=0 2024-06-28T15:21:54.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.844+0000 7f3fb21ba980 10 manifest: total_size = 10485760 2024-06-28T15:21:54.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.844+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.12565385435276146935 2024-06-28T15:21:54.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.844+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d727c70 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x56149d73c9f8 s->prefetch_data=0 2024-06-28T15:21:54.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.844+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d73c9f8 2024-06-28T15:21:54.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.844+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.14.0 2024-06-28T15:21:54.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.852+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.852+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.852+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.852+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.852+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.12565385435276146935 2024-06-28T15:21:54.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.852+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.852+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.14__shadow_.u9cb5U65wT55S3n5QR45N7N_TbXC4EC_0 2024-06-28T15:21:54.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.852+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 338 2024-06-28T15:21:54.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.852+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.14__shadow_.u9cb5U65wT55S3n5QR45N7N_TbXC4EC_1 2024-06-28T15:21:54.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.852+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 575 2024-06-28T15:21:54.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.852+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.852+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.12565385435276146935 2024-06-28T15:21:54.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.852+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-06-28T15:21:54.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.856+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d727740 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x56149d73c9f8 s->prefetch_data=0 2024-06-28T15:21:54.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.856+0000 7f3fb21ba980 10 manifest: total_size = 10485760 2024-06-28T15:21:54.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.856+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.1680904577117860548 2024-06-28T15:21:54.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.856+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d727740 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x56149d73c9f8 s->prefetch_data=0 2024-06-28T15:21:54.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.856+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d73c9f8 2024-06-28T15:21:54.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.856+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.14.0 2024-06-28T15:21:54.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.860+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.860+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.860+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.860+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.860+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.1680904577117860548 2024-06-28T15:21:54.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.860+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.860+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.14__shadow_.-1KUIHHeqH8ixt3tVlNOgPh1nuKG74M_0 2024-06-28T15:21:54.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.860+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 339 2024-06-28T15:21:54.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.860+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.14__shadow_.-1KUIHHeqH8ixt3tVlNOgPh1nuKG74M_1 2024-06-28T15:21:54.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.860+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 578 2024-06-28T15:21:54.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.860+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.860+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.1680904577117860548 2024-06-28T15:21:54.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.864+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x56149d73c9f8 s->prefetch_data=0 2024-06-28T15:21:54.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.864+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-06-28T15:21:54.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.864+0000 7f3fb21ba980 10 manifest: total_size = 3312 2024-06-28T15:21:54.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.864+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.8489236377501723306 2024-06-28T15:21:54.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.864+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa0030450 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x56149d73c9f8 s->prefetch_data=0 2024-06-28T15:21:54.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.864+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d73c9f8 2024-06-28T15:21:54.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.864+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.14.0 2024-06-28T15:21:54.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.872+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.872+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=3312 stripe_ofs=3312 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.872+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.8489236377501723306 2024-06-28T15:21:54.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.872+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.872+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.14__shadow_.HRrjuKiJfl1-hn79QLqwfDgeRPybSoP_0 2024-06-28T15:21:54.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.872+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 330 2024-06-28T15:21:54.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.872+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.872+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.8489236377501723306 2024-06-28T15:21:54.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.876+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-06-28T15:21:54.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.876+0000 7f3fb21ba980 20 list_objects_ordered: starting attempt 1 2024-06-28T15:21:54.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.876+0000 7f3fb21ba980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-06-28T15:21:54.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.876+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-06-28T15:21:54.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.876+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-06-28T15:21:54.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.876+0000 7f3fb21ba980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-06-28T15:21:54.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.876+0000 7f3fb21ba980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-06-28T15:21:54.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.876+0000 7f3fb21ba980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[843e24ce-085a-4043-8f55-38e05370e790.4686.14])"; uploads.size()=0; is_truncated=0 2024-06-28T15:21:54.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.880+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2024-06-28T15:21:54.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.880+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.880+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=228 2024-06-28T15:21:54.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.880+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x11 2024-06-28T15:21:54.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.880+0000 7f3fb21ba980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2024-06-28T15:21:54.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.884+0000 7f3fb21ba980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2024-06-28T15:21:54.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.884+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1, ofs0, ns] 2024-06-28T15:21:54.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.884+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660506 cookie 94646540640160 notifier 6036 bl.length()=184 2024-06-28T15:21:54.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.892+0000 7f3fb21ba980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:843e24ce-085a-4043-8f55-38e05370e790.4686.14 from cache 2024-06-28T15:21:54.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.892+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:843e24ce-085a-4043-8f55-38e05370e790.4686.14, ofs0, ns] 2024-06-28T15:21:54.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.892+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660487 cookie 94646540654496 notifier 6036 bl.length()=242 2024-06-28T15:21:54.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.896+0000 7f3fb21ba980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/843e24ce-085a-4043-8f55-38e05370e790.4686.14 : miss 2024-06-28T15:21:54.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.896+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.896+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:54.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.896+0000 7f3fb21ba980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/843e24ce-085a-4043-8f55-38e05370e790.4686.14 info.flags=0x0 2024-06-28T15:21:54.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.896+0000 7f3fb21ba980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/843e24ce-085a-4043-8f55-38e05370e790.4686.14 to cache LRU end 2024-06-28T15:21:54.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:843e24ce-085a-4043-8f55-38e05370e790.4686.7 : miss 2024-06-28T15:21:54.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=437 2024-06-28T15:21:54.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:843e24ce-085a-4043-8f55-38e05370e790.4686.7 info.flags=0x17 2024-06-28T15:21:54.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:843e24ce-085a-4043-8f55-38e05370e790.4686.7 to cache LRU end 2024-06-28T15:21:54.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:21:54.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-06-28T15:21:54.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:843e24ce-085a-4043-8f55-38e05370e790.4686.7 2024-06-28T15:21:54.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got __ 2024-06-28T15:21:54.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got __ _ 2024-06-28T15:21:54.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got ___ 2024-06-28T15:21:54.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got obj 2024-06-28T15:21:54.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d726bf0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:54.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 10 manifest: total_size = 0 2024-06-28T15:21:54.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.14999287471024616920 2024-06-28T15:21:54.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d726bf0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:54.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d724c48 2024-06-28T15:21:54.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.904+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.7.0 2024-06-28T15:21:54.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.912+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d726bf0 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:54.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.912+0000 7f3fb21ba980 10 manifest: total_size = 0 2024-06-28T15:21:54.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.912+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.10217590709429197572 2024-06-28T15:21:54.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.912+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d726bf0 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:54.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.912+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d724c48 2024-06-28T15:21:54.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.912+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.7.0 2024-06-28T15:21:54.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.916+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=__ 2024-06-28T15:21:54.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.920+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d726bf0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:54.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.920+0000 7f3fb21ba980 10 manifest: total_size = 0 2024-06-28T15:21:54.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.920+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.997306730980876216 2024-06-28T15:21:54.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.920+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d726bf0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:54.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.920+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d724c48 2024-06-28T15:21:54.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.920+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.7.0 2024-06-28T15:21:54.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.924+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=__ _ 2024-06-28T15:21:54.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.928+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d726bf0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:54.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.928+0000 7f3fb21ba980 10 manifest: total_size = 0 2024-06-28T15:21:54.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.928+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.13888673404152937201 2024-06-28T15:21:54.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.928+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d726bf0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:54.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.928+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d724c48 2024-06-28T15:21:54.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.928+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.7.0 2024-06-28T15:21:54.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.932+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=___ 2024-06-28T15:21:54.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.936+0000 7f3fb21ba980 20 list_objects_ordered: starting attempt 1 2024-06-28T15:21:54.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.936+0000 7f3fb21ba980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-06-28T15:21:54.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.936+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=obj 2024-06-28T15:21:54.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.936+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-06-28T15:21:54.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.936+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-06-28T15:21:54.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.936+0000 7f3fb21ba980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-06-28T15:21:54.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.936+0000 7f3fb21ba980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-06-28T15:21:54.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.936+0000 7f3fb21ba980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[843e24ce-085a-4043-8f55-38e05370e790.4686.7])"; uploads.size()=0; is_truncated=0 2024-06-28T15:21:54.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.940+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2024-06-28T15:21:54.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.940+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.940+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=223 2024-06-28T15:21:54.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.940+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x11 2024-06-28T15:21:54.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.940+0000 7f3fb21ba980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2024-06-28T15:21:54.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.944+0000 7f3fb21ba980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2024-06-28T15:21:54.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.944+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.1, ofs0, ns] 2024-06-28T15:21:54.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.944+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660579 cookie 94646540639104 notifier 6036 bl.length()=181 2024-06-28T15:21:54.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.952+0000 7f3fb21ba980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:843e24ce-085a-4043-8f55-38e05370e790.4686.7 from cache 2024-06-28T15:21:54.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.952+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:843e24ce-085a-4043-8f55-38e05370e790.4686.7, ofs0, ns] 2024-06-28T15:21:54.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.952+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660503 cookie 94646540644992 notifier 6036 bl.length()=238 2024-06-28T15:21:54.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.956+0000 7f3fb21ba980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/843e24ce-085a-4043-8f55-38e05370e790.4686.7 : miss 2024-06-28T15:21:54.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.956+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.956+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:54.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.956+0000 7f3fb21ba980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/843e24ce-085a-4043-8f55-38e05370e790.4686.7 info.flags=0x0 2024-06-28T15:21:54.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.956+0000 7f3fb21ba980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/843e24ce-085a-4043-8f55-38e05370e790.4686.7 to cache LRU end 2024-06-28T15:21:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.964+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:843e24ce-085a-4043-8f55-38e05370e790.4686.8 : miss 2024-06-28T15:21:54.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.964+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.964+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=437 2024-06-28T15:21:54.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.964+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:843e24ce-085a-4043-8f55-38e05370e790.4686.8 info.flags=0x17 2024-06-28T15:21:54.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.964+0000 7f3fb21ba980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:843e24ce-085a-4043-8f55-38e05370e790.4686.8 to cache LRU end 2024-06-28T15:21:54.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.964+0000 7f3fb21ba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:21:54.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.964+0000 7f3fb21ba980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-06-28T15:21:54.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.964+0000 7f3fb21ba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:843e24ce-085a-4043-8f55-38e05370e790.4686.8 2024-06-28T15:21:54.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.964+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got __ 2024-06-28T15:21:54.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.968+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got __ _ 2024-06-28T15:21:54.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.968+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got ___ 2024-06-28T15:21:54.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.968+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got obj 2024-06-28T15:21:54.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.968+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa8100240 obj=test-client.0-r-test-small-obj-data.2:__ state=0x56149d731cd8 s->prefetch_data=0 2024-06-28T15:21:54.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.968+0000 7f3fb21ba980 10 manifest: total_size = 38 2024-06-28T15:21:54.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.968+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.2163557565277149311 2024-06-28T15:21:54.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.968+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa8100240 obj=test-client.0-r-test-small-obj-data.2:__ state=0x56149d731cd8 s->prefetch_data=0 2024-06-28T15:21:54.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.968+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d731cd8 2024-06-28T15:21:54.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.968+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.8.0 2024-06-28T15:21:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.972+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.972+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=38 stripe_ofs=38 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.972+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.2163557565277149311 2024-06-28T15:21:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.972+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.972+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.8__shadow_.97ZlBFmz2V3ll9O3l4nMwEiPBkrCtnW_0 2024-06-28T15:21:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.972+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-06-28T15:21:54.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.972+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.972+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.2163557565277149311 2024-06-28T15:21:54.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.976+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=__ 2024-06-28T15:21:54.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.976+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa8100240 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x56149d731cd8 s->prefetch_data=0 2024-06-28T15:21:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.976+0000 7f3fb21ba980 10 manifest: total_size = 2078 2024-06-28T15:21:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.976+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.1783670138229329555 2024-06-28T15:21:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.976+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa8100240 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x56149d731cd8 s->prefetch_data=0 2024-06-28T15:21:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.976+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d731cd8 2024-06-28T15:21:54.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.976+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.8.0 2024-06-28T15:21:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.984+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.984+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=2078 stripe_ofs=2078 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.984+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.1783670138229329555 2024-06-28T15:21:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.984+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.984+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.8__shadow_.sfhDN1Hh3xzn35Xuq_7hRU225YRVCQ5_0 2024-06-28T15:21:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.984+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-06-28T15:21:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.984+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.984+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.15tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.1783670138229329555 2024-06-28T15:21:54.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.984+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=__ _ 2024-06-28T15:21:54.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.988+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa8100240 obj=test-client.0-r-test-small-obj-data.2:___ state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.988+0000 7f3fb21ba980 10 manifest: total_size = 524288 2024-06-28T15:21:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.988+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.17043748717799448144 2024-06-28T15:21:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.988+0000 7f3fb21ba980 20 get_obj_state: octx=0x7f3fa8100240 obj=test-client.0-r-test-small-obj-data.2:___ state=0x56149d72a6e8 s->prefetch_data=0 2024-06-28T15:21:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.988+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d72a6e8 2024-06-28T15:21:54.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.988+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.8.0 2024-06-28T15:21:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.992+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.992+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=524288 stripe_ofs=524288 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.992+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.17043748717799448144 2024-06-28T15:21:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.992+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.992+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.8__shadow_.GXaGwg-JTGEkPfW_ypUGwT7Do7wSvf0_0 2024-06-28T15:21:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.992+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-06-28T15:21:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.992+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.992+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.17043748717799448144 2024-06-28T15:21:54.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.996+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=___ 2024-06-28T15:21:54.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.996+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d736940 obj=test-client.0-r-test-small-obj-data.2:obj state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:54.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.996+0000 7f3fb21ba980 10 manifest: total_size = 524288 2024-06-28T15:21:54.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.996+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.175067232702430969 2024-06-28T15:21:54.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.996+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d736940 obj=test-client.0-r-test-small-obj-data.2:obj state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:54.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.996+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d724c48 2024-06-28T15:21:54.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:54.996+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.8.0 2024-06-28T15:21:54.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.004+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:54.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.004+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=524288 stripe_ofs=524288 part_ofs=0 rule->part_size=0 2024-06-28T15:21:54.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.004+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.175067232702430969 2024-06-28T15:21:54.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.004+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:54.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.004+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.8__shadow_.AZ0NaE_t4-pu0BYAuqLXpV6kCj-rjHL_0 2024-06-28T15:21:54.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.004+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 327 2024-06-28T15:21:54.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.004+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:54.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.004+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.8tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.175067232702430969 2024-06-28T15:21:54.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.008+0000 7f3fb21ba980 20 list_objects_ordered: starting attempt 1 2024-06-28T15:21:54.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.008+0000 7f3fb21ba980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-06-28T15:21:54.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.008+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=obj 2024-06-28T15:21:54.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.008+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-06-28T15:21:54.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.008+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-06-28T15:21:54.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.008+0000 7f3fb21ba980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-06-28T15:21:54.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.008+0000 7f3fb21ba980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-06-28T15:21:54.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.008+0000 7f3fb21ba980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[843e24ce-085a-4043-8f55-38e05370e790.4686.8])"; uploads.size()=0; is_truncated=0 2024-06-28T15:21:54.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.012+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2024-06-28T15:21:54.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.012+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:54.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.012+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=223 2024-06-28T15:21:54.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.012+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x11 2024-06-28T15:21:54.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.012+0000 7f3fb21ba980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2024-06-28T15:21:54.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.012+0000 7f3fb21ba980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2024-06-28T15:21:54.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.012+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.2, ofs0, ns] 2024-06-28T15:21:54.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.016+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660580 cookie 94646540639104 notifier 6036 bl.length()=181 2024-06-28T15:21:55.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.024+0000 7f3fb21ba980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:843e24ce-085a-4043-8f55-38e05370e790.4686.8 from cache 2024-06-28T15:21:55.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.024+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:843e24ce-085a-4043-8f55-38e05370e790.4686.8, ofs0, ns] 2024-06-28T15:21:55.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.024+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660504 cookie 94646540644992 notifier 6036 bl.length()=238 2024-06-28T15:21:55.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.028+0000 7f3fb21ba980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/843e24ce-085a-4043-8f55-38e05370e790.4686.8 : miss 2024-06-28T15:21:55.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.028+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.028+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:55.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.028+0000 7f3fb21ba980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/843e24ce-085a-4043-8f55-38e05370e790.4686.8 info.flags=0x0 2024-06-28T15:21:55.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.028+0000 7f3fb21ba980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/843e24ce-085a-4043-8f55-38e05370e790.4686.8 to cache LRU end 2024-06-28T15:21:55.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:843e24ce-085a-4043-8f55-38e05370e790.4686.9 : miss 2024-06-28T15:21:55.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=437 2024-06-28T15:21:55.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:843e24ce-085a-4043-8f55-38e05370e790.4686.9 info.flags=0x17 2024-06-28T15:21:55.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:843e24ce-085a-4043-8f55-38e05370e790.4686.9 to cache LRU end 2024-06-28T15:21:55.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:21:55.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-06-28T15:21:55.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:843e24ce-085a-4043-8f55-38e05370e790.4686.9 2024-06-28T15:21:55.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got __ 2024-06-28T15:21:55.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got __ _ 2024-06-28T15:21:55.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got ___ 2024-06-28T15:21:55.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 10 cls_bucket_list_unordered: got obj 2024-06-28T15:21:55.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d727630 obj=test-client.0-r-test-small-obj-data.3:__ state=0x56149d730378 s->prefetch_data=0 2024-06-28T15:21:55.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 10 manifest: total_size = 55 2024-06-28T15:21:55.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.18391893484726116323 2024-06-28T15:21:55.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d727630 obj=test-client.0-r-test-small-obj-data.3:__ state=0x56149d730378 s->prefetch_data=0 2024-06-28T15:21:55.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d730378 2024-06-28T15:21:55.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.036+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.9.0 2024-06-28T15:21:55.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.044+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:55.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.044+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=55 stripe_ofs=55 part_ofs=0 rule->part_size=0 2024-06-28T15:21:55.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.044+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.18391893484726116323 2024-06-28T15:21:55.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.044+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:55.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.044+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.9__shadow_.YeRwfleGbdAziA7fF6xOcgncV56Lqrq_0 2024-06-28T15:21:55.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.044+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 329 2024-06-28T15:21:55.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.044+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:55.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.044+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.18391893484726116323 2024-06-28T15:21:55.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.048+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=__ 2024-06-28T15:21:55.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.048+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d727630 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:55.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.048+0000 7f3fb21ba980 10 manifest: total_size = 1035 2024-06-28T15:21:55.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.048+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.36165453831064552 2024-06-28T15:21:55.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.048+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d727630 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:55.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.048+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d724c48 2024-06-28T15:21:55.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.048+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.9.0 2024-06-28T15:21:55.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.052+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:55.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.052+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=1035 stripe_ofs=1035 part_ofs=0 rule->part_size=0 2024-06-28T15:21:55.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.052+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.36165453831064552 2024-06-28T15:21:55.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.052+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:55.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.052+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.9__shadow_.nj_8oRrkOL8CYVoJxq_pwy74Eg8CcvO_0 2024-06-28T15:21:55.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.052+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 326 2024-06-28T15:21:55.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.052+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:55.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.052+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.10tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.36165453831064552 2024-06-28T15:21:55.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.056+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d727630 obj=test-client.0-r-test-small-obj-data.3:___ state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:55.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.056+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=__ _ 2024-06-28T15:21:55.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.056+0000 7f3fb21ba980 10 manifest: total_size = 49187 2024-06-28T15:21:55.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.056+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.5708589557636172717 2024-06-28T15:21:55.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.056+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d727630 obj=test-client.0-r-test-small-obj-data.3:___ state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:55.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.056+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d724c48 2024-06-28T15:21:55.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.056+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.9.0 2024-06-28T15:21:55.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.064+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:55.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.064+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=49187 stripe_ofs=49187 part_ofs=0 rule->part_size=0 2024-06-28T15:21:55.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.064+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.5708589557636172717 2024-06-28T15:21:55.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.064+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:55.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.064+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.9__shadow_.xMwOjFVEFVekjC9hn-Y-lDanvSRb0wf_0 2024-06-28T15:21:55.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.064+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-06-28T15:21:55.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.064+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:55.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.064+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.5708589557636172717 2024-06-28T15:21:55.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.068+0000 7f3e8effd640 20 handle_completion(): completion ok for obj=___ 2024-06-28T15:21:55.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.068+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d727630 obj=test-client.0-r-test-small-obj-data.3:obj state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:55.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.068+0000 7f3fb21ba980 10 manifest: total_size = 1048576 2024-06-28T15:21:55.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.068+0000 7f3fb21ba980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.9911405742747893466 2024-06-28T15:21:55.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.068+0000 7f3fb21ba980 20 get_obj_state: octx=0x56149d727630 obj=test-client.0-r-test-small-obj-data.3:obj state=0x56149d724c48 s->prefetch_data=0 2024-06-28T15:21:55.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.068+0000 7f3fb21ba980 20 prepare_atomic_modification: state is not atomic. state=0x56149d724c48 2024-06-28T15:21:55.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.068+0000 7f3fb21ba980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.9.0 2024-06-28T15:21:55.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.076+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:21:55.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.076+0000 7f3fb21ba980 20 RGWObjManifest::operator++(): result: ofs=1048576 stripe_ofs=1048576 part_ofs=0 rule->part_size=0 2024-06-28T15:21:55.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.076+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.9911405742747893466 2024-06-28T15:21:55.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.076+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-06-28T15:21:55.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.076+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 843e24ce-085a-4043-8f55-38e05370e790.4686.9__shadow_.PCm6GjSJvqSjZyWzLIvOlNXVaJKxgrK_0 2024-06-28T15:21:55.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.076+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 328 2024-06-28T15:21:55.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.076+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-06-28T15:21:55.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.076+0000 7f3fb21ba980 20 garbage collection: RGWGC::send_chain - on object name: gc.21tag is: 843e24ce-085a-4043-8f55-38e05370e790.4686.9911405742747893466 2024-06-28T15:21:55.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.080+0000 7f3e8e7fc640 20 handle_completion(): completion ok for obj=obj 2024-06-28T15:21:55.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.080+0000 7f3fb21ba980 20 list_objects_ordered: starting attempt 1 2024-06-28T15:21:55.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.080+0000 7f3fb21ba980 10 cls_bucket_list_ordered: request from each of 1 shard(s) for 1001 entries to get 1001 total entries 2024-06-28T15:21:55.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.080+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-06-28T15:21:55.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.080+0000 7f3fb21ba980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-06-28T15:21:55.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.080+0000 7f3fb21ba980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-06-28T15:21:55.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.080+0000 7f3fb21ba980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-06-28T15:21:55.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.080+0000 7f3fb21ba980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[843e24ce-085a-4043-8f55-38e05370e790.4686.9])"; uploads.size()=0; is_truncated=0 2024-06-28T15:21:55.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.084+0000 7f3fb21ba980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2024-06-28T15:21:55.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.084+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.084+0000 7f3fb21ba980 20 rados_obj.operate() r=0 bl.length=223 2024-06-28T15:21:55.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.084+0000 7f3fb21ba980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x11 2024-06-28T15:21:55.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.084+0000 7f3fb21ba980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2024-06-28T15:21:55.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.088+0000 7f3fb21ba980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2024-06-28T15:21:55.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.088+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.3, ofs0, ns] 2024-06-28T15:21:55.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.088+0000 7f3e8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660581 cookie 94646540639104 notifier 6036 bl.length()=181 2024-06-28T15:21:55.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.096+0000 7f3fb21ba980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:843e24ce-085a-4043-8f55-38e05370e790.4686.9 from cache 2024-06-28T15:21:55.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.096+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:843e24ce-085a-4043-8f55-38e05370e790.4686.9, ofs0, ns] 2024-06-28T15:21:55.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.096+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660505 cookie 94646540644992 notifier 6036 bl.length()=238 2024-06-28T15:21:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.100+0000 7f3fb21ba980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/843e24ce-085a-4043-8f55-38e05370e790.4686.9 : miss 2024-06-28T15:21:55.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.100+0000 7f3fb21ba980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.104+0000 7f3fb21ba980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.104+0000 7f3fb21ba980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/843e24ce-085a-4043-8f55-38e05370e790.4686.9 info.flags=0x0 2024-06-28T15:21:55.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.104+0000 7f3fb21ba980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/843e24ce-085a-4043-8f55-38e05370e790.4686.9 to cache LRU end 2024-06-28T15:21:55.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.108+0000 7f3fb21ba980 10 removing key index: DCOBIBMLAMCBAZSEXBWO 2024-06-28T15:21:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.108+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.keys:DCOBIBMLAMCBAZSEXBWO, ofs0, ns] 2024-06-28T15:21:55.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.108+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660506 cookie 94646540644992 notifier 6036 bl.length()=170 2024-06-28T15:21:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.112+0000 7f3fb21ba980 10 removing email index: ragweed.client.0+test@test.test 2024-06-28T15:21:55.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.116+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2024-06-28T15:21:55.098 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.116+0000 7f3e8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660507 cookie 94646540640160 notifier 6036 bl.length()=182 2024-06-28T15:21:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.120+0000 7f3fb21ba980 10 removing user buckets index 2024-06-28T15:21:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.120+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0.buckets, ofs0, ns] 2024-06-28T15:21:55.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.124+0000 7f3e8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660502 cookie 94646540649792 notifier 6036 bl.length()=173 2024-06-28T15:21:55.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.124+0000 7f3fb21ba980 10 removing user index: ragweed.client.0 2024-06-28T15:21:55.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.128+0000 7f3fb21ba980 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2024-06-28T15:21:55.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.128+0000 7f3fb21ba980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2024-06-28T15:21:55.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.128+0000 7f3e8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660518 cookie 94646540646048 notifier 6036 bl.length()=165 2024-06-28T15:21:55.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.136+0000 7f3e65fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:21:55.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.140+0000 7f3fb21ba980 20 remove_watcher() i=0 2024-06-28T15:21:55.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.140+0000 7f3fb21ba980 2 removed watcher, disabling cache 2024-06-28T15:21:55.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.144+0000 7f3fb21ba980 20 remove_watcher() i=1 2024-06-28T15:21:55.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.148+0000 7f3fb21ba980 20 remove_watcher() i=2 2024-06-28T15:21:55.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.148+0000 7f3fb21ba980 20 remove_watcher() i=3 2024-06-28T15:21:55.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.152+0000 7f3fb21ba980 20 remove_watcher() i=4 2024-06-28T15:21:55.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.156+0000 7f3fb21ba980 20 remove_watcher() i=5 2024-06-28T15:21:55.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.156+0000 7f3fb21ba980 20 remove_watcher() i=6 2024-06-28T15:21:55.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.156+0000 7f3fb21ba980 20 remove_watcher() i=7 2024-06-28T15:21:55.157 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2024-06-28T15:21:55.236 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:21:55.236 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:21:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.296+0000 7f0ea6368980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.296+0000 7f0ea6368980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.296+0000 7f0ea6368980 20 realm 2024-06-28T15:21:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.296+0000 7f0ea6368980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.296+0000 7f0ea6368980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.296+0000 7f0ea6368980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:21:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.296+0000 7f0ea6368980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.296+0000 7f0ea6368980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.296+0000 7f0ea6368980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.300+0000 7f0ea6368980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:21:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.300+0000 7f0ea6368980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.300+0000 7f0ea6368980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:21:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.300+0000 7f0ea6368980 20 searching for the correct realm 2024-06-28T15:21:55.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.304+0000 7f0ea467a640 0 -- 172.21.15.178:0/4062044489 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f0e98053950 msgr2=0x7f0e98029410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:21:55.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.520+0000 7f0ea3e79640 0 -- 172.21.15.178:0/4062044489 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55c925dad440 msgr2=0x55c925daf870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:21:55.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.528+0000 7f0ea6368980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:21:55.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.528+0000 7f0ea6368980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:21:55.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.528+0000 7f0ea6368980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:21:55.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.528+0000 7f0ea6368980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:21:55.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.528+0000 7f0ea6368980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:21:55.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.528+0000 7f0ea6368980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:21:55.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.528+0000 7f0ea6368980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.528+0000 7f0ea6368980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:55.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.528+0000 7f0ea6368980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.528+0000 7f0ea6368980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:21:55.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.528+0000 7f0ea6368980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:21:55.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 20 zone default found 2024-06-28T15:21:55.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 4 Realm: () 2024-06-28T15:21:55.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:21:55.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:21:55.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:21:55.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 20 zonegroup default 2024-06-28T15:21:55.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:55.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:55.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:21:55.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 20 started sync module instance, tier type = 2024-06-28T15:21:55.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.532+0000 7f0ea6368980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:21:55.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.556+0000 7f0ea6368980 20 add_watcher() i=0 2024-06-28T15:21:55.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.556+0000 7f0ea6368980 20 add_watcher() i=1 2024-06-28T15:21:55.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.556+0000 7f0ea6368980 20 add_watcher() i=2 2024-06-28T15:21:55.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.556+0000 7f0ea6368980 20 add_watcher() i=3 2024-06-28T15:21:55.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.556+0000 7f0ea6368980 20 add_watcher() i=4 2024-06-28T15:21:55.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.556+0000 7f0ea6368980 20 add_watcher() i=5 2024-06-28T15:21:55.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.556+0000 7f0ea6368980 20 add_watcher() i=6 2024-06-28T15:21:55.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.560+0000 7f0ea6368980 20 add_watcher() i=7 2024-06-28T15:21:55.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.560+0000 7f0ea6368980 2 all 8 watchers are set, enabling cache 2024-06-28T15:21:55.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.564+0000 7f0d59fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:21:55.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.564+0000 7f0ea6368980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:21:55.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.564+0000 7f0ea6368980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:21:55.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.652+0000 7f0d58fe1640 20 reqs_thread_entry: start 2024-06-28T15:21:55.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.652+0000 7f0ea6368980 20 init_complete bucket index max shards: 1 2024-06-28T15:21:55.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.652+0000 7f0ea6368980 20 Filter name: none 2024-06-28T15:21:55.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.652+0000 7f0cd27fc640 20 reqs_thread_entry: start 2024-06-28T15:21:55.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.652+0000 7f0ea6368980 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2024-06-28T15:21:55.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.652+0000 7f0ea6368980 20 rados->read ofs=0 len=0 2024-06-28T15:21:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.652+0000 7f0ea6368980 20 rados_obj.operate() r=0 bl.length=486 2024-06-28T15:21:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.652+0000 7f0ea6368980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x13 2024-06-28T15:21:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.652+0000 7f0ea6368980 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-06-28T15:21:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.652+0000 7f0ea6368980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:21:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.652+0000 7f0ea6368980 10 removing key index: YECNJEQCOQGRZLQMVUYL 2024-06-28T15:21:55.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.656+0000 7f0ea6368980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.keys:YECNJEQCOQGRZLQMVUYL, ofs0, ns] 2024-06-28T15:21:55.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.656+0000 7f0d82ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660519 cookie 94322411931904 notifier 5729 bl.length()=170 2024-06-28T15:21:55.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.660+0000 7f0ea6368980 10 removing email index: sysuser.client.0+test@test.test 2024-06-28T15:21:55.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.664+0000 7f0ea6368980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2024-06-28T15:21:55.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.664+0000 7f0d82ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660520 cookie 94322411931904 notifier 5729 bl.length()=182 2024-06-28T15:21:55.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.664+0000 7f0ea6368980 10 removing user buckets index 2024-06-28T15:21:55.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.668+0000 7f0ea6368980 10 removing user index: sysuser.client.0 2024-06-28T15:21:55.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.668+0000 7f0ea6368980 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2024-06-28T15:21:55.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.668+0000 7f0ea6368980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2024-06-28T15:21:55.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.672+0000 7f0d827fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1095216660508 cookie 94322411923488 notifier 5729 bl.length()=165 2024-06-28T15:21:55.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.676+0000 7f0d59fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:21:55.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.680+0000 7f0ea6368980 20 remove_watcher() i=0 2024-06-28T15:21:55.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.680+0000 7f0ea6368980 2 removed watcher, disabling cache 2024-06-28T15:21:55.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.680+0000 7f0ea6368980 20 remove_watcher() i=1 2024-06-28T15:21:55.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.684+0000 7f0ea6368980 20 remove_watcher() i=2 2024-06-28T15:21:55.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.684+0000 7f0ea6368980 20 remove_watcher() i=3 2024-06-28T15:21:55.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.688+0000 7f0ea6368980 20 remove_watcher() i=4 2024-06-28T15:21:55.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.688+0000 7f0ea6368980 20 remove_watcher() i=5 2024-06-28T15:21:55.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.692+0000 7f0ea6368980 20 remove_watcher() i=6 2024-06-28T15:21:55.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:21:55.692+0000 7f0ea6368980 20 remove_watcher() i=7 2024-06-28T15:21:55.690 INFO:tasks.ragweed:Removing ragweed... 2024-06-28T15:21:55.690 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/ragweed.client.0 2024-06-28T15:21:55.779 INFO:teuthology.run_tasks:Running task workunit... 2024-06-28T15:21:55.790 INFO:tasks.workunit:Pulling workunits from ref ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T15:21:55.790 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-28T15:21:55.790 DEBUG:teuthology.orchestra.run.smithi178:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:21:55.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T15:21:55.796 INFO:teuthology.orchestra.run.smithi178.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-06-28T15:21:55.796 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:21:55.844 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-06-28T15:21:55.844 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-06-28T15:21:55.893 INFO:tasks.workunit:timeout=3h 2024-06-28T15:21:55.893 INFO:tasks.workunit:cleanup=True 2024-06-28T15:21:55.894 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ec2a38ba1b69b5357de8f0e8f96b00b4a577169d 2024-06-28T15:21:55.942 INFO:tasks.workunit.client.0.smithi178.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-28T15:22:50.632 INFO:tasks.workunit.client.0.smithi178.stderr:Note: switching to 'ec2a38ba1b69b5357de8f0e8f96b00b4a577169d'. 2024-06-28T15:22:50.633 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:22:50.633 INFO:tasks.workunit.client.0.smithi178.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-28T15:22:50.633 INFO:tasks.workunit.client.0.smithi178.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-28T15:22:50.633 INFO:tasks.workunit.client.0.smithi178.stderr:state without impacting any branches by switching back to a branch. 2024-06-28T15:22:50.633 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:22:50.633 INFO:tasks.workunit.client.0.smithi178.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-28T15:22:50.633 INFO:tasks.workunit.client.0.smithi178.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-28T15:22:50.633 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:22:50.633 INFO:tasks.workunit.client.0.smithi178.stderr: git switch -c 2024-06-28T15:22:50.634 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:22:50.634 INFO:tasks.workunit.client.0.smithi178.stderr:Or undo this operation with: 2024-06-28T15:22:50.634 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:22:50.634 INFO:tasks.workunit.client.0.smithi178.stderr: git switch - 2024-06-28T15:22:50.634 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:22:50.634 INFO:tasks.workunit.client.0.smithi178.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-28T15:22:50.634 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:22:50.634 INFO:tasks.workunit.client.0.smithi178.stderr:HEAD is now at ec2a38ba1b6 Merge branches 'pr-56792', 'pr-57537', 'pr-57858', 'pr-57985', 'pr-58081' and 'pr-58288' into wip-cbodley-testing 2024-06-28T15:22:50.640 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-28T15:22:50.689 INFO:tasks.workunit.client.0.smithi178.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-28T15:22:50.692 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-28T15:22:50.692 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-28T15:22:50.751 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-28T15:22:50.807 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-28T15:22:50.854 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-28T15:22:50.855 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-28T15:22:50.856 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-28T15:22:50.898 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-28T15:22:50.901 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T15:22:50.901 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-28T15:22:50.949 INFO:tasks.workunit:Running workunits matching rgw/run-reshard.sh on client.0... 2024-06-28T15:22:50.950 INFO:tasks.workunit:Running workunit rgw/run-reshard.sh... 2024-06-28T15:22:50.950 DEBUG:teuthology.orchestra.run.smithi178:workunit test rgw/run-reshard.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ec2a38ba1b69b5357de8f0e8f96b00b4a577169d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2024-06-28T15:22:51.004 INFO:tasks.workunit.client.0.smithi178.stderr:+ ulimit -c 0 2024-06-28T15:22:51.005 INFO:tasks.workunit.client.0.smithi178.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2024-06-28T15:22:51.005 INFO:tasks.workunit.client.0.smithi178.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-06-28T15:22:51.005 INFO:tasks.workunit.client.0.smithi178.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-06-28T15:22:53.894 INFO:tasks.workunit.client.0.smithi178.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-06-28T15:22:53.895 INFO:tasks.workunit.client.0.smithi178.stderr:++ deactivate nondestructive 2024-06-28T15:22:53.895 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n '' ']' 2024-06-28T15:22:53.895 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n '' ']' 2024-06-28T15:22:53.895 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-06-28T15:22:53.895 INFO:tasks.workunit.client.0.smithi178.stderr:++ hash -r 2024-06-28T15:22:53.895 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n '' ']' 2024-06-28T15:22:53.895 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset VIRTUAL_ENV 2024-06-28T15:22:53.895 INFO:tasks.workunit.client.0.smithi178.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-06-28T15:22:53.895 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-06-28T15:22:53.895 INFO:tasks.workunit.client.0.smithi178.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-06-28T15:22:53.895 INFO:tasks.workunit.client.0.smithi178.stderr:++ export VIRTUAL_ENV 2024-06-28T15:22:53.895 INFO:tasks.workunit.client.0.smithi178.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-06-28T15:22:53.896 INFO:tasks.workunit.client.0.smithi178.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-06-28T15:22:53.896 INFO:tasks.workunit.client.0.smithi178.stderr:++ export PATH 2024-06-28T15:22:53.896 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n '' ']' 2024-06-28T15:22:53.896 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -z '' ']' 2024-06-28T15:22:53.896 INFO:tasks.workunit.client.0.smithi178.stderr:++ _OLD_VIRTUAL_PS1= 2024-06-28T15:22:53.896 INFO:tasks.workunit.client.0.smithi178.stderr:++ PS1='(rgw) ' 2024-06-28T15:22:53.896 INFO:tasks.workunit.client.0.smithi178.stderr:++ export PS1 2024-06-28T15:22:53.896 INFO:tasks.workunit.client.0.smithi178.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-06-28T15:22:53.896 INFO:tasks.workunit.client.0.smithi178.stderr:++ export VIRTUAL_ENV_PROMPT 2024-06-28T15:22:53.896 INFO:tasks.workunit.client.0.smithi178.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-06-28T15:22:53.896 INFO:tasks.workunit.client.0.smithi178.stderr:++ hash -r 2024-06-28T15:22:53.897 INFO:tasks.workunit.client.0.smithi178.stderr:+ pip install pip --upgrade 2024-06-28T15:22:54.291 INFO:tasks.workunit.client.0.smithi178.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-06-28T15:22:54.633 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting pip 2024-06-28T15:22:54.639 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached pip-24.1.1-py3-none-any.whl (1.8 MB) 2024-06-28T15:22:54.675 INFO:tasks.workunit.client.0.smithi178.stdout:Installing collected packages: pip 2024-06-28T15:22:54.675 INFO:tasks.workunit.client.0.smithi178.stdout: Attempting uninstall: pip 2024-06-28T15:22:54.675 INFO:tasks.workunit.client.0.smithi178.stdout: Found existing installation: pip 22.0.2 2024-06-28T15:22:54.933 INFO:tasks.workunit.client.0.smithi178.stdout: Uninstalling pip-22.0.2: 2024-06-28T15:22:54.939 INFO:tasks.workunit.client.0.smithi178.stdout: Successfully uninstalled pip-22.0.2 2024-06-28T15:22:55.700 INFO:tasks.workunit.client.0.smithi178.stdout:Successfully installed pip-24.1.1 2024-06-28T15:22:55.763 INFO:tasks.workunit.client.0.smithi178.stderr:+ pip install boto3 2024-06-28T15:22:57.125 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting boto3 2024-06-28T15:22:57.126 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached boto3-1.34.135-py3-none-any.whl.metadata (6.6 kB) 2024-06-28T15:22:57.687 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting botocore<1.35.0,>=1.34.135 (from boto3) 2024-06-28T15:22:57.689 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached botocore-1.34.135-py3-none-any.whl.metadata (5.7 kB) 2024-06-28T15:22:57.779 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-06-28T15:22:57.780 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-06-28T15:22:57.868 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-06-28T15:22:57.870 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached s3transfer-0.10.2-py3-none-any.whl.metadata (1.7 kB) 2024-06-28T15:22:57.971 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.135->boto3) 2024-06-28T15:22:57.974 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-06-28T15:22:58.133 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.35.0,>=1.34.135->boto3) 2024-06-28T15:22:58.135 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB) 2024-06-28T15:22:58.368 INFO:tasks.workunit.client.0.smithi178.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.135->boto3) 2024-06-28T15:22:58.370 INFO:tasks.workunit.client.0.smithi178.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-06-28T15:22:58.374 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached boto3-1.34.135-py3-none-any.whl (139 kB) 2024-06-28T15:22:58.376 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached botocore-1.34.135-py3-none-any.whl (12.3 MB) 2024-06-28T15:22:58.402 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-06-28T15:22:58.403 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached s3transfer-0.10.2-py3-none-any.whl (82 kB) 2024-06-28T15:22:58.405 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-06-28T15:22:58.407 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached urllib3-2.2.2-py3-none-any.whl (121 kB) 2024-06-28T15:22:58.408 INFO:tasks.workunit.client.0.smithi178.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-06-28T15:22:58.480 INFO:tasks.workunit.client.0.smithi178.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-06-28T15:22:59.633 INFO:tasks.workunit.client.0.smithi178.stdout:Successfully installed boto3-1.34.135 botocore-1.34.135 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.2 six-1.16.0 urllib3-2.2.2 2024-06-28T15:22:59.768 INFO:tasks.workunit.client.0.smithi178.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_reshard.py 2024-06-28T15:22:59.909 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin user create --uid tester --display-name "Testing" --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn 2024-06-28T15:22:59.946 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:22:59.947 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:22:59.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.006+0000 7fdeb979e980 20 rados->read ofs=0 len=0 2024-06-28T15:22:59.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.006+0000 7fdeb979e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:22:59.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.006+0000 7fdeb979e980 20 realm 2024-06-28T15:22:59.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.006+0000 7fdeb979e980 20 rados->read ofs=0 len=0 2024-06-28T15:22:59.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.006+0000 7fdeb979e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:22:59.991 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.006+0000 7fdeb979e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:22:59.991 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.006+0000 7fdeb979e980 20 rados->read ofs=0 len=0 2024-06-28T15:22:59.991 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.006+0000 7fdeb979e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:22:59.991 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.006+0000 7fdeb979e980 20 rados->read ofs=0 len=0 2024-06-28T15:22:59.992 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.010+0000 7fdeb979e980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:22:59.992 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.010+0000 7fdeb979e980 20 rados->read ofs=0 len=0 2024-06-28T15:22:59.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.010+0000 7fdeb979e980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:22:59.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.010+0000 7fdeb979e980 20 searching for the correct realm 2024-06-28T15:23:00.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.018+0000 7fdeb7ab0640 0 -- 172.21.15.178:0/4187026362 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fdea00f3960 msgr2=0x7fdea00e2410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:00.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.234+0000 7fdeb72af640 0 -- 172.21.15.178:0/4187026362 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x5638f43148f0 msgr2=0x5638f4316d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:00.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.242+0000 7fdeb979e980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:00.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.242+0000 7fdeb979e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:00.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.242+0000 7fdeb979e980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:00.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.242+0000 7fdeb979e980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:00.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.242+0000 7fdeb979e980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:00.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.242+0000 7fdeb979e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:00.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.242+0000 7fdeb979e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:00.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.242+0000 7fdeb979e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:00.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.242+0000 7fdeb979e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:00.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.242+0000 7fdeb979e980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:00.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.242+0000 7fdeb979e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:00.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:00.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 20 zone default found 2024-06-28T15:23:00.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 4 Realm: () 2024-06-28T15:23:00.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:00.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:00.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:00.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 20 zonegroup default 2024-06-28T15:23:00.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:00.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:00.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:00.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:00.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:00.230 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:00.230 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 20 started sync module instance, tier type = 2024-06-28T15:23:00.230 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.246+0000 7fdeb979e980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:00.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.270+0000 7fdeb979e980 20 add_watcher() i=0 2024-06-28T15:23:00.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.270+0000 7fdeb979e980 20 add_watcher() i=1 2024-06-28T15:23:00.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.270+0000 7fdeb979e980 20 add_watcher() i=2 2024-06-28T15:23:00.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.270+0000 7fdeb979e980 20 add_watcher() i=3 2024-06-28T15:23:00.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.270+0000 7fdeb979e980 20 add_watcher() i=4 2024-06-28T15:23:00.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.270+0000 7fdeb979e980 20 add_watcher() i=5 2024-06-28T15:23:00.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.270+0000 7fdeb979e980 20 add_watcher() i=6 2024-06-28T15:23:00.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.274+0000 7fdeb979e980 20 add_watcher() i=7 2024-06-28T15:23:00.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.274+0000 7fdeb979e980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:00.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.278+0000 7fdd6dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:00.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.278+0000 7fdeb979e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:00.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.278+0000 7fdeb979e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:00.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.278+0000 7fdeb979e980 5 note: GC not initialized 2024-06-28T15:23:00.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.278+0000 7fdd6cfe1640 20 reqs_thread_entry: start 2024-06-28T15:23:00.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.278+0000 7fdeb979e980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:00.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.278+0000 7fdeb979e980 20 Filter name: none 2024-06-28T15:23:00.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.278+0000 7fdce67fc640 20 reqs_thread_entry: start 2024-06-28T15:23:00.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 10 cache get: name=default.rgw.meta+users.uid+tester : miss 2024-06-28T15:23:00.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:00.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:00.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 10 cache put: name=default.rgw.meta+users.uid+tester info.flags=0x0 2024-06-28T15:23:00.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 10 adding default.rgw.meta+users.uid+tester to cache LRU end 2024-06-28T15:23:00.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-06-28T15:23:00.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:00.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 20 rados_obj.operate() r=0 bl.length=22 2024-06-28T15:23:00.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x5 2024-06-28T15:23:00.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-06-28T15:23:00.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 10 cache get: name=default.rgw.meta+users.uid+rgw_datacache_user : miss 2024-06-28T15:23:00.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:00.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 20 rados_obj.operate() r=0 bl.length=420 2024-06-28T15:23:00.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 10 cache put: name=default.rgw.meta+users.uid+rgw_datacache_user info.flags=0x13 2024-06-28T15:23:00.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 10 adding default.rgw.meta+users.uid+rgw_datacache_user to cache LRU end 2024-06-28T15:23:00.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:00.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+rgw_datacache_user 2024-06-28T15:23:00.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdeb979e980 20 get_user_info_from_index found 0 xattrs 2024-06-28T15:23:00.269 INFO:tasks.workunit.client.0.smithi178.stderr:could not create user: unable to parse parameters, duplicate key provided 2024-06-28T15:23:00.270 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.282+0000 7fdd6dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:00.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.290+0000 7fdeb979e980 20 remove_watcher() i=0 2024-06-28T15:23:00.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.290+0000 7fdeb979e980 2 removed watcher, disabling cache 2024-06-28T15:23:00.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.290+0000 7fdeb979e980 20 remove_watcher() i=1 2024-06-28T15:23:00.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.294+0000 7fdeb979e980 20 remove_watcher() i=2 2024-06-28T15:23:00.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.294+0000 7fdeb979e980 20 remove_watcher() i=3 2024-06-28T15:23:00.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.298+0000 7fdeb979e980 20 remove_watcher() i=4 2024-06-28T15:23:00.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.298+0000 7fdeb979e980 20 remove_watcher() i=5 2024-06-28T15:23:00.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.302+0000 7fdeb979e980 20 remove_watcher() i=6 2024-06-28T15:23:00.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:00.302+0000 7fdeb979e980 20 remove_watcher() i=7 2024-06-28T15:23:00.948 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: reshard add 2024-06-28T15:23:00.949 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:00.949 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-06-28T15:23:00.982 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:00.982 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:01.029 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.046+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.046+0000 7f17acc5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.046+0000 7f17acc5b980 20 realm 2024-06-28T15:23:01.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.046+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.046+0000 7f17acc5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.046+0000 7f17acc5b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:01.031 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.046+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.046+0000 7f17acc5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.046+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.033 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.050+0000 7f17acc5b980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:01.033 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.050+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.034 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.050+0000 7f17acc5b980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:01.034 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.050+0000 7f17acc5b980 20 searching for the correct realm 2024-06-28T15:23:01.040 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.058+0000 7f17aaf6d640 0 -- 172.21.15.178:0/2940403408 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x558b73df24f0 msgr2=0x558b73df4940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:01.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.062+0000 7f17a9f6b640 0 -- 172.21.15.178:0/2940403408 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f179c01a040 msgr2=0x7f179c04b930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:01.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.070+0000 7f17acc5b980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:01.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.070+0000 7f17acc5b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:01.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.070+0000 7f17acc5b980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:01.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.070+0000 7f17acc5b980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:01.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.070+0000 7f17acc5b980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:01.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.070+0000 7f17acc5b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:01.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.070+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.070+0000 7f17acc5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.070+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.070+0000 7f17acc5b980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:01.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.070+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:01.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 20 zone default found 2024-06-28T15:23:01.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 4 Realm: () 2024-06-28T15:23:01.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:01.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:01.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:01.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 20 zonegroup default 2024-06-28T15:23:01.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 20 started sync module instance, tier type = 2024-06-28T15:23:01.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.074+0000 7f17acc5b980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:01.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.090+0000 7f17acc5b980 20 add_watcher() i=0 2024-06-28T15:23:01.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.090+0000 7f17acc5b980 20 add_watcher() i=1 2024-06-28T15:23:01.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.090+0000 7f17acc5b980 20 add_watcher() i=2 2024-06-28T15:23:01.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.090+0000 7f17acc5b980 20 add_watcher() i=3 2024-06-28T15:23:01.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.090+0000 7f17acc5b980 20 add_watcher() i=4 2024-06-28T15:23:01.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.090+0000 7f17acc5b980 20 add_watcher() i=5 2024-06-28T15:23:01.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.090+0000 7f17acc5b980 20 add_watcher() i=6 2024-06-28T15:23:01.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.090+0000 7f17acc5b980 20 add_watcher() i=7 2024-06-28T15:23:01.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.090+0000 7f17acc5b980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:01.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f1660fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:01.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f17acc5b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:01.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f17acc5b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:01.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f17acc5b980 5 note: GC not initialized 2024-06-28T15:23:01.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f15db7fe640 20 reqs_thread_entry: start 2024-06-28T15:23:01.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f17acc5b980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:01.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f17acc5b980 20 Filter name: none 2024-06-28T15:23:01.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f15d97fa640 20 reqs_thread_entry: start 2024-06-28T15:23:01.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.078 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f17acc5b980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:23:01.078 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.078 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f17acc5b980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:23:01.079 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f17acc5b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:01.079 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.079 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f17acc5b980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:23:01.079 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.094+0000 7f17acc5b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.080 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.098+0000 7f17acc5b980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:23:01.080 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.098+0000 7f17acc5b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:01.081 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.098+0000 7f1660fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:01.086 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.102+0000 7f17acc5b980 20 remove_watcher() i=0 2024-06-28T15:23:01.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.102+0000 7f17acc5b980 2 removed watcher, disabling cache 2024-06-28T15:23:01.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.106+0000 7f17acc5b980 20 remove_watcher() i=1 2024-06-28T15:23:01.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.106+0000 7f17acc5b980 20 remove_watcher() i=2 2024-06-28T15:23:01.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.110+0000 7f17acc5b980 20 remove_watcher() i=3 2024-06-28T15:23:01.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.110+0000 7f17acc5b980 20 remove_watcher() i=4 2024-06-28T15:23:01.096 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.114+0000 7f17acc5b980 20 remove_watcher() i=5 2024-06-28T15:23:01.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.114+0000 7f17acc5b980 20 remove_watcher() i=6 2024-06-28T15:23:01.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.114+0000 7f17acc5b980 20 remove_watcher() i=7 2024-06-28T15:23:01.112 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:01.112 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:01.112 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: a-bucket id: 843e24ce-085a-4043-8f55-38e05370e790.4686.18 num_objects: 0 size_kb: 0 num_shards: 1 2024-06-28T15:23:01.112 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:01.113 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 2 2024-06-28T15:23:01.141 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:01.141 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:01.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.202+0000 7f233fcde980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.185 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.202+0000 7f233fcde980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.186 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.202+0000 7f233fcde980 20 realm 2024-06-28T15:23:01.186 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.202+0000 7f233fcde980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.186 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.202+0000 7f233fcde980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.186 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.202+0000 7f233fcde980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:01.186 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.202+0000 7f233fcde980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.186 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.202+0000 7f233fcde980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.186 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.202+0000 7f233fcde980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.187 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.206+0000 7f233fcde980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:01.188 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.206+0000 7f233fcde980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.188 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.206+0000 7f233fcde980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:01.188 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.206+0000 7f233fcde980 20 searching for the correct realm 2024-06-28T15:23:01.195 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.210+0000 7f233dff0640 0 -- 172.21.15.178:0/3036249862 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f23340030a0 msgr2=0x7f2334135a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:01.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.426+0000 7f233d7ef640 0 -- 172.21.15.178:0/3036249862 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x559471e92bc0 msgr2=0x559471e94fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:01.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.434+0000 7f233fcde980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:01.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.434+0000 7f233fcde980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:01.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.434+0000 7f233fcde980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:01.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.434+0000 7f233fcde980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:01.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.434+0000 7f233fcde980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:01.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.434+0000 7f233fcde980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:01.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.434+0000 7f233fcde980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.434+0000 7f233fcde980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.434+0000 7f233fcde980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:01.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:01.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 20 zone default found 2024-06-28T15:23:01.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 4 Realm: () 2024-06-28T15:23:01.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:01.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:01.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:01.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 20 zonegroup default 2024-06-28T15:23:01.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.421 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 20 started sync module instance, tier type = 2024-06-28T15:23:01.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.438+0000 7f233fcde980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:01.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.462+0000 7f233fcde980 20 add_watcher() i=0 2024-06-28T15:23:01.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.462+0000 7f233fcde980 20 add_watcher() i=1 2024-06-28T15:23:01.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.462+0000 7f233fcde980 20 add_watcher() i=2 2024-06-28T15:23:01.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.462+0000 7f233fcde980 20 add_watcher() i=3 2024-06-28T15:23:01.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.462+0000 7f233fcde980 20 add_watcher() i=4 2024-06-28T15:23:01.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.462+0000 7f233fcde980 20 add_watcher() i=5 2024-06-28T15:23:01.447 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.462+0000 7f233fcde980 20 add_watcher() i=6 2024-06-28T15:23:01.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.466+0000 7f233fcde980 20 add_watcher() i=7 2024-06-28T15:23:01.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.466+0000 7f233fcde980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:01.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.470+0000 7f21f37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:01.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.470+0000 7f233fcde980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:01.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.470+0000 7f233fcde980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:01.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.470+0000 7f233fcde980 5 note: GC not initialized 2024-06-28T15:23:01.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.470+0000 7f21f27e4640 20 reqs_thread_entry: start 2024-06-28T15:23:01.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.470+0000 7f233fcde980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:01.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.470+0000 7f233fcde980 20 Filter name: none 2024-06-28T15:23:01.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.470+0000 7f2163fff640 20 reqs_thread_entry: start 2024-06-28T15:23:01.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.470+0000 7f233fcde980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-06-28T15:23:01.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.470+0000 7f233fcde980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.456 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.474+0000 7f233fcde980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:23:01.456 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.474+0000 7f233fcde980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-06-28T15:23:01.456 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.474+0000 7f233fcde980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-06-28T15:23:01.456 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.474+0000 7f233fcde980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 : miss 2024-06-28T15:23:01.456 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.474+0000 7f233fcde980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.474+0000 7f233fcde980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:23:01.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.474+0000 7f233fcde980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:23:01.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.474+0000 7f233fcde980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:23:01.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.474+0000 7f233fcde980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:01.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.474+0000 7f233fcde980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:23:01.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.474+0000 7f233fcde980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 2024-06-28T15:23:01.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.478+0000 7f21f37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:01.465 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.482+0000 7f233fcde980 20 remove_watcher() i=0 2024-06-28T15:23:01.465 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.482+0000 7f233fcde980 2 removed watcher, disabling cache 2024-06-28T15:23:01.467 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.482+0000 7f233fcde980 20 remove_watcher() i=1 2024-06-28T15:23:01.470 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.486+0000 7f233fcde980 20 remove_watcher() i=2 2024-06-28T15:23:01.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.490+0000 7f233fcde980 20 remove_watcher() i=3 2024-06-28T15:23:01.474 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.490+0000 7f233fcde980 20 remove_watcher() i=4 2024-06-28T15:23:01.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.490+0000 7f233fcde980 20 remove_watcher() i=5 2024-06-28T15:23:01.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.494+0000 7f233fcde980 20 remove_watcher() i=6 2024-06-28T15:23:01.478 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.494+0000 7f233fcde980 20 remove_watcher() i=7 2024-06-28T15:23:01.500 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin reshard list 2024-06-28T15:23:01.526 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:01.526 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:01.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.586+0000 7f82e6646980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.590+0000 7f82e6646980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.590+0000 7f82e6646980 20 realm 2024-06-28T15:23:01.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.590+0000 7f82e6646980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.590+0000 7f82e6646980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.590+0000 7f82e6646980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:01.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.590+0000 7f82e6646980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.590+0000 7f82e6646980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.590+0000 7f82e6646980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.575 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.594+0000 7f82e6646980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:01.575 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.594+0000 7f82e6646980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.576 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.594+0000 7f82e6646980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:01.576 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.594+0000 7f82e6646980 20 searching for the correct realm 2024-06-28T15:23:01.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.598+0000 7f82dbfff640 0 -- 172.21.15.178:0/2330849816 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f82d4001910 msgr2=0x7f82d4123190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:01.588 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.606+0000 7f82dbfff640 0 -- 172.21.15.178:0/2330849816 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f81ac07fd50 msgr2=0x7f81ac082180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:01.595 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.610+0000 7f82e6646980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:01.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.610+0000 7f82e6646980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:01.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.610+0000 7f82e6646980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:01.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.610+0000 7f82e6646980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:01.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.610+0000 7f82e6646980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:01.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.610+0000 7f82e6646980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:01.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.610+0000 7f82e6646980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:01.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:01.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 zone default found 2024-06-28T15:23:01.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 4 Realm: () 2024-06-28T15:23:01.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:01.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:01.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:01.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 zonegroup default 2024-06-28T15:23:01.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 started sync module instance, tier type = 2024-06-28T15:23:01.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.614+0000 7f82e6646980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:01.613 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.630+0000 7f82e6646980 20 add_watcher() i=0 2024-06-28T15:23:01.613 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.630+0000 7f82e6646980 20 add_watcher() i=1 2024-06-28T15:23:01.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.630+0000 7f82e6646980 20 add_watcher() i=2 2024-06-28T15:23:01.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.630+0000 7f82e6646980 20 add_watcher() i=3 2024-06-28T15:23:01.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.630+0000 7f82e6646980 20 add_watcher() i=4 2024-06-28T15:23:01.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.630+0000 7f82e6646980 20 add_watcher() i=5 2024-06-28T15:23:01.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.630+0000 7f82e6646980 20 add_watcher() i=6 2024-06-28T15:23:01.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.630+0000 7f82e6646980 20 add_watcher() i=7 2024-06-28T15:23:01.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.630+0000 7f82e6646980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:01.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.634+0000 7f819afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:01.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.634+0000 7f82e6646980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:01.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.634+0000 7f82e6646980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:01.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.634+0000 7f82e6646980 5 note: GC not initialized 2024-06-28T15:23:01.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.634+0000 7f8199fdb640 20 reqs_thread_entry: start 2024-06-28T15:23:01.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.634+0000 7f82e6646980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:01.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.634+0000 7f82e6646980 20 Filter name: none 2024-06-28T15:23:01.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.634+0000 7f810b7fe640 20 reqs_thread_entry: start 2024-06-28T15:23:01.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.646+0000 7f819afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:01.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.650+0000 7f82e6646980 20 remove_watcher() i=0 2024-06-28T15:23:01.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.650+0000 7f82e6646980 2 removed watcher, disabling cache 2024-06-28T15:23:01.635 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.654+0000 7f82e6646980 20 remove_watcher() i=1 2024-06-28T15:23:01.637 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.654+0000 7f82e6646980 20 remove_watcher() i=2 2024-06-28T15:23:01.639 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.654+0000 7f82e6646980 20 remove_watcher() i=3 2024-06-28T15:23:01.641 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.658+0000 7f82e6646980 20 remove_watcher() i=4 2024-06-28T15:23:01.643 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.658+0000 7f82e6646980 20 remove_watcher() i=5 2024-06-28T15:23:01.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.662+0000 7f82e6646980 20 remove_watcher() i=6 2024-06-28T15:23:01.645 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.662+0000 7f82e6646980 20 remove_watcher() i=7 2024-06-28T15:23:01.660 INFO:tasks.workunit.client.0.smithi178.stderr:bucket name a-bucket 2024-06-28T15:23:01.660 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: reshard process 2024-06-28T15:23:01.660 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:01.661 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin reshard process 2024-06-28T15:23:01.685 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:01.685 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:01.729 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.746+0000 7f5b4b7c2980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.746+0000 7f5b4b7c2980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.746+0000 7f5b4b7c2980 20 realm 2024-06-28T15:23:01.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.746+0000 7f5b4b7c2980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.746+0000 7f5b4b7c2980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.746+0000 7f5b4b7c2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:01.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.746+0000 7f5b4b7c2980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.731 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.746+0000 7f5b4b7c2980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.731 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.746+0000 7f5b4b7c2980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.732 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.750+0000 7f5b4b7c2980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:01.732 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.750+0000 7f5b4b7c2980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.733 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.750+0000 7f5b4b7c2980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:01.733 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.750+0000 7f5b4b7c2980 20 searching for the correct realm 2024-06-28T15:23:01.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.754+0000 7f5b48ad2640 0 -- 172.21.15.178:0/3988339768 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f5b3c11d4f0 msgr2=0x7f5b3c113400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:01.745 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.762+0000 7f5b49ad4640 0 -- 172.21.15.178:0/3988339768 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f5b40050450 msgr2=0x7f5b400508b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:23:01.753 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:01.753 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:01.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:01.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:01.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:01.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:01.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.755 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.755 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:01.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:01.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 zone default found 2024-06-28T15:23:01.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 4 Realm: () 2024-06-28T15:23:01.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:01.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:01.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:01.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 zonegroup default 2024-06-28T15:23:01.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.770+0000 7f5b4b7c2980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.774+0000 7f5b4b7c2980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:01.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.774+0000 7f5b4b7c2980 20 started sync module instance, tier type = 2024-06-28T15:23:01.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.774+0000 7f5b4b7c2980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:01.771 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.786+0000 7f5b4b7c2980 20 add_watcher() i=0 2024-06-28T15:23:01.771 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.786+0000 7f5b4b7c2980 20 add_watcher() i=1 2024-06-28T15:23:01.772 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.786+0000 7f5b4b7c2980 20 add_watcher() i=2 2024-06-28T15:23:01.772 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.786+0000 7f5b4b7c2980 20 add_watcher() i=3 2024-06-28T15:23:01.772 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.786+0000 7f5b4b7c2980 20 add_watcher() i=4 2024-06-28T15:23:01.772 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.786+0000 7f5b4b7c2980 20 add_watcher() i=5 2024-06-28T15:23:01.772 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.786+0000 7f5b4b7c2980 20 add_watcher() i=6 2024-06-28T15:23:01.773 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.790+0000 7f5b4b7c2980 20 add_watcher() i=7 2024-06-28T15:23:01.773 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.790+0000 7f5b4b7c2980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:01.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.790+0000 7f59ff7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:01.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.790+0000 7f5b4b7c2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:01.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.790+0000 7f5b4b7c2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:01.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.790+0000 7f5b4b7c2980 5 note: GC not initialized 2024-06-28T15:23:01.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.794+0000 7f59fe7e4640 20 reqs_thread_entry: start 2024-06-28T15:23:01.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.794+0000 7f5b4b7c2980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:01.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.794+0000 7f5b4b7c2980 20 Filter name: none 2024-06-28T15:23:01.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.794+0000 7f596ffff640 20 reqs_thread_entry: start 2024-06-28T15:23:01.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.794+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000000 2024-06-28T15:23:01.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.798+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000000 , ret = 0 2024-06-28T15:23:01.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.798+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000001 2024-06-28T15:23:01.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.802+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000001 , ret = 0 2024-06-28T15:23:01.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.802+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000002 2024-06-28T15:23:01.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.806+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000002 , ret = 0 2024-06-28T15:23:01.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.806+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000003 2024-06-28T15:23:01.794 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.810+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000003 , ret = 0 2024-06-28T15:23:01.794 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.810+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000004 2024-06-28T15:23:01.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.818+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000004 , ret = 0 2024-06-28T15:23:01.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.818+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000005 2024-06-28T15:23:01.806 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.822+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000005 , ret = 0 2024-06-28T15:23:01.806 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.822+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000006 2024-06-28T15:23:01.808 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.826+0000 7f5b4b7c2980 20 process_entry resharding a-bucket 2024-06-28T15:23:01.808 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.826+0000 7f5b4b7c2980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-06-28T15:23:01.808 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.826+0000 7f5b4b7c2980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.809 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.826+0000 7f5b4b7c2980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:23:01.809 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.826+0000 7f5b4b7c2980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2024-06-28T15:23:01.809 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.826+0000 7f5b4b7c2980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-06-28T15:23:01.809 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.826+0000 7f5b4b7c2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 : miss 2024-06-28T15:23:01.809 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.826+0000 7f5b4b7c2980 20 rados->read ofs=0 len=0 2024-06-28T15:23:01.810 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.826+0000 7f5b4b7c2980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:23:01.810 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.826+0000 7f5b4b7c2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:23:01.810 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.826+0000 7f5b4b7c2980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:23:01.810 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.826+0000 7f5b4b7c2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:01.810 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.826+0000 7f5b4b7c2980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:23:01.810 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.826+0000 7f5b4b7c2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 2024-06-28T15:23:01.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.838+0000 7f5b4b7c2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:23:01.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.838+0000 7f5b4b7c2980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:23:01.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.838+0000 7f5b4b7c2980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:23:01.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.838+0000 7f5b4b7c2980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18, ofs0, ns] 2024-06-28T15:23:01.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.838+0000 7f5a2cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595102 cookie 94522921164704 notifier 5810 bl.length()=903 2024-06-28T15:23:01.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.838+0000 7f5a2cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:23:01.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.838+0000 7f5a2cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:23:01.822 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.838+0000 7f5a2cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:23:01.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.842+0000 7f5b4b7c2980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): orig_sources= new_sources= 2024-06-28T15:23:01.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.842+0000 7f5b4b7c2980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): potential sources added=[] removed=[] 2024-06-28T15:23:01.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.842+0000 7f5b4b7c2980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): orig_dests= new_dests= 2024-06-28T15:23:01.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.842+0000 7f5b4b7c2980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): potential dests added=[] removed=[] 2024-06-28T15:23:01.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.846+0000 7f5b4b7c2980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.1.0 2024-06-28T15:23:01.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.846+0000 7f5b4b7c2980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.1.0 2024-06-28T15:23:01.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.846+0000 7f5b4b7c2980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.1.1 2024-06-28T15:23:01.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.846+0000 7f5b4b7c2980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.1.1 2024-06-28T15:23:01.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.846+0000 7f5b4b7c2980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.0 2024-06-28T15:23:01.829 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.846+0000 7f5b4b7c2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 : hit (requested=0x17, cached=0x17) 2024-06-28T15:23:01.829 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.846+0000 7f5b4b7c2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 2024-06-28T15:23:01.831 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.846+0000 7f5b4b7c2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:23:01.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.846+0000 7f5b4b7c2980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:23:01.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.846+0000 7f5b4b7c2980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:23:01.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.846+0000 7f5b4b7c2980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18, ofs0, ns] 2024-06-28T15:23:01.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.850+0000 7f5a1bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595103 cookie 94522921164704 notifier 5810 bl.length()=871 2024-06-28T15:23:01.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.850+0000 7f5a1bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:23:01.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.850+0000 7f5a1bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:23:01.833 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.850+0000 7f5a1bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:23:01.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.850+0000 7f5b4b7c2980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): orig_sources= new_sources= 2024-06-28T15:23:01.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.850+0000 7f5b4b7c2980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): potential sources added=[] removed=[] 2024-06-28T15:23:01.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.850+0000 7f5b4b7c2980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): orig_dests= new_dests= 2024-06-28T15:23:01.835 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.850+0000 7f5b4b7c2980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): potential dests added=[] removed=[] 2024-06-28T15:23:01.838 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.854+0000 7f5b4b7c2980 1 execute INFO: reshard of bucket "a-bucket" from 2 shards to 2 shards completed successfully 2024-06-28T15:23:01.840 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.858+0000 7f5b4b7c2980 20 process_entry removing reshard queue entry for bucket a-bucket 2024-06-28T15:23:01.845 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.862+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000006 , ret = 0 2024-06-28T15:23:01.845 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.862+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000007 2024-06-28T15:23:01.851 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.866+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000007 , ret = 0 2024-06-28T15:23:01.851 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.866+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000008 2024-06-28T15:23:01.857 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.874+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000008 , ret = 0 2024-06-28T15:23:01.858 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.874+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000009 2024-06-28T15:23:01.864 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.882+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000009 , ret = 0 2024-06-28T15:23:01.864 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.882+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000010 2024-06-28T15:23:01.874 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.890+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000010 , ret = 0 2024-06-28T15:23:01.874 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.890+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000011 2024-06-28T15:23:01.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.898+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000011 , ret = 0 2024-06-28T15:23:01.881 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.898+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000012 2024-06-28T15:23:01.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.902+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000012 , ret = 0 2024-06-28T15:23:01.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.902+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000013 2024-06-28T15:23:01.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.910+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000013 , ret = 0 2024-06-28T15:23:01.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.910+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000014 2024-06-28T15:23:01.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.918+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000014 , ret = 0 2024-06-28T15:23:01.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.918+0000 7f5b4b7c2980 20 processing logshard = reshard.0000000015 2024-06-28T15:23:01.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.922+0000 7f5b4b7c2980 20 finish processing logshard = reshard.0000000015 , ret = 0 2024-06-28T15:23:01.909 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.926+0000 7f59ff7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:01.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.930+0000 7f5b4b7c2980 20 remove_watcher() i=0 2024-06-28T15:23:01.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.930+0000 7f5b4b7c2980 2 removed watcher, disabling cache 2024-06-28T15:23:01.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.934+0000 7f5b4b7c2980 20 remove_watcher() i=1 2024-06-28T15:23:01.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.938+0000 7f5b4b7c2980 20 remove_watcher() i=2 2024-06-28T15:23:01.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.938+0000 7f5b4b7c2980 20 remove_watcher() i=3 2024-06-28T15:23:01.925 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.942+0000 7f5b4b7c2980 20 remove_watcher() i=4 2024-06-28T15:23:01.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.946+0000 7f5b4b7c2980 20 remove_watcher() i=5 2024-06-28T15:23:01.930 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.946+0000 7f5b4b7c2980 20 remove_watcher() i=6 2024-06-28T15:23:01.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:01.950+0000 7f5b4b7c2980 20 remove_watcher() i=7 2024-06-28T15:23:06.959 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-06-28T15:23:06.991 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:06.991 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:07.041 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.058+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.058+0000 7f2ca40c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:07.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.058+0000 7f2ca40c4980 20 realm 2024-06-28T15:23:07.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.058+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.058+0000 7f2ca40c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:07.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.058+0000 7f2ca40c4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:07.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.058+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.058+0000 7f2ca40c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:07.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.058+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.062+0000 7f2ca40c4980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:07.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.062+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.062+0000 7f2ca40c4980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:07.046 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.062+0000 7f2ca40c4980 20 searching for the correct realm 2024-06-28T15:23:07.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.070+0000 7f2ca13d4640 0 -- 172.21.15.178:0/2650072435 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f2c94004040 msgr2=0x7f2c94125400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:07.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.074+0000 7f2ca13d4640 0 -- 172.21.15.178:0/2650072435 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f2b6c07fd50 msgr2=0x7f2b6c082180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:07.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:07.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:07.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:07.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:07.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:07.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:07.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:07.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:07.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:07.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 zone default found 2024-06-28T15:23:07.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 4 Realm: () 2024-06-28T15:23:07.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:07.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:07.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:07.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 zonegroup default 2024-06-28T15:23:07.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:07.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.082+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.086+0000 7f2ca40c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:07.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.086+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.086+0000 7f2ca40c4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:07.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.086+0000 7f2ca40c4980 20 started sync module instance, tier type = 2024-06-28T15:23:07.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.086+0000 7f2ca40c4980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:07.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.102+0000 7f2ca40c4980 20 add_watcher() i=0 2024-06-28T15:23:07.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.102+0000 7f2ca40c4980 20 add_watcher() i=1 2024-06-28T15:23:07.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.102+0000 7f2ca40c4980 20 add_watcher() i=2 2024-06-28T15:23:07.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.102+0000 7f2ca40c4980 20 add_watcher() i=3 2024-06-28T15:23:07.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.102+0000 7f2ca40c4980 20 add_watcher() i=4 2024-06-28T15:23:07.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.102+0000 7f2ca40c4980 20 add_watcher() i=5 2024-06-28T15:23:07.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.102+0000 7f2ca40c4980 20 add_watcher() i=6 2024-06-28T15:23:07.086 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.102+0000 7f2ca40c4980 20 add_watcher() i=7 2024-06-28T15:23:07.086 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.102+0000 7f2ca40c4980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:07.088 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.106+0000 7f2b537fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:07.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.106+0000 7f2ca40c4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:07.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.106+0000 7f2ca40c4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:07.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.106+0000 7f2ca40c4980 5 note: GC not initialized 2024-06-28T15:23:07.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.106+0000 7f2b527fc640 20 reqs_thread_entry: start 2024-06-28T15:23:07.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.106+0000 7f2ca40c4980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:07.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.106+0000 7f2ca40c4980 20 Filter name: none 2024-06-28T15:23:07.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.106+0000 7f2ac3fff640 20 reqs_thread_entry: start 2024-06-28T15:23:07.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.106+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.106+0000 7f2ca40c4980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:23:07.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.106+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.106+0000 7f2ca40c4980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:23:07.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.106+0000 7f2ca40c4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:07.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.106+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.110+0000 7f2ca40c4980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:23:07.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.110+0000 7f2ca40c4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.093 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.110+0000 7f2ca40c4980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:23:07.093 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.110+0000 7f2ca40c4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:07.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.110+0000 7f2b537fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:07.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.114+0000 7f2ca40c4980 20 remove_watcher() i=0 2024-06-28T15:23:07.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.114+0000 7f2ca40c4980 2 removed watcher, disabling cache 2024-06-28T15:23:07.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.118+0000 7f2ca40c4980 20 remove_watcher() i=1 2024-06-28T15:23:07.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.122+0000 7f2ca40c4980 20 remove_watcher() i=2 2024-06-28T15:23:07.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.122+0000 7f2ca40c4980 20 remove_watcher() i=3 2024-06-28T15:23:07.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.126+0000 7f2ca40c4980 20 remove_watcher() i=4 2024-06-28T15:23:07.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.126+0000 7f2ca40c4980 20 remove_watcher() i=5 2024-06-28T15:23:07.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.126+0000 7f2ca40c4980 20 remove_watcher() i=6 2024-06-28T15:23:07.112 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.130+0000 7f2ca40c4980 20 remove_watcher() i=7 2024-06-28T15:23:07.127 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:07.127 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:07.127 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: a-bucket id: 843e24ce-085a-4043-8f55-38e05370e790.4686.18 num_objects: 0 size_kb: 0 num_shards: 2 2024-06-28T15:23:07.127 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:07.127 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: reshard add non empty bucket 2024-06-28T15:23:07.128 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:07.842 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-06-28T15:23:07.874 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:07.874 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:07.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.934+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.938+0000 7f1508990980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:07.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.938+0000 7f1508990980 20 realm 2024-06-28T15:23:07.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.938+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.938+0000 7f1508990980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:07.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.938+0000 7f1508990980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:07.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.938+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.938+0000 7f1508990980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:07.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.938+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.938+0000 7f1508990980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:07.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.938+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:07.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.942+0000 7f1508990980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:07.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.942+0000 7f1508990980 20 searching for the correct realm 2024-06-28T15:23:07.930 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:07.946+0000 7f1506ca2640 0 -- 172.21.15.178:0/1688606570 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f14fc008040 msgr2=0x7f14fc06d9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:08.146 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.162+0000 7f15064a1640 0 -- 172.21.15.178:0/1688606570 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55f7f0f382b0 msgr2=0x55f7f0f3a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:08.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.170+0000 7f1508990980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:08.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.170+0000 7f1508990980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:08.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.170+0000 7f1508990980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:08.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.170+0000 7f1508990980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:08.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.170+0000 7f1508990980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:08.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.170+0000 7f1508990980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:08.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.170+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.154 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.170+0000 7f1508990980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.155 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.170+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.155 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.170+0000 7f1508990980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:08.155 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.170+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.157 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:08.157 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 20 zone default found 2024-06-28T15:23:08.157 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 4 Realm: () 2024-06-28T15:23:08.157 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:08.157 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:08.157 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:08.157 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 20 zonegroup default 2024-06-28T15:23:08.157 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.157 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.158 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.158 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.158 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 20 started sync module instance, tier type = 2024-06-28T15:23:08.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.174+0000 7f1508990980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:08.183 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.198+0000 7f1508990980 20 add_watcher() i=0 2024-06-28T15:23:08.183 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.198+0000 7f1508990980 20 add_watcher() i=1 2024-06-28T15:23:08.183 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.198+0000 7f1508990980 20 add_watcher() i=2 2024-06-28T15:23:08.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.198+0000 7f1508990980 20 add_watcher() i=3 2024-06-28T15:23:08.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.198+0000 7f1508990980 20 add_watcher() i=4 2024-06-28T15:23:08.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.198+0000 7f1508990980 20 add_watcher() i=5 2024-06-28T15:23:08.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.198+0000 7f1508990980 20 add_watcher() i=6 2024-06-28T15:23:08.186 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.202+0000 7f1508990980 20 add_watcher() i=7 2024-06-28T15:23:08.186 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.202+0000 7f1508990980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:08.190 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.206+0000 7f13bcfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:08.191 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.206+0000 7f1508990980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:08.191 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.206+0000 7f1508990980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:08.191 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.206+0000 7f1508990980 5 note: GC not initialized 2024-06-28T15:23:08.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.210+0000 7f13377fe640 20 reqs_thread_entry: start 2024-06-28T15:23:08.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.210+0000 7f1508990980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:08.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.210+0000 7f1508990980 20 Filter name: none 2024-06-28T15:23:08.193 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.210+0000 7f13357fa640 20 reqs_thread_entry: start 2024-06-28T15:23:08.193 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.210+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.193 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.210+0000 7f1508990980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:23:08.193 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.210+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.194 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.210+0000 7f1508990980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:23:08.194 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.210+0000 7f1508990980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:08.194 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.210+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.195 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.210+0000 7f1508990980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:23:08.195 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.210+0000 7f1508990980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.196 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.214+0000 7f1508990980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:23:08.196 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.214+0000 7f1508990980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:08.198 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.214+0000 7f13bcfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:08.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.218+0000 7f1508990980 20 remove_watcher() i=0 2024-06-28T15:23:08.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.218+0000 7f1508990980 2 removed watcher, disabling cache 2024-06-28T15:23:08.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.222+0000 7f1508990980 20 remove_watcher() i=1 2024-06-28T15:23:08.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.226+0000 7f1508990980 20 remove_watcher() i=2 2024-06-28T15:23:08.210 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.226+0000 7f1508990980 20 remove_watcher() i=3 2024-06-28T15:23:08.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.230+0000 7f1508990980 20 remove_watcher() i=4 2024-06-28T15:23:08.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.230+0000 7f1508990980 20 remove_watcher() i=5 2024-06-28T15:23:08.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.234+0000 7f1508990980 20 remove_watcher() i=6 2024-06-28T15:23:08.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.234+0000 7f1508990980 20 remove_watcher() i=7 2024-06-28T15:23:08.235 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:08.235 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:08.235 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: a-bucket id: 843e24ce-085a-4043-8f55-38e05370e790.4686.18 num_objects: 8 size_kb: 1 num_shards: 2 2024-06-28T15:23:08.235 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:08.236 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 3 2024-06-28T15:23:08.268 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:08.269 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:08.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.330+0000 7f85b5b8b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.314 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.330+0000 7f85b5b8b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.330+0000 7f85b5b8b980 20 realm 2024-06-28T15:23:08.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.330+0000 7f85b5b8b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.330+0000 7f85b5b8b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.330+0000 7f85b5b8b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:08.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.330+0000 7f85b5b8b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.330+0000 7f85b5b8b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.330+0000 7f85b5b8b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.316 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.334+0000 7f85b5b8b980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:08.316 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.334+0000 7f85b5b8b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.317 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.334+0000 7f85b5b8b980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:08.317 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.334+0000 7f85b5b8b980 20 searching for the correct realm 2024-06-28T15:23:08.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.338+0000 7f85b3e9d640 0 -- 172.21.15.178:0/747000191 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x56222cf23240 msgr2=0x56222cf256c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:08.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.346+0000 7f85b2e9b640 0 -- 172.21.15.178:0/747000191 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f85a4001af0 msgr2=0x7f85a40b8460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:08.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:08.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:08.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:08.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:08.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:08.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:08.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:08.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:08.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 zone default found 2024-06-28T15:23:08.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 4 Realm: () 2024-06-28T15:23:08.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:08.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:08.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:08.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 zonegroup default 2024-06-28T15:23:08.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.354+0000 7f85b5b8b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.358+0000 7f85b5b8b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.358+0000 7f85b5b8b980 20 started sync module instance, tier type = 2024-06-28T15:23:08.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.358+0000 7f85b5b8b980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:08.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.370+0000 7f85b5b8b980 20 add_watcher() i=0 2024-06-28T15:23:08.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.370+0000 7f85b5b8b980 20 add_watcher() i=1 2024-06-28T15:23:08.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.370+0000 7f85b5b8b980 20 add_watcher() i=2 2024-06-28T15:23:08.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.370+0000 7f85b5b8b980 20 add_watcher() i=3 2024-06-28T15:23:08.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.370+0000 7f85b5b8b980 20 add_watcher() i=4 2024-06-28T15:23:08.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.370+0000 7f85b5b8b980 20 add_watcher() i=5 2024-06-28T15:23:08.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.370+0000 7f85b5b8b980 20 add_watcher() i=6 2024-06-28T15:23:08.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.374+0000 7f85b5b8b980 20 add_watcher() i=7 2024-06-28T15:23:08.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.374+0000 7f85b5b8b980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:08.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.374+0000 7f8469fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:08.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.374+0000 7f85b5b8b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:08.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.374+0000 7f85b5b8b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:08.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.374+0000 7f85b5b8b980 5 note: GC not initialized 2024-06-28T15:23:08.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f8468fe1640 20 reqs_thread_entry: start 2024-06-28T15:23:08.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:08.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 20 Filter name: none 2024-06-28T15:23:08.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f83e27fc640 20 reqs_thread_entry: start 2024-06-28T15:23:08.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-06-28T15:23:08.361 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.361 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:23:08.361 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-06-28T15:23:08.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-06-28T15:23:08.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 : miss 2024-06-28T15:23:08.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:23:08.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:23:08.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:23:08.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:08.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:23:08.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.378+0000 7f85b5b8b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 2024-06-28T15:23:08.364 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.382+0000 7f8469fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:08.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.386+0000 7f85b5b8b980 20 remove_watcher() i=0 2024-06-28T15:23:08.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.386+0000 7f85b5b8b980 2 removed watcher, disabling cache 2024-06-28T15:23:08.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.386+0000 7f85b5b8b980 20 remove_watcher() i=1 2024-06-28T15:23:08.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.390+0000 7f85b5b8b980 20 remove_watcher() i=2 2024-06-28T15:23:08.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.390+0000 7f85b5b8b980 20 remove_watcher() i=3 2024-06-28T15:23:08.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.394+0000 7f85b5b8b980 20 remove_watcher() i=4 2024-06-28T15:23:08.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.394+0000 7f85b5b8b980 20 remove_watcher() i=5 2024-06-28T15:23:08.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.394+0000 7f85b5b8b980 20 remove_watcher() i=6 2024-06-28T15:23:08.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.398+0000 7f85b5b8b980 20 remove_watcher() i=7 2024-06-28T15:23:08.394 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin reshard list 2024-06-28T15:23:08.419 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:08.419 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:08.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.478+0000 7fd0c288c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.462 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.478+0000 7fd0c288c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.478+0000 7fd0c288c980 20 realm 2024-06-28T15:23:08.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.478+0000 7fd0c288c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.478+0000 7fd0c288c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.478+0000 7fd0c288c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:08.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.478+0000 7fd0c288c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.478+0000 7fd0c288c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.478+0000 7fd0c288c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.482+0000 7fd0c288c980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:08.465 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.482+0000 7fd0c288c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.465 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.482+0000 7fd0c288c980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:08.465 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.482+0000 7fd0c288c980 20 searching for the correct realm 2024-06-28T15:23:08.470 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.486+0000 7fd0b7fff640 0 -- 172.21.15.178:0/1222322038 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x55a13d8413d0 msgr2=0x55a13d843870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:23:08.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.494+0000 7fd0c039d640 0 -- 172.21.15.178:0/1222322038 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fd0a8008040 msgr2=0x7fd0a804c3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:08.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:08.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:08.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:08.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:08.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:08.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:08.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:08.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:08.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 zone default found 2024-06-28T15:23:08.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 4 Realm: () 2024-06-28T15:23:08.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:08.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:08.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:08.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 zonegroup default 2024-06-28T15:23:08.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.502+0000 7fd0c288c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.506+0000 7fd0c288c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.506+0000 7fd0c288c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.506+0000 7fd0c288c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.506+0000 7fd0c288c980 20 started sync module instance, tier type = 2024-06-28T15:23:08.488 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.506+0000 7fd0c288c980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:08.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.518+0000 7fd0c288c980 20 add_watcher() i=0 2024-06-28T15:23:08.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.518+0000 7fd0c288c980 20 add_watcher() i=1 2024-06-28T15:23:08.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.518+0000 7fd0c288c980 20 add_watcher() i=2 2024-06-28T15:23:08.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.518+0000 7fd0c288c980 20 add_watcher() i=3 2024-06-28T15:23:08.504 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.518+0000 7fd0c288c980 20 add_watcher() i=4 2024-06-28T15:23:08.504 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.518+0000 7fd0c288c980 20 add_watcher() i=5 2024-06-28T15:23:08.504 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.518+0000 7fd0c288c980 20 add_watcher() i=6 2024-06-28T15:23:08.504 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.522+0000 7fd0c288c980 20 add_watcher() i=7 2024-06-28T15:23:08.504 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.522+0000 7fd0c288c980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:08.507 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.522+0000 7fcf767e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:08.507 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.522+0000 7fd0c288c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:08.507 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.522+0000 7fd0c288c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:08.507 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.522+0000 7fd0c288c980 5 note: GC not initialized 2024-06-28T15:23:08.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.526+0000 7fcf757e2640 20 reqs_thread_entry: start 2024-06-28T15:23:08.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.526+0000 7fd0c288c980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:08.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.526+0000 7fd0c288c980 20 Filter name: none 2024-06-28T15:23:08.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.526+0000 7fceeeffd640 20 reqs_thread_entry: start 2024-06-28T15:23:08.517 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.534+0000 7fcf767e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:08.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.538+0000 7fd0c288c980 20 remove_watcher() i=0 2024-06-28T15:23:08.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.538+0000 7fd0c288c980 2 removed watcher, disabling cache 2024-06-28T15:23:08.523 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.542+0000 7fd0c288c980 20 remove_watcher() i=1 2024-06-28T15:23:08.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.542+0000 7fd0c288c980 20 remove_watcher() i=2 2024-06-28T15:23:08.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.546+0000 7fd0c288c980 20 remove_watcher() i=3 2024-06-28T15:23:08.529 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.546+0000 7fd0c288c980 20 remove_watcher() i=4 2024-06-28T15:23:08.530 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.546+0000 7fd0c288c980 20 remove_watcher() i=5 2024-06-28T15:23:08.532 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.550+0000 7fd0c288c980 20 remove_watcher() i=6 2024-06-28T15:23:08.533 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.550+0000 7fd0c288c980 20 remove_watcher() i=7 2024-06-28T15:23:08.547 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: reshard process non empty bucket 2024-06-28T15:23:08.547 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:08.548 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin reshard process 2024-06-28T15:23:08.572 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:08.572 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:08.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.634+0000 7eff2d068980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.634+0000 7eff2d068980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.634+0000 7eff2d068980 20 realm 2024-06-28T15:23:08.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.634+0000 7eff2d068980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.634+0000 7eff2d068980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.634+0000 7eff2d068980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:08.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.634+0000 7eff2d068980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.634+0000 7eff2d068980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:08.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.634+0000 7eff2d068980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.638+0000 7eff2d068980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:08.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.638+0000 7eff2d068980 20 rados->read ofs=0 len=0 2024-06-28T15:23:08.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.638+0000 7eff2d068980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:08.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.638+0000 7eff2d068980 20 searching for the correct realm 2024-06-28T15:23:08.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.638+0000 7eff2a378640 0 -- 172.21.15.178:0/3353634461 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7eff1c10f070 msgr2=0x7eff1c129160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:08.836 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:08.854+0000 7eff2ab79640 0 -- 172.21.15.178:0/3353634461 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7eff20051960 msgr2=0x7eff200093d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:23:09.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.070+0000 7eff2d068980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:09.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.070+0000 7eff2d068980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:09.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.070+0000 7eff2d068980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:09.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.070+0000 7eff2d068980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:09.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.070+0000 7eff2d068980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:09.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.070+0000 7eff2d068980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:09.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.070+0000 7eff2d068980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.070+0000 7eff2d068980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:09.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.070+0000 7eff2d068980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.074+0000 7eff2d068980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:09.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.074+0000 7eff2d068980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.074+0000 7eff2d068980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:09.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.074+0000 7eff2d068980 20 zone default found 2024-06-28T15:23:09.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.074+0000 7eff2d068980 4 Realm: () 2024-06-28T15:23:09.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.074+0000 7eff2d068980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:09.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.074+0000 7eff2d068980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:09.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.074+0000 7eff2d068980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:09.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.074+0000 7eff2d068980 20 zonegroup default 2024-06-28T15:23:09.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.074+0000 7eff2d068980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.074+0000 7eff2d068980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:09.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.074+0000 7eff2d068980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.074+0000 7eff2d068980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:09.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.074+0000 7eff2d068980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.078+0000 7eff2d068980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:09.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.078+0000 7eff2d068980 20 started sync module instance, tier type = 2024-06-28T15:23:09.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.078+0000 7eff2d068980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:09.082 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.098+0000 7eff2d068980 20 add_watcher() i=0 2024-06-28T15:23:09.083 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.098+0000 7eff2d068980 20 add_watcher() i=1 2024-06-28T15:23:09.083 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.098+0000 7eff2d068980 20 add_watcher() i=2 2024-06-28T15:23:09.083 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.098+0000 7eff2d068980 20 add_watcher() i=3 2024-06-28T15:23:09.083 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.098+0000 7eff2d068980 20 add_watcher() i=4 2024-06-28T15:23:09.083 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.098+0000 7eff2d068980 20 add_watcher() i=5 2024-06-28T15:23:09.083 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.098+0000 7eff2d068980 20 add_watcher() i=6 2024-06-28T15:23:09.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.102+0000 7eff2d068980 20 add_watcher() i=7 2024-06-28T15:23:09.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.102+0000 7eff2d068980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:09.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.106+0000 7efde17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:09.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.106+0000 7eff2d068980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:09.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.106+0000 7eff2d068980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:09.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.106+0000 7eff2d068980 5 note: GC not initialized 2024-06-28T15:23:09.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.106+0000 7efd5bfff640 20 reqs_thread_entry: start 2024-06-28T15:23:09.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.106+0000 7eff2d068980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:09.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.106+0000 7eff2d068980 20 Filter name: none 2024-06-28T15:23:09.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.106+0000 7efd59ffb640 20 reqs_thread_entry: start 2024-06-28T15:23:09.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.106+0000 7eff2d068980 20 processing logshard = reshard.0000000000 2024-06-28T15:23:09.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.114+0000 7eff2d068980 20 finish processing logshard = reshard.0000000000 , ret = 0 2024-06-28T15:23:09.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.114+0000 7eff2d068980 20 processing logshard = reshard.0000000001 2024-06-28T15:23:09.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.118+0000 7eff2d068980 20 finish processing logshard = reshard.0000000001 , ret = 0 2024-06-28T15:23:09.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.118+0000 7eff2d068980 20 processing logshard = reshard.0000000002 2024-06-28T15:23:09.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.122+0000 7eff2d068980 20 finish processing logshard = reshard.0000000002 , ret = 0 2024-06-28T15:23:09.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.122+0000 7eff2d068980 20 processing logshard = reshard.0000000003 2024-06-28T15:23:09.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.130+0000 7eff2d068980 20 finish processing logshard = reshard.0000000003 , ret = 0 2024-06-28T15:23:09.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.130+0000 7eff2d068980 20 processing logshard = reshard.0000000004 2024-06-28T15:23:09.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.134+0000 7eff2d068980 20 finish processing logshard = reshard.0000000004 , ret = 0 2024-06-28T15:23:09.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.134+0000 7eff2d068980 20 processing logshard = reshard.0000000005 2024-06-28T15:23:09.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.142+0000 7eff2d068980 20 finish processing logshard = reshard.0000000005 , ret = 0 2024-06-28T15:23:09.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.142+0000 7eff2d068980 20 processing logshard = reshard.0000000006 2024-06-28T15:23:09.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.146+0000 7eff2d068980 20 process_entry resharding a-bucket 2024-06-28T15:23:09.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.146+0000 7eff2d068980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-06-28T15:23:09.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.146+0000 7eff2d068980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.146+0000 7eff2d068980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:23:09.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.146+0000 7eff2d068980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2024-06-28T15:23:09.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.146+0000 7eff2d068980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-06-28T15:23:09.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.146+0000 7eff2d068980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 : miss 2024-06-28T15:23:09.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.146+0000 7eff2d068980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.146+0000 7eff2d068980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:23:09.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.146+0000 7eff2d068980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:23:09.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.146+0000 7eff2d068980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:23:09.130 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.146+0000 7eff2d068980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:09.131 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.146+0000 7eff2d068980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:23:09.131 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.146+0000 7eff2d068980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 2024-06-28T15:23:09.143 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.158+0000 7eff2d068980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:23:09.143 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.158+0000 7eff2d068980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:23:09.143 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.158+0000 7eff2d068980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:23:09.143 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.158+0000 7eff2d068980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18, ofs0, ns] 2024-06-28T15:23:09.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.162+0000 7efe0e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595104 cookie 93868733976832 notifier 5888 bl.length()=903 2024-06-28T15:23:09.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.162+0000 7efe0e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:23:09.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.162+0000 7efe0e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:23:09.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.162+0000 7efe0e7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:23:09.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.162+0000 7eff2d068980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): orig_sources= new_sources= 2024-06-28T15:23:09.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.162+0000 7eff2d068980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): potential sources added=[] removed=[] 2024-06-28T15:23:09.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.162+0000 7eff2d068980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): orig_dests= new_dests= 2024-06-28T15:23:09.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.162+0000 7eff2d068980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): potential dests added=[] removed=[] 2024-06-28T15:23:09.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.166+0000 7eff2d068980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.0 2024-06-28T15:23:09.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.166+0000 7eff2d068980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.0 2024-06-28T15:23:09.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.166+0000 7eff2d068980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.1 2024-06-28T15:23:09.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.166+0000 7eff2d068980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.1 2024-06-28T15:23:09.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.166+0000 7eff2d068980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.2 2024-06-28T15:23:09.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.166+0000 7eff2d068980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.2 2024-06-28T15:23:09.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.166+0000 7eff2d068980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.1.0 2024-06-28T15:23:09.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.166+0000 7eff2d068980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.1.0 2024-06-28T15:23:09.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.166+0000 7eff2d068980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.1.1 2024-06-28T15:23:09.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.166+0000 7eff2d068980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.1.1 2024-06-28T15:23:09.155 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.170+0000 7eff2d068980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 : hit (requested=0x17, cached=0x17) 2024-06-28T15:23:09.155 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.170+0000 7eff2d068980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 2024-06-28T15:23:09.158 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.174+0000 7eff2d068980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:23:09.158 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.174+0000 7eff2d068980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:23:09.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.174+0000 7eff2d068980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:23:09.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.174+0000 7eff2d068980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18, ofs0, ns] 2024-06-28T15:23:09.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.174+0000 7efe0e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595105 cookie 93868733976832 notifier 5888 bl.length()=871 2024-06-28T15:23:09.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.174+0000 7efe0e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:23:09.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.174+0000 7efe0e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:23:09.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.174+0000 7efe0e7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:23:09.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.178+0000 7eff2d068980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): orig_sources= new_sources= 2024-06-28T15:23:09.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.178+0000 7eff2d068980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): potential sources added=[] removed=[] 2024-06-28T15:23:09.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.178+0000 7eff2d068980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): orig_dests= new_dests= 2024-06-28T15:23:09.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.178+0000 7eff2d068980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): potential dests added=[] removed=[] 2024-06-28T15:23:09.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.182+0000 7eff2d068980 1 execute INFO: reshard of bucket "a-bucket" from 3 shards to 3 shards completed successfully 2024-06-28T15:23:09.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.186+0000 7eff2d068980 20 process_entry removing reshard queue entry for bucket a-bucket 2024-06-28T15:23:09.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.190+0000 7eff2d068980 20 finish processing logshard = reshard.0000000006 , ret = 0 2024-06-28T15:23:09.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.190+0000 7eff2d068980 20 processing logshard = reshard.0000000007 2024-06-28T15:23:09.178 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.194+0000 7eff2d068980 20 finish processing logshard = reshard.0000000007 , ret = 0 2024-06-28T15:23:09.178 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.194+0000 7eff2d068980 20 processing logshard = reshard.0000000008 2024-06-28T15:23:09.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.202+0000 7eff2d068980 20 finish processing logshard = reshard.0000000008 , ret = 0 2024-06-28T15:23:09.185 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.202+0000 7eff2d068980 20 processing logshard = reshard.0000000009 2024-06-28T15:23:09.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.210+0000 7eff2d068980 20 finish processing logshard = reshard.0000000009 , ret = 0 2024-06-28T15:23:09.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.210+0000 7eff2d068980 20 processing logshard = reshard.0000000010 2024-06-28T15:23:09.197 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.214+0000 7eff2d068980 20 finish processing logshard = reshard.0000000010 , ret = 0 2024-06-28T15:23:09.197 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.214+0000 7eff2d068980 20 processing logshard = reshard.0000000011 2024-06-28T15:23:09.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.218+0000 7eff2d068980 20 finish processing logshard = reshard.0000000011 , ret = 0 2024-06-28T15:23:09.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.218+0000 7eff2d068980 20 processing logshard = reshard.0000000012 2024-06-28T15:23:09.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.226+0000 7eff2d068980 20 finish processing logshard = reshard.0000000012 , ret = 0 2024-06-28T15:23:09.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.226+0000 7eff2d068980 20 processing logshard = reshard.0000000013 2024-06-28T15:23:09.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.234+0000 7eff2d068980 20 finish processing logshard = reshard.0000000013 , ret = 0 2024-06-28T15:23:09.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.234+0000 7eff2d068980 20 processing logshard = reshard.0000000014 2024-06-28T15:23:09.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.238+0000 7eff2d068980 20 finish processing logshard = reshard.0000000014 , ret = 0 2024-06-28T15:23:09.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.238+0000 7eff2d068980 20 processing logshard = reshard.0000000015 2024-06-28T15:23:09.228 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.246+0000 7eff2d068980 20 finish processing logshard = reshard.0000000015 , ret = 0 2024-06-28T15:23:09.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.246+0000 7efde17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:09.234 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.250+0000 7eff2d068980 20 remove_watcher() i=0 2024-06-28T15:23:09.234 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.250+0000 7eff2d068980 2 removed watcher, disabling cache 2024-06-28T15:23:09.236 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.254+0000 7eff2d068980 20 remove_watcher() i=1 2024-06-28T15:23:09.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.254+0000 7eff2d068980 20 remove_watcher() i=2 2024-06-28T15:23:09.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.258+0000 7eff2d068980 20 remove_watcher() i=3 2024-06-28T15:23:09.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.258+0000 7eff2d068980 20 remove_watcher() i=4 2024-06-28T15:23:09.245 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.262+0000 7eff2d068980 20 remove_watcher() i=5 2024-06-28T15:23:09.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.262+0000 7eff2d068980 20 remove_watcher() i=6 2024-06-28T15:23:09.248 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.266+0000 7eff2d068980 20 remove_watcher() i=7 2024-06-28T15:23:09.268 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-06-28T15:23:09.301 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:09.301 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:09.346 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.362+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.366+0000 7f7a72194980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:09.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.366+0000 7f7a72194980 20 realm 2024-06-28T15:23:09.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.366+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.366+0000 7f7a72194980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:09.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.366+0000 7f7a72194980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:09.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.366+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.366+0000 7f7a72194980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:09.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.366+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.366+0000 7f7a72194980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:09.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.366+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.366+0000 7f7a72194980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:09.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.366+0000 7f7a72194980 20 searching for the correct realm 2024-06-28T15:23:09.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.370+0000 7f7a6f4a4640 0 -- 172.21.15.178:0/236449542 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f7a60038040 msgr2=0x7f7a6004f470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:09.362 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.378+0000 7f7a6fca5640 0 -- 172.21.15.178:0/236449542 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f7a64067390 msgr2=0x7f7a640625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:09.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.594+0000 7f7a72194980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:09.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.594+0000 7f7a72194980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:09.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.594+0000 7f7a72194980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:09.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.594+0000 7f7a72194980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:09.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.594+0000 7f7a72194980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:09.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.594+0000 7f7a72194980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:09.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.594+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.594+0000 7f7a72194980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:09.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.594+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.598+0000 7f7a72194980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:09.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.598+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.598+0000 7f7a72194980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:09.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.598+0000 7f7a72194980 20 zone default found 2024-06-28T15:23:09.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.598+0000 7f7a72194980 4 Realm: () 2024-06-28T15:23:09.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.598+0000 7f7a72194980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:09.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.598+0000 7f7a72194980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:09.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.598+0000 7f7a72194980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:09.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.598+0000 7f7a72194980 20 zonegroup default 2024-06-28T15:23:09.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.598+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.583 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.598+0000 7f7a72194980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:09.583 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.598+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.583 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.598+0000 7f7a72194980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:09.583 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.598+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.602+0000 7f7a72194980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:09.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.602+0000 7f7a72194980 20 started sync module instance, tier type = 2024-06-28T15:23:09.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.602+0000 7f7a72194980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:09.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.622+0000 7f7a72194980 20 add_watcher() i=0 2024-06-28T15:23:09.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.622+0000 7f7a72194980 20 add_watcher() i=1 2024-06-28T15:23:09.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.622+0000 7f7a72194980 20 add_watcher() i=2 2024-06-28T15:23:09.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.622+0000 7f7a72194980 20 add_watcher() i=3 2024-06-28T15:23:09.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.622+0000 7f7a72194980 20 add_watcher() i=4 2024-06-28T15:23:09.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.622+0000 7f7a72194980 20 add_watcher() i=5 2024-06-28T15:23:09.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.622+0000 7f7a72194980 20 add_watcher() i=6 2024-06-28T15:23:09.610 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.626+0000 7f7a72194980 20 add_watcher() i=7 2024-06-28T15:23:09.610 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.626+0000 7f7a72194980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:09.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.630+0000 7f7925fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:09.615 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.630+0000 7f7a72194980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:09.615 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.630+0000 7f7a72194980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:09.615 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.630+0000 7f7a72194980 5 note: GC not initialized 2024-06-28T15:23:09.615 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.630+0000 7f7924fe1640 20 reqs_thread_entry: start 2024-06-28T15:23:09.615 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.634+0000 7f7a72194980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:09.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.634+0000 7f7a72194980 20 Filter name: none 2024-06-28T15:23:09.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.634+0000 7f789e7fc640 20 reqs_thread_entry: start 2024-06-28T15:23:09.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.634+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.634+0000 7f7a72194980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:23:09.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.634+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.634+0000 7f7a72194980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:23:09.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.634+0000 7f7a72194980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:09.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.634+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.634+0000 7f7a72194980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:23:09.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.634+0000 7f7a72194980 20 rados->read ofs=0 len=0 2024-06-28T15:23:09.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.638+0000 7f7a72194980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:23:09.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.638+0000 7f7a72194980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:09.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.638+0000 7f7925fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:09.627 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.642+0000 7f7a72194980 20 remove_watcher() i=0 2024-06-28T15:23:09.627 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.642+0000 7f7a72194980 2 removed watcher, disabling cache 2024-06-28T15:23:09.629 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.646+0000 7f7a72194980 20 remove_watcher() i=1 2024-06-28T15:23:09.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.646+0000 7f7a72194980 20 remove_watcher() i=2 2024-06-28T15:23:09.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.650+0000 7f7a72194980 20 remove_watcher() i=3 2024-06-28T15:23:09.636 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.654+0000 7f7a72194980 20 remove_watcher() i=4 2024-06-28T15:23:09.637 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.654+0000 7f7a72194980 20 remove_watcher() i=5 2024-06-28T15:23:09.639 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.654+0000 7f7a72194980 20 remove_watcher() i=6 2024-06-28T15:23:09.640 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:09.658+0000 7f7a72194980 20 remove_watcher() i=7 2024-06-28T15:23:09.657 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:09.657 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:09.657 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: a-bucket id: 843e24ce-085a-4043-8f55-38e05370e790.4686.18 num_objects: 8 size_kb: 1 num_shards: 3 2024-06-28T15:23:09.657 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:09.657 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: reshard bucket with EIO injected at set_target_layout 2024-06-28T15:23:09.657 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:11.750 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-06-28T15:23:11.783 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:11.783 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:11.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.846+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.830 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.846+0000 7f44fcf50980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:11.830 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.846+0000 7f44fcf50980 20 realm 2024-06-28T15:23:11.830 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.846+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.831 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.846+0000 7f44fcf50980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:11.831 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.846+0000 7f44fcf50980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:11.831 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.846+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.831 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.846+0000 7f44fcf50980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:11.831 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.846+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.850+0000 7f44fcf50980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:11.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.850+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.833 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.850+0000 7f44fcf50980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:11.833 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.850+0000 7f44fcf50980 20 searching for the correct realm 2024-06-28T15:23:11.838 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.854+0000 7f44fa260640 0 -- 172.21.15.178:0/2982552105 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x55af5f1aaa00 msgr2=0x55af5f1acdf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:11.844 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.862+0000 7f44faa61640 0 -- 172.21.15.178:0/2982552105 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f44e4062c20 msgr2=0x7f44e40595e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:11.852 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:11.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:11.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:11.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:11.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:11.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:11.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:11.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:11.854 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.854 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:11.854 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 zone default found 2024-06-28T15:23:11.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 4 Realm: () 2024-06-28T15:23:11.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:11.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:11.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:11.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 zonegroup default 2024-06-28T15:23:11.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:11.855 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.856 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:11.856 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.870+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.856 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.874+0000 7f44fcf50980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:11.856 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.874+0000 7f44fcf50980 20 started sync module instance, tier type = 2024-06-28T15:23:11.856 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.874+0000 7f44fcf50980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:11.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.886+0000 7f44fcf50980 20 add_watcher() i=0 2024-06-28T15:23:11.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.886+0000 7f44fcf50980 20 add_watcher() i=1 2024-06-28T15:23:11.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.886+0000 7f44fcf50980 20 add_watcher() i=2 2024-06-28T15:23:11.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.886+0000 7f44fcf50980 20 add_watcher() i=3 2024-06-28T15:23:11.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.886+0000 7f44fcf50980 20 add_watcher() i=4 2024-06-28T15:23:11.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.886+0000 7f44fcf50980 20 add_watcher() i=5 2024-06-28T15:23:11.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.886+0000 7f44fcf50980 20 add_watcher() i=6 2024-06-28T15:23:11.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.890+0000 7f44fcf50980 20 add_watcher() i=7 2024-06-28T15:23:11.872 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.890+0000 7f44fcf50980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:11.875 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.890+0000 7f43b17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:11.875 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.890+0000 7f44fcf50980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:11.875 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.890+0000 7f44fcf50980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:11.876 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.890+0000 7f44fcf50980 5 note: GC not initialized 2024-06-28T15:23:11.876 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.894+0000 7f432bfff640 20 reqs_thread_entry: start 2024-06-28T15:23:11.876 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.894+0000 7f44fcf50980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:11.876 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.894+0000 7f44fcf50980 20 Filter name: none 2024-06-28T15:23:11.876 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.894+0000 7f4329ffb640 20 reqs_thread_entry: start 2024-06-28T15:23:11.876 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.894+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.876 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.894+0000 7f44fcf50980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:11.877 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.894+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.878 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.894+0000 7f44fcf50980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:11.878 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.894+0000 7f44fcf50980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:11.878 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.894+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.878 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.894+0000 7f44fcf50980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:11.878 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.894+0000 7f44fcf50980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.894+0000 7f44fcf50980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:11.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.894+0000 7f44fcf50980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:11.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.898+0000 7f43b17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:11.885 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.902+0000 7f44fcf50980 20 remove_watcher() i=0 2024-06-28T15:23:11.885 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.902+0000 7f44fcf50980 2 removed watcher, disabling cache 2024-06-28T15:23:11.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.902+0000 7f44fcf50980 20 remove_watcher() i=1 2024-06-28T15:23:11.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.906+0000 7f44fcf50980 20 remove_watcher() i=2 2024-06-28T15:23:11.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.906+0000 7f44fcf50980 20 remove_watcher() i=3 2024-06-28T15:23:11.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.910+0000 7f44fcf50980 20 remove_watcher() i=4 2024-06-28T15:23:11.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.910+0000 7f44fcf50980 20 remove_watcher() i=5 2024-06-28T15:23:11.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.914+0000 7f44fcf50980 20 remove_watcher() i=6 2024-06-28T15:23:11.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:11.914+0000 7f44fcf50980 20 remove_watcher() i=7 2024-06-28T15:23:11.911 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:11.912 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:11.912 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-set-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.20 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:23:11.912 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:11.912 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout 2024-06-28T15:23:11.943 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:11.943 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:11.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.002+0000 7faed159c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.002+0000 7faed159c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:11.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.002+0000 7faed159c980 20 realm 2024-06-28T15:23:11.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.002+0000 7faed159c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.002+0000 7faed159c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:11.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.002+0000 7faed159c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:11.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.006+0000 7faed159c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.006+0000 7faed159c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:11.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.006+0000 7faed159c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.006+0000 7faed159c980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:11.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.006+0000 7faed159c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:11.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.006+0000 7faed159c980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:11.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.006+0000 7faed159c980 20 searching for the correct realm 2024-06-28T15:23:11.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.014+0000 7faecf8ae640 0 -- 172.21.15.178:0/1093137437 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7faec402c040 msgr2=0x7faec40426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:12.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.230+0000 7faecf0ad640 0 -- 172.21.15.178:0/1093137437 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x555fa481ee80 msgr2=0x555fa48212b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:12.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.238+0000 7faed159c980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:12.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.238+0000 7faed159c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:12.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.238+0000 7faed159c980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:12.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.238+0000 7faed159c980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:12.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.238+0000 7faed159c980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:12.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.238+0000 7faed159c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:12.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.238+0000 7faed159c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.238+0000 7faed159c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.238+0000 7faed159c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.238+0000 7faed159c980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:12.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.238+0000 7faed159c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:12.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 20 zone default found 2024-06-28T15:23:12.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 4 Realm: () 2024-06-28T15:23:12.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:12.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:12.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:12.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 20 zonegroup default 2024-06-28T15:23:12.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 20 started sync module instance, tier type = 2024-06-28T15:23:12.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.242+0000 7faed159c980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:12.251 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.266+0000 7faed159c980 20 add_watcher() i=0 2024-06-28T15:23:12.251 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.266+0000 7faed159c980 20 add_watcher() i=1 2024-06-28T15:23:12.251 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.266+0000 7faed159c980 20 add_watcher() i=2 2024-06-28T15:23:12.251 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.266+0000 7faed159c980 20 add_watcher() i=3 2024-06-28T15:23:12.251 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.266+0000 7faed159c980 20 add_watcher() i=4 2024-06-28T15:23:12.251 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.266+0000 7faed159c980 20 add_watcher() i=5 2024-06-28T15:23:12.251 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.266+0000 7faed159c980 20 add_watcher() i=6 2024-06-28T15:23:12.253 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.270+0000 7faed159c980 20 add_watcher() i=7 2024-06-28T15:23:12.253 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.270+0000 7faed159c980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:12.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.274+0000 7fad85fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:12.258 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.274+0000 7faed159c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:12.258 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.274+0000 7faed159c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:12.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.274+0000 7faed159c980 5 note: GC not initialized 2024-06-28T15:23:12.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.274+0000 7fad84fe1640 20 reqs_thread_entry: start 2024-06-28T15:23:12.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.274+0000 7faed159c980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:12.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7faed159c980 20 Filter name: none 2024-06-28T15:23:12.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7facfe7fc640 20 reqs_thread_entry: start 2024-06-28T15:23:12.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7faed159c980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-06-28T15:23:12.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7faed159c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7faed159c980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:12.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7faed159c980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-06-28T15:23:12.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7faed159c980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-06-28T15:23:12.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7faed159c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 : miss 2024-06-28T15:23:12.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7faed159c980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7faed159c980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:12.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7faed159c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 info.flags=0x17 2024-06-28T15:23:12.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7faed159c980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 to cache LRU end 2024-06-28T15:23:12.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7faed159c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:12.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7faed159c980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:12.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.278+0000 7faed159c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 2024-06-28T15:23:12.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.286+0000 7faed159c980 -1 Injecting error=-5 at location=set_target_layout 2024-06-28T15:23:12.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.286+0000 7faed159c980 0 ERROR: init_target_layout failed to write target index layout to bucket info: (5) Input/output error 2024-06-28T15:23:12.276 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.294+0000 7fad85fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:12.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.298+0000 7faed159c980 20 remove_watcher() i=0 2024-06-28T15:23:12.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.298+0000 7faed159c980 2 removed watcher, disabling cache 2024-06-28T15:23:12.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.298+0000 7faed159c980 20 remove_watcher() i=1 2024-06-28T15:23:12.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.302+0000 7faed159c980 20 remove_watcher() i=2 2024-06-28T15:23:12.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.302+0000 7faed159c980 20 remove_watcher() i=3 2024-06-28T15:23:12.289 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.306+0000 7faed159c980 20 remove_watcher() i=4 2024-06-28T15:23:12.294 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.310+0000 7faed159c980 20 remove_watcher() i=5 2024-06-28T15:23:12.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.314+0000 7faed159c980 20 remove_watcher() i=6 2024-06-28T15:23:12.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.314+0000 7faed159c980 20 remove_watcher() i=7 2024-06-28T15:23:12.313 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-06-28T15:23:12.346 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:12.346 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:12.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.402+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.388 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.402+0000 7fbd1d33e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.388 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.402+0000 7fbd1d33e980 20 realm 2024-06-28T15:23:12.388 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.402+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.406+0000 7fbd1d33e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.406+0000 7fbd1d33e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:12.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.406+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.406+0000 7fbd1d33e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.406+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.406+0000 7fbd1d33e980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:12.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.406+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.406+0000 7fbd1d33e980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:12.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.406+0000 7fbd1d33e980 20 searching for the correct realm 2024-06-28T15:23:12.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.414+0000 7fbd1b650640 0 -- 172.21.15.178:0/1558606388 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fbd1006c040 msgr2=0x7fbd10064a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:12.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.630+0000 7fbd1ae4f640 0 -- 172.21.15.178:0/1558606388 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x561a065654e0 msgr2=0x561a065678d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:12.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.638+0000 7fbd1d33e980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:12.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.638+0000 7fbd1d33e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:12.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.638+0000 7fbd1d33e980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:12.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.638+0000 7fbd1d33e980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:12.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.638+0000 7fbd1d33e980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:12.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.638+0000 7fbd1d33e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:12.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.638+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.638+0000 7fbd1d33e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.638+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.638+0000 7fbd1d33e980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:12.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.638+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:12.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 20 zone default found 2024-06-28T15:23:12.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 4 Realm: () 2024-06-28T15:23:12.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:12.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:12.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:12.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 20 zonegroup default 2024-06-28T15:23:12.624 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.625 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.625 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.625 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.625 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 20 started sync module instance, tier type = 2024-06-28T15:23:12.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.642+0000 7fbd1d33e980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:12.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.666+0000 7fbd1d33e980 20 add_watcher() i=0 2024-06-28T15:23:12.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.666+0000 7fbd1d33e980 20 add_watcher() i=1 2024-06-28T15:23:12.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.666+0000 7fbd1d33e980 20 add_watcher() i=2 2024-06-28T15:23:12.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.666+0000 7fbd1d33e980 20 add_watcher() i=3 2024-06-28T15:23:12.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.666+0000 7fbd1d33e980 20 add_watcher() i=4 2024-06-28T15:23:12.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.666+0000 7fbd1d33e980 20 add_watcher() i=5 2024-06-28T15:23:12.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.666+0000 7fbd1d33e980 20 add_watcher() i=6 2024-06-28T15:23:12.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.666+0000 7fbd1d33e980 20 add_watcher() i=7 2024-06-28T15:23:12.651 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.666+0000 7fbd1d33e980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:12.654 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.670+0000 7fbbd17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:12.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.670+0000 7fbd1d33e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:12.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.670+0000 7fbd1d33e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:12.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.670+0000 7fbd1d33e980 5 note: GC not initialized 2024-06-28T15:23:12.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.670+0000 7fbb4bfff640 20 reqs_thread_entry: start 2024-06-28T15:23:12.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.674+0000 7fbd1d33e980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:12.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.674+0000 7fbd1d33e980 20 Filter name: none 2024-06-28T15:23:12.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.674+0000 7fbb49ffb640 20 reqs_thread_entry: start 2024-06-28T15:23:12.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.674+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.657 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.674+0000 7fbd1d33e980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:12.657 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.674+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.658 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.674+0000 7fbd1d33e980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:12.658 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.674+0000 7fbd1d33e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:12.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.674+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.674+0000 7fbd1d33e980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:12.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.674+0000 7fbd1d33e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.678+0000 7fbd1d33e980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:12.659 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.678+0000 7fbd1d33e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:12.661 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.678+0000 7fbbd17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:12.666 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.682+0000 7fbd1d33e980 20 remove_watcher() i=0 2024-06-28T15:23:12.667 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.682+0000 7fbd1d33e980 2 removed watcher, disabling cache 2024-06-28T15:23:12.669 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.686+0000 7fbd1d33e980 20 remove_watcher() i=1 2024-06-28T15:23:12.671 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.690+0000 7fbd1d33e980 20 remove_watcher() i=2 2024-06-28T15:23:12.674 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.690+0000 7fbd1d33e980 20 remove_watcher() i=3 2024-06-28T15:23:12.676 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.694+0000 7fbd1d33e980 20 remove_watcher() i=4 2024-06-28T15:23:12.677 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.694+0000 7fbd1d33e980 20 remove_watcher() i=5 2024-06-28T15:23:12.679 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.694+0000 7fbd1d33e980 20 remove_watcher() i=6 2024-06-28T15:23:12.681 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.698+0000 7fbd1d33e980 20 remove_watcher() i=7 2024-06-28T15:23:12.699 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:12.700 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:12.700 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-set-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.20 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:23:12.700 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:12.837 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-06-28T15:23:12.873 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:12.873 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:12.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.934+0000 7efc6d94f980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.934+0000 7efc6d94f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.934+0000 7efc6d94f980 20 realm 2024-06-28T15:23:12.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.934+0000 7efc6d94f980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.938+0000 7efc6d94f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.938+0000 7efc6d94f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:12.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.938+0000 7efc6d94f980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.938+0000 7efc6d94f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.920 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.938+0000 7efc6d94f980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.938+0000 7efc6d94f980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:12.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.938+0000 7efc6d94f980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.938+0000 7efc6d94f980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:12.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.938+0000 7efc6d94f980 20 searching for the correct realm 2024-06-28T15:23:12.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.946+0000 7efc6bc61640 0 -- 172.21.15.178:0/2927984778 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x55dc4acf5140 msgr2=0x55dc4acf75d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:12.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.950+0000 7efc6ac5f640 0 -- 172.21.15.178:0/2927984778 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7efc5c027040 msgr2=0x7efc5c02a040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:12.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:12.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:12.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:12.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:12.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:12.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:12.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:12.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:12.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 zone default found 2024-06-28T15:23:12.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 4 Realm: () 2024-06-28T15:23:12.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:12.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:12.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:12.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 zonegroup default 2024-06-28T15:23:12.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.958+0000 7efc6d94f980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.962+0000 7efc6d94f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.962+0000 7efc6d94f980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.962+0000 7efc6d94f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.962+0000 7efc6d94f980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.962+0000 7efc6d94f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:12.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.962+0000 7efc6d94f980 20 started sync module instance, tier type = 2024-06-28T15:23:12.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.962+0000 7efc6d94f980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:12.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.978+0000 7efc6d94f980 20 add_watcher() i=0 2024-06-28T15:23:12.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.978+0000 7efc6d94f980 20 add_watcher() i=1 2024-06-28T15:23:12.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.978+0000 7efc6d94f980 20 add_watcher() i=2 2024-06-28T15:23:12.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.978+0000 7efc6d94f980 20 add_watcher() i=3 2024-06-28T15:23:12.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.978+0000 7efc6d94f980 20 add_watcher() i=4 2024-06-28T15:23:12.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.978+0000 7efc6d94f980 20 add_watcher() i=5 2024-06-28T15:23:12.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.978+0000 7efc6d94f980 20 add_watcher() i=6 2024-06-28T15:23:12.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.978+0000 7efc6d94f980 20 add_watcher() i=7 2024-06-28T15:23:12.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.978+0000 7efc6d94f980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:12.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efb21fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:12.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:12.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:12.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 5 note: GC not initialized 2024-06-28T15:23:12.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efb20fe1640 20 reqs_thread_entry: start 2024-06-28T15:23:12.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:12.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 20 Filter name: none 2024-06-28T15:23:12.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efa9a7fc640 20 reqs_thread_entry: start 2024-06-28T15:23:12.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-06-28T15:23:12.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:12.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-06-28T15:23:12.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-06-28T15:23:12.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 : miss 2024-06-28T15:23:12.969 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 20 rados->read ofs=0 len=0 2024-06-28T15:23:12.969 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:12.969 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 info.flags=0x17 2024-06-28T15:23:12.969 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 to cache LRU end 2024-06-28T15:23:12.969 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:12.969 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:12.969 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.982+0000 7efc6d94f980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 2024-06-28T15:23:12.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.990+0000 7efc6d94f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 info.flags=0x17 2024-06-28T15:23:12.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.990+0000 7efc6d94f980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 to cache LRU end 2024-06-28T15:23:12.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.990+0000 7efc6d94f980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:12.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.990+0000 7efc6d94f980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20, ofs0, ns] 2024-06-28T15:23:12.975 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.990+0000 7efb4effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595107 cookie 94404636317264 notifier 5975 bl.length()=1001 2024-06-28T15:23:12.975 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.990+0000 7efb4effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 info.flags=0x17 2024-06-28T15:23:12.975 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.990+0000 7efb4effd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 to cache LRU end 2024-06-28T15:23:12.975 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.990+0000 7efb4effd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:12.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.994+0000 7efc6d94f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.20]): orig_sources= new_sources= 2024-06-28T15:23:12.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.994+0000 7efc6d94f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.20]): potential sources added=[] removed=[] 2024-06-28T15:23:12.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.994+0000 7efc6d94f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.20]): orig_dests= new_dests= 2024-06-28T15:23:12.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.994+0000 7efc6d94f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.20]): potential dests added=[] removed=[] 2024-06-28T15:23:12.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.994+0000 7efc6d94f980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.20.1.0 2024-06-28T15:23:12.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.994+0000 7efc6d94f980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.20.1.0 2024-06-28T15:23:12.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.994+0000 7efc6d94f980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.20.1.1 2024-06-28T15:23:12.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.994+0000 7efc6d94f980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.20.1.1 2024-06-28T15:23:12.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.994+0000 7efc6d94f980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.20.0 2024-06-28T15:23:12.982 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.998+0000 7efc6d94f980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 : hit (requested=0x17, cached=0x17) 2024-06-28T15:23:12.982 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:12.998+0000 7efc6d94f980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 2024-06-28T15:23:12.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.002+0000 7efc6d94f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 info.flags=0x17 2024-06-28T15:23:12.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.002+0000 7efc6d94f980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 to cache LRU end 2024-06-28T15:23:12.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.002+0000 7efc6d94f980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:12.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.002+0000 7efc6d94f980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20, ofs0, ns] 2024-06-28T15:23:12.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.002+0000 7efb4effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595108 cookie 94404636317264 notifier 5975 bl.length()=969 2024-06-28T15:23:12.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.002+0000 7efb4effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 info.flags=0x17 2024-06-28T15:23:12.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.002+0000 7efb4effd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.20 to cache LRU end 2024-06-28T15:23:12.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.002+0000 7efb4effd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:12.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.006+0000 7efc6d94f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.20]): orig_sources= new_sources= 2024-06-28T15:23:12.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.006+0000 7efc6d94f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.20]): potential sources added=[] removed=[] 2024-06-28T15:23:12.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.006+0000 7efc6d94f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.20]): orig_dests= new_dests= 2024-06-28T15:23:12.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.006+0000 7efc6d94f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.20]): potential dests added=[] removed=[] 2024-06-28T15:23:12.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.006+0000 7efc6d94f980 1 execute INFO: reshard of bucket "error-at-set-target-layout" from 2 shards to 2 shards completed successfully 2024-06-28T15:23:12.992 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.010+0000 7efb21fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:12.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.014+0000 7efc6d94f980 20 remove_watcher() i=0 2024-06-28T15:23:12.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.014+0000 7efc6d94f980 2 removed watcher, disabling cache 2024-06-28T15:23:12.999 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.014+0000 7efc6d94f980 20 remove_watcher() i=1 2024-06-28T15:23:13.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.018+0000 7efc6d94f980 20 remove_watcher() i=2 2024-06-28T15:23:13.003 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.022+0000 7efc6d94f980 20 remove_watcher() i=3 2024-06-28T15:23:13.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.022+0000 7efc6d94f980 20 remove_watcher() i=4 2024-06-28T15:23:13.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.022+0000 7efc6d94f980 20 remove_watcher() i=5 2024-06-28T15:23:13.008 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.026+0000 7efc6d94f980 20 remove_watcher() i=6 2024-06-28T15:23:13.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.026+0000 7efc6d94f980 20 remove_watcher() i=7 2024-06-28T15:23:13.025 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-06-28T15:23:13.056 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:13.056 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:13.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.118+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.118+0000 7f859ee47980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:13.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.118+0000 7f859ee47980 20 realm 2024-06-28T15:23:13.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.118+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.118+0000 7f859ee47980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:13.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.118+0000 7f859ee47980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:13.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.118+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.118+0000 7f859ee47980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:13.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.118+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.122+0000 7f859ee47980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:13.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.122+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.122+0000 7f859ee47980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:13.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.122+0000 7f859ee47980 20 searching for the correct realm 2024-06-28T15:23:13.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.126+0000 7f859c157640 0 -- 172.21.15.178:0/3445508235 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x55b6b792fe40 msgr2=0x55b6b7932230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:13.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.134+0000 7f859c958640 0 -- 172.21.15.178:0/3445508235 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f859011cb20 msgr2=0x7f85901311b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:13.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.138+0000 7f859ee47980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:13.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.138+0000 7f859ee47980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:13.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.138+0000 7f859ee47980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:13.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.138+0000 7f859ee47980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:13.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.138+0000 7f859ee47980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:13.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.138+0000 7f859ee47980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:13.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.138+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.138+0000 7f859ee47980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:13.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.138+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:13.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:13.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 20 zone default found 2024-06-28T15:23:13.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 4 Realm: () 2024-06-28T15:23:13.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:13.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:13.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:13.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 20 zonegroup default 2024-06-28T15:23:13.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:13.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:13.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:13.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 20 started sync module instance, tier type = 2024-06-28T15:23:13.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.142+0000 7f859ee47980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:13.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.162+0000 7f859ee47980 20 add_watcher() i=0 2024-06-28T15:23:13.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.162+0000 7f859ee47980 20 add_watcher() i=1 2024-06-28T15:23:13.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.162+0000 7f859ee47980 20 add_watcher() i=2 2024-06-28T15:23:13.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.162+0000 7f859ee47980 20 add_watcher() i=3 2024-06-28T15:23:13.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.162+0000 7f859ee47980 20 add_watcher() i=4 2024-06-28T15:23:13.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.162+0000 7f859ee47980 20 add_watcher() i=5 2024-06-28T15:23:13.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.162+0000 7f859ee47980 20 add_watcher() i=6 2024-06-28T15:23:13.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.162+0000 7f859ee47980 20 add_watcher() i=7 2024-06-28T15:23:13.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.162+0000 7f859ee47980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:13.147 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.162+0000 7f8452fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:13.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:13.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:13.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 5 note: GC not initialized 2024-06-28T15:23:13.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f8451fe3640 20 reqs_thread_entry: start 2024-06-28T15:23:13.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:13.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 20 Filter name: none 2024-06-28T15:23:13.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f83c37fe640 20 reqs_thread_entry: start 2024-06-28T15:23:13.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:13.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:13.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:13.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:13.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 20 rados->read ofs=0 len=0 2024-06-28T15:23:13.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:13.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.166+0000 7f859ee47980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:13.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.170+0000 7f8452fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:13.160 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.178+0000 7f859ee47980 20 remove_watcher() i=0 2024-06-28T15:23:13.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.178+0000 7f859ee47980 2 removed watcher, disabling cache 2024-06-28T15:23:13.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.178+0000 7f859ee47980 20 remove_watcher() i=1 2024-06-28T15:23:13.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.182+0000 7f859ee47980 20 remove_watcher() i=2 2024-06-28T15:23:13.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.182+0000 7f859ee47980 20 remove_watcher() i=3 2024-06-28T15:23:13.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.186+0000 7f859ee47980 20 remove_watcher() i=4 2024-06-28T15:23:13.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.190+0000 7f859ee47980 20 remove_watcher() i=5 2024-06-28T15:23:13.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.190+0000 7f859ee47980 20 remove_watcher() i=6 2024-06-28T15:23:13.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:13.190+0000 7f859ee47980 20 remove_watcher() i=7 2024-06-28T15:23:13.188 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:13.188 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:13.188 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-set-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.20 num_objects: 19 size_kb: 1 num_shards: 2 2024-06-28T15:23:13.188 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:13.983 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: reshard bucket with ECANCELED injected at set_target_layout 2024-06-28T15:23:13.983 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:16.135 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-06-28T15:23:16.170 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:16.170 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:16.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.234+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.238+0000 7f7d80f55980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:16.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.238+0000 7f7d80f55980 20 realm 2024-06-28T15:23:16.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.238+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.238+0000 7f7d80f55980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:16.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.238+0000 7f7d80f55980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:16.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.238+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.238+0000 7f7d80f55980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:16.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.238+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.238+0000 7f7d80f55980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:16.224 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.238+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.242+0000 7f7d80f55980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:16.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.242+0000 7f7d80f55980 20 searching for the correct realm 2024-06-28T15:23:16.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.246+0000 7f7d7f267640 0 -- 172.21.15.178:0/541512682 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f7d6810e260 msgr2=0x7f7d681151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:16.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.462+0000 7f7d7ea66640 0 -- 172.21.15.178:0/541512682 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x5607d448c9d0 msgr2=0x5607d448edc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:16.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.470+0000 7f7d80f55980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:16.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.470+0000 7f7d80f55980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:16.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.470+0000 7f7d80f55980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:16.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.470+0000 7f7d80f55980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:16.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.470+0000 7f7d80f55980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:16.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.470+0000 7f7d80f55980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:16.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.470+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.470+0000 7f7d80f55980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:16.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.470+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.470+0000 7f7d80f55980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:16.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.470+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:16.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 20 zone default found 2024-06-28T15:23:16.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 4 Realm: () 2024-06-28T15:23:16.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:16.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:16.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:16.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 20 zonegroup default 2024-06-28T15:23:16.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:16.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:16.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:16.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 20 started sync module instance, tier type = 2024-06-28T15:23:16.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.474+0000 7f7d80f55980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:16.483 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.498+0000 7f7d80f55980 20 add_watcher() i=0 2024-06-28T15:23:16.483 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.498+0000 7f7d80f55980 20 add_watcher() i=1 2024-06-28T15:23:16.484 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.498+0000 7f7d80f55980 20 add_watcher() i=2 2024-06-28T15:23:16.484 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.498+0000 7f7d80f55980 20 add_watcher() i=3 2024-06-28T15:23:16.484 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.498+0000 7f7d80f55980 20 add_watcher() i=4 2024-06-28T15:23:16.484 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.498+0000 7f7d80f55980 20 add_watcher() i=5 2024-06-28T15:23:16.484 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.498+0000 7f7d80f55980 20 add_watcher() i=6 2024-06-28T15:23:16.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.506+0000 7f7d80f55980 20 add_watcher() i=7 2024-06-28T15:23:16.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.506+0000 7f7d80f55980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:16.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.510+0000 7f7c357e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:16.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.510+0000 7f7d80f55980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:16.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.510+0000 7f7d80f55980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:16.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.510+0000 7f7d80f55980 5 note: GC not initialized 2024-06-28T15:23:16.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.510+0000 7f7baffff640 20 reqs_thread_entry: start 2024-06-28T15:23:16.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.510+0000 7f7d80f55980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:16.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.514+0000 7f7d80f55980 20 Filter name: none 2024-06-28T15:23:16.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.514+0000 7f7badffb640 20 reqs_thread_entry: start 2024-06-28T15:23:16.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.514+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.514+0000 7f7d80f55980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:16.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.514+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.514+0000 7f7d80f55980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:16.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.514+0000 7f7d80f55980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:16.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.514+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.514+0000 7f7d80f55980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:16.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.514+0000 7f7d80f55980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.518+0000 7f7d80f55980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:16.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.518+0000 7f7d80f55980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:16.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.518+0000 7f7c357e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:16.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.526+0000 7f7d80f55980 20 remove_watcher() i=0 2024-06-28T15:23:16.508 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.526+0000 7f7d80f55980 2 removed watcher, disabling cache 2024-06-28T15:23:16.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.526+0000 7f7d80f55980 20 remove_watcher() i=1 2024-06-28T15:23:16.512 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.530+0000 7f7d80f55980 20 remove_watcher() i=2 2024-06-28T15:23:16.514 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.530+0000 7f7d80f55980 20 remove_watcher() i=3 2024-06-28T15:23:16.516 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.534+0000 7f7d80f55980 20 remove_watcher() i=4 2024-06-28T15:23:16.518 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.534+0000 7f7d80f55980 20 remove_watcher() i=5 2024-06-28T15:23:16.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.534+0000 7f7d80f55980 20 remove_watcher() i=6 2024-06-28T15:23:16.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.538+0000 7f7d80f55980 20 remove_watcher() i=7 2024-06-28T15:23:16.542 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:16.542 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:16.542 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-set-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.21 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:23:16.543 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:16.543 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout --inject-error-code 125 2024-06-28T15:23:16.582 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:16.582 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:16.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.646+0000 7fbf4e9f0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.646+0000 7fbf4e9f0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:16.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.646+0000 7fbf4e9f0980 20 realm 2024-06-28T15:23:16.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.646+0000 7fbf4e9f0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.646+0000 7fbf4e9f0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:16.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.646+0000 7fbf4e9f0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:16.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.646+0000 7fbf4e9f0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.646+0000 7fbf4e9f0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:16.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.646+0000 7fbf4e9f0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.650+0000 7fbf4e9f0980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:16.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.650+0000 7fbf4e9f0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:16.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.650+0000 7fbf4e9f0980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:16.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.650+0000 7fbf4e9f0980 20 searching for the correct realm 2024-06-28T15:23:16.636 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.654+0000 7fbf4bd00640 0 -- 172.21.15.178:0/2383619174 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fbf3c062040 msgr2=0x7fbf3c083240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:16.853 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:16.870+0000 7fbf4c501640 0 -- 172.21.15.178:0/2383619174 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fbf400980e0 msgr2=0x7fbf40022780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:17.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.086+0000 7fbf4e9f0980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:17.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.086+0000 7fbf4e9f0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:17.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.086+0000 7fbf4e9f0980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:17.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.086+0000 7fbf4e9f0980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:17.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.086+0000 7fbf4e9f0980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:17.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.086+0000 7fbf4e9f0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:17.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.086+0000 7fbf4e9f0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.086+0000 7fbf4e9f0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:17.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.086+0000 7fbf4e9f0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:17.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:17.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 20 zone default found 2024-06-28T15:23:17.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 4 Realm: () 2024-06-28T15:23:17.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:17.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:17.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:17.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 20 zonegroup default 2024-06-28T15:23:17.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:17.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:17.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:17.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 20 started sync module instance, tier type = 2024-06-28T15:23:17.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.090+0000 7fbf4e9f0980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:17.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.114+0000 7fbf4e9f0980 20 add_watcher() i=0 2024-06-28T15:23:17.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.114+0000 7fbf4e9f0980 20 add_watcher() i=1 2024-06-28T15:23:17.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.114+0000 7fbf4e9f0980 20 add_watcher() i=2 2024-06-28T15:23:17.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.114+0000 7fbf4e9f0980 20 add_watcher() i=3 2024-06-28T15:23:17.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.114+0000 7fbf4e9f0980 20 add_watcher() i=4 2024-06-28T15:23:17.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.114+0000 7fbf4e9f0980 20 add_watcher() i=5 2024-06-28T15:23:17.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.114+0000 7fbf4e9f0980 20 add_watcher() i=6 2024-06-28T15:23:17.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.114+0000 7fbf4e9f0980 20 add_watcher() i=7 2024-06-28T15:23:17.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.114+0000 7fbf4e9f0980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:17.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.118+0000 7fbe02fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:17.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:17.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:17.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 5 note: GC not initialized 2024-06-28T15:23:17.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbe01fdb640 20 reqs_thread_entry: start 2024-06-28T15:23:17.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:17.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 20 Filter name: none 2024-06-28T15:23:17.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbd737fe640 20 reqs_thread_entry: start 2024-06-28T15:23:17.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-06-28T15:23:17.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:17.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-06-28T15:23:17.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-06-28T15:23:17.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21 : miss 2024-06-28T15:23:17.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:17.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21 info.flags=0x17 2024-06-28T15:23:17.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21 to cache LRU end 2024-06-28T15:23:17.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:17.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:17.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.122+0000 7fbf4e9f0980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21 2024-06-28T15:23:17.112 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.130+0000 7fbf4e9f0980 -1 Injecting error=-125 at location=set_target_layout 2024-06-28T15:23:17.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.134+0000 7fbf4e9f0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21 info.flags=0x17 2024-06-28T15:23:17.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.134+0000 7fbf4e9f0980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21 to cache LRU end 2024-06-28T15:23:17.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.134+0000 7fbf4e9f0980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:17.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.134+0000 7fbf4e9f0980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21, ofs0, ns] 2024-06-28T15:23:17.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.134+0000 7fbe237fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595111 cookie 94009119204800 notifier 6390 bl.length()=1001 2024-06-28T15:23:17.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.134+0000 7fbe237fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21 info.flags=0x17 2024-06-28T15:23:17.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.134+0000 7fbe237fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21 to cache LRU end 2024-06-28T15:23:17.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.134+0000 7fbe237fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:17.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.138+0000 7fbf4e9f0980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.21]): orig_sources= new_sources= 2024-06-28T15:23:17.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.138+0000 7fbf4e9f0980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.21]): potential sources added=[] removed=[] 2024-06-28T15:23:17.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.138+0000 7fbf4e9f0980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.21]): orig_dests= new_dests= 2024-06-28T15:23:17.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.138+0000 7fbf4e9f0980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.21]): potential dests added=[] removed=[] 2024-06-28T15:23:17.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.138+0000 7fbf4e9f0980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.21.1.0 2024-06-28T15:23:17.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.138+0000 7fbf4e9f0980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.21.1.0 2024-06-28T15:23:17.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.138+0000 7fbf4e9f0980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.21.1.1 2024-06-28T15:23:17.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.138+0000 7fbf4e9f0980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.21.1.1 2024-06-28T15:23:17.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.138+0000 7fbf4e9f0980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.21.0 2024-06-28T15:23:17.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.146+0000 7fbf4e9f0980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21 : hit (requested=0x17, cached=0x17) 2024-06-28T15:23:17.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.146+0000 7fbf4e9f0980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21 2024-06-28T15:23:17.131 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.146+0000 7fbf4e9f0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21 info.flags=0x17 2024-06-28T15:23:17.131 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.146+0000 7fbf4e9f0980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21 to cache LRU end 2024-06-28T15:23:17.131 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.146+0000 7fbf4e9f0980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:17.131 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.146+0000 7fbf4e9f0980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21, ofs0, ns] 2024-06-28T15:23:17.132 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.150+0000 7fbe22ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595112 cookie 94009119204800 notifier 6390 bl.length()=969 2024-06-28T15:23:17.132 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.150+0000 7fbe22ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21 info.flags=0x17 2024-06-28T15:23:17.132 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.150+0000 7fbe22ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.21 to cache LRU end 2024-06-28T15:23:17.132 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.150+0000 7fbe22ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:17.134 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.150+0000 7fbf4e9f0980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.21]): orig_sources= new_sources= 2024-06-28T15:23:17.134 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.150+0000 7fbf4e9f0980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.21]): potential sources added=[] removed=[] 2024-06-28T15:23:17.134 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.150+0000 7fbf4e9f0980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.21]): orig_dests= new_dests= 2024-06-28T15:23:17.134 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.150+0000 7fbf4e9f0980 20 handle_bi_update(): bucket=:error-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.21]): potential dests added=[] removed=[] 2024-06-28T15:23:17.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.154+0000 7fbf4e9f0980 1 execute INFO: reshard of bucket "error-at-set-target-layout" from 2 shards to 2 shards completed successfully 2024-06-28T15:23:17.141 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.158+0000 7fbe02fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:17.146 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.162+0000 7fbf4e9f0980 20 remove_watcher() i=0 2024-06-28T15:23:17.146 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.162+0000 7fbf4e9f0980 2 removed watcher, disabling cache 2024-06-28T15:23:17.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.166+0000 7fbf4e9f0980 20 remove_watcher() i=1 2024-06-28T15:23:17.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.166+0000 7fbf4e9f0980 20 remove_watcher() i=2 2024-06-28T15:23:17.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.170+0000 7fbf4e9f0980 20 remove_watcher() i=3 2024-06-28T15:23:17.155 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.174+0000 7fbf4e9f0980 20 remove_watcher() i=4 2024-06-28T15:23:17.157 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.174+0000 7fbf4e9f0980 20 remove_watcher() i=5 2024-06-28T15:23:17.158 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.174+0000 7fbf4e9f0980 20 remove_watcher() i=6 2024-06-28T15:23:17.160 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.178+0000 7fbf4e9f0980 20 remove_watcher() i=7 2024-06-28T15:23:17.183 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-06-28T15:23:17.214 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:17.214 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:17.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.274+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.278+0000 7fb92586b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:17.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.278+0000 7fb92586b980 20 realm 2024-06-28T15:23:17.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.278+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.278+0000 7fb92586b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:17.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.278+0000 7fb92586b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:17.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.278+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.278+0000 7fb92586b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:17.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.278+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.278+0000 7fb92586b980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:17.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.278+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.282+0000 7fb92586b980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:17.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.282+0000 7fb92586b980 20 searching for the correct realm 2024-06-28T15:23:17.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.286+0000 7fb91bfff640 0 -- 172.21.15.178:0/3420391151 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fb91402a060 msgr2=0x7fb91405ba60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:17.275 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.290+0000 7fb91bfff640 0 -- 172.21.15.178:0/3420391151 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fb7ec07fdd0 msgr2=0x7fb7ec082200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:17.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.298+0000 7fb92586b980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:17.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.298+0000 7fb92586b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:17.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.298+0000 7fb92586b980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:17.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.298+0000 7fb92586b980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:17.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.298+0000 7fb92586b980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:17.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.298+0000 7fb92586b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:17.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.298+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.298+0000 7fb92586b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:17.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.298+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.298+0000 7fb92586b980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:17.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.298+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:17.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 20 zone default found 2024-06-28T15:23:17.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 4 Realm: () 2024-06-28T15:23:17.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:17.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:17.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:17.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 20 zonegroup default 2024-06-28T15:23:17.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:17.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:17.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:17.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 20 started sync module instance, tier type = 2024-06-28T15:23:17.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.302+0000 7fb92586b980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:17.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.318+0000 7fb92586b980 20 add_watcher() i=0 2024-06-28T15:23:17.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.318+0000 7fb92586b980 20 add_watcher() i=1 2024-06-28T15:23:17.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.318+0000 7fb92586b980 20 add_watcher() i=2 2024-06-28T15:23:17.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.318+0000 7fb92586b980 20 add_watcher() i=3 2024-06-28T15:23:17.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.318+0000 7fb92586b980 20 add_watcher() i=4 2024-06-28T15:23:17.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.318+0000 7fb92586b980 20 add_watcher() i=5 2024-06-28T15:23:17.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.318+0000 7fb92586b980 20 add_watcher() i=6 2024-06-28T15:23:17.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.318+0000 7fb92586b980 20 add_watcher() i=7 2024-06-28T15:23:17.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.318+0000 7fb92586b980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:17.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.322+0000 7fb7d9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:17.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.322+0000 7fb92586b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:17.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.322+0000 7fb92586b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:17.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.322+0000 7fb92586b980 5 note: GC not initialized 2024-06-28T15:23:17.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.322+0000 7fb7d8fe1640 20 reqs_thread_entry: start 2024-06-28T15:23:17.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.322+0000 7fb92586b980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:17.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.322+0000 7fb92586b980 20 Filter name: none 2024-06-28T15:23:17.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.322+0000 7fb7527fc640 20 reqs_thread_entry: start 2024-06-28T15:23:17.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.322+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.322+0000 7fb92586b980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:17.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.322+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.322+0000 7fb92586b980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:17.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.322+0000 7fb92586b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:17.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.322+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.326+0000 7fb92586b980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:17.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.326+0000 7fb92586b980 20 rados->read ofs=0 len=0 2024-06-28T15:23:17.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.326+0000 7fb92586b980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:17.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.326+0000 7fb92586b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:17.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.326+0000 7fb7d9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:17.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.330+0000 7fb92586b980 20 remove_watcher() i=0 2024-06-28T15:23:17.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.330+0000 7fb92586b980 2 removed watcher, disabling cache 2024-06-28T15:23:17.317 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.334+0000 7fb92586b980 20 remove_watcher() i=1 2024-06-28T15:23:17.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.334+0000 7fb92586b980 20 remove_watcher() i=2 2024-06-28T15:23:17.320 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.338+0000 7fb92586b980 20 remove_watcher() i=3 2024-06-28T15:23:17.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.338+0000 7fb92586b980 20 remove_watcher() i=4 2024-06-28T15:23:17.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.342+0000 7fb92586b980 20 remove_watcher() i=5 2024-06-28T15:23:17.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.342+0000 7fb92586b980 20 remove_watcher() i=6 2024-06-28T15:23:17.326 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:17.342+0000 7fb92586b980 20 remove_watcher() i=7 2024-06-28T15:23:17.340 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:17.340 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:17.340 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-set-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.21 num_objects: 20 size_kb: 1 num_shards: 2 2024-06-28T15:23:17.340 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:18.076 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: reshard bucket with abort at set_target_layout 2024-06-28T15:23:18.076 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.253 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-06-28T15:23:20.286 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:20.286 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:20.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.346+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.350+0000 7fdbac1a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.350+0000 7fdbac1a0980 20 realm 2024-06-28T15:23:20.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.350+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.350+0000 7fdbac1a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.350+0000 7fdbac1a0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:20.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.350+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.350+0000 7fdbac1a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.350+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.350+0000 7fdbac1a0980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:20.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.350+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.354+0000 7fdbac1a0980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:20.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.354+0000 7fdbac1a0980 20 searching for the correct realm 2024-06-28T15:23:20.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.358+0000 7fdbaa4b2640 0 -- 172.21.15.178:0/1893969111 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x55f3ccfcc860 msgr2=0x55f3ccfcec50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:20.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.366+0000 7fdba94b0640 0 -- 172.21.15.178:0/1893969111 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fdb9c091ad0 msgr2=0x7fdb9c062c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:20.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.370+0000 7fdbac1a0980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:20.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.370+0000 7fdbac1a0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:20.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.370+0000 7fdbac1a0980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:20.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.370+0000 7fdbac1a0980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:20.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.370+0000 7fdbac1a0980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:20.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.370+0000 7fdbac1a0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:20.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.370+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.370+0000 7fdbac1a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.370+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:20.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:20.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 20 zone default found 2024-06-28T15:23:20.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 4 Realm: () 2024-06-28T15:23:20.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:20.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:20.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:20.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 20 zonegroup default 2024-06-28T15:23:20.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 20 started sync module instance, tier type = 2024-06-28T15:23:20.360 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.374+0000 7fdbac1a0980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:20.373 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.390+0000 7fdbac1a0980 20 add_watcher() i=0 2024-06-28T15:23:20.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.390+0000 7fdbac1a0980 20 add_watcher() i=1 2024-06-28T15:23:20.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.390+0000 7fdbac1a0980 20 add_watcher() i=2 2024-06-28T15:23:20.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.390+0000 7fdbac1a0980 20 add_watcher() i=3 2024-06-28T15:23:20.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.390+0000 7fdbac1a0980 20 add_watcher() i=4 2024-06-28T15:23:20.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.390+0000 7fdbac1a0980 20 add_watcher() i=5 2024-06-28T15:23:20.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.390+0000 7fdbac1a0980 20 add_watcher() i=6 2024-06-28T15:23:20.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.390+0000 7fdbac1a0980 20 add_watcher() i=7 2024-06-28T15:23:20.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.390+0000 7fdbac1a0980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:20.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.394+0000 7fda5b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:20.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.394+0000 7fdbac1a0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:20.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.394+0000 7fdbac1a0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:20.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.394+0000 7fdbac1a0980 5 note: GC not initialized 2024-06-28T15:23:20.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.394+0000 7fda5a7fc640 20 reqs_thread_entry: start 2024-06-28T15:23:20.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.394+0000 7fdbac1a0980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:20.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.394+0000 7fdbac1a0980 20 Filter name: none 2024-06-28T15:23:20.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.394+0000 7fd9cbfff640 20 reqs_thread_entry: start 2024-06-28T15:23:20.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.394+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.394+0000 7fdbac1a0980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:20.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.394+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.398+0000 7fdbac1a0980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:20.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.398+0000 7fdbac1a0980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:20.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.398+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.398+0000 7fdbac1a0980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:20.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.398+0000 7fdbac1a0980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.398+0000 7fdbac1a0980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:20.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.398+0000 7fdbac1a0980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:20.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.402+0000 7fda5b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:20.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.406+0000 7fdbac1a0980 20 remove_watcher() i=0 2024-06-28T15:23:20.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.406+0000 7fdbac1a0980 2 removed watcher, disabling cache 2024-06-28T15:23:20.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.406+0000 7fdbac1a0980 20 remove_watcher() i=1 2024-06-28T15:23:20.393 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.410+0000 7fdbac1a0980 20 remove_watcher() i=2 2024-06-28T15:23:20.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.410+0000 7fdbac1a0980 20 remove_watcher() i=3 2024-06-28T15:23:20.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.414+0000 7fdbac1a0980 20 remove_watcher() i=4 2024-06-28T15:23:20.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.414+0000 7fdbac1a0980 20 remove_watcher() i=5 2024-06-28T15:23:20.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.418+0000 7fdbac1a0980 20 remove_watcher() i=6 2024-06-28T15:23:20.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.418+0000 7fdbac1a0980 20 remove_watcher() i=7 2024-06-28T15:23:20.415 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.415 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:20.415 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: abort-at-set-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.22 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:23:20.415 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.415 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at set_target_layout 2024-06-28T15:23:20.447 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:20.447 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:20.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.506+0000 7fa42206d980 20 realm 2024-06-28T15:23:20.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.506+0000 7fa42206d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:20.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:20.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:20.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.510+0000 7fa42206d980 20 searching for the correct realm 2024-06-28T15:23:20.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.518+0000 7fa42037f640 0 -- 172.21.15.178:0/1630651460 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fa4140343a0 msgr2=0x7fa414034800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:20.718 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.734+0000 7fa41fb7e640 0 -- 172.21.15.178:0/1630651460 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55d571965d20 msgr2=0x55d571968130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:20.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:20.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:20.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:20.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:20.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:20.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:20.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.742+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.742+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.742+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:20.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:20.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 20 zone default found 2024-06-28T15:23:20.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 4 Realm: () 2024-06-28T15:23:20.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:20.729 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:20.729 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:20.729 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 20 zonegroup default 2024-06-28T15:23:20.729 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.729 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.729 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 20 started sync module instance, tier type = 2024-06-28T15:23:20.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.746+0000 7fa42206d980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:20.750 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=0 2024-06-28T15:23:20.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=1 2024-06-28T15:23:20.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=2 2024-06-28T15:23:20.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=3 2024-06-28T15:23:20.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=4 2024-06-28T15:23:20.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=5 2024-06-28T15:23:20.751 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=6 2024-06-28T15:23:20.752 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.770+0000 7fa42206d980 20 add_watcher() i=7 2024-06-28T15:23:20.752 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.770+0000 7fa42206d980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:20.755 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa2d67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:20.756 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa42206d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:20.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa42206d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:20.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa42206d980 5 note: GC not initialized 2024-06-28T15:23:20.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa2d57e2640 20 reqs_thread_entry: start 2024-06-28T15:23:20.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa42206d980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:20.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa42206d980 20 Filter name: none 2024-06-28T15:23:20.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa2ce7fc640 20 reqs_thread_entry: start 2024-06-28T15:23:20.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa42206d980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-06-28T15:23:20.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:20.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa42206d980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-06-28T15:23:20.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa42206d980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-06-28T15:23:20.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa42206d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 : miss 2024-06-28T15:23:20.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.774+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.778+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:20.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.778+0000 7fa42206d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 info.flags=0x17 2024-06-28T15:23:20.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.778+0000 7fa42206d980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 to cache LRU end 2024-06-28T15:23:20.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.778+0000 7fa42206d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:20.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.778+0000 7fa42206d980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:20.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.778+0000 7fa42206d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 2024-06-28T15:23:20.765 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fa42206d980 time 2024-06-28T15:23:20.787106+0000 2024-06-28T15:23:20.765 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:23:20.767 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:23:20.767 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fa425944c91] 2024-06-28T15:23:20.767 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fa425944e45] 2024-06-28T15:23:20.767 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d5700124b1] 2024-06-28T15:23:20.767 INFO:tasks.workunit.client.0.smithi178.stderr: 4: radosgw-admin(+0x9acc0a) [0x55d57001ec0a] 2024-06-28T15:23:20.767 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x260) [0x55d5700239e0] 2024-06-28T15:23:20.767 INFO:tasks.workunit.client.0.smithi178.stderr: 6: main() 2024-06-28T15:23:20.767 INFO:tasks.workunit.client.0.smithi178.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa4247add90] 2024-06-28T15:23:20.767 INFO:tasks.workunit.client.0.smithi178.stderr: 8: __libc_start_main() 2024-06-28T15:23:20.767 INFO:tasks.workunit.client.0.smithi178.stderr: 9: _start() 2024-06-28T15:23:20.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.782+0000 7fa42206d980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fa42206d980 time 2024-06-28T15:23:20.787106+0000 2024-06-28T15:23:20.768 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:23:20.768 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.768 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:23:20.768 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fa425944c91] 2024-06-28T15:23:20.768 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fa425944e45] 2024-06-28T15:23:20.768 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d5700124b1] 2024-06-28T15:23:20.768 INFO:tasks.workunit.client.0.smithi178.stderr: 4: radosgw-admin(+0x9acc0a) [0x55d57001ec0a] 2024-06-28T15:23:20.768 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x260) [0x55d5700239e0] 2024-06-28T15:23:20.768 INFO:tasks.workunit.client.0.smithi178.stderr: 6: main() 2024-06-28T15:23:20.768 INFO:tasks.workunit.client.0.smithi178.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa4247add90] 2024-06-28T15:23:20.768 INFO:tasks.workunit.client.0.smithi178.stderr: 8: __libc_start_main() 2024-06-28T15:23:20.768 INFO:tasks.workunit.client.0.smithi178.stderr: 9: _start() 2024-06-28T15:23:20.768 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.768 INFO:tasks.workunit.client.0.smithi178.stderr:*** Caught signal (Aborted) ** 2024-06-28T15:23:20.769 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7fa42206d980 thread_name:radosgw-admin 2024-06-28T15:23:20.780 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:23:20.780 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fa4247c6520] 2024-06-28T15:23:20.780 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:23:20.780 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:23:20.780 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:23:20.780 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fa425944ceb] 2024-06-28T15:23:20.780 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fa425944e45] 2024-06-28T15:23:20.780 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d5700124b1] 2024-06-28T15:23:20.780 INFO:tasks.workunit.client.0.smithi178.stderr: 8: radosgw-admin(+0x9acc0a) [0x55d57001ec0a] 2024-06-28T15:23:20.780 INFO:tasks.workunit.client.0.smithi178.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x260) [0x55d5700239e0] 2024-06-28T15:23:20.780 INFO:tasks.workunit.client.0.smithi178.stderr: 10: main() 2024-06-28T15:23:20.781 INFO:tasks.workunit.client.0.smithi178.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa4247add90] 2024-06-28T15:23:20.781 INFO:tasks.workunit.client.0.smithi178.stderr: 12: __libc_start_main() 2024-06-28T15:23:20.781 INFO:tasks.workunit.client.0.smithi178.stderr: 13: _start() 2024-06-28T15:23:20.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.786+0000 7fa42206d980 -1 *** Caught signal (Aborted) ** 2024-06-28T15:23:20.781 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7fa42206d980 thread_name:radosgw-admin 2024-06-28T15:23:20.781 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.781 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:23:20.781 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fa4247c6520] 2024-06-28T15:23:20.781 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:23:20.781 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:23:20.781 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:23:20.781 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fa425944ceb] 2024-06-28T15:23:20.781 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fa425944e45] 2024-06-28T15:23:20.782 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d5700124b1] 2024-06-28T15:23:20.782 INFO:tasks.workunit.client.0.smithi178.stderr: 8: radosgw-admin(+0x9acc0a) [0x55d57001ec0a] 2024-06-28T15:23:20.782 INFO:tasks.workunit.client.0.smithi178.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x260) [0x55d5700239e0] 2024-06-28T15:23:20.782 INFO:tasks.workunit.client.0.smithi178.stderr: 10: main() 2024-06-28T15:23:20.782 INFO:tasks.workunit.client.0.smithi178.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa4247add90] 2024-06-28T15:23:20.782 INFO:tasks.workunit.client.0.smithi178.stderr: 12: __libc_start_main() 2024-06-28T15:23:20.782 INFO:tasks.workunit.client.0.smithi178.stderr: 13: _start() 2024-06-28T15:23:20.782 INFO:tasks.workunit.client.0.smithi178.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-28T15:23:20.782 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.782 INFO:tasks.workunit.client.0.smithi178.stderr:--- begin dump of recent events --- 2024-06-28T15:23:20.782 INFO:tasks.workunit.client.0.smithi178.stderr: -499> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.782 INFO:tasks.workunit.client.0.smithi178.stderr: -498> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.783 INFO:tasks.workunit.client.0.smithi178.stderr: -497> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.783 INFO:tasks.workunit.client.0.smithi178.stderr: -496> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): _renew_subs 2024-06-28T15:23:20.783 INFO:tasks.workunit.client.0.smithi178.stderr: -495> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): authenticate will time out at 2343.948354s 2024-06-28T15:23:20.783 INFO:tasks.workunit.client.0.smithi178.stderr: -494> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request con 0x55d571704940 auth_method 0 2024-06-28T15:23:20.783 INFO:tasks.workunit.client.0.smithi178.stderr: -493> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.783 INFO:tasks.workunit.client.0.smithi178.stderr: -492> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.783 INFO:tasks.workunit.client.0.smithi178.stderr: -491> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.783 INFO:tasks.workunit.client.0.smithi178.stderr: -490> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): get_auth_request con 0x55d57187ecf0 auth_method 0 2024-06-28T15:23:20.783 INFO:tasks.workunit.client.0.smithi178.stderr: -489> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.783 INFO:tasks.workunit.client.0.smithi178.stderr: -488> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.783 INFO:tasks.workunit.client.0.smithi178.stderr: -487> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.783 INFO:tasks.workunit.client.0.smithi178.stderr: -486> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.783 INFO:tasks.workunit.client.0.smithi178.stderr: -485> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.783 INFO:tasks.workunit.client.0.smithi178.stderr: -484> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.784 INFO:tasks.workunit.client.0.smithi178.stderr: -483> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.784 INFO:tasks.workunit.client.0.smithi178.stderr: -482> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.784 INFO:tasks.workunit.client.0.smithi178.stderr: -481> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.784 INFO:tasks.workunit.client.0.smithi178.stderr: -480> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_done global_id 6435 payload 274 2024-06-28T15:23:20.784 INFO:tasks.workunit.client.0.smithi178.stderr: -479> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient: _finish_hunting 0 2024-06-28T15:23:20.784 INFO:tasks.workunit.client.0.smithi178.stderr: -478> 2024-06-28T15:23:20.486+0000 7fa42037f640 1 monclient: found mon.noname-a 2024-06-28T15:23:20.784 INFO:tasks.workunit.client.0.smithi178.stderr: -477> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.784 INFO:tasks.workunit.client.0.smithi178.stderr: -476> 2024-06-28T15:23:20.486+0000 7fa41f37d640 10 monclient: get_auth_request con 0x55d571726ed0 auth_method 0 2024-06-28T15:23:20.784 INFO:tasks.workunit.client.0.smithi178.stderr: -475> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.784 INFO:tasks.workunit.client.0.smithi178.stderr: -474> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:23:20.784 INFO:tasks.workunit.client.0.smithi178.stderr: -473> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: dump: 2024-06-28T15:23:20.784 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.784 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.784 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.785 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.785 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.785 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.785 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.785 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.785 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.785 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.785 INFO:tasks.workunit.client.0.smithi178.stderr: -472> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: _finish_auth 0 2024-06-28T15:23:20.785 INFO:tasks.workunit.client.0.smithi178.stderr: -471> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: _check_auth_tickets 2024-06-28T15:23:20.785 INFO:tasks.workunit.client.0.smithi178.stderr: -470> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: handle_config config(0 keys) 2024-06-28T15:23:20.785 INFO:tasks.workunit.client.0.smithi178.stderr: -469> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.785 INFO:tasks.workunit.client.0.smithi178.stderr: -468> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:23:20.785 INFO:tasks.workunit.client.0.smithi178.stderr: -467> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: dump: 2024-06-28T15:23:20.785 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.786 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.786 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.786 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.786 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.786 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.786 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.786 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.786 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.786 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.786 INFO:tasks.workunit.client.0.smithi178.stderr: -466> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 monclient: authenticate success, global_id 6435 2024-06-28T15:23:20.786 INFO:tasks.workunit.client.0.smithi178.stderr: -465> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient: get_monmap_and_config success 2024-06-28T15:23:20.786 INFO:tasks.workunit.client.0.smithi178.stderr: -464> 2024-06-28T15:23:20.486+0000 7fa42206d980 4 set_mon_vals no callback set 2024-06-28T15:23:20.786 INFO:tasks.workunit.client.0.smithi178.stderr: -463> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient: shutdown 2024-06-28T15:23:20.787 INFO:tasks.workunit.client.0.smithi178.stderr: -462> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 asok(0x55d57166b5e0) unregister_commands rotate-key 2024-06-28T15:23:20.787 INFO:tasks.workunit.client.0.smithi178.stderr: -461> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient: build_initial_monmap 2024-06-28T15:23:20.787 INFO:tasks.workunit.client.0.smithi178.stderr: -460> 2024-06-28T15:23:20.486+0000 7fa42206d980 1 build_initial for_mkfs: 0 2024-06-28T15:23:20.787 INFO:tasks.workunit.client.0.smithi178.stderr: -459> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient: monmap: 2024-06-28T15:23:20.787 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:23:20.787 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:23:20.787 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:23:20.787 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:23:20.787 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:23:20.787 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.787 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:23:20.787 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:23:20.787 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:23:20.787 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.788 INFO:tasks.workunit.client.0.smithi178.stderr: -458> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding auth protocol: cephx 2024-06-28T15:23:20.788 INFO:tasks.workunit.client.0.smithi178.stderr: -457> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding auth protocol: cephx 2024-06-28T15:23:20.788 INFO:tasks.workunit.client.0.smithi178.stderr: -456> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding auth protocol: cephx 2024-06-28T15:23:20.788 INFO:tasks.workunit.client.0.smithi178.stderr: -455> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: secure 2024-06-28T15:23:20.788 INFO:tasks.workunit.client.0.smithi178.stderr: -454> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: crc 2024-06-28T15:23:20.788 INFO:tasks.workunit.client.0.smithi178.stderr: -453> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: secure 2024-06-28T15:23:20.788 INFO:tasks.workunit.client.0.smithi178.stderr: -452> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: crc 2024-06-28T15:23:20.788 INFO:tasks.workunit.client.0.smithi178.stderr: -451> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: secure 2024-06-28T15:23:20.788 INFO:tasks.workunit.client.0.smithi178.stderr: -450> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: crc 2024-06-28T15:23:20.788 INFO:tasks.workunit.client.0.smithi178.stderr: -449> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: crc 2024-06-28T15:23:20.788 INFO:tasks.workunit.client.0.smithi178.stderr: -448> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: secure 2024-06-28T15:23:20.788 INFO:tasks.workunit.client.0.smithi178.stderr: -447> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: crc 2024-06-28T15:23:20.788 INFO:tasks.workunit.client.0.smithi178.stderr: -446> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: secure 2024-06-28T15:23:20.788 INFO:tasks.workunit.client.0.smithi178.stderr: -445> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: crc 2024-06-28T15:23:20.789 INFO:tasks.workunit.client.0.smithi178.stderr: -444> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: secure 2024-06-28T15:23:20.789 INFO:tasks.workunit.client.0.smithi178.stderr: -443> 2024-06-28T15:23:20.486+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.789 INFO:tasks.workunit.client.0.smithi178.stderr: -442> 2024-06-28T15:23:20.486+0000 7fa42206d980 1 librados: starting msgr at 2024-06-28T15:23:20.789 INFO:tasks.workunit.client.0.smithi178.stderr: -441> 2024-06-28T15:23:20.486+0000 7fa42206d980 1 librados: starting objecter 2024-06-28T15:23:20.789 INFO:tasks.workunit.client.0.smithi178.stderr: -440> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command objecter_requests hook 0x55d571694f30 2024-06-28T15:23:20.789 INFO:tasks.workunit.client.0.smithi178.stderr: -439> 2024-06-28T15:23:20.486+0000 7fa42206d980 1 librados: setting wanted keys 2024-06-28T15:23:20.789 INFO:tasks.workunit.client.0.smithi178.stderr: -438> 2024-06-28T15:23:20.486+0000 7fa42206d980 1 librados: calling monclient init 2024-06-28T15:23:20.789 INFO:tasks.workunit.client.0.smithi178.stderr: -437> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient: init 2024-06-28T15:23:20.789 INFO:tasks.workunit.client.0.smithi178.stderr: -436> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding auth protocol: cephx 2024-06-28T15:23:20.789 INFO:tasks.workunit.client.0.smithi178.stderr: -435> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding auth protocol: cephx 2024-06-28T15:23:20.789 INFO:tasks.workunit.client.0.smithi178.stderr: -434> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding auth protocol: cephx 2024-06-28T15:23:20.789 INFO:tasks.workunit.client.0.smithi178.stderr: -433> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: secure 2024-06-28T15:23:20.789 INFO:tasks.workunit.client.0.smithi178.stderr: -432> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: crc 2024-06-28T15:23:20.790 INFO:tasks.workunit.client.0.smithi178.stderr: -431> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: secure 2024-06-28T15:23:20.790 INFO:tasks.workunit.client.0.smithi178.stderr: -430> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: crc 2024-06-28T15:23:20.790 INFO:tasks.workunit.client.0.smithi178.stderr: -429> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: secure 2024-06-28T15:23:20.790 INFO:tasks.workunit.client.0.smithi178.stderr: -428> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: crc 2024-06-28T15:23:20.790 INFO:tasks.workunit.client.0.smithi178.stderr: -427> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: crc 2024-06-28T15:23:20.790 INFO:tasks.workunit.client.0.smithi178.stderr: -426> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: secure 2024-06-28T15:23:20.790 INFO:tasks.workunit.client.0.smithi178.stderr: -425> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: crc 2024-06-28T15:23:20.790 INFO:tasks.workunit.client.0.smithi178.stderr: -424> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: secure 2024-06-28T15:23:20.790 INFO:tasks.workunit.client.0.smithi178.stderr: -423> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: crc 2024-06-28T15:23:20.790 INFO:tasks.workunit.client.0.smithi178.stderr: -422> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: secure 2024-06-28T15:23:20.790 INFO:tasks.workunit.client.0.smithi178.stderr: -421> 2024-06-28T15:23:20.486+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.790 INFO:tasks.workunit.client.0.smithi178.stderr: -420> 2024-06-28T15:23:20.486+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.790 INFO:tasks.workunit.client.0.smithi178.stderr: -419> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command rotate-key hook 0x55d571878bc8 2024-06-28T15:23:20.790 INFO:tasks.workunit.client.0.smithi178.stderr: -418> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient: _reopen_session rank -1 2024-06-28T15:23:20.791 INFO:tasks.workunit.client.0.smithi178.stderr: -417> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient: _add_conns ranks=[2,1,0] 2024-06-28T15:23:20.791 INFO:tasks.workunit.client.0.smithi178.stderr: -416> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-c con 0x55d571704940 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:23:20.791 INFO:tasks.workunit.client.0.smithi178.stderr: -415> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-b con 0x55d571726ed0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:23:20.791 INFO:tasks.workunit.client.0.smithi178.stderr: -414> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-a con 0x55d57187ecf0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:23:20.791 INFO:tasks.workunit.client.0.smithi178.stderr: -413> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.791 INFO:tasks.workunit.client.0.smithi178.stderr: -412> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.791 INFO:tasks.workunit.client.0.smithi178.stderr: -411> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.791 INFO:tasks.workunit.client.0.smithi178.stderr: -410> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): _renew_subs 2024-06-28T15:23:20.791 INFO:tasks.workunit.client.0.smithi178.stderr: -409> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): authenticate will time out at 2343.951467s 2024-06-28T15:23:20.791 INFO:tasks.workunit.client.0.smithi178.stderr: -408> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request con 0x55d571704940 auth_method 0 2024-06-28T15:23:20.791 INFO:tasks.workunit.client.0.smithi178.stderr: -407> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.791 INFO:tasks.workunit.client.0.smithi178.stderr: -406> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.791 INFO:tasks.workunit.client.0.smithi178.stderr: -405> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.792 INFO:tasks.workunit.client.0.smithi178.stderr: -404> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): get_auth_request con 0x55d57187ecf0 auth_method 0 2024-06-28T15:23:20.792 INFO:tasks.workunit.client.0.smithi178.stderr: -403> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.792 INFO:tasks.workunit.client.0.smithi178.stderr: -402> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.792 INFO:tasks.workunit.client.0.smithi178.stderr: -401> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.792 INFO:tasks.workunit.client.0.smithi178.stderr: -400> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.792 INFO:tasks.workunit.client.0.smithi178.stderr: -399> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.792 INFO:tasks.workunit.client.0.smithi178.stderr: -398> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.792 INFO:tasks.workunit.client.0.smithi178.stderr: -397> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.792 INFO:tasks.workunit.client.0.smithi178.stderr: -396> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.792 INFO:tasks.workunit.client.0.smithi178.stderr: -395> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.792 INFO:tasks.workunit.client.0.smithi178.stderr: -394> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_done global_id 6438 payload 824 2024-06-28T15:23:20.792 INFO:tasks.workunit.client.0.smithi178.stderr: -393> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient: _finish_hunting 0 2024-06-28T15:23:20.792 INFO:tasks.workunit.client.0.smithi178.stderr: -392> 2024-06-28T15:23:20.486+0000 7fa42037f640 1 monclient: found mon.noname-a 2024-06-28T15:23:20.792 INFO:tasks.workunit.client.0.smithi178.stderr: -391> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.793 INFO:tasks.workunit.client.0.smithi178.stderr: -390> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.793 INFO:tasks.workunit.client.0.smithi178.stderr: -389> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:23:20.793 INFO:tasks.workunit.client.0.smithi178.stderr: -388> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: dump: 2024-06-28T15:23:20.793 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.793 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.793 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.793 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.793 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.793 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.793 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.793 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.793 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.793 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.793 INFO:tasks.workunit.client.0.smithi178.stderr: -387> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: _finish_auth 0 2024-06-28T15:23:20.794 INFO:tasks.workunit.client.0.smithi178.stderr: -386> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: _check_auth_tickets 2024-06-28T15:23:20.794 INFO:tasks.workunit.client.0.smithi178.stderr: -385> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: handle_config config(0 keys) 2024-06-28T15:23:20.794 INFO:tasks.workunit.client.0.smithi178.stderr: -384> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.794 INFO:tasks.workunit.client.0.smithi178.stderr: -383> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:23:20.794 INFO:tasks.workunit.client.0.smithi178.stderr: -382> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: dump: 2024-06-28T15:23:20.794 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.794 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.794 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.794 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.794 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.794 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.794 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.794 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.794 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.795 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.795 INFO:tasks.workunit.client.0.smithi178.stderr: -381> 2024-06-28T15:23:20.490+0000 7fa42206d980 5 monclient: authenticate success, global_id 6438 2024-06-28T15:23:20.795 INFO:tasks.workunit.client.0.smithi178.stderr: -380> 2024-06-28T15:23:20.490+0000 7fa30de0b640 4 set_mon_vals no callback set 2024-06-28T15:23:20.795 INFO:tasks.workunit.client.0.smithi178.stderr: -379> 2024-06-28T15:23:20.490+0000 7fa42206d980 10 monclient: _renew_subs 2024-06-28T15:23:20.795 INFO:tasks.workunit.client.0.smithi178.stderr: -378> 2024-06-28T15:23:20.490+0000 7fa42206d980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.795 INFO:tasks.workunit.client.0.smithi178.stderr: -377> 2024-06-28T15:23:20.490+0000 7fa42206d980 10 monclient: _renew_subs 2024-06-28T15:23:20.795 INFO:tasks.workunit.client.0.smithi178.stderr: -376> 2024-06-28T15:23:20.490+0000 7fa42206d980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.795 INFO:tasks.workunit.client.0.smithi178.stderr: -375> 2024-06-28T15:23:20.490+0000 7fa42206d980 1 librados: init done 2024-06-28T15:23:20.795 INFO:tasks.workunit.client.0.smithi178.stderr: -374> 2024-06-28T15:23:20.490+0000 7fa307fff640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:23:20.795 INFO:tasks.workunit.client.0.smithi178.stderr: -373> 2024-06-28T15:23:20.490+0000 7fa307fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:23:20.795 INFO:tasks.workunit.client.0.smithi178.stderr: -372> 2024-06-28T15:23:20.490+0000 7fa307fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:23:20.795 INFO:tasks.workunit.client.0.smithi178.stderr: -371> 2024-06-28T15:23:20.490+0000 7fa41fb7e640 10 monclient: get_auth_request con 0x7fa30806f780 auth_method 0 2024-06-28T15:23:20.795 INFO:tasks.workunit.client.0.smithi178.stderr: -370> 2024-06-28T15:23:20.490+0000 7fa41f37d640 10 monclient: get_auth_request con 0x55d5718a1490 auth_method 0 2024-06-28T15:23:20.796 INFO:tasks.workunit.client.0.smithi178.stderr: -369> 2024-06-28T15:23:20.490+0000 7fa41fb7e640 10 monclient: get_auth_request con 0x55d5718c35d0 auth_method 0 2024-06-28T15:23:20.796 INFO:tasks.workunit.client.0.smithi178.stderr: -368> 2024-06-28T15:23:20.494+0000 7fa41f37d640 10 monclient: get_auth_request con 0x55d5718c77c0 auth_method 0 2024-06-28T15:23:20.796 INFO:tasks.workunit.client.0.smithi178.stderr: -367> 2024-06-28T15:23:20.494+0000 7fa42206d980 10 monclient: build_initial_monmap 2024-06-28T15:23:20.796 INFO:tasks.workunit.client.0.smithi178.stderr: -366> 2024-06-28T15:23:20.494+0000 7fa42206d980 1 build_initial for_mkfs: 0 2024-06-28T15:23:20.796 INFO:tasks.workunit.client.0.smithi178.stderr: -365> 2024-06-28T15:23:20.494+0000 7fa42206d980 10 monclient: monmap: 2024-06-28T15:23:20.796 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:23:20.796 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:23:20.796 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:23:20.796 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:23:20.796 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:23:20.796 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.796 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:23:20.796 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:23:20.796 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:23:20.797 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.797 INFO:tasks.workunit.client.0.smithi178.stderr: -364> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding auth protocol: cephx 2024-06-28T15:23:20.797 INFO:tasks.workunit.client.0.smithi178.stderr: -363> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding auth protocol: cephx 2024-06-28T15:23:20.797 INFO:tasks.workunit.client.0.smithi178.stderr: -362> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding auth protocol: cephx 2024-06-28T15:23:20.797 INFO:tasks.workunit.client.0.smithi178.stderr: -361> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: secure 2024-06-28T15:23:20.797 INFO:tasks.workunit.client.0.smithi178.stderr: -360> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: crc 2024-06-28T15:23:20.797 INFO:tasks.workunit.client.0.smithi178.stderr: -359> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: secure 2024-06-28T15:23:20.797 INFO:tasks.workunit.client.0.smithi178.stderr: -358> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: crc 2024-06-28T15:23:20.797 INFO:tasks.workunit.client.0.smithi178.stderr: -357> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: secure 2024-06-28T15:23:20.797 INFO:tasks.workunit.client.0.smithi178.stderr: -356> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: crc 2024-06-28T15:23:20.797 INFO:tasks.workunit.client.0.smithi178.stderr: -355> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: crc 2024-06-28T15:23:20.797 INFO:tasks.workunit.client.0.smithi178.stderr: -354> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: secure 2024-06-28T15:23:20.797 INFO:tasks.workunit.client.0.smithi178.stderr: -353> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: crc 2024-06-28T15:23:20.797 INFO:tasks.workunit.client.0.smithi178.stderr: -352> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: secure 2024-06-28T15:23:20.798 INFO:tasks.workunit.client.0.smithi178.stderr: -351> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: crc 2024-06-28T15:23:20.798 INFO:tasks.workunit.client.0.smithi178.stderr: -350> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: secure 2024-06-28T15:23:20.798 INFO:tasks.workunit.client.0.smithi178.stderr: -349> 2024-06-28T15:23:20.494+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.798 INFO:tasks.workunit.client.0.smithi178.stderr: -348> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command objecter_requests cmddesc objecter_requests hook 0x55d5718cfa10 EEXIST 2024-06-28T15:23:20.798 INFO:tasks.workunit.client.0.smithi178.stderr: -347> 2024-06-28T15:23:20.494+0000 7fa42206d980 10 monclient: init 2024-06-28T15:23:20.798 INFO:tasks.workunit.client.0.smithi178.stderr: -346> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding auth protocol: cephx 2024-06-28T15:23:20.798 INFO:tasks.workunit.client.0.smithi178.stderr: -345> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding auth protocol: cephx 2024-06-28T15:23:20.798 INFO:tasks.workunit.client.0.smithi178.stderr: -344> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding auth protocol: cephx 2024-06-28T15:23:20.798 INFO:tasks.workunit.client.0.smithi178.stderr: -343> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: secure 2024-06-28T15:23:20.798 INFO:tasks.workunit.client.0.smithi178.stderr: -342> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: crc 2024-06-28T15:23:20.798 INFO:tasks.workunit.client.0.smithi178.stderr: -341> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: secure 2024-06-28T15:23:20.798 INFO:tasks.workunit.client.0.smithi178.stderr: -340> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: crc 2024-06-28T15:23:20.798 INFO:tasks.workunit.client.0.smithi178.stderr: -339> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: secure 2024-06-28T15:23:20.798 INFO:tasks.workunit.client.0.smithi178.stderr: -338> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: crc 2024-06-28T15:23:20.799 INFO:tasks.workunit.client.0.smithi178.stderr: -337> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: crc 2024-06-28T15:23:20.799 INFO:tasks.workunit.client.0.smithi178.stderr: -336> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: secure 2024-06-28T15:23:20.799 INFO:tasks.workunit.client.0.smithi178.stderr: -335> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: crc 2024-06-28T15:23:20.799 INFO:tasks.workunit.client.0.smithi178.stderr: -334> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: secure 2024-06-28T15:23:20.799 INFO:tasks.workunit.client.0.smithi178.stderr: -333> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: crc 2024-06-28T15:23:20.799 INFO:tasks.workunit.client.0.smithi178.stderr: -332> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: secure 2024-06-28T15:23:20.799 INFO:tasks.workunit.client.0.smithi178.stderr: -331> 2024-06-28T15:23:20.498+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.799 INFO:tasks.workunit.client.0.smithi178.stderr: -330> 2024-06-28T15:23:20.498+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.799 INFO:tasks.workunit.client.0.smithi178.stderr: -329> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command rotate-key cmddesc rotate-key hook 0x55d5718cabd0 EEXIST 2024-06-28T15:23:20.799 INFO:tasks.workunit.client.0.smithi178.stderr: -328> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: _reopen_session rank -1 2024-06-28T15:23:20.799 INFO:tasks.workunit.client.0.smithi178.stderr: -327> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: _add_conns ranks=[2,0,1] 2024-06-28T15:23:20.799 INFO:tasks.workunit.client.0.smithi178.stderr: -326> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-c con 0x55d5718d5fa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:23:20.799 INFO:tasks.workunit.client.0.smithi178.stderr: -325> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-a con 0x55d5718d68d0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:23:20.800 INFO:tasks.workunit.client.0.smithi178.stderr: -324> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-b con 0x55d5718d6380 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:23:20.800 INFO:tasks.workunit.client.0.smithi178.stderr: -323> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.800 INFO:tasks.workunit.client.0.smithi178.stderr: -322> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.800 INFO:tasks.workunit.client.0.smithi178.stderr: -321> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.800 INFO:tasks.workunit.client.0.smithi178.stderr: -320> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): _renew_subs 2024-06-28T15:23:20.800 INFO:tasks.workunit.client.0.smithi178.stderr: -319> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): authenticate will time out at 2343.960501s 2024-06-28T15:23:20.800 INFO:tasks.workunit.client.0.smithi178.stderr: -318> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient(hunting): get_auth_request con 0x55d5718d68d0 auth_method 0 2024-06-28T15:23:20.800 INFO:tasks.workunit.client.0.smithi178.stderr: -317> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.800 INFO:tasks.workunit.client.0.smithi178.stderr: -316> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.800 INFO:tasks.workunit.client.0.smithi178.stderr: -315> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.800 INFO:tasks.workunit.client.0.smithi178.stderr: -314> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request con 0x55d5718d5fa0 auth_method 0 2024-06-28T15:23:20.801 INFO:tasks.workunit.client.0.smithi178.stderr: -313> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.801 INFO:tasks.workunit.client.0.smithi178.stderr: -312> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.801 INFO:tasks.workunit.client.0.smithi178.stderr: -311> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.801 INFO:tasks.workunit.client.0.smithi178.stderr: -310> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.801 INFO:tasks.workunit.client.0.smithi178.stderr: -309> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.801 INFO:tasks.workunit.client.0.smithi178.stderr: -308> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.801 INFO:tasks.workunit.client.0.smithi178.stderr: -307> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.801 INFO:tasks.workunit.client.0.smithi178.stderr: -306> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.801 INFO:tasks.workunit.client.0.smithi178.stderr: -305> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.801 INFO:tasks.workunit.client.0.smithi178.stderr: -304> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_done global_id 6068 payload 824 2024-06-28T15:23:20.801 INFO:tasks.workunit.client.0.smithi178.stderr: -303> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient: _finish_hunting 0 2024-06-28T15:23:20.801 INFO:tasks.workunit.client.0.smithi178.stderr: -302> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 1 monclient: found mon.noname-c 2024-06-28T15:23:20.801 INFO:tasks.workunit.client.0.smithi178.stderr: -301> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:23:20.801 INFO:tasks.workunit.client.0.smithi178.stderr: -300> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.802 INFO:tasks.workunit.client.0.smithi178.stderr: -299> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:23:20.802 INFO:tasks.workunit.client.0.smithi178.stderr: -298> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: dump: 2024-06-28T15:23:20.802 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.802 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.802 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.802 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.802 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.802 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.802 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.802 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.802 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.802 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.802 INFO:tasks.workunit.client.0.smithi178.stderr: -297> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: _finish_auth 0 2024-06-28T15:23:20.802 INFO:tasks.workunit.client.0.smithi178.stderr: -296> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: _check_auth_tickets 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr: -295> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 monclient: authenticate success, global_id 6068 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr: -294> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: _renew_subs 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr: -293> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr: -292> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: handle_config config(0 keys) 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr: -291> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr: -290> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr: -289> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: dump: 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.803 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.804 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.804 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.804 INFO:tasks.workunit.client.0.smithi178.stderr: -288> 2024-06-28T15:23:20.498+0000 7fa41eb7c640 4 set_mon_vals no callback set 2024-06-28T15:23:20.804 INFO:tasks.workunit.client.0.smithi178.stderr: -287> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: _renew_subs 2024-06-28T15:23:20.804 INFO:tasks.workunit.client.0.smithi178.stderr: -286> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:23:20.804 INFO:tasks.workunit.client.0.smithi178.stderr: -285> 2024-06-28T15:23:20.498+0000 7fa3057fa640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:23:20.804 INFO:tasks.workunit.client.0.smithi178.stderr: -284> 2024-06-28T15:23:20.498+0000 7fa3057fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:23:20.804 INFO:tasks.workunit.client.0.smithi178.stderr: -283> 2024-06-28T15:23:20.498+0000 7fa3057fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:23:20.804 INFO:tasks.workunit.client.0.smithi178.stderr: -282> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient: get_auth_request con 0x7fa2f806f690 auth_method 0 2024-06-28T15:23:20.804 INFO:tasks.workunit.client.0.smithi178.stderr: -281> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: get_monmap_and_config 2024-06-28T15:23:20.804 INFO:tasks.workunit.client.0.smithi178.stderr: -280> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: build_initial_monmap 2024-06-28T15:23:20.804 INFO:tasks.workunit.client.0.smithi178.stderr: -279> 2024-06-28T15:23:20.498+0000 7fa42206d980 1 build_initial for_mkfs: 0 2024-06-28T15:23:20.804 INFO:tasks.workunit.client.0.smithi178.stderr: -278> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: monmap: 2024-06-28T15:23:20.804 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:23:20.805 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:23:20.805 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:23:20.805 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:23:20.805 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:23:20.805 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.805 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:23:20.805 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:23:20.805 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:23:20.805 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.805 INFO:tasks.workunit.client.0.smithi178.stderr: -277> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding auth protocol: cephx 2024-06-28T15:23:20.805 INFO:tasks.workunit.client.0.smithi178.stderr: -276> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding auth protocol: cephx 2024-06-28T15:23:20.805 INFO:tasks.workunit.client.0.smithi178.stderr: -275> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding auth protocol: cephx 2024-06-28T15:23:20.805 INFO:tasks.workunit.client.0.smithi178.stderr: -274> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: secure 2024-06-28T15:23:20.805 INFO:tasks.workunit.client.0.smithi178.stderr: -273> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: crc 2024-06-28T15:23:20.806 INFO:tasks.workunit.client.0.smithi178.stderr: -272> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: secure 2024-06-28T15:23:20.806 INFO:tasks.workunit.client.0.smithi178.stderr: -271> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: crc 2024-06-28T15:23:20.806 INFO:tasks.workunit.client.0.smithi178.stderr: -270> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: secure 2024-06-28T15:23:20.806 INFO:tasks.workunit.client.0.smithi178.stderr: -269> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: crc 2024-06-28T15:23:20.806 INFO:tasks.workunit.client.0.smithi178.stderr: -268> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: crc 2024-06-28T15:23:20.806 INFO:tasks.workunit.client.0.smithi178.stderr: -267> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: secure 2024-06-28T15:23:20.806 INFO:tasks.workunit.client.0.smithi178.stderr: -266> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: crc 2024-06-28T15:23:20.806 INFO:tasks.workunit.client.0.smithi178.stderr: -265> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: secure 2024-06-28T15:23:20.806 INFO:tasks.workunit.client.0.smithi178.stderr: -264> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: crc 2024-06-28T15:23:20.806 INFO:tasks.workunit.client.0.smithi178.stderr: -263> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: secure 2024-06-28T15:23:20.806 INFO:tasks.workunit.client.0.smithi178.stderr: -262> 2024-06-28T15:23:20.502+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.806 INFO:tasks.workunit.client.0.smithi178.stderr: -261> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: init 2024-06-28T15:23:20.806 INFO:tasks.workunit.client.0.smithi178.stderr: -260> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding auth protocol: cephx 2024-06-28T15:23:20.806 INFO:tasks.workunit.client.0.smithi178.stderr: -259> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding auth protocol: cephx 2024-06-28T15:23:20.807 INFO:tasks.workunit.client.0.smithi178.stderr: -258> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding auth protocol: cephx 2024-06-28T15:23:20.807 INFO:tasks.workunit.client.0.smithi178.stderr: -257> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: secure 2024-06-28T15:23:20.807 INFO:tasks.workunit.client.0.smithi178.stderr: -256> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: crc 2024-06-28T15:23:20.807 INFO:tasks.workunit.client.0.smithi178.stderr: -255> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: secure 2024-06-28T15:23:20.807 INFO:tasks.workunit.client.0.smithi178.stderr: -254> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: crc 2024-06-28T15:23:20.807 INFO:tasks.workunit.client.0.smithi178.stderr: -253> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: secure 2024-06-28T15:23:20.807 INFO:tasks.workunit.client.0.smithi178.stderr: -252> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: crc 2024-06-28T15:23:20.807 INFO:tasks.workunit.client.0.smithi178.stderr: -251> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: crc 2024-06-28T15:23:20.807 INFO:tasks.workunit.client.0.smithi178.stderr: -250> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: secure 2024-06-28T15:23:20.807 INFO:tasks.workunit.client.0.smithi178.stderr: -249> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: crc 2024-06-28T15:23:20.807 INFO:tasks.workunit.client.0.smithi178.stderr: -248> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: secure 2024-06-28T15:23:20.807 INFO:tasks.workunit.client.0.smithi178.stderr: -247> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: crc 2024-06-28T15:23:20.807 INFO:tasks.workunit.client.0.smithi178.stderr: -246> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: secure 2024-06-28T15:23:20.807 INFO:tasks.workunit.client.0.smithi178.stderr: -245> 2024-06-28T15:23:20.502+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.808 INFO:tasks.workunit.client.0.smithi178.stderr: -244> 2024-06-28T15:23:20.502+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.808 INFO:tasks.workunit.client.0.smithi178.stderr: -243> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command rotate-key cmddesc rotate-key hook 0x7ffcc88df108 EEXIST 2024-06-28T15:23:20.808 INFO:tasks.workunit.client.0.smithi178.stderr: -242> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: _reopen_session rank -1 2024-06-28T15:23:20.808 INFO:tasks.workunit.client.0.smithi178.stderr: -241> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: _add_conns ranks=[0,2,1] 2024-06-28T15:23:20.808 INFO:tasks.workunit.client.0.smithi178.stderr: -240> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-a con 0x55d5719439a0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:23:20.808 INFO:tasks.workunit.client.0.smithi178.stderr: -239> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-c con 0x55d571943d80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:23:20.808 INFO:tasks.workunit.client.0.smithi178.stderr: -238> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-b con 0x55d571919160 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:23:20.808 INFO:tasks.workunit.client.0.smithi178.stderr: -237> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.808 INFO:tasks.workunit.client.0.smithi178.stderr: -236> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.808 INFO:tasks.workunit.client.0.smithi178.stderr: -235> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.808 INFO:tasks.workunit.client.0.smithi178.stderr: -234> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): _renew_subs 2024-06-28T15:23:20.808 INFO:tasks.workunit.client.0.smithi178.stderr: -233> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): authenticate will time out at 2343.964937s 2024-06-28T15:23:20.808 INFO:tasks.workunit.client.0.smithi178.stderr: -232> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request con 0x55d5719439a0 auth_method 0 2024-06-28T15:23:20.808 INFO:tasks.workunit.client.0.smithi178.stderr: -231> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -230> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -229> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -228> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): get_auth_request con 0x55d571943d80 auth_method 0 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -227> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -226> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -225> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -224> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -223> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -222> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -221> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -220> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -219> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -218> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_done global_id 6444 payload 274 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -217> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient: _finish_hunting 0 2024-06-28T15:23:20.809 INFO:tasks.workunit.client.0.smithi178.stderr: -216> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 1 monclient: found mon.noname-a 2024-06-28T15:23:20.810 INFO:tasks.workunit.client.0.smithi178.stderr: -215> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.810 INFO:tasks.workunit.client.0.smithi178.stderr: -214> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.810 INFO:tasks.workunit.client.0.smithi178.stderr: -213> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:23:20.810 INFO:tasks.workunit.client.0.smithi178.stderr: -212> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: dump: 2024-06-28T15:23:20.810 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.810 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.810 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.810 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.810 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.810 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.810 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.810 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.810 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.810 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.811 INFO:tasks.workunit.client.0.smithi178.stderr: -211> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: _finish_auth 0 2024-06-28T15:23:20.811 INFO:tasks.workunit.client.0.smithi178.stderr: -210> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: _check_auth_tickets 2024-06-28T15:23:20.811 INFO:tasks.workunit.client.0.smithi178.stderr: -209> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: handle_config config(0 keys) 2024-06-28T15:23:20.811 INFO:tasks.workunit.client.0.smithi178.stderr: -208> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 monclient: authenticate success, global_id 6444 2024-06-28T15:23:20.811 INFO:tasks.workunit.client.0.smithi178.stderr: -207> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: get_monmap_and_config success 2024-06-28T15:23:20.811 INFO:tasks.workunit.client.0.smithi178.stderr: -206> 2024-06-28T15:23:20.502+0000 7fa42206d980 4 set_mon_vals no callback set 2024-06-28T15:23:20.811 INFO:tasks.workunit.client.0.smithi178.stderr: -205> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.811 INFO:tasks.workunit.client.0.smithi178.stderr: -204> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:23:20.811 INFO:tasks.workunit.client.0.smithi178.stderr: -203> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: dump: 2024-06-28T15:23:20.811 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.811 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.811 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.811 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.811 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.812 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.812 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.812 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.812 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.812 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.812 INFO:tasks.workunit.client.0.smithi178.stderr: -202> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: shutdown 2024-06-28T15:23:20.812 INFO:tasks.workunit.client.0.smithi178.stderr: -201> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: build_initial_monmap 2024-06-28T15:23:20.812 INFO:tasks.workunit.client.0.smithi178.stderr: -200> 2024-06-28T15:23:20.502+0000 7fa42206d980 1 build_initial for_mkfs: 0 2024-06-28T15:23:20.812 INFO:tasks.workunit.client.0.smithi178.stderr: -199> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: monmap: 2024-06-28T15:23:20.812 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:23:20.812 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:23:20.812 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:23:20.812 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:23:20.812 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr: -198> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding auth protocol: cephx 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr: -197> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding auth protocol: cephx 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr: -196> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding auth protocol: cephx 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr: -195> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: secure 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr: -194> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: crc 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr: -193> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: secure 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr: -192> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: crc 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr: -191> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: secure 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr: -190> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: crc 2024-06-28T15:23:20.813 INFO:tasks.workunit.client.0.smithi178.stderr: -189> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: crc 2024-06-28T15:23:20.814 INFO:tasks.workunit.client.0.smithi178.stderr: -188> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: secure 2024-06-28T15:23:20.814 INFO:tasks.workunit.client.0.smithi178.stderr: -187> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: crc 2024-06-28T15:23:20.814 INFO:tasks.workunit.client.0.smithi178.stderr: -186> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: secure 2024-06-28T15:23:20.814 INFO:tasks.workunit.client.0.smithi178.stderr: -185> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: crc 2024-06-28T15:23:20.814 INFO:tasks.workunit.client.0.smithi178.stderr: -184> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: secure 2024-06-28T15:23:20.814 INFO:tasks.workunit.client.0.smithi178.stderr: -183> 2024-06-28T15:23:20.502+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.814 INFO:tasks.workunit.client.0.smithi178.stderr: -182> 2024-06-28T15:23:20.502+0000 7fa42206d980 1 librados: starting msgr at 2024-06-28T15:23:20.814 INFO:tasks.workunit.client.0.smithi178.stderr: -181> 2024-06-28T15:23:20.502+0000 7fa42206d980 1 librados: starting objecter 2024-06-28T15:23:20.814 INFO:tasks.workunit.client.0.smithi178.stderr: -180> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command objecter_requests cmddesc objecter_requests hook 0x55d571941f90 EEXIST 2024-06-28T15:23:20.814 INFO:tasks.workunit.client.0.smithi178.stderr: -179> 2024-06-28T15:23:20.502+0000 7fa42206d980 1 librados: setting wanted keys 2024-06-28T15:23:20.814 INFO:tasks.workunit.client.0.smithi178.stderr: -178> 2024-06-28T15:23:20.502+0000 7fa42206d980 1 librados: calling monclient init 2024-06-28T15:23:20.814 INFO:tasks.workunit.client.0.smithi178.stderr: -177> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: init 2024-06-28T15:23:20.814 INFO:tasks.workunit.client.0.smithi178.stderr: -176> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding auth protocol: cephx 2024-06-28T15:23:20.815 INFO:tasks.workunit.client.0.smithi178.stderr: -175> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding auth protocol: cephx 2024-06-28T15:23:20.815 INFO:tasks.workunit.client.0.smithi178.stderr: -174> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding auth protocol: cephx 2024-06-28T15:23:20.815 INFO:tasks.workunit.client.0.smithi178.stderr: -173> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: secure 2024-06-28T15:23:20.815 INFO:tasks.workunit.client.0.smithi178.stderr: -172> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: crc 2024-06-28T15:23:20.815 INFO:tasks.workunit.client.0.smithi178.stderr: -171> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: secure 2024-06-28T15:23:20.815 INFO:tasks.workunit.client.0.smithi178.stderr: -170> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: crc 2024-06-28T15:23:20.815 INFO:tasks.workunit.client.0.smithi178.stderr: -169> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: secure 2024-06-28T15:23:20.815 INFO:tasks.workunit.client.0.smithi178.stderr: -168> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: crc 2024-06-28T15:23:20.815 INFO:tasks.workunit.client.0.smithi178.stderr: -167> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: crc 2024-06-28T15:23:20.815 INFO:tasks.workunit.client.0.smithi178.stderr: -166> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: secure 2024-06-28T15:23:20.815 INFO:tasks.workunit.client.0.smithi178.stderr: -165> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: crc 2024-06-28T15:23:20.815 INFO:tasks.workunit.client.0.smithi178.stderr: -164> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: secure 2024-06-28T15:23:20.815 INFO:tasks.workunit.client.0.smithi178.stderr: -163> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: crc 2024-06-28T15:23:20.816 INFO:tasks.workunit.client.0.smithi178.stderr: -162> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: secure 2024-06-28T15:23:20.816 INFO:tasks.workunit.client.0.smithi178.stderr: -161> 2024-06-28T15:23:20.502+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.816 INFO:tasks.workunit.client.0.smithi178.stderr: -160> 2024-06-28T15:23:20.502+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.816 INFO:tasks.workunit.client.0.smithi178.stderr: -159> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command rotate-key cmddesc rotate-key hook 0x55d57190bb48 EEXIST 2024-06-28T15:23:20.816 INFO:tasks.workunit.client.0.smithi178.stderr: -158> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: _reopen_session rank -1 2024-06-28T15:23:20.816 INFO:tasks.workunit.client.0.smithi178.stderr: -157> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: _add_conns ranks=[0,2,1] 2024-06-28T15:23:20.816 INFO:tasks.workunit.client.0.smithi178.stderr: -156> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-a con 0x55d571919160 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:23:20.816 INFO:tasks.workunit.client.0.smithi178.stderr: -155> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-c con 0x55d5719439a0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:23:20.816 INFO:tasks.workunit.client.0.smithi178.stderr: -154> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-b con 0x55d5719157a0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:23:20.816 INFO:tasks.workunit.client.0.smithi178.stderr: -153> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.816 INFO:tasks.workunit.client.0.smithi178.stderr: -152> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.816 INFO:tasks.workunit.client.0.smithi178.stderr: -151> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.816 INFO:tasks.workunit.client.0.smithi178.stderr: -150> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): _renew_subs 2024-06-28T15:23:20.816 INFO:tasks.workunit.client.0.smithi178.stderr: -149> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): authenticate will time out at 2343.967549s 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -148> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request con 0x55d571919160 auth_method 0 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -147> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -146> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -145> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -144> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): get_auth_request con 0x55d5719439a0 auth_method 0 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -143> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -142> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -141> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -140> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -139> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -138> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -137> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -136> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -135> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.817 INFO:tasks.workunit.client.0.smithi178.stderr: -134> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_done global_id 6447 payload 824 2024-06-28T15:23:20.818 INFO:tasks.workunit.client.0.smithi178.stderr: -133> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient: _finish_hunting 0 2024-06-28T15:23:20.818 INFO:tasks.workunit.client.0.smithi178.stderr: -132> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 1 monclient: found mon.noname-a 2024-06-28T15:23:20.818 INFO:tasks.workunit.client.0.smithi178.stderr: -131> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.818 INFO:tasks.workunit.client.0.smithi178.stderr: -130> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.818 INFO:tasks.workunit.client.0.smithi178.stderr: -129> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:23:20.818 INFO:tasks.workunit.client.0.smithi178.stderr: -128> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: dump: 2024-06-28T15:23:20.818 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.818 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.818 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.818 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.818 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.818 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.818 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.818 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.819 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.819 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.819 INFO:tasks.workunit.client.0.smithi178.stderr: -127> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: _finish_auth 0 2024-06-28T15:23:20.819 INFO:tasks.workunit.client.0.smithi178.stderr: -126> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: _check_auth_tickets 2024-06-28T15:23:20.819 INFO:tasks.workunit.client.0.smithi178.stderr: -125> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: handle_config config(0 keys) 2024-06-28T15:23:20.819 INFO:tasks.workunit.client.0.smithi178.stderr: -124> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.819 INFO:tasks.workunit.client.0.smithi178.stderr: -123> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:23:20.819 INFO:tasks.workunit.client.0.smithi178.stderr: -122> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: dump: 2024-06-28T15:23:20.819 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.819 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.819 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.819 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.819 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.819 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.820 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.820 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.820 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.820 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.820 INFO:tasks.workunit.client.0.smithi178.stderr: -121> 2024-06-28T15:23:20.506+0000 7fa2feffd640 4 set_mon_vals no callback set 2024-06-28T15:23:20.820 INFO:tasks.workunit.client.0.smithi178.stderr: -120> 2024-06-28T15:23:20.506+0000 7fa42206d980 5 monclient: authenticate success, global_id 6447 2024-06-28T15:23:20.820 INFO:tasks.workunit.client.0.smithi178.stderr: -119> 2024-06-28T15:23:20.506+0000 7fa42206d980 10 monclient: _renew_subs 2024-06-28T15:23:20.820 INFO:tasks.workunit.client.0.smithi178.stderr: -118> 2024-06-28T15:23:20.506+0000 7fa42206d980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.820 INFO:tasks.workunit.client.0.smithi178.stderr: -117> 2024-06-28T15:23:20.506+0000 7fa42206d980 10 monclient: _renew_subs 2024-06-28T15:23:20.820 INFO:tasks.workunit.client.0.smithi178.stderr: -116> 2024-06-28T15:23:20.506+0000 7fa42206d980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.820 INFO:tasks.workunit.client.0.smithi178.stderr: -115> 2024-06-28T15:23:20.506+0000 7fa42206d980 1 librados: init done 2024-06-28T15:23:20.820 INFO:tasks.workunit.client.0.smithi178.stderr: -114> 2024-06-28T15:23:20.506+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command cr dump hook 0x55d57194c218 2024-06-28T15:23:20.820 INFO:tasks.workunit.client.0.smithi178.stderr: -113> 2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.820 INFO:tasks.workunit.client.0.smithi178.stderr: -112> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:23:20.821 INFO:tasks.workunit.client.0.smithi178.stderr: -111> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:23:20.821 INFO:tasks.workunit.client.0.smithi178.stderr: -110> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:23:20.821 INFO:tasks.workunit.client.0.smithi178.stderr: -109> 2024-06-28T15:23:20.506+0000 7fa41f37d640 10 monclient: get_auth_request con 0x7fa2e806f780 auth_method 0 2024-06-28T15:23:20.821 INFO:tasks.workunit.client.0.smithi178.stderr: -108> 2024-06-28T15:23:20.506+0000 7fa42037f640 10 monclient: get_auth_request con 0x55d571961ad0 auth_method 0 2024-06-28T15:23:20.821 INFO:tasks.workunit.client.0.smithi178.stderr: -107> 2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.821 INFO:tasks.workunit.client.0.smithi178.stderr: -106> 2024-06-28T15:23:20.506+0000 7fa42206d980 20 realm 2024-06-28T15:23:20.821 INFO:tasks.workunit.client.0.smithi178.stderr: -105> 2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.821 INFO:tasks.workunit.client.0.smithi178.stderr: -104> 2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.821 INFO:tasks.workunit.client.0.smithi178.stderr: -103> 2024-06-28T15:23:20.506+0000 7fa42206d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:20.821 INFO:tasks.workunit.client.0.smithi178.stderr: -102> 2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.821 INFO:tasks.workunit.client.0.smithi178.stderr: -101> 2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.821 INFO:tasks.workunit.client.0.smithi178.stderr: -100> 2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.821 INFO:tasks.workunit.client.0.smithi178.stderr: -99> 2024-06-28T15:23:20.510+0000 7fa41fb7e640 10 monclient: get_auth_request con 0x55d571965d20 auth_method 0 2024-06-28T15:23:20.821 INFO:tasks.workunit.client.0.smithi178.stderr: -98> 2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:20.822 INFO:tasks.workunit.client.0.smithi178.stderr: -97> 2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.822 INFO:tasks.workunit.client.0.smithi178.stderr: -96> 2024-06-28T15:23:20.510+0000 7fa41f37d640 10 monclient: get_auth_request con 0x55d571969560 auth_method 0 2024-06-28T15:23:20.822 INFO:tasks.workunit.client.0.smithi178.stderr: -95> 2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:20.822 INFO:tasks.workunit.client.0.smithi178.stderr: -94> 2024-06-28T15:23:20.510+0000 7fa42206d980 20 searching for the correct realm 2024-06-28T15:23:20.822 INFO:tasks.workunit.client.0.smithi178.stderr: -93> 2024-06-28T15:23:20.510+0000 7fa42037f640 10 monclient: get_auth_request con 0x7fa414008040 auth_method 0 2024-06-28T15:23:20.822 INFO:tasks.workunit.client.0.smithi178.stderr: -92> 2024-06-28T15:23:20.514+0000 7fa41fb7e640 10 monclient: get_auth_request con 0x7fa4180ab070 auth_method 0 2024-06-28T15:23:20.822 INFO:tasks.workunit.client.0.smithi178.stderr: -91> 2024-06-28T15:23:20.514+0000 7fa41f37d640 10 monclient: get_auth_request con 0x7fa410067030 auth_method 0 2024-06-28T15:23:20.822 INFO:tasks.workunit.client.0.smithi178.stderr: -90> 2024-06-28T15:23:20.518+0000 7fa42037f640 10 monclient: get_auth_request con 0x7fa4140343a0 auth_method 0 2024-06-28T15:23:20.822 INFO:tasks.workunit.client.0.smithi178.stderr: -89> 2024-06-28T15:23:20.518+0000 7fa42037f640 0 -- 172.21.15.178:0/1630651460 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fa4140343a0 msgr2=0x7fa414034800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:20.822 INFO:tasks.workunit.client.0.smithi178.stderr: -88> 2024-06-28T15:23:20.722+0000 7fa42037f640 10 monclient: get_auth_request con 0x7fa4140343a0 auth_method 0 2024-06-28T15:23:20.822 INFO:tasks.workunit.client.0.smithi178.stderr: -87> 2024-06-28T15:23:20.730+0000 7fa41fb7e640 10 monclient: get_auth_request con 0x7fa41404e8e0 auth_method 0 2024-06-28T15:23:20.822 INFO:tasks.workunit.client.0.smithi178.stderr: -86> 2024-06-28T15:23:20.734+0000 7fa41fb7e640 0 -- 172.21.15.178:0/1630651460 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55d571965d20 msgr2=0x55d571968130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:20.822 INFO:tasks.workunit.client.0.smithi178.stderr: -85> 2024-06-28T15:23:20.734+0000 7fa2fd7fa640 1 client.6447.objecter ms_handle_reset 0x55d571965d20 session 0x55d5719656a0 osd.5 2024-06-28T15:23:20.822 INFO:tasks.workunit.client.0.smithi178.stderr: -84> 2024-06-28T15:23:20.734+0000 7fa2fd7fa640 10 monclient: _renew_subs 2024-06-28T15:23:20.823 INFO:tasks.workunit.client.0.smithi178.stderr: -83> 2024-06-28T15:23:20.734+0000 7fa2fd7fa640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.823 INFO:tasks.workunit.client.0.smithi178.stderr: -82> 2024-06-28T15:23:20.734+0000 7fa41fb7e640 10 monclient: get_auth_request con 0x7fa2e807fda0 auth_method 0 2024-06-28T15:23:20.823 INFO:tasks.workunit.client.0.smithi178.stderr: -81> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:20.823 INFO:tasks.workunit.client.0.smithi178.stderr: -80> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:20.823 INFO:tasks.workunit.client.0.smithi178.stderr: -79> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:20.823 INFO:tasks.workunit.client.0.smithi178.stderr: -78> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:20.823 INFO:tasks.workunit.client.0.smithi178.stderr: -77> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:20.823 INFO:tasks.workunit.client.0.smithi178.stderr: -76> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:20.823 INFO:tasks.workunit.client.0.smithi178.stderr: -75> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.823 INFO:tasks.workunit.client.0.smithi178.stderr: -74> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.823 INFO:tasks.workunit.client.0.smithi178.stderr: -73> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.823 INFO:tasks.workunit.client.0.smithi178.stderr: -72> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:20.823 INFO:tasks.workunit.client.0.smithi178.stderr: -71> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.824 INFO:tasks.workunit.client.0.smithi178.stderr: -70> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:20.824 INFO:tasks.workunit.client.0.smithi178.stderr: -69> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 zone default found 2024-06-28T15:23:20.824 INFO:tasks.workunit.client.0.smithi178.stderr: -68> 2024-06-28T15:23:20.746+0000 7fa42206d980 4 Realm: () 2024-06-28T15:23:20.824 INFO:tasks.workunit.client.0.smithi178.stderr: -67> 2024-06-28T15:23:20.746+0000 7fa42206d980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:20.824 INFO:tasks.workunit.client.0.smithi178.stderr: -66> 2024-06-28T15:23:20.746+0000 7fa42206d980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:20.824 INFO:tasks.workunit.client.0.smithi178.stderr: -65> 2024-06-28T15:23:20.746+0000 7fa42206d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:20.824 INFO:tasks.workunit.client.0.smithi178.stderr: -64> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 zonegroup default 2024-06-28T15:23:20.824 INFO:tasks.workunit.client.0.smithi178.stderr: -63> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.824 INFO:tasks.workunit.client.0.smithi178.stderr: -62> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.824 INFO:tasks.workunit.client.0.smithi178.stderr: -61> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.824 INFO:tasks.workunit.client.0.smithi178.stderr: -60> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.824 INFO:tasks.workunit.client.0.smithi178.stderr: -59> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.824 INFO:tasks.workunit.client.0.smithi178.stderr: -58> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.824 INFO:tasks.workunit.client.0.smithi178.stderr: -57> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 started sync module instance, tier type = 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -56> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -55> 2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=0 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -54> 2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=1 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -53> 2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=2 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -52> 2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=3 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -51> 2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=4 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -50> 2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=5 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -49> 2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=6 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -48> 2024-06-28T15:23:20.770+0000 7fa42206d980 20 add_watcher() i=7 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -47> 2024-06-28T15:23:20.770+0000 7fa42206d980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -46> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding auth protocol: cephx 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -45> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding auth protocol: cephx 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -44> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding auth protocol: cephx 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -43> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: secure 2024-06-28T15:23:20.825 INFO:tasks.workunit.client.0.smithi178.stderr: -42> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: crc 2024-06-28T15:23:20.826 INFO:tasks.workunit.client.0.smithi178.stderr: -41> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: secure 2024-06-28T15:23:20.826 INFO:tasks.workunit.client.0.smithi178.stderr: -40> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: crc 2024-06-28T15:23:20.826 INFO:tasks.workunit.client.0.smithi178.stderr: -39> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: secure 2024-06-28T15:23:20.826 INFO:tasks.workunit.client.0.smithi178.stderr: -38> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: crc 2024-06-28T15:23:20.826 INFO:tasks.workunit.client.0.smithi178.stderr: -37> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: crc 2024-06-28T15:23:20.826 INFO:tasks.workunit.client.0.smithi178.stderr: -36> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: secure 2024-06-28T15:23:20.826 INFO:tasks.workunit.client.0.smithi178.stderr: -35> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: crc 2024-06-28T15:23:20.826 INFO:tasks.workunit.client.0.smithi178.stderr: -34> 2024-06-28T15:23:20.774+0000 7fa2d67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:20.826 INFO:tasks.workunit.client.0.smithi178.stderr: -33> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: secure 2024-06-28T15:23:20.826 INFO:tasks.workunit.client.0.smithi178.stderr: -32> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: crc 2024-06-28T15:23:20.826 INFO:tasks.workunit.client.0.smithi178.stderr: -31> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: secure 2024-06-28T15:23:20.826 INFO:tasks.workunit.client.0.smithi178.stderr: -30> 2024-06-28T15:23:20.774+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.826 INFO:tasks.workunit.client.0.smithi178.stderr: -29> 2024-06-28T15:23:20.774+0000 7fa42206d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:20.827 INFO:tasks.workunit.client.0.smithi178.stderr: -28> 2024-06-28T15:23:20.774+0000 7fa42206d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:20.827 INFO:tasks.workunit.client.0.smithi178.stderr: -27> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command cache list hook 0x55d57193cf20 2024-06-28T15:23:20.827 INFO:tasks.workunit.client.0.smithi178.stderr: -26> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command cache inspect hook 0x55d57193cf20 2024-06-28T15:23:20.827 INFO:tasks.workunit.client.0.smithi178.stderr: -25> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command cache erase hook 0x55d57193cf20 2024-06-28T15:23:20.827 INFO:tasks.workunit.client.0.smithi178.stderr: -24> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command cache zap hook 0x55d57193cf20 2024-06-28T15:23:20.827 INFO:tasks.workunit.client.0.smithi178.stderr: -23> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 note: GC not initialized 2024-06-28T15:23:20.827 INFO:tasks.workunit.client.0.smithi178.stderr: -22> 2024-06-28T15:23:20.774+0000 7fa2d57e2640 20 reqs_thread_entry: start 2024-06-28T15:23:20.827 INFO:tasks.workunit.client.0.smithi178.stderr: -21> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command sync trace show hook 0x7fa4140088e0 2024-06-28T15:23:20.827 INFO:tasks.workunit.client.0.smithi178.stderr: -20> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command sync trace history hook 0x7fa4140088e0 2024-06-28T15:23:20.827 INFO:tasks.workunit.client.0.smithi178.stderr: -19> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command sync trace active hook 0x7fa4140088e0 2024-06-28T15:23:20.827 INFO:tasks.workunit.client.0.smithi178.stderr: -18> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command sync trace active_short hook 0x7fa4140088e0 2024-06-28T15:23:20.827 INFO:tasks.workunit.client.0.smithi178.stderr: -17> 2024-06-28T15:23:20.774+0000 7fa42206d980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:20.827 INFO:tasks.workunit.client.0.smithi178.stderr: -16> 2024-06-28T15:23:20.774+0000 7fa42206d980 20 Filter name: none 2024-06-28T15:23:20.827 INFO:tasks.workunit.client.0.smithi178.stderr: -15> 2024-06-28T15:23:20.774+0000 7fa2ce7fc640 20 reqs_thread_entry: start 2024-06-28T15:23:20.828 INFO:tasks.workunit.client.0.smithi178.stderr: -14> 2024-06-28T15:23:20.774+0000 7fa42206d980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-06-28T15:23:20.828 INFO:tasks.workunit.client.0.smithi178.stderr: -13> 2024-06-28T15:23:20.774+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.828 INFO:tasks.workunit.client.0.smithi178.stderr: -12> 2024-06-28T15:23:20.774+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:20.828 INFO:tasks.workunit.client.0.smithi178.stderr: -11> 2024-06-28T15:23:20.774+0000 7fa42206d980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-06-28T15:23:20.828 INFO:tasks.workunit.client.0.smithi178.stderr: -10> 2024-06-28T15:23:20.774+0000 7fa42206d980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-06-28T15:23:20.828 INFO:tasks.workunit.client.0.smithi178.stderr: -9> 2024-06-28T15:23:20.774+0000 7fa42206d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 : miss 2024-06-28T15:23:20.828 INFO:tasks.workunit.client.0.smithi178.stderr: -8> 2024-06-28T15:23:20.774+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.828 INFO:tasks.workunit.client.0.smithi178.stderr: -7> 2024-06-28T15:23:20.778+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:20.828 INFO:tasks.workunit.client.0.smithi178.stderr: -6> 2024-06-28T15:23:20.778+0000 7fa42206d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 info.flags=0x17 2024-06-28T15:23:20.828 INFO:tasks.workunit.client.0.smithi178.stderr: -5> 2024-06-28T15:23:20.778+0000 7fa42206d980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 to cache LRU end 2024-06-28T15:23:20.828 INFO:tasks.workunit.client.0.smithi178.stderr: -4> 2024-06-28T15:23:20.778+0000 7fa42206d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:20.828 INFO:tasks.workunit.client.0.smithi178.stderr: -3> 2024-06-28T15:23:20.778+0000 7fa42206d980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:20.828 INFO:tasks.workunit.client.0.smithi178.stderr: -2> 2024-06-28T15:23:20.778+0000 7fa42206d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 2024-06-28T15:23:20.828 INFO:tasks.workunit.client.0.smithi178.stderr: -1> 2024-06-28T15:23:20.782+0000 7fa42206d980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fa42206d980 time 2024-06-28T15:23:20.787106+0000 2024-06-28T15:23:20.829 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:23:20.829 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.829 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:23:20.829 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fa425944c91] 2024-06-28T15:23:20.829 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fa425944e45] 2024-06-28T15:23:20.829 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d5700124b1] 2024-06-28T15:23:20.829 INFO:tasks.workunit.client.0.smithi178.stderr: 4: radosgw-admin(+0x9acc0a) [0x55d57001ec0a] 2024-06-28T15:23:20.829 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x260) [0x55d5700239e0] 2024-06-28T15:23:20.829 INFO:tasks.workunit.client.0.smithi178.stderr: 6: main() 2024-06-28T15:23:20.829 INFO:tasks.workunit.client.0.smithi178.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa4247add90] 2024-06-28T15:23:20.829 INFO:tasks.workunit.client.0.smithi178.stderr: 8: __libc_start_main() 2024-06-28T15:23:20.829 INFO:tasks.workunit.client.0.smithi178.stderr: 9: _start() 2024-06-28T15:23:20.829 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.830 INFO:tasks.workunit.client.0.smithi178.stderr: 0> 2024-06-28T15:23:20.786+0000 7fa42206d980 -1 *** Caught signal (Aborted) ** 2024-06-28T15:23:20.830 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7fa42206d980 thread_name:radosgw-admin 2024-06-28T15:23:20.830 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.830 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:23:20.830 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fa4247c6520] 2024-06-28T15:23:20.830 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:23:20.830 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:23:20.830 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:23:20.830 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fa425944ceb] 2024-06-28T15:23:20.830 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fa425944e45] 2024-06-28T15:23:20.830 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d5700124b1] 2024-06-28T15:23:20.830 INFO:tasks.workunit.client.0.smithi178.stderr: 8: radosgw-admin(+0x9acc0a) [0x55d57001ec0a] 2024-06-28T15:23:20.830 INFO:tasks.workunit.client.0.smithi178.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x260) [0x55d5700239e0] 2024-06-28T15:23:20.830 INFO:tasks.workunit.client.0.smithi178.stderr: 10: main() 2024-06-28T15:23:20.831 INFO:tasks.workunit.client.0.smithi178.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa4247add90] 2024-06-28T15:23:20.831 INFO:tasks.workunit.client.0.smithi178.stderr: 12: __libc_start_main() 2024-06-28T15:23:20.831 INFO:tasks.workunit.client.0.smithi178.stderr: 13: _start() 2024-06-28T15:23:20.831 INFO:tasks.workunit.client.0.smithi178.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-28T15:23:20.831 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.831 INFO:tasks.workunit.client.0.smithi178.stderr:--- logging levels --- 2024-06-28T15:23:20.831 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 none 2024-06-28T15:23:20.831 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 lockdep 2024-06-28T15:23:20.831 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 context 2024-06-28T15:23:20.831 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 crush 2024-06-28T15:23:20.831 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds 2024-06-28T15:23:20.831 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_balancer 2024-06-28T15:23:20.831 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_locker 2024-06-28T15:23:20.831 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log_expire 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_migrator 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 3/ 5 mds_quiesce 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 buffer 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 timer 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 filer 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 striper 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 objecter 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rados 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_mirror 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_replay 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_pwl 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 journaler 2024-06-28T15:23:20.832 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objectcacher 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 immutable_obj_cache 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 client 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 osd 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 optracker 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objclass 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 filestore 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 journal 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 ms 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mon 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 0/10 monc 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 paxos 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 tp 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 auth 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 crypto 2024-06-28T15:23:20.833 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 finisher 2024-06-28T15:23:20.834 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 reserver 2024-06-28T15:23:20.834 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 heartbeatmap 2024-06-28T15:23:20.834 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 perfcounter 2024-06-28T15:23:20.834 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw 2024-06-28T15:23:20.834 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_sync 2024-06-28T15:23:20.834 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_datacache 2024-06-28T15:23:20.834 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_access 2024-06-28T15:23:20.834 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_dbstore 2024-06-28T15:23:20.834 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_flight 2024-06-28T15:23:20.834 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_lifecycle 2024-06-28T15:23:20.834 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw_notification 2024-06-28T15:23:20.834 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 javaclient 2024-06-28T15:23:20.834 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 asok 2024-06-28T15:23:20.834 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 throttle 2024-06-28T15:23:20.835 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 refs 2024-06-28T15:23:20.835 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 compressor 2024-06-28T15:23:20.835 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluestore 2024-06-28T15:23:20.835 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluefs 2024-06-28T15:23:20.835 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 bdev 2024-06-28T15:23:20.835 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 kstore 2024-06-28T15:23:20.835 INFO:tasks.workunit.client.0.smithi178.stderr: 4/ 5 rocksdb 2024-06-28T15:23:20.835 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 fuse 2024-06-28T15:23:20.835 INFO:tasks.workunit.client.0.smithi178.stderr: 2/ 5 mgr 2024-06-28T15:23:20.835 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mgrc 2024-06-28T15:23:20.835 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 dpdk 2024-06-28T15:23:20.835 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 eventtrace 2024-06-28T15:23:20.835 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 prioritycache 2024-06-28T15:23:20.835 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 test 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephfs_mirror 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephsqlite 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 crimson_interrupt 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_onode 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_odata 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_omap 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_tm 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_t 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cleaner 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_epm 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_lba 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_fixedkv_tree 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cache 2024-06-28T15:23:20.836 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_journal 2024-06-28T15:23:20.837 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_device 2024-06-28T15:23:20.837 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_backref 2024-06-28T15:23:20.837 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 alienstore 2024-06-28T15:23:20.837 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mclock 2024-06-28T15:23:20.837 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cyanstore 2024-06-28T15:23:20.837 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 ceph_exporter 2024-06-28T15:23:20.837 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 memstore 2024-06-28T15:23:20.837 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 trace 2024-06-28T15:23:20.837 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 ceph_dedup 2024-06-28T15:23:20.837 INFO:tasks.workunit.client.0.smithi178.stderr: -2/-2 (syslog threshold) 2024-06-28T15:23:20.837 INFO:tasks.workunit.client.0.smithi178.stderr: 99/99 (stderr threshold) 2024-06-28T15:23:20.837 INFO:tasks.workunit.client.0.smithi178.stderr:--- pthread ID / name mapping for recent threads --- 2024-06-28T15:23:20.837 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa2ce7fc640 / http_manager 2024-06-28T15:23:20.837 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa2d57e2640 / http_manager 2024-06-28T15:23:20.838 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa2d67e4640 / rgw_dt_lg_renew 2024-06-28T15:23:20.838 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa2fd7fa640 / ms_dispatch 2024-06-28T15:23:20.838 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa2feffd640 / io_context_pool 2024-06-28T15:23:20.838 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa3057fa640 / ms_dispatch 2024-06-28T15:23:20.838 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa307fff640 / ms_dispatch 2024-06-28T15:23:20.838 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa30de0b640 / io_context_pool 2024-06-28T15:23:20.838 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa41eb7c640 / io_context_pool 2024-06-28T15:23:20.838 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa41f37d640 / msgr-worker-2 2024-06-28T15:23:20.838 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa41fb7e640 / msgr-worker-1 2024-06-28T15:23:20.838 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa42037f640 / msgr-worker-0 2024-06-28T15:23:20.838 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa42206d980 / radosgw-admin 2024-06-28T15:23:20.838 INFO:tasks.workunit.client.0.smithi178.stderr: max_recent 500 2024-06-28T15:23:20.838 INFO:tasks.workunit.client.0.smithi178.stderr: max_new 1000 2024-06-28T15:23:20.838 INFO:tasks.workunit.client.0.smithi178.stderr: log_file /var/log/ceph/ceph-client.admin.81910.log 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr:--- end dump of recent events --- 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr:--- begin dump of recent events --- 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr: -499> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr: -498> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr: -497> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr: -496> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): _renew_subs 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr: -495> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): authenticate will time out at 2343.948354s 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr: -494> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request con 0x55d571704940 auth_method 0 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr: -493> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr: -492> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr: -491> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr: -490> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): get_auth_request con 0x55d57187ecf0 auth_method 0 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr: -489> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr: -488> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.839 INFO:tasks.workunit.client.0.smithi178.stderr: -487> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.840 INFO:tasks.workunit.client.0.smithi178.stderr: -486> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.840 INFO:tasks.workunit.client.0.smithi178.stderr: -485> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.840 INFO:tasks.workunit.client.0.smithi178.stderr: -484> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.840 INFO:tasks.workunit.client.0.smithi178.stderr: -483> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.840 INFO:tasks.workunit.client.0.smithi178.stderr: -482> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.840 INFO:tasks.workunit.client.0.smithi178.stderr: -481> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.840 INFO:tasks.workunit.client.0.smithi178.stderr: -480> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_done global_id 6435 payload 274 2024-06-28T15:23:20.840 INFO:tasks.workunit.client.0.smithi178.stderr: -479> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient: _finish_hunting 0 2024-06-28T15:23:20.840 INFO:tasks.workunit.client.0.smithi178.stderr: -478> 2024-06-28T15:23:20.486+0000 7fa42037f640 1 monclient: found mon.noname-a 2024-06-28T15:23:20.840 INFO:tasks.workunit.client.0.smithi178.stderr: -477> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.840 INFO:tasks.workunit.client.0.smithi178.stderr: -476> 2024-06-28T15:23:20.486+0000 7fa41f37d640 10 monclient: get_auth_request con 0x55d571726ed0 auth_method 0 2024-06-28T15:23:20.840 INFO:tasks.workunit.client.0.smithi178.stderr: -475> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.840 INFO:tasks.workunit.client.0.smithi178.stderr: -474> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:23:20.840 INFO:tasks.workunit.client.0.smithi178.stderr: -473> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: dump: 2024-06-28T15:23:20.841 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.841 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.841 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.841 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.841 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.841 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.841 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.841 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.841 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.841 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.841 INFO:tasks.workunit.client.0.smithi178.stderr: -472> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: _finish_auth 0 2024-06-28T15:23:20.841 INFO:tasks.workunit.client.0.smithi178.stderr: -471> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: _check_auth_tickets 2024-06-28T15:23:20.841 INFO:tasks.workunit.client.0.smithi178.stderr: -470> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: handle_config config(0 keys) 2024-06-28T15:23:20.841 INFO:tasks.workunit.client.0.smithi178.stderr: -469> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr: -468> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr: -467> 2024-06-28T15:23:20.486+0000 7fa30de0b640 10 monclient: dump: 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr: -466> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 monclient: authenticate success, global_id 6435 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr: -465> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient: get_monmap_and_config success 2024-06-28T15:23:20.842 INFO:tasks.workunit.client.0.smithi178.stderr: -464> 2024-06-28T15:23:20.486+0000 7fa42206d980 4 set_mon_vals no callback set 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr: -463> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient: shutdown 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr: -462> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 asok(0x55d57166b5e0) unregister_commands rotate-key 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr: -461> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient: build_initial_monmap 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr: -460> 2024-06-28T15:23:20.486+0000 7fa42206d980 1 build_initial for_mkfs: 0 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr: -459> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient: monmap: 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:23:20.843 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.844 INFO:tasks.workunit.client.0.smithi178.stderr: -458> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding auth protocol: cephx 2024-06-28T15:23:20.844 INFO:tasks.workunit.client.0.smithi178.stderr: -457> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding auth protocol: cephx 2024-06-28T15:23:20.844 INFO:tasks.workunit.client.0.smithi178.stderr: -456> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding auth protocol: cephx 2024-06-28T15:23:20.844 INFO:tasks.workunit.client.0.smithi178.stderr: -455> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: secure 2024-06-28T15:23:20.844 INFO:tasks.workunit.client.0.smithi178.stderr: -454> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: crc 2024-06-28T15:23:20.844 INFO:tasks.workunit.client.0.smithi178.stderr: -453> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: secure 2024-06-28T15:23:20.844 INFO:tasks.workunit.client.0.smithi178.stderr: -452> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: crc 2024-06-28T15:23:20.844 INFO:tasks.workunit.client.0.smithi178.stderr: -451> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: secure 2024-06-28T15:23:20.844 INFO:tasks.workunit.client.0.smithi178.stderr: -450> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: crc 2024-06-28T15:23:20.844 INFO:tasks.workunit.client.0.smithi178.stderr: -449> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: crc 2024-06-28T15:23:20.845 INFO:tasks.workunit.client.0.smithi178.stderr: -448> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: secure 2024-06-28T15:23:20.845 INFO:tasks.workunit.client.0.smithi178.stderr: -447> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: crc 2024-06-28T15:23:20.845 INFO:tasks.workunit.client.0.smithi178.stderr: -446> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: secure 2024-06-28T15:23:20.845 INFO:tasks.workunit.client.0.smithi178.stderr: -445> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: crc 2024-06-28T15:23:20.845 INFO:tasks.workunit.client.0.smithi178.stderr: -444> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d57187ded0) adding con mode: secure 2024-06-28T15:23:20.845 INFO:tasks.workunit.client.0.smithi178.stderr: -443> 2024-06-28T15:23:20.486+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.845 INFO:tasks.workunit.client.0.smithi178.stderr: -442> 2024-06-28T15:23:20.486+0000 7fa42206d980 1 librados: starting msgr at 2024-06-28T15:23:20.845 INFO:tasks.workunit.client.0.smithi178.stderr: -441> 2024-06-28T15:23:20.486+0000 7fa42206d980 1 librados: starting objecter 2024-06-28T15:23:20.845 INFO:tasks.workunit.client.0.smithi178.stderr: -440> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command objecter_requests hook 0x55d571694f30 2024-06-28T15:23:20.845 INFO:tasks.workunit.client.0.smithi178.stderr: -439> 2024-06-28T15:23:20.486+0000 7fa42206d980 1 librados: setting wanted keys 2024-06-28T15:23:20.845 INFO:tasks.workunit.client.0.smithi178.stderr: -438> 2024-06-28T15:23:20.486+0000 7fa42206d980 1 librados: calling monclient init 2024-06-28T15:23:20.845 INFO:tasks.workunit.client.0.smithi178.stderr: -437> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient: init 2024-06-28T15:23:20.845 INFO:tasks.workunit.client.0.smithi178.stderr: -436> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding auth protocol: cephx 2024-06-28T15:23:20.845 INFO:tasks.workunit.client.0.smithi178.stderr: -435> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding auth protocol: cephx 2024-06-28T15:23:20.846 INFO:tasks.workunit.client.0.smithi178.stderr: -434> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding auth protocol: cephx 2024-06-28T15:23:20.846 INFO:tasks.workunit.client.0.smithi178.stderr: -433> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: secure 2024-06-28T15:23:20.846 INFO:tasks.workunit.client.0.smithi178.stderr: -432> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: crc 2024-06-28T15:23:20.846 INFO:tasks.workunit.client.0.smithi178.stderr: -431> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: secure 2024-06-28T15:23:20.846 INFO:tasks.workunit.client.0.smithi178.stderr: -430> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: crc 2024-06-28T15:23:20.846 INFO:tasks.workunit.client.0.smithi178.stderr: -429> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: secure 2024-06-28T15:23:20.846 INFO:tasks.workunit.client.0.smithi178.stderr: -428> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: crc 2024-06-28T15:23:20.846 INFO:tasks.workunit.client.0.smithi178.stderr: -427> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: crc 2024-06-28T15:23:20.846 INFO:tasks.workunit.client.0.smithi178.stderr: -426> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: secure 2024-06-28T15:23:20.846 INFO:tasks.workunit.client.0.smithi178.stderr: -425> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: crc 2024-06-28T15:23:20.846 INFO:tasks.workunit.client.0.smithi178.stderr: -424> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: secure 2024-06-28T15:23:20.846 INFO:tasks.workunit.client.0.smithi178.stderr: -423> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: crc 2024-06-28T15:23:20.846 INFO:tasks.workunit.client.0.smithi178.stderr: -422> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 AuthRegistry(0x55d571878a80) adding con mode: secure 2024-06-28T15:23:20.846 INFO:tasks.workunit.client.0.smithi178.stderr: -421> 2024-06-28T15:23:20.486+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.847 INFO:tasks.workunit.client.0.smithi178.stderr: -420> 2024-06-28T15:23:20.486+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.847 INFO:tasks.workunit.client.0.smithi178.stderr: -419> 2024-06-28T15:23:20.486+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command rotate-key hook 0x55d571878bc8 2024-06-28T15:23:20.847 INFO:tasks.workunit.client.0.smithi178.stderr: -418> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient: _reopen_session rank -1 2024-06-28T15:23:20.847 INFO:tasks.workunit.client.0.smithi178.stderr: -417> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient: _add_conns ranks=[2,1,0] 2024-06-28T15:23:20.847 INFO:tasks.workunit.client.0.smithi178.stderr: -416> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-c con 0x55d571704940 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:23:20.847 INFO:tasks.workunit.client.0.smithi178.stderr: -415> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-b con 0x55d571726ed0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:23:20.847 INFO:tasks.workunit.client.0.smithi178.stderr: -414> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-a con 0x55d57187ecf0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:23:20.847 INFO:tasks.workunit.client.0.smithi178.stderr: -413> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.847 INFO:tasks.workunit.client.0.smithi178.stderr: -412> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.847 INFO:tasks.workunit.client.0.smithi178.stderr: -411> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.847 INFO:tasks.workunit.client.0.smithi178.stderr: -410> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): _renew_subs 2024-06-28T15:23:20.847 INFO:tasks.workunit.client.0.smithi178.stderr: -409> 2024-06-28T15:23:20.486+0000 7fa42206d980 10 monclient(hunting): authenticate will time out at 2343.951467s 2024-06-28T15:23:20.847 INFO:tasks.workunit.client.0.smithi178.stderr: -408> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request con 0x55d571704940 auth_method 0 2024-06-28T15:23:20.847 INFO:tasks.workunit.client.0.smithi178.stderr: -407> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -406> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -405> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -404> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): get_auth_request con 0x55d57187ecf0 auth_method 0 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -403> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -402> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -401> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -400> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -399> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -398> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -397> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -396> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -395> 2024-06-28T15:23:20.486+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -394> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient(hunting): handle_auth_done global_id 6438 payload 824 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -393> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient: _finish_hunting 0 2024-06-28T15:23:20.848 INFO:tasks.workunit.client.0.smithi178.stderr: -392> 2024-06-28T15:23:20.486+0000 7fa42037f640 1 monclient: found mon.noname-a 2024-06-28T15:23:20.849 INFO:tasks.workunit.client.0.smithi178.stderr: -391> 2024-06-28T15:23:20.486+0000 7fa42037f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.849 INFO:tasks.workunit.client.0.smithi178.stderr: -390> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.849 INFO:tasks.workunit.client.0.smithi178.stderr: -389> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:23:20.849 INFO:tasks.workunit.client.0.smithi178.stderr: -388> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: dump: 2024-06-28T15:23:20.849 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.849 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.849 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.849 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.849 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.849 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.849 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.849 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.849 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.849 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.850 INFO:tasks.workunit.client.0.smithi178.stderr: -387> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: _finish_auth 0 2024-06-28T15:23:20.850 INFO:tasks.workunit.client.0.smithi178.stderr: -386> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: _check_auth_tickets 2024-06-28T15:23:20.850 INFO:tasks.workunit.client.0.smithi178.stderr: -385> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: handle_config config(0 keys) 2024-06-28T15:23:20.850 INFO:tasks.workunit.client.0.smithi178.stderr: -384> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.850 INFO:tasks.workunit.client.0.smithi178.stderr: -383> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:23:20.850 INFO:tasks.workunit.client.0.smithi178.stderr: -382> 2024-06-28T15:23:20.490+0000 7fa307fff640 10 monclient: dump: 2024-06-28T15:23:20.850 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.850 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.850 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.850 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.850 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.850 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.850 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.850 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.851 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.851 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.851 INFO:tasks.workunit.client.0.smithi178.stderr: -381> 2024-06-28T15:23:20.490+0000 7fa42206d980 5 monclient: authenticate success, global_id 6438 2024-06-28T15:23:20.851 INFO:tasks.workunit.client.0.smithi178.stderr: -380> 2024-06-28T15:23:20.490+0000 7fa30de0b640 4 set_mon_vals no callback set 2024-06-28T15:23:20.851 INFO:tasks.workunit.client.0.smithi178.stderr: -379> 2024-06-28T15:23:20.490+0000 7fa42206d980 10 monclient: _renew_subs 2024-06-28T15:23:20.851 INFO:tasks.workunit.client.0.smithi178.stderr: -378> 2024-06-28T15:23:20.490+0000 7fa42206d980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.851 INFO:tasks.workunit.client.0.smithi178.stderr: -377> 2024-06-28T15:23:20.490+0000 7fa42206d980 10 monclient: _renew_subs 2024-06-28T15:23:20.851 INFO:tasks.workunit.client.0.smithi178.stderr: -376> 2024-06-28T15:23:20.490+0000 7fa42206d980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.851 INFO:tasks.workunit.client.0.smithi178.stderr: -375> 2024-06-28T15:23:20.490+0000 7fa42206d980 1 librados: init done 2024-06-28T15:23:20.851 INFO:tasks.workunit.client.0.smithi178.stderr: -374> 2024-06-28T15:23:20.490+0000 7fa307fff640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:23:20.851 INFO:tasks.workunit.client.0.smithi178.stderr: -373> 2024-06-28T15:23:20.490+0000 7fa307fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:23:20.851 INFO:tasks.workunit.client.0.smithi178.stderr: -372> 2024-06-28T15:23:20.490+0000 7fa307fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:23:20.851 INFO:tasks.workunit.client.0.smithi178.stderr: -371> 2024-06-28T15:23:20.490+0000 7fa41fb7e640 10 monclient: get_auth_request con 0x7fa30806f780 auth_method 0 2024-06-28T15:23:20.852 INFO:tasks.workunit.client.0.smithi178.stderr: -370> 2024-06-28T15:23:20.490+0000 7fa41f37d640 10 monclient: get_auth_request con 0x55d5718a1490 auth_method 0 2024-06-28T15:23:20.852 INFO:tasks.workunit.client.0.smithi178.stderr: -369> 2024-06-28T15:23:20.490+0000 7fa41fb7e640 10 monclient: get_auth_request con 0x55d5718c35d0 auth_method 0 2024-06-28T15:23:20.852 INFO:tasks.workunit.client.0.smithi178.stderr: -368> 2024-06-28T15:23:20.494+0000 7fa41f37d640 10 monclient: get_auth_request con 0x55d5718c77c0 auth_method 0 2024-06-28T15:23:20.852 INFO:tasks.workunit.client.0.smithi178.stderr: -367> 2024-06-28T15:23:20.494+0000 7fa42206d980 10 monclient: build_initial_monmap 2024-06-28T15:23:20.852 INFO:tasks.workunit.client.0.smithi178.stderr: -366> 2024-06-28T15:23:20.494+0000 7fa42206d980 1 build_initial for_mkfs: 0 2024-06-28T15:23:20.852 INFO:tasks.workunit.client.0.smithi178.stderr: -365> 2024-06-28T15:23:20.494+0000 7fa42206d980 10 monclient: monmap: 2024-06-28T15:23:20.852 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:23:20.852 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:23:20.852 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:23:20.852 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:23:20.852 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:23:20.852 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.852 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:23:20.852 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:23:20.853 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:23:20.853 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.853 INFO:tasks.workunit.client.0.smithi178.stderr: -364> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding auth protocol: cephx 2024-06-28T15:23:20.853 INFO:tasks.workunit.client.0.smithi178.stderr: -363> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding auth protocol: cephx 2024-06-28T15:23:20.853 INFO:tasks.workunit.client.0.smithi178.stderr: -362> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding auth protocol: cephx 2024-06-28T15:23:20.853 INFO:tasks.workunit.client.0.smithi178.stderr: -361> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: secure 2024-06-28T15:23:20.853 INFO:tasks.workunit.client.0.smithi178.stderr: -360> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: crc 2024-06-28T15:23:20.853 INFO:tasks.workunit.client.0.smithi178.stderr: -359> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: secure 2024-06-28T15:23:20.853 INFO:tasks.workunit.client.0.smithi178.stderr: -358> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: crc 2024-06-28T15:23:20.853 INFO:tasks.workunit.client.0.smithi178.stderr: -357> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: secure 2024-06-28T15:23:20.853 INFO:tasks.workunit.client.0.smithi178.stderr: -356> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: crc 2024-06-28T15:23:20.853 INFO:tasks.workunit.client.0.smithi178.stderr: -355> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: crc 2024-06-28T15:23:20.853 INFO:tasks.workunit.client.0.smithi178.stderr: -354> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: secure 2024-06-28T15:23:20.853 INFO:tasks.workunit.client.0.smithi178.stderr: -353> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: crc 2024-06-28T15:23:20.854 INFO:tasks.workunit.client.0.smithi178.stderr: -352> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: secure 2024-06-28T15:23:20.854 INFO:tasks.workunit.client.0.smithi178.stderr: -351> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: crc 2024-06-28T15:23:20.854 INFO:tasks.workunit.client.0.smithi178.stderr: -350> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718cc580) adding con mode: secure 2024-06-28T15:23:20.854 INFO:tasks.workunit.client.0.smithi178.stderr: -349> 2024-06-28T15:23:20.494+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.854 INFO:tasks.workunit.client.0.smithi178.stderr: -348> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command objecter_requests cmddesc objecter_requests hook 0x55d5718cfa10 EEXIST 2024-06-28T15:23:20.854 INFO:tasks.workunit.client.0.smithi178.stderr: -347> 2024-06-28T15:23:20.494+0000 7fa42206d980 10 monclient: init 2024-06-28T15:23:20.854 INFO:tasks.workunit.client.0.smithi178.stderr: -346> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding auth protocol: cephx 2024-06-28T15:23:20.854 INFO:tasks.workunit.client.0.smithi178.stderr: -345> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding auth protocol: cephx 2024-06-28T15:23:20.854 INFO:tasks.workunit.client.0.smithi178.stderr: -344> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding auth protocol: cephx 2024-06-28T15:23:20.854 INFO:tasks.workunit.client.0.smithi178.stderr: -343> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: secure 2024-06-28T15:23:20.854 INFO:tasks.workunit.client.0.smithi178.stderr: -342> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: crc 2024-06-28T15:23:20.854 INFO:tasks.workunit.client.0.smithi178.stderr: -341> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: secure 2024-06-28T15:23:20.854 INFO:tasks.workunit.client.0.smithi178.stderr: -340> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: crc 2024-06-28T15:23:20.854 INFO:tasks.workunit.client.0.smithi178.stderr: -339> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: secure 2024-06-28T15:23:20.855 INFO:tasks.workunit.client.0.smithi178.stderr: -338> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: crc 2024-06-28T15:23:20.855 INFO:tasks.workunit.client.0.smithi178.stderr: -337> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: crc 2024-06-28T15:23:20.855 INFO:tasks.workunit.client.0.smithi178.stderr: -336> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: secure 2024-06-28T15:23:20.855 INFO:tasks.workunit.client.0.smithi178.stderr: -335> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: crc 2024-06-28T15:23:20.855 INFO:tasks.workunit.client.0.smithi178.stderr: -334> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: secure 2024-06-28T15:23:20.855 INFO:tasks.workunit.client.0.smithi178.stderr: -333> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: crc 2024-06-28T15:23:20.855 INFO:tasks.workunit.client.0.smithi178.stderr: -332> 2024-06-28T15:23:20.494+0000 7fa42206d980 5 AuthRegistry(0x55d5718caa88) adding con mode: secure 2024-06-28T15:23:20.855 INFO:tasks.workunit.client.0.smithi178.stderr: -331> 2024-06-28T15:23:20.498+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.855 INFO:tasks.workunit.client.0.smithi178.stderr: -330> 2024-06-28T15:23:20.498+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.855 INFO:tasks.workunit.client.0.smithi178.stderr: -329> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command rotate-key cmddesc rotate-key hook 0x55d5718cabd0 EEXIST 2024-06-28T15:23:20.855 INFO:tasks.workunit.client.0.smithi178.stderr: -328> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: _reopen_session rank -1 2024-06-28T15:23:20.855 INFO:tasks.workunit.client.0.smithi178.stderr: -327> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: _add_conns ranks=[2,0,1] 2024-06-28T15:23:20.855 INFO:tasks.workunit.client.0.smithi178.stderr: -326> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-c con 0x55d5718d5fa0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:23:20.855 INFO:tasks.workunit.client.0.smithi178.stderr: -325> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-a con 0x55d5718d68d0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:23:20.856 INFO:tasks.workunit.client.0.smithi178.stderr: -324> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-b con 0x55d5718d6380 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:23:20.856 INFO:tasks.workunit.client.0.smithi178.stderr: -323> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.856 INFO:tasks.workunit.client.0.smithi178.stderr: -322> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.856 INFO:tasks.workunit.client.0.smithi178.stderr: -321> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.856 INFO:tasks.workunit.client.0.smithi178.stderr: -320> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): _renew_subs 2024-06-28T15:23:20.856 INFO:tasks.workunit.client.0.smithi178.stderr: -319> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient(hunting): authenticate will time out at 2343.960501s 2024-06-28T15:23:20.856 INFO:tasks.workunit.client.0.smithi178.stderr: -318> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient(hunting): get_auth_request con 0x55d5718d68d0 auth_method 0 2024-06-28T15:23:20.856 INFO:tasks.workunit.client.0.smithi178.stderr: -317> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.856 INFO:tasks.workunit.client.0.smithi178.stderr: -316> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.856 INFO:tasks.workunit.client.0.smithi178.stderr: -315> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.856 INFO:tasks.workunit.client.0.smithi178.stderr: -314> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request con 0x55d5718d5fa0 auth_method 0 2024-06-28T15:23:20.856 INFO:tasks.workunit.client.0.smithi178.stderr: -313> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.856 INFO:tasks.workunit.client.0.smithi178.stderr: -312> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.856 INFO:tasks.workunit.client.0.smithi178.stderr: -311> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr: -310> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr: -309> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr: -308> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr: -307> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr: -306> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr: -305> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr: -304> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_done global_id 6068 payload 824 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr: -303> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient: _finish_hunting 0 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr: -302> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 1 monclient: found mon.noname-c 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr: -301> 2024-06-28T15:23:20.498+0000 7fa41fb7e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr: -300> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr: -299> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr: -298> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: dump: 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.857 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.858 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.858 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.858 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.858 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.858 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.858 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.858 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.858 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.858 INFO:tasks.workunit.client.0.smithi178.stderr: -297> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: _finish_auth 0 2024-06-28T15:23:20.858 INFO:tasks.workunit.client.0.smithi178.stderr: -296> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: _check_auth_tickets 2024-06-28T15:23:20.858 INFO:tasks.workunit.client.0.smithi178.stderr: -295> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 monclient: authenticate success, global_id 6068 2024-06-28T15:23:20.858 INFO:tasks.workunit.client.0.smithi178.stderr: -294> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: _renew_subs 2024-06-28T15:23:20.858 INFO:tasks.workunit.client.0.smithi178.stderr: -293> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr: -292> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: handle_config config(0 keys) 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr: -291> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr: -290> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr: -289> 2024-06-28T15:23:20.498+0000 7fa3057fa640 10 monclient: dump: 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.859 INFO:tasks.workunit.client.0.smithi178.stderr: -288> 2024-06-28T15:23:20.498+0000 7fa41eb7c640 4 set_mon_vals no callback set 2024-06-28T15:23:20.860 INFO:tasks.workunit.client.0.smithi178.stderr: -287> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: _renew_subs 2024-06-28T15:23:20.860 INFO:tasks.workunit.client.0.smithi178.stderr: -286> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:23:20.860 INFO:tasks.workunit.client.0.smithi178.stderr: -285> 2024-06-28T15:23:20.498+0000 7fa3057fa640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:23:20.860 INFO:tasks.workunit.client.0.smithi178.stderr: -284> 2024-06-28T15:23:20.498+0000 7fa3057fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:23:20.860 INFO:tasks.workunit.client.0.smithi178.stderr: -283> 2024-06-28T15:23:20.498+0000 7fa3057fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:23:20.860 INFO:tasks.workunit.client.0.smithi178.stderr: -282> 2024-06-28T15:23:20.498+0000 7fa41f37d640 10 monclient: get_auth_request con 0x7fa2f806f690 auth_method 0 2024-06-28T15:23:20.860 INFO:tasks.workunit.client.0.smithi178.stderr: -281> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: get_monmap_and_config 2024-06-28T15:23:20.860 INFO:tasks.workunit.client.0.smithi178.stderr: -280> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: build_initial_monmap 2024-06-28T15:23:20.860 INFO:tasks.workunit.client.0.smithi178.stderr: -279> 2024-06-28T15:23:20.498+0000 7fa42206d980 1 build_initial for_mkfs: 0 2024-06-28T15:23:20.860 INFO:tasks.workunit.client.0.smithi178.stderr: -278> 2024-06-28T15:23:20.498+0000 7fa42206d980 10 monclient: monmap: 2024-06-28T15:23:20.860 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:23:20.860 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:23:20.860 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:23:20.860 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:23:20.861 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:23:20.861 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.861 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:23:20.861 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:23:20.861 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:23:20.861 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.861 INFO:tasks.workunit.client.0.smithi178.stderr: -277> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding auth protocol: cephx 2024-06-28T15:23:20.861 INFO:tasks.workunit.client.0.smithi178.stderr: -276> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding auth protocol: cephx 2024-06-28T15:23:20.861 INFO:tasks.workunit.client.0.smithi178.stderr: -275> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding auth protocol: cephx 2024-06-28T15:23:20.861 INFO:tasks.workunit.client.0.smithi178.stderr: -274> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: secure 2024-06-28T15:23:20.861 INFO:tasks.workunit.client.0.smithi178.stderr: -273> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: crc 2024-06-28T15:23:20.861 INFO:tasks.workunit.client.0.smithi178.stderr: -272> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: secure 2024-06-28T15:23:20.861 INFO:tasks.workunit.client.0.smithi178.stderr: -271> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: crc 2024-06-28T15:23:20.861 INFO:tasks.workunit.client.0.smithi178.stderr: -270> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: secure 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -269> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: crc 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -268> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: crc 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -267> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: secure 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -266> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: crc 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -265> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: secure 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -264> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: crc 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -263> 2024-06-28T15:23:20.498+0000 7fa42206d980 5 AuthRegistry(0x55d571913b00) adding con mode: secure 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -262> 2024-06-28T15:23:20.502+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -261> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: init 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -260> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding auth protocol: cephx 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -259> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding auth protocol: cephx 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -258> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding auth protocol: cephx 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -257> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: secure 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -256> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: crc 2024-06-28T15:23:20.862 INFO:tasks.workunit.client.0.smithi178.stderr: -255> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: secure 2024-06-28T15:23:20.863 INFO:tasks.workunit.client.0.smithi178.stderr: -254> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: crc 2024-06-28T15:23:20.863 INFO:tasks.workunit.client.0.smithi178.stderr: -253> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: secure 2024-06-28T15:23:20.863 INFO:tasks.workunit.client.0.smithi178.stderr: -252> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: crc 2024-06-28T15:23:20.863 INFO:tasks.workunit.client.0.smithi178.stderr: -251> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: crc 2024-06-28T15:23:20.863 INFO:tasks.workunit.client.0.smithi178.stderr: -250> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: secure 2024-06-28T15:23:20.863 INFO:tasks.workunit.client.0.smithi178.stderr: -249> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: crc 2024-06-28T15:23:20.863 INFO:tasks.workunit.client.0.smithi178.stderr: -248> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: secure 2024-06-28T15:23:20.863 INFO:tasks.workunit.client.0.smithi178.stderr: -247> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: crc 2024-06-28T15:23:20.863 INFO:tasks.workunit.client.0.smithi178.stderr: -246> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88defc0) adding con mode: secure 2024-06-28T15:23:20.863 INFO:tasks.workunit.client.0.smithi178.stderr: -245> 2024-06-28T15:23:20.502+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.863 INFO:tasks.workunit.client.0.smithi178.stderr: -244> 2024-06-28T15:23:20.502+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.863 INFO:tasks.workunit.client.0.smithi178.stderr: -243> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command rotate-key cmddesc rotate-key hook 0x7ffcc88df108 EEXIST 2024-06-28T15:23:20.863 INFO:tasks.workunit.client.0.smithi178.stderr: -242> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: _reopen_session rank -1 2024-06-28T15:23:20.863 INFO:tasks.workunit.client.0.smithi178.stderr: -241> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: _add_conns ranks=[0,2,1] 2024-06-28T15:23:20.864 INFO:tasks.workunit.client.0.smithi178.stderr: -240> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-a con 0x55d5719439a0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:23:20.864 INFO:tasks.workunit.client.0.smithi178.stderr: -239> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-c con 0x55d571943d80 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:23:20.864 INFO:tasks.workunit.client.0.smithi178.stderr: -238> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-b con 0x55d571919160 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:23:20.864 INFO:tasks.workunit.client.0.smithi178.stderr: -237> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.864 INFO:tasks.workunit.client.0.smithi178.stderr: -236> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.864 INFO:tasks.workunit.client.0.smithi178.stderr: -235> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.864 INFO:tasks.workunit.client.0.smithi178.stderr: -234> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): _renew_subs 2024-06-28T15:23:20.864 INFO:tasks.workunit.client.0.smithi178.stderr: -233> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): authenticate will time out at 2343.964937s 2024-06-28T15:23:20.864 INFO:tasks.workunit.client.0.smithi178.stderr: -232> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request con 0x55d5719439a0 auth_method 0 2024-06-28T15:23:20.864 INFO:tasks.workunit.client.0.smithi178.stderr: -231> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.864 INFO:tasks.workunit.client.0.smithi178.stderr: -230> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.864 INFO:tasks.workunit.client.0.smithi178.stderr: -229> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.864 INFO:tasks.workunit.client.0.smithi178.stderr: -228> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): get_auth_request con 0x55d571943d80 auth_method 0 2024-06-28T15:23:20.864 INFO:tasks.workunit.client.0.smithi178.stderr: -227> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.865 INFO:tasks.workunit.client.0.smithi178.stderr: -226> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.865 INFO:tasks.workunit.client.0.smithi178.stderr: -225> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.865 INFO:tasks.workunit.client.0.smithi178.stderr: -224> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.865 INFO:tasks.workunit.client.0.smithi178.stderr: -223> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.865 INFO:tasks.workunit.client.0.smithi178.stderr: -222> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.865 INFO:tasks.workunit.client.0.smithi178.stderr: -221> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.865 INFO:tasks.workunit.client.0.smithi178.stderr: -220> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.865 INFO:tasks.workunit.client.0.smithi178.stderr: -219> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.865 INFO:tasks.workunit.client.0.smithi178.stderr: -218> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_done global_id 6444 payload 274 2024-06-28T15:23:20.865 INFO:tasks.workunit.client.0.smithi178.stderr: -217> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient: _finish_hunting 0 2024-06-28T15:23:20.865 INFO:tasks.workunit.client.0.smithi178.stderr: -216> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 1 monclient: found mon.noname-a 2024-06-28T15:23:20.866 INFO:tasks.workunit.client.0.smithi178.stderr: -215> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.866 INFO:tasks.workunit.client.0.smithi178.stderr: -214> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.866 INFO:tasks.workunit.client.0.smithi178.stderr: -213> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:23:20.866 INFO:tasks.workunit.client.0.smithi178.stderr: -212> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: dump: 2024-06-28T15:23:20.866 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.866 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.866 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.866 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.866 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.866 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.866 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.866 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.866 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.866 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.867 INFO:tasks.workunit.client.0.smithi178.stderr: -211> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: _finish_auth 0 2024-06-28T15:23:20.867 INFO:tasks.workunit.client.0.smithi178.stderr: -210> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: _check_auth_tickets 2024-06-28T15:23:20.867 INFO:tasks.workunit.client.0.smithi178.stderr: -209> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: handle_config config(0 keys) 2024-06-28T15:23:20.867 INFO:tasks.workunit.client.0.smithi178.stderr: -208> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 monclient: authenticate success, global_id 6444 2024-06-28T15:23:20.867 INFO:tasks.workunit.client.0.smithi178.stderr: -207> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: get_monmap_and_config success 2024-06-28T15:23:20.867 INFO:tasks.workunit.client.0.smithi178.stderr: -206> 2024-06-28T15:23:20.502+0000 7fa42206d980 4 set_mon_vals no callback set 2024-06-28T15:23:20.867 INFO:tasks.workunit.client.0.smithi178.stderr: -205> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.867 INFO:tasks.workunit.client.0.smithi178.stderr: -204> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:23:20.867 INFO:tasks.workunit.client.0.smithi178.stderr: -203> 2024-06-28T15:23:20.502+0000 7fa2feffd640 10 monclient: dump: 2024-06-28T15:23:20.867 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.867 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.867 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.867 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.867 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.868 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.868 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.868 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.868 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.868 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.868 INFO:tasks.workunit.client.0.smithi178.stderr: -202> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: shutdown 2024-06-28T15:23:20.868 INFO:tasks.workunit.client.0.smithi178.stderr: -201> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: build_initial_monmap 2024-06-28T15:23:20.868 INFO:tasks.workunit.client.0.smithi178.stderr: -200> 2024-06-28T15:23:20.502+0000 7fa42206d980 1 build_initial for_mkfs: 0 2024-06-28T15:23:20.868 INFO:tasks.workunit.client.0.smithi178.stderr: -199> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: monmap: 2024-06-28T15:23:20.868 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:23:20.868 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:23:20.868 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:23:20.868 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:23:20.868 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:23:20.869 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.869 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:23:20.869 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:23:20.869 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:23:20.869 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.869 INFO:tasks.workunit.client.0.smithi178.stderr: -198> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding auth protocol: cephx 2024-06-28T15:23:20.869 INFO:tasks.workunit.client.0.smithi178.stderr: -197> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding auth protocol: cephx 2024-06-28T15:23:20.869 INFO:tasks.workunit.client.0.smithi178.stderr: -196> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding auth protocol: cephx 2024-06-28T15:23:20.869 INFO:tasks.workunit.client.0.smithi178.stderr: -195> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: secure 2024-06-28T15:23:20.869 INFO:tasks.workunit.client.0.smithi178.stderr: -194> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: crc 2024-06-28T15:23:20.869 INFO:tasks.workunit.client.0.smithi178.stderr: -193> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: secure 2024-06-28T15:23:20.869 INFO:tasks.workunit.client.0.smithi178.stderr: -192> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: crc 2024-06-28T15:23:20.869 INFO:tasks.workunit.client.0.smithi178.stderr: -191> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: secure 2024-06-28T15:23:20.869 INFO:tasks.workunit.client.0.smithi178.stderr: -190> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: crc 2024-06-28T15:23:20.870 INFO:tasks.workunit.client.0.smithi178.stderr: -189> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: crc 2024-06-28T15:23:20.870 INFO:tasks.workunit.client.0.smithi178.stderr: -188> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: secure 2024-06-28T15:23:20.870 INFO:tasks.workunit.client.0.smithi178.stderr: -187> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: crc 2024-06-28T15:23:20.870 INFO:tasks.workunit.client.0.smithi178.stderr: -186> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: secure 2024-06-28T15:23:20.870 INFO:tasks.workunit.client.0.smithi178.stderr: -185> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: crc 2024-06-28T15:23:20.870 INFO:tasks.workunit.client.0.smithi178.stderr: -184> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d571947790) adding con mode: secure 2024-06-28T15:23:20.870 INFO:tasks.workunit.client.0.smithi178.stderr: -183> 2024-06-28T15:23:20.502+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.870 INFO:tasks.workunit.client.0.smithi178.stderr: -182> 2024-06-28T15:23:20.502+0000 7fa42206d980 1 librados: starting msgr at 2024-06-28T15:23:20.870 INFO:tasks.workunit.client.0.smithi178.stderr: -181> 2024-06-28T15:23:20.502+0000 7fa42206d980 1 librados: starting objecter 2024-06-28T15:23:20.870 INFO:tasks.workunit.client.0.smithi178.stderr: -180> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command objecter_requests cmddesc objecter_requests hook 0x55d571941f90 EEXIST 2024-06-28T15:23:20.870 INFO:tasks.workunit.client.0.smithi178.stderr: -179> 2024-06-28T15:23:20.502+0000 7fa42206d980 1 librados: setting wanted keys 2024-06-28T15:23:20.870 INFO:tasks.workunit.client.0.smithi178.stderr: -178> 2024-06-28T15:23:20.502+0000 7fa42206d980 1 librados: calling monclient init 2024-06-28T15:23:20.870 INFO:tasks.workunit.client.0.smithi178.stderr: -177> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: init 2024-06-28T15:23:20.870 INFO:tasks.workunit.client.0.smithi178.stderr: -176> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding auth protocol: cephx 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -175> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding auth protocol: cephx 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -174> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding auth protocol: cephx 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -173> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: secure 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -172> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: crc 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -171> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: secure 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -170> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: crc 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -169> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: secure 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -168> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: crc 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -167> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: crc 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -166> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: secure 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -165> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: crc 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -164> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: secure 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -163> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: crc 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -162> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 AuthRegistry(0x55d57190ba00) adding con mode: secure 2024-06-28T15:23:20.871 INFO:tasks.workunit.client.0.smithi178.stderr: -161> 2024-06-28T15:23:20.502+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.872 INFO:tasks.workunit.client.0.smithi178.stderr: -160> 2024-06-28T15:23:20.502+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.872 INFO:tasks.workunit.client.0.smithi178.stderr: -159> 2024-06-28T15:23:20.502+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command rotate-key cmddesc rotate-key hook 0x55d57190bb48 EEXIST 2024-06-28T15:23:20.872 INFO:tasks.workunit.client.0.smithi178.stderr: -158> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: _reopen_session rank -1 2024-06-28T15:23:20.872 INFO:tasks.workunit.client.0.smithi178.stderr: -157> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient: _add_conns ranks=[0,2,1] 2024-06-28T15:23:20.872 INFO:tasks.workunit.client.0.smithi178.stderr: -156> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-a con 0x55d571919160 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:23:20.872 INFO:tasks.workunit.client.0.smithi178.stderr: -155> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-c con 0x55d5719439a0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:23:20.872 INFO:tasks.workunit.client.0.smithi178.stderr: -154> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): picked mon.noname-b con 0x55d5719157a0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:23:20.872 INFO:tasks.workunit.client.0.smithi178.stderr: -153> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.872 INFO:tasks.workunit.client.0.smithi178.stderr: -152> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.872 INFO:tasks.workunit.client.0.smithi178.stderr: -151> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:23:20.872 INFO:tasks.workunit.client.0.smithi178.stderr: -150> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): _renew_subs 2024-06-28T15:23:20.872 INFO:tasks.workunit.client.0.smithi178.stderr: -149> 2024-06-28T15:23:20.502+0000 7fa42206d980 10 monclient(hunting): authenticate will time out at 2343.967549s 2024-06-28T15:23:20.872 INFO:tasks.workunit.client.0.smithi178.stderr: -148> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request con 0x55d571919160 auth_method 0 2024-06-28T15:23:20.872 INFO:tasks.workunit.client.0.smithi178.stderr: -147> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.873 INFO:tasks.workunit.client.0.smithi178.stderr: -146> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.873 INFO:tasks.workunit.client.0.smithi178.stderr: -145> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.873 INFO:tasks.workunit.client.0.smithi178.stderr: -144> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): get_auth_request con 0x55d5719439a0 auth_method 0 2024-06-28T15:23:20.873 INFO:tasks.workunit.client.0.smithi178.stderr: -143> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:23:20.873 INFO:tasks.workunit.client.0.smithi178.stderr: -142> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:23:20.873 INFO:tasks.workunit.client.0.smithi178.stderr: -141> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:23:20.873 INFO:tasks.workunit.client.0.smithi178.stderr: -140> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.873 INFO:tasks.workunit.client.0.smithi178.stderr: -139> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.873 INFO:tasks.workunit.client.0.smithi178.stderr: -138> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.873 INFO:tasks.workunit.client.0.smithi178.stderr: -137> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:23:20.873 INFO:tasks.workunit.client.0.smithi178.stderr: -136> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:23:20.873 INFO:tasks.workunit.client.0.smithi178.stderr: -135> 2024-06-28T15:23:20.502+0000 7fa41f37d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:23:20.873 INFO:tasks.workunit.client.0.smithi178.stderr: -134> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient(hunting): handle_auth_done global_id 6447 payload 824 2024-06-28T15:23:20.873 INFO:tasks.workunit.client.0.smithi178.stderr: -133> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient: _finish_hunting 0 2024-06-28T15:23:20.874 INFO:tasks.workunit.client.0.smithi178.stderr: -132> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 1 monclient: found mon.noname-a 2024-06-28T15:23:20.874 INFO:tasks.workunit.client.0.smithi178.stderr: -131> 2024-06-28T15:23:20.502+0000 7fa41fb7e640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.874 INFO:tasks.workunit.client.0.smithi178.stderr: -130> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.874 INFO:tasks.workunit.client.0.smithi178.stderr: -129> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:23:20.874 INFO:tasks.workunit.client.0.smithi178.stderr: -128> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: dump: 2024-06-28T15:23:20.874 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.874 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.874 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.874 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.874 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.874 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.874 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.874 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.875 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.875 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.875 INFO:tasks.workunit.client.0.smithi178.stderr: -127> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: _finish_auth 0 2024-06-28T15:23:20.875 INFO:tasks.workunit.client.0.smithi178.stderr: -126> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: _check_auth_tickets 2024-06-28T15:23:20.875 INFO:tasks.workunit.client.0.smithi178.stderr: -125> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: handle_config config(0 keys) 2024-06-28T15:23:20.875 INFO:tasks.workunit.client.0.smithi178.stderr: -124> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:23:20.875 INFO:tasks.workunit.client.0.smithi178.stderr: -123> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:23:20.875 INFO:tasks.workunit.client.0.smithi178.stderr: -122> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 10 monclient: dump: 2024-06-28T15:23:20.875 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:23:20.875 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:23:20.875 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.875 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:23:20.875 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:23:20.875 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:23:20.876 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:23:20.876 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:23:20.876 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:23:20.876 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.876 INFO:tasks.workunit.client.0.smithi178.stderr: -121> 2024-06-28T15:23:20.506+0000 7fa2feffd640 4 set_mon_vals no callback set 2024-06-28T15:23:20.876 INFO:tasks.workunit.client.0.smithi178.stderr: -120> 2024-06-28T15:23:20.506+0000 7fa42206d980 5 monclient: authenticate success, global_id 6447 2024-06-28T15:23:20.876 INFO:tasks.workunit.client.0.smithi178.stderr: -119> 2024-06-28T15:23:20.506+0000 7fa42206d980 10 monclient: _renew_subs 2024-06-28T15:23:20.876 INFO:tasks.workunit.client.0.smithi178.stderr: -118> 2024-06-28T15:23:20.506+0000 7fa42206d980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.876 INFO:tasks.workunit.client.0.smithi178.stderr: -117> 2024-06-28T15:23:20.506+0000 7fa42206d980 10 monclient: _renew_subs 2024-06-28T15:23:20.876 INFO:tasks.workunit.client.0.smithi178.stderr: -116> 2024-06-28T15:23:20.506+0000 7fa42206d980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.876 INFO:tasks.workunit.client.0.smithi178.stderr: -115> 2024-06-28T15:23:20.506+0000 7fa42206d980 1 librados: init done 2024-06-28T15:23:20.876 INFO:tasks.workunit.client.0.smithi178.stderr: -114> 2024-06-28T15:23:20.506+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command cr dump hook 0x55d57194c218 2024-06-28T15:23:20.876 INFO:tasks.workunit.client.0.smithi178.stderr: -113> 2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.876 INFO:tasks.workunit.client.0.smithi178.stderr: -112> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -111> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -110> 2024-06-28T15:23:20.506+0000 7fa2fd7fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -109> 2024-06-28T15:23:20.506+0000 7fa41f37d640 10 monclient: get_auth_request con 0x7fa2e806f780 auth_method 0 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -108> 2024-06-28T15:23:20.506+0000 7fa42037f640 10 monclient: get_auth_request con 0x55d571961ad0 auth_method 0 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -107> 2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -106> 2024-06-28T15:23:20.506+0000 7fa42206d980 20 realm 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -105> 2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -104> 2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -103> 2024-06-28T15:23:20.506+0000 7fa42206d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -102> 2024-06-28T15:23:20.506+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -101> 2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -100> 2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -99> 2024-06-28T15:23:20.510+0000 7fa41fb7e640 10 monclient: get_auth_request con 0x55d571965d20 auth_method 0 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -98> 2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:20.877 INFO:tasks.workunit.client.0.smithi178.stderr: -97> 2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.878 INFO:tasks.workunit.client.0.smithi178.stderr: -96> 2024-06-28T15:23:20.510+0000 7fa41f37d640 10 monclient: get_auth_request con 0x55d571969560 auth_method 0 2024-06-28T15:23:20.878 INFO:tasks.workunit.client.0.smithi178.stderr: -95> 2024-06-28T15:23:20.510+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:20.878 INFO:tasks.workunit.client.0.smithi178.stderr: -94> 2024-06-28T15:23:20.510+0000 7fa42206d980 20 searching for the correct realm 2024-06-28T15:23:20.878 INFO:tasks.workunit.client.0.smithi178.stderr: -93> 2024-06-28T15:23:20.510+0000 7fa42037f640 10 monclient: get_auth_request con 0x7fa414008040 auth_method 0 2024-06-28T15:23:20.878 INFO:tasks.workunit.client.0.smithi178.stderr: -92> 2024-06-28T15:23:20.514+0000 7fa41fb7e640 10 monclient: get_auth_request con 0x7fa4180ab070 auth_method 0 2024-06-28T15:23:20.878 INFO:tasks.workunit.client.0.smithi178.stderr: -91> 2024-06-28T15:23:20.514+0000 7fa41f37d640 10 monclient: get_auth_request con 0x7fa410067030 auth_method 0 2024-06-28T15:23:20.878 INFO:tasks.workunit.client.0.smithi178.stderr: -90> 2024-06-28T15:23:20.518+0000 7fa42037f640 10 monclient: get_auth_request con 0x7fa4140343a0 auth_method 0 2024-06-28T15:23:20.878 INFO:tasks.workunit.client.0.smithi178.stderr: -89> 2024-06-28T15:23:20.518+0000 7fa42037f640 0 -- 172.21.15.178:0/1630651460 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fa4140343a0 msgr2=0x7fa414034800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:20.878 INFO:tasks.workunit.client.0.smithi178.stderr: -88> 2024-06-28T15:23:20.722+0000 7fa42037f640 10 monclient: get_auth_request con 0x7fa4140343a0 auth_method 0 2024-06-28T15:23:20.878 INFO:tasks.workunit.client.0.smithi178.stderr: -87> 2024-06-28T15:23:20.730+0000 7fa41fb7e640 10 monclient: get_auth_request con 0x7fa41404e8e0 auth_method 0 2024-06-28T15:23:20.878 INFO:tasks.workunit.client.0.smithi178.stderr: -86> 2024-06-28T15:23:20.734+0000 7fa41fb7e640 0 -- 172.21.15.178:0/1630651460 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55d571965d20 msgr2=0x55d571968130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:20.878 INFO:tasks.workunit.client.0.smithi178.stderr: -85> 2024-06-28T15:23:20.734+0000 7fa2fd7fa640 1 client.6447.objecter ms_handle_reset 0x55d571965d20 session 0x55d5719656a0 osd.5 2024-06-28T15:23:20.878 INFO:tasks.workunit.client.0.smithi178.stderr: -84> 2024-06-28T15:23:20.734+0000 7fa2fd7fa640 10 monclient: _renew_subs 2024-06-28T15:23:20.878 INFO:tasks.workunit.client.0.smithi178.stderr: -83> 2024-06-28T15:23:20.734+0000 7fa2fd7fa640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:23:20.879 INFO:tasks.workunit.client.0.smithi178.stderr: -82> 2024-06-28T15:23:20.734+0000 7fa41fb7e640 10 monclient: get_auth_request con 0x7fa2e807fda0 auth_method 0 2024-06-28T15:23:20.879 INFO:tasks.workunit.client.0.smithi178.stderr: -81> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:20.879 INFO:tasks.workunit.client.0.smithi178.stderr: -80> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:20.879 INFO:tasks.workunit.client.0.smithi178.stderr: -79> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:20.879 INFO:tasks.workunit.client.0.smithi178.stderr: -78> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:20.879 INFO:tasks.workunit.client.0.smithi178.stderr: -77> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:20.879 INFO:tasks.workunit.client.0.smithi178.stderr: -76> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:20.879 INFO:tasks.workunit.client.0.smithi178.stderr: -75> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.879 INFO:tasks.workunit.client.0.smithi178.stderr: -74> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.879 INFO:tasks.workunit.client.0.smithi178.stderr: -73> 2024-06-28T15:23:20.742+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.879 INFO:tasks.workunit.client.0.smithi178.stderr: -72> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:20.879 INFO:tasks.workunit.client.0.smithi178.stderr: -71> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.879 INFO:tasks.workunit.client.0.smithi178.stderr: -70> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:20.879 INFO:tasks.workunit.client.0.smithi178.stderr: -69> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 zone default found 2024-06-28T15:23:20.880 INFO:tasks.workunit.client.0.smithi178.stderr: -68> 2024-06-28T15:23:20.746+0000 7fa42206d980 4 Realm: () 2024-06-28T15:23:20.880 INFO:tasks.workunit.client.0.smithi178.stderr: -67> 2024-06-28T15:23:20.746+0000 7fa42206d980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:20.880 INFO:tasks.workunit.client.0.smithi178.stderr: -66> 2024-06-28T15:23:20.746+0000 7fa42206d980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:20.880 INFO:tasks.workunit.client.0.smithi178.stderr: -65> 2024-06-28T15:23:20.746+0000 7fa42206d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:20.880 INFO:tasks.workunit.client.0.smithi178.stderr: -64> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 zonegroup default 2024-06-28T15:23:20.880 INFO:tasks.workunit.client.0.smithi178.stderr: -63> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.880 INFO:tasks.workunit.client.0.smithi178.stderr: -62> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.880 INFO:tasks.workunit.client.0.smithi178.stderr: -61> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.880 INFO:tasks.workunit.client.0.smithi178.stderr: -60> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.880 INFO:tasks.workunit.client.0.smithi178.stderr: -59> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.880 INFO:tasks.workunit.client.0.smithi178.stderr: -58> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.880 INFO:tasks.workunit.client.0.smithi178.stderr: -57> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 started sync module instance, tier type = 2024-06-28T15:23:20.880 INFO:tasks.workunit.client.0.smithi178.stderr: -56> 2024-06-28T15:23:20.746+0000 7fa42206d980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:20.880 INFO:tasks.workunit.client.0.smithi178.stderr: -55> 2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=0 2024-06-28T15:23:20.881 INFO:tasks.workunit.client.0.smithi178.stderr: -54> 2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=1 2024-06-28T15:23:20.881 INFO:tasks.workunit.client.0.smithi178.stderr: -53> 2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=2 2024-06-28T15:23:20.881 INFO:tasks.workunit.client.0.smithi178.stderr: -52> 2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=3 2024-06-28T15:23:20.881 INFO:tasks.workunit.client.0.smithi178.stderr: -51> 2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=4 2024-06-28T15:23:20.881 INFO:tasks.workunit.client.0.smithi178.stderr: -50> 2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=5 2024-06-28T15:23:20.881 INFO:tasks.workunit.client.0.smithi178.stderr: -49> 2024-06-28T15:23:20.766+0000 7fa42206d980 20 add_watcher() i=6 2024-06-28T15:23:20.881 INFO:tasks.workunit.client.0.smithi178.stderr: -48> 2024-06-28T15:23:20.770+0000 7fa42206d980 20 add_watcher() i=7 2024-06-28T15:23:20.881 INFO:tasks.workunit.client.0.smithi178.stderr: -47> 2024-06-28T15:23:20.770+0000 7fa42206d980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:20.881 INFO:tasks.workunit.client.0.smithi178.stderr: -46> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding auth protocol: cephx 2024-06-28T15:23:20.881 INFO:tasks.workunit.client.0.smithi178.stderr: -45> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding auth protocol: cephx 2024-06-28T15:23:20.881 INFO:tasks.workunit.client.0.smithi178.stderr: -44> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding auth protocol: cephx 2024-06-28T15:23:20.881 INFO:tasks.workunit.client.0.smithi178.stderr: -43> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: secure 2024-06-28T15:23:20.881 INFO:tasks.workunit.client.0.smithi178.stderr: -42> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: crc 2024-06-28T15:23:20.881 INFO:tasks.workunit.client.0.smithi178.stderr: -41> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: secure 2024-06-28T15:23:20.882 INFO:tasks.workunit.client.0.smithi178.stderr: -40> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: crc 2024-06-28T15:23:20.882 INFO:tasks.workunit.client.0.smithi178.stderr: -39> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: secure 2024-06-28T15:23:20.882 INFO:tasks.workunit.client.0.smithi178.stderr: -38> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: crc 2024-06-28T15:23:20.882 INFO:tasks.workunit.client.0.smithi178.stderr: -37> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: crc 2024-06-28T15:23:20.882 INFO:tasks.workunit.client.0.smithi178.stderr: -36> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: secure 2024-06-28T15:23:20.882 INFO:tasks.workunit.client.0.smithi178.stderr: -35> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: crc 2024-06-28T15:23:20.882 INFO:tasks.workunit.client.0.smithi178.stderr: -34> 2024-06-28T15:23:20.774+0000 7fa2d67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:20.882 INFO:tasks.workunit.client.0.smithi178.stderr: -33> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: secure 2024-06-28T15:23:20.882 INFO:tasks.workunit.client.0.smithi178.stderr: -32> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: crc 2024-06-28T15:23:20.882 INFO:tasks.workunit.client.0.smithi178.stderr: -31> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 AuthRegistry(0x7ffcc88df4a0) adding con mode: secure 2024-06-28T15:23:20.882 INFO:tasks.workunit.client.0.smithi178.stderr: -30> 2024-06-28T15:23:20.774+0000 7fa42206d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:23:20.882 INFO:tasks.workunit.client.0.smithi178.stderr: -29> 2024-06-28T15:23:20.774+0000 7fa42206d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:20.882 INFO:tasks.workunit.client.0.smithi178.stderr: -28> 2024-06-28T15:23:20.774+0000 7fa42206d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:20.882 INFO:tasks.workunit.client.0.smithi178.stderr: -27> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command cache list hook 0x55d57193cf20 2024-06-28T15:23:20.883 INFO:tasks.workunit.client.0.smithi178.stderr: -26> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command cache inspect hook 0x55d57193cf20 2024-06-28T15:23:20.883 INFO:tasks.workunit.client.0.smithi178.stderr: -25> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command cache erase hook 0x55d57193cf20 2024-06-28T15:23:20.883 INFO:tasks.workunit.client.0.smithi178.stderr: -24> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command cache zap hook 0x55d57193cf20 2024-06-28T15:23:20.883 INFO:tasks.workunit.client.0.smithi178.stderr: -23> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 note: GC not initialized 2024-06-28T15:23:20.883 INFO:tasks.workunit.client.0.smithi178.stderr: -22> 2024-06-28T15:23:20.774+0000 7fa2d57e2640 20 reqs_thread_entry: start 2024-06-28T15:23:20.883 INFO:tasks.workunit.client.0.smithi178.stderr: -21> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command sync trace show hook 0x7fa4140088e0 2024-06-28T15:23:20.883 INFO:tasks.workunit.client.0.smithi178.stderr: -20> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command sync trace history hook 0x7fa4140088e0 2024-06-28T15:23:20.883 INFO:tasks.workunit.client.0.smithi178.stderr: -19> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command sync trace active hook 0x7fa4140088e0 2024-06-28T15:23:20.883 INFO:tasks.workunit.client.0.smithi178.stderr: -18> 2024-06-28T15:23:20.774+0000 7fa42206d980 5 asok(0x55d57166b5e0) register_command sync trace active_short hook 0x7fa4140088e0 2024-06-28T15:23:20.883 INFO:tasks.workunit.client.0.smithi178.stderr: -17> 2024-06-28T15:23:20.774+0000 7fa42206d980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:20.883 INFO:tasks.workunit.client.0.smithi178.stderr: -16> 2024-06-28T15:23:20.774+0000 7fa42206d980 20 Filter name: none 2024-06-28T15:23:20.883 INFO:tasks.workunit.client.0.smithi178.stderr: -15> 2024-06-28T15:23:20.774+0000 7fa2ce7fc640 20 reqs_thread_entry: start 2024-06-28T15:23:20.883 INFO:tasks.workunit.client.0.smithi178.stderr: -14> 2024-06-28T15:23:20.774+0000 7fa42206d980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-06-28T15:23:20.883 INFO:tasks.workunit.client.0.smithi178.stderr: -13> 2024-06-28T15:23:20.774+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.884 INFO:tasks.workunit.client.0.smithi178.stderr: -12> 2024-06-28T15:23:20.774+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:20.884 INFO:tasks.workunit.client.0.smithi178.stderr: -11> 2024-06-28T15:23:20.774+0000 7fa42206d980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-06-28T15:23:20.884 INFO:tasks.workunit.client.0.smithi178.stderr: -10> 2024-06-28T15:23:20.774+0000 7fa42206d980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-06-28T15:23:20.884 INFO:tasks.workunit.client.0.smithi178.stderr: -9> 2024-06-28T15:23:20.774+0000 7fa42206d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 : miss 2024-06-28T15:23:20.884 INFO:tasks.workunit.client.0.smithi178.stderr: -8> 2024-06-28T15:23:20.774+0000 7fa42206d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.884 INFO:tasks.workunit.client.0.smithi178.stderr: -7> 2024-06-28T15:23:20.778+0000 7fa42206d980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:20.884 INFO:tasks.workunit.client.0.smithi178.stderr: -6> 2024-06-28T15:23:20.778+0000 7fa42206d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 info.flags=0x17 2024-06-28T15:23:20.884 INFO:tasks.workunit.client.0.smithi178.stderr: -5> 2024-06-28T15:23:20.778+0000 7fa42206d980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 to cache LRU end 2024-06-28T15:23:20.884 INFO:tasks.workunit.client.0.smithi178.stderr: -4> 2024-06-28T15:23:20.778+0000 7fa42206d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:20.884 INFO:tasks.workunit.client.0.smithi178.stderr: -3> 2024-06-28T15:23:20.778+0000 7fa42206d980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:20.884 INFO:tasks.workunit.client.0.smithi178.stderr: -2> 2024-06-28T15:23:20.778+0000 7fa42206d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 2024-06-28T15:23:20.884 INFO:tasks.workunit.client.0.smithi178.stderr: -1> 2024-06-28T15:23:20.782+0000 7fa42206d980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fa42206d980 time 2024-06-28T15:23:20.787106+0000 2024-06-28T15:23:20.884 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:23:20.884 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.885 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:23:20.885 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fa425944c91] 2024-06-28T15:23:20.885 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fa425944e45] 2024-06-28T15:23:20.885 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d5700124b1] 2024-06-28T15:23:20.885 INFO:tasks.workunit.client.0.smithi178.stderr: 4: radosgw-admin(+0x9acc0a) [0x55d57001ec0a] 2024-06-28T15:23:20.885 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x260) [0x55d5700239e0] 2024-06-28T15:23:20.885 INFO:tasks.workunit.client.0.smithi178.stderr: 6: main() 2024-06-28T15:23:20.885 INFO:tasks.workunit.client.0.smithi178.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa4247add90] 2024-06-28T15:23:20.885 INFO:tasks.workunit.client.0.smithi178.stderr: 8: __libc_start_main() 2024-06-28T15:23:20.885 INFO:tasks.workunit.client.0.smithi178.stderr: 9: _start() 2024-06-28T15:23:20.885 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.885 INFO:tasks.workunit.client.0.smithi178.stderr: 0> 2024-06-28T15:23:20.786+0000 7fa42206d980 -1 *** Caught signal (Aborted) ** 2024-06-28T15:23:20.885 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7fa42206d980 thread_name:radosgw-admin 2024-06-28T15:23:20.885 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.886 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:23:20.886 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fa4247c6520] 2024-06-28T15:23:20.886 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:23:20.886 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:23:20.886 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:23:20.886 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fa425944ceb] 2024-06-28T15:23:20.886 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fa425944e45] 2024-06-28T15:23:20.886 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55d5700124b1] 2024-06-28T15:23:20.886 INFO:tasks.workunit.client.0.smithi178.stderr: 8: radosgw-admin(+0x9acc0a) [0x55d57001ec0a] 2024-06-28T15:23:20.886 INFO:tasks.workunit.client.0.smithi178.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x260) [0x55d5700239e0] 2024-06-28T15:23:20.886 INFO:tasks.workunit.client.0.smithi178.stderr: 10: main() 2024-06-28T15:23:20.886 INFO:tasks.workunit.client.0.smithi178.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fa4247add90] 2024-06-28T15:23:20.886 INFO:tasks.workunit.client.0.smithi178.stderr: 12: __libc_start_main() 2024-06-28T15:23:20.886 INFO:tasks.workunit.client.0.smithi178.stderr: 13: _start() 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr:--- logging levels --- 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 none 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 lockdep 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 context 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 crush 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_balancer 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_locker 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log_expire 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_migrator 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr: 3/ 5 mds_quiesce 2024-06-28T15:23:20.887 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 buffer 2024-06-28T15:23:20.888 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 timer 2024-06-28T15:23:20.888 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 filer 2024-06-28T15:23:20.888 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 striper 2024-06-28T15:23:20.888 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 objecter 2024-06-28T15:23:20.888 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rados 2024-06-28T15:23:20.888 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd 2024-06-28T15:23:20.888 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_mirror 2024-06-28T15:23:20.888 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_replay 2024-06-28T15:23:20.888 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_pwl 2024-06-28T15:23:20.888 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 journaler 2024-06-28T15:23:20.888 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objectcacher 2024-06-28T15:23:20.888 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 immutable_obj_cache 2024-06-28T15:23:20.888 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 client 2024-06-28T15:23:20.888 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 osd 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 optracker 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objclass 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 filestore 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 journal 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 ms 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mon 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 0/10 monc 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 paxos 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 tp 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 auth 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 crypto 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 finisher 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 reserver 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 heartbeatmap 2024-06-28T15:23:20.889 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 perfcounter 2024-06-28T15:23:20.890 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw 2024-06-28T15:23:20.890 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_sync 2024-06-28T15:23:20.890 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_datacache 2024-06-28T15:23:20.890 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_access 2024-06-28T15:23:20.890 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_dbstore 2024-06-28T15:23:20.890 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_flight 2024-06-28T15:23:20.890 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_lifecycle 2024-06-28T15:23:20.890 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw_notification 2024-06-28T15:23:20.890 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 javaclient 2024-06-28T15:23:20.890 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 asok 2024-06-28T15:23:20.890 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 throttle 2024-06-28T15:23:20.890 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 refs 2024-06-28T15:23:20.890 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 compressor 2024-06-28T15:23:20.890 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluestore 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluefs 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 bdev 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 kstore 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 4/ 5 rocksdb 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 fuse 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 2/ 5 mgr 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mgrc 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 dpdk 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 eventtrace 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 prioritycache 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 test 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephfs_mirror 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephsqlite 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 crimson_interrupt 2024-06-28T15:23:20.891 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore 2024-06-28T15:23:20.892 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_onode 2024-06-28T15:23:20.892 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_odata 2024-06-28T15:23:20.892 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_omap 2024-06-28T15:23:20.892 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_tm 2024-06-28T15:23:20.892 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_t 2024-06-28T15:23:20.892 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cleaner 2024-06-28T15:23:20.892 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_epm 2024-06-28T15:23:20.892 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_lba 2024-06-28T15:23:20.892 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_fixedkv_tree 2024-06-28T15:23:20.892 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cache 2024-06-28T15:23:20.892 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_journal 2024-06-28T15:23:20.892 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_device 2024-06-28T15:23:20.892 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_backref 2024-06-28T15:23:20.892 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 alienstore 2024-06-28T15:23:20.893 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mclock 2024-06-28T15:23:20.893 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cyanstore 2024-06-28T15:23:20.893 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 ceph_exporter 2024-06-28T15:23:20.893 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 memstore 2024-06-28T15:23:20.893 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 trace 2024-06-28T15:23:20.893 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 ceph_dedup 2024-06-28T15:23:20.893 INFO:tasks.workunit.client.0.smithi178.stderr: -2/-2 (syslog threshold) 2024-06-28T15:23:20.893 INFO:tasks.workunit.client.0.smithi178.stderr: 99/99 (stderr threshold) 2024-06-28T15:23:20.893 INFO:tasks.workunit.client.0.smithi178.stderr:--- pthread ID / name mapping for recent threads --- 2024-06-28T15:23:20.893 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa2ce7fc640 / http_manager 2024-06-28T15:23:20.893 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa2d57e2640 / http_manager 2024-06-28T15:23:20.893 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa2d67e4640 / rgw_dt_lg_renew 2024-06-28T15:23:20.893 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa2fd7fa640 / ms_dispatch 2024-06-28T15:23:20.893 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa2feffd640 / io_context_pool 2024-06-28T15:23:20.894 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa3057fa640 / ms_dispatch 2024-06-28T15:23:20.894 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa307fff640 / ms_dispatch 2024-06-28T15:23:20.894 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa30de0b640 / io_context_pool 2024-06-28T15:23:20.894 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa41eb7c640 / io_context_pool 2024-06-28T15:23:20.894 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa41f37d640 / msgr-worker-2 2024-06-28T15:23:20.894 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa41fb7e640 / msgr-worker-1 2024-06-28T15:23:20.894 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa42037f640 / msgr-worker-0 2024-06-28T15:23:20.894 INFO:tasks.workunit.client.0.smithi178.stderr: 7fa42206d980 / radosgw-admin 2024-06-28T15:23:20.894 INFO:tasks.workunit.client.0.smithi178.stderr: max_recent 500 2024-06-28T15:23:20.894 INFO:tasks.workunit.client.0.smithi178.stderr: max_new 1000 2024-06-28T15:23:20.894 INFO:tasks.workunit.client.0.smithi178.stderr: log_file /var/lib/ceph/crash/2024-06-28T15:23:20.789569Z_fa831b5e-8806-4ff5-ab21-f7ed29f78afc/log 2024-06-28T15:23:20.894 INFO:tasks.workunit.client.0.smithi178.stderr:--- end dump of recent events --- 2024-06-28T15:23:20.897 INFO:tasks.workunit.client.0.smithi178.stderr:Aborted 2024-06-28T15:23:20.897 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-06-28T15:23:20.897 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:20.897 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:20.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.874+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.874+0000 7f4bdde13980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.874+0000 7f4bdde13980 20 realm 2024-06-28T15:23:20.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.874+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.874+0000 7f4bdde13980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.874+0000 7f4bdde13980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:20.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.874+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.874+0000 7f4bdde13980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.874+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.878+0000 7f4bdde13980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:20.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.878+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.878+0000 7f4bdde13980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:20.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.878+0000 7f4bdde13980 20 searching for the correct realm 2024-06-28T15:23:20.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.882+0000 7f4bdc125640 0 -- 172.21.15.178:0/2329404729 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x561589cf9190 msgr2=0x561589cfb580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:20.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.890+0000 7f4bdb123640 0 -- 172.21.15.178:0/2329404729 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f4bcc008070 msgr2=0x7f4bcc008450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:20.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:20.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:20.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:20.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:20.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 zone default found 2024-06-28T15:23:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 4 Realm: () 2024-06-28T15:23:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 zonegroup default 2024-06-28T15:23:20.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:20.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 started sync module instance, tier type = 2024-06-28T15:23:20.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.898+0000 7f4bdde13980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:20.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.914+0000 7f4bdde13980 20 add_watcher() i=0 2024-06-28T15:23:20.901 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.914+0000 7f4bdde13980 20 add_watcher() i=1 2024-06-28T15:23:20.901 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.914+0000 7f4bdde13980 20 add_watcher() i=2 2024-06-28T15:23:20.901 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.914+0000 7f4bdde13980 20 add_watcher() i=3 2024-06-28T15:23:20.901 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.914+0000 7f4bdde13980 20 add_watcher() i=4 2024-06-28T15:23:20.901 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.914+0000 7f4bdde13980 20 add_watcher() i=5 2024-06-28T15:23:20.901 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.914+0000 7f4bdde13980 20 add_watcher() i=6 2024-06-28T15:23:20.901 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.914+0000 7f4bdde13980 20 add_watcher() i=7 2024-06-28T15:23:20.901 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.914+0000 7f4bdde13980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:20.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.918+0000 7f4a91fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:20.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.918+0000 7f4bdde13980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:20.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.918+0000 7f4bdde13980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:20.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.918+0000 7f4bdde13980 5 note: GC not initialized 2024-06-28T15:23:20.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.918+0000 7f4a90fe1640 20 reqs_thread_entry: start 2024-06-28T15:23:20.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.918+0000 7f4bdde13980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:20.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.918+0000 7f4bdde13980 20 Filter name: none 2024-06-28T15:23:20.903 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.918+0000 7f4a0a7fc640 20 reqs_thread_entry: start 2024-06-28T15:23:20.903 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.918+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.903 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.918+0000 7f4bdde13980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:20.903 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.918+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.904 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.922+0000 7f4bdde13980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:20.904 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.922+0000 7f4bdde13980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:20.904 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.922+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.905 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.922+0000 7f4bdde13980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:20.905 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.922+0000 7f4bdde13980 20 rados->read ofs=0 len=0 2024-06-28T15:23:20.906 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.922+0000 7f4bdde13980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:20.906 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.922+0000 7f4bdde13980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:20.908 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.926+0000 7f4a91fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:20.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.930+0000 7f4bdde13980 20 remove_watcher() i=0 2024-06-28T15:23:20.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.930+0000 7f4bdde13980 2 removed watcher, disabling cache 2024-06-28T15:23:20.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.930+0000 7f4bdde13980 20 remove_watcher() i=1 2024-06-28T15:23:20.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.934+0000 7f4bdde13980 20 remove_watcher() i=2 2024-06-28T15:23:20.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.934+0000 7f4bdde13980 20 remove_watcher() i=3 2024-06-28T15:23:20.921 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.938+0000 7f4bdde13980 20 remove_watcher() i=4 2024-06-28T15:23:20.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.938+0000 7f4bdde13980 20 remove_watcher() i=5 2024-06-28T15:23:20.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.942+0000 7f4bdde13980 20 remove_watcher() i=6 2024-06-28T15:23:20.924 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:20.942+0000 7f4bdde13980 20 remove_watcher() i=7 2024-06-28T15:23:20.938 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:20.938 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:20.938 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: abort-at-set-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.22 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:23:20.938 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:21.037 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-06-28T15:23:21.073 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:21.073 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:21.115 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.130+0000 7f8c5e246980 20 rados->read ofs=0 len=0 2024-06-28T15:23:21.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.134+0000 7f8c5e246980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:21.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.134+0000 7f8c5e246980 20 realm 2024-06-28T15:23:21.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.134+0000 7f8c5e246980 20 rados->read ofs=0 len=0 2024-06-28T15:23:21.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.134+0000 7f8c5e246980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:21.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.134+0000 7f8c5e246980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:21.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.134+0000 7f8c5e246980 20 rados->read ofs=0 len=0 2024-06-28T15:23:21.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.134+0000 7f8c5e246980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:21.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.134+0000 7f8c5e246980 20 rados->read ofs=0 len=0 2024-06-28T15:23:21.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.134+0000 7f8c5e246980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:21.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.134+0000 7f8c5e246980 20 rados->read ofs=0 len=0 2024-06-28T15:23:21.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.138+0000 7f8c5e246980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:21.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.138+0000 7f8c5e246980 20 searching for the correct realm 2024-06-28T15:23:21.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.142+0000 7f8c5c558640 0 -- 172.21.15.178:0/2885457881 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f8c50008070 msgr2=0x7f8c50038920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:23:21.342 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.358+0000 7f8c5bd57640 0 -- 172.21.15.178:0/2885457881 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55fc60734e80 msgr2=0x55fc60737270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:21.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.366+0000 7f8c5e246980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:21.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.366+0000 7f8c5e246980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:21.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.366+0000 7f8c5e246980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:21.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.366+0000 7f8c5e246980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:21.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.366+0000 7f8c5e246980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:21.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.366+0000 7f8c5e246980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:21.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.366+0000 7f8c5e246980 20 rados->read ofs=0 len=0 2024-06-28T15:23:21.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.366+0000 7f8c5e246980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:21.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.366+0000 7f8c5e246980 20 rados->read ofs=0 len=0 2024-06-28T15:23:21.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:21.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 20 rados->read ofs=0 len=0 2024-06-28T15:23:21.352 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:21.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 20 zone default found 2024-06-28T15:23:21.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 4 Realm: () 2024-06-28T15:23:21.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:21.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:21.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:21.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 20 zonegroup default 2024-06-28T15:23:21.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 20 rados->read ofs=0 len=0 2024-06-28T15:23:21.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:21.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 20 rados->read ofs=0 len=0 2024-06-28T15:23:21.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:21.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 20 rados->read ofs=0 len=0 2024-06-28T15:23:21.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:21.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 20 started sync module instance, tier type = 2024-06-28T15:23:21.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.370+0000 7f8c5e246980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:21.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.394+0000 7f8c5e246980 20 add_watcher() i=0 2024-06-28T15:23:21.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.394+0000 7f8c5e246980 20 add_watcher() i=1 2024-06-28T15:23:21.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.394+0000 7f8c5e246980 20 add_watcher() i=2 2024-06-28T15:23:21.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.394+0000 7f8c5e246980 20 add_watcher() i=3 2024-06-28T15:23:21.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.394+0000 7f8c5e246980 20 add_watcher() i=4 2024-06-28T15:23:21.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.394+0000 7f8c5e246980 20 add_watcher() i=5 2024-06-28T15:23:21.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.394+0000 7f8c5e246980 20 add_watcher() i=6 2024-06-28T15:23:21.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.398+0000 7f8c5e246980 20 add_watcher() i=7 2024-06-28T15:23:21.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.398+0000 7f8c5e246980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:21.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.402+0000 7f8b11fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:21.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.402+0000 7f8c5e246980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:21.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.402+0000 7f8c5e246980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:21.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.402+0000 7f8c5e246980 5 note: GC not initialized 2024-06-28T15:23:21.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.402+0000 7f8b10fe1640 20 reqs_thread_entry: start 2024-06-28T15:23:21.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.402+0000 7f8c5e246980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:21.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.402+0000 7f8c5e246980 20 Filter name: none 2024-06-28T15:23:21.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.402+0000 7f8a8a7fc640 20 reqs_thread_entry: start 2024-06-28T15:23:21.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.402+0000 7f8c5e246980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-06-28T15:23:21.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.402+0000 7f8c5e246980 20 rados->read ofs=0 len=0 2024-06-28T15:23:21.388 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.406+0000 7f8c5e246980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:21.388 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.406+0000 7f8c5e246980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-06-28T15:23:21.388 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.406+0000 7f8c5e246980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-06-28T15:23:21.388 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.406+0000 7f8c5e246980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 : miss 2024-06-28T15:23:21.388 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.406+0000 7f8c5e246980 20 rados->read ofs=0 len=0 2024-06-28T15:23:21.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.406+0000 7f8c5e246980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:21.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.406+0000 7f8c5e246980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 info.flags=0x17 2024-06-28T15:23:21.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.406+0000 7f8c5e246980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 to cache LRU end 2024-06-28T15:23:21.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.406+0000 7f8c5e246980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:21.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.406+0000 7f8c5e246980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:21.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.406+0000 7f8c5e246980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 2024-06-28T15:23:21.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.406+0000 7f8c5e246980 0 INFO: RGWReshardLock::lock found lock on abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 to be held by another RGW process; skipping for now 2024-06-28T15:23:21.392 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.410+0000 7f8b11fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:21.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.414+0000 7f8c5e246980 20 remove_watcher() i=0 2024-06-28T15:23:21.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.414+0000 7f8c5e246980 2 removed watcher, disabling cache 2024-06-28T15:23:21.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.414+0000 7f8c5e246980 20 remove_watcher() i=1 2024-06-28T15:23:21.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.418+0000 7f8c5e246980 20 remove_watcher() i=2 2024-06-28T15:23:21.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.418+0000 7f8c5e246980 20 remove_watcher() i=3 2024-06-28T15:23:21.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.422+0000 7f8c5e246980 20 remove_watcher() i=4 2024-06-28T15:23:21.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.422+0000 7f8c5e246980 20 remove_watcher() i=5 2024-06-28T15:23:21.407 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.426+0000 7f8c5e246980 20 remove_watcher() i=6 2024-06-28T15:23:21.409 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:21.426+0000 7f8c5e246980 20 remove_watcher() i=7 2024-06-28T15:23:21.423 INFO:tasks.workunit.client.0.smithi178.stderr:waiting 30 seconds for reshard lock to expire... 2024-06-28T15:23:51.441 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-06-28T15:23:51.477 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:51.477 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:51.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.538+0000 7f6a56dc8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.542+0000 7f6a56dc8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:51.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.542+0000 7f6a56dc8980 20 realm 2024-06-28T15:23:51.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.542+0000 7f6a56dc8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.542+0000 7f6a56dc8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:51.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.542+0000 7f6a56dc8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:51.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.542+0000 7f6a56dc8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.542+0000 7f6a56dc8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:51.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.542+0000 7f6a56dc8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.542+0000 7f6a56dc8980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:51.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.542+0000 7f6a56dc8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.542+0000 7f6a56dc8980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:51.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.542+0000 7f6a56dc8980 20 searching for the correct realm 2024-06-28T15:23:51.533 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.550+0000 7f6a540d8640 0 -- 172.21.15.178:0/2305019360 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f6a4400b930 msgr2=0x7f6a4403f230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:51.539 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.554+0000 7f6a540d8640 0 -- 172.21.15.178:0/2305019360 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f692007fda0 msgr2=0x7f69200821d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:51.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.562+0000 7f6a56dc8980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:51.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.562+0000 7f6a56dc8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:51.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.562+0000 7f6a56dc8980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:51.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.562+0000 7f6a56dc8980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:51.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.562+0000 7f6a56dc8980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:51.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.562+0000 7f6a56dc8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:51.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.562+0000 7f6a56dc8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.562+0000 7f6a56dc8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:51.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.562+0000 7f6a56dc8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.562+0000 7f6a56dc8980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:51.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.562+0000 7f6a56dc8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:51.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 20 zone default found 2024-06-28T15:23:51.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 4 Realm: () 2024-06-28T15:23:51.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:51.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:51.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:51.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 20 zonegroup default 2024-06-28T15:23:51.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:51.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:51.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:51.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 20 started sync module instance, tier type = 2024-06-28T15:23:51.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.566+0000 7f6a56dc8980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:51.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.582+0000 7f6a56dc8980 20 add_watcher() i=0 2024-06-28T15:23:51.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.582+0000 7f6a56dc8980 20 add_watcher() i=1 2024-06-28T15:23:51.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.582+0000 7f6a56dc8980 20 add_watcher() i=2 2024-06-28T15:23:51.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.582+0000 7f6a56dc8980 20 add_watcher() i=3 2024-06-28T15:23:51.565 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.582+0000 7f6a56dc8980 20 add_watcher() i=4 2024-06-28T15:23:51.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.582+0000 7f6a56dc8980 20 add_watcher() i=5 2024-06-28T15:23:51.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.582+0000 7f6a56dc8980 20 add_watcher() i=6 2024-06-28T15:23:51.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.582+0000 7f6a56dc8980 20 add_watcher() i=7 2024-06-28T15:23:51.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.582+0000 7f6a56dc8980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:51.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f690afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:51.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f6a56dc8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:51.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f6a56dc8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:51.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f6a56dc8980 5 note: GC not initialized 2024-06-28T15:23:51.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f6909fe3640 20 reqs_thread_entry: start 2024-06-28T15:23:51.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f6a56dc8980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:51.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f6a56dc8980 20 Filter name: none 2024-06-28T15:23:51.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f687b7fe640 20 reqs_thread_entry: start 2024-06-28T15:23:51.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f6a56dc8980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-06-28T15:23:51.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f6a56dc8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f6a56dc8980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:51.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f6a56dc8980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-06-28T15:23:51.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f6a56dc8980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-06-28T15:23:51.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f6a56dc8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 : miss 2024-06-28T15:23:51.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.586+0000 7f6a56dc8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.590+0000 7f6a56dc8980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:51.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.590+0000 7f6a56dc8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 info.flags=0x17 2024-06-28T15:23:51.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.590+0000 7f6a56dc8980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 to cache LRU end 2024-06-28T15:23:51.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.590+0000 7f6a56dc8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:51.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.590+0000 7f6a56dc8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:51.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.590+0000 7f6a56dc8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 2024-06-28T15:23:51.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.598+0000 7f6a56dc8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 info.flags=0x17 2024-06-28T15:23:51.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.598+0000 7f6a56dc8980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 to cache LRU end 2024-06-28T15:23:51.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.598+0000 7f6a56dc8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:51.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.598+0000 7f6a56dc8980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22, ofs0, ns] 2024-06-28T15:23:51.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.598+0000 7f692bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595081 cookie 94191211235440 notifier 6507 bl.length()=1001 2024-06-28T15:23:51.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.598+0000 7f692bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 info.flags=0x17 2024-06-28T15:23:51.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.598+0000 7f692bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 to cache LRU end 2024-06-28T15:23:51.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.598+0000 7f692bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:51.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.598+0000 7f6a56dc8980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.22]): orig_sources= new_sources= 2024-06-28T15:23:51.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.598+0000 7f6a56dc8980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.22]): potential sources added=[] removed=[] 2024-06-28T15:23:51.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.598+0000 7f6a56dc8980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.22]): orig_dests= new_dests= 2024-06-28T15:23:51.582 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.598+0000 7f6a56dc8980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.22]): potential dests added=[] removed=[] 2024-06-28T15:23:51.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.602+0000 7f6a56dc8980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.22.1.0 2024-06-28T15:23:51.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.602+0000 7f6a56dc8980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.22.1.0 2024-06-28T15:23:51.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.602+0000 7f6a56dc8980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.22.1.1 2024-06-28T15:23:51.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.602+0000 7f6a56dc8980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.22.1.1 2024-06-28T15:23:51.585 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.602+0000 7f6a56dc8980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.22.0 2024-06-28T15:23:51.590 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.606+0000 7f6a56dc8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 : hit (requested=0x17, cached=0x17) 2024-06-28T15:23:51.590 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.606+0000 7f6a56dc8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 2024-06-28T15:23:51.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.610+0000 7f6a56dc8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 info.flags=0x17 2024-06-28T15:23:51.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.610+0000 7f6a56dc8980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 to cache LRU end 2024-06-28T15:23:51.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.610+0000 7f6a56dc8980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:51.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.610+0000 7f6a56dc8980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22, ofs0, ns] 2024-06-28T15:23:51.595 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.610+0000 7f692b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595082 cookie 94191211235440 notifier 6507 bl.length()=969 2024-06-28T15:23:51.595 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.610+0000 7f692b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 info.flags=0x17 2024-06-28T15:23:51.595 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.610+0000 7f692b7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.22 to cache LRU end 2024-06-28T15:23:51.595 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.610+0000 7f692b7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:51.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.614+0000 7f6a56dc8980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.22]): orig_sources= new_sources= 2024-06-28T15:23:51.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.614+0000 7f6a56dc8980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.22]): potential sources added=[] removed=[] 2024-06-28T15:23:51.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.614+0000 7f6a56dc8980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.22]): orig_dests= new_dests= 2024-06-28T15:23:51.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.614+0000 7f6a56dc8980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.22]): potential dests added=[] removed=[] 2024-06-28T15:23:51.599 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.618+0000 7f6a56dc8980 1 execute INFO: reshard of bucket "abort-at-set-target-layout" from 2 shards to 2 shards completed successfully 2024-06-28T15:23:51.602 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.618+0000 7f690afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:51.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.626+0000 7f6a56dc8980 20 remove_watcher() i=0 2024-06-28T15:23:51.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.626+0000 7f6a56dc8980 2 removed watcher, disabling cache 2024-06-28T15:23:51.610 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.626+0000 7f6a56dc8980 20 remove_watcher() i=1 2024-06-28T15:23:51.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.630+0000 7f6a56dc8980 20 remove_watcher() i=2 2024-06-28T15:23:51.614 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.630+0000 7f6a56dc8980 20 remove_watcher() i=3 2024-06-28T15:23:51.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.634+0000 7f6a56dc8980 20 remove_watcher() i=4 2024-06-28T15:23:51.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.634+0000 7f6a56dc8980 20 remove_watcher() i=5 2024-06-28T15:23:51.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.634+0000 7f6a56dc8980 20 remove_watcher() i=6 2024-06-28T15:23:51.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.638+0000 7f6a56dc8980 20 remove_watcher() i=7 2024-06-28T15:23:51.634 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-06-28T15:23:51.660 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:51.661 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:51.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.722+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.705 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.722+0000 7fb59c8b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:51.705 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.722+0000 7fb59c8b1980 20 realm 2024-06-28T15:23:51.705 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.722+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.706 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.722+0000 7fb59c8b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:51.706 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.722+0000 7fb59c8b1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:51.706 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.722+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.706 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.722+0000 7fb59c8b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:51.706 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.722+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.707 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.726+0000 7fb59c8b1980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:51.708 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.726+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.709 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.726+0000 7fb59c8b1980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:51.709 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.726+0000 7fb59c8b1980 20 searching for the correct realm 2024-06-28T15:23:51.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.730+0000 7fb59abc3640 0 -- 172.21.15.178:0/1411160794 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fb590029070 msgr2=0x7fb59004e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:23:51.929 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.946+0000 7fb59a3c2640 0 -- 172.21.15.178:0/1411160794 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55909dd2da20 msgr2=0x55909dd2fe10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:51.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:51.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:51.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:51.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:51.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:51.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:51.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:51.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:51.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:51.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 zone default found 2024-06-28T15:23:51.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 4 Realm: () 2024-06-28T15:23:51.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:51.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:51.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:51.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 zonegroup default 2024-06-28T15:23:51.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:51.940 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.954+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.940 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.958+0000 7fb59c8b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:51.940 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.958+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.958+0000 7fb59c8b1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:51.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.958+0000 7fb59c8b1980 20 started sync module instance, tier type = 2024-06-28T15:23:51.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.958+0000 7fb59c8b1980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:51.964 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.982+0000 7fb59c8b1980 20 add_watcher() i=0 2024-06-28T15:23:51.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.982+0000 7fb59c8b1980 20 add_watcher() i=1 2024-06-28T15:23:51.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.982+0000 7fb59c8b1980 20 add_watcher() i=2 2024-06-28T15:23:51.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.982+0000 7fb59c8b1980 20 add_watcher() i=3 2024-06-28T15:23:51.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.982+0000 7fb59c8b1980 20 add_watcher() i=4 2024-06-28T15:23:51.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.982+0000 7fb59c8b1980 20 add_watcher() i=5 2024-06-28T15:23:51.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.982+0000 7fb59c8b1980 20 add_watcher() i=6 2024-06-28T15:23:51.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.982+0000 7fb59c8b1980 20 add_watcher() i=7 2024-06-28T15:23:51.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.982+0000 7fb59c8b1980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:51.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.986+0000 7fb450fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:51.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.990+0000 7fb59c8b1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:51.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.990+0000 7fb59c8b1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:51.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.990+0000 7fb59c8b1980 5 note: GC not initialized 2024-06-28T15:23:51.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.990+0000 7fb3cb7fe640 20 reqs_thread_entry: start 2024-06-28T15:23:51.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.990+0000 7fb59c8b1980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:51.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.990+0000 7fb59c8b1980 20 Filter name: none 2024-06-28T15:23:51.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.990+0000 7fb3c97fa640 20 reqs_thread_entry: start 2024-06-28T15:23:51.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.990+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.990+0000 7fb59c8b1980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:51.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.990+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.975 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.990+0000 7fb59c8b1980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:51.975 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.990+0000 7fb59c8b1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:51.975 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.990+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.994+0000 7fb59c8b1980 20 rados_obj.operate() r=0 bl.length=218 2024-06-28T15:23:51.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.994+0000 7fb59c8b1980 20 rados->read ofs=0 len=0 2024-06-28T15:23:51.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.994+0000 7fb59c8b1980 20 rados_obj.operate() r=0 bl.length=432 2024-06-28T15:23:51.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.994+0000 7fb59c8b1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:51.980 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:51.998+0000 7fb450fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:51.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:52.002+0000 7fb59c8b1980 20 remove_watcher() i=0 2024-06-28T15:23:51.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:52.002+0000 7fb59c8b1980 2 removed watcher, disabling cache 2024-06-28T15:23:51.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:52.006+0000 7fb59c8b1980 20 remove_watcher() i=1 2024-06-28T15:23:51.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:52.006+0000 7fb59c8b1980 20 remove_watcher() i=2 2024-06-28T15:23:51.992 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:52.010+0000 7fb59c8b1980 20 remove_watcher() i=3 2024-06-28T15:23:51.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:52.010+0000 7fb59c8b1980 20 remove_watcher() i=4 2024-06-28T15:23:51.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:52.014+0000 7fb59c8b1980 20 remove_watcher() i=5 2024-06-28T15:23:51.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:52.014+0000 7fb59c8b1980 20 remove_watcher() i=6 2024-06-28T15:23:51.999 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:52.014+0000 7fb59c8b1980 20 remove_watcher() i=7 2024-06-28T15:23:52.015 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:52.015 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:52.015 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: abort-at-set-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.22 num_objects: 19 size_kb: 1 num_shards: 2 2024-06-28T15:23:52.016 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:52.820 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: reshard bucket with EIO injected at block_writes 2024-06-28T15:23:52.821 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:54.972 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-06-28T15:23:55.004 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:55.004 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:55.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.065+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.065+0000 7fa9e2bb8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.065+0000 7fa9e2bb8980 20 realm 2024-06-28T15:23:55.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.065+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.065+0000 7fa9e2bb8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.065+0000 7fa9e2bb8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:55.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.065+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.065+0000 7fa9e2bb8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.051 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.065+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.069+0000 7fa9e2bb8980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:55.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.069+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.069+0000 7fa9e2bb8980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:55.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.069+0000 7fa9e2bb8980 20 searching for the correct realm 2024-06-28T15:23:55.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.077+0000 7fa9e0eca640 0 -- 172.21.15.178:0/2699658771 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x559fa72efec0 msgr2=0x559fa72f2360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:55.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.081+0000 7fa9dfec8640 0 -- 172.21.15.178:0/2699658771 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fa9d00702e0 msgr2=0x7fa9d0009d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:23:55.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.089+0000 7fa9e2bb8980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:55.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.089+0000 7fa9e2bb8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:55.074 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.089+0000 7fa9e2bb8980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:55.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.089+0000 7fa9e2bb8980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:55.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.089+0000 7fa9e2bb8980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:55.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.089+0000 7fa9e2bb8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:55.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.089+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.089+0000 7fa9e2bb8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.089+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.089+0000 7fa9e2bb8980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:55.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.089+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:55.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 20 zone default found 2024-06-28T15:23:55.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 4 Realm: () 2024-06-28T15:23:55.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:55.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:55.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:55.076 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 20 zonegroup default 2024-06-28T15:23:55.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 20 started sync module instance, tier type = 2024-06-28T15:23:55.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.093+0000 7fa9e2bb8980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:55.093 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.109+0000 7fa9e2bb8980 20 add_watcher() i=0 2024-06-28T15:23:55.093 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.109+0000 7fa9e2bb8980 20 add_watcher() i=1 2024-06-28T15:23:55.093 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.109+0000 7fa9e2bb8980 20 add_watcher() i=2 2024-06-28T15:23:55.093 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.109+0000 7fa9e2bb8980 20 add_watcher() i=3 2024-06-28T15:23:55.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.109+0000 7fa9e2bb8980 20 add_watcher() i=4 2024-06-28T15:23:55.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.109+0000 7fa9e2bb8980 20 add_watcher() i=5 2024-06-28T15:23:55.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.109+0000 7fa9e2bb8980 20 add_watcher() i=6 2024-06-28T15:23:55.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.109+0000 7fa9e2bb8980 20 add_watcher() i=7 2024-06-28T15:23:55.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.109+0000 7fa9e2bb8980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:55.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.113+0000 7fa896fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:55.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.113+0000 7fa9e2bb8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:55.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.113+0000 7fa9e2bb8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:55.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.113+0000 7fa9e2bb8980 5 note: GC not initialized 2024-06-28T15:23:55.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.113+0000 7fa895fdb640 20 reqs_thread_entry: start 2024-06-28T15:23:55.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.113+0000 7fa9e2bb8980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:55.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.113+0000 7fa9e2bb8980 20 Filter name: none 2024-06-28T15:23:55.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.113+0000 7fa8077fe640 20 reqs_thread_entry: start 2024-06-28T15:23:55.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.113+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.113+0000 7fa9e2bb8980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:23:55.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.113+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.117+0000 7fa9e2bb8980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:23:55.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.117+0000 7fa9e2bb8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:55.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.117+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.101 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.117+0000 7fa9e2bb8980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:23:55.101 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.117+0000 7fa9e2bb8980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.117+0000 7fa9e2bb8980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:23:55.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.117+0000 7fa9e2bb8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:55.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.121+0000 7fa896fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:55.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.125+0000 7fa9e2bb8980 20 remove_watcher() i=0 2024-06-28T15:23:55.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.125+0000 7fa9e2bb8980 2 removed watcher, disabling cache 2024-06-28T15:23:55.112 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.129+0000 7fa9e2bb8980 20 remove_watcher() i=1 2024-06-28T15:23:55.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.129+0000 7fa9e2bb8980 20 remove_watcher() i=2 2024-06-28T15:23:55.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.133+0000 7fa9e2bb8980 20 remove_watcher() i=3 2024-06-28T15:23:55.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.133+0000 7fa9e2bb8980 20 remove_watcher() i=4 2024-06-28T15:23:55.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.133+0000 7fa9e2bb8980 20 remove_watcher() i=5 2024-06-28T15:23:55.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.137+0000 7fa9e2bb8980 20 remove_watcher() i=6 2024-06-28T15:23:55.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.137+0000 7fa9e2bb8980 20 remove_watcher() i=7 2024-06-28T15:23:55.135 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:55.135 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:55.135 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-block-writes id: 843e24ce-085a-4043-8f55-38e05370e790.4686.23 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:23:55.135 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:55.135 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at block_writes 2024-06-28T15:23:55.168 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:55.168 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:55.211 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.229+0000 7f6307b95980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.229+0000 7f6307b95980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.229+0000 7f6307b95980 20 realm 2024-06-28T15:23:55.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.229+0000 7f6307b95980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.229+0000 7f6307b95980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.229+0000 7f6307b95980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:55.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.229+0000 7f6307b95980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.229+0000 7f6307b95980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.229+0000 7f6307b95980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.229+0000 7f6307b95980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:55.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.229+0000 7f6307b95980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.233+0000 7f6307b95980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:55.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.233+0000 7f6307b95980 20 searching for the correct realm 2024-06-28T15:23:55.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.237+0000 7f6304ea5640 0 -- 172.21.15.178:0/494377858 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f62f803abe0 msgr2=0x7f62f804a940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:55.427 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.445+0000 7f63056a6640 0 -- 172.21.15.178:0/494377858 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f62f00c78f0 msgr2=0x7f62f0105040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:23:55.647 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.661+0000 7f6307b95980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:55.647 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.661+0000 7f6307b95980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:55.647 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.661+0000 7f6307b95980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:55.647 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.661+0000 7f6307b95980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:55.647 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.661+0000 7f6307b95980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:55.647 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.661+0000 7f6307b95980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:55.648 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.661+0000 7f6307b95980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.648 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.661+0000 7f6307b95980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.648 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.661+0000 7f6307b95980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:55.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:55.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 20 zone default found 2024-06-28T15:23:55.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 4 Realm: () 2024-06-28T15:23:55.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:55.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:55.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:55.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 20 zonegroup default 2024-06-28T15:23:55.651 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.651 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.651 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 20 started sync module instance, tier type = 2024-06-28T15:23:55.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.665+0000 7f6307b95980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:55.674 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.689+0000 7f6307b95980 20 add_watcher() i=0 2024-06-28T15:23:55.674 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.689+0000 7f6307b95980 20 add_watcher() i=1 2024-06-28T15:23:55.674 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.689+0000 7f6307b95980 20 add_watcher() i=2 2024-06-28T15:23:55.674 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.689+0000 7f6307b95980 20 add_watcher() i=3 2024-06-28T15:23:55.674 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.689+0000 7f6307b95980 20 add_watcher() i=4 2024-06-28T15:23:55.674 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.689+0000 7f6307b95980 20 add_watcher() i=5 2024-06-28T15:23:55.675 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.689+0000 7f6307b95980 20 add_watcher() i=6 2024-06-28T15:23:55.676 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.693+0000 7f6307b95980 20 add_watcher() i=7 2024-06-28T15:23:55.676 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.693+0000 7f6307b95980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:55.680 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.697+0000 7f61bb7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:55.681 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.697+0000 7f6307b95980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:55.681 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.697+0000 7f6307b95980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:55.681 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.697+0000 7f6307b95980 5 note: GC not initialized 2024-06-28T15:23:55.681 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.697+0000 7f61ba7e4640 20 reqs_thread_entry: start 2024-06-28T15:23:55.682 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.697+0000 7f6307b95980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:55.682 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.697+0000 7f6307b95980 20 Filter name: none 2024-06-28T15:23:55.682 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.697+0000 7f612bfff640 20 reqs_thread_entry: start 2024-06-28T15:23:55.682 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.697+0000 7f6307b95980 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2024-06-28T15:23:55.682 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.697+0000 7f6307b95980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.683 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.697+0000 7f6307b95980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:23:55.684 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.697+0000 7f6307b95980 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2024-06-28T15:23:55.684 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.697+0000 7f6307b95980 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2024-06-28T15:23:55.684 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.697+0000 7f6307b95980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 : miss 2024-06-28T15:23:55.684 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.701+0000 7f6307b95980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.684 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.701+0000 7f6307b95980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:23:55.684 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.701+0000 7f6307b95980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 info.flags=0x17 2024-06-28T15:23:55.685 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.701+0000 7f6307b95980 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 to cache LRU end 2024-06-28T15:23:55.685 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.701+0000 7f6307b95980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:55.685 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.701+0000 7f6307b95980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:55.685 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.701+0000 7f6307b95980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 2024-06-28T15:23:55.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.709+0000 7f6307b95980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 info.flags=0x17 2024-06-28T15:23:55.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.709+0000 7f6307b95980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 to cache LRU end 2024-06-28T15:23:55.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.709+0000 7f6307b95980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:55.693 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.709+0000 7f6307b95980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23, ofs0, ns] 2024-06-28T15:23:55.694 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.709+0000 7f61e8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595115 cookie 94877192176368 notifier 6176 bl.length()=991 2024-06-28T15:23:55.694 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.709+0000 7f61e8ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 info.flags=0x17 2024-06-28T15:23:55.694 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.709+0000 7f61e8ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 to cache LRU end 2024-06-28T15:23:55.695 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.709+0000 7f61e8ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:55.697 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.713+0000 7f6307b95980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): orig_sources= new_sources= 2024-06-28T15:23:55.697 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.713+0000 7f6307b95980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): potential sources added=[] removed=[] 2024-06-28T15:23:55.698 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.713+0000 7f6307b95980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): orig_dests= new_dests= 2024-06-28T15:23:55.698 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.713+0000 7f6307b95980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): potential dests added=[] removed=[] 2024-06-28T15:23:55.698 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.713+0000 7f6307b95980 -1 Injecting error=-5 at location=block_writes 2024-06-28T15:23:55.698 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.713+0000 7f6307b95980 0 ERROR: init_reshard failed to pause writes to the current index: (5) Input/output error 2024-06-28T15:23:55.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.717+0000 7f6307b95980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 : hit (requested=0x17, cached=0x17) 2024-06-28T15:23:55.700 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.717+0000 7f6307b95980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 2024-06-28T15:23:55.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.717+0000 7f6307b95980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 info.flags=0x17 2024-06-28T15:23:55.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.717+0000 7f6307b95980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 to cache LRU end 2024-06-28T15:23:55.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.717+0000 7f6307b95980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:55.703 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.717+0000 7f6307b95980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23, ofs0, ns] 2024-06-28T15:23:55.704 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.721+0000 7f61e8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595116 cookie 94877192176368 notifier 6176 bl.length()=959 2024-06-28T15:23:55.704 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.721+0000 7f61e8ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 info.flags=0x17 2024-06-28T15:23:55.704 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.721+0000 7f61e8ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 to cache LRU end 2024-06-28T15:23:55.704 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.721+0000 7f61e8ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:55.707 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.725+0000 7f6307b95980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): orig_sources= new_sources= 2024-06-28T15:23:55.707 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.725+0000 7f6307b95980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): potential sources added=[] removed=[] 2024-06-28T15:23:55.707 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.725+0000 7f6307b95980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): orig_dests= new_dests= 2024-06-28T15:23:55.708 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.725+0000 7f6307b95980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): potential dests added=[] removed=[] 2024-06-28T15:23:55.712 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.729+0000 7f61bb7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:55.717 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.733+0000 7f6307b95980 20 remove_watcher() i=0 2024-06-28T15:23:55.717 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.733+0000 7f6307b95980 2 removed watcher, disabling cache 2024-06-28T15:23:55.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.737+0000 7f6307b95980 20 remove_watcher() i=1 2024-06-28T15:23:55.722 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.737+0000 7f6307b95980 20 remove_watcher() i=2 2024-06-28T15:23:55.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.741+0000 7f6307b95980 20 remove_watcher() i=3 2024-06-28T15:23:55.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.741+0000 7f6307b95980 20 remove_watcher() i=4 2024-06-28T15:23:55.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.745+0000 7f6307b95980 20 remove_watcher() i=5 2024-06-28T15:23:55.730 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.745+0000 7f6307b95980 20 remove_watcher() i=6 2024-06-28T15:23:55.731 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.749+0000 7f6307b95980 20 remove_watcher() i=7 2024-06-28T15:23:55.748 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-06-28T15:23:55.779 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:55.779 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:55.823 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.837+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.841+0000 7ff64a16e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.841+0000 7ff64a16e980 20 realm 2024-06-28T15:23:55.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.841+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.825 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.841+0000 7ff64a16e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.841+0000 7ff64a16e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:55.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.841+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.841+0000 7ff64a16e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:55.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.841+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.841+0000 7ff64a16e980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:55.827 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.841+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:55.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.845+0000 7ff64a16e980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:55.828 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.845+0000 7ff64a16e980 20 searching for the correct realm 2024-06-28T15:23:55.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:55.849+0000 7ff648480640 0 -- 172.21.15.178:0/2596317155 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7ff63c0fa950 msgr2=0x7ff63c0d43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:56.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.065+0000 7ff647c7f640 0 -- 172.21.15.178:0/2596317155 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x5635f1012660 msgr2=0x5635f1014a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:56.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:56.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:56.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:56.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:56.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:56.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:56.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:56.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:56.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 zone default found 2024-06-28T15:23:56.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 4 Realm: () 2024-06-28T15:23:56.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:56.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:56.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:56.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 zonegroup default 2024-06-28T15:23:56.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.073+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.077+0000 7ff64a16e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.060 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.077+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.061 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.077+0000 7ff64a16e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.061 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.077+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.061 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.077+0000 7ff64a16e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.077+0000 7ff64a16e980 20 started sync module instance, tier type = 2024-06-28T15:23:56.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.077+0000 7ff64a16e980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:56.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.101+0000 7ff64a16e980 20 add_watcher() i=0 2024-06-28T15:23:56.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.101+0000 7ff64a16e980 20 add_watcher() i=1 2024-06-28T15:23:56.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.101+0000 7ff64a16e980 20 add_watcher() i=2 2024-06-28T15:23:56.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.101+0000 7ff64a16e980 20 add_watcher() i=3 2024-06-28T15:23:56.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.101+0000 7ff64a16e980 20 add_watcher() i=4 2024-06-28T15:23:56.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.101+0000 7ff64a16e980 20 add_watcher() i=5 2024-06-28T15:23:56.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.101+0000 7ff64a16e980 20 add_watcher() i=6 2024-06-28T15:23:56.086 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.101+0000 7ff64a16e980 20 add_watcher() i=7 2024-06-28T15:23:56.087 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.101+0000 7ff64a16e980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:56.090 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.105+0000 7ff4fdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:56.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.105+0000 7ff64a16e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:56.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.105+0000 7ff64a16e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:56.091 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.105+0000 7ff64a16e980 5 note: GC not initialized 2024-06-28T15:23:56.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.109+0000 7ff4fcfe1640 20 reqs_thread_entry: start 2024-06-28T15:23:56.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.109+0000 7ff64a16e980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:56.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.109+0000 7ff64a16e980 20 Filter name: none 2024-06-28T15:23:56.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.109+0000 7ff4767fc640 20 reqs_thread_entry: start 2024-06-28T15:23:56.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.109+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.093 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.109+0000 7ff64a16e980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:23:56.093 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.109+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.109+0000 7ff64a16e980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:23:56.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.109+0000 7ff64a16e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:56.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.109+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.113+0000 7ff64a16e980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:23:56.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.113+0000 7ff64a16e980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.113+0000 7ff64a16e980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:23:56.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.113+0000 7ff64a16e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:56.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.117+0000 7ff4fdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:56.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.121+0000 7ff64a16e980 20 remove_watcher() i=0 2024-06-28T15:23:56.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.121+0000 7ff64a16e980 2 removed watcher, disabling cache 2024-06-28T15:23:56.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.121+0000 7ff64a16e980 20 remove_watcher() i=1 2024-06-28T15:23:56.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.125+0000 7ff64a16e980 20 remove_watcher() i=2 2024-06-28T15:23:56.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.125+0000 7ff64a16e980 20 remove_watcher() i=3 2024-06-28T15:23:56.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.129+0000 7ff64a16e980 20 remove_watcher() i=4 2024-06-28T15:23:56.114 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.129+0000 7ff64a16e980 20 remove_watcher() i=5 2024-06-28T15:23:56.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.133+0000 7ff64a16e980 20 remove_watcher() i=6 2024-06-28T15:23:56.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.133+0000 7ff64a16e980 20 remove_watcher() i=7 2024-06-28T15:23:56.132 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:56.132 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:56.132 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-block-writes id: 843e24ce-085a-4043-8f55-38e05370e790.4686.23 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:23:56.132 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:56.245 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-06-28T15:23:56.279 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:56.280 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:56.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.337+0000 7f7ca3020980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.341+0000 7f7ca3020980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.341+0000 7f7ca3020980 20 realm 2024-06-28T15:23:56.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.341+0000 7f7ca3020980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.341+0000 7f7ca3020980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.341+0000 7f7ca3020980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:56.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.341+0000 7f7ca3020980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.341+0000 7f7ca3020980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.341+0000 7f7ca3020980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.326 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.341+0000 7f7ca3020980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:56.326 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.341+0000 7f7ca3020980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.345+0000 7f7ca3020980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:56.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.345+0000 7f7ca3020980 20 searching for the correct realm 2024-06-28T15:23:56.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.349+0000 7f7ca0330640 0 -- 172.21.15.178:0/1343583934 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f7c90031920 msgr2=0x7f7c9003b400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:56.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.357+0000 7f7ca0330640 0 -- 172.21.15.178:0/1343583934 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f7b6c07fda0 msgr2=0x7f7b6c0821d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:56.347 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.361+0000 7f7ca3020980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:56.347 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.361+0000 7f7ca3020980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:56.347 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.361+0000 7f7ca3020980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:56.347 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.361+0000 7f7ca3020980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:56.347 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.361+0000 7f7ca3020980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:56.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.361+0000 7f7ca3020980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:56.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.361+0000 7f7ca3020980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.361+0000 7f7ca3020980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.361+0000 7f7ca3020980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:56.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:56.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 20 zone default found 2024-06-28T15:23:56.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 4 Realm: () 2024-06-28T15:23:56.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:56.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:56.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:56.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 20 zonegroup default 2024-06-28T15:23:56.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 20 started sync module instance, tier type = 2024-06-28T15:23:56.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.365+0000 7f7ca3020980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:56.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.381+0000 7f7ca3020980 20 add_watcher() i=0 2024-06-28T15:23:56.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.381+0000 7f7ca3020980 20 add_watcher() i=1 2024-06-28T15:23:56.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.381+0000 7f7ca3020980 20 add_watcher() i=2 2024-06-28T15:23:56.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.381+0000 7f7ca3020980 20 add_watcher() i=3 2024-06-28T15:23:56.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.381+0000 7f7ca3020980 20 add_watcher() i=4 2024-06-28T15:23:56.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.381+0000 7f7ca3020980 20 add_watcher() i=5 2024-06-28T15:23:56.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.381+0000 7f7ca3020980 20 add_watcher() i=6 2024-06-28T15:23:56.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.381+0000 7f7ca3020980 20 add_watcher() i=7 2024-06-28T15:23:56.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.381+0000 7f7ca3020980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:56.368 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7b577e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:56.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:56.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:56.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 5 note: GC not initialized 2024-06-28T15:23:56.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7b567e4640 20 reqs_thread_entry: start 2024-06-28T15:23:56.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:56.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 20 Filter name: none 2024-06-28T15:23:56.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7b4f7fe640 20 reqs_thread_entry: start 2024-06-28T15:23:56.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2024-06-28T15:23:56.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:23:56.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2024-06-28T15:23:56.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2024-06-28T15:23:56.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 : miss 2024-06-28T15:23:56.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:23:56.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 info.flags=0x17 2024-06-28T15:23:56.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 to cache LRU end 2024-06-28T15:23:56.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:56.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:56.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.385+0000 7f7ca3020980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 2024-06-28T15:23:56.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.393+0000 7f7ca3020980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 info.flags=0x17 2024-06-28T15:23:56.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.393+0000 7f7ca3020980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 to cache LRU end 2024-06-28T15:23:56.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.393+0000 7f7ca3020980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:56.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.393+0000 7f7ca3020980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23, ofs0, ns] 2024-06-28T15:23:56.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.397+0000 7f7b77fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595117 cookie 94727291860816 notifier 6609 bl.length()=991 2024-06-28T15:23:56.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.397+0000 7f7b77fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 info.flags=0x17 2024-06-28T15:23:56.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.397+0000 7f7b77fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 to cache LRU end 2024-06-28T15:23:56.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.397+0000 7f7b77fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:56.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.397+0000 7f7ca3020980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): orig_sources= new_sources= 2024-06-28T15:23:56.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.397+0000 7f7ca3020980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): potential sources added=[] removed=[] 2024-06-28T15:23:56.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.397+0000 7f7ca3020980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): orig_dests= new_dests= 2024-06-28T15:23:56.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.397+0000 7f7ca3020980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): potential dests added=[] removed=[] 2024-06-28T15:23:56.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.401+0000 7f7ca3020980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.23.1.0 2024-06-28T15:23:56.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.401+0000 7f7ca3020980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.23.1.0 2024-06-28T15:23:56.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.401+0000 7f7ca3020980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.23.1.1 2024-06-28T15:23:56.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.401+0000 7f7ca3020980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.23.1.1 2024-06-28T15:23:56.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.401+0000 7f7ca3020980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.23.0 2024-06-28T15:23:56.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.405+0000 7f7ca3020980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 : hit (requested=0x17, cached=0x17) 2024-06-28T15:23:56.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.405+0000 7f7ca3020980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 2024-06-28T15:23:56.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.409+0000 7f7ca3020980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 info.flags=0x17 2024-06-28T15:23:56.392 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.409+0000 7f7ca3020980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 to cache LRU end 2024-06-28T15:23:56.392 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.409+0000 7f7ca3020980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:56.392 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.409+0000 7f7ca3020980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23, ofs0, ns] 2024-06-28T15:23:56.392 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.409+0000 7f7b777fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595118 cookie 94727291860816 notifier 6609 bl.length()=959 2024-06-28T15:23:56.392 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.409+0000 7f7b777fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 info.flags=0x17 2024-06-28T15:23:56.392 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.409+0000 7f7b777fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.23 to cache LRU end 2024-06-28T15:23:56.393 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.409+0000 7f7b777fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:56.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.409+0000 7f7ca3020980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): orig_sources= new_sources= 2024-06-28T15:23:56.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.409+0000 7f7ca3020980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): potential sources added=[] removed=[] 2024-06-28T15:23:56.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.409+0000 7f7ca3020980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): orig_dests= new_dests= 2024-06-28T15:23:56.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.409+0000 7f7ca3020980 20 handle_bi_update(): bucket=:error-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.23]): potential dests added=[] removed=[] 2024-06-28T15:23:56.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.413+0000 7f7ca3020980 1 execute INFO: reshard of bucket "error-at-block-writes" from 2 shards to 2 shards completed successfully 2024-06-28T15:23:56.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.417+0000 7f7b577e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:56.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.421+0000 7f7ca3020980 20 remove_watcher() i=0 2024-06-28T15:23:56.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.421+0000 7f7ca3020980 2 removed watcher, disabling cache 2024-06-28T15:23:56.407 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.425+0000 7f7ca3020980 20 remove_watcher() i=1 2024-06-28T15:23:56.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.425+0000 7f7ca3020980 20 remove_watcher() i=2 2024-06-28T15:23:56.411 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.429+0000 7f7ca3020980 20 remove_watcher() i=3 2024-06-28T15:23:56.414 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.429+0000 7f7ca3020980 20 remove_watcher() i=4 2024-06-28T15:23:56.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.433+0000 7f7ca3020980 20 remove_watcher() i=5 2024-06-28T15:23:56.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.433+0000 7f7ca3020980 20 remove_watcher() i=6 2024-06-28T15:23:56.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.433+0000 7f7ca3020980 20 remove_watcher() i=7 2024-06-28T15:23:56.432 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-06-28T15:23:56.458 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:56.458 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:56.500 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.517+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.517+0000 7f1179539980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.517+0000 7f1179539980 20 realm 2024-06-28T15:23:56.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.517+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.517+0000 7f1179539980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.517+0000 7f1179539980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:56.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.517+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.517+0000 7f1179539980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.502 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.517+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.521+0000 7f1179539980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:56.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.521+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.504 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.521+0000 7f1179539980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:56.505 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.521+0000 7f1179539980 20 searching for the correct realm 2024-06-28T15:23:56.510 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.525+0000 7f117784b640 0 -- 172.21.15.178:0/3944964988 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x558eb7dce6c0 msgr2=0x558eb7dd0b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:56.516 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.533+0000 7f116ffff640 0 -- 172.21.15.178:0/3944964988 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f11680c0020 msgr2=0x7f11680030a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:23:56.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:56.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:56.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:56.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:56.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:56.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:56.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.524 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:56.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:56.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 zone default found 2024-06-28T15:23:56.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 4 Realm: () 2024-06-28T15:23:56.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:56.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:56.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:56.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 zonegroup default 2024-06-28T15:23:56.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:56.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 started sync module instance, tier type = 2024-06-28T15:23:56.528 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.541+0000 7f1179539980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:56.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.557+0000 7f1179539980 20 add_watcher() i=0 2024-06-28T15:23:56.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.557+0000 7f1179539980 20 add_watcher() i=1 2024-06-28T15:23:56.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.557+0000 7f1179539980 20 add_watcher() i=2 2024-06-28T15:23:56.542 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.557+0000 7f1179539980 20 add_watcher() i=3 2024-06-28T15:23:56.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.557+0000 7f1179539980 20 add_watcher() i=4 2024-06-28T15:23:56.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.557+0000 7f1179539980 20 add_watcher() i=5 2024-06-28T15:23:56.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.557+0000 7f1179539980 20 add_watcher() i=6 2024-06-28T15:23:56.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.561+0000 7f1179539980 20 add_watcher() i=7 2024-06-28T15:23:56.543 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.561+0000 7f1179539980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:56.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.561+0000 7f102dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:56.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.561+0000 7f1179539980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:56.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.561+0000 7f1179539980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:56.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.561+0000 7f1179539980 5 note: GC not initialized 2024-06-28T15:23:56.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.561+0000 7f102cfe1640 20 reqs_thread_entry: start 2024-06-28T15:23:56.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.561+0000 7f1179539980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:56.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.561+0000 7f1179539980 20 Filter name: none 2024-06-28T15:23:56.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.561+0000 7f0fa67fc640 20 reqs_thread_entry: start 2024-06-28T15:23:56.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.561+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.565+0000 7f1179539980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:23:56.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.565+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.565+0000 7f1179539980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:23:56.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.565+0000 7f1179539980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:56.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.565+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.565+0000 7f1179539980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:23:56.550 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.565+0000 7f1179539980 20 rados->read ofs=0 len=0 2024-06-28T15:23:56.551 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.569+0000 7f1179539980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:23:56.551 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.569+0000 7f1179539980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:56.553 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.569+0000 7f102dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:56.558 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.573+0000 7f1179539980 20 remove_watcher() i=0 2024-06-28T15:23:56.558 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.573+0000 7f1179539980 2 removed watcher, disabling cache 2024-06-28T15:23:56.560 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.577+0000 7f1179539980 20 remove_watcher() i=1 2024-06-28T15:23:56.562 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.577+0000 7f1179539980 20 remove_watcher() i=2 2024-06-28T15:23:56.564 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.581+0000 7f1179539980 20 remove_watcher() i=3 2024-06-28T15:23:56.566 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.581+0000 7f1179539980 20 remove_watcher() i=4 2024-06-28T15:23:56.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.581+0000 7f1179539980 20 remove_watcher() i=5 2024-06-28T15:23:56.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.585+0000 7f1179539980 20 remove_watcher() i=6 2024-06-28T15:23:56.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:56.585+0000 7f1179539980 20 remove_watcher() i=7 2024-06-28T15:23:56.582 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:56.582 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:56.582 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-block-writes id: 843e24ce-085a-4043-8f55-38e05370e790.4686.23 num_objects: 19 size_kb: 1 num_shards: 2 2024-06-28T15:23:56.582 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:57.308 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: reshard bucket with abort at block_writes 2024-06-28T15:23:57.308 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:59.468 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-06-28T15:23:59.500 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:59.500 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:59.544 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.561+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.561+0000 7f78d45a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:59.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.561+0000 7f78d45a4980 20 realm 2024-06-28T15:23:59.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.561+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.546 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.561+0000 7f78d45a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:59.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.561+0000 7f78d45a4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:59.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.561+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.561+0000 7f78d45a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:59.547 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.561+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.565+0000 7f78d45a4980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:59.548 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.565+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.565+0000 7f78d45a4980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:59.549 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.565+0000 7f78d45a4980 20 searching for the correct realm 2024-06-28T15:23:59.555 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.569+0000 7f78d28b6640 0 -- 172.21.15.178:0/2457421931 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f78c800a040 msgr2=0x7f78c807c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:23:59.768 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.785+0000 7f78d20b5640 0 -- 172.21.15.178:0/2457421931 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55df1f640840 msgr2=0x55df1f642c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:59.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:59.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:59.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:59.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:59.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:59.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:59.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:59.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:59.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:59.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 zone default found 2024-06-28T15:23:59.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 4 Realm: () 2024-06-28T15:23:59.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:59.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:59.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:59.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 zonegroup default 2024-06-28T15:23:59.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.779 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:59.779 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.793+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.779 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.797+0000 7f78d45a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:59.779 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.797+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.797+0000 7f78d45a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:59.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.797+0000 7f78d45a4980 20 started sync module instance, tier type = 2024-06-28T15:23:59.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.797+0000 7f78d45a4980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:59.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.821+0000 7f78d45a4980 20 add_watcher() i=0 2024-06-28T15:23:59.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.821+0000 7f78d45a4980 20 add_watcher() i=1 2024-06-28T15:23:59.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.821+0000 7f78d45a4980 20 add_watcher() i=2 2024-06-28T15:23:59.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.821+0000 7f78d45a4980 20 add_watcher() i=3 2024-06-28T15:23:59.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.821+0000 7f78d45a4980 20 add_watcher() i=4 2024-06-28T15:23:59.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.821+0000 7f78d45a4980 20 add_watcher() i=5 2024-06-28T15:23:59.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.821+0000 7f78d45a4980 20 add_watcher() i=6 2024-06-28T15:23:59.806 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.821+0000 7f78d45a4980 20 add_watcher() i=7 2024-06-28T15:23:59.806 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.821+0000 7f78d45a4980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:59.810 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.825+0000 7f7788fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:59.811 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.825+0000 7f78d45a4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:59.812 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.825+0000 7f78d45a4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:59.812 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.825+0000 7f78d45a4980 5 note: GC not initialized 2024-06-28T15:23:59.812 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.829+0000 7f77037fe640 20 reqs_thread_entry: start 2024-06-28T15:23:59.812 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.829+0000 7f78d45a4980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:59.812 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.829+0000 7f78d45a4980 20 Filter name: none 2024-06-28T15:23:59.813 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.829+0000 7f77017fa640 20 reqs_thread_entry: start 2024-06-28T15:23:59.813 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.829+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.813 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.829+0000 7f78d45a4980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:23:59.814 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.829+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.829+0000 7f78d45a4980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:23:59.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.829+0000 7f78d45a4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:59.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.829+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.833+0000 7f78d45a4980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:23:59.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.833+0000 7f78d45a4980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.816 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.833+0000 7f78d45a4980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:23:59.817 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.833+0000 7f78d45a4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:23:59.819 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.833+0000 7f7788fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:59.824 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.841+0000 7f78d45a4980 20 remove_watcher() i=0 2024-06-28T15:23:59.824 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.841+0000 7f78d45a4980 2 removed watcher, disabling cache 2024-06-28T15:23:59.826 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.841+0000 7f78d45a4980 20 remove_watcher() i=1 2024-06-28T15:23:59.829 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.845+0000 7f78d45a4980 20 remove_watcher() i=2 2024-06-28T15:23:59.831 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.849+0000 7f78d45a4980 20 remove_watcher() i=3 2024-06-28T15:23:59.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.849+0000 7f78d45a4980 20 remove_watcher() i=4 2024-06-28T15:23:59.838 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.853+0000 7f78d45a4980 20 remove_watcher() i=5 2024-06-28T15:23:59.838 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.853+0000 7f78d45a4980 20 remove_watcher() i=6 2024-06-28T15:23:59.839 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.853+0000 7f78d45a4980 20 remove_watcher() i=7 2024-06-28T15:23:59.853 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:59.853 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:23:59.853 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: abort-at-block-writes id: 843e24ce-085a-4043-8f55-38e05370e790.4686.24 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:23:59.853 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:59.854 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at block_writes 2024-06-28T15:23:59.889 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:23:59.889 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:23:59.931 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:59.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 realm 2024-06-28T15:23:59.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:59.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.949+0000 7f2f2ef5d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:23:59.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:59.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:59.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.953+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:23:59.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.953+0000 7f2f2ef5d980 20 searching for the correct realm 2024-06-28T15:23:59.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.957+0000 7f2f2c26d640 0 -- 172.21.15.178:0/2278050652 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f2f1c0dc060 msgr2=0x7f2f1c0c83d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:59.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.965+0000 7f2f2c26d640 0 -- 172.21.15.178:0/2278050652 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f2df807fda0 msgr2=0x7f2df80821d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:23:59.954 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:23:59.955 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:23:59.955 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:23:59.955 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:23:59.955 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:23:59.955 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:23:59.955 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.955 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:59.955 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:23:59.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:23:59.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 zone default found 2024-06-28T15:23:59.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 4 Realm: () 2024-06-28T15:23:59.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:23:59.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:23:59.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:23:59.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 zonegroup default 2024-06-28T15:23:59.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:59.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:59.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:23:59.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 started sync module instance, tier type = 2024-06-28T15:23:59.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:23:59.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=0 2024-06-28T15:23:59.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=1 2024-06-28T15:23:59.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=2 2024-06-28T15:23:59.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=3 2024-06-28T15:23:59.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=4 2024-06-28T15:23:59.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=5 2024-06-28T15:23:59.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=6 2024-06-28T15:23:59.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=7 2024-06-28T15:23:59.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.989+0000 7f2f2ef5d980 2 all 8 watchers are set, enabling cache 2024-06-28T15:23:59.976 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2de37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:23:59.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:23:59.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:23:59.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 note: GC not initialized 2024-06-28T15:23:59.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2de27e4640 20 reqs_thread_entry: start 2024-06-28T15:23:59.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 init_complete bucket index max shards: 1 2024-06-28T15:23:59.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 Filter name: none 2024-06-28T15:23:59.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2ddbfff640 20 reqs_thread_entry: start 2024-06-28T15:23:59.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-06-28T15:23:59.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:23:59.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-06-28T15:23:59.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-06-28T15:23:59.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 : miss 2024-06-28T15:23:59.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:23:59.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:23:59.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:23:59.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:23:59.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:23:59.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:59.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 2024-06-28T15:23:59.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.001+0000 7f2f2ef5d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:23:59.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.001+0000 7f2f2ef5d980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:23:59.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.001+0000 7f2f2ef5d980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:59.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.001+0000 7f2f2ef5d980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24, ofs0, ns] 2024-06-28T15:23:59.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.001+0000 7f2e03fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595085 cookie 94507079483440 notifier 6260 bl.length()=991 2024-06-28T15:23:59.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.001+0000 7f2e03fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:23:59.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.001+0000 7f2e03fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:23:59.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.001+0000 7f2e03fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:23:59.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.005+0000 7f2f2ef5d980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): orig_sources= new_sources= 2024-06-28T15:23:59.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.005+0000 7f2f2ef5d980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): potential sources added=[] removed=[] 2024-06-28T15:23:59.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.005+0000 7f2f2ef5d980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): orig_dests= new_dests= 2024-06-28T15:23:59.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.005+0000 7f2f2ef5d980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): potential dests added=[] removed=[] 2024-06-28T15:23:59.989 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f2f2ef5d980 time 2024-06-28T15:24:00.010620+0000 2024-06-28T15:23:59.989 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:23:59.990 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:23:59.990 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f2f32834c91] 2024-06-28T15:23:59.990 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f2f32834e45] 2024-06-28T15:23:59.990 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55f421d834b1] 2024-06-28T15:23:59.990 INFO:tasks.workunit.client.0.smithi178.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2bc) [0x55f421d94a3c] 2024-06-28T15:23:59.990 INFO:tasks.workunit.client.0.smithi178.stderr: 5: main() 2024-06-28T15:23:59.990 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2f3169dd90] 2024-06-28T15:23:59.990 INFO:tasks.workunit.client.0.smithi178.stderr: 7: __libc_start_main() 2024-06-28T15:23:59.990 INFO:tasks.workunit.client.0.smithi178.stderr: 8: _start() 2024-06-28T15:23:59.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.005+0000 7f2f2ef5d980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f2f2ef5d980 time 2024-06-28T15:24:00.010620+0000 2024-06-28T15:23:59.990 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:23:59.990 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:59.991 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:23:59.991 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f2f32834c91] 2024-06-28T15:23:59.991 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f2f32834e45] 2024-06-28T15:23:59.991 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55f421d834b1] 2024-06-28T15:23:59.991 INFO:tasks.workunit.client.0.smithi178.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2bc) [0x55f421d94a3c] 2024-06-28T15:23:59.991 INFO:tasks.workunit.client.0.smithi178.stderr: 5: main() 2024-06-28T15:23:59.991 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2f3169dd90] 2024-06-28T15:23:59.991 INFO:tasks.workunit.client.0.smithi178.stderr: 7: __libc_start_main() 2024-06-28T15:23:59.991 INFO:tasks.workunit.client.0.smithi178.stderr: 8: _start() 2024-06-28T15:23:59.991 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:23:59.991 INFO:tasks.workunit.client.0.smithi178.stderr:*** Caught signal (Aborted) ** 2024-06-28T15:23:59.991 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7f2f2ef5d980 thread_name:radosgw-admin 2024-06-28T15:24:00.008 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:24:00.008 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f2f316b6520] 2024-06-28T15:24:00.008 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:24:00.009 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:24:00.009 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:24:00.009 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f2f32834ceb] 2024-06-28T15:24:00.009 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f2f32834e45] 2024-06-28T15:24:00.009 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55f421d834b1] 2024-06-28T15:24:00.009 INFO:tasks.workunit.client.0.smithi178.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2bc) [0x55f421d94a3c] 2024-06-28T15:24:00.009 INFO:tasks.workunit.client.0.smithi178.stderr: 9: main() 2024-06-28T15:24:00.009 INFO:tasks.workunit.client.0.smithi178.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2f3169dd90] 2024-06-28T15:24:00.009 INFO:tasks.workunit.client.0.smithi178.stderr: 11: __libc_start_main() 2024-06-28T15:24:00.009 INFO:tasks.workunit.client.0.smithi178.stderr: 12: _start() 2024-06-28T15:24:00.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.005+0000 7f2f2ef5d980 -1 *** Caught signal (Aborted) ** 2024-06-28T15:24:00.009 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7f2f2ef5d980 thread_name:radosgw-admin 2024-06-28T15:24:00.009 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.010 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:24:00.010 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f2f316b6520] 2024-06-28T15:24:00.010 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:24:00.010 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:24:00.010 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:24:00.010 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f2f32834ceb] 2024-06-28T15:24:00.010 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f2f32834e45] 2024-06-28T15:24:00.010 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55f421d834b1] 2024-06-28T15:24:00.010 INFO:tasks.workunit.client.0.smithi178.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2bc) [0x55f421d94a3c] 2024-06-28T15:24:00.010 INFO:tasks.workunit.client.0.smithi178.stderr: 9: main() 2024-06-28T15:24:00.010 INFO:tasks.workunit.client.0.smithi178.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2f3169dd90] 2024-06-28T15:24:00.010 INFO:tasks.workunit.client.0.smithi178.stderr: 11: __libc_start_main() 2024-06-28T15:24:00.010 INFO:tasks.workunit.client.0.smithi178.stderr: 12: _start() 2024-06-28T15:24:00.010 INFO:tasks.workunit.client.0.smithi178.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-28T15:24:00.011 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.011 INFO:tasks.workunit.client.0.smithi178.stderr:--- begin dump of recent events --- 2024-06-28T15:24:00.011 INFO:tasks.workunit.client.0.smithi178.stderr: -499> 2024-06-28T15:23:59.925+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.011 INFO:tasks.workunit.client.0.smithi178.stderr: -498> 2024-06-28T15:23:59.925+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.011 INFO:tasks.workunit.client.0.smithi178.stderr: -497> 2024-06-28T15:23:59.925+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_done global_id 6654 payload 274 2024-06-28T15:24:00.011 INFO:tasks.workunit.client.0.smithi178.stderr: -496> 2024-06-28T15:23:59.925+0000 7f2f2d26f640 10 monclient: _finish_hunting 0 2024-06-28T15:24:00.011 INFO:tasks.workunit.client.0.smithi178.stderr: -495> 2024-06-28T15:23:59.925+0000 7f2f2d26f640 1 monclient: found mon.noname-a 2024-06-28T15:24:00.011 INFO:tasks.workunit.client.0.smithi178.stderr: -494> 2024-06-28T15:23:59.925+0000 7f2f2d26f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:24:00.011 INFO:tasks.workunit.client.0.smithi178.stderr: -493> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.011 INFO:tasks.workunit.client.0.smithi178.stderr: -492> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:24:00.011 INFO:tasks.workunit.client.0.smithi178.stderr: -491> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: dump: 2024-06-28T15:24:00.011 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.011 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.011 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.012 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.012 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.012 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.012 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.012 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.012 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.012 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.012 INFO:tasks.workunit.client.0.smithi178.stderr: -490> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: _finish_auth 0 2024-06-28T15:24:00.012 INFO:tasks.workunit.client.0.smithi178.stderr: -489> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: _check_auth_tickets 2024-06-28T15:24:00.013 INFO:tasks.workunit.client.0.smithi178.stderr: -488> 2024-06-28T15:23:59.925+0000 7f2f2ef5d980 5 monclient: authenticate success, global_id 6654 2024-06-28T15:24:00.013 INFO:tasks.workunit.client.0.smithi178.stderr: -487> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:00.013 INFO:tasks.workunit.client.0.smithi178.stderr: -486> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.013 INFO:tasks.workunit.client.0.smithi178.stderr: -485> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:24:00.013 INFO:tasks.workunit.client.0.smithi178.stderr: -484> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: dump: 2024-06-28T15:24:00.013 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.013 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.013 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.014 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.014 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.014 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.014 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.014 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.014 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.014 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.014 INFO:tasks.workunit.client.0.smithi178.stderr: -483> 2024-06-28T15:23:59.925+0000 7f2f2ef5d980 10 monclient: get_monmap_and_config success 2024-06-28T15:24:00.014 INFO:tasks.workunit.client.0.smithi178.stderr: -482> 2024-06-28T15:23:59.925+0000 7f2f2ef5d980 4 set_mon_vals no callback set 2024-06-28T15:24:00.014 INFO:tasks.workunit.client.0.smithi178.stderr: -481> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: shutdown 2024-06-28T15:24:00.014 INFO:tasks.workunit.client.0.smithi178.stderr: -480> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) unregister_commands rotate-key 2024-06-28T15:24:00.014 INFO:tasks.workunit.client.0.smithi178.stderr: -479> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: build_initial_monmap 2024-06-28T15:24:00.014 INFO:tasks.workunit.client.0.smithi178.stderr: -478> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 1 build_initial for_mkfs: 0 2024-06-28T15:24:00.015 INFO:tasks.workunit.client.0.smithi178.stderr: -477> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: monmap: 2024-06-28T15:24:00.015 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:00.015 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:00.015 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:00.015 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:00.015 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:00.015 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.015 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:00.015 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:00.015 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:00.015 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.015 INFO:tasks.workunit.client.0.smithi178.stderr: -476> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding auth protocol: cephx 2024-06-28T15:24:00.016 INFO:tasks.workunit.client.0.smithi178.stderr: -475> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding auth protocol: cephx 2024-06-28T15:24:00.016 INFO:tasks.workunit.client.0.smithi178.stderr: -474> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding auth protocol: cephx 2024-06-28T15:24:00.016 INFO:tasks.workunit.client.0.smithi178.stderr: -473> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: secure 2024-06-28T15:24:00.016 INFO:tasks.workunit.client.0.smithi178.stderr: -472> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: crc 2024-06-28T15:24:00.016 INFO:tasks.workunit.client.0.smithi178.stderr: -471> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: secure 2024-06-28T15:24:00.016 INFO:tasks.workunit.client.0.smithi178.stderr: -470> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: crc 2024-06-28T15:24:00.016 INFO:tasks.workunit.client.0.smithi178.stderr: -469> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: secure 2024-06-28T15:24:00.016 INFO:tasks.workunit.client.0.smithi178.stderr: -468> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: crc 2024-06-28T15:24:00.016 INFO:tasks.workunit.client.0.smithi178.stderr: -467> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: crc 2024-06-28T15:24:00.016 INFO:tasks.workunit.client.0.smithi178.stderr: -466> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: secure 2024-06-28T15:24:00.016 INFO:tasks.workunit.client.0.smithi178.stderr: -465> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: crc 2024-06-28T15:24:00.016 INFO:tasks.workunit.client.0.smithi178.stderr: -464> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: secure 2024-06-28T15:24:00.016 INFO:tasks.workunit.client.0.smithi178.stderr: -463> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: crc 2024-06-28T15:24:00.017 INFO:tasks.workunit.client.0.smithi178.stderr: -462> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: secure 2024-06-28T15:24:00.017 INFO:tasks.workunit.client.0.smithi178.stderr: -461> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.017 INFO:tasks.workunit.client.0.smithi178.stderr: -460> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 1 librados: starting msgr at 2024-06-28T15:24:00.017 INFO:tasks.workunit.client.0.smithi178.stderr: -459> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 1 librados: starting objecter 2024-06-28T15:24:00.017 INFO:tasks.workunit.client.0.smithi178.stderr: -458> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command objecter_requests hook 0x55f424da1eb0 2024-06-28T15:24:00.017 INFO:tasks.workunit.client.0.smithi178.stderr: -457> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 1 librados: setting wanted keys 2024-06-28T15:24:00.017 INFO:tasks.workunit.client.0.smithi178.stderr: -456> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 1 librados: calling monclient init 2024-06-28T15:24:00.017 INFO:tasks.workunit.client.0.smithi178.stderr: -455> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: init 2024-06-28T15:24:00.018 INFO:tasks.workunit.client.0.smithi178.stderr: -454> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding auth protocol: cephx 2024-06-28T15:24:00.018 INFO:tasks.workunit.client.0.smithi178.stderr: -453> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding auth protocol: cephx 2024-06-28T15:24:00.018 INFO:tasks.workunit.client.0.smithi178.stderr: -452> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding auth protocol: cephx 2024-06-28T15:24:00.018 INFO:tasks.workunit.client.0.smithi178.stderr: -451> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: secure 2024-06-28T15:24:00.018 INFO:tasks.workunit.client.0.smithi178.stderr: -450> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: crc 2024-06-28T15:24:00.018 INFO:tasks.workunit.client.0.smithi178.stderr: -449> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: secure 2024-06-28T15:24:00.018 INFO:tasks.workunit.client.0.smithi178.stderr: -448> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: crc 2024-06-28T15:24:00.018 INFO:tasks.workunit.client.0.smithi178.stderr: -447> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: secure 2024-06-28T15:24:00.018 INFO:tasks.workunit.client.0.smithi178.stderr: -446> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: crc 2024-06-28T15:24:00.018 INFO:tasks.workunit.client.0.smithi178.stderr: -445> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: crc 2024-06-28T15:24:00.018 INFO:tasks.workunit.client.0.smithi178.stderr: -444> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: secure 2024-06-28T15:24:00.018 INFO:tasks.workunit.client.0.smithi178.stderr: -443> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: crc 2024-06-28T15:24:00.019 INFO:tasks.workunit.client.0.smithi178.stderr: -442> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: secure 2024-06-28T15:24:00.019 INFO:tasks.workunit.client.0.smithi178.stderr: -441> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: crc 2024-06-28T15:24:00.019 INFO:tasks.workunit.client.0.smithi178.stderr: -440> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: secure 2024-06-28T15:24:00.019 INFO:tasks.workunit.client.0.smithi178.stderr: -439> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.019 INFO:tasks.workunit.client.0.smithi178.stderr: -438> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.019 INFO:tasks.workunit.client.0.smithi178.stderr: -437> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command rotate-key hook 0x55f424d6f3d8 2024-06-28T15:24:00.019 INFO:tasks.workunit.client.0.smithi178.stderr: -436> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:00.019 INFO:tasks.workunit.client.0.smithi178.stderr: -435> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: _add_conns ranks=[1,2,0] 2024-06-28T15:24:00.019 INFO:tasks.workunit.client.0.smithi178.stderr: -434> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-b con 0x55f424c1f8c0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:00.020 INFO:tasks.workunit.client.0.smithi178.stderr: -433> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-c con 0x55f424c20300 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:00.020 INFO:tasks.workunit.client.0.smithi178.stderr: -432> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-a con 0x55f424c29270 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:00.020 INFO:tasks.workunit.client.0.smithi178.stderr: -431> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.020 INFO:tasks.workunit.client.0.smithi178.stderr: -430> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.020 INFO:tasks.workunit.client.0.smithi178.stderr: -429> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.020 INFO:tasks.workunit.client.0.smithi178.stderr: -428> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:00.020 INFO:tasks.workunit.client.0.smithi178.stderr: -427> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): authenticate will time out at 2383.394019s 2024-06-28T15:24:00.020 INFO:tasks.workunit.client.0.smithi178.stderr: -426> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): get_auth_request con 0x55f424c20300 auth_method 0 2024-06-28T15:24:00.020 INFO:tasks.workunit.client.0.smithi178.stderr: -425> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.020 INFO:tasks.workunit.client.0.smithi178.stderr: -424> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.020 INFO:tasks.workunit.client.0.smithi178.stderr: -423> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.021 INFO:tasks.workunit.client.0.smithi178.stderr: -422> 2024-06-28T15:23:59.929+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request con 0x55f424c29270 auth_method 0 2024-06-28T15:24:00.021 INFO:tasks.workunit.client.0.smithi178.stderr: -421> 2024-06-28T15:23:59.929+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.021 INFO:tasks.workunit.client.0.smithi178.stderr: -420> 2024-06-28T15:23:59.929+0000 7f2f2d26f640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.021 INFO:tasks.workunit.client.0.smithi178.stderr: -419> 2024-06-28T15:23:59.929+0000 7f2f2d26f640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.021 INFO:tasks.workunit.client.0.smithi178.stderr: -418> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.021 INFO:tasks.workunit.client.0.smithi178.stderr: -417> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.021 INFO:tasks.workunit.client.0.smithi178.stderr: -416> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.021 INFO:tasks.workunit.client.0.smithi178.stderr: -415> 2024-06-28T15:23:59.929+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.021 INFO:tasks.workunit.client.0.smithi178.stderr: -414> 2024-06-28T15:23:59.929+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.022 INFO:tasks.workunit.client.0.smithi178.stderr: -413> 2024-06-28T15:23:59.929+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.022 INFO:tasks.workunit.client.0.smithi178.stderr: -412> 2024-06-28T15:23:59.929+0000 7f2f2ca6e640 10 monclient(hunting): get_auth_request con 0x55f424c1f8c0 auth_method 0 2024-06-28T15:24:00.022 INFO:tasks.workunit.client.0.smithi178.stderr: -411> 2024-06-28T15:23:59.929+0000 7f2f2ca6e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.022 INFO:tasks.workunit.client.0.smithi178.stderr: -410> 2024-06-28T15:23:59.929+0000 7f2f2ca6e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.022 INFO:tasks.workunit.client.0.smithi178.stderr: -409> 2024-06-28T15:23:59.929+0000 7f2f2ca6e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.022 INFO:tasks.workunit.client.0.smithi178.stderr: -408> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_done global_id 6251 payload 824 2024-06-28T15:24:00.022 INFO:tasks.workunit.client.0.smithi178.stderr: -407> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient: _finish_hunting 0 2024-06-28T15:24:00.022 INFO:tasks.workunit.client.0.smithi178.stderr: -406> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 1 monclient: found mon.noname-c 2024-06-28T15:24:00.022 INFO:tasks.workunit.client.0.smithi178.stderr: -405> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.022 INFO:tasks.workunit.client.0.smithi178.stderr: -404> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.022 INFO:tasks.workunit.client.0.smithi178.stderr: -403> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:24:00.022 INFO:tasks.workunit.client.0.smithi178.stderr: -402> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: dump: 2024-06-28T15:24:00.022 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.023 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.023 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.023 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.023 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.023 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.023 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.023 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.023 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.023 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.023 INFO:tasks.workunit.client.0.smithi178.stderr: -401> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: _finish_auth 0 2024-06-28T15:24:00.023 INFO:tasks.workunit.client.0.smithi178.stderr: -400> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: _check_auth_tickets 2024-06-28T15:24:00.023 INFO:tasks.workunit.client.0.smithi178.stderr: -399> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:00.023 INFO:tasks.workunit.client.0.smithi178.stderr: -398> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.023 INFO:tasks.workunit.client.0.smithi178.stderr: -397> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr: -396> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: dump: 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr: -395> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 monclient: authenticate success, global_id 6251 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr: -394> 2024-06-28T15:23:59.929+0000 7f2e1ae0d640 4 set_mon_vals no callback set 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr: -393> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: _renew_subs 2024-06-28T15:24:00.024 INFO:tasks.workunit.client.0.smithi178.stderr: -392> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.025 INFO:tasks.workunit.client.0.smithi178.stderr: -391> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: _renew_subs 2024-06-28T15:24:00.025 INFO:tasks.workunit.client.0.smithi178.stderr: -390> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.025 INFO:tasks.workunit.client.0.smithi178.stderr: -389> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 1 librados: init done 2024-06-28T15:24:00.025 INFO:tasks.workunit.client.0.smithi178.stderr: -388> 2024-06-28T15:23:59.933+0000 7f2e1960a640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:24:00.025 INFO:tasks.workunit.client.0.smithi178.stderr: -387> 2024-06-28T15:23:59.933+0000 7f2e1960a640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:00.025 INFO:tasks.workunit.client.0.smithi178.stderr: -386> 2024-06-28T15:23:59.933+0000 7f2e1960a640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:00.025 INFO:tasks.workunit.client.0.smithi178.stderr: -385> 2024-06-28T15:23:59.933+0000 7f2f2ca6e640 10 monclient: get_auth_request con 0x7f2e1406f7b0 auth_method 0 2024-06-28T15:24:00.025 INFO:tasks.workunit.client.0.smithi178.stderr: -384> 2024-06-28T15:23:59.933+0000 7f2f2d26f640 10 monclient: get_auth_request con 0x55f424d97f30 auth_method 0 2024-06-28T15:24:00.025 INFO:tasks.workunit.client.0.smithi178.stderr: -383> 2024-06-28T15:23:59.933+0000 7f2f2ca6e640 10 monclient: get_auth_request con 0x55f424dbaab0 auth_method 0 2024-06-28T15:24:00.025 INFO:tasks.workunit.client.0.smithi178.stderr: -382> 2024-06-28T15:23:59.933+0000 7f2f2c26d640 10 monclient: get_auth_request con 0x55f424dbee10 auth_method 0 2024-06-28T15:24:00.025 INFO:tasks.workunit.client.0.smithi178.stderr: -381> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient: build_initial_monmap 2024-06-28T15:24:00.026 INFO:tasks.workunit.client.0.smithi178.stderr: -380> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 1 build_initial for_mkfs: 0 2024-06-28T15:24:00.026 INFO:tasks.workunit.client.0.smithi178.stderr: -379> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient: monmap: 2024-06-28T15:24:00.026 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:00.026 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:00.026 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:00.026 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:00.026 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:00.026 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.026 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:00.026 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:00.026 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:00.026 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.026 INFO:tasks.workunit.client.0.smithi178.stderr: -378> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding auth protocol: cephx 2024-06-28T15:24:00.027 INFO:tasks.workunit.client.0.smithi178.stderr: -377> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding auth protocol: cephx 2024-06-28T15:24:00.027 INFO:tasks.workunit.client.0.smithi178.stderr: -376> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding auth protocol: cephx 2024-06-28T15:24:00.027 INFO:tasks.workunit.client.0.smithi178.stderr: -375> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: secure 2024-06-28T15:24:00.027 INFO:tasks.workunit.client.0.smithi178.stderr: -374> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: crc 2024-06-28T15:24:00.027 INFO:tasks.workunit.client.0.smithi178.stderr: -373> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: secure 2024-06-28T15:24:00.027 INFO:tasks.workunit.client.0.smithi178.stderr: -372> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: crc 2024-06-28T15:24:00.027 INFO:tasks.workunit.client.0.smithi178.stderr: -371> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: secure 2024-06-28T15:24:00.027 INFO:tasks.workunit.client.0.smithi178.stderr: -370> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: crc 2024-06-28T15:24:00.027 INFO:tasks.workunit.client.0.smithi178.stderr: -369> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: crc 2024-06-28T15:24:00.027 INFO:tasks.workunit.client.0.smithi178.stderr: -368> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: secure 2024-06-28T15:24:00.027 INFO:tasks.workunit.client.0.smithi178.stderr: -367> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: crc 2024-06-28T15:24:00.027 INFO:tasks.workunit.client.0.smithi178.stderr: -366> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: secure 2024-06-28T15:24:00.028 INFO:tasks.workunit.client.0.smithi178.stderr: -365> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: crc 2024-06-28T15:24:00.028 INFO:tasks.workunit.client.0.smithi178.stderr: -364> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: secure 2024-06-28T15:24:00.028 INFO:tasks.workunit.client.0.smithi178.stderr: -363> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.028 INFO:tasks.workunit.client.0.smithi178.stderr: -362> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command objecter_requests cmddesc objecter_requests hook 0x55f424dc7240 EEXIST 2024-06-28T15:24:00.028 INFO:tasks.workunit.client.0.smithi178.stderr: -361> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient: init 2024-06-28T15:24:00.028 INFO:tasks.workunit.client.0.smithi178.stderr: -360> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding auth protocol: cephx 2024-06-28T15:24:00.028 INFO:tasks.workunit.client.0.smithi178.stderr: -359> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding auth protocol: cephx 2024-06-28T15:24:00.028 INFO:tasks.workunit.client.0.smithi178.stderr: -358> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding auth protocol: cephx 2024-06-28T15:24:00.028 INFO:tasks.workunit.client.0.smithi178.stderr: -357> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: secure 2024-06-28T15:24:00.028 INFO:tasks.workunit.client.0.smithi178.stderr: -356> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: crc 2024-06-28T15:24:00.028 INFO:tasks.workunit.client.0.smithi178.stderr: -355> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: secure 2024-06-28T15:24:00.028 INFO:tasks.workunit.client.0.smithi178.stderr: -354> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: crc 2024-06-28T15:24:00.028 INFO:tasks.workunit.client.0.smithi178.stderr: -353> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: secure 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -352> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: crc 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -351> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: crc 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -350> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: secure 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -349> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: crc 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -348> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: secure 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -347> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: crc 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -346> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: secure 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -345> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -344> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -343> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command rotate-key cmddesc rotate-key hook 0x55f424dc2480 EEXIST 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -342> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -341> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient: _add_conns ranks=[2,1,0] 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -340> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-c con 0x55f424dcd7d0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -339> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-b con 0x55f424dce100 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:00.029 INFO:tasks.workunit.client.0.smithi178.stderr: -338> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-a con 0x55f424dcdbb0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:00.030 INFO:tasks.workunit.client.0.smithi178.stderr: -337> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.030 INFO:tasks.workunit.client.0.smithi178.stderr: -336> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.030 INFO:tasks.workunit.client.0.smithi178.stderr: -335> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.030 INFO:tasks.workunit.client.0.smithi178.stderr: -334> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:00.030 INFO:tasks.workunit.client.0.smithi178.stderr: -333> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): authenticate will time out at 2383.403309s 2024-06-28T15:24:00.030 INFO:tasks.workunit.client.0.smithi178.stderr: -332> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): get_auth_request con 0x55f424dcd7d0 auth_method 0 2024-06-28T15:24:00.030 INFO:tasks.workunit.client.0.smithi178.stderr: -331> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.030 INFO:tasks.workunit.client.0.smithi178.stderr: -330> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.030 INFO:tasks.workunit.client.0.smithi178.stderr: -329> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.030 INFO:tasks.workunit.client.0.smithi178.stderr: -328> 2024-06-28T15:23:59.937+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request con 0x55f424dcdbb0 auth_method 0 2024-06-28T15:24:00.030 INFO:tasks.workunit.client.0.smithi178.stderr: -327> 2024-06-28T15:23:59.937+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.030 INFO:tasks.workunit.client.0.smithi178.stderr: -326> 2024-06-28T15:23:59.937+0000 7f2f2d26f640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.030 INFO:tasks.workunit.client.0.smithi178.stderr: -325> 2024-06-28T15:23:59.937+0000 7f2f2d26f640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.031 INFO:tasks.workunit.client.0.smithi178.stderr: -324> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.031 INFO:tasks.workunit.client.0.smithi178.stderr: -323> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.031 INFO:tasks.workunit.client.0.smithi178.stderr: -322> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.031 INFO:tasks.workunit.client.0.smithi178.stderr: -321> 2024-06-28T15:23:59.937+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.031 INFO:tasks.workunit.client.0.smithi178.stderr: -320> 2024-06-28T15:23:59.937+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.031 INFO:tasks.workunit.client.0.smithi178.stderr: -319> 2024-06-28T15:23:59.937+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.031 INFO:tasks.workunit.client.0.smithi178.stderr: -318> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_done global_id 6254 payload 824 2024-06-28T15:24:00.031 INFO:tasks.workunit.client.0.smithi178.stderr: -317> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient: _finish_hunting 0 2024-06-28T15:24:00.031 INFO:tasks.workunit.client.0.smithi178.stderr: -316> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 1 monclient: found mon.noname-c 2024-06-28T15:24:00.031 INFO:tasks.workunit.client.0.smithi178.stderr: -315> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.031 INFO:tasks.workunit.client.0.smithi178.stderr: -314> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.032 INFO:tasks.workunit.client.0.smithi178.stderr: -313> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:24:00.032 INFO:tasks.workunit.client.0.smithi178.stderr: -312> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: dump: 2024-06-28T15:24:00.032 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.032 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.032 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.032 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.032 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.032 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.032 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.032 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.033 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.033 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.033 INFO:tasks.workunit.client.0.smithi178.stderr: -311> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: _finish_auth 0 2024-06-28T15:24:00.033 INFO:tasks.workunit.client.0.smithi178.stderr: -310> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: _check_auth_tickets 2024-06-28T15:24:00.033 INFO:tasks.workunit.client.0.smithi178.stderr: -309> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:00.033 INFO:tasks.workunit.client.0.smithi178.stderr: -308> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.033 INFO:tasks.workunit.client.0.smithi178.stderr: -307> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:24:00.033 INFO:tasks.workunit.client.0.smithi178.stderr: -306> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: dump: 2024-06-28T15:24:00.033 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.033 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.033 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.033 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.033 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.033 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.034 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.034 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.034 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.034 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.034 INFO:tasks.workunit.client.0.smithi178.stderr: -305> 2024-06-28T15:23:59.941+0000 7f2f2da70640 4 set_mon_vals no callback set 2024-06-28T15:24:00.034 INFO:tasks.workunit.client.0.smithi178.stderr: -304> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 monclient: authenticate success, global_id 6254 2024-06-28T15:24:00.034 INFO:tasks.workunit.client.0.smithi178.stderr: -303> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: _renew_subs 2024-06-28T15:24:00.034 INFO:tasks.workunit.client.0.smithi178.stderr: -302> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.034 INFO:tasks.workunit.client.0.smithi178.stderr: -301> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: _renew_subs 2024-06-28T15:24:00.034 INFO:tasks.workunit.client.0.smithi178.stderr: -300> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.034 INFO:tasks.workunit.client.0.smithi178.stderr: -299> 2024-06-28T15:23:59.941+0000 7f2e127fc640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:24:00.034 INFO:tasks.workunit.client.0.smithi178.stderr: -298> 2024-06-28T15:23:59.941+0000 7f2e127fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:00.034 INFO:tasks.workunit.client.0.smithi178.stderr: -297> 2024-06-28T15:23:59.941+0000 7f2e127fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:00.035 INFO:tasks.workunit.client.0.smithi178.stderr: -296> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient: get_auth_request con 0x7f2e0406f690 auth_method 0 2024-06-28T15:24:00.035 INFO:tasks.workunit.client.0.smithi178.stderr: -295> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: get_monmap_and_config 2024-06-28T15:24:00.035 INFO:tasks.workunit.client.0.smithi178.stderr: -294> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: build_initial_monmap 2024-06-28T15:24:00.035 INFO:tasks.workunit.client.0.smithi178.stderr: -293> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 1 build_initial for_mkfs: 0 2024-06-28T15:24:00.035 INFO:tasks.workunit.client.0.smithi178.stderr: -292> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: monmap: 2024-06-28T15:24:00.035 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:00.035 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:00.035 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:00.035 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:00.035 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:00.035 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.035 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:00.035 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:00.036 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:00.036 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.036 INFO:tasks.workunit.client.0.smithi178.stderr: -291> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding auth protocol: cephx 2024-06-28T15:24:00.036 INFO:tasks.workunit.client.0.smithi178.stderr: -290> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding auth protocol: cephx 2024-06-28T15:24:00.036 INFO:tasks.workunit.client.0.smithi178.stderr: -289> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding auth protocol: cephx 2024-06-28T15:24:00.036 INFO:tasks.workunit.client.0.smithi178.stderr: -288> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: secure 2024-06-28T15:24:00.036 INFO:tasks.workunit.client.0.smithi178.stderr: -287> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: crc 2024-06-28T15:24:00.036 INFO:tasks.workunit.client.0.smithi178.stderr: -286> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: secure 2024-06-28T15:24:00.036 INFO:tasks.workunit.client.0.smithi178.stderr: -285> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: crc 2024-06-28T15:24:00.036 INFO:tasks.workunit.client.0.smithi178.stderr: -284> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: secure 2024-06-28T15:24:00.036 INFO:tasks.workunit.client.0.smithi178.stderr: -283> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: crc 2024-06-28T15:24:00.036 INFO:tasks.workunit.client.0.smithi178.stderr: -282> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: crc 2024-06-28T15:24:00.036 INFO:tasks.workunit.client.0.smithi178.stderr: -281> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: secure 2024-06-28T15:24:00.036 INFO:tasks.workunit.client.0.smithi178.stderr: -280> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: crc 2024-06-28T15:24:00.037 INFO:tasks.workunit.client.0.smithi178.stderr: -279> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: secure 2024-06-28T15:24:00.037 INFO:tasks.workunit.client.0.smithi178.stderr: -278> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: crc 2024-06-28T15:24:00.037 INFO:tasks.workunit.client.0.smithi178.stderr: -277> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: secure 2024-06-28T15:24:00.037 INFO:tasks.workunit.client.0.smithi178.stderr: -276> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.037 INFO:tasks.workunit.client.0.smithi178.stderr: -275> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: init 2024-06-28T15:24:00.037 INFO:tasks.workunit.client.0.smithi178.stderr: -274> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding auth protocol: cephx 2024-06-28T15:24:00.037 INFO:tasks.workunit.client.0.smithi178.stderr: -273> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding auth protocol: cephx 2024-06-28T15:24:00.037 INFO:tasks.workunit.client.0.smithi178.stderr: -272> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding auth protocol: cephx 2024-06-28T15:24:00.038 INFO:tasks.workunit.client.0.smithi178.stderr: -271> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: secure 2024-06-28T15:24:00.038 INFO:tasks.workunit.client.0.smithi178.stderr: -270> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: crc 2024-06-28T15:24:00.038 INFO:tasks.workunit.client.0.smithi178.stderr: -269> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: secure 2024-06-28T15:24:00.038 INFO:tasks.workunit.client.0.smithi178.stderr: -268> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: crc 2024-06-28T15:24:00.038 INFO:tasks.workunit.client.0.smithi178.stderr: -267> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: secure 2024-06-28T15:24:00.038 INFO:tasks.workunit.client.0.smithi178.stderr: -266> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: crc 2024-06-28T15:24:00.039 INFO:tasks.workunit.client.0.smithi178.stderr: -265> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: crc 2024-06-28T15:24:00.039 INFO:tasks.workunit.client.0.smithi178.stderr: -264> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: secure 2024-06-28T15:24:00.039 INFO:tasks.workunit.client.0.smithi178.stderr: -263> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: crc 2024-06-28T15:24:00.039 INFO:tasks.workunit.client.0.smithi178.stderr: -262> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: secure 2024-06-28T15:24:00.039 INFO:tasks.workunit.client.0.smithi178.stderr: -261> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: crc 2024-06-28T15:24:00.039 INFO:tasks.workunit.client.0.smithi178.stderr: -260> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: secure 2024-06-28T15:24:00.040 INFO:tasks.workunit.client.0.smithi178.stderr: -259> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.040 INFO:tasks.workunit.client.0.smithi178.stderr: -258> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.040 INFO:tasks.workunit.client.0.smithi178.stderr: -257> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command rotate-key cmddesc rotate-key hook 0x7fff993815b8 EEXIST 2024-06-28T15:24:00.040 INFO:tasks.workunit.client.0.smithi178.stderr: -256> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:00.040 INFO:tasks.workunit.client.0.smithi178.stderr: -255> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: _add_conns ranks=[1,0,2] 2024-06-28T15:24:00.040 INFO:tasks.workunit.client.0.smithi178.stderr: -254> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-b con 0x55f424e3b170 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:00.041 INFO:tasks.workunit.client.0.smithi178.stderr: -253> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-a con 0x55f424e3b550 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:00.041 INFO:tasks.workunit.client.0.smithi178.stderr: -252> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-c con 0x55f424e39c60 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:00.041 INFO:tasks.workunit.client.0.smithi178.stderr: -251> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.041 INFO:tasks.workunit.client.0.smithi178.stderr: -250> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.041 INFO:tasks.workunit.client.0.smithi178.stderr: -249> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.041 INFO:tasks.workunit.client.0.smithi178.stderr: -248> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:00.042 INFO:tasks.workunit.client.0.smithi178.stderr: -247> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): authenticate will time out at 2383.407396s 2024-06-28T15:24:00.042 INFO:tasks.workunit.client.0.smithi178.stderr: -246> 2024-06-28T15:23:59.941+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request con 0x55f424e39c60 auth_method 0 2024-06-28T15:24:00.042 INFO:tasks.workunit.client.0.smithi178.stderr: -245> 2024-06-28T15:23:59.941+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.042 INFO:tasks.workunit.client.0.smithi178.stderr: -244> 2024-06-28T15:23:59.941+0000 7f2f2d26f640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.042 INFO:tasks.workunit.client.0.smithi178.stderr: -243> 2024-06-28T15:23:59.941+0000 7f2f2d26f640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.042 INFO:tasks.workunit.client.0.smithi178.stderr: -242> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): get_auth_request con 0x55f424e3b550 auth_method 0 2024-06-28T15:24:00.043 INFO:tasks.workunit.client.0.smithi178.stderr: -241> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.043 INFO:tasks.workunit.client.0.smithi178.stderr: -240> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.043 INFO:tasks.workunit.client.0.smithi178.stderr: -239> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.043 INFO:tasks.workunit.client.0.smithi178.stderr: -238> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.043 INFO:tasks.workunit.client.0.smithi178.stderr: -237> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.043 INFO:tasks.workunit.client.0.smithi178.stderr: -236> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.044 INFO:tasks.workunit.client.0.smithi178.stderr: -235> 2024-06-28T15:23:59.941+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.044 INFO:tasks.workunit.client.0.smithi178.stderr: -234> 2024-06-28T15:23:59.941+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.044 INFO:tasks.workunit.client.0.smithi178.stderr: -233> 2024-06-28T15:23:59.941+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.044 INFO:tasks.workunit.client.0.smithi178.stderr: -232> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_done global_id 6663 payload 274 2024-06-28T15:24:00.044 INFO:tasks.workunit.client.0.smithi178.stderr: -231> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient: _finish_hunting 0 2024-06-28T15:24:00.044 INFO:tasks.workunit.client.0.smithi178.stderr: -230> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 1 monclient: found mon.noname-a 2024-06-28T15:24:00.045 INFO:tasks.workunit.client.0.smithi178.stderr: -229> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:24:00.045 INFO:tasks.workunit.client.0.smithi178.stderr: -228> 2024-06-28T15:23:59.941+0000 7f2f2ca6e640 10 monclient: get_auth_request con 0x55f424e3b170 auth_method 0 2024-06-28T15:24:00.045 INFO:tasks.workunit.client.0.smithi178.stderr: -227> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.045 INFO:tasks.workunit.client.0.smithi178.stderr: -226> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:24:00.045 INFO:tasks.workunit.client.0.smithi178.stderr: -225> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: dump: 2024-06-28T15:24:00.045 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.046 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.046 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.046 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.046 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.046 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.047 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.047 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.047 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.047 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.047 INFO:tasks.workunit.client.0.smithi178.stderr: -224> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: _finish_auth 0 2024-06-28T15:24:00.047 INFO:tasks.workunit.client.0.smithi178.stderr: -223> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: _check_auth_tickets 2024-06-28T15:24:00.048 INFO:tasks.workunit.client.0.smithi178.stderr: -222> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:00.048 INFO:tasks.workunit.client.0.smithi178.stderr: -221> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 monclient: authenticate success, global_id 6663 2024-06-28T15:24:00.048 INFO:tasks.workunit.client.0.smithi178.stderr: -220> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: get_monmap_and_config success 2024-06-28T15:24:00.048 INFO:tasks.workunit.client.0.smithi178.stderr: -219> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 4 set_mon_vals no callback set 2024-06-28T15:24:00.048 INFO:tasks.workunit.client.0.smithi178.stderr: -218> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.048 INFO:tasks.workunit.client.0.smithi178.stderr: -217> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:24:00.049 INFO:tasks.workunit.client.0.smithi178.stderr: -216> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: dump: 2024-06-28T15:24:00.049 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.049 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.049 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.049 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.049 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.050 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.050 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.050 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.050 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.050 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.050 INFO:tasks.workunit.client.0.smithi178.stderr: -215> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: shutdown 2024-06-28T15:24:00.051 INFO:tasks.workunit.client.0.smithi178.stderr: -214> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: build_initial_monmap 2024-06-28T15:24:00.051 INFO:tasks.workunit.client.0.smithi178.stderr: -213> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 1 build_initial for_mkfs: 0 2024-06-28T15:24:00.051 INFO:tasks.workunit.client.0.smithi178.stderr: -212> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: monmap: 2024-06-28T15:24:00.051 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:00.052 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:00.052 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:00.052 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:00.052 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:00.052 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.053 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:00.053 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:00.053 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:00.053 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.053 INFO:tasks.workunit.client.0.smithi178.stderr: -211> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding auth protocol: cephx 2024-06-28T15:24:00.053 INFO:tasks.workunit.client.0.smithi178.stderr: -210> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding auth protocol: cephx 2024-06-28T15:24:00.054 INFO:tasks.workunit.client.0.smithi178.stderr: -209> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding auth protocol: cephx 2024-06-28T15:24:00.054 INFO:tasks.workunit.client.0.smithi178.stderr: -208> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: secure 2024-06-28T15:24:00.054 INFO:tasks.workunit.client.0.smithi178.stderr: -207> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: crc 2024-06-28T15:24:00.054 INFO:tasks.workunit.client.0.smithi178.stderr: -206> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: secure 2024-06-28T15:24:00.054 INFO:tasks.workunit.client.0.smithi178.stderr: -205> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: crc 2024-06-28T15:24:00.054 INFO:tasks.workunit.client.0.smithi178.stderr: -204> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: secure 2024-06-28T15:24:00.055 INFO:tasks.workunit.client.0.smithi178.stderr: -203> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: crc 2024-06-28T15:24:00.055 INFO:tasks.workunit.client.0.smithi178.stderr: -202> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: crc 2024-06-28T15:24:00.055 INFO:tasks.workunit.client.0.smithi178.stderr: -201> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: secure 2024-06-28T15:24:00.055 INFO:tasks.workunit.client.0.smithi178.stderr: -200> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: crc 2024-06-28T15:24:00.055 INFO:tasks.workunit.client.0.smithi178.stderr: -199> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: secure 2024-06-28T15:24:00.055 INFO:tasks.workunit.client.0.smithi178.stderr: -198> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: crc 2024-06-28T15:24:00.056 INFO:tasks.workunit.client.0.smithi178.stderr: -197> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: secure 2024-06-28T15:24:00.056 INFO:tasks.workunit.client.0.smithi178.stderr: -196> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.056 INFO:tasks.workunit.client.0.smithi178.stderr: -195> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 1 librados: starting msgr at 2024-06-28T15:24:00.056 INFO:tasks.workunit.client.0.smithi178.stderr: -194> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 1 librados: starting objecter 2024-06-28T15:24:00.056 INFO:tasks.workunit.client.0.smithi178.stderr: -193> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command objecter_requests cmddesc objecter_requests hook 0x55f424dcd390 EEXIST 2024-06-28T15:24:00.056 INFO:tasks.workunit.client.0.smithi178.stderr: -192> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 1 librados: setting wanted keys 2024-06-28T15:24:00.057 INFO:tasks.workunit.client.0.smithi178.stderr: -191> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 1 librados: calling monclient init 2024-06-28T15:24:00.057 INFO:tasks.workunit.client.0.smithi178.stderr: -190> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: init 2024-06-28T15:24:00.057 INFO:tasks.workunit.client.0.smithi178.stderr: -189> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding auth protocol: cephx 2024-06-28T15:24:00.057 INFO:tasks.workunit.client.0.smithi178.stderr: -188> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding auth protocol: cephx 2024-06-28T15:24:00.057 INFO:tasks.workunit.client.0.smithi178.stderr: -187> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding auth protocol: cephx 2024-06-28T15:24:00.057 INFO:tasks.workunit.client.0.smithi178.stderr: -186> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: secure 2024-06-28T15:24:00.058 INFO:tasks.workunit.client.0.smithi178.stderr: -185> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: crc 2024-06-28T15:24:00.058 INFO:tasks.workunit.client.0.smithi178.stderr: -184> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: secure 2024-06-28T15:24:00.058 INFO:tasks.workunit.client.0.smithi178.stderr: -183> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: crc 2024-06-28T15:24:00.058 INFO:tasks.workunit.client.0.smithi178.stderr: -182> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: secure 2024-06-28T15:24:00.058 INFO:tasks.workunit.client.0.smithi178.stderr: -181> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: crc 2024-06-28T15:24:00.059 INFO:tasks.workunit.client.0.smithi178.stderr: -180> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: crc 2024-06-28T15:24:00.059 INFO:tasks.workunit.client.0.smithi178.stderr: -179> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: secure 2024-06-28T15:24:00.059 INFO:tasks.workunit.client.0.smithi178.stderr: -178> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: crc 2024-06-28T15:24:00.059 INFO:tasks.workunit.client.0.smithi178.stderr: -177> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: secure 2024-06-28T15:24:00.059 INFO:tasks.workunit.client.0.smithi178.stderr: -176> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: crc 2024-06-28T15:24:00.059 INFO:tasks.workunit.client.0.smithi178.stderr: -175> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: secure 2024-06-28T15:24:00.060 INFO:tasks.workunit.client.0.smithi178.stderr: -174> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.060 INFO:tasks.workunit.client.0.smithi178.stderr: -173> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.060 INFO:tasks.workunit.client.0.smithi178.stderr: -172> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command rotate-key cmddesc rotate-key hook 0x55f424e03378 EEXIST 2024-06-28T15:24:00.060 INFO:tasks.workunit.client.0.smithi178.stderr: -171> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:00.060 INFO:tasks.workunit.client.0.smithi178.stderr: -170> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: _add_conns ranks=[2,1,0] 2024-06-28T15:24:00.061 INFO:tasks.workunit.client.0.smithi178.stderr: -169> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-c con 0x55f424e39c60 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:00.061 INFO:tasks.workunit.client.0.smithi178.stderr: -168> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-b con 0x55f424e3b170 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:00.061 INFO:tasks.workunit.client.0.smithi178.stderr: -167> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-a con 0x55f424e37760 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:00.061 INFO:tasks.workunit.client.0.smithi178.stderr: -166> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.061 INFO:tasks.workunit.client.0.smithi178.stderr: -165> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.061 INFO:tasks.workunit.client.0.smithi178.stderr: -164> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.062 INFO:tasks.workunit.client.0.smithi178.stderr: -163> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:00.062 INFO:tasks.workunit.client.0.smithi178.stderr: -162> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): authenticate will time out at 2383.410001s 2024-06-28T15:24:00.062 INFO:tasks.workunit.client.0.smithi178.stderr: -161> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): get_auth_request con 0x55f424e39c60 auth_method 0 2024-06-28T15:24:00.062 INFO:tasks.workunit.client.0.smithi178.stderr: -160> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.062 INFO:tasks.workunit.client.0.smithi178.stderr: -159> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.062 INFO:tasks.workunit.client.0.smithi178.stderr: -158> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.063 INFO:tasks.workunit.client.0.smithi178.stderr: -157> 2024-06-28T15:23:59.945+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request con 0x55f424e37760 auth_method 0 2024-06-28T15:24:00.063 INFO:tasks.workunit.client.0.smithi178.stderr: -156> 2024-06-28T15:23:59.945+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.063 INFO:tasks.workunit.client.0.smithi178.stderr: -155> 2024-06-28T15:23:59.945+0000 7f2f2d26f640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.063 INFO:tasks.workunit.client.0.smithi178.stderr: -154> 2024-06-28T15:23:59.945+0000 7f2f2d26f640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.063 INFO:tasks.workunit.client.0.smithi178.stderr: -153> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.063 INFO:tasks.workunit.client.0.smithi178.stderr: -152> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.064 INFO:tasks.workunit.client.0.smithi178.stderr: -151> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.064 INFO:tasks.workunit.client.0.smithi178.stderr: -150> 2024-06-28T15:23:59.945+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.064 INFO:tasks.workunit.client.0.smithi178.stderr: -149> 2024-06-28T15:23:59.945+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.064 INFO:tasks.workunit.client.0.smithi178.stderr: -148> 2024-06-28T15:23:59.945+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.064 INFO:tasks.workunit.client.0.smithi178.stderr: -147> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_done global_id 6260 payload 824 2024-06-28T15:24:00.064 INFO:tasks.workunit.client.0.smithi178.stderr: -146> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient: _finish_hunting 0 2024-06-28T15:24:00.065 INFO:tasks.workunit.client.0.smithi178.stderr: -145> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 1 monclient: found mon.noname-c 2024-06-28T15:24:00.065 INFO:tasks.workunit.client.0.smithi178.stderr: -144> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.065 INFO:tasks.workunit.client.0.smithi178.stderr: -143> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.065 INFO:tasks.workunit.client.0.smithi178.stderr: -142> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:24:00.065 INFO:tasks.workunit.client.0.smithi178.stderr: -141> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: dump: 2024-06-28T15:24:00.065 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.066 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.066 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.066 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.066 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.066 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.066 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.066 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.066 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.067 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.067 INFO:tasks.workunit.client.0.smithi178.stderr: -140> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: _finish_auth 0 2024-06-28T15:24:00.067 INFO:tasks.workunit.client.0.smithi178.stderr: -139> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: _check_auth_tickets 2024-06-28T15:24:00.067 INFO:tasks.workunit.client.0.smithi178.stderr: -138> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:00.067 INFO:tasks.workunit.client.0.smithi178.stderr: -137> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.067 INFO:tasks.workunit.client.0.smithi178.stderr: -136> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:24:00.067 INFO:tasks.workunit.client.0.smithi178.stderr: -135> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: dump: 2024-06-28T15:24:00.067 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.068 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.068 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.068 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.068 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.068 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.068 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.068 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.068 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.069 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.069 INFO:tasks.workunit.client.0.smithi178.stderr: -134> 2024-06-28T15:23:59.945+0000 7f2e03fff640 4 set_mon_vals no callback set 2024-06-28T15:24:00.069 INFO:tasks.workunit.client.0.smithi178.stderr: -133> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 monclient: authenticate success, global_id 6260 2024-06-28T15:24:00.069 INFO:tasks.workunit.client.0.smithi178.stderr: -132> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: _renew_subs 2024-06-28T15:24:00.069 INFO:tasks.workunit.client.0.smithi178.stderr: -131> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.069 INFO:tasks.workunit.client.0.smithi178.stderr: -130> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: _renew_subs 2024-06-28T15:24:00.069 INFO:tasks.workunit.client.0.smithi178.stderr: -129> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.069 INFO:tasks.workunit.client.0.smithi178.stderr: -128> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 1 librados: init done 2024-06-28T15:24:00.070 INFO:tasks.workunit.client.0.smithi178.stderr: -127> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command cr dump hook 0x55f424e34308 2024-06-28T15:24:00.070 INFO:tasks.workunit.client.0.smithi178.stderr: -126> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.070 INFO:tasks.workunit.client.0.smithi178.stderr: -125> 2024-06-28T15:23:59.949+0000 7f2e027fc640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:24:00.070 INFO:tasks.workunit.client.0.smithi178.stderr: -124> 2024-06-28T15:23:59.949+0000 7f2e027fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:00.070 INFO:tasks.workunit.client.0.smithi178.stderr: -123> 2024-06-28T15:23:59.949+0000 7f2e027fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:00.070 INFO:tasks.workunit.client.0.smithi178.stderr: -122> 2024-06-28T15:23:59.949+0000 7f2f2c26d640 10 monclient: get_auth_request con 0x7f2df806f780 auth_method 0 2024-06-28T15:24:00.070 INFO:tasks.workunit.client.0.smithi178.stderr: -121> 2024-06-28T15:23:59.949+0000 7f2f2d26f640 10 monclient: get_auth_request con 0x55f424e593e0 auth_method 0 2024-06-28T15:24:00.070 INFO:tasks.workunit.client.0.smithi178.stderr: -120> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.070 INFO:tasks.workunit.client.0.smithi178.stderr: -119> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 realm 2024-06-28T15:24:00.070 INFO:tasks.workunit.client.0.smithi178.stderr: -118> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.070 INFO:tasks.workunit.client.0.smithi178.stderr: -117> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.071 INFO:tasks.workunit.client.0.smithi178.stderr: -116> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:00.071 INFO:tasks.workunit.client.0.smithi178.stderr: -115> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.071 INFO:tasks.workunit.client.0.smithi178.stderr: -114> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.071 INFO:tasks.workunit.client.0.smithi178.stderr: -113> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.071 INFO:tasks.workunit.client.0.smithi178.stderr: -112> 2024-06-28T15:23:59.949+0000 7f2f2ca6e640 10 monclient: get_auth_request con 0x55f424e5d830 auth_method 0 2024-06-28T15:24:00.071 INFO:tasks.workunit.client.0.smithi178.stderr: -111> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:00.071 INFO:tasks.workunit.client.0.smithi178.stderr: -110> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.071 INFO:tasks.workunit.client.0.smithi178.stderr: -109> 2024-06-28T15:23:59.953+0000 7f2f2c26d640 10 monclient: get_auth_request con 0x55f424e60f70 auth_method 0 2024-06-28T15:24:00.071 INFO:tasks.workunit.client.0.smithi178.stderr: -108> 2024-06-28T15:23:59.953+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:00.071 INFO:tasks.workunit.client.0.smithi178.stderr: -107> 2024-06-28T15:23:59.953+0000 7f2f2ef5d980 20 searching for the correct realm 2024-06-28T15:24:00.071 INFO:tasks.workunit.client.0.smithi178.stderr: -106> 2024-06-28T15:23:59.953+0000 7f2f2d26f640 10 monclient: get_auth_request con 0x7f2f14085040 auth_method 0 2024-06-28T15:24:00.071 INFO:tasks.workunit.client.0.smithi178.stderr: -105> 2024-06-28T15:23:59.953+0000 7f2f2ca6e640 10 monclient: get_auth_request con 0x7f2f2402a040 auth_method 0 2024-06-28T15:24:00.071 INFO:tasks.workunit.client.0.smithi178.stderr: -104> 2024-06-28T15:23:59.953+0000 7f2f2c26d640 10 monclient: get_auth_request con 0x7f2f1c0dc060 auth_method 0 2024-06-28T15:24:00.072 INFO:tasks.workunit.client.0.smithi178.stderr: -103> 2024-06-28T15:23:59.957+0000 7f2f2c26d640 0 -- 172.21.15.178:0/2278050652 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f2f1c0dc060 msgr2=0x7f2f1c0c83d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:00.072 INFO:tasks.workunit.client.0.smithi178.stderr: -102> 2024-06-28T15:23:59.957+0000 7f2e027fc640 1 client.6260.objecter ms_handle_reset 0x7f2f1c0dc060 session 0x7f2f1c0cd050 osd.4 2024-06-28T15:24:00.072 INFO:tasks.workunit.client.0.smithi178.stderr: -101> 2024-06-28T15:23:59.957+0000 7f2e027fc640 10 monclient: _renew_subs 2024-06-28T15:24:00.072 INFO:tasks.workunit.client.0.smithi178.stderr: -100> 2024-06-28T15:23:59.957+0000 7f2e027fc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.072 INFO:tasks.workunit.client.0.smithi178.stderr: -99> 2024-06-28T15:23:59.957+0000 7f2f2d26f640 10 monclient: get_auth_request con 0x7f2f1408d380 auth_method 0 2024-06-28T15:24:00.072 INFO:tasks.workunit.client.0.smithi178.stderr: -98> 2024-06-28T15:23:59.957+0000 7f2f2c26d640 10 monclient: get_auth_request con 0x7f2df807fda0 auth_method 0 2024-06-28T15:24:00.072 INFO:tasks.workunit.client.0.smithi178.stderr: -97> 2024-06-28T15:23:59.961+0000 7f2f2ca6e640 10 monclient: get_auth_request con 0x7f2f14060790 auth_method 0 2024-06-28T15:24:00.072 INFO:tasks.workunit.client.0.smithi178.stderr: -96> 2024-06-28T15:23:59.965+0000 7f2f2c26d640 0 -- 172.21.15.178:0/2278050652 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f2df807fda0 msgr2=0x7f2df80821d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:00.072 INFO:tasks.workunit.client.0.smithi178.stderr: -95> 2024-06-28T15:23:59.965+0000 7f2e027fc640 1 client.6260.objecter ms_handle_reset 0x7f2df807fda0 session 0x7f2f1c0cd050 osd.4 2024-06-28T15:24:00.072 INFO:tasks.workunit.client.0.smithi178.stderr: -94> 2024-06-28T15:23:59.965+0000 7f2f2c26d640 10 monclient: get_auth_request con 0x7f2f1c0dc060 auth_method 0 2024-06-28T15:24:00.072 INFO:tasks.workunit.client.0.smithi178.stderr: -93> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:00.072 INFO:tasks.workunit.client.0.smithi178.stderr: -92> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:00.073 INFO:tasks.workunit.client.0.smithi178.stderr: -91> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:00.073 INFO:tasks.workunit.client.0.smithi178.stderr: -90> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:00.073 INFO:tasks.workunit.client.0.smithi178.stderr: -89> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:00.073 INFO:tasks.workunit.client.0.smithi178.stderr: -88> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:00.073 INFO:tasks.workunit.client.0.smithi178.stderr: -87> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.073 INFO:tasks.workunit.client.0.smithi178.stderr: -86> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.073 INFO:tasks.workunit.client.0.smithi178.stderr: -85> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.073 INFO:tasks.workunit.client.0.smithi178.stderr: -84> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:00.073 INFO:tasks.workunit.client.0.smithi178.stderr: -83> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.073 INFO:tasks.workunit.client.0.smithi178.stderr: -82> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:00.073 INFO:tasks.workunit.client.0.smithi178.stderr: -81> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 zone default found 2024-06-28T15:24:00.073 INFO:tasks.workunit.client.0.smithi178.stderr: -80> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 4 Realm: () 2024-06-28T15:24:00.073 INFO:tasks.workunit.client.0.smithi178.stderr: -79> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:00.073 INFO:tasks.workunit.client.0.smithi178.stderr: -78> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:00.074 INFO:tasks.workunit.client.0.smithi178.stderr: -77> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:00.074 INFO:tasks.workunit.client.0.smithi178.stderr: -76> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 zonegroup default 2024-06-28T15:24:00.074 INFO:tasks.workunit.client.0.smithi178.stderr: -75> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.074 INFO:tasks.workunit.client.0.smithi178.stderr: -74> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.074 INFO:tasks.workunit.client.0.smithi178.stderr: -73> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.074 INFO:tasks.workunit.client.0.smithi178.stderr: -72> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.074 INFO:tasks.workunit.client.0.smithi178.stderr: -71> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.074 INFO:tasks.workunit.client.0.smithi178.stderr: -70> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.074 INFO:tasks.workunit.client.0.smithi178.stderr: -69> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 started sync module instance, tier type = 2024-06-28T15:24:00.074 INFO:tasks.workunit.client.0.smithi178.stderr: -68> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:00.074 INFO:tasks.workunit.client.0.smithi178.stderr: -67> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=0 2024-06-28T15:24:00.074 INFO:tasks.workunit.client.0.smithi178.stderr: -66> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=1 2024-06-28T15:24:00.074 INFO:tasks.workunit.client.0.smithi178.stderr: -65> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=2 2024-06-28T15:24:00.075 INFO:tasks.workunit.client.0.smithi178.stderr: -64> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=3 2024-06-28T15:24:00.075 INFO:tasks.workunit.client.0.smithi178.stderr: -63> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=4 2024-06-28T15:24:00.075 INFO:tasks.workunit.client.0.smithi178.stderr: -62> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=5 2024-06-28T15:24:00.075 INFO:tasks.workunit.client.0.smithi178.stderr: -61> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=6 2024-06-28T15:24:00.075 INFO:tasks.workunit.client.0.smithi178.stderr: -60> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=7 2024-06-28T15:24:00.075 INFO:tasks.workunit.client.0.smithi178.stderr: -59> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:00.075 INFO:tasks.workunit.client.0.smithi178.stderr: -58> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding auth protocol: cephx 2024-06-28T15:24:00.075 INFO:tasks.workunit.client.0.smithi178.stderr: -57> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding auth protocol: cephx 2024-06-28T15:24:00.075 INFO:tasks.workunit.client.0.smithi178.stderr: -56> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding auth protocol: cephx 2024-06-28T15:24:00.075 INFO:tasks.workunit.client.0.smithi178.stderr: -55> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: secure 2024-06-28T15:24:00.075 INFO:tasks.workunit.client.0.smithi178.stderr: -54> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: crc 2024-06-28T15:24:00.075 INFO:tasks.workunit.client.0.smithi178.stderr: -53> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: secure 2024-06-28T15:24:00.075 INFO:tasks.workunit.client.0.smithi178.stderr: -52> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: crc 2024-06-28T15:24:00.075 INFO:tasks.workunit.client.0.smithi178.stderr: -51> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: secure 2024-06-28T15:24:00.076 INFO:tasks.workunit.client.0.smithi178.stderr: -50> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: crc 2024-06-28T15:24:00.076 INFO:tasks.workunit.client.0.smithi178.stderr: -49> 2024-06-28T15:23:59.993+0000 7f2de37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:00.076 INFO:tasks.workunit.client.0.smithi178.stderr: -48> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: crc 2024-06-28T15:24:00.076 INFO:tasks.workunit.client.0.smithi178.stderr: -47> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: secure 2024-06-28T15:24:00.076 INFO:tasks.workunit.client.0.smithi178.stderr: -46> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: crc 2024-06-28T15:24:00.076 INFO:tasks.workunit.client.0.smithi178.stderr: -45> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: secure 2024-06-28T15:24:00.076 INFO:tasks.workunit.client.0.smithi178.stderr: -44> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: crc 2024-06-28T15:24:00.076 INFO:tasks.workunit.client.0.smithi178.stderr: -43> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: secure 2024-06-28T15:24:00.076 INFO:tasks.workunit.client.0.smithi178.stderr: -42> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.076 INFO:tasks.workunit.client.0.smithi178.stderr: -41> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:00.076 INFO:tasks.workunit.client.0.smithi178.stderr: -40> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:00.076 INFO:tasks.workunit.client.0.smithi178.stderr: -39> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command cache list hook 0x55f424e4da90 2024-06-28T15:24:00.076 INFO:tasks.workunit.client.0.smithi178.stderr: -38> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command cache inspect hook 0x55f424e4da90 2024-06-28T15:24:00.076 INFO:tasks.workunit.client.0.smithi178.stderr: -37> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command cache erase hook 0x55f424e4da90 2024-06-28T15:24:00.077 INFO:tasks.workunit.client.0.smithi178.stderr: -36> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command cache zap hook 0x55f424e4da90 2024-06-28T15:24:00.077 INFO:tasks.workunit.client.0.smithi178.stderr: -35> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 note: GC not initialized 2024-06-28T15:24:00.077 INFO:tasks.workunit.client.0.smithi178.stderr: -34> 2024-06-28T15:23:59.993+0000 7f2de27e4640 20 reqs_thread_entry: start 2024-06-28T15:24:00.077 INFO:tasks.workunit.client.0.smithi178.stderr: -33> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command sync trace show hook 0x7f2f1c004510 2024-06-28T15:24:00.077 INFO:tasks.workunit.client.0.smithi178.stderr: -32> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command sync trace history hook 0x7f2f1c004510 2024-06-28T15:24:00.077 INFO:tasks.workunit.client.0.smithi178.stderr: -31> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command sync trace active hook 0x7f2f1c004510 2024-06-28T15:24:00.077 INFO:tasks.workunit.client.0.smithi178.stderr: -30> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command sync trace active_short hook 0x7f2f1c004510 2024-06-28T15:24:00.077 INFO:tasks.workunit.client.0.smithi178.stderr: -29> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:00.077 INFO:tasks.workunit.client.0.smithi178.stderr: -28> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 Filter name: none 2024-06-28T15:24:00.077 INFO:tasks.workunit.client.0.smithi178.stderr: -27> 2024-06-28T15:23:59.993+0000 7f2ddbfff640 20 reqs_thread_entry: start 2024-06-28T15:24:00.077 INFO:tasks.workunit.client.0.smithi178.stderr: -26> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-06-28T15:24:00.077 INFO:tasks.workunit.client.0.smithi178.stderr: -25> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.077 INFO:tasks.workunit.client.0.smithi178.stderr: -24> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:24:00.077 INFO:tasks.workunit.client.0.smithi178.stderr: -23> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-06-28T15:24:00.078 INFO:tasks.workunit.client.0.smithi178.stderr: -22> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-06-28T15:24:00.078 INFO:tasks.workunit.client.0.smithi178.stderr: -21> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 : miss 2024-06-28T15:24:00.078 INFO:tasks.workunit.client.0.smithi178.stderr: -20> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.078 INFO:tasks.workunit.client.0.smithi178.stderr: -19> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:24:00.078 INFO:tasks.workunit.client.0.smithi178.stderr: -18> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:24:00.078 INFO:tasks.workunit.client.0.smithi178.stderr: -17> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:24:00.078 INFO:tasks.workunit.client.0.smithi178.stderr: -16> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:24:00.078 INFO:tasks.workunit.client.0.smithi178.stderr: -15> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:00.078 INFO:tasks.workunit.client.0.smithi178.stderr: -14> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 2024-06-28T15:24:00.078 INFO:tasks.workunit.client.0.smithi178.stderr: -13> 2024-06-28T15:24:00.001+0000 7f2f2ef5d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:24:00.078 INFO:tasks.workunit.client.0.smithi178.stderr: -12> 2024-06-28T15:24:00.001+0000 7f2f2ef5d980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:24:00.078 INFO:tasks.workunit.client.0.smithi178.stderr: -11> 2024-06-28T15:24:00.001+0000 7f2f2ef5d980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:00.078 INFO:tasks.workunit.client.0.smithi178.stderr: -10> 2024-06-28T15:24:00.001+0000 7f2f2ef5d980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24, ofs0, ns] 2024-06-28T15:24:00.079 INFO:tasks.workunit.client.0.smithi178.stderr: -9> 2024-06-28T15:24:00.001+0000 7f2e03fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595085 cookie 94507079483440 notifier 6260 bl.length()=991 2024-06-28T15:24:00.079 INFO:tasks.workunit.client.0.smithi178.stderr: -8> 2024-06-28T15:24:00.001+0000 7f2e03fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:24:00.079 INFO:tasks.workunit.client.0.smithi178.stderr: -7> 2024-06-28T15:24:00.001+0000 7f2e03fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:24:00.079 INFO:tasks.workunit.client.0.smithi178.stderr: -6> 2024-06-28T15:24:00.001+0000 7f2e03fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:00.079 INFO:tasks.workunit.client.0.smithi178.stderr: -5> 2024-06-28T15:24:00.005+0000 7f2f2ef5d980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): orig_sources= new_sources= 2024-06-28T15:24:00.079 INFO:tasks.workunit.client.0.smithi178.stderr: -4> 2024-06-28T15:24:00.005+0000 7f2f2ef5d980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): potential sources added=[] removed=[] 2024-06-28T15:24:00.079 INFO:tasks.workunit.client.0.smithi178.stderr: -3> 2024-06-28T15:24:00.005+0000 7f2f2ef5d980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): orig_dests= new_dests= 2024-06-28T15:24:00.079 INFO:tasks.workunit.client.0.smithi178.stderr: -2> 2024-06-28T15:24:00.005+0000 7f2f2ef5d980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): potential dests added=[] removed=[] 2024-06-28T15:24:00.079 INFO:tasks.workunit.client.0.smithi178.stderr: -1> 2024-06-28T15:24:00.005+0000 7f2f2ef5d980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f2f2ef5d980 time 2024-06-28T15:24:00.010620+0000 2024-06-28T15:24:00.079 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:24:00.079 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.079 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:24:00.079 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f2f32834c91] 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f2f32834e45] 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55f421d834b1] 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2bc) [0x55f421d94a3c] 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: 5: main() 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2f3169dd90] 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: 7: __libc_start_main() 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: 8: _start() 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: 0> 2024-06-28T15:24:00.005+0000 7f2f2ef5d980 -1 *** Caught signal (Aborted) ** 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7f2f2ef5d980 thread_name:radosgw-admin 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f2f316b6520] 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:24:00.080 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:24:00.081 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:24:00.081 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f2f32834ceb] 2024-06-28T15:24:00.081 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f2f32834e45] 2024-06-28T15:24:00.081 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55f421d834b1] 2024-06-28T15:24:00.081 INFO:tasks.workunit.client.0.smithi178.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2bc) [0x55f421d94a3c] 2024-06-28T15:24:00.081 INFO:tasks.workunit.client.0.smithi178.stderr: 9: main() 2024-06-28T15:24:00.081 INFO:tasks.workunit.client.0.smithi178.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2f3169dd90] 2024-06-28T15:24:00.081 INFO:tasks.workunit.client.0.smithi178.stderr: 11: __libc_start_main() 2024-06-28T15:24:00.081 INFO:tasks.workunit.client.0.smithi178.stderr: 12: _start() 2024-06-28T15:24:00.081 INFO:tasks.workunit.client.0.smithi178.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-28T15:24:00.081 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.081 INFO:tasks.workunit.client.0.smithi178.stderr:--- logging levels --- 2024-06-28T15:24:00.081 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 none 2024-06-28T15:24:00.081 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 lockdep 2024-06-28T15:24:00.082 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 context 2024-06-28T15:24:00.082 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 crush 2024-06-28T15:24:00.082 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds 2024-06-28T15:24:00.082 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_balancer 2024-06-28T15:24:00.082 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_locker 2024-06-28T15:24:00.082 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log 2024-06-28T15:24:00.082 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log_expire 2024-06-28T15:24:00.082 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_migrator 2024-06-28T15:24:00.082 INFO:tasks.workunit.client.0.smithi178.stderr: 3/ 5 mds_quiesce 2024-06-28T15:24:00.082 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 buffer 2024-06-28T15:24:00.082 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 timer 2024-06-28T15:24:00.082 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 filer 2024-06-28T15:24:00.083 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 striper 2024-06-28T15:24:00.083 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 objecter 2024-06-28T15:24:00.083 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rados 2024-06-28T15:24:00.083 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd 2024-06-28T15:24:00.083 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_mirror 2024-06-28T15:24:00.083 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_replay 2024-06-28T15:24:00.083 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_pwl 2024-06-28T15:24:00.083 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 journaler 2024-06-28T15:24:00.083 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objectcacher 2024-06-28T15:24:00.083 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 immutable_obj_cache 2024-06-28T15:24:00.083 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 client 2024-06-28T15:24:00.083 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 osd 2024-06-28T15:24:00.083 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 optracker 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objclass 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 filestore 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 journal 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 ms 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mon 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 0/10 monc 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 paxos 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 tp 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 auth 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 crypto 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 finisher 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 reserver 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 heartbeatmap 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 perfcounter 2024-06-28T15:24:00.084 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw 2024-06-28T15:24:00.085 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_sync 2024-06-28T15:24:00.085 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_datacache 2024-06-28T15:24:00.085 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_access 2024-06-28T15:24:00.085 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_dbstore 2024-06-28T15:24:00.085 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_flight 2024-06-28T15:24:00.085 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_lifecycle 2024-06-28T15:24:00.085 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw_notification 2024-06-28T15:24:00.085 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 javaclient 2024-06-28T15:24:00.085 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 asok 2024-06-28T15:24:00.085 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 throttle 2024-06-28T15:24:00.085 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 refs 2024-06-28T15:24:00.085 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 compressor 2024-06-28T15:24:00.085 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluestore 2024-06-28T15:24:00.085 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluefs 2024-06-28T15:24:00.086 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 bdev 2024-06-28T15:24:00.086 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 kstore 2024-06-28T15:24:00.086 INFO:tasks.workunit.client.0.smithi178.stderr: 4/ 5 rocksdb 2024-06-28T15:24:00.086 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 fuse 2024-06-28T15:24:00.086 INFO:tasks.workunit.client.0.smithi178.stderr: 2/ 5 mgr 2024-06-28T15:24:00.086 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mgrc 2024-06-28T15:24:00.086 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 dpdk 2024-06-28T15:24:00.086 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 eventtrace 2024-06-28T15:24:00.086 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 prioritycache 2024-06-28T15:24:00.086 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 test 2024-06-28T15:24:00.086 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephfs_mirror 2024-06-28T15:24:00.086 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephsqlite 2024-06-28T15:24:00.086 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 crimson_interrupt 2024-06-28T15:24:00.087 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore 2024-06-28T15:24:00.087 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_onode 2024-06-28T15:24:00.087 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_odata 2024-06-28T15:24:00.087 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_omap 2024-06-28T15:24:00.087 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_tm 2024-06-28T15:24:00.087 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_t 2024-06-28T15:24:00.087 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cleaner 2024-06-28T15:24:00.087 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_epm 2024-06-28T15:24:00.087 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_lba 2024-06-28T15:24:00.087 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_fixedkv_tree 2024-06-28T15:24:00.087 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cache 2024-06-28T15:24:00.087 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_journal 2024-06-28T15:24:00.087 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_device 2024-06-28T15:24:00.087 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_backref 2024-06-28T15:24:00.088 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 alienstore 2024-06-28T15:24:00.088 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mclock 2024-06-28T15:24:00.088 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cyanstore 2024-06-28T15:24:00.088 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 ceph_exporter 2024-06-28T15:24:00.088 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 memstore 2024-06-28T15:24:00.088 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 trace 2024-06-28T15:24:00.088 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 ceph_dedup 2024-06-28T15:24:00.088 INFO:tasks.workunit.client.0.smithi178.stderr: -2/-2 (syslog threshold) 2024-06-28T15:24:00.088 INFO:tasks.workunit.client.0.smithi178.stderr: 99/99 (stderr threshold) 2024-06-28T15:24:00.088 INFO:tasks.workunit.client.0.smithi178.stderr:--- pthread ID / name mapping for recent threads --- 2024-06-28T15:24:00.088 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2ddbfff640 / http_manager 2024-06-28T15:24:00.088 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2de27e4640 / http_manager 2024-06-28T15:24:00.088 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2de37e6640 / rgw_dt_lg_renew 2024-06-28T15:24:00.088 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2e027fc640 / ms_dispatch 2024-06-28T15:24:00.089 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2e03fff640 / io_context_pool 2024-06-28T15:24:00.089 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2e127fc640 / ms_dispatch 2024-06-28T15:24:00.089 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2e1960a640 / ms_dispatch 2024-06-28T15:24:00.089 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2e1ae0d640 / io_context_pool 2024-06-28T15:24:00.089 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2f2c26d640 / msgr-worker-2 2024-06-28T15:24:00.089 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2f2ca6e640 / msgr-worker-1 2024-06-28T15:24:00.089 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2f2d26f640 / msgr-worker-0 2024-06-28T15:24:00.089 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2f2da70640 / io_context_pool 2024-06-28T15:24:00.089 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2f2ef5d980 / radosgw-admin 2024-06-28T15:24:00.089 INFO:tasks.workunit.client.0.smithi178.stderr: max_recent 500 2024-06-28T15:24:00.089 INFO:tasks.workunit.client.0.smithi178.stderr: max_new 1000 2024-06-28T15:24:00.089 INFO:tasks.workunit.client.0.smithi178.stderr: log_file /var/log/ceph/ceph-client.admin.88401.log 2024-06-28T15:24:00.089 INFO:tasks.workunit.client.0.smithi178.stderr:--- end dump of recent events --- 2024-06-28T15:24:00.090 INFO:tasks.workunit.client.0.smithi178.stderr:--- begin dump of recent events --- 2024-06-28T15:24:00.090 INFO:tasks.workunit.client.0.smithi178.stderr: -499> 2024-06-28T15:23:59.925+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.090 INFO:tasks.workunit.client.0.smithi178.stderr: -498> 2024-06-28T15:23:59.925+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.090 INFO:tasks.workunit.client.0.smithi178.stderr: -497> 2024-06-28T15:23:59.925+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_done global_id 6654 payload 274 2024-06-28T15:24:00.090 INFO:tasks.workunit.client.0.smithi178.stderr: -496> 2024-06-28T15:23:59.925+0000 7f2f2d26f640 10 monclient: _finish_hunting 0 2024-06-28T15:24:00.090 INFO:tasks.workunit.client.0.smithi178.stderr: -495> 2024-06-28T15:23:59.925+0000 7f2f2d26f640 1 monclient: found mon.noname-a 2024-06-28T15:24:00.090 INFO:tasks.workunit.client.0.smithi178.stderr: -494> 2024-06-28T15:23:59.925+0000 7f2f2d26f640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:24:00.090 INFO:tasks.workunit.client.0.smithi178.stderr: -493> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.090 INFO:tasks.workunit.client.0.smithi178.stderr: -492> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:24:00.090 INFO:tasks.workunit.client.0.smithi178.stderr: -491> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: dump: 2024-06-28T15:24:00.090 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.090 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.091 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.091 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.091 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.091 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.091 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.091 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.091 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.091 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.091 INFO:tasks.workunit.client.0.smithi178.stderr: -490> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: _finish_auth 0 2024-06-28T15:24:00.091 INFO:tasks.workunit.client.0.smithi178.stderr: -489> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: _check_auth_tickets 2024-06-28T15:24:00.091 INFO:tasks.workunit.client.0.smithi178.stderr: -488> 2024-06-28T15:23:59.925+0000 7f2f2ef5d980 5 monclient: authenticate success, global_id 6654 2024-06-28T15:24:00.091 INFO:tasks.workunit.client.0.smithi178.stderr: -487> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:00.092 INFO:tasks.workunit.client.0.smithi178.stderr: -486> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.092 INFO:tasks.workunit.client.0.smithi178.stderr: -485> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:24:00.092 INFO:tasks.workunit.client.0.smithi178.stderr: -484> 2024-06-28T15:23:59.925+0000 7f2e1ae0d640 10 monclient: dump: 2024-06-28T15:24:00.092 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.092 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.092 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.092 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.092 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.092 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.092 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.092 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.092 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.092 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.093 INFO:tasks.workunit.client.0.smithi178.stderr: -483> 2024-06-28T15:23:59.925+0000 7f2f2ef5d980 10 monclient: get_monmap_and_config success 2024-06-28T15:24:00.093 INFO:tasks.workunit.client.0.smithi178.stderr: -482> 2024-06-28T15:23:59.925+0000 7f2f2ef5d980 4 set_mon_vals no callback set 2024-06-28T15:24:00.093 INFO:tasks.workunit.client.0.smithi178.stderr: -481> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: shutdown 2024-06-28T15:24:00.093 INFO:tasks.workunit.client.0.smithi178.stderr: -480> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) unregister_commands rotate-key 2024-06-28T15:24:00.093 INFO:tasks.workunit.client.0.smithi178.stderr: -479> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: build_initial_monmap 2024-06-28T15:24:00.093 INFO:tasks.workunit.client.0.smithi178.stderr: -478> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 1 build_initial for_mkfs: 0 2024-06-28T15:24:00.093 INFO:tasks.workunit.client.0.smithi178.stderr: -477> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: monmap: 2024-06-28T15:24:00.093 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:00.093 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:00.093 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:00.093 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:00.093 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:00.093 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.093 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:00.094 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:00.094 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:00.094 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.094 INFO:tasks.workunit.client.0.smithi178.stderr: -476> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding auth protocol: cephx 2024-06-28T15:24:00.094 INFO:tasks.workunit.client.0.smithi178.stderr: -475> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding auth protocol: cephx 2024-06-28T15:24:00.094 INFO:tasks.workunit.client.0.smithi178.stderr: -474> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding auth protocol: cephx 2024-06-28T15:24:00.094 INFO:tasks.workunit.client.0.smithi178.stderr: -473> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: secure 2024-06-28T15:24:00.094 INFO:tasks.workunit.client.0.smithi178.stderr: -472> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: crc 2024-06-28T15:24:00.094 INFO:tasks.workunit.client.0.smithi178.stderr: -471> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: secure 2024-06-28T15:24:00.094 INFO:tasks.workunit.client.0.smithi178.stderr: -470> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: crc 2024-06-28T15:24:00.094 INFO:tasks.workunit.client.0.smithi178.stderr: -469> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: secure 2024-06-28T15:24:00.095 INFO:tasks.workunit.client.0.smithi178.stderr: -468> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: crc 2024-06-28T15:24:00.095 INFO:tasks.workunit.client.0.smithi178.stderr: -467> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: crc 2024-06-28T15:24:00.095 INFO:tasks.workunit.client.0.smithi178.stderr: -466> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: secure 2024-06-28T15:24:00.095 INFO:tasks.workunit.client.0.smithi178.stderr: -465> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: crc 2024-06-28T15:24:00.095 INFO:tasks.workunit.client.0.smithi178.stderr: -464> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: secure 2024-06-28T15:24:00.095 INFO:tasks.workunit.client.0.smithi178.stderr: -463> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: crc 2024-06-28T15:24:00.095 INFO:tasks.workunit.client.0.smithi178.stderr: -462> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d75d90) adding con mode: secure 2024-06-28T15:24:00.095 INFO:tasks.workunit.client.0.smithi178.stderr: -461> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.095 INFO:tasks.workunit.client.0.smithi178.stderr: -460> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 1 librados: starting msgr at 2024-06-28T15:24:00.095 INFO:tasks.workunit.client.0.smithi178.stderr: -459> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 1 librados: starting objecter 2024-06-28T15:24:00.095 INFO:tasks.workunit.client.0.smithi178.stderr: -458> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command objecter_requests hook 0x55f424da1eb0 2024-06-28T15:24:00.095 INFO:tasks.workunit.client.0.smithi178.stderr: -457> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 1 librados: setting wanted keys 2024-06-28T15:24:00.095 INFO:tasks.workunit.client.0.smithi178.stderr: -456> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 1 librados: calling monclient init 2024-06-28T15:24:00.095 INFO:tasks.workunit.client.0.smithi178.stderr: -455> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: init 2024-06-28T15:24:00.096 INFO:tasks.workunit.client.0.smithi178.stderr: -454> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding auth protocol: cephx 2024-06-28T15:24:00.096 INFO:tasks.workunit.client.0.smithi178.stderr: -453> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding auth protocol: cephx 2024-06-28T15:24:00.096 INFO:tasks.workunit.client.0.smithi178.stderr: -452> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding auth protocol: cephx 2024-06-28T15:24:00.096 INFO:tasks.workunit.client.0.smithi178.stderr: -451> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: secure 2024-06-28T15:24:00.096 INFO:tasks.workunit.client.0.smithi178.stderr: -450> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: crc 2024-06-28T15:24:00.096 INFO:tasks.workunit.client.0.smithi178.stderr: -449> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: secure 2024-06-28T15:24:00.096 INFO:tasks.workunit.client.0.smithi178.stderr: -448> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: crc 2024-06-28T15:24:00.096 INFO:tasks.workunit.client.0.smithi178.stderr: -447> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: secure 2024-06-28T15:24:00.096 INFO:tasks.workunit.client.0.smithi178.stderr: -446> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: crc 2024-06-28T15:24:00.096 INFO:tasks.workunit.client.0.smithi178.stderr: -445> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: crc 2024-06-28T15:24:00.096 INFO:tasks.workunit.client.0.smithi178.stderr: -444> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: secure 2024-06-28T15:24:00.096 INFO:tasks.workunit.client.0.smithi178.stderr: -443> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: crc 2024-06-28T15:24:00.096 INFO:tasks.workunit.client.0.smithi178.stderr: -442> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: secure 2024-06-28T15:24:00.096 INFO:tasks.workunit.client.0.smithi178.stderr: -441> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: crc 2024-06-28T15:24:00.097 INFO:tasks.workunit.client.0.smithi178.stderr: -440> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424d6f290) adding con mode: secure 2024-06-28T15:24:00.097 INFO:tasks.workunit.client.0.smithi178.stderr: -439> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.097 INFO:tasks.workunit.client.0.smithi178.stderr: -438> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.097 INFO:tasks.workunit.client.0.smithi178.stderr: -437> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command rotate-key hook 0x55f424d6f3d8 2024-06-28T15:24:00.097 INFO:tasks.workunit.client.0.smithi178.stderr: -436> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:00.097 INFO:tasks.workunit.client.0.smithi178.stderr: -435> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: _add_conns ranks=[1,2,0] 2024-06-28T15:24:00.097 INFO:tasks.workunit.client.0.smithi178.stderr: -434> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-b con 0x55f424c1f8c0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:00.097 INFO:tasks.workunit.client.0.smithi178.stderr: -433> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-c con 0x55f424c20300 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:00.097 INFO:tasks.workunit.client.0.smithi178.stderr: -432> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-a con 0x55f424c29270 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:00.097 INFO:tasks.workunit.client.0.smithi178.stderr: -431> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.097 INFO:tasks.workunit.client.0.smithi178.stderr: -430> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.097 INFO:tasks.workunit.client.0.smithi178.stderr: -429> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.097 INFO:tasks.workunit.client.0.smithi178.stderr: -428> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:00.097 INFO:tasks.workunit.client.0.smithi178.stderr: -427> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient(hunting): authenticate will time out at 2383.394019s 2024-06-28T15:24:00.098 INFO:tasks.workunit.client.0.smithi178.stderr: -426> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): get_auth_request con 0x55f424c20300 auth_method 0 2024-06-28T15:24:00.098 INFO:tasks.workunit.client.0.smithi178.stderr: -425> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.098 INFO:tasks.workunit.client.0.smithi178.stderr: -424> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.098 INFO:tasks.workunit.client.0.smithi178.stderr: -423> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.098 INFO:tasks.workunit.client.0.smithi178.stderr: -422> 2024-06-28T15:23:59.929+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request con 0x55f424c29270 auth_method 0 2024-06-28T15:24:00.098 INFO:tasks.workunit.client.0.smithi178.stderr: -421> 2024-06-28T15:23:59.929+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.098 INFO:tasks.workunit.client.0.smithi178.stderr: -420> 2024-06-28T15:23:59.929+0000 7f2f2d26f640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.098 INFO:tasks.workunit.client.0.smithi178.stderr: -419> 2024-06-28T15:23:59.929+0000 7f2f2d26f640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.098 INFO:tasks.workunit.client.0.smithi178.stderr: -418> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.098 INFO:tasks.workunit.client.0.smithi178.stderr: -417> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.098 INFO:tasks.workunit.client.0.smithi178.stderr: -416> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.098 INFO:tasks.workunit.client.0.smithi178.stderr: -415> 2024-06-28T15:23:59.929+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.098 INFO:tasks.workunit.client.0.smithi178.stderr: -414> 2024-06-28T15:23:59.929+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.099 INFO:tasks.workunit.client.0.smithi178.stderr: -413> 2024-06-28T15:23:59.929+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.099 INFO:tasks.workunit.client.0.smithi178.stderr: -412> 2024-06-28T15:23:59.929+0000 7f2f2ca6e640 10 monclient(hunting): get_auth_request con 0x55f424c1f8c0 auth_method 0 2024-06-28T15:24:00.099 INFO:tasks.workunit.client.0.smithi178.stderr: -411> 2024-06-28T15:23:59.929+0000 7f2f2ca6e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.099 INFO:tasks.workunit.client.0.smithi178.stderr: -410> 2024-06-28T15:23:59.929+0000 7f2f2ca6e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.099 INFO:tasks.workunit.client.0.smithi178.stderr: -409> 2024-06-28T15:23:59.929+0000 7f2f2ca6e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.099 INFO:tasks.workunit.client.0.smithi178.stderr: -408> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_done global_id 6251 payload 824 2024-06-28T15:24:00.099 INFO:tasks.workunit.client.0.smithi178.stderr: -407> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient: _finish_hunting 0 2024-06-28T15:24:00.099 INFO:tasks.workunit.client.0.smithi178.stderr: -406> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 1 monclient: found mon.noname-c 2024-06-28T15:24:00.099 INFO:tasks.workunit.client.0.smithi178.stderr: -405> 2024-06-28T15:23:59.929+0000 7f2f2c26d640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.099 INFO:tasks.workunit.client.0.smithi178.stderr: -404> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.099 INFO:tasks.workunit.client.0.smithi178.stderr: -403> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:24:00.099 INFO:tasks.workunit.client.0.smithi178.stderr: -402> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: dump: 2024-06-28T15:24:00.099 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.100 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.100 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.100 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.100 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.100 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.100 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.100 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.100 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.100 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.100 INFO:tasks.workunit.client.0.smithi178.stderr: -401> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: _finish_auth 0 2024-06-28T15:24:00.100 INFO:tasks.workunit.client.0.smithi178.stderr: -400> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: _check_auth_tickets 2024-06-28T15:24:00.100 INFO:tasks.workunit.client.0.smithi178.stderr: -399> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:00.100 INFO:tasks.workunit.client.0.smithi178.stderr: -398> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.100 INFO:tasks.workunit.client.0.smithi178.stderr: -397> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr: -396> 2024-06-28T15:23:59.929+0000 7f2e1960a640 10 monclient: dump: 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr: -395> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 5 monclient: authenticate success, global_id 6251 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr: -394> 2024-06-28T15:23:59.929+0000 7f2e1ae0d640 4 set_mon_vals no callback set 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr: -393> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: _renew_subs 2024-06-28T15:24:00.101 INFO:tasks.workunit.client.0.smithi178.stderr: -392> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.102 INFO:tasks.workunit.client.0.smithi178.stderr: -391> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: _renew_subs 2024-06-28T15:24:00.102 INFO:tasks.workunit.client.0.smithi178.stderr: -390> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.102 INFO:tasks.workunit.client.0.smithi178.stderr: -389> 2024-06-28T15:23:59.929+0000 7f2f2ef5d980 1 librados: init done 2024-06-28T15:24:00.102 INFO:tasks.workunit.client.0.smithi178.stderr: -388> 2024-06-28T15:23:59.933+0000 7f2e1960a640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:24:00.102 INFO:tasks.workunit.client.0.smithi178.stderr: -387> 2024-06-28T15:23:59.933+0000 7f2e1960a640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:00.102 INFO:tasks.workunit.client.0.smithi178.stderr: -386> 2024-06-28T15:23:59.933+0000 7f2e1960a640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:00.102 INFO:tasks.workunit.client.0.smithi178.stderr: -385> 2024-06-28T15:23:59.933+0000 7f2f2ca6e640 10 monclient: get_auth_request con 0x7f2e1406f7b0 auth_method 0 2024-06-28T15:24:00.102 INFO:tasks.workunit.client.0.smithi178.stderr: -384> 2024-06-28T15:23:59.933+0000 7f2f2d26f640 10 monclient: get_auth_request con 0x55f424d97f30 auth_method 0 2024-06-28T15:24:00.102 INFO:tasks.workunit.client.0.smithi178.stderr: -383> 2024-06-28T15:23:59.933+0000 7f2f2ca6e640 10 monclient: get_auth_request con 0x55f424dbaab0 auth_method 0 2024-06-28T15:24:00.102 INFO:tasks.workunit.client.0.smithi178.stderr: -382> 2024-06-28T15:23:59.933+0000 7f2f2c26d640 10 monclient: get_auth_request con 0x55f424dbee10 auth_method 0 2024-06-28T15:24:00.102 INFO:tasks.workunit.client.0.smithi178.stderr: -381> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient: build_initial_monmap 2024-06-28T15:24:00.102 INFO:tasks.workunit.client.0.smithi178.stderr: -380> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 1 build_initial for_mkfs: 0 2024-06-28T15:24:00.102 INFO:tasks.workunit.client.0.smithi178.stderr: -379> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient: monmap: 2024-06-28T15:24:00.103 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:00.103 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:00.103 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:00.103 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:00.103 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:00.103 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.103 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:00.103 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:00.103 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:00.103 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.103 INFO:tasks.workunit.client.0.smithi178.stderr: -378> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding auth protocol: cephx 2024-06-28T15:24:00.103 INFO:tasks.workunit.client.0.smithi178.stderr: -377> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding auth protocol: cephx 2024-06-28T15:24:00.103 INFO:tasks.workunit.client.0.smithi178.stderr: -376> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding auth protocol: cephx 2024-06-28T15:24:00.103 INFO:tasks.workunit.client.0.smithi178.stderr: -375> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: secure 2024-06-28T15:24:00.104 INFO:tasks.workunit.client.0.smithi178.stderr: -374> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: crc 2024-06-28T15:24:00.104 INFO:tasks.workunit.client.0.smithi178.stderr: -373> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: secure 2024-06-28T15:24:00.104 INFO:tasks.workunit.client.0.smithi178.stderr: -372> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: crc 2024-06-28T15:24:00.104 INFO:tasks.workunit.client.0.smithi178.stderr: -371> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: secure 2024-06-28T15:24:00.104 INFO:tasks.workunit.client.0.smithi178.stderr: -370> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: crc 2024-06-28T15:24:00.104 INFO:tasks.workunit.client.0.smithi178.stderr: -369> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: crc 2024-06-28T15:24:00.104 INFO:tasks.workunit.client.0.smithi178.stderr: -368> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: secure 2024-06-28T15:24:00.104 INFO:tasks.workunit.client.0.smithi178.stderr: -367> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: crc 2024-06-28T15:24:00.104 INFO:tasks.workunit.client.0.smithi178.stderr: -366> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: secure 2024-06-28T15:24:00.104 INFO:tasks.workunit.client.0.smithi178.stderr: -365> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: crc 2024-06-28T15:24:00.104 INFO:tasks.workunit.client.0.smithi178.stderr: -364> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc3c60) adding con mode: secure 2024-06-28T15:24:00.105 INFO:tasks.workunit.client.0.smithi178.stderr: -363> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.105 INFO:tasks.workunit.client.0.smithi178.stderr: -362> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command objecter_requests cmddesc objecter_requests hook 0x55f424dc7240 EEXIST 2024-06-28T15:24:00.105 INFO:tasks.workunit.client.0.smithi178.stderr: -361> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient: init 2024-06-28T15:24:00.105 INFO:tasks.workunit.client.0.smithi178.stderr: -360> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding auth protocol: cephx 2024-06-28T15:24:00.105 INFO:tasks.workunit.client.0.smithi178.stderr: -359> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding auth protocol: cephx 2024-06-28T15:24:00.105 INFO:tasks.workunit.client.0.smithi178.stderr: -358> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding auth protocol: cephx 2024-06-28T15:24:00.105 INFO:tasks.workunit.client.0.smithi178.stderr: -357> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: secure 2024-06-28T15:24:00.105 INFO:tasks.workunit.client.0.smithi178.stderr: -356> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: crc 2024-06-28T15:24:00.105 INFO:tasks.workunit.client.0.smithi178.stderr: -355> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: secure 2024-06-28T15:24:00.105 INFO:tasks.workunit.client.0.smithi178.stderr: -354> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: crc 2024-06-28T15:24:00.105 INFO:tasks.workunit.client.0.smithi178.stderr: -353> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: secure 2024-06-28T15:24:00.105 INFO:tasks.workunit.client.0.smithi178.stderr: -352> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: crc 2024-06-28T15:24:00.105 INFO:tasks.workunit.client.0.smithi178.stderr: -351> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: crc 2024-06-28T15:24:00.106 INFO:tasks.workunit.client.0.smithi178.stderr: -350> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: secure 2024-06-28T15:24:00.106 INFO:tasks.workunit.client.0.smithi178.stderr: -349> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: crc 2024-06-28T15:24:00.106 INFO:tasks.workunit.client.0.smithi178.stderr: -348> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: secure 2024-06-28T15:24:00.106 INFO:tasks.workunit.client.0.smithi178.stderr: -347> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: crc 2024-06-28T15:24:00.106 INFO:tasks.workunit.client.0.smithi178.stderr: -346> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424dc2338) adding con mode: secure 2024-06-28T15:24:00.106 INFO:tasks.workunit.client.0.smithi178.stderr: -345> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.106 INFO:tasks.workunit.client.0.smithi178.stderr: -344> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.106 INFO:tasks.workunit.client.0.smithi178.stderr: -343> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command rotate-key cmddesc rotate-key hook 0x55f424dc2480 EEXIST 2024-06-28T15:24:00.106 INFO:tasks.workunit.client.0.smithi178.stderr: -342> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:00.106 INFO:tasks.workunit.client.0.smithi178.stderr: -341> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient: _add_conns ranks=[2,1,0] 2024-06-28T15:24:00.106 INFO:tasks.workunit.client.0.smithi178.stderr: -340> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-c con 0x55f424dcd7d0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:00.106 INFO:tasks.workunit.client.0.smithi178.stderr: -339> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-b con 0x55f424dce100 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:00.107 INFO:tasks.workunit.client.0.smithi178.stderr: -338> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-a con 0x55f424dcdbb0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:00.107 INFO:tasks.workunit.client.0.smithi178.stderr: -337> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.107 INFO:tasks.workunit.client.0.smithi178.stderr: -336> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.107 INFO:tasks.workunit.client.0.smithi178.stderr: -335> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.107 INFO:tasks.workunit.client.0.smithi178.stderr: -334> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:00.107 INFO:tasks.workunit.client.0.smithi178.stderr: -333> 2024-06-28T15:23:59.937+0000 7f2f2ef5d980 10 monclient(hunting): authenticate will time out at 2383.403309s 2024-06-28T15:24:00.107 INFO:tasks.workunit.client.0.smithi178.stderr: -332> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): get_auth_request con 0x55f424dcd7d0 auth_method 0 2024-06-28T15:24:00.108 INFO:tasks.workunit.client.0.smithi178.stderr: -331> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.108 INFO:tasks.workunit.client.0.smithi178.stderr: -330> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.108 INFO:tasks.workunit.client.0.smithi178.stderr: -329> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.108 INFO:tasks.workunit.client.0.smithi178.stderr: -328> 2024-06-28T15:23:59.937+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request con 0x55f424dcdbb0 auth_method 0 2024-06-28T15:24:00.108 INFO:tasks.workunit.client.0.smithi178.stderr: -327> 2024-06-28T15:23:59.937+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.108 INFO:tasks.workunit.client.0.smithi178.stderr: -326> 2024-06-28T15:23:59.937+0000 7f2f2d26f640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.108 INFO:tasks.workunit.client.0.smithi178.stderr: -325> 2024-06-28T15:23:59.937+0000 7f2f2d26f640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.109 INFO:tasks.workunit.client.0.smithi178.stderr: -324> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.109 INFO:tasks.workunit.client.0.smithi178.stderr: -323> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.109 INFO:tasks.workunit.client.0.smithi178.stderr: -322> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.109 INFO:tasks.workunit.client.0.smithi178.stderr: -321> 2024-06-28T15:23:59.937+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.109 INFO:tasks.workunit.client.0.smithi178.stderr: -320> 2024-06-28T15:23:59.937+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.109 INFO:tasks.workunit.client.0.smithi178.stderr: -319> 2024-06-28T15:23:59.937+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.110 INFO:tasks.workunit.client.0.smithi178.stderr: -318> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_done global_id 6254 payload 824 2024-06-28T15:24:00.110 INFO:tasks.workunit.client.0.smithi178.stderr: -317> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient: _finish_hunting 0 2024-06-28T15:24:00.110 INFO:tasks.workunit.client.0.smithi178.stderr: -316> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 1 monclient: found mon.noname-c 2024-06-28T15:24:00.110 INFO:tasks.workunit.client.0.smithi178.stderr: -315> 2024-06-28T15:23:59.937+0000 7f2f2ca6e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.110 INFO:tasks.workunit.client.0.smithi178.stderr: -314> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.110 INFO:tasks.workunit.client.0.smithi178.stderr: -313> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:24:00.111 INFO:tasks.workunit.client.0.smithi178.stderr: -312> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: dump: 2024-06-28T15:24:00.111 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.111 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.111 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.111 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.111 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.112 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.112 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.112 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.112 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.112 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.112 INFO:tasks.workunit.client.0.smithi178.stderr: -311> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: _finish_auth 0 2024-06-28T15:24:00.113 INFO:tasks.workunit.client.0.smithi178.stderr: -310> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: _check_auth_tickets 2024-06-28T15:24:00.113 INFO:tasks.workunit.client.0.smithi178.stderr: -309> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:00.113 INFO:tasks.workunit.client.0.smithi178.stderr: -308> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.113 INFO:tasks.workunit.client.0.smithi178.stderr: -307> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:24:00.113 INFO:tasks.workunit.client.0.smithi178.stderr: -306> 2024-06-28T15:23:59.941+0000 7f2e127fc640 10 monclient: dump: 2024-06-28T15:24:00.113 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.113 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.114 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.114 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.114 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.114 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.114 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.114 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.115 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.115 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.115 INFO:tasks.workunit.client.0.smithi178.stderr: -305> 2024-06-28T15:23:59.941+0000 7f2f2da70640 4 set_mon_vals no callback set 2024-06-28T15:24:00.115 INFO:tasks.workunit.client.0.smithi178.stderr: -304> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 monclient: authenticate success, global_id 6254 2024-06-28T15:24:00.115 INFO:tasks.workunit.client.0.smithi178.stderr: -303> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: _renew_subs 2024-06-28T15:24:00.115 INFO:tasks.workunit.client.0.smithi178.stderr: -302> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.116 INFO:tasks.workunit.client.0.smithi178.stderr: -301> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: _renew_subs 2024-06-28T15:24:00.116 INFO:tasks.workunit.client.0.smithi178.stderr: -300> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.116 INFO:tasks.workunit.client.0.smithi178.stderr: -299> 2024-06-28T15:23:59.941+0000 7f2e127fc640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:24:00.116 INFO:tasks.workunit.client.0.smithi178.stderr: -298> 2024-06-28T15:23:59.941+0000 7f2e127fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:00.116 INFO:tasks.workunit.client.0.smithi178.stderr: -297> 2024-06-28T15:23:59.941+0000 7f2e127fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:00.117 INFO:tasks.workunit.client.0.smithi178.stderr: -296> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient: get_auth_request con 0x7f2e0406f690 auth_method 0 2024-06-28T15:24:00.117 INFO:tasks.workunit.client.0.smithi178.stderr: -295> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: get_monmap_and_config 2024-06-28T15:24:00.117 INFO:tasks.workunit.client.0.smithi178.stderr: -294> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: build_initial_monmap 2024-06-28T15:24:00.117 INFO:tasks.workunit.client.0.smithi178.stderr: -293> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 1 build_initial for_mkfs: 0 2024-06-28T15:24:00.117 INFO:tasks.workunit.client.0.smithi178.stderr: -292> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: monmap: 2024-06-28T15:24:00.117 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:00.118 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:00.118 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:00.118 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:00.118 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:00.118 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.118 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:00.119 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:00.119 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:00.119 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.119 INFO:tasks.workunit.client.0.smithi178.stderr: -291> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding auth protocol: cephx 2024-06-28T15:24:00.119 INFO:tasks.workunit.client.0.smithi178.stderr: -290> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding auth protocol: cephx 2024-06-28T15:24:00.119 INFO:tasks.workunit.client.0.smithi178.stderr: -289> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding auth protocol: cephx 2024-06-28T15:24:00.119 INFO:tasks.workunit.client.0.smithi178.stderr: -288> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: secure 2024-06-28T15:24:00.119 INFO:tasks.workunit.client.0.smithi178.stderr: -287> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: crc 2024-06-28T15:24:00.120 INFO:tasks.workunit.client.0.smithi178.stderr: -286> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: secure 2024-06-28T15:24:00.120 INFO:tasks.workunit.client.0.smithi178.stderr: -285> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: crc 2024-06-28T15:24:00.120 INFO:tasks.workunit.client.0.smithi178.stderr: -284> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: secure 2024-06-28T15:24:00.120 INFO:tasks.workunit.client.0.smithi178.stderr: -283> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: crc 2024-06-28T15:24:00.120 INFO:tasks.workunit.client.0.smithi178.stderr: -282> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: crc 2024-06-28T15:24:00.120 INFO:tasks.workunit.client.0.smithi178.stderr: -281> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: secure 2024-06-28T15:24:00.120 INFO:tasks.workunit.client.0.smithi178.stderr: -280> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: crc 2024-06-28T15:24:00.120 INFO:tasks.workunit.client.0.smithi178.stderr: -279> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: secure 2024-06-28T15:24:00.120 INFO:tasks.workunit.client.0.smithi178.stderr: -278> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: crc 2024-06-28T15:24:00.121 INFO:tasks.workunit.client.0.smithi178.stderr: -277> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e0b2a0) adding con mode: secure 2024-06-28T15:24:00.121 INFO:tasks.workunit.client.0.smithi178.stderr: -276> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.121 INFO:tasks.workunit.client.0.smithi178.stderr: -275> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: init 2024-06-28T15:24:00.121 INFO:tasks.workunit.client.0.smithi178.stderr: -274> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding auth protocol: cephx 2024-06-28T15:24:00.121 INFO:tasks.workunit.client.0.smithi178.stderr: -273> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding auth protocol: cephx 2024-06-28T15:24:00.121 INFO:tasks.workunit.client.0.smithi178.stderr: -272> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding auth protocol: cephx 2024-06-28T15:24:00.121 INFO:tasks.workunit.client.0.smithi178.stderr: -271> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: secure 2024-06-28T15:24:00.122 INFO:tasks.workunit.client.0.smithi178.stderr: -270> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: crc 2024-06-28T15:24:00.122 INFO:tasks.workunit.client.0.smithi178.stderr: -269> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: secure 2024-06-28T15:24:00.122 INFO:tasks.workunit.client.0.smithi178.stderr: -268> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: crc 2024-06-28T15:24:00.122 INFO:tasks.workunit.client.0.smithi178.stderr: -267> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: secure 2024-06-28T15:24:00.122 INFO:tasks.workunit.client.0.smithi178.stderr: -266> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: crc 2024-06-28T15:24:00.122 INFO:tasks.workunit.client.0.smithi178.stderr: -265> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: crc 2024-06-28T15:24:00.123 INFO:tasks.workunit.client.0.smithi178.stderr: -264> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: secure 2024-06-28T15:24:00.123 INFO:tasks.workunit.client.0.smithi178.stderr: -263> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: crc 2024-06-28T15:24:00.123 INFO:tasks.workunit.client.0.smithi178.stderr: -262> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: secure 2024-06-28T15:24:00.123 INFO:tasks.workunit.client.0.smithi178.stderr: -261> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: crc 2024-06-28T15:24:00.123 INFO:tasks.workunit.client.0.smithi178.stderr: -260> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381470) adding con mode: secure 2024-06-28T15:24:00.123 INFO:tasks.workunit.client.0.smithi178.stderr: -259> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.123 INFO:tasks.workunit.client.0.smithi178.stderr: -258> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.123 INFO:tasks.workunit.client.0.smithi178.stderr: -257> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command rotate-key cmddesc rotate-key hook 0x7fff993815b8 EEXIST 2024-06-28T15:24:00.123 INFO:tasks.workunit.client.0.smithi178.stderr: -256> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:00.123 INFO:tasks.workunit.client.0.smithi178.stderr: -255> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient: _add_conns ranks=[1,0,2] 2024-06-28T15:24:00.123 INFO:tasks.workunit.client.0.smithi178.stderr: -254> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-b con 0x55f424e3b170 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:00.123 INFO:tasks.workunit.client.0.smithi178.stderr: -253> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-a con 0x55f424e3b550 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:00.123 INFO:tasks.workunit.client.0.smithi178.stderr: -252> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-c con 0x55f424e39c60 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:00.123 INFO:tasks.workunit.client.0.smithi178.stderr: -251> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.124 INFO:tasks.workunit.client.0.smithi178.stderr: -250> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.124 INFO:tasks.workunit.client.0.smithi178.stderr: -249> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.124 INFO:tasks.workunit.client.0.smithi178.stderr: -248> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:00.124 INFO:tasks.workunit.client.0.smithi178.stderr: -247> 2024-06-28T15:23:59.941+0000 7f2f2ef5d980 10 monclient(hunting): authenticate will time out at 2383.407396s 2024-06-28T15:24:00.124 INFO:tasks.workunit.client.0.smithi178.stderr: -246> 2024-06-28T15:23:59.941+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request con 0x55f424e39c60 auth_method 0 2024-06-28T15:24:00.124 INFO:tasks.workunit.client.0.smithi178.stderr: -245> 2024-06-28T15:23:59.941+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.124 INFO:tasks.workunit.client.0.smithi178.stderr: -244> 2024-06-28T15:23:59.941+0000 7f2f2d26f640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.124 INFO:tasks.workunit.client.0.smithi178.stderr: -243> 2024-06-28T15:23:59.941+0000 7f2f2d26f640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.124 INFO:tasks.workunit.client.0.smithi178.stderr: -242> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): get_auth_request con 0x55f424e3b550 auth_method 0 2024-06-28T15:24:00.124 INFO:tasks.workunit.client.0.smithi178.stderr: -241> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.124 INFO:tasks.workunit.client.0.smithi178.stderr: -240> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.124 INFO:tasks.workunit.client.0.smithi178.stderr: -239> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.124 INFO:tasks.workunit.client.0.smithi178.stderr: -238> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.124 INFO:tasks.workunit.client.0.smithi178.stderr: -237> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.125 INFO:tasks.workunit.client.0.smithi178.stderr: -236> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.125 INFO:tasks.workunit.client.0.smithi178.stderr: -235> 2024-06-28T15:23:59.941+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.125 INFO:tasks.workunit.client.0.smithi178.stderr: -234> 2024-06-28T15:23:59.941+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.125 INFO:tasks.workunit.client.0.smithi178.stderr: -233> 2024-06-28T15:23:59.941+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.125 INFO:tasks.workunit.client.0.smithi178.stderr: -232> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient(hunting): handle_auth_done global_id 6663 payload 274 2024-06-28T15:24:00.125 INFO:tasks.workunit.client.0.smithi178.stderr: -231> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient: _finish_hunting 0 2024-06-28T15:24:00.125 INFO:tasks.workunit.client.0.smithi178.stderr: -230> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 1 monclient: found mon.noname-a 2024-06-28T15:24:00.125 INFO:tasks.workunit.client.0.smithi178.stderr: -229> 2024-06-28T15:23:59.941+0000 7f2f2c26d640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:24:00.125 INFO:tasks.workunit.client.0.smithi178.stderr: -228> 2024-06-28T15:23:59.941+0000 7f2f2ca6e640 10 monclient: get_auth_request con 0x55f424e3b170 auth_method 0 2024-06-28T15:24:00.125 INFO:tasks.workunit.client.0.smithi178.stderr: -227> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.125 INFO:tasks.workunit.client.0.smithi178.stderr: -226> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:24:00.126 INFO:tasks.workunit.client.0.smithi178.stderr: -225> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: dump: 2024-06-28T15:24:00.126 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.126 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.126 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.126 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.126 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.126 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.126 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.126 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.126 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.126 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.126 INFO:tasks.workunit.client.0.smithi178.stderr: -224> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: _finish_auth 0 2024-06-28T15:24:00.126 INFO:tasks.workunit.client.0.smithi178.stderr: -223> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: _check_auth_tickets 2024-06-28T15:24:00.127 INFO:tasks.workunit.client.0.smithi178.stderr: -222> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:00.127 INFO:tasks.workunit.client.0.smithi178.stderr: -221> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 monclient: authenticate success, global_id 6663 2024-06-28T15:24:00.127 INFO:tasks.workunit.client.0.smithi178.stderr: -220> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: get_monmap_and_config success 2024-06-28T15:24:00.127 INFO:tasks.workunit.client.0.smithi178.stderr: -219> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 4 set_mon_vals no callback set 2024-06-28T15:24:00.127 INFO:tasks.workunit.client.0.smithi178.stderr: -218> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.127 INFO:tasks.workunit.client.0.smithi178.stderr: -217> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:24:00.127 INFO:tasks.workunit.client.0.smithi178.stderr: -216> 2024-06-28T15:23:59.945+0000 7f2e03fff640 10 monclient: dump: 2024-06-28T15:24:00.127 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.127 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.127 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.127 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.127 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.127 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.128 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.128 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.128 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.128 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.128 INFO:tasks.workunit.client.0.smithi178.stderr: -215> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: shutdown 2024-06-28T15:24:00.128 INFO:tasks.workunit.client.0.smithi178.stderr: -214> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: build_initial_monmap 2024-06-28T15:24:00.128 INFO:tasks.workunit.client.0.smithi178.stderr: -213> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 1 build_initial for_mkfs: 0 2024-06-28T15:24:00.128 INFO:tasks.workunit.client.0.smithi178.stderr: -212> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: monmap: 2024-06-28T15:24:00.128 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:00.128 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:00.128 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:00.129 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:00.129 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:00.129 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.129 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:00.129 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:00.129 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:00.129 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.129 INFO:tasks.workunit.client.0.smithi178.stderr: -211> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding auth protocol: cephx 2024-06-28T15:24:00.129 INFO:tasks.workunit.client.0.smithi178.stderr: -210> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding auth protocol: cephx 2024-06-28T15:24:00.129 INFO:tasks.workunit.client.0.smithi178.stderr: -209> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding auth protocol: cephx 2024-06-28T15:24:00.129 INFO:tasks.workunit.client.0.smithi178.stderr: -208> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: secure 2024-06-28T15:24:00.129 INFO:tasks.workunit.client.0.smithi178.stderr: -207> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: crc 2024-06-28T15:24:00.129 INFO:tasks.workunit.client.0.smithi178.stderr: -206> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: secure 2024-06-28T15:24:00.129 INFO:tasks.workunit.client.0.smithi178.stderr: -205> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: crc 2024-06-28T15:24:00.130 INFO:tasks.workunit.client.0.smithi178.stderr: -204> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: secure 2024-06-28T15:24:00.130 INFO:tasks.workunit.client.0.smithi178.stderr: -203> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: crc 2024-06-28T15:24:00.130 INFO:tasks.workunit.client.0.smithi178.stderr: -202> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: crc 2024-06-28T15:24:00.130 INFO:tasks.workunit.client.0.smithi178.stderr: -201> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: secure 2024-06-28T15:24:00.130 INFO:tasks.workunit.client.0.smithi178.stderr: -200> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: crc 2024-06-28T15:24:00.130 INFO:tasks.workunit.client.0.smithi178.stderr: -199> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: secure 2024-06-28T15:24:00.130 INFO:tasks.workunit.client.0.smithi178.stderr: -198> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: crc 2024-06-28T15:24:00.130 INFO:tasks.workunit.client.0.smithi178.stderr: -197> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e3ef40) adding con mode: secure 2024-06-28T15:24:00.130 INFO:tasks.workunit.client.0.smithi178.stderr: -196> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.130 INFO:tasks.workunit.client.0.smithi178.stderr: -195> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 1 librados: starting msgr at 2024-06-28T15:24:00.130 INFO:tasks.workunit.client.0.smithi178.stderr: -194> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 1 librados: starting objecter 2024-06-28T15:24:00.130 INFO:tasks.workunit.client.0.smithi178.stderr: -193> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command objecter_requests cmddesc objecter_requests hook 0x55f424dcd390 EEXIST 2024-06-28T15:24:00.130 INFO:tasks.workunit.client.0.smithi178.stderr: -192> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 1 librados: setting wanted keys 2024-06-28T15:24:00.131 INFO:tasks.workunit.client.0.smithi178.stderr: -191> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 1 librados: calling monclient init 2024-06-28T15:24:00.131 INFO:tasks.workunit.client.0.smithi178.stderr: -190> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: init 2024-06-28T15:24:00.131 INFO:tasks.workunit.client.0.smithi178.stderr: -189> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding auth protocol: cephx 2024-06-28T15:24:00.131 INFO:tasks.workunit.client.0.smithi178.stderr: -188> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding auth protocol: cephx 2024-06-28T15:24:00.131 INFO:tasks.workunit.client.0.smithi178.stderr: -187> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding auth protocol: cephx 2024-06-28T15:24:00.131 INFO:tasks.workunit.client.0.smithi178.stderr: -186> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: secure 2024-06-28T15:24:00.131 INFO:tasks.workunit.client.0.smithi178.stderr: -185> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: crc 2024-06-28T15:24:00.131 INFO:tasks.workunit.client.0.smithi178.stderr: -184> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: secure 2024-06-28T15:24:00.131 INFO:tasks.workunit.client.0.smithi178.stderr: -183> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: crc 2024-06-28T15:24:00.131 INFO:tasks.workunit.client.0.smithi178.stderr: -182> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: secure 2024-06-28T15:24:00.131 INFO:tasks.workunit.client.0.smithi178.stderr: -181> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: crc 2024-06-28T15:24:00.131 INFO:tasks.workunit.client.0.smithi178.stderr: -180> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: crc 2024-06-28T15:24:00.131 INFO:tasks.workunit.client.0.smithi178.stderr: -179> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: secure 2024-06-28T15:24:00.131 INFO:tasks.workunit.client.0.smithi178.stderr: -178> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: crc 2024-06-28T15:24:00.132 INFO:tasks.workunit.client.0.smithi178.stderr: -177> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: secure 2024-06-28T15:24:00.132 INFO:tasks.workunit.client.0.smithi178.stderr: -176> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: crc 2024-06-28T15:24:00.132 INFO:tasks.workunit.client.0.smithi178.stderr: -175> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 AuthRegistry(0x55f424e03230) adding con mode: secure 2024-06-28T15:24:00.132 INFO:tasks.workunit.client.0.smithi178.stderr: -174> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.132 INFO:tasks.workunit.client.0.smithi178.stderr: -173> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.132 INFO:tasks.workunit.client.0.smithi178.stderr: -172> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command rotate-key cmddesc rotate-key hook 0x55f424e03378 EEXIST 2024-06-28T15:24:00.132 INFO:tasks.workunit.client.0.smithi178.stderr: -171> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:00.132 INFO:tasks.workunit.client.0.smithi178.stderr: -170> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: _add_conns ranks=[2,1,0] 2024-06-28T15:24:00.132 INFO:tasks.workunit.client.0.smithi178.stderr: -169> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-c con 0x55f424e39c60 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:00.132 INFO:tasks.workunit.client.0.smithi178.stderr: -168> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-b con 0x55f424e3b170 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:00.132 INFO:tasks.workunit.client.0.smithi178.stderr: -167> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): picked mon.noname-a con 0x55f424e37760 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:00.132 INFO:tasks.workunit.client.0.smithi178.stderr: -166> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.132 INFO:tasks.workunit.client.0.smithi178.stderr: -165> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.133 INFO:tasks.workunit.client.0.smithi178.stderr: -164> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:00.133 INFO:tasks.workunit.client.0.smithi178.stderr: -163> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:00.133 INFO:tasks.workunit.client.0.smithi178.stderr: -162> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient(hunting): authenticate will time out at 2383.410001s 2024-06-28T15:24:00.133 INFO:tasks.workunit.client.0.smithi178.stderr: -161> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): get_auth_request con 0x55f424e39c60 auth_method 0 2024-06-28T15:24:00.133 INFO:tasks.workunit.client.0.smithi178.stderr: -160> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.133 INFO:tasks.workunit.client.0.smithi178.stderr: -159> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.133 INFO:tasks.workunit.client.0.smithi178.stderr: -158> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.133 INFO:tasks.workunit.client.0.smithi178.stderr: -157> 2024-06-28T15:23:59.945+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request con 0x55f424e37760 auth_method 0 2024-06-28T15:24:00.133 INFO:tasks.workunit.client.0.smithi178.stderr: -156> 2024-06-28T15:23:59.945+0000 7f2f2d26f640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:00.133 INFO:tasks.workunit.client.0.smithi178.stderr: -155> 2024-06-28T15:23:59.945+0000 7f2f2d26f640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:00.133 INFO:tasks.workunit.client.0.smithi178.stderr: -154> 2024-06-28T15:23:59.945+0000 7f2f2d26f640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:00.133 INFO:tasks.workunit.client.0.smithi178.stderr: -153> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.133 INFO:tasks.workunit.client.0.smithi178.stderr: -152> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.134 INFO:tasks.workunit.client.0.smithi178.stderr: -151> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.134 INFO:tasks.workunit.client.0.smithi178.stderr: -150> 2024-06-28T15:23:59.945+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:00.134 INFO:tasks.workunit.client.0.smithi178.stderr: -149> 2024-06-28T15:23:59.945+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:00.134 INFO:tasks.workunit.client.0.smithi178.stderr: -148> 2024-06-28T15:23:59.945+0000 7f2f2d26f640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:00.134 INFO:tasks.workunit.client.0.smithi178.stderr: -147> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient(hunting): handle_auth_done global_id 6260 payload 824 2024-06-28T15:24:00.134 INFO:tasks.workunit.client.0.smithi178.stderr: -146> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient: _finish_hunting 0 2024-06-28T15:24:00.134 INFO:tasks.workunit.client.0.smithi178.stderr: -145> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 1 monclient: found mon.noname-c 2024-06-28T15:24:00.134 INFO:tasks.workunit.client.0.smithi178.stderr: -144> 2024-06-28T15:23:59.945+0000 7f2f2ca6e640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.134 INFO:tasks.workunit.client.0.smithi178.stderr: -143> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.134 INFO:tasks.workunit.client.0.smithi178.stderr: -142> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:24:00.134 INFO:tasks.workunit.client.0.smithi178.stderr: -141> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: dump: 2024-06-28T15:24:00.134 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.135 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.135 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.135 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.135 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.135 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.135 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.135 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.135 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.135 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.135 INFO:tasks.workunit.client.0.smithi178.stderr: -140> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: _finish_auth 0 2024-06-28T15:24:00.135 INFO:tasks.workunit.client.0.smithi178.stderr: -139> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: _check_auth_tickets 2024-06-28T15:24:00.135 INFO:tasks.workunit.client.0.smithi178.stderr: -138> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:00.135 INFO:tasks.workunit.client.0.smithi178.stderr: -137> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:00.136 INFO:tasks.workunit.client.0.smithi178.stderr: -136> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:24:00.136 INFO:tasks.workunit.client.0.smithi178.stderr: -135> 2024-06-28T15:23:59.945+0000 7f2e027fc640 10 monclient: dump: 2024-06-28T15:24:00.136 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:00.136 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:00.136 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.136 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:00.136 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:00.136 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:00.136 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:00.136 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:00.136 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:00.136 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.136 INFO:tasks.workunit.client.0.smithi178.stderr: -134> 2024-06-28T15:23:59.945+0000 7f2e03fff640 4 set_mon_vals no callback set 2024-06-28T15:24:00.136 INFO:tasks.workunit.client.0.smithi178.stderr: -133> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 monclient: authenticate success, global_id 6260 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -132> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: _renew_subs 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -131> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -130> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: _renew_subs 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -129> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -128> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 1 librados: init done 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -127> 2024-06-28T15:23:59.945+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command cr dump hook 0x55f424e34308 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -126> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -125> 2024-06-28T15:23:59.949+0000 7f2e027fc640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -124> 2024-06-28T15:23:59.949+0000 7f2e027fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -123> 2024-06-28T15:23:59.949+0000 7f2e027fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -122> 2024-06-28T15:23:59.949+0000 7f2f2c26d640 10 monclient: get_auth_request con 0x7f2df806f780 auth_method 0 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -121> 2024-06-28T15:23:59.949+0000 7f2f2d26f640 10 monclient: get_auth_request con 0x55f424e593e0 auth_method 0 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -120> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -119> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 realm 2024-06-28T15:24:00.137 INFO:tasks.workunit.client.0.smithi178.stderr: -118> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.138 INFO:tasks.workunit.client.0.smithi178.stderr: -117> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.138 INFO:tasks.workunit.client.0.smithi178.stderr: -116> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:00.138 INFO:tasks.workunit.client.0.smithi178.stderr: -115> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.138 INFO:tasks.workunit.client.0.smithi178.stderr: -114> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.138 INFO:tasks.workunit.client.0.smithi178.stderr: -113> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.138 INFO:tasks.workunit.client.0.smithi178.stderr: -112> 2024-06-28T15:23:59.949+0000 7f2f2ca6e640 10 monclient: get_auth_request con 0x55f424e5d830 auth_method 0 2024-06-28T15:24:00.139 INFO:tasks.workunit.client.0.smithi178.stderr: -111> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:00.139 INFO:tasks.workunit.client.0.smithi178.stderr: -110> 2024-06-28T15:23:59.949+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.139 INFO:tasks.workunit.client.0.smithi178.stderr: -109> 2024-06-28T15:23:59.953+0000 7f2f2c26d640 10 monclient: get_auth_request con 0x55f424e60f70 auth_method 0 2024-06-28T15:24:00.139 INFO:tasks.workunit.client.0.smithi178.stderr: -108> 2024-06-28T15:23:59.953+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:00.139 INFO:tasks.workunit.client.0.smithi178.stderr: -107> 2024-06-28T15:23:59.953+0000 7f2f2ef5d980 20 searching for the correct realm 2024-06-28T15:24:00.139 INFO:tasks.workunit.client.0.smithi178.stderr: -106> 2024-06-28T15:23:59.953+0000 7f2f2d26f640 10 monclient: get_auth_request con 0x7f2f14085040 auth_method 0 2024-06-28T15:24:00.140 INFO:tasks.workunit.client.0.smithi178.stderr: -105> 2024-06-28T15:23:59.953+0000 7f2f2ca6e640 10 monclient: get_auth_request con 0x7f2f2402a040 auth_method 0 2024-06-28T15:24:00.140 INFO:tasks.workunit.client.0.smithi178.stderr: -104> 2024-06-28T15:23:59.953+0000 7f2f2c26d640 10 monclient: get_auth_request con 0x7f2f1c0dc060 auth_method 0 2024-06-28T15:24:00.140 INFO:tasks.workunit.client.0.smithi178.stderr: -103> 2024-06-28T15:23:59.957+0000 7f2f2c26d640 0 -- 172.21.15.178:0/2278050652 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f2f1c0dc060 msgr2=0x7f2f1c0c83d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:00.140 INFO:tasks.workunit.client.0.smithi178.stderr: -102> 2024-06-28T15:23:59.957+0000 7f2e027fc640 1 client.6260.objecter ms_handle_reset 0x7f2f1c0dc060 session 0x7f2f1c0cd050 osd.4 2024-06-28T15:24:00.140 INFO:tasks.workunit.client.0.smithi178.stderr: -101> 2024-06-28T15:23:59.957+0000 7f2e027fc640 10 monclient: _renew_subs 2024-06-28T15:24:00.140 INFO:tasks.workunit.client.0.smithi178.stderr: -100> 2024-06-28T15:23:59.957+0000 7f2e027fc640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:00.140 INFO:tasks.workunit.client.0.smithi178.stderr: -99> 2024-06-28T15:23:59.957+0000 7f2f2d26f640 10 monclient: get_auth_request con 0x7f2f1408d380 auth_method 0 2024-06-28T15:24:00.141 INFO:tasks.workunit.client.0.smithi178.stderr: -98> 2024-06-28T15:23:59.957+0000 7f2f2c26d640 10 monclient: get_auth_request con 0x7f2df807fda0 auth_method 0 2024-06-28T15:24:00.141 INFO:tasks.workunit.client.0.smithi178.stderr: -97> 2024-06-28T15:23:59.961+0000 7f2f2ca6e640 10 monclient: get_auth_request con 0x7f2f14060790 auth_method 0 2024-06-28T15:24:00.141 INFO:tasks.workunit.client.0.smithi178.stderr: -96> 2024-06-28T15:23:59.965+0000 7f2f2c26d640 0 -- 172.21.15.178:0/2278050652 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f2df807fda0 msgr2=0x7f2df80821d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:00.141 INFO:tasks.workunit.client.0.smithi178.stderr: -95> 2024-06-28T15:23:59.965+0000 7f2e027fc640 1 client.6260.objecter ms_handle_reset 0x7f2df807fda0 session 0x7f2f1c0cd050 osd.4 2024-06-28T15:24:00.141 INFO:tasks.workunit.client.0.smithi178.stderr: -94> 2024-06-28T15:23:59.965+0000 7f2f2c26d640 10 monclient: get_auth_request con 0x7f2f1c0dc060 auth_method 0 2024-06-28T15:24:00.141 INFO:tasks.workunit.client.0.smithi178.stderr: -93> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:00.141 INFO:tasks.workunit.client.0.smithi178.stderr: -92> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:00.142 INFO:tasks.workunit.client.0.smithi178.stderr: -91> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:00.142 INFO:tasks.workunit.client.0.smithi178.stderr: -90> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:00.142 INFO:tasks.workunit.client.0.smithi178.stderr: -89> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:00.142 INFO:tasks.workunit.client.0.smithi178.stderr: -88> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:00.142 INFO:tasks.workunit.client.0.smithi178.stderr: -87> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.142 INFO:tasks.workunit.client.0.smithi178.stderr: -86> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.142 INFO:tasks.workunit.client.0.smithi178.stderr: -85> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.142 INFO:tasks.workunit.client.0.smithi178.stderr: -84> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:00.143 INFO:tasks.workunit.client.0.smithi178.stderr: -83> 2024-06-28T15:23:59.969+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.143 INFO:tasks.workunit.client.0.smithi178.stderr: -82> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:00.143 INFO:tasks.workunit.client.0.smithi178.stderr: -81> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 zone default found 2024-06-28T15:24:00.143 INFO:tasks.workunit.client.0.smithi178.stderr: -80> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 4 Realm: () 2024-06-28T15:24:00.143 INFO:tasks.workunit.client.0.smithi178.stderr: -79> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:00.143 INFO:tasks.workunit.client.0.smithi178.stderr: -78> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:00.144 INFO:tasks.workunit.client.0.smithi178.stderr: -77> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:00.144 INFO:tasks.workunit.client.0.smithi178.stderr: -76> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 zonegroup default 2024-06-28T15:24:00.144 INFO:tasks.workunit.client.0.smithi178.stderr: -75> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.144 INFO:tasks.workunit.client.0.smithi178.stderr: -74> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.144 INFO:tasks.workunit.client.0.smithi178.stderr: -73> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.144 INFO:tasks.workunit.client.0.smithi178.stderr: -72> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.145 INFO:tasks.workunit.client.0.smithi178.stderr: -71> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.145 INFO:tasks.workunit.client.0.smithi178.stderr: -70> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.145 INFO:tasks.workunit.client.0.smithi178.stderr: -69> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 started sync module instance, tier type = 2024-06-28T15:24:00.145 INFO:tasks.workunit.client.0.smithi178.stderr: -68> 2024-06-28T15:23:59.973+0000 7f2f2ef5d980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:00.145 INFO:tasks.workunit.client.0.smithi178.stderr: -67> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=0 2024-06-28T15:24:00.145 INFO:tasks.workunit.client.0.smithi178.stderr: -66> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=1 2024-06-28T15:24:00.145 INFO:tasks.workunit.client.0.smithi178.stderr: -65> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=2 2024-06-28T15:24:00.146 INFO:tasks.workunit.client.0.smithi178.stderr: -64> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=3 2024-06-28T15:24:00.146 INFO:tasks.workunit.client.0.smithi178.stderr: -63> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=4 2024-06-28T15:24:00.146 INFO:tasks.workunit.client.0.smithi178.stderr: -62> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=5 2024-06-28T15:24:00.146 INFO:tasks.workunit.client.0.smithi178.stderr: -61> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=6 2024-06-28T15:24:00.146 INFO:tasks.workunit.client.0.smithi178.stderr: -60> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 20 add_watcher() i=7 2024-06-28T15:24:00.146 INFO:tasks.workunit.client.0.smithi178.stderr: -59> 2024-06-28T15:23:59.989+0000 7f2f2ef5d980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:00.146 INFO:tasks.workunit.client.0.smithi178.stderr: -58> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding auth protocol: cephx 2024-06-28T15:24:00.146 INFO:tasks.workunit.client.0.smithi178.stderr: -57> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding auth protocol: cephx 2024-06-28T15:24:00.147 INFO:tasks.workunit.client.0.smithi178.stderr: -56> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding auth protocol: cephx 2024-06-28T15:24:00.147 INFO:tasks.workunit.client.0.smithi178.stderr: -55> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: secure 2024-06-28T15:24:00.147 INFO:tasks.workunit.client.0.smithi178.stderr: -54> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: crc 2024-06-28T15:24:00.147 INFO:tasks.workunit.client.0.smithi178.stderr: -53> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: secure 2024-06-28T15:24:00.147 INFO:tasks.workunit.client.0.smithi178.stderr: -52> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: crc 2024-06-28T15:24:00.147 INFO:tasks.workunit.client.0.smithi178.stderr: -51> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: secure 2024-06-28T15:24:00.147 INFO:tasks.workunit.client.0.smithi178.stderr: -50> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: crc 2024-06-28T15:24:00.147 INFO:tasks.workunit.client.0.smithi178.stderr: -49> 2024-06-28T15:23:59.993+0000 7f2de37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:00.147 INFO:tasks.workunit.client.0.smithi178.stderr: -48> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: crc 2024-06-28T15:24:00.147 INFO:tasks.workunit.client.0.smithi178.stderr: -47> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: secure 2024-06-28T15:24:00.147 INFO:tasks.workunit.client.0.smithi178.stderr: -46> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: crc 2024-06-28T15:24:00.147 INFO:tasks.workunit.client.0.smithi178.stderr: -45> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: secure 2024-06-28T15:24:00.148 INFO:tasks.workunit.client.0.smithi178.stderr: -44> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: crc 2024-06-28T15:24:00.148 INFO:tasks.workunit.client.0.smithi178.stderr: -43> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 AuthRegistry(0x7fff99381950) adding con mode: secure 2024-06-28T15:24:00.148 INFO:tasks.workunit.client.0.smithi178.stderr: -42> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:00.148 INFO:tasks.workunit.client.0.smithi178.stderr: -41> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:00.148 INFO:tasks.workunit.client.0.smithi178.stderr: -40> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:00.148 INFO:tasks.workunit.client.0.smithi178.stderr: -39> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command cache list hook 0x55f424e4da90 2024-06-28T15:24:00.148 INFO:tasks.workunit.client.0.smithi178.stderr: -38> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command cache inspect hook 0x55f424e4da90 2024-06-28T15:24:00.148 INFO:tasks.workunit.client.0.smithi178.stderr: -37> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command cache erase hook 0x55f424e4da90 2024-06-28T15:24:00.148 INFO:tasks.workunit.client.0.smithi178.stderr: -36> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command cache zap hook 0x55f424e4da90 2024-06-28T15:24:00.149 INFO:tasks.workunit.client.0.smithi178.stderr: -35> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 note: GC not initialized 2024-06-28T15:24:00.149 INFO:tasks.workunit.client.0.smithi178.stderr: -34> 2024-06-28T15:23:59.993+0000 7f2de27e4640 20 reqs_thread_entry: start 2024-06-28T15:24:00.149 INFO:tasks.workunit.client.0.smithi178.stderr: -33> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command sync trace show hook 0x7f2f1c004510 2024-06-28T15:24:00.149 INFO:tasks.workunit.client.0.smithi178.stderr: -32> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command sync trace history hook 0x7f2f1c004510 2024-06-28T15:24:00.149 INFO:tasks.workunit.client.0.smithi178.stderr: -31> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command sync trace active hook 0x7f2f1c004510 2024-06-28T15:24:00.149 INFO:tasks.workunit.client.0.smithi178.stderr: -30> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 5 asok(0x55f424b655e0) register_command sync trace active_short hook 0x7f2f1c004510 2024-06-28T15:24:00.149 INFO:tasks.workunit.client.0.smithi178.stderr: -29> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:00.149 INFO:tasks.workunit.client.0.smithi178.stderr: -28> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 Filter name: none 2024-06-28T15:24:00.149 INFO:tasks.workunit.client.0.smithi178.stderr: -27> 2024-06-28T15:23:59.993+0000 7f2ddbfff640 20 reqs_thread_entry: start 2024-06-28T15:24:00.149 INFO:tasks.workunit.client.0.smithi178.stderr: -26> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-06-28T15:24:00.149 INFO:tasks.workunit.client.0.smithi178.stderr: -25> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.150 INFO:tasks.workunit.client.0.smithi178.stderr: -24> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:24:00.150 INFO:tasks.workunit.client.0.smithi178.stderr: -23> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-06-28T15:24:00.150 INFO:tasks.workunit.client.0.smithi178.stderr: -22> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-06-28T15:24:00.150 INFO:tasks.workunit.client.0.smithi178.stderr: -21> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 : miss 2024-06-28T15:24:00.150 INFO:tasks.workunit.client.0.smithi178.stderr: -20> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.150 INFO:tasks.workunit.client.0.smithi178.stderr: -19> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:24:00.150 INFO:tasks.workunit.client.0.smithi178.stderr: -18> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:24:00.150 INFO:tasks.workunit.client.0.smithi178.stderr: -17> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:24:00.150 INFO:tasks.workunit.client.0.smithi178.stderr: -16> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:24:00.150 INFO:tasks.workunit.client.0.smithi178.stderr: -15> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:00.150 INFO:tasks.workunit.client.0.smithi178.stderr: -14> 2024-06-28T15:23:59.993+0000 7f2f2ef5d980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 2024-06-28T15:24:00.150 INFO:tasks.workunit.client.0.smithi178.stderr: -13> 2024-06-28T15:24:00.001+0000 7f2f2ef5d980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:24:00.150 INFO:tasks.workunit.client.0.smithi178.stderr: -12> 2024-06-28T15:24:00.001+0000 7f2f2ef5d980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:24:00.151 INFO:tasks.workunit.client.0.smithi178.stderr: -11> 2024-06-28T15:24:00.001+0000 7f2f2ef5d980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:00.151 INFO:tasks.workunit.client.0.smithi178.stderr: -10> 2024-06-28T15:24:00.001+0000 7f2f2ef5d980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24, ofs0, ns] 2024-06-28T15:24:00.151 INFO:tasks.workunit.client.0.smithi178.stderr: -9> 2024-06-28T15:24:00.001+0000 7f2e03fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595085 cookie 94507079483440 notifier 6260 bl.length()=991 2024-06-28T15:24:00.151 INFO:tasks.workunit.client.0.smithi178.stderr: -8> 2024-06-28T15:24:00.001+0000 7f2e03fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:24:00.151 INFO:tasks.workunit.client.0.smithi178.stderr: -7> 2024-06-28T15:24:00.001+0000 7f2e03fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:24:00.151 INFO:tasks.workunit.client.0.smithi178.stderr: -6> 2024-06-28T15:24:00.001+0000 7f2e03fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:00.151 INFO:tasks.workunit.client.0.smithi178.stderr: -5> 2024-06-28T15:24:00.005+0000 7f2f2ef5d980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): orig_sources= new_sources= 2024-06-28T15:24:00.151 INFO:tasks.workunit.client.0.smithi178.stderr: -4> 2024-06-28T15:24:00.005+0000 7f2f2ef5d980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): potential sources added=[] removed=[] 2024-06-28T15:24:00.151 INFO:tasks.workunit.client.0.smithi178.stderr: -3> 2024-06-28T15:24:00.005+0000 7f2f2ef5d980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): orig_dests= new_dests= 2024-06-28T15:24:00.151 INFO:tasks.workunit.client.0.smithi178.stderr: -2> 2024-06-28T15:24:00.005+0000 7f2f2ef5d980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): potential dests added=[] removed=[] 2024-06-28T15:24:00.152 INFO:tasks.workunit.client.0.smithi178.stderr: -1> 2024-06-28T15:24:00.005+0000 7f2f2ef5d980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f2f2ef5d980 time 2024-06-28T15:24:00.010620+0000 2024-06-28T15:24:00.152 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:24:00.152 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.152 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:24:00.152 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f2f32834c91] 2024-06-28T15:24:00.152 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f2f32834e45] 2024-06-28T15:24:00.152 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55f421d834b1] 2024-06-28T15:24:00.152 INFO:tasks.workunit.client.0.smithi178.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2bc) [0x55f421d94a3c] 2024-06-28T15:24:00.152 INFO:tasks.workunit.client.0.smithi178.stderr: 5: main() 2024-06-28T15:24:00.152 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2f3169dd90] 2024-06-28T15:24:00.152 INFO:tasks.workunit.client.0.smithi178.stderr: 7: __libc_start_main() 2024-06-28T15:24:00.153 INFO:tasks.workunit.client.0.smithi178.stderr: 8: _start() 2024-06-28T15:24:00.153 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.153 INFO:tasks.workunit.client.0.smithi178.stderr: 0> 2024-06-28T15:24:00.005+0000 7f2f2ef5d980 -1 *** Caught signal (Aborted) ** 2024-06-28T15:24:00.153 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7f2f2ef5d980 thread_name:radosgw-admin 2024-06-28T15:24:00.153 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.153 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:24:00.153 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f2f316b6520] 2024-06-28T15:24:00.153 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:24:00.153 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:24:00.153 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:24:00.153 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f2f32834ceb] 2024-06-28T15:24:00.154 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f2f32834e45] 2024-06-28T15:24:00.154 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55f421d834b1] 2024-06-28T15:24:00.154 INFO:tasks.workunit.client.0.smithi178.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2bc) [0x55f421d94a3c] 2024-06-28T15:24:00.154 INFO:tasks.workunit.client.0.smithi178.stderr: 9: main() 2024-06-28T15:24:00.154 INFO:tasks.workunit.client.0.smithi178.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f2f3169dd90] 2024-06-28T15:24:00.154 INFO:tasks.workunit.client.0.smithi178.stderr: 11: __libc_start_main() 2024-06-28T15:24:00.154 INFO:tasks.workunit.client.0.smithi178.stderr: 12: _start() 2024-06-28T15:24:00.154 INFO:tasks.workunit.client.0.smithi178.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-28T15:24:00.154 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.154 INFO:tasks.workunit.client.0.smithi178.stderr:--- logging levels --- 2024-06-28T15:24:00.155 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 none 2024-06-28T15:24:00.155 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 lockdep 2024-06-28T15:24:00.155 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 context 2024-06-28T15:24:00.155 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 crush 2024-06-28T15:24:00.155 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds 2024-06-28T15:24:00.155 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_balancer 2024-06-28T15:24:00.155 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_locker 2024-06-28T15:24:00.155 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log 2024-06-28T15:24:00.155 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log_expire 2024-06-28T15:24:00.155 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_migrator 2024-06-28T15:24:00.155 INFO:tasks.workunit.client.0.smithi178.stderr: 3/ 5 mds_quiesce 2024-06-28T15:24:00.156 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 buffer 2024-06-28T15:24:00.156 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 timer 2024-06-28T15:24:00.156 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 filer 2024-06-28T15:24:00.156 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 striper 2024-06-28T15:24:00.156 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 objecter 2024-06-28T15:24:00.156 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rados 2024-06-28T15:24:00.156 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd 2024-06-28T15:24:00.156 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_mirror 2024-06-28T15:24:00.156 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_replay 2024-06-28T15:24:00.156 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_pwl 2024-06-28T15:24:00.157 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 journaler 2024-06-28T15:24:00.157 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objectcacher 2024-06-28T15:24:00.157 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 immutable_obj_cache 2024-06-28T15:24:00.157 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 client 2024-06-28T15:24:00.157 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 osd 2024-06-28T15:24:00.157 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 optracker 2024-06-28T15:24:00.157 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objclass 2024-06-28T15:24:00.157 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 filestore 2024-06-28T15:24:00.157 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 journal 2024-06-28T15:24:00.157 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 ms 2024-06-28T15:24:00.157 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mon 2024-06-28T15:24:00.158 INFO:tasks.workunit.client.0.smithi178.stderr: 0/10 monc 2024-06-28T15:24:00.158 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 paxos 2024-06-28T15:24:00.158 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 tp 2024-06-28T15:24:00.158 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 auth 2024-06-28T15:24:00.158 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 crypto 2024-06-28T15:24:00.158 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 finisher 2024-06-28T15:24:00.158 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 reserver 2024-06-28T15:24:00.158 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 heartbeatmap 2024-06-28T15:24:00.158 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 perfcounter 2024-06-28T15:24:00.158 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw 2024-06-28T15:24:00.158 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_sync 2024-06-28T15:24:00.159 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_datacache 2024-06-28T15:24:00.159 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_access 2024-06-28T15:24:00.159 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_dbstore 2024-06-28T15:24:00.159 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_flight 2024-06-28T15:24:00.159 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_lifecycle 2024-06-28T15:24:00.159 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw_notification 2024-06-28T15:24:00.159 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 javaclient 2024-06-28T15:24:00.159 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 asok 2024-06-28T15:24:00.159 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 throttle 2024-06-28T15:24:00.159 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 refs 2024-06-28T15:24:00.160 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 compressor 2024-06-28T15:24:00.160 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluestore 2024-06-28T15:24:00.160 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluefs 2024-06-28T15:24:00.160 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 bdev 2024-06-28T15:24:00.160 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 kstore 2024-06-28T15:24:00.160 INFO:tasks.workunit.client.0.smithi178.stderr: 4/ 5 rocksdb 2024-06-28T15:24:00.160 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 fuse 2024-06-28T15:24:00.160 INFO:tasks.workunit.client.0.smithi178.stderr: 2/ 5 mgr 2024-06-28T15:24:00.160 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mgrc 2024-06-28T15:24:00.160 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 dpdk 2024-06-28T15:24:00.160 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 eventtrace 2024-06-28T15:24:00.161 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 prioritycache 2024-06-28T15:24:00.161 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 test 2024-06-28T15:24:00.161 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephfs_mirror 2024-06-28T15:24:00.161 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephsqlite 2024-06-28T15:24:00.161 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 crimson_interrupt 2024-06-28T15:24:00.161 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore 2024-06-28T15:24:00.161 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_onode 2024-06-28T15:24:00.161 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_odata 2024-06-28T15:24:00.161 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_omap 2024-06-28T15:24:00.161 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_tm 2024-06-28T15:24:00.161 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_t 2024-06-28T15:24:00.161 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cleaner 2024-06-28T15:24:00.161 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_epm 2024-06-28T15:24:00.161 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_lba 2024-06-28T15:24:00.162 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_fixedkv_tree 2024-06-28T15:24:00.162 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cache 2024-06-28T15:24:00.162 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_journal 2024-06-28T15:24:00.162 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_device 2024-06-28T15:24:00.162 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_backref 2024-06-28T15:24:00.162 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 alienstore 2024-06-28T15:24:00.162 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mclock 2024-06-28T15:24:00.162 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cyanstore 2024-06-28T15:24:00.162 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 ceph_exporter 2024-06-28T15:24:00.162 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 memstore 2024-06-28T15:24:00.162 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 trace 2024-06-28T15:24:00.162 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 ceph_dedup 2024-06-28T15:24:00.162 INFO:tasks.workunit.client.0.smithi178.stderr: -2/-2 (syslog threshold) 2024-06-28T15:24:00.163 INFO:tasks.workunit.client.0.smithi178.stderr: 99/99 (stderr threshold) 2024-06-28T15:24:00.163 INFO:tasks.workunit.client.0.smithi178.stderr:--- pthread ID / name mapping for recent threads --- 2024-06-28T15:24:00.163 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2ddbfff640 / http_manager 2024-06-28T15:24:00.163 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2de27e4640 / http_manager 2024-06-28T15:24:00.163 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2de37e6640 / rgw_dt_lg_renew 2024-06-28T15:24:00.163 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2e027fc640 / ms_dispatch 2024-06-28T15:24:00.163 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2e03fff640 / io_context_pool 2024-06-28T15:24:00.163 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2e127fc640 / ms_dispatch 2024-06-28T15:24:00.163 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2e1960a640 / ms_dispatch 2024-06-28T15:24:00.163 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2e1ae0d640 / io_context_pool 2024-06-28T15:24:00.163 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2f2c26d640 / msgr-worker-2 2024-06-28T15:24:00.163 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2f2ca6e640 / msgr-worker-1 2024-06-28T15:24:00.163 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2f2d26f640 / msgr-worker-0 2024-06-28T15:24:00.163 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2f2da70640 / io_context_pool 2024-06-28T15:24:00.164 INFO:tasks.workunit.client.0.smithi178.stderr: 7f2f2ef5d980 / radosgw-admin 2024-06-28T15:24:00.164 INFO:tasks.workunit.client.0.smithi178.stderr: max_recent 500 2024-06-28T15:24:00.164 INFO:tasks.workunit.client.0.smithi178.stderr: max_new 1000 2024-06-28T15:24:00.164 INFO:tasks.workunit.client.0.smithi178.stderr: log_file /var/lib/ceph/crash/2024-06-28T15:24:00.012610Z_19bb4fe9-3f5f-4add-9e0e-9417202ccdf0/log 2024-06-28T15:24:00.164 INFO:tasks.workunit.client.0.smithi178.stderr:--- end dump of recent events --- 2024-06-28T15:24:00.164 INFO:tasks.workunit.client.0.smithi178.stderr:Aborted 2024-06-28T15:24:00.164 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-06-28T15:24:00.169 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:00.169 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:00.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.085+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.085+0000 7f1a6264f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.085+0000 7f1a6264f980 20 realm 2024-06-28T15:24:00.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.085+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.085+0000 7f1a6264f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.085+0000 7f1a6264f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:00.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.085+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.085+0000 7f1a6264f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.085+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.089+0000 7f1a6264f980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:00.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.089+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.089+0000 7f1a6264f980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:00.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.089+0000 7f1a6264f980 20 searching for the correct realm 2024-06-28T15:24:00.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.093+0000 7f1a5f95f640 0 -- 172.21.15.178:0/300657273 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x55d08ec7a310 msgr2=0x55d08ec7c700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:00.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.097+0000 7f1a60160640 0 -- 172.21.15.178:0/300657273 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f1a54003b90 msgr2=0x7f1a54072040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:00.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.105+0000 7f1a6264f980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:00.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.105+0000 7f1a6264f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:00.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.105+0000 7f1a6264f980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:00.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.105+0000 7f1a6264f980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:00.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.105+0000 7f1a6264f980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:00.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.105+0000 7f1a6264f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:00.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.105+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.105+0000 7f1a6264f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.105+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.105+0000 7f1a6264f980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:00.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.105+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:00.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 20 zone default found 2024-06-28T15:24:00.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 4 Realm: () 2024-06-28T15:24:00.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:00.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:00.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:00.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 20 zonegroup default 2024-06-28T15:24:00.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 20 started sync module instance, tier type = 2024-06-28T15:24:00.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.109+0000 7f1a6264f980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:00.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.125+0000 7f1a6264f980 20 add_watcher() i=0 2024-06-28T15:24:00.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.125+0000 7f1a6264f980 20 add_watcher() i=1 2024-06-28T15:24:00.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.125+0000 7f1a6264f980 20 add_watcher() i=2 2024-06-28T15:24:00.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.125+0000 7f1a6264f980 20 add_watcher() i=3 2024-06-28T15:24:00.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.125+0000 7f1a6264f980 20 add_watcher() i=4 2024-06-28T15:24:00.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.125+0000 7f1a6264f980 20 add_watcher() i=5 2024-06-28T15:24:00.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.125+0000 7f1a6264f980 20 add_watcher() i=6 2024-06-28T15:24:00.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.125+0000 7f1a6264f980 20 add_watcher() i=7 2024-06-28T15:24:00.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.125+0000 7f1a6264f980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:00.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.125+0000 7f1916fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:00.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:00.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:00.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 5 note: GC not initialized 2024-06-28T15:24:00.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1915fdb640 20 reqs_thread_entry: start 2024-06-28T15:24:00.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:00.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 20 Filter name: none 2024-06-28T15:24:00.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f18877fe640 20 reqs_thread_entry: start 2024-06-28T15:24:00.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:24:00.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 20 rados_obj.operate() r=0 bl.length=459 2024-06-28T15:24:00.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:00.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:24:00.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 20 rados_obj.operate() r=0 bl.length=459 2024-06-28T15:24:00.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.129+0000 7f1a6264f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:00.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.133+0000 7f1916fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:00.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.137+0000 7f1a6264f980 20 remove_watcher() i=0 2024-06-28T15:24:00.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.137+0000 7f1a6264f980 2 removed watcher, disabling cache 2024-06-28T15:24:00.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.137+0000 7f1a6264f980 20 remove_watcher() i=1 2024-06-28T15:24:00.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.137+0000 7f1a6264f980 20 remove_watcher() i=2 2024-06-28T15:24:00.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.141+0000 7f1a6264f980 20 remove_watcher() i=3 2024-06-28T15:24:00.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.141+0000 7f1a6264f980 20 remove_watcher() i=4 2024-06-28T15:24:00.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.141+0000 7f1a6264f980 20 remove_watcher() i=5 2024-06-28T15:24:00.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.145+0000 7f1a6264f980 20 remove_watcher() i=6 2024-06-28T15:24:00.176 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.145+0000 7f1a6264f980 20 remove_watcher() i=7 2024-06-28T15:24:00.176 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.176 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:24:00.176 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: abort-at-block-writes id: 843e24ce-085a-4043-8f55-38e05370e790.4686.24 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:24:00.176 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:00.241 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-06-28T15:24:00.275 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:00.275 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:00.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.537+0000 7efc64c65980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.541+0000 7efc64c65980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.541+0000 7efc64c65980 20 realm 2024-06-28T15:24:00.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.541+0000 7efc64c65980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.541+0000 7efc64c65980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.525 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.541+0000 7efc64c65980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:00.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.541+0000 7efc64c65980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.541+0000 7efc64c65980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.526 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.541+0000 7efc64c65980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.541+0000 7efc64c65980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:00.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.541+0000 7efc64c65980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.545+0000 7efc64c65980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:00.527 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.545+0000 7efc64c65980 20 searching for the correct realm 2024-06-28T15:24:00.532 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.549+0000 7efc61f75640 0 -- 172.21.15.178:0/4188325452 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7efc540c5070 msgr2=0x7efc540fe9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:00.539 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.553+0000 7efc62776640 0 -- 172.21.15.178:0/4188325452 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7efc5804d040 msgr2=0x7efc5804d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:00.755 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.769+0000 7efc64c65980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:00.755 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.769+0000 7efc64c65980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:00.755 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.769+0000 7efc64c65980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:00.755 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.769+0000 7efc64c65980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:00.755 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.769+0000 7efc64c65980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:00.755 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.769+0000 7efc64c65980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:00.755 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.769+0000 7efc64c65980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.756 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.769+0000 7efc64c65980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.756 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.769+0000 7efc64c65980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.756 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.773+0000 7efc64c65980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:00.756 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.773+0000 7efc64c65980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.773+0000 7efc64c65980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:00.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.773+0000 7efc64c65980 20 zone default found 2024-06-28T15:24:00.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.773+0000 7efc64c65980 4 Realm: () 2024-06-28T15:24:00.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.773+0000 7efc64c65980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:00.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.773+0000 7efc64c65980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:00.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.773+0000 7efc64c65980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:00.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.773+0000 7efc64c65980 20 zonegroup default 2024-06-28T15:24:00.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.773+0000 7efc64c65980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.773+0000 7efc64c65980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.773+0000 7efc64c65980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.773+0000 7efc64c65980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.773+0000 7efc64c65980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.777+0000 7efc64c65980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:00.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.777+0000 7efc64c65980 20 started sync module instance, tier type = 2024-06-28T15:24:00.759 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.777+0000 7efc64c65980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:00.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.797+0000 7efc64c65980 20 add_watcher() i=0 2024-06-28T15:24:00.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.797+0000 7efc64c65980 20 add_watcher() i=1 2024-06-28T15:24:00.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.797+0000 7efc64c65980 20 add_watcher() i=2 2024-06-28T15:24:00.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.797+0000 7efc64c65980 20 add_watcher() i=3 2024-06-28T15:24:00.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.797+0000 7efc64c65980 20 add_watcher() i=4 2024-06-28T15:24:00.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.797+0000 7efc64c65980 20 add_watcher() i=5 2024-06-28T15:24:00.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.797+0000 7efc64c65980 20 add_watcher() i=6 2024-06-28T15:24:00.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.801+0000 7efc64c65980 20 add_watcher() i=7 2024-06-28T15:24:00.785 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.801+0000 7efc64c65980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:00.789 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.805+0000 7efb18fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:00.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.805+0000 7efc64c65980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:00.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.805+0000 7efc64c65980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:00.790 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.805+0000 7efc64c65980 5 note: GC not initialized 2024-06-28T15:24:00.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.805+0000 7efa937fe640 20 reqs_thread_entry: start 2024-06-28T15:24:00.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.805+0000 7efc64c65980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:00.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.805+0000 7efc64c65980 20 Filter name: none 2024-06-28T15:24:00.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.809+0000 7efa917fa640 20 reqs_thread_entry: start 2024-06-28T15:24:00.791 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.809+0000 7efc64c65980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-06-28T15:24:00.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.809+0000 7efc64c65980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.809+0000 7efc64c65980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:24:00.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.809+0000 7efc64c65980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-06-28T15:24:00.792 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.809+0000 7efc64c65980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-06-28T15:24:00.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.809+0000 7efc64c65980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 : miss 2024-06-28T15:24:00.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.809+0000 7efc64c65980 20 rados->read ofs=0 len=0 2024-06-28T15:24:00.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.809+0000 7efc64c65980 20 rados_obj.operate() r=0 bl.length=459 2024-06-28T15:24:00.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.809+0000 7efc64c65980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:24:00.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.809+0000 7efc64c65980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:24:00.793 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.809+0000 7efc64c65980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:24:00.794 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.809+0000 7efc64c65980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:00.794 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.809+0000 7efc64c65980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 2024-06-28T15:24:00.795 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.813+0000 7efc64c65980 0 INFO: RGWReshardLock::lock found lock on abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to be held by another RGW process; skipping for now 2024-06-28T15:24:00.797 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.813+0000 7efb18fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:00.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.817+0000 7efc64c65980 20 remove_watcher() i=0 2024-06-28T15:24:00.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.817+0000 7efc64c65980 2 removed watcher, disabling cache 2024-06-28T15:24:00.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.821+0000 7efc64c65980 20 remove_watcher() i=1 2024-06-28T15:24:00.807 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.821+0000 7efc64c65980 20 remove_watcher() i=2 2024-06-28T15:24:00.809 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.825+0000 7efc64c65980 20 remove_watcher() i=3 2024-06-28T15:24:00.811 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.825+0000 7efc64c65980 20 remove_watcher() i=4 2024-06-28T15:24:00.812 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.829+0000 7efc64c65980 20 remove_watcher() i=5 2024-06-28T15:24:00.814 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.829+0000 7efc64c65980 20 remove_watcher() i=6 2024-06-28T15:24:00.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:00.833+0000 7efc64c65980 20 remove_watcher() i=7 2024-06-28T15:24:00.832 INFO:tasks.workunit.client.0.smithi178.stderr:waiting 30 seconds for reshard lock to expire... 2024-06-28T15:24:30.864 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-06-28T15:24:30.900 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:30.900 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:30.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.961+0000 7fc01c032980 20 rados->read ofs=0 len=0 2024-06-28T15:24:30.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.961+0000 7fc01c032980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:30.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.961+0000 7fc01c032980 20 realm 2024-06-28T15:24:30.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.961+0000 7fc01c032980 20 rados->read ofs=0 len=0 2024-06-28T15:24:30.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.965+0000 7fc01c032980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:30.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.965+0000 7fc01c032980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:30.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.965+0000 7fc01c032980 20 rados->read ofs=0 len=0 2024-06-28T15:24:30.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.965+0000 7fc01c032980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:30.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.965+0000 7fc01c032980 20 rados->read ofs=0 len=0 2024-06-28T15:24:30.949 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.965+0000 7fc01c032980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:30.949 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.965+0000 7fc01c032980 20 rados->read ofs=0 len=0 2024-06-28T15:24:30.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.965+0000 7fc01c032980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:30.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.965+0000 7fc01c032980 20 searching for the correct realm 2024-06-28T15:24:30.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.973+0000 7fc019342640 0 -- 172.21.15.178:0/1177309331 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fc00c0a4160 msgr2=0x7fc00c0f6140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:30.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.977+0000 7fc01a344640 0 -- 172.21.15.178:0/1177309331 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fc010056920 msgr2=0x7fc0100373f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:24:30.970 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.985+0000 7fc01c032980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:30.970 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.985+0000 7fc01c032980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:30.970 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.985+0000 7fc01c032980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:30.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.985+0000 7fc01c032980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:30.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.985+0000 7fc01c032980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:30.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.985+0000 7fc01c032980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:30.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.985+0000 7fc01c032980 20 rados->read ofs=0 len=0 2024-06-28T15:24:30.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.985+0000 7fc01c032980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:30.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.985+0000 7fc01c032980 20 rados->read ofs=0 len=0 2024-06-28T15:24:30.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:30.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 20 rados->read ofs=0 len=0 2024-06-28T15:24:30.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:30.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 20 zone default found 2024-06-28T15:24:30.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 4 Realm: () 2024-06-28T15:24:30.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:30.972 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:30.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:30.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 20 zonegroup default 2024-06-28T15:24:30.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 20 rados->read ofs=0 len=0 2024-06-28T15:24:30.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:30.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 20 rados->read ofs=0 len=0 2024-06-28T15:24:30.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:30.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 20 rados->read ofs=0 len=0 2024-06-28T15:24:30.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:30.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 20 started sync module instance, tier type = 2024-06-28T15:24:30.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:30.989+0000 7fc01c032980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:30.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.005+0000 7fc01c032980 20 add_watcher() i=0 2024-06-28T15:24:30.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.005+0000 7fc01c032980 20 add_watcher() i=1 2024-06-28T15:24:30.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.005+0000 7fc01c032980 20 add_watcher() i=2 2024-06-28T15:24:30.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.005+0000 7fc01c032980 20 add_watcher() i=3 2024-06-28T15:24:30.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.005+0000 7fc01c032980 20 add_watcher() i=4 2024-06-28T15:24:30.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.005+0000 7fc01c032980 20 add_watcher() i=5 2024-06-28T15:24:30.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.005+0000 7fc01c032980 20 add_watcher() i=6 2024-06-28T15:24:30.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.005+0000 7fc01c032980 20 add_watcher() i=7 2024-06-28T15:24:30.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.005+0000 7fc01c032980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:30.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fbecbfff640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:30.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fc01c032980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:30.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fc01c032980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:30.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fc01c032980 5 note: GC not initialized 2024-06-28T15:24:30.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fbecaffd640 20 reqs_thread_entry: start 2024-06-28T15:24:30.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fc01c032980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:30.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fc01c032980 20 Filter name: none 2024-06-28T15:24:30.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fbec8ff9640 20 reqs_thread_entry: start 2024-06-28T15:24:30.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fc01c032980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-06-28T15:24:30.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fc01c032980 20 rados->read ofs=0 len=0 2024-06-28T15:24:30.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fc01c032980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:24:30.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fc01c032980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-06-28T15:24:30.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fc01c032980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-06-28T15:24:30.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fc01c032980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 : miss 2024-06-28T15:24:30.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.009+0000 7fc01c032980 20 rados->read ofs=0 len=0 2024-06-28T15:24:30.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.013+0000 7fc01c032980 20 rados_obj.operate() r=0 bl.length=459 2024-06-28T15:24:30.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.013+0000 7fc01c032980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:24:30.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.013+0000 7fc01c032980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:24:30.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.013+0000 7fc01c032980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:24:30.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.013+0000 7fc01c032980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:30.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.013+0000 7fc01c032980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 2024-06-28T15:24:30.998 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.013+0000 7fc01c032980 10 init_target_layout removing existing target index objects from a previous reshard attempt 2024-06-28T15:24:31.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.021+0000 7fc01c032980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:24:31.006 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.021+0000 7fc01c032980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:24:31.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.021+0000 7fc01c032980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:31.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.021+0000 7fc01c032980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24, ofs0, ns] 2024-06-28T15:24:31.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.021+0000 7fbefd7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595086 cookie 94359769665696 notifier 6308 bl.length()=991 2024-06-28T15:24:31.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.021+0000 7fbefd7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:24:31.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.021+0000 7fbefd7fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:24:31.007 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.021+0000 7fbefd7fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:31.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.025+0000 7fc01c032980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): orig_sources= new_sources= 2024-06-28T15:24:31.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.025+0000 7fc01c032980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): potential sources added=[] removed=[] 2024-06-28T15:24:31.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.025+0000 7fc01c032980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): orig_dests= new_dests= 2024-06-28T15:24:31.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.025+0000 7fc01c032980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): potential dests added=[] removed=[] 2024-06-28T15:24:31.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.025+0000 7fc01c032980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.24.2.0 2024-06-28T15:24:31.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.025+0000 7fc01c032980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.24.2.0 2024-06-28T15:24:31.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.025+0000 7fc01c032980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.24.2.1 2024-06-28T15:24:31.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.025+0000 7fc01c032980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.24.2.1 2024-06-28T15:24:31.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.025+0000 7fc01c032980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.24.0 2024-06-28T15:24:31.015 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.033+0000 7fc01c032980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 : hit (requested=0x17, cached=0x17) 2024-06-28T15:24:31.015 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.033+0000 7fc01c032980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 2024-06-28T15:24:31.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.033+0000 7fc01c032980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:24:31.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.033+0000 7fc01c032980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:24:31.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.033+0000 7fc01c032980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:31.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.033+0000 7fc01c032980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24, ofs0, ns] 2024-06-28T15:24:31.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.033+0000 7fbefcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595087 cookie 94359769665696 notifier 6308 bl.length()=959 2024-06-28T15:24:31.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.033+0000 7fbefcff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 info.flags=0x17 2024-06-28T15:24:31.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.033+0000 7fbefcff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:843e24ce-085a-4043-8f55-38e05370e790.4686.24 to cache LRU end 2024-06-28T15:24:31.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.033+0000 7fbefcff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:31.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.037+0000 7fc01c032980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): orig_sources= new_sources= 2024-06-28T15:24:31.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.037+0000 7fc01c032980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): potential sources added=[] removed=[] 2024-06-28T15:24:31.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.037+0000 7fc01c032980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): orig_dests= new_dests= 2024-06-28T15:24:31.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.037+0000 7fc01c032980 20 handle_bi_update(): bucket=:abort-at-block-writes[843e24ce-085a-4043-8f55-38e05370e790.4686.24]): potential dests added=[] removed=[] 2024-06-28T15:24:31.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.037+0000 7fc01c032980 1 execute INFO: reshard of bucket "abort-at-block-writes" from 2 shards to 2 shards completed successfully 2024-06-28T15:24:31.025 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.041+0000 7fbecbfff640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:31.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.045+0000 7fc01c032980 20 remove_watcher() i=0 2024-06-28T15:24:31.030 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.045+0000 7fc01c032980 2 removed watcher, disabling cache 2024-06-28T15:24:31.032 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.049+0000 7fc01c032980 20 remove_watcher() i=1 2024-06-28T15:24:31.034 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.049+0000 7fc01c032980 20 remove_watcher() i=2 2024-06-28T15:24:31.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.053+0000 7fc01c032980 20 remove_watcher() i=3 2024-06-28T15:24:31.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.053+0000 7fc01c032980 20 remove_watcher() i=4 2024-06-28T15:24:31.040 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.057+0000 7fc01c032980 20 remove_watcher() i=5 2024-06-28T15:24:31.041 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.057+0000 7fc01c032980 20 remove_watcher() i=6 2024-06-28T15:24:31.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.057+0000 7fc01c032980 20 remove_watcher() i=7 2024-06-28T15:24:31.056 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-06-28T15:24:31.082 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:31.082 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:31.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.141+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.141+0000 7f3698178980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:31.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.141+0000 7f3698178980 20 realm 2024-06-28T15:24:31.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.141+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.141+0000 7f3698178980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:31.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.141+0000 7f3698178980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:31.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.141+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.141+0000 7f3698178980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:31.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.141+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.145+0000 7f3698178980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:31.128 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.145+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.145+0000 7f3698178980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:31.129 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.145+0000 7f3698178980 20 searching for the correct realm 2024-06-28T15:24:31.135 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.153+0000 7f3695488640 0 -- 172.21.15.178:0/3458497457 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f3688008050 msgr2=0x7f368806a9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:31.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.157+0000 7f369648a640 0 -- 172.21.15.178:0/3458497457 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f36800673e0 msgr2=0x7f36800808e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:31.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:31.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:31.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:31.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:31.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:31.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:31.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:31.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:31.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:31.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 zone default found 2024-06-28T15:24:31.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 4 Realm: () 2024-06-28T15:24:31.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:31.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:31.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:31.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 zonegroup default 2024-06-28T15:24:31.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:31.150 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:31.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:31.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 started sync module instance, tier type = 2024-06-28T15:24:31.152 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.165+0000 7f3698178980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:31.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.181+0000 7f3698178980 20 add_watcher() i=0 2024-06-28T15:24:31.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.181+0000 7f3698178980 20 add_watcher() i=1 2024-06-28T15:24:31.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.181+0000 7f3698178980 20 add_watcher() i=2 2024-06-28T15:24:31.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.181+0000 7f3698178980 20 add_watcher() i=3 2024-06-28T15:24:31.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.181+0000 7f3698178980 20 add_watcher() i=4 2024-06-28T15:24:31.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.181+0000 7f3698178980 20 add_watcher() i=5 2024-06-28T15:24:31.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.181+0000 7f3698178980 20 add_watcher() i=6 2024-06-28T15:24:31.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.185+0000 7f3698178980 20 add_watcher() i=7 2024-06-28T15:24:31.167 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.185+0000 7f3698178980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:31.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.185+0000 7f35477fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:31.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.185+0000 7f3698178980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:31.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.185+0000 7f3698178980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:31.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.185+0000 7f3698178980 5 note: GC not initialized 2024-06-28T15:24:31.174 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.185+0000 7f35467fc640 20 reqs_thread_entry: start 2024-06-28T15:24:31.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.185+0000 7f3698178980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:31.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.185+0000 7f3698178980 20 Filter name: none 2024-06-28T15:24:31.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.185+0000 7f34b7fff640 20 reqs_thread_entry: start 2024-06-28T15:24:31.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.185+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.185+0000 7f3698178980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:24:31.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.185+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.189+0000 7f3698178980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:24:31.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.189+0000 7f3698178980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:31.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.189+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.189+0000 7f3698178980 20 rados_obj.operate() r=0 bl.length=213 2024-06-28T15:24:31.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.189+0000 7f3698178980 20 rados->read ofs=0 len=0 2024-06-28T15:24:31.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.189+0000 7f3698178980 20 rados_obj.operate() r=0 bl.length=427 2024-06-28T15:24:31.175 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.189+0000 7f3698178980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:31.176 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.193+0000 7f35477fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:31.180 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.197+0000 7f3698178980 20 remove_watcher() i=0 2024-06-28T15:24:31.180 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.197+0000 7f3698178980 2 removed watcher, disabling cache 2024-06-28T15:24:31.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.197+0000 7f3698178980 20 remove_watcher() i=1 2024-06-28T15:24:31.184 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.201+0000 7f3698178980 20 remove_watcher() i=2 2024-06-28T15:24:31.186 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.201+0000 7f3698178980 20 remove_watcher() i=3 2024-06-28T15:24:31.188 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.205+0000 7f3698178980 20 remove_watcher() i=4 2024-06-28T15:24:31.190 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.205+0000 7f3698178980 20 remove_watcher() i=5 2024-06-28T15:24:31.191 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.205+0000 7f3698178980 20 remove_watcher() i=6 2024-06-28T15:24:31.192 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:31.209+0000 7f3698178980 20 remove_watcher() i=7 2024-06-28T15:24:31.205 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:31.205 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:24:31.205 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: abort-at-block-writes id: 843e24ce-085a-4043-8f55-38e05370e790.4686.24 num_objects: 19 size_kb: 1 num_shards: 2 2024-06-28T15:24:31.205 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:31.958 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: reshard bucket with EIO injected at commit_target_layout 2024-06-28T15:24:31.958 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:34.177 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-06-28T15:24:34.209 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:34.209 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:34.254 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.269+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.273+0000 7fd658da6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.273+0000 7fd658da6980 20 realm 2024-06-28T15:24:34.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.273+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.273+0000 7fd658da6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.273+0000 7fd658da6980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:34.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.273+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.273+0000 7fd658da6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.273+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.258 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.273+0000 7fd658da6980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:34.258 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.273+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.277+0000 7fd658da6980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:34.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.277+0000 7fd658da6980 20 searching for the correct realm 2024-06-28T15:24:34.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.281+0000 7fd6570b8640 0 -- 172.21.15.178:0/350419839 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x5618e6ddcad0 msgr2=0x5618e6ddef70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:34.271 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.285+0000 7fd6568b7640 0 -- 172.21.15.178:0/350419839 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fd64c02d920 msgr2=0x7fd64c030400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:34.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.293+0000 7fd658da6980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:34.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.293+0000 7fd658da6980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:34.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.293+0000 7fd658da6980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:34.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.293+0000 7fd658da6980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:34.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.293+0000 7fd658da6980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:34.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.293+0000 7fd658da6980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:34.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.293+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.293+0000 7fd658da6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.293+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.279 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:34.280 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.280 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:34.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 20 zone default found 2024-06-28T15:24:34.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 4 Realm: () 2024-06-28T15:24:34.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:34.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:34.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:34.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 20 zonegroup default 2024-06-28T15:24:34.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 20 started sync module instance, tier type = 2024-06-28T15:24:34.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.297+0000 7fd658da6980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:34.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.313+0000 7fd658da6980 20 add_watcher() i=0 2024-06-28T15:24:34.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.313+0000 7fd658da6980 20 add_watcher() i=1 2024-06-28T15:24:34.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.313+0000 7fd658da6980 20 add_watcher() i=2 2024-06-28T15:24:34.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.313+0000 7fd658da6980 20 add_watcher() i=3 2024-06-28T15:24:34.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.313+0000 7fd658da6980 20 add_watcher() i=4 2024-06-28T15:24:34.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.313+0000 7fd658da6980 20 add_watcher() i=5 2024-06-28T15:24:34.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.313+0000 7fd658da6980 20 add_watcher() i=6 2024-06-28T15:24:34.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.317+0000 7fd658da6980 20 add_watcher() i=7 2024-06-28T15:24:34.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.317+0000 7fd658da6980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:34.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.317+0000 7fd50d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:34.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.317+0000 7fd658da6980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:34.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.317+0000 7fd658da6980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:34.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.317+0000 7fd658da6980 5 note: GC not initialized 2024-06-28T15:24:34.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.317+0000 7fd487fff640 20 reqs_thread_entry: start 2024-06-28T15:24:34.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.317+0000 7fd658da6980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:34.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.317+0000 7fd658da6980 20 Filter name: none 2024-06-28T15:24:34.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.317+0000 7fd485ffb640 20 reqs_thread_entry: start 2024-06-28T15:24:34.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.321+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.321+0000 7fd658da6980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:34.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.321+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.321+0000 7fd658da6980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:34.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.321+0000 7fd658da6980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:34.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.321+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.321+0000 7fd658da6980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:34.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.321+0000 7fd658da6980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.321+0000 7fd658da6980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:34.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.321+0000 7fd658da6980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:34.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.325+0000 7fd50d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:34.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.329+0000 7fd658da6980 20 remove_watcher() i=0 2024-06-28T15:24:34.313 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.329+0000 7fd658da6980 2 removed watcher, disabling cache 2024-06-28T15:24:34.315 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.333+0000 7fd658da6980 20 remove_watcher() i=1 2024-06-28T15:24:34.317 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.333+0000 7fd658da6980 20 remove_watcher() i=2 2024-06-28T15:24:34.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.333+0000 7fd658da6980 20 remove_watcher() i=3 2024-06-28T15:24:34.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.337+0000 7fd658da6980 20 remove_watcher() i=4 2024-06-28T15:24:34.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.337+0000 7fd658da6980 20 remove_watcher() i=5 2024-06-28T15:24:34.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.341+0000 7fd658da6980 20 remove_watcher() i=6 2024-06-28T15:24:34.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.341+0000 7fd658da6980 20 remove_watcher() i=7 2024-06-28T15:24:34.338 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:34.338 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:24:34.338 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-commit-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.25 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:24:34.339 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:34.339 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout 2024-06-28T15:24:34.370 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:34.370 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:34.413 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.429+0000 7fdb40620980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.429+0000 7fdb40620980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.429+0000 7fdb40620980 20 realm 2024-06-28T15:24:34.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.429+0000 7fdb40620980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.429+0000 7fdb40620980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.429+0000 7fdb40620980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:34.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.429+0000 7fdb40620980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.429+0000 7fdb40620980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.415 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.433+0000 7fdb40620980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.433+0000 7fdb40620980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:34.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.433+0000 7fdb40620980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.433+0000 7fdb40620980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:34.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.433+0000 7fdb40620980 20 searching for the correct realm 2024-06-28T15:24:34.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.441+0000 7fdb3d930640 0 -- 172.21.15.178:0/1568146158 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fdb30017070 msgr2=0x7fdb3005a1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:34.429 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.445+0000 7fdb3d930640 0 -- 172.21.15.178:0/1568146158 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fda0807fda0 msgr2=0x7fda080821d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:34.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:34.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:34.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:34.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:34.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:34.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:34.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:34.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:34.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 zone default found 2024-06-28T15:24:34.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 4 Realm: () 2024-06-28T15:24:34.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:34.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:34.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:34.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 zonegroup default 2024-06-28T15:24:34.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 started sync module instance, tier type = 2024-06-28T15:24:34.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.453+0000 7fdb40620980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:34.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.469+0000 7fdb40620980 20 add_watcher() i=0 2024-06-28T15:24:34.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.469+0000 7fdb40620980 20 add_watcher() i=1 2024-06-28T15:24:34.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.469+0000 7fdb40620980 20 add_watcher() i=2 2024-06-28T15:24:34.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.469+0000 7fdb40620980 20 add_watcher() i=3 2024-06-28T15:24:34.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.469+0000 7fdb40620980 20 add_watcher() i=4 2024-06-28T15:24:34.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.469+0000 7fdb40620980 20 add_watcher() i=5 2024-06-28T15:24:34.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.469+0000 7fdb40620980 20 add_watcher() i=6 2024-06-28T15:24:34.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.473+0000 7fdb40620980 20 add_watcher() i=7 2024-06-28T15:24:34.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.473+0000 7fdb40620980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:34.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.473+0000 7fd9f57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:34.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.473+0000 7fdb40620980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:34.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.473+0000 7fdb40620980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:34.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.473+0000 7fdb40620980 5 note: GC not initialized 2024-06-28T15:24:34.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.473+0000 7fd9ef7fe640 20 reqs_thread_entry: start 2024-06-28T15:24:34.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.473+0000 7fdb40620980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:34.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.473+0000 7fdb40620980 20 Filter name: none 2024-06-28T15:24:34.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.473+0000 7fd9ed7fa640 20 reqs_thread_entry: start 2024-06-28T15:24:34.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.477+0000 7fdb40620980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-06-28T15:24:34.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.477+0000 7fdb40620980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.477+0000 7fdb40620980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:34.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.477+0000 7fdb40620980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-06-28T15:24:34.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.477+0000 7fdb40620980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-06-28T15:24:34.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.477+0000 7fdb40620980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 : miss 2024-06-28T15:24:34.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.477+0000 7fdb40620980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.477+0000 7fdb40620980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:34.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.477+0000 7fdb40620980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 info.flags=0x17 2024-06-28T15:24:34.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.477+0000 7fdb40620980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 to cache LRU end 2024-06-28T15:24:34.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.477+0000 7fdb40620980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:24:34.462 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.477+0000 7fdb40620980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:34.462 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.477+0000 7fdb40620980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 2024-06-28T15:24:34.467 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.481+0000 7fdb40620980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 info.flags=0x17 2024-06-28T15:24:34.467 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.481+0000 7fdb40620980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 to cache LRU end 2024-06-28T15:24:34.467 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.481+0000 7fdb40620980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:34.467 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.481+0000 7fdb40620980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25, ofs0, ns] 2024-06-28T15:24:34.468 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.485+0000 7fda21ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595109 cookie 94801347250768 notifier 6359 bl.length()=1007 2024-06-28T15:24:34.468 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.485+0000 7fda21ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 info.flags=0x17 2024-06-28T15:24:34.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.485+0000 7fda21ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 to cache LRU end 2024-06-28T15:24:34.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.485+0000 7fda21ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:34.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.485+0000 7fdb40620980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.25]): orig_sources= new_sources= 2024-06-28T15:24:34.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.485+0000 7fdb40620980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.25]): potential sources added=[] removed=[] 2024-06-28T15:24:34.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.485+0000 7fdb40620980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.25]): orig_dests= new_dests= 2024-06-28T15:24:34.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.485+0000 7fdb40620980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.25]): potential dests added=[] removed=[] 2024-06-28T15:24:34.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.489+0000 7fdb40620980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.25.1.0 2024-06-28T15:24:34.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.489+0000 7fdb40620980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.25.1.0 2024-06-28T15:24:34.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.489+0000 7fdb40620980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.25.1.1 2024-06-28T15:24:34.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.489+0000 7fdb40620980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.25.1.1 2024-06-28T15:24:34.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.489+0000 7fdb40620980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.25.0 2024-06-28T15:24:34.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.493+0000 7fdb40620980 -1 Injecting error=-5 at location=commit_target_layout 2024-06-28T15:24:34.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.493+0000 7fdb40620980 0 ERROR: commit_reshard failed to commit target index layout: (5) Input/output error 2024-06-28T15:24:34.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.497+0000 7fd9f57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:34.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.501+0000 7fdb40620980 20 remove_watcher() i=0 2024-06-28T15:24:34.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.501+0000 7fdb40620980 2 removed watcher, disabling cache 2024-06-28T15:24:34.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.501+0000 7fdb40620980 20 remove_watcher() i=1 2024-06-28T15:24:34.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.505+0000 7fdb40620980 20 remove_watcher() i=2 2024-06-28T15:24:34.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.505+0000 7fdb40620980 20 remove_watcher() i=3 2024-06-28T15:24:34.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.509+0000 7fdb40620980 20 remove_watcher() i=4 2024-06-28T15:24:34.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.509+0000 7fdb40620980 20 remove_watcher() i=5 2024-06-28T15:24:34.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.509+0000 7fdb40620980 20 remove_watcher() i=6 2024-06-28T15:24:34.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.513+0000 7fdb40620980 20 remove_watcher() i=7 2024-06-28T15:24:34.511 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-06-28T15:24:34.537 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:34.537 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:34.579 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.593+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.597+0000 7f9fd10a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.597+0000 7f9fd10a7980 20 realm 2024-06-28T15:24:34.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.597+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.597+0000 7f9fd10a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.597+0000 7f9fd10a7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:34.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.597+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.597+0000 7f9fd10a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.581 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.597+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.583 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.597+0000 7f9fd10a7980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:34.583 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.597+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.583 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.601+0000 7f9fd10a7980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:34.584 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.601+0000 7f9fd10a7980 20 searching for the correct realm 2024-06-28T15:24:34.589 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.605+0000 7f9fce3b7640 0 -- 172.21.15.178:0/2324880728 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x55aa00d3f400 msgr2=0x55aa00d41810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:34.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.613+0000 7f9fce3b7640 0 -- 172.21.15.178:0/2324880728 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f9fc0037030 msgr2=0x7f9fc0037410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:34.603 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:34.603 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:34.603 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:34.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:34.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:34.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:34.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:34.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.605 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:34.605 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 zone default found 2024-06-28T15:24:34.605 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 4 Realm: () 2024-06-28T15:24:34.605 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:34.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:34.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:34.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 zonegroup default 2024-06-28T15:24:34.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 started sync module instance, tier type = 2024-06-28T15:24:34.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.621+0000 7f9fd10a7980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:34.621 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.637+0000 7f9fd10a7980 20 add_watcher() i=0 2024-06-28T15:24:34.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.637+0000 7f9fd10a7980 20 add_watcher() i=1 2024-06-28T15:24:34.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.637+0000 7f9fd10a7980 20 add_watcher() i=2 2024-06-28T15:24:34.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.637+0000 7f9fd10a7980 20 add_watcher() i=3 2024-06-28T15:24:34.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.637+0000 7f9fd10a7980 20 add_watcher() i=4 2024-06-28T15:24:34.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.637+0000 7f9fd10a7980 20 add_watcher() i=5 2024-06-28T15:24:34.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.637+0000 7f9fd10a7980 20 add_watcher() i=6 2024-06-28T15:24:34.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.637+0000 7f9fd10a7980 20 add_watcher() i=7 2024-06-28T15:24:34.623 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.637+0000 7f9fd10a7980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:34.625 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.641+0000 7f9e857e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:34.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.641+0000 7f9fd10a7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:34.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.641+0000 7f9fd10a7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:34.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.641+0000 7f9fd10a7980 5 note: GC not initialized 2024-06-28T15:24:34.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.641+0000 7f9dfffff640 20 reqs_thread_entry: start 2024-06-28T15:24:34.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.641+0000 7f9fd10a7980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:34.626 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.641+0000 7f9fd10a7980 20 Filter name: none 2024-06-28T15:24:34.627 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.641+0000 7f9dfdffb640 20 reqs_thread_entry: start 2024-06-28T15:24:34.627 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.641+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.627 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.645+0000 7f9fd10a7980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:34.627 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.645+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.645+0000 7f9fd10a7980 20 rados_obj.operate() r=0 bl.length=467 2024-06-28T15:24:34.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.645+0000 7f9fd10a7980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:34.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.645+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.645+0000 7f9fd10a7980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:34.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.645+0000 7f9fd10a7980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.629 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.645+0000 7f9fd10a7980 20 rados_obj.operate() r=0 bl.length=467 2024-06-28T15:24:34.629 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.645+0000 7f9fd10a7980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:34.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.645+0000 7f9e857e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:34.635 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.653+0000 7f9fd10a7980 20 remove_watcher() i=0 2024-06-28T15:24:34.635 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.653+0000 7f9fd10a7980 2 removed watcher, disabling cache 2024-06-28T15:24:34.637 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.653+0000 7f9fd10a7980 20 remove_watcher() i=1 2024-06-28T15:24:34.639 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.657+0000 7f9fd10a7980 20 remove_watcher() i=2 2024-06-28T15:24:34.641 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.657+0000 7f9fd10a7980 20 remove_watcher() i=3 2024-06-28T15:24:34.643 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.661+0000 7f9fd10a7980 20 remove_watcher() i=4 2024-06-28T15:24:34.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.661+0000 7f9fd10a7980 20 remove_watcher() i=5 2024-06-28T15:24:34.645 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.661+0000 7f9fd10a7980 20 remove_watcher() i=6 2024-06-28T15:24:34.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.661+0000 7f9fd10a7980 20 remove_watcher() i=7 2024-06-28T15:24:34.659 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:34.660 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:24:34.660 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-commit-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.25 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:24:34.660 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:34.757 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-06-28T15:24:34.789 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:34.789 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:34.832 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.849+0000 7f1966df9980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.849+0000 7f1966df9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.849+0000 7f1966df9980 20 realm 2024-06-28T15:24:34.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.849+0000 7f1966df9980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.849+0000 7f1966df9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.849+0000 7f1966df9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:34.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.849+0000 7f1966df9980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.849+0000 7f1966df9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:34.834 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.849+0000 7f1966df9980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.836 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.853+0000 7f1966df9980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:34.836 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.853+0000 7f1966df9980 20 rados->read ofs=0 len=0 2024-06-28T15:24:34.836 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.853+0000 7f1966df9980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:34.837 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.853+0000 7f1966df9980 20 searching for the correct realm 2024-06-28T15:24:34.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:34.861+0000 7f196510b640 0 -- 172.21.15.178:0/2759140845 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f19580661a0 msgr2=0x7f1958084040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:35.061 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.077+0000 7f196490a640 0 -- 172.21.15.178:0/2759140845 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55b67fa9d050 msgr2=0x55b67fa9f480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:35.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.085+0000 7f1966df9980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:35.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.085+0000 7f1966df9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:35.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.085+0000 7f1966df9980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:35.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.085+0000 7f1966df9980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:35.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.085+0000 7f1966df9980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:35.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.085+0000 7f1966df9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:35.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.085+0000 7f1966df9980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.085+0000 7f1966df9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:35.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.085+0000 7f1966df9980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.085+0000 7f1966df9980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:35.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.085+0000 7f1966df9980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:35.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 20 zone default found 2024-06-28T15:24:35.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 4 Realm: () 2024-06-28T15:24:35.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:35.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:35.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:35.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 20 zonegroup default 2024-06-28T15:24:35.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:35.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:35.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:35.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 20 started sync module instance, tier type = 2024-06-28T15:24:35.073 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.089+0000 7f1966df9980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:35.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.109+0000 7f1966df9980 20 add_watcher() i=0 2024-06-28T15:24:35.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.109+0000 7f1966df9980 20 add_watcher() i=1 2024-06-28T15:24:35.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.109+0000 7f1966df9980 20 add_watcher() i=2 2024-06-28T15:24:35.096 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.109+0000 7f1966df9980 20 add_watcher() i=3 2024-06-28T15:24:35.096 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.109+0000 7f1966df9980 20 add_watcher() i=4 2024-06-28T15:24:35.096 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.109+0000 7f1966df9980 20 add_watcher() i=5 2024-06-28T15:24:35.096 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.109+0000 7f1966df9980 20 add_watcher() i=6 2024-06-28T15:24:35.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.113+0000 7f1966df9980 20 add_watcher() i=7 2024-06-28T15:24:35.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.113+0000 7f1966df9980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:35.101 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.117+0000 7f181afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:35.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.117+0000 7f1966df9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:35.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.117+0000 7f1966df9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:35.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.117+0000 7f1966df9980 5 note: GC not initialized 2024-06-28T15:24:35.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.117+0000 7f1819fe3640 20 reqs_thread_entry: start 2024-06-28T15:24:35.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.117+0000 7f1966df9980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:35.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.117+0000 7f1966df9980 20 Filter name: none 2024-06-28T15:24:35.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.117+0000 7f178b7fe640 20 reqs_thread_entry: start 2024-06-28T15:24:35.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.117+0000 7f1966df9980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-06-28T15:24:35.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.117+0000 7f1966df9980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.121+0000 7f1966df9980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:35.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.121+0000 7f1966df9980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-06-28T15:24:35.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.121+0000 7f1966df9980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-06-28T15:24:35.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.121+0000 7f1966df9980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 : miss 2024-06-28T15:24:35.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.121+0000 7f1966df9980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.121+0000 7f1966df9980 20 rados_obj.operate() r=0 bl.length=467 2024-06-28T15:24:35.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.121+0000 7f1966df9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 info.flags=0x17 2024-06-28T15:24:35.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.121+0000 7f1966df9980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 to cache LRU end 2024-06-28T15:24:35.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.121+0000 7f1966df9980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:24:35.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.121+0000 7f1966df9980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:35.105 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.121+0000 7f1966df9980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 2024-06-28T15:24:35.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.121+0000 7f1966df9980 10 init_target_layout removing existing target index objects from a previous reshard attempt 2024-06-28T15:24:35.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.129+0000 7f1966df9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 info.flags=0x17 2024-06-28T15:24:35.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.129+0000 7f1966df9980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 to cache LRU end 2024-06-28T15:24:35.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.129+0000 7f1966df9980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:35.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.129+0000 7f1966df9980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25, ofs0, ns] 2024-06-28T15:24:35.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.129+0000 7f183bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595110 cookie 94242314282432 notifier 6819 bl.length()=1007 2024-06-28T15:24:35.113 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.129+0000 7f183bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 info.flags=0x17 2024-06-28T15:24:35.114 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.129+0000 7f183bfff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 to cache LRU end 2024-06-28T15:24:35.114 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.129+0000 7f183bfff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:35.115 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.133+0000 7f1966df9980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.25]): orig_sources= new_sources= 2024-06-28T15:24:35.115 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.133+0000 7f1966df9980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.25]): potential sources added=[] removed=[] 2024-06-28T15:24:35.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.133+0000 7f1966df9980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.25]): orig_dests= new_dests= 2024-06-28T15:24:35.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.133+0000 7f1966df9980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.25]): potential dests added=[] removed=[] 2024-06-28T15:24:35.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.133+0000 7f1966df9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.25.2.0 2024-06-28T15:24:35.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.133+0000 7f1966df9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.25.2.0 2024-06-28T15:24:35.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.133+0000 7f1966df9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.25.2.1 2024-06-28T15:24:35.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.133+0000 7f1966df9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.25.2.1 2024-06-28T15:24:35.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.133+0000 7f1966df9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.25.0 2024-06-28T15:24:35.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.137+0000 7f1966df9980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 : hit (requested=0x17, cached=0x17) 2024-06-28T15:24:35.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.137+0000 7f1966df9980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 2024-06-28T15:24:35.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.137+0000 7f1966df9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 info.flags=0x17 2024-06-28T15:24:35.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.137+0000 7f1966df9980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 to cache LRU end 2024-06-28T15:24:35.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.137+0000 7f1966df9980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:35.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.137+0000 7f1966df9980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25, ofs0, ns] 2024-06-28T15:24:35.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.141+0000 7f183b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595111 cookie 94242314282432 notifier 6819 bl.length()=975 2024-06-28T15:24:35.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.141+0000 7f183b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 info.flags=0x17 2024-06-28T15:24:35.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.141+0000 7f183b7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.25 to cache LRU end 2024-06-28T15:24:35.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.141+0000 7f183b7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:35.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.141+0000 7f1966df9980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.25]): orig_sources= new_sources= 2024-06-28T15:24:35.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.141+0000 7f1966df9980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.25]): potential sources added=[] removed=[] 2024-06-28T15:24:35.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.141+0000 7f1966df9980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.25]): orig_dests= new_dests= 2024-06-28T15:24:35.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.141+0000 7f1966df9980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.25]): potential dests added=[] removed=[] 2024-06-28T15:24:35.127 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.145+0000 7f1966df9980 1 execute INFO: reshard of bucket "error-at-commit-target-layout" from 2 shards to 2 shards completed successfully 2024-06-28T15:24:35.131 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.145+0000 7f181afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:35.135 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.153+0000 7f1966df9980 20 remove_watcher() i=0 2024-06-28T15:24:35.135 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.153+0000 7f1966df9980 2 removed watcher, disabling cache 2024-06-28T15:24:35.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.153+0000 7f1966df9980 20 remove_watcher() i=1 2024-06-28T15:24:35.140 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.157+0000 7f1966df9980 20 remove_watcher() i=2 2024-06-28T15:24:35.141 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.157+0000 7f1966df9980 20 remove_watcher() i=3 2024-06-28T15:24:35.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.161+0000 7f1966df9980 20 remove_watcher() i=4 2024-06-28T15:24:35.145 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.161+0000 7f1966df9980 20 remove_watcher() i=5 2024-06-28T15:24:35.146 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.161+0000 7f1966df9980 20 remove_watcher() i=6 2024-06-28T15:24:35.148 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.165+0000 7f1966df9980 20 remove_watcher() i=7 2024-06-28T15:24:35.165 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-06-28T15:24:35.194 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:35.194 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:35.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.253+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.257+0000 7f3942132980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:35.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.257+0000 7f3942132980 20 realm 2024-06-28T15:24:35.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.257+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.257+0000 7f3942132980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:35.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.257+0000 7f3942132980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:35.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.257+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.257+0000 7f3942132980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:35.241 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.257+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.257+0000 7f3942132980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:35.242 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.257+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.261+0000 7f3942132980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:35.243 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.261+0000 7f3942132980 20 searching for the correct realm 2024-06-28T15:24:35.249 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.265+0000 7f393f442640 0 -- 172.21.15.178:0/383927782 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f393008e140 msgr2=0x7f39300872f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:35.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.273+0000 7f393f442640 0 -- 172.21.15.178:0/383927782 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f380807fda0 msgr2=0x7f38080821d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:35.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.277+0000 7f3942132980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:35.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.277+0000 7f3942132980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:35.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.277+0000 7f3942132980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:35.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.277+0000 7f3942132980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:35.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.277+0000 7f3942132980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:35.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.277+0000 7f3942132980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:35.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.277+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.277+0000 7f3942132980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:35.263 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.277+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:35.264 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:35.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 20 zone default found 2024-06-28T15:24:35.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 4 Realm: () 2024-06-28T15:24:35.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:35.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:35.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:35.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 20 zonegroup default 2024-06-28T15:24:35.265 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:35.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:35.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:35.266 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 20 started sync module instance, tier type = 2024-06-28T15:24:35.267 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.281+0000 7f3942132980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:35.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.297+0000 7f3942132980 20 add_watcher() i=0 2024-06-28T15:24:35.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.297+0000 7f3942132980 20 add_watcher() i=1 2024-06-28T15:24:35.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.297+0000 7f3942132980 20 add_watcher() i=2 2024-06-28T15:24:35.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.297+0000 7f3942132980 20 add_watcher() i=3 2024-06-28T15:24:35.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.297+0000 7f3942132980 20 add_watcher() i=4 2024-06-28T15:24:35.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.297+0000 7f3942132980 20 add_watcher() i=5 2024-06-28T15:24:35.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.297+0000 7f3942132980 20 add_watcher() i=6 2024-06-28T15:24:35.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.301+0000 7f3942132980 20 add_watcher() i=7 2024-06-28T15:24:35.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.301+0000 7f3942132980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:35.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.301+0000 7f37f5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:35.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.301+0000 7f3942132980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:35.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.301+0000 7f3942132980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:35.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.301+0000 7f3942132980 5 note: GC not initialized 2024-06-28T15:24:35.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.301+0000 7f37f4fe1640 20 reqs_thread_entry: start 2024-06-28T15:24:35.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.301+0000 7f3942132980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:35.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.301+0000 7f3942132980 20 Filter name: none 2024-06-28T15:24:35.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.301+0000 7f376e7fc640 20 reqs_thread_entry: start 2024-06-28T15:24:35.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.301+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.305+0000 7f3942132980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:35.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.305+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.305+0000 7f3942132980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:35.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.305+0000 7f3942132980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:35.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.305+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.289 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.305+0000 7f3942132980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:35.289 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.305+0000 7f3942132980 20 rados->read ofs=0 len=0 2024-06-28T15:24:35.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.305+0000 7f3942132980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:35.290 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.305+0000 7f3942132980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:35.291 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.309+0000 7f37f5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:35.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.313+0000 7f3942132980 20 remove_watcher() i=0 2024-06-28T15:24:35.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.313+0000 7f3942132980 2 removed watcher, disabling cache 2024-06-28T15:24:35.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.317+0000 7f3942132980 20 remove_watcher() i=1 2024-06-28T15:24:35.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.321+0000 7f3942132980 20 remove_watcher() i=2 2024-06-28T15:24:35.305 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.321+0000 7f3942132980 20 remove_watcher() i=3 2024-06-28T15:24:35.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.325+0000 7f3942132980 20 remove_watcher() i=4 2024-06-28T15:24:35.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.325+0000 7f3942132980 20 remove_watcher() i=5 2024-06-28T15:24:35.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.325+0000 7f3942132980 20 remove_watcher() i=6 2024-06-28T15:24:35.311 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:35.329+0000 7f3942132980 20 remove_watcher() i=7 2024-06-28T15:24:35.325 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:35.325 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:24:35.325 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-commit-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.25 num_objects: 19 size_kb: 1 num_shards: 2 2024-06-28T15:24:35.325 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:36.012 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: reshard bucket with ECANCELED injected at commit_target_layout 2024-06-28T15:24:36.013 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:38.158 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-06-28T15:24:38.191 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:38.191 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:38.235 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.253+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.237 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.253+0000 7f0ea8443980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.237 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.253+0000 7f0ea8443980 20 realm 2024-06-28T15:24:38.237 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.253+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.253+0000 7f0ea8443980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.253+0000 7f0ea8443980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:38.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.253+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.253+0000 7f0ea8443980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.253+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.257+0000 7f0ea8443980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:38.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.257+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.257+0000 7f0ea8443980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:38.240 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.257+0000 7f0ea8443980 20 searching for the correct realm 2024-06-28T15:24:38.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.261+0000 7f0ea5753640 0 -- 172.21.15.178:0/297779491 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f0e980c0020 msgr2=0x7f0e981108d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:38.252 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.269+0000 7f0ea5753640 0 -- 172.21.15.178:0/297779491 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f0d7007fd20 msgr2=0x7f0d70082150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:38.258 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.273+0000 7f0ea8443980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:38.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.273+0000 7f0ea8443980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:38.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.273+0000 7f0ea8443980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:38.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.273+0000 7f0ea8443980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:38.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.273+0000 7f0ea8443980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:38.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.273+0000 7f0ea8443980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:38.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.273+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.273+0000 7f0ea8443980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.259 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.273+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:38.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:38.260 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 20 zone default found 2024-06-28T15:24:38.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 4 Realm: () 2024-06-28T15:24:38.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:38.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:38.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:38.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 20 zonegroup default 2024-06-28T15:24:38.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.261 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 20 started sync module instance, tier type = 2024-06-28T15:24:38.262 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.277+0000 7f0ea8443980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:38.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.293+0000 7f0ea8443980 20 add_watcher() i=0 2024-06-28T15:24:38.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.293+0000 7f0ea8443980 20 add_watcher() i=1 2024-06-28T15:24:38.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.293+0000 7f0ea8443980 20 add_watcher() i=2 2024-06-28T15:24:38.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.293+0000 7f0ea8443980 20 add_watcher() i=3 2024-06-28T15:24:38.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.293+0000 7f0ea8443980 20 add_watcher() i=4 2024-06-28T15:24:38.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.293+0000 7f0ea8443980 20 add_watcher() i=5 2024-06-28T15:24:38.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.293+0000 7f0ea8443980 20 add_watcher() i=6 2024-06-28T15:24:38.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.293+0000 7f0ea8443980 20 add_watcher() i=7 2024-06-28T15:24:38.278 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.293+0000 7f0ea8443980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:38.280 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.297+0000 7f0d577fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:38.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.297+0000 7f0ea8443980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:38.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.297+0000 7f0ea8443980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:38.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.297+0000 7f0ea8443980 5 note: GC not initialized 2024-06-28T15:24:38.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.297+0000 7f0d567fc640 20 reqs_thread_entry: start 2024-06-28T15:24:38.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.297+0000 7f0ea8443980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:38.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.297+0000 7f0ea8443980 20 Filter name: none 2024-06-28T15:24:38.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.297+0000 7f0cc7fff640 20 reqs_thread_entry: start 2024-06-28T15:24:38.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.297+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.297+0000 7f0ea8443980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:38.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.297+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.301+0000 7f0ea8443980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:38.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.301+0000 7f0ea8443980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:38.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.301+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.301+0000 7f0ea8443980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:38.285 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.301+0000 7f0ea8443980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.301+0000 7f0ea8443980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:38.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.301+0000 7f0ea8443980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:38.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.305+0000 7f0d577fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:38.292 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.309+0000 7f0ea8443980 20 remove_watcher() i=0 2024-06-28T15:24:38.292 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.309+0000 7f0ea8443980 2 removed watcher, disabling cache 2024-06-28T15:24:38.294 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.309+0000 7f0ea8443980 20 remove_watcher() i=1 2024-06-28T15:24:38.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.313+0000 7f0ea8443980 20 remove_watcher() i=2 2024-06-28T15:24:38.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.313+0000 7f0ea8443980 20 remove_watcher() i=3 2024-06-28T15:24:38.300 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.317+0000 7f0ea8443980 20 remove_watcher() i=4 2024-06-28T15:24:38.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.317+0000 7f0ea8443980 20 remove_watcher() i=5 2024-06-28T15:24:38.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.317+0000 7f0ea8443980 20 remove_watcher() i=6 2024-06-28T15:24:38.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.321+0000 7f0ea8443980 20 remove_watcher() i=7 2024-06-28T15:24:38.317 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:38.317 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:24:38.317 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-commit-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.26 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:24:38.317 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:38.318 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout --inject-error-code 125 2024-06-28T15:24:38.351 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:38.351 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:38.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.409+0000 7faca0d70980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.413+0000 7faca0d70980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.413+0000 7faca0d70980 20 realm 2024-06-28T15:24:38.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.413+0000 7faca0d70980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.413+0000 7faca0d70980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.413+0000 7faca0d70980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:38.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.413+0000 7faca0d70980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.413+0000 7faca0d70980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.396 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.413+0000 7faca0d70980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.413+0000 7faca0d70980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:38.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.413+0000 7faca0d70980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.413+0000 7faca0d70980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:38.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.413+0000 7faca0d70980 20 searching for the correct realm 2024-06-28T15:24:38.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.421+0000 7fac9f082640 0 -- 172.21.15.178:0/361071365 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x55f748b428e0 msgr2=0x55f748b44d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:38.409 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.425+0000 7fac9e881640 0 -- 172.21.15.178:0/361071365 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fac9411c7a0 msgr2=0x7fac9411a040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:38.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:38.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:38.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:38.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:38.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:38.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:38.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:38.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:38.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 zone default found 2024-06-28T15:24:38.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 4 Realm: () 2024-06-28T15:24:38.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:38.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:38.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:38.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 zonegroup default 2024-06-28T15:24:38.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.433+0000 7faca0d70980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.437+0000 7faca0d70980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.437+0000 7faca0d70980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.437+0000 7faca0d70980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.437+0000 7faca0d70980 20 started sync module instance, tier type = 2024-06-28T15:24:38.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.437+0000 7faca0d70980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:38.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.449+0000 7faca0d70980 20 add_watcher() i=0 2024-06-28T15:24:38.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.449+0000 7faca0d70980 20 add_watcher() i=1 2024-06-28T15:24:38.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.449+0000 7faca0d70980 20 add_watcher() i=2 2024-06-28T15:24:38.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.449+0000 7faca0d70980 20 add_watcher() i=3 2024-06-28T15:24:38.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.449+0000 7faca0d70980 20 add_watcher() i=4 2024-06-28T15:24:38.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.449+0000 7faca0d70980 20 add_watcher() i=5 2024-06-28T15:24:38.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.449+0000 7faca0d70980 20 add_watcher() i=6 2024-06-28T15:24:38.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.453+0000 7faca0d70980 20 add_watcher() i=7 2024-06-28T15:24:38.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.453+0000 7faca0d70980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:38.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.453+0000 7fab557e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:38.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.453+0000 7faca0d70980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:38.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.453+0000 7faca0d70980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:38.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 5 note: GC not initialized 2024-06-28T15:24:38.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faacffff640 20 reqs_thread_entry: start 2024-06-28T15:24:38.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:38.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 20 Filter name: none 2024-06-28T15:24:38.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faacdffb640 20 reqs_thread_entry: start 2024-06-28T15:24:38.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-06-28T15:24:38.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:38.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-06-28T15:24:38.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-06-28T15:24:38.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26 : miss 2024-06-28T15:24:38.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:38.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26 info.flags=0x17 2024-06-28T15:24:38.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26 to cache LRU end 2024-06-28T15:24:38.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:24:38.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:38.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.457+0000 7faca0d70980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26 2024-06-28T15:24:38.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.465+0000 7faca0d70980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26 info.flags=0x17 2024-06-28T15:24:38.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.465+0000 7faca0d70980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26 to cache LRU end 2024-06-28T15:24:38.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.465+0000 7faca0d70980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:38.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.465+0000 7faca0d70980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26, ofs0, ns] 2024-06-28T15:24:38.451 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.465+0000 7fab827fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595114 cookie 94520565103376 notifier 6873 bl.length()=1007 2024-06-28T15:24:38.451 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.465+0000 7fab827fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26 info.flags=0x17 2024-06-28T15:24:38.451 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.465+0000 7fab827fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26 to cache LRU end 2024-06-28T15:24:38.451 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.465+0000 7fab827fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:38.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.469+0000 7faca0d70980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.26]): orig_sources= new_sources= 2024-06-28T15:24:38.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.469+0000 7faca0d70980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.26]): potential sources added=[] removed=[] 2024-06-28T15:24:38.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.469+0000 7faca0d70980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.26]): orig_dests= new_dests= 2024-06-28T15:24:38.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.469+0000 7faca0d70980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.26]): potential dests added=[] removed=[] 2024-06-28T15:24:38.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.469+0000 7faca0d70980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.26.1.0 2024-06-28T15:24:38.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.469+0000 7faca0d70980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.26.1.0 2024-06-28T15:24:38.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.469+0000 7faca0d70980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.26.1.1 2024-06-28T15:24:38.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.469+0000 7faca0d70980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.26.1.1 2024-06-28T15:24:38.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.469+0000 7faca0d70980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.26.0 2024-06-28T15:24:38.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.473+0000 7faca0d70980 -1 Injecting error=-125 at location=commit_target_layout 2024-06-28T15:24:38.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.473+0000 7faca0d70980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26 : hit (requested=0x17, cached=0x17) 2024-06-28T15:24:38.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.473+0000 7faca0d70980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26 2024-06-28T15:24:38.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.477+0000 7faca0d70980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26 info.flags=0x17 2024-06-28T15:24:38.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.477+0000 7faca0d70980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26 to cache LRU end 2024-06-28T15:24:38.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.477+0000 7faca0d70980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:38.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.477+0000 7faca0d70980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26, ofs0, ns] 2024-06-28T15:24:38.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.477+0000 7fab81ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595115 cookie 94520565103376 notifier 6873 bl.length()=975 2024-06-28T15:24:38.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.477+0000 7fab81ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26 info.flags=0x17 2024-06-28T15:24:38.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.477+0000 7fab81ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.26 to cache LRU end 2024-06-28T15:24:38.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.477+0000 7fab81ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:38.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.481+0000 7faca0d70980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.26]): orig_sources= new_sources= 2024-06-28T15:24:38.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.481+0000 7faca0d70980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.26]): potential sources added=[] removed=[] 2024-06-28T15:24:38.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.481+0000 7faca0d70980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.26]): orig_dests= new_dests= 2024-06-28T15:24:38.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.481+0000 7faca0d70980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.26]): potential dests added=[] removed=[] 2024-06-28T15:24:38.466 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.481+0000 7faca0d70980 1 execute INFO: reshard of bucket "error-at-commit-target-layout" from 2 shards to 2 shards completed successfully 2024-06-28T15:24:38.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.485+0000 7fab557e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:38.473 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.489+0000 7faca0d70980 20 remove_watcher() i=0 2024-06-28T15:24:38.474 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.489+0000 7faca0d70980 2 removed watcher, disabling cache 2024-06-28T15:24:38.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.493+0000 7faca0d70980 20 remove_watcher() i=1 2024-06-28T15:24:38.478 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.493+0000 7faca0d70980 20 remove_watcher() i=2 2024-06-28T15:24:38.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.497+0000 7faca0d70980 20 remove_watcher() i=3 2024-06-28T15:24:38.482 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.497+0000 7faca0d70980 20 remove_watcher() i=4 2024-06-28T15:24:38.484 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.501+0000 7faca0d70980 20 remove_watcher() i=5 2024-06-28T15:24:38.485 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.501+0000 7faca0d70980 20 remove_watcher() i=6 2024-06-28T15:24:38.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.501+0000 7faca0d70980 20 remove_watcher() i=7 2024-06-28T15:24:38.499 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-06-28T15:24:38.527 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:38.527 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:38.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.585+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.589+0000 7fddfad35980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.589+0000 7fddfad35980 20 realm 2024-06-28T15:24:38.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.589+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.589+0000 7fddfad35980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.589+0000 7fddfad35980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:38.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.589+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.589+0000 7fddfad35980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.589+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.589+0000 7fddfad35980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:38.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.589+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.589+0000 7fddfad35980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:38.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.589+0000 7fddfad35980 20 searching for the correct realm 2024-06-28T15:24:38.580 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.597+0000 7fddf8045640 0 -- 172.21.15.178:0/2652562579 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x561a0464bea0 msgr2=0x561a0464e290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:38.586 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.601+0000 7fddf8846640 0 -- 172.21.15.178:0/2652562579 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fddec055040 msgr2=0x7fddec054440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:38.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.609+0000 7fddfad35980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:38.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.609+0000 7fddfad35980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:38.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.609+0000 7fddfad35980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:38.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.609+0000 7fddfad35980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:38.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.609+0000 7fddfad35980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:38.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.609+0000 7fddfad35980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:38.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.609+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.594 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.609+0000 7fddfad35980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.595 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.609+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.609+0000 7fddfad35980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:38.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.609+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:38.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 20 zone default found 2024-06-28T15:24:38.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 4 Realm: () 2024-06-28T15:24:38.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:38.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:38.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:38.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 20 zonegroup default 2024-06-28T15:24:38.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.596 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:38.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 20 started sync module instance, tier type = 2024-06-28T15:24:38.597 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.613+0000 7fddfad35980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:38.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.629+0000 7fddfad35980 20 add_watcher() i=0 2024-06-28T15:24:38.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.629+0000 7fddfad35980 20 add_watcher() i=1 2024-06-28T15:24:38.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.629+0000 7fddfad35980 20 add_watcher() i=2 2024-06-28T15:24:38.612 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.629+0000 7fddfad35980 20 add_watcher() i=3 2024-06-28T15:24:38.613 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.629+0000 7fddfad35980 20 add_watcher() i=4 2024-06-28T15:24:38.613 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.629+0000 7fddfad35980 20 add_watcher() i=5 2024-06-28T15:24:38.613 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.629+0000 7fddfad35980 20 add_watcher() i=6 2024-06-28T15:24:38.613 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.629+0000 7fddfad35980 20 add_watcher() i=7 2024-06-28T15:24:38.613 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.629+0000 7fddfad35980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:38.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.633+0000 7fdcaefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:38.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.633+0000 7fddfad35980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:38.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.633+0000 7fddfad35980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:38.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.633+0000 7fddfad35980 5 note: GC not initialized 2024-06-28T15:24:38.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.633+0000 7fdcadfe3640 20 reqs_thread_entry: start 2024-06-28T15:24:38.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.633+0000 7fddfad35980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:38.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.633+0000 7fddfad35980 20 Filter name: none 2024-06-28T15:24:38.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.633+0000 7fdc1f7fe640 20 reqs_thread_entry: start 2024-06-28T15:24:38.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.633+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.633+0000 7fddfad35980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:38.618 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.633+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.633+0000 7fddfad35980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:38.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.633+0000 7fddfad35980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:38.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.633+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.637+0000 7fddfad35980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:38.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.637+0000 7fddfad35980 20 rados->read ofs=0 len=0 2024-06-28T15:24:38.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.637+0000 7fddfad35980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:38.620 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.637+0000 7fddfad35980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:38.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.637+0000 7fdcaefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:38.627 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.641+0000 7fddfad35980 20 remove_watcher() i=0 2024-06-28T15:24:38.627 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.641+0000 7fddfad35980 2 removed watcher, disabling cache 2024-06-28T15:24:38.628 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.645+0000 7fddfad35980 20 remove_watcher() i=1 2024-06-28T15:24:38.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.645+0000 7fddfad35980 20 remove_watcher() i=2 2024-06-28T15:24:38.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.649+0000 7fddfad35980 20 remove_watcher() i=3 2024-06-28T15:24:38.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.649+0000 7fddfad35980 20 remove_watcher() i=4 2024-06-28T15:24:38.636 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.653+0000 7fddfad35980 20 remove_watcher() i=5 2024-06-28T15:24:38.637 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.653+0000 7fddfad35980 20 remove_watcher() i=6 2024-06-28T15:24:38.638 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:38.653+0000 7fddfad35980 20 remove_watcher() i=7 2024-06-28T15:24:38.651 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:38.651 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:24:38.651 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-commit-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.26 num_objects: 20 size_kb: 1 num_shards: 2 2024-06-28T15:24:38.651 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:39.455 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: reshard bucket with abort at commit_target_layout 2024-06-28T15:24:39.455 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:41.661 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-06-28T15:24:41.693 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:41.693 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:41.739 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.753+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:41.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.757+0000 7f1c534c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:41.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.757+0000 7f1c534c1980 20 realm 2024-06-28T15:24:41.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.757+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:41.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.757+0000 7f1c534c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:41.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.757+0000 7f1c534c1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:41.741 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.757+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:41.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.757+0000 7f1c534c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:41.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.757+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:41.743 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.757+0000 7f1c534c1980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:41.743 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.757+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:41.743 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.761+0000 7f1c534c1980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:41.743 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.761+0000 7f1c534c1980 20 searching for the correct realm 2024-06-28T15:24:41.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.761+0000 7f1c507d1640 0 -- 172.21.15.178:0/533304776 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f1c400e8020 msgr2=0x7f1c400e1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:24:41.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:41.977+0000 7f1c50fd2640 0 -- 172.21.15.178:0/533304776 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f1c44087220 msgr2=0x7f1c44087760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:24:42.178 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.193+0000 7f1c534c1980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:42.178 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.193+0000 7f1c534c1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:42.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.193+0000 7f1c534c1980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:42.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.193+0000 7f1c534c1980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:42.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.193+0000 7f1c534c1980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:42.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.193+0000 7f1c534c1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:42.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.193+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.193+0000 7f1c534c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.179 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.193+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.180 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.197+0000 7f1c534c1980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:42.180 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.197+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.197+0000 7f1c534c1980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:42.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.197+0000 7f1c534c1980 20 zone default found 2024-06-28T15:24:42.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.197+0000 7f1c534c1980 4 Realm: () 2024-06-28T15:24:42.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.197+0000 7f1c534c1980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:42.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.197+0000 7f1c534c1980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:42.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.197+0000 7f1c534c1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:42.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.197+0000 7f1c534c1980 20 zonegroup default 2024-06-28T15:24:42.181 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.197+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.197+0000 7f1c534c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.197+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.197+0000 7f1c534c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.182 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.197+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.183 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.201+0000 7f1c534c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.183 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.201+0000 7f1c534c1980 20 started sync module instance, tier type = 2024-06-28T15:24:42.183 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.201+0000 7f1c534c1980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:42.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.221+0000 7f1c534c1980 20 add_watcher() i=0 2024-06-28T15:24:42.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.221+0000 7f1c534c1980 20 add_watcher() i=1 2024-06-28T15:24:42.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.221+0000 7f1c534c1980 20 add_watcher() i=2 2024-06-28T15:24:42.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.221+0000 7f1c534c1980 20 add_watcher() i=3 2024-06-28T15:24:42.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.221+0000 7f1c534c1980 20 add_watcher() i=4 2024-06-28T15:24:42.207 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.221+0000 7f1c534c1980 20 add_watcher() i=5 2024-06-28T15:24:42.207 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.221+0000 7f1c534c1980 20 add_watcher() i=6 2024-06-28T15:24:42.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.225+0000 7f1c534c1980 20 add_watcher() i=7 2024-06-28T15:24:42.209 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.225+0000 7f1c534c1980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:42.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.229+0000 7f1b06fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:42.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.229+0000 7f1c534c1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:42.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.229+0000 7f1c534c1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:42.214 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.229+0000 7f1c534c1980 5 note: GC not initialized 2024-06-28T15:24:42.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.229+0000 7f1b05fe3640 20 reqs_thread_entry: start 2024-06-28T15:24:42.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.229+0000 7f1c534c1980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:42.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.229+0000 7f1c534c1980 20 Filter name: none 2024-06-28T15:24:42.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.229+0000 7f1a777fe640 20 reqs_thread_entry: start 2024-06-28T15:24:42.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.229+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.233+0000 7f1c534c1980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:42.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.233+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.233+0000 7f1c534c1980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:42.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.233+0000 7f1c534c1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:42.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.233+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.233+0000 7f1c534c1980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:42.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.233+0000 7f1c534c1980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.233+0000 7f1c534c1980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:42.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.233+0000 7f1c534c1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:42.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.237+0000 7f1b06fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:42.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.241+0000 7f1c534c1980 20 remove_watcher() i=0 2024-06-28T15:24:42.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.241+0000 7f1c534c1980 2 removed watcher, disabling cache 2024-06-28T15:24:42.227 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.245+0000 7f1c534c1980 20 remove_watcher() i=1 2024-06-28T15:24:42.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.245+0000 7f1c534c1980 20 remove_watcher() i=2 2024-06-28T15:24:42.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.249+0000 7f1c534c1980 20 remove_watcher() i=3 2024-06-28T15:24:42.234 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.249+0000 7f1c534c1980 20 remove_watcher() i=4 2024-06-28T15:24:42.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.253+0000 7f1c534c1980 20 remove_watcher() i=5 2024-06-28T15:24:42.238 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.253+0000 7f1c534c1980 20 remove_watcher() i=6 2024-06-28T15:24:42.239 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.253+0000 7f1c534c1980 20 remove_watcher() i=7 2024-06-28T15:24:42.254 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.255 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:24:42.255 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: abort-at-commit-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.27 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:24:42.255 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.255 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at commit_target_layout 2024-06-28T15:24:42.292 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:42.292 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:42.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.349+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 realm 2024-06-28T15:24:42.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.353+0000 7fcea6bc5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:42.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.338 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.357+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:42.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.357+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.357+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:42.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.357+0000 7fcea6bc5980 20 searching for the correct realm 2024-06-28T15:24:42.346 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.361+0000 7fcea4ed7640 0 -- 172.21.15.178:0/2581271606 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fce8c04a960 msgr2=0x7fce8c029410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:42.561 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.577+0000 7fcea46d6640 0 -- 172.21.15.178:0/2581271606 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x56328a17c0d0 msgr2=0x56328a17e4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:42.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:42.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:42.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:42.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:42.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:42.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:42.569 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.570 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:42.571 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:42.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 zone default found 2024-06-28T15:24:42.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 4 Realm: () 2024-06-28T15:24:42.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:42.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:42.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:42.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 zonegroup default 2024-06-28T15:24:42.572 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.573 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 started sync module instance, tier type = 2024-06-28T15:24:42.574 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:42.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=0 2024-06-28T15:24:42.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=1 2024-06-28T15:24:42.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=2 2024-06-28T15:24:42.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=3 2024-06-28T15:24:42.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=4 2024-06-28T15:24:42.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=5 2024-06-28T15:24:42.598 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=6 2024-06-28T15:24:42.600 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.617+0000 7fcea6bc5980 20 add_watcher() i=7 2024-06-28T15:24:42.601 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.617+0000 7fcea6bc5980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:42.604 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.621+0000 7fcd5afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:42.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.621+0000 7fcea6bc5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:42.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.621+0000 7fcea6bc5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:42.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 note: GC not initialized 2024-06-28T15:24:42.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.621+0000 7fcd59fdb640 20 reqs_thread_entry: start 2024-06-28T15:24:42.606 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.621+0000 7fcea6bc5980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:42.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.621+0000 7fcea6bc5980 20 Filter name: none 2024-06-28T15:24:42.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.625+0000 7fcccb7fe640 20 reqs_thread_entry: start 2024-06-28T15:24:42.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-06-28T15:24:42.607 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.625+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.625+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:42.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-06-28T15:24:42.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-06-28T15:24:42.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 : miss 2024-06-28T15:24:42.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.625+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.625+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:42.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 info.flags=0x17 2024-06-28T15:24:42.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 to cache LRU end 2024-06-28T15:24:42.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:24:42.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:42.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 2024-06-28T15:24:42.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.633+0000 7fcea6bc5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 info.flags=0x17 2024-06-28T15:24:42.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.633+0000 7fcea6bc5980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 to cache LRU end 2024-06-28T15:24:42.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.633+0000 7fcea6bc5980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:42.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.633+0000 7fcea6bc5980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27, ofs0, ns] 2024-06-28T15:24:42.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.633+0000 7fcd7b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595117 cookie 94775065204384 notifier 6927 bl.length()=1007 2024-06-28T15:24:42.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.633+0000 7fcd7b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 info.flags=0x17 2024-06-28T15:24:42.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.633+0000 7fcd7b7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 to cache LRU end 2024-06-28T15:24:42.617 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.633+0000 7fcd7b7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:42.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): orig_sources= new_sources= 2024-06-28T15:24:42.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): potential sources added=[] removed=[] 2024-06-28T15:24:42.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): orig_dests= new_dests= 2024-06-28T15:24:42.619 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): potential dests added=[] removed=[] 2024-06-28T15:24:42.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.637+0000 7fcea6bc5980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.0 2024-06-28T15:24:42.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.0 2024-06-28T15:24:42.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.637+0000 7fcea6bc5980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.1 2024-06-28T15:24:42.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.1 2024-06-28T15:24:42.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.0 2024-06-28T15:24:42.627 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fcea6bc5980 time 2024-06-28T15:24:42.649045+0000 2024-06-28T15:24:42.627 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:24:42.629 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:24:42.630 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fceaa49cc91] 2024-06-28T15:24:42.630 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fceaa49ce45] 2024-06-28T15:24:42.630 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x563287a014b1] 2024-06-28T15:24:42.630 INFO:tasks.workunit.client.0.smithi178.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x563287a12cf7] 2024-06-28T15:24:42.631 INFO:tasks.workunit.client.0.smithi178.stderr: 5: main() 2024-06-28T15:24:42.631 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fcea9305d90] 2024-06-28T15:24:42.631 INFO:tasks.workunit.client.0.smithi178.stderr: 7: __libc_start_main() 2024-06-28T15:24:42.631 INFO:tasks.workunit.client.0.smithi178.stderr: 8: _start() 2024-06-28T15:24:42.631 INFO:tasks.workunit.client.0.smithi178.stderr:*** Caught signal (Aborted) ** 2024-06-28T15:24:42.631 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7fcea6bc5980 thread_name:radosgw-admin 2024-06-28T15:24:42.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.645+0000 7fcea6bc5980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fcea6bc5980 time 2024-06-28T15:24:42.649045+0000 2024-06-28T15:24:42.631 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:24:42.631 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.631 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:24:42.631 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fceaa49cc91] 2024-06-28T15:24:42.631 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fceaa49ce45] 2024-06-28T15:24:42.632 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x563287a014b1] 2024-06-28T15:24:42.632 INFO:tasks.workunit.client.0.smithi178.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x563287a12cf7] 2024-06-28T15:24:42.632 INFO:tasks.workunit.client.0.smithi178.stderr: 5: main() 2024-06-28T15:24:42.632 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fcea9305d90] 2024-06-28T15:24:42.632 INFO:tasks.workunit.client.0.smithi178.stderr: 7: __libc_start_main() 2024-06-28T15:24:42.632 INFO:tasks.workunit.client.0.smithi178.stderr: 8: _start() 2024-06-28T15:24:42.632 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.633 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:24:42.633 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fcea931e520] 2024-06-28T15:24:42.633 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:24:42.633 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:24:42.633 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:24:42.633 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fceaa49cceb] 2024-06-28T15:24:42.633 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fceaa49ce45] 2024-06-28T15:24:42.633 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x563287a014b1] 2024-06-28T15:24:42.633 INFO:tasks.workunit.client.0.smithi178.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x563287a12cf7] 2024-06-28T15:24:42.633 INFO:tasks.workunit.client.0.smithi178.stderr: 9: main() 2024-06-28T15:24:42.633 INFO:tasks.workunit.client.0.smithi178.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fcea9305d90] 2024-06-28T15:24:42.634 INFO:tasks.workunit.client.0.smithi178.stderr: 11: __libc_start_main() 2024-06-28T15:24:42.634 INFO:tasks.workunit.client.0.smithi178.stderr: 12: _start() 2024-06-28T15:24:42.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.649+0000 7fcea6bc5980 -1 *** Caught signal (Aborted) ** 2024-06-28T15:24:42.634 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7fcea6bc5980 thread_name:radosgw-admin 2024-06-28T15:24:42.634 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.634 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:24:42.634 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fcea931e520] 2024-06-28T15:24:42.634 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:24:42.634 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:24:42.634 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:24:42.634 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fceaa49cceb] 2024-06-28T15:24:42.634 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fceaa49ce45] 2024-06-28T15:24:42.634 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x563287a014b1] 2024-06-28T15:24:42.635 INFO:tasks.workunit.client.0.smithi178.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x563287a12cf7] 2024-06-28T15:24:42.635 INFO:tasks.workunit.client.0.smithi178.stderr: 9: main() 2024-06-28T15:24:42.635 INFO:tasks.workunit.client.0.smithi178.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fcea9305d90] 2024-06-28T15:24:42.635 INFO:tasks.workunit.client.0.smithi178.stderr: 11: __libc_start_main() 2024-06-28T15:24:42.635 INFO:tasks.workunit.client.0.smithi178.stderr: 12: _start() 2024-06-28T15:24:42.635 INFO:tasks.workunit.client.0.smithi178.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-28T15:24:42.635 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.635 INFO:tasks.workunit.client.0.smithi178.stderr:--- begin dump of recent events --- 2024-06-28T15:24:42.682 INFO:tasks.workunit.client.0.smithi178.stderr: -499> 2024-06-28T15:24:42.329+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.682 INFO:tasks.workunit.client.0.smithi178.stderr: -498> 2024-06-28T15:24:42.329+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.682 INFO:tasks.workunit.client.0.smithi178.stderr: -497> 2024-06-28T15:24:42.329+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.682 INFO:tasks.workunit.client.0.smithi178.stderr: -496> 2024-06-28T15:24:42.329+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_done global_id 6491 payload 274 2024-06-28T15:24:42.682 INFO:tasks.workunit.client.0.smithi178.stderr: -495> 2024-06-28T15:24:42.329+0000 7fcea4ed7640 10 monclient: _finish_hunting 0 2024-06-28T15:24:42.682 INFO:tasks.workunit.client.0.smithi178.stderr: -494> 2024-06-28T15:24:42.329+0000 7fcea4ed7640 1 monclient: found mon.noname-c 2024-06-28T15:24:42.682 INFO:tasks.workunit.client.0.smithi178.stderr: -493> 2024-06-28T15:24:42.329+0000 7fcea4ed7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:24:42.682 INFO:tasks.workunit.client.0.smithi178.stderr: -492> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.683 INFO:tasks.workunit.client.0.smithi178.stderr: -491> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:24:42.683 INFO:tasks.workunit.client.0.smithi178.stderr: -490> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: dump: 2024-06-28T15:24:42.683 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.683 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.683 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.683 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.683 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.683 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.683 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.683 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.683 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.683 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.683 INFO:tasks.workunit.client.0.smithi178.stderr: -489> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: _finish_auth 0 2024-06-28T15:24:42.684 INFO:tasks.workunit.client.0.smithi178.stderr: -488> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: _check_auth_tickets 2024-06-28T15:24:42.684 INFO:tasks.workunit.client.0.smithi178.stderr: -487> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:42.684 INFO:tasks.workunit.client.0.smithi178.stderr: -486> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.684 INFO:tasks.workunit.client.0.smithi178.stderr: -485> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:24:42.684 INFO:tasks.workunit.client.0.smithi178.stderr: -484> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: dump: 2024-06-28T15:24:42.684 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.684 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.684 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.684 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.684 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.684 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.684 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.684 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.685 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.685 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.685 INFO:tasks.workunit.client.0.smithi178.stderr: -483> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 monclient: authenticate success, global_id 6491 2024-06-28T15:24:42.685 INFO:tasks.workunit.client.0.smithi178.stderr: -482> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 10 monclient: get_monmap_and_config success 2024-06-28T15:24:42.685 INFO:tasks.workunit.client.0.smithi178.stderr: -481> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 4 set_mon_vals no callback set 2024-06-28T15:24:42.685 INFO:tasks.workunit.client.0.smithi178.stderr: -480> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 10 monclient: shutdown 2024-06-28T15:24:42.685 INFO:tasks.workunit.client.0.smithi178.stderr: -479> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 asok(0x563289e825e0) unregister_commands rotate-key 2024-06-28T15:24:42.685 INFO:tasks.workunit.client.0.smithi178.stderr: -478> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 10 monclient: build_initial_monmap 2024-06-28T15:24:42.685 INFO:tasks.workunit.client.0.smithi178.stderr: -477> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 1 build_initial for_mkfs: 0 2024-06-28T15:24:42.685 INFO:tasks.workunit.client.0.smithi178.stderr: -476> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 10 monclient: monmap: 2024-06-28T15:24:42.685 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:42.685 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:42.685 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:42.685 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:42.686 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:42.686 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.686 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:42.686 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:42.686 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:42.686 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.686 INFO:tasks.workunit.client.0.smithi178.stderr: -475> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding auth protocol: cephx 2024-06-28T15:24:42.686 INFO:tasks.workunit.client.0.smithi178.stderr: -474> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding auth protocol: cephx 2024-06-28T15:24:42.686 INFO:tasks.workunit.client.0.smithi178.stderr: -473> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding auth protocol: cephx 2024-06-28T15:24:42.686 INFO:tasks.workunit.client.0.smithi178.stderr: -472> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: secure 2024-06-28T15:24:42.686 INFO:tasks.workunit.client.0.smithi178.stderr: -471> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: crc 2024-06-28T15:24:42.686 INFO:tasks.workunit.client.0.smithi178.stderr: -470> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: secure 2024-06-28T15:24:42.686 INFO:tasks.workunit.client.0.smithi178.stderr: -469> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: crc 2024-06-28T15:24:42.686 INFO:tasks.workunit.client.0.smithi178.stderr: -468> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: secure 2024-06-28T15:24:42.687 INFO:tasks.workunit.client.0.smithi178.stderr: -467> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: crc 2024-06-28T15:24:42.687 INFO:tasks.workunit.client.0.smithi178.stderr: -466> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: crc 2024-06-28T15:24:42.687 INFO:tasks.workunit.client.0.smithi178.stderr: -465> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: secure 2024-06-28T15:24:42.687 INFO:tasks.workunit.client.0.smithi178.stderr: -464> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: crc 2024-06-28T15:24:42.687 INFO:tasks.workunit.client.0.smithi178.stderr: -463> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: secure 2024-06-28T15:24:42.687 INFO:tasks.workunit.client.0.smithi178.stderr: -462> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: crc 2024-06-28T15:24:42.687 INFO:tasks.workunit.client.0.smithi178.stderr: -461> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: secure 2024-06-28T15:24:42.687 INFO:tasks.workunit.client.0.smithi178.stderr: -460> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.687 INFO:tasks.workunit.client.0.smithi178.stderr: -459> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 1 librados: starting msgr at 2024-06-28T15:24:42.687 INFO:tasks.workunit.client.0.smithi178.stderr: -458> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 1 librados: starting objecter 2024-06-28T15:24:42.687 INFO:tasks.workunit.client.0.smithi178.stderr: -457> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command objecter_requests hook 0x563289e62090 2024-06-28T15:24:42.687 INFO:tasks.workunit.client.0.smithi178.stderr: -456> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 1 librados: setting wanted keys 2024-06-28T15:24:42.687 INFO:tasks.workunit.client.0.smithi178.stderr: -455> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 1 librados: calling monclient init 2024-06-28T15:24:42.687 INFO:tasks.workunit.client.0.smithi178.stderr: -454> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient: init 2024-06-28T15:24:42.688 INFO:tasks.workunit.client.0.smithi178.stderr: -453> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding auth protocol: cephx 2024-06-28T15:24:42.688 INFO:tasks.workunit.client.0.smithi178.stderr: -452> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding auth protocol: cephx 2024-06-28T15:24:42.688 INFO:tasks.workunit.client.0.smithi178.stderr: -451> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding auth protocol: cephx 2024-06-28T15:24:42.688 INFO:tasks.workunit.client.0.smithi178.stderr: -450> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: secure 2024-06-28T15:24:42.688 INFO:tasks.workunit.client.0.smithi178.stderr: -449> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: crc 2024-06-28T15:24:42.688 INFO:tasks.workunit.client.0.smithi178.stderr: -448> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: secure 2024-06-28T15:24:42.688 INFO:tasks.workunit.client.0.smithi178.stderr: -447> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: crc 2024-06-28T15:24:42.688 INFO:tasks.workunit.client.0.smithi178.stderr: -446> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: secure 2024-06-28T15:24:42.688 INFO:tasks.workunit.client.0.smithi178.stderr: -445> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: crc 2024-06-28T15:24:42.688 INFO:tasks.workunit.client.0.smithi178.stderr: -444> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: crc 2024-06-28T15:24:42.688 INFO:tasks.workunit.client.0.smithi178.stderr: -443> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: secure 2024-06-28T15:24:42.688 INFO:tasks.workunit.client.0.smithi178.stderr: -442> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: crc 2024-06-28T15:24:42.688 INFO:tasks.workunit.client.0.smithi178.stderr: -441> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: secure 2024-06-28T15:24:42.688 INFO:tasks.workunit.client.0.smithi178.stderr: -440> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: crc 2024-06-28T15:24:42.689 INFO:tasks.workunit.client.0.smithi178.stderr: -439> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: secure 2024-06-28T15:24:42.689 INFO:tasks.workunit.client.0.smithi178.stderr: -438> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.689 INFO:tasks.workunit.client.0.smithi178.stderr: -437> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.689 INFO:tasks.workunit.client.0.smithi178.stderr: -436> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command rotate-key hook 0x56328a08f8c8 2024-06-28T15:24:42.689 INFO:tasks.workunit.client.0.smithi178.stderr: -435> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:42.689 INFO:tasks.workunit.client.0.smithi178.stderr: -434> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient: _add_conns ranks=[2,1,0] 2024-06-28T15:24:42.689 INFO:tasks.workunit.client.0.smithi178.stderr: -433> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-c con 0x563289f3fd30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:42.689 INFO:tasks.workunit.client.0.smithi178.stderr: -432> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-b con 0x563289f49d80 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:42.689 INFO:tasks.workunit.client.0.smithi178.stderr: -431> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-a con 0x56328a0ba2d0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:42.689 INFO:tasks.workunit.client.0.smithi178.stderr: -430> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.689 INFO:tasks.workunit.client.0.smithi178.stderr: -429> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.689 INFO:tasks.workunit.client.0.smithi178.stderr: -428> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.689 INFO:tasks.workunit.client.0.smithi178.stderr: -427> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:42.689 INFO:tasks.workunit.client.0.smithi178.stderr: -426> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): authenticate will time out at 2425.796960s 2024-06-28T15:24:42.690 INFO:tasks.workunit.client.0.smithi178.stderr: -425> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): get_auth_request con 0x563289f3fd30 auth_method 0 2024-06-28T15:24:42.690 INFO:tasks.workunit.client.0.smithi178.stderr: -424> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.690 INFO:tasks.workunit.client.0.smithi178.stderr: -423> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.690 INFO:tasks.workunit.client.0.smithi178.stderr: -422> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.690 INFO:tasks.workunit.client.0.smithi178.stderr: -421> 2024-06-28T15:24:42.333+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request con 0x56328a0ba2d0 auth_method 0 2024-06-28T15:24:42.690 INFO:tasks.workunit.client.0.smithi178.stderr: -420> 2024-06-28T15:24:42.333+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.690 INFO:tasks.workunit.client.0.smithi178.stderr: -419> 2024-06-28T15:24:42.333+0000 7fcea4ed7640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.690 INFO:tasks.workunit.client.0.smithi178.stderr: -418> 2024-06-28T15:24:42.333+0000 7fcea4ed7640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.690 INFO:tasks.workunit.client.0.smithi178.stderr: -417> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.690 INFO:tasks.workunit.client.0.smithi178.stderr: -416> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.690 INFO:tasks.workunit.client.0.smithi178.stderr: -415> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.690 INFO:tasks.workunit.client.0.smithi178.stderr: -414> 2024-06-28T15:24:42.333+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.690 INFO:tasks.workunit.client.0.smithi178.stderr: -413> 2024-06-28T15:24:42.333+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.691 INFO:tasks.workunit.client.0.smithi178.stderr: -412> 2024-06-28T15:24:42.333+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.691 INFO:tasks.workunit.client.0.smithi178.stderr: -411> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): handle_auth_done global_id 6494 payload 824 2024-06-28T15:24:42.691 INFO:tasks.workunit.client.0.smithi178.stderr: -410> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient: _finish_hunting 0 2024-06-28T15:24:42.691 INFO:tasks.workunit.client.0.smithi178.stderr: -409> 2024-06-28T15:24:42.333+0000 7fcea46d6640 1 monclient: found mon.noname-c 2024-06-28T15:24:42.691 INFO:tasks.workunit.client.0.smithi178.stderr: -408> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:24:42.691 INFO:tasks.workunit.client.0.smithi178.stderr: -407> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.691 INFO:tasks.workunit.client.0.smithi178.stderr: -406> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:24:42.691 INFO:tasks.workunit.client.0.smithi178.stderr: -405> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: dump: 2024-06-28T15:24:42.691 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.691 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.691 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.691 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.691 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.691 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.692 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.692 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.692 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.692 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.692 INFO:tasks.workunit.client.0.smithi178.stderr: -404> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: _finish_auth 0 2024-06-28T15:24:42.692 INFO:tasks.workunit.client.0.smithi178.stderr: -403> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: _check_auth_tickets 2024-06-28T15:24:42.692 INFO:tasks.workunit.client.0.smithi178.stderr: -402> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:42.692 INFO:tasks.workunit.client.0.smithi178.stderr: -401> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.692 INFO:tasks.workunit.client.0.smithi178.stderr: -400> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:24:42.692 INFO:tasks.workunit.client.0.smithi178.stderr: -399> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: dump: 2024-06-28T15:24:42.692 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.692 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.692 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.693 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.693 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.693 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.693 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.693 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.693 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.693 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.693 INFO:tasks.workunit.client.0.smithi178.stderr: -398> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 monclient: authenticate success, global_id 6494 2024-06-28T15:24:42.693 INFO:tasks.workunit.client.0.smithi178.stderr: -397> 2024-06-28T15:24:42.333+0000 7fcd9260c640 4 set_mon_vals no callback set 2024-06-28T15:24:42.693 INFO:tasks.workunit.client.0.smithi178.stderr: -396> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient: _renew_subs 2024-06-28T15:24:42.693 INFO:tasks.workunit.client.0.smithi178.stderr: -395> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:42.693 INFO:tasks.workunit.client.0.smithi178.stderr: -394> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient: _renew_subs 2024-06-28T15:24:42.693 INFO:tasks.workunit.client.0.smithi178.stderr: -393> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:42.693 INFO:tasks.workunit.client.0.smithi178.stderr: -392> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 1 librados: init done 2024-06-28T15:24:42.694 INFO:tasks.workunit.client.0.smithi178.stderr: -391> 2024-06-28T15:24:42.333+0000 7fcd90e09640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:24:42.694 INFO:tasks.workunit.client.0.smithi178.stderr: -390> 2024-06-28T15:24:42.333+0000 7fcd90e09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:42.694 INFO:tasks.workunit.client.0.smithi178.stderr: -389> 2024-06-28T15:24:42.333+0000 7fcd90e09640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:42.694 INFO:tasks.workunit.client.0.smithi178.stderr: -388> 2024-06-28T15:24:42.333+0000 7fcea3ed5640 10 monclient: get_auth_request con 0x7fcd8c06f780 auth_method 0 2024-06-28T15:24:42.694 INFO:tasks.workunit.client.0.smithi178.stderr: -387> 2024-06-28T15:24:42.337+0000 7fcea4ed7640 10 monclient: get_auth_request con 0x56328a0b7ea0 auth_method 0 2024-06-28T15:24:42.694 INFO:tasks.workunit.client.0.smithi178.stderr: -386> 2024-06-28T15:24:42.337+0000 7fcea46d6640 10 monclient: get_auth_request con 0x56328a0da740 auth_method 0 2024-06-28T15:24:42.694 INFO:tasks.workunit.client.0.smithi178.stderr: -385> 2024-06-28T15:24:42.337+0000 7fcea3ed5640 10 monclient: get_auth_request con 0x56328a0deaa0 auth_method 0 2024-06-28T15:24:42.694 INFO:tasks.workunit.client.0.smithi178.stderr: -384> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: build_initial_monmap 2024-06-28T15:24:42.694 INFO:tasks.workunit.client.0.smithi178.stderr: -383> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 1 build_initial for_mkfs: 0 2024-06-28T15:24:42.694 INFO:tasks.workunit.client.0.smithi178.stderr: -382> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: monmap: 2024-06-28T15:24:42.694 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:42.694 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:42.694 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:42.694 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:42.695 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:42.695 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.695 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:42.695 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:42.695 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:42.695 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.695 INFO:tasks.workunit.client.0.smithi178.stderr: -381> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding auth protocol: cephx 2024-06-28T15:24:42.695 INFO:tasks.workunit.client.0.smithi178.stderr: -380> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding auth protocol: cephx 2024-06-28T15:24:42.695 INFO:tasks.workunit.client.0.smithi178.stderr: -379> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding auth protocol: cephx 2024-06-28T15:24:42.695 INFO:tasks.workunit.client.0.smithi178.stderr: -378> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: secure 2024-06-28T15:24:42.695 INFO:tasks.workunit.client.0.smithi178.stderr: -377> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: crc 2024-06-28T15:24:42.695 INFO:tasks.workunit.client.0.smithi178.stderr: -376> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: secure 2024-06-28T15:24:42.695 INFO:tasks.workunit.client.0.smithi178.stderr: -375> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: crc 2024-06-28T15:24:42.696 INFO:tasks.workunit.client.0.smithi178.stderr: -374> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: secure 2024-06-28T15:24:42.696 INFO:tasks.workunit.client.0.smithi178.stderr: -373> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: crc 2024-06-28T15:24:42.696 INFO:tasks.workunit.client.0.smithi178.stderr: -372> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: crc 2024-06-28T15:24:42.696 INFO:tasks.workunit.client.0.smithi178.stderr: -371> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: secure 2024-06-28T15:24:42.696 INFO:tasks.workunit.client.0.smithi178.stderr: -370> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: crc 2024-06-28T15:24:42.696 INFO:tasks.workunit.client.0.smithi178.stderr: -369> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: secure 2024-06-28T15:24:42.696 INFO:tasks.workunit.client.0.smithi178.stderr: -368> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: crc 2024-06-28T15:24:42.696 INFO:tasks.workunit.client.0.smithi178.stderr: -367> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: secure 2024-06-28T15:24:42.696 INFO:tasks.workunit.client.0.smithi178.stderr: -366> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.696 INFO:tasks.workunit.client.0.smithi178.stderr: -365> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command objecter_requests cmddesc objecter_requests hook 0x56328a0e6ec0 EEXIST 2024-06-28T15:24:42.696 INFO:tasks.workunit.client.0.smithi178.stderr: -364> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: init 2024-06-28T15:24:42.696 INFO:tasks.workunit.client.0.smithi178.stderr: -363> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding auth protocol: cephx 2024-06-28T15:24:42.696 INFO:tasks.workunit.client.0.smithi178.stderr: -362> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding auth protocol: cephx 2024-06-28T15:24:42.697 INFO:tasks.workunit.client.0.smithi178.stderr: -361> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding auth protocol: cephx 2024-06-28T15:24:42.697 INFO:tasks.workunit.client.0.smithi178.stderr: -360> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: secure 2024-06-28T15:24:42.697 INFO:tasks.workunit.client.0.smithi178.stderr: -359> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: crc 2024-06-28T15:24:42.697 INFO:tasks.workunit.client.0.smithi178.stderr: -358> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: secure 2024-06-28T15:24:42.697 INFO:tasks.workunit.client.0.smithi178.stderr: -357> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: crc 2024-06-28T15:24:42.697 INFO:tasks.workunit.client.0.smithi178.stderr: -356> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: secure 2024-06-28T15:24:42.697 INFO:tasks.workunit.client.0.smithi178.stderr: -355> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: crc 2024-06-28T15:24:42.697 INFO:tasks.workunit.client.0.smithi178.stderr: -354> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: crc 2024-06-28T15:24:42.697 INFO:tasks.workunit.client.0.smithi178.stderr: -353> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: secure 2024-06-28T15:24:42.697 INFO:tasks.workunit.client.0.smithi178.stderr: -352> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: crc 2024-06-28T15:24:42.697 INFO:tasks.workunit.client.0.smithi178.stderr: -351> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: secure 2024-06-28T15:24:42.697 INFO:tasks.workunit.client.0.smithi178.stderr: -350> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: crc 2024-06-28T15:24:42.697 INFO:tasks.workunit.client.0.smithi178.stderr: -349> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: secure 2024-06-28T15:24:42.697 INFO:tasks.workunit.client.0.smithi178.stderr: -348> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.698 INFO:tasks.workunit.client.0.smithi178.stderr: -347> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.698 INFO:tasks.workunit.client.0.smithi178.stderr: -346> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command rotate-key cmddesc rotate-key hook 0x56328a0e2160 EEXIST 2024-06-28T15:24:42.698 INFO:tasks.workunit.client.0.smithi178.stderr: -345> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:42.698 INFO:tasks.workunit.client.0.smithi178.stderr: -344> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: _add_conns ranks=[0,1,2] 2024-06-28T15:24:42.698 INFO:tasks.workunit.client.0.smithi178.stderr: -343> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-a con 0x56328a0ed450 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:42.698 INFO:tasks.workunit.client.0.smithi178.stderr: -342> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-b con 0x56328a0edd80 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:42.698 INFO:tasks.workunit.client.0.smithi178.stderr: -341> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-c con 0x56328a0ed830 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:42.698 INFO:tasks.workunit.client.0.smithi178.stderr: -340> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.698 INFO:tasks.workunit.client.0.smithi178.stderr: -339> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.698 INFO:tasks.workunit.client.0.smithi178.stderr: -338> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.698 INFO:tasks.workunit.client.0.smithi178.stderr: -337> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:42.698 INFO:tasks.workunit.client.0.smithi178.stderr: -336> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): authenticate will time out at 2425.806415s 2024-06-28T15:24:42.698 INFO:tasks.workunit.client.0.smithi178.stderr: -335> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): get_auth_request con 0x56328a0ed450 auth_method 0 2024-06-28T15:24:42.699 INFO:tasks.workunit.client.0.smithi178.stderr: -334> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.699 INFO:tasks.workunit.client.0.smithi178.stderr: -333> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.699 INFO:tasks.workunit.client.0.smithi178.stderr: -332> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.699 INFO:tasks.workunit.client.0.smithi178.stderr: -331> 2024-06-28T15:24:42.341+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request con 0x56328a0ed830 auth_method 0 2024-06-28T15:24:42.699 INFO:tasks.workunit.client.0.smithi178.stderr: -330> 2024-06-28T15:24:42.341+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.699 INFO:tasks.workunit.client.0.smithi178.stderr: -329> 2024-06-28T15:24:42.341+0000 7fcea4ed7640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.699 INFO:tasks.workunit.client.0.smithi178.stderr: -328> 2024-06-28T15:24:42.341+0000 7fcea4ed7640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.699 INFO:tasks.workunit.client.0.smithi178.stderr: -327> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.699 INFO:tasks.workunit.client.0.smithi178.stderr: -326> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.699 INFO:tasks.workunit.client.0.smithi178.stderr: -325> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.699 INFO:tasks.workunit.client.0.smithi178.stderr: -324> 2024-06-28T15:24:42.341+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.699 INFO:tasks.workunit.client.0.smithi178.stderr: -323> 2024-06-28T15:24:42.341+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.699 INFO:tasks.workunit.client.0.smithi178.stderr: -322> 2024-06-28T15:24:42.341+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.700 INFO:tasks.workunit.client.0.smithi178.stderr: -321> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): handle_auth_done global_id 6921 payload 824 2024-06-28T15:24:42.700 INFO:tasks.workunit.client.0.smithi178.stderr: -320> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient: _finish_hunting 0 2024-06-28T15:24:42.700 INFO:tasks.workunit.client.0.smithi178.stderr: -319> 2024-06-28T15:24:42.341+0000 7fcea46d6640 1 monclient: found mon.noname-a 2024-06-28T15:24:42.700 INFO:tasks.workunit.client.0.smithi178.stderr: -318> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:24:42.700 INFO:tasks.workunit.client.0.smithi178.stderr: -317> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.700 INFO:tasks.workunit.client.0.smithi178.stderr: -316> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:24:42.700 INFO:tasks.workunit.client.0.smithi178.stderr: -315> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: dump: 2024-06-28T15:24:42.700 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.700 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.700 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.700 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.700 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.700 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.700 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.701 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.701 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.701 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.701 INFO:tasks.workunit.client.0.smithi178.stderr: -314> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: _finish_auth 0 2024-06-28T15:24:42.701 INFO:tasks.workunit.client.0.smithi178.stderr: -313> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: _check_auth_tickets 2024-06-28T15:24:42.701 INFO:tasks.workunit.client.0.smithi178.stderr: -312> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 monclient: authenticate success, global_id 6921 2024-06-28T15:24:42.701 INFO:tasks.workunit.client.0.smithi178.stderr: -311> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:42.701 INFO:tasks.workunit.client.0.smithi178.stderr: -310> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.701 INFO:tasks.workunit.client.0.smithi178.stderr: -309> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:24:42.701 INFO:tasks.workunit.client.0.smithi178.stderr: -308> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: dump: 2024-06-28T15:24:42.701 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.701 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.701 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.701 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.702 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.702 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.702 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.702 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.702 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.702 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.702 INFO:tasks.workunit.client.0.smithi178.stderr: -307> 2024-06-28T15:24:42.341+0000 7fcea56d8640 4 set_mon_vals no callback set 2024-06-28T15:24:42.702 INFO:tasks.workunit.client.0.smithi178.stderr: -306> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: _renew_subs 2024-06-28T15:24:42.702 INFO:tasks.workunit.client.0.smithi178.stderr: -305> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:24:42.702 INFO:tasks.workunit.client.0.smithi178.stderr: -304> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: _renew_subs 2024-06-28T15:24:42.702 INFO:tasks.workunit.client.0.smithi178.stderr: -303> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:24:42.702 INFO:tasks.workunit.client.0.smithi178.stderr: -302> 2024-06-28T15:24:42.345+0000 7fcd89ffb640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:24:42.702 INFO:tasks.workunit.client.0.smithi178.stderr: -301> 2024-06-28T15:24:42.345+0000 7fcd89ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:42.703 INFO:tasks.workunit.client.0.smithi178.stderr: -300> 2024-06-28T15:24:42.345+0000 7fcd89ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:42.703 INFO:tasks.workunit.client.0.smithi178.stderr: -299> 2024-06-28T15:24:42.345+0000 7fcea3ed5640 10 monclient: get_auth_request con 0x7fcd7c06f690 auth_method 0 2024-06-28T15:24:42.703 INFO:tasks.workunit.client.0.smithi178.stderr: -298> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient: get_monmap_and_config 2024-06-28T15:24:42.703 INFO:tasks.workunit.client.0.smithi178.stderr: -297> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient: build_initial_monmap 2024-06-28T15:24:42.703 INFO:tasks.workunit.client.0.smithi178.stderr: -296> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 1 build_initial for_mkfs: 0 2024-06-28T15:24:42.703 INFO:tasks.workunit.client.0.smithi178.stderr: -295> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient: monmap: 2024-06-28T15:24:42.703 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:42.703 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:42.703 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:42.703 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:42.703 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:42.703 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.703 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:42.703 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:42.704 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:42.704 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.704 INFO:tasks.workunit.client.0.smithi178.stderr: -294> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding auth protocol: cephx 2024-06-28T15:24:42.704 INFO:tasks.workunit.client.0.smithi178.stderr: -293> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding auth protocol: cephx 2024-06-28T15:24:42.704 INFO:tasks.workunit.client.0.smithi178.stderr: -292> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding auth protocol: cephx 2024-06-28T15:24:42.704 INFO:tasks.workunit.client.0.smithi178.stderr: -291> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: secure 2024-06-28T15:24:42.704 INFO:tasks.workunit.client.0.smithi178.stderr: -290> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: crc 2024-06-28T15:24:42.704 INFO:tasks.workunit.client.0.smithi178.stderr: -289> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: secure 2024-06-28T15:24:42.704 INFO:tasks.workunit.client.0.smithi178.stderr: -288> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: crc 2024-06-28T15:24:42.704 INFO:tasks.workunit.client.0.smithi178.stderr: -287> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: secure 2024-06-28T15:24:42.704 INFO:tasks.workunit.client.0.smithi178.stderr: -286> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: crc 2024-06-28T15:24:42.704 INFO:tasks.workunit.client.0.smithi178.stderr: -285> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: crc 2024-06-28T15:24:42.704 INFO:tasks.workunit.client.0.smithi178.stderr: -284> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: secure 2024-06-28T15:24:42.705 INFO:tasks.workunit.client.0.smithi178.stderr: -283> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: crc 2024-06-28T15:24:42.705 INFO:tasks.workunit.client.0.smithi178.stderr: -282> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: secure 2024-06-28T15:24:42.705 INFO:tasks.workunit.client.0.smithi178.stderr: -281> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: crc 2024-06-28T15:24:42.705 INFO:tasks.workunit.client.0.smithi178.stderr: -280> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: secure 2024-06-28T15:24:42.705 INFO:tasks.workunit.client.0.smithi178.stderr: -279> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.705 INFO:tasks.workunit.client.0.smithi178.stderr: -278> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient: init 2024-06-28T15:24:42.705 INFO:tasks.workunit.client.0.smithi178.stderr: -277> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding auth protocol: cephx 2024-06-28T15:24:42.705 INFO:tasks.workunit.client.0.smithi178.stderr: -276> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding auth protocol: cephx 2024-06-28T15:24:42.705 INFO:tasks.workunit.client.0.smithi178.stderr: -275> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding auth protocol: cephx 2024-06-28T15:24:42.705 INFO:tasks.workunit.client.0.smithi178.stderr: -274> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: secure 2024-06-28T15:24:42.705 INFO:tasks.workunit.client.0.smithi178.stderr: -273> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: crc 2024-06-28T15:24:42.705 INFO:tasks.workunit.client.0.smithi178.stderr: -272> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: secure 2024-06-28T15:24:42.705 INFO:tasks.workunit.client.0.smithi178.stderr: -271> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: crc 2024-06-28T15:24:42.705 INFO:tasks.workunit.client.0.smithi178.stderr: -270> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: secure 2024-06-28T15:24:42.706 INFO:tasks.workunit.client.0.smithi178.stderr: -269> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: crc 2024-06-28T15:24:42.706 INFO:tasks.workunit.client.0.smithi178.stderr: -268> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: crc 2024-06-28T15:24:42.706 INFO:tasks.workunit.client.0.smithi178.stderr: -267> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: secure 2024-06-28T15:24:42.706 INFO:tasks.workunit.client.0.smithi178.stderr: -266> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: crc 2024-06-28T15:24:42.706 INFO:tasks.workunit.client.0.smithi178.stderr: -265> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: secure 2024-06-28T15:24:42.706 INFO:tasks.workunit.client.0.smithi178.stderr: -264> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: crc 2024-06-28T15:24:42.706 INFO:tasks.workunit.client.0.smithi178.stderr: -263> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: secure 2024-06-28T15:24:42.706 INFO:tasks.workunit.client.0.smithi178.stderr: -262> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.706 INFO:tasks.workunit.client.0.smithi178.stderr: -261> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.706 INFO:tasks.workunit.client.0.smithi178.stderr: -260> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command rotate-key cmddesc rotate-key hook 0x7ffd239b9378 EEXIST 2024-06-28T15:24:42.706 INFO:tasks.workunit.client.0.smithi178.stderr: -259> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:42.706 INFO:tasks.workunit.client.0.smithi178.stderr: -258> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient: _add_conns ranks=[2,1,0] 2024-06-28T15:24:42.706 INFO:tasks.workunit.client.0.smithi178.stderr: -257> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-c con 0x56328a1593b0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:42.707 INFO:tasks.workunit.client.0.smithi178.stderr: -256> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-b con 0x56328a164760 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:42.707 INFO:tasks.workunit.client.0.smithi178.stderr: -255> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-a con 0x56328a151220 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:42.707 INFO:tasks.workunit.client.0.smithi178.stderr: -254> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.707 INFO:tasks.workunit.client.0.smithi178.stderr: -253> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.707 INFO:tasks.workunit.client.0.smithi178.stderr: -252> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.707 INFO:tasks.workunit.client.0.smithi178.stderr: -251> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:42.707 INFO:tasks.workunit.client.0.smithi178.stderr: -250> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): authenticate will time out at 2425.810827s 2024-06-28T15:24:42.707 INFO:tasks.workunit.client.0.smithi178.stderr: -249> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): get_auth_request con 0x56328a1593b0 auth_method 0 2024-06-28T15:24:42.707 INFO:tasks.workunit.client.0.smithi178.stderr: -248> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.707 INFO:tasks.workunit.client.0.smithi178.stderr: -247> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.707 INFO:tasks.workunit.client.0.smithi178.stderr: -246> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.707 INFO:tasks.workunit.client.0.smithi178.stderr: -245> 2024-06-28T15:24:42.345+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request con 0x56328a151220 auth_method 0 2024-06-28T15:24:42.707 INFO:tasks.workunit.client.0.smithi178.stderr: -244> 2024-06-28T15:24:42.345+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.707 INFO:tasks.workunit.client.0.smithi178.stderr: -243> 2024-06-28T15:24:42.345+0000 7fcea4ed7640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.708 INFO:tasks.workunit.client.0.smithi178.stderr: -242> 2024-06-28T15:24:42.345+0000 7fcea4ed7640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.708 INFO:tasks.workunit.client.0.smithi178.stderr: -241> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.708 INFO:tasks.workunit.client.0.smithi178.stderr: -240> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.708 INFO:tasks.workunit.client.0.smithi178.stderr: -239> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.708 INFO:tasks.workunit.client.0.smithi178.stderr: -238> 2024-06-28T15:24:42.345+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.708 INFO:tasks.workunit.client.0.smithi178.stderr: -237> 2024-06-28T15:24:42.345+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.708 INFO:tasks.workunit.client.0.smithi178.stderr: -236> 2024-06-28T15:24:42.345+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.708 INFO:tasks.workunit.client.0.smithi178.stderr: -235> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): handle_auth_done global_id 6500 payload 274 2024-06-28T15:24:42.708 INFO:tasks.workunit.client.0.smithi178.stderr: -234> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient: _finish_hunting 0 2024-06-28T15:24:42.708 INFO:tasks.workunit.client.0.smithi178.stderr: -233> 2024-06-28T15:24:42.345+0000 7fcea46d6640 1 monclient: found mon.noname-c 2024-06-28T15:24:42.708 INFO:tasks.workunit.client.0.smithi178.stderr: -232> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:24:42.708 INFO:tasks.workunit.client.0.smithi178.stderr: -231> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.708 INFO:tasks.workunit.client.0.smithi178.stderr: -230> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:24:42.709 INFO:tasks.workunit.client.0.smithi178.stderr: -229> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: dump: 2024-06-28T15:24:42.709 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.709 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.709 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.709 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.709 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.709 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.709 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.709 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.709 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.709 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.709 INFO:tasks.workunit.client.0.smithi178.stderr: -228> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: _finish_auth 0 2024-06-28T15:24:42.709 INFO:tasks.workunit.client.0.smithi178.stderr: -227> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: _check_auth_tickets 2024-06-28T15:24:42.710 INFO:tasks.workunit.client.0.smithi178.stderr: -226> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:42.710 INFO:tasks.workunit.client.0.smithi178.stderr: -225> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 monclient: authenticate success, global_id 6500 2024-06-28T15:24:42.710 INFO:tasks.workunit.client.0.smithi178.stderr: -224> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient: get_monmap_and_config success 2024-06-28T15:24:42.710 INFO:tasks.workunit.client.0.smithi178.stderr: -223> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 4 set_mon_vals no callback set 2024-06-28T15:24:42.710 INFO:tasks.workunit.client.0.smithi178.stderr: -222> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.710 INFO:tasks.workunit.client.0.smithi178.stderr: -221> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:24:42.710 INFO:tasks.workunit.client.0.smithi178.stderr: -220> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: dump: 2024-06-28T15:24:42.710 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.710 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.710 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.710 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.710 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.710 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.711 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.711 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.711 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.711 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.711 INFO:tasks.workunit.client.0.smithi178.stderr: -219> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: shutdown 2024-06-28T15:24:42.711 INFO:tasks.workunit.client.0.smithi178.stderr: -218> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: build_initial_monmap 2024-06-28T15:24:42.711 INFO:tasks.workunit.client.0.smithi178.stderr: -217> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 1 build_initial for_mkfs: 0 2024-06-28T15:24:42.711 INFO:tasks.workunit.client.0.smithi178.stderr: -216> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: monmap: 2024-06-28T15:24:42.711 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:42.711 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:42.711 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:42.711 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:42.711 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:42.712 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.712 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:42.712 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:42.712 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:42.712 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.712 INFO:tasks.workunit.client.0.smithi178.stderr: -215> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding auth protocol: cephx 2024-06-28T15:24:42.712 INFO:tasks.workunit.client.0.smithi178.stderr: -214> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding auth protocol: cephx 2024-06-28T15:24:42.712 INFO:tasks.workunit.client.0.smithi178.stderr: -213> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding auth protocol: cephx 2024-06-28T15:24:42.712 INFO:tasks.workunit.client.0.smithi178.stderr: -212> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: secure 2024-06-28T15:24:42.712 INFO:tasks.workunit.client.0.smithi178.stderr: -211> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: crc 2024-06-28T15:24:42.712 INFO:tasks.workunit.client.0.smithi178.stderr: -210> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: secure 2024-06-28T15:24:42.712 INFO:tasks.workunit.client.0.smithi178.stderr: -209> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: crc 2024-06-28T15:24:42.713 INFO:tasks.workunit.client.0.smithi178.stderr: -208> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: secure 2024-06-28T15:24:42.713 INFO:tasks.workunit.client.0.smithi178.stderr: -207> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: crc 2024-06-28T15:24:42.713 INFO:tasks.workunit.client.0.smithi178.stderr: -206> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: crc 2024-06-28T15:24:42.713 INFO:tasks.workunit.client.0.smithi178.stderr: -205> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: secure 2024-06-28T15:24:42.713 INFO:tasks.workunit.client.0.smithi178.stderr: -204> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: crc 2024-06-28T15:24:42.713 INFO:tasks.workunit.client.0.smithi178.stderr: -203> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: secure 2024-06-28T15:24:42.713 INFO:tasks.workunit.client.0.smithi178.stderr: -202> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: crc 2024-06-28T15:24:42.713 INFO:tasks.workunit.client.0.smithi178.stderr: -201> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: secure 2024-06-28T15:24:42.713 INFO:tasks.workunit.client.0.smithi178.stderr: -200> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.713 INFO:tasks.workunit.client.0.smithi178.stderr: -199> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 1 librados: starting msgr at 2024-06-28T15:24:42.713 INFO:tasks.workunit.client.0.smithi178.stderr: -198> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 1 librados: starting objecter 2024-06-28T15:24:42.713 INFO:tasks.workunit.client.0.smithi178.stderr: -197> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command objecter_requests cmddesc objecter_requests hook 0x56328a0ef780 EEXIST 2024-06-28T15:24:42.714 INFO:tasks.workunit.client.0.smithi178.stderr: -196> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 1 librados: setting wanted keys 2024-06-28T15:24:42.714 INFO:tasks.workunit.client.0.smithi178.stderr: -195> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 1 librados: calling monclient init 2024-06-28T15:24:42.714 INFO:tasks.workunit.client.0.smithi178.stderr: -194> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: init 2024-06-28T15:24:42.714 INFO:tasks.workunit.client.0.smithi178.stderr: -193> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding auth protocol: cephx 2024-06-28T15:24:42.714 INFO:tasks.workunit.client.0.smithi178.stderr: -192> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding auth protocol: cephx 2024-06-28T15:24:42.714 INFO:tasks.workunit.client.0.smithi178.stderr: -191> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding auth protocol: cephx 2024-06-28T15:24:42.714 INFO:tasks.workunit.client.0.smithi178.stderr: -190> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: secure 2024-06-28T15:24:42.714 INFO:tasks.workunit.client.0.smithi178.stderr: -189> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: crc 2024-06-28T15:24:42.714 INFO:tasks.workunit.client.0.smithi178.stderr: -188> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: secure 2024-06-28T15:24:42.714 INFO:tasks.workunit.client.0.smithi178.stderr: -187> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: crc 2024-06-28T15:24:42.714 INFO:tasks.workunit.client.0.smithi178.stderr: -186> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: secure 2024-06-28T15:24:42.714 INFO:tasks.workunit.client.0.smithi178.stderr: -185> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: crc 2024-06-28T15:24:42.714 INFO:tasks.workunit.client.0.smithi178.stderr: -184> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: crc 2024-06-28T15:24:42.715 INFO:tasks.workunit.client.0.smithi178.stderr: -183> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: secure 2024-06-28T15:24:42.715 INFO:tasks.workunit.client.0.smithi178.stderr: -182> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: crc 2024-06-28T15:24:42.715 INFO:tasks.workunit.client.0.smithi178.stderr: -181> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: secure 2024-06-28T15:24:42.715 INFO:tasks.workunit.client.0.smithi178.stderr: -180> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: crc 2024-06-28T15:24:42.715 INFO:tasks.workunit.client.0.smithi178.stderr: -179> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: secure 2024-06-28T15:24:42.715 INFO:tasks.workunit.client.0.smithi178.stderr: -178> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.715 INFO:tasks.workunit.client.0.smithi178.stderr: -177> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.715 INFO:tasks.workunit.client.0.smithi178.stderr: -176> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command rotate-key cmddesc rotate-key hook 0x56328a122ff8 EEXIST 2024-06-28T15:24:42.715 INFO:tasks.workunit.client.0.smithi178.stderr: -175> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:42.715 INFO:tasks.workunit.client.0.smithi178.stderr: -174> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: _add_conns ranks=[0,1,2] 2024-06-28T15:24:42.715 INFO:tasks.workunit.client.0.smithi178.stderr: -173> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-a con 0x56328a151220 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:42.715 INFO:tasks.workunit.client.0.smithi178.stderr: -172> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-b con 0x56328a1593b0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:42.715 INFO:tasks.workunit.client.0.smithi178.stderr: -171> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-c con 0x56328a164760 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:42.716 INFO:tasks.workunit.client.0.smithi178.stderr: -170> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.716 INFO:tasks.workunit.client.0.smithi178.stderr: -169> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.716 INFO:tasks.workunit.client.0.smithi178.stderr: -168> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.716 INFO:tasks.workunit.client.0.smithi178.stderr: -167> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:42.716 INFO:tasks.workunit.client.0.smithi178.stderr: -166> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): authenticate will time out at 2425.813391s 2024-06-28T15:24:42.716 INFO:tasks.workunit.client.0.smithi178.stderr: -165> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): get_auth_request con 0x56328a151220 auth_method 0 2024-06-28T15:24:42.716 INFO:tasks.workunit.client.0.smithi178.stderr: -164> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.716 INFO:tasks.workunit.client.0.smithi178.stderr: -163> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.716 INFO:tasks.workunit.client.0.smithi178.stderr: -162> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.716 INFO:tasks.workunit.client.0.smithi178.stderr: -161> 2024-06-28T15:24:42.349+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request con 0x56328a164760 auth_method 0 2024-06-28T15:24:42.716 INFO:tasks.workunit.client.0.smithi178.stderr: -160> 2024-06-28T15:24:42.349+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.716 INFO:tasks.workunit.client.0.smithi178.stderr: -159> 2024-06-28T15:24:42.349+0000 7fcea4ed7640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.716 INFO:tasks.workunit.client.0.smithi178.stderr: -158> 2024-06-28T15:24:42.349+0000 7fcea4ed7640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.717 INFO:tasks.workunit.client.0.smithi178.stderr: -157> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.717 INFO:tasks.workunit.client.0.smithi178.stderr: -156> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.717 INFO:tasks.workunit.client.0.smithi178.stderr: -155> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.717 INFO:tasks.workunit.client.0.smithi178.stderr: -154> 2024-06-28T15:24:42.349+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.717 INFO:tasks.workunit.client.0.smithi178.stderr: -153> 2024-06-28T15:24:42.349+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.717 INFO:tasks.workunit.client.0.smithi178.stderr: -152> 2024-06-28T15:24:42.349+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.717 INFO:tasks.workunit.client.0.smithi178.stderr: -151> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): handle_auth_done global_id 6927 payload 824 2024-06-28T15:24:42.717 INFO:tasks.workunit.client.0.smithi178.stderr: -150> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient: _finish_hunting 0 2024-06-28T15:24:42.717 INFO:tasks.workunit.client.0.smithi178.stderr: -149> 2024-06-28T15:24:42.349+0000 7fcea46d6640 1 monclient: found mon.noname-a 2024-06-28T15:24:42.717 INFO:tasks.workunit.client.0.smithi178.stderr: -148> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:24:42.717 INFO:tasks.workunit.client.0.smithi178.stderr: -147> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.717 INFO:tasks.workunit.client.0.smithi178.stderr: -146> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:24:42.717 INFO:tasks.workunit.client.0.smithi178.stderr: -145> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: dump: 2024-06-28T15:24:42.718 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.718 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.718 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.718 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.718 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.718 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.718 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.718 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.718 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.718 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.718 INFO:tasks.workunit.client.0.smithi178.stderr: -144> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: _finish_auth 0 2024-06-28T15:24:42.718 INFO:tasks.workunit.client.0.smithi178.stderr: -143> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: _check_auth_tickets 2024-06-28T15:24:42.718 INFO:tasks.workunit.client.0.smithi178.stderr: -142> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:42.718 INFO:tasks.workunit.client.0.smithi178.stderr: -141> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr: -140> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr: -139> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: dump: 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr: -138> 2024-06-28T15:24:42.349+0000 7fcd7b7fe640 4 set_mon_vals no callback set 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr: -137> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 monclient: authenticate success, global_id 6927 2024-06-28T15:24:42.719 INFO:tasks.workunit.client.0.smithi178.stderr: -136> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: _renew_subs 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -135> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -134> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: _renew_subs 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -133> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -132> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 1 librados: init done 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -131> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command cr dump hook 0x56328a154338 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -130> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -129> 2024-06-28T15:24:42.353+0000 7fcd79ffb640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -128> 2024-06-28T15:24:42.353+0000 7fcd79ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -127> 2024-06-28T15:24:42.353+0000 7fcd79ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -126> 2024-06-28T15:24:42.353+0000 7fcea3ed5640 10 monclient: get_auth_request con 0x7fcd6c06f780 auth_method 0 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -125> 2024-06-28T15:24:42.353+0000 7fcea4ed7640 10 monclient: get_auth_request con 0x56328a177ee0 auth_method 0 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -124> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -123> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 realm 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -122> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.720 INFO:tasks.workunit.client.0.smithi178.stderr: -121> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -120> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -119> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -118> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -117> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -116> 2024-06-28T15:24:42.353+0000 7fcea46d6640 10 monclient: get_auth_request con 0x56328a17c0d0 auth_method 0 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -115> 2024-06-28T15:24:42.357+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -114> 2024-06-28T15:24:42.357+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -113> 2024-06-28T15:24:42.357+0000 7fcea3ed5640 10 monclient: get_auth_request con 0x56328a17f910 auth_method 0 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -112> 2024-06-28T15:24:42.357+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -111> 2024-06-28T15:24:42.357+0000 7fcea6bc5980 20 searching for the correct realm 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -110> 2024-06-28T15:24:42.357+0000 7fcea4ed7640 10 monclient: get_auth_request con 0x7fce8c008070 auth_method 0 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -109> 2024-06-28T15:24:42.357+0000 7fcea46d6640 10 monclient: get_auth_request con 0x7fce9c024040 auth_method 0 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -108> 2024-06-28T15:24:42.361+0000 7fcea3ed5640 10 monclient: get_auth_request con 0x7fce940f1bd0 auth_method 0 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -107> 2024-06-28T15:24:42.361+0000 7fcea4ed7640 10 monclient: get_auth_request con 0x7fce8c04a960 auth_method 0 2024-06-28T15:24:42.721 INFO:tasks.workunit.client.0.smithi178.stderr: -106> 2024-06-28T15:24:42.361+0000 7fcea4ed7640 0 -- 172.21.15.178:0/2581271606 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fce8c04a960 msgr2=0x7fce8c029410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:42.722 INFO:tasks.workunit.client.0.smithi178.stderr: -105> 2024-06-28T15:24:42.565+0000 7fcea4ed7640 10 monclient: get_auth_request con 0x7fce8c04a960 auth_method 0 2024-06-28T15:24:42.722 INFO:tasks.workunit.client.0.smithi178.stderr: -104> 2024-06-28T15:24:42.569+0000 7fcea46d6640 10 monclient: get_auth_request con 0x7fce8c035400 auth_method 0 2024-06-28T15:24:42.722 INFO:tasks.workunit.client.0.smithi178.stderr: -103> 2024-06-28T15:24:42.577+0000 7fcea46d6640 0 -- 172.21.15.178:0/2581271606 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x56328a17c0d0 msgr2=0x56328a17e4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:42.722 INFO:tasks.workunit.client.0.smithi178.stderr: -102> 2024-06-28T15:24:42.577+0000 7fcd79ffb640 1 client.6927.objecter ms_handle_reset 0x56328a17c0d0 session 0x56328a17ba50 osd.5 2024-06-28T15:24:42.722 INFO:tasks.workunit.client.0.smithi178.stderr: -101> 2024-06-28T15:24:42.577+0000 7fcd79ffb640 10 monclient: _renew_subs 2024-06-28T15:24:42.722 INFO:tasks.workunit.client.0.smithi178.stderr: -100> 2024-06-28T15:24:42.577+0000 7fcd79ffb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:24:42.722 INFO:tasks.workunit.client.0.smithi178.stderr: -99> 2024-06-28T15:24:42.577+0000 7fcea46d6640 10 monclient: get_auth_request con 0x7fcd6c07fda0 auth_method 0 2024-06-28T15:24:42.722 INFO:tasks.workunit.client.0.smithi178.stderr: -98> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:42.722 INFO:tasks.workunit.client.0.smithi178.stderr: -97> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:42.722 INFO:tasks.workunit.client.0.smithi178.stderr: -96> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:42.722 INFO:tasks.workunit.client.0.smithi178.stderr: -95> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:42.722 INFO:tasks.workunit.client.0.smithi178.stderr: -94> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:42.722 INFO:tasks.workunit.client.0.smithi178.stderr: -93> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:42.723 INFO:tasks.workunit.client.0.smithi178.stderr: -92> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.723 INFO:tasks.workunit.client.0.smithi178.stderr: -91> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.723 INFO:tasks.workunit.client.0.smithi178.stderr: -90> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.723 INFO:tasks.workunit.client.0.smithi178.stderr: -89> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:42.723 INFO:tasks.workunit.client.0.smithi178.stderr: -88> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.723 INFO:tasks.workunit.client.0.smithi178.stderr: -87> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:42.723 INFO:tasks.workunit.client.0.smithi178.stderr: -86> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 zone default found 2024-06-28T15:24:42.723 INFO:tasks.workunit.client.0.smithi178.stderr: -85> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 4 Realm: () 2024-06-28T15:24:42.723 INFO:tasks.workunit.client.0.smithi178.stderr: -84> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:42.723 INFO:tasks.workunit.client.0.smithi178.stderr: -83> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:42.723 INFO:tasks.workunit.client.0.smithi178.stderr: -82> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:42.723 INFO:tasks.workunit.client.0.smithi178.stderr: -81> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 zonegroup default 2024-06-28T15:24:42.723 INFO:tasks.workunit.client.0.smithi178.stderr: -80> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.723 INFO:tasks.workunit.client.0.smithi178.stderr: -79> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.724 INFO:tasks.workunit.client.0.smithi178.stderr: -78> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.724 INFO:tasks.workunit.client.0.smithi178.stderr: -77> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.724 INFO:tasks.workunit.client.0.smithi178.stderr: -76> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.724 INFO:tasks.workunit.client.0.smithi178.stderr: -75> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.724 INFO:tasks.workunit.client.0.smithi178.stderr: -74> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 started sync module instance, tier type = 2024-06-28T15:24:42.724 INFO:tasks.workunit.client.0.smithi178.stderr: -73> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:42.724 INFO:tasks.workunit.client.0.smithi178.stderr: -72> 2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=0 2024-06-28T15:24:42.724 INFO:tasks.workunit.client.0.smithi178.stderr: -71> 2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=1 2024-06-28T15:24:42.724 INFO:tasks.workunit.client.0.smithi178.stderr: -70> 2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=2 2024-06-28T15:24:42.724 INFO:tasks.workunit.client.0.smithi178.stderr: -69> 2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=3 2024-06-28T15:24:42.724 INFO:tasks.workunit.client.0.smithi178.stderr: -68> 2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=4 2024-06-28T15:24:42.724 INFO:tasks.workunit.client.0.smithi178.stderr: -67> 2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=5 2024-06-28T15:24:42.724 INFO:tasks.workunit.client.0.smithi178.stderr: -66> 2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=6 2024-06-28T15:24:42.725 INFO:tasks.workunit.client.0.smithi178.stderr: -65> 2024-06-28T15:24:42.617+0000 7fcea6bc5980 20 add_watcher() i=7 2024-06-28T15:24:42.725 INFO:tasks.workunit.client.0.smithi178.stderr: -64> 2024-06-28T15:24:42.617+0000 7fcea6bc5980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:42.725 INFO:tasks.workunit.client.0.smithi178.stderr: -63> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding auth protocol: cephx 2024-06-28T15:24:42.725 INFO:tasks.workunit.client.0.smithi178.stderr: -62> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding auth protocol: cephx 2024-06-28T15:24:42.725 INFO:tasks.workunit.client.0.smithi178.stderr: -61> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding auth protocol: cephx 2024-06-28T15:24:42.725 INFO:tasks.workunit.client.0.smithi178.stderr: -60> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: secure 2024-06-28T15:24:42.725 INFO:tasks.workunit.client.0.smithi178.stderr: -59> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: crc 2024-06-28T15:24:42.725 INFO:tasks.workunit.client.0.smithi178.stderr: -58> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: secure 2024-06-28T15:24:42.725 INFO:tasks.workunit.client.0.smithi178.stderr: -57> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: crc 2024-06-28T15:24:42.725 INFO:tasks.workunit.client.0.smithi178.stderr: -56> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: secure 2024-06-28T15:24:42.725 INFO:tasks.workunit.client.0.smithi178.stderr: -55> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: crc 2024-06-28T15:24:42.725 INFO:tasks.workunit.client.0.smithi178.stderr: -54> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: crc 2024-06-28T15:24:42.725 INFO:tasks.workunit.client.0.smithi178.stderr: -53> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: secure 2024-06-28T15:24:42.725 INFO:tasks.workunit.client.0.smithi178.stderr: -52> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: crc 2024-06-28T15:24:42.726 INFO:tasks.workunit.client.0.smithi178.stderr: -51> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: secure 2024-06-28T15:24:42.726 INFO:tasks.workunit.client.0.smithi178.stderr: -50> 2024-06-28T15:24:42.621+0000 7fcd5afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:42.726 INFO:tasks.workunit.client.0.smithi178.stderr: -49> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: crc 2024-06-28T15:24:42.726 INFO:tasks.workunit.client.0.smithi178.stderr: -48> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: secure 2024-06-28T15:24:42.726 INFO:tasks.workunit.client.0.smithi178.stderr: -47> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.726 INFO:tasks.workunit.client.0.smithi178.stderr: -46> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:42.726 INFO:tasks.workunit.client.0.smithi178.stderr: -45> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:42.726 INFO:tasks.workunit.client.0.smithi178.stderr: -44> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command cache list hook 0x56328a151bb0 2024-06-28T15:24:42.726 INFO:tasks.workunit.client.0.smithi178.stderr: -43> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command cache inspect hook 0x56328a151bb0 2024-06-28T15:24:42.726 INFO:tasks.workunit.client.0.smithi178.stderr: -42> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command cache erase hook 0x56328a151bb0 2024-06-28T15:24:42.726 INFO:tasks.workunit.client.0.smithi178.stderr: -41> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command cache zap hook 0x56328a151bb0 2024-06-28T15:24:42.726 INFO:tasks.workunit.client.0.smithi178.stderr: -40> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 note: GC not initialized 2024-06-28T15:24:42.726 INFO:tasks.workunit.client.0.smithi178.stderr: -39> 2024-06-28T15:24:42.621+0000 7fcd59fdb640 20 reqs_thread_entry: start 2024-06-28T15:24:42.727 INFO:tasks.workunit.client.0.smithi178.stderr: -38> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command sync trace show hook 0x7fce94112070 2024-06-28T15:24:42.727 INFO:tasks.workunit.client.0.smithi178.stderr: -37> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command sync trace history hook 0x7fce94112070 2024-06-28T15:24:42.727 INFO:tasks.workunit.client.0.smithi178.stderr: -36> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command sync trace active hook 0x7fce94112070 2024-06-28T15:24:42.727 INFO:tasks.workunit.client.0.smithi178.stderr: -35> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command sync trace active_short hook 0x7fce94112070 2024-06-28T15:24:42.727 INFO:tasks.workunit.client.0.smithi178.stderr: -34> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:42.727 INFO:tasks.workunit.client.0.smithi178.stderr: -33> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 20 Filter name: none 2024-06-28T15:24:42.727 INFO:tasks.workunit.client.0.smithi178.stderr: -32> 2024-06-28T15:24:42.625+0000 7fcccb7fe640 20 reqs_thread_entry: start 2024-06-28T15:24:42.727 INFO:tasks.workunit.client.0.smithi178.stderr: -31> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-06-28T15:24:42.727 INFO:tasks.workunit.client.0.smithi178.stderr: -30> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.727 INFO:tasks.workunit.client.0.smithi178.stderr: -29> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:42.727 INFO:tasks.workunit.client.0.smithi178.stderr: -28> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-06-28T15:24:42.727 INFO:tasks.workunit.client.0.smithi178.stderr: -27> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-06-28T15:24:42.727 INFO:tasks.workunit.client.0.smithi178.stderr: -26> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 : miss 2024-06-28T15:24:42.727 INFO:tasks.workunit.client.0.smithi178.stderr: -25> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.728 INFO:tasks.workunit.client.0.smithi178.stderr: -24> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:42.728 INFO:tasks.workunit.client.0.smithi178.stderr: -23> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 info.flags=0x17 2024-06-28T15:24:42.728 INFO:tasks.workunit.client.0.smithi178.stderr: -22> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 to cache LRU end 2024-06-28T15:24:42.728 INFO:tasks.workunit.client.0.smithi178.stderr: -21> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:24:42.728 INFO:tasks.workunit.client.0.smithi178.stderr: -20> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:42.728 INFO:tasks.workunit.client.0.smithi178.stderr: -19> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 2024-06-28T15:24:42.728 INFO:tasks.workunit.client.0.smithi178.stderr: -18> 2024-06-28T15:24:42.633+0000 7fcea6bc5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 info.flags=0x17 2024-06-28T15:24:42.728 INFO:tasks.workunit.client.0.smithi178.stderr: -17> 2024-06-28T15:24:42.633+0000 7fcea6bc5980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 to cache LRU end 2024-06-28T15:24:42.728 INFO:tasks.workunit.client.0.smithi178.stderr: -16> 2024-06-28T15:24:42.633+0000 7fcea6bc5980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:42.728 INFO:tasks.workunit.client.0.smithi178.stderr: -15> 2024-06-28T15:24:42.633+0000 7fcea6bc5980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27, ofs0, ns] 2024-06-28T15:24:42.728 INFO:tasks.workunit.client.0.smithi178.stderr: -14> 2024-06-28T15:24:42.633+0000 7fcd7b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595117 cookie 94775065204384 notifier 6927 bl.length()=1007 2024-06-28T15:24:42.728 INFO:tasks.workunit.client.0.smithi178.stderr: -13> 2024-06-28T15:24:42.633+0000 7fcd7b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 info.flags=0x17 2024-06-28T15:24:42.728 INFO:tasks.workunit.client.0.smithi178.stderr: -12> 2024-06-28T15:24:42.633+0000 7fcd7b7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 to cache LRU end 2024-06-28T15:24:42.729 INFO:tasks.workunit.client.0.smithi178.stderr: -11> 2024-06-28T15:24:42.633+0000 7fcd7b7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:42.729 INFO:tasks.workunit.client.0.smithi178.stderr: -10> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): orig_sources= new_sources= 2024-06-28T15:24:42.729 INFO:tasks.workunit.client.0.smithi178.stderr: -9> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): potential sources added=[] removed=[] 2024-06-28T15:24:42.729 INFO:tasks.workunit.client.0.smithi178.stderr: -8> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): orig_dests= new_dests= 2024-06-28T15:24:42.729 INFO:tasks.workunit.client.0.smithi178.stderr: -7> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): potential dests added=[] removed=[] 2024-06-28T15:24:42.729 INFO:tasks.workunit.client.0.smithi178.stderr: -6> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.0 2024-06-28T15:24:42.729 INFO:tasks.workunit.client.0.smithi178.stderr: -5> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.0 2024-06-28T15:24:42.729 INFO:tasks.workunit.client.0.smithi178.stderr: -4> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.1 2024-06-28T15:24:42.729 INFO:tasks.workunit.client.0.smithi178.stderr: -3> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.1 2024-06-28T15:24:42.729 INFO:tasks.workunit.client.0.smithi178.stderr: -2> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.0 2024-06-28T15:24:42.729 INFO:tasks.workunit.client.0.smithi178.stderr: -1> 2024-06-28T15:24:42.645+0000 7fcea6bc5980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fcea6bc5980 time 2024-06-28T15:24:42.649045+0000 2024-06-28T15:24:42.729 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:24:42.729 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.730 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:24:42.730 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fceaa49cc91] 2024-06-28T15:24:42.730 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fceaa49ce45] 2024-06-28T15:24:42.730 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x563287a014b1] 2024-06-28T15:24:42.730 INFO:tasks.workunit.client.0.smithi178.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x563287a12cf7] 2024-06-28T15:24:42.730 INFO:tasks.workunit.client.0.smithi178.stderr: 5: main() 2024-06-28T15:24:42.730 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fcea9305d90] 2024-06-28T15:24:42.730 INFO:tasks.workunit.client.0.smithi178.stderr: 7: __libc_start_main() 2024-06-28T15:24:42.730 INFO:tasks.workunit.client.0.smithi178.stderr: 8: _start() 2024-06-28T15:24:42.730 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.730 INFO:tasks.workunit.client.0.smithi178.stderr: 0> 2024-06-28T15:24:42.649+0000 7fcea6bc5980 -1 *** Caught signal (Aborted) ** 2024-06-28T15:24:42.730 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7fcea6bc5980 thread_name:radosgw-admin 2024-06-28T15:24:42.730 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.731 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:24:42.731 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fcea931e520] 2024-06-28T15:24:42.731 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:24:42.731 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:24:42.731 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:24:42.731 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fceaa49cceb] 2024-06-28T15:24:42.731 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fceaa49ce45] 2024-06-28T15:24:42.731 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x563287a014b1] 2024-06-28T15:24:42.731 INFO:tasks.workunit.client.0.smithi178.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x563287a12cf7] 2024-06-28T15:24:42.731 INFO:tasks.workunit.client.0.smithi178.stderr: 9: main() 2024-06-28T15:24:42.731 INFO:tasks.workunit.client.0.smithi178.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fcea9305d90] 2024-06-28T15:24:42.731 INFO:tasks.workunit.client.0.smithi178.stderr: 11: __libc_start_main() 2024-06-28T15:24:42.731 INFO:tasks.workunit.client.0.smithi178.stderr: 12: _start() 2024-06-28T15:24:42.731 INFO:tasks.workunit.client.0.smithi178.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr:--- logging levels --- 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 none 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 lockdep 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 context 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 crush 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_balancer 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_locker 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log_expire 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_migrator 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr: 3/ 5 mds_quiesce 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 buffer 2024-06-28T15:24:42.732 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 timer 2024-06-28T15:24:42.733 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 filer 2024-06-28T15:24:42.733 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 striper 2024-06-28T15:24:42.733 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 objecter 2024-06-28T15:24:42.733 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rados 2024-06-28T15:24:42.733 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd 2024-06-28T15:24:42.733 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_mirror 2024-06-28T15:24:42.733 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_replay 2024-06-28T15:24:42.733 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_pwl 2024-06-28T15:24:42.733 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 journaler 2024-06-28T15:24:42.733 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objectcacher 2024-06-28T15:24:42.733 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 immutable_obj_cache 2024-06-28T15:24:42.733 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 client 2024-06-28T15:24:42.733 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 osd 2024-06-28T15:24:42.733 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 optracker 2024-06-28T15:24:42.734 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objclass 2024-06-28T15:24:42.734 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 filestore 2024-06-28T15:24:42.734 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 journal 2024-06-28T15:24:42.734 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 ms 2024-06-28T15:24:42.734 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mon 2024-06-28T15:24:42.734 INFO:tasks.workunit.client.0.smithi178.stderr: 0/10 monc 2024-06-28T15:24:42.734 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 paxos 2024-06-28T15:24:42.734 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 tp 2024-06-28T15:24:42.734 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 auth 2024-06-28T15:24:42.734 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 crypto 2024-06-28T15:24:42.734 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 finisher 2024-06-28T15:24:42.734 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 reserver 2024-06-28T15:24:42.734 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 heartbeatmap 2024-06-28T15:24:42.735 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 perfcounter 2024-06-28T15:24:42.735 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw 2024-06-28T15:24:42.735 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_sync 2024-06-28T15:24:42.735 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_datacache 2024-06-28T15:24:42.735 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_access 2024-06-28T15:24:42.735 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_dbstore 2024-06-28T15:24:42.735 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_flight 2024-06-28T15:24:42.735 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_lifecycle 2024-06-28T15:24:42.735 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw_notification 2024-06-28T15:24:42.735 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 javaclient 2024-06-28T15:24:42.735 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 asok 2024-06-28T15:24:42.735 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 throttle 2024-06-28T15:24:42.735 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 refs 2024-06-28T15:24:42.735 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 compressor 2024-06-28T15:24:42.736 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluestore 2024-06-28T15:24:42.736 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluefs 2024-06-28T15:24:42.736 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 bdev 2024-06-28T15:24:42.736 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 kstore 2024-06-28T15:24:42.736 INFO:tasks.workunit.client.0.smithi178.stderr: 4/ 5 rocksdb 2024-06-28T15:24:42.736 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 fuse 2024-06-28T15:24:42.736 INFO:tasks.workunit.client.0.smithi178.stderr: 2/ 5 mgr 2024-06-28T15:24:42.736 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mgrc 2024-06-28T15:24:42.736 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 dpdk 2024-06-28T15:24:42.736 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 eventtrace 2024-06-28T15:24:42.736 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 prioritycache 2024-06-28T15:24:42.736 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 test 2024-06-28T15:24:42.736 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephfs_mirror 2024-06-28T15:24:42.736 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephsqlite 2024-06-28T15:24:42.737 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 crimson_interrupt 2024-06-28T15:24:42.737 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore 2024-06-28T15:24:42.737 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_onode 2024-06-28T15:24:42.737 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_odata 2024-06-28T15:24:42.737 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_omap 2024-06-28T15:24:42.737 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_tm 2024-06-28T15:24:42.737 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_t 2024-06-28T15:24:42.737 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cleaner 2024-06-28T15:24:42.737 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_epm 2024-06-28T15:24:42.737 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_lba 2024-06-28T15:24:42.737 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_fixedkv_tree 2024-06-28T15:24:42.737 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cache 2024-06-28T15:24:42.737 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_journal 2024-06-28T15:24:42.737 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_device 2024-06-28T15:24:42.738 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_backref 2024-06-28T15:24:42.738 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 alienstore 2024-06-28T15:24:42.738 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mclock 2024-06-28T15:24:42.738 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cyanstore 2024-06-28T15:24:42.738 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 ceph_exporter 2024-06-28T15:24:42.738 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 memstore 2024-06-28T15:24:42.738 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 trace 2024-06-28T15:24:42.738 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 ceph_dedup 2024-06-28T15:24:42.738 INFO:tasks.workunit.client.0.smithi178.stderr: -2/-2 (syslog threshold) 2024-06-28T15:24:42.738 INFO:tasks.workunit.client.0.smithi178.stderr: 99/99 (stderr threshold) 2024-06-28T15:24:42.738 INFO:tasks.workunit.client.0.smithi178.stderr:--- pthread ID / name mapping for recent threads --- 2024-06-28T15:24:42.738 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcccb7fe640 / http_manager 2024-06-28T15:24:42.738 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcd59fdb640 / http_manager 2024-06-28T15:24:42.738 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcd5afdd640 / rgw_dt_lg_renew 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcd79ffb640 / ms_dispatch 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcd7b7fe640 / io_context_pool 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcd89ffb640 / ms_dispatch 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcd90e09640 / ms_dispatch 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcd9260c640 / io_context_pool 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcea3ed5640 / msgr-worker-2 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcea46d6640 / msgr-worker-1 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcea4ed7640 / msgr-worker-0 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcea56d8640 / io_context_pool 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcea6bc5980 / radosgw-admin 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr: max_recent 500 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr: max_new 1000 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr: log_file /var/log/ceph/ceph-client.admin.96661.log 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr:--- end dump of recent events --- 2024-06-28T15:24:42.739 INFO:tasks.workunit.client.0.smithi178.stderr:--- begin dump of recent events --- 2024-06-28T15:24:42.740 INFO:tasks.workunit.client.0.smithi178.stderr: -499> 2024-06-28T15:24:42.329+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.740 INFO:tasks.workunit.client.0.smithi178.stderr: -498> 2024-06-28T15:24:42.329+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.740 INFO:tasks.workunit.client.0.smithi178.stderr: -497> 2024-06-28T15:24:42.329+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.740 INFO:tasks.workunit.client.0.smithi178.stderr: -496> 2024-06-28T15:24:42.329+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_done global_id 6491 payload 274 2024-06-28T15:24:42.740 INFO:tasks.workunit.client.0.smithi178.stderr: -495> 2024-06-28T15:24:42.329+0000 7fcea4ed7640 10 monclient: _finish_hunting 0 2024-06-28T15:24:42.740 INFO:tasks.workunit.client.0.smithi178.stderr: -494> 2024-06-28T15:24:42.329+0000 7fcea4ed7640 1 monclient: found mon.noname-c 2024-06-28T15:24:42.740 INFO:tasks.workunit.client.0.smithi178.stderr: -493> 2024-06-28T15:24:42.329+0000 7fcea4ed7640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:24:42.740 INFO:tasks.workunit.client.0.smithi178.stderr: -492> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.740 INFO:tasks.workunit.client.0.smithi178.stderr: -491> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:24:42.740 INFO:tasks.workunit.client.0.smithi178.stderr: -490> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: dump: 2024-06-28T15:24:42.740 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.740 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.740 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.741 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.741 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.741 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.741 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.741 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.741 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.741 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.741 INFO:tasks.workunit.client.0.smithi178.stderr: -489> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: _finish_auth 0 2024-06-28T15:24:42.741 INFO:tasks.workunit.client.0.smithi178.stderr: -488> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: _check_auth_tickets 2024-06-28T15:24:42.741 INFO:tasks.workunit.client.0.smithi178.stderr: -487> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:42.741 INFO:tasks.workunit.client.0.smithi178.stderr: -486> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.741 INFO:tasks.workunit.client.0.smithi178.stderr: -485> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:24:42.741 INFO:tasks.workunit.client.0.smithi178.stderr: -484> 2024-06-28T15:24:42.329+0000 7fcd9260c640 10 monclient: dump: 2024-06-28T15:24:42.741 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.742 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.742 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.742 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.742 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.742 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.742 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.742 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.742 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.742 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.742 INFO:tasks.workunit.client.0.smithi178.stderr: -483> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 monclient: authenticate success, global_id 6491 2024-06-28T15:24:42.742 INFO:tasks.workunit.client.0.smithi178.stderr: -482> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 10 monclient: get_monmap_and_config success 2024-06-28T15:24:42.742 INFO:tasks.workunit.client.0.smithi178.stderr: -481> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 4 set_mon_vals no callback set 2024-06-28T15:24:42.742 INFO:tasks.workunit.client.0.smithi178.stderr: -480> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 10 monclient: shutdown 2024-06-28T15:24:42.742 INFO:tasks.workunit.client.0.smithi178.stderr: -479> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 asok(0x563289e825e0) unregister_commands rotate-key 2024-06-28T15:24:42.743 INFO:tasks.workunit.client.0.smithi178.stderr: -478> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 10 monclient: build_initial_monmap 2024-06-28T15:24:42.743 INFO:tasks.workunit.client.0.smithi178.stderr: -477> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 1 build_initial for_mkfs: 0 2024-06-28T15:24:42.743 INFO:tasks.workunit.client.0.smithi178.stderr: -476> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 10 monclient: monmap: 2024-06-28T15:24:42.743 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:42.743 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:42.743 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:42.743 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:42.743 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:42.743 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.743 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:42.743 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:42.743 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:42.743 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.743 INFO:tasks.workunit.client.0.smithi178.stderr: -475> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding auth protocol: cephx 2024-06-28T15:24:42.744 INFO:tasks.workunit.client.0.smithi178.stderr: -474> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding auth protocol: cephx 2024-06-28T15:24:42.744 INFO:tasks.workunit.client.0.smithi178.stderr: -473> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding auth protocol: cephx 2024-06-28T15:24:42.744 INFO:tasks.workunit.client.0.smithi178.stderr: -472> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: secure 2024-06-28T15:24:42.744 INFO:tasks.workunit.client.0.smithi178.stderr: -471> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: crc 2024-06-28T15:24:42.744 INFO:tasks.workunit.client.0.smithi178.stderr: -470> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: secure 2024-06-28T15:24:42.744 INFO:tasks.workunit.client.0.smithi178.stderr: -469> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: crc 2024-06-28T15:24:42.744 INFO:tasks.workunit.client.0.smithi178.stderr: -468> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: secure 2024-06-28T15:24:42.744 INFO:tasks.workunit.client.0.smithi178.stderr: -467> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: crc 2024-06-28T15:24:42.744 INFO:tasks.workunit.client.0.smithi178.stderr: -466> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: crc 2024-06-28T15:24:42.744 INFO:tasks.workunit.client.0.smithi178.stderr: -465> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: secure 2024-06-28T15:24:42.744 INFO:tasks.workunit.client.0.smithi178.stderr: -464> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: crc 2024-06-28T15:24:42.744 INFO:tasks.workunit.client.0.smithi178.stderr: -463> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: secure 2024-06-28T15:24:42.744 INFO:tasks.workunit.client.0.smithi178.stderr: -462> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: crc 2024-06-28T15:24:42.744 INFO:tasks.workunit.client.0.smithi178.stderr: -461> 2024-06-28T15:24:42.329+0000 7fcea6bc5980 5 AuthRegistry(0x56328a091570) adding con mode: secure 2024-06-28T15:24:42.745 INFO:tasks.workunit.client.0.smithi178.stderr: -460> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.745 INFO:tasks.workunit.client.0.smithi178.stderr: -459> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 1 librados: starting msgr at 2024-06-28T15:24:42.745 INFO:tasks.workunit.client.0.smithi178.stderr: -458> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 1 librados: starting objecter 2024-06-28T15:24:42.745 INFO:tasks.workunit.client.0.smithi178.stderr: -457> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command objecter_requests hook 0x563289e62090 2024-06-28T15:24:42.745 INFO:tasks.workunit.client.0.smithi178.stderr: -456> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 1 librados: setting wanted keys 2024-06-28T15:24:42.745 INFO:tasks.workunit.client.0.smithi178.stderr: -455> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 1 librados: calling monclient init 2024-06-28T15:24:42.745 INFO:tasks.workunit.client.0.smithi178.stderr: -454> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient: init 2024-06-28T15:24:42.745 INFO:tasks.workunit.client.0.smithi178.stderr: -453> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding auth protocol: cephx 2024-06-28T15:24:42.745 INFO:tasks.workunit.client.0.smithi178.stderr: -452> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding auth protocol: cephx 2024-06-28T15:24:42.745 INFO:tasks.workunit.client.0.smithi178.stderr: -451> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding auth protocol: cephx 2024-06-28T15:24:42.745 INFO:tasks.workunit.client.0.smithi178.stderr: -450> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: secure 2024-06-28T15:24:42.745 INFO:tasks.workunit.client.0.smithi178.stderr: -449> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: crc 2024-06-28T15:24:42.745 INFO:tasks.workunit.client.0.smithi178.stderr: -448> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: secure 2024-06-28T15:24:42.746 INFO:tasks.workunit.client.0.smithi178.stderr: -447> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: crc 2024-06-28T15:24:42.746 INFO:tasks.workunit.client.0.smithi178.stderr: -446> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: secure 2024-06-28T15:24:42.746 INFO:tasks.workunit.client.0.smithi178.stderr: -445> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: crc 2024-06-28T15:24:42.746 INFO:tasks.workunit.client.0.smithi178.stderr: -444> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: crc 2024-06-28T15:24:42.746 INFO:tasks.workunit.client.0.smithi178.stderr: -443> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: secure 2024-06-28T15:24:42.746 INFO:tasks.workunit.client.0.smithi178.stderr: -442> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: crc 2024-06-28T15:24:42.746 INFO:tasks.workunit.client.0.smithi178.stderr: -441> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: secure 2024-06-28T15:24:42.746 INFO:tasks.workunit.client.0.smithi178.stderr: -440> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: crc 2024-06-28T15:24:42.746 INFO:tasks.workunit.client.0.smithi178.stderr: -439> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 AuthRegistry(0x56328a08f780) adding con mode: secure 2024-06-28T15:24:42.746 INFO:tasks.workunit.client.0.smithi178.stderr: -438> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.746 INFO:tasks.workunit.client.0.smithi178.stderr: -437> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.746 INFO:tasks.workunit.client.0.smithi178.stderr: -436> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command rotate-key hook 0x56328a08f8c8 2024-06-28T15:24:42.746 INFO:tasks.workunit.client.0.smithi178.stderr: -435> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:42.746 INFO:tasks.workunit.client.0.smithi178.stderr: -434> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient: _add_conns ranks=[2,1,0] 2024-06-28T15:24:42.747 INFO:tasks.workunit.client.0.smithi178.stderr: -433> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-c con 0x563289f3fd30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:42.747 INFO:tasks.workunit.client.0.smithi178.stderr: -432> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-b con 0x563289f49d80 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:42.747 INFO:tasks.workunit.client.0.smithi178.stderr: -431> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-a con 0x56328a0ba2d0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:42.747 INFO:tasks.workunit.client.0.smithi178.stderr: -430> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.747 INFO:tasks.workunit.client.0.smithi178.stderr: -429> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.747 INFO:tasks.workunit.client.0.smithi178.stderr: -428> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.747 INFO:tasks.workunit.client.0.smithi178.stderr: -427> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:42.747 INFO:tasks.workunit.client.0.smithi178.stderr: -426> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient(hunting): authenticate will time out at 2425.796960s 2024-06-28T15:24:42.747 INFO:tasks.workunit.client.0.smithi178.stderr: -425> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): get_auth_request con 0x563289f3fd30 auth_method 0 2024-06-28T15:24:42.747 INFO:tasks.workunit.client.0.smithi178.stderr: -424> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.747 INFO:tasks.workunit.client.0.smithi178.stderr: -423> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.747 INFO:tasks.workunit.client.0.smithi178.stderr: -422> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.747 INFO:tasks.workunit.client.0.smithi178.stderr: -421> 2024-06-28T15:24:42.333+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request con 0x56328a0ba2d0 auth_method 0 2024-06-28T15:24:42.748 INFO:tasks.workunit.client.0.smithi178.stderr: -420> 2024-06-28T15:24:42.333+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.748 INFO:tasks.workunit.client.0.smithi178.stderr: -419> 2024-06-28T15:24:42.333+0000 7fcea4ed7640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.748 INFO:tasks.workunit.client.0.smithi178.stderr: -418> 2024-06-28T15:24:42.333+0000 7fcea4ed7640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.748 INFO:tasks.workunit.client.0.smithi178.stderr: -417> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.748 INFO:tasks.workunit.client.0.smithi178.stderr: -416> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.748 INFO:tasks.workunit.client.0.smithi178.stderr: -415> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.748 INFO:tasks.workunit.client.0.smithi178.stderr: -414> 2024-06-28T15:24:42.333+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.748 INFO:tasks.workunit.client.0.smithi178.stderr: -413> 2024-06-28T15:24:42.333+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.748 INFO:tasks.workunit.client.0.smithi178.stderr: -412> 2024-06-28T15:24:42.333+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.748 INFO:tasks.workunit.client.0.smithi178.stderr: -411> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient(hunting): handle_auth_done global_id 6494 payload 824 2024-06-28T15:24:42.748 INFO:tasks.workunit.client.0.smithi178.stderr: -410> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient: _finish_hunting 0 2024-06-28T15:24:42.748 INFO:tasks.workunit.client.0.smithi178.stderr: -409> 2024-06-28T15:24:42.333+0000 7fcea46d6640 1 monclient: found mon.noname-c 2024-06-28T15:24:42.748 INFO:tasks.workunit.client.0.smithi178.stderr: -408> 2024-06-28T15:24:42.333+0000 7fcea46d6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:24:42.748 INFO:tasks.workunit.client.0.smithi178.stderr: -407> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.749 INFO:tasks.workunit.client.0.smithi178.stderr: -406> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:24:42.749 INFO:tasks.workunit.client.0.smithi178.stderr: -405> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: dump: 2024-06-28T15:24:42.749 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.749 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.749 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.749 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.749 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.749 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.749 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.749 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.749 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.749 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.749 INFO:tasks.workunit.client.0.smithi178.stderr: -404> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: _finish_auth 0 2024-06-28T15:24:42.749 INFO:tasks.workunit.client.0.smithi178.stderr: -403> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: _check_auth_tickets 2024-06-28T15:24:42.750 INFO:tasks.workunit.client.0.smithi178.stderr: -402> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:42.750 INFO:tasks.workunit.client.0.smithi178.stderr: -401> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.750 INFO:tasks.workunit.client.0.smithi178.stderr: -400> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:24:42.750 INFO:tasks.workunit.client.0.smithi178.stderr: -399> 2024-06-28T15:24:42.333+0000 7fcd90e09640 10 monclient: dump: 2024-06-28T15:24:42.750 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.750 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.750 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.750 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.750 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.750 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.750 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.750 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.750 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.751 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.751 INFO:tasks.workunit.client.0.smithi178.stderr: -398> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 5 monclient: authenticate success, global_id 6494 2024-06-28T15:24:42.751 INFO:tasks.workunit.client.0.smithi178.stderr: -397> 2024-06-28T15:24:42.333+0000 7fcd9260c640 4 set_mon_vals no callback set 2024-06-28T15:24:42.751 INFO:tasks.workunit.client.0.smithi178.stderr: -396> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient: _renew_subs 2024-06-28T15:24:42.751 INFO:tasks.workunit.client.0.smithi178.stderr: -395> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:42.751 INFO:tasks.workunit.client.0.smithi178.stderr: -394> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient: _renew_subs 2024-06-28T15:24:42.751 INFO:tasks.workunit.client.0.smithi178.stderr: -393> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:24:42.751 INFO:tasks.workunit.client.0.smithi178.stderr: -392> 2024-06-28T15:24:42.333+0000 7fcea6bc5980 1 librados: init done 2024-06-28T15:24:42.751 INFO:tasks.workunit.client.0.smithi178.stderr: -391> 2024-06-28T15:24:42.333+0000 7fcd90e09640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:24:42.751 INFO:tasks.workunit.client.0.smithi178.stderr: -390> 2024-06-28T15:24:42.333+0000 7fcd90e09640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:42.751 INFO:tasks.workunit.client.0.smithi178.stderr: -389> 2024-06-28T15:24:42.333+0000 7fcd90e09640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:42.751 INFO:tasks.workunit.client.0.smithi178.stderr: -388> 2024-06-28T15:24:42.333+0000 7fcea3ed5640 10 monclient: get_auth_request con 0x7fcd8c06f780 auth_method 0 2024-06-28T15:24:42.751 INFO:tasks.workunit.client.0.smithi178.stderr: -387> 2024-06-28T15:24:42.337+0000 7fcea4ed7640 10 monclient: get_auth_request con 0x56328a0b7ea0 auth_method 0 2024-06-28T15:24:42.751 INFO:tasks.workunit.client.0.smithi178.stderr: -386> 2024-06-28T15:24:42.337+0000 7fcea46d6640 10 monclient: get_auth_request con 0x56328a0da740 auth_method 0 2024-06-28T15:24:42.752 INFO:tasks.workunit.client.0.smithi178.stderr: -385> 2024-06-28T15:24:42.337+0000 7fcea3ed5640 10 monclient: get_auth_request con 0x56328a0deaa0 auth_method 0 2024-06-28T15:24:42.752 INFO:tasks.workunit.client.0.smithi178.stderr: -384> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: build_initial_monmap 2024-06-28T15:24:42.752 INFO:tasks.workunit.client.0.smithi178.stderr: -383> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 1 build_initial for_mkfs: 0 2024-06-28T15:24:42.752 INFO:tasks.workunit.client.0.smithi178.stderr: -382> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: monmap: 2024-06-28T15:24:42.752 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:42.752 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:42.752 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:42.752 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:42.752 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:42.752 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.752 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:42.752 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:42.752 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:42.752 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.753 INFO:tasks.workunit.client.0.smithi178.stderr: -381> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding auth protocol: cephx 2024-06-28T15:24:42.753 INFO:tasks.workunit.client.0.smithi178.stderr: -380> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding auth protocol: cephx 2024-06-28T15:24:42.753 INFO:tasks.workunit.client.0.smithi178.stderr: -379> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding auth protocol: cephx 2024-06-28T15:24:42.753 INFO:tasks.workunit.client.0.smithi178.stderr: -378> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: secure 2024-06-28T15:24:42.753 INFO:tasks.workunit.client.0.smithi178.stderr: -377> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: crc 2024-06-28T15:24:42.753 INFO:tasks.workunit.client.0.smithi178.stderr: -376> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: secure 2024-06-28T15:24:42.753 INFO:tasks.workunit.client.0.smithi178.stderr: -375> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: crc 2024-06-28T15:24:42.753 INFO:tasks.workunit.client.0.smithi178.stderr: -374> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: secure 2024-06-28T15:24:42.753 INFO:tasks.workunit.client.0.smithi178.stderr: -373> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: crc 2024-06-28T15:24:42.753 INFO:tasks.workunit.client.0.smithi178.stderr: -372> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: crc 2024-06-28T15:24:42.753 INFO:tasks.workunit.client.0.smithi178.stderr: -371> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: secure 2024-06-28T15:24:42.753 INFO:tasks.workunit.client.0.smithi178.stderr: -370> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: crc 2024-06-28T15:24:42.753 INFO:tasks.workunit.client.0.smithi178.stderr: -369> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: secure 2024-06-28T15:24:42.753 INFO:tasks.workunit.client.0.smithi178.stderr: -368> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: crc 2024-06-28T15:24:42.754 INFO:tasks.workunit.client.0.smithi178.stderr: -367> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e3940) adding con mode: secure 2024-06-28T15:24:42.754 INFO:tasks.workunit.client.0.smithi178.stderr: -366> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.754 INFO:tasks.workunit.client.0.smithi178.stderr: -365> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command objecter_requests cmddesc objecter_requests hook 0x56328a0e6ec0 EEXIST 2024-06-28T15:24:42.754 INFO:tasks.workunit.client.0.smithi178.stderr: -364> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: init 2024-06-28T15:24:42.754 INFO:tasks.workunit.client.0.smithi178.stderr: -363> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding auth protocol: cephx 2024-06-28T15:24:42.754 INFO:tasks.workunit.client.0.smithi178.stderr: -362> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding auth protocol: cephx 2024-06-28T15:24:42.754 INFO:tasks.workunit.client.0.smithi178.stderr: -361> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding auth protocol: cephx 2024-06-28T15:24:42.754 INFO:tasks.workunit.client.0.smithi178.stderr: -360> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: secure 2024-06-28T15:24:42.754 INFO:tasks.workunit.client.0.smithi178.stderr: -359> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: crc 2024-06-28T15:24:42.754 INFO:tasks.workunit.client.0.smithi178.stderr: -358> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: secure 2024-06-28T15:24:42.754 INFO:tasks.workunit.client.0.smithi178.stderr: -357> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: crc 2024-06-28T15:24:42.754 INFO:tasks.workunit.client.0.smithi178.stderr: -356> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: secure 2024-06-28T15:24:42.754 INFO:tasks.workunit.client.0.smithi178.stderr: -355> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: crc 2024-06-28T15:24:42.754 INFO:tasks.workunit.client.0.smithi178.stderr: -354> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: crc 2024-06-28T15:24:42.755 INFO:tasks.workunit.client.0.smithi178.stderr: -353> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: secure 2024-06-28T15:24:42.755 INFO:tasks.workunit.client.0.smithi178.stderr: -352> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: crc 2024-06-28T15:24:42.755 INFO:tasks.workunit.client.0.smithi178.stderr: -351> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: secure 2024-06-28T15:24:42.755 INFO:tasks.workunit.client.0.smithi178.stderr: -350> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: crc 2024-06-28T15:24:42.755 INFO:tasks.workunit.client.0.smithi178.stderr: -349> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 AuthRegistry(0x56328a0e2018) adding con mode: secure 2024-06-28T15:24:42.755 INFO:tasks.workunit.client.0.smithi178.stderr: -348> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.755 INFO:tasks.workunit.client.0.smithi178.stderr: -347> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.755 INFO:tasks.workunit.client.0.smithi178.stderr: -346> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command rotate-key cmddesc rotate-key hook 0x56328a0e2160 EEXIST 2024-06-28T15:24:42.755 INFO:tasks.workunit.client.0.smithi178.stderr: -345> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:42.755 INFO:tasks.workunit.client.0.smithi178.stderr: -344> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: _add_conns ranks=[0,1,2] 2024-06-28T15:24:42.755 INFO:tasks.workunit.client.0.smithi178.stderr: -343> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-a con 0x56328a0ed450 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:42.755 INFO:tasks.workunit.client.0.smithi178.stderr: -342> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-b con 0x56328a0edd80 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:42.755 INFO:tasks.workunit.client.0.smithi178.stderr: -341> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-c con 0x56328a0ed830 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:42.756 INFO:tasks.workunit.client.0.smithi178.stderr: -340> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.756 INFO:tasks.workunit.client.0.smithi178.stderr: -339> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.756 INFO:tasks.workunit.client.0.smithi178.stderr: -338> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.756 INFO:tasks.workunit.client.0.smithi178.stderr: -337> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:42.756 INFO:tasks.workunit.client.0.smithi178.stderr: -336> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient(hunting): authenticate will time out at 2425.806415s 2024-06-28T15:24:42.756 INFO:tasks.workunit.client.0.smithi178.stderr: -335> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): get_auth_request con 0x56328a0ed450 auth_method 0 2024-06-28T15:24:42.756 INFO:tasks.workunit.client.0.smithi178.stderr: -334> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.756 INFO:tasks.workunit.client.0.smithi178.stderr: -333> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.756 INFO:tasks.workunit.client.0.smithi178.stderr: -332> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.756 INFO:tasks.workunit.client.0.smithi178.stderr: -331> 2024-06-28T15:24:42.341+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request con 0x56328a0ed830 auth_method 0 2024-06-28T15:24:42.756 INFO:tasks.workunit.client.0.smithi178.stderr: -330> 2024-06-28T15:24:42.341+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.756 INFO:tasks.workunit.client.0.smithi178.stderr: -329> 2024-06-28T15:24:42.341+0000 7fcea4ed7640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.756 INFO:tasks.workunit.client.0.smithi178.stderr: -328> 2024-06-28T15:24:42.341+0000 7fcea4ed7640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.757 INFO:tasks.workunit.client.0.smithi178.stderr: -327> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.757 INFO:tasks.workunit.client.0.smithi178.stderr: -326> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.757 INFO:tasks.workunit.client.0.smithi178.stderr: -325> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.757 INFO:tasks.workunit.client.0.smithi178.stderr: -324> 2024-06-28T15:24:42.341+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.757 INFO:tasks.workunit.client.0.smithi178.stderr: -323> 2024-06-28T15:24:42.341+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.757 INFO:tasks.workunit.client.0.smithi178.stderr: -322> 2024-06-28T15:24:42.341+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.757 INFO:tasks.workunit.client.0.smithi178.stderr: -321> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient(hunting): handle_auth_done global_id 6921 payload 824 2024-06-28T15:24:42.757 INFO:tasks.workunit.client.0.smithi178.stderr: -320> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient: _finish_hunting 0 2024-06-28T15:24:42.757 INFO:tasks.workunit.client.0.smithi178.stderr: -319> 2024-06-28T15:24:42.341+0000 7fcea46d6640 1 monclient: found mon.noname-a 2024-06-28T15:24:42.757 INFO:tasks.workunit.client.0.smithi178.stderr: -318> 2024-06-28T15:24:42.341+0000 7fcea46d6640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:24:42.757 INFO:tasks.workunit.client.0.smithi178.stderr: -317> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.757 INFO:tasks.workunit.client.0.smithi178.stderr: -316> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:24:42.757 INFO:tasks.workunit.client.0.smithi178.stderr: -315> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: dump: 2024-06-28T15:24:42.758 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.758 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.758 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.758 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.758 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.758 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.758 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.758 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.758 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.758 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.758 INFO:tasks.workunit.client.0.smithi178.stderr: -314> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: _finish_auth 0 2024-06-28T15:24:42.758 INFO:tasks.workunit.client.0.smithi178.stderr: -313> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: _check_auth_tickets 2024-06-28T15:24:42.758 INFO:tasks.workunit.client.0.smithi178.stderr: -312> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 5 monclient: authenticate success, global_id 6921 2024-06-28T15:24:42.758 INFO:tasks.workunit.client.0.smithi178.stderr: -311> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:42.759 INFO:tasks.workunit.client.0.smithi178.stderr: -310> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.759 INFO:tasks.workunit.client.0.smithi178.stderr: -309> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:24:42.759 INFO:tasks.workunit.client.0.smithi178.stderr: -308> 2024-06-28T15:24:42.341+0000 7fcd89ffb640 10 monclient: dump: 2024-06-28T15:24:42.759 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.759 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.759 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.759 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.759 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.759 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.759 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.759 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.759 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.759 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.759 INFO:tasks.workunit.client.0.smithi178.stderr: -307> 2024-06-28T15:24:42.341+0000 7fcea56d8640 4 set_mon_vals no callback set 2024-06-28T15:24:42.760 INFO:tasks.workunit.client.0.smithi178.stderr: -306> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: _renew_subs 2024-06-28T15:24:42.760 INFO:tasks.workunit.client.0.smithi178.stderr: -305> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:24:42.760 INFO:tasks.workunit.client.0.smithi178.stderr: -304> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: _renew_subs 2024-06-28T15:24:42.760 INFO:tasks.workunit.client.0.smithi178.stderr: -303> 2024-06-28T15:24:42.341+0000 7fcea6bc5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:24:42.760 INFO:tasks.workunit.client.0.smithi178.stderr: -302> 2024-06-28T15:24:42.345+0000 7fcd89ffb640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:24:42.760 INFO:tasks.workunit.client.0.smithi178.stderr: -301> 2024-06-28T15:24:42.345+0000 7fcd89ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:42.760 INFO:tasks.workunit.client.0.smithi178.stderr: -300> 2024-06-28T15:24:42.345+0000 7fcd89ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:42.760 INFO:tasks.workunit.client.0.smithi178.stderr: -299> 2024-06-28T15:24:42.345+0000 7fcea3ed5640 10 monclient: get_auth_request con 0x7fcd7c06f690 auth_method 0 2024-06-28T15:24:42.760 INFO:tasks.workunit.client.0.smithi178.stderr: -298> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient: get_monmap_and_config 2024-06-28T15:24:42.760 INFO:tasks.workunit.client.0.smithi178.stderr: -297> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient: build_initial_monmap 2024-06-28T15:24:42.760 INFO:tasks.workunit.client.0.smithi178.stderr: -296> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 1 build_initial for_mkfs: 0 2024-06-28T15:24:42.760 INFO:tasks.workunit.client.0.smithi178.stderr: -295> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient: monmap: 2024-06-28T15:24:42.760 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:42.761 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:42.761 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:42.761 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:42.761 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:42.761 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.761 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:42.761 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:42.761 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:42.761 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.761 INFO:tasks.workunit.client.0.smithi178.stderr: -294> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding auth protocol: cephx 2024-06-28T15:24:42.761 INFO:tasks.workunit.client.0.smithi178.stderr: -293> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding auth protocol: cephx 2024-06-28T15:24:42.761 INFO:tasks.workunit.client.0.smithi178.stderr: -292> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding auth protocol: cephx 2024-06-28T15:24:42.761 INFO:tasks.workunit.client.0.smithi178.stderr: -291> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: secure 2024-06-28T15:24:42.761 INFO:tasks.workunit.client.0.smithi178.stderr: -290> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: crc 2024-06-28T15:24:42.762 INFO:tasks.workunit.client.0.smithi178.stderr: -289> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: secure 2024-06-28T15:24:42.762 INFO:tasks.workunit.client.0.smithi178.stderr: -288> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: crc 2024-06-28T15:24:42.762 INFO:tasks.workunit.client.0.smithi178.stderr: -287> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: secure 2024-06-28T15:24:42.762 INFO:tasks.workunit.client.0.smithi178.stderr: -286> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: crc 2024-06-28T15:24:42.762 INFO:tasks.workunit.client.0.smithi178.stderr: -285> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: crc 2024-06-28T15:24:42.762 INFO:tasks.workunit.client.0.smithi178.stderr: -284> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: secure 2024-06-28T15:24:42.762 INFO:tasks.workunit.client.0.smithi178.stderr: -283> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: crc 2024-06-28T15:24:42.762 INFO:tasks.workunit.client.0.smithi178.stderr: -282> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: secure 2024-06-28T15:24:42.762 INFO:tasks.workunit.client.0.smithi178.stderr: -281> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: crc 2024-06-28T15:24:42.762 INFO:tasks.workunit.client.0.smithi178.stderr: -280> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12ac30) adding con mode: secure 2024-06-28T15:24:42.762 INFO:tasks.workunit.client.0.smithi178.stderr: -279> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.762 INFO:tasks.workunit.client.0.smithi178.stderr: -278> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient: init 2024-06-28T15:24:42.762 INFO:tasks.workunit.client.0.smithi178.stderr: -277> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding auth protocol: cephx 2024-06-28T15:24:42.763 INFO:tasks.workunit.client.0.smithi178.stderr: -276> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding auth protocol: cephx 2024-06-28T15:24:42.763 INFO:tasks.workunit.client.0.smithi178.stderr: -275> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding auth protocol: cephx 2024-06-28T15:24:42.763 INFO:tasks.workunit.client.0.smithi178.stderr: -274> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: secure 2024-06-28T15:24:42.763 INFO:tasks.workunit.client.0.smithi178.stderr: -273> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: crc 2024-06-28T15:24:42.763 INFO:tasks.workunit.client.0.smithi178.stderr: -272> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: secure 2024-06-28T15:24:42.763 INFO:tasks.workunit.client.0.smithi178.stderr: -271> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: crc 2024-06-28T15:24:42.763 INFO:tasks.workunit.client.0.smithi178.stderr: -270> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: secure 2024-06-28T15:24:42.763 INFO:tasks.workunit.client.0.smithi178.stderr: -269> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: crc 2024-06-28T15:24:42.763 INFO:tasks.workunit.client.0.smithi178.stderr: -268> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: crc 2024-06-28T15:24:42.763 INFO:tasks.workunit.client.0.smithi178.stderr: -267> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: secure 2024-06-28T15:24:42.763 INFO:tasks.workunit.client.0.smithi178.stderr: -266> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: crc 2024-06-28T15:24:42.763 INFO:tasks.workunit.client.0.smithi178.stderr: -265> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: secure 2024-06-28T15:24:42.763 INFO:tasks.workunit.client.0.smithi178.stderr: -264> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: crc 2024-06-28T15:24:42.763 INFO:tasks.workunit.client.0.smithi178.stderr: -263> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9230) adding con mode: secure 2024-06-28T15:24:42.764 INFO:tasks.workunit.client.0.smithi178.stderr: -262> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.764 INFO:tasks.workunit.client.0.smithi178.stderr: -261> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.764 INFO:tasks.workunit.client.0.smithi178.stderr: -260> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command rotate-key cmddesc rotate-key hook 0x7ffd239b9378 EEXIST 2024-06-28T15:24:42.764 INFO:tasks.workunit.client.0.smithi178.stderr: -259> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:42.764 INFO:tasks.workunit.client.0.smithi178.stderr: -258> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient: _add_conns ranks=[2,1,0] 2024-06-28T15:24:42.764 INFO:tasks.workunit.client.0.smithi178.stderr: -257> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-c con 0x56328a1593b0 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:42.764 INFO:tasks.workunit.client.0.smithi178.stderr: -256> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-b con 0x56328a164760 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:42.764 INFO:tasks.workunit.client.0.smithi178.stderr: -255> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-a con 0x56328a151220 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:42.764 INFO:tasks.workunit.client.0.smithi178.stderr: -254> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.764 INFO:tasks.workunit.client.0.smithi178.stderr: -253> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.764 INFO:tasks.workunit.client.0.smithi178.stderr: -252> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.764 INFO:tasks.workunit.client.0.smithi178.stderr: -251> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:42.764 INFO:tasks.workunit.client.0.smithi178.stderr: -250> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient(hunting): authenticate will time out at 2425.810827s 2024-06-28T15:24:42.764 INFO:tasks.workunit.client.0.smithi178.stderr: -249> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): get_auth_request con 0x56328a1593b0 auth_method 0 2024-06-28T15:24:42.765 INFO:tasks.workunit.client.0.smithi178.stderr: -248> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.765 INFO:tasks.workunit.client.0.smithi178.stderr: -247> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.765 INFO:tasks.workunit.client.0.smithi178.stderr: -246> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.765 INFO:tasks.workunit.client.0.smithi178.stderr: -245> 2024-06-28T15:24:42.345+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request con 0x56328a151220 auth_method 0 2024-06-28T15:24:42.765 INFO:tasks.workunit.client.0.smithi178.stderr: -244> 2024-06-28T15:24:42.345+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.765 INFO:tasks.workunit.client.0.smithi178.stderr: -243> 2024-06-28T15:24:42.345+0000 7fcea4ed7640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.765 INFO:tasks.workunit.client.0.smithi178.stderr: -242> 2024-06-28T15:24:42.345+0000 7fcea4ed7640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.765 INFO:tasks.workunit.client.0.smithi178.stderr: -241> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.765 INFO:tasks.workunit.client.0.smithi178.stderr: -240> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.765 INFO:tasks.workunit.client.0.smithi178.stderr: -239> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.765 INFO:tasks.workunit.client.0.smithi178.stderr: -238> 2024-06-28T15:24:42.345+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.765 INFO:tasks.workunit.client.0.smithi178.stderr: -237> 2024-06-28T15:24:42.345+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.765 INFO:tasks.workunit.client.0.smithi178.stderr: -236> 2024-06-28T15:24:42.345+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.765 INFO:tasks.workunit.client.0.smithi178.stderr: -235> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient(hunting): handle_auth_done global_id 6500 payload 274 2024-06-28T15:24:42.766 INFO:tasks.workunit.client.0.smithi178.stderr: -234> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient: _finish_hunting 0 2024-06-28T15:24:42.766 INFO:tasks.workunit.client.0.smithi178.stderr: -233> 2024-06-28T15:24:42.345+0000 7fcea46d6640 1 monclient: found mon.noname-c 2024-06-28T15:24:42.766 INFO:tasks.workunit.client.0.smithi178.stderr: -232> 2024-06-28T15:24:42.345+0000 7fcea46d6640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:24:42.766 INFO:tasks.workunit.client.0.smithi178.stderr: -231> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.766 INFO:tasks.workunit.client.0.smithi178.stderr: -230> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:24:42.766 INFO:tasks.workunit.client.0.smithi178.stderr: -229> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: dump: 2024-06-28T15:24:42.766 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.766 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.766 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.766 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.766 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.766 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.766 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.767 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.767 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.767 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.767 INFO:tasks.workunit.client.0.smithi178.stderr: -228> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: _finish_auth 0 2024-06-28T15:24:42.767 INFO:tasks.workunit.client.0.smithi178.stderr: -227> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: _check_auth_tickets 2024-06-28T15:24:42.767 INFO:tasks.workunit.client.0.smithi178.stderr: -226> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:42.767 INFO:tasks.workunit.client.0.smithi178.stderr: -225> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 5 monclient: authenticate success, global_id 6500 2024-06-28T15:24:42.767 INFO:tasks.workunit.client.0.smithi178.stderr: -224> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 10 monclient: get_monmap_and_config success 2024-06-28T15:24:42.767 INFO:tasks.workunit.client.0.smithi178.stderr: -223> 2024-06-28T15:24:42.345+0000 7fcea6bc5980 4 set_mon_vals no callback set 2024-06-28T15:24:42.767 INFO:tasks.workunit.client.0.smithi178.stderr: -222> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.767 INFO:tasks.workunit.client.0.smithi178.stderr: -221> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:24:42.767 INFO:tasks.workunit.client.0.smithi178.stderr: -220> 2024-06-28T15:24:42.345+0000 7fcd7b7fe640 10 monclient: dump: 2024-06-28T15:24:42.767 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.767 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.768 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.768 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.768 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.768 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.768 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.768 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.768 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.768 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.768 INFO:tasks.workunit.client.0.smithi178.stderr: -219> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: shutdown 2024-06-28T15:24:42.768 INFO:tasks.workunit.client.0.smithi178.stderr: -218> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: build_initial_monmap 2024-06-28T15:24:42.768 INFO:tasks.workunit.client.0.smithi178.stderr: -217> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 1 build_initial for_mkfs: 0 2024-06-28T15:24:42.768 INFO:tasks.workunit.client.0.smithi178.stderr: -216> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: monmap: 2024-06-28T15:24:42.768 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:24:42.768 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:24:42.769 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:24:42.769 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:24:42.769 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:24:42.769 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.769 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:24:42.769 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:24:42.769 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:24:42.769 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.769 INFO:tasks.workunit.client.0.smithi178.stderr: -215> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding auth protocol: cephx 2024-06-28T15:24:42.769 INFO:tasks.workunit.client.0.smithi178.stderr: -214> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding auth protocol: cephx 2024-06-28T15:24:42.769 INFO:tasks.workunit.client.0.smithi178.stderr: -213> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding auth protocol: cephx 2024-06-28T15:24:42.769 INFO:tasks.workunit.client.0.smithi178.stderr: -212> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: secure 2024-06-28T15:24:42.769 INFO:tasks.workunit.client.0.smithi178.stderr: -211> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: crc 2024-06-28T15:24:42.769 INFO:tasks.workunit.client.0.smithi178.stderr: -210> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: secure 2024-06-28T15:24:42.770 INFO:tasks.workunit.client.0.smithi178.stderr: -209> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: crc 2024-06-28T15:24:42.770 INFO:tasks.workunit.client.0.smithi178.stderr: -208> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: secure 2024-06-28T15:24:42.770 INFO:tasks.workunit.client.0.smithi178.stderr: -207> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: crc 2024-06-28T15:24:42.770 INFO:tasks.workunit.client.0.smithi178.stderr: -206> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: crc 2024-06-28T15:24:42.770 INFO:tasks.workunit.client.0.smithi178.stderr: -205> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: secure 2024-06-28T15:24:42.770 INFO:tasks.workunit.client.0.smithi178.stderr: -204> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: crc 2024-06-28T15:24:42.770 INFO:tasks.workunit.client.0.smithi178.stderr: -203> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: secure 2024-06-28T15:24:42.770 INFO:tasks.workunit.client.0.smithi178.stderr: -202> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: crc 2024-06-28T15:24:42.770 INFO:tasks.workunit.client.0.smithi178.stderr: -201> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a12eb20) adding con mode: secure 2024-06-28T15:24:42.770 INFO:tasks.workunit.client.0.smithi178.stderr: -200> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.770 INFO:tasks.workunit.client.0.smithi178.stderr: -199> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 1 librados: starting msgr at 2024-06-28T15:24:42.770 INFO:tasks.workunit.client.0.smithi178.stderr: -198> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 1 librados: starting objecter 2024-06-28T15:24:42.770 INFO:tasks.workunit.client.0.smithi178.stderr: -197> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command objecter_requests cmddesc objecter_requests hook 0x56328a0ef780 EEXIST 2024-06-28T15:24:42.771 INFO:tasks.workunit.client.0.smithi178.stderr: -196> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 1 librados: setting wanted keys 2024-06-28T15:24:42.771 INFO:tasks.workunit.client.0.smithi178.stderr: -195> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 1 librados: calling monclient init 2024-06-28T15:24:42.771 INFO:tasks.workunit.client.0.smithi178.stderr: -194> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: init 2024-06-28T15:24:42.771 INFO:tasks.workunit.client.0.smithi178.stderr: -193> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding auth protocol: cephx 2024-06-28T15:24:42.771 INFO:tasks.workunit.client.0.smithi178.stderr: -192> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding auth protocol: cephx 2024-06-28T15:24:42.771 INFO:tasks.workunit.client.0.smithi178.stderr: -191> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding auth protocol: cephx 2024-06-28T15:24:42.771 INFO:tasks.workunit.client.0.smithi178.stderr: -190> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: secure 2024-06-28T15:24:42.771 INFO:tasks.workunit.client.0.smithi178.stderr: -189> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: crc 2024-06-28T15:24:42.771 INFO:tasks.workunit.client.0.smithi178.stderr: -188> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: secure 2024-06-28T15:24:42.771 INFO:tasks.workunit.client.0.smithi178.stderr: -187> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: crc 2024-06-28T15:24:42.771 INFO:tasks.workunit.client.0.smithi178.stderr: -186> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: secure 2024-06-28T15:24:42.771 INFO:tasks.workunit.client.0.smithi178.stderr: -185> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: crc 2024-06-28T15:24:42.771 INFO:tasks.workunit.client.0.smithi178.stderr: -184> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: crc 2024-06-28T15:24:42.771 INFO:tasks.workunit.client.0.smithi178.stderr: -183> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: secure 2024-06-28T15:24:42.772 INFO:tasks.workunit.client.0.smithi178.stderr: -182> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: crc 2024-06-28T15:24:42.772 INFO:tasks.workunit.client.0.smithi178.stderr: -181> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: secure 2024-06-28T15:24:42.772 INFO:tasks.workunit.client.0.smithi178.stderr: -180> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: crc 2024-06-28T15:24:42.772 INFO:tasks.workunit.client.0.smithi178.stderr: -179> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 AuthRegistry(0x56328a122eb0) adding con mode: secure 2024-06-28T15:24:42.772 INFO:tasks.workunit.client.0.smithi178.stderr: -178> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.772 INFO:tasks.workunit.client.0.smithi178.stderr: -177> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.772 INFO:tasks.workunit.client.0.smithi178.stderr: -176> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command rotate-key cmddesc rotate-key hook 0x56328a122ff8 EEXIST 2024-06-28T15:24:42.772 INFO:tasks.workunit.client.0.smithi178.stderr: -175> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: _reopen_session rank -1 2024-06-28T15:24:42.772 INFO:tasks.workunit.client.0.smithi178.stderr: -174> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: _add_conns ranks=[0,1,2] 2024-06-28T15:24:42.772 INFO:tasks.workunit.client.0.smithi178.stderr: -173> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-a con 0x56328a151220 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:24:42.772 INFO:tasks.workunit.client.0.smithi178.stderr: -172> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-b con 0x56328a1593b0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:24:42.772 INFO:tasks.workunit.client.0.smithi178.stderr: -171> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): picked mon.noname-c con 0x56328a164760 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:24:42.772 INFO:tasks.workunit.client.0.smithi178.stderr: -170> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.772 INFO:tasks.workunit.client.0.smithi178.stderr: -169> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.773 INFO:tasks.workunit.client.0.smithi178.stderr: -168> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): start opening mon connection 2024-06-28T15:24:42.773 INFO:tasks.workunit.client.0.smithi178.stderr: -167> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): _renew_subs 2024-06-28T15:24:42.773 INFO:tasks.workunit.client.0.smithi178.stderr: -166> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient(hunting): authenticate will time out at 2425.813391s 2024-06-28T15:24:42.773 INFO:tasks.workunit.client.0.smithi178.stderr: -165> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): get_auth_request con 0x56328a151220 auth_method 0 2024-06-28T15:24:42.773 INFO:tasks.workunit.client.0.smithi178.stderr: -164> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.773 INFO:tasks.workunit.client.0.smithi178.stderr: -163> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.773 INFO:tasks.workunit.client.0.smithi178.stderr: -162> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.773 INFO:tasks.workunit.client.0.smithi178.stderr: -161> 2024-06-28T15:24:42.349+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request con 0x56328a164760 auth_method 0 2024-06-28T15:24:42.773 INFO:tasks.workunit.client.0.smithi178.stderr: -160> 2024-06-28T15:24:42.349+0000 7fcea4ed7640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:24:42.773 INFO:tasks.workunit.client.0.smithi178.stderr: -159> 2024-06-28T15:24:42.349+0000 7fcea4ed7640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:24:42.773 INFO:tasks.workunit.client.0.smithi178.stderr: -158> 2024-06-28T15:24:42.349+0000 7fcea4ed7640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:24:42.773 INFO:tasks.workunit.client.0.smithi178.stderr: -157> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.773 INFO:tasks.workunit.client.0.smithi178.stderr: -156> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.773 INFO:tasks.workunit.client.0.smithi178.stderr: -155> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.774 INFO:tasks.workunit.client.0.smithi178.stderr: -154> 2024-06-28T15:24:42.349+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:24:42.774 INFO:tasks.workunit.client.0.smithi178.stderr: -153> 2024-06-28T15:24:42.349+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:24:42.774 INFO:tasks.workunit.client.0.smithi178.stderr: -152> 2024-06-28T15:24:42.349+0000 7fcea4ed7640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:24:42.774 INFO:tasks.workunit.client.0.smithi178.stderr: -151> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient(hunting): handle_auth_done global_id 6927 payload 824 2024-06-28T15:24:42.774 INFO:tasks.workunit.client.0.smithi178.stderr: -150> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient: _finish_hunting 0 2024-06-28T15:24:42.774 INFO:tasks.workunit.client.0.smithi178.stderr: -149> 2024-06-28T15:24:42.349+0000 7fcea46d6640 1 monclient: found mon.noname-a 2024-06-28T15:24:42.774 INFO:tasks.workunit.client.0.smithi178.stderr: -148> 2024-06-28T15:24:42.349+0000 7fcea46d6640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:24:42.774 INFO:tasks.workunit.client.0.smithi178.stderr: -147> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.774 INFO:tasks.workunit.client.0.smithi178.stderr: -146> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:24:42.774 INFO:tasks.workunit.client.0.smithi178.stderr: -145> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: dump: 2024-06-28T15:24:42.774 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.774 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.774 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.775 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.775 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.775 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.775 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.775 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.775 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.775 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.775 INFO:tasks.workunit.client.0.smithi178.stderr: -144> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: _finish_auth 0 2024-06-28T15:24:42.775 INFO:tasks.workunit.client.0.smithi178.stderr: -143> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: _check_auth_tickets 2024-06-28T15:24:42.775 INFO:tasks.workunit.client.0.smithi178.stderr: -142> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: handle_config config(0 keys) 2024-06-28T15:24:42.775 INFO:tasks.workunit.client.0.smithi178.stderr: -141> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:24:42.775 INFO:tasks.workunit.client.0.smithi178.stderr: -140> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:24:42.775 INFO:tasks.workunit.client.0.smithi178.stderr: -139> 2024-06-28T15:24:42.349+0000 7fcd79ffb640 10 monclient: dump: 2024-06-28T15:24:42.776 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:24:42.776 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:24:42.776 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.776 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:24:42.776 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:24:42.776 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:24:42.776 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:24:42.776 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:24:42.776 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:24:42.776 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.776 INFO:tasks.workunit.client.0.smithi178.stderr: -138> 2024-06-28T15:24:42.349+0000 7fcd7b7fe640 4 set_mon_vals no callback set 2024-06-28T15:24:42.777 INFO:tasks.workunit.client.0.smithi178.stderr: -137> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 monclient: authenticate success, global_id 6927 2024-06-28T15:24:42.777 INFO:tasks.workunit.client.0.smithi178.stderr: -136> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: _renew_subs 2024-06-28T15:24:42.777 INFO:tasks.workunit.client.0.smithi178.stderr: -135> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:24:42.777 INFO:tasks.workunit.client.0.smithi178.stderr: -134> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: _renew_subs 2024-06-28T15:24:42.777 INFO:tasks.workunit.client.0.smithi178.stderr: -133> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:24:42.777 INFO:tasks.workunit.client.0.smithi178.stderr: -132> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 1 librados: init done 2024-06-28T15:24:42.777 INFO:tasks.workunit.client.0.smithi178.stderr: -131> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command cr dump hook 0x56328a154338 2024-06-28T15:24:42.777 INFO:tasks.workunit.client.0.smithi178.stderr: -130> 2024-06-28T15:24:42.349+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.777 INFO:tasks.workunit.client.0.smithi178.stderr: -129> 2024-06-28T15:24:42.353+0000 7fcd79ffb640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:24:42.777 INFO:tasks.workunit.client.0.smithi178.stderr: -128> 2024-06-28T15:24:42.353+0000 7fcd79ffb640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:42.777 INFO:tasks.workunit.client.0.smithi178.stderr: -127> 2024-06-28T15:24:42.353+0000 7fcd79ffb640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:24:42.777 INFO:tasks.workunit.client.0.smithi178.stderr: -126> 2024-06-28T15:24:42.353+0000 7fcea3ed5640 10 monclient: get_auth_request con 0x7fcd6c06f780 auth_method 0 2024-06-28T15:24:42.778 INFO:tasks.workunit.client.0.smithi178.stderr: -125> 2024-06-28T15:24:42.353+0000 7fcea4ed7640 10 monclient: get_auth_request con 0x56328a177ee0 auth_method 0 2024-06-28T15:24:42.778 INFO:tasks.workunit.client.0.smithi178.stderr: -124> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.778 INFO:tasks.workunit.client.0.smithi178.stderr: -123> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 realm 2024-06-28T15:24:42.778 INFO:tasks.workunit.client.0.smithi178.stderr: -122> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.778 INFO:tasks.workunit.client.0.smithi178.stderr: -121> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.778 INFO:tasks.workunit.client.0.smithi178.stderr: -120> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:42.778 INFO:tasks.workunit.client.0.smithi178.stderr: -119> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.778 INFO:tasks.workunit.client.0.smithi178.stderr: -118> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.778 INFO:tasks.workunit.client.0.smithi178.stderr: -117> 2024-06-28T15:24:42.353+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.778 INFO:tasks.workunit.client.0.smithi178.stderr: -116> 2024-06-28T15:24:42.353+0000 7fcea46d6640 10 monclient: get_auth_request con 0x56328a17c0d0 auth_method 0 2024-06-28T15:24:42.778 INFO:tasks.workunit.client.0.smithi178.stderr: -115> 2024-06-28T15:24:42.357+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:42.778 INFO:tasks.workunit.client.0.smithi178.stderr: -114> 2024-06-28T15:24:42.357+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.778 INFO:tasks.workunit.client.0.smithi178.stderr: -113> 2024-06-28T15:24:42.357+0000 7fcea3ed5640 10 monclient: get_auth_request con 0x56328a17f910 auth_method 0 2024-06-28T15:24:42.779 INFO:tasks.workunit.client.0.smithi178.stderr: -112> 2024-06-28T15:24:42.357+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:42.779 INFO:tasks.workunit.client.0.smithi178.stderr: -111> 2024-06-28T15:24:42.357+0000 7fcea6bc5980 20 searching for the correct realm 2024-06-28T15:24:42.779 INFO:tasks.workunit.client.0.smithi178.stderr: -110> 2024-06-28T15:24:42.357+0000 7fcea4ed7640 10 monclient: get_auth_request con 0x7fce8c008070 auth_method 0 2024-06-28T15:24:42.779 INFO:tasks.workunit.client.0.smithi178.stderr: -109> 2024-06-28T15:24:42.357+0000 7fcea46d6640 10 monclient: get_auth_request con 0x7fce9c024040 auth_method 0 2024-06-28T15:24:42.779 INFO:tasks.workunit.client.0.smithi178.stderr: -108> 2024-06-28T15:24:42.361+0000 7fcea3ed5640 10 monclient: get_auth_request con 0x7fce940f1bd0 auth_method 0 2024-06-28T15:24:42.779 INFO:tasks.workunit.client.0.smithi178.stderr: -107> 2024-06-28T15:24:42.361+0000 7fcea4ed7640 10 monclient: get_auth_request con 0x7fce8c04a960 auth_method 0 2024-06-28T15:24:42.779 INFO:tasks.workunit.client.0.smithi178.stderr: -106> 2024-06-28T15:24:42.361+0000 7fcea4ed7640 0 -- 172.21.15.178:0/2581271606 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fce8c04a960 msgr2=0x7fce8c029410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:42.779 INFO:tasks.workunit.client.0.smithi178.stderr: -105> 2024-06-28T15:24:42.565+0000 7fcea4ed7640 10 monclient: get_auth_request con 0x7fce8c04a960 auth_method 0 2024-06-28T15:24:42.779 INFO:tasks.workunit.client.0.smithi178.stderr: -104> 2024-06-28T15:24:42.569+0000 7fcea46d6640 10 monclient: get_auth_request con 0x7fce8c035400 auth_method 0 2024-06-28T15:24:42.779 INFO:tasks.workunit.client.0.smithi178.stderr: -103> 2024-06-28T15:24:42.577+0000 7fcea46d6640 0 -- 172.21.15.178:0/2581271606 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x56328a17c0d0 msgr2=0x56328a17e4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:42.779 INFO:tasks.workunit.client.0.smithi178.stderr: -102> 2024-06-28T15:24:42.577+0000 7fcd79ffb640 1 client.6927.objecter ms_handle_reset 0x56328a17c0d0 session 0x56328a17ba50 osd.5 2024-06-28T15:24:42.779 INFO:tasks.workunit.client.0.smithi178.stderr: -101> 2024-06-28T15:24:42.577+0000 7fcd79ffb640 10 monclient: _renew_subs 2024-06-28T15:24:42.779 INFO:tasks.workunit.client.0.smithi178.stderr: -100> 2024-06-28T15:24:42.577+0000 7fcd79ffb640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.178:3300/0 2024-06-28T15:24:42.779 INFO:tasks.workunit.client.0.smithi178.stderr: -99> 2024-06-28T15:24:42.577+0000 7fcea46d6640 10 monclient: get_auth_request con 0x7fcd6c07fda0 auth_method 0 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -98> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -97> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -96> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -95> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -94> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -93> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -92> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -91> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -90> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -89> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -88> 2024-06-28T15:24:42.585+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -87> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -86> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 zone default found 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -85> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 4 Realm: () 2024-06-28T15:24:42.780 INFO:tasks.workunit.client.0.smithi178.stderr: -84> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:42.781 INFO:tasks.workunit.client.0.smithi178.stderr: -83> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:42.781 INFO:tasks.workunit.client.0.smithi178.stderr: -82> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:42.781 INFO:tasks.workunit.client.0.smithi178.stderr: -81> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 zonegroup default 2024-06-28T15:24:42.781 INFO:tasks.workunit.client.0.smithi178.stderr: -80> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.781 INFO:tasks.workunit.client.0.smithi178.stderr: -79> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.781 INFO:tasks.workunit.client.0.smithi178.stderr: -78> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.781 INFO:tasks.workunit.client.0.smithi178.stderr: -77> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.781 INFO:tasks.workunit.client.0.smithi178.stderr: -76> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.781 INFO:tasks.workunit.client.0.smithi178.stderr: -75> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.781 INFO:tasks.workunit.client.0.smithi178.stderr: -74> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 started sync module instance, tier type = 2024-06-28T15:24:42.781 INFO:tasks.workunit.client.0.smithi178.stderr: -73> 2024-06-28T15:24:42.589+0000 7fcea6bc5980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:42.781 INFO:tasks.workunit.client.0.smithi178.stderr: -72> 2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=0 2024-06-28T15:24:42.781 INFO:tasks.workunit.client.0.smithi178.stderr: -71> 2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=1 2024-06-28T15:24:42.781 INFO:tasks.workunit.client.0.smithi178.stderr: -70> 2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=2 2024-06-28T15:24:42.782 INFO:tasks.workunit.client.0.smithi178.stderr: -69> 2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=3 2024-06-28T15:24:42.782 INFO:tasks.workunit.client.0.smithi178.stderr: -68> 2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=4 2024-06-28T15:24:42.782 INFO:tasks.workunit.client.0.smithi178.stderr: -67> 2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=5 2024-06-28T15:24:42.782 INFO:tasks.workunit.client.0.smithi178.stderr: -66> 2024-06-28T15:24:42.613+0000 7fcea6bc5980 20 add_watcher() i=6 2024-06-28T15:24:42.782 INFO:tasks.workunit.client.0.smithi178.stderr: -65> 2024-06-28T15:24:42.617+0000 7fcea6bc5980 20 add_watcher() i=7 2024-06-28T15:24:42.782 INFO:tasks.workunit.client.0.smithi178.stderr: -64> 2024-06-28T15:24:42.617+0000 7fcea6bc5980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:42.782 INFO:tasks.workunit.client.0.smithi178.stderr: -63> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding auth protocol: cephx 2024-06-28T15:24:42.782 INFO:tasks.workunit.client.0.smithi178.stderr: -62> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding auth protocol: cephx 2024-06-28T15:24:42.782 INFO:tasks.workunit.client.0.smithi178.stderr: -61> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding auth protocol: cephx 2024-06-28T15:24:42.782 INFO:tasks.workunit.client.0.smithi178.stderr: -60> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: secure 2024-06-28T15:24:42.782 INFO:tasks.workunit.client.0.smithi178.stderr: -59> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: crc 2024-06-28T15:24:42.782 INFO:tasks.workunit.client.0.smithi178.stderr: -58> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: secure 2024-06-28T15:24:42.782 INFO:tasks.workunit.client.0.smithi178.stderr: -57> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: crc 2024-06-28T15:24:42.783 INFO:tasks.workunit.client.0.smithi178.stderr: -56> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: secure 2024-06-28T15:24:42.783 INFO:tasks.workunit.client.0.smithi178.stderr: -55> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: crc 2024-06-28T15:24:42.783 INFO:tasks.workunit.client.0.smithi178.stderr: -54> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: crc 2024-06-28T15:24:42.783 INFO:tasks.workunit.client.0.smithi178.stderr: -53> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: secure 2024-06-28T15:24:42.783 INFO:tasks.workunit.client.0.smithi178.stderr: -52> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: crc 2024-06-28T15:24:42.783 INFO:tasks.workunit.client.0.smithi178.stderr: -51> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: secure 2024-06-28T15:24:42.783 INFO:tasks.workunit.client.0.smithi178.stderr: -50> 2024-06-28T15:24:42.621+0000 7fcd5afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:42.783 INFO:tasks.workunit.client.0.smithi178.stderr: -49> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: crc 2024-06-28T15:24:42.783 INFO:tasks.workunit.client.0.smithi178.stderr: -48> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 AuthRegistry(0x7ffd239b9710) adding con mode: secure 2024-06-28T15:24:42.783 INFO:tasks.workunit.client.0.smithi178.stderr: -47> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:24:42.783 INFO:tasks.workunit.client.0.smithi178.stderr: -46> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:42.783 INFO:tasks.workunit.client.0.smithi178.stderr: -45> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:42.783 INFO:tasks.workunit.client.0.smithi178.stderr: -44> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command cache list hook 0x56328a151bb0 2024-06-28T15:24:42.783 INFO:tasks.workunit.client.0.smithi178.stderr: -43> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command cache inspect hook 0x56328a151bb0 2024-06-28T15:24:42.784 INFO:tasks.workunit.client.0.smithi178.stderr: -42> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command cache erase hook 0x56328a151bb0 2024-06-28T15:24:42.784 INFO:tasks.workunit.client.0.smithi178.stderr: -41> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command cache zap hook 0x56328a151bb0 2024-06-28T15:24:42.784 INFO:tasks.workunit.client.0.smithi178.stderr: -40> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 note: GC not initialized 2024-06-28T15:24:42.784 INFO:tasks.workunit.client.0.smithi178.stderr: -39> 2024-06-28T15:24:42.621+0000 7fcd59fdb640 20 reqs_thread_entry: start 2024-06-28T15:24:42.784 INFO:tasks.workunit.client.0.smithi178.stderr: -38> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command sync trace show hook 0x7fce94112070 2024-06-28T15:24:42.784 INFO:tasks.workunit.client.0.smithi178.stderr: -37> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command sync trace history hook 0x7fce94112070 2024-06-28T15:24:42.784 INFO:tasks.workunit.client.0.smithi178.stderr: -36> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command sync trace active hook 0x7fce94112070 2024-06-28T15:24:42.784 INFO:tasks.workunit.client.0.smithi178.stderr: -35> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 5 asok(0x563289e825e0) register_command sync trace active_short hook 0x7fce94112070 2024-06-28T15:24:42.784 INFO:tasks.workunit.client.0.smithi178.stderr: -34> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:42.784 INFO:tasks.workunit.client.0.smithi178.stderr: -33> 2024-06-28T15:24:42.621+0000 7fcea6bc5980 20 Filter name: none 2024-06-28T15:24:42.784 INFO:tasks.workunit.client.0.smithi178.stderr: -32> 2024-06-28T15:24:42.625+0000 7fcccb7fe640 20 reqs_thread_entry: start 2024-06-28T15:24:42.784 INFO:tasks.workunit.client.0.smithi178.stderr: -31> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-06-28T15:24:42.784 INFO:tasks.workunit.client.0.smithi178.stderr: -30> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.784 INFO:tasks.workunit.client.0.smithi178.stderr: -29> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:42.785 INFO:tasks.workunit.client.0.smithi178.stderr: -28> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-06-28T15:24:42.785 INFO:tasks.workunit.client.0.smithi178.stderr: -27> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-06-28T15:24:42.785 INFO:tasks.workunit.client.0.smithi178.stderr: -26> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 : miss 2024-06-28T15:24:42.785 INFO:tasks.workunit.client.0.smithi178.stderr: -25> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.785 INFO:tasks.workunit.client.0.smithi178.stderr: -24> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:24:42.785 INFO:tasks.workunit.client.0.smithi178.stderr: -23> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 info.flags=0x17 2024-06-28T15:24:42.785 INFO:tasks.workunit.client.0.smithi178.stderr: -22> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 to cache LRU end 2024-06-28T15:24:42.785 INFO:tasks.workunit.client.0.smithi178.stderr: -21> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:24:42.785 INFO:tasks.workunit.client.0.smithi178.stderr: -20> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:42.785 INFO:tasks.workunit.client.0.smithi178.stderr: -19> 2024-06-28T15:24:42.625+0000 7fcea6bc5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 2024-06-28T15:24:42.785 INFO:tasks.workunit.client.0.smithi178.stderr: -18> 2024-06-28T15:24:42.633+0000 7fcea6bc5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 info.flags=0x17 2024-06-28T15:24:42.785 INFO:tasks.workunit.client.0.smithi178.stderr: -17> 2024-06-28T15:24:42.633+0000 7fcea6bc5980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 to cache LRU end 2024-06-28T15:24:42.785 INFO:tasks.workunit.client.0.smithi178.stderr: -16> 2024-06-28T15:24:42.633+0000 7fcea6bc5980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:42.785 INFO:tasks.workunit.client.0.smithi178.stderr: -15> 2024-06-28T15:24:42.633+0000 7fcea6bc5980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27, ofs0, ns] 2024-06-28T15:24:42.786 INFO:tasks.workunit.client.0.smithi178.stderr: -14> 2024-06-28T15:24:42.633+0000 7fcd7b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595117 cookie 94775065204384 notifier 6927 bl.length()=1007 2024-06-28T15:24:42.786 INFO:tasks.workunit.client.0.smithi178.stderr: -13> 2024-06-28T15:24:42.633+0000 7fcd7b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 info.flags=0x17 2024-06-28T15:24:42.786 INFO:tasks.workunit.client.0.smithi178.stderr: -12> 2024-06-28T15:24:42.633+0000 7fcd7b7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 to cache LRU end 2024-06-28T15:24:42.786 INFO:tasks.workunit.client.0.smithi178.stderr: -11> 2024-06-28T15:24:42.633+0000 7fcd7b7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:24:42.786 INFO:tasks.workunit.client.0.smithi178.stderr: -10> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): orig_sources= new_sources= 2024-06-28T15:24:42.786 INFO:tasks.workunit.client.0.smithi178.stderr: -9> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): potential sources added=[] removed=[] 2024-06-28T15:24:42.786 INFO:tasks.workunit.client.0.smithi178.stderr: -8> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): orig_dests= new_dests= 2024-06-28T15:24:42.786 INFO:tasks.workunit.client.0.smithi178.stderr: -7> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): potential dests added=[] removed=[] 2024-06-28T15:24:42.786 INFO:tasks.workunit.client.0.smithi178.stderr: -6> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.0 2024-06-28T15:24:42.786 INFO:tasks.workunit.client.0.smithi178.stderr: -5> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.0 2024-06-28T15:24:42.786 INFO:tasks.workunit.client.0.smithi178.stderr: -4> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.1 2024-06-28T15:24:42.786 INFO:tasks.workunit.client.0.smithi178.stderr: -3> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.1 2024-06-28T15:24:42.786 INFO:tasks.workunit.client.0.smithi178.stderr: -2> 2024-06-28T15:24:42.637+0000 7fcea6bc5980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.0 2024-06-28T15:24:42.786 INFO:tasks.workunit.client.0.smithi178.stderr: -1> 2024-06-28T15:24:42.645+0000 7fcea6bc5980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7fcea6bc5980 time 2024-06-28T15:24:42.649045+0000 2024-06-28T15:24:42.787 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:24:42.787 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.787 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:24:42.787 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7fceaa49cc91] 2024-06-28T15:24:42.787 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fceaa49ce45] 2024-06-28T15:24:42.787 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x563287a014b1] 2024-06-28T15:24:42.787 INFO:tasks.workunit.client.0.smithi178.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x563287a12cf7] 2024-06-28T15:24:42.787 INFO:tasks.workunit.client.0.smithi178.stderr: 5: main() 2024-06-28T15:24:42.787 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fcea9305d90] 2024-06-28T15:24:42.787 INFO:tasks.workunit.client.0.smithi178.stderr: 7: __libc_start_main() 2024-06-28T15:24:42.787 INFO:tasks.workunit.client.0.smithi178.stderr: 8: _start() 2024-06-28T15:24:42.787 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.787 INFO:tasks.workunit.client.0.smithi178.stderr: 0> 2024-06-28T15:24:42.649+0000 7fcea6bc5980 -1 *** Caught signal (Aborted) ** 2024-06-28T15:24:42.788 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7fcea6bc5980 thread_name:radosgw-admin 2024-06-28T15:24:42.788 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.788 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:24:42.788 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7fcea931e520] 2024-06-28T15:24:42.788 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:24:42.788 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:24:42.788 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:24:42.788 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7fceaa49cceb] 2024-06-28T15:24:42.788 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7fceaa49ce45] 2024-06-28T15:24:42.788 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x563287a014b1] 2024-06-28T15:24:42.788 INFO:tasks.workunit.client.0.smithi178.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x577) [0x563287a12cf7] 2024-06-28T15:24:42.788 INFO:tasks.workunit.client.0.smithi178.stderr: 9: main() 2024-06-28T15:24:42.788 INFO:tasks.workunit.client.0.smithi178.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7fcea9305d90] 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr: 11: __libc_start_main() 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr: 12: _start() 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr:--- logging levels --- 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 none 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 lockdep 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 context 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 crush 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_balancer 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_locker 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log_expire 2024-06-28T15:24:42.789 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_migrator 2024-06-28T15:24:42.790 INFO:tasks.workunit.client.0.smithi178.stderr: 3/ 5 mds_quiesce 2024-06-28T15:24:42.790 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 buffer 2024-06-28T15:24:42.790 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 timer 2024-06-28T15:24:42.790 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 filer 2024-06-28T15:24:42.790 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 striper 2024-06-28T15:24:42.790 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 objecter 2024-06-28T15:24:42.790 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rados 2024-06-28T15:24:42.790 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd 2024-06-28T15:24:42.790 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_mirror 2024-06-28T15:24:42.790 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_replay 2024-06-28T15:24:42.790 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_pwl 2024-06-28T15:24:42.790 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 journaler 2024-06-28T15:24:42.790 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objectcacher 2024-06-28T15:24:42.790 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 immutable_obj_cache 2024-06-28T15:24:42.791 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 client 2024-06-28T15:24:42.791 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 osd 2024-06-28T15:24:42.791 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 optracker 2024-06-28T15:24:42.791 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objclass 2024-06-28T15:24:42.791 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 filestore 2024-06-28T15:24:42.791 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 journal 2024-06-28T15:24:42.791 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 ms 2024-06-28T15:24:42.791 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mon 2024-06-28T15:24:42.791 INFO:tasks.workunit.client.0.smithi178.stderr: 0/10 monc 2024-06-28T15:24:42.791 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 paxos 2024-06-28T15:24:42.791 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 tp 2024-06-28T15:24:42.791 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 auth 2024-06-28T15:24:42.791 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 crypto 2024-06-28T15:24:42.791 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 finisher 2024-06-28T15:24:42.792 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 reserver 2024-06-28T15:24:42.792 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 heartbeatmap 2024-06-28T15:24:42.792 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 perfcounter 2024-06-28T15:24:42.792 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw 2024-06-28T15:24:42.792 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_sync 2024-06-28T15:24:42.792 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_datacache 2024-06-28T15:24:42.792 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_access 2024-06-28T15:24:42.792 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_dbstore 2024-06-28T15:24:42.792 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_flight 2024-06-28T15:24:42.792 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_lifecycle 2024-06-28T15:24:42.792 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw_notification 2024-06-28T15:24:42.792 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 javaclient 2024-06-28T15:24:42.792 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 asok 2024-06-28T15:24:42.793 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 throttle 2024-06-28T15:24:42.793 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 refs 2024-06-28T15:24:42.793 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 compressor 2024-06-28T15:24:42.793 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluestore 2024-06-28T15:24:42.793 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluefs 2024-06-28T15:24:42.793 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 bdev 2024-06-28T15:24:42.793 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 kstore 2024-06-28T15:24:42.793 INFO:tasks.workunit.client.0.smithi178.stderr: 4/ 5 rocksdb 2024-06-28T15:24:42.793 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 fuse 2024-06-28T15:24:42.793 INFO:tasks.workunit.client.0.smithi178.stderr: 2/ 5 mgr 2024-06-28T15:24:42.793 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mgrc 2024-06-28T15:24:42.793 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 dpdk 2024-06-28T15:24:42.793 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 eventtrace 2024-06-28T15:24:42.793 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 prioritycache 2024-06-28T15:24:42.794 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 test 2024-06-28T15:24:42.794 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephfs_mirror 2024-06-28T15:24:42.794 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephsqlite 2024-06-28T15:24:42.794 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 crimson_interrupt 2024-06-28T15:24:42.794 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore 2024-06-28T15:24:42.794 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_onode 2024-06-28T15:24:42.794 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_odata 2024-06-28T15:24:42.794 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_omap 2024-06-28T15:24:42.794 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_tm 2024-06-28T15:24:42.794 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_t 2024-06-28T15:24:42.794 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cleaner 2024-06-28T15:24:42.794 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_epm 2024-06-28T15:24:42.794 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_lba 2024-06-28T15:24:42.794 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_fixedkv_tree 2024-06-28T15:24:42.795 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cache 2024-06-28T15:24:42.795 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_journal 2024-06-28T15:24:42.795 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_device 2024-06-28T15:24:42.795 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_backref 2024-06-28T15:24:42.795 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 alienstore 2024-06-28T15:24:42.795 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mclock 2024-06-28T15:24:42.795 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cyanstore 2024-06-28T15:24:42.795 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 ceph_exporter 2024-06-28T15:24:42.795 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 memstore 2024-06-28T15:24:42.795 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 trace 2024-06-28T15:24:42.795 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 ceph_dedup 2024-06-28T15:24:42.795 INFO:tasks.workunit.client.0.smithi178.stderr: -2/-2 (syslog threshold) 2024-06-28T15:24:42.795 INFO:tasks.workunit.client.0.smithi178.stderr: 99/99 (stderr threshold) 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr:--- pthread ID / name mapping for recent threads --- 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcccb7fe640 / http_manager 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcd59fdb640 / http_manager 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcd5afdd640 / rgw_dt_lg_renew 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcd79ffb640 / ms_dispatch 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcd7b7fe640 / io_context_pool 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcd89ffb640 / ms_dispatch 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcd90e09640 / ms_dispatch 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcd9260c640 / io_context_pool 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcea3ed5640 / msgr-worker-2 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcea46d6640 / msgr-worker-1 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcea4ed7640 / msgr-worker-0 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcea56d8640 / io_context_pool 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr: 7fcea6bc5980 / radosgw-admin 2024-06-28T15:24:42.796 INFO:tasks.workunit.client.0.smithi178.stderr: max_recent 500 2024-06-28T15:24:42.797 INFO:tasks.workunit.client.0.smithi178.stderr: max_new 1000 2024-06-28T15:24:42.797 INFO:tasks.workunit.client.0.smithi178.stderr: log_file /var/lib/ceph/crash/2024-06-28T15:24:42.654702Z_2ce4fd3c-6bb5-408f-bc7e-bd5a7379ab26/log 2024-06-28T15:24:42.797 INFO:tasks.workunit.client.0.smithi178.stderr:--- end dump of recent events --- 2024-06-28T15:24:42.797 INFO:tasks.workunit.client.0.smithi178.stderr:Aborted 2024-06-28T15:24:42.797 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-06-28T15:24:42.797 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:24:42.797 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:24:42.798 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.737+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.798 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.737+0000 7f81da6a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.737+0000 7f81da6a8980 20 realm 2024-06-28T15:24:42.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.737+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.737+0000 7f81da6a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.737+0000 7f81da6a8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:24:42.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.737+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.737+0000 7f81da6a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.737+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.741+0000 7f81da6a8980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:42.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.741+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.741+0000 7f81da6a8980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:24:42.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.741+0000 7f81da6a8980 20 searching for the correct realm 2024-06-28T15:24:42.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.749+0000 7f81d89ba640 0 -- 172.21.15.178:0/1705784450 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f81cc03e3f0 msgr2=0x7f81cc03e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:24:42.949 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.965+0000 7f81d81b9640 0 -- 172.21.15.178:0/1705784450 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x56154b7963d0 msgr2=0x56154b7987c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:24:42.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.973+0000 7f81da6a8980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:24:42.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.973+0000 7f81da6a8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:24:42.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.973+0000 7f81da6a8980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:24:42.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.973+0000 7f81da6a8980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:24:42.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.973+0000 7f81da6a8980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:24:42.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.973+0000 7f81da6a8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:24:42.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.973+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.958 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.973+0000 7f81da6a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.958 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.973+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.958 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.973+0000 7f81da6a8980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:24:42.958 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.973+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:24:42.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 20 zone default found 2024-06-28T15:24:42.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 4 Realm: () 2024-06-28T15:24:42.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:24:42.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:24:42.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:24:42.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 20 zonegroup default 2024-06-28T15:24:42.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:42.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:24:42.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 20 started sync module instance, tier type = 2024-06-28T15:24:42.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:42.977+0000 7f81da6a8980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:24:42.992 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.009+0000 7f81da6a8980 20 add_watcher() i=0 2024-06-28T15:24:42.992 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.009+0000 7f81da6a8980 20 add_watcher() i=1 2024-06-28T15:24:42.992 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.009+0000 7f81da6a8980 20 add_watcher() i=2 2024-06-28T15:24:42.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.009+0000 7f81da6a8980 20 add_watcher() i=3 2024-06-28T15:24:42.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.009+0000 7f81da6a8980 20 add_watcher() i=4 2024-06-28T15:24:42.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.009+0000 7f81da6a8980 20 add_watcher() i=5 2024-06-28T15:24:42.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.009+0000 7f81da6a8980 20 add_watcher() i=6 2024-06-28T15:24:42.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.009+0000 7f81da6a8980 20 add_watcher() i=7 2024-06-28T15:24:42.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.009+0000 7f81da6a8980 2 all 8 watchers are set, enabling cache 2024-06-28T15:24:43.000 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.013+0000 7f808efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:43.000 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f81da6a8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:24:43.000 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f81da6a8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:24:43.000 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f81da6a8980 5 note: GC not initialized 2024-06-28T15:24:43.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f808dfdb640 20 reqs_thread_entry: start 2024-06-28T15:24:43.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f81da6a8980 20 init_complete bucket index max shards: 1 2024-06-28T15:24:43.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f81da6a8980 20 Filter name: none 2024-06-28T15:24:43.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f7fff7fe640 20 reqs_thread_entry: start 2024-06-28T15:24:43.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:43.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f81da6a8980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:43.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:43.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f81da6a8980 20 rados_obj.operate() r=0 bl.length=467 2024-06-28T15:24:43.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f81da6a8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:43.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:43.004 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f81da6a8980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:24:43.004 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.017+0000 7f81da6a8980 20 rados->read ofs=0 len=0 2024-06-28T15:24:43.004 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.021+0000 7f81da6a8980 20 rados_obj.operate() r=0 bl.length=467 2024-06-28T15:24:43.004 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.021+0000 7f81da6a8980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:24:43.005 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.021+0000 7f808efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:24:43.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.025+0000 7f81da6a8980 20 remove_watcher() i=0 2024-06-28T15:24:43.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.025+0000 7f81da6a8980 2 removed watcher, disabling cache 2024-06-28T15:24:43.013 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.029+0000 7f81da6a8980 20 remove_watcher() i=1 2024-06-28T15:24:43.015 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.033+0000 7f81da6a8980 20 remove_watcher() i=2 2024-06-28T15:24:43.017 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.033+0000 7f81da6a8980 20 remove_watcher() i=3 2024-06-28T15:24:43.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.037+0000 7f81da6a8980 20 remove_watcher() i=4 2024-06-28T15:24:43.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.037+0000 7f81da6a8980 20 remove_watcher() i=5 2024-06-28T15:24:43.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.037+0000 7f81da6a8980 20 remove_watcher() i=6 2024-06-28T15:24:43.024 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:24:43.041+0000 7f81da6a8980 20 remove_watcher() i=7 2024-06-28T15:24:43.040 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:24:43.040 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:24:43.040 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: abort-at-commit-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.27 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:24:43.040 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:13.374 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-06-28T15:25:13.409 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:13.409 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:13.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.472+0000 7fa2413dc980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.476+0000 7fa2413dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:13.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.476+0000 7fa2413dc980 20 realm 2024-06-28T15:25:13.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.476+0000 7fa2413dc980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.476+0000 7fa2413dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:13.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.476+0000 7fa2413dc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:13.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.476+0000 7fa2413dc980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.476+0000 7fa2413dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:13.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.476+0000 7fa2413dc980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.480+0000 7fa2413dc980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:13.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.480+0000 7fa2413dc980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.480+0000 7fa2413dc980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:13.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.480+0000 7fa2413dc980 20 searching for the correct realm 2024-06-28T15:25:13.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.484+0000 7fa23f6ee640 0 -- 172.21.15.178:0/2642681692 >> [v2:172.21.15.195:6824/1278553753,v1:172.21.15.195:6825/1278553753] conn(0x7fa234017020 msgr2=0x7fa23405a300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:13.470 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.488+0000 7fa23eeed640 0 -- 172.21.15.178:0/2642681692 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fa23407d870 msgr2=0x7fa234090040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:13.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.704+0000 7fa2413dc980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:13.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.704+0000 7fa2413dc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:13.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.704+0000 7fa2413dc980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:13.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.704+0000 7fa2413dc980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:13.687 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.704+0000 7fa2413dc980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:13.688 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.704+0000 7fa2413dc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:13.688 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.688 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:13.688 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.688 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:13.688 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.689 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:13.689 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 20 zone default found 2024-06-28T15:25:13.689 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 4 Realm: () 2024-06-28T15:25:13.689 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:13.689 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:13.690 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:13.690 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 20 zonegroup default 2024-06-28T15:25:13.690 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.690 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:13.690 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.708+0000 7fa2413dc980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.691 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.712+0000 7fa2413dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:13.691 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.712+0000 7fa2413dc980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.692 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.712+0000 7fa2413dc980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:13.692 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.712+0000 7fa2413dc980 20 started sync module instance, tier type = 2024-06-28T15:25:13.692 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.712+0000 7fa2413dc980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:13.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.732+0000 7fa2413dc980 20 add_watcher() i=0 2024-06-28T15:25:13.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.732+0000 7fa2413dc980 20 add_watcher() i=1 2024-06-28T15:25:13.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.732+0000 7fa2413dc980 20 add_watcher() i=2 2024-06-28T15:25:13.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.732+0000 7fa2413dc980 20 add_watcher() i=3 2024-06-28T15:25:13.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.732+0000 7fa2413dc980 20 add_watcher() i=4 2024-06-28T15:25:13.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.732+0000 7fa2413dc980 20 add_watcher() i=5 2024-06-28T15:25:13.715 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.732+0000 7fa2413dc980 20 add_watcher() i=6 2024-06-28T15:25:13.717 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.736+0000 7fa2413dc980 20 add_watcher() i=7 2024-06-28T15:25:13.717 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.736+0000 7fa2413dc980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:13.721 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.740+0000 7fa0f57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:13.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.740+0000 7fa2413dc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:13.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.740+0000 7fa2413dc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:13.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.740+0000 7fa2413dc980 5 note: GC not initialized 2024-06-28T15:25:13.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa06ffff640 20 reqs_thread_entry: start 2024-06-28T15:25:13.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:13.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 20 Filter name: none 2024-06-28T15:25:13.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa06dffb640 20 reqs_thread_entry: start 2024-06-28T15:25:13.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-06-28T15:25:13.724 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:25:13.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-06-28T15:25:13.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-06-28T15:25:13.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 : miss 2024-06-28T15:25:13.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:25:13.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 info.flags=0x17 2024-06-28T15:25:13.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 to cache LRU end 2024-06-28T15:25:13.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:25:13.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:13.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.744+0000 7fa2413dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 2024-06-28T15:25:13.732 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.752+0000 7fa2413dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 info.flags=0x17 2024-06-28T15:25:13.732 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.752+0000 7fa2413dc980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 to cache LRU end 2024-06-28T15:25:13.732 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.752+0000 7fa2413dc980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:13.732 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.752+0000 7fa2413dc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27, ofs0, ns] 2024-06-28T15:25:13.733 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.752+0000 7fa121ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595119 cookie 94861797301168 notifier 6963 bl.length()=1007 2024-06-28T15:25:13.733 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.752+0000 7fa121ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 info.flags=0x17 2024-06-28T15:25:13.733 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.752+0000 7fa121ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 to cache LRU end 2024-06-28T15:25:13.733 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.752+0000 7fa121ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:13.736 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.756+0000 7fa2413dc980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): orig_sources= new_sources= 2024-06-28T15:25:13.736 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.756+0000 7fa2413dc980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): potential sources added=[] removed=[] 2024-06-28T15:25:13.736 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.756+0000 7fa2413dc980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): orig_dests= new_dests= 2024-06-28T15:25:13.736 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.756+0000 7fa2413dc980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): potential dests added=[] removed=[] 2024-06-28T15:25:13.738 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.756+0000 7fa2413dc980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.0 2024-06-28T15:25:13.738 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.756+0000 7fa2413dc980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.0 2024-06-28T15:25:13.738 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.756+0000 7fa2413dc980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.1 2024-06-28T15:25:13.738 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.756+0000 7fa2413dc980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.1.1 2024-06-28T15:25:13.738 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.756+0000 7fa2413dc980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.27.0 2024-06-28T15:25:13.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.760+0000 7fa2413dc980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 : hit (requested=0x17, cached=0x17) 2024-06-28T15:25:13.742 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.760+0000 7fa2413dc980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 2024-06-28T15:25:13.744 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.764+0000 7fa2413dc980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 info.flags=0x17 2024-06-28T15:25:13.744 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.764+0000 7fa2413dc980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 to cache LRU end 2024-06-28T15:25:13.744 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.764+0000 7fa2413dc980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:13.745 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.764+0000 7fa2413dc980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27, ofs0, ns] 2024-06-28T15:25:13.745 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.764+0000 7fa1227fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595120 cookie 94861797301168 notifier 6963 bl.length()=975 2024-06-28T15:25:13.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.764+0000 7fa1227fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 info.flags=0x17 2024-06-28T15:25:13.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.764+0000 7fa1227fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:843e24ce-085a-4043-8f55-38e05370e790.4686.27 to cache LRU end 2024-06-28T15:25:13.746 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.764+0000 7fa1227fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:13.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.768+0000 7fa2413dc980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): orig_sources= new_sources= 2024-06-28T15:25:13.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.768+0000 7fa2413dc980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): potential sources added=[] removed=[] 2024-06-28T15:25:13.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.768+0000 7fa2413dc980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): orig_dests= new_dests= 2024-06-28T15:25:13.748 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.768+0000 7fa2413dc980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[843e24ce-085a-4043-8f55-38e05370e790.4686.27]): potential dests added=[] removed=[] 2024-06-28T15:25:13.750 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.768+0000 7fa2413dc980 1 execute INFO: reshard of bucket "abort-at-commit-target-layout" from 2 shards to 2 shards completed successfully 2024-06-28T15:25:13.753 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.772+0000 7fa0f57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:13.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.776+0000 7fa2413dc980 20 remove_watcher() i=0 2024-06-28T15:25:13.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.776+0000 7fa2413dc980 2 removed watcher, disabling cache 2024-06-28T15:25:13.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.780+0000 7fa2413dc980 20 remove_watcher() i=1 2024-06-28T15:25:13.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.784+0000 7fa2413dc980 20 remove_watcher() i=2 2024-06-28T15:25:13.771 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.792+0000 7fa2413dc980 20 remove_watcher() i=3 2024-06-28T15:25:13.773 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.792+0000 7fa2413dc980 20 remove_watcher() i=4 2024-06-28T15:25:13.775 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.792+0000 7fa2413dc980 20 remove_watcher() i=5 2024-06-28T15:25:13.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.796+0000 7fa2413dc980 20 remove_watcher() i=6 2024-06-28T15:25:13.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.796+0000 7fa2413dc980 20 remove_watcher() i=7 2024-06-28T15:25:13.793 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-06-28T15:25:13.822 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:13.822 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:13.865 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.884+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.866 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.884+0000 7fc6974a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:13.867 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.884+0000 7fc6974a4980 20 realm 2024-06-28T15:25:13.867 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.884+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.867 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.884+0000 7fc6974a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:13.867 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.884+0000 7fc6974a4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:13.867 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.884+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.867 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.884+0000 7fc6974a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:13.867 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.884+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.868 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.888+0000 7fc6974a4980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:13.868 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.888+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.869 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.888+0000 7fc6974a4980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:13.869 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.888+0000 7fc6974a4980 20 searching for the correct realm 2024-06-28T15:25:13.875 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.892+0000 7fc68ffff640 0 -- 172.21.15.178:0/447209925 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fc688039070 msgr2=0x7fc688043400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:13.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.900+0000 7fc68ffff640 0 -- 172.21.15.178:0/447209925 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fc56007fda0 msgr2=0x7fc5600821d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:13.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:13.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:13.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:13.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:13.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:13.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:13.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:13.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:13.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:13.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 zone default found 2024-06-28T15:25:13.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 4 Realm: () 2024-06-28T15:25:13.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:13.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:13.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:13.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 zonegroup default 2024-06-28T15:25:13.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:13.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.908+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.912+0000 7fc6974a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:13.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.912+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.912+0000 7fc6974a4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:13.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.912+0000 7fc6974a4980 20 started sync module instance, tier type = 2024-06-28T15:25:13.893 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.912+0000 7fc6974a4980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:13.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.928+0000 7fc6974a4980 20 add_watcher() i=0 2024-06-28T15:25:13.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.928+0000 7fc6974a4980 20 add_watcher() i=1 2024-06-28T15:25:13.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.928+0000 7fc6974a4980 20 add_watcher() i=2 2024-06-28T15:25:13.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.928+0000 7fc6974a4980 20 add_watcher() i=3 2024-06-28T15:25:13.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.928+0000 7fc6974a4980 20 add_watcher() i=4 2024-06-28T15:25:13.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.928+0000 7fc6974a4980 20 add_watcher() i=5 2024-06-28T15:25:13.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.928+0000 7fc6974a4980 20 add_watcher() i=6 2024-06-28T15:25:13.908 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.928+0000 7fc6974a4980 20 add_watcher() i=7 2024-06-28T15:25:13.908 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.928+0000 7fc6974a4980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:13.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.928+0000 7fc54b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:13.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.928+0000 7fc6974a4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:13.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.928+0000 7fc6974a4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:13.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.928+0000 7fc6974a4980 5 note: GC not initialized 2024-06-28T15:25:13.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc54a7e4640 20 reqs_thread_entry: start 2024-06-28T15:25:13.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc6974a4980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:13.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc6974a4980 20 Filter name: none 2024-06-28T15:25:13.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc5437fe640 20 reqs_thread_entry: start 2024-06-28T15:25:13.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc6974a4980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:25:13.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc6974a4980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:25:13.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc6974a4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:13.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc6974a4980 20 rados_obj.operate() r=0 bl.length=221 2024-06-28T15:25:13.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc6974a4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:13.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc6974a4980 20 rados_obj.operate() r=0 bl.length=435 2024-06-28T15:25:13.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc6974a4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:13.914 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.932+0000 7fc54b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:13.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.940+0000 7fc6974a4980 20 remove_watcher() i=0 2024-06-28T15:25:13.919 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.940+0000 7fc6974a4980 2 removed watcher, disabling cache 2024-06-28T15:25:13.924 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.944+0000 7fc6974a4980 20 remove_watcher() i=1 2024-06-28T15:25:13.926 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.944+0000 7fc6974a4980 20 remove_watcher() i=2 2024-06-28T15:25:13.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.948+0000 7fc6974a4980 20 remove_watcher() i=3 2024-06-28T15:25:13.930 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.948+0000 7fc6974a4980 20 remove_watcher() i=4 2024-06-28T15:25:13.931 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.952+0000 7fc6974a4980 20 remove_watcher() i=5 2024-06-28T15:25:13.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.952+0000 7fc6974a4980 20 remove_watcher() i=6 2024-06-28T15:25:13.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:13.952+0000 7fc6974a4980 20 remove_watcher() i=7 2024-06-28T15:25:13.947 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:13.947 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:25:13.947 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: abort-at-commit-target-layout id: 843e24ce-085a-4043-8f55-38e05370e790.4686.27 num_objects: 19 size_kb: 1 num_shards: 2 2024-06-28T15:25:13.947 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:14.675 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: reshard bucket with EIO injected at do_reshard 2024-06-28T15:25:14.676 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:16.857 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-06-28T15:25:16.890 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:16.890 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:16.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:16.952+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:16.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:16.956+0000 7efdc71ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:16.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:16.956+0000 7efdc71ec980 20 realm 2024-06-28T15:25:16.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:16.956+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:16.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:16.956+0000 7efdc71ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:16.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:16.956+0000 7efdc71ec980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:16.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:16.956+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:16.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:16.956+0000 7efdc71ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:16.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:16.956+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:16.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:16.956+0000 7efdc71ec980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:16.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:16.956+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:16.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:16.956+0000 7efdc71ec980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:16.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:16.956+0000 7efdc71ec980 20 searching for the correct realm 2024-06-28T15:25:16.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:16.964+0000 7efdc54fe640 0 -- 172.21.15.178:0/3948163957 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7efdb802c2a0 msgr2=0x7efdb808c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:17.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.180+0000 7efdc4cfd640 0 -- 172.21.15.178:0/3948163957 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55e332357020 msgr2=0x55e332359410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:17.168 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:17.168 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:17.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:17.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:17.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:17.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:17.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.169 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:17.170 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:17.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 zone default found 2024-06-28T15:25:17.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 4 Realm: () 2024-06-28T15:25:17.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:17.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:17.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:17.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 zonegroup default 2024-06-28T15:25:17.171 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.188+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.192+0000 7efdc71ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.192+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.192+0000 7efdc71ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.192+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.192+0000 7efdc71ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.192+0000 7efdc71ec980 20 started sync module instance, tier type = 2024-06-28T15:25:17.173 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.192+0000 7efdc71ec980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:17.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.220+0000 7efdc71ec980 20 add_watcher() i=0 2024-06-28T15:25:17.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.220+0000 7efdc71ec980 20 add_watcher() i=1 2024-06-28T15:25:17.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.220+0000 7efdc71ec980 20 add_watcher() i=2 2024-06-28T15:25:17.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.220+0000 7efdc71ec980 20 add_watcher() i=3 2024-06-28T15:25:17.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.220+0000 7efdc71ec980 20 add_watcher() i=4 2024-06-28T15:25:17.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.220+0000 7efdc71ec980 20 add_watcher() i=5 2024-06-28T15:25:17.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.220+0000 7efdc71ec980 20 add_watcher() i=6 2024-06-28T15:25:17.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.228+0000 7efdc71ec980 20 add_watcher() i=7 2024-06-28T15:25:17.208 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.228+0000 7efdc71ec980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:17.212 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.232+0000 7efc7afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:17.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.232+0000 7efdc71ec980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:17.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.232+0000 7efdc71ec980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:17.213 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.232+0000 7efdc71ec980 5 note: GC not initialized 2024-06-28T15:25:17.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.232+0000 7efc79fe3640 20 reqs_thread_entry: start 2024-06-28T15:25:17.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.232+0000 7efdc71ec980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:17.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.232+0000 7efdc71ec980 20 Filter name: none 2024-06-28T15:25:17.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.232+0000 7efbeb7fe640 20 reqs_thread_entry: start 2024-06-28T15:25:17.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.232+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.215 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.236+0000 7efdc71ec980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:17.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.236+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.236+0000 7efdc71ec980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:17.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.236+0000 7efdc71ec980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:17.216 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.236+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.236+0000 7efdc71ec980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:17.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.236+0000 7efdc71ec980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.217 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.236+0000 7efdc71ec980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:17.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.236+0000 7efdc71ec980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:17.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.240+0000 7efc7afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:17.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.244+0000 7efdc71ec980 20 remove_watcher() i=0 2024-06-28T15:25:17.225 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.244+0000 7efdc71ec980 2 removed watcher, disabling cache 2024-06-28T15:25:17.226 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.244+0000 7efdc71ec980 20 remove_watcher() i=1 2024-06-28T15:25:17.229 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.248+0000 7efdc71ec980 20 remove_watcher() i=2 2024-06-28T15:25:17.231 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.248+0000 7efdc71ec980 20 remove_watcher() i=3 2024-06-28T15:25:17.233 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.252+0000 7efdc71ec980 20 remove_watcher() i=4 2024-06-28T15:25:17.234 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.252+0000 7efdc71ec980 20 remove_watcher() i=5 2024-06-28T15:25:17.235 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.256+0000 7efdc71ec980 20 remove_watcher() i=6 2024-06-28T15:25:17.237 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.256+0000 7efdc71ec980 20 remove_watcher() i=7 2024-06-28T15:25:17.253 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:17.253 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:25:17.253 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-do-reshard id: 843e24ce-085a-4043-8f55-38e05370e790.4686.28 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:25:17.253 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:17.254 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-error-at do_reshard 2024-06-28T15:25:17.286 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:17.287 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:17.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.348+0000 7f0c09768980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.352+0000 7f0c09768980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.352+0000 7f0c09768980 20 realm 2024-06-28T15:25:17.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.352+0000 7f0c09768980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.352+0000 7f0c09768980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.352+0000 7f0c09768980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:17.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.352+0000 7f0c09768980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.352+0000 7f0c09768980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.352+0000 7f0c09768980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.352+0000 7f0c09768980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:17.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.352+0000 7f0c09768980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.352+0000 7f0c09768980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:17.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.352+0000 7f0c09768980 20 searching for the correct realm 2024-06-28T15:25:17.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.360+0000 7f0c06a78640 0 -- 172.21.15.178:0/3908192786 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f0bf80f0930 msgr2=0x7f0bf80c3410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:17.346 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.364+0000 7f0c07a7a640 0 -- 172.21.15.178:0/3908192786 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f0bfc01fb60 msgr2=0x7f0bfc0677b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:17.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.372+0000 7f0c09768980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:17.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.372+0000 7f0c09768980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:17.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.372+0000 7f0c09768980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:17.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.372+0000 7f0c09768980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:17.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.372+0000 7f0c09768980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:17.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.372+0000 7f0c09768980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:17.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.372+0000 7f0c09768980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.372+0000 7f0c09768980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.372+0000 7f0c09768980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.372+0000 7f0c09768980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:17.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.372+0000 7f0c09768980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:17.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 20 zone default found 2024-06-28T15:25:17.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 4 Realm: () 2024-06-28T15:25:17.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:17.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:17.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:17.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 20 zonegroup default 2024-06-28T15:25:17.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 20 started sync module instance, tier type = 2024-06-28T15:25:17.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.376+0000 7f0c09768980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:17.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.392+0000 7f0c09768980 20 add_watcher() i=0 2024-06-28T15:25:17.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.392+0000 7f0c09768980 20 add_watcher() i=1 2024-06-28T15:25:17.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.392+0000 7f0c09768980 20 add_watcher() i=2 2024-06-28T15:25:17.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.392+0000 7f0c09768980 20 add_watcher() i=3 2024-06-28T15:25:17.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.392+0000 7f0c09768980 20 add_watcher() i=4 2024-06-28T15:25:17.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.392+0000 7f0c09768980 20 add_watcher() i=5 2024-06-28T15:25:17.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.392+0000 7f0c09768980 20 add_watcher() i=6 2024-06-28T15:25:17.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.392+0000 7f0c09768980 20 add_watcher() i=7 2024-06-28T15:25:17.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.392+0000 7f0c09768980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:17.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.392+0000 7f0abdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:17.375 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:17.375 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:17.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 5 note: GC not initialized 2024-06-28T15:25:17.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0abcfe1640 20 reqs_thread_entry: start 2024-06-28T15:25:17.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:17.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 20 Filter name: none 2024-06-28T15:25:17.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0a367fc640 20 reqs_thread_entry: start 2024-06-28T15:25:17.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2024-06-28T15:25:17.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:17.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2024-06-28T15:25:17.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2024-06-28T15:25:17.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 : miss 2024-06-28T15:25:17.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:17.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 info.flags=0x17 2024-06-28T15:25:17.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 to cache LRU end 2024-06-28T15:25:17.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:25:17.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:17.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.396+0000 7f0c09768980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 2024-06-28T15:25:17.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.400+0000 7f0c09768980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 info.flags=0x17 2024-06-28T15:25:17.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.400+0000 7f0c09768980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 to cache LRU end 2024-06-28T15:25:17.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.400+0000 7f0c09768980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:17.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.400+0000 7f0c09768980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28, ofs0, ns] 2024-06-28T15:25:17.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.400+0000 7f0aeaffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595090 cookie 94058022898336 notifier 7020 bl.length()=987 2024-06-28T15:25:17.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.400+0000 7f0aeaffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 info.flags=0x17 2024-06-28T15:25:17.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.400+0000 7f0aeaffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 to cache LRU end 2024-06-28T15:25:17.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.400+0000 7f0aeaffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:17.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.404+0000 7f0c09768980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): orig_sources= new_sources= 2024-06-28T15:25:17.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.404+0000 7f0c09768980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): potential sources added=[] removed=[] 2024-06-28T15:25:17.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.404+0000 7f0c09768980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): orig_dests= new_dests= 2024-06-28T15:25:17.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.404+0000 7f0c09768980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): potential dests added=[] removed=[] 2024-06-28T15:25:17.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.404+0000 7f0c09768980 -1 Injecting error=-5 at location=do_reshard 2024-06-28T15:25:17.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.408+0000 7f0c09768980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 : hit (requested=0x17, cached=0x17) 2024-06-28T15:25:17.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.408+0000 7f0c09768980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 2024-06-28T15:25:17.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.408+0000 7f0c09768980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 info.flags=0x17 2024-06-28T15:25:17.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.408+0000 7f0c09768980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 to cache LRU end 2024-06-28T15:25:17.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.408+0000 7f0c09768980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:17.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.408+0000 7f0c09768980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28, ofs0, ns] 2024-06-28T15:25:17.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.408+0000 7f0aea7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595091 cookie 94058022898336 notifier 7020 bl.length()=955 2024-06-28T15:25:17.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.408+0000 7f0aea7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 info.flags=0x17 2024-06-28T15:25:17.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.408+0000 7f0aea7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 to cache LRU end 2024-06-28T15:25:17.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.408+0000 7f0aea7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:17.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.412+0000 7f0c09768980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): orig_sources= new_sources= 2024-06-28T15:25:17.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.412+0000 7f0c09768980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): potential sources added=[] removed=[] 2024-06-28T15:25:17.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.412+0000 7f0c09768980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): orig_dests= new_dests= 2024-06-28T15:25:17.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.412+0000 7f0c09768980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): potential dests added=[] removed=[] 2024-06-28T15:25:17.391 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.412+0000 7f0c09768980 1 execute INFO: reshard of bucket "error-at-do-reshard" canceled due to errors 2024-06-28T15:25:17.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.412+0000 7f0abdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:17.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.416+0000 7f0c09768980 20 remove_watcher() i=0 2024-06-28T15:25:17.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.416+0000 7f0c09768980 2 removed watcher, disabling cache 2024-06-28T15:25:17.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.420+0000 7f0c09768980 20 remove_watcher() i=1 2024-06-28T15:25:17.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.424+0000 7f0c09768980 20 remove_watcher() i=2 2024-06-28T15:25:17.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.424+0000 7f0c09768980 20 remove_watcher() i=3 2024-06-28T15:25:17.407 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.428+0000 7f0c09768980 20 remove_watcher() i=4 2024-06-28T15:25:17.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.428+0000 7f0c09768980 20 remove_watcher() i=5 2024-06-28T15:25:17.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.428+0000 7f0c09768980 20 remove_watcher() i=6 2024-06-28T15:25:17.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.428+0000 7f0c09768980 20 remove_watcher() i=7 2024-06-28T15:25:17.424 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-06-28T15:25:17.449 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:17.450 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:17.491 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.512+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.512+0000 7f1108c03980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.512+0000 7f1108c03980 20 realm 2024-06-28T15:25:17.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.512+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.512+0000 7f1108c03980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.512+0000 7f1108c03980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:17.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.512+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.512+0000 7f1108c03980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.512+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.516+0000 7f1108c03980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:17.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.516+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.516+0000 7f1108c03980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:17.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.516+0000 7f1108c03980 20 searching for the correct realm 2024-06-28T15:25:17.503 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.520+0000 7f1106f15640 0 -- 172.21.15.178:0/2266639779 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f10f0008050 msgr2=0x7f10f009b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:17.717 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.736+0000 7f1106714640 0 -- 172.21.15.178:0/2266639779 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x5635c3cc7870 msgr2=0x5635c3cc9c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:17.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.744+0000 7f1108c03980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:17.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.744+0000 7f1108c03980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:17.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.744+0000 7f1108c03980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:17.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.744+0000 7f1108c03980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:17.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.744+0000 7f1108c03980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:17.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.744+0000 7f1108c03980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:17.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.744+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.744+0000 7f1108c03980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.725 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.744+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.744+0000 7f1108c03980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:17.726 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.744+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:17.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 20 zone default found 2024-06-28T15:25:17.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 4 Realm: () 2024-06-28T15:25:17.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:17.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:17.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:17.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 20 zonegroup default 2024-06-28T15:25:17.727 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.728 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.729 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.729 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 20 started sync module instance, tier type = 2024-06-28T15:25:17.729 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.748+0000 7f1108c03980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:17.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.772+0000 7f1108c03980 20 add_watcher() i=0 2024-06-28T15:25:17.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.772+0000 7f1108c03980 20 add_watcher() i=1 2024-06-28T15:25:17.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.772+0000 7f1108c03980 20 add_watcher() i=2 2024-06-28T15:25:17.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.772+0000 7f1108c03980 20 add_watcher() i=3 2024-06-28T15:25:17.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.772+0000 7f1108c03980 20 add_watcher() i=4 2024-06-28T15:25:17.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.772+0000 7f1108c03980 20 add_watcher() i=5 2024-06-28T15:25:17.754 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.772+0000 7f1108c03980 20 add_watcher() i=6 2024-06-28T15:25:17.756 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.776+0000 7f1108c03980 20 add_watcher() i=7 2024-06-28T15:25:17.757 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.776+0000 7f1108c03980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:17.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.780+0000 7f0fbcfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:17.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.780+0000 7f1108c03980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:17.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.780+0000 7f1108c03980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:17.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.780+0000 7f1108c03980 5 note: GC not initialized 2024-06-28T15:25:17.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.780+0000 7f0f377fe640 20 reqs_thread_entry: start 2024-06-28T15:25:17.762 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.780+0000 7f1108c03980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:17.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.780+0000 7f1108c03980 20 Filter name: none 2024-06-28T15:25:17.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.780+0000 7f0f357fa640 20 reqs_thread_entry: start 2024-06-28T15:25:17.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.784+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.784+0000 7f1108c03980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:17.763 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.784+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.764 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.784+0000 7f1108c03980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:17.764 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.784+0000 7f1108c03980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:17.764 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.784+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.784+0000 7f1108c03980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:17.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.784+0000 7f1108c03980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.784+0000 7f1108c03980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:17.765 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.784+0000 7f1108c03980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:17.767 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.784+0000 7f0fbcfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:17.772 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.792+0000 7f1108c03980 20 remove_watcher() i=0 2024-06-28T15:25:17.772 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.792+0000 7f1108c03980 2 removed watcher, disabling cache 2024-06-28T15:25:17.774 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.792+0000 7f1108c03980 20 remove_watcher() i=1 2024-06-28T15:25:17.776 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.796+0000 7f1108c03980 20 remove_watcher() i=2 2024-06-28T15:25:17.777 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.796+0000 7f1108c03980 20 remove_watcher() i=3 2024-06-28T15:25:17.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.800+0000 7f1108c03980 20 remove_watcher() i=4 2024-06-28T15:25:17.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.800+0000 7f1108c03980 20 remove_watcher() i=5 2024-06-28T15:25:17.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.800+0000 7f1108c03980 20 remove_watcher() i=6 2024-06-28T15:25:17.784 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:17.804+0000 7f1108c03980 20 remove_watcher() i=7 2024-06-28T15:25:17.798 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:17.798 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:25:17.798 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-do-reshard id: 843e24ce-085a-4043-8f55-38e05370e790.4686.28 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:25:17.798 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:17.913 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-06-28T15:25:17.949 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:17.949 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:17.993 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.012+0000 7f5cf55a0980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.016+0000 7f5cf55a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.016+0000 7f5cf55a0980 20 realm 2024-06-28T15:25:17.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.016+0000 7f5cf55a0980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.016+0000 7f5cf55a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.016+0000 7f5cf55a0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:17.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.016+0000 7f5cf55a0980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.016+0000 7f5cf55a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:17.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.016+0000 7f5cf55a0980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.016+0000 7f5cf55a0980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:17.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.016+0000 7f5cf55a0980 20 rados->read ofs=0 len=0 2024-06-28T15:25:17.998 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.016+0000 7f5cf55a0980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:17.998 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.016+0000 7f5cf55a0980 20 searching for the correct realm 2024-06-28T15:25:18.004 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.024+0000 7f5cf38b2640 0 -- 172.21.15.178:0/1996115255 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x56287e9d9c70 msgr2=0x56287e9dc0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:18.009 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.028+0000 7f5cf28b0640 0 -- 172.21.15.178:0/1996115255 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f5ce406c8b0 msgr2=0x7f5ce4062c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:18.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.036+0000 7f5cf55a0980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:18.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.036+0000 7f5cf55a0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:18.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.036+0000 7f5cf55a0980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:18.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.036+0000 7f5cf55a0980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:18.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.036+0000 7f5cf55a0980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:18.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.036+0000 7f5cf55a0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:18.018 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.036+0000 7f5cf55a0980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.036+0000 7f5cf55a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:18.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.036+0000 7f5cf55a0980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:18.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:18.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 20 zone default found 2024-06-28T15:25:18.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 4 Realm: () 2024-06-28T15:25:18.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:18.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:18.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:18.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 20 zonegroup default 2024-06-28T15:25:18.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:18.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:18.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:18.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 20 started sync module instance, tier type = 2024-06-28T15:25:18.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.040+0000 7f5cf55a0980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:18.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.056+0000 7f5cf55a0980 20 add_watcher() i=0 2024-06-28T15:25:18.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.056+0000 7f5cf55a0980 20 add_watcher() i=1 2024-06-28T15:25:18.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.056+0000 7f5cf55a0980 20 add_watcher() i=2 2024-06-28T15:25:18.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.056+0000 7f5cf55a0980 20 add_watcher() i=3 2024-06-28T15:25:18.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.056+0000 7f5cf55a0980 20 add_watcher() i=4 2024-06-28T15:25:18.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.056+0000 7f5cf55a0980 20 add_watcher() i=5 2024-06-28T15:25:18.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.056+0000 7f5cf55a0980 20 add_watcher() i=6 2024-06-28T15:25:18.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.056+0000 7f5cf55a0980 20 add_watcher() i=7 2024-06-28T15:25:18.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.056+0000 7f5cf55a0980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:18.041 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5ba9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:18.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:18.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:18.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 5 note: GC not initialized 2024-06-28T15:25:18.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5ba8fe1640 20 reqs_thread_entry: start 2024-06-28T15:25:18.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:18.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 20 Filter name: none 2024-06-28T15:25:18.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5b227fc640 20 reqs_thread_entry: start 2024-06-28T15:25:18.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2024-06-28T15:25:18.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:18.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2024-06-28T15:25:18.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2024-06-28T15:25:18.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 : miss 2024-06-28T15:25:18.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:18.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 info.flags=0x17 2024-06-28T15:25:18.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 to cache LRU end 2024-06-28T15:25:18.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:25:18.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:18.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.060+0000 7f5cf55a0980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 2024-06-28T15:25:18.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.068+0000 7f5cf55a0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 info.flags=0x17 2024-06-28T15:25:18.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.068+0000 7f5cf55a0980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 to cache LRU end 2024-06-28T15:25:18.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.068+0000 7f5cf55a0980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:18.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.068+0000 7f5cf55a0980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28, ofs0, ns] 2024-06-28T15:25:18.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.068+0000 7f5bd6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595092 cookie 94731923006256 notifier 7056 bl.length()=987 2024-06-28T15:25:18.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.068+0000 7f5bd6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 info.flags=0x17 2024-06-28T15:25:18.048 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.068+0000 7f5bd6ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 to cache LRU end 2024-06-28T15:25:18.049 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.068+0000 7f5bd6ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:18.050 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.068+0000 7f5cf55a0980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): orig_sources= new_sources= 2024-06-28T15:25:18.050 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.068+0000 7f5cf55a0980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): potential sources added=[] removed=[] 2024-06-28T15:25:18.050 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.068+0000 7f5cf55a0980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): orig_dests= new_dests= 2024-06-28T15:25:18.050 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.068+0000 7f5cf55a0980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): potential dests added=[] removed=[] 2024-06-28T15:25:18.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.072+0000 7f5cf55a0980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.28.1.0 2024-06-28T15:25:18.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.072+0000 7f5cf55a0980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.28.1.0 2024-06-28T15:25:18.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.072+0000 7f5cf55a0980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.28.1.1 2024-06-28T15:25:18.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.072+0000 7f5cf55a0980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.28.1.1 2024-06-28T15:25:18.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.072+0000 7f5cf55a0980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.28.0 2024-06-28T15:25:18.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.072+0000 7f5cf55a0980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 : hit (requested=0x17, cached=0x17) 2024-06-28T15:25:18.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.072+0000 7f5cf55a0980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 2024-06-28T15:25:18.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.076+0000 7f5cf55a0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 info.flags=0x17 2024-06-28T15:25:18.055 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.076+0000 7f5cf55a0980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 to cache LRU end 2024-06-28T15:25:18.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.076+0000 7f5cf55a0980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:18.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.076+0000 7f5cf55a0980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28, ofs0, ns] 2024-06-28T15:25:18.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.076+0000 7f5bd67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595093 cookie 94731923006256 notifier 7056 bl.length()=955 2024-06-28T15:25:18.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.076+0000 7f5bd67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 info.flags=0x17 2024-06-28T15:25:18.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.076+0000 7f5bd67fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.28 to cache LRU end 2024-06-28T15:25:18.056 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.076+0000 7f5bd67fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:18.057 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.076+0000 7f5cf55a0980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): orig_sources= new_sources= 2024-06-28T15:25:18.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.076+0000 7f5cf55a0980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): potential sources added=[] removed=[] 2024-06-28T15:25:18.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.076+0000 7f5cf55a0980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): orig_dests= new_dests= 2024-06-28T15:25:18.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.076+0000 7f5cf55a0980 20 handle_bi_update(): bucket=:error-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.28]): potential dests added=[] removed=[] 2024-06-28T15:25:18.059 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.080+0000 7f5cf55a0980 1 execute INFO: reshard of bucket "error-at-do-reshard" from 2 shards to 2 shards completed successfully 2024-06-28T15:25:18.062 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.080+0000 7f5ba9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:18.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.084+0000 7f5cf55a0980 20 remove_watcher() i=0 2024-06-28T15:25:18.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.084+0000 7f5cf55a0980 2 removed watcher, disabling cache 2024-06-28T15:25:18.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.088+0000 7f5cf55a0980 20 remove_watcher() i=1 2024-06-28T15:25:18.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.088+0000 7f5cf55a0980 20 remove_watcher() i=2 2024-06-28T15:25:18.072 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.092+0000 7f5cf55a0980 20 remove_watcher() i=3 2024-06-28T15:25:18.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.092+0000 7f5cf55a0980 20 remove_watcher() i=4 2024-06-28T15:25:18.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.096+0000 7f5cf55a0980 20 remove_watcher() i=5 2024-06-28T15:25:18.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.096+0000 7f5cf55a0980 20 remove_watcher() i=6 2024-06-28T15:25:18.078 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.096+0000 7f5cf55a0980 20 remove_watcher() i=7 2024-06-28T15:25:18.091 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-06-28T15:25:18.118 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:18.118 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:18.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.180+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.180+0000 7f524bdda980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:18.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.180+0000 7f524bdda980 20 realm 2024-06-28T15:25:18.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.180+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.180+0000 7f524bdda980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:18.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.180+0000 7f524bdda980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:18.163 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.180+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.164 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.180+0000 7f524bdda980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:18.164 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.180+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.164 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.184+0000 7f524bdda980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:18.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.184+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.165 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.184+0000 7f524bdda980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:18.166 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.184+0000 7f524bdda980 20 searching for the correct realm 2024-06-28T15:25:18.172 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.192+0000 7f524a0ec640 0 -- 172.21.15.178:0/1746411410 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f5240054040 msgr2=0x7f5240062470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:18.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.408+0000 7f52498eb640 0 -- 172.21.15.178:0/1746411410 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x56427351c250 msgr2=0x56427351e710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:18.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.416+0000 7f524bdda980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:18.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.416+0000 7f524bdda980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:18.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.416+0000 7f524bdda980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:18.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.416+0000 7f524bdda980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:18.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.416+0000 7f524bdda980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:18.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.416+0000 7f524bdda980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:18.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.416+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.416+0000 7f524bdda980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:18.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.416+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.416+0000 7f524bdda980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:18.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:18.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 20 zone default found 2024-06-28T15:25:18.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 4 Realm: () 2024-06-28T15:25:18.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:18.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:18.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:18.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 20 zonegroup default 2024-06-28T15:25:18.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:18.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:18.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:18.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 20 started sync module instance, tier type = 2024-06-28T15:25:18.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.420+0000 7f524bdda980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:18.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.444+0000 7f524bdda980 20 add_watcher() i=0 2024-06-28T15:25:18.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.444+0000 7f524bdda980 20 add_watcher() i=1 2024-06-28T15:25:18.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.444+0000 7f524bdda980 20 add_watcher() i=2 2024-06-28T15:25:18.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.444+0000 7f524bdda980 20 add_watcher() i=3 2024-06-28T15:25:18.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.444+0000 7f524bdda980 20 add_watcher() i=4 2024-06-28T15:25:18.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.444+0000 7f524bdda980 20 add_watcher() i=5 2024-06-28T15:25:18.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.444+0000 7f524bdda980 20 add_watcher() i=6 2024-06-28T15:25:18.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.444+0000 7f524bdda980 20 add_watcher() i=7 2024-06-28T15:25:18.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.444+0000 7f524bdda980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:18.430 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.448+0000 7f50fb7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:18.431 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.448+0000 7f524bdda980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:18.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.448+0000 7f524bdda980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:18.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f524bdda980 5 note: GC not initialized 2024-06-28T15:25:18.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f50fa7fc640 20 reqs_thread_entry: start 2024-06-28T15:25:18.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f524bdda980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:18.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f524bdda980 20 Filter name: none 2024-06-28T15:25:18.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f506bfff640 20 reqs_thread_entry: start 2024-06-28T15:25:18.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.433 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f524bdda980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:18.433 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.434 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f524bdda980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:18.434 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f524bdda980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:18.434 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.434 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f524bdda980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:18.434 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f524bdda980 20 rados->read ofs=0 len=0 2024-06-28T15:25:18.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f524bdda980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:18.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.452+0000 7f524bdda980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:18.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.456+0000 7f50fb7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:18.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.460+0000 7f524bdda980 20 remove_watcher() i=0 2024-06-28T15:25:18.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.460+0000 7f524bdda980 2 removed watcher, disabling cache 2024-06-28T15:25:18.444 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.464+0000 7f524bdda980 20 remove_watcher() i=1 2024-06-28T15:25:18.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.464+0000 7f524bdda980 20 remove_watcher() i=2 2024-06-28T15:25:18.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.468+0000 7f524bdda980 20 remove_watcher() i=3 2024-06-28T15:25:18.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.468+0000 7f524bdda980 20 remove_watcher() i=4 2024-06-28T15:25:18.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.472+0000 7f524bdda980 20 remove_watcher() i=5 2024-06-28T15:25:18.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.472+0000 7f524bdda980 20 remove_watcher() i=6 2024-06-28T15:25:18.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:18.476+0000 7f524bdda980 20 remove_watcher() i=7 2024-06-28T15:25:18.474 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:18.474 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:25:18.474 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: error-at-do-reshard id: 843e24ce-085a-4043-8f55-38e05370e790.4686.28 num_objects: 19 size_kb: 1 num_shards: 2 2024-06-28T15:25:18.474 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:19.219 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: reshard bucket with abort at do_reshard 2024-06-28T15:25:19.220 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:21.319 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-06-28T15:25:21.352 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:21.352 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:21.398 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.416+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.420+0000 7f5d93f4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:21.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.420+0000 7f5d93f4f980 20 realm 2024-06-28T15:25:21.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.420+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.420+0000 7f5d93f4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:21.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.420+0000 7f5d93f4f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:21.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.420+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.420+0000 7f5d93f4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:21.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.420+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.420+0000 7f5d93f4f980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:21.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.420+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.403 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.424+0000 7f5d93f4f980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:21.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.424+0000 7f5d93f4f980 20 searching for the correct realm 2024-06-28T15:25:21.407 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.428+0000 7f5d9125f640 0 -- 172.21.15.178:0/3695208907 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f5d84008070 msgr2=0x7f5d840722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:21.625 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.644+0000 7f5d91a60640 0 -- 172.21.15.178:0/3695208907 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f5d8801b3c0 msgr2=0x7f5d8801b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:21.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.860+0000 7f5d93f4f980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:21.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.860+0000 7f5d93f4f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:21.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.860+0000 7f5d93f4f980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:21.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.860+0000 7f5d93f4f980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:21.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.860+0000 7f5d93f4f980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:21.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.860+0000 7f5d93f4f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:21.843 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.860+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.844 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:21.844 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.845 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:21.845 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.846 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:21.846 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 20 zone default found 2024-06-28T15:25:21.846 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 4 Realm: () 2024-06-28T15:25:21.846 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:21.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:21.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:21.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 20 zonegroup default 2024-06-28T15:25:21.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:21.847 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.849 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:21.849 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.849 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.864+0000 7f5d93f4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:21.849 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.868+0000 7f5d93f4f980 20 started sync module instance, tier type = 2024-06-28T15:25:21.850 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.868+0000 7f5d93f4f980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:21.870 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.888+0000 7f5d93f4f980 20 add_watcher() i=0 2024-06-28T15:25:21.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.888+0000 7f5d93f4f980 20 add_watcher() i=1 2024-06-28T15:25:21.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.888+0000 7f5d93f4f980 20 add_watcher() i=2 2024-06-28T15:25:21.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.888+0000 7f5d93f4f980 20 add_watcher() i=3 2024-06-28T15:25:21.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.888+0000 7f5d93f4f980 20 add_watcher() i=4 2024-06-28T15:25:21.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.888+0000 7f5d93f4f980 20 add_watcher() i=5 2024-06-28T15:25:21.871 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.888+0000 7f5d93f4f980 20 add_watcher() i=6 2024-06-28T15:25:21.873 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.892+0000 7f5d93f4f980 20 add_watcher() i=7 2024-06-28T15:25:21.873 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.892+0000 7f5d93f4f980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:21.877 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.896+0000 7f5c437fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:21.878 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.896+0000 7f5d93f4f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:21.878 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.896+0000 7f5d93f4f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:21.878 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.896+0000 7f5d93f4f980 5 note: GC not initialized 2024-06-28T15:25:21.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.896+0000 7f5c427fc640 20 reqs_thread_entry: start 2024-06-28T15:25:21.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.896+0000 7f5d93f4f980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:21.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.896+0000 7f5d93f4f980 20 Filter name: none 2024-06-28T15:25:21.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.900+0000 7f5bb3fff640 20 reqs_thread_entry: start 2024-06-28T15:25:21.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.900+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.900+0000 7f5d93f4f980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:21.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.900+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.900+0000 7f5d93f4f980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:21.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.900+0000 7f5d93f4f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:21.882 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.900+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.883 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.900+0000 7f5d93f4f980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:21.883 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.900+0000 7f5d93f4f980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.884 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.904+0000 7f5d93f4f980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:21.884 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.904+0000 7f5d93f4f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:21.886 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.904+0000 7f5c437fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:21.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.908+0000 7f5d93f4f980 20 remove_watcher() i=0 2024-06-28T15:25:21.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.908+0000 7f5d93f4f980 2 removed watcher, disabling cache 2024-06-28T15:25:21.892 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.912+0000 7f5d93f4f980 20 remove_watcher() i=1 2024-06-28T15:25:21.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.912+0000 7f5d93f4f980 20 remove_watcher() i=2 2024-06-28T15:25:21.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.916+0000 7f5d93f4f980 20 remove_watcher() i=3 2024-06-28T15:25:21.899 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.920+0000 7f5d93f4f980 20 remove_watcher() i=4 2024-06-28T15:25:21.900 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.920+0000 7f5d93f4f980 20 remove_watcher() i=5 2024-06-28T15:25:21.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.920+0000 7f5d93f4f980 20 remove_watcher() i=6 2024-06-28T15:25:21.903 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:21.924+0000 7f5d93f4f980 20 remove_watcher() i=7 2024-06-28T15:25:21.918 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:21.918 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:25:21.918 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: abort-at-do-reshard id: 843e24ce-085a-4043-8f55-38e05370e790.4686.29 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:25:21.918 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:21.918 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at do_reshard 2024-06-28T15:25:21.952 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:21.952 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:21.995 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:21.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.016+0000 7f842baa9980 20 realm 2024-06-28T15:25:21.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:21.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.016+0000 7f842baa9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:21.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:21.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.998 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:21.998 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:21.999 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.020+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:21.999 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.020+0000 7f842baa9980 20 searching for the correct realm 2024-06-28T15:25:22.005 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.024+0000 7f8429dbb640 0 -- 172.21.15.178:0/636795418 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x558a7110e330 msgr2=0x558a711107f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:22.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.028+0000 7f8428db9640 0 -- 172.21.15.178:0/636795418 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f841c06d020 msgr2=0x7f841c05a560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:22.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:22.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:22.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:22.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:22.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:22.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:22.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.036+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.036+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.019 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.036+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:22.020 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:22.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 20 zone default found 2024-06-28T15:25:22.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 4 Realm: () 2024-06-28T15:25:22.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:22.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:22.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:22.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 20 zonegroup default 2024-06-28T15:25:22.021 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.022 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.023 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 20 started sync module instance, tier type = 2024-06-28T15:25:22.023 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.040+0000 7f842baa9980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:22.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=0 2024-06-28T15:25:22.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=1 2024-06-28T15:25:22.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=2 2024-06-28T15:25:22.037 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=3 2024-06-28T15:25:22.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=4 2024-06-28T15:25:22.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=5 2024-06-28T15:25:22.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=6 2024-06-28T15:25:22.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=7 2024-06-28T15:25:22.038 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.056+0000 7f842baa9980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:22.041 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.060+0000 7f82df7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:22.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.060+0000 7f842baa9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:22.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.060+0000 7f842baa9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:22.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.060+0000 7f842baa9980 5 note: GC not initialized 2024-06-28T15:25:22.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.060+0000 7f82de7e4640 20 reqs_thread_entry: start 2024-06-28T15:25:22.042 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.060+0000 7f842baa9980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:22.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.060+0000 7f842baa9980 20 Filter name: none 2024-06-28T15:25:22.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.060+0000 7f824ffff640 20 reqs_thread_entry: start 2024-06-28T15:25:22.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.060+0000 7f842baa9980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-06-28T15:25:22.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.060+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.064+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:22.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.064+0000 7f842baa9980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-06-28T15:25:22.043 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.064+0000 7f842baa9980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-06-28T15:25:22.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.064+0000 7f842baa9980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 : miss 2024-06-28T15:25:22.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.064+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.064+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:22.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.064+0000 7f842baa9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 info.flags=0x17 2024-06-28T15:25:22.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.064+0000 7f842baa9980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 to cache LRU end 2024-06-28T15:25:22.044 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.064+0000 7f842baa9980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:25:22.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.064+0000 7f842baa9980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:22.045 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.064+0000 7f842baa9980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 2024-06-28T15:25:22.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.072+0000 7f842baa9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 info.flags=0x17 2024-06-28T15:25:22.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.072+0000 7f842baa9980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 to cache LRU end 2024-06-28T15:25:22.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.072+0000 7f842baa9980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:22.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.072+0000 7f842baa9980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29, ofs0, ns] 2024-06-28T15:25:22.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.072+0000 7f830cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595097 cookie 94053090840352 notifier 6677 bl.length()=987 2024-06-28T15:25:22.052 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.072+0000 7f830cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 info.flags=0x17 2024-06-28T15:25:22.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.072+0000 7f830cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 to cache LRU end 2024-06-28T15:25:22.053 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.072+0000 7f830cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:22.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.072+0000 7f842baa9980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): orig_sources= new_sources= 2024-06-28T15:25:22.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.072+0000 7f842baa9980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): potential sources added=[] removed=[] 2024-06-28T15:25:22.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.072+0000 7f842baa9980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): orig_dests= new_dests= 2024-06-28T15:25:22.054 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.072+0000 7f842baa9980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): potential dests added=[] removed=[] 2024-06-28T15:25:22.056 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f842baa9980 time 2024-06-28T15:25:22.078313+0000 2024-06-28T15:25:22.056 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:25:22.057 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:25:22.057 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f842f380c91] 2024-06-28T15:25:22.057 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f842f380e45] 2024-06-28T15:25:22.057 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558a6f5424b1] 2024-06-28T15:25:22.057 INFO:tasks.workunit.client.0.smithi178.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x32c) [0x558a6f553aac] 2024-06-28T15:25:22.058 INFO:tasks.workunit.client.0.smithi178.stderr: 5: main() 2024-06-28T15:25:22.058 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f842e1e9d90] 2024-06-28T15:25:22.058 INFO:tasks.workunit.client.0.smithi178.stderr: 7: __libc_start_main() 2024-06-28T15:25:22.058 INFO:tasks.workunit.client.0.smithi178.stderr: 8: _start() 2024-06-28T15:25:22.058 INFO:tasks.workunit.client.0.smithi178.stderr:*** Caught signal (Aborted) ** 2024-06-28T15:25:22.058 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7f842baa9980 thread_name:radosgw-admin 2024-06-28T15:25:22.058 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.076+0000 7f842baa9980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f842baa9980 time 2024-06-28T15:25:22.078313+0000 2024-06-28T15:25:22.058 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:25:22.058 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.058 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:25:22.058 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f842f380c91] 2024-06-28T15:25:22.058 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f842f380e45] 2024-06-28T15:25:22.058 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558a6f5424b1] 2024-06-28T15:25:22.059 INFO:tasks.workunit.client.0.smithi178.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x32c) [0x558a6f553aac] 2024-06-28T15:25:22.059 INFO:tasks.workunit.client.0.smithi178.stderr: 5: main() 2024-06-28T15:25:22.059 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f842e1e9d90] 2024-06-28T15:25:22.059 INFO:tasks.workunit.client.0.smithi178.stderr: 7: __libc_start_main() 2024-06-28T15:25:22.059 INFO:tasks.workunit.client.0.smithi178.stderr: 8: _start() 2024-06-28T15:25:22.059 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.077 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:25:22.077 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f842e202520] 2024-06-28T15:25:22.077 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:25:22.077 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:25:22.077 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:25:22.078 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f842f380ceb] 2024-06-28T15:25:22.078 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f842f380e45] 2024-06-28T15:25:22.078 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558a6f5424b1] 2024-06-28T15:25:22.078 INFO:tasks.workunit.client.0.smithi178.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x32c) [0x558a6f553aac] 2024-06-28T15:25:22.078 INFO:tasks.workunit.client.0.smithi178.stderr: 9: main() 2024-06-28T15:25:22.078 INFO:tasks.workunit.client.0.smithi178.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f842e1e9d90] 2024-06-28T15:25:22.078 INFO:tasks.workunit.client.0.smithi178.stderr: 11: __libc_start_main() 2024-06-28T15:25:22.078 INFO:tasks.workunit.client.0.smithi178.stderr: 12: _start() 2024-06-28T15:25:22.078 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.076+0000 7f842baa9980 -1 *** Caught signal (Aborted) ** 2024-06-28T15:25:22.078 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7f842baa9980 thread_name:radosgw-admin 2024-06-28T15:25:22.078 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.078 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:25:22.078 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f842e202520] 2024-06-28T15:25:22.079 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:25:22.079 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:25:22.079 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:25:22.079 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f842f380ceb] 2024-06-28T15:25:22.079 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f842f380e45] 2024-06-28T15:25:22.079 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558a6f5424b1] 2024-06-28T15:25:22.079 INFO:tasks.workunit.client.0.smithi178.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x32c) [0x558a6f553aac] 2024-06-28T15:25:22.079 INFO:tasks.workunit.client.0.smithi178.stderr: 9: main() 2024-06-28T15:25:22.079 INFO:tasks.workunit.client.0.smithi178.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f842e1e9d90] 2024-06-28T15:25:22.079 INFO:tasks.workunit.client.0.smithi178.stderr: 11: __libc_start_main() 2024-06-28T15:25:22.079 INFO:tasks.workunit.client.0.smithi178.stderr: 12: _start() 2024-06-28T15:25:22.079 INFO:tasks.workunit.client.0.smithi178.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-28T15:25:22.079 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.079 INFO:tasks.workunit.client.0.smithi178.stderr:--- begin dump of recent events --- 2024-06-28T15:25:22.080 INFO:tasks.workunit.client.0.smithi178.stderr: -499> 2024-06-28T15:25:21.992+0000 7f8428db9640 1 monclient: found mon.noname-a 2024-06-28T15:25:22.080 INFO:tasks.workunit.client.0.smithi178.stderr: -498> 2024-06-28T15:25:21.992+0000 7f8428db9640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:25:22.080 INFO:tasks.workunit.client.0.smithi178.stderr: -497> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.080 INFO:tasks.workunit.client.0.smithi178.stderr: -496> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:25:22.080 INFO:tasks.workunit.client.0.smithi178.stderr: -495> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: dump: 2024-06-28T15:25:22.080 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.080 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.080 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.080 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.080 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.080 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.080 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.080 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.081 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.081 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.081 INFO:tasks.workunit.client.0.smithi178.stderr: -494> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: _finish_auth 0 2024-06-28T15:25:22.081 INFO:tasks.workunit.client.0.smithi178.stderr: -493> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: _check_auth_tickets 2024-06-28T15:25:22.081 INFO:tasks.workunit.client.0.smithi178.stderr: -492> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: handle_config config(0 keys) 2024-06-28T15:25:22.081 INFO:tasks.workunit.client.0.smithi178.stderr: -491> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.081 INFO:tasks.workunit.client.0.smithi178.stderr: -490> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:25:22.081 INFO:tasks.workunit.client.0.smithi178.stderr: -489> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: dump: 2024-06-28T15:25:22.081 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.081 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.081 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.081 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.081 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.081 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.082 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.082 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.082 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.082 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.082 INFO:tasks.workunit.client.0.smithi178.stderr: -488> 2024-06-28T15:25:21.992+0000 7f842baa9980 5 monclient: authenticate success, global_id 7098 2024-06-28T15:25:22.082 INFO:tasks.workunit.client.0.smithi178.stderr: -487> 2024-06-28T15:25:21.992+0000 7f842baa9980 10 monclient: get_monmap_and_config success 2024-06-28T15:25:22.082 INFO:tasks.workunit.client.0.smithi178.stderr: -486> 2024-06-28T15:25:21.992+0000 7f842baa9980 4 set_mon_vals no callback set 2024-06-28T15:25:22.082 INFO:tasks.workunit.client.0.smithi178.stderr: -485> 2024-06-28T15:25:21.992+0000 7f842baa9980 10 monclient: shutdown 2024-06-28T15:25:22.082 INFO:tasks.workunit.client.0.smithi178.stderr: -484> 2024-06-28T15:25:21.992+0000 7f842baa9980 5 asok(0x558a70e185e0) unregister_commands rotate-key 2024-06-28T15:25:22.082 INFO:tasks.workunit.client.0.smithi178.stderr: -483> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: build_initial_monmap 2024-06-28T15:25:22.082 INFO:tasks.workunit.client.0.smithi178.stderr: -482> 2024-06-28T15:25:21.996+0000 7f842baa9980 1 build_initial for_mkfs: 0 2024-06-28T15:25:22.082 INFO:tasks.workunit.client.0.smithi178.stderr: -481> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: monmap: 2024-06-28T15:25:22.082 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:25:22.083 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:25:22.083 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:25:22.083 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:25:22.083 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:25:22.083 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.083 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:25:22.083 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:25:22.083 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:25:22.083 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.083 INFO:tasks.workunit.client.0.smithi178.stderr: -480> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding auth protocol: cephx 2024-06-28T15:25:22.083 INFO:tasks.workunit.client.0.smithi178.stderr: -479> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding auth protocol: cephx 2024-06-28T15:25:22.083 INFO:tasks.workunit.client.0.smithi178.stderr: -478> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding auth protocol: cephx 2024-06-28T15:25:22.083 INFO:tasks.workunit.client.0.smithi178.stderr: -477> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: secure 2024-06-28T15:25:22.083 INFO:tasks.workunit.client.0.smithi178.stderr: -476> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: crc 2024-06-28T15:25:22.084 INFO:tasks.workunit.client.0.smithi178.stderr: -475> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: secure 2024-06-28T15:25:22.084 INFO:tasks.workunit.client.0.smithi178.stderr: -474> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: crc 2024-06-28T15:25:22.084 INFO:tasks.workunit.client.0.smithi178.stderr: -473> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: secure 2024-06-28T15:25:22.084 INFO:tasks.workunit.client.0.smithi178.stderr: -472> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: crc 2024-06-28T15:25:22.084 INFO:tasks.workunit.client.0.smithi178.stderr: -471> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: crc 2024-06-28T15:25:22.084 INFO:tasks.workunit.client.0.smithi178.stderr: -470> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: secure 2024-06-28T15:25:22.084 INFO:tasks.workunit.client.0.smithi178.stderr: -469> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: crc 2024-06-28T15:25:22.084 INFO:tasks.workunit.client.0.smithi178.stderr: -468> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: secure 2024-06-28T15:25:22.084 INFO:tasks.workunit.client.0.smithi178.stderr: -467> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: crc 2024-06-28T15:25:22.084 INFO:tasks.workunit.client.0.smithi178.stderr: -466> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: secure 2024-06-28T15:25:22.084 INFO:tasks.workunit.client.0.smithi178.stderr: -465> 2024-06-28T15:25:21.996+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.084 INFO:tasks.workunit.client.0.smithi178.stderr: -464> 2024-06-28T15:25:21.996+0000 7f842baa9980 1 librados: starting msgr at 2024-06-28T15:25:22.084 INFO:tasks.workunit.client.0.smithi178.stderr: -463> 2024-06-28T15:25:21.996+0000 7f842baa9980 1 librados: starting objecter 2024-06-28T15:25:22.084 INFO:tasks.workunit.client.0.smithi178.stderr: -462> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command objecter_requests hook 0x558a7102a0e0 2024-06-28T15:25:22.085 INFO:tasks.workunit.client.0.smithi178.stderr: -461> 2024-06-28T15:25:21.996+0000 7f842baa9980 1 librados: setting wanted keys 2024-06-28T15:25:22.085 INFO:tasks.workunit.client.0.smithi178.stderr: -460> 2024-06-28T15:25:21.996+0000 7f842baa9980 1 librados: calling monclient init 2024-06-28T15:25:22.085 INFO:tasks.workunit.client.0.smithi178.stderr: -459> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: init 2024-06-28T15:25:22.085 INFO:tasks.workunit.client.0.smithi178.stderr: -458> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding auth protocol: cephx 2024-06-28T15:25:22.085 INFO:tasks.workunit.client.0.smithi178.stderr: -457> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding auth protocol: cephx 2024-06-28T15:25:22.085 INFO:tasks.workunit.client.0.smithi178.stderr: -456> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding auth protocol: cephx 2024-06-28T15:25:22.085 INFO:tasks.workunit.client.0.smithi178.stderr: -455> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: secure 2024-06-28T15:25:22.085 INFO:tasks.workunit.client.0.smithi178.stderr: -454> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: crc 2024-06-28T15:25:22.085 INFO:tasks.workunit.client.0.smithi178.stderr: -453> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: secure 2024-06-28T15:25:22.085 INFO:tasks.workunit.client.0.smithi178.stderr: -452> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: crc 2024-06-28T15:25:22.085 INFO:tasks.workunit.client.0.smithi178.stderr: -451> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: secure 2024-06-28T15:25:22.085 INFO:tasks.workunit.client.0.smithi178.stderr: -450> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: crc 2024-06-28T15:25:22.085 INFO:tasks.workunit.client.0.smithi178.stderr: -449> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: crc 2024-06-28T15:25:22.086 INFO:tasks.workunit.client.0.smithi178.stderr: -448> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: secure 2024-06-28T15:25:22.086 INFO:tasks.workunit.client.0.smithi178.stderr: -447> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: crc 2024-06-28T15:25:22.086 INFO:tasks.workunit.client.0.smithi178.stderr: -446> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: secure 2024-06-28T15:25:22.086 INFO:tasks.workunit.client.0.smithi178.stderr: -445> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: crc 2024-06-28T15:25:22.086 INFO:tasks.workunit.client.0.smithi178.stderr: -444> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: secure 2024-06-28T15:25:22.086 INFO:tasks.workunit.client.0.smithi178.stderr: -443> 2024-06-28T15:25:21.996+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.086 INFO:tasks.workunit.client.0.smithi178.stderr: -442> 2024-06-28T15:25:21.996+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.086 INFO:tasks.workunit.client.0.smithi178.stderr: -441> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command rotate-key hook 0x558a71025788 2024-06-28T15:25:22.086 INFO:tasks.workunit.client.0.smithi178.stderr: -440> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: _reopen_session rank -1 2024-06-28T15:25:22.086 INFO:tasks.workunit.client.0.smithi178.stderr: -439> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: _add_conns ranks=[2,0,1] 2024-06-28T15:25:22.086 INFO:tasks.workunit.client.0.smithi178.stderr: -438> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-c con 0x558a70ed5d30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:25:22.086 INFO:tasks.workunit.client.0.smithi178.stderr: -437> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-a con 0x558a70f6f110 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:25:22.086 INFO:tasks.workunit.client.0.smithi178.stderr: -436> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-b con 0x558a7102af70 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:25:22.087 INFO:tasks.workunit.client.0.smithi178.stderr: -435> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.087 INFO:tasks.workunit.client.0.smithi178.stderr: -434> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.087 INFO:tasks.workunit.client.0.smithi178.stderr: -433> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.087 INFO:tasks.workunit.client.0.smithi178.stderr: -432> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): _renew_subs 2024-06-28T15:25:22.087 INFO:tasks.workunit.client.0.smithi178.stderr: -431> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): authenticate will time out at 2465.457165s 2024-06-28T15:25:22.087 INFO:tasks.workunit.client.0.smithi178.stderr: -430> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): get_auth_request con 0x558a70ed5d30 auth_method 0 2024-06-28T15:25:22.087 INFO:tasks.workunit.client.0.smithi178.stderr: -429> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.087 INFO:tasks.workunit.client.0.smithi178.stderr: -428> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.087 INFO:tasks.workunit.client.0.smithi178.stderr: -427> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.087 INFO:tasks.workunit.client.0.smithi178.stderr: -426> 2024-06-28T15:25:21.996+0000 7f8428db9640 10 monclient(hunting): get_auth_request con 0x558a70f6f110 auth_method 0 2024-06-28T15:25:22.087 INFO:tasks.workunit.client.0.smithi178.stderr: -425> 2024-06-28T15:25:21.996+0000 7f8428db9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.087 INFO:tasks.workunit.client.0.smithi178.stderr: -424> 2024-06-28T15:25:21.996+0000 7f8428db9640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.088 INFO:tasks.workunit.client.0.smithi178.stderr: -423> 2024-06-28T15:25:21.996+0000 7f8428db9640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.088 INFO:tasks.workunit.client.0.smithi178.stderr: -422> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.088 INFO:tasks.workunit.client.0.smithi178.stderr: -421> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.088 INFO:tasks.workunit.client.0.smithi178.stderr: -420> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.088 INFO:tasks.workunit.client.0.smithi178.stderr: -419> 2024-06-28T15:25:21.996+0000 7f8428db9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.088 INFO:tasks.workunit.client.0.smithi178.stderr: -418> 2024-06-28T15:25:21.996+0000 7f8428db9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.088 INFO:tasks.workunit.client.0.smithi178.stderr: -417> 2024-06-28T15:25:21.996+0000 7f8428db9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.088 INFO:tasks.workunit.client.0.smithi178.stderr: -416> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): handle_auth_done global_id 6668 payload 824 2024-06-28T15:25:22.088 INFO:tasks.workunit.client.0.smithi178.stderr: -415> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient: _finish_hunting 0 2024-06-28T15:25:22.088 INFO:tasks.workunit.client.0.smithi178.stderr: -414> 2024-06-28T15:25:21.996+0000 7f84295ba640 1 monclient: found mon.noname-c 2024-06-28T15:25:22.088 INFO:tasks.workunit.client.0.smithi178.stderr: -413> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.088 INFO:tasks.workunit.client.0.smithi178.stderr: -412> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.088 INFO:tasks.workunit.client.0.smithi178.stderr: -411> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:25:22.088 INFO:tasks.workunit.client.0.smithi178.stderr: -410> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: dump: 2024-06-28T15:25:22.089 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.089 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.089 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.089 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.089 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.089 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.089 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.089 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.089 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.089 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.089 INFO:tasks.workunit.client.0.smithi178.stderr: -409> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: _finish_auth 0 2024-06-28T15:25:22.089 INFO:tasks.workunit.client.0.smithi178.stderr: -408> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: _check_auth_tickets 2024-06-28T15:25:22.089 INFO:tasks.workunit.client.0.smithi178.stderr: -407> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: handle_config config(0 keys) 2024-06-28T15:25:22.089 INFO:tasks.workunit.client.0.smithi178.stderr: -406> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.090 INFO:tasks.workunit.client.0.smithi178.stderr: -405> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:25:22.090 INFO:tasks.workunit.client.0.smithi178.stderr: -404> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: dump: 2024-06-28T15:25:22.090 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.090 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.090 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.090 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.090 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.090 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.090 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.090 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.090 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.090 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.090 INFO:tasks.workunit.client.0.smithi178.stderr: -403> 2024-06-28T15:25:21.996+0000 7f8317606640 4 set_mon_vals no callback set 2024-06-28T15:25:22.090 INFO:tasks.workunit.client.0.smithi178.stderr: -402> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 monclient: authenticate success, global_id 6668 2024-06-28T15:25:22.091 INFO:tasks.workunit.client.0.smithi178.stderr: -401> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: _renew_subs 2024-06-28T15:25:22.091 INFO:tasks.workunit.client.0.smithi178.stderr: -400> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.091 INFO:tasks.workunit.client.0.smithi178.stderr: -399> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: _renew_subs 2024-06-28T15:25:22.091 INFO:tasks.workunit.client.0.smithi178.stderr: -398> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.091 INFO:tasks.workunit.client.0.smithi178.stderr: -397> 2024-06-28T15:25:21.996+0000 7f842baa9980 1 librados: init done 2024-06-28T15:25:22.091 INFO:tasks.workunit.client.0.smithi178.stderr: -396> 2024-06-28T15:25:21.996+0000 7f8315e03640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:25:22.091 INFO:tasks.workunit.client.0.smithi178.stderr: -395> 2024-06-28T15:25:21.996+0000 7f8315e03640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:25:22.091 INFO:tasks.workunit.client.0.smithi178.stderr: -394> 2024-06-28T15:25:21.996+0000 7f8315e03640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:25:22.091 INFO:tasks.workunit.client.0.smithi178.stderr: -393> 2024-06-28T15:25:22.000+0000 7f8428db9640 10 monclient: get_auth_request con 0x7f831006f7b0 auth_method 0 2024-06-28T15:25:22.091 INFO:tasks.workunit.client.0.smithi178.stderr: -392> 2024-06-28T15:25:22.000+0000 7f8429dbb640 10 monclient: get_auth_request con 0x558a71066c90 auth_method 0 2024-06-28T15:25:22.091 INFO:tasks.workunit.client.0.smithi178.stderr: -391> 2024-06-28T15:25:22.000+0000 7f84295ba640 10 monclient: get_auth_request con 0x558a7106ff70 auth_method 0 2024-06-28T15:25:22.091 INFO:tasks.workunit.client.0.smithi178.stderr: -390> 2024-06-28T15:25:22.000+0000 7f8428db9640 10 monclient: get_auth_request con 0x558a710742d0 auth_method 0 2024-06-28T15:25:22.091 INFO:tasks.workunit.client.0.smithi178.stderr: -389> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: build_initial_monmap 2024-06-28T15:25:22.091 INFO:tasks.workunit.client.0.smithi178.stderr: -388> 2024-06-28T15:25:22.004+0000 7f842baa9980 1 build_initial for_mkfs: 0 2024-06-28T15:25:22.092 INFO:tasks.workunit.client.0.smithi178.stderr: -387> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: monmap: 2024-06-28T15:25:22.092 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:25:22.092 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:25:22.092 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:25:22.092 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:25:22.092 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:25:22.092 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.092 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:25:22.092 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:25:22.092 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:25:22.092 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.092 INFO:tasks.workunit.client.0.smithi178.stderr: -386> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding auth protocol: cephx 2024-06-28T15:25:22.092 INFO:tasks.workunit.client.0.smithi178.stderr: -385> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding auth protocol: cephx 2024-06-28T15:25:22.092 INFO:tasks.workunit.client.0.smithi178.stderr: -384> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding auth protocol: cephx 2024-06-28T15:25:22.093 INFO:tasks.workunit.client.0.smithi178.stderr: -383> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: secure 2024-06-28T15:25:22.093 INFO:tasks.workunit.client.0.smithi178.stderr: -382> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: crc 2024-06-28T15:25:22.093 INFO:tasks.workunit.client.0.smithi178.stderr: -381> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: secure 2024-06-28T15:25:22.093 INFO:tasks.workunit.client.0.smithi178.stderr: -380> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: crc 2024-06-28T15:25:22.093 INFO:tasks.workunit.client.0.smithi178.stderr: -379> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: secure 2024-06-28T15:25:22.093 INFO:tasks.workunit.client.0.smithi178.stderr: -378> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: crc 2024-06-28T15:25:22.093 INFO:tasks.workunit.client.0.smithi178.stderr: -377> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: crc 2024-06-28T15:25:22.093 INFO:tasks.workunit.client.0.smithi178.stderr: -376> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: secure 2024-06-28T15:25:22.093 INFO:tasks.workunit.client.0.smithi178.stderr: -375> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: crc 2024-06-28T15:25:22.093 INFO:tasks.workunit.client.0.smithi178.stderr: -374> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: secure 2024-06-28T15:25:22.093 INFO:tasks.workunit.client.0.smithi178.stderr: -373> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: crc 2024-06-28T15:25:22.093 INFO:tasks.workunit.client.0.smithi178.stderr: -372> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: secure 2024-06-28T15:25:22.093 INFO:tasks.workunit.client.0.smithi178.stderr: -371> 2024-06-28T15:25:22.004+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.093 INFO:tasks.workunit.client.0.smithi178.stderr: -370> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command objecter_requests cmddesc objecter_requests hook 0x558a7107c7c0 EEXIST 2024-06-28T15:25:22.094 INFO:tasks.workunit.client.0.smithi178.stderr: -369> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: init 2024-06-28T15:25:22.094 INFO:tasks.workunit.client.0.smithi178.stderr: -368> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding auth protocol: cephx 2024-06-28T15:25:22.094 INFO:tasks.workunit.client.0.smithi178.stderr: -367> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding auth protocol: cephx 2024-06-28T15:25:22.094 INFO:tasks.workunit.client.0.smithi178.stderr: -366> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding auth protocol: cephx 2024-06-28T15:25:22.094 INFO:tasks.workunit.client.0.smithi178.stderr: -365> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: secure 2024-06-28T15:25:22.094 INFO:tasks.workunit.client.0.smithi178.stderr: -364> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: crc 2024-06-28T15:25:22.094 INFO:tasks.workunit.client.0.smithi178.stderr: -363> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: secure 2024-06-28T15:25:22.094 INFO:tasks.workunit.client.0.smithi178.stderr: -362> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: crc 2024-06-28T15:25:22.094 INFO:tasks.workunit.client.0.smithi178.stderr: -361> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: secure 2024-06-28T15:25:22.094 INFO:tasks.workunit.client.0.smithi178.stderr: -360> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: crc 2024-06-28T15:25:22.094 INFO:tasks.workunit.client.0.smithi178.stderr: -359> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: crc 2024-06-28T15:25:22.094 INFO:tasks.workunit.client.0.smithi178.stderr: -358> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: secure 2024-06-28T15:25:22.094 INFO:tasks.workunit.client.0.smithi178.stderr: -357> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: crc 2024-06-28T15:25:22.094 INFO:tasks.workunit.client.0.smithi178.stderr: -356> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: secure 2024-06-28T15:25:22.095 INFO:tasks.workunit.client.0.smithi178.stderr: -355> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: crc 2024-06-28T15:25:22.095 INFO:tasks.workunit.client.0.smithi178.stderr: -354> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: secure 2024-06-28T15:25:22.095 INFO:tasks.workunit.client.0.smithi178.stderr: -353> 2024-06-28T15:25:22.004+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.095 INFO:tasks.workunit.client.0.smithi178.stderr: -352> 2024-06-28T15:25:22.004+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.095 INFO:tasks.workunit.client.0.smithi178.stderr: -351> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command rotate-key cmddesc rotate-key hook 0x558a71077a00 EEXIST 2024-06-28T15:25:22.095 INFO:tasks.workunit.client.0.smithi178.stderr: -350> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: _reopen_session rank -1 2024-06-28T15:25:22.095 INFO:tasks.workunit.client.0.smithi178.stderr: -349> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: _add_conns ranks=[0,2,1] 2024-06-28T15:25:22.095 INFO:tasks.workunit.client.0.smithi178.stderr: -348> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-a con 0x558a71082d00 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:25:22.095 INFO:tasks.workunit.client.0.smithi178.stderr: -347> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-c con 0x558a71083630 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:25:22.095 INFO:tasks.workunit.client.0.smithi178.stderr: -346> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-b con 0x558a710830e0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:25:22.095 INFO:tasks.workunit.client.0.smithi178.stderr: -345> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.095 INFO:tasks.workunit.client.0.smithi178.stderr: -344> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.095 INFO:tasks.workunit.client.0.smithi178.stderr: -343> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.095 INFO:tasks.workunit.client.0.smithi178.stderr: -342> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): _renew_subs 2024-06-28T15:25:22.096 INFO:tasks.workunit.client.0.smithi178.stderr: -341> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): authenticate will time out at 2465.466204s 2024-06-28T15:25:22.096 INFO:tasks.workunit.client.0.smithi178.stderr: -340> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): get_auth_request con 0x558a71083630 auth_method 0 2024-06-28T15:25:22.096 INFO:tasks.workunit.client.0.smithi178.stderr: -339> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.096 INFO:tasks.workunit.client.0.smithi178.stderr: -338> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.096 INFO:tasks.workunit.client.0.smithi178.stderr: -337> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.096 INFO:tasks.workunit.client.0.smithi178.stderr: -336> 2024-06-28T15:25:22.004+0000 7f84295ba640 10 monclient(hunting): get_auth_request con 0x558a71082d00 auth_method 0 2024-06-28T15:25:22.096 INFO:tasks.workunit.client.0.smithi178.stderr: -335> 2024-06-28T15:25:22.004+0000 7f84295ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.096 INFO:tasks.workunit.client.0.smithi178.stderr: -334> 2024-06-28T15:25:22.004+0000 7f84295ba640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.096 INFO:tasks.workunit.client.0.smithi178.stderr: -333> 2024-06-28T15:25:22.004+0000 7f84295ba640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.096 INFO:tasks.workunit.client.0.smithi178.stderr: -332> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.096 INFO:tasks.workunit.client.0.smithi178.stderr: -331> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.096 INFO:tasks.workunit.client.0.smithi178.stderr: -330> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.096 INFO:tasks.workunit.client.0.smithi178.stderr: -329> 2024-06-28T15:25:22.004+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.097 INFO:tasks.workunit.client.0.smithi178.stderr: -328> 2024-06-28T15:25:22.004+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.097 INFO:tasks.workunit.client.0.smithi178.stderr: -327> 2024-06-28T15:25:22.004+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.097 INFO:tasks.workunit.client.0.smithi178.stderr: -326> 2024-06-28T15:25:22.004+0000 7f8429dbb640 10 monclient(hunting): get_auth_request con 0x558a710830e0 auth_method 0 2024-06-28T15:25:22.097 INFO:tasks.workunit.client.0.smithi178.stderr: -325> 2024-06-28T15:25:22.004+0000 7f8429dbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.097 INFO:tasks.workunit.client.0.smithi178.stderr: -324> 2024-06-28T15:25:22.004+0000 7f8429dbb640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.097 INFO:tasks.workunit.client.0.smithi178.stderr: -323> 2024-06-28T15:25:22.004+0000 7f8429dbb640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.097 INFO:tasks.workunit.client.0.smithi178.stderr: -322> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): handle_auth_done global_id 6671 payload 824 2024-06-28T15:25:22.097 INFO:tasks.workunit.client.0.smithi178.stderr: -321> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient: _finish_hunting 0 2024-06-28T15:25:22.097 INFO:tasks.workunit.client.0.smithi178.stderr: -320> 2024-06-28T15:25:22.004+0000 7f8428db9640 1 monclient: found mon.noname-c 2024-06-28T15:25:22.097 INFO:tasks.workunit.client.0.smithi178.stderr: -319> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.097 INFO:tasks.workunit.client.0.smithi178.stderr: -318> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.097 INFO:tasks.workunit.client.0.smithi178.stderr: -317> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:25:22.097 INFO:tasks.workunit.client.0.smithi178.stderr: -316> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: dump: 2024-06-28T15:25:22.097 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.098 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.098 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.098 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.098 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.098 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.098 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.098 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.098 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.098 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.098 INFO:tasks.workunit.client.0.smithi178.stderr: -315> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: _finish_auth 0 2024-06-28T15:25:22.098 INFO:tasks.workunit.client.0.smithi178.stderr: -314> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: _check_auth_tickets 2024-06-28T15:25:22.098 INFO:tasks.workunit.client.0.smithi178.stderr: -313> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: handle_config config(0 keys) 2024-06-28T15:25:22.098 INFO:tasks.workunit.client.0.smithi178.stderr: -312> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.099 INFO:tasks.workunit.client.0.smithi178.stderr: -311> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:25:22.099 INFO:tasks.workunit.client.0.smithi178.stderr: -310> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: dump: 2024-06-28T15:25:22.099 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.099 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.099 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.099 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.099 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.099 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.099 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.099 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.099 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.099 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.099 INFO:tasks.workunit.client.0.smithi178.stderr: -309> 2024-06-28T15:25:22.004+0000 7f842a5bc640 4 set_mon_vals no callback set 2024-06-28T15:25:22.100 INFO:tasks.workunit.client.0.smithi178.stderr: -308> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 monclient: authenticate success, global_id 6671 2024-06-28T15:25:22.100 INFO:tasks.workunit.client.0.smithi178.stderr: -307> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: _renew_subs 2024-06-28T15:25:22.100 INFO:tasks.workunit.client.0.smithi178.stderr: -306> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.100 INFO:tasks.workunit.client.0.smithi178.stderr: -305> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: _renew_subs 2024-06-28T15:25:22.100 INFO:tasks.workunit.client.0.smithi178.stderr: -304> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.100 INFO:tasks.workunit.client.0.smithi178.stderr: -303> 2024-06-28T15:25:22.008+0000 7f830effd640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:25:22.100 INFO:tasks.workunit.client.0.smithi178.stderr: -302> 2024-06-28T15:25:22.008+0000 7f830effd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:25:22.100 INFO:tasks.workunit.client.0.smithi178.stderr: -301> 2024-06-28T15:25:22.008+0000 7f830effd640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:25:22.100 INFO:tasks.workunit.client.0.smithi178.stderr: -300> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient: get_auth_request con 0x7f830406f690 auth_method 0 2024-06-28T15:25:22.100 INFO:tasks.workunit.client.0.smithi178.stderr: -299> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient: get_monmap_and_config 2024-06-28T15:25:22.100 INFO:tasks.workunit.client.0.smithi178.stderr: -298> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient: build_initial_monmap 2024-06-28T15:25:22.100 INFO:tasks.workunit.client.0.smithi178.stderr: -297> 2024-06-28T15:25:22.008+0000 7f842baa9980 1 build_initial for_mkfs: 0 2024-06-28T15:25:22.100 INFO:tasks.workunit.client.0.smithi178.stderr: -296> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient: monmap: 2024-06-28T15:25:22.100 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:25:22.101 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:25:22.101 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:25:22.101 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:25:22.101 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:25:22.101 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.101 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:25:22.101 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:25:22.101 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:25:22.101 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.101 INFO:tasks.workunit.client.0.smithi178.stderr: -295> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding auth protocol: cephx 2024-06-28T15:25:22.101 INFO:tasks.workunit.client.0.smithi178.stderr: -294> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding auth protocol: cephx 2024-06-28T15:25:22.101 INFO:tasks.workunit.client.0.smithi178.stderr: -293> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding auth protocol: cephx 2024-06-28T15:25:22.101 INFO:tasks.workunit.client.0.smithi178.stderr: -292> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: secure 2024-06-28T15:25:22.102 INFO:tasks.workunit.client.0.smithi178.stderr: -291> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: crc 2024-06-28T15:25:22.102 INFO:tasks.workunit.client.0.smithi178.stderr: -290> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: secure 2024-06-28T15:25:22.102 INFO:tasks.workunit.client.0.smithi178.stderr: -289> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: crc 2024-06-28T15:25:22.102 INFO:tasks.workunit.client.0.smithi178.stderr: -288> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: secure 2024-06-28T15:25:22.102 INFO:tasks.workunit.client.0.smithi178.stderr: -287> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: crc 2024-06-28T15:25:22.102 INFO:tasks.workunit.client.0.smithi178.stderr: -286> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: crc 2024-06-28T15:25:22.102 INFO:tasks.workunit.client.0.smithi178.stderr: -285> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: secure 2024-06-28T15:25:22.102 INFO:tasks.workunit.client.0.smithi178.stderr: -284> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: crc 2024-06-28T15:25:22.102 INFO:tasks.workunit.client.0.smithi178.stderr: -283> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: secure 2024-06-28T15:25:22.102 INFO:tasks.workunit.client.0.smithi178.stderr: -282> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: crc 2024-06-28T15:25:22.102 INFO:tasks.workunit.client.0.smithi178.stderr: -281> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: secure 2024-06-28T15:25:22.102 INFO:tasks.workunit.client.0.smithi178.stderr: -280> 2024-06-28T15:25:22.008+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.102 INFO:tasks.workunit.client.0.smithi178.stderr: -279> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient: init 2024-06-28T15:25:22.103 INFO:tasks.workunit.client.0.smithi178.stderr: -278> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding auth protocol: cephx 2024-06-28T15:25:22.103 INFO:tasks.workunit.client.0.smithi178.stderr: -277> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding auth protocol: cephx 2024-06-28T15:25:22.103 INFO:tasks.workunit.client.0.smithi178.stderr: -276> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding auth protocol: cephx 2024-06-28T15:25:22.103 INFO:tasks.workunit.client.0.smithi178.stderr: -275> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: secure 2024-06-28T15:25:22.103 INFO:tasks.workunit.client.0.smithi178.stderr: -274> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: crc 2024-06-28T15:25:22.103 INFO:tasks.workunit.client.0.smithi178.stderr: -273> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: secure 2024-06-28T15:25:22.103 INFO:tasks.workunit.client.0.smithi178.stderr: -272> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: crc 2024-06-28T15:25:22.103 INFO:tasks.workunit.client.0.smithi178.stderr: -271> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: secure 2024-06-28T15:25:22.103 INFO:tasks.workunit.client.0.smithi178.stderr: -270> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: crc 2024-06-28T15:25:22.103 INFO:tasks.workunit.client.0.smithi178.stderr: -269> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: crc 2024-06-28T15:25:22.103 INFO:tasks.workunit.client.0.smithi178.stderr: -268> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: secure 2024-06-28T15:25:22.103 INFO:tasks.workunit.client.0.smithi178.stderr: -267> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: crc 2024-06-28T15:25:22.103 INFO:tasks.workunit.client.0.smithi178.stderr: -266> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: secure 2024-06-28T15:25:22.104 INFO:tasks.workunit.client.0.smithi178.stderr: -265> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: crc 2024-06-28T15:25:22.104 INFO:tasks.workunit.client.0.smithi178.stderr: -264> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: secure 2024-06-28T15:25:22.104 INFO:tasks.workunit.client.0.smithi178.stderr: -263> 2024-06-28T15:25:22.008+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.104 INFO:tasks.workunit.client.0.smithi178.stderr: -262> 2024-06-28T15:25:22.008+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.104 INFO:tasks.workunit.client.0.smithi178.stderr: -261> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command rotate-key cmddesc rotate-key hook 0x7ffe25e5c8e8 EEXIST 2024-06-28T15:25:22.104 INFO:tasks.workunit.client.0.smithi178.stderr: -260> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient: _reopen_session rank -1 2024-06-28T15:25:22.104 INFO:tasks.workunit.client.0.smithi178.stderr: -259> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient: _add_conns ranks=[0,1,2] 2024-06-28T15:25:22.104 INFO:tasks.workunit.client.0.smithi178.stderr: -258> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-a con 0x558a710f0130 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:25:22.104 INFO:tasks.workunit.client.0.smithi178.stderr: -257> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-b con 0x558a710f0510 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:25:22.104 INFO:tasks.workunit.client.0.smithi178.stderr: -256> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-c con 0x558a710f4b10 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:25:22.104 INFO:tasks.workunit.client.0.smithi178.stderr: -255> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.104 INFO:tasks.workunit.client.0.smithi178.stderr: -254> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.104 INFO:tasks.workunit.client.0.smithi178.stderr: -253> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.104 INFO:tasks.workunit.client.0.smithi178.stderr: -252> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): _renew_subs 2024-06-28T15:25:22.105 INFO:tasks.workunit.client.0.smithi178.stderr: -251> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): authenticate will time out at 2465.470946s 2024-06-28T15:25:22.105 INFO:tasks.workunit.client.0.smithi178.stderr: -250> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): get_auth_request con 0x558a710f0130 auth_method 0 2024-06-28T15:25:22.105 INFO:tasks.workunit.client.0.smithi178.stderr: -249> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.105 INFO:tasks.workunit.client.0.smithi178.stderr: -248> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.105 INFO:tasks.workunit.client.0.smithi178.stderr: -247> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.105 INFO:tasks.workunit.client.0.smithi178.stderr: -246> 2024-06-28T15:25:22.008+0000 7f8429dbb640 10 monclient(hunting): get_auth_request con 0x558a710f4b10 auth_method 0 2024-06-28T15:25:22.105 INFO:tasks.workunit.client.0.smithi178.stderr: -245> 2024-06-28T15:25:22.008+0000 7f8429dbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.105 INFO:tasks.workunit.client.0.smithi178.stderr: -244> 2024-06-28T15:25:22.008+0000 7f8429dbb640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.105 INFO:tasks.workunit.client.0.smithi178.stderr: -243> 2024-06-28T15:25:22.008+0000 7f8429dbb640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.105 INFO:tasks.workunit.client.0.smithi178.stderr: -242> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.105 INFO:tasks.workunit.client.0.smithi178.stderr: -241> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.105 INFO:tasks.workunit.client.0.smithi178.stderr: -240> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.105 INFO:tasks.workunit.client.0.smithi178.stderr: -239> 2024-06-28T15:25:22.008+0000 7f8429dbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.105 INFO:tasks.workunit.client.0.smithi178.stderr: -238> 2024-06-28T15:25:22.008+0000 7f8429dbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.106 INFO:tasks.workunit.client.0.smithi178.stderr: -237> 2024-06-28T15:25:22.008+0000 7f8429dbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.106 INFO:tasks.workunit.client.0.smithi178.stderr: -236> 2024-06-28T15:25:22.008+0000 7f8428db9640 10 monclient(hunting): get_auth_request con 0x558a710f0510 auth_method 0 2024-06-28T15:25:22.106 INFO:tasks.workunit.client.0.smithi178.stderr: -235> 2024-06-28T15:25:22.008+0000 7f8428db9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.106 INFO:tasks.workunit.client.0.smithi178.stderr: -234> 2024-06-28T15:25:22.008+0000 7f8428db9640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.106 INFO:tasks.workunit.client.0.smithi178.stderr: -233> 2024-06-28T15:25:22.008+0000 7f8428db9640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.106 INFO:tasks.workunit.client.0.smithi178.stderr: -232> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): handle_auth_done global_id 7107 payload 274 2024-06-28T15:25:22.106 INFO:tasks.workunit.client.0.smithi178.stderr: -231> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient: _finish_hunting 0 2024-06-28T15:25:22.106 INFO:tasks.workunit.client.0.smithi178.stderr: -230> 2024-06-28T15:25:22.008+0000 7f84295ba640 1 monclient: found mon.noname-a 2024-06-28T15:25:22.106 INFO:tasks.workunit.client.0.smithi178.stderr: -229> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:25:22.106 INFO:tasks.workunit.client.0.smithi178.stderr: -228> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.106 INFO:tasks.workunit.client.0.smithi178.stderr: -227> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:25:22.106 INFO:tasks.workunit.client.0.smithi178.stderr: -226> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: dump: 2024-06-28T15:25:22.106 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.107 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.107 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.107 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.107 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.107 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.107 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.107 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.107 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.107 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.107 INFO:tasks.workunit.client.0.smithi178.stderr: -225> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: _finish_auth 0 2024-06-28T15:25:22.107 INFO:tasks.workunit.client.0.smithi178.stderr: -224> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: _check_auth_tickets 2024-06-28T15:25:22.107 INFO:tasks.workunit.client.0.smithi178.stderr: -223> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: handle_config config(0 keys) 2024-06-28T15:25:22.107 INFO:tasks.workunit.client.0.smithi178.stderr: -222> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr: -221> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr: -220> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: dump: 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr: -219> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 monclient: authenticate success, global_id 7107 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr: -218> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient: get_monmap_and_config success 2024-06-28T15:25:22.108 INFO:tasks.workunit.client.0.smithi178.stderr: -217> 2024-06-28T15:25:22.008+0000 7f842baa9980 4 set_mon_vals no callback set 2024-06-28T15:25:22.109 INFO:tasks.workunit.client.0.smithi178.stderr: -216> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: shutdown 2024-06-28T15:25:22.109 INFO:tasks.workunit.client.0.smithi178.stderr: -215> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: build_initial_monmap 2024-06-28T15:25:22.109 INFO:tasks.workunit.client.0.smithi178.stderr: -214> 2024-06-28T15:25:22.012+0000 7f842baa9980 1 build_initial for_mkfs: 0 2024-06-28T15:25:22.109 INFO:tasks.workunit.client.0.smithi178.stderr: -213> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: monmap: 2024-06-28T15:25:22.109 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:25:22.109 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:25:22.109 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:25:22.109 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:25:22.109 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:25:22.109 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.109 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:25:22.109 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:25:22.109 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:25:22.109 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.110 INFO:tasks.workunit.client.0.smithi178.stderr: -212> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding auth protocol: cephx 2024-06-28T15:25:22.110 INFO:tasks.workunit.client.0.smithi178.stderr: -211> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding auth protocol: cephx 2024-06-28T15:25:22.110 INFO:tasks.workunit.client.0.smithi178.stderr: -210> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding auth protocol: cephx 2024-06-28T15:25:22.110 INFO:tasks.workunit.client.0.smithi178.stderr: -209> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: secure 2024-06-28T15:25:22.110 INFO:tasks.workunit.client.0.smithi178.stderr: -208> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: crc 2024-06-28T15:25:22.110 INFO:tasks.workunit.client.0.smithi178.stderr: -207> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: secure 2024-06-28T15:25:22.110 INFO:tasks.workunit.client.0.smithi178.stderr: -206> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: crc 2024-06-28T15:25:22.110 INFO:tasks.workunit.client.0.smithi178.stderr: -205> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: secure 2024-06-28T15:25:22.110 INFO:tasks.workunit.client.0.smithi178.stderr: -204> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: crc 2024-06-28T15:25:22.110 INFO:tasks.workunit.client.0.smithi178.stderr: -203> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: crc 2024-06-28T15:25:22.110 INFO:tasks.workunit.client.0.smithi178.stderr: -202> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: secure 2024-06-28T15:25:22.110 INFO:tasks.workunit.client.0.smithi178.stderr: -201> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: crc 2024-06-28T15:25:22.110 INFO:tasks.workunit.client.0.smithi178.stderr: -200> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: secure 2024-06-28T15:25:22.110 INFO:tasks.workunit.client.0.smithi178.stderr: -199> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: crc 2024-06-28T15:25:22.111 INFO:tasks.workunit.client.0.smithi178.stderr: -198> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: secure 2024-06-28T15:25:22.111 INFO:tasks.workunit.client.0.smithi178.stderr: -197> 2024-06-28T15:25:22.012+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.111 INFO:tasks.workunit.client.0.smithi178.stderr: -196> 2024-06-28T15:25:22.012+0000 7f842baa9980 1 librados: starting msgr at 2024-06-28T15:25:22.111 INFO:tasks.workunit.client.0.smithi178.stderr: -195> 2024-06-28T15:25:22.012+0000 7f842baa9980 1 librados: starting objecter 2024-06-28T15:25:22.111 INFO:tasks.workunit.client.0.smithi178.stderr: -194> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command objecter_requests cmddesc objecter_requests hook 0x558a710c2300 EEXIST 2024-06-28T15:25:22.111 INFO:tasks.workunit.client.0.smithi178.stderr: -193> 2024-06-28T15:25:22.012+0000 7f842baa9980 1 librados: setting wanted keys 2024-06-28T15:25:22.111 INFO:tasks.workunit.client.0.smithi178.stderr: -192> 2024-06-28T15:25:22.012+0000 7f842baa9980 1 librados: calling monclient init 2024-06-28T15:25:22.111 INFO:tasks.workunit.client.0.smithi178.stderr: -191> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: init 2024-06-28T15:25:22.111 INFO:tasks.workunit.client.0.smithi178.stderr: -190> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding auth protocol: cephx 2024-06-28T15:25:22.111 INFO:tasks.workunit.client.0.smithi178.stderr: -189> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding auth protocol: cephx 2024-06-28T15:25:22.111 INFO:tasks.workunit.client.0.smithi178.stderr: -188> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding auth protocol: cephx 2024-06-28T15:25:22.111 INFO:tasks.workunit.client.0.smithi178.stderr: -187> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: secure 2024-06-28T15:25:22.111 INFO:tasks.workunit.client.0.smithi178.stderr: -186> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: crc 2024-06-28T15:25:22.111 INFO:tasks.workunit.client.0.smithi178.stderr: -185> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: secure 2024-06-28T15:25:22.112 INFO:tasks.workunit.client.0.smithi178.stderr: -184> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: crc 2024-06-28T15:25:22.112 INFO:tasks.workunit.client.0.smithi178.stderr: -183> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: secure 2024-06-28T15:25:22.112 INFO:tasks.workunit.client.0.smithi178.stderr: -182> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: crc 2024-06-28T15:25:22.112 INFO:tasks.workunit.client.0.smithi178.stderr: -181> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: crc 2024-06-28T15:25:22.112 INFO:tasks.workunit.client.0.smithi178.stderr: -180> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: secure 2024-06-28T15:25:22.112 INFO:tasks.workunit.client.0.smithi178.stderr: -179> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: crc 2024-06-28T15:25:22.112 INFO:tasks.workunit.client.0.smithi178.stderr: -178> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: secure 2024-06-28T15:25:22.112 INFO:tasks.workunit.client.0.smithi178.stderr: -177> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: crc 2024-06-28T15:25:22.112 INFO:tasks.workunit.client.0.smithi178.stderr: -176> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: secure 2024-06-28T15:25:22.112 INFO:tasks.workunit.client.0.smithi178.stderr: -175> 2024-06-28T15:25:22.012+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.112 INFO:tasks.workunit.client.0.smithi178.stderr: -174> 2024-06-28T15:25:22.012+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.112 INFO:tasks.workunit.client.0.smithi178.stderr: -173> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command rotate-key cmddesc rotate-key hook 0x558a710b8888 EEXIST 2024-06-28T15:25:22.112 INFO:tasks.workunit.client.0.smithi178.stderr: -172> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: _reopen_session rank -1 2024-06-28T15:25:22.112 INFO:tasks.workunit.client.0.smithi178.stderr: -171> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: _add_conns ranks=[2,1,0] 2024-06-28T15:25:22.113 INFO:tasks.workunit.client.0.smithi178.stderr: -170> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-c con 0x558a710f0130 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:25:22.113 INFO:tasks.workunit.client.0.smithi178.stderr: -169> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-b con 0x558a710f4b10 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:25:22.113 INFO:tasks.workunit.client.0.smithi178.stderr: -168> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-a con 0x558a710f1cc0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:25:22.113 INFO:tasks.workunit.client.0.smithi178.stderr: -167> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.113 INFO:tasks.workunit.client.0.smithi178.stderr: -166> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.113 INFO:tasks.workunit.client.0.smithi178.stderr: -165> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.113 INFO:tasks.workunit.client.0.smithi178.stderr: -164> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): _renew_subs 2024-06-28T15:25:22.113 INFO:tasks.workunit.client.0.smithi178.stderr: -163> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): authenticate will time out at 2465.473553s 2024-06-28T15:25:22.113 INFO:tasks.workunit.client.0.smithi178.stderr: -162> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): get_auth_request con 0x558a710f0130 auth_method 0 2024-06-28T15:25:22.113 INFO:tasks.workunit.client.0.smithi178.stderr: -161> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.113 INFO:tasks.workunit.client.0.smithi178.stderr: -160> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.113 INFO:tasks.workunit.client.0.smithi178.stderr: -159> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.113 INFO:tasks.workunit.client.0.smithi178.stderr: -158> 2024-06-28T15:25:22.012+0000 7f8429dbb640 10 monclient(hunting): get_auth_request con 0x558a710f1cc0 auth_method 0 2024-06-28T15:25:22.113 INFO:tasks.workunit.client.0.smithi178.stderr: -157> 2024-06-28T15:25:22.012+0000 7f8429dbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.114 INFO:tasks.workunit.client.0.smithi178.stderr: -156> 2024-06-28T15:25:22.012+0000 7f8429dbb640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.114 INFO:tasks.workunit.client.0.smithi178.stderr: -155> 2024-06-28T15:25:22.012+0000 7f8429dbb640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.114 INFO:tasks.workunit.client.0.smithi178.stderr: -154> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.114 INFO:tasks.workunit.client.0.smithi178.stderr: -153> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.114 INFO:tasks.workunit.client.0.smithi178.stderr: -152> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.114 INFO:tasks.workunit.client.0.smithi178.stderr: -151> 2024-06-28T15:25:22.012+0000 7f8429dbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.114 INFO:tasks.workunit.client.0.smithi178.stderr: -150> 2024-06-28T15:25:22.012+0000 7f8429dbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.114 INFO:tasks.workunit.client.0.smithi178.stderr: -149> 2024-06-28T15:25:22.012+0000 7f8429dbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.114 INFO:tasks.workunit.client.0.smithi178.stderr: -148> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): handle_auth_done global_id 6677 payload 824 2024-06-28T15:25:22.114 INFO:tasks.workunit.client.0.smithi178.stderr: -147> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient: _finish_hunting 0 2024-06-28T15:25:22.114 INFO:tasks.workunit.client.0.smithi178.stderr: -146> 2024-06-28T15:25:22.012+0000 7f84295ba640 1 monclient: found mon.noname-c 2024-06-28T15:25:22.114 INFO:tasks.workunit.client.0.smithi178.stderr: -145> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.114 INFO:tasks.workunit.client.0.smithi178.stderr: -144> 2024-06-28T15:25:22.012+0000 7f8428db9640 10 monclient: get_auth_request con 0x558a710f4b10 auth_method 0 2024-06-28T15:25:22.114 INFO:tasks.workunit.client.0.smithi178.stderr: -143> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.115 INFO:tasks.workunit.client.0.smithi178.stderr: -142> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:25:22.115 INFO:tasks.workunit.client.0.smithi178.stderr: -141> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: dump: 2024-06-28T15:25:22.115 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.115 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.115 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.115 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.115 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.115 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.115 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.115 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.115 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.115 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.115 INFO:tasks.workunit.client.0.smithi178.stderr: -140> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: _finish_auth 0 2024-06-28T15:25:22.115 INFO:tasks.workunit.client.0.smithi178.stderr: -139> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: _check_auth_tickets 2024-06-28T15:25:22.116 INFO:tasks.workunit.client.0.smithi178.stderr: -138> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 monclient: authenticate success, global_id 6677 2024-06-28T15:25:22.116 INFO:tasks.workunit.client.0.smithi178.stderr: -137> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: _renew_subs 2024-06-28T15:25:22.116 INFO:tasks.workunit.client.0.smithi178.stderr: -136> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.116 INFO:tasks.workunit.client.0.smithi178.stderr: -135> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: handle_config config(0 keys) 2024-06-28T15:25:22.116 INFO:tasks.workunit.client.0.smithi178.stderr: -134> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.116 INFO:tasks.workunit.client.0.smithi178.stderr: -133> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:25:22.116 INFO:tasks.workunit.client.0.smithi178.stderr: -132> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: dump: 2024-06-28T15:25:22.116 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.116 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.116 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.116 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.116 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.116 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.116 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.117 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.117 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.117 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.117 INFO:tasks.workunit.client.0.smithi178.stderr: -131> 2024-06-28T15:25:22.012+0000 7f830cff9640 4 set_mon_vals no callback set 2024-06-28T15:25:22.117 INFO:tasks.workunit.client.0.smithi178.stderr: -130> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: _renew_subs 2024-06-28T15:25:22.117 INFO:tasks.workunit.client.0.smithi178.stderr: -129> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.117 INFO:tasks.workunit.client.0.smithi178.stderr: -128> 2024-06-28T15:25:22.012+0000 7f842baa9980 1 librados: init done 2024-06-28T15:25:22.117 INFO:tasks.workunit.client.0.smithi178.stderr: -127> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command cr dump hook 0x558a710f3d58 2024-06-28T15:25:22.117 INFO:tasks.workunit.client.0.smithi178.stderr: -126> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.117 INFO:tasks.workunit.client.0.smithi178.stderr: -125> 2024-06-28T15:25:22.016+0000 7f82faffd640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:25:22.117 INFO:tasks.workunit.client.0.smithi178.stderr: -124> 2024-06-28T15:25:22.016+0000 7f82faffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:25:22.117 INFO:tasks.workunit.client.0.smithi178.stderr: -123> 2024-06-28T15:25:22.016+0000 7f82faffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:25:22.117 INFO:tasks.workunit.client.0.smithi178.stderr: -122> 2024-06-28T15:25:22.016+0000 7f8428db9640 10 monclient: get_auth_request con 0x7f82f406f7b0 auth_method 0 2024-06-28T15:25:22.117 INFO:tasks.workunit.client.0.smithi178.stderr: -121> 2024-06-28T15:25:22.016+0000 7f8429dbb640 10 monclient: get_auth_request con 0x558a7110e330 auth_method 0 2024-06-28T15:25:22.118 INFO:tasks.workunit.client.0.smithi178.stderr: -120> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.118 INFO:tasks.workunit.client.0.smithi178.stderr: -119> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 realm 2024-06-28T15:25:22.118 INFO:tasks.workunit.client.0.smithi178.stderr: -118> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.118 INFO:tasks.workunit.client.0.smithi178.stderr: -117> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.118 INFO:tasks.workunit.client.0.smithi178.stderr: -116> 2024-06-28T15:25:22.016+0000 7f842baa9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:22.118 INFO:tasks.workunit.client.0.smithi178.stderr: -115> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.118 INFO:tasks.workunit.client.0.smithi178.stderr: -114> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.118 INFO:tasks.workunit.client.0.smithi178.stderr: -113> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.118 INFO:tasks.workunit.client.0.smithi178.stderr: -112> 2024-06-28T15:25:22.016+0000 7f84295ba640 10 monclient: get_auth_request con 0x558a71112620 auth_method 0 2024-06-28T15:25:22.118 INFO:tasks.workunit.client.0.smithi178.stderr: -111> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:22.118 INFO:tasks.workunit.client.0.smithi178.stderr: -110> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.118 INFO:tasks.workunit.client.0.smithi178.stderr: -109> 2024-06-28T15:25:22.016+0000 7f8428db9640 10 monclient: get_auth_request con 0x558a71115ec0 auth_method 0 2024-06-28T15:25:22.118 INFO:tasks.workunit.client.0.smithi178.stderr: -108> 2024-06-28T15:25:22.020+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:22.119 INFO:tasks.workunit.client.0.smithi178.stderr: -107> 2024-06-28T15:25:22.020+0000 7f842baa9980 20 searching for the correct realm 2024-06-28T15:25:22.119 INFO:tasks.workunit.client.0.smithi178.stderr: -106> 2024-06-28T15:25:22.020+0000 7f8429dbb640 10 monclient: get_auth_request con 0x7f84200e4060 auth_method 0 2024-06-28T15:25:22.119 INFO:tasks.workunit.client.0.smithi178.stderr: -105> 2024-06-28T15:25:22.020+0000 7f84295ba640 10 monclient: get_auth_request con 0x7f8424020040 auth_method 0 2024-06-28T15:25:22.119 INFO:tasks.workunit.client.0.smithi178.stderr: -104> 2024-06-28T15:25:22.020+0000 7f8428db9640 10 monclient: get_auth_request con 0x7f841c06d020 auth_method 0 2024-06-28T15:25:22.119 INFO:tasks.workunit.client.0.smithi178.stderr: -103> 2024-06-28T15:25:22.024+0000 7f8429dbb640 0 -- 172.21.15.178:0/636795418 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x558a7110e330 msgr2=0x558a711107f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:22.119 INFO:tasks.workunit.client.0.smithi178.stderr: -102> 2024-06-28T15:25:22.024+0000 7f82faffd640 1 client.6677.objecter ms_handle_reset 0x558a7110e330 session 0x558a7110dc60 osd.1 2024-06-28T15:25:22.119 INFO:tasks.workunit.client.0.smithi178.stderr: -101> 2024-06-28T15:25:22.024+0000 7f82faffd640 10 monclient: _renew_subs 2024-06-28T15:25:22.119 INFO:tasks.workunit.client.0.smithi178.stderr: -100> 2024-06-28T15:25:22.024+0000 7f82faffd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.119 INFO:tasks.workunit.client.0.smithi178.stderr: -99> 2024-06-28T15:25:22.024+0000 7f8429dbb640 10 monclient: get_auth_request con 0x7f82f407fdd0 auth_method 0 2024-06-28T15:25:22.119 INFO:tasks.workunit.client.0.smithi178.stderr: -98> 2024-06-28T15:25:22.024+0000 7f8429dbb640 10 monclient: get_auth_request con 0x7f84200cd040 auth_method 0 2024-06-28T15:25:22.119 INFO:tasks.workunit.client.0.smithi178.stderr: -97> 2024-06-28T15:25:22.028+0000 7f84295ba640 10 monclient: get_auth_request con 0x7f8420107b00 auth_method 0 2024-06-28T15:25:22.119 INFO:tasks.workunit.client.0.smithi178.stderr: -96> 2024-06-28T15:25:22.028+0000 7f8428db9640 0 -- 172.21.15.178:0/636795418 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f841c06d020 msgr2=0x7f841c05a560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:22.119 INFO:tasks.workunit.client.0.smithi178.stderr: -95> 2024-06-28T15:25:22.028+0000 7f82faffd640 1 client.6677.objecter ms_handle_reset 0x7f841c06d020 session 0x7f841c047040 osd.4 2024-06-28T15:25:22.120 INFO:tasks.workunit.client.0.smithi178.stderr: -94> 2024-06-28T15:25:22.032+0000 7f8428db9640 10 monclient: get_auth_request con 0x558a7110e330 auth_method 0 2024-06-28T15:25:22.120 INFO:tasks.workunit.client.0.smithi178.stderr: -93> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:22.120 INFO:tasks.workunit.client.0.smithi178.stderr: -92> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:22.120 INFO:tasks.workunit.client.0.smithi178.stderr: -91> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:22.120 INFO:tasks.workunit.client.0.smithi178.stderr: -90> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:22.120 INFO:tasks.workunit.client.0.smithi178.stderr: -89> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:22.120 INFO:tasks.workunit.client.0.smithi178.stderr: -88> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:22.120 INFO:tasks.workunit.client.0.smithi178.stderr: -87> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.120 INFO:tasks.workunit.client.0.smithi178.stderr: -86> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.120 INFO:tasks.workunit.client.0.smithi178.stderr: -85> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.120 INFO:tasks.workunit.client.0.smithi178.stderr: -84> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:22.120 INFO:tasks.workunit.client.0.smithi178.stderr: -83> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.120 INFO:tasks.workunit.client.0.smithi178.stderr: -82> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:22.120 INFO:tasks.workunit.client.0.smithi178.stderr: -81> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 zone default found 2024-06-28T15:25:22.121 INFO:tasks.workunit.client.0.smithi178.stderr: -80> 2024-06-28T15:25:22.040+0000 7f842baa9980 4 Realm: () 2024-06-28T15:25:22.121 INFO:tasks.workunit.client.0.smithi178.stderr: -79> 2024-06-28T15:25:22.040+0000 7f842baa9980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:22.121 INFO:tasks.workunit.client.0.smithi178.stderr: -78> 2024-06-28T15:25:22.040+0000 7f842baa9980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:22.121 INFO:tasks.workunit.client.0.smithi178.stderr: -77> 2024-06-28T15:25:22.040+0000 7f842baa9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:22.121 INFO:tasks.workunit.client.0.smithi178.stderr: -76> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 zonegroup default 2024-06-28T15:25:22.121 INFO:tasks.workunit.client.0.smithi178.stderr: -75> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.121 INFO:tasks.workunit.client.0.smithi178.stderr: -74> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.121 INFO:tasks.workunit.client.0.smithi178.stderr: -73> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.121 INFO:tasks.workunit.client.0.smithi178.stderr: -72> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.121 INFO:tasks.workunit.client.0.smithi178.stderr: -71> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.121 INFO:tasks.workunit.client.0.smithi178.stderr: -70> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.121 INFO:tasks.workunit.client.0.smithi178.stderr: -69> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 started sync module instance, tier type = 2024-06-28T15:25:22.121 INFO:tasks.workunit.client.0.smithi178.stderr: -68> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:22.121 INFO:tasks.workunit.client.0.smithi178.stderr: -67> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=0 2024-06-28T15:25:22.122 INFO:tasks.workunit.client.0.smithi178.stderr: -66> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=1 2024-06-28T15:25:22.122 INFO:tasks.workunit.client.0.smithi178.stderr: -65> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=2 2024-06-28T15:25:22.122 INFO:tasks.workunit.client.0.smithi178.stderr: -64> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=3 2024-06-28T15:25:22.122 INFO:tasks.workunit.client.0.smithi178.stderr: -63> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=4 2024-06-28T15:25:22.122 INFO:tasks.workunit.client.0.smithi178.stderr: -62> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=5 2024-06-28T15:25:22.122 INFO:tasks.workunit.client.0.smithi178.stderr: -61> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=6 2024-06-28T15:25:22.122 INFO:tasks.workunit.client.0.smithi178.stderr: -60> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=7 2024-06-28T15:25:22.122 INFO:tasks.workunit.client.0.smithi178.stderr: -59> 2024-06-28T15:25:22.056+0000 7f842baa9980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:22.122 INFO:tasks.workunit.client.0.smithi178.stderr: -58> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding auth protocol: cephx 2024-06-28T15:25:22.122 INFO:tasks.workunit.client.0.smithi178.stderr: -57> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding auth protocol: cephx 2024-06-28T15:25:22.122 INFO:tasks.workunit.client.0.smithi178.stderr: -56> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding auth protocol: cephx 2024-06-28T15:25:22.122 INFO:tasks.workunit.client.0.smithi178.stderr: -55> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: secure 2024-06-28T15:25:22.122 INFO:tasks.workunit.client.0.smithi178.stderr: -54> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: crc 2024-06-28T15:25:22.122 INFO:tasks.workunit.client.0.smithi178.stderr: -53> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: secure 2024-06-28T15:25:22.123 INFO:tasks.workunit.client.0.smithi178.stderr: -52> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: crc 2024-06-28T15:25:22.123 INFO:tasks.workunit.client.0.smithi178.stderr: -51> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: secure 2024-06-28T15:25:22.123 INFO:tasks.workunit.client.0.smithi178.stderr: -50> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: crc 2024-06-28T15:25:22.123 INFO:tasks.workunit.client.0.smithi178.stderr: -49> 2024-06-28T15:25:22.060+0000 7f82df7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:22.123 INFO:tasks.workunit.client.0.smithi178.stderr: -48> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: crc 2024-06-28T15:25:22.123 INFO:tasks.workunit.client.0.smithi178.stderr: -47> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: secure 2024-06-28T15:25:22.123 INFO:tasks.workunit.client.0.smithi178.stderr: -46> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: crc 2024-06-28T15:25:22.123 INFO:tasks.workunit.client.0.smithi178.stderr: -45> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: secure 2024-06-28T15:25:22.123 INFO:tasks.workunit.client.0.smithi178.stderr: -44> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: crc 2024-06-28T15:25:22.123 INFO:tasks.workunit.client.0.smithi178.stderr: -43> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: secure 2024-06-28T15:25:22.123 INFO:tasks.workunit.client.0.smithi178.stderr: -42> 2024-06-28T15:25:22.060+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.123 INFO:tasks.workunit.client.0.smithi178.stderr: -41> 2024-06-28T15:25:22.060+0000 7f842baa9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:22.123 INFO:tasks.workunit.client.0.smithi178.stderr: -40> 2024-06-28T15:25:22.060+0000 7f842baa9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:22.123 INFO:tasks.workunit.client.0.smithi178.stderr: -39> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command cache list hook 0x558a710f2100 2024-06-28T15:25:22.124 INFO:tasks.workunit.client.0.smithi178.stderr: -38> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command cache inspect hook 0x558a710f2100 2024-06-28T15:25:22.124 INFO:tasks.workunit.client.0.smithi178.stderr: -37> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command cache erase hook 0x558a710f2100 2024-06-28T15:25:22.124 INFO:tasks.workunit.client.0.smithi178.stderr: -36> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command cache zap hook 0x558a710f2100 2024-06-28T15:25:22.124 INFO:tasks.workunit.client.0.smithi178.stderr: -35> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 note: GC not initialized 2024-06-28T15:25:22.124 INFO:tasks.workunit.client.0.smithi178.stderr: -34> 2024-06-28T15:25:22.060+0000 7f82de7e4640 20 reqs_thread_entry: start 2024-06-28T15:25:22.124 INFO:tasks.workunit.client.0.smithi178.stderr: -33> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command sync trace show hook 0x7f841c001250 2024-06-28T15:25:22.124 INFO:tasks.workunit.client.0.smithi178.stderr: -32> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command sync trace history hook 0x7f841c001250 2024-06-28T15:25:22.124 INFO:tasks.workunit.client.0.smithi178.stderr: -31> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command sync trace active hook 0x7f841c001250 2024-06-28T15:25:22.124 INFO:tasks.workunit.client.0.smithi178.stderr: -30> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command sync trace active_short hook 0x7f841c001250 2024-06-28T15:25:22.124 INFO:tasks.workunit.client.0.smithi178.stderr: -29> 2024-06-28T15:25:22.060+0000 7f842baa9980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:22.124 INFO:tasks.workunit.client.0.smithi178.stderr: -28> 2024-06-28T15:25:22.060+0000 7f842baa9980 20 Filter name: none 2024-06-28T15:25:22.124 INFO:tasks.workunit.client.0.smithi178.stderr: -27> 2024-06-28T15:25:22.060+0000 7f824ffff640 20 reqs_thread_entry: start 2024-06-28T15:25:22.124 INFO:tasks.workunit.client.0.smithi178.stderr: -26> 2024-06-28T15:25:22.060+0000 7f842baa9980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-06-28T15:25:22.125 INFO:tasks.workunit.client.0.smithi178.stderr: -25> 2024-06-28T15:25:22.060+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.125 INFO:tasks.workunit.client.0.smithi178.stderr: -24> 2024-06-28T15:25:22.064+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:22.125 INFO:tasks.workunit.client.0.smithi178.stderr: -23> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-06-28T15:25:22.125 INFO:tasks.workunit.client.0.smithi178.stderr: -22> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-06-28T15:25:22.125 INFO:tasks.workunit.client.0.smithi178.stderr: -21> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 : miss 2024-06-28T15:25:22.125 INFO:tasks.workunit.client.0.smithi178.stderr: -20> 2024-06-28T15:25:22.064+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.125 INFO:tasks.workunit.client.0.smithi178.stderr: -19> 2024-06-28T15:25:22.064+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:22.125 INFO:tasks.workunit.client.0.smithi178.stderr: -18> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 info.flags=0x17 2024-06-28T15:25:22.125 INFO:tasks.workunit.client.0.smithi178.stderr: -17> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 to cache LRU end 2024-06-28T15:25:22.125 INFO:tasks.workunit.client.0.smithi178.stderr: -16> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:25:22.125 INFO:tasks.workunit.client.0.smithi178.stderr: -15> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:22.125 INFO:tasks.workunit.client.0.smithi178.stderr: -14> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 2024-06-28T15:25:22.125 INFO:tasks.workunit.client.0.smithi178.stderr: -13> 2024-06-28T15:25:22.072+0000 7f842baa9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 info.flags=0x17 2024-06-28T15:25:22.125 INFO:tasks.workunit.client.0.smithi178.stderr: -12> 2024-06-28T15:25:22.072+0000 7f842baa9980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 to cache LRU end 2024-06-28T15:25:22.126 INFO:tasks.workunit.client.0.smithi178.stderr: -11> 2024-06-28T15:25:22.072+0000 7f842baa9980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:22.126 INFO:tasks.workunit.client.0.smithi178.stderr: -10> 2024-06-28T15:25:22.072+0000 7f842baa9980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29, ofs0, ns] 2024-06-28T15:25:22.126 INFO:tasks.workunit.client.0.smithi178.stderr: -9> 2024-06-28T15:25:22.072+0000 7f830cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595097 cookie 94053090840352 notifier 6677 bl.length()=987 2024-06-28T15:25:22.126 INFO:tasks.workunit.client.0.smithi178.stderr: -8> 2024-06-28T15:25:22.072+0000 7f830cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 info.flags=0x17 2024-06-28T15:25:22.126 INFO:tasks.workunit.client.0.smithi178.stderr: -7> 2024-06-28T15:25:22.072+0000 7f830cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 to cache LRU end 2024-06-28T15:25:22.126 INFO:tasks.workunit.client.0.smithi178.stderr: -6> 2024-06-28T15:25:22.072+0000 7f830cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:22.126 INFO:tasks.workunit.client.0.smithi178.stderr: -5> 2024-06-28T15:25:22.072+0000 7f842baa9980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): orig_sources= new_sources= 2024-06-28T15:25:22.126 INFO:tasks.workunit.client.0.smithi178.stderr: -4> 2024-06-28T15:25:22.072+0000 7f842baa9980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): potential sources added=[] removed=[] 2024-06-28T15:25:22.126 INFO:tasks.workunit.client.0.smithi178.stderr: -3> 2024-06-28T15:25:22.072+0000 7f842baa9980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): orig_dests= new_dests= 2024-06-28T15:25:22.126 INFO:tasks.workunit.client.0.smithi178.stderr: -2> 2024-06-28T15:25:22.072+0000 7f842baa9980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): potential dests added=[] removed=[] 2024-06-28T15:25:22.126 INFO:tasks.workunit.client.0.smithi178.stderr: -1> 2024-06-28T15:25:22.076+0000 7f842baa9980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f842baa9980 time 2024-06-28T15:25:22.078313+0000 2024-06-28T15:25:22.126 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:25:22.126 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.127 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:25:22.127 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f842f380c91] 2024-06-28T15:25:22.127 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f842f380e45] 2024-06-28T15:25:22.127 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558a6f5424b1] 2024-06-28T15:25:22.127 INFO:tasks.workunit.client.0.smithi178.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x32c) [0x558a6f553aac] 2024-06-28T15:25:22.127 INFO:tasks.workunit.client.0.smithi178.stderr: 5: main() 2024-06-28T15:25:22.127 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f842e1e9d90] 2024-06-28T15:25:22.127 INFO:tasks.workunit.client.0.smithi178.stderr: 7: __libc_start_main() 2024-06-28T15:25:22.127 INFO:tasks.workunit.client.0.smithi178.stderr: 8: _start() 2024-06-28T15:25:22.127 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.127 INFO:tasks.workunit.client.0.smithi178.stderr: 0> 2024-06-28T15:25:22.076+0000 7f842baa9980 -1 *** Caught signal (Aborted) ** 2024-06-28T15:25:22.127 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7f842baa9980 thread_name:radosgw-admin 2024-06-28T15:25:22.127 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.127 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:25:22.128 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f842e202520] 2024-06-28T15:25:22.128 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:25:22.128 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:25:22.128 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:25:22.128 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f842f380ceb] 2024-06-28T15:25:22.128 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f842f380e45] 2024-06-28T15:25:22.128 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558a6f5424b1] 2024-06-28T15:25:22.128 INFO:tasks.workunit.client.0.smithi178.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x32c) [0x558a6f553aac] 2024-06-28T15:25:22.128 INFO:tasks.workunit.client.0.smithi178.stderr: 9: main() 2024-06-28T15:25:22.128 INFO:tasks.workunit.client.0.smithi178.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f842e1e9d90] 2024-06-28T15:25:22.128 INFO:tasks.workunit.client.0.smithi178.stderr: 11: __libc_start_main() 2024-06-28T15:25:22.128 INFO:tasks.workunit.client.0.smithi178.stderr: 12: _start() 2024-06-28T15:25:22.128 INFO:tasks.workunit.client.0.smithi178.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-28T15:25:22.128 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.129 INFO:tasks.workunit.client.0.smithi178.stderr:--- logging levels --- 2024-06-28T15:25:22.129 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 none 2024-06-28T15:25:22.129 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 lockdep 2024-06-28T15:25:22.129 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 context 2024-06-28T15:25:22.129 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 crush 2024-06-28T15:25:22.129 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds 2024-06-28T15:25:22.129 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_balancer 2024-06-28T15:25:22.129 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_locker 2024-06-28T15:25:22.129 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log 2024-06-28T15:25:22.129 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log_expire 2024-06-28T15:25:22.129 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_migrator 2024-06-28T15:25:22.129 INFO:tasks.workunit.client.0.smithi178.stderr: 3/ 5 mds_quiesce 2024-06-28T15:25:22.129 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 buffer 2024-06-28T15:25:22.129 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 timer 2024-06-28T15:25:22.130 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 filer 2024-06-28T15:25:22.130 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 striper 2024-06-28T15:25:22.130 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 objecter 2024-06-28T15:25:22.130 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rados 2024-06-28T15:25:22.130 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd 2024-06-28T15:25:22.130 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_mirror 2024-06-28T15:25:22.130 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_replay 2024-06-28T15:25:22.130 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_pwl 2024-06-28T15:25:22.130 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 journaler 2024-06-28T15:25:22.130 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objectcacher 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 immutable_obj_cache 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 client 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 osd 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 optracker 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objclass 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 filestore 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 journal 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 ms 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mon 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 0/10 monc 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 paxos 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 tp 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 auth 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 crypto 2024-06-28T15:25:22.131 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 finisher 2024-06-28T15:25:22.132 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 reserver 2024-06-28T15:25:22.132 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 heartbeatmap 2024-06-28T15:25:22.132 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 perfcounter 2024-06-28T15:25:22.132 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw 2024-06-28T15:25:22.132 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_sync 2024-06-28T15:25:22.132 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_datacache 2024-06-28T15:25:22.132 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_access 2024-06-28T15:25:22.132 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_dbstore 2024-06-28T15:25:22.132 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_flight 2024-06-28T15:25:22.132 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_lifecycle 2024-06-28T15:25:22.132 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw_notification 2024-06-28T15:25:22.132 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 javaclient 2024-06-28T15:25:22.132 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 asok 2024-06-28T15:25:22.132 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 throttle 2024-06-28T15:25:22.133 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 refs 2024-06-28T15:25:22.133 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 compressor 2024-06-28T15:25:22.133 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluestore 2024-06-28T15:25:22.133 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluefs 2024-06-28T15:25:22.133 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 bdev 2024-06-28T15:25:22.133 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 kstore 2024-06-28T15:25:22.133 INFO:tasks.workunit.client.0.smithi178.stderr: 4/ 5 rocksdb 2024-06-28T15:25:22.133 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 fuse 2024-06-28T15:25:22.133 INFO:tasks.workunit.client.0.smithi178.stderr: 2/ 5 mgr 2024-06-28T15:25:22.133 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mgrc 2024-06-28T15:25:22.133 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 dpdk 2024-06-28T15:25:22.133 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 eventtrace 2024-06-28T15:25:22.133 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 prioritycache 2024-06-28T15:25:22.134 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 test 2024-06-28T15:25:22.134 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephfs_mirror 2024-06-28T15:25:22.134 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephsqlite 2024-06-28T15:25:22.134 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 crimson_interrupt 2024-06-28T15:25:22.134 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore 2024-06-28T15:25:22.134 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_onode 2024-06-28T15:25:22.134 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_odata 2024-06-28T15:25:22.134 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_omap 2024-06-28T15:25:22.134 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_tm 2024-06-28T15:25:22.134 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_t 2024-06-28T15:25:22.134 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cleaner 2024-06-28T15:25:22.134 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_epm 2024-06-28T15:25:22.134 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_lba 2024-06-28T15:25:22.134 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_fixedkv_tree 2024-06-28T15:25:22.135 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cache 2024-06-28T15:25:22.135 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_journal 2024-06-28T15:25:22.135 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_device 2024-06-28T15:25:22.135 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_backref 2024-06-28T15:25:22.135 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 alienstore 2024-06-28T15:25:22.135 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mclock 2024-06-28T15:25:22.135 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cyanstore 2024-06-28T15:25:22.135 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 ceph_exporter 2024-06-28T15:25:22.135 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 memstore 2024-06-28T15:25:22.135 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 trace 2024-06-28T15:25:22.135 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 ceph_dedup 2024-06-28T15:25:22.135 INFO:tasks.workunit.client.0.smithi178.stderr: -2/-2 (syslog threshold) 2024-06-28T15:25:22.135 INFO:tasks.workunit.client.0.smithi178.stderr: 99/99 (stderr threshold) 2024-06-28T15:25:22.135 INFO:tasks.workunit.client.0.smithi178.stderr:--- pthread ID / name mapping for recent threads --- 2024-06-28T15:25:22.136 INFO:tasks.workunit.client.0.smithi178.stderr: 7f824ffff640 / http_manager 2024-06-28T15:25:22.136 INFO:tasks.workunit.client.0.smithi178.stderr: 7f82de7e4640 / http_manager 2024-06-28T15:25:22.136 INFO:tasks.workunit.client.0.smithi178.stderr: 7f82df7e6640 / rgw_dt_lg_renew 2024-06-28T15:25:22.136 INFO:tasks.workunit.client.0.smithi178.stderr: 7f82faffd640 / ms_dispatch 2024-06-28T15:25:22.136 INFO:tasks.workunit.client.0.smithi178.stderr: 7f830cff9640 / io_context_pool 2024-06-28T15:25:22.136 INFO:tasks.workunit.client.0.smithi178.stderr: 7f830effd640 / ms_dispatch 2024-06-28T15:25:22.136 INFO:tasks.workunit.client.0.smithi178.stderr: 7f8315e03640 / ms_dispatch 2024-06-28T15:25:22.136 INFO:tasks.workunit.client.0.smithi178.stderr: 7f8317606640 / io_context_pool 2024-06-28T15:25:22.136 INFO:tasks.workunit.client.0.smithi178.stderr: 7f8428db9640 / msgr-worker-2 2024-06-28T15:25:22.136 INFO:tasks.workunit.client.0.smithi178.stderr: 7f84295ba640 / msgr-worker-1 2024-06-28T15:25:22.136 INFO:tasks.workunit.client.0.smithi178.stderr: 7f8429dbb640 / msgr-worker-0 2024-06-28T15:25:22.136 INFO:tasks.workunit.client.0.smithi178.stderr: 7f842a5bc640 / io_context_pool 2024-06-28T15:25:22.136 INFO:tasks.workunit.client.0.smithi178.stderr: 7f842baa9980 / radosgw-admin 2024-06-28T15:25:22.136 INFO:tasks.workunit.client.0.smithi178.stderr: max_recent 500 2024-06-28T15:25:22.137 INFO:tasks.workunit.client.0.smithi178.stderr: max_new 1000 2024-06-28T15:25:22.137 INFO:tasks.workunit.client.0.smithi178.stderr: log_file /var/log/ceph/ceph-client.admin.102564.log 2024-06-28T15:25:22.137 INFO:tasks.workunit.client.0.smithi178.stderr:--- end dump of recent events --- 2024-06-28T15:25:22.137 INFO:tasks.workunit.client.0.smithi178.stderr:--- begin dump of recent events --- 2024-06-28T15:25:22.137 INFO:tasks.workunit.client.0.smithi178.stderr: -499> 2024-06-28T15:25:21.992+0000 7f8428db9640 1 monclient: found mon.noname-a 2024-06-28T15:25:22.137 INFO:tasks.workunit.client.0.smithi178.stderr: -498> 2024-06-28T15:25:21.992+0000 7f8428db9640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:25:22.137 INFO:tasks.workunit.client.0.smithi178.stderr: -497> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.137 INFO:tasks.workunit.client.0.smithi178.stderr: -496> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:25:22.137 INFO:tasks.workunit.client.0.smithi178.stderr: -495> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: dump: 2024-06-28T15:25:22.137 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.137 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.137 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.137 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.137 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.138 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.138 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.138 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.138 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.138 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.138 INFO:tasks.workunit.client.0.smithi178.stderr: -494> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: _finish_auth 0 2024-06-28T15:25:22.138 INFO:tasks.workunit.client.0.smithi178.stderr: -493> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: _check_auth_tickets 2024-06-28T15:25:22.138 INFO:tasks.workunit.client.0.smithi178.stderr: -492> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: handle_config config(0 keys) 2024-06-28T15:25:22.138 INFO:tasks.workunit.client.0.smithi178.stderr: -491> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.138 INFO:tasks.workunit.client.0.smithi178.stderr: -490> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:25:22.138 INFO:tasks.workunit.client.0.smithi178.stderr: -489> 2024-06-28T15:25:21.992+0000 7f8317606640 10 monclient: dump: 2024-06-28T15:25:22.138 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.138 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.139 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.139 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.139 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.139 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.139 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.139 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.139 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.139 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.139 INFO:tasks.workunit.client.0.smithi178.stderr: -488> 2024-06-28T15:25:21.992+0000 7f842baa9980 5 monclient: authenticate success, global_id 7098 2024-06-28T15:25:22.139 INFO:tasks.workunit.client.0.smithi178.stderr: -487> 2024-06-28T15:25:21.992+0000 7f842baa9980 10 monclient: get_monmap_and_config success 2024-06-28T15:25:22.139 INFO:tasks.workunit.client.0.smithi178.stderr: -486> 2024-06-28T15:25:21.992+0000 7f842baa9980 4 set_mon_vals no callback set 2024-06-28T15:25:22.139 INFO:tasks.workunit.client.0.smithi178.stderr: -485> 2024-06-28T15:25:21.992+0000 7f842baa9980 10 monclient: shutdown 2024-06-28T15:25:22.139 INFO:tasks.workunit.client.0.smithi178.stderr: -484> 2024-06-28T15:25:21.992+0000 7f842baa9980 5 asok(0x558a70e185e0) unregister_commands rotate-key 2024-06-28T15:25:22.139 INFO:tasks.workunit.client.0.smithi178.stderr: -483> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: build_initial_monmap 2024-06-28T15:25:22.140 INFO:tasks.workunit.client.0.smithi178.stderr: -482> 2024-06-28T15:25:21.996+0000 7f842baa9980 1 build_initial for_mkfs: 0 2024-06-28T15:25:22.140 INFO:tasks.workunit.client.0.smithi178.stderr: -481> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: monmap: 2024-06-28T15:25:22.140 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:25:22.140 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:25:22.140 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:25:22.140 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:25:22.140 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:25:22.140 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.140 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:25:22.140 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:25:22.140 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:25:22.140 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.140 INFO:tasks.workunit.client.0.smithi178.stderr: -480> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding auth protocol: cephx 2024-06-28T15:25:22.140 INFO:tasks.workunit.client.0.smithi178.stderr: -479> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding auth protocol: cephx 2024-06-28T15:25:22.141 INFO:tasks.workunit.client.0.smithi178.stderr: -478> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding auth protocol: cephx 2024-06-28T15:25:22.141 INFO:tasks.workunit.client.0.smithi178.stderr: -477> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: secure 2024-06-28T15:25:22.141 INFO:tasks.workunit.client.0.smithi178.stderr: -476> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: crc 2024-06-28T15:25:22.141 INFO:tasks.workunit.client.0.smithi178.stderr: -475> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: secure 2024-06-28T15:25:22.141 INFO:tasks.workunit.client.0.smithi178.stderr: -474> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: crc 2024-06-28T15:25:22.141 INFO:tasks.workunit.client.0.smithi178.stderr: -473> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: secure 2024-06-28T15:25:22.141 INFO:tasks.workunit.client.0.smithi178.stderr: -472> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: crc 2024-06-28T15:25:22.141 INFO:tasks.workunit.client.0.smithi178.stderr: -471> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: crc 2024-06-28T15:25:22.141 INFO:tasks.workunit.client.0.smithi178.stderr: -470> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: secure 2024-06-28T15:25:22.141 INFO:tasks.workunit.client.0.smithi178.stderr: -469> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: crc 2024-06-28T15:25:22.141 INFO:tasks.workunit.client.0.smithi178.stderr: -468> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: secure 2024-06-28T15:25:22.141 INFO:tasks.workunit.client.0.smithi178.stderr: -467> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: crc 2024-06-28T15:25:22.141 INFO:tasks.workunit.client.0.smithi178.stderr: -466> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a7102b420) adding con mode: secure 2024-06-28T15:25:22.141 INFO:tasks.workunit.client.0.smithi178.stderr: -465> 2024-06-28T15:25:21.996+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.142 INFO:tasks.workunit.client.0.smithi178.stderr: -464> 2024-06-28T15:25:21.996+0000 7f842baa9980 1 librados: starting msgr at 2024-06-28T15:25:22.142 INFO:tasks.workunit.client.0.smithi178.stderr: -463> 2024-06-28T15:25:21.996+0000 7f842baa9980 1 librados: starting objecter 2024-06-28T15:25:22.142 INFO:tasks.workunit.client.0.smithi178.stderr: -462> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command objecter_requests hook 0x558a7102a0e0 2024-06-28T15:25:22.142 INFO:tasks.workunit.client.0.smithi178.stderr: -461> 2024-06-28T15:25:21.996+0000 7f842baa9980 1 librados: setting wanted keys 2024-06-28T15:25:22.142 INFO:tasks.workunit.client.0.smithi178.stderr: -460> 2024-06-28T15:25:21.996+0000 7f842baa9980 1 librados: calling monclient init 2024-06-28T15:25:22.142 INFO:tasks.workunit.client.0.smithi178.stderr: -459> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: init 2024-06-28T15:25:22.142 INFO:tasks.workunit.client.0.smithi178.stderr: -458> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding auth protocol: cephx 2024-06-28T15:25:22.142 INFO:tasks.workunit.client.0.smithi178.stderr: -457> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding auth protocol: cephx 2024-06-28T15:25:22.142 INFO:tasks.workunit.client.0.smithi178.stderr: -456> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding auth protocol: cephx 2024-06-28T15:25:22.142 INFO:tasks.workunit.client.0.smithi178.stderr: -455> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: secure 2024-06-28T15:25:22.142 INFO:tasks.workunit.client.0.smithi178.stderr: -454> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: crc 2024-06-28T15:25:22.142 INFO:tasks.workunit.client.0.smithi178.stderr: -453> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: secure 2024-06-28T15:25:22.142 INFO:tasks.workunit.client.0.smithi178.stderr: -452> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: crc 2024-06-28T15:25:22.142 INFO:tasks.workunit.client.0.smithi178.stderr: -451> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: secure 2024-06-28T15:25:22.143 INFO:tasks.workunit.client.0.smithi178.stderr: -450> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: crc 2024-06-28T15:25:22.143 INFO:tasks.workunit.client.0.smithi178.stderr: -449> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: crc 2024-06-28T15:25:22.143 INFO:tasks.workunit.client.0.smithi178.stderr: -448> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: secure 2024-06-28T15:25:22.143 INFO:tasks.workunit.client.0.smithi178.stderr: -447> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: crc 2024-06-28T15:25:22.143 INFO:tasks.workunit.client.0.smithi178.stderr: -446> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: secure 2024-06-28T15:25:22.143 INFO:tasks.workunit.client.0.smithi178.stderr: -445> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: crc 2024-06-28T15:25:22.143 INFO:tasks.workunit.client.0.smithi178.stderr: -444> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 AuthRegistry(0x558a71025640) adding con mode: secure 2024-06-28T15:25:22.143 INFO:tasks.workunit.client.0.smithi178.stderr: -443> 2024-06-28T15:25:21.996+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.143 INFO:tasks.workunit.client.0.smithi178.stderr: -442> 2024-06-28T15:25:21.996+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.143 INFO:tasks.workunit.client.0.smithi178.stderr: -441> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command rotate-key hook 0x558a71025788 2024-06-28T15:25:22.143 INFO:tasks.workunit.client.0.smithi178.stderr: -440> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: _reopen_session rank -1 2024-06-28T15:25:22.143 INFO:tasks.workunit.client.0.smithi178.stderr: -439> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: _add_conns ranks=[2,0,1] 2024-06-28T15:25:22.143 INFO:tasks.workunit.client.0.smithi178.stderr: -438> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-c con 0x558a70ed5d30 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:25:22.143 INFO:tasks.workunit.client.0.smithi178.stderr: -437> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-a con 0x558a70f6f110 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:25:22.144 INFO:tasks.workunit.client.0.smithi178.stderr: -436> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-b con 0x558a7102af70 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:25:22.144 INFO:tasks.workunit.client.0.smithi178.stderr: -435> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.144 INFO:tasks.workunit.client.0.smithi178.stderr: -434> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.144 INFO:tasks.workunit.client.0.smithi178.stderr: -433> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.144 INFO:tasks.workunit.client.0.smithi178.stderr: -432> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): _renew_subs 2024-06-28T15:25:22.144 INFO:tasks.workunit.client.0.smithi178.stderr: -431> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient(hunting): authenticate will time out at 2465.457165s 2024-06-28T15:25:22.144 INFO:tasks.workunit.client.0.smithi178.stderr: -430> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): get_auth_request con 0x558a70ed5d30 auth_method 0 2024-06-28T15:25:22.144 INFO:tasks.workunit.client.0.smithi178.stderr: -429> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.144 INFO:tasks.workunit.client.0.smithi178.stderr: -428> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.144 INFO:tasks.workunit.client.0.smithi178.stderr: -427> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.144 INFO:tasks.workunit.client.0.smithi178.stderr: -426> 2024-06-28T15:25:21.996+0000 7f8428db9640 10 monclient(hunting): get_auth_request con 0x558a70f6f110 auth_method 0 2024-06-28T15:25:22.144 INFO:tasks.workunit.client.0.smithi178.stderr: -425> 2024-06-28T15:25:21.996+0000 7f8428db9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.144 INFO:tasks.workunit.client.0.smithi178.stderr: -424> 2024-06-28T15:25:21.996+0000 7f8428db9640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.144 INFO:tasks.workunit.client.0.smithi178.stderr: -423> 2024-06-28T15:25:21.996+0000 7f8428db9640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.145 INFO:tasks.workunit.client.0.smithi178.stderr: -422> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.145 INFO:tasks.workunit.client.0.smithi178.stderr: -421> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.145 INFO:tasks.workunit.client.0.smithi178.stderr: -420> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.145 INFO:tasks.workunit.client.0.smithi178.stderr: -419> 2024-06-28T15:25:21.996+0000 7f8428db9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.145 INFO:tasks.workunit.client.0.smithi178.stderr: -418> 2024-06-28T15:25:21.996+0000 7f8428db9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.145 INFO:tasks.workunit.client.0.smithi178.stderr: -417> 2024-06-28T15:25:21.996+0000 7f8428db9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.145 INFO:tasks.workunit.client.0.smithi178.stderr: -416> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient(hunting): handle_auth_done global_id 6668 payload 824 2024-06-28T15:25:22.145 INFO:tasks.workunit.client.0.smithi178.stderr: -415> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient: _finish_hunting 0 2024-06-28T15:25:22.145 INFO:tasks.workunit.client.0.smithi178.stderr: -414> 2024-06-28T15:25:21.996+0000 7f84295ba640 1 monclient: found mon.noname-c 2024-06-28T15:25:22.145 INFO:tasks.workunit.client.0.smithi178.stderr: -413> 2024-06-28T15:25:21.996+0000 7f84295ba640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.145 INFO:tasks.workunit.client.0.smithi178.stderr: -412> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.145 INFO:tasks.workunit.client.0.smithi178.stderr: -411> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:25:22.145 INFO:tasks.workunit.client.0.smithi178.stderr: -410> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: dump: 2024-06-28T15:25:22.145 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.146 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.146 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.146 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.146 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.146 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.146 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.146 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.146 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.146 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.146 INFO:tasks.workunit.client.0.smithi178.stderr: -409> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: _finish_auth 0 2024-06-28T15:25:22.146 INFO:tasks.workunit.client.0.smithi178.stderr: -408> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: _check_auth_tickets 2024-06-28T15:25:22.146 INFO:tasks.workunit.client.0.smithi178.stderr: -407> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: handle_config config(0 keys) 2024-06-28T15:25:22.146 INFO:tasks.workunit.client.0.smithi178.stderr: -406> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.147 INFO:tasks.workunit.client.0.smithi178.stderr: -405> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:25:22.147 INFO:tasks.workunit.client.0.smithi178.stderr: -404> 2024-06-28T15:25:21.996+0000 7f8315e03640 10 monclient: dump: 2024-06-28T15:25:22.147 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.147 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.147 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.147 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.147 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.147 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.147 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.147 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.147 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.147 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.147 INFO:tasks.workunit.client.0.smithi178.stderr: -403> 2024-06-28T15:25:21.996+0000 7f8317606640 4 set_mon_vals no callback set 2024-06-28T15:25:22.147 INFO:tasks.workunit.client.0.smithi178.stderr: -402> 2024-06-28T15:25:21.996+0000 7f842baa9980 5 monclient: authenticate success, global_id 6668 2024-06-28T15:25:22.148 INFO:tasks.workunit.client.0.smithi178.stderr: -401> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: _renew_subs 2024-06-28T15:25:22.148 INFO:tasks.workunit.client.0.smithi178.stderr: -400> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.148 INFO:tasks.workunit.client.0.smithi178.stderr: -399> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: _renew_subs 2024-06-28T15:25:22.148 INFO:tasks.workunit.client.0.smithi178.stderr: -398> 2024-06-28T15:25:21.996+0000 7f842baa9980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.148 INFO:tasks.workunit.client.0.smithi178.stderr: -397> 2024-06-28T15:25:21.996+0000 7f842baa9980 1 librados: init done 2024-06-28T15:25:22.148 INFO:tasks.workunit.client.0.smithi178.stderr: -396> 2024-06-28T15:25:21.996+0000 7f8315e03640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:25:22.148 INFO:tasks.workunit.client.0.smithi178.stderr: -395> 2024-06-28T15:25:21.996+0000 7f8315e03640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:25:22.148 INFO:tasks.workunit.client.0.smithi178.stderr: -394> 2024-06-28T15:25:21.996+0000 7f8315e03640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:25:22.148 INFO:tasks.workunit.client.0.smithi178.stderr: -393> 2024-06-28T15:25:22.000+0000 7f8428db9640 10 monclient: get_auth_request con 0x7f831006f7b0 auth_method 0 2024-06-28T15:25:22.148 INFO:tasks.workunit.client.0.smithi178.stderr: -392> 2024-06-28T15:25:22.000+0000 7f8429dbb640 10 monclient: get_auth_request con 0x558a71066c90 auth_method 0 2024-06-28T15:25:22.148 INFO:tasks.workunit.client.0.smithi178.stderr: -391> 2024-06-28T15:25:22.000+0000 7f84295ba640 10 monclient: get_auth_request con 0x558a7106ff70 auth_method 0 2024-06-28T15:25:22.148 INFO:tasks.workunit.client.0.smithi178.stderr: -390> 2024-06-28T15:25:22.000+0000 7f8428db9640 10 monclient: get_auth_request con 0x558a710742d0 auth_method 0 2024-06-28T15:25:22.148 INFO:tasks.workunit.client.0.smithi178.stderr: -389> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: build_initial_monmap 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr: -388> 2024-06-28T15:25:22.004+0000 7f842baa9980 1 build_initial for_mkfs: 0 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr: -387> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: monmap: 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr: -386> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding auth protocol: cephx 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr: -385> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding auth protocol: cephx 2024-06-28T15:25:22.149 INFO:tasks.workunit.client.0.smithi178.stderr: -384> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding auth protocol: cephx 2024-06-28T15:25:22.150 INFO:tasks.workunit.client.0.smithi178.stderr: -383> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: secure 2024-06-28T15:25:22.150 INFO:tasks.workunit.client.0.smithi178.stderr: -382> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: crc 2024-06-28T15:25:22.150 INFO:tasks.workunit.client.0.smithi178.stderr: -381> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: secure 2024-06-28T15:25:22.150 INFO:tasks.workunit.client.0.smithi178.stderr: -380> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: crc 2024-06-28T15:25:22.150 INFO:tasks.workunit.client.0.smithi178.stderr: -379> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: secure 2024-06-28T15:25:22.150 INFO:tasks.workunit.client.0.smithi178.stderr: -378> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: crc 2024-06-28T15:25:22.150 INFO:tasks.workunit.client.0.smithi178.stderr: -377> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: crc 2024-06-28T15:25:22.150 INFO:tasks.workunit.client.0.smithi178.stderr: -376> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: secure 2024-06-28T15:25:22.150 INFO:tasks.workunit.client.0.smithi178.stderr: -375> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: crc 2024-06-28T15:25:22.150 INFO:tasks.workunit.client.0.smithi178.stderr: -374> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: secure 2024-06-28T15:25:22.150 INFO:tasks.workunit.client.0.smithi178.stderr: -373> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: crc 2024-06-28T15:25:22.150 INFO:tasks.workunit.client.0.smithi178.stderr: -372> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710791e0) adding con mode: secure 2024-06-28T15:25:22.150 INFO:tasks.workunit.client.0.smithi178.stderr: -371> 2024-06-28T15:25:22.004+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.151 INFO:tasks.workunit.client.0.smithi178.stderr: -370> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command objecter_requests cmddesc objecter_requests hook 0x558a7107c7c0 EEXIST 2024-06-28T15:25:22.151 INFO:tasks.workunit.client.0.smithi178.stderr: -369> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: init 2024-06-28T15:25:22.151 INFO:tasks.workunit.client.0.smithi178.stderr: -368> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding auth protocol: cephx 2024-06-28T15:25:22.151 INFO:tasks.workunit.client.0.smithi178.stderr: -367> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding auth protocol: cephx 2024-06-28T15:25:22.151 INFO:tasks.workunit.client.0.smithi178.stderr: -366> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding auth protocol: cephx 2024-06-28T15:25:22.151 INFO:tasks.workunit.client.0.smithi178.stderr: -365> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: secure 2024-06-28T15:25:22.151 INFO:tasks.workunit.client.0.smithi178.stderr: -364> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: crc 2024-06-28T15:25:22.151 INFO:tasks.workunit.client.0.smithi178.stderr: -363> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: secure 2024-06-28T15:25:22.151 INFO:tasks.workunit.client.0.smithi178.stderr: -362> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: crc 2024-06-28T15:25:22.151 INFO:tasks.workunit.client.0.smithi178.stderr: -361> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: secure 2024-06-28T15:25:22.151 INFO:tasks.workunit.client.0.smithi178.stderr: -360> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: crc 2024-06-28T15:25:22.151 INFO:tasks.workunit.client.0.smithi178.stderr: -359> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: crc 2024-06-28T15:25:22.151 INFO:tasks.workunit.client.0.smithi178.stderr: -358> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: secure 2024-06-28T15:25:22.152 INFO:tasks.workunit.client.0.smithi178.stderr: -357> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: crc 2024-06-28T15:25:22.152 INFO:tasks.workunit.client.0.smithi178.stderr: -356> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: secure 2024-06-28T15:25:22.152 INFO:tasks.workunit.client.0.smithi178.stderr: -355> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: crc 2024-06-28T15:25:22.152 INFO:tasks.workunit.client.0.smithi178.stderr: -354> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 AuthRegistry(0x558a710778b8) adding con mode: secure 2024-06-28T15:25:22.152 INFO:tasks.workunit.client.0.smithi178.stderr: -353> 2024-06-28T15:25:22.004+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.152 INFO:tasks.workunit.client.0.smithi178.stderr: -352> 2024-06-28T15:25:22.004+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.152 INFO:tasks.workunit.client.0.smithi178.stderr: -351> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command rotate-key cmddesc rotate-key hook 0x558a71077a00 EEXIST 2024-06-28T15:25:22.152 INFO:tasks.workunit.client.0.smithi178.stderr: -350> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: _reopen_session rank -1 2024-06-28T15:25:22.152 INFO:tasks.workunit.client.0.smithi178.stderr: -349> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: _add_conns ranks=[0,2,1] 2024-06-28T15:25:22.152 INFO:tasks.workunit.client.0.smithi178.stderr: -348> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-a con 0x558a71082d00 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:25:22.152 INFO:tasks.workunit.client.0.smithi178.stderr: -347> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-c con 0x558a71083630 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:25:22.152 INFO:tasks.workunit.client.0.smithi178.stderr: -346> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-b con 0x558a710830e0 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:25:22.152 INFO:tasks.workunit.client.0.smithi178.stderr: -345> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.153 INFO:tasks.workunit.client.0.smithi178.stderr: -344> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.153 INFO:tasks.workunit.client.0.smithi178.stderr: -343> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.153 INFO:tasks.workunit.client.0.smithi178.stderr: -342> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): _renew_subs 2024-06-28T15:25:22.153 INFO:tasks.workunit.client.0.smithi178.stderr: -341> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient(hunting): authenticate will time out at 2465.466204s 2024-06-28T15:25:22.153 INFO:tasks.workunit.client.0.smithi178.stderr: -340> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): get_auth_request con 0x558a71083630 auth_method 0 2024-06-28T15:25:22.153 INFO:tasks.workunit.client.0.smithi178.stderr: -339> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.153 INFO:tasks.workunit.client.0.smithi178.stderr: -338> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.153 INFO:tasks.workunit.client.0.smithi178.stderr: -337> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.153 INFO:tasks.workunit.client.0.smithi178.stderr: -336> 2024-06-28T15:25:22.004+0000 7f84295ba640 10 monclient(hunting): get_auth_request con 0x558a71082d00 auth_method 0 2024-06-28T15:25:22.153 INFO:tasks.workunit.client.0.smithi178.stderr: -335> 2024-06-28T15:25:22.004+0000 7f84295ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.153 INFO:tasks.workunit.client.0.smithi178.stderr: -334> 2024-06-28T15:25:22.004+0000 7f84295ba640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.153 INFO:tasks.workunit.client.0.smithi178.stderr: -333> 2024-06-28T15:25:22.004+0000 7f84295ba640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.153 INFO:tasks.workunit.client.0.smithi178.stderr: -332> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.153 INFO:tasks.workunit.client.0.smithi178.stderr: -331> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.154 INFO:tasks.workunit.client.0.smithi178.stderr: -330> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.154 INFO:tasks.workunit.client.0.smithi178.stderr: -329> 2024-06-28T15:25:22.004+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.154 INFO:tasks.workunit.client.0.smithi178.stderr: -328> 2024-06-28T15:25:22.004+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.154 INFO:tasks.workunit.client.0.smithi178.stderr: -327> 2024-06-28T15:25:22.004+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.154 INFO:tasks.workunit.client.0.smithi178.stderr: -326> 2024-06-28T15:25:22.004+0000 7f8429dbb640 10 monclient(hunting): get_auth_request con 0x558a710830e0 auth_method 0 2024-06-28T15:25:22.154 INFO:tasks.workunit.client.0.smithi178.stderr: -325> 2024-06-28T15:25:22.004+0000 7f8429dbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.154 INFO:tasks.workunit.client.0.smithi178.stderr: -324> 2024-06-28T15:25:22.004+0000 7f8429dbb640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.154 INFO:tasks.workunit.client.0.smithi178.stderr: -323> 2024-06-28T15:25:22.004+0000 7f8429dbb640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.154 INFO:tasks.workunit.client.0.smithi178.stderr: -322> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient(hunting): handle_auth_done global_id 6671 payload 824 2024-06-28T15:25:22.154 INFO:tasks.workunit.client.0.smithi178.stderr: -321> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient: _finish_hunting 0 2024-06-28T15:25:22.154 INFO:tasks.workunit.client.0.smithi178.stderr: -320> 2024-06-28T15:25:22.004+0000 7f8428db9640 1 monclient: found mon.noname-c 2024-06-28T15:25:22.154 INFO:tasks.workunit.client.0.smithi178.stderr: -319> 2024-06-28T15:25:22.004+0000 7f8428db9640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.154 INFO:tasks.workunit.client.0.smithi178.stderr: -318> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.155 INFO:tasks.workunit.client.0.smithi178.stderr: -317> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:25:22.155 INFO:tasks.workunit.client.0.smithi178.stderr: -316> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: dump: 2024-06-28T15:25:22.155 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.155 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.155 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.155 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.155 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.155 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.155 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.155 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.155 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.155 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.155 INFO:tasks.workunit.client.0.smithi178.stderr: -315> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: _finish_auth 0 2024-06-28T15:25:22.155 INFO:tasks.workunit.client.0.smithi178.stderr: -314> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: _check_auth_tickets 2024-06-28T15:25:22.156 INFO:tasks.workunit.client.0.smithi178.stderr: -313> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: handle_config config(0 keys) 2024-06-28T15:25:22.156 INFO:tasks.workunit.client.0.smithi178.stderr: -312> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.156 INFO:tasks.workunit.client.0.smithi178.stderr: -311> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:25:22.156 INFO:tasks.workunit.client.0.smithi178.stderr: -310> 2024-06-28T15:25:22.004+0000 7f830effd640 10 monclient: dump: 2024-06-28T15:25:22.156 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.156 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.156 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.156 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.156 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.156 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.156 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.156 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.156 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.156 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.157 INFO:tasks.workunit.client.0.smithi178.stderr: -309> 2024-06-28T15:25:22.004+0000 7f842a5bc640 4 set_mon_vals no callback set 2024-06-28T15:25:22.157 INFO:tasks.workunit.client.0.smithi178.stderr: -308> 2024-06-28T15:25:22.004+0000 7f842baa9980 5 monclient: authenticate success, global_id 6671 2024-06-28T15:25:22.157 INFO:tasks.workunit.client.0.smithi178.stderr: -307> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: _renew_subs 2024-06-28T15:25:22.157 INFO:tasks.workunit.client.0.smithi178.stderr: -306> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.157 INFO:tasks.workunit.client.0.smithi178.stderr: -305> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: _renew_subs 2024-06-28T15:25:22.157 INFO:tasks.workunit.client.0.smithi178.stderr: -304> 2024-06-28T15:25:22.004+0000 7f842baa9980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.157 INFO:tasks.workunit.client.0.smithi178.stderr: -303> 2024-06-28T15:25:22.008+0000 7f830effd640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:25:22.157 INFO:tasks.workunit.client.0.smithi178.stderr: -302> 2024-06-28T15:25:22.008+0000 7f830effd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:25:22.157 INFO:tasks.workunit.client.0.smithi178.stderr: -301> 2024-06-28T15:25:22.008+0000 7f830effd640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:25:22.157 INFO:tasks.workunit.client.0.smithi178.stderr: -300> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient: get_auth_request con 0x7f830406f690 auth_method 0 2024-06-28T15:25:22.157 INFO:tasks.workunit.client.0.smithi178.stderr: -299> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient: get_monmap_and_config 2024-06-28T15:25:22.157 INFO:tasks.workunit.client.0.smithi178.stderr: -298> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient: build_initial_monmap 2024-06-28T15:25:22.157 INFO:tasks.workunit.client.0.smithi178.stderr: -297> 2024-06-28T15:25:22.008+0000 7f842baa9980 1 build_initial for_mkfs: 0 2024-06-28T15:25:22.157 INFO:tasks.workunit.client.0.smithi178.stderr: -296> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient: monmap: 2024-06-28T15:25:22.158 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:25:22.158 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:25:22.158 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:25:22.158 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:25:22.158 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:25:22.158 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.158 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:25:22.158 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:25:22.158 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:25:22.158 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.158 INFO:tasks.workunit.client.0.smithi178.stderr: -295> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding auth protocol: cephx 2024-06-28T15:25:22.158 INFO:tasks.workunit.client.0.smithi178.stderr: -294> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding auth protocol: cephx 2024-06-28T15:25:22.158 INFO:tasks.workunit.client.0.smithi178.stderr: -293> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding auth protocol: cephx 2024-06-28T15:25:22.158 INFO:tasks.workunit.client.0.smithi178.stderr: -292> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: secure 2024-06-28T15:25:22.159 INFO:tasks.workunit.client.0.smithi178.stderr: -291> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: crc 2024-06-28T15:25:22.159 INFO:tasks.workunit.client.0.smithi178.stderr: -290> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: secure 2024-06-28T15:25:22.159 INFO:tasks.workunit.client.0.smithi178.stderr: -289> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: crc 2024-06-28T15:25:22.159 INFO:tasks.workunit.client.0.smithi178.stderr: -288> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: secure 2024-06-28T15:25:22.159 INFO:tasks.workunit.client.0.smithi178.stderr: -287> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: crc 2024-06-28T15:25:22.159 INFO:tasks.workunit.client.0.smithi178.stderr: -286> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: crc 2024-06-28T15:25:22.159 INFO:tasks.workunit.client.0.smithi178.stderr: -285> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: secure 2024-06-28T15:25:22.159 INFO:tasks.workunit.client.0.smithi178.stderr: -284> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: crc 2024-06-28T15:25:22.159 INFO:tasks.workunit.client.0.smithi178.stderr: -283> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: secure 2024-06-28T15:25:22.159 INFO:tasks.workunit.client.0.smithi178.stderr: -282> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: crc 2024-06-28T15:25:22.159 INFO:tasks.workunit.client.0.smithi178.stderr: -281> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x558a710c02e0) adding con mode: secure 2024-06-28T15:25:22.159 INFO:tasks.workunit.client.0.smithi178.stderr: -280> 2024-06-28T15:25:22.008+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.159 INFO:tasks.workunit.client.0.smithi178.stderr: -279> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient: init 2024-06-28T15:25:22.159 INFO:tasks.workunit.client.0.smithi178.stderr: -278> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding auth protocol: cephx 2024-06-28T15:25:22.160 INFO:tasks.workunit.client.0.smithi178.stderr: -277> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding auth protocol: cephx 2024-06-28T15:25:22.160 INFO:tasks.workunit.client.0.smithi178.stderr: -276> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding auth protocol: cephx 2024-06-28T15:25:22.160 INFO:tasks.workunit.client.0.smithi178.stderr: -275> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: secure 2024-06-28T15:25:22.160 INFO:tasks.workunit.client.0.smithi178.stderr: -274> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: crc 2024-06-28T15:25:22.160 INFO:tasks.workunit.client.0.smithi178.stderr: -273> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: secure 2024-06-28T15:25:22.160 INFO:tasks.workunit.client.0.smithi178.stderr: -272> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: crc 2024-06-28T15:25:22.160 INFO:tasks.workunit.client.0.smithi178.stderr: -271> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: secure 2024-06-28T15:25:22.160 INFO:tasks.workunit.client.0.smithi178.stderr: -270> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: crc 2024-06-28T15:25:22.160 INFO:tasks.workunit.client.0.smithi178.stderr: -269> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: crc 2024-06-28T15:25:22.160 INFO:tasks.workunit.client.0.smithi178.stderr: -268> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: secure 2024-06-28T15:25:22.160 INFO:tasks.workunit.client.0.smithi178.stderr: -267> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: crc 2024-06-28T15:25:22.160 INFO:tasks.workunit.client.0.smithi178.stderr: -266> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: secure 2024-06-28T15:25:22.160 INFO:tasks.workunit.client.0.smithi178.stderr: -265> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: crc 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -264> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5c7a0) adding con mode: secure 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -263> 2024-06-28T15:25:22.008+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -262> 2024-06-28T15:25:22.008+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -261> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command rotate-key cmddesc rotate-key hook 0x7ffe25e5c8e8 EEXIST 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -260> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient: _reopen_session rank -1 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -259> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient: _add_conns ranks=[0,1,2] 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -258> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-a con 0x558a710f0130 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -257> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-b con 0x558a710f0510 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -256> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-c con 0x558a710f4b10 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -255> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -254> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -253> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -252> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): _renew_subs 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -251> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient(hunting): authenticate will time out at 2465.470946s 2024-06-28T15:25:22.161 INFO:tasks.workunit.client.0.smithi178.stderr: -250> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): get_auth_request con 0x558a710f0130 auth_method 0 2024-06-28T15:25:22.162 INFO:tasks.workunit.client.0.smithi178.stderr: -249> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.162 INFO:tasks.workunit.client.0.smithi178.stderr: -248> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.162 INFO:tasks.workunit.client.0.smithi178.stderr: -247> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.162 INFO:tasks.workunit.client.0.smithi178.stderr: -246> 2024-06-28T15:25:22.008+0000 7f8429dbb640 10 monclient(hunting): get_auth_request con 0x558a710f4b10 auth_method 0 2024-06-28T15:25:22.162 INFO:tasks.workunit.client.0.smithi178.stderr: -245> 2024-06-28T15:25:22.008+0000 7f8429dbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.162 INFO:tasks.workunit.client.0.smithi178.stderr: -244> 2024-06-28T15:25:22.008+0000 7f8429dbb640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.162 INFO:tasks.workunit.client.0.smithi178.stderr: -243> 2024-06-28T15:25:22.008+0000 7f8429dbb640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.162 INFO:tasks.workunit.client.0.smithi178.stderr: -242> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.162 INFO:tasks.workunit.client.0.smithi178.stderr: -241> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.162 INFO:tasks.workunit.client.0.smithi178.stderr: -240> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.162 INFO:tasks.workunit.client.0.smithi178.stderr: -239> 2024-06-28T15:25:22.008+0000 7f8429dbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.162 INFO:tasks.workunit.client.0.smithi178.stderr: -238> 2024-06-28T15:25:22.008+0000 7f8429dbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.162 INFO:tasks.workunit.client.0.smithi178.stderr: -237> 2024-06-28T15:25:22.008+0000 7f8429dbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.162 INFO:tasks.workunit.client.0.smithi178.stderr: -236> 2024-06-28T15:25:22.008+0000 7f8428db9640 10 monclient(hunting): get_auth_request con 0x558a710f0510 auth_method 0 2024-06-28T15:25:22.163 INFO:tasks.workunit.client.0.smithi178.stderr: -235> 2024-06-28T15:25:22.008+0000 7f8428db9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.163 INFO:tasks.workunit.client.0.smithi178.stderr: -234> 2024-06-28T15:25:22.008+0000 7f8428db9640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.163 INFO:tasks.workunit.client.0.smithi178.stderr: -233> 2024-06-28T15:25:22.008+0000 7f8428db9640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.163 INFO:tasks.workunit.client.0.smithi178.stderr: -232> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient(hunting): handle_auth_done global_id 7107 payload 274 2024-06-28T15:25:22.163 INFO:tasks.workunit.client.0.smithi178.stderr: -231> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient: _finish_hunting 0 2024-06-28T15:25:22.163 INFO:tasks.workunit.client.0.smithi178.stderr: -230> 2024-06-28T15:25:22.008+0000 7f84295ba640 1 monclient: found mon.noname-a 2024-06-28T15:25:22.163 INFO:tasks.workunit.client.0.smithi178.stderr: -229> 2024-06-28T15:25:22.008+0000 7f84295ba640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.178:3300/0 2024-06-28T15:25:22.163 INFO:tasks.workunit.client.0.smithi178.stderr: -228> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.163 INFO:tasks.workunit.client.0.smithi178.stderr: -227> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-06-28T15:25:22.163 INFO:tasks.workunit.client.0.smithi178.stderr: -226> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: dump: 2024-06-28T15:25:22.163 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.163 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.163 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.164 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.164 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.164 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.164 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.164 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.164 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.164 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.164 INFO:tasks.workunit.client.0.smithi178.stderr: -225> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: _finish_auth 0 2024-06-28T15:25:22.164 INFO:tasks.workunit.client.0.smithi178.stderr: -224> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: _check_auth_tickets 2024-06-28T15:25:22.164 INFO:tasks.workunit.client.0.smithi178.stderr: -223> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: handle_config config(0 keys) 2024-06-28T15:25:22.164 INFO:tasks.workunit.client.0.smithi178.stderr: -222> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.164 INFO:tasks.workunit.client.0.smithi178.stderr: -221> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-06-28T15:25:22.164 INFO:tasks.workunit.client.0.smithi178.stderr: -220> 2024-06-28T15:25:22.008+0000 7f830cff9640 10 monclient: dump: 2024-06-28T15:25:22.165 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.165 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.165 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.165 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.165 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.165 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.165 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.165 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.165 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.165 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.165 INFO:tasks.workunit.client.0.smithi178.stderr: -219> 2024-06-28T15:25:22.008+0000 7f842baa9980 5 monclient: authenticate success, global_id 7107 2024-06-28T15:25:22.165 INFO:tasks.workunit.client.0.smithi178.stderr: -218> 2024-06-28T15:25:22.008+0000 7f842baa9980 10 monclient: get_monmap_and_config success 2024-06-28T15:25:22.165 INFO:tasks.workunit.client.0.smithi178.stderr: -217> 2024-06-28T15:25:22.008+0000 7f842baa9980 4 set_mon_vals no callback set 2024-06-28T15:25:22.165 INFO:tasks.workunit.client.0.smithi178.stderr: -216> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: shutdown 2024-06-28T15:25:22.166 INFO:tasks.workunit.client.0.smithi178.stderr: -215> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: build_initial_monmap 2024-06-28T15:25:22.166 INFO:tasks.workunit.client.0.smithi178.stderr: -214> 2024-06-28T15:25:22.012+0000 7f842baa9980 1 build_initial for_mkfs: 0 2024-06-28T15:25:22.166 INFO:tasks.workunit.client.0.smithi178.stderr: -213> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: monmap: 2024-06-28T15:25:22.166 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 0 2024-06-28T15:25:22.166 INFO:tasks.workunit.client.0.smithi178.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-06-28T15:25:22.166 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 0.000000 2024-06-28T15:25:22.166 INFO:tasks.workunit.client.0.smithi178.stderr:created 0.000000 2024-06-28T15:25:22.166 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 0 (unknown) 2024-06-28T15:25:22.166 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.166 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.noname-a 2024-06-28T15:25:22.166 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.noname-b 2024-06-28T15:25:22.166 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.noname-c 2024-06-28T15:25:22.166 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.167 INFO:tasks.workunit.client.0.smithi178.stderr: -212> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding auth protocol: cephx 2024-06-28T15:25:22.167 INFO:tasks.workunit.client.0.smithi178.stderr: -211> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding auth protocol: cephx 2024-06-28T15:25:22.167 INFO:tasks.workunit.client.0.smithi178.stderr: -210> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding auth protocol: cephx 2024-06-28T15:25:22.167 INFO:tasks.workunit.client.0.smithi178.stderr: -209> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: secure 2024-06-28T15:25:22.167 INFO:tasks.workunit.client.0.smithi178.stderr: -208> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: crc 2024-06-28T15:25:22.167 INFO:tasks.workunit.client.0.smithi178.stderr: -207> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: secure 2024-06-28T15:25:22.167 INFO:tasks.workunit.client.0.smithi178.stderr: -206> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: crc 2024-06-28T15:25:22.167 INFO:tasks.workunit.client.0.smithi178.stderr: -205> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: secure 2024-06-28T15:25:22.167 INFO:tasks.workunit.client.0.smithi178.stderr: -204> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: crc 2024-06-28T15:25:22.167 INFO:tasks.workunit.client.0.smithi178.stderr: -203> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: crc 2024-06-28T15:25:22.167 INFO:tasks.workunit.client.0.smithi178.stderr: -202> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: secure 2024-06-28T15:25:22.167 INFO:tasks.workunit.client.0.smithi178.stderr: -201> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: crc 2024-06-28T15:25:22.167 INFO:tasks.workunit.client.0.smithi178.stderr: -200> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: secure 2024-06-28T15:25:22.167 INFO:tasks.workunit.client.0.smithi178.stderr: -199> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: crc 2024-06-28T15:25:22.168 INFO:tasks.workunit.client.0.smithi178.stderr: -198> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710c5020) adding con mode: secure 2024-06-28T15:25:22.168 INFO:tasks.workunit.client.0.smithi178.stderr: -197> 2024-06-28T15:25:22.012+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.168 INFO:tasks.workunit.client.0.smithi178.stderr: -196> 2024-06-28T15:25:22.012+0000 7f842baa9980 1 librados: starting msgr at 2024-06-28T15:25:22.168 INFO:tasks.workunit.client.0.smithi178.stderr: -195> 2024-06-28T15:25:22.012+0000 7f842baa9980 1 librados: starting objecter 2024-06-28T15:25:22.168 INFO:tasks.workunit.client.0.smithi178.stderr: -194> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command objecter_requests cmddesc objecter_requests hook 0x558a710c2300 EEXIST 2024-06-28T15:25:22.168 INFO:tasks.workunit.client.0.smithi178.stderr: -193> 2024-06-28T15:25:22.012+0000 7f842baa9980 1 librados: setting wanted keys 2024-06-28T15:25:22.168 INFO:tasks.workunit.client.0.smithi178.stderr: -192> 2024-06-28T15:25:22.012+0000 7f842baa9980 1 librados: calling monclient init 2024-06-28T15:25:22.168 INFO:tasks.workunit.client.0.smithi178.stderr: -191> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: init 2024-06-28T15:25:22.168 INFO:tasks.workunit.client.0.smithi178.stderr: -190> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding auth protocol: cephx 2024-06-28T15:25:22.168 INFO:tasks.workunit.client.0.smithi178.stderr: -189> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding auth protocol: cephx 2024-06-28T15:25:22.168 INFO:tasks.workunit.client.0.smithi178.stderr: -188> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding auth protocol: cephx 2024-06-28T15:25:22.168 INFO:tasks.workunit.client.0.smithi178.stderr: -187> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: secure 2024-06-28T15:25:22.168 INFO:tasks.workunit.client.0.smithi178.stderr: -186> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: crc 2024-06-28T15:25:22.168 INFO:tasks.workunit.client.0.smithi178.stderr: -185> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: secure 2024-06-28T15:25:22.169 INFO:tasks.workunit.client.0.smithi178.stderr: -184> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: crc 2024-06-28T15:25:22.169 INFO:tasks.workunit.client.0.smithi178.stderr: -183> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: secure 2024-06-28T15:25:22.169 INFO:tasks.workunit.client.0.smithi178.stderr: -182> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: crc 2024-06-28T15:25:22.169 INFO:tasks.workunit.client.0.smithi178.stderr: -181> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: crc 2024-06-28T15:25:22.169 INFO:tasks.workunit.client.0.smithi178.stderr: -180> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: secure 2024-06-28T15:25:22.169 INFO:tasks.workunit.client.0.smithi178.stderr: -179> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: crc 2024-06-28T15:25:22.169 INFO:tasks.workunit.client.0.smithi178.stderr: -178> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: secure 2024-06-28T15:25:22.169 INFO:tasks.workunit.client.0.smithi178.stderr: -177> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: crc 2024-06-28T15:25:22.169 INFO:tasks.workunit.client.0.smithi178.stderr: -176> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 AuthRegistry(0x558a710b8740) adding con mode: secure 2024-06-28T15:25:22.169 INFO:tasks.workunit.client.0.smithi178.stderr: -175> 2024-06-28T15:25:22.012+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.169 INFO:tasks.workunit.client.0.smithi178.stderr: -174> 2024-06-28T15:25:22.012+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.169 INFO:tasks.workunit.client.0.smithi178.stderr: -173> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command rotate-key cmddesc rotate-key hook 0x558a710b8888 EEXIST 2024-06-28T15:25:22.169 INFO:tasks.workunit.client.0.smithi178.stderr: -172> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: _reopen_session rank -1 2024-06-28T15:25:22.170 INFO:tasks.workunit.client.0.smithi178.stderr: -171> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: _add_conns ranks=[2,1,0] 2024-06-28T15:25:22.170 INFO:tasks.workunit.client.0.smithi178.stderr: -170> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-c con 0x558a710f0130 addr [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] 2024-06-28T15:25:22.170 INFO:tasks.workunit.client.0.smithi178.stderr: -169> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-b con 0x558a710f4b10 addr [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-06-28T15:25:22.170 INFO:tasks.workunit.client.0.smithi178.stderr: -168> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): picked mon.noname-a con 0x558a710f1cc0 addr [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-28T15:25:22.170 INFO:tasks.workunit.client.0.smithi178.stderr: -167> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.170 INFO:tasks.workunit.client.0.smithi178.stderr: -166> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.170 INFO:tasks.workunit.client.0.smithi178.stderr: -165> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): start opening mon connection 2024-06-28T15:25:22.170 INFO:tasks.workunit.client.0.smithi178.stderr: -164> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): _renew_subs 2024-06-28T15:25:22.170 INFO:tasks.workunit.client.0.smithi178.stderr: -163> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient(hunting): authenticate will time out at 2465.473553s 2024-06-28T15:25:22.170 INFO:tasks.workunit.client.0.smithi178.stderr: -162> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): get_auth_request con 0x558a710f0130 auth_method 0 2024-06-28T15:25:22.170 INFO:tasks.workunit.client.0.smithi178.stderr: -161> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.170 INFO:tasks.workunit.client.0.smithi178.stderr: -160> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.170 INFO:tasks.workunit.client.0.smithi178.stderr: -159> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.170 INFO:tasks.workunit.client.0.smithi178.stderr: -158> 2024-06-28T15:25:22.012+0000 7f8429dbb640 10 monclient(hunting): get_auth_request con 0x558a710f1cc0 auth_method 0 2024-06-28T15:25:22.171 INFO:tasks.workunit.client.0.smithi178.stderr: -157> 2024-06-28T15:25:22.012+0000 7f8429dbb640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-06-28T15:25:22.171 INFO:tasks.workunit.client.0.smithi178.stderr: -156> 2024-06-28T15:25:22.012+0000 7f8429dbb640 10 monclient(hunting): _init_auth method 2 2024-06-28T15:25:22.171 INFO:tasks.workunit.client.0.smithi178.stderr: -155> 2024-06-28T15:25:22.012+0000 7f8429dbb640 10 monclient(hunting): _init_auth creating new auth 2024-06-28T15:25:22.171 INFO:tasks.workunit.client.0.smithi178.stderr: -154> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.171 INFO:tasks.workunit.client.0.smithi178.stderr: -153> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.171 INFO:tasks.workunit.client.0.smithi178.stderr: -152> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.171 INFO:tasks.workunit.client.0.smithi178.stderr: -151> 2024-06-28T15:25:22.012+0000 7f8429dbb640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-06-28T15:25:22.171 INFO:tasks.workunit.client.0.smithi178.stderr: -150> 2024-06-28T15:25:22.012+0000 7f8429dbb640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-06-28T15:25:22.171 INFO:tasks.workunit.client.0.smithi178.stderr: -149> 2024-06-28T15:25:22.012+0000 7f8429dbb640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-06-28T15:25:22.171 INFO:tasks.workunit.client.0.smithi178.stderr: -148> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient(hunting): handle_auth_done global_id 6677 payload 824 2024-06-28T15:25:22.171 INFO:tasks.workunit.client.0.smithi178.stderr: -147> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient: _finish_hunting 0 2024-06-28T15:25:22.171 INFO:tasks.workunit.client.0.smithi178.stderr: -146> 2024-06-28T15:25:22.012+0000 7f84295ba640 1 monclient: found mon.noname-c 2024-06-28T15:25:22.171 INFO:tasks.workunit.client.0.smithi178.stderr: -145> 2024-06-28T15:25:22.012+0000 7f84295ba640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.172 INFO:tasks.workunit.client.0.smithi178.stderr: -144> 2024-06-28T15:25:22.012+0000 7f8428db9640 10 monclient: get_auth_request con 0x558a710f4b10 auth_method 0 2024-06-28T15:25:22.172 INFO:tasks.workunit.client.0.smithi178.stderr: -143> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.172 INFO:tasks.workunit.client.0.smithi178.stderr: -142> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-06-28T15:25:22.172 INFO:tasks.workunit.client.0.smithi178.stderr: -141> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: dump: 2024-06-28T15:25:22.172 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.172 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.172 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.172 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.172 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.172 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.172 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.172 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.172 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.172 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.173 INFO:tasks.workunit.client.0.smithi178.stderr: -140> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: _finish_auth 0 2024-06-28T15:25:22.173 INFO:tasks.workunit.client.0.smithi178.stderr: -139> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: _check_auth_tickets 2024-06-28T15:25:22.173 INFO:tasks.workunit.client.0.smithi178.stderr: -138> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 monclient: authenticate success, global_id 6677 2024-06-28T15:25:22.173 INFO:tasks.workunit.client.0.smithi178.stderr: -137> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: _renew_subs 2024-06-28T15:25:22.173 INFO:tasks.workunit.client.0.smithi178.stderr: -136> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.173 INFO:tasks.workunit.client.0.smithi178.stderr: -135> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: handle_config config(0 keys) 2024-06-28T15:25:22.173 INFO:tasks.workunit.client.0.smithi178.stderr: -134> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-06-28T15:25:22.173 INFO:tasks.workunit.client.0.smithi178.stderr: -133> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-06-28T15:25:22.173 INFO:tasks.workunit.client.0.smithi178.stderr: -132> 2024-06-28T15:25:22.012+0000 7f82faffd640 10 monclient: dump: 2024-06-28T15:25:22.173 INFO:tasks.workunit.client.0.smithi178.stderr:epoch 1 2024-06-28T15:25:22.173 INFO:tasks.workunit.client.0.smithi178.stderr:fsid cc348236-88e8-4481-a265-c0b687a883fe 2024-06-28T15:25:22.173 INFO:tasks.workunit.client.0.smithi178.stderr:last_changed 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.173 INFO:tasks.workunit.client.0.smithi178.stderr:created 2024-06-28T14:59:18.862098+0000 2024-06-28T15:25:22.173 INFO:tasks.workunit.client.0.smithi178.stderr:min_mon_release 19 (squid) 2024-06-28T15:25:22.174 INFO:tasks.workunit.client.0.smithi178.stderr:election_strategy: 1 2024-06-28T15:25:22.174 INFO:tasks.workunit.client.0.smithi178.stderr:0: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.a 2024-06-28T15:25:22.174 INFO:tasks.workunit.client.0.smithi178.stderr:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-06-28T15:25:22.174 INFO:tasks.workunit.client.0.smithi178.stderr:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2024-06-28T15:25:22.174 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.174 INFO:tasks.workunit.client.0.smithi178.stderr: -131> 2024-06-28T15:25:22.012+0000 7f830cff9640 4 set_mon_vals no callback set 2024-06-28T15:25:22.174 INFO:tasks.workunit.client.0.smithi178.stderr: -130> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: _renew_subs 2024-06-28T15:25:22.174 INFO:tasks.workunit.client.0.smithi178.stderr: -129> 2024-06-28T15:25:22.012+0000 7f842baa9980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.174 INFO:tasks.workunit.client.0.smithi178.stderr: -128> 2024-06-28T15:25:22.012+0000 7f842baa9980 1 librados: init done 2024-06-28T15:25:22.174 INFO:tasks.workunit.client.0.smithi178.stderr: -127> 2024-06-28T15:25:22.012+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command cr dump hook 0x558a710f3d58 2024-06-28T15:25:22.174 INFO:tasks.workunit.client.0.smithi178.stderr: -126> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.174 INFO:tasks.workunit.client.0.smithi178.stderr: -125> 2024-06-28T15:25:22.016+0000 7f82faffd640 4 mgrc handle_mgr_map Got map version 5 2024-06-28T15:25:22.174 INFO:tasks.workunit.client.0.smithi178.stderr: -124> 2024-06-28T15:25:22.016+0000 7f82faffd640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:25:22.175 INFO:tasks.workunit.client.0.smithi178.stderr: -123> 2024-06-28T15:25:22.016+0000 7f82faffd640 4 mgrc reconnect Starting new session with [v2:172.21.15.178:6832/886789604,v1:172.21.15.178:6833/886789604] 2024-06-28T15:25:22.175 INFO:tasks.workunit.client.0.smithi178.stderr: -122> 2024-06-28T15:25:22.016+0000 7f8428db9640 10 monclient: get_auth_request con 0x7f82f406f7b0 auth_method 0 2024-06-28T15:25:22.175 INFO:tasks.workunit.client.0.smithi178.stderr: -121> 2024-06-28T15:25:22.016+0000 7f8429dbb640 10 monclient: get_auth_request con 0x558a7110e330 auth_method 0 2024-06-28T15:25:22.175 INFO:tasks.workunit.client.0.smithi178.stderr: -120> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.175 INFO:tasks.workunit.client.0.smithi178.stderr: -119> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 realm 2024-06-28T15:25:22.175 INFO:tasks.workunit.client.0.smithi178.stderr: -118> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.175 INFO:tasks.workunit.client.0.smithi178.stderr: -117> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.175 INFO:tasks.workunit.client.0.smithi178.stderr: -116> 2024-06-28T15:25:22.016+0000 7f842baa9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:22.175 INFO:tasks.workunit.client.0.smithi178.stderr: -115> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.175 INFO:tasks.workunit.client.0.smithi178.stderr: -114> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.175 INFO:tasks.workunit.client.0.smithi178.stderr: -113> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.175 INFO:tasks.workunit.client.0.smithi178.stderr: -112> 2024-06-28T15:25:22.016+0000 7f84295ba640 10 monclient: get_auth_request con 0x558a71112620 auth_method 0 2024-06-28T15:25:22.175 INFO:tasks.workunit.client.0.smithi178.stderr: -111> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:22.175 INFO:tasks.workunit.client.0.smithi178.stderr: -110> 2024-06-28T15:25:22.016+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.176 INFO:tasks.workunit.client.0.smithi178.stderr: -109> 2024-06-28T15:25:22.016+0000 7f8428db9640 10 monclient: get_auth_request con 0x558a71115ec0 auth_method 0 2024-06-28T15:25:22.176 INFO:tasks.workunit.client.0.smithi178.stderr: -108> 2024-06-28T15:25:22.020+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:22.176 INFO:tasks.workunit.client.0.smithi178.stderr: -107> 2024-06-28T15:25:22.020+0000 7f842baa9980 20 searching for the correct realm 2024-06-28T15:25:22.176 INFO:tasks.workunit.client.0.smithi178.stderr: -106> 2024-06-28T15:25:22.020+0000 7f8429dbb640 10 monclient: get_auth_request con 0x7f84200e4060 auth_method 0 2024-06-28T15:25:22.176 INFO:tasks.workunit.client.0.smithi178.stderr: -105> 2024-06-28T15:25:22.020+0000 7f84295ba640 10 monclient: get_auth_request con 0x7f8424020040 auth_method 0 2024-06-28T15:25:22.176 INFO:tasks.workunit.client.0.smithi178.stderr: -104> 2024-06-28T15:25:22.020+0000 7f8428db9640 10 monclient: get_auth_request con 0x7f841c06d020 auth_method 0 2024-06-28T15:25:22.176 INFO:tasks.workunit.client.0.smithi178.stderr: -103> 2024-06-28T15:25:22.024+0000 7f8429dbb640 0 -- 172.21.15.178:0/636795418 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x558a7110e330 msgr2=0x558a711107f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:22.176 INFO:tasks.workunit.client.0.smithi178.stderr: -102> 2024-06-28T15:25:22.024+0000 7f82faffd640 1 client.6677.objecter ms_handle_reset 0x558a7110e330 session 0x558a7110dc60 osd.1 2024-06-28T15:25:22.176 INFO:tasks.workunit.client.0.smithi178.stderr: -101> 2024-06-28T15:25:22.024+0000 7f82faffd640 10 monclient: _renew_subs 2024-06-28T15:25:22.176 INFO:tasks.workunit.client.0.smithi178.stderr: -100> 2024-06-28T15:25:22.024+0000 7f82faffd640 10 monclient: _send_mon_message to mon.c at v2:172.21.15.178:3301/0 2024-06-28T15:25:22.176 INFO:tasks.workunit.client.0.smithi178.stderr: -99> 2024-06-28T15:25:22.024+0000 7f8429dbb640 10 monclient: get_auth_request con 0x7f82f407fdd0 auth_method 0 2024-06-28T15:25:22.176 INFO:tasks.workunit.client.0.smithi178.stderr: -98> 2024-06-28T15:25:22.024+0000 7f8429dbb640 10 monclient: get_auth_request con 0x7f84200cd040 auth_method 0 2024-06-28T15:25:22.176 INFO:tasks.workunit.client.0.smithi178.stderr: -97> 2024-06-28T15:25:22.028+0000 7f84295ba640 10 monclient: get_auth_request con 0x7f8420107b00 auth_method 0 2024-06-28T15:25:22.177 INFO:tasks.workunit.client.0.smithi178.stderr: -96> 2024-06-28T15:25:22.028+0000 7f8428db9640 0 -- 172.21.15.178:0/636795418 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f841c06d020 msgr2=0x7f841c05a560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:22.177 INFO:tasks.workunit.client.0.smithi178.stderr: -95> 2024-06-28T15:25:22.028+0000 7f82faffd640 1 client.6677.objecter ms_handle_reset 0x7f841c06d020 session 0x7f841c047040 osd.4 2024-06-28T15:25:22.177 INFO:tasks.workunit.client.0.smithi178.stderr: -94> 2024-06-28T15:25:22.032+0000 7f8428db9640 10 monclient: get_auth_request con 0x558a7110e330 auth_method 0 2024-06-28T15:25:22.177 INFO:tasks.workunit.client.0.smithi178.stderr: -93> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:22.177 INFO:tasks.workunit.client.0.smithi178.stderr: -92> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:22.177 INFO:tasks.workunit.client.0.smithi178.stderr: -91> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:22.177 INFO:tasks.workunit.client.0.smithi178.stderr: -90> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:22.177 INFO:tasks.workunit.client.0.smithi178.stderr: -89> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:22.177 INFO:tasks.workunit.client.0.smithi178.stderr: -88> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:22.177 INFO:tasks.workunit.client.0.smithi178.stderr: -87> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.177 INFO:tasks.workunit.client.0.smithi178.stderr: -86> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.177 INFO:tasks.workunit.client.0.smithi178.stderr: -85> 2024-06-28T15:25:22.036+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.177 INFO:tasks.workunit.client.0.smithi178.stderr: -84> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:22.178 INFO:tasks.workunit.client.0.smithi178.stderr: -83> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.178 INFO:tasks.workunit.client.0.smithi178.stderr: -82> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:22.178 INFO:tasks.workunit.client.0.smithi178.stderr: -81> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 zone default found 2024-06-28T15:25:22.178 INFO:tasks.workunit.client.0.smithi178.stderr: -80> 2024-06-28T15:25:22.040+0000 7f842baa9980 4 Realm: () 2024-06-28T15:25:22.178 INFO:tasks.workunit.client.0.smithi178.stderr: -79> 2024-06-28T15:25:22.040+0000 7f842baa9980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:22.178 INFO:tasks.workunit.client.0.smithi178.stderr: -78> 2024-06-28T15:25:22.040+0000 7f842baa9980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:22.178 INFO:tasks.workunit.client.0.smithi178.stderr: -77> 2024-06-28T15:25:22.040+0000 7f842baa9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:22.178 INFO:tasks.workunit.client.0.smithi178.stderr: -76> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 zonegroup default 2024-06-28T15:25:22.178 INFO:tasks.workunit.client.0.smithi178.stderr: -75> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.178 INFO:tasks.workunit.client.0.smithi178.stderr: -74> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.178 INFO:tasks.workunit.client.0.smithi178.stderr: -73> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.178 INFO:tasks.workunit.client.0.smithi178.stderr: -72> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.178 INFO:tasks.workunit.client.0.smithi178.stderr: -71> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.179 INFO:tasks.workunit.client.0.smithi178.stderr: -70> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.179 INFO:tasks.workunit.client.0.smithi178.stderr: -69> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 started sync module instance, tier type = 2024-06-28T15:25:22.179 INFO:tasks.workunit.client.0.smithi178.stderr: -68> 2024-06-28T15:25:22.040+0000 7f842baa9980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:22.179 INFO:tasks.workunit.client.0.smithi178.stderr: -67> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=0 2024-06-28T15:25:22.179 INFO:tasks.workunit.client.0.smithi178.stderr: -66> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=1 2024-06-28T15:25:22.179 INFO:tasks.workunit.client.0.smithi178.stderr: -65> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=2 2024-06-28T15:25:22.179 INFO:tasks.workunit.client.0.smithi178.stderr: -64> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=3 2024-06-28T15:25:22.179 INFO:tasks.workunit.client.0.smithi178.stderr: -63> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=4 2024-06-28T15:25:22.179 INFO:tasks.workunit.client.0.smithi178.stderr: -62> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=5 2024-06-28T15:25:22.179 INFO:tasks.workunit.client.0.smithi178.stderr: -61> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=6 2024-06-28T15:25:22.179 INFO:tasks.workunit.client.0.smithi178.stderr: -60> 2024-06-28T15:25:22.056+0000 7f842baa9980 20 add_watcher() i=7 2024-06-28T15:25:22.179 INFO:tasks.workunit.client.0.smithi178.stderr: -59> 2024-06-28T15:25:22.056+0000 7f842baa9980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:22.179 INFO:tasks.workunit.client.0.smithi178.stderr: -58> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding auth protocol: cephx 2024-06-28T15:25:22.180 INFO:tasks.workunit.client.0.smithi178.stderr: -57> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding auth protocol: cephx 2024-06-28T15:25:22.180 INFO:tasks.workunit.client.0.smithi178.stderr: -56> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding auth protocol: cephx 2024-06-28T15:25:22.180 INFO:tasks.workunit.client.0.smithi178.stderr: -55> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: secure 2024-06-28T15:25:22.180 INFO:tasks.workunit.client.0.smithi178.stderr: -54> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: crc 2024-06-28T15:25:22.180 INFO:tasks.workunit.client.0.smithi178.stderr: -53> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: secure 2024-06-28T15:25:22.180 INFO:tasks.workunit.client.0.smithi178.stderr: -52> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: crc 2024-06-28T15:25:22.180 INFO:tasks.workunit.client.0.smithi178.stderr: -51> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: secure 2024-06-28T15:25:22.180 INFO:tasks.workunit.client.0.smithi178.stderr: -50> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: crc 2024-06-28T15:25:22.180 INFO:tasks.workunit.client.0.smithi178.stderr: -49> 2024-06-28T15:25:22.060+0000 7f82df7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:22.180 INFO:tasks.workunit.client.0.smithi178.stderr: -48> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: crc 2024-06-28T15:25:22.180 INFO:tasks.workunit.client.0.smithi178.stderr: -47> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: secure 2024-06-28T15:25:22.180 INFO:tasks.workunit.client.0.smithi178.stderr: -46> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: crc 2024-06-28T15:25:22.180 INFO:tasks.workunit.client.0.smithi178.stderr: -45> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: secure 2024-06-28T15:25:22.180 INFO:tasks.workunit.client.0.smithi178.stderr: -44> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: crc 2024-06-28T15:25:22.181 INFO:tasks.workunit.client.0.smithi178.stderr: -43> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 AuthRegistry(0x7ffe25e5cc80) adding con mode: secure 2024-06-28T15:25:22.181 INFO:tasks.workunit.client.0.smithi178.stderr: -42> 2024-06-28T15:25:22.060+0000 7f842baa9980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-06-28T15:25:22.181 INFO:tasks.workunit.client.0.smithi178.stderr: -41> 2024-06-28T15:25:22.060+0000 7f842baa9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:22.181 INFO:tasks.workunit.client.0.smithi178.stderr: -40> 2024-06-28T15:25:22.060+0000 7f842baa9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:22.181 INFO:tasks.workunit.client.0.smithi178.stderr: -39> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command cache list hook 0x558a710f2100 2024-06-28T15:25:22.181 INFO:tasks.workunit.client.0.smithi178.stderr: -38> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command cache inspect hook 0x558a710f2100 2024-06-28T15:25:22.181 INFO:tasks.workunit.client.0.smithi178.stderr: -37> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command cache erase hook 0x558a710f2100 2024-06-28T15:25:22.181 INFO:tasks.workunit.client.0.smithi178.stderr: -36> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command cache zap hook 0x558a710f2100 2024-06-28T15:25:22.181 INFO:tasks.workunit.client.0.smithi178.stderr: -35> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 note: GC not initialized 2024-06-28T15:25:22.181 INFO:tasks.workunit.client.0.smithi178.stderr: -34> 2024-06-28T15:25:22.060+0000 7f82de7e4640 20 reqs_thread_entry: start 2024-06-28T15:25:22.181 INFO:tasks.workunit.client.0.smithi178.stderr: -33> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command sync trace show hook 0x7f841c001250 2024-06-28T15:25:22.181 INFO:tasks.workunit.client.0.smithi178.stderr: -32> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command sync trace history hook 0x7f841c001250 2024-06-28T15:25:22.181 INFO:tasks.workunit.client.0.smithi178.stderr: -31> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command sync trace active hook 0x7f841c001250 2024-06-28T15:25:22.181 INFO:tasks.workunit.client.0.smithi178.stderr: -30> 2024-06-28T15:25:22.060+0000 7f842baa9980 5 asok(0x558a70e185e0) register_command sync trace active_short hook 0x7f841c001250 2024-06-28T15:25:22.182 INFO:tasks.workunit.client.0.smithi178.stderr: -29> 2024-06-28T15:25:22.060+0000 7f842baa9980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:22.182 INFO:tasks.workunit.client.0.smithi178.stderr: -28> 2024-06-28T15:25:22.060+0000 7f842baa9980 20 Filter name: none 2024-06-28T15:25:22.182 INFO:tasks.workunit.client.0.smithi178.stderr: -27> 2024-06-28T15:25:22.060+0000 7f824ffff640 20 reqs_thread_entry: start 2024-06-28T15:25:22.182 INFO:tasks.workunit.client.0.smithi178.stderr: -26> 2024-06-28T15:25:22.060+0000 7f842baa9980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-06-28T15:25:22.182 INFO:tasks.workunit.client.0.smithi178.stderr: -25> 2024-06-28T15:25:22.060+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.182 INFO:tasks.workunit.client.0.smithi178.stderr: -24> 2024-06-28T15:25:22.064+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:22.182 INFO:tasks.workunit.client.0.smithi178.stderr: -23> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-06-28T15:25:22.182 INFO:tasks.workunit.client.0.smithi178.stderr: -22> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-06-28T15:25:22.182 INFO:tasks.workunit.client.0.smithi178.stderr: -21> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 : miss 2024-06-28T15:25:22.182 INFO:tasks.workunit.client.0.smithi178.stderr: -20> 2024-06-28T15:25:22.064+0000 7f842baa9980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.182 INFO:tasks.workunit.client.0.smithi178.stderr: -19> 2024-06-28T15:25:22.064+0000 7f842baa9980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:22.182 INFO:tasks.workunit.client.0.smithi178.stderr: -18> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 info.flags=0x17 2024-06-28T15:25:22.183 INFO:tasks.workunit.client.0.smithi178.stderr: -17> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 to cache LRU end 2024-06-28T15:25:22.183 INFO:tasks.workunit.client.0.smithi178.stderr: -16> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:25:22.183 INFO:tasks.workunit.client.0.smithi178.stderr: -15> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:22.183 INFO:tasks.workunit.client.0.smithi178.stderr: -14> 2024-06-28T15:25:22.064+0000 7f842baa9980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 2024-06-28T15:25:22.183 INFO:tasks.workunit.client.0.smithi178.stderr: -13> 2024-06-28T15:25:22.072+0000 7f842baa9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 info.flags=0x17 2024-06-28T15:25:22.183 INFO:tasks.workunit.client.0.smithi178.stderr: -12> 2024-06-28T15:25:22.072+0000 7f842baa9980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 to cache LRU end 2024-06-28T15:25:22.183 INFO:tasks.workunit.client.0.smithi178.stderr: -11> 2024-06-28T15:25:22.072+0000 7f842baa9980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:22.183 INFO:tasks.workunit.client.0.smithi178.stderr: -10> 2024-06-28T15:25:22.072+0000 7f842baa9980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29, ofs0, ns] 2024-06-28T15:25:22.183 INFO:tasks.workunit.client.0.smithi178.stderr: -9> 2024-06-28T15:25:22.072+0000 7f830cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595097 cookie 94053090840352 notifier 6677 bl.length()=987 2024-06-28T15:25:22.183 INFO:tasks.workunit.client.0.smithi178.stderr: -8> 2024-06-28T15:25:22.072+0000 7f830cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 info.flags=0x17 2024-06-28T15:25:22.183 INFO:tasks.workunit.client.0.smithi178.stderr: -7> 2024-06-28T15:25:22.072+0000 7f830cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 to cache LRU end 2024-06-28T15:25:22.183 INFO:tasks.workunit.client.0.smithi178.stderr: -6> 2024-06-28T15:25:22.072+0000 7f830cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:22.183 INFO:tasks.workunit.client.0.smithi178.stderr: -5> 2024-06-28T15:25:22.072+0000 7f842baa9980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): orig_sources= new_sources= 2024-06-28T15:25:22.183 INFO:tasks.workunit.client.0.smithi178.stderr: -4> 2024-06-28T15:25:22.072+0000 7f842baa9980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): potential sources added=[] removed=[] 2024-06-28T15:25:22.184 INFO:tasks.workunit.client.0.smithi178.stderr: -3> 2024-06-28T15:25:22.072+0000 7f842baa9980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): orig_dests= new_dests= 2024-06-28T15:25:22.184 INFO:tasks.workunit.client.0.smithi178.stderr: -2> 2024-06-28T15:25:22.072+0000 7f842baa9980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): potential dests added=[] removed=[] 2024-06-28T15:25:22.184 INFO:tasks.workunit.client.0.smithi178.stderr: -1> 2024-06-28T15:25:22.076+0000 7f842baa9980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f842baa9980 time 2024-06-28T15:25:22.078313+0000 2024-06-28T15:25:22.184 INFO:tasks.workunit.client.0.smithi178.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-06-28T15:25:22.185 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.185 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:25:22.185 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f842f380c91] 2024-06-28T15:25:22.185 INFO:tasks.workunit.client.0.smithi178.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f842f380e45] 2024-06-28T15:25:22.185 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558a6f5424b1] 2024-06-28T15:25:22.185 INFO:tasks.workunit.client.0.smithi178.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x32c) [0x558a6f553aac] 2024-06-28T15:25:22.186 INFO:tasks.workunit.client.0.smithi178.stderr: 5: main() 2024-06-28T15:25:22.186 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f842e1e9d90] 2024-06-28T15:25:22.186 INFO:tasks.workunit.client.0.smithi178.stderr: 7: __libc_start_main() 2024-06-28T15:25:22.186 INFO:tasks.workunit.client.0.smithi178.stderr: 8: _start() 2024-06-28T15:25:22.186 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.186 INFO:tasks.workunit.client.0.smithi178.stderr: 0> 2024-06-28T15:25:22.076+0000 7f842baa9980 -1 *** Caught signal (Aborted) ** 2024-06-28T15:25:22.186 INFO:tasks.workunit.client.0.smithi178.stderr: in thread 7f842baa9980 thread_name:radosgw-admin 2024-06-28T15:25:22.186 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.186 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 19.0.0-4592-gec2a38ba (ec2a38ba1b69b5357de8f0e8f96b00b4a577169d) squid (dev) 2024-06-28T15:25:22.186 INFO:tasks.workunit.client.0.smithi178.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f842e202520] 2024-06-28T15:25:22.186 INFO:tasks.workunit.client.0.smithi178.stderr: 2: pthread_kill() 2024-06-28T15:25:22.186 INFO:tasks.workunit.client.0.smithi178.stderr: 3: raise() 2024-06-28T15:25:22.186 INFO:tasks.workunit.client.0.smithi178.stderr: 4: abort() 2024-06-28T15:25:22.187 INFO:tasks.workunit.client.0.smithi178.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f842f380ceb] 2024-06-28T15:25:22.187 INFO:tasks.workunit.client.0.smithi178.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x182e45) [0x7f842f380e45] 2024-06-28T15:25:22.187 INFO:tasks.workunit.client.0.smithi178.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x558a6f5424b1] 2024-06-28T15:25:22.187 INFO:tasks.workunit.client.0.smithi178.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, cls_rgw_reshard_initiator, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x32c) [0x558a6f553aac] 2024-06-28T15:25:22.187 INFO:tasks.workunit.client.0.smithi178.stderr: 9: main() 2024-06-28T15:25:22.187 INFO:tasks.workunit.client.0.smithi178.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f842e1e9d90] 2024-06-28T15:25:22.187 INFO:tasks.workunit.client.0.smithi178.stderr: 11: __libc_start_main() 2024-06-28T15:25:22.187 INFO:tasks.workunit.client.0.smithi178.stderr: 12: _start() 2024-06-28T15:25:22.187 INFO:tasks.workunit.client.0.smithi178.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-28T15:25:22.187 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.187 INFO:tasks.workunit.client.0.smithi178.stderr:--- logging levels --- 2024-06-28T15:25:22.187 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 none 2024-06-28T15:25:22.187 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 lockdep 2024-06-28T15:25:22.187 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 context 2024-06-28T15:25:22.188 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 crush 2024-06-28T15:25:22.188 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds 2024-06-28T15:25:22.188 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_balancer 2024-06-28T15:25:22.188 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_locker 2024-06-28T15:25:22.188 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log 2024-06-28T15:25:22.188 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_log_expire 2024-06-28T15:25:22.188 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mds_migrator 2024-06-28T15:25:22.188 INFO:tasks.workunit.client.0.smithi178.stderr: 3/ 5 mds_quiesce 2024-06-28T15:25:22.188 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 buffer 2024-06-28T15:25:22.188 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 timer 2024-06-28T15:25:22.188 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 filer 2024-06-28T15:25:22.188 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 striper 2024-06-28T15:25:22.189 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 1 objecter 2024-06-28T15:25:22.189 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rados 2024-06-28T15:25:22.189 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd 2024-06-28T15:25:22.189 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_mirror 2024-06-28T15:25:22.189 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_replay 2024-06-28T15:25:22.189 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 rbd_pwl 2024-06-28T15:25:22.189 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 journaler 2024-06-28T15:25:22.189 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objectcacher 2024-06-28T15:25:22.189 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 immutable_obj_cache 2024-06-28T15:25:22.189 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 client 2024-06-28T15:25:22.189 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 osd 2024-06-28T15:25:22.189 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 optracker 2024-06-28T15:25:22.189 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 objclass 2024-06-28T15:25:22.189 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 filestore 2024-06-28T15:25:22.190 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 journal 2024-06-28T15:25:22.190 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 ms 2024-06-28T15:25:22.190 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mon 2024-06-28T15:25:22.190 INFO:tasks.workunit.client.0.smithi178.stderr: 0/10 monc 2024-06-28T15:25:22.190 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 paxos 2024-06-28T15:25:22.190 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 tp 2024-06-28T15:25:22.190 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 auth 2024-06-28T15:25:22.190 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 crypto 2024-06-28T15:25:22.190 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 finisher 2024-06-28T15:25:22.190 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 reserver 2024-06-28T15:25:22.190 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 heartbeatmap 2024-06-28T15:25:22.190 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 perfcounter 2024-06-28T15:25:22.190 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw 2024-06-28T15:25:22.191 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_sync 2024-06-28T15:25:22.191 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_datacache 2024-06-28T15:25:22.191 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_access 2024-06-28T15:25:22.191 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_dbstore 2024-06-28T15:25:22.191 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_flight 2024-06-28T15:25:22.191 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 rgw_lifecycle 2024-06-28T15:25:22.191 INFO:tasks.workunit.client.0.smithi178.stderr: 20/20 rgw_notification 2024-06-28T15:25:22.191 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 javaclient 2024-06-28T15:25:22.191 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 asok 2024-06-28T15:25:22.191 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 1 throttle 2024-06-28T15:25:22.191 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 0 refs 2024-06-28T15:25:22.191 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 compressor 2024-06-28T15:25:22.191 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluestore 2024-06-28T15:25:22.191 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 bluefs 2024-06-28T15:25:22.192 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 3 bdev 2024-06-28T15:25:22.192 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 kstore 2024-06-28T15:25:22.192 INFO:tasks.workunit.client.0.smithi178.stderr: 4/ 5 rocksdb 2024-06-28T15:25:22.192 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 fuse 2024-06-28T15:25:22.192 INFO:tasks.workunit.client.0.smithi178.stderr: 2/ 5 mgr 2024-06-28T15:25:22.192 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mgrc 2024-06-28T15:25:22.192 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 dpdk 2024-06-28T15:25:22.192 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 eventtrace 2024-06-28T15:25:22.192 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 prioritycache 2024-06-28T15:25:22.192 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 test 2024-06-28T15:25:22.192 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephfs_mirror 2024-06-28T15:25:22.192 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cephsqlite 2024-06-28T15:25:22.192 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 crimson_interrupt 2024-06-28T15:25:22.192 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore 2024-06-28T15:25:22.193 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_onode 2024-06-28T15:25:22.193 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_odata 2024-06-28T15:25:22.193 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_omap 2024-06-28T15:25:22.193 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_tm 2024-06-28T15:25:22.193 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_t 2024-06-28T15:25:22.193 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cleaner 2024-06-28T15:25:22.193 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_epm 2024-06-28T15:25:22.193 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_lba 2024-06-28T15:25:22.193 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_fixedkv_tree 2024-06-28T15:25:22.193 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_cache 2024-06-28T15:25:22.193 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_journal 2024-06-28T15:25:22.193 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_device 2024-06-28T15:25:22.193 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 seastore_backref 2024-06-28T15:25:22.193 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 alienstore 2024-06-28T15:25:22.194 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 mclock 2024-06-28T15:25:22.194 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 cyanstore 2024-06-28T15:25:22.194 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 ceph_exporter 2024-06-28T15:25:22.194 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 memstore 2024-06-28T15:25:22.194 INFO:tasks.workunit.client.0.smithi178.stderr: 1/ 5 trace 2024-06-28T15:25:22.194 INFO:tasks.workunit.client.0.smithi178.stderr: 0/ 5 ceph_dedup 2024-06-28T15:25:22.194 INFO:tasks.workunit.client.0.smithi178.stderr: -2/-2 (syslog threshold) 2024-06-28T15:25:22.194 INFO:tasks.workunit.client.0.smithi178.stderr: 99/99 (stderr threshold) 2024-06-28T15:25:22.194 INFO:tasks.workunit.client.0.smithi178.stderr:--- pthread ID / name mapping for recent threads --- 2024-06-28T15:25:22.194 INFO:tasks.workunit.client.0.smithi178.stderr: 7f824ffff640 / http_manager 2024-06-28T15:25:22.194 INFO:tasks.workunit.client.0.smithi178.stderr: 7f82de7e4640 / http_manager 2024-06-28T15:25:22.194 INFO:tasks.workunit.client.0.smithi178.stderr: 7f82df7e6640 / rgw_dt_lg_renew 2024-06-28T15:25:22.194 INFO:tasks.workunit.client.0.smithi178.stderr: 7f82faffd640 / ms_dispatch 2024-06-28T15:25:22.194 INFO:tasks.workunit.client.0.smithi178.stderr: 7f830cff9640 / io_context_pool 2024-06-28T15:25:22.195 INFO:tasks.workunit.client.0.smithi178.stderr: 7f830effd640 / ms_dispatch 2024-06-28T15:25:22.195 INFO:tasks.workunit.client.0.smithi178.stderr: 7f8315e03640 / ms_dispatch 2024-06-28T15:25:22.195 INFO:tasks.workunit.client.0.smithi178.stderr: 7f8317606640 / io_context_pool 2024-06-28T15:25:22.195 INFO:tasks.workunit.client.0.smithi178.stderr: 7f8428db9640 / msgr-worker-2 2024-06-28T15:25:22.195 INFO:tasks.workunit.client.0.smithi178.stderr: 7f84295ba640 / msgr-worker-1 2024-06-28T15:25:22.195 INFO:tasks.workunit.client.0.smithi178.stderr: 7f8429dbb640 / msgr-worker-0 2024-06-28T15:25:22.195 INFO:tasks.workunit.client.0.smithi178.stderr: 7f842a5bc640 / io_context_pool 2024-06-28T15:25:22.195 INFO:tasks.workunit.client.0.smithi178.stderr: 7f842baa9980 / radosgw-admin 2024-06-28T15:25:22.195 INFO:tasks.workunit.client.0.smithi178.stderr: max_recent 500 2024-06-28T15:25:22.195 INFO:tasks.workunit.client.0.smithi178.stderr: max_new 1000 2024-06-28T15:25:22.195 INFO:tasks.workunit.client.0.smithi178.stderr: log_file /var/lib/ceph/crash/2024-06-28T15:25:22.080287Z_8fb65c4a-05c8-415a-a87c-65fed1123164/log 2024-06-28T15:25:22.195 INFO:tasks.workunit.client.0.smithi178.stderr:--- end dump of recent events --- 2024-06-28T15:25:22.195 INFO:tasks.workunit.client.0.smithi178.stderr:Aborted 2024-06-28T15:25:22.195 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-06-28T15:25:22.200 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:22.200 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:22.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.156+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.156+0000 7f07330f5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.156+0000 7f07330f5980 20 realm 2024-06-28T15:25:22.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.156+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.156+0000 7f07330f5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.156+0000 7f07330f5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:22.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.156+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.156+0000 7f07330f5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.156+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.200 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.160+0000 7f07330f5980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:22.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.160+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.160+0000 7f07330f5980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:22.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.160+0000 7f07330f5980 20 searching for the correct realm 2024-06-28T15:25:22.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.164+0000 7f072bfff640 0 -- 172.21.15.178:0/1003255188 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x560193e4ef00 msgr2=0x560193e51390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:22.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.172+0000 7f0730c06640 0 -- 172.21.15.178:0/1003255188 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f071c08ce60 msgr2=0x7f071c001000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:22.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.176+0000 7f07330f5980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:22.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.176+0000 7f07330f5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:22.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.176+0000 7f07330f5980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:22.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.176+0000 7f07330f5980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:22.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.176+0000 7f07330f5980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:22.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.176+0000 7f07330f5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:22.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.176+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.176+0000 7f07330f5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.201 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.176+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:22.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:22.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 20 zone default found 2024-06-28T15:25:22.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 4 Realm: () 2024-06-28T15:25:22.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:22.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:22.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:22.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 20 zonegroup default 2024-06-28T15:25:22.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.202 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 20 started sync module instance, tier type = 2024-06-28T15:25:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.180+0000 7f07330f5980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.196+0000 7f07330f5980 20 add_watcher() i=0 2024-06-28T15:25:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.196+0000 7f07330f5980 20 add_watcher() i=1 2024-06-28T15:25:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.196+0000 7f07330f5980 20 add_watcher() i=2 2024-06-28T15:25:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.196+0000 7f07330f5980 20 add_watcher() i=3 2024-06-28T15:25:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.196+0000 7f07330f5980 20 add_watcher() i=4 2024-06-28T15:25:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.196+0000 7f07330f5980 20 add_watcher() i=5 2024-06-28T15:25:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.196+0000 7f07330f5980 20 add_watcher() i=6 2024-06-28T15:25:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.200+0000 7f07330f5980 20 add_watcher() i=7 2024-06-28T15:25:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.200+0000 7f07330f5980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:22.203 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.200+0000 7f05e6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:22.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.200+0000 7f07330f5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:22.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.200+0000 7f07330f5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:22.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.200+0000 7f07330f5980 5 note: GC not initialized 2024-06-28T15:25:22.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.200+0000 7f05e5fe3640 20 reqs_thread_entry: start 2024-06-28T15:25:22.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.200+0000 7f07330f5980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:22.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.200+0000 7f07330f5980 20 Filter name: none 2024-06-28T15:25:22.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.200+0000 7f05577fe640 20 reqs_thread_entry: start 2024-06-28T15:25:22.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.200+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.204+0000 7f07330f5980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:22.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.204+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.204+0000 7f07330f5980 20 rados_obj.operate() r=0 bl.length=457 2024-06-28T15:25:22.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.204+0000 7f07330f5980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:22.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.204+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.204 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.204+0000 7f07330f5980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:22.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.204+0000 7f07330f5980 20 rados->read ofs=0 len=0 2024-06-28T15:25:22.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.204+0000 7f07330f5980 20 rados_obj.operate() r=0 bl.length=457 2024-06-28T15:25:22.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.204+0000 7f07330f5980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:22.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.208+0000 7f05e6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:22.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.212+0000 7f07330f5980 20 remove_watcher() i=0 2024-06-28T15:25:22.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.212+0000 7f07330f5980 2 removed watcher, disabling cache 2024-06-28T15:25:22.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.212+0000 7f07330f5980 20 remove_watcher() i=1 2024-06-28T15:25:22.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.216+0000 7f07330f5980 20 remove_watcher() i=2 2024-06-28T15:25:22.205 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.216+0000 7f07330f5980 20 remove_watcher() i=3 2024-06-28T15:25:22.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.220+0000 7f07330f5980 20 remove_watcher() i=4 2024-06-28T15:25:22.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.220+0000 7f07330f5980 20 remove_watcher() i=5 2024-06-28T15:25:22.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.224+0000 7f07330f5980 20 remove_watcher() i=6 2024-06-28T15:25:22.206 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:22.224+0000 7f07330f5980 20 remove_watcher() i=7 2024-06-28T15:25:22.217 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:22.217 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:25:22.217 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: abort-at-do-reshard id: 843e24ce-085a-4043-8f55-38e05370e790.4686.29 num_objects: 20 size_kb: 1 num_shards: 1 2024-06-28T15:25:22.217 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:52.490 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 2 --rgw-reshard-bucket-lock-duration 30 2024-06-28T15:25:52.525 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:52.525 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:52.752 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.772+0000 7f980c240640 0 --2- 172.21.15.178:0/574208453 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x5647843e94e0 0x564784555710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-06-28T15:25:52.778 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.796+0000 7f980ef30980 20 rados->read ofs=0 len=0 2024-06-28T15:25:52.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.800+0000 7f980ef30980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:52.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.800+0000 7f980ef30980 20 realm 2024-06-28T15:25:52.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.800+0000 7f980ef30980 20 rados->read ofs=0 len=0 2024-06-28T15:25:52.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.800+0000 7f980ef30980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:52.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.800+0000 7f980ef30980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:52.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.800+0000 7f980ef30980 20 rados->read ofs=0 len=0 2024-06-28T15:25:52.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.800+0000 7f980ef30980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:52.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.800+0000 7f980ef30980 20 rados->read ofs=0 len=0 2024-06-28T15:25:52.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.800+0000 7f980ef30980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:52.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.800+0000 7f980ef30980 20 rados->read ofs=0 len=0 2024-06-28T15:25:52.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.800+0000 7f980ef30980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:52.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.800+0000 7f980ef30980 20 searching for the correct realm 2024-06-28T15:25:52.786 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:52.804+0000 7f980c240640 0 -- 172.21.15.178:0/1991617616 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f9804088040 msgr2=0x7f9804071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:53.001 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.020+0000 7f980ca41640 0 -- 172.21.15.178:0/1991617616 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f98000cb940 msgr2=0x7f98000be3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:53.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.236+0000 7f980ef30980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:53.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.236+0000 7f980ef30980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:53.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.236+0000 7f980ef30980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:53.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.236+0000 7f980ef30980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:53.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.236+0000 7f980ef30980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:53.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.236+0000 7f980ef30980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:53.218 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.236+0000 7f980ef30980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.236+0000 7f980ef30980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:53.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.236+0000 7f980ef30980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:53.219 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:53.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 20 zone default found 2024-06-28T15:25:53.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 4 Realm: () 2024-06-28T15:25:53.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:53.220 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:53.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:53.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 20 zonegroup default 2024-06-28T15:25:53.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:53.221 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:53.222 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:53.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 20 started sync module instance, tier type = 2024-06-28T15:25:53.223 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.240+0000 7f980ef30980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:53.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.264+0000 7f980ef30980 20 add_watcher() i=0 2024-06-28T15:25:53.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.264+0000 7f980ef30980 20 add_watcher() i=1 2024-06-28T15:25:53.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.264+0000 7f980ef30980 20 add_watcher() i=2 2024-06-28T15:25:53.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.264+0000 7f980ef30980 20 add_watcher() i=3 2024-06-28T15:25:53.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.264+0000 7f980ef30980 20 add_watcher() i=4 2024-06-28T15:25:53.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.264+0000 7f980ef30980 20 add_watcher() i=5 2024-06-28T15:25:53.246 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.264+0000 7f980ef30980 20 add_watcher() i=6 2024-06-28T15:25:53.248 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.268+0000 7f980ef30980 20 add_watcher() i=7 2024-06-28T15:25:53.248 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.268+0000 7f980ef30980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:53.252 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.272+0000 7f96c2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:53.253 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.272+0000 7f980ef30980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:53.253 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.272+0000 7f980ef30980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:53.253 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.272+0000 7f980ef30980 5 note: GC not initialized 2024-06-28T15:25:53.254 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.272+0000 7f96c1fe3640 20 reqs_thread_entry: start 2024-06-28T15:25:53.254 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.272+0000 7f980ef30980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:53.254 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.272+0000 7f980ef30980 20 Filter name: none 2024-06-28T15:25:53.254 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.272+0000 7f96337fe640 20 reqs_thread_entry: start 2024-06-28T15:25:53.254 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.272+0000 7f980ef30980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-06-28T15:25:53.254 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.272+0000 7f980ef30980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.255 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.276+0000 7f980ef30980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:53.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.276+0000 7f980ef30980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-06-28T15:25:53.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.276+0000 7f980ef30980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-06-28T15:25:53.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.276+0000 7f980ef30980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 : miss 2024-06-28T15:25:53.256 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.276+0000 7f980ef30980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.276+0000 7f980ef30980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:53.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.276+0000 7f980ef30980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 info.flags=0x17 2024-06-28T15:25:53.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.276+0000 7f980ef30980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 to cache LRU end 2024-06-28T15:25:53.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.276+0000 7f980ef30980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:25:53.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.276+0000 7f980ef30980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:53.257 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.276+0000 7f980ef30980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 2024-06-28T15:25:53.267 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.284+0000 7f980ef30980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 info.flags=0x17 2024-06-28T15:25:53.267 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.284+0000 7f980ef30980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 to cache LRU end 2024-06-28T15:25:53.268 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.284+0000 7f980ef30980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:53.268 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.284+0000 7f980ef30980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29, ofs0, ns] 2024-06-28T15:25:53.268 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.284+0000 7f96e3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595099 cookie 94865163703856 notifier 6710 bl.length()=987 2024-06-28T15:25:53.268 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.284+0000 7f96e3fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 info.flags=0x17 2024-06-28T15:25:53.268 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.284+0000 7f96e3fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 to cache LRU end 2024-06-28T15:25:53.268 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.284+0000 7f96e3fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:53.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.288+0000 7f980ef30980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): orig_sources= new_sources= 2024-06-28T15:25:53.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.288+0000 7f980ef30980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): potential sources added=[] removed=[] 2024-06-28T15:25:53.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.288+0000 7f980ef30980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): orig_dests= new_dests= 2024-06-28T15:25:53.269 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.288+0000 7f980ef30980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): potential dests added=[] removed=[] 2024-06-28T15:25:53.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.292+0000 7f980ef30980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.29.1.0 2024-06-28T15:25:53.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.292+0000 7f980ef30980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.29.1.0 2024-06-28T15:25:53.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.292+0000 7f980ef30980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.29.1.1 2024-06-28T15:25:53.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.292+0000 7f980ef30980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.29.1.1 2024-06-28T15:25:53.272 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.292+0000 7f980ef30980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.29.0 2024-06-28T15:25:53.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.296+0000 7f980ef30980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 : hit (requested=0x17, cached=0x17) 2024-06-28T15:25:53.277 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.296+0000 7f980ef30980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 2024-06-28T15:25:53.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.300+0000 7f980ef30980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 info.flags=0x17 2024-06-28T15:25:53.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.300+0000 7f980ef30980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 to cache LRU end 2024-06-28T15:25:53.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.300+0000 7f980ef30980 10 updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:53.281 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.300+0000 7f980ef30980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29, ofs0, ns] 2024-06-28T15:25:53.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.300+0000 7f96e37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595100 cookie 94865163703856 notifier 6710 bl.length()=955 2024-06-28T15:25:53.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.300+0000 7f96e37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 info.flags=0x17 2024-06-28T15:25:53.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.300+0000 7f96e37fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:843e24ce-085a-4043-8f55-38e05370e790.4686.29 to cache LRU end 2024-06-28T15:25:53.282 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.300+0000 7f96e37fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=265 2024-06-28T15:25:53.283 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.304+0000 7f980ef30980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): orig_sources= new_sources= 2024-06-28T15:25:53.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.304+0000 7f980ef30980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): potential sources added=[] removed=[] 2024-06-28T15:25:53.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.304+0000 7f980ef30980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): orig_dests= new_dests= 2024-06-28T15:25:53.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.304+0000 7f980ef30980 20 handle_bi_update(): bucket=:abort-at-do-reshard[843e24ce-085a-4043-8f55-38e05370e790.4686.29]): potential dests added=[] removed=[] 2024-06-28T15:25:53.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.304+0000 7f980ef30980 1 execute INFO: reshard of bucket "abort-at-do-reshard" from 2 shards to 2 shards completed successfully 2024-06-28T15:25:53.289 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.308+0000 7f96c2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:53.294 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.312+0000 7f980ef30980 20 remove_watcher() i=0 2024-06-28T15:25:53.294 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.312+0000 7f980ef30980 2 removed watcher, disabling cache 2024-06-28T15:25:53.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.316+0000 7f980ef30980 20 remove_watcher() i=1 2024-06-28T15:25:53.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.316+0000 7f980ef30980 20 remove_watcher() i=2 2024-06-28T15:25:53.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.320+0000 7f980ef30980 20 remove_watcher() i=3 2024-06-28T15:25:53.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.320+0000 7f980ef30980 20 remove_watcher() i=4 2024-06-28T15:25:53.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.324+0000 7f980ef30980 20 remove_watcher() i=5 2024-06-28T15:25:53.306 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.324+0000 7f980ef30980 20 remove_watcher() i=6 2024-06-28T15:25:53.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.328+0000 7f980ef30980 20 remove_watcher() i=7 2024-06-28T15:25:53.322 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-06-28T15:25:53.353 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:53.353 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:53.397 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.416+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.416+0000 7f735a78e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:53.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.416+0000 7f735a78e980 20 realm 2024-06-28T15:25:53.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.416+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.420+0000 7f735a78e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:53.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.420+0000 7f735a78e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:53.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.420+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.420+0000 7f735a78e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:53.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.420+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.420+0000 7f735a78e980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:53.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.420+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.420+0000 7f735a78e980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:53.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.420+0000 7f735a78e980 20 searching for the correct realm 2024-06-28T15:25:53.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.424+0000 7f735829f640 0 -- 172.21.15.178:0/3979686366 >> [v2:172.21.15.178:6823/1880098663,v1:172.21.15.178:6825/1880098663] conn(0x55ed88ac9010 msgr2=0x7f735010f0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:53.412 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.432+0000 7f7358aa0640 0 -- 172.21.15.178:0/3979686366 >> [v2:172.21.15.195:6824/1278553753,v1:172.21.15.195:6825/1278553753] conn(0x7f734c036b20 msgr2=0x7f734c02d920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:53.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.440+0000 7f735a78e980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:53.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.440+0000 7f735a78e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:53.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.440+0000 7f735a78e980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:53.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.440+0000 7f735a78e980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:53.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.440+0000 7f735a78e980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:53.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.440+0000 7f735a78e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:53.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.440+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.440+0000 7f735a78e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:53.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.440+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:53.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:53.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 20 zone default found 2024-06-28T15:25:53.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 4 Realm: () 2024-06-28T15:25:53.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:53.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:53.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:53.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 20 zonegroup default 2024-06-28T15:25:53.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:53.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:53.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:53.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 20 started sync module instance, tier type = 2024-06-28T15:25:53.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.444+0000 7f735a78e980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:53.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.460+0000 7f735a78e980 20 add_watcher() i=0 2024-06-28T15:25:53.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.460+0000 7f735a78e980 20 add_watcher() i=1 2024-06-28T15:25:53.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.460+0000 7f735a78e980 20 add_watcher() i=2 2024-06-28T15:25:53.440 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.460+0000 7f735a78e980 20 add_watcher() i=3 2024-06-28T15:25:53.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.460+0000 7f735a78e980 20 add_watcher() i=4 2024-06-28T15:25:53.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.460+0000 7f735a78e980 20 add_watcher() i=5 2024-06-28T15:25:53.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.460+0000 7f735a78e980 20 add_watcher() i=6 2024-06-28T15:25:53.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.460+0000 7f735a78e980 20 add_watcher() i=7 2024-06-28T15:25:53.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.460+0000 7f735a78e980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:53.444 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.464+0000 7f720efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:53.444 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.464+0000 7f735a78e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:53.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.464+0000 7f735a78e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:53.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.464+0000 7f735a78e980 5 note: GC not initialized 2024-06-28T15:25:53.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.464+0000 7f720dfdb640 20 reqs_thread_entry: start 2024-06-28T15:25:53.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.464+0000 7f735a78e980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:53.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.464+0000 7f735a78e980 20 Filter name: none 2024-06-28T15:25:53.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.464+0000 7f717f7fe640 20 reqs_thread_entry: start 2024-06-28T15:25:53.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.464+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.464+0000 7f735a78e980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:53.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.464+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.468+0000 7f735a78e980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:53.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.468+0000 7f735a78e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:53.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.468+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.449 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.468+0000 7f735a78e980 20 rados_obj.operate() r=0 bl.length=211 2024-06-28T15:25:53.449 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.468+0000 7f735a78e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:53.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.468+0000 7f735a78e980 20 rados_obj.operate() r=0 bl.length=425 2024-06-28T15:25:53.450 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.468+0000 7f735a78e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:53.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.472+0000 7f720efdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:53.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.476+0000 7f735a78e980 20 remove_watcher() i=0 2024-06-28T15:25:53.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.476+0000 7f735a78e980 2 removed watcher, disabling cache 2024-06-28T15:25:53.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.476+0000 7f735a78e980 20 remove_watcher() i=1 2024-06-28T15:25:53.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.480+0000 7f735a78e980 20 remove_watcher() i=2 2024-06-28T15:25:53.463 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.480+0000 7f735a78e980 20 remove_watcher() i=3 2024-06-28T15:25:53.464 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.484+0000 7f735a78e980 20 remove_watcher() i=4 2024-06-28T15:25:53.466 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.484+0000 7f735a78e980 20 remove_watcher() i=5 2024-06-28T15:25:53.467 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.484+0000 7f735a78e980 20 remove_watcher() i=6 2024-06-28T15:25:53.468 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:53.488+0000 7f735a78e980 20 remove_watcher() i=7 2024-06-28T15:25:53.481 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:53.482 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:25:53.482 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: abort-at-do-reshard id: 843e24ce-085a-4043-8f55-38e05370e790.4686.29 num_objects: 19 size_kb: 1 num_shards: 2 2024-06-28T15:25:53.482 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:54.216 INFO:tasks.workunit.client.0.smithi178.stderr: test: reshard versioned bucket 2024-06-28T15:25:54.216 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket myver 2024-06-28T15:25:54.249 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:54.249 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:54.293 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.312+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.316+0000 7fd7a9dca980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:54.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.316+0000 7fd7a9dca980 20 realm 2024-06-28T15:25:54.295 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.316+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.316+0000 7fd7a9dca980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:54.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.316+0000 7fd7a9dca980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:54.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.316+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.316+0000 7fd7a9dca980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:54.296 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.316+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.316+0000 7fd7a9dca980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:54.297 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.316+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.316+0000 7fd7a9dca980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:54.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.316+0000 7fd7a9dca980 20 searching for the correct realm 2024-06-28T15:25:54.304 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.324+0000 7fd7a80dc640 0 -- 172.21.15.178:0/365650941 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x55aff2d92240 msgr2=0x55aff2d946c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:54.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.328+0000 7fd7a78db640 0 -- 172.21.15.178:0/365650941 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fd7900611e0 msgr2=0x7fd79008e710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:54.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:54.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:54.318 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:54.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:54.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:54.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:54.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:54.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:54.319 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:54.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 zone default found 2024-06-28T15:25:54.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 4 Realm: () 2024-06-28T15:25:54.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:54.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:54.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:54.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 zonegroup default 2024-06-28T15:25:54.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.336+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.340+0000 7fd7a9dca980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:54.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.340+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.340+0000 7fd7a9dca980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:54.322 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.340+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.340+0000 7fd7a9dca980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:54.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.340+0000 7fd7a9dca980 20 started sync module instance, tier type = 2024-06-28T15:25:54.323 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.340+0000 7fd7a9dca980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:54.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.356+0000 7fd7a9dca980 20 add_watcher() i=0 2024-06-28T15:25:54.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.356+0000 7fd7a9dca980 20 add_watcher() i=1 2024-06-28T15:25:54.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.356+0000 7fd7a9dca980 20 add_watcher() i=2 2024-06-28T15:25:54.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.356+0000 7fd7a9dca980 20 add_watcher() i=3 2024-06-28T15:25:54.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.356+0000 7fd7a9dca980 20 add_watcher() i=4 2024-06-28T15:25:54.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.356+0000 7fd7a9dca980 20 add_watcher() i=5 2024-06-28T15:25:54.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.356+0000 7fd7a9dca980 20 add_watcher() i=6 2024-06-28T15:25:54.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.356+0000 7fd7a9dca980 20 add_watcher() i=7 2024-06-28T15:25:54.337 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.356+0000 7fd7a9dca980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:54.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.356+0000 7fd65dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:54.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:54.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:54.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 5 note: GC not initialized 2024-06-28T15:25:54.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd65cfe1640 20 reqs_thread_entry: start 2024-06-28T15:25:54.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:54.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 20 Filter name: none 2024-06-28T15:25:54.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd5d67fc640 20 reqs_thread_entry: start 2024-06-28T15:25:54.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:25:54.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:25:54.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:54.342 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.342 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:25:54.344 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.344 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:25:54.344 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd7a9dca980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:54.345 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.360+0000 7fd65dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:54.347 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.368+0000 7fd7a9dca980 20 remove_watcher() i=0 2024-06-28T15:25:54.347 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.368+0000 7fd7a9dca980 2 removed watcher, disabling cache 2024-06-28T15:25:54.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.368+0000 7fd7a9dca980 20 remove_watcher() i=1 2024-06-28T15:25:54.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.372+0000 7fd7a9dca980 20 remove_watcher() i=2 2024-06-28T15:25:54.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.372+0000 7fd7a9dca980 20 remove_watcher() i=3 2024-06-28T15:25:54.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.376+0000 7fd7a9dca980 20 remove_watcher() i=4 2024-06-28T15:25:54.356 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.376+0000 7fd7a9dca980 20 remove_watcher() i=5 2024-06-28T15:25:54.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.376+0000 7fd7a9dca980 20 remove_watcher() i=6 2024-06-28T15:25:54.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.376+0000 7fd7a9dca980 20 remove_watcher() i=7 2024-06-28T15:25:54.372 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:54.372 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:25:54.372 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: myver id: 843e24ce-085a-4043-8f55-38e05370e790.4686.19 num_objects: 0 size_kb: 0 num_shards: 1 2024-06-28T15:25:54.373 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:54.373 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 2 2024-06-28T15:25:54.400 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:54.401 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:54.444 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.464+0000 7f0fb59a6980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.464+0000 7f0fb59a6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:54.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.464+0000 7f0fb59a6980 20 realm 2024-06-28T15:25:54.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.464+0000 7f0fb59a6980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.464+0000 7f0fb59a6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:54.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.464+0000 7f0fb59a6980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:54.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.464+0000 7f0fb59a6980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.464+0000 7f0fb59a6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:54.446 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.464+0000 7f0fb59a6980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.447 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.468+0000 7f0fb59a6980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:54.447 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.468+0000 7f0fb59a6980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.468+0000 7f0fb59a6980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:54.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.468+0000 7f0fb59a6980 20 searching for the correct realm 2024-06-28T15:25:54.451 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.472+0000 7f0fb2cb6640 0 -- 172.21.15.178:0/376935366 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f0fa4010040 msgr2=0x7f0fa405af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:54.669 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.688+0000 7f0fb34b7640 0 -- 172.21.15.178:0/376935366 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f0fa81017e0 msgr2=0x7f0fa81084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:54.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.904+0000 7f0fb59a6980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:54.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.904+0000 7f0fb59a6980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:54.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.904+0000 7f0fb59a6980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:54.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.904+0000 7f0fb59a6980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:54.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.904+0000 7f0fb59a6980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:54.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.904+0000 7f0fb59a6980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:54.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.904+0000 7f0fb59a6980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.904+0000 7f0fb59a6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:54.887 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.904+0000 7f0fb59a6980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.888 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:54.888 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:54.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 20 zone default found 2024-06-28T15:25:54.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 4 Realm: () 2024-06-28T15:25:54.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:54.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:54.889 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:54.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 20 zonegroup default 2024-06-28T15:25:54.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:54.890 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:54.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:54.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 20 started sync module instance, tier type = 2024-06-28T15:25:54.891 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.908+0000 7f0fb59a6980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:54.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.932+0000 7f0fb59a6980 20 add_watcher() i=0 2024-06-28T15:25:54.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.932+0000 7f0fb59a6980 20 add_watcher() i=1 2024-06-28T15:25:54.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.932+0000 7f0fb59a6980 20 add_watcher() i=2 2024-06-28T15:25:54.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.932+0000 7f0fb59a6980 20 add_watcher() i=3 2024-06-28T15:25:54.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.932+0000 7f0fb59a6980 20 add_watcher() i=4 2024-06-28T15:25:54.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.932+0000 7f0fb59a6980 20 add_watcher() i=5 2024-06-28T15:25:54.916 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.932+0000 7f0fb59a6980 20 add_watcher() i=6 2024-06-28T15:25:54.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.936+0000 7f0fb59a6980 20 add_watcher() i=7 2024-06-28T15:25:54.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.936+0000 7f0fb59a6980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:54.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.940+0000 7f0e69fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:54.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.940+0000 7f0fb59a6980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:54.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.940+0000 7f0fb59a6980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:54.923 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.940+0000 7f0fb59a6980 5 note: GC not initialized 2024-06-28T15:25:54.926 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0e68fe1640 20 reqs_thread_entry: start 2024-06-28T15:25:54.926 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:54.926 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 20 Filter name: none 2024-06-28T15:25:54.926 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0de27fc640 20 reqs_thread_entry: start 2024-06-28T15:25:54.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-06-28T15:25:54.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:25:54.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-06-28T15:25:54.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-06-28T15:25:54.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 : miss 2024-06-28T15:25:54.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 20 rados->read ofs=0 len=0 2024-06-28T15:25:54.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:25:54.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:25:54.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 10 adding default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:25:54.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:25:54.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:25:54.929 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.944+0000 7f0fb59a6980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 2024-06-28T15:25:54.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.952+0000 7f0fb59a6980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:25:54.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.952+0000 7f0fb59a6980 10 moving default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:25:54.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.952+0000 7f0fb59a6980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:25:54.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.952+0000 7f0fb59a6980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19, ofs0, ns] 2024-06-28T15:25:54.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.952+0000 7f0e96ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595117 cookie 94284820203648 notifier 6755 bl.length()=897 2024-06-28T15:25:54.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.952+0000 7f0e96ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:25:54.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.952+0000 7f0e96ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:25:54.933 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.952+0000 7f0e96ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:25:54.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.952+0000 7f0fb59a6980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): orig_sources= new_sources= 2024-06-28T15:25:54.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.952+0000 7f0fb59a6980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): potential sources added=[] removed=[] 2024-06-28T15:25:54.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.952+0000 7f0fb59a6980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): orig_dests= new_dests= 2024-06-28T15:25:54.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.952+0000 7f0fb59a6980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): potential dests added=[] removed=[] 2024-06-28T15:25:54.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.956+0000 7f0fb59a6980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.1.0 2024-06-28T15:25:54.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.956+0000 7f0fb59a6980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.1.0 2024-06-28T15:25:54.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.956+0000 7f0fb59a6980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.1.1 2024-06-28T15:25:54.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.956+0000 7f0fb59a6980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.1.1 2024-06-28T15:25:54.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.956+0000 7f0fb59a6980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.0 2024-06-28T15:25:54.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.956+0000 7f0fb59a6980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 : hit (requested=0x17, cached=0x17) 2024-06-28T15:25:54.938 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.956+0000 7f0fb59a6980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 2024-06-28T15:25:54.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.960+0000 7f0fb59a6980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:25:54.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.960+0000 7f0fb59a6980 10 moving default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:25:54.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.960+0000 7f0fb59a6980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:25:54.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.960+0000 7f0fb59a6980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19, ofs0, ns] 2024-06-28T15:25:54.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.960+0000 7f0e967fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595118 cookie 94284820203648 notifier 6755 bl.length()=865 2024-06-28T15:25:54.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.960+0000 7f0e967fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:25:54.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.960+0000 7f0e967fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:25:54.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.960+0000 7f0e967fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:25:54.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.960+0000 7f0fb59a6980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): orig_sources= new_sources= 2024-06-28T15:25:54.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.960+0000 7f0fb59a6980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): potential sources added=[] removed=[] 2024-06-28T15:25:54.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.960+0000 7f0fb59a6980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): orig_dests= new_dests= 2024-06-28T15:25:54.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.960+0000 7f0fb59a6980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): potential dests added=[] removed=[] 2024-06-28T15:25:54.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.964+0000 7f0fb59a6980 1 execute INFO: reshard of bucket "myver" from 2 shards to 2 shards completed successfully 2024-06-28T15:25:54.949 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.968+0000 7f0e69fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:54.954 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.972+0000 7f0fb59a6980 20 remove_watcher() i=0 2024-06-28T15:25:54.954 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.972+0000 7f0fb59a6980 2 removed watcher, disabling cache 2024-06-28T15:25:54.956 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.976+0000 7f0fb59a6980 20 remove_watcher() i=1 2024-06-28T15:25:54.958 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.976+0000 7f0fb59a6980 20 remove_watcher() i=2 2024-06-28T15:25:54.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.980+0000 7f0fb59a6980 20 remove_watcher() i=3 2024-06-28T15:25:54.963 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.980+0000 7f0fb59a6980 20 remove_watcher() i=4 2024-06-28T15:25:54.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.984+0000 7f0fb59a6980 20 remove_watcher() i=5 2024-06-28T15:25:54.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.984+0000 7f0fb59a6980 20 remove_watcher() i=6 2024-06-28T15:25:54.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:54.988+0000 7f0fb59a6980 20 remove_watcher() i=7 2024-06-28T15:25:54.985 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket myver 2024-06-28T15:25:55.015 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:55.015 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:55.061 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.080+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.080+0000 7f6b2cc76980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:55.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.080+0000 7f6b2cc76980 20 realm 2024-06-28T15:25:55.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.080+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.063 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.080+0000 7f6b2cc76980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:55.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.080+0000 7f6b2cc76980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:55.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.080+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.080+0000 7f6b2cc76980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:55.064 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.080+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.084+0000 7f6b2cc76980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:55.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.084+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.084+0000 7f6b2cc76980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:55.066 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.084+0000 7f6b2cc76980 20 searching for the correct realm 2024-06-28T15:25:55.071 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.088+0000 7f6b29f86640 0 -- 172.21.15.178:0/477685585 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f6b1c02a060 msgr2=0x7f6b1c07da20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:55.077 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.096+0000 7f6b29f86640 0 -- 172.21.15.178:0/477685585 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f69f407fda0 msgr2=0x7f69f40821d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:55.083 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.100+0000 7f6b2cc76980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:55.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.100+0000 7f6b2cc76980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:55.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.100+0000 7f6b2cc76980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:55.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.100+0000 7f6b2cc76980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:55.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.100+0000 7f6b2cc76980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:55.084 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.100+0000 7f6b2cc76980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:55.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.100+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:55.085 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.087 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:55.087 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.087 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:55.087 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 zone default found 2024-06-28T15:25:55.087 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 4 Realm: () 2024-06-28T15:25:55.088 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:55.088 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:55.088 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:55.088 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 zonegroup default 2024-06-28T15:25:55.088 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.088 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:55.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:55.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:55.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 started sync module instance, tier type = 2024-06-28T15:25:55.089 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.104+0000 7f6b2cc76980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:55.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.120+0000 7f6b2cc76980 20 add_watcher() i=0 2024-06-28T15:25:55.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.120+0000 7f6b2cc76980 20 add_watcher() i=1 2024-06-28T15:25:55.102 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.120+0000 7f6b2cc76980 20 add_watcher() i=2 2024-06-28T15:25:55.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.120+0000 7f6b2cc76980 20 add_watcher() i=3 2024-06-28T15:25:55.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.120+0000 7f6b2cc76980 20 add_watcher() i=4 2024-06-28T15:25:55.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.120+0000 7f6b2cc76980 20 add_watcher() i=5 2024-06-28T15:25:55.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.120+0000 7f6b2cc76980 20 add_watcher() i=6 2024-06-28T15:25:55.103 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.120+0000 7f6b2cc76980 20 add_watcher() i=7 2024-06-28T15:25:55.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.120+0000 7f6b2cc76980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:55.107 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.124+0000 7f69e0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:55.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.124+0000 7f6b2cc76980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:55.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.124+0000 7f6b2cc76980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:55.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.124+0000 7f6b2cc76980 5 note: GC not initialized 2024-06-28T15:25:55.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.124+0000 7f695b7fe640 20 reqs_thread_entry: start 2024-06-28T15:25:55.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.124+0000 7f6b2cc76980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:55.108 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.124+0000 7f6b2cc76980 20 Filter name: none 2024-06-28T15:25:55.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.124+0000 7f69597fa640 20 reqs_thread_entry: start 2024-06-28T15:25:55.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.124+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.124+0000 7f6b2cc76980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:25:55.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.124+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.124+0000 7f6b2cc76980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:25:55.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.124+0000 7f6b2cc76980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:55.109 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.124+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.128+0000 7f6b2cc76980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:25:55.110 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.128+0000 7f6b2cc76980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.128+0000 7f6b2cc76980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:25:55.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.128+0000 7f6b2cc76980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:55.111 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.128+0000 7f69e0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:55.114 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.132+0000 7f6b2cc76980 20 remove_watcher() i=0 2024-06-28T15:25:55.114 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.132+0000 7f6b2cc76980 2 removed watcher, disabling cache 2024-06-28T15:25:55.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.136+0000 7f6b2cc76980 20 remove_watcher() i=1 2024-06-28T15:25:55.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.136+0000 7f6b2cc76980 20 remove_watcher() i=2 2024-06-28T15:25:55.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.140+0000 7f6b2cc76980 20 remove_watcher() i=3 2024-06-28T15:25:55.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.140+0000 7f6b2cc76980 20 remove_watcher() i=4 2024-06-28T15:25:55.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.144+0000 7f6b2cc76980 20 remove_watcher() i=5 2024-06-28T15:25:55.124 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.144+0000 7f6b2cc76980 20 remove_watcher() i=6 2024-06-28T15:25:55.126 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.144+0000 7f6b2cc76980 20 remove_watcher() i=7 2024-06-28T15:25:55.139 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:55.139 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:25:55.140 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: myver id: 843e24ce-085a-4043-8f55-38e05370e790.4686.19 num_objects: 0 size_kb: 0 num_shards: 2 2024-06-28T15:25:55.140 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:55.220 INFO:tasks.workunit.client.0.smithi178.stderr: test: reshard removes olh entries with empty name 2024-06-28T15:25:55.637 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-06-28T15:25:55.669 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:55.669 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:55.713 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.732+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.718 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.736+0000 7ff6cdf14980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:55.718 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.736+0000 7ff6cdf14980 20 realm 2024-06-28T15:25:55.718 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.736+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.718 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.736+0000 7ff6cdf14980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:55.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.736+0000 7ff6cdf14980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:55.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.736+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.736+0000 7ff6cdf14980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:55.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.736+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.736+0000 7ff6cdf14980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:55.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.736+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.736+0000 7ff6cdf14980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:55.719 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.736+0000 7ff6cdf14980 20 searching for the correct realm 2024-06-28T15:25:55.723 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.744+0000 7ff6cc226640 0 -- 172.21.15.178:0/3853752523 >> [v2:172.21.15.195:6824/1278553753,v1:172.21.15.195:6825/1278553753] conn(0x7ff6c0064330 msgr2=0x7ff6c006a570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:55.729 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.748+0000 7ff6cba25640 0 -- 172.21.15.178:0/3853752523 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7ff6c008f280 msgr2=0x7ff6c001d7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:55.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.964+0000 7ff6cdf14980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:55.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.964+0000 7ff6cdf14980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:55.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.964+0000 7ff6cdf14980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:55.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.964+0000 7ff6cdf14980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:55.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.964+0000 7ff6cdf14980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:55.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.964+0000 7ff6cdf14980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:55.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.964+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.968+0000 7ff6cdf14980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:55.948 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.968+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.949 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.968+0000 7ff6cdf14980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:55.949 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.968+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.968+0000 7ff6cdf14980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:55.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.968+0000 7ff6cdf14980 20 zone default found 2024-06-28T15:25:55.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.968+0000 7ff6cdf14980 4 Realm: () 2024-06-28T15:25:55.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.968+0000 7ff6cdf14980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:55.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.968+0000 7ff6cdf14980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:55.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.968+0000 7ff6cdf14980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:55.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.968+0000 7ff6cdf14980 20 zonegroup default 2024-06-28T15:25:55.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.968+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.951 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.968+0000 7ff6cdf14980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:55.951 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.968+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.952 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.972+0000 7ff6cdf14980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:55.952 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.972+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.953 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.972+0000 7ff6cdf14980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:55.953 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.972+0000 7ff6cdf14980 20 started sync module instance, tier type = 2024-06-28T15:25:55.953 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.972+0000 7ff6cdf14980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:55.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.996+0000 7ff6cdf14980 20 add_watcher() i=0 2024-06-28T15:25:55.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.996+0000 7ff6cdf14980 20 add_watcher() i=1 2024-06-28T15:25:55.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.996+0000 7ff6cdf14980 20 add_watcher() i=2 2024-06-28T15:25:55.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.996+0000 7ff6cdf14980 20 add_watcher() i=3 2024-06-28T15:25:55.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.996+0000 7ff6cdf14980 20 add_watcher() i=4 2024-06-28T15:25:55.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.996+0000 7ff6cdf14980 20 add_watcher() i=5 2024-06-28T15:25:55.978 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:55.996+0000 7ff6cdf14980 20 add_watcher() i=6 2024-06-28T15:25:55.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.000+0000 7ff6cdf14980 20 add_watcher() i=7 2024-06-28T15:25:55.980 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.000+0000 7ff6cdf14980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:55.984 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.004+0000 7ff581fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:55.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.004+0000 7ff6cdf14980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:55.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.004+0000 7ff6cdf14980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:55.985 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.004+0000 7ff6cdf14980 5 note: GC not initialized 2024-06-28T15:25:55.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.004+0000 7ff580fe1640 20 reqs_thread_entry: start 2024-06-28T15:25:55.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.004+0000 7ff6cdf14980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:55.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.004+0000 7ff6cdf14980 20 Filter name: none 2024-06-28T15:25:55.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.004+0000 7ff4fa7fc640 20 reqs_thread_entry: start 2024-06-28T15:25:55.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.004+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.008+0000 7ff6cdf14980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:25:55.987 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.008+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.008+0000 7ff6cdf14980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:25:55.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.008+0000 7ff6cdf14980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:55.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.008+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.008+0000 7ff6cdf14980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:25:55.989 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.008+0000 7ff6cdf14980 20 rados->read ofs=0 len=0 2024-06-28T15:25:55.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.008+0000 7ff6cdf14980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:25:55.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.008+0000 7ff6cdf14980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:55.992 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.012+0000 7ff581fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:55.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.016+0000 7ff6cdf14980 20 remove_watcher() i=0 2024-06-28T15:25:55.998 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.016+0000 7ff6cdf14980 2 removed watcher, disabling cache 2024-06-28T15:25:56.000 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.020+0000 7ff6cdf14980 20 remove_watcher() i=1 2024-06-28T15:25:56.002 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.020+0000 7ff6cdf14980 20 remove_watcher() i=2 2024-06-28T15:25:56.004 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.024+0000 7ff6cdf14980 20 remove_watcher() i=3 2024-06-28T15:25:56.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.024+0000 7ff6cdf14980 20 remove_watcher() i=4 2024-06-28T15:25:56.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.028+0000 7ff6cdf14980 20 remove_watcher() i=5 2024-06-28T15:25:56.010 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.028+0000 7ff6cdf14980 20 remove_watcher() i=6 2024-06-28T15:25:56.011 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.028+0000 7ff6cdf14980 20 remove_watcher() i=7 2024-06-28T15:25:56.026 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:56.026 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:25:56.026 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: a-bucket id: 843e24ce-085a-4043-8f55-38e05370e790.4686.18 num_objects: 0 size_kb: 0 num_shards: 3 2024-06-28T15:25:56.027 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:56.027 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket layout --bucket a-bucket 2024-06-28T15:25:56.053 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:56.053 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:56.096 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.116+0000 7fe7210b4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.116+0000 7fe7210b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.116+0000 7fe7210b4980 20 realm 2024-06-28T15:25:56.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.116+0000 7fe7210b4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.116+0000 7fe7210b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.116+0000 7fe7210b4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:56.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.116+0000 7fe7210b4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.116+0000 7fe7210b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.116+0000 7fe7210b4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.120+0000 7fe7210b4980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:56.099 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.120+0000 7fe7210b4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.120+0000 7fe7210b4980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:56.100 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.120+0000 7fe7210b4980 20 searching for the correct realm 2024-06-28T15:25:56.106 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.124+0000 7fe71e3c4640 0 -- 172.21.15.178:0/32428448 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fe71006f070 msgr2=0x7fe710065930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:56.112 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.132+0000 7fe71e3c4640 0 -- 172.21.15.178:0/32428448 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fe5e807fda0 msgr2=0x7fe5e80821d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:56.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.136+0000 7fe7210b4980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:56.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.136+0000 7fe7210b4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:56.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.136+0000 7fe7210b4980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:56.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.136+0000 7fe7210b4980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:56.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.136+0000 7fe7210b4980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:56.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.136+0000 7fe7210b4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:56.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.136+0000 7fe7210b4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.136+0000 7fe7210b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.136+0000 7fe7210b4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:56.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:56.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 20 zone default found 2024-06-28T15:25:56.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 4 Realm: () 2024-06-28T15:25:56.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:56.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:56.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:56.121 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 20 zonegroup default 2024-06-28T15:25:56.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.122 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 20 started sync module instance, tier type = 2024-06-28T15:25:56.123 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.140+0000 7fe7210b4980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:56.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.156+0000 7fe7210b4980 20 add_watcher() i=0 2024-06-28T15:25:56.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.156+0000 7fe7210b4980 20 add_watcher() i=1 2024-06-28T15:25:56.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.156+0000 7fe7210b4980 20 add_watcher() i=2 2024-06-28T15:25:56.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.156+0000 7fe7210b4980 20 add_watcher() i=3 2024-06-28T15:25:56.137 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.156+0000 7fe7210b4980 20 add_watcher() i=4 2024-06-28T15:25:56.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.156+0000 7fe7210b4980 20 add_watcher() i=5 2024-06-28T15:25:56.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.156+0000 7fe7210b4980 20 add_watcher() i=6 2024-06-28T15:25:56.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.156+0000 7fe7210b4980 20 add_watcher() i=7 2024-06-28T15:25:56.138 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.156+0000 7fe7210b4980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:56.140 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.160+0000 7fe5d57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:56.141 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.160+0000 7fe7210b4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:56.141 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.160+0000 7fe7210b4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:56.141 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.160+0000 7fe7210b4980 5 note: GC not initialized 2024-06-28T15:25:56.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.160+0000 7fe54ffff640 20 reqs_thread_entry: start 2024-06-28T15:25:56.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.160+0000 7fe7210b4980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:56.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.160+0000 7fe7210b4980 20 Filter name: none 2024-06-28T15:25:56.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.160+0000 7fe54dffb640 20 reqs_thread_entry: start 2024-06-28T15:25:56.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.160+0000 7fe7210b4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.160+0000 7fe7210b4980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:25:56.142 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.160+0000 7fe7210b4980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.143 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.160+0000 7fe7210b4980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:25:56.143 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.160+0000 7fe7210b4980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:56.144 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.164+0000 7fe5d57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:56.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.168+0000 7fe7210b4980 20 remove_watcher() i=0 2024-06-28T15:25:56.149 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.168+0000 7fe7210b4980 2 removed watcher, disabling cache 2024-06-28T15:25:56.151 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.168+0000 7fe7210b4980 20 remove_watcher() i=1 2024-06-28T15:25:56.153 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.172+0000 7fe7210b4980 20 remove_watcher() i=2 2024-06-28T15:25:56.155 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.172+0000 7fe7210b4980 20 remove_watcher() i=3 2024-06-28T15:25:56.157 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.176+0000 7fe7210b4980 20 remove_watcher() i=4 2024-06-28T15:25:56.158 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.176+0000 7fe7210b4980 20 remove_watcher() i=5 2024-06-28T15:25:56.159 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.180+0000 7fe7210b4980 20 remove_watcher() i=6 2024-06-28T15:25:56.161 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.180+0000 7fe7210b4980 20 remove_watcher() i=7 2024-06-28T15:25:56.174 INFO:tasks.workunit.client.0.smithi178.stderr:rados -p default.rgw.buckets.index setomapval .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.0 --omap-key-file /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_key < /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_val 2024-06-28T15:25:56.200 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:56.200 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:56.218 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket a-bucket 2024-06-28T15:25:56.243 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:56.243 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:56.284 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.304+0000 7f88c03d1980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.304+0000 7f88c03d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.304+0000 7f88c03d1980 20 realm 2024-06-28T15:25:56.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.304+0000 7f88c03d1980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.304+0000 7f88c03d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.304+0000 7f88c03d1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:56.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.304+0000 7f88c03d1980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.286 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.304+0000 7f88c03d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.287 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.304+0000 7f88c03d1980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.308+0000 7f88c03d1980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:56.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.308+0000 7f88c03d1980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.288 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.308+0000 7f88c03d1980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:56.289 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.308+0000 7f88c03d1980 20 searching for the correct realm 2024-06-28T15:25:56.294 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.312+0000 7f88be6e3640 0 -- 172.21.15.178:0/226394567 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x555bf1f67d20 msgr2=0x555bf1f6a150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:56.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.320+0000 7f88bdee2640 0 -- 172.21.15.178:0/226394567 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f88b406b040 msgr2=0x7f88b4061310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:56.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.324+0000 7f88c03d1980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:56.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.324+0000 7f88c03d1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:56.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.324+0000 7f88c03d1980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:56.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.324+0000 7f88c03d1980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:56.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.324+0000 7f88c03d1980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:56.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.324+0000 7f88c03d1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:56.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.324+0000 7f88c03d1980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.324+0000 7f88c03d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.307 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.324+0000 7f88c03d1980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:56.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.308 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:56.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 20 zone default found 2024-06-28T15:25:56.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 4 Realm: () 2024-06-28T15:25:56.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:56.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:56.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:56.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 20 zonegroup default 2024-06-28T15:25:56.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.309 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 20 started sync module instance, tier type = 2024-06-28T15:25:56.310 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.328+0000 7f88c03d1980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:56.324 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.344+0000 7f88c03d1980 20 add_watcher() i=0 2024-06-28T15:25:56.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.344+0000 7f88c03d1980 20 add_watcher() i=1 2024-06-28T15:25:56.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.344+0000 7f88c03d1980 20 add_watcher() i=2 2024-06-28T15:25:56.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.344+0000 7f88c03d1980 20 add_watcher() i=3 2024-06-28T15:25:56.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.344+0000 7f88c03d1980 20 add_watcher() i=4 2024-06-28T15:25:56.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.344+0000 7f88c03d1980 20 add_watcher() i=5 2024-06-28T15:25:56.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.344+0000 7f88c03d1980 20 add_watcher() i=6 2024-06-28T15:25:56.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.344+0000 7f88c03d1980 20 add_watcher() i=7 2024-06-28T15:25:56.326 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.344+0000 7f88c03d1980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:56.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f876f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:56.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f88c03d1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:56.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f88c03d1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:56.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f88c03d1980 5 note: GC not initialized 2024-06-28T15:25:56.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f876e7fc640 20 reqs_thread_entry: start 2024-06-28T15:25:56.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f88c03d1980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:56.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f88c03d1980 20 Filter name: none 2024-06-28T15:25:56.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f86dffff640 20 reqs_thread_entry: start 2024-06-28T15:25:56.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f88c03d1980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f88c03d1980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:25:56.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f88c03d1980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f88c03d1980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:25:56.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f88c03d1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:56.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f88c03d1980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-06-28T15:25:56.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f88c03d1980 20 INFO: main: starting shard=0 2024-06-28T15:25:56.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f88c03d1980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.0 2024-06-28T15:25:56.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.348+0000 7f88c03d1980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.0 2024-06-28T15:25:56.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.352+0000 7f88c03d1980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-06-28T15:25:56.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.352+0000 7f88c03d1980 20 INFO: main: starting shard=1 2024-06-28T15:25:56.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.352+0000 7f88c03d1980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.1 2024-06-28T15:25:56.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.352+0000 7f88c03d1980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.1 2024-06-28T15:25:56.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.352+0000 7f88c03d1980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-06-28T15:25:56.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.352+0000 7f88c03d1980 20 INFO: main: starting shard=2 2024-06-28T15:25:56.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.352+0000 7f88c03d1980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.2 2024-06-28T15:25:56.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.352+0000 7f88c03d1980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.2 2024-06-28T15:25:56.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.352+0000 7f88c03d1980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-06-28T15:25:56.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.352+0000 7f88c03d1980 20 INFO: main: done 2024-06-28T15:25:56.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.352+0000 7f876f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:56.339 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.360+0000 7f88c03d1980 20 remove_watcher() i=0 2024-06-28T15:25:56.340 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.360+0000 7f88c03d1980 2 removed watcher, disabling cache 2024-06-28T15:25:56.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.360+0000 7f88c03d1980 20 remove_watcher() i=1 2024-06-28T15:25:56.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.364+0000 7f88c03d1980 20 remove_watcher() i=2 2024-06-28T15:25:56.345 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.364+0000 7f88c03d1980 20 remove_watcher() i=3 2024-06-28T15:25:56.347 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.368+0000 7f88c03d1980 20 remove_watcher() i=4 2024-06-28T15:25:56.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.368+0000 7f88c03d1980 20 remove_watcher() i=5 2024-06-28T15:25:56.349 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.368+0000 7f88c03d1980 20 remove_watcher() i=6 2024-06-28T15:25:56.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.368+0000 7f88c03d1980 20 remove_watcher() i=7 2024-06-28T15:25:56.364 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket a-bucket --num-shards 1 --yes-i-really-mean-it 2024-06-28T15:25:56.393 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:56.393 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:56.434 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.452+0000 7eff192af980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.452+0000 7eff192af980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.452+0000 7eff192af980 20 realm 2024-06-28T15:25:56.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.452+0000 7eff192af980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.456+0000 7eff192af980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.456+0000 7eff192af980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:56.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.456+0000 7eff192af980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.456+0000 7eff192af980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.456+0000 7eff192af980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.456+0000 7eff192af980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:56.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.456+0000 7eff192af980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.456+0000 7eff192af980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:56.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.456+0000 7eff192af980 20 searching for the correct realm 2024-06-28T15:25:56.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.464+0000 7eff165bf640 0 -- 172.21.15.178:0/3448808116 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x5605656de8f0 msgr2=0x5605656e0d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:56.449 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.468+0000 7eff165bf640 0 -- 172.21.15.178:0/3448808116 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7eff08033040 msgr2=0x7eff08041a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:56.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:56.457 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:56.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:56.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:56.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:56.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:56.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.459 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:56.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:56.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 zone default found 2024-06-28T15:25:56.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 4 Realm: () 2024-06-28T15:25:56.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:56.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:56.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:56.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 zonegroup default 2024-06-28T15:25:56.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.476+0000 7eff192af980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.480+0000 7eff192af980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.480+0000 7eff192af980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.480+0000 7eff192af980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.480+0000 7eff192af980 20 started sync module instance, tier type = 2024-06-28T15:25:56.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.480+0000 7eff192af980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:56.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.492+0000 7eff192af980 20 add_watcher() i=0 2024-06-28T15:25:56.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.492+0000 7eff192af980 20 add_watcher() i=1 2024-06-28T15:25:56.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.492+0000 7eff192af980 20 add_watcher() i=2 2024-06-28T15:25:56.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.492+0000 7eff192af980 20 add_watcher() i=3 2024-06-28T15:25:56.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.492+0000 7eff192af980 20 add_watcher() i=4 2024-06-28T15:25:56.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.492+0000 7eff192af980 20 add_watcher() i=5 2024-06-28T15:25:56.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.492+0000 7eff192af980 20 add_watcher() i=6 2024-06-28T15:25:56.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.496+0000 7eff192af980 20 add_watcher() i=7 2024-06-28T15:25:56.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.496+0000 7eff192af980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:56.478 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.496+0000 7efdcd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:56.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.496+0000 7eff192af980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:56.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.496+0000 7eff192af980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:56.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.496+0000 7eff192af980 5 note: GC not initialized 2024-06-28T15:25:56.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7efd47fff640 20 reqs_thread_entry: start 2024-06-28T15:25:56.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:56.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 20 Filter name: none 2024-06-28T15:25:56.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7efd45ffb640 20 reqs_thread_entry: start 2024-06-28T15:25:56.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-06-28T15:25:56.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:25:56.481 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-06-28T15:25:56.481 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-06-28T15:25:56.481 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 : miss 2024-06-28T15:25:56.481 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.481 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:25:56.481 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:25:56.481 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:25:56.481 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:25:56.482 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:25:56.482 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.500+0000 7eff192af980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 2024-06-28T15:25:56.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.504+0000 7eff192af980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:25:56.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.504+0000 7eff192af980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:25:56.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.504+0000 7eff192af980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:25:56.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.504+0000 7eff192af980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18, ofs0, ns] 2024-06-28T15:25:56.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.508+0000 7efdf9ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595122 cookie 94581176583696 notifier 7314 bl.length()=903 2024-06-28T15:25:56.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.508+0000 7efdf9ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:25:56.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.508+0000 7efdf9ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:25:56.487 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.508+0000 7efdf9ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:25:56.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.508+0000 7eff192af980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): orig_sources= new_sources= 2024-06-28T15:25:56.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.508+0000 7eff192af980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): potential sources added=[] removed=[] 2024-06-28T15:25:56.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.508+0000 7eff192af980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): orig_dests= new_dests= 2024-06-28T15:25:56.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.508+0000 7eff192af980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): potential dests added=[] removed=[] 2024-06-28T15:25:56.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.512+0000 7eff192af980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.3.0 2024-06-28T15:25:56.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.512+0000 7eff192af980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.3.0 2024-06-28T15:25:56.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.512+0000 7eff192af980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.0 2024-06-28T15:25:56.493 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.512+0000 7eff192af980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.0 2024-06-28T15:25:56.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.512+0000 7eff192af980 10 Dropping entry with empty name, idx=�1001_04/57/0457f727ec113e418d5b16d206b200ed068c0533554883ce811df7c932a3df68/2018_12_11/2889999/3386469/metadata.gz 2024-06-28T15:25:56.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.512+0000 7eff192af980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.1 2024-06-28T15:25:56.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.512+0000 7eff192af980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.1 2024-06-28T15:25:56.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.516+0000 7eff192af980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.2 2024-06-28T15:25:56.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.516+0000 7eff192af980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.2.2 2024-06-28T15:25:56.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.516+0000 7eff192af980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 : hit (requested=0x17, cached=0x17) 2024-06-28T15:25:56.496 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.516+0000 7eff192af980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 2024-06-28T15:25:56.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.516+0000 7eff192af980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:25:56.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.516+0000 7eff192af980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:25:56.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.516+0000 7eff192af980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:25:56.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.516+0000 7eff192af980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18, ofs0, ns] 2024-06-28T15:25:56.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.516+0000 7efdf9ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595123 cookie 94581176583696 notifier 7314 bl.length()=871 2024-06-28T15:25:56.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.516+0000 7efdf9ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 info.flags=0x17 2024-06-28T15:25:56.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.516+0000 7efdf9ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:843e24ce-085a-4043-8f55-38e05370e790.4686.18 to cache LRU end 2024-06-28T15:25:56.499 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.516+0000 7efdf9ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:25:56.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.520+0000 7eff192af980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): orig_sources= new_sources= 2024-06-28T15:25:56.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.520+0000 7eff192af980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): potential sources added=[] removed=[] 2024-06-28T15:25:56.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.520+0000 7eff192af980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): orig_dests= new_dests= 2024-06-28T15:25:56.501 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.520+0000 7eff192af980 20 handle_bi_update(): bucket=:a-bucket[843e24ce-085a-4043-8f55-38e05370e790.4686.18]): potential dests added=[] removed=[] 2024-06-28T15:25:56.504 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.524+0000 7eff192af980 1 execute INFO: reshard of bucket "a-bucket" from 1 shards to 1 shards completed successfully 2024-06-28T15:25:56.506 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.524+0000 7efdcd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:56.511 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.532+0000 7eff192af980 20 remove_watcher() i=0 2024-06-28T15:25:56.511 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.532+0000 7eff192af980 2 removed watcher, disabling cache 2024-06-28T15:25:56.513 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.532+0000 7eff192af980 20 remove_watcher() i=1 2024-06-28T15:25:56.515 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.536+0000 7eff192af980 20 remove_watcher() i=2 2024-06-28T15:25:56.517 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.536+0000 7eff192af980 20 remove_watcher() i=3 2024-06-28T15:25:56.519 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.540+0000 7eff192af980 20 remove_watcher() i=4 2024-06-28T15:25:56.521 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.540+0000 7eff192af980 20 remove_watcher() i=5 2024-06-28T15:25:56.522 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.540+0000 7eff192af980 20 remove_watcher() i=6 2024-06-28T15:25:56.523 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.544+0000 7eff192af980 20 remove_watcher() i=7 2024-06-28T15:25:56.537 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket a-bucket 2024-06-28T15:25:56.563 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:56.563 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:56.605 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.624+0000 7ff675bfb980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.628+0000 7ff675bfb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.628+0000 7ff675bfb980 20 realm 2024-06-28T15:25:56.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.628+0000 7ff675bfb980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.628+0000 7ff675bfb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.628+0000 7ff675bfb980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:56.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.628+0000 7ff675bfb980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.628+0000 7ff675bfb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.608 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.628+0000 7ff675bfb980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.609 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.628+0000 7ff675bfb980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:56.610 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.628+0000 7ff675bfb980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.610 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.628+0000 7ff675bfb980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:56.610 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.628+0000 7ff675bfb980 20 searching for the correct realm 2024-06-28T15:25:56.616 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.636+0000 7ff672f0b640 0 -- 172.21.15.178:0/3342399964 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x55c056640ca0 msgr2=0x55c0566430c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:56.622 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.640+0000 7ff672f0b640 0 -- 172.21.15.178:0/3342399964 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7ff664035070 msgr2=0x7ff664041410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:56.630 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.648+0000 7ff675bfb980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:56.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.648+0000 7ff675bfb980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:56.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.648+0000 7ff675bfb980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:56.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.648+0000 7ff675bfb980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:56.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.648+0000 7ff675bfb980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:56.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.648+0000 7ff675bfb980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:56.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.648+0000 7ff675bfb980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.648+0000 7ff675bfb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.631 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.648+0000 7ff675bfb980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.648+0000 7ff675bfb980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:56.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.648+0000 7ff675bfb980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:56.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 20 zone default found 2024-06-28T15:25:56.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 4 Realm: () 2024-06-28T15:25:56.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:56.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:56.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:56.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 20 zonegroup default 2024-06-28T15:25:56.632 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.633 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 20 started sync module instance, tier type = 2024-06-28T15:25:56.634 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.652+0000 7ff675bfb980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:56.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.668+0000 7ff675bfb980 20 add_watcher() i=0 2024-06-28T15:25:56.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.668+0000 7ff675bfb980 20 add_watcher() i=1 2024-06-28T15:25:56.649 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.668+0000 7ff675bfb980 20 add_watcher() i=2 2024-06-28T15:25:56.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.668+0000 7ff675bfb980 20 add_watcher() i=3 2024-06-28T15:25:56.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.668+0000 7ff675bfb980 20 add_watcher() i=4 2024-06-28T15:25:56.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.668+0000 7ff675bfb980 20 add_watcher() i=5 2024-06-28T15:25:56.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.668+0000 7ff675bfb980 20 add_watcher() i=6 2024-06-28T15:25:56.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.668+0000 7ff675bfb980 20 add_watcher() i=7 2024-06-28T15:25:56.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.668+0000 7ff675bfb980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:56.652 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff529fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:56.653 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff675bfb980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:56.653 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff675bfb980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:56.654 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff675bfb980 5 note: GC not initialized 2024-06-28T15:25:56.654 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff528fe1640 20 reqs_thread_entry: start 2024-06-28T15:25:56.654 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff675bfb980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:56.654 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff675bfb980 20 Filter name: none 2024-06-28T15:25:56.654 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff4a27fc640 20 reqs_thread_entry: start 2024-06-28T15:25:56.654 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff675bfb980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff675bfb980 20 rados_obj.operate() r=0 bl.length=200 2024-06-28T15:25:56.655 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff675bfb980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff675bfb980 20 rados_obj.operate() r=0 bl.length=414 2024-06-28T15:25:56.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff675bfb980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:56.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff675bfb980 20 INFO: main: max_entries=1000, index=gen=3, max_shards=1 2024-06-28T15:25:56.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff675bfb980 20 INFO: main: starting shard=0 2024-06-28T15:25:56.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff675bfb980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.3.0 2024-06-28T15:25:56.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.672+0000 7ff675bfb980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.18.3.0 2024-06-28T15:25:56.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.676+0000 7ff675bfb980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-06-28T15:25:56.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.676+0000 7ff675bfb980 20 INFO: main: done 2024-06-28T15:25:56.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.676+0000 7ff529fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:56.662 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.680+0000 7ff675bfb980 20 remove_watcher() i=0 2024-06-28T15:25:56.662 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.680+0000 7ff675bfb980 2 removed watcher, disabling cache 2024-06-28T15:25:56.663 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.680+0000 7ff675bfb980 20 remove_watcher() i=1 2024-06-28T15:25:56.665 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.684+0000 7ff675bfb980 20 remove_watcher() i=2 2024-06-28T15:25:56.667 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.684+0000 7ff675bfb980 20 remove_watcher() i=3 2024-06-28T15:25:56.669 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.688+0000 7ff675bfb980 20 remove_watcher() i=4 2024-06-28T15:25:56.670 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.688+0000 7ff675bfb980 20 remove_watcher() i=5 2024-06-28T15:25:56.671 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.692+0000 7ff675bfb980 20 remove_watcher() i=6 2024-06-28T15:25:56.672 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.692+0000 7ff675bfb980 20 remove_watcher() i=7 2024-06-28T15:25:56.686 INFO:tasks.workunit.client.0.smithi178.stderr: test: PUT succeeds during reshard 2024-06-28T15:25:56.686 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket myver 2024-06-28T15:25:56.712 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:56.713 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:56.756 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.776+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.776+0000 7f1e4b67e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.776+0000 7f1e4b67e980 20 realm 2024-06-28T15:25:56.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.776+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.776+0000 7f1e4b67e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.776+0000 7f1e4b67e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:56.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.776+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.776+0000 7f1e4b67e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.758 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.776+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.780+0000 7f1e4b67e980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:56.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.780+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.760 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.780+0000 7f1e4b67e980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:56.761 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.780+0000 7f1e4b67e980 20 searching for the correct realm 2024-06-28T15:25:56.766 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.784+0000 7f1e49990640 0 -- 172.21.15.178:0/583944801 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x55b723de1790 msgr2=0x55b723de3c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:25:56.772 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.792+0000 7f1e4898e640 0 -- 172.21.15.178:0/583944801 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f1e3c0ed060 msgr2=0x7f1e3c0c9960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:56.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:56.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:56.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:56.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:56.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:56.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:56.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.780 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:56.781 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:56.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 zone default found 2024-06-28T15:25:56.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 4 Realm: () 2024-06-28T15:25:56.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:56.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:56.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:56.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 zonegroup default 2024-06-28T15:25:56.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.782 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:56.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 started sync module instance, tier type = 2024-06-28T15:25:56.783 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.800+0000 7f1e4b67e980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:56.798 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.816+0000 7f1e4b67e980 20 add_watcher() i=0 2024-06-28T15:25:56.798 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.816+0000 7f1e4b67e980 20 add_watcher() i=1 2024-06-28T15:25:56.798 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.816+0000 7f1e4b67e980 20 add_watcher() i=2 2024-06-28T15:25:56.798 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.816+0000 7f1e4b67e980 20 add_watcher() i=3 2024-06-28T15:25:56.798 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.816+0000 7f1e4b67e980 20 add_watcher() i=4 2024-06-28T15:25:56.798 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.816+0000 7f1e4b67e980 20 add_watcher() i=5 2024-06-28T15:25:56.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.816+0000 7f1e4b67e980 20 add_watcher() i=6 2024-06-28T15:25:56.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.816+0000 7f1e4b67e980 20 add_watcher() i=7 2024-06-28T15:25:56.799 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.816+0000 7f1e4b67e980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:56.801 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.820+0000 7f1cff7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:56.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.820+0000 7f1e4b67e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:56.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.820+0000 7f1e4b67e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:56.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.820+0000 7f1e4b67e980 5 note: GC not initialized 2024-06-28T15:25:56.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.820+0000 7f1cfe7e4640 20 reqs_thread_entry: start 2024-06-28T15:25:56.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.820+0000 7f1e4b67e980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:56.802 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.820+0000 7f1e4b67e980 20 Filter name: none 2024-06-28T15:25:56.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.820+0000 7f1c6ffff640 20 reqs_thread_entry: start 2024-06-28T15:25:56.803 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.820+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.820+0000 7f1e4b67e980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:25:56.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.820+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.820+0000 7f1e4b67e980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:25:56.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.820+0000 7f1e4b67e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:56.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.820+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.824+0000 7f1e4b67e980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:25:56.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.824+0000 7f1e4b67e980 20 rados->read ofs=0 len=0 2024-06-28T15:25:56.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.824+0000 7f1e4b67e980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:25:56.804 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.824+0000 7f1e4b67e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:25:56.805 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.824+0000 7f1cff7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:56.810 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.828+0000 7f1e4b67e980 20 remove_watcher() i=0 2024-06-28T15:25:56.810 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.828+0000 7f1e4b67e980 2 removed watcher, disabling cache 2024-06-28T15:25:56.812 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.832+0000 7f1e4b67e980 20 remove_watcher() i=1 2024-06-28T15:25:56.814 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.832+0000 7f1e4b67e980 20 remove_watcher() i=2 2024-06-28T15:25:56.815 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.836+0000 7f1e4b67e980 20 remove_watcher() i=3 2024-06-28T15:25:56.817 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.836+0000 7f1e4b67e980 20 remove_watcher() i=4 2024-06-28T15:25:56.819 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.836+0000 7f1e4b67e980 20 remove_watcher() i=5 2024-06-28T15:25:56.820 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.840+0000 7f1e4b67e980 20 remove_watcher() i=6 2024-06-28T15:25:56.821 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:56.840+0000 7f1e4b67e980 20 remove_watcher() i=7 2024-06-28T15:25:56.835 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:56.835 INFO:tasks.workunit.client.0.smithi178.stderr:BUCKET_STATS: 2024-06-28T15:25:56.835 INFO:tasks.workunit.client.0.smithi178.stderr:bucket: myver id: 843e24ce-085a-4043-8f55-38e05370e790.4686.19 num_objects: 0 size_kb: 0 num_shards: 2 2024-06-28T15:25:56.835 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:25:56.835 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin --inject-delay-at=do_reshard --inject-delay-ms=5000 bucket reshard --bucket myver --num-shards 3 2024-06-28T15:25:56.865 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:25:56.865 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:25:57.116 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.136+0000 7f6b43950980 20 rados->read ofs=0 len=0 2024-06-28T15:25:57.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.136+0000 7f6b43950980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:57.117 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.136+0000 7f6b43950980 20 realm 2024-06-28T15:25:57.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.136+0000 7f6b43950980 20 rados->read ofs=0 len=0 2024-06-28T15:25:57.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.136+0000 7f6b43950980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:57.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.136+0000 7f6b43950980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:25:57.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.136+0000 7f6b43950980 20 rados->read ofs=0 len=0 2024-06-28T15:25:57.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.136+0000 7f6b43950980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:57.118 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.136+0000 7f6b43950980 20 rados->read ofs=0 len=0 2024-06-28T15:25:57.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.140+0000 7f6b43950980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:57.119 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.140+0000 7f6b43950980 20 rados->read ofs=0 len=0 2024-06-28T15:25:57.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.140+0000 7f6b43950980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:25:57.120 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.140+0000 7f6b43950980 20 searching for the correct realm 2024-06-28T15:25:57.125 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.144+0000 7f6b41c62640 0 -- 172.21.15.178:0/58503338 >> [v2:172.21.15.195:6824/1278553753,v1:172.21.15.195:6825/1278553753] conn(0x7f6b38032060 msgr2=0x7f6b38061720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:57.133 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.152+0000 7f6b41461640 0 -- 172.21.15.178:0/58503338 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f6b38060e20 msgr2=0x7f6b38095700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:25:57.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.368+0000 7f6b43950980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:25:57.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.368+0000 7f6b43950980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:25:57.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.368+0000 7f6b43950980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:25:57.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.368+0000 7f6b43950980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:25:57.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.368+0000 7f6b43950980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:25:57.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.368+0000 7f6b43950980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:25:57.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.368+0000 7f6b43950980 20 rados->read ofs=0 len=0 2024-06-28T15:25:57.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.368+0000 7f6b43950980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:57.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.368+0000 7f6b43950980 20 rados->read ofs=0 len=0 2024-06-28T15:25:57.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:25:57.351 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 20 rados->read ofs=0 len=0 2024-06-28T15:25:57.352 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:25:57.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 20 zone default found 2024-06-28T15:25:57.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 4 Realm: () 2024-06-28T15:25:57.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:25:57.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:25:57.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:25:57.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 20 zonegroup default 2024-06-28T15:25:57.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 20 rados->read ofs=0 len=0 2024-06-28T15:25:57.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:57.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 20 rados->read ofs=0 len=0 2024-06-28T15:25:57.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:57.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 20 rados->read ofs=0 len=0 2024-06-28T15:25:57.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:25:57.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 20 started sync module instance, tier type = 2024-06-28T15:25:57.355 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.372+0000 7f6b43950980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:25:57.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.396+0000 7f6b43950980 20 add_watcher() i=0 2024-06-28T15:25:57.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.396+0000 7f6b43950980 20 add_watcher() i=1 2024-06-28T15:25:57.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.396+0000 7f6b43950980 20 add_watcher() i=2 2024-06-28T15:25:57.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.396+0000 7f6b43950980 20 add_watcher() i=3 2024-06-28T15:25:57.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.396+0000 7f6b43950980 20 add_watcher() i=4 2024-06-28T15:25:57.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.396+0000 7f6b43950980 20 add_watcher() i=5 2024-06-28T15:25:57.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.396+0000 7f6b43950980 20 add_watcher() i=6 2024-06-28T15:25:57.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.400+0000 7f6b43950980 20 add_watcher() i=7 2024-06-28T15:25:57.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.400+0000 7f6b43950980 2 all 8 watchers are set, enabling cache 2024-06-28T15:25:57.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.404+0000 7f69f77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:25:57.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.404+0000 7f6b43950980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:25:57.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.404+0000 7f6b43950980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:25:57.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.404+0000 7f6b43950980 5 note: GC not initialized 2024-06-28T15:25:57.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.404+0000 7f69f67e4640 20 reqs_thread_entry: start 2024-06-28T15:25:57.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.404+0000 7f6b43950980 20 init_complete bucket index max shards: 1 2024-06-28T15:25:57.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.404+0000 7f6b43950980 20 Filter name: none 2024-06-28T15:25:57.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.404+0000 7f6967fff640 20 reqs_thread_entry: start 2024-06-28T15:25:57.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.404+0000 7f6b43950980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-06-28T15:25:57.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.404+0000 7f6b43950980 20 rados->read ofs=0 len=0 2024-06-28T15:25:57.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.404+0000 7f6b43950980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:25:57.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.404+0000 7f6b43950980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-06-28T15:25:57.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.404+0000 7f6b43950980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-06-28T15:25:57.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.404+0000 7f6b43950980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 : miss 2024-06-28T15:25:57.388 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.408+0000 7f6b43950980 20 rados->read ofs=0 len=0 2024-06-28T15:25:57.388 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.408+0000 7f6b43950980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:25:57.388 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.408+0000 7f6b43950980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:25:57.388 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.408+0000 7f6b43950980 10 adding default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:25:57.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.408+0000 7f6b43950980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:25:57.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.408+0000 7f6b43950980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:25:57.389 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.408+0000 7f6b43950980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 2024-06-28T15:25:57.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.412+0000 7f6b43950980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:25:57.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.412+0000 7f6b43950980 10 moving default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:25:57.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.412+0000 7f6b43950980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:25:57.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.412+0000 7f6b43950980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19, ofs0, ns] 2024-06-28T15:25:57.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.412+0000 7f6a24ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595119 cookie 94017538608784 notifier 6893 bl.length()=897 2024-06-28T15:25:57.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.412+0000 7f6a24ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:25:57.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.412+0000 7f6a24ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:25:57.395 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.412+0000 7f6a24ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:25:57.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.420+0000 7f6b43950980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): orig_sources= new_sources= 2024-06-28T15:25:57.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.420+0000 7f6b43950980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): potential sources added=[] removed=[] 2024-06-28T15:25:57.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.420+0000 7f6b43950980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): orig_dests= new_dests= 2024-06-28T15:25:57.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.420+0000 7f6b43950980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): potential dests added=[] removed=[] 2024-06-28T15:25:57.404 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:25:57.424+0000 7f6b43950980 -1 Injecting delay=5.000000000s at location=do_reshard 2024-06-28T15:26:02.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.424+0000 7f6b43950980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.0 2024-06-28T15:26:02.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.424+0000 7f6b43950980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.0 2024-06-28T15:26:02.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.424+0000 7f6b43950980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.1 2024-06-28T15:26:02.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.424+0000 7f6b43950980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.1 2024-06-28T15:26:02.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.424+0000 7f6b43950980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.2 2024-06-28T15:26:02.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.424+0000 7f6b43950980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.2 2024-06-28T15:26:02.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.424+0000 7f6b43950980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.1.0 2024-06-28T15:26:02.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.424+0000 7f6b43950980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.1.0 2024-06-28T15:26:02.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.424+0000 7f6b43950980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.1.1 2024-06-28T15:26:02.406 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.424+0000 7f6b43950980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.1.1 2024-06-28T15:26:02.407 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.424+0000 7f6b43950980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 : hit (requested=0x17, cached=0x17) 2024-06-28T15:26:02.407 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.424+0000 7f6b43950980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 2024-06-28T15:26:02.411 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.428+0000 7f6b43950980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:26:02.411 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.428+0000 7f6b43950980 10 moving default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:26:02.411 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.428+0000 7f6b43950980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:26:02.411 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.428+0000 7f6b43950980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19, ofs0, ns] 2024-06-28T15:26:02.412 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.432+0000 7f6a24ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595120 cookie 94017538608784 notifier 6893 bl.length()=865 2024-06-28T15:26:02.412 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.432+0000 7f6a24ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:26:02.412 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.432+0000 7f6a24ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:26:02.412 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.432+0000 7f6a24ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:26:02.416 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.436+0000 7f6b43950980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): orig_sources= new_sources= 2024-06-28T15:26:02.416 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.436+0000 7f6b43950980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): potential sources added=[] removed=[] 2024-06-28T15:26:02.416 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.436+0000 7f6b43950980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): orig_dests= new_dests= 2024-06-28T15:26:02.416 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.436+0000 7f6b43950980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): potential dests added=[] removed=[] 2024-06-28T15:26:02.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.440+0000 7f6b43950980 1 execute INFO: reshard of bucket "myver" from 3 shards to 3 shards completed successfully 2024-06-28T15:26:02.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.444+0000 7f69f77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:26:02.430 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.448+0000 7f6b43950980 20 remove_watcher() i=0 2024-06-28T15:26:02.430 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.448+0000 7f6b43950980 2 removed watcher, disabling cache 2024-06-28T15:26:02.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.452+0000 7f6b43950980 20 remove_watcher() i=1 2024-06-28T15:26:02.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.456+0000 7f6b43950980 20 remove_watcher() i=2 2024-06-28T15:26:02.438 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.456+0000 7f6b43950980 20 remove_watcher() i=3 2024-06-28T15:26:02.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.460+0000 7f6b43950980 20 remove_watcher() i=4 2024-06-28T15:26:02.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.460+0000 7f6b43950980 20 remove_watcher() i=5 2024-06-28T15:26:02.445 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.464+0000 7f6b43950980 20 remove_watcher() i=6 2024-06-28T15:26:02.447 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:02.468+0000 7f6b43950980 20 remove_watcher() i=7 2024-06-28T15:26:03.053 INFO:tasks.workunit.client.0.smithi178.stderr:put object successful 2024-06-28T15:26:03.053 INFO:tasks.workunit.client.0.smithi178.stderr:TEST: check that bucket stats are correct after reshard with unlinked entries 2024-06-28T15:26:03.053 INFO:tasks.workunit.client.0.smithi178.stderr: 2024-06-28T15:26:03.970 INFO:tasks.workunit.client.0.smithi178.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-06-28T15:26:04.408 INFO:tasks.workunit.client.0.smithi178.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-06-28T15:26:05.984 INFO:tasks.workunit.client.0.smithi178.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-06-28T15:26:05.985 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket myver --object x 2024-06-28T15:26:06.021 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:26:06.021 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:26:06.065 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.084+0000 7fe91b010980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.088+0000 7fe91b010980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:06.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.088+0000 7fe91b010980 20 realm 2024-06-28T15:26:06.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.088+0000 7fe91b010980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.067 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.088+0000 7fe91b010980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:06.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.088+0000 7fe91b010980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:26:06.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.088+0000 7fe91b010980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.088+0000 7fe91b010980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:06.068 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.088+0000 7fe91b010980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.088+0000 7fe91b010980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:26:06.069 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.088+0000 7fe91b010980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.088+0000 7fe91b010980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:26:06.070 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.088+0000 7fe91b010980 20 searching for the correct realm 2024-06-28T15:26:06.075 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.092+0000 7fe919322640 0 -- 172.21.15.178:0/1742753824 >> [v2:172.21.15.195:6824/1278553753,v1:172.21.15.195:6825/1278553753] conn(0x7fe908029070 msgr2=0x7fe9080356f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:26:06.081 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.100+0000 7fe918b21640 0 -- 172.21.15.178:0/1742753824 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fe9080935a0 msgr2=0x7fe908093a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:26:06.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.316+0000 7fe91b010980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:26:06.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.316+0000 7fe91b010980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:26:06.298 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.316+0000 7fe91b010980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:26:06.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.316+0000 7fe91b010980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:26:06.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.316+0000 7fe91b010980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:26:06.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.316+0000 7fe91b010980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:26:06.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.316+0000 7fe91b010980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.316+0000 7fe91b010980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:06.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.316+0000 7fe91b010980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.299 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:26:06.300 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:26:06.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 20 zone default found 2024-06-28T15:26:06.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 4 Realm: () 2024-06-28T15:26:06.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:26:06.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:26:06.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:26:06.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 20 zonegroup default 2024-06-28T15:26:06.301 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:06.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:06.302 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:06.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 20 started sync module instance, tier type = 2024-06-28T15:26:06.303 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.320+0000 7fe91b010980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:26:06.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.344+0000 7fe91b010980 20 add_watcher() i=0 2024-06-28T15:26:06.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.344+0000 7fe91b010980 20 add_watcher() i=1 2024-06-28T15:26:06.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.344+0000 7fe91b010980 20 add_watcher() i=2 2024-06-28T15:26:06.325 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.344+0000 7fe91b010980 20 add_watcher() i=3 2024-06-28T15:26:06.326 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.344+0000 7fe91b010980 20 add_watcher() i=4 2024-06-28T15:26:06.326 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.344+0000 7fe91b010980 20 add_watcher() i=5 2024-06-28T15:26:06.326 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.344+0000 7fe91b010980 20 add_watcher() i=6 2024-06-28T15:26:06.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.344+0000 7fe91b010980 20 add_watcher() i=7 2024-06-28T15:26:06.327 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.344+0000 7fe91b010980 2 all 8 watchers are set, enabling cache 2024-06-28T15:26:06.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.348+0000 7fe7cefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:26:06.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.348+0000 7fe91b010980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:26:06.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.348+0000 7fe91b010980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:26:06.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.348+0000 7fe91b010980 5 note: GC not initialized 2024-06-28T15:26:06.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.348+0000 7fe7cdfe3640 20 reqs_thread_entry: start 2024-06-28T15:26:06.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.348+0000 7fe91b010980 20 init_complete bucket index max shards: 1 2024-06-28T15:26:06.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 Filter name: none 2024-06-28T15:26:06.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe73f7fe640 20 reqs_thread_entry: start 2024-06-28T15:26:06.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:26:06.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:26:06.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:26:06.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-06-28T15:26:06.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 INFO: main: starting shard=0 2024-06-28T15:26:06.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.0 2024-06-28T15:26:06.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.0 2024-06-28T15:26:06.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-06-28T15:26:06.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 INFO: main: starting shard=1 2024-06-28T15:26:06.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.1 2024-06-28T15:26:06.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.1 2024-06-28T15:26:06.334 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-06-28T15:26:06.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 INFO: main: starting shard=2 2024-06-28T15:26:06.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.2 2024-06-28T15:26:06.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.352+0000 7fe91b010980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.2 2024-06-28T15:26:06.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.356+0000 7fe91b010980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-06-28T15:26:06.335 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.356+0000 7fe91b010980 20 INFO: main: done 2024-06-28T15:26:06.336 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.356+0000 7fe7cefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:26:06.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.360+0000 7fe91b010980 20 remove_watcher() i=0 2024-06-28T15:26:06.341 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.360+0000 7fe91b010980 2 removed watcher, disabling cache 2024-06-28T15:26:06.343 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.364+0000 7fe91b010980 20 remove_watcher() i=1 2024-06-28T15:26:06.346 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.364+0000 7fe91b010980 20 remove_watcher() i=2 2024-06-28T15:26:06.348 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.368+0000 7fe91b010980 20 remove_watcher() i=3 2024-06-28T15:26:06.350 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.368+0000 7fe91b010980 20 remove_watcher() i=4 2024-06-28T15:26:06.352 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.372+0000 7fe91b010980 20 remove_watcher() i=5 2024-06-28T15:26:06.353 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.372+0000 7fe91b010980 20 remove_watcher() i=6 2024-06-28T15:26:06.354 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.372+0000 7fe91b010980 20 remove_watcher() i=7 2024-06-28T15:26:06.560 INFO:tasks.workunit.client.0.smithi178.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-06-28T15:26:06.561 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket myver --object y 2024-06-28T15:26:06.596 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:26:06.596 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:26:06.641 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.660+0000 7f56451f7980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.643 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.660+0000 7f56451f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:06.643 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.660+0000 7f56451f7980 20 realm 2024-06-28T15:26:06.643 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.660+0000 7f56451f7980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.643 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.660+0000 7f56451f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:06.643 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.660+0000 7f56451f7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:26:06.643 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.660+0000 7f56451f7980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.643 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.660+0000 7f56451f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:06.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.660+0000 7f56451f7980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.644 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.664+0000 7f56451f7980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:26:06.645 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.664+0000 7f56451f7980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.645 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.664+0000 7f56451f7980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:26:06.645 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.664+0000 7f56451f7980 20 searching for the correct realm 2024-06-28T15:26:06.650 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.668+0000 7f5643509640 0 -- 172.21.15.178:0/949348455 >> [v2:172.21.15.195:6824/1278553753,v1:172.21.15.195:6825/1278553753] conn(0x7f563803f060 msgr2=0x7f5638065e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:26:06.656 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.676+0000 7f5642d08640 0 -- 172.21.15.178:0/949348455 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f5638008040 msgr2=0x7f56380800d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:26:06.874 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.892+0000 7f56451f7980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:26:06.874 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.892+0000 7f56451f7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:26:06.874 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.892+0000 7f56451f7980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:26:06.877 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.892+0000 7f56451f7980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:26:06.877 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.892+0000 7f56451f7980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:26:06.877 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.892+0000 7f56451f7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:26:06.877 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.892+0000 7f56451f7980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.892+0000 7f56451f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:06.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.892+0000 7f56451f7980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:26:06.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:26:06.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 20 zone default found 2024-06-28T15:26:06.879 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 4 Realm: () 2024-06-28T15:26:06.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:26:06.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:26:06.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:26:06.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 20 zonegroup default 2024-06-28T15:26:06.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:06.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:06.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:06.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 20 started sync module instance, tier type = 2024-06-28T15:26:06.880 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.896+0000 7f56451f7980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:26:06.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.920+0000 7f56451f7980 20 add_watcher() i=0 2024-06-28T15:26:06.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.920+0000 7f56451f7980 20 add_watcher() i=1 2024-06-28T15:26:06.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.920+0000 7f56451f7980 20 add_watcher() i=2 2024-06-28T15:26:06.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.920+0000 7f56451f7980 20 add_watcher() i=3 2024-06-28T15:26:06.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.920+0000 7f56451f7980 20 add_watcher() i=4 2024-06-28T15:26:06.902 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.920+0000 7f56451f7980 20 add_watcher() i=5 2024-06-28T15:26:06.903 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.920+0000 7f56451f7980 20 add_watcher() i=6 2024-06-28T15:26:06.904 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.924+0000 7f56451f7980 20 add_watcher() i=7 2024-06-28T15:26:06.904 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.924+0000 7f56451f7980 2 all 8 watchers are set, enabling cache 2024-06-28T15:26:06.908 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.928+0000 7f54f97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:26:06.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.928+0000 7f56451f7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:26:06.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.928+0000 7f56451f7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:26:06.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.928+0000 7f56451f7980 5 note: GC not initialized 2024-06-28T15:26:06.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.928+0000 7f5473fff640 20 reqs_thread_entry: start 2024-06-28T15:26:06.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.928+0000 7f56451f7980 20 init_complete bucket index max shards: 1 2024-06-28T15:26:06.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.928+0000 7f56451f7980 20 Filter name: none 2024-06-28T15:26:06.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.928+0000 7f5471ffb640 20 reqs_thread_entry: start 2024-06-28T15:26:06.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.928+0000 7f56451f7980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:26:06.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 20 rados->read ofs=0 len=0 2024-06-28T15:26:06.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:26:06.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:26:06.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-06-28T15:26:06.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 20 INFO: main: starting shard=0 2024-06-28T15:26:06.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.0 2024-06-28T15:26:06.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.0 2024-06-28T15:26:06.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-06-28T15:26:06.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 20 INFO: main: starting shard=1 2024-06-28T15:26:06.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.1 2024-06-28T15:26:06.913 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.1 2024-06-28T15:26:06.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-06-28T15:26:06.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 20 INFO: main: starting shard=2 2024-06-28T15:26:06.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.2 2024-06-28T15:26:06.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.932+0000 7f56451f7980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.2 2024-06-28T15:26:06.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.936+0000 7f56451f7980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-06-28T15:26:06.915 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.936+0000 7f56451f7980 20 INFO: main: done 2024-06-28T15:26:06.917 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.936+0000 7f54f97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:26:06.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.940+0000 7f56451f7980 20 remove_watcher() i=0 2024-06-28T15:26:06.922 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.940+0000 7f56451f7980 2 removed watcher, disabling cache 2024-06-28T15:26:06.924 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.944+0000 7f56451f7980 20 remove_watcher() i=1 2024-06-28T15:26:06.926 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.944+0000 7f56451f7980 20 remove_watcher() i=2 2024-06-28T15:26:06.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.948+0000 7f56451f7980 20 remove_watcher() i=3 2024-06-28T15:26:06.931 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.948+0000 7f56451f7980 20 remove_watcher() i=4 2024-06-28T15:26:06.932 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.952+0000 7f56451f7980 20 remove_watcher() i=5 2024-06-28T15:26:06.934 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.952+0000 7f56451f7980 20 remove_watcher() i=6 2024-06-28T15:26:06.935 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:06.956+0000 7f56451f7980 20 remove_watcher() i=7 2024-06-28T15:26:07.136 INFO:tasks.workunit.client.0.smithi178.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-06-28T15:26:07.137 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bi list --bucket myver --object z 2024-06-28T15:26:07.172 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:26:07.172 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:26:07.417 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.436+0000 7f9d61253640 0 --2- 172.21.15.178:0/799375519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555ba5938380 0x555ba59284b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-06-28T15:26:07.422 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.440+0000 7f9d63f43980 20 rados->read ofs=0 len=0 2024-06-28T15:26:07.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.444+0000 7f9d63f43980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:07.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.444+0000 7f9d63f43980 20 realm 2024-06-28T15:26:07.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.444+0000 7f9d63f43980 20 rados->read ofs=0 len=0 2024-06-28T15:26:07.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.444+0000 7f9d63f43980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:07.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.444+0000 7f9d63f43980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:26:07.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.444+0000 7f9d63f43980 20 rados->read ofs=0 len=0 2024-06-28T15:26:07.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.444+0000 7f9d63f43980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:07.426 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.444+0000 7f9d63f43980 20 rados->read ofs=0 len=0 2024-06-28T15:26:07.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.448+0000 7f9d63f43980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:26:07.428 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.448+0000 7f9d63f43980 20 rados->read ofs=0 len=0 2024-06-28T15:26:07.429 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.448+0000 7f9d63f43980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:26:07.429 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.448+0000 7f9d63f43980 20 searching for the correct realm 2024-06-28T15:26:07.432 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.452+0000 7f9d61a54640 0 -- 172.21.15.178:0/799375519 >> [v2:172.21.15.178:6823/1880098663,v1:172.21.15.178:6825/1880098663] conn(0x7f9d5401e060 msgr2=0x7f9d54062ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:26:07.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.460+0000 7f9d62255640 0 -- 172.21.15.178:0/799375519 >> [v2:172.21.15.195:6824/1278553753,v1:172.21.15.195:6825/1278553753] conn(0x7f9d58008050 msgr2=0x7f9d58048400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:26:07.451 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:26:07.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:26:07.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:26:07.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:26:07.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:26:07.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:26:07.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 rados->read ofs=0 len=0 2024-06-28T15:26:07.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:07.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 rados->read ofs=0 len=0 2024-06-28T15:26:07.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:26:07.452 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 rados->read ofs=0 len=0 2024-06-28T15:26:07.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:26:07.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 zone default found 2024-06-28T15:26:07.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 4 Realm: () 2024-06-28T15:26:07.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:26:07.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:26:07.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:26:07.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 zonegroup default 2024-06-28T15:26:07.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 rados->read ofs=0 len=0 2024-06-28T15:26:07.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:07.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 rados->read ofs=0 len=0 2024-06-28T15:26:07.454 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:07.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 rados->read ofs=0 len=0 2024-06-28T15:26:07.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:07.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 started sync module instance, tier type = 2024-06-28T15:26:07.455 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.472+0000 7f9d63f43980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:26:07.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.488+0000 7f9d63f43980 20 add_watcher() i=0 2024-06-28T15:26:07.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.488+0000 7f9d63f43980 20 add_watcher() i=1 2024-06-28T15:26:07.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.488+0000 7f9d63f43980 20 add_watcher() i=2 2024-06-28T15:26:07.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.488+0000 7f9d63f43980 20 add_watcher() i=3 2024-06-28T15:26:07.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.488+0000 7f9d63f43980 20 add_watcher() i=4 2024-06-28T15:26:07.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.488+0000 7f9d63f43980 20 add_watcher() i=5 2024-06-28T15:26:07.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.488+0000 7f9d63f43980 20 add_watcher() i=6 2024-06-28T15:26:07.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.492+0000 7f9d63f43980 20 add_watcher() i=7 2024-06-28T15:26:07.472 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.492+0000 7f9d63f43980 2 all 8 watchers are set, enabling cache 2024-06-28T15:26:07.474 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.492+0000 7f9c137fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:26:07.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.492+0000 7f9d63f43980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:26:07.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.492+0000 7f9d63f43980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:26:07.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.492+0000 7f9d63f43980 5 note: GC not initialized 2024-06-28T15:26:07.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9c127fc640 20 reqs_thread_entry: start 2024-06-28T15:26:07.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 init_complete bucket index max shards: 1 2024-06-28T15:26:07.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 Filter name: none 2024-06-28T15:26:07.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9b83fff640 20 reqs_thread_entry: start 2024-06-28T15:26:07.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 rados->read ofs=0 len=0 2024-06-28T15:26:07.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:26:07.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 rados->read ofs=0 len=0 2024-06-28T15:26:07.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:26:07.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:26:07.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=3 2024-06-28T15:26:07.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 INFO: main: starting shard=0 2024-06-28T15:26:07.477 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.0 2024-06-28T15:26:07.478 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.0 2024-06-28T15:26:07.478 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-06-28T15:26:07.478 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 INFO: main: starting shard=1 2024-06-28T15:26:07.478 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.1 2024-06-28T15:26:07.478 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.1 2024-06-28T15:26:07.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-06-28T15:26:07.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 INFO: main: starting shard=2 2024-06-28T15:26:07.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.2 2024-06-28T15:26:07.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.496+0000 7f9d63f43980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.2 2024-06-28T15:26:07.479 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.500+0000 7f9d63f43980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-06-28T15:26:07.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.500+0000 7f9d63f43980 20 INFO: main: done 2024-06-28T15:26:07.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.500+0000 7f9c137fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:26:07.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.504+0000 7f9d63f43980 20 remove_watcher() i=0 2024-06-28T15:26:07.486 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.504+0000 7f9d63f43980 2 removed watcher, disabling cache 2024-06-28T15:26:07.489 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.508+0000 7f9d63f43980 20 remove_watcher() i=1 2024-06-28T15:26:07.490 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.508+0000 7f9d63f43980 20 remove_watcher() i=2 2024-06-28T15:26:07.492 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.512+0000 7f9d63f43980 20 remove_watcher() i=3 2024-06-28T15:26:07.494 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.512+0000 7f9d63f43980 20 remove_watcher() i=4 2024-06-28T15:26:07.495 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.516+0000 7f9d63f43980 20 remove_watcher() i=5 2024-06-28T15:26:07.497 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.516+0000 7f9d63f43980 20 remove_watcher() i=6 2024-06-28T15:26:07.498 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:07.516+0000 7f9d63f43980 20 remove_watcher() i=7 2024-06-28T15:26:07.539 INFO:tasks.workunit.client.0.smithi178.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-06-28T15:26:07.969 INFO:tasks.workunit.client.0.smithi178.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-06-28T15:26:08.389 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 17 --yes-i-really-mean-it 2024-06-28T15:26:08.424 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:26:08.424 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:26:08.467 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.484+0000 7fec8d941980 20 rados->read ofs=0 len=0 2024-06-28T15:26:08.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.488+0000 7fec8d941980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:08.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.488+0000 7fec8d941980 20 realm 2024-06-28T15:26:08.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.488+0000 7fec8d941980 20 rados->read ofs=0 len=0 2024-06-28T15:26:08.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.488+0000 7fec8d941980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:08.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.488+0000 7fec8d941980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:26:08.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.488+0000 7fec8d941980 20 rados->read ofs=0 len=0 2024-06-28T15:26:08.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.488+0000 7fec8d941980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:08.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.488+0000 7fec8d941980 20 rados->read ofs=0 len=0 2024-06-28T15:26:08.470 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.488+0000 7fec8d941980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:26:08.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.488+0000 7fec8d941980 20 rados->read ofs=0 len=0 2024-06-28T15:26:08.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.488+0000 7fec8d941980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:26:08.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.488+0000 7fec8d941980 20 searching for the correct realm 2024-06-28T15:26:08.474 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.492+0000 7fec8ac51640 0 -- 172.21.15.178:0/304903136 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fec74002380 msgr2=0x7fec7c05bfc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:26:08.688 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.707+0000 7fec8b452640 0 -- 172.21.15.178:0/304903136 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fec800b5040 msgr2=0x7fec800b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:26:08.906 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.923+0000 7fec8d941980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:26:08.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.923+0000 7fec8d941980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:26:08.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.923+0000 7fec8d941980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:26:08.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.923+0000 7fec8d941980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:26:08.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.923+0000 7fec8d941980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:26:08.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.923+0000 7fec8d941980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:26:08.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.923+0000 7fec8d941980 20 rados->read ofs=0 len=0 2024-06-28T15:26:08.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.923+0000 7fec8d941980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:08.907 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.923+0000 7fec8d941980 20 rados->read ofs=0 len=0 2024-06-28T15:26:08.908 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.927+0000 7fec8d941980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:26:08.908 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.927+0000 7fec8d941980 20 rados->read ofs=0 len=0 2024-06-28T15:26:08.909 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.927+0000 7fec8d941980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:26:08.909 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.927+0000 7fec8d941980 20 zone default found 2024-06-28T15:26:08.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.927+0000 7fec8d941980 4 Realm: () 2024-06-28T15:26:08.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.927+0000 7fec8d941980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:26:08.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.927+0000 7fec8d941980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:26:08.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.927+0000 7fec8d941980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:26:08.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.927+0000 7fec8d941980 20 zonegroup default 2024-06-28T15:26:08.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.927+0000 7fec8d941980 20 rados->read ofs=0 len=0 2024-06-28T15:26:08.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.927+0000 7fec8d941980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:08.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.927+0000 7fec8d941980 20 rados->read ofs=0 len=0 2024-06-28T15:26:08.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.927+0000 7fec8d941980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:08.911 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.931+0000 7fec8d941980 20 rados->read ofs=0 len=0 2024-06-28T15:26:08.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.931+0000 7fec8d941980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:08.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.931+0000 7fec8d941980 20 started sync module instance, tier type = 2024-06-28T15:26:08.912 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.931+0000 7fec8d941980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:26:08.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.955+0000 7fec8d941980 20 add_watcher() i=0 2024-06-28T15:26:08.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.955+0000 7fec8d941980 20 add_watcher() i=1 2024-06-28T15:26:08.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.955+0000 7fec8d941980 20 add_watcher() i=2 2024-06-28T15:26:08.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.955+0000 7fec8d941980 20 add_watcher() i=3 2024-06-28T15:26:08.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.955+0000 7fec8d941980 20 add_watcher() i=4 2024-06-28T15:26:08.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.955+0000 7fec8d941980 20 add_watcher() i=5 2024-06-28T15:26:08.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.955+0000 7fec8d941980 20 add_watcher() i=6 2024-06-28T15:26:08.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.955+0000 7fec8d941980 20 add_watcher() i=7 2024-06-28T15:26:08.939 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.955+0000 7fec8d941980 2 all 8 watchers are set, enabling cache 2024-06-28T15:26:08.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.959+0000 7feb41fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:26:08.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:26:08.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:26:08.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 5 note: GC not initialized 2024-06-28T15:26:08.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7feb40fe1640 20 reqs_thread_entry: start 2024-06-28T15:26:08.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 20 init_complete bucket index max shards: 1 2024-06-28T15:26:08.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 20 Filter name: none 2024-06-28T15:26:08.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7feaba7fc640 20 reqs_thread_entry: start 2024-06-28T15:26:08.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-06-28T15:26:08.945 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 20 rados->read ofs=0 len=0 2024-06-28T15:26:08.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:26:08.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-06-28T15:26:08.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-06-28T15:26:08.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 : miss 2024-06-28T15:26:08.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 20 rados->read ofs=0 len=0 2024-06-28T15:26:08.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:26:08.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:26:08.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 10 adding default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:26:08.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:26:08.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:26:08.947 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.963+0000 7fec8d941980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 2024-06-28T15:26:08.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.979+0000 7fec8d941980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:26:08.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.979+0000 7fec8d941980 10 moving default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:26:08.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.979+0000 7fec8d941980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:26:08.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.979+0000 7fec8d941980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19, ofs0, ns] 2024-06-28T15:26:08.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.979+0000 7feb6e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595121 cookie 94072676570144 notifier 7479 bl.length()=897 2024-06-28T15:26:08.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.979+0000 7feb6e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:26:08.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.979+0000 7feb6e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:26:08.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.979+0000 7feb6e7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:26:08.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.979+0000 7fec8d941980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): orig_sources= new_sources= 2024-06-28T15:26:08.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.979+0000 7fec8d941980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): potential sources added=[] removed=[] 2024-06-28T15:26:08.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.979+0000 7fec8d941980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): orig_dests= new_dests= 2024-06-28T15:26:08.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.979+0000 7fec8d941980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): potential dests added=[] removed=[] 2024-06-28T15:26:08.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.0 2024-06-28T15:26:08.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.0 2024-06-28T15:26:08.965 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.1 2024-06-28T15:26:08.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.1 2024-06-28T15:26:08.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.2 2024-06-28T15:26:08.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.2 2024-06-28T15:26:08.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.3 2024-06-28T15:26:08.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.3 2024-06-28T15:26:08.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.4 2024-06-28T15:26:08.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.4 2024-06-28T15:26:08.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.5 2024-06-28T15:26:08.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.5 2024-06-28T15:26:08.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.6 2024-06-28T15:26:08.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.6 2024-06-28T15:26:08.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.7 2024-06-28T15:26:08.966 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.7 2024-06-28T15:26:08.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.8 2024-06-28T15:26:08.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.8 2024-06-28T15:26:08.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:08.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:08.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.10 2024-06-28T15:26:08.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.10 2024-06-28T15:26:08.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.11 2024-06-28T15:26:08.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.11 2024-06-28T15:26:08.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.12 2024-06-28T15:26:08.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.12 2024-06-28T15:26:08.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.13 2024-06-28T15:26:08.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.13 2024-06-28T15:26:08.967 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.14 2024-06-28T15:26:08.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.14 2024-06-28T15:26:08.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.15 2024-06-28T15:26:08.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.15 2024-06-28T15:26:08.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.16 2024-06-28T15:26:08.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.16 2024-06-28T15:26:08.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.0 2024-06-28T15:26:08.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.0 2024-06-28T15:26:08.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.1 2024-06-28T15:26:08.968 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.983+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.1 2024-06-28T15:26:08.969 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.987+0000 7fec8d941980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.2 2024-06-28T15:26:08.969 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.987+0000 7fec8d941980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.2.2 2024-06-28T15:26:08.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.987+0000 7fec8d941980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 : hit (requested=0x17, cached=0x17) 2024-06-28T15:26:08.971 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.987+0000 7fec8d941980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 2024-06-28T15:26:08.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.991+0000 7fec8d941980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:26:08.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.991+0000 7fec8d941980 10 moving default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:26:08.973 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.991+0000 7fec8d941980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:26:08.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.991+0000 7fec8d941980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19, ofs0, ns] 2024-06-28T15:26:08.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.991+0000 7feb6e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595122 cookie 94072676570144 notifier 7479 bl.length()=865 2024-06-28T15:26:08.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.991+0000 7feb6e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:26:08.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.991+0000 7feb6e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:26:08.974 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.991+0000 7feb6e7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:26:08.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.995+0000 7fec8d941980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): orig_sources= new_sources= 2024-06-28T15:26:08.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.995+0000 7fec8d941980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): potential sources added=[] removed=[] 2024-06-28T15:26:08.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.995+0000 7fec8d941980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): orig_dests= new_dests= 2024-06-28T15:26:08.977 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.995+0000 7fec8d941980 20 handle_bi_update(): bucket=:myver[843e24ce-085a-4043-8f55-38e05370e790.4686.19]): potential dests added=[] removed=[] 2024-06-28T15:26:08.979 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.999+0000 7fec8d941980 1 execute INFO: reshard of bucket "myver" from 17 shards to 17 shards completed successfully 2024-06-28T15:26:08.982 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:08.999+0000 7feb41fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:26:08.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.003+0000 7fec8d941980 20 remove_watcher() i=0 2024-06-28T15:26:08.986 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.003+0000 7fec8d941980 2 removed watcher, disabling cache 2024-06-28T15:26:08.988 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.007+0000 7fec8d941980 20 remove_watcher() i=1 2024-06-28T15:26:08.990 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.007+0000 7fec8d941980 20 remove_watcher() i=2 2024-06-28T15:26:08.992 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.011+0000 7fec8d941980 20 remove_watcher() i=3 2024-06-28T15:26:08.994 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.011+0000 7fec8d941980 20 remove_watcher() i=4 2024-06-28T15:26:08.996 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.015+0000 7fec8d941980 20 remove_watcher() i=5 2024-06-28T15:26:08.997 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.015+0000 7fec8d941980 20 remove_watcher() i=6 2024-06-28T15:26:08.998 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.015+0000 7fec8d941980 20 remove_watcher() i=7 2024-06-28T15:26:09.012 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket check unlinked --bucket myver --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-06-28T15:26:09.049 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:26:09.050 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:26:09.092 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.111+0000 7f44c45c9980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.094 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.111+0000 7f44c45c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:09.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.111+0000 7f44c45c9980 20 realm 2024-06-28T15:26:09.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.111+0000 7f44c45c9980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.111+0000 7f44c45c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:09.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.111+0000 7f44c45c9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:26:09.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.111+0000 7f44c45c9980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.095 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.111+0000 7f44c45c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:09.096 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.111+0000 7f44c45c9980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.096 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.115+0000 7f44c45c9980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:26:09.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.115+0000 7f44c45c9980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.097 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.115+0000 7f44c45c9980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:26:09.098 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.115+0000 7f44c45c9980 20 searching for the correct realm 2024-06-28T15:26:09.104 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.123+0000 7f44c28db640 0 -- 172.21.15.178:0/2407132947 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f44ac008040 msgr2=0x7f44ac046920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:26:09.321 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.339+0000 7f44c20da640 0 -- 172.21.15.178:0/2407132947 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x562589148b40 msgr2=0x56258914af30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:26:09.328 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:26:09.328 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:26:09.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:26:09.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:26:09.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:26:09.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:26:09.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:09.329 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:26:09.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:26:09.330 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 zone default found 2024-06-28T15:26:09.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 4 Realm: () 2024-06-28T15:26:09.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:26:09.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:26:09.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:26:09.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 zonegroup default 2024-06-28T15:26:09.331 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:09.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.347+0000 7f44c45c9980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.351+0000 7f44c45c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:09.332 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.351+0000 7f44c45c9980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.351+0000 7f44c45c9980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:09.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.351+0000 7f44c45c9980 20 started sync module instance, tier type = 2024-06-28T15:26:09.333 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.351+0000 7f44c45c9980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:26:09.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.375+0000 7f44c45c9980 20 add_watcher() i=0 2024-06-28T15:26:09.357 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.375+0000 7f44c45c9980 20 add_watcher() i=1 2024-06-28T15:26:09.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.375+0000 7f44c45c9980 20 add_watcher() i=2 2024-06-28T15:26:09.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.375+0000 7f44c45c9980 20 add_watcher() i=3 2024-06-28T15:26:09.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.375+0000 7f44c45c9980 20 add_watcher() i=4 2024-06-28T15:26:09.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.375+0000 7f44c45c9980 20 add_watcher() i=5 2024-06-28T15:26:09.358 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.375+0000 7f44c45c9980 20 add_watcher() i=6 2024-06-28T15:26:09.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.379+0000 7f44c45c9980 20 add_watcher() i=7 2024-06-28T15:26:09.359 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.379+0000 7f44c45c9980 2 all 8 watchers are set, enabling cache 2024-06-28T15:26:09.364 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f4378fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:26:09.365 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f44c45c9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:26:09.365 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f44c45c9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:26:09.365 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f44c45c9980 5 note: GC not initialized 2024-06-28T15:26:09.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f42f37fe640 20 reqs_thread_entry: start 2024-06-28T15:26:09.366 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f44c45c9980 20 init_complete bucket index max shards: 1 2024-06-28T15:26:09.367 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f44c45c9980 20 Filter name: none 2024-06-28T15:26:09.367 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f42f17fa640 20 reqs_thread_entry: start 2024-06-28T15:26:09.367 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f44c45c9980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-06-28T15:26:09.367 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f44c45c9980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.367 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f44c45c9980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:26:09.368 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f44c45c9980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-06-28T15:26:09.368 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f44c45c9980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-06-28T15:26:09.368 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f44c45c9980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 : miss 2024-06-28T15:26:09.368 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.383+0000 7f44c45c9980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:26:09.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 info.flags=0x17 2024-06-28T15:26:09.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 10 adding default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 to cache LRU end 2024-06-28T15:26:09.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:26:09.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-06-28T15:26:09.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:843e24ce-085a-4043-8f55-38e05370e790.4686.19 2024-06-28T15:26:09.369 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.0 2024-06-28T15:26:09.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.0 2024-06-28T15:26:09.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 1 NOTICE: finished shard 0 (0 entries removed) 2024-06-28T15:26:09.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.1 2024-06-28T15:26:09.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.1 2024-06-28T15:26:09.370 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 1 NOTICE: finished shard 1 (0 entries removed) 2024-06-28T15:26:09.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.2 2024-06-28T15:26:09.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.2 2024-06-28T15:26:09.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 1 NOTICE: finished shard 2 (0 entries removed) 2024-06-28T15:26:09.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.3 2024-06-28T15:26:09.371 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.387+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.3 2024-06-28T15:26:09.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.391+0000 7f44c45c9980 1 NOTICE: finished shard 3 (0 entries removed) 2024-06-28T15:26:09.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.391+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.4 2024-06-28T15:26:09.372 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.391+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.4 2024-06-28T15:26:09.373 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.391+0000 7f44c45c9980 1 NOTICE: finished shard 4 (0 entries removed) 2024-06-28T15:26:09.373 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.391+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.5 2024-06-28T15:26:09.373 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.391+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.5 2024-06-28T15:26:09.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.391+0000 7f44c45c9980 1 NOTICE: finished shard 5 (0 entries removed) 2024-06-28T15:26:09.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.391+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.6 2024-06-28T15:26:09.374 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.391+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.6 2024-06-28T15:26:09.375 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.391+0000 7f44c45c9980 1 NOTICE: finished shard 6 (0 entries removed) 2024-06-28T15:26:09.375 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.391+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.7 2024-06-28T15:26:09.375 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.391+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.7 2024-06-28T15:26:09.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.395+0000 7f44c45c9980 1 NOTICE: finished shard 7 (0 entries removed) 2024-06-28T15:26:09.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.395+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.8 2024-06-28T15:26:09.376 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.395+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.8 2024-06-28T15:26:09.377 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.395+0000 7f44c45c9980 1 NOTICE: finished shard 8 (0 entries removed) 2024-06-28T15:26:09.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.395+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.395+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.395+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.10 2024-06-28T15:26:09.378 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.395+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.10 2024-06-28T15:26:09.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.395+0000 7f44c45c9980 1 NOTICE: finished shard 10 (0 entries removed) 2024-06-28T15:26:09.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.395+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.11 2024-06-28T15:26:09.379 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.395+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.11 2024-06-28T15:26:09.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.399+0000 7f44c45c9980 1 NOTICE: finished shard 11 (0 entries removed) 2024-06-28T15:26:09.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.399+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.12 2024-06-28T15:26:09.380 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.399+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.12 2024-06-28T15:26:09.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.399+0000 7f44c45c9980 1 NOTICE: finished shard 12 (0 entries removed) 2024-06-28T15:26:09.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.399+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.13 2024-06-28T15:26:09.381 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.399+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.13 2024-06-28T15:26:09.382 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.399+0000 7f44c45c9980 1 NOTICE: finished shard 13 (0 entries removed) 2024-06-28T15:26:09.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.399+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.14 2024-06-28T15:26:09.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.399+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.14 2024-06-28T15:26:09.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.403+0000 7f44c45c9980 1 NOTICE: finished shard 14 (0 entries removed) 2024-06-28T15:26:09.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.403+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.15 2024-06-28T15:26:09.383 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.403+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.15 2024-06-28T15:26:09.384 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.403+0000 7f44c45c9980 1 NOTICE: finished shard 15 (0 entries removed) 2024-06-28T15:26:09.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.403+0000 7f44c45c9980 10 bucket_obj is .dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.16 2024-06-28T15:26:09.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.403+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.16 2024-06-28T15:26:09.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.403+0000 7f44c45c9980 1 NOTICE: finished shard 16 (0 entries removed) 2024-06-28T15:26:09.385 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.403+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.386 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.403+0000 7f44c45c9980 20 get_obj_state: octx=0x562588ed7770 obj=myver:x state=0x562589185058 s->prefetch_data=0 2024-06-28T15:26:09.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.403+0000 7f44c45c9980 20 get_obj_state: setting s->obj_tag to p8de4jy8s73tyyljrn6a1g7774b97zmz 2024-06-28T15:26:09.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.403+0000 7f44c45c9980 20 get_obj_state_impl: setting s->olh_tag to 0wb8rafcxl0rodn2cstpeubatmufcs6l 2024-06-28T15:26:09.387 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.403+0000 7f44c45c9980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=0wb8rafcxl0rodn2cstpeubatmufcs6l 2024-06-28T15:26:09.390 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.407+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.392 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.411+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.392 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.411+0000 7f44c45c9980 20 olh_log_entry: epoch=2 op=2 key=x[] 2024-06-28T15:26:09.393 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.411+0000 7f44c45c9980 20 olh_log_entry: epoch=2 op=3 key=x[dCz8UB6v5dTNxkpBEc4qo8rTMqOKrdB] 2024-06-28T15:26:09.393 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.411+0000 7f44c45c9980 20 get_obj_state: octx=0x562588ed7770 obj=myver:_:dCz8UB6v5dTNxkpBEc4qo8rTMqOKrdB_x state=0x5625891ada58 s->prefetch_data=0 2024-06-28T15:26:09.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.411+0000 7f44c45c9980 10 manifest: total_size = 9 2024-06-28T15:26:09.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.411+0000 7f44c45c9980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.18272209509167285727 2024-06-28T15:26:09.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.411+0000 7f44c45c9980 20 get_obj_state: octx=0x562588ed7770 obj=myver:_:dCz8UB6v5dTNxkpBEc4qo8rTMqOKrdB_x state=0x5625891ada58 s->prefetch_data=0 2024-06-28T15:26:09.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.411+0000 7f44c45c9980 20 prepare_atomic_modification: state is not atomic. state=0x5625891ada58 2024-06-28T15:26:09.394 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.411+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.399 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.419+0000 7f44c45c9980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:26:09.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.419+0000 7f44c45c9980 20 RGWObjManifest::operator++(): result: ofs=9 stripe_ofs=9 part_ofs=0 rule->part_size=0 2024-06-28T15:26:09.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.419+0000 7f44c45c9980 0 deleting objects inline since gc isn't initialized 2024-06-28T15:26:09.400 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.419+0000 7f44c45c9980 5 delete_objs_inline: removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.19__shadow:dCz8UB6v5dTNxkpBEc4qo8rTMqOKrdB_.y7S21P5-YeSHfDToimU2_H1aeTn7IY4_0 2024-06-28T15:26:09.401 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.419+0000 7f43a5ffb640 20 handle_completion(): completion ok for obj=x[dCz8UB6v5dTNxkpBEc4qo8rTMqOKrdB] 2024-06-28T15:26:09.402 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.419+0000 7f44c45c9980 5 delete_objs_inline: refcount put returned error -2 2024-06-28T15:26:09.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.423+0000 7f44c45c9980 20 get_obj_state: octx=0x562588ed7770 obj=myver:x state=0x562589185058 s->prefetch_data=0 2024-06-28T15:26:09.405 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.423+0000 7f44c45c9980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=0wb8rafcxl0rodn2cstpeubatmufcs6l 2024-06-28T15:26:09.410 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.427+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.412 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.431+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.413 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.431+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.413 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.431+0000 7f44c45c9980 20 get_obj_state: octx=0x562588ed7770 obj=myver:y state=0x5625891ada58 s->prefetch_data=0 2024-06-28T15:26:09.414 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.431+0000 7f44c45c9980 20 get_obj_state: setting s->obj_tag to tmsrznfq9b07ns0ipyyn3q2dnuun45tu 2024-06-28T15:26:09.414 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.431+0000 7f44c45c9980 20 get_obj_state_impl: setting s->olh_tag to 3psgj0rvqbt2mfg9ke0p37sz1pni9g7i 2024-06-28T15:26:09.414 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.431+0000 7f44c45c9980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=3psgj0rvqbt2mfg9ke0p37sz1pni9g7i 2024-06-28T15:26:09.416 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.435+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.418 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.435+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.435+0000 7f44c45c9980 20 olh_log_entry: epoch=2 op=2 key=y[] 2024-06-28T15:26:09.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.435+0000 7f44c45c9980 20 olh_log_entry: epoch=2 op=3 key=y[Y6YTYpDN.rT6yPrkJSDuX0NPG-ayVQy] 2024-06-28T15:26:09.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.435+0000 7f44c45c9980 20 get_obj_state: octx=0x562588ed7770 obj=myver:_:Y6YTYpDN.rT6yPrkJSDuX0NPG-ayVQy_y state=0x56258917b968 s->prefetch_data=0 2024-06-28T15:26:09.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.439+0000 7f44c45c9980 10 manifest: total_size = 9 2024-06-28T15:26:09.419 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.439+0000 7f44c45c9980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.10673890093870267038 2024-06-28T15:26:09.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.439+0000 7f44c45c9980 20 get_obj_state: octx=0x562588ed7770 obj=myver:_:Y6YTYpDN.rT6yPrkJSDuX0NPG-ayVQy_y state=0x56258917b968 s->prefetch_data=0 2024-06-28T15:26:09.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.439+0000 7f44c45c9980 20 prepare_atomic_modification: state is not atomic. state=0x56258917b968 2024-06-28T15:26:09.420 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.439+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.423 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.443+0000 7f44c45c9980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:26:09.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.443+0000 7f44c45c9980 20 RGWObjManifest::operator++(): result: ofs=9 stripe_ofs=9 part_ofs=0 rule->part_size=0 2024-06-28T15:26:09.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.443+0000 7f44c45c9980 0 deleting objects inline since gc isn't initialized 2024-06-28T15:26:09.424 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.443+0000 7f44c45c9980 5 delete_objs_inline: removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.19__shadow:Y6YTYpDN.rT6yPrkJSDuX0NPG-ayVQy_.Pc9b6Gfs_0z5h90r6X6f9uaYri7bsGH_0 2024-06-28T15:26:09.425 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.443+0000 7f43a57fa640 20 handle_completion(): completion ok for obj=y[Y6YTYpDN.rT6yPrkJSDuX0NPG-ayVQy] 2024-06-28T15:26:09.427 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.443+0000 7f44c45c9980 5 delete_objs_inline: refcount put returned error -2 2024-06-28T15:26:09.429 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.447+0000 7f44c45c9980 20 get_obj_state: octx=0x562588ed7770 obj=myver:y state=0x5625891ada58 s->prefetch_data=0 2024-06-28T15:26:09.429 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.447+0000 7f44c45c9980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=3psgj0rvqbt2mfg9ke0p37sz1pni9g7i 2024-06-28T15:26:09.434 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.451+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.435 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.455+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.455+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.436 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.455+0000 7f44c45c9980 20 get_obj_state: octx=0x562588ed7770 obj=myver:z state=0x5625891ada58 s->prefetch_data=0 2024-06-28T15:26:09.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.455+0000 7f44c45c9980 20 get_obj_state: setting s->obj_tag to 1bs4nk8fnbqu5ueraqymusxdwyx4pv8q 2024-06-28T15:26:09.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.455+0000 7f44c45c9980 20 get_obj_state_impl: setting s->olh_tag to pzzhkmpp1d63iuh5kqpzusgn3q4vnnda 2024-06-28T15:26:09.437 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.455+0000 7f44c45c9980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=pzzhkmpp1d63iuh5kqpzusgn3q4vnnda 2024-06-28T15:26:09.439 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.459+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.441 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.459+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.459+0000 7f44c45c9980 20 olh_log_entry: epoch=2 op=2 key=z[] 2024-06-28T15:26:09.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.459+0000 7f44c45c9980 20 olh_log_entry: epoch=2 op=3 key=z[ZkMPa2dqAIwbQqFniUj8k3LguEMqoKc] 2024-06-28T15:26:09.442 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.459+0000 7f44c45c9980 20 get_obj_state: octx=0x562588ed7770 obj=myver:_:ZkMPa2dqAIwbQqFniUj8k3LguEMqoKc_z state=0x56258917b968 s->prefetch_data=0 2024-06-28T15:26:09.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.459+0000 7f44c45c9980 10 manifest: total_size = 12 2024-06-28T15:26:09.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.459+0000 7f44c45c9980 20 get_obj_state: setting s->obj_tag to 843e24ce-085a-4043-8f55-38e05370e790.4686.11941711285015805242 2024-06-28T15:26:09.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.459+0000 7f44c45c9980 20 get_obj_state: octx=0x562588ed7770 obj=myver:_:ZkMPa2dqAIwbQqFniUj8k3LguEMqoKc_z state=0x56258917b968 s->prefetch_data=0 2024-06-28T15:26:09.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.459+0000 7f44c45c9980 20 prepare_atomic_modification: state is not atomic. state=0x56258917b968 2024-06-28T15:26:09.443 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.459+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.467+0000 7f44c45c9980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-06-28T15:26:09.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.467+0000 7f44c45c9980 20 RGWObjManifest::operator++(): result: ofs=12 stripe_ofs=12 part_ofs=0 rule->part_size=0 2024-06-28T15:26:09.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.467+0000 7f44c45c9980 0 deleting objects inline since gc isn't initialized 2024-06-28T15:26:09.448 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.467+0000 7f44c45c9980 5 delete_objs_inline: removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.19__shadow:ZkMPa2dqAIwbQqFniUj8k3LguEMqoKc_.O9kfXkZHZ6L_U6XMBGSpn23d_GKnCns_0 2024-06-28T15:26:09.449 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.467+0000 7f43a5ffb640 20 handle_completion(): completion ok for obj=z[ZkMPa2dqAIwbQqFniUj8k3LguEMqoKc] 2024-06-28T15:26:09.451 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.471+0000 7f44c45c9980 5 delete_objs_inline: refcount put returned error -2 2024-06-28T15:26:09.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.471+0000 7f44c45c9980 20 get_obj_state: octx=0x562588ed7770 obj=myver:z state=0x5625891ada58 s->prefetch_data=0 2024-06-28T15:26:09.453 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.471+0000 7f44c45c9980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=pzzhkmpp1d63iuh5kqpzusgn3q4vnnda 2024-06-28T15:26:09.458 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.475+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.479+0000 7f44c45c9980 20 bucket index object: :.dir.843e24ce-085a-4043-8f55-38e05370e790.4686.19.3.9 2024-06-28T15:26:09.460 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.479+0000 7f44c45c9980 1 NOTICE: finished shard 9 (3 entries removed) 2024-06-28T15:26:09.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.479+0000 7f44c45c9980 1 NOTICE: finished all shards (3 entries removed) 2024-06-28T15:26:09.461 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.479+0000 7f4378fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:26:09.466 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.483+0000 7f44c45c9980 20 remove_watcher() i=0 2024-06-28T15:26:09.466 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.483+0000 7f44c45c9980 2 removed watcher, disabling cache 2024-06-28T15:26:09.469 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.487+0000 7f44c45c9980 20 remove_watcher() i=1 2024-06-28T15:26:09.471 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.487+0000 7f44c45c9980 20 remove_watcher() i=2 2024-06-28T15:26:09.473 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.491+0000 7f44c45c9980 20 remove_watcher() i=3 2024-06-28T15:26:09.475 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.491+0000 7f44c45c9980 20 remove_watcher() i=4 2024-06-28T15:26:09.476 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.495+0000 7f44c45c9980 20 remove_watcher() i=5 2024-06-28T15:26:09.478 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.495+0000 7f44c45c9980 20 remove_watcher() i=6 2024-06-28T15:26:09.480 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.499+0000 7f44c45c9980 20 remove_watcher() i=7 2024-06-28T15:26:09.817 INFO:tasks.workunit.client.0.smithi178.stderr:radosgw-admin bucket stats --bucket myver 2024-06-28T15:26:09.849 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:26:09.849 INFO:tasks.workunit.client.0.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:26:09.894 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.911+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.915+0000 7f81dcccc980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:09.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.915+0000 7f81dcccc980 20 realm 2024-06-28T15:26:09.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.915+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.915+0000 7f81dcccc980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:09.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.915+0000 7f81dcccc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:26:09.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.915+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.915+0000 7f81dcccc980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:09.896 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.915+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.915+0000 7f81dcccc980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:26:09.897 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.915+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.915+0000 7f81dcccc980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:26:09.898 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.915+0000 7f81dcccc980 20 searching for the correct realm 2024-06-28T15:26:09.904 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.923+0000 7f81d9fdc640 0 -- 172.21.15.178:0/4098009640 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x558ca68f76b0 msgr2=0x558ca68f9b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:26:09.910 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.927+0000 7f81da7dd640 0 -- 172.21.15.178:0/4098009640 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f81d005ba30 msgr2=0x7f81d001a710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:26:09.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:26:09.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:26:09.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:26:09.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:26:09.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:26:09.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:26:09.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:09.918 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:26:09.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:26:09.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 zone default found 2024-06-28T15:26:09.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 4 Realm: () 2024-06-28T15:26:09.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:26:09.927 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:26:09.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:26:09.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 zonegroup default 2024-06-28T15:26:09.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.935+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.939+0000 7f81dcccc980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:09.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.939+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.939+0000 7f81dcccc980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:09.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.939+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.939+0000 7f81dcccc980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:26:09.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.939+0000 7f81dcccc980 20 started sync module instance, tier type = 2024-06-28T15:26:09.928 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.939+0000 7f81dcccc980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:26:09.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.955+0000 7f81dcccc980 20 add_watcher() i=0 2024-06-28T15:26:09.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.955+0000 7f81dcccc980 20 add_watcher() i=1 2024-06-28T15:26:09.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.955+0000 7f81dcccc980 20 add_watcher() i=2 2024-06-28T15:26:09.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.955+0000 7f81dcccc980 20 add_watcher() i=3 2024-06-28T15:26:09.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.955+0000 7f81dcccc980 20 add_watcher() i=4 2024-06-28T15:26:09.936 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.955+0000 7f81dcccc980 20 add_watcher() i=5 2024-06-28T15:26:09.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.955+0000 7f81dcccc980 20 add_watcher() i=6 2024-06-28T15:26:09.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.955+0000 7f81dcccc980 20 add_watcher() i=7 2024-06-28T15:26:09.937 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.955+0000 7f81dcccc980 2 all 8 watchers are set, enabling cache 2024-06-28T15:26:09.940 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f8090fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:26:09.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:26:09.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:26:09.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 5 note: GC not initialized 2024-06-28T15:26:09.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f800b7fe640 20 reqs_thread_entry: start 2024-06-28T15:26:09.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 20 init_complete bucket index max shards: 1 2024-06-28T15:26:09.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 20 Filter name: none 2024-06-28T15:26:09.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f80097fa640 20 reqs_thread_entry: start 2024-06-28T15:26:09.941 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:26:09.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.942 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:26:09.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:26:09.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 20 rados_obj.operate() r=0 bl.length=197 2024-06-28T15:26:09.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 20 rados->read ofs=0 len=0 2024-06-28T15:26:09.943 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 20 rados_obj.operate() r=0 bl.length=411 2024-06-28T15:26:09.944 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.959+0000 7f81dcccc980 20 couldn't put binfo cache entry, might have raced with data changes 2024-06-28T15:26:09.946 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.963+0000 7f8090fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:26:09.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.967+0000 7f81dcccc980 20 remove_watcher() i=0 2024-06-28T15:26:09.950 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.967+0000 7f81dcccc980 2 removed watcher, disabling cache 2024-06-28T15:26:09.953 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.971+0000 7f81dcccc980 20 remove_watcher() i=1 2024-06-28T15:26:09.955 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.971+0000 7f81dcccc980 20 remove_watcher() i=2 2024-06-28T15:26:09.957 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.975+0000 7f81dcccc980 20 remove_watcher() i=3 2024-06-28T15:26:09.959 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.975+0000 7f81dcccc980 20 remove_watcher() i=4 2024-06-28T15:26:09.960 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.979+0000 7f81dcccc980 20 remove_watcher() i=5 2024-06-28T15:26:09.961 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.979+0000 7f81dcccc980 20 remove_watcher() i=6 2024-06-28T15:26:09.962 INFO:tasks.workunit.client.0.smithi178.stderr:2024-06-28T15:26:09.979+0000 7f81dcccc980 20 remove_watcher() i=7 2024-06-28T15:26:09.976 INFO:tasks.workunit.client.0.smithi178.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2024-06-28T15:26:09.977 INFO:tasks.workunit.client.0.smithi178.stderr:Deleting bucket a-bucket 2024-06-28T15:26:10.078 INFO:tasks.workunit.client.0.smithi178.stderr:Deleting bucket myver 2024-06-28T15:26:10.654 INFO:tasks.workunit.client.0.smithi178.stderr:Completed resharding tests 2024-06-28T15:26:10.654 INFO:tasks.workunit.client.0.smithi178.stdout:connected to http://localhost:80 2024-06-28T15:26:10.758 INFO:tasks.workunit.client.0.smithi178.stderr:+ deactivate 2024-06-28T15:26:10.758 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-06-28T15:26:10.758 INFO:tasks.workunit.client.0.smithi178.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-06-28T15:26:10.758 INFO:tasks.workunit.client.0.smithi178.stderr:+ export PATH 2024-06-28T15:26:10.758 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset _OLD_VIRTUAL_PATH 2024-06-28T15:26:10.758 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -n '' ']' 2024-06-28T15:26:10.759 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-06-28T15:26:10.759 INFO:tasks.workunit.client.0.smithi178.stderr:+ hash -r 2024-06-28T15:26:10.759 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' -n '' ']' 2024-06-28T15:26:10.759 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset VIRTUAL_ENV 2024-06-28T15:26:10.759 INFO:tasks.workunit.client.0.smithi178.stdout:OK. 2024-06-28T15:26:10.760 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-06-28T15:26:10.760 INFO:tasks.workunit.client.0.smithi178.stderr:+ '[' '!' '' = nondestructive ']' 2024-06-28T15:26:10.760 INFO:tasks.workunit.client.0.smithi178.stderr:+ unset -f deactivate 2024-06-28T15:26:10.760 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo OK. 2024-06-28T15:26:10.761 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-28T15:26:10.761 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-28T15:26:10.816 INFO:tasks.workunit:Stopping ['rgw/run-reshard.sh'] on client.0... 2024-06-28T15:26:10.816 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-28T15:26:11.245 DEBUG:teuthology.parallel:result is None 2024-06-28T15:26:11.246 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:26:11.257 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:26:11.258 DEBUG:teuthology.orchestra.run.smithi178:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-06-28T15:26:11.304 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-06-28T15:26:11.304 INFO:teuthology.run_tasks:Running task s3tests-java... 2024-06-28T15:26:11.320 DEBUG:tasks.s3tests_java:S3 Tests Java: __INIT__ 2024-06-28T15:26:11.321 DEBUG:tasks.s3tests_java:S3 Tests Java: SETUP 2024-06-28T15:26:11.321 INFO:tasks.s3tests_java:S3 Tests Java: Downloading test suite... 2024-06-28T15:26:11.321 DEBUG:teuthology.orchestra.run.smithi178:> git clone -b ceph-master https://github.com/ceph/java_s3tests.git /home/ubuntu/cephtest/s3-tests-java 2024-06-28T15:26:11.351 INFO:teuthology.orchestra.run.smithi178.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-java'... 2024-06-28T15:26:47.516 INFO:tasks.s3tests_java:S3 Tests Java: Installing required packages... 2024-06-28T15:26:47.517 DEBUG:teuthology.orchestra.run.smithi178:> /home/ubuntu/cephtest/s3-tests-java/bootstrap.sh 2024-06-28T15:26:47.609 INFO:teuthology.orchestra.run.smithi178.stderr:/home/ubuntu/cephtest/s3-tests-java/bootstrap.sh: missing required DEB packages. Installing via sudo. 2024-06-28T15:26:47.669 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-06-28T15:26:47.879 INFO:teuthology.orchestra.run.smithi178.stdout:Building dependency tree... 2024-06-28T15:26:47.880 INFO:teuthology.orchestra.run.smithi178.stdout:Reading state information... 2024-06-28T15:26:48.076 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages were automatically installed and are no longer required: 2024-06-28T15:26:48.077 INFO:teuthology.orchestra.run.smithi178.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-06-28T15:26:48.077 INFO:teuthology.orchestra.run.smithi178.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-06-28T15:26:48.077 INFO:teuthology.orchestra.run.smithi178.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-06-28T15:26:48.077 INFO:teuthology.orchestra.run.smithi178.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-06-28T15:26:48.078 INFO:teuthology.orchestra.run.smithi178.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-06-28T15:26:48.078 INFO:teuthology.orchestra.run.smithi178.stdout:Use 'sudo apt autoremove' to remove them. 2024-06-28T15:26:48.078 INFO:teuthology.orchestra.run.smithi178.stdout:The following additional packages will be installed: 2024-06-28T15:26:48.079 INFO:teuthology.orchestra.run.smithi178.stdout: libgail-common libgail18 libgtk2.0-0 libgtk2.0-bin libgtk2.0-common 2024-06-28T15:26:48.079 INFO:teuthology.orchestra.run.smithi178.stdout: openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless 2024-06-28T15:26:48.080 INFO:teuthology.orchestra.run.smithi178.stdout:Suggested packages: 2024-06-28T15:26:48.080 INFO:teuthology.orchestra.run.smithi178.stdout: gvfs openjdk-8-demo openjdk-8-source visualvm libnss-mdns fonts-nanum 2024-06-28T15:26:48.080 INFO:teuthology.orchestra.run.smithi178.stdout: fonts-ipafont-gothic fonts-ipafont-mincho fonts-wqy-microhei 2024-06-28T15:26:48.080 INFO:teuthology.orchestra.run.smithi178.stdout: fonts-wqy-zenhei fonts-indic 2024-06-28T15:26:48.111 INFO:teuthology.orchestra.run.smithi178.stdout:The following NEW packages will be installed: 2024-06-28T15:26:48.112 INFO:teuthology.orchestra.run.smithi178.stdout: libgail-common libgail18 libgtk2.0-0 libgtk2.0-bin libgtk2.0-common 2024-06-28T15:26:48.113 INFO:teuthology.orchestra.run.smithi178.stdout: openjdk-8-jdk openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless 2024-06-28T15:26:48.211 INFO:teuthology.orchestra.run.smithi178.stdout:0 upgraded, 9 newly installed, 0 to remove and 315 not upgraded. 2024-06-28T15:26:48.212 INFO:teuthology.orchestra.run.smithi178.stdout:Need to get 46.0 MB of archives. 2024-06-28T15:26:48.212 INFO:teuthology.orchestra.run.smithi178.stdout:After this operation, 155 MB of additional disk space will be used. 2024-06-28T15:26:48.212 INFO:teuthology.orchestra.run.smithi178.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-common all 2.24.33-2ubuntu2 [125 kB] 2024-06-28T15:26:48.376 INFO:teuthology.orchestra.run.smithi178.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-0 amd64 2.24.33-2ubuntu2 [2,037 kB] 2024-06-28T15:26:48.536 INFO:teuthology.orchestra.run.smithi178.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgail18 amd64 2.24.33-2ubuntu2 [15.9 kB] 2024-06-28T15:26:48.536 INFO:teuthology.orchestra.run.smithi178.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgail-common amd64 2.24.33-2ubuntu2 [132 kB] 2024-06-28T15:26:48.540 INFO:teuthology.orchestra.run.smithi178.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-bin amd64 2.24.33-2ubuntu2 [7,932 B] 2024-06-28T15:26:48.540 INFO:teuthology.orchestra.run.smithi178.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jre-headless amd64 8u412-ga-1~22.04.1 [30.8 MB] 2024-06-28T15:26:49.246 INFO:teuthology.orchestra.run.smithi178.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jre amd64 8u412-ga-1~22.04.1 [75.3 kB] 2024-06-28T15:26:49.248 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jdk-headless amd64 8u412-ga-1~22.04.1 [8,857 kB] 2024-06-28T15:26:49.383 INFO:teuthology.orchestra.run.smithi178.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jdk amd64 8u412-ga-1~22.04.1 [4,024 kB] 2024-06-28T15:26:49.775 INFO:teuthology.orchestra.run.smithi178.stderr:debconf: unable to initialize frontend: Dialog 2024-06-28T15:26:49.775 INFO:teuthology.orchestra.run.smithi178.stderr:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-06-28T15:26:49.775 INFO:teuthology.orchestra.run.smithi178.stderr:debconf: falling back to frontend: Readline 2024-06-28T15:26:49.786 INFO:teuthology.orchestra.run.smithi178.stderr:debconf: unable to initialize frontend: Readline 2024-06-28T15:26:49.786 INFO:teuthology.orchestra.run.smithi178.stderr:debconf: (This frontend requires a controlling tty.) 2024-06-28T15:26:49.787 INFO:teuthology.orchestra.run.smithi178.stderr:debconf: falling back to frontend: Teletype 2024-06-28T15:26:49.790 INFO:teuthology.orchestra.run.smithi178.stderr:dpkg-preconfigure: unable to re-open stdin: 2024-06-28T15:26:49.823 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 46.0 MB in 1s (34.2 MB/s) 2024-06-28T15:26:49.938 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libgtk2.0-common. 2024-06-28T15:26:50.005 INFO:teuthology.orchestra.run.smithi178.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 177223 files and directories currently installed.) 2024-06-28T15:26:50.010 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../0-libgtk2.0-common_2.24.33-2ubuntu2_all.deb ... 2024-06-28T15:26:50.054 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libgtk2.0-common (2.24.33-2ubuntu2) ... 2024-06-28T15:26:50.400 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libgtk2.0-0:amd64. 2024-06-28T15:26:50.429 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../1-libgtk2.0-0_2.24.33-2ubuntu2_amd64.deb ... 2024-06-28T15:26:50.491 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libgtk2.0-0:amd64 (2.24.33-2ubuntu2) ... 2024-06-28T15:26:50.953 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libgail18:amd64. 2024-06-28T15:26:50.982 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../2-libgail18_2.24.33-2ubuntu2_amd64.deb ... 2024-06-28T15:26:51.020 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libgail18:amd64 (2.24.33-2ubuntu2) ... 2024-06-28T15:26:51.239 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libgail-common:amd64. 2024-06-28T15:26:51.257 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../3-libgail-common_2.24.33-2ubuntu2_amd64.deb ... 2024-06-28T15:26:51.289 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libgail-common:amd64 (2.24.33-2ubuntu2) ... 2024-06-28T15:26:51.567 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libgtk2.0-bin. 2024-06-28T15:26:51.584 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../4-libgtk2.0-bin_2.24.33-2ubuntu2_amd64.deb ... 2024-06-28T15:26:51.616 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libgtk2.0-bin (2.24.33-2ubuntu2) ... 2024-06-28T15:26:52.020 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package openjdk-8-jre-headless:amd64. 2024-06-28T15:26:52.049 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../5-openjdk-8-jre-headless_8u412-ga-1~22.04.1_amd64.deb ... 2024-06-28T15:26:52.115 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking openjdk-8-jre-headless:amd64 (8u412-ga-1~22.04.1) ... 2024-06-28T15:26:53.767 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package openjdk-8-jre:amd64. 2024-06-28T15:26:53.783 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../6-openjdk-8-jre_8u412-ga-1~22.04.1_amd64.deb ... 2024-06-28T15:26:53.820 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking openjdk-8-jre:amd64 (8u412-ga-1~22.04.1) ... 2024-06-28T15:26:54.271 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package openjdk-8-jdk-headless:amd64. 2024-06-28T15:26:54.300 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../7-openjdk-8-jdk-headless_8u412-ga-1~22.04.1_amd64.deb ... 2024-06-28T15:26:54.341 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking openjdk-8-jdk-headless:amd64 (8u412-ga-1~22.04.1) ... 2024-06-28T15:26:55.150 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package openjdk-8-jdk:amd64. 2024-06-28T15:26:55.179 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../8-openjdk-8-jdk_8u412-ga-1~22.04.1_amd64.deb ... 2024-06-28T15:26:55.220 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking openjdk-8-jdk:amd64 (8u412-ga-1~22.04.1) ... 2024-06-28T15:26:55.578 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up openjdk-8-jre-headless:amd64 (8u412-ga-1~22.04.1) ... 2024-06-28T15:26:56.716 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/orbd to provide /usr/bin/orbd (orbd) in auto mode 2024-06-28T15:26:56.742 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/servertool to provide /usr/bin/servertool (servertool) in auto mode 2024-06-28T15:26:56.760 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/tnameserv to provide /usr/bin/tnameserv (tnameserv) in auto mode 2024-06-28T15:26:57.125 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libgtk2.0-common (2.24.33-2ubuntu2) ... 2024-06-28T15:26:57.293 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libgtk2.0-0:amd64 (2.24.33-2ubuntu2) ... 2024-06-28T15:26:57.678 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up openjdk-8-jre:amd64 (8u412-ga-1~22.04.1) ... 2024-06-28T15:26:57.769 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/policytool to provide /usr/bin/policytool (policytool) in auto mode 2024-06-28T15:26:57.838 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up openjdk-8-jdk-headless:amd64 (8u412-ga-1~22.04.1) ... 2024-06-28T15:26:57.930 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/clhsdb to provide /usr/bin/clhsdb (clhsdb) in auto mode 2024-06-28T15:26:57.958 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/extcheck to provide /usr/bin/extcheck (extcheck) in auto mode 2024-06-28T15:26:57.984 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/hsdb to provide /usr/bin/hsdb (hsdb) in auto mode 2024-06-28T15:26:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/idlj to provide /usr/bin/idlj (idlj) in auto mode 2024-06-28T15:26:58.136 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/javah to provide /usr/bin/javah (javah) in auto mode 2024-06-28T15:26:58.288 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jhat to provide /usr/bin/jhat (jhat) in auto mode 2024-06-28T15:26:58.390 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jsadebugd to provide /usr/bin/jsadebugd (jsadebugd) in auto mode 2024-06-28T15:26:58.491 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/native2ascii to provide /usr/bin/native2ascii (native2ascii) in auto mode 2024-06-28T15:26:58.542 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/schemagen to provide /usr/bin/schemagen (schemagen) in auto mode 2024-06-28T15:26:58.585 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsgen to provide /usr/bin/wsgen (wsgen) in auto mode 2024-06-28T15:26:58.613 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsimport to provide /usr/bin/wsimport (wsimport) in auto mode 2024-06-28T15:26:58.636 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/xjc to provide /usr/bin/xjc (xjc) in auto mode 2024-06-28T15:26:58.701 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libgail18:amd64 (2.24.33-2ubuntu2) ... 2024-06-28T15:26:58.860 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libgtk2.0-bin (2.24.33-2ubuntu2) ... 2024-06-28T15:26:58.986 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up openjdk-8-jdk:amd64 (8u412-ga-1~22.04.1) ... 2024-06-28T15:26:59.078 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/appletviewer to provide /usr/bin/appletviewer (appletviewer) in auto mode 2024-06-28T15:26:59.172 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libgail-common:amd64 (2.24.33-2ubuntu2) ... 2024-06-28T15:26:59.324 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-06-28T15:27:00.424 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-06-28T15:27:00.525 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-06-28T15:27:00.769 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-06-28T15:27:01.205 INFO:teuthology.orchestra.run.smithi178.stdout: Gradle will be installed under /opt/gradle 2024-06-28T15:27:01.237 INFO:teuthology.orchestra.run.smithi178.stderr:--2024-06-28 15:27:01-- https://services.gradle.org/distributions/gradle-6.0.1-bin.zip 2024-06-28T15:27:01.241 INFO:teuthology.orchestra.run.smithi178.stderr:Resolving services.gradle.org (services.gradle.org)... 104.16.72.101, 104.16.73.101, 2606:4700::6810:4865, ... 2024-06-28T15:27:01.254 INFO:teuthology.orchestra.run.smithi178.stderr:Connecting to services.gradle.org (services.gradle.org)|104.16.72.101|:443... connected. 2024-06-28T15:27:01.316 INFO:teuthology.orchestra.run.smithi178.stderr:HTTP request sent, awaiting response... 301 Moved Permanently 2024-06-28T15:27:01.316 INFO:teuthology.orchestra.run.smithi178.stderr:Location: https://github.com/gradle/gradle-distributions/releases/download/v6.0.1/gradle-6.0.1-bin.zip [following] 2024-06-28T15:27:01.316 INFO:teuthology.orchestra.run.smithi178.stderr:--2024-06-28 15:27:01-- https://github.com/gradle/gradle-distributions/releases/download/v6.0.1/gradle-6.0.1-bin.zip 2024-06-28T15:27:01.317 INFO:teuthology.orchestra.run.smithi178.stderr:Resolving github.com (github.com)... 140.82.114.3 2024-06-28T15:27:01.359 INFO:teuthology.orchestra.run.smithi178.stderr:Connecting to github.com (github.com)|140.82.114.3|:443... connected. 2024-06-28T15:27:01.493 INFO:teuthology.orchestra.run.smithi178.stderr:HTTP request sent, awaiting response... 302 Found 2024-06-28T15:27:01.493 INFO:teuthology.orchestra.run.smithi178.stderr:Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/696192900/8f418ba1-c336-448c-95c6-5180382113cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240628%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240628T152656Z&X-Amz-Expires=300&X-Amz-Signature=282c1971b0f225b807c73e1e74494bf04ba97f2495251b6c6e52af345093d3fb&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=696192900&response-content-disposition=attachment%3B%20filename%3Dgradle-6.0.1-bin.zip&response-content-type=application%2Foctet-stream [following] 2024-06-28T15:27:01.493 INFO:teuthology.orchestra.run.smithi178.stderr:--2024-06-28 15:27:01-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/696192900/8f418ba1-c336-448c-95c6-5180382113cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240628%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240628T152656Z&X-Amz-Expires=300&X-Amz-Signature=282c1971b0f225b807c73e1e74494bf04ba97f2495251b6c6e52af345093d3fb&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=696192900&response-content-disposition=attachment%3B%20filename%3Dgradle-6.0.1-bin.zip&response-content-type=application%2Foctet-stream 2024-06-28T15:27:01.495 INFO:teuthology.orchestra.run.smithi178.stderr:Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.110.133, ... 2024-06-28T15:27:01.545 INFO:teuthology.orchestra.run.smithi178.stderr:Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.111.133|:443... connected. 2024-06-28T15:27:01.733 INFO:teuthology.orchestra.run.smithi178.stderr:HTTP request sent, awaiting response... 200 OK 2024-06-28T15:27:01.733 INFO:teuthology.orchestra.run.smithi178.stderr:Length: 95129672 (91M) [application/octet-stream] 2024-06-28T15:27:01.733 INFO:teuthology.orchestra.run.smithi178.stderr:Saving to: 'gradle-6.0.1-bin.zip' 2024-06-28T15:27:01.733 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-06-28T15:27:01.778 INFO:teuthology.orchestra.run.smithi178.stderr: 0K .......... .......... .......... .......... .......... 0% 1.11M 81s 2024-06-28T15:27:01.815 INFO:teuthology.orchestra.run.smithi178.stderr: 50K .......... .......... .......... .......... .......... 0% 1.27M 76s 2024-06-28T15:27:01.825 INFO:teuthology.orchestra.run.smithi178.stderr: 100K .......... .......... .......... .......... .......... 0% 5.06M 57s 2024-06-28T15:27:01.835 INFO:teuthology.orchestra.run.smithi178.stderr: 150K .......... .......... .......... .......... .......... 0% 4.77M 47s 2024-06-28T15:27:01.853 INFO:teuthology.orchestra.run.smithi178.stderr: 200K .......... .......... .......... .......... .......... 0% 2.68M 45s 2024-06-28T15:27:01.861 INFO:teuthology.orchestra.run.smithi178.stderr: 250K .......... .......... .......... .......... .......... 0% 8.90M 39s 2024-06-28T15:27:01.867 INFO:teuthology.orchestra.run.smithi178.stderr: 300K .......... .......... .......... .......... .......... 0% 6.27M 35s 2024-06-28T15:27:01.872 INFO:teuthology.orchestra.run.smithi178.stderr: 350K .......... .......... .......... .......... .......... 0% 8.99M 32s 2024-06-28T15:27:01.879 INFO:teuthology.orchestra.run.smithi178.stderr: 400K .......... .......... .......... .......... .......... 0% 7.25M 30s 2024-06-28T15:27:01.898 INFO:teuthology.orchestra.run.smithi178.stderr: 450K .......... .......... .......... .......... .......... 0% 2.52M 31s 2024-06-28T15:27:01.902 INFO:teuthology.orchestra.run.smithi178.stderr: 500K .......... .......... .......... .......... .......... 0% 12.4M 28s 2024-06-28T15:27:01.907 INFO:teuthology.orchestra.run.smithi178.stderr: 550K .......... .......... .......... .......... .......... 0% 9.73M 27s 2024-06-28T15:27:01.911 INFO:teuthology.orchestra.run.smithi178.stderr: 600K .......... .......... .......... .......... .......... 0% 13.4M 25s 2024-06-28T15:27:01.914 INFO:teuthology.orchestra.run.smithi178.stderr: 650K .......... .......... .......... .......... .......... 0% 20.0M 24s 2024-06-28T15:27:01.918 INFO:teuthology.orchestra.run.smithi178.stderr: 700K .......... .......... .......... .......... .......... 0% 12.4M 23s 2024-06-28T15:27:01.921 INFO:teuthology.orchestra.run.smithi178.stderr: 750K .......... .......... .......... .......... .......... 0% 14.2M 22s 2024-06-28T15:27:01.922 INFO:teuthology.orchestra.run.smithi178.stderr: 800K .......... .......... .......... .......... .......... 0% 30.8M 21s 2024-06-28T15:27:01.929 INFO:teuthology.orchestra.run.smithi178.stderr: 850K .......... .......... .......... .......... .......... 0% 7.77M 20s 2024-06-28T15:27:01.931 INFO:teuthology.orchestra.run.smithi178.stderr: 900K .......... .......... .......... .......... .......... 1% 23.1M 19s 2024-06-28T15:27:01.933 INFO:teuthology.orchestra.run.smithi178.stderr: 950K .......... .......... .......... .......... .......... 1% 21.7M 18s 2024-06-28T15:27:01.949 INFO:teuthology.orchestra.run.smithi178.stderr: 1000K .......... .......... .......... .......... .......... 1% 3.17M 19s 2024-06-28T15:27:01.952 INFO:teuthology.orchestra.run.smithi178.stderr: 1050K .......... .......... .......... .......... .......... 1% 15.0M 18s 2024-06-28T15:27:01.955 INFO:teuthology.orchestra.run.smithi178.stderr: 1100K .......... .......... .......... .......... .......... 1% 12.5M 18s 2024-06-28T15:27:01.961 INFO:teuthology.orchestra.run.smithi178.stderr: 1150K .......... .......... .......... .......... .......... 1% 9.85M 17s 2024-06-28T15:27:01.963 INFO:teuthology.orchestra.run.smithi178.stderr: 1200K .......... .......... .......... .......... .......... 1% 19.1M 17s 2024-06-28T15:27:01.966 INFO:teuthology.orchestra.run.smithi178.stderr: 1250K .......... .......... .......... .......... .......... 1% 14.2M 16s 2024-06-28T15:27:01.969 INFO:teuthology.orchestra.run.smithi178.stderr: 1300K .......... .......... .......... .......... .......... 1% 17.4M 16s 2024-06-28T15:27:01.972 INFO:teuthology.orchestra.run.smithi178.stderr: 1350K .......... .......... .......... .......... .......... 1% 15.8M 16s 2024-06-28T15:27:01.975 INFO:teuthology.orchestra.run.smithi178.stderr: 1400K .......... .......... .......... .......... .......... 1% 21.1M 15s 2024-06-28T15:27:01.978 INFO:teuthology.orchestra.run.smithi178.stderr: 1450K .......... .......... .......... .......... .......... 1% 17.4M 15s 2024-06-28T15:27:01.980 INFO:teuthology.orchestra.run.smithi178.stderr: 1500K .......... .......... .......... .......... .......... 1% 17.5M 15s 2024-06-28T15:27:01.984 INFO:teuthology.orchestra.run.smithi178.stderr: 1550K .......... .......... .......... .......... .......... 1% 15.7M 14s 2024-06-28T15:27:01.998 INFO:teuthology.orchestra.run.smithi178.stderr: 1600K .......... .......... .......... .......... .......... 1% 3.31M 15s 2024-06-28T15:27:02.002 INFO:teuthology.orchestra.run.smithi178.stderr: 1650K .......... .......... .......... .......... .......... 1% 11.4M 14s 2024-06-28T15:27:02.005 INFO:teuthology.orchestra.run.smithi178.stderr: 1700K .......... .......... .......... .......... .......... 1% 21.9M 14s 2024-06-28T15:27:02.007 INFO:teuthology.orchestra.run.smithi178.stderr: 1750K .......... .......... .......... .......... .......... 1% 18.1M 14s 2024-06-28T15:27:02.011 INFO:teuthology.orchestra.run.smithi178.stderr: 1800K .......... .......... .......... .......... .......... 1% 12.2M 14s 2024-06-28T15:27:02.016 INFO:teuthology.orchestra.run.smithi178.stderr: 1850K .......... .......... .......... .......... .......... 2% 17.2M 13s 2024-06-28T15:27:02.019 INFO:teuthology.orchestra.run.smithi178.stderr: 1900K .......... .......... .......... .......... .......... 2% 10.7M 13s 2024-06-28T15:27:02.021 INFO:teuthology.orchestra.run.smithi178.stderr: 1950K .......... .......... .......... .......... .......... 2% 18.3M 13s 2024-06-28T15:27:02.026 INFO:teuthology.orchestra.run.smithi178.stderr: 2000K .......... .......... .......... .......... .......... 2% 16.7M 13s 2024-06-28T15:27:02.031 INFO:teuthology.orchestra.run.smithi178.stderr: 2050K .......... .......... .......... .......... .......... 2% 11.6M 13s 2024-06-28T15:27:02.031 INFO:teuthology.orchestra.run.smithi178.stderr: 2100K .......... .......... .......... .......... .......... 2% 19.9M 13s 2024-06-28T15:27:02.034 INFO:teuthology.orchestra.run.smithi178.stderr: 2150K .......... .......... .......... .......... .......... 2% 19.4M 12s 2024-06-28T15:27:02.037 INFO:teuthology.orchestra.run.smithi178.stderr: 2200K .......... .......... .......... .......... .......... 2% 17.3M 12s 2024-06-28T15:27:02.045 INFO:teuthology.orchestra.run.smithi178.stderr: 2250K .......... .......... .......... .......... .......... 2% 5.63M 12s 2024-06-28T15:27:02.050 INFO:teuthology.orchestra.run.smithi178.stderr: 2300K .......... .......... .......... .......... .......... 2% 11.3M 12s 2024-06-28T15:27:02.053 INFO:teuthology.orchestra.run.smithi178.stderr: 2350K .......... .......... .......... .......... .......... 2% 13.0M 12s 2024-06-28T15:27:02.058 INFO:teuthology.orchestra.run.smithi178.stderr: 2400K .......... .......... .......... .......... .......... 2% 10.3M 12s 2024-06-28T15:27:02.060 INFO:teuthology.orchestra.run.smithi178.stderr: 2450K .......... .......... .......... .......... .......... 2% 23.5M 12s 2024-06-28T15:27:02.064 INFO:teuthology.orchestra.run.smithi178.stderr: 2500K .......... .......... .......... .......... .......... 2% 13.2M 12s 2024-06-28T15:27:02.066 INFO:teuthology.orchestra.run.smithi178.stderr: 2550K .......... .......... .......... .......... .......... 2% 18.1M 12s 2024-06-28T15:27:02.069 INFO:teuthology.orchestra.run.smithi178.stderr: 2600K .......... .......... .......... .......... .......... 2% 16.6M 11s 2024-06-28T15:27:02.074 INFO:teuthology.orchestra.run.smithi178.stderr: 2650K .......... .......... .......... .......... .......... 2% 13.8M 11s 2024-06-28T15:27:02.078 INFO:teuthology.orchestra.run.smithi178.stderr: 2700K .......... .......... .......... .......... .......... 2% 11.3M 11s 2024-06-28T15:27:02.081 INFO:teuthology.orchestra.run.smithi178.stderr: 2750K .......... .......... .......... .......... .......... 3% 14.7M 11s 2024-06-28T15:27:02.083 INFO:teuthology.orchestra.run.smithi178.stderr: 2800K .......... .......... .......... .......... .......... 3% 18.9M 11s 2024-06-28T15:27:02.086 INFO:teuthology.orchestra.run.smithi178.stderr: 2850K .......... .......... .......... .......... .......... 3% 15.6M 11s 2024-06-28T15:27:02.092 INFO:teuthology.orchestra.run.smithi178.stderr: 2900K .......... .......... .......... .......... .......... 3% 8.22M 11s 2024-06-28T15:27:02.095 INFO:teuthology.orchestra.run.smithi178.stderr: 2950K .......... .......... .......... .......... .......... 3% 17.4M 11s 2024-06-28T15:27:02.098 INFO:teuthology.orchestra.run.smithi178.stderr: 3000K .......... .......... .......... .......... .......... 3% 15.9M 11s 2024-06-28T15:27:02.104 INFO:teuthology.orchestra.run.smithi178.stderr: 3050K .......... .......... .......... .......... .......... 3% 15.5M 11s 2024-06-28T15:27:02.106 INFO:teuthology.orchestra.run.smithi178.stderr: 3100K .......... .......... .......... .......... .......... 3% 9.75M 11s 2024-06-28T15:27:02.108 INFO:teuthology.orchestra.run.smithi178.stderr: 3150K .......... .......... .......... .......... .......... 3% 21.3M 11s 2024-06-28T15:27:02.112 INFO:teuthology.orchestra.run.smithi178.stderr: 3200K .......... .......... .......... .......... .......... 3% 14.7M 10s 2024-06-28T15:27:02.115 INFO:teuthology.orchestra.run.smithi178.stderr: 3250K .......... .......... .......... .......... .......... 3% 12.8M 10s 2024-06-28T15:27:02.120 INFO:teuthology.orchestra.run.smithi178.stderr: 3300K .......... .......... .......... .......... .......... 3% 10.3M 10s 2024-06-28T15:27:02.124 INFO:teuthology.orchestra.run.smithi178.stderr: 3350K .......... .......... .......... .......... .......... 3% 12.5M 10s 2024-06-28T15:27:02.129 INFO:teuthology.orchestra.run.smithi178.stderr: 3400K .......... .......... .......... .......... .......... 3% 10.9M 10s 2024-06-28T15:27:02.132 INFO:teuthology.orchestra.run.smithi178.stderr: 3450K .......... .......... .......... .......... .......... 3% 16.8M 10s 2024-06-28T15:27:02.134 INFO:teuthology.orchestra.run.smithi178.stderr: 3500K .......... .......... .......... .......... .......... 3% 23.8M 10s 2024-06-28T15:27:02.135 INFO:teuthology.orchestra.run.smithi178.stderr: 3550K .......... .......... .......... .......... .......... 3% 29.6M 10s 2024-06-28T15:27:02.142 INFO:teuthology.orchestra.run.smithi178.stderr: 3600K .......... .......... .......... .......... .......... 3% 7.83M 10s 2024-06-28T15:27:02.144 INFO:teuthology.orchestra.run.smithi178.stderr: 3650K .......... .......... .......... .......... .......... 3% 15.6M 10s 2024-06-28T15:27:02.148 INFO:teuthology.orchestra.run.smithi178.stderr: 3700K .......... .......... .......... .......... .......... 4% 14.8M 10s 2024-06-28T15:27:02.150 INFO:teuthology.orchestra.run.smithi178.stderr: 3750K .......... .......... .......... .......... .......... 4% 17.9M 10s 2024-06-28T15:27:02.155 INFO:teuthology.orchestra.run.smithi178.stderr: 3800K .......... .......... .......... .......... .......... 4% 9.96M 10s 2024-06-28T15:27:02.159 INFO:teuthology.orchestra.run.smithi178.stderr: 3850K .......... .......... .......... .......... .......... 4% 20.6M 10s 2024-06-28T15:27:02.162 INFO:teuthology.orchestra.run.smithi178.stderr: 3900K .......... .......... .......... .......... .......... 4% 13.4M 10s 2024-06-28T15:27:02.164 INFO:teuthology.orchestra.run.smithi178.stderr: 3950K .......... .......... .......... .......... .......... 4% 19.6M 10s 2024-06-28T15:27:02.169 INFO:teuthology.orchestra.run.smithi178.stderr: 4000K .......... .......... .......... .......... .......... 4% 10.2M 10s 2024-06-28T15:27:02.173 INFO:teuthology.orchestra.run.smithi178.stderr: 4050K .......... .......... .......... .......... .......... 4% 12.1M 10s 2024-06-28T15:27:02.175 INFO:teuthology.orchestra.run.smithi178.stderr: 4100K .......... .......... .......... .......... .......... 4% 25.9M 9s 2024-06-28T15:27:02.179 INFO:teuthology.orchestra.run.smithi178.stderr: 4150K .......... .......... .......... .......... .......... 4% 12.5M 9s 2024-06-28T15:27:02.181 INFO:teuthology.orchestra.run.smithi178.stderr: 4200K .......... .......... .......... .......... .......... 4% 17.0M 9s 2024-06-28T15:27:02.185 INFO:teuthology.orchestra.run.smithi178.stderr: 4250K .......... .......... .......... .......... .......... 4% 19.3M 9s 2024-06-28T15:27:02.187 INFO:teuthology.orchestra.run.smithi178.stderr: 4300K .......... .......... .......... .......... .......... 4% 15.2M 9s 2024-06-28T15:27:02.191 INFO:teuthology.orchestra.run.smithi178.stderr: 4350K .......... .......... .......... .......... .......... 4% 12.6M 9s 2024-06-28T15:27:02.194 INFO:teuthology.orchestra.run.smithi178.stderr: 4400K .......... .......... .......... .......... .......... 4% 16.7M 9s 2024-06-28T15:27:02.196 INFO:teuthology.orchestra.run.smithi178.stderr: 4450K .......... .......... .......... .......... .......... 4% 24.8M 9s 2024-06-28T15:27:02.199 INFO:teuthology.orchestra.run.smithi178.stderr: 4500K .......... .......... .......... .......... .......... 4% 19.3M 9s 2024-06-28T15:27:02.202 INFO:teuthology.orchestra.run.smithi178.stderr: 4550K .......... .......... .......... .......... .......... 4% 15.4M 9s 2024-06-28T15:27:02.204 INFO:teuthology.orchestra.run.smithi178.stderr: 4600K .......... .......... .......... .......... .......... 5% 17.2M 9s 2024-06-28T15:27:02.208 INFO:teuthology.orchestra.run.smithi178.stderr: 4650K .......... .......... .......... .......... .......... 5% 15.8M 9s 2024-06-28T15:27:02.211 INFO:teuthology.orchestra.run.smithi178.stderr: 4700K .......... .......... .......... .......... .......... 5% 12.7M 9s 2024-06-28T15:27:02.215 INFO:teuthology.orchestra.run.smithi178.stderr: 4750K .......... .......... .......... .......... .......... 5% 16.1M 9s 2024-06-28T15:27:02.218 INFO:teuthology.orchestra.run.smithi178.stderr: 4800K .......... .......... .......... .......... .......... 5% 14.1M 9s 2024-06-28T15:27:02.220 INFO:teuthology.orchestra.run.smithi178.stderr: 4850K .......... .......... .......... .......... .......... 5% 26.9M 9s 2024-06-28T15:27:02.223 INFO:teuthology.orchestra.run.smithi178.stderr: 4900K .......... .......... .......... .......... .......... 5% 16.6M 9s 2024-06-28T15:27:02.225 INFO:teuthology.orchestra.run.smithi178.stderr: 4950K .......... .......... .......... .......... .......... 5% 19.4M 9s 2024-06-28T15:27:02.227 INFO:teuthology.orchestra.run.smithi178.stderr: 5000K .......... .......... .......... .......... .......... 5% 31.3M 9s 2024-06-28T15:27:02.229 INFO:teuthology.orchestra.run.smithi178.stderr: 5050K .......... .......... .......... .......... .......... 5% 21.2M 9s 2024-06-28T15:27:02.232 INFO:teuthology.orchestra.run.smithi178.stderr: 5100K .......... .......... .......... .......... .......... 5% 14.8M 9s 2024-06-28T15:27:02.238 INFO:teuthology.orchestra.run.smithi178.stderr: 5150K .......... .......... .......... .......... .......... 5% 16.7M 8s 2024-06-28T15:27:02.238 INFO:teuthology.orchestra.run.smithi178.stderr: 5200K .......... .......... .......... .......... .......... 5% 22.8M 8s 2024-06-28T15:27:02.240 INFO:teuthology.orchestra.run.smithi178.stderr: 5250K .......... .......... .......... .......... .......... 5% 19.3M 8s 2024-06-28T15:27:02.243 INFO:teuthology.orchestra.run.smithi178.stderr: 5300K .......... .......... .......... .......... .......... 5% 17.7M 8s 2024-06-28T15:27:02.246 INFO:teuthology.orchestra.run.smithi178.stderr: 5350K .......... .......... .......... .......... .......... 5% 15.4M 8s 2024-06-28T15:27:02.248 INFO:teuthology.orchestra.run.smithi178.stderr: 5400K .......... .......... .......... .......... .......... 5% 28.0M 8s 2024-06-28T15:27:02.251 INFO:teuthology.orchestra.run.smithi178.stderr: 5450K .......... .......... .......... .......... .......... 5% 15.1M 8s 2024-06-28T15:27:02.253 INFO:teuthology.orchestra.run.smithi178.stderr: 5500K .......... .......... .......... .......... .......... 5% 19.9M 8s 2024-06-28T15:27:02.256 INFO:teuthology.orchestra.run.smithi178.stderr: 5550K .......... .......... .......... .......... .......... 6% 17.5M 8s 2024-06-28T15:27:02.258 INFO:teuthology.orchestra.run.smithi178.stderr: 5600K .......... .......... .......... .......... .......... 6% 20.5M 8s 2024-06-28T15:27:02.261 INFO:teuthology.orchestra.run.smithi178.stderr: 5650K .......... .......... .......... .......... .......... 6% 19.8M 8s 2024-06-28T15:27:02.264 INFO:teuthology.orchestra.run.smithi178.stderr: 5700K .......... .......... .......... .......... .......... 6% 15.1M 8s 2024-06-28T15:27:02.268 INFO:teuthology.orchestra.run.smithi178.stderr: 5750K .......... .......... .......... .......... .......... 6% 13.5M 8s 2024-06-28T15:27:02.270 INFO:teuthology.orchestra.run.smithi178.stderr: 5800K .......... .......... .......... .......... .......... 6% 21.8M 8s 2024-06-28T15:27:02.273 INFO:teuthology.orchestra.run.smithi178.stderr: 5850K .......... .......... .......... .......... .......... 6% 19.0M 8s 2024-06-28T15:27:02.276 INFO:teuthology.orchestra.run.smithi178.stderr: 5900K .......... .......... .......... .......... .......... 6% 16.0M 8s 2024-06-28T15:27:02.277 INFO:teuthology.orchestra.run.smithi178.stderr: 5950K .......... .......... .......... .......... .......... 6% 30.5M 8s 2024-06-28T15:27:02.280 INFO:teuthology.orchestra.run.smithi178.stderr: 6000K .......... .......... .......... .......... .......... 6% 15.3M 8s 2024-06-28T15:27:02.284 INFO:teuthology.orchestra.run.smithi178.stderr: 6050K .......... .......... .......... .......... .......... 6% 15.7M 8s 2024-06-28T15:27:02.285 INFO:teuthology.orchestra.run.smithi178.stderr: 6100K .......... .......... .......... .......... .......... 6% 31.5M 8s 2024-06-28T15:27:02.287 INFO:teuthology.orchestra.run.smithi178.stderr: 6150K .......... .......... .......... .......... .......... 6% 35.4M 8s 2024-06-28T15:27:02.289 INFO:teuthology.orchestra.run.smithi178.stderr: 6200K .......... .......... .......... .......... .......... 6% 18.3M 8s 2024-06-28T15:27:02.293 INFO:teuthology.orchestra.run.smithi178.stderr: 6250K .......... .......... .......... .......... .......... 6% 22.3M 8s 2024-06-28T15:27:02.295 INFO:teuthology.orchestra.run.smithi178.stderr: 6300K .......... .......... .......... .......... .......... 6% 13.2M 8s 2024-06-28T15:27:02.298 INFO:teuthology.orchestra.run.smithi178.stderr: 6350K .......... .......... .......... .......... .......... 6% 19.3M 8s 2024-06-28T15:27:02.299 INFO:teuthology.orchestra.run.smithi178.stderr: 6400K .......... .......... .......... .......... .......... 6% 41.5M 8s 2024-06-28T15:27:02.302 INFO:teuthology.orchestra.run.smithi178.stderr: 6450K .......... .......... .......... .......... .......... 6% 14.5M 8s 2024-06-28T15:27:02.305 INFO:teuthology.orchestra.run.smithi178.stderr: 6500K .......... .......... .......... .......... .......... 7% 18.5M 8s 2024-06-28T15:27:02.307 INFO:teuthology.orchestra.run.smithi178.stderr: 6550K .......... .......... .......... .......... .......... 7% 23.6M 8s 2024-06-28T15:27:02.309 INFO:teuthology.orchestra.run.smithi178.stderr: 6600K .......... .......... .......... .......... .......... 7% 23.5M 7s 2024-06-28T15:27:02.314 INFO:teuthology.orchestra.run.smithi178.stderr: 6650K .......... .......... .......... .......... .......... 7% 13.2M 7s 2024-06-28T15:27:02.316 INFO:teuthology.orchestra.run.smithi178.stderr: 6700K .......... .......... .......... .......... .......... 7% 13.2M 7s 2024-06-28T15:27:02.319 INFO:teuthology.orchestra.run.smithi178.stderr: 6750K .......... .......... .......... .......... .......... 7% 17.7M 7s 2024-06-28T15:27:02.322 INFO:teuthology.orchestra.run.smithi178.stderr: 6800K .......... .......... .......... .......... .......... 7% 18.5M 7s 2024-06-28T15:27:02.325 INFO:teuthology.orchestra.run.smithi178.stderr: 6850K .......... .......... .......... .......... .......... 7% 16.9M 7s 2024-06-28T15:27:02.329 INFO:teuthology.orchestra.run.smithi178.stderr: 6900K .......... .......... .......... .......... .......... 7% 11.6M 7s 2024-06-28T15:27:02.330 INFO:teuthology.orchestra.run.smithi178.stderr: 6950K .......... .......... .......... .......... .......... 7% 29.2M 7s 2024-06-28T15:27:02.332 INFO:teuthology.orchestra.run.smithi178.stderr: 7000K .......... .......... .......... .......... .......... 7% 28.7M 7s 2024-06-28T15:27:02.335 INFO:teuthology.orchestra.run.smithi178.stderr: 7050K .......... .......... .......... .......... .......... 7% 20.0M 7s 2024-06-28T15:27:02.337 INFO:teuthology.orchestra.run.smithi178.stderr: 7100K .......... .......... .......... .......... .......... 7% 22.4M 7s 2024-06-28T15:27:02.340 INFO:teuthology.orchestra.run.smithi178.stderr: 7150K .......... .......... .......... .......... .......... 7% 19.5M 7s 2024-06-28T15:27:02.341 INFO:teuthology.orchestra.run.smithi178.stderr: 7200K .......... .......... .......... .......... .......... 7% 30.4M 7s 2024-06-28T15:27:02.343 INFO:teuthology.orchestra.run.smithi178.stderr: 7250K .......... .......... .......... .......... .......... 7% 22.7M 7s 2024-06-28T15:27:02.346 INFO:teuthology.orchestra.run.smithi178.stderr: 7300K .......... .......... .......... .......... .......... 7% 19.1M 7s 2024-06-28T15:27:02.348 INFO:teuthology.orchestra.run.smithi178.stderr: 7350K .......... .......... .......... .......... .......... 7% 21.1M 7s 2024-06-28T15:27:02.351 INFO:teuthology.orchestra.run.smithi178.stderr: 7400K .......... .......... .......... .......... .......... 8% 16.9M 7s 2024-06-28T15:27:02.353 INFO:teuthology.orchestra.run.smithi178.stderr: 7450K .......... .......... .......... .......... .......... 8% 40.7M 7s 2024-06-28T15:27:02.355 INFO:teuthology.orchestra.run.smithi178.stderr: 7500K .......... .......... .......... .......... .......... 8% 19.5M 7s 2024-06-28T15:27:02.357 INFO:teuthology.orchestra.run.smithi178.stderr: 7550K .......... .......... .......... .......... .......... 8% 17.8M 7s 2024-06-28T15:27:02.360 INFO:teuthology.orchestra.run.smithi178.stderr: 7600K .......... .......... .......... .......... .......... 8% 18.9M 7s 2024-06-28T15:27:02.362 INFO:teuthology.orchestra.run.smithi178.stderr: 7650K .......... .......... .......... .......... .......... 8% 22.4M 7s 2024-06-28T15:27:02.363 INFO:teuthology.orchestra.run.smithi178.stderr: 7700K .......... .......... .......... .......... .......... 8% 34.5M 7s 2024-06-28T15:27:02.368 INFO:teuthology.orchestra.run.smithi178.stderr: 7750K .......... .......... .......... .......... .......... 8% 12.0M 7s 2024-06-28T15:27:02.371 INFO:teuthology.orchestra.run.smithi178.stderr: 7800K .......... .......... .......... .......... .......... 8% 14.8M 7s 2024-06-28T15:27:02.373 INFO:teuthology.orchestra.run.smithi178.stderr: 7850K .......... .......... .......... .......... .......... 8% 43.8M 7s 2024-06-28T15:27:02.375 INFO:teuthology.orchestra.run.smithi178.stderr: 7900K .......... .......... .......... .......... .......... 8% 17.4M 7s 2024-06-28T15:27:02.378 INFO:teuthology.orchestra.run.smithi178.stderr: 7950K .......... .......... .......... .......... .......... 8% 16.4M 7s 2024-06-28T15:27:02.380 INFO:teuthology.orchestra.run.smithi178.stderr: 8000K .......... .......... .......... .......... .......... 8% 23.3M 7s 2024-06-28T15:27:02.382 INFO:teuthology.orchestra.run.smithi178.stderr: 8050K .......... .......... .......... .......... .......... 8% 27.8M 7s 2024-06-28T15:27:02.384 INFO:teuthology.orchestra.run.smithi178.stderr: 8100K .......... .......... .......... .......... .......... 8% 20.8M 7s 2024-06-28T15:27:02.387 INFO:teuthology.orchestra.run.smithi178.stderr: 8150K .......... .......... .......... .......... .......... 8% 16.7M 7s 2024-06-28T15:27:02.388 INFO:teuthology.orchestra.run.smithi178.stderr: 8200K .......... .......... .......... .......... .......... 8% 27.8M 7s 2024-06-28T15:27:02.391 INFO:teuthology.orchestra.run.smithi178.stderr: 8250K .......... .......... .......... .......... .......... 8% 29.6M 7s 2024-06-28T15:27:02.392 INFO:teuthology.orchestra.run.smithi178.stderr: 8300K .......... .......... .......... .......... .......... 8% 24.3M 7s 2024-06-28T15:27:02.395 INFO:teuthology.orchestra.run.smithi178.stderr: 8350K .......... .......... .......... .......... .......... 9% 15.8M 7s 2024-06-28T15:27:02.398 INFO:teuthology.orchestra.run.smithi178.stderr: 8400K .......... .......... .......... .......... .......... 9% 20.1M 7s 2024-06-28T15:27:02.399 INFO:teuthology.orchestra.run.smithi178.stderr: 8450K .......... .......... .......... .......... .......... 9% 29.7M 7s 2024-06-28T15:27:02.401 INFO:teuthology.orchestra.run.smithi178.stderr: 8500K .......... .......... .......... .......... .......... 9% 30.7M 7s 2024-06-28T15:27:02.405 INFO:teuthology.orchestra.run.smithi178.stderr: 8550K .......... .......... .......... .......... .......... 9% 14.0M 7s 2024-06-28T15:27:02.407 INFO:teuthology.orchestra.run.smithi178.stderr: 8600K .......... .......... .......... .......... .......... 9% 21.4M 7s 2024-06-28T15:27:02.409 INFO:teuthology.orchestra.run.smithi178.stderr: 8650K .......... .......... .......... .......... .......... 9% 26.5M 7s 2024-06-28T15:27:02.413 INFO:teuthology.orchestra.run.smithi178.stderr: 8700K .......... .......... .......... .......... .......... 9% 11.9M 7s 2024-06-28T15:27:02.418 INFO:teuthology.orchestra.run.smithi178.stderr: 8750K .......... .......... .......... .......... .......... 9% 11.4M 7s 2024-06-28T15:27:02.420 INFO:teuthology.orchestra.run.smithi178.stderr: 8800K .......... .......... .......... .......... .......... 9% 31.0M 7s 2024-06-28T15:27:02.423 INFO:teuthology.orchestra.run.smithi178.stderr: 8850K .......... .......... .......... .......... .......... 9% 20.2M 6s 2024-06-28T15:27:02.423 INFO:teuthology.orchestra.run.smithi178.stderr: 8900K .......... .......... .......... .......... .......... 9% 35.3M 6s 2024-06-28T15:27:02.425 INFO:teuthology.orchestra.run.smithi178.stderr: 8950K .......... .......... .......... .......... .......... 9% 28.6M 6s 2024-06-28T15:27:02.427 INFO:teuthology.orchestra.run.smithi178.stderr: 9000K .......... .......... .......... .......... .......... 9% 24.1M 6s 2024-06-28T15:27:02.430 INFO:teuthology.orchestra.run.smithi178.stderr: 9050K .......... .......... .......... .......... .......... 9% 24.0M 6s 2024-06-28T15:27:02.432 INFO:teuthology.orchestra.run.smithi178.stderr: 9100K .......... .......... .......... .......... .......... 9% 11.6M 6s 2024-06-28T15:27:02.434 INFO:teuthology.orchestra.run.smithi178.stderr: 9150K .......... .......... .......... .......... .......... 9% 27.9M 6s 2024-06-28T15:27:02.437 INFO:teuthology.orchestra.run.smithi178.stderr: 9200K .......... .......... .......... .......... .......... 9% 19.7M 6s 2024-06-28T15:27:02.439 INFO:teuthology.orchestra.run.smithi178.stderr: 9250K .......... .......... .......... .......... .......... 10% 21.9M 6s 2024-06-28T15:27:02.451 INFO:teuthology.orchestra.run.smithi178.stderr: 9300K .......... .......... .......... .......... .......... 10% 21.6M 6s 2024-06-28T15:27:02.451 INFO:teuthology.orchestra.run.smithi178.stderr: 9350K .......... .......... .......... .......... .......... 10% 31.9M 6s 2024-06-28T15:27:02.451 INFO:teuthology.orchestra.run.smithi178.stderr: 9400K .......... .......... .......... .......... .......... 10% 22.2M 6s 2024-06-28T15:27:02.451 INFO:teuthology.orchestra.run.smithi178.stderr: 9450K .......... .......... .......... .......... .......... 10% 15.8M 6s 2024-06-28T15:27:02.451 INFO:teuthology.orchestra.run.smithi178.stderr: 9500K .......... .......... .......... .......... .......... 10% 21.7M 6s 2024-06-28T15:27:02.453 INFO:teuthology.orchestra.run.smithi178.stderr: 9550K .......... .......... .......... .......... .......... 10% 19.4M 6s 2024-06-28T15:27:02.454 INFO:teuthology.orchestra.run.smithi178.stderr: 9600K .......... .......... .......... .......... .......... 10% 32.6M 6s 2024-06-28T15:27:02.455 INFO:teuthology.orchestra.run.smithi178.stderr: 9650K .......... .......... .......... .......... .......... 10% 37.5M 6s 2024-06-28T15:27:02.459 INFO:teuthology.orchestra.run.smithi178.stderr: 9700K .......... .......... .......... .......... .......... 10% 13.7M 6s 2024-06-28T15:27:02.462 INFO:teuthology.orchestra.run.smithi178.stderr: 9750K .......... .......... .......... .......... .......... 10% 15.3M 6s 2024-06-28T15:27:02.464 INFO:teuthology.orchestra.run.smithi178.stderr: 9800K .......... .......... .......... .......... .......... 10% 24.8M 6s 2024-06-28T15:27:02.468 INFO:teuthology.orchestra.run.smithi178.stderr: 9850K .......... .......... .......... .......... .......... 10% 14.4M 6s 2024-06-28T15:27:02.470 INFO:teuthology.orchestra.run.smithi178.stderr: 9900K .......... .......... .......... .......... .......... 10% 25.2M 6s 2024-06-28T15:27:02.471 INFO:teuthology.orchestra.run.smithi178.stderr: 9950K .......... .......... .......... .......... .......... 10% 28.7M 6s 2024-06-28T15:27:02.474 INFO:teuthology.orchestra.run.smithi178.stderr: 10000K .......... .......... .......... .......... .......... 10% 19.7M 6s 2024-06-28T15:27:02.475 INFO:teuthology.orchestra.run.smithi178.stderr: 10050K .......... .......... .......... .......... .......... 10% 35.0M 6s 2024-06-28T15:27:02.479 INFO:teuthology.orchestra.run.smithi178.stderr: 10100K .......... .......... .......... .......... .......... 10% 13.4M 6s 2024-06-28T15:27:02.481 INFO:teuthology.orchestra.run.smithi178.stderr: 10150K .......... .......... .......... .......... .......... 10% 21.2M 6s 2024-06-28T15:27:02.483 INFO:teuthology.orchestra.run.smithi178.stderr: 10200K .......... .......... .......... .......... .......... 11% 27.2M 6s 2024-06-28T15:27:02.486 INFO:teuthology.orchestra.run.smithi178.stderr: 10250K .......... .......... .......... .......... .......... 11% 24.9M 6s 2024-06-28T15:27:02.488 INFO:teuthology.orchestra.run.smithi178.stderr: 10300K .......... .......... .......... .......... .......... 11% 21.3M 6s 2024-06-28T15:27:02.489 INFO:teuthology.orchestra.run.smithi178.stderr: 10350K .......... .......... .......... .......... .......... 11% 38.6M 6s 2024-06-28T15:27:02.490 INFO:teuthology.orchestra.run.smithi178.stderr: 10400K .......... .......... .......... .......... .......... 11% 26.4M 6s 2024-06-28T15:27:02.492 INFO:teuthology.orchestra.run.smithi178.stderr: 10450K .......... .......... .......... .......... .......... 11% 36.2M 6s 2024-06-28T15:27:02.494 INFO:teuthology.orchestra.run.smithi178.stderr: 10500K .......... .......... .......... .......... .......... 11% 22.8M 6s 2024-06-28T15:27:02.496 INFO:teuthology.orchestra.run.smithi178.stderr: 10550K .......... .......... .......... .......... .......... 11% 18.1M 6s 2024-06-28T15:27:02.499 INFO:teuthology.orchestra.run.smithi178.stderr: 10600K .......... .......... .......... .......... .......... 11% 17.7M 6s 2024-06-28T15:27:02.501 INFO:teuthology.orchestra.run.smithi178.stderr: 10650K .......... .......... .......... .......... .......... 11% 33.3M 6s 2024-06-28T15:27:02.504 INFO:teuthology.orchestra.run.smithi178.stderr: 10700K .......... .......... .......... .......... .......... 11% 16.2M 6s 2024-06-28T15:27:02.506 INFO:teuthology.orchestra.run.smithi178.stderr: 10750K .......... .......... .......... .......... .......... 11% 20.9M 6s 2024-06-28T15:27:02.509 INFO:teuthology.orchestra.run.smithi178.stderr: 10800K .......... .......... .......... .......... .......... 11% 16.9M 6s 2024-06-28T15:27:02.511 INFO:teuthology.orchestra.run.smithi178.stderr: 10850K .......... .......... .......... .......... .......... 11% 25.0M 6s 2024-06-28T15:27:02.513 INFO:teuthology.orchestra.run.smithi178.stderr: 10900K .......... .......... .......... .......... .......... 11% 21.3M 6s 2024-06-28T15:27:02.514 INFO:teuthology.orchestra.run.smithi178.stderr: 10950K .......... .......... .......... .......... .......... 11% 48.8M 6s 2024-06-28T15:27:02.516 INFO:teuthology.orchestra.run.smithi178.stderr: 11000K .......... .......... .......... .......... .......... 11% 24.7M 6s 2024-06-28T15:27:02.519 INFO:teuthology.orchestra.run.smithi178.stderr: 11050K .......... .......... .......... .......... .......... 11% 26.0M 6s 2024-06-28T15:27:02.521 INFO:teuthology.orchestra.run.smithi178.stderr: 11100K .......... .......... .......... .......... .......... 12% 15.2M 6s 2024-06-28T15:27:02.524 INFO:teuthology.orchestra.run.smithi178.stderr: 11150K .......... .......... .......... .......... .......... 12% 19.2M 6s 2024-06-28T15:27:02.526 INFO:teuthology.orchestra.run.smithi178.stderr: 11200K .......... .......... .......... .......... .......... 12% 17.8M 6s 2024-06-28T15:27:02.528 INFO:teuthology.orchestra.run.smithi178.stderr: 11250K .......... .......... .......... .......... .......... 12% 27.9M 6s 2024-06-28T15:27:02.529 INFO:teuthology.orchestra.run.smithi178.stderr: 11300K .......... .......... .......... .......... .......... 12% 39.2M 6s 2024-06-28T15:27:02.532 INFO:teuthology.orchestra.run.smithi178.stderr: 11350K .......... .......... .......... .......... .......... 12% 16.3M 6s 2024-06-28T15:27:02.533 INFO:teuthology.orchestra.run.smithi178.stderr: 11400K .......... .......... .......... .......... .......... 12% 53.9M 6s 2024-06-28T15:27:02.538 INFO:teuthology.orchestra.run.smithi178.stderr: 11450K .......... .......... .......... .......... .......... 12% 17.1M 6s 2024-06-28T15:27:02.539 INFO:teuthology.orchestra.run.smithi178.stderr: 11500K .......... .......... .......... .......... .......... 12% 20.4M 6s 2024-06-28T15:27:02.540 INFO:teuthology.orchestra.run.smithi178.stderr: 11550K .......... .......... .......... .......... .......... 12% 27.3M 6s 2024-06-28T15:27:02.544 INFO:teuthology.orchestra.run.smithi178.stderr: 11600K .......... .......... .......... .......... .......... 12% 20.6M 6s 2024-06-28T15:27:02.544 INFO:teuthology.orchestra.run.smithi178.stderr: 11650K .......... .......... .......... .......... .......... 12% 61.8M 6s 2024-06-28T15:27:02.546 INFO:teuthology.orchestra.run.smithi178.stderr: 11700K .......... .......... .......... .......... .......... 12% 17.0M 6s 2024-06-28T15:27:02.548 INFO:teuthology.orchestra.run.smithi178.stderr: 11750K .......... .......... .......... .......... .......... 12% 35.3M 6s 2024-06-28T15:27:02.550 INFO:teuthology.orchestra.run.smithi178.stderr: 11800K .......... .......... .......... .......... .......... 12% 20.0M 6s 2024-06-28T15:27:02.554 INFO:teuthology.orchestra.run.smithi178.stderr: 11850K .......... .......... .......... .......... .......... 12% 25.7M 6s 2024-06-28T15:27:02.557 INFO:teuthology.orchestra.run.smithi178.stderr: 11900K .......... .......... .......... .......... .......... 12% 17.2M 6s 2024-06-28T15:27:02.560 INFO:teuthology.orchestra.run.smithi178.stderr: 11950K .......... .......... .......... .......... .......... 12% 17.8M 6s 2024-06-28T15:27:02.561 INFO:teuthology.orchestra.run.smithi178.stderr: 12000K .......... .......... .......... .......... .......... 12% 50.8M 6s 2024-06-28T15:27:02.562 INFO:teuthology.orchestra.run.smithi178.stderr: 12050K .......... .......... .......... .......... .......... 13% 18.7M 6s 2024-06-28T15:27:02.563 INFO:teuthology.orchestra.run.smithi178.stderr: 12100K .......... .......... .......... .......... .......... 13% 34.8M 6s 2024-06-28T15:27:02.565 INFO:teuthology.orchestra.run.smithi178.stderr: 12150K .......... .......... .......... .......... .......... 13% 21.7M 6s 2024-06-28T15:27:02.568 INFO:teuthology.orchestra.run.smithi178.stderr: 12200K .......... .......... .......... .......... .......... 13% 20.0M 5s 2024-06-28T15:27:02.571 INFO:teuthology.orchestra.run.smithi178.stderr: 12250K .......... .......... .......... .......... .......... 13% 15.3M 5s 2024-06-28T15:27:02.574 INFO:teuthology.orchestra.run.smithi178.stderr: 12300K .......... .......... .......... .......... .......... 13% 15.5M 5s 2024-06-28T15:27:02.578 INFO:teuthology.orchestra.run.smithi178.stderr: 12350K .......... .......... .......... .......... .......... 13% 31.0M 5s 2024-06-28T15:27:02.578 INFO:teuthology.orchestra.run.smithi178.stderr: 12400K .......... .......... .......... .......... .......... 13% 32.7M 5s 2024-06-28T15:27:02.579 INFO:teuthology.orchestra.run.smithi178.stderr: 12450K .......... .......... .......... .......... .......... 13% 31.7M 5s 2024-06-28T15:27:02.581 INFO:teuthology.orchestra.run.smithi178.stderr: 12500K .......... .......... .......... .......... .......... 13% 21.5M 5s 2024-06-28T15:27:02.583 INFO:teuthology.orchestra.run.smithi178.stderr: 12550K .......... .......... .......... .......... .......... 13% 18.1M 5s 2024-06-28T15:27:02.585 INFO:teuthology.orchestra.run.smithi178.stderr: 12600K .......... .......... .......... .......... .......... 13% 29.4M 5s 2024-06-28T15:27:02.587 INFO:teuthology.orchestra.run.smithi178.stderr: 12650K .......... .......... .......... .......... .......... 13% 26.9M 5s 2024-06-28T15:27:02.589 INFO:teuthology.orchestra.run.smithi178.stderr: 12700K .......... .......... .......... .......... .......... 13% 20.9M 5s 2024-06-28T15:27:02.591 INFO:teuthology.orchestra.run.smithi178.stderr: 12750K .......... .......... .......... .......... .......... 13% 31.4M 5s 2024-06-28T15:27:02.593 INFO:teuthology.orchestra.run.smithi178.stderr: 12800K .......... .......... .......... .......... .......... 13% 22.5M 5s 2024-06-28T15:27:02.594 INFO:teuthology.orchestra.run.smithi178.stderr: 12850K .......... .......... .......... .......... .......... 13% 26.6M 5s 2024-06-28T15:27:02.596 INFO:teuthology.orchestra.run.smithi178.stderr: 12900K .......... .......... .......... .......... .......... 13% 36.0M 5s 2024-06-28T15:27:02.599 INFO:teuthology.orchestra.run.smithi178.stderr: 12950K .......... .......... .......... .......... .......... 13% 16.9M 5s 2024-06-28T15:27:02.601 INFO:teuthology.orchestra.run.smithi178.stderr: 13000K .......... .......... .......... .......... .......... 14% 25.2M 5s 2024-06-28T15:27:02.603 INFO:teuthology.orchestra.run.smithi178.stderr: 13050K .......... .......... .......... .......... .......... 14% 25.2M 5s 2024-06-28T15:27:02.605 INFO:teuthology.orchestra.run.smithi178.stderr: 13100K .......... .......... .......... .......... .......... 14% 21.2M 5s 2024-06-28T15:27:02.606 INFO:teuthology.orchestra.run.smithi178.stderr: 13150K .......... .......... .......... .......... .......... 14% 54.2M 5s 2024-06-28T15:27:02.609 INFO:teuthology.orchestra.run.smithi178.stderr: 13200K .......... .......... .......... .......... .......... 14% 17.5M 5s 2024-06-28T15:27:02.611 INFO:teuthology.orchestra.run.smithi178.stderr: 13250K .......... .......... .......... .......... .......... 14% 25.0M 5s 2024-06-28T15:27:02.613 INFO:teuthology.orchestra.run.smithi178.stderr: 13300K .......... .......... .......... .......... .......... 14% 25.1M 5s 2024-06-28T15:27:02.615 INFO:teuthology.orchestra.run.smithi178.stderr: 13350K .......... .......... .......... .......... .......... 14% 22.0M 5s 2024-06-28T15:27:02.617 INFO:teuthology.orchestra.run.smithi178.stderr: 13400K .......... .......... .......... .......... .......... 14% 26.1M 5s 2024-06-28T15:27:02.620 INFO:teuthology.orchestra.run.smithi178.stderr: 13450K .......... .......... .......... .......... .......... 14% 19.7M 5s 2024-06-28T15:27:02.623 INFO:teuthology.orchestra.run.smithi178.stderr: 13500K .......... .......... .......... .......... .......... 14% 12.0M 5s 2024-06-28T15:27:02.624 INFO:teuthology.orchestra.run.smithi178.stderr: 13550K .......... .......... .......... .......... .......... 14% 54.9M 5s 2024-06-28T15:27:02.626 INFO:teuthology.orchestra.run.smithi178.stderr: 13600K .......... .......... .......... .......... .......... 14% 22.2M 5s 2024-06-28T15:27:02.629 INFO:teuthology.orchestra.run.smithi178.stderr: 13650K .......... .......... .......... .......... .......... 14% 16.6M 5s 2024-06-28T15:27:02.632 INFO:teuthology.orchestra.run.smithi178.stderr: 13700K .......... .......... .......... .......... .......... 14% 22.3M 5s 2024-06-28T15:27:02.632 INFO:teuthology.orchestra.run.smithi178.stderr: 13750K .......... .......... .......... .......... .......... 14% 47.9M 5s 2024-06-28T15:27:02.636 INFO:teuthology.orchestra.run.smithi178.stderr: 13800K .......... .......... .......... .......... .......... 14% 18.0M 5s 2024-06-28T15:27:02.637 INFO:teuthology.orchestra.run.smithi178.stderr: 13850K .......... .......... .......... .......... .......... 14% 73.2M 5s 2024-06-28T15:27:02.639 INFO:teuthology.orchestra.run.smithi178.stderr: 13900K .......... .......... .......... .......... .......... 15% 20.2M 5s 2024-06-28T15:27:02.641 INFO:teuthology.orchestra.run.smithi178.stderr: 13950K .......... .......... .......... .......... .......... 15% 20.5M 5s 2024-06-28T15:27:02.642 INFO:teuthology.orchestra.run.smithi178.stderr: 14000K .......... .......... .......... .......... .......... 15% 30.1M 5s 2024-06-28T15:27:02.646 INFO:teuthology.orchestra.run.smithi178.stderr: 14050K .......... .......... .......... .......... .......... 15% 12.8M 5s 2024-06-28T15:27:02.647 INFO:teuthology.orchestra.run.smithi178.stderr: 14100K .......... .......... .......... .......... .......... 15% 38.8M 5s 2024-06-28T15:27:02.649 INFO:teuthology.orchestra.run.smithi178.stderr: 14150K .......... .......... .......... .......... .......... 15% 24.9M 5s 2024-06-28T15:27:02.651 INFO:teuthology.orchestra.run.smithi178.stderr: 14200K .......... .......... .......... .......... .......... 15% 22.1M 5s 2024-06-28T15:27:02.654 INFO:teuthology.orchestra.run.smithi178.stderr: 14250K .......... .......... .......... .......... .......... 15% 24.6M 5s 2024-06-28T15:27:02.656 INFO:teuthology.orchestra.run.smithi178.stderr: 14300K .......... .......... .......... .......... .......... 15% 23.7M 5s 2024-06-28T15:27:02.657 INFO:teuthology.orchestra.run.smithi178.stderr: 14350K .......... .......... .......... .......... .......... 15% 51.6M 5s 2024-06-28T15:27:02.659 INFO:teuthology.orchestra.run.smithi178.stderr: 14400K .......... .......... .......... .......... .......... 15% 21.6M 5s 2024-06-28T15:27:02.660 INFO:teuthology.orchestra.run.smithi178.stderr: 14450K .......... .......... .......... .......... .......... 15% 37.5M 5s 2024-06-28T15:27:02.663 INFO:teuthology.orchestra.run.smithi178.stderr: 14500K .......... .......... .......... .......... .......... 15% 19.3M 5s 2024-06-28T15:27:02.665 INFO:teuthology.orchestra.run.smithi178.stderr: 14550K .......... .......... .......... .......... .......... 15% 28.0M 5s 2024-06-28T15:27:02.667 INFO:teuthology.orchestra.run.smithi178.stderr: 14600K .......... .......... .......... .......... .......... 15% 20.8M 5s 2024-06-28T15:27:02.669 INFO:teuthology.orchestra.run.smithi178.stderr: 14650K .......... .......... .......... .......... .......... 15% 37.4M 5s 2024-06-28T15:27:02.671 INFO:teuthology.orchestra.run.smithi178.stderr: 14700K .......... .......... .......... .......... .......... 15% 14.7M 5s 2024-06-28T15:27:02.673 INFO:teuthology.orchestra.run.smithi178.stderr: 14750K .......... .......... .......... .......... .......... 15% 34.3M 5s 2024-06-28T15:27:02.674 INFO:teuthology.orchestra.run.smithi178.stderr: 14800K .......... .......... .......... .......... .......... 15% 37.1M 5s 2024-06-28T15:27:02.677 INFO:teuthology.orchestra.run.smithi178.stderr: 14850K .......... .......... .......... .......... .......... 16% 20.4M 5s 2024-06-28T15:27:02.680 INFO:teuthology.orchestra.run.smithi178.stderr: 14900K .......... .......... .......... .......... .......... 16% 15.2M 5s 2024-06-28T15:27:02.682 INFO:teuthology.orchestra.run.smithi178.stderr: 14950K .......... .......... .......... .......... .......... 16% 25.5M 5s 2024-06-28T15:27:02.683 INFO:teuthology.orchestra.run.smithi178.stderr: 15000K .......... .......... .......... .......... .......... 16% 36.2M 5s 2024-06-28T15:27:02.686 INFO:teuthology.orchestra.run.smithi178.stderr: 15050K .......... .......... .......... .......... .......... 16% 20.6M 5s 2024-06-28T15:27:02.687 INFO:teuthology.orchestra.run.smithi178.stderr: 15100K .......... .......... .......... .......... .......... 16% 35.7M 5s 2024-06-28T15:27:02.688 INFO:teuthology.orchestra.run.smithi178.stderr: 15150K .......... .......... .......... .......... .......... 16% 31.8M 5s 2024-06-28T15:27:02.694 INFO:teuthology.orchestra.run.smithi178.stderr: 15200K .......... .......... .......... .......... .......... 16% 8.84M 5s 2024-06-28T15:27:02.696 INFO:teuthology.orchestra.run.smithi178.stderr: 15250K .......... .......... .......... .......... .......... 16% 24.5M 5s 2024-06-28T15:27:02.699 INFO:teuthology.orchestra.run.smithi178.stderr: 15300K .......... .......... .......... .......... .......... 16% 15.5M 5s 2024-06-28T15:27:02.700 INFO:teuthology.orchestra.run.smithi178.stderr: 15350K .......... .......... .......... .......... .......... 16% 38.3M 5s 2024-06-28T15:27:02.702 INFO:teuthology.orchestra.run.smithi178.stderr: 15400K .......... .......... .......... .......... .......... 16% 44.0M 5s 2024-06-28T15:27:02.703 INFO:teuthology.orchestra.run.smithi178.stderr: 15450K .......... .......... .......... .......... .......... 16% 25.6M 5s 2024-06-28T15:27:02.705 INFO:teuthology.orchestra.run.smithi178.stderr: 15500K .......... .......... .......... .......... .......... 16% 28.2M 5s 2024-06-28T15:27:02.707 INFO:teuthology.orchestra.run.smithi178.stderr: 15550K .......... .......... .......... .......... .......... 16% 22.6M 5s 2024-06-28T15:27:02.709 INFO:teuthology.orchestra.run.smithi178.stderr: 15600K .......... .......... .......... .......... .......... 16% 22.0M 5s 2024-06-28T15:27:02.710 INFO:teuthology.orchestra.run.smithi178.stderr: 15650K .......... .......... .......... .......... .......... 16% 38.3M 5s 2024-06-28T15:27:02.712 INFO:teuthology.orchestra.run.smithi178.stderr: 15700K .......... .......... .......... .......... .......... 16% 26.2M 5s 2024-06-28T15:27:02.715 INFO:teuthology.orchestra.run.smithi178.stderr: 15750K .......... .......... .......... .......... .......... 17% 21.5M 5s 2024-06-28T15:27:02.717 INFO:teuthology.orchestra.run.smithi178.stderr: 15800K .......... .......... .......... .......... .......... 17% 21.8M 5s 2024-06-28T15:27:02.719 INFO:teuthology.orchestra.run.smithi178.stderr: 15850K .......... .......... .......... .......... .......... 17% 23.2M 5s 2024-06-28T15:27:02.722 INFO:teuthology.orchestra.run.smithi178.stderr: 15900K .......... .......... .......... .......... .......... 17% 17.3M 5s 2024-06-28T15:27:02.724 INFO:teuthology.orchestra.run.smithi178.stderr: 15950K .......... .......... .......... .......... .......... 17% 25.6M 5s 2024-06-28T15:27:02.725 INFO:teuthology.orchestra.run.smithi178.stderr: 16000K .......... .......... .......... .......... .......... 17% 51.0M 5s 2024-06-28T15:27:02.727 INFO:teuthology.orchestra.run.smithi178.stderr: 16050K .......... .......... .......... .......... .......... 17% 28.4M 5s 2024-06-28T15:27:02.728 INFO:teuthology.orchestra.run.smithi178.stderr: 16100K .......... .......... .......... .......... .......... 17% 37.6M 5s 2024-06-28T15:27:02.731 INFO:teuthology.orchestra.run.smithi178.stderr: 16150K .......... .......... .......... .......... .......... 17% 37.6M 5s 2024-06-28T15:27:02.739 INFO:teuthology.orchestra.run.smithi178.stderr: 16200K .......... .......... .......... .......... .......... 17% 19.8M 5s 2024-06-28T15:27:02.739 INFO:teuthology.orchestra.run.smithi178.stderr: 16250K .......... .......... .......... .......... .......... 17% 32.4M 5s 2024-06-28T15:27:02.739 INFO:teuthology.orchestra.run.smithi178.stderr: 16300K .......... .......... .......... .......... .......... 17% 22.7M 5s 2024-06-28T15:27:02.739 INFO:teuthology.orchestra.run.smithi178.stderr: 16350K .......... .......... .......... .......... .......... 17% 40.2M 5s 2024-06-28T15:27:02.739 INFO:teuthology.orchestra.run.smithi178.stderr: 16400K .......... .......... .......... .......... .......... 17% 44.2M 5s 2024-06-28T15:27:02.739 INFO:teuthology.orchestra.run.smithi178.stderr: 16450K .......... .......... .......... .......... .......... 17% 68.8M 5s 2024-06-28T15:27:02.744 INFO:teuthology.orchestra.run.smithi178.stderr: 16500K .......... .......... .......... .......... .......... 17% 7.92M 5s 2024-06-28T15:27:02.746 INFO:teuthology.orchestra.run.smithi178.stderr: 16550K .......... .......... .......... .......... .......... 17% 57.0M 5s 2024-06-28T15:27:02.749 INFO:teuthology.orchestra.run.smithi178.stderr: 16600K .......... .......... .......... .......... .......... 17% 16.0M 5s 2024-06-28T15:27:02.752 INFO:teuthology.orchestra.run.smithi178.stderr: 16650K .......... .......... .......... .......... .......... 17% 36.6M 5s 2024-06-28T15:27:02.752 INFO:teuthology.orchestra.run.smithi178.stderr: 16700K .......... .......... .......... .......... .......... 18% 24.3M 5s 2024-06-28T15:27:02.754 INFO:teuthology.orchestra.run.smithi178.stderr: 16750K .......... .......... .......... .......... .......... 18% 31.1M 5s 2024-06-28T15:27:02.755 INFO:teuthology.orchestra.run.smithi178.stderr: 16800K .......... .......... .......... .......... .......... 18% 32.2M 5s 2024-06-28T15:27:02.757 INFO:teuthology.orchestra.run.smithi178.stderr: 16850K .......... .......... .......... .......... .......... 18% 21.8M 5s 2024-06-28T15:27:02.759 INFO:teuthology.orchestra.run.smithi178.stderr: 16900K .......... .......... .......... .......... .......... 18% 26.3M 5s 2024-06-28T15:27:02.762 INFO:teuthology.orchestra.run.smithi178.stderr: 16950K .......... .......... .......... .......... .......... 18% 16.5M 5s 2024-06-28T15:27:02.764 INFO:teuthology.orchestra.run.smithi178.stderr: 17000K .......... .......... .......... .......... .......... 18% 24.6M 5s 2024-06-28T15:27:02.767 INFO:teuthology.orchestra.run.smithi178.stderr: 17050K .......... .......... .......... .......... .......... 18% 14.8M 5s 2024-06-28T15:27:02.769 INFO:teuthology.orchestra.run.smithi178.stderr: 17100K .......... .......... .......... .......... .......... 18% 22.6M 5s 2024-06-28T15:27:02.771 INFO:teuthology.orchestra.run.smithi178.stderr: 17150K .......... .......... .......... .......... .......... 18% 34.9M 5s 2024-06-28T15:27:02.775 INFO:teuthology.orchestra.run.smithi178.stderr: 17200K .......... .......... .......... .......... .......... 18% 19.4M 5s 2024-06-28T15:27:02.775 INFO:teuthology.orchestra.run.smithi178.stderr: 17250K .......... .......... .......... .......... .......... 18% 47.9M 5s 2024-06-28T15:27:02.779 INFO:teuthology.orchestra.run.smithi178.stderr: 17300K .......... .......... .......... .......... .......... 18% 28.3M 5s 2024-06-28T15:27:02.779 INFO:teuthology.orchestra.run.smithi178.stderr: 17350K .......... .......... .......... .......... .......... 18% 83.1M 5s 2024-06-28T15:27:02.779 INFO:teuthology.orchestra.run.smithi178.stderr: 17400K .......... .......... .......... .......... .......... 18% 18.5M 5s 2024-06-28T15:27:02.782 INFO:teuthology.orchestra.run.smithi178.stderr: 17450K .......... .......... .......... .......... .......... 18% 44.5M 5s 2024-06-28T15:27:02.783 INFO:teuthology.orchestra.run.smithi178.stderr: 17500K .......... .......... .......... .......... .......... 18% 18.8M 5s 2024-06-28T15:27:02.785 INFO:teuthology.orchestra.run.smithi178.stderr: 17550K .......... .......... .......... .......... .......... 18% 22.8M 5s 2024-06-28T15:27:02.786 INFO:teuthology.orchestra.run.smithi178.stderr: 17600K .......... .......... .......... .......... .......... 18% 33.0M 4s 2024-06-28T15:27:02.788 INFO:teuthology.orchestra.run.smithi178.stderr: 17650K .......... .......... .......... .......... .......... 19% 29.5M 4s 2024-06-28T15:27:02.790 INFO:teuthology.orchestra.run.smithi178.stderr: 17700K .......... .......... .......... .......... .......... 19% 37.1M 4s 2024-06-28T15:27:02.791 INFO:teuthology.orchestra.run.smithi178.stderr: 17750K .......... .......... .......... .......... .......... 19% 36.0M 4s 2024-06-28T15:27:02.792 INFO:teuthology.orchestra.run.smithi178.stderr: 17800K .......... .......... .......... .......... .......... 19% 49.6M 4s 2024-06-28T15:27:02.794 INFO:teuthology.orchestra.run.smithi178.stderr: 17850K .......... .......... .......... .......... .......... 19% 26.4M 4s 2024-06-28T15:27:02.800 INFO:teuthology.orchestra.run.smithi178.stderr: 17900K .......... .......... .......... .......... .......... 19% 7.39M 4s 2024-06-28T15:27:02.803 INFO:teuthology.orchestra.run.smithi178.stderr: 17950K .......... .......... .......... .......... .......... 19% 19.8M 4s 2024-06-28T15:27:02.805 INFO:teuthology.orchestra.run.smithi178.stderr: 18000K .......... .......... .......... .......... .......... 19% 27.6M 4s 2024-06-28T15:27:02.805 INFO:teuthology.orchestra.run.smithi178.stderr: 18050K .......... .......... .......... .......... .......... 19% 60.5M 4s 2024-06-28T15:27:02.808 INFO:teuthology.orchestra.run.smithi178.stderr: 18100K .......... .......... .......... .......... .......... 19% 22.0M 4s 2024-06-28T15:27:02.808 INFO:teuthology.orchestra.run.smithi178.stderr: 18150K .......... .......... .......... .......... .......... 19% 66.7M 4s 2024-06-28T15:27:02.810 INFO:teuthology.orchestra.run.smithi178.stderr: 18200K .......... .......... .......... .......... .......... 19% 26.6M 4s 2024-06-28T15:27:02.813 INFO:teuthology.orchestra.run.smithi178.stderr: 18250K .......... .......... .......... .......... .......... 19% 18.7M 4s 2024-06-28T15:27:02.814 INFO:teuthology.orchestra.run.smithi178.stderr: 18300K .......... .......... .......... .......... .......... 19% 26.1M 4s 2024-06-28T15:27:02.819 INFO:teuthology.orchestra.run.smithi178.stderr: 18350K .......... .......... .......... .......... .......... 19% 14.4M 4s 2024-06-28T15:27:02.819 INFO:teuthology.orchestra.run.smithi178.stderr: 18400K .......... .......... .......... .......... .......... 19% 79.0M 4s 2024-06-28T15:27:02.822 INFO:teuthology.orchestra.run.smithi178.stderr: 18450K .......... .......... .......... .......... .......... 19% 15.1M 4s 2024-06-28T15:27:02.823 INFO:teuthology.orchestra.run.smithi178.stderr: 18500K .......... .......... .......... .......... .......... 19% 57.1M 4s 2024-06-28T15:27:02.826 INFO:teuthology.orchestra.run.smithi178.stderr: 18550K .......... .......... .......... .......... .......... 20% 14.2M 4s 2024-06-28T15:27:02.832 INFO:teuthology.orchestra.run.smithi178.stderr: 18600K .......... .......... .......... .......... .......... 20% 11.4M 4s 2024-06-28T15:27:02.832 INFO:teuthology.orchestra.run.smithi178.stderr: 18650K .......... .......... .......... .......... .......... 20% 65.7M 4s 2024-06-28T15:27:02.832 INFO:teuthology.orchestra.run.smithi178.stderr: 18700K .......... .......... .......... .......... .......... 20% 36.3M 4s 2024-06-28T15:27:02.834 INFO:teuthology.orchestra.run.smithi178.stderr: 18750K .......... .......... .......... .......... .......... 20% 40.5M 4s 2024-06-28T15:27:02.836 INFO:teuthology.orchestra.run.smithi178.stderr: 18800K .......... .......... .......... .......... .......... 20% 20.7M 4s 2024-06-28T15:27:02.838 INFO:teuthology.orchestra.run.smithi178.stderr: 18850K .......... .......... .......... .......... .......... 20% 30.5M 4s 2024-06-28T15:27:02.839 INFO:teuthology.orchestra.run.smithi178.stderr: 18900K .......... .......... .......... .......... .......... 20% 44.8M 4s 2024-06-28T15:27:02.840 INFO:teuthology.orchestra.run.smithi178.stderr: 18950K .......... .......... .......... .......... .......... 20% 35.3M 4s 2024-06-28T15:27:02.841 INFO:teuthology.orchestra.run.smithi178.stderr: 19000K .......... .......... .......... .......... .......... 20% 44.0M 4s 2024-06-28T15:27:02.843 INFO:teuthology.orchestra.run.smithi178.stderr: 19050K .......... .......... .......... .......... .......... 20% 34.8M 4s 2024-06-28T15:27:02.844 INFO:teuthology.orchestra.run.smithi178.stderr: 19100K .......... .......... .......... .......... .......... 20% 38.7M 4s 2024-06-28T15:27:02.845 INFO:teuthology.orchestra.run.smithi178.stderr: 19150K .......... .......... .......... .......... .......... 20% 55.5M 4s 2024-06-28T15:27:02.846 INFO:teuthology.orchestra.run.smithi178.stderr: 19200K .......... .......... .......... .......... .......... 20% 37.6M 4s 2024-06-28T15:27:02.847 INFO:teuthology.orchestra.run.smithi178.stderr: 19250K .......... .......... .......... .......... .......... 20% 52.4M 4s 2024-06-28T15:27:02.856 INFO:teuthology.orchestra.run.smithi178.stderr: 19300K .......... .......... .......... .......... .......... 20% 5.56M 4s 2024-06-28T15:27:02.856 INFO:teuthology.orchestra.run.smithi178.stderr: 19350K .......... .......... .......... .......... .......... 20% 50.6M 4s 2024-06-28T15:27:02.859 INFO:teuthology.orchestra.run.smithi178.stderr: 19400K .......... .......... .......... .......... .......... 20% 21.6M 4s 2024-06-28T15:27:02.860 INFO:teuthology.orchestra.run.smithi178.stderr: 19450K .......... .......... .......... .......... .......... 20% 51.0M 4s 2024-06-28T15:27:02.862 INFO:teuthology.orchestra.run.smithi178.stderr: 19500K .......... .......... .......... .......... .......... 21% 24.8M 4s 2024-06-28T15:27:02.863 INFO:teuthology.orchestra.run.smithi178.stderr: 19550K .......... .......... .......... .......... .......... 21% 42.9M 4s 2024-06-28T15:27:02.864 INFO:teuthology.orchestra.run.smithi178.stderr: 19600K .......... .......... .......... .......... .......... 21% 59.8M 4s 2024-06-28T15:27:02.865 INFO:teuthology.orchestra.run.smithi178.stderr: 19650K .......... .......... .......... .......... .......... 21% 41.6M 4s 2024-06-28T15:27:02.867 INFO:teuthology.orchestra.run.smithi178.stderr: 19700K .......... .......... .......... .......... .......... 21% 24.9M 4s 2024-06-28T15:27:02.869 INFO:teuthology.orchestra.run.smithi178.stderr: 19750K .......... .......... .......... .......... .......... 21% 22.6M 4s 2024-06-28T15:27:02.870 INFO:teuthology.orchestra.run.smithi178.stderr: 19800K .......... .......... .......... .......... .......... 21% 43.5M 4s 2024-06-28T15:27:02.874 INFO:teuthology.orchestra.run.smithi178.stderr: 19850K .......... .......... .......... .......... .......... 21% 14.2M 4s 2024-06-28T15:27:02.876 INFO:teuthology.orchestra.run.smithi178.stderr: 19900K .......... .......... .......... .......... .......... 21% 21.4M 4s 2024-06-28T15:27:02.877 INFO:teuthology.orchestra.run.smithi178.stderr: 19950K .......... .......... .......... .......... .......... 21% 53.4M 4s 2024-06-28T15:27:02.878 INFO:teuthology.orchestra.run.smithi178.stderr: 20000K .......... .......... .......... .......... .......... 21% 36.2M 4s 2024-06-28T15:27:02.880 INFO:teuthology.orchestra.run.smithi178.stderr: 20050K .......... .......... .......... .......... .......... 21% 26.1M 4s 2024-06-28T15:27:02.886 INFO:teuthology.orchestra.run.smithi178.stderr: 20100K .......... .......... .......... .......... .......... 21% 8.45M 4s 2024-06-28T15:27:02.887 INFO:teuthology.orchestra.run.smithi178.stderr: 20150K .......... .......... .......... .......... .......... 21% 41.3M 4s 2024-06-28T15:27:02.889 INFO:teuthology.orchestra.run.smithi178.stderr: 20200K .......... .......... .......... .......... .......... 21% 21.0M 4s 2024-06-28T15:27:02.891 INFO:teuthology.orchestra.run.smithi178.stderr: 20250K .......... .......... .......... .......... .......... 21% 51.2M 4s 2024-06-28T15:27:02.892 INFO:teuthology.orchestra.run.smithi178.stderr: 20300K .......... .......... .......... .......... .......... 21% 26.8M 4s 2024-06-28T15:27:02.893 INFO:teuthology.orchestra.run.smithi178.stderr: 20350K .......... .......... .......... .......... .......... 21% 41.3M 4s 2024-06-28T15:27:02.894 INFO:teuthology.orchestra.run.smithi178.stderr: 20400K .......... .......... .......... .......... .......... 22% 40.9M 4s 2024-06-28T15:27:02.896 INFO:teuthology.orchestra.run.smithi178.stderr: 20450K .......... .......... .......... .......... .......... 22% 35.8M 4s 2024-06-28T15:27:02.897 INFO:teuthology.orchestra.run.smithi178.stderr: 20500K .......... .......... .......... .......... .......... 22% 59.9M 4s 2024-06-28T15:27:02.898 INFO:teuthology.orchestra.run.smithi178.stderr: 20550K .......... .......... .......... .......... .......... 22% 34.6M 4s 2024-06-28T15:27:02.900 INFO:teuthology.orchestra.run.smithi178.stderr: 20600K .......... .......... .......... .......... .......... 22% 20.7M 4s 2024-06-28T15:27:02.902 INFO:teuthology.orchestra.run.smithi178.stderr: 20650K .......... .......... .......... .......... .......... 22% 28.5M 4s 2024-06-28T15:27:02.905 INFO:teuthology.orchestra.run.smithi178.stderr: 20700K .......... .......... .......... .......... .......... 22% 19.4M 4s 2024-06-28T15:27:02.906 INFO:teuthology.orchestra.run.smithi178.stderr: 20750K .......... .......... .......... .......... .......... 22% 37.1M 4s 2024-06-28T15:27:02.908 INFO:teuthology.orchestra.run.smithi178.stderr: 20800K .......... .......... .......... .......... .......... 22% 21.1M 4s 2024-06-28T15:27:02.911 INFO:teuthology.orchestra.run.smithi178.stderr: 20850K .......... .......... .......... .......... .......... 22% 17.7M 4s 2024-06-28T15:27:02.913 INFO:teuthology.orchestra.run.smithi178.stderr: 20900K .......... .......... .......... .......... .......... 22% 31.5M 4s 2024-06-28T15:27:02.914 INFO:teuthology.orchestra.run.smithi178.stderr: 20950K .......... .......... .......... .......... .......... 22% 42.7M 4s 2024-06-28T15:27:02.915 INFO:teuthology.orchestra.run.smithi178.stderr: 21000K .......... .......... .......... .......... .......... 22% 35.7M 4s 2024-06-28T15:27:02.918 INFO:teuthology.orchestra.run.smithi178.stderr: 21050K .......... .......... .......... .......... .......... 22% 30.4M 4s 2024-06-28T15:27:02.919 INFO:teuthology.orchestra.run.smithi178.stderr: 21100K .......... .......... .......... .......... .......... 22% 20.8M 4s 2024-06-28T15:27:02.921 INFO:teuthology.orchestra.run.smithi178.stderr: 21150K .......... .......... .......... .......... .......... 22% 27.0M 4s 2024-06-28T15:27:02.922 INFO:teuthology.orchestra.run.smithi178.stderr: 21200K .......... .......... .......... .......... .......... 22% 46.4M 4s 2024-06-28T15:27:02.924 INFO:teuthology.orchestra.run.smithi178.stderr: 21250K .......... .......... .......... .......... .......... 22% 22.4M 4s 2024-06-28T15:27:02.925 INFO:teuthology.orchestra.run.smithi178.stderr: 21300K .......... .......... .......... .......... .......... 22% 35.4M 4s 2024-06-28T15:27:02.929 INFO:teuthology.orchestra.run.smithi178.stderr: 21350K .......... .......... .......... .......... .......... 23% 17.4M 4s 2024-06-28T15:27:02.929 INFO:teuthology.orchestra.run.smithi178.stderr: 21400K .......... .......... .......... .......... .......... 23% 79.6M 4s 2024-06-28T15:27:02.931 INFO:teuthology.orchestra.run.smithi178.stderr: 21450K .......... .......... .......... .......... .......... 23% 29.6M 4s 2024-06-28T15:27:02.933 INFO:teuthology.orchestra.run.smithi178.stderr: 21500K .......... .......... .......... .......... .......... 23% 21.9M 4s 2024-06-28T15:27:02.935 INFO:teuthology.orchestra.run.smithi178.stderr: 21550K .......... .......... .......... .......... .......... 23% 35.5M 4s 2024-06-28T15:27:02.937 INFO:teuthology.orchestra.run.smithi178.stderr: 21600K .......... .......... .......... .......... .......... 23% 22.5M 4s 2024-06-28T15:27:02.942 INFO:teuthology.orchestra.run.smithi178.stderr: 21650K .......... .......... .......... .......... .......... 23% 9.99M 4s 2024-06-28T15:27:02.942 INFO:teuthology.orchestra.run.smithi178.stderr: 21700K .......... .......... .......... .......... .......... 23% 90.5M 4s 2024-06-28T15:27:02.944 INFO:teuthology.orchestra.run.smithi178.stderr: 21750K .......... .......... .......... .......... .......... 23% 21.8M 4s 2024-06-28T15:27:02.945 INFO:teuthology.orchestra.run.smithi178.stderr: 21800K .......... .......... .......... .......... .......... 23% 63.3M 4s 2024-06-28T15:27:02.946 INFO:teuthology.orchestra.run.smithi178.stderr: 21850K .......... .......... .......... .......... .......... 23% 32.1M 4s 2024-06-28T15:27:02.948 INFO:teuthology.orchestra.run.smithi178.stderr: 21900K .......... .......... .......... .......... .......... 23% 26.3M 4s 2024-06-28T15:27:02.949 INFO:teuthology.orchestra.run.smithi178.stderr: 21950K .......... .......... .......... .......... .......... 23% 50.6M 4s 2024-06-28T15:27:02.950 INFO:teuthology.orchestra.run.smithi178.stderr: 22000K .......... .......... .......... .......... .......... 23% 48.3M 4s 2024-06-28T15:27:02.951 INFO:teuthology.orchestra.run.smithi178.stderr: 22050K .......... .......... .......... .......... .......... 23% 39.8M 4s 2024-06-28T15:27:02.953 INFO:teuthology.orchestra.run.smithi178.stderr: 22100K .......... .......... .......... .......... .......... 23% 33.8M 4s 2024-06-28T15:27:02.955 INFO:teuthology.orchestra.run.smithi178.stderr: 22150K .......... .......... .......... .......... .......... 23% 56.8M 4s 2024-06-28T15:27:02.955 INFO:teuthology.orchestra.run.smithi178.stderr: 22200K .......... .......... .......... .......... .......... 23% 46.8M 4s 2024-06-28T15:27:02.958 INFO:teuthology.orchestra.run.smithi178.stderr: 22250K .......... .......... .......... .......... .......... 24% 80.0M 4s 2024-06-28T15:27:02.958 INFO:teuthology.orchestra.run.smithi178.stderr: 22300K .......... .......... .......... .......... .......... 24% 19.8M 4s 2024-06-28T15:27:02.961 INFO:teuthology.orchestra.run.smithi178.stderr: 22350K .......... .......... .......... .......... .......... 24% 17.2M 4s 2024-06-28T15:27:02.962 INFO:teuthology.orchestra.run.smithi178.stderr: 22400K .......... .......... .......... .......... .......... 24% 40.2M 4s 2024-06-28T15:27:02.964 INFO:teuthology.orchestra.run.smithi178.stderr: 22450K .......... .......... .......... .......... .......... 24% 23.6M 4s 2024-06-28T15:27:02.966 INFO:teuthology.orchestra.run.smithi178.stderr: 22500K .......... .......... .......... .......... .......... 24% 27.2M 4s 2024-06-28T15:27:02.968 INFO:teuthology.orchestra.run.smithi178.stderr: 22550K .......... .......... .......... .......... .......... 24% 23.3M 4s 2024-06-28T15:27:02.969 INFO:teuthology.orchestra.run.smithi178.stderr: 22600K .......... .......... .......... .......... .......... 24% 34.6M 4s 2024-06-28T15:27:02.972 INFO:teuthology.orchestra.run.smithi178.stderr: 22650K .......... .......... .......... .......... .......... 24% 17.6M 4s 2024-06-28T15:27:02.974 INFO:teuthology.orchestra.run.smithi178.stderr: 22700K .......... .......... .......... .......... .......... 24% 25.6M 4s 2024-06-28T15:27:02.976 INFO:teuthology.orchestra.run.smithi178.stderr: 22750K .......... .......... .......... .......... .......... 24% 49.5M 4s 2024-06-28T15:27:02.976 INFO:teuthology.orchestra.run.smithi178.stderr: 22800K .......... .......... .......... .......... .......... 24% 35.5M 4s 2024-06-28T15:27:02.977 INFO:teuthology.orchestra.run.smithi178.stderr: 22850K .......... .......... .......... .......... .......... 24% 47.2M 4s 2024-06-28T15:27:02.978 INFO:teuthology.orchestra.run.smithi178.stderr: 22900K .......... .......... .......... .......... .......... 24% 35.7M 4s 2024-06-28T15:27:02.980 INFO:teuthology.orchestra.run.smithi178.stderr: 22950K .......... .......... .......... .......... .......... 24% 43.9M 4s 2024-06-28T15:27:02.982 INFO:teuthology.orchestra.run.smithi178.stderr: 23000K .......... .......... .......... .......... .......... 24% 23.5M 4s 2024-06-28T15:27:02.985 INFO:teuthology.orchestra.run.smithi178.stderr: 23050K .......... .......... .......... .......... .......... 24% 31.5M 4s 2024-06-28T15:27:02.987 INFO:teuthology.orchestra.run.smithi178.stderr: 23100K .......... .......... .......... .......... .......... 24% 17.7M 4s 2024-06-28T15:27:02.988 INFO:teuthology.orchestra.run.smithi178.stderr: 23150K .......... .......... .......... .......... .......... 24% 34.3M 4s 2024-06-28T15:27:02.989 INFO:teuthology.orchestra.run.smithi178.stderr: 23200K .......... .......... .......... .......... .......... 25% 35.3M 4s 2024-06-28T15:27:02.992 INFO:teuthology.orchestra.run.smithi178.stderr: 23250K .......... .......... .......... .......... .......... 25% 13.9M 4s 2024-06-28T15:27:02.995 INFO:teuthology.orchestra.run.smithi178.stderr: 23300K .......... .......... .......... .......... .......... 25% 31.3M 4s 2024-06-28T15:27:02.996 INFO:teuthology.orchestra.run.smithi178.stderr: 23350K .......... .......... .......... .......... .......... 25% 24.9M 4s 2024-06-28T15:27:02.997 INFO:teuthology.orchestra.run.smithi178.stderr: 23400K .......... .......... .......... .......... .......... 25% 41.3M 4s 2024-06-28T15:27:02.998 INFO:teuthology.orchestra.run.smithi178.stderr: 23450K .......... .......... .......... .......... .......... 25% 51.4M 4s 2024-06-28T15:27:02.999 INFO:teuthology.orchestra.run.smithi178.stderr: 23500K .......... .......... .......... .......... .......... 25% 38.3M 4s 2024-06-28T15:27:03.000 INFO:teuthology.orchestra.run.smithi178.stderr: 23550K .......... .......... .......... .......... .......... 25% 58.3M 4s 2024-06-28T15:27:03.004 INFO:teuthology.orchestra.run.smithi178.stderr: 23600K .......... .......... .......... .......... .......... 25% 28.7M 4s 2024-06-28T15:27:03.005 INFO:teuthology.orchestra.run.smithi178.stderr: 23650K .......... .......... .......... .......... .......... 25% 37.6M 4s 2024-06-28T15:27:03.005 INFO:teuthology.orchestra.run.smithi178.stderr: 23700K .......... .......... .......... .......... .......... 25% 24.4M 4s 2024-06-28T15:27:03.007 INFO:teuthology.orchestra.run.smithi178.stderr: 23750K .......... .......... .......... .......... .......... 25% 31.3M 4s 2024-06-28T15:27:03.009 INFO:teuthology.orchestra.run.smithi178.stderr: 23800K .......... .......... .......... .......... .......... 25% 26.7M 4s 2024-06-28T15:27:03.010 INFO:teuthology.orchestra.run.smithi178.stderr: 23850K .......... .......... .......... .......... .......... 25% 64.2M 4s 2024-06-28T15:27:03.011 INFO:teuthology.orchestra.run.smithi178.stderr: 23900K .......... .......... .......... .......... .......... 25% 30.1M 4s 2024-06-28T15:27:03.014 INFO:teuthology.orchestra.run.smithi178.stderr: 23950K .......... .......... .......... .......... .......... 25% 14.7M 4s 2024-06-28T15:27:03.016 INFO:teuthology.orchestra.run.smithi178.stderr: 24000K .......... .......... .......... .......... .......... 25% 46.5M 4s 2024-06-28T15:27:03.017 INFO:teuthology.orchestra.run.smithi178.stderr: 24050K .......... .......... .......... .......... .......... 25% 38.5M 4s 2024-06-28T15:27:03.018 INFO:teuthology.orchestra.run.smithi178.stderr: 24100K .......... .......... .......... .......... .......... 25% 41.0M 4s 2024-06-28T15:27:03.021 INFO:teuthology.orchestra.run.smithi178.stderr: 24150K .......... .......... .......... .......... .......... 26% 18.0M 4s 2024-06-28T15:27:03.021 INFO:teuthology.orchestra.run.smithi178.stderr: 24200K .......... .......... .......... .......... .......... 26% 61.3M 4s 2024-06-28T15:27:03.023 INFO:teuthology.orchestra.run.smithi178.stderr: 24250K .......... .......... .......... .......... .......... 26% 29.2M 4s 2024-06-28T15:27:03.026 INFO:teuthology.orchestra.run.smithi178.stderr: 24300K .......... .......... .......... .......... .......... 26% 15.0M 4s 2024-06-28T15:27:03.028 INFO:teuthology.orchestra.run.smithi178.stderr: 24350K .......... .......... .......... .......... .......... 26% 36.8M 4s 2024-06-28T15:27:03.029 INFO:teuthology.orchestra.run.smithi178.stderr: 24400K .......... .......... .......... .......... .......... 26% 30.3M 4s 2024-06-28T15:27:03.031 INFO:teuthology.orchestra.run.smithi178.stderr: 24450K .......... .......... .......... .......... .......... 26% 42.0M 4s 2024-06-28T15:27:03.031 INFO:teuthology.orchestra.run.smithi178.stderr: 24500K .......... .......... .......... .......... .......... 26% 48.0M 4s 2024-06-28T15:27:03.032 INFO:teuthology.orchestra.run.smithi178.stderr: 24550K .......... .......... .......... .......... .......... 26% 41.4M 4s 2024-06-28T15:27:03.035 INFO:teuthology.orchestra.run.smithi178.stderr: 24600K .......... .......... .......... .......... .......... 26% 24.3M 4s 2024-06-28T15:27:03.037 INFO:teuthology.orchestra.run.smithi178.stderr: 24650K .......... .......... .......... .......... .......... 26% 18.9M 4s 2024-06-28T15:27:03.039 INFO:teuthology.orchestra.run.smithi178.stderr: 24700K .......... .......... .......... .......... .......... 26% 23.5M 4s 2024-06-28T15:27:03.040 INFO:teuthology.orchestra.run.smithi178.stderr: 24750K .......... .......... .......... .......... .......... 26% 40.2M 4s 2024-06-28T15:27:03.042 INFO:teuthology.orchestra.run.smithi178.stderr: 24800K .......... .......... .......... .......... .......... 26% 37.7M 4s 2024-06-28T15:27:03.043 INFO:teuthology.orchestra.run.smithi178.stderr: 24850K .......... .......... .......... .......... .......... 26% 32.7M 4s 2024-06-28T15:27:03.045 INFO:teuthology.orchestra.run.smithi178.stderr: 24900K .......... .......... .......... .......... .......... 26% 28.2M 4s 2024-06-28T15:27:03.046 INFO:teuthology.orchestra.run.smithi178.stderr: 24950K .......... .......... .......... .......... .......... 26% 30.2M 4s 2024-06-28T15:27:03.050 INFO:teuthology.orchestra.run.smithi178.stderr: 25000K .......... .......... .......... .......... .......... 26% 14.7M 4s 2024-06-28T15:27:03.117 INFO:teuthology.orchestra.run.smithi178.stderr: 25050K .......... .......... .......... .......... .......... 27% 760K 4s 2024-06-28T15:27:03.117 INFO:teuthology.orchestra.run.smithi178.stderr: 25100K .......... .......... .......... .......... .......... 27% 107M 4s 2024-06-28T15:27:03.121 INFO:teuthology.orchestra.run.smithi178.stderr: 25150K .......... .......... .......... .......... .......... 27% 13.2M 4s 2024-06-28T15:27:03.121 INFO:teuthology.orchestra.run.smithi178.stderr: 25200K .......... .......... .......... .......... .......... 27% 127M 4s 2024-06-28T15:27:03.174 INFO:teuthology.orchestra.run.smithi178.stderr: 25250K .......... .......... .......... .......... .......... 27% 953K 4s 2024-06-28T15:27:03.175 INFO:teuthology.orchestra.run.smithi178.stderr: 25300K .......... .......... .......... .......... .......... 27% 113M 4s 2024-06-28T15:27:03.175 INFO:teuthology.orchestra.run.smithi178.stderr: 25350K .......... .......... .......... .......... .......... 27% 145M 4s 2024-06-28T15:27:03.180 INFO:teuthology.orchestra.run.smithi178.stderr: 25400K .......... .......... .......... .......... .......... 27% 129M 4s 2024-06-28T15:27:03.181 INFO:teuthology.orchestra.run.smithi178.stderr: 25450K .......... .......... .......... .......... .......... 27% 143M 4s 2024-06-28T15:27:03.181 INFO:teuthology.orchestra.run.smithi178.stderr: 25500K .......... .......... .......... .......... .......... 27% 119M 4s 2024-06-28T15:27:03.181 INFO:teuthology.orchestra.run.smithi178.stderr: 25550K .......... .......... .......... .......... .......... 27% 145M 4s 2024-06-28T15:27:03.181 INFO:teuthology.orchestra.run.smithi178.stderr: 25600K .......... .......... .......... .......... .......... 27% 146M 4s 2024-06-28T15:27:03.181 INFO:teuthology.orchestra.run.smithi178.stderr: 25650K .......... .......... .......... .......... .......... 27% 137M 4s 2024-06-28T15:27:03.181 INFO:teuthology.orchestra.run.smithi178.stderr: 25700K .......... .......... .......... .......... .......... 27% 133M 4s 2024-06-28T15:27:03.221 INFO:teuthology.orchestra.run.smithi178.stderr: 25750K .......... .......... .......... .......... .......... 27% 148M 4s 2024-06-28T15:27:03.221 INFO:teuthology.orchestra.run.smithi178.stderr: 25800K .......... .......... .......... .......... .......... 27% 147M 4s 2024-06-28T15:27:03.221 INFO:teuthology.orchestra.run.smithi178.stderr: 25850K .......... .......... .......... .......... .......... 27% 145M 4s 2024-06-28T15:27:03.221 INFO:teuthology.orchestra.run.smithi178.stderr: 25900K .......... .......... .......... .......... .......... 27% 121M 4s 2024-06-28T15:27:03.221 INFO:teuthology.orchestra.run.smithi178.stderr: 25950K .......... .......... .......... .......... .......... 27% 144M 4s 2024-06-28T15:27:03.221 INFO:teuthology.orchestra.run.smithi178.stderr: 26000K .......... .......... .......... .......... .......... 28% 145M 4s 2024-06-28T15:27:03.221 INFO:teuthology.orchestra.run.smithi178.stderr: 26050K .......... .......... .......... .......... .......... 28% 145M 4s 2024-06-28T15:27:03.221 INFO:teuthology.orchestra.run.smithi178.stderr: 26100K .......... .......... .......... .......... .......... 28% 126M 4s 2024-06-28T15:27:03.221 INFO:teuthology.orchestra.run.smithi178.stderr: 26150K .......... .......... .......... .......... .......... 28% 146M 4s 2024-06-28T15:27:03.221 INFO:teuthology.orchestra.run.smithi178.stderr: 26200K .......... .......... .......... .......... .......... 28% 145M 4s 2024-06-28T15:27:03.222 INFO:teuthology.orchestra.run.smithi178.stderr: 26250K .......... .......... .......... .......... .......... 28% 138M 4s 2024-06-28T15:27:03.222 INFO:teuthology.orchestra.run.smithi178.stderr: 26300K .......... .......... .......... .......... .......... 28% 121M 4s 2024-06-28T15:27:03.222 INFO:teuthology.orchestra.run.smithi178.stderr: 26350K .......... .......... .......... .......... .......... 28% 142M 4s 2024-06-28T15:27:03.222 INFO:teuthology.orchestra.run.smithi178.stderr: 26400K .......... .......... .......... .......... .......... 28% 143M 4s 2024-06-28T15:27:03.222 INFO:teuthology.orchestra.run.smithi178.stderr: 26450K .......... .......... .......... .......... .......... 28% 147M 4s 2024-06-28T15:27:03.222 INFO:teuthology.orchestra.run.smithi178.stderr: 26500K .......... .......... .......... .......... .......... 28% 133M 4s 2024-06-28T15:27:03.222 INFO:teuthology.orchestra.run.smithi178.stderr: 26550K .......... .......... .......... .......... .......... 28% 145M 4s 2024-06-28T15:27:03.222 INFO:teuthology.orchestra.run.smithi178.stderr: 26600K .......... .......... .......... .......... .......... 28% 143M 4s 2024-06-28T15:27:03.222 INFO:teuthology.orchestra.run.smithi178.stderr: 26650K .......... .......... .......... .......... .......... 28% 150M 4s 2024-06-28T15:27:03.222 INFO:teuthology.orchestra.run.smithi178.stderr: 26700K .......... .......... .......... .......... .......... 28% 123M 4s 2024-06-28T15:27:03.222 INFO:teuthology.orchestra.run.smithi178.stderr: 26750K .......... .......... .......... .......... .......... 28% 155M 4s 2024-06-28T15:27:03.222 INFO:teuthology.orchestra.run.smithi178.stderr: 26800K .......... .......... .......... .......... .......... 28% 146M 4s 2024-06-28T15:27:03.223 INFO:teuthology.orchestra.run.smithi178.stderr: 26850K .......... .......... .......... .......... .......... 28% 136M 4s 2024-06-28T15:27:03.223 INFO:teuthology.orchestra.run.smithi178.stderr: 26900K .......... .......... .......... .......... .......... 29% 130M 4s 2024-06-28T15:27:03.223 INFO:teuthology.orchestra.run.smithi178.stderr: 26950K .......... .......... .......... .......... .......... 29% 146M 4s 2024-06-28T15:27:03.223 INFO:teuthology.orchestra.run.smithi178.stderr: 27000K .......... .......... .......... .......... .......... 29% 146M 4s 2024-06-28T15:27:03.223 INFO:teuthology.orchestra.run.smithi178.stderr: 27050K .......... .......... .......... .......... .......... 29% 148M 4s 2024-06-28T15:27:03.223 INFO:teuthology.orchestra.run.smithi178.stderr: 27100K .......... .......... .......... .......... .......... 29% 120M 4s 2024-06-28T15:27:03.223 INFO:teuthology.orchestra.run.smithi178.stderr: 27150K .......... .......... .......... .......... .......... 29% 148M 4s 2024-06-28T15:27:03.223 INFO:teuthology.orchestra.run.smithi178.stderr: 27200K .......... .......... .......... .......... .......... 29% 147M 4s 2024-06-28T15:27:03.223 INFO:teuthology.orchestra.run.smithi178.stderr: 27250K .......... .......... .......... .......... .......... 29% 149M 3s 2024-06-28T15:27:03.223 INFO:teuthology.orchestra.run.smithi178.stderr: 27300K .......... .......... .......... .......... .......... 29% 130M 3s 2024-06-28T15:27:03.223 INFO:teuthology.orchestra.run.smithi178.stderr: 27350K .......... .......... .......... .......... .......... 29% 141M 3s 2024-06-28T15:27:03.223 INFO:teuthology.orchestra.run.smithi178.stderr: 27400K .......... .......... .......... .......... .......... 29% 138M 3s 2024-06-28T15:27:03.223 INFO:teuthology.orchestra.run.smithi178.stderr: 27450K .......... .......... .......... .......... .......... 29% 146M 3s 2024-06-28T15:27:03.224 INFO:teuthology.orchestra.run.smithi178.stderr: 27500K .......... .......... .......... .......... .......... 29% 102M 3s 2024-06-28T15:27:03.224 INFO:teuthology.orchestra.run.smithi178.stderr: 27550K .......... .......... .......... .......... .......... 29% 145M 3s 2024-06-28T15:27:03.224 INFO:teuthology.orchestra.run.smithi178.stderr: 27600K .......... .......... .......... .......... .......... 29% 142M 3s 2024-06-28T15:27:03.224 INFO:teuthology.orchestra.run.smithi178.stderr: 27650K .......... .......... .......... .......... .......... 29% 145M 3s 2024-06-28T15:27:03.224 INFO:teuthology.orchestra.run.smithi178.stderr: 27700K .......... .......... .......... .......... .......... 29% 130M 3s 2024-06-28T15:27:03.224 INFO:teuthology.orchestra.run.smithi178.stderr: 27750K .......... .......... .......... .......... .......... 29% 146M 3s 2024-06-28T15:27:03.224 INFO:teuthology.orchestra.run.smithi178.stderr: 27800K .......... .......... .......... .......... .......... 29% 145M 3s 2024-06-28T15:27:03.224 INFO:teuthology.orchestra.run.smithi178.stderr: 27850K .......... .......... .......... .......... .......... 30% 142M 3s 2024-06-28T15:27:03.224 INFO:teuthology.orchestra.run.smithi178.stderr: 27900K .......... .......... .......... .......... .......... 30% 123M 3s 2024-06-28T15:27:03.224 INFO:teuthology.orchestra.run.smithi178.stderr: 27950K .......... .......... .......... .......... .......... 30% 140M 3s 2024-06-28T15:27:03.224 INFO:teuthology.orchestra.run.smithi178.stderr: 28000K .......... .......... .......... .......... .......... 30% 148M 3s 2024-06-28T15:27:03.224 INFO:teuthology.orchestra.run.smithi178.stderr: 28050K .......... .......... .......... .......... .......... 30% 149M 3s 2024-06-28T15:27:03.225 INFO:teuthology.orchestra.run.smithi178.stderr: 28100K .......... .......... .......... .......... .......... 30% 3.81M 3s 2024-06-28T15:27:03.235 INFO:teuthology.orchestra.run.smithi178.stderr: 28150K .......... .......... .......... .......... .......... 30% 1.63M 3s 2024-06-28T15:27:03.237 INFO:teuthology.orchestra.run.smithi178.stderr: 28200K .......... .......... .......... .......... .......... 30% 31.8M 3s 2024-06-28T15:27:03.239 INFO:teuthology.orchestra.run.smithi178.stderr: 28250K .......... .......... .......... .......... .......... 30% 31.7M 3s 2024-06-28T15:27:03.241 INFO:teuthology.orchestra.run.smithi178.stderr: 28300K .......... .......... .......... .......... .......... 30% 23.3M 3s 2024-06-28T15:27:03.242 INFO:teuthology.orchestra.run.smithi178.stderr: 28350K .......... .......... .......... .......... .......... 30% 32.3M 3s 2024-06-28T15:27:03.244 INFO:teuthology.orchestra.run.smithi178.stderr: 28400K .......... .......... .......... .......... .......... 30% 24.3M 3s 2024-06-28T15:27:03.245 INFO:teuthology.orchestra.run.smithi178.stderr: 28450K .......... .......... .......... .......... .......... 30% 34.1M 3s 2024-06-28T15:27:03.247 INFO:teuthology.orchestra.run.smithi178.stderr: 28500K .......... .......... .......... .......... .......... 30% 30.0M 3s 2024-06-28T15:27:03.249 INFO:teuthology.orchestra.run.smithi178.stderr: 28550K .......... .......... .......... .......... .......... 30% 29.3M 3s 2024-06-28T15:27:03.251 INFO:teuthology.orchestra.run.smithi178.stderr: 28600K .......... .......... .......... .......... .......... 30% 24.2M 3s 2024-06-28T15:27:03.253 INFO:teuthology.orchestra.run.smithi178.stderr: 28650K .......... .......... .......... .......... .......... 30% 31.9M 3s 2024-06-28T15:27:03.254 INFO:teuthology.orchestra.run.smithi178.stderr: 28700K .......... .......... .......... .......... .......... 30% 22.4M 3s 2024-06-28T15:27:03.258 INFO:teuthology.orchestra.run.smithi178.stderr: 28750K .......... .......... .......... .......... .......... 31% 12.6M 3s 2024-06-28T15:27:03.260 INFO:teuthology.orchestra.run.smithi178.stderr: 28800K .......... .......... .......... .......... .......... 31% 27.9M 3s 2024-06-28T15:27:03.262 INFO:teuthology.orchestra.run.smithi178.stderr: 28850K .......... .......... .......... .......... .......... 31% 23.4M 3s 2024-06-28T15:27:03.263 INFO:teuthology.orchestra.run.smithi178.stderr: 28900K .......... .......... .......... .......... .......... 31% 47.8M 3s 2024-06-28T15:27:03.266 INFO:teuthology.orchestra.run.smithi178.stderr: 28950K .......... .......... .......... .......... .......... 31% 16.4M 3s 2024-06-28T15:27:03.267 INFO:teuthology.orchestra.run.smithi178.stderr: 29000K .......... .......... .......... .......... .......... 31% 47.9M 3s 2024-06-28T15:27:03.273 INFO:teuthology.orchestra.run.smithi178.stderr: 29050K .......... .......... .......... .......... .......... 31% 8.28M 3s 2024-06-28T15:27:03.276 INFO:teuthology.orchestra.run.smithi178.stderr: 29100K .......... .......... .......... .......... .......... 31% 22.1M 3s 2024-06-28T15:27:03.277 INFO:teuthology.orchestra.run.smithi178.stderr: 29150K .......... .......... .......... .......... .......... 31% 31.1M 3s 2024-06-28T15:27:03.279 INFO:teuthology.orchestra.run.smithi178.stderr: 29200K .......... .......... .......... .......... .......... 31% 26.8M 3s 2024-06-28T15:27:03.280 INFO:teuthology.orchestra.run.smithi178.stderr: 29250K .......... .......... .......... .......... .......... 31% 27.1M 3s 2024-06-28T15:27:03.283 INFO:teuthology.orchestra.run.smithi178.stderr: 29300K .......... .......... .......... .......... .......... 31% 19.7M 3s 2024-06-28T15:27:03.284 INFO:teuthology.orchestra.run.smithi178.stderr: 29350K .......... .......... .......... .......... .......... 31% 32.9M 3s 2024-06-28T15:27:03.287 INFO:teuthology.orchestra.run.smithi178.stderr: 29400K .......... .......... .......... .......... .......... 31% 19.9M 3s 2024-06-28T15:27:03.289 INFO:teuthology.orchestra.run.smithi178.stderr: 29450K .......... .......... .......... .......... .......... 31% 31.6M 3s 2024-06-28T15:27:03.291 INFO:teuthology.orchestra.run.smithi178.stderr: 29500K .......... .......... .......... .......... .......... 31% 20.8M 3s 2024-06-28T15:27:03.293 INFO:teuthology.orchestra.run.smithi178.stderr: 29550K .......... .......... .......... .......... .......... 31% 23.9M 3s 2024-06-28T15:27:03.294 INFO:teuthology.orchestra.run.smithi178.stderr: 29600K .......... .......... .......... .......... .......... 31% 34.8M 3s 2024-06-28T15:27:03.298 INFO:teuthology.orchestra.run.smithi178.stderr: 29650K .......... .......... .......... .......... .......... 31% 14.5M 3s 2024-06-28T15:27:03.300 INFO:teuthology.orchestra.run.smithi178.stderr: 29700K .......... .......... .......... .......... .......... 32% 17.1M 3s 2024-06-28T15:27:03.302 INFO:teuthology.orchestra.run.smithi178.stderr: 29750K .......... .......... .......... .......... .......... 32% 34.2M 3s 2024-06-28T15:27:03.304 INFO:teuthology.orchestra.run.smithi178.stderr: 29800K .......... .......... .......... .......... .......... 32% 22.4M 3s 2024-06-28T15:27:03.306 INFO:teuthology.orchestra.run.smithi178.stderr: 29850K .......... .......... .......... .......... .......... 32% 25.2M 3s 2024-06-28T15:27:03.310 INFO:teuthology.orchestra.run.smithi178.stderr: 29900K .......... .......... .......... .......... .......... 32% 10.7M 3s 2024-06-28T15:27:03.313 INFO:teuthology.orchestra.run.smithi178.stderr: 29950K .......... .......... .......... .......... .......... 32% 20.8M 3s 2024-06-28T15:27:03.315 INFO:teuthology.orchestra.run.smithi178.stderr: 30000K .......... .......... .......... .......... .......... 32% 28.1M 3s 2024-06-28T15:27:03.317 INFO:teuthology.orchestra.run.smithi178.stderr: 30050K .......... .......... .......... .......... .......... 32% 19.9M 3s 2024-06-28T15:27:03.318 INFO:teuthology.orchestra.run.smithi178.stderr: 30100K .......... .......... .......... .......... .......... 32% 35.7M 3s 2024-06-28T15:27:03.321 INFO:teuthology.orchestra.run.smithi178.stderr: 30150K .......... .......... .......... .......... .......... 32% 19.6M 3s 2024-06-28T15:27:03.323 INFO:teuthology.orchestra.run.smithi178.stderr: 30200K .......... .......... .......... .......... .......... 32% 27.8M 3s 2024-06-28T15:27:03.326 INFO:teuthology.orchestra.run.smithi178.stderr: 30250K .......... .......... .......... .......... .......... 32% 17.5M 3s 2024-06-28T15:27:03.329 INFO:teuthology.orchestra.run.smithi178.stderr: 30300K .......... .......... .......... .......... .......... 32% 15.5M 3s 2024-06-28T15:27:03.330 INFO:teuthology.orchestra.run.smithi178.stderr: 30350K .......... .......... .......... .......... .......... 32% 40.8M 3s 2024-06-28T15:27:03.333 INFO:teuthology.orchestra.run.smithi178.stderr: 30400K .......... .......... .......... .......... .......... 32% 15.8M 3s 2024-06-28T15:27:03.335 INFO:teuthology.orchestra.run.smithi178.stderr: 30450K .......... .......... .......... .......... .......... 32% 36.0M 3s 2024-06-28T15:27:03.338 INFO:teuthology.orchestra.run.smithi178.stderr: 30500K .......... .......... .......... .......... .......... 32% 13.8M 3s 2024-06-28T15:27:03.339 INFO:teuthology.orchestra.run.smithi178.stderr: 30550K .......... .......... .......... .......... .......... 32% 30.1M 3s 2024-06-28T15:27:03.341 INFO:teuthology.orchestra.run.smithi178.stderr: 30600K .......... .......... .......... .......... .......... 32% 24.2M 3s 2024-06-28T15:27:03.343 INFO:teuthology.orchestra.run.smithi178.stderr: 30650K .......... .......... .......... .......... .......... 33% 37.4M 3s 2024-06-28T15:27:03.345 INFO:teuthology.orchestra.run.smithi178.stderr: 30700K .......... .......... .......... .......... .......... 33% 23.4M 3s 2024-06-28T15:27:03.347 INFO:teuthology.orchestra.run.smithi178.stderr: 30750K .......... .......... .......... .......... .......... 33% 21.9M 3s 2024-06-28T15:27:03.350 INFO:teuthology.orchestra.run.smithi178.stderr: 30800K .......... .......... .......... .......... .......... 33% 16.8M 3s 2024-06-28T15:27:03.352 INFO:teuthology.orchestra.run.smithi178.stderr: 30850K .......... .......... .......... .......... .......... 33% 29.9M 3s 2024-06-28T15:27:03.353 INFO:teuthology.orchestra.run.smithi178.stderr: 30900K .......... .......... .......... .......... .......... 33% 33.1M 3s 2024-06-28T15:27:03.357 INFO:teuthology.orchestra.run.smithi178.stderr: 30950K .......... .......... .......... .......... .......... 33% 19.4M 3s 2024-06-28T15:27:03.357 INFO:teuthology.orchestra.run.smithi178.stderr: 31000K .......... .......... .......... .......... .......... 33% 40.8M 3s 2024-06-28T15:27:03.359 INFO:teuthology.orchestra.run.smithi178.stderr: 31050K .......... .......... .......... .......... .......... 33% 24.4M 3s 2024-06-28T15:27:03.362 INFO:teuthology.orchestra.run.smithi178.stderr: 31100K .......... .......... .......... .......... .......... 33% 15.8M 3s 2024-06-28T15:27:03.364 INFO:teuthology.orchestra.run.smithi178.stderr: 31150K .......... .......... .......... .......... .......... 33% 26.8M 3s 2024-06-28T15:27:03.367 INFO:teuthology.orchestra.run.smithi178.stderr: 31200K .......... .......... .......... .......... .......... 33% 14.9M 3s 2024-06-28T15:27:03.369 INFO:teuthology.orchestra.run.smithi178.stderr: 31250K .......... .......... .......... .......... .......... 33% 25.1M 3s 2024-06-28T15:27:03.372 INFO:teuthology.orchestra.run.smithi178.stderr: 31300K .......... .......... .......... .......... .......... 33% 21.2M 3s 2024-06-28T15:27:03.376 INFO:teuthology.orchestra.run.smithi178.stderr: 31350K .......... .......... .......... .......... .......... 33% 21.8M 3s 2024-06-28T15:27:03.376 INFO:teuthology.orchestra.run.smithi178.stderr: 31400K .......... .......... .......... .......... .......... 33% 23.5M 3s 2024-06-28T15:27:03.379 INFO:teuthology.orchestra.run.smithi178.stderr: 31450K .......... .......... .......... .......... .......... 33% 19.7M 3s 2024-06-28T15:27:03.382 INFO:teuthology.orchestra.run.smithi178.stderr: 31500K .......... .......... .......... .......... .......... 33% 16.1M 3s 2024-06-28T15:27:03.387 INFO:teuthology.orchestra.run.smithi178.stderr: 31550K .......... .......... .......... .......... .......... 34% 28.4M 3s 2024-06-28T15:27:03.387 INFO:teuthology.orchestra.run.smithi178.stderr: 31600K .......... .......... .......... .......... .......... 34% 23.2M 3s 2024-06-28T15:27:03.387 INFO:teuthology.orchestra.run.smithi178.stderr: 31650K .......... .......... .......... .......... .......... 34% 30.1M 3s 2024-06-28T15:27:03.390 INFO:teuthology.orchestra.run.smithi178.stderr: 31700K .......... .......... .......... .......... .......... 34% 16.5M 3s 2024-06-28T15:27:03.393 INFO:teuthology.orchestra.run.smithi178.stderr: 31750K .......... .......... .......... .......... .......... 34% 21.0M 3s 2024-06-28T15:27:03.395 INFO:teuthology.orchestra.run.smithi178.stderr: 31800K .......... .......... .......... .......... .......... 34% 26.5M 3s 2024-06-28T15:27:03.398 INFO:teuthology.orchestra.run.smithi178.stderr: 31850K .......... .......... .......... .......... .......... 34% 23.4M 3s 2024-06-28T15:27:03.398 INFO:teuthology.orchestra.run.smithi178.stderr: 31900K .......... .......... .......... .......... .......... 34% 21.1M 3s 2024-06-28T15:27:03.400 INFO:teuthology.orchestra.run.smithi178.stderr: 31950K .......... .......... .......... .......... .......... 34% 27.5M 3s 2024-06-28T15:27:03.405 INFO:teuthology.orchestra.run.smithi178.stderr: 32000K .......... .......... .......... .......... .......... 34% 14.2M 3s 2024-06-28T15:27:03.407 INFO:teuthology.orchestra.run.smithi178.stderr: 32050K .......... .......... .......... .......... .......... 34% 19.1M 3s 2024-06-28T15:27:03.409 INFO:teuthology.orchestra.run.smithi178.stderr: 32100K .......... .......... .......... .......... .......... 34% 25.7M 3s 2024-06-28T15:27:03.410 INFO:teuthology.orchestra.run.smithi178.stderr: 32150K .......... .......... .......... .......... .......... 34% 35.3M 3s 2024-06-28T15:27:03.413 INFO:teuthology.orchestra.run.smithi178.stderr: 32200K .......... .......... .......... .......... .......... 34% 17.3M 3s 2024-06-28T15:27:03.415 INFO:teuthology.orchestra.run.smithi178.stderr: 32250K .......... .......... .......... .......... .......... 34% 20.5M 3s 2024-06-28T15:27:03.418 INFO:teuthology.orchestra.run.smithi178.stderr: 32300K .......... .......... .......... .......... .......... 34% 15.6M 3s 2024-06-28T15:27:03.420 INFO:teuthology.orchestra.run.smithi178.stderr: 32350K .......... .......... .......... .......... .......... 34% 22.0M 3s 2024-06-28T15:27:03.424 INFO:teuthology.orchestra.run.smithi178.stderr: 32400K .......... .......... .......... .......... .......... 34% 19.6M 3s 2024-06-28T15:27:03.424 INFO:teuthology.orchestra.run.smithi178.stderr: 32450K .......... .......... .......... .......... .......... 34% 43.9M 3s 2024-06-28T15:27:03.428 INFO:teuthology.orchestra.run.smithi178.stderr: 32500K .......... .......... .......... .......... .......... 35% 13.2M 3s 2024-06-28T15:27:03.429 INFO:teuthology.orchestra.run.smithi178.stderr: 32550K .......... .......... .......... .......... .......... 35% 32.0M 3s 2024-06-28T15:27:03.433 INFO:teuthology.orchestra.run.smithi178.stderr: 32600K .......... .......... .......... .......... .......... 35% 12.7M 3s 2024-06-28T15:27:03.435 INFO:teuthology.orchestra.run.smithi178.stderr: 32650K .......... .......... .......... .......... .......... 35% 27.3M 3s 2024-06-28T15:27:03.437 INFO:teuthology.orchestra.run.smithi178.stderr: 32700K .......... .......... .......... .......... .......... 35% 21.9M 3s 2024-06-28T15:27:03.440 INFO:teuthology.orchestra.run.smithi178.stderr: 32750K .......... .......... .......... .......... .......... 35% 15.5M 3s 2024-06-28T15:27:03.441 INFO:teuthology.orchestra.run.smithi178.stderr: 32800K .......... .......... .......... .......... .......... 35% 28.5M 3s 2024-06-28T15:27:03.444 INFO:teuthology.orchestra.run.smithi178.stderr: 32850K .......... .......... .......... .......... .......... 35% 16.1M 3s 2024-06-28T15:27:03.447 INFO:teuthology.orchestra.run.smithi178.stderr: 32900K .......... .......... .......... .......... .......... 35% 16.2M 3s 2024-06-28T15:27:03.450 INFO:teuthology.orchestra.run.smithi178.stderr: 32950K .......... .......... .......... .......... .......... 35% 19.9M 3s 2024-06-28T15:27:03.452 INFO:teuthology.orchestra.run.smithi178.stderr: 33000K .......... .......... .......... .......... .......... 35% 33.0M 3s 2024-06-28T15:27:03.454 INFO:teuthology.orchestra.run.smithi178.stderr: 33050K .......... .......... .......... .......... .......... 35% 22.6M 3s 2024-06-28T15:27:03.456 INFO:teuthology.orchestra.run.smithi178.stderr: 33100K .......... .......... .......... .......... .......... 35% 18.4M 3s 2024-06-28T15:27:03.457 INFO:teuthology.orchestra.run.smithi178.stderr: 33150K .......... .......... .......... .......... .......... 35% 37.5M 3s 2024-06-28T15:27:03.460 INFO:teuthology.orchestra.run.smithi178.stderr: 33200K .......... .......... .......... .......... .......... 35% 21.0M 3s 2024-06-28T15:27:03.463 INFO:teuthology.orchestra.run.smithi178.stderr: 33250K .......... .......... .......... .......... .......... 35% 28.6M 3s 2024-06-28T15:27:03.464 INFO:teuthology.orchestra.run.smithi178.stderr: 33300K .......... .......... .......... .......... .......... 35% 21.1M 3s 2024-06-28T15:27:03.465 INFO:teuthology.orchestra.run.smithi178.stderr: 33350K .......... .......... .......... .......... .......... 35% 30.0M 3s 2024-06-28T15:27:03.467 INFO:teuthology.orchestra.run.smithi178.stderr: 33400K .......... .......... .......... .......... .......... 36% 22.1M 3s 2024-06-28T15:27:03.471 INFO:teuthology.orchestra.run.smithi178.stderr: 33450K .......... .......... .......... .......... .......... 36% 20.9M 3s 2024-06-28T15:27:03.473 INFO:teuthology.orchestra.run.smithi178.stderr: 33500K .......... .......... .......... .......... .......... 36% 14.1M 3s 2024-06-28T15:27:03.476 INFO:teuthology.orchestra.run.smithi178.stderr: 33550K .......... .......... .......... .......... .......... 36% 28.0M 3s 2024-06-28T15:27:03.477 INFO:teuthology.orchestra.run.smithi178.stderr: 33600K .......... .......... .......... .......... .......... 36% 20.8M 3s 2024-06-28T15:27:03.480 INFO:teuthology.orchestra.run.smithi178.stderr: 33650K .......... .......... .......... .......... .......... 36% 20.5M 3s 2024-06-28T15:27:03.481 INFO:teuthology.orchestra.run.smithi178.stderr: 33700K .......... .......... .......... .......... .......... 36% 34.5M 3s 2024-06-28T15:27:03.484 INFO:teuthology.orchestra.run.smithi178.stderr: 33750K .......... .......... .......... .......... .......... 36% 16.4M 3s 2024-06-28T15:27:03.487 INFO:teuthology.orchestra.run.smithi178.stderr: 33800K .......... .......... .......... .......... .......... 36% 15.7M 3s 2024-06-28T15:27:03.489 INFO:teuthology.orchestra.run.smithi178.stderr: 33850K .......... .......... .......... .......... .......... 36% 23.4M 3s 2024-06-28T15:27:03.492 INFO:teuthology.orchestra.run.smithi178.stderr: 33900K .......... .......... .......... .......... .......... 36% 20.6M 3s 2024-06-28T15:27:03.494 INFO:teuthology.orchestra.run.smithi178.stderr: 33950K .......... .......... .......... .......... .......... 36% 19.5M 3s 2024-06-28T15:27:03.496 INFO:teuthology.orchestra.run.smithi178.stderr: 34000K .......... .......... .......... .......... .......... 36% 30.0M 3s 2024-06-28T15:27:03.498 INFO:teuthology.orchestra.run.smithi178.stderr: 34050K .......... .......... .......... .......... .......... 36% 25.3M 3s 2024-06-28T15:27:03.499 INFO:teuthology.orchestra.run.smithi178.stderr: 34100K .......... .......... .......... .......... .......... 36% 34.1M 3s 2024-06-28T15:27:03.501 INFO:teuthology.orchestra.run.smithi178.stderr: 34150K .......... .......... .......... .......... .......... 36% 22.2M 3s 2024-06-28T15:27:03.503 INFO:teuthology.orchestra.run.smithi178.stderr: 34200K .......... .......... .......... .......... .......... 36% 32.3M 3s 2024-06-28T15:27:03.507 INFO:teuthology.orchestra.run.smithi178.stderr: 34250K .......... .......... .......... .......... .......... 36% 30.4M 3s 2024-06-28T15:27:03.508 INFO:teuthology.orchestra.run.smithi178.stderr: 34300K .......... .......... .......... .......... .......... 36% 13.4M 3s 2024-06-28T15:27:03.511 INFO:teuthology.orchestra.run.smithi178.stderr: 34350K .......... .......... .......... .......... .......... 37% 19.3M 3s 2024-06-28T15:27:03.514 INFO:teuthology.orchestra.run.smithi178.stderr: 34400K .......... .......... .......... .......... .......... 37% 18.0M 3s 2024-06-28T15:27:03.514 INFO:teuthology.orchestra.run.smithi178.stderr: 34450K .......... .......... .......... .......... .......... 37% 52.9M 3s 2024-06-28T15:27:03.516 INFO:teuthology.orchestra.run.smithi178.stderr: 34500K .......... .......... .......... .......... .......... 37% 25.4M 3s 2024-06-28T15:27:03.519 INFO:teuthology.orchestra.run.smithi178.stderr: 34550K .......... .......... .......... .......... .......... 37% 16.5M 3s 2024-06-28T15:27:03.522 INFO:teuthology.orchestra.run.smithi178.stderr: 34600K .......... .......... .......... .......... .......... 37% 18.9M 3s 2024-06-28T15:27:03.527 INFO:teuthology.orchestra.run.smithi178.stderr: 34650K .......... .......... .......... .......... .......... 37% 18.2M 3s 2024-06-28T15:27:03.528 INFO:teuthology.orchestra.run.smithi178.stderr: 34700K .......... .......... .......... .......... .......... 37% 15.7M 3s 2024-06-28T15:27:03.530 INFO:teuthology.orchestra.run.smithi178.stderr: 34750K .......... .......... .......... .......... .......... 37% 23.3M 3s 2024-06-28T15:27:03.532 INFO:teuthology.orchestra.run.smithi178.stderr: 34800K .......... .......... .......... .......... .......... 37% 25.5M 3s 2024-06-28T15:27:03.534 INFO:teuthology.orchestra.run.smithi178.stderr: 34850K .......... .......... .......... .......... .......... 37% 23.5M 3s 2024-06-28T15:27:03.536 INFO:teuthology.orchestra.run.smithi178.stderr: 34900K .......... .......... .......... .......... .......... 37% 22.7M 3s 2024-06-28T15:27:03.538 INFO:teuthology.orchestra.run.smithi178.stderr: 34950K .......... .......... .......... .......... .......... 37% 19.6M 3s 2024-06-28T15:27:03.540 INFO:teuthology.orchestra.run.smithi178.stderr: 35000K .......... .......... .......... .......... .......... 37% 25.2M 3s 2024-06-28T15:27:03.542 INFO:teuthology.orchestra.run.smithi178.stderr: 35050K .......... .......... .......... .......... .......... 37% 22.7M 3s 2024-06-28T15:27:03.545 INFO:teuthology.orchestra.run.smithi178.stderr: 35100K .......... .......... .......... .......... .......... 37% 19.9M 3s 2024-06-28T15:27:03.547 INFO:teuthology.orchestra.run.smithi178.stderr: 35150K .......... .......... .......... .......... .......... 37% 24.0M 3s 2024-06-28T15:27:03.550 INFO:teuthology.orchestra.run.smithi178.stderr: 35200K .......... .......... .......... .......... .......... 37% 16.7M 3s 2024-06-28T15:27:03.552 INFO:teuthology.orchestra.run.smithi178.stderr: 35250K .......... .......... .......... .......... .......... 37% 24.2M 3s 2024-06-28T15:27:03.554 INFO:teuthology.orchestra.run.smithi178.stderr: 35300K .......... .......... .......... .......... .......... 38% 18.3M 3s 2024-06-28T15:27:03.556 INFO:teuthology.orchestra.run.smithi178.stderr: 35350K .......... .......... .......... .......... .......... 38% 40.6M 3s 2024-06-28T15:27:03.557 INFO:teuthology.orchestra.run.smithi178.stderr: 35400K .......... .......... .......... .......... .......... 38% 31.3M 3s 2024-06-28T15:27:03.560 INFO:teuthology.orchestra.run.smithi178.stderr: 35450K .......... .......... .......... .......... .......... 38% 20.2M 3s 2024-06-28T15:27:03.562 INFO:teuthology.orchestra.run.smithi178.stderr: 35500K .......... .......... .......... .......... .......... 38% 26.4M 3s 2024-06-28T15:27:03.565 INFO:teuthology.orchestra.run.smithi178.stderr: 35550K .......... .......... .......... .......... .......... 38% 15.6M 3s 2024-06-28T15:27:03.568 INFO:teuthology.orchestra.run.smithi178.stderr: 35600K .......... .......... .......... .......... .......... 38% 15.2M 3s 2024-06-28T15:27:03.570 INFO:teuthology.orchestra.run.smithi178.stderr: 35650K .......... .......... .......... .......... .......... 38% 21.8M 3s 2024-06-28T15:27:03.572 INFO:teuthology.orchestra.run.smithi178.stderr: 35700K .......... .......... .......... .......... .......... 38% 23.9M 3s 2024-06-28T15:27:03.573 INFO:teuthology.orchestra.run.smithi178.stderr: 35750K .......... .......... .......... .......... .......... 38% 50.7M 3s 2024-06-28T15:27:03.576 INFO:teuthology.orchestra.run.smithi178.stderr: 35800K .......... .......... .......... .......... .......... 38% 18.0M 3s 2024-06-28T15:27:03.578 INFO:teuthology.orchestra.run.smithi178.stderr: 35850K .......... .......... .......... .......... .......... 38% 22.4M 3s 2024-06-28T15:27:03.581 INFO:teuthology.orchestra.run.smithi178.stderr: 35900K .......... .......... .......... .......... .......... 38% 18.7M 3s 2024-06-28T15:27:03.582 INFO:teuthology.orchestra.run.smithi178.stderr: 35950K .......... .......... .......... .......... .......... 38% 37.2M 3s 2024-06-28T15:27:03.583 INFO:teuthology.orchestra.run.smithi178.stderr: 36000K .......... .......... .......... .......... .......... 38% 33.7M 3s 2024-06-28T15:27:03.585 INFO:teuthology.orchestra.run.smithi178.stderr: 36050K .......... .......... .......... .......... .......... 38% 29.2M 3s 2024-06-28T15:27:03.588 INFO:teuthology.orchestra.run.smithi178.stderr: 36100K .......... .......... .......... .......... .......... 38% 16.1M 3s 2024-06-28T15:27:03.591 INFO:teuthology.orchestra.run.smithi178.stderr: 36150K .......... .......... .......... .......... .......... 38% 25.3M 3s 2024-06-28T15:27:03.591 INFO:teuthology.orchestra.run.smithi178.stderr: 36200K .......... .......... .......... .......... .......... 39% 65.8M 3s 2024-06-28T15:27:03.594 INFO:teuthology.orchestra.run.smithi178.stderr: 36250K .......... .......... .......... .......... .......... 39% 18.9M 3s 2024-06-28T15:27:03.595 INFO:teuthology.orchestra.run.smithi178.stderr: 36300K .......... .......... .......... .......... .......... 39% 25.3M 3s 2024-06-28T15:27:03.599 INFO:teuthology.orchestra.run.smithi178.stderr: 36350K .......... .......... .......... .......... .......... 39% 15.7M 3s 2024-06-28T15:27:03.601 INFO:teuthology.orchestra.run.smithi178.stderr: 36400K .......... .......... .......... .......... .......... 39% 18.7M 3s 2024-06-28T15:27:03.602 INFO:teuthology.orchestra.run.smithi178.stderr: 36450K .......... .......... .......... .......... .......... 39% 33.8M 3s 2024-06-28T15:27:03.608 INFO:teuthology.orchestra.run.smithi178.stderr: 36500K .......... .......... .......... .......... .......... 39% 10.6M 3s 2024-06-28T15:27:03.609 INFO:teuthology.orchestra.run.smithi178.stderr: 36550K .......... .......... .......... .......... .......... 39% 35.1M 3s 2024-06-28T15:27:03.610 INFO:teuthology.orchestra.run.smithi178.stderr: 36600K .......... .......... .......... .......... .......... 39% 26.6M 3s 2024-06-28T15:27:03.612 INFO:teuthology.orchestra.run.smithi178.stderr: 36650K .......... .......... .......... .......... .......... 39% 41.0M 3s 2024-06-28T15:27:03.615 INFO:teuthology.orchestra.run.smithi178.stderr: 36700K .......... .......... .......... .......... .......... 39% 13.1M 3s 2024-06-28T15:27:03.617 INFO:teuthology.orchestra.run.smithi178.stderr: 36750K .......... .......... .......... .......... .......... 39% 26.1M 3s 2024-06-28T15:27:03.620 INFO:teuthology.orchestra.run.smithi178.stderr: 36800K .......... .......... .......... .......... .......... 39% 18.0M 3s 2024-06-28T15:27:03.621 INFO:teuthology.orchestra.run.smithi178.stderr: 36850K .......... .......... .......... .......... .......... 39% 36.2M 3s 2024-06-28T15:27:03.622 INFO:teuthology.orchestra.run.smithi178.stderr: 36900K .......... .......... .......... .......... .......... 39% 39.9M 3s 2024-06-28T15:27:03.626 INFO:teuthology.orchestra.run.smithi178.stderr: 36950K .......... .......... .......... .......... .......... 39% 15.6M 3s 2024-06-28T15:27:03.627 INFO:teuthology.orchestra.run.smithi178.stderr: 37000K .......... .......... .......... .......... .......... 39% 28.1M 3s 2024-06-28T15:27:03.629 INFO:teuthology.orchestra.run.smithi178.stderr: 37050K .......... .......... .......... .......... .......... 39% 27.8M 3s 2024-06-28T15:27:03.631 INFO:teuthology.orchestra.run.smithi178.stderr: 37100K .......... .......... .......... .......... .......... 39% 20.3M 3s 2024-06-28T15:27:03.633 INFO:teuthology.orchestra.run.smithi178.stderr: 37150K .......... .......... .......... .......... .......... 40% 22.6M 3s 2024-06-28T15:27:03.635 INFO:teuthology.orchestra.run.smithi178.stderr: 37200K .......... .......... .......... .......... .......... 40% 28.7M 3s 2024-06-28T15:27:03.637 INFO:teuthology.orchestra.run.smithi178.stderr: 37250K .......... .......... .......... .......... .......... 40% 28.1M 3s 2024-06-28T15:27:03.639 INFO:teuthology.orchestra.run.smithi178.stderr: 37300K .......... .......... .......... .......... .......... 40% 22.0M 3s 2024-06-28T15:27:03.641 INFO:teuthology.orchestra.run.smithi178.stderr: 37350K .......... .......... .......... .......... .......... 40% 26.9M 3s 2024-06-28T15:27:03.644 INFO:teuthology.orchestra.run.smithi178.stderr: 37400K .......... .......... .......... .......... .......... 40% 14.4M 3s 2024-06-28T15:27:03.647 INFO:teuthology.orchestra.run.smithi178.stderr: 37450K .......... .......... .......... .......... .......... 40% 36.3M 3s 2024-06-28T15:27:03.649 INFO:teuthology.orchestra.run.smithi178.stderr: 37500K .......... .......... .......... .......... .......... 40% 18.6M 3s 2024-06-28T15:27:03.650 INFO:teuthology.orchestra.run.smithi178.stderr: 37550K .......... .......... .......... .......... .......... 40% 32.6M 3s 2024-06-28T15:27:03.651 INFO:teuthology.orchestra.run.smithi178.stderr: 37600K .......... .......... .......... .......... .......... 40% 35.1M 3s 2024-06-28T15:27:03.654 INFO:teuthology.orchestra.run.smithi178.stderr: 37650K .......... .......... .......... .......... .......... 40% 19.7M 3s 2024-06-28T15:27:03.656 INFO:teuthology.orchestra.run.smithi178.stderr: 37700K .......... .......... .......... .......... .......... 40% 22.6M 3s 2024-06-28T15:27:03.658 INFO:teuthology.orchestra.run.smithi178.stderr: 37750K .......... .......... .......... .......... .......... 40% 21.8M 3s 2024-06-28T15:27:03.659 INFO:teuthology.orchestra.run.smithi178.stderr: 37800K .......... .......... .......... .......... .......... 40% 52.3M 3s 2024-06-28T15:27:03.663 INFO:teuthology.orchestra.run.smithi178.stderr: 37850K .......... .......... .......... .......... .......... 40% 12.7M 3s 2024-06-28T15:27:03.666 INFO:teuthology.orchestra.run.smithi178.stderr: 37900K .......... .......... .......... .......... .......... 40% 18.4M 3s 2024-06-28T15:27:03.666 INFO:teuthology.orchestra.run.smithi178.stderr: 37950K .......... .......... .......... .......... .......... 40% 49.9M 3s 2024-06-28T15:27:03.669 INFO:teuthology.orchestra.run.smithi178.stderr: 38000K .......... .......... .......... .......... .......... 40% 23.8M 3s 2024-06-28T15:27:03.671 INFO:teuthology.orchestra.run.smithi178.stderr: 38050K .......... .......... .......... .......... .......... 41% 19.7M 3s 2024-06-28T15:27:03.673 INFO:teuthology.orchestra.run.smithi178.stderr: 38100K .......... .......... .......... .......... .......... 41% 25.6M 3s 2024-06-28T15:27:03.676 INFO:teuthology.orchestra.run.smithi178.stderr: 38150K .......... .......... .......... .......... .......... 41% 36.7M 3s 2024-06-28T15:27:03.677 INFO:teuthology.orchestra.run.smithi178.stderr: 38200K .......... .......... .......... .......... .......... 41% 19.6M 3s 2024-06-28T15:27:03.680 INFO:teuthology.orchestra.run.smithi178.stderr: 38250K .......... .......... .......... .......... .......... 41% 27.6M 3s 2024-06-28T15:27:03.681 INFO:teuthology.orchestra.run.smithi178.stderr: 38300K .......... .......... .......... .......... .......... 41% 21.3M 3s 2024-06-28T15:27:03.686 INFO:teuthology.orchestra.run.smithi178.stderr: 38350K .......... .......... .......... .......... .......... 41% 11.8M 3s 2024-06-28T15:27:03.687 INFO:teuthology.orchestra.run.smithi178.stderr: 38400K .......... .......... .......... .......... .......... 41% 28.0M 3s 2024-06-28T15:27:03.689 INFO:teuthology.orchestra.run.smithi178.stderr: 38450K .......... .......... .......... .......... .......... 41% 22.2M 3s 2024-06-28T15:27:03.690 INFO:teuthology.orchestra.run.smithi178.stderr: 38500K .......... .......... .......... .......... .......... 41% 38.9M 3s 2024-06-28T15:27:03.693 INFO:teuthology.orchestra.run.smithi178.stderr: 38550K .......... .......... .......... .......... .......... 41% 19.1M 3s 2024-06-28T15:27:03.694 INFO:teuthology.orchestra.run.smithi178.stderr: 38600K .......... .......... .......... .......... .......... 41% 46.0M 3s 2024-06-28T15:27:03.697 INFO:teuthology.orchestra.run.smithi178.stderr: 38650K .......... .......... .......... .......... .......... 41% 16.7M 3s 2024-06-28T15:27:03.699 INFO:teuthology.orchestra.run.smithi178.stderr: 38700K .......... .......... .......... .......... .......... 41% 28.9M 3s 2024-06-28T15:27:03.700 INFO:teuthology.orchestra.run.smithi178.stderr: 38750K .......... .......... .......... .......... .......... 41% 24.7M 3s 2024-06-28T15:27:03.703 INFO:teuthology.orchestra.run.smithi178.stderr: 38800K .......... .......... .......... .......... .......... 41% 19.0M 3s 2024-06-28T15:27:03.705 INFO:teuthology.orchestra.run.smithi178.stderr: 38850K .......... .......... .......... .......... .......... 41% 25.7M 3s 2024-06-28T15:27:03.707 INFO:teuthology.orchestra.run.smithi178.stderr: 38900K .......... .......... .......... .......... .......... 41% 19.2M 3s 2024-06-28T15:27:03.709 INFO:teuthology.orchestra.run.smithi178.stderr: 38950K .......... .......... .......... .......... .......... 41% 40.3M 3s 2024-06-28T15:27:03.713 INFO:teuthology.orchestra.run.smithi178.stderr: 39000K .......... .......... .......... .......... .......... 42% 11.5M 3s 2024-06-28T15:27:03.715 INFO:teuthology.orchestra.run.smithi178.stderr: 39050K .......... .......... .......... .......... .......... 42% 33.7M 3s 2024-06-28T15:27:03.717 INFO:teuthology.orchestra.run.smithi178.stderr: 39100K .......... .......... .......... .......... .......... 42% 17.6M 3s 2024-06-28T15:27:03.721 INFO:teuthology.orchestra.run.smithi178.stderr: 39150K .......... .......... .......... .......... .......... 42% 11.9M 3s 2024-06-28T15:27:03.723 INFO:teuthology.orchestra.run.smithi178.stderr: 39200K .......... .......... .......... .......... .......... 42% 27.0M 3s 2024-06-28T15:27:03.726 INFO:teuthology.orchestra.run.smithi178.stderr: 39250K .......... .......... .......... .......... .......... 42% 14.0M 3s 2024-06-28T15:27:03.728 INFO:teuthology.orchestra.run.smithi178.stderr: 39300K .......... .......... .......... .......... .......... 42% 28.1M 3s 2024-06-28T15:27:03.731 INFO:teuthology.orchestra.run.smithi178.stderr: 39350K .......... .......... .......... .......... .......... 42% 21.2M 3s 2024-06-28T15:27:03.732 INFO:teuthology.orchestra.run.smithi178.stderr: 39400K .......... .......... .......... .......... .......... 42% 28.8M 3s 2024-06-28T15:27:03.735 INFO:teuthology.orchestra.run.smithi178.stderr: 39450K .......... .......... .......... .......... .......... 42% 28.4M 3s 2024-06-28T15:27:03.736 INFO:teuthology.orchestra.run.smithi178.stderr: 39500K .......... .......... .......... .......... .......... 42% 20.6M 3s 2024-06-28T15:27:03.738 INFO:teuthology.orchestra.run.smithi178.stderr: 39550K .......... .......... .......... .......... .......... 42% 24.9M 3s 2024-06-28T15:27:03.741 INFO:teuthology.orchestra.run.smithi178.stderr: 39600K .......... .......... .......... .......... .......... 42% 18.4M 3s 2024-06-28T15:27:03.742 INFO:teuthology.orchestra.run.smithi178.stderr: 39650K .......... .......... .......... .......... .......... 42% 30.1M 3s 2024-06-28T15:27:03.745 INFO:teuthology.orchestra.run.smithi178.stderr: 39700K .......... .......... .......... .......... .......... 42% 23.0M 3s 2024-06-28T15:27:03.748 INFO:teuthology.orchestra.run.smithi178.stderr: 39750K .......... .......... .......... .......... .......... 42% 17.3M 3s 2024-06-28T15:27:03.748 INFO:teuthology.orchestra.run.smithi178.stderr: 39800K .......... .......... .......... .......... .......... 42% 41.0M 3s 2024-06-28T15:27:03.752 INFO:teuthology.orchestra.run.smithi178.stderr: 39850K .......... .......... .......... .......... .......... 42% 18.2M 3s 2024-06-28T15:27:03.754 INFO:teuthology.orchestra.run.smithi178.stderr: 39900K .......... .......... .......... .......... .......... 43% 16.6M 3s 2024-06-28T15:27:03.757 INFO:teuthology.orchestra.run.smithi178.stderr: 39950K .......... .......... .......... .......... .......... 43% 20.4M 3s 2024-06-28T15:27:03.760 INFO:teuthology.orchestra.run.smithi178.stderr: 40000K .......... .......... .......... .......... .......... 43% 15.6M 3s 2024-06-28T15:27:03.762 INFO:teuthology.orchestra.run.smithi178.stderr: 40050K .......... .......... .......... .......... .......... 43% 25.4M 3s 2024-06-28T15:27:03.765 INFO:teuthology.orchestra.run.smithi178.stderr: 40100K .......... .......... .......... .......... .......... 43% 14.3M 3s 2024-06-28T15:27:03.767 INFO:teuthology.orchestra.run.smithi178.stderr: 40150K .......... .......... .......... .......... .......... 43% 23.0M 3s 2024-06-28T15:27:03.771 INFO:teuthology.orchestra.run.smithi178.stderr: 40200K .......... .......... .......... .......... .......... 43% 14.2M 3s 2024-06-28T15:27:03.774 INFO:teuthology.orchestra.run.smithi178.stderr: 40250K .......... .......... .......... .......... .......... 43% 21.4M 3s 2024-06-28T15:27:03.776 INFO:teuthology.orchestra.run.smithi178.stderr: 40300K .......... .......... .......... .......... .......... 43% 17.6M 3s 2024-06-28T15:27:03.778 INFO:teuthology.orchestra.run.smithi178.stderr: 40350K .......... .......... .......... .......... .......... 43% 20.8M 3s 2024-06-28T15:27:03.780 INFO:teuthology.orchestra.run.smithi178.stderr: 40400K .......... .......... .......... .......... .......... 43% 23.1M 3s 2024-06-28T15:27:03.782 INFO:teuthology.orchestra.run.smithi178.stderr: 40450K .......... .......... .......... .......... .......... 43% 28.1M 3s 2024-06-28T15:27:03.784 INFO:teuthology.orchestra.run.smithi178.stderr: 40500K .......... .......... .......... .......... .......... 43% 23.1M 3s 2024-06-28T15:27:03.786 INFO:teuthology.orchestra.run.smithi178.stderr: 40550K .......... .......... .......... .......... .......... 43% 31.2M 3s 2024-06-28T15:27:03.787 INFO:teuthology.orchestra.run.smithi178.stderr: 40600K .......... .......... .......... .......... .......... 43% 24.6M 3s 2024-06-28T15:27:03.790 INFO:teuthology.orchestra.run.smithi178.stderr: 40650K .......... .......... .......... .......... .......... 43% 26.0M 3s 2024-06-28T15:27:03.791 INFO:teuthology.orchestra.run.smithi178.stderr: 40700K .......... .......... .......... .......... .......... 43% 22.8M 3s 2024-06-28T15:27:03.793 INFO:teuthology.orchestra.run.smithi178.stderr: 40750K .......... .......... .......... .......... .......... 43% 27.2M 3s 2024-06-28T15:27:03.796 INFO:teuthology.orchestra.run.smithi178.stderr: 40800K .......... .......... .......... .......... .......... 43% 19.3M 3s 2024-06-28T15:27:03.798 INFO:teuthology.orchestra.run.smithi178.stderr: 40850K .......... .......... .......... .......... .......... 44% 20.2M 3s 2024-06-28T15:27:03.801 INFO:teuthology.orchestra.run.smithi178.stderr: 40900K .......... .......... .......... .......... .......... 44% 19.1M 3s 2024-06-28T15:27:03.803 INFO:teuthology.orchestra.run.smithi178.stderr: 40950K .......... .......... .......... .......... .......... 44% 24.7M 3s 2024-06-28T15:27:03.807 INFO:teuthology.orchestra.run.smithi178.stderr: 41000K .......... .......... .......... .......... .......... 44% 12.7M 3s 2024-06-28T15:27:03.809 INFO:teuthology.orchestra.run.smithi178.stderr: 41050K .......... .......... .......... .......... .......... 44% 32.8M 3s 2024-06-28T15:27:03.813 INFO:teuthology.orchestra.run.smithi178.stderr: 41100K .......... .......... .......... .......... .......... 44% 10.4M 3s 2024-06-28T15:27:03.815 INFO:teuthology.orchestra.run.smithi178.stderr: 41150K .......... .......... .......... .......... .......... 44% 24.6M 3s 2024-06-28T15:27:03.818 INFO:teuthology.orchestra.run.smithi178.stderr: 41200K .......... .......... .......... .......... .......... 44% 30.3M 3s 2024-06-28T15:27:03.820 INFO:teuthology.orchestra.run.smithi178.stderr: 41250K .......... .......... .......... .......... .......... 44% 16.7M 3s 2024-06-28T15:27:03.822 INFO:teuthology.orchestra.run.smithi178.stderr: 41300K .......... .......... .......... .......... .......... 44% 20.9M 3s 2024-06-28T15:27:03.823 INFO:teuthology.orchestra.run.smithi178.stderr: 41350K .......... .......... .......... .......... .......... 44% 33.6M 3s 2024-06-28T15:27:03.825 INFO:teuthology.orchestra.run.smithi178.stderr: 41400K .......... .......... .......... .......... .......... 44% 27.2M 3s 2024-06-28T15:27:03.831 INFO:teuthology.orchestra.run.smithi178.stderr: 41450K .......... .......... .......... .......... .......... 44% 9.15M 3s 2024-06-28T15:27:03.832 INFO:teuthology.orchestra.run.smithi178.stderr: 41500K .......... .......... .......... .......... .......... 44% 69.7M 3s 2024-06-28T15:27:03.832 INFO:teuthology.orchestra.run.smithi178.stderr: 41550K .......... .......... .......... .......... .......... 44% 33.3M 3s 2024-06-28T15:27:03.834 INFO:teuthology.orchestra.run.smithi178.stderr: 41600K .......... .......... .......... .......... .......... 44% 31.4M 3s 2024-06-28T15:27:03.836 INFO:teuthology.orchestra.run.smithi178.stderr: 41650K .......... .......... .......... .......... .......... 44% 27.3M 3s 2024-06-28T15:27:03.839 INFO:teuthology.orchestra.run.smithi178.stderr: 41700K .......... .......... .......... .......... .......... 44% 17.7M 3s 2024-06-28T15:27:03.841 INFO:teuthology.orchestra.run.smithi178.stderr: 41750K .......... .......... .......... .......... .......... 44% 20.8M 3s 2024-06-28T15:27:03.845 INFO:teuthology.orchestra.run.smithi178.stderr: 41800K .......... .......... .......... .......... .......... 45% 11.7M 3s 2024-06-28T15:27:03.848 INFO:teuthology.orchestra.run.smithi178.stderr: 41850K .......... .......... .......... .......... .......... 45% 15.6M 3s 2024-06-28T15:27:03.852 INFO:teuthology.orchestra.run.smithi178.stderr: 41900K .......... .......... .......... .......... .......... 45% 13.0M 3s 2024-06-28T15:27:03.854 INFO:teuthology.orchestra.run.smithi178.stderr: 41950K .......... .......... .......... .......... .......... 45% 33.8M 3s 2024-06-28T15:27:03.855 INFO:teuthology.orchestra.run.smithi178.stderr: 42000K .......... .......... .......... .......... .......... 45% 35.8M 3s 2024-06-28T15:27:03.858 INFO:teuthology.orchestra.run.smithi178.stderr: 42050K .......... .......... .......... .......... .......... 45% 14.3M 3s 2024-06-28T15:27:03.863 INFO:teuthology.orchestra.run.smithi178.stderr: 42100K .......... .......... .......... .......... .......... 45% 11.2M 3s 2024-06-28T15:27:03.865 INFO:teuthology.orchestra.run.smithi178.stderr: 42150K .......... .......... .......... .......... .......... 45% 24.1M 3s 2024-06-28T15:27:03.866 INFO:teuthology.orchestra.run.smithi178.stderr: 42200K .......... .......... .......... .......... .......... 45% 26.7M 3s 2024-06-28T15:27:03.871 INFO:teuthology.orchestra.run.smithi178.stderr: 42250K .......... .......... .......... .......... .......... 45% 26.9M 3s 2024-06-28T15:27:03.871 INFO:teuthology.orchestra.run.smithi178.stderr: 42300K .......... .......... .......... .......... .......... 45% 21.2M 3s 2024-06-28T15:27:03.876 INFO:teuthology.orchestra.run.smithi178.stderr: 42350K .......... .......... .......... .......... .......... 45% 9.84M 3s 2024-06-28T15:27:03.877 INFO:teuthology.orchestra.run.smithi178.stderr: 42400K .......... .......... .......... .......... .......... 45% 34.6M 3s 2024-06-28T15:27:03.880 INFO:teuthology.orchestra.run.smithi178.stderr: 42450K .......... .......... .......... .......... .......... 45% 17.8M 3s 2024-06-28T15:27:03.882 INFO:teuthology.orchestra.run.smithi178.stderr: 42500K .......... .......... .......... .......... .......... 45% 24.8M 3s 2024-06-28T15:27:03.884 INFO:teuthology.orchestra.run.smithi178.stderr: 42550K .......... .......... .......... .......... .......... 45% 28.2M 3s 2024-06-28T15:27:03.884 INFO:teuthology.orchestra.run.smithi178.stderr: 42600K .......... .......... .......... .......... .......... 45% 45.2M 3s 2024-06-28T15:27:03.886 INFO:teuthology.orchestra.run.smithi178.stderr: 42650K .......... .......... .......... .......... .......... 45% 54.8M 3s 2024-06-28T15:27:03.889 INFO:teuthology.orchestra.run.smithi178.stderr: 42700K .......... .......... .......... .......... .......... 46% 12.2M 3s 2024-06-28T15:27:03.893 INFO:teuthology.orchestra.run.smithi178.stderr: 42750K .......... .......... .......... .......... .......... 46% 14.9M 3s 2024-06-28T15:27:03.896 INFO:teuthology.orchestra.run.smithi178.stderr: 42800K .......... .......... .......... .......... .......... 46% 16.7M 3s 2024-06-28T15:27:03.899 INFO:teuthology.orchestra.run.smithi178.stderr: 42850K .......... .......... .......... .......... .......... 46% 14.7M 3s 2024-06-28T15:27:03.901 INFO:teuthology.orchestra.run.smithi178.stderr: 42900K .......... .......... .......... .......... .......... 46% 18.4M 3s 2024-06-28T15:27:03.903 INFO:teuthology.orchestra.run.smithi178.stderr: 42950K .......... .......... .......... .......... .......... 46% 27.2M 3s 2024-06-28T15:27:03.905 INFO:teuthology.orchestra.run.smithi178.stderr: 43000K .......... .......... .......... .......... .......... 46% 25.1M 3s 2024-06-28T15:27:03.907 INFO:teuthology.orchestra.run.smithi178.stderr: 43050K .......... .......... .......... .......... .......... 46% 27.5M 3s 2024-06-28T15:27:03.908 INFO:teuthology.orchestra.run.smithi178.stderr: 43100K .......... .......... .......... .......... .......... 46% 32.2M 3s 2024-06-28T15:27:03.912 INFO:teuthology.orchestra.run.smithi178.stderr: 43150K .......... .......... .......... .......... .......... 46% 16.2M 3s 2024-06-28T15:27:03.914 INFO:teuthology.orchestra.run.smithi178.stderr: 43200K .......... .......... .......... .......... .......... 46% 29.4M 3s 2024-06-28T15:27:03.914 INFO:teuthology.orchestra.run.smithi178.stderr: 43250K .......... .......... .......... .......... .......... 46% 57.8M 2s 2024-06-28T15:27:03.917 INFO:teuthology.orchestra.run.smithi178.stderr: 43300K .......... .......... .......... .......... .......... 46% 15.6M 2s 2024-06-28T15:27:03.920 INFO:teuthology.orchestra.run.smithi178.stderr: 43350K .......... .......... .......... .......... .......... 46% 20.7M 2s 2024-06-28T15:27:03.921 INFO:teuthology.orchestra.run.smithi178.stderr: 43400K .......... .......... .......... .......... .......... 46% 24.8M 2s 2024-06-28T15:27:03.924 INFO:teuthology.orchestra.run.smithi178.stderr: 43450K .......... .......... .......... .......... .......... 46% 27.1M 2s 2024-06-28T15:27:03.926 INFO:teuthology.orchestra.run.smithi178.stderr: 43500K .......... .......... .......... .......... .......... 46% 20.4M 2s 2024-06-28T15:27:03.928 INFO:teuthology.orchestra.run.smithi178.stderr: 43550K .......... .......... .......... .......... .......... 46% 34.3M 2s 2024-06-28T15:27:03.929 INFO:teuthology.orchestra.run.smithi178.stderr: 43600K .......... .......... .......... .......... .......... 46% 34.6M 2s 2024-06-28T15:27:03.932 INFO:teuthology.orchestra.run.smithi178.stderr: 43650K .......... .......... .......... .......... .......... 47% 12.8M 2s 2024-06-28T15:27:03.934 INFO:teuthology.orchestra.run.smithi178.stderr: 43700K .......... .......... .......... .......... .......... 47% 28.8M 2s 2024-06-28T15:27:03.936 INFO:teuthology.orchestra.run.smithi178.stderr: 43750K .......... .......... .......... .......... .......... 47% 24.1M 2s 2024-06-28T15:27:03.939 INFO:teuthology.orchestra.run.smithi178.stderr: 43800K .......... .......... .......... .......... .......... 47% 17.5M 2s 2024-06-28T15:27:03.942 INFO:teuthology.orchestra.run.smithi178.stderr: 43850K .......... .......... .......... .......... .......... 47% 21.6M 2s 2024-06-28T15:27:03.944 INFO:teuthology.orchestra.run.smithi178.stderr: 43900K .......... .......... .......... .......... .......... 47% 19.1M 2s 2024-06-28T15:27:03.945 INFO:teuthology.orchestra.run.smithi178.stderr: 43950K .......... .......... .......... .......... .......... 47% 27.3M 2s 2024-06-28T15:27:03.947 INFO:teuthology.orchestra.run.smithi178.stderr: 44000K .......... .......... .......... .......... .......... 47% 32.2M 2s 2024-06-28T15:27:03.949 INFO:teuthology.orchestra.run.smithi178.stderr: 44050K .......... .......... .......... .......... .......... 47% 22.4M 2s 2024-06-28T15:27:03.950 INFO:teuthology.orchestra.run.smithi178.stderr: 44100K .......... .......... .......... .......... .......... 47% 40.8M 2s 2024-06-28T15:27:03.952 INFO:teuthology.orchestra.run.smithi178.stderr: 44150K .......... .......... .......... .......... .......... 47% 32.1M 2s 2024-06-28T15:27:03.954 INFO:teuthology.orchestra.run.smithi178.stderr: 44200K .......... .......... .......... .......... .......... 47% 26.3M 2s 2024-06-28T15:27:03.957 INFO:teuthology.orchestra.run.smithi178.stderr: 44250K .......... .......... .......... .......... .......... 47% 14.2M 2s 2024-06-28T15:27:03.962 INFO:teuthology.orchestra.run.smithi178.stderr: 44300K .......... .......... .......... .......... .......... 47% 12.7M 2s 2024-06-28T15:27:03.963 INFO:teuthology.orchestra.run.smithi178.stderr: 44350K .......... .......... .......... .......... .......... 47% 46.6M 2s 2024-06-28T15:27:03.963 INFO:teuthology.orchestra.run.smithi178.stderr: 44400K .......... .......... .......... .......... .......... 47% 49.8M 2s 2024-06-28T15:27:03.965 INFO:teuthology.orchestra.run.smithi178.stderr: 44450K .......... .......... .......... .......... .......... 47% 35.6M 2s 2024-06-28T15:27:03.966 INFO:teuthology.orchestra.run.smithi178.stderr: 44500K .......... .......... .......... .......... .......... 47% 42.1M 2s 2024-06-28T15:27:03.967 INFO:teuthology.orchestra.run.smithi178.stderr: 44550K .......... .......... .......... .......... .......... 48% 40.0M 2s 2024-06-28T15:27:03.969 INFO:teuthology.orchestra.run.smithi178.stderr: 44600K .......... .......... .......... .......... .......... 48% 27.2M 2s 2024-06-28T15:27:03.972 INFO:teuthology.orchestra.run.smithi178.stderr: 44650K .......... .......... .......... .......... .......... 48% 19.1M 2s 2024-06-28T15:27:03.975 INFO:teuthology.orchestra.run.smithi178.stderr: 44700K .......... .......... .......... .......... .......... 48% 14.4M 2s 2024-06-28T15:27:03.978 INFO:teuthology.orchestra.run.smithi178.stderr: 44750K .......... .......... .......... .......... .......... 48% 17.0M 2s 2024-06-28T15:27:03.979 INFO:teuthology.orchestra.run.smithi178.stderr: 44800K .......... .......... .......... .......... .......... 48% 30.4M 2s 2024-06-28T15:27:03.981 INFO:teuthology.orchestra.run.smithi178.stderr: 44850K .......... .......... .......... .......... .......... 48% 19.5M 2s 2024-06-28T15:27:03.983 INFO:teuthology.orchestra.run.smithi178.stderr: 44900K .......... .......... .......... .......... .......... 48% 32.1M 2s 2024-06-28T15:27:03.985 INFO:teuthology.orchestra.run.smithi178.stderr: 44950K .......... .......... .......... .......... .......... 48% 23.9M 2s 2024-06-28T15:27:03.987 INFO:teuthology.orchestra.run.smithi178.stderr: 45000K .......... .......... .......... .......... .......... 48% 26.5M 2s 2024-06-28T15:27:03.990 INFO:teuthology.orchestra.run.smithi178.stderr: 45050K .......... .......... .......... .......... .......... 48% 22.3M 2s 2024-06-28T15:27:03.992 INFO:teuthology.orchestra.run.smithi178.stderr: 45100K .......... .......... .......... .......... .......... 48% 16.8M 2s 2024-06-28T15:27:03.996 INFO:teuthology.orchestra.run.smithi178.stderr: 45150K .......... .......... .......... .......... .......... 48% 13.1M 2s 2024-06-28T15:27:03.999 INFO:teuthology.orchestra.run.smithi178.stderr: 45200K .......... .......... .......... .......... .......... 48% 23.2M 2s 2024-06-28T15:27:04.002 INFO:teuthology.orchestra.run.smithi178.stderr: 45250K .......... .......... .......... .......... .......... 48% 19.4M 2s 2024-06-28T15:27:04.002 INFO:teuthology.orchestra.run.smithi178.stderr: 45300K .......... .......... .......... .......... .......... 48% 41.5M 2s 2024-06-28T15:27:04.004 INFO:teuthology.orchestra.run.smithi178.stderr: 45350K .......... .......... .......... .......... .......... 48% 19.6M 2s 2024-06-28T15:27:04.005 INFO:teuthology.orchestra.run.smithi178.stderr: 45400K .......... .......... .......... .......... .......... 48% 37.0M 2s 2024-06-28T15:27:04.008 INFO:teuthology.orchestra.run.smithi178.stderr: 45450K .......... .......... .......... .......... .......... 48% 27.7M 2s 2024-06-28T15:27:04.009 INFO:teuthology.orchestra.run.smithi178.stderr: 45500K .......... .......... .......... .......... .......... 49% 27.4M 2s 2024-06-28T15:27:04.011 INFO:teuthology.orchestra.run.smithi178.stderr: 45550K .......... .......... .......... .......... .......... 49% 30.4M 2s 2024-06-28T15:27:04.013 INFO:teuthology.orchestra.run.smithi178.stderr: 45600K .......... .......... .......... .......... .......... 49% 20.1M 2s 2024-06-28T15:27:04.015 INFO:teuthology.orchestra.run.smithi178.stderr: 45650K .......... .......... .......... .......... .......... 49% 20.2M 2s 2024-06-28T15:27:04.017 INFO:teuthology.orchestra.run.smithi178.stderr: 45700K .......... .......... .......... .......... .......... 49% 30.0M 2s 2024-06-28T15:27:04.019 INFO:teuthology.orchestra.run.smithi178.stderr: 45750K .......... .......... .......... .......... .......... 49% 34.2M 2s 2024-06-28T15:27:04.021 INFO:teuthology.orchestra.run.smithi178.stderr: 45800K .......... .......... .......... .......... .......... 49% 21.7M 2s 2024-06-28T15:27:04.024 INFO:teuthology.orchestra.run.smithi178.stderr: 45850K .......... .......... .......... .......... .......... 49% 24.0M 2s 2024-06-28T15:27:04.027 INFO:teuthology.orchestra.run.smithi178.stderr: 45900K .......... .......... .......... .......... .......... 49% 12.7M 2s 2024-06-28T15:27:04.030 INFO:teuthology.orchestra.run.smithi178.stderr: 45950K .......... .......... .......... .......... .......... 49% 17.4M 2s 2024-06-28T15:27:04.031 INFO:teuthology.orchestra.run.smithi178.stderr: 46000K .......... .......... .......... .......... .......... 49% 38.8M 2s 2024-06-28T15:27:04.035 INFO:teuthology.orchestra.run.smithi178.stderr: 46050K .......... .......... .......... .......... .......... 49% 12.4M 2s 2024-06-28T15:27:04.037 INFO:teuthology.orchestra.run.smithi178.stderr: 46100K .......... .......... .......... .......... .......... 49% 26.0M 2s 2024-06-28T15:27:04.039 INFO:teuthology.orchestra.run.smithi178.stderr: 46150K .......... .......... .......... .......... .......... 49% 19.2M 2s 2024-06-28T15:27:04.041 INFO:teuthology.orchestra.run.smithi178.stderr: 46200K .......... .......... .......... .......... .......... 49% 27.1M 2s 2024-06-28T15:27:04.043 INFO:teuthology.orchestra.run.smithi178.stderr: 46250K .......... .......... .......... .......... .......... 49% 27.1M 2s 2024-06-28T15:27:04.045 INFO:teuthology.orchestra.run.smithi178.stderr: 46300K .......... .......... .......... .......... .......... 49% 19.2M 2s 2024-06-28T15:27:04.046 INFO:teuthology.orchestra.run.smithi178.stderr: 46350K .......... .......... .......... .......... .......... 49% 49.0M 2s 2024-06-28T15:27:04.048 INFO:teuthology.orchestra.run.smithi178.stderr: 46400K .......... .......... .......... .......... .......... 49% 34.7M 2s 2024-06-28T15:27:04.049 INFO:teuthology.orchestra.run.smithi178.stderr: 46450K .......... .......... .......... .......... .......... 50% 45.5M 2s 2024-06-28T15:27:04.050 INFO:teuthology.orchestra.run.smithi178.stderr: 46500K .......... .......... .......... .......... .......... 50% 32.4M 2s 2024-06-28T15:27:04.051 INFO:teuthology.orchestra.run.smithi178.stderr: 46550K .......... .......... .......... .......... .......... 50% 33.3M 2s 2024-06-28T15:27:04.054 INFO:teuthology.orchestra.run.smithi178.stderr: 46600K .......... .......... .......... .......... .......... 50% 22.6M 2s 2024-06-28T15:27:04.056 INFO:teuthology.orchestra.run.smithi178.stderr: 46650K .......... .......... .......... .......... .......... 50% 35.7M 2s 2024-06-28T15:27:04.058 INFO:teuthology.orchestra.run.smithi178.stderr: 46700K .......... .......... .......... .......... .......... 50% 25.4M 2s 2024-06-28T15:27:04.059 INFO:teuthology.orchestra.run.smithi178.stderr: 46750K .......... .......... .......... .......... .......... 50% 40.3M 2s 2024-06-28T15:27:04.063 INFO:teuthology.orchestra.run.smithi178.stderr: 46800K .......... .......... .......... .......... .......... 50% 11.4M 2s 2024-06-28T15:27:04.064 INFO:teuthology.orchestra.run.smithi178.stderr: 46850K .......... .......... .......... .......... .......... 50% 30.5M 2s 2024-06-28T15:27:04.066 INFO:teuthology.orchestra.run.smithi178.stderr: 46900K .......... .......... .......... .......... .......... 50% 22.2M 2s 2024-06-28T15:27:04.070 INFO:teuthology.orchestra.run.smithi178.stderr: 46950K .......... .......... .......... .......... .......... 50% 14.4M 2s 2024-06-28T15:27:04.071 INFO:teuthology.orchestra.run.smithi178.stderr: 47000K .......... .......... .......... .......... .......... 50% 33.5M 2s 2024-06-28T15:27:04.077 INFO:teuthology.orchestra.run.smithi178.stderr: 47050K .......... .......... .......... .......... .......... 50% 9.09M 2s 2024-06-28T15:27:04.079 INFO:teuthology.orchestra.run.smithi178.stderr: 47100K .......... .......... .......... .......... .......... 50% 23.6M 2s 2024-06-28T15:27:04.082 INFO:teuthology.orchestra.run.smithi178.stderr: 47150K .......... .......... .......... .......... .......... 50% 13.0M 2s 2024-06-28T15:27:04.084 INFO:teuthology.orchestra.run.smithi178.stderr: 47200K .......... .......... .......... .......... .......... 50% 28.4M 2s 2024-06-28T15:27:04.085 INFO:teuthology.orchestra.run.smithi178.stderr: 47250K .......... .......... .......... .......... .......... 50% 40.5M 2s 2024-06-28T15:27:04.088 INFO:teuthology.orchestra.run.smithi178.stderr: 47300K .......... .......... .......... .......... .......... 50% 21.3M 2s 2024-06-28T15:27:04.090 INFO:teuthology.orchestra.run.smithi178.stderr: 47350K .......... .......... .......... .......... .......... 51% 26.6M 2s 2024-06-28T15:27:04.091 INFO:teuthology.orchestra.run.smithi178.stderr: 47400K .......... .......... .......... .......... .......... 51% 39.0M 2s 2024-06-28T15:27:04.092 INFO:teuthology.orchestra.run.smithi178.stderr: 47450K .......... .......... .......... .......... .......... 51% 45.7M 2s 2024-06-28T15:27:04.093 INFO:teuthology.orchestra.run.smithi178.stderr: 47500K .......... .......... .......... .......... .......... 51% 43.4M 2s 2024-06-28T15:27:04.095 INFO:teuthology.orchestra.run.smithi178.stderr: 47550K .......... .......... .......... .......... .......... 51% 35.2M 2s 2024-06-28T15:27:04.099 INFO:teuthology.orchestra.run.smithi178.stderr: 47600K .......... .......... .......... .......... .......... 51% 13.8M 2s 2024-06-28T15:27:04.099 INFO:teuthology.orchestra.run.smithi178.stderr: 47650K .......... .......... .......... .......... .......... 51% 56.3M 2s 2024-06-28T15:27:04.100 INFO:teuthology.orchestra.run.smithi178.stderr: 47700K .......... .......... .......... .......... .......... 51% 30.5M 2s 2024-06-28T15:27:04.103 INFO:teuthology.orchestra.run.smithi178.stderr: 47750K .......... .......... .......... .......... .......... 51% 18.6M 2s 2024-06-28T15:27:04.104 INFO:teuthology.orchestra.run.smithi178.stderr: 47800K .......... .......... .......... .......... .......... 51% 37.6M 2s 2024-06-28T15:27:04.106 INFO:teuthology.orchestra.run.smithi178.stderr: 47850K .......... .......... .......... .......... .......... 51% 40.2M 2s 2024-06-28T15:27:04.108 INFO:teuthology.orchestra.run.smithi178.stderr: 47900K .......... .......... .......... .......... .......... 51% 18.6M 2s 2024-06-28T15:27:04.114 INFO:teuthology.orchestra.run.smithi178.stderr: 47950K .......... .......... .......... .......... .......... 51% 8.85M 2s 2024-06-28T15:27:04.116 INFO:teuthology.orchestra.run.smithi178.stderr: 48000K .......... .......... .......... .......... .......... 51% 16.7M 2s 2024-06-28T15:27:04.118 INFO:teuthology.orchestra.run.smithi178.stderr: 48050K .......... .......... .......... .......... .......... 51% 36.8M 2s 2024-06-28T15:27:04.124 INFO:teuthology.orchestra.run.smithi178.stderr: 48100K .......... .......... .......... .......... .......... 51% 10.3M 2s 2024-06-28T15:27:04.125 INFO:teuthology.orchestra.run.smithi178.stderr: 48150K .......... .......... .......... .......... .......... 51% 23.1M 2s 2024-06-28T15:27:04.125 INFO:teuthology.orchestra.run.smithi178.stderr: 48200K .......... .......... .......... .......... .......... 51% 48.5M 2s 2024-06-28T15:27:04.127 INFO:teuthology.orchestra.run.smithi178.stderr: 48250K .......... .......... .......... .......... .......... 51% 30.8M 2s 2024-06-28T15:27:04.128 INFO:teuthology.orchestra.run.smithi178.stderr: 48300K .......... .......... .......... .......... .......... 52% 39.4M 2s 2024-06-28T15:27:04.130 INFO:teuthology.orchestra.run.smithi178.stderr: 48350K .......... .......... .......... .......... .......... 52% 47.2M 2s 2024-06-28T15:27:04.133 INFO:teuthology.orchestra.run.smithi178.stderr: 48400K .......... .......... .......... .......... .......... 52% 14.1M 2s 2024-06-28T15:27:04.135 INFO:teuthology.orchestra.run.smithi178.stderr: 48450K .......... .......... .......... .......... .......... 52% 23.1M 2s 2024-06-28T15:27:04.138 INFO:teuthology.orchestra.run.smithi178.stderr: 48500K .......... .......... .......... .......... .......... 52% 18.6M 2s 2024-06-28T15:27:04.139 INFO:teuthology.orchestra.run.smithi178.stderr: 48550K .......... .......... .......... .......... .......... 52% 38.5M 2s 2024-06-28T15:27:04.142 INFO:teuthology.orchestra.run.smithi178.stderr: 48600K .......... .......... .......... .......... .......... 52% 15.6M 2s 2024-06-28T15:27:04.145 INFO:teuthology.orchestra.run.smithi178.stderr: 48650K .......... .......... .......... .......... .......... 52% 29.7M 2s 2024-06-28T15:27:04.147 INFO:teuthology.orchestra.run.smithi178.stderr: 48700K .......... .......... .......... .......... .......... 52% 17.1M 2s 2024-06-28T15:27:04.149 INFO:teuthology.orchestra.run.smithi178.stderr: 48750K .......... .......... .......... .......... .......... 52% 28.5M 2s 2024-06-28T15:27:04.150 INFO:teuthology.orchestra.run.smithi178.stderr: 48800K .......... .......... .......... .......... .......... 52% 38.3M 2s 2024-06-28T15:27:04.151 INFO:teuthology.orchestra.run.smithi178.stderr: 48850K .......... .......... .......... .......... .......... 52% 38.6M 2s 2024-06-28T15:27:04.152 INFO:teuthology.orchestra.run.smithi178.stderr: 48900K .......... .......... .......... .......... .......... 52% 49.1M 2s 2024-06-28T15:27:04.156 INFO:teuthology.orchestra.run.smithi178.stderr: 48950K .......... .......... .......... .......... .......... 52% 13.0M 2s 2024-06-28T15:27:04.157 INFO:teuthology.orchestra.run.smithi178.stderr: 49000K .......... .......... .......... .......... .......... 52% 37.7M 2s 2024-06-28T15:27:04.160 INFO:teuthology.orchestra.run.smithi178.stderr: 49050K .......... .......... .......... .......... .......... 52% 22.9M 2s 2024-06-28T15:27:04.163 INFO:teuthology.orchestra.run.smithi178.stderr: 49100K .......... .......... .......... .......... .......... 52% 16.6M 2s 2024-06-28T15:27:04.164 INFO:teuthology.orchestra.run.smithi178.stderr: 49150K .......... .......... .......... .......... .......... 52% 51.5M 2s 2024-06-28T15:27:04.164 INFO:teuthology.orchestra.run.smithi178.stderr: 49200K .......... .......... .......... .......... .......... 53% 37.2M 2s 2024-06-28T15:27:04.167 INFO:teuthology.orchestra.run.smithi178.stderr: 49250K .......... .......... .......... .......... .......... 53% 16.3M 2s 2024-06-28T15:27:04.170 INFO:teuthology.orchestra.run.smithi178.stderr: 49300K .......... .......... .......... .......... .......... 53% 17.6M 2s 2024-06-28T15:27:04.171 INFO:teuthology.orchestra.run.smithi178.stderr: 49350K .......... .......... .......... .......... .......... 53% 38.5M 2s 2024-06-28T15:27:04.177 INFO:teuthology.orchestra.run.smithi178.stderr: 49400K .......... .......... .......... .......... .......... 53% 8.03M 2s 2024-06-28T15:27:04.181 INFO:teuthology.orchestra.run.smithi178.stderr: 49450K .......... .......... .......... .......... .......... 53% 27.2M 2s 2024-06-28T15:27:04.182 INFO:teuthology.orchestra.run.smithi178.stderr: 49500K .......... .......... .......... .......... .......... 53% 17.9M 2s 2024-06-28T15:27:04.185 INFO:teuthology.orchestra.run.smithi178.stderr: 49550K .......... .......... .......... .......... .......... 53% 17.7M 2s 2024-06-28T15:27:04.186 INFO:teuthology.orchestra.run.smithi178.stderr: 49600K .......... .......... .......... .......... .......... 53% 34.1M 2s 2024-06-28T15:27:04.188 INFO:teuthology.orchestra.run.smithi178.stderr: 49650K .......... .......... .......... .......... .......... 53% 22.5M 2s 2024-06-28T15:27:04.190 INFO:teuthology.orchestra.run.smithi178.stderr: 49700K .......... .......... .......... .......... .......... 53% 35.7M 2s 2024-06-28T15:27:04.191 INFO:teuthology.orchestra.run.smithi178.stderr: 49750K .......... .......... .......... .......... .......... 53% 29.9M 2s 2024-06-28T15:27:04.193 INFO:teuthology.orchestra.run.smithi178.stderr: 49800K .......... .......... .......... .......... .......... 53% 28.4M 2s 2024-06-28T15:27:04.196 INFO:teuthology.orchestra.run.smithi178.stderr: 49850K .......... .......... .......... .......... .......... 53% 17.5M 2s 2024-06-28T15:27:04.199 INFO:teuthology.orchestra.run.smithi178.stderr: 49900K .......... .......... .......... .......... .......... 53% 17.5M 2s 2024-06-28T15:27:04.200 INFO:teuthology.orchestra.run.smithi178.stderr: 49950K .......... .......... .......... .......... .......... 53% 33.9M 2s 2024-06-28T15:27:04.203 INFO:teuthology.orchestra.run.smithi178.stderr: 50000K .......... .......... .......... .......... .......... 53% 31.6M 2s 2024-06-28T15:27:04.203 INFO:teuthology.orchestra.run.smithi178.stderr: 50050K .......... .......... .......... .......... .......... 53% 88.0M 2s 2024-06-28T15:27:04.203 INFO:teuthology.orchestra.run.smithi178.stderr: 50100K .......... .......... .......... .......... .......... 53% 49.8M 2s 2024-06-28T15:27:04.204 INFO:teuthology.orchestra.run.smithi178.stderr: 50150K .......... .......... .......... .......... .......... 54% 53.5M 2s 2024-06-28T15:27:04.209 INFO:teuthology.orchestra.run.smithi178.stderr: 50200K .......... .......... .......... .......... .......... 54% 9.06M 2s 2024-06-28T15:27:04.212 INFO:teuthology.orchestra.run.smithi178.stderr: 50250K .......... .......... .......... .......... .......... 54% 45.6M 2s 2024-06-28T15:27:04.213 INFO:teuthology.orchestra.run.smithi178.stderr: 50300K .......... .......... .......... .......... .......... 54% 18.2M 2s 2024-06-28T15:27:04.217 INFO:teuthology.orchestra.run.smithi178.stderr: 50350K .......... .......... .......... .......... .......... 54% 13.3M 2s 2024-06-28T15:27:04.219 INFO:teuthology.orchestra.run.smithi178.stderr: 50400K .......... .......... .......... .......... .......... 54% 26.8M 2s 2024-06-28T15:27:04.221 INFO:teuthology.orchestra.run.smithi178.stderr: 50450K .......... .......... .......... .......... .......... 54% 19.0M 2s 2024-06-28T15:27:04.225 INFO:teuthology.orchestra.run.smithi178.stderr: 50500K .......... .......... .......... .......... .......... 54% 14.6M 2s 2024-06-28T15:27:04.226 INFO:teuthology.orchestra.run.smithi178.stderr: 50550K .......... .......... .......... .......... .......... 54% 37.6M 2s 2024-06-28T15:27:04.227 INFO:teuthology.orchestra.run.smithi178.stderr: 50600K .......... .......... .......... .......... .......... 54% 45.9M 2s 2024-06-28T15:27:04.229 INFO:teuthology.orchestra.run.smithi178.stderr: 50650K .......... .......... .......... .......... .......... 54% 26.9M 2s 2024-06-28T15:27:04.230 INFO:teuthology.orchestra.run.smithi178.stderr: 50700K .......... .......... .......... .......... .......... 54% 26.4M 2s 2024-06-28T15:27:04.233 INFO:teuthology.orchestra.run.smithi178.stderr: 50750K .......... .......... .......... .......... .......... 54% 24.8M 2s 2024-06-28T15:27:04.233 INFO:teuthology.orchestra.run.smithi178.stderr: 50800K .......... .......... .......... .......... .......... 54% 82.0M 2s 2024-06-28T15:27:04.235 INFO:teuthology.orchestra.run.smithi178.stderr: 50850K .......... .......... .......... .......... .......... 54% 28.1M 2s 2024-06-28T15:27:04.236 INFO:teuthology.orchestra.run.smithi178.stderr: 50900K .......... .......... .......... .......... .......... 54% 44.1M 2s 2024-06-28T15:27:04.237 INFO:teuthology.orchestra.run.smithi178.stderr: 50950K .......... .......... .......... .......... .......... 54% 49.7M 2s 2024-06-28T15:27:04.238 INFO:teuthology.orchestra.run.smithi178.stderr: 51000K .......... .......... .......... .......... .......... 54% 57.6M 2s 2024-06-28T15:27:04.240 INFO:teuthology.orchestra.run.smithi178.stderr: 51050K .......... .......... .......... .......... .......... 55% 21.3M 2s 2024-06-28T15:27:04.243 INFO:teuthology.orchestra.run.smithi178.stderr: 51100K .......... .......... .......... .......... .......... 55% 15.1M 2s 2024-06-28T15:27:04.246 INFO:teuthology.orchestra.run.smithi178.stderr: 51150K .......... .......... .......... .......... .......... 55% 19.1M 2s 2024-06-28T15:27:04.249 INFO:teuthology.orchestra.run.smithi178.stderr: 51200K .......... .......... .......... .......... .......... 55% 17.6M 2s 2024-06-28T15:27:04.250 INFO:teuthology.orchestra.run.smithi178.stderr: 51250K .......... .......... .......... .......... .......... 55% 28.1M 2s 2024-06-28T15:27:04.254 INFO:teuthology.orchestra.run.smithi178.stderr: 51300K .......... .......... .......... .......... .......... 55% 15.8M 2s 2024-06-28T15:27:04.258 INFO:teuthology.orchestra.run.smithi178.stderr: 51350K .......... .......... .......... .......... .......... 55% 10.4M 2s 2024-06-28T15:27:04.261 INFO:teuthology.orchestra.run.smithi178.stderr: 51400K .......... .......... .......... .......... .......... 55% 16.2M 2s 2024-06-28T15:27:04.264 INFO:teuthology.orchestra.run.smithi178.stderr: 51450K .......... .......... .......... .......... .......... 55% 31.4M 2s 2024-06-28T15:27:04.265 INFO:teuthology.orchestra.run.smithi178.stderr: 51500K .......... .......... .......... .......... .......... 55% 19.1M 2s 2024-06-28T15:27:04.268 INFO:teuthology.orchestra.run.smithi178.stderr: 51550K .......... .......... .......... .......... .......... 55% 22.4M 2s 2024-06-28T15:27:04.268 INFO:teuthology.orchestra.run.smithi178.stderr: 51600K .......... .......... .......... .......... .......... 55% 51.0M 2s 2024-06-28T15:27:04.270 INFO:teuthology.orchestra.run.smithi178.stderr: 51650K .......... .......... .......... .......... .......... 55% 29.3M 2s 2024-06-28T15:27:04.273 INFO:teuthology.orchestra.run.smithi178.stderr: 51700K .......... .......... .......... .......... .......... 55% 32.3M 2s 2024-06-28T15:27:04.273 INFO:teuthology.orchestra.run.smithi178.stderr: 51750K .......... .......... .......... .......... .......... 55% 51.6M 2s 2024-06-28T15:27:04.274 INFO:teuthology.orchestra.run.smithi178.stderr: 51800K .......... .......... .......... .......... .......... 55% 62.2M 2s 2024-06-28T15:27:04.276 INFO:teuthology.orchestra.run.smithi178.stderr: 51850K .......... .......... .......... .......... .......... 55% 21.3M 2s 2024-06-28T15:27:04.278 INFO:teuthology.orchestra.run.smithi178.stderr: 51900K .......... .......... .......... .......... .......... 55% 25.4M 2s 2024-06-28T15:27:04.279 INFO:teuthology.orchestra.run.smithi178.stderr: 51950K .......... .......... .......... .......... .......... 55% 37.0M 2s 2024-06-28T15:27:04.283 INFO:teuthology.orchestra.run.smithi178.stderr: 52000K .......... .......... .......... .......... .......... 56% 12.2M 2s 2024-06-28T15:27:04.284 INFO:teuthology.orchestra.run.smithi178.stderr: 52050K .......... .......... .......... .......... .......... 56% 44.1M 2s 2024-06-28T15:27:04.287 INFO:teuthology.orchestra.run.smithi178.stderr: 52100K .......... .......... .......... .......... .......... 56% 19.0M 2s 2024-06-28T15:27:04.288 INFO:teuthology.orchestra.run.smithi178.stderr: 52150K .......... .......... .......... .......... .......... 56% 27.8M 2s 2024-06-28T15:27:04.292 INFO:teuthology.orchestra.run.smithi178.stderr: 52200K .......... .......... .......... .......... .......... 56% 13.6M 2s 2024-06-28T15:27:04.294 INFO:teuthology.orchestra.run.smithi178.stderr: 52250K .......... .......... .......... .......... .......... 56% 23.8M 2s 2024-06-28T15:27:04.299 INFO:teuthology.orchestra.run.smithi178.stderr: 52300K .......... .......... .......... .......... .......... 56% 10.4M 2s 2024-06-28T15:27:04.305 INFO:teuthology.orchestra.run.smithi178.stderr: 52350K .......... .......... .......... .......... .......... 56% 8.22M 2s 2024-06-28T15:27:04.307 INFO:teuthology.orchestra.run.smithi178.stderr: 52400K .......... .......... .......... .......... .......... 56% 20.7M 2s 2024-06-28T15:27:04.310 INFO:teuthology.orchestra.run.smithi178.stderr: 52450K .......... .......... .......... .......... .......... 56% 21.0M 2s 2024-06-28T15:27:04.310 INFO:teuthology.orchestra.run.smithi178.stderr: 52500K .......... .......... .......... .......... .......... 56% 53.1M 2s 2024-06-28T15:27:04.312 INFO:teuthology.orchestra.run.smithi178.stderr: 52550K .......... .......... .......... .......... .......... 56% 31.3M 2s 2024-06-28T15:27:04.313 INFO:teuthology.orchestra.run.smithi178.stderr: 52600K .......... .......... .......... .......... .......... 56% 41.6M 2s 2024-06-28T15:27:04.315 INFO:teuthology.orchestra.run.smithi178.stderr: 52650K .......... .......... .......... .......... .......... 56% 38.3M 2s 2024-06-28T15:27:04.316 INFO:teuthology.orchestra.run.smithi178.stderr: 52700K .......... .......... .......... .......... .......... 56% 34.1M 2s 2024-06-28T15:27:04.318 INFO:teuthology.orchestra.run.smithi178.stderr: 52750K .......... .......... .......... .......... .......... 56% 30.0M 2s 2024-06-28T15:27:04.319 INFO:teuthology.orchestra.run.smithi178.stderr: 52800K .......... .......... .......... .......... .......... 56% 28.5M 2s 2024-06-28T15:27:04.321 INFO:teuthology.orchestra.run.smithi178.stderr: 52850K .......... .......... .......... .......... .......... 56% 30.4M 2s 2024-06-28T15:27:04.323 INFO:teuthology.orchestra.run.smithi178.stderr: 52900K .......... .......... .......... .......... .......... 56% 21.3M 2s 2024-06-28T15:27:04.324 INFO:teuthology.orchestra.run.smithi178.stderr: 52950K .......... .......... .......... .......... .......... 57% 29.7M 2s 2024-06-28T15:27:04.326 INFO:teuthology.orchestra.run.smithi178.stderr: 53000K .......... .......... .......... .......... .......... 57% 29.6M 2s 2024-06-28T15:27:04.329 INFO:teuthology.orchestra.run.smithi178.stderr: 53050K .......... .......... .......... .......... .......... 57% 24.4M 2s 2024-06-28T15:27:04.332 INFO:teuthology.orchestra.run.smithi178.stderr: 53100K .......... .......... .......... .......... .......... 57% 14.0M 2s 2024-06-28T15:27:04.333 INFO:teuthology.orchestra.run.smithi178.stderr: 53150K .......... .......... .......... .......... .......... 57% 39.3M 2s 2024-06-28T15:27:04.335 INFO:teuthology.orchestra.run.smithi178.stderr: 53200K .......... .......... .......... .......... .......... 57% 24.8M 2s 2024-06-28T15:27:04.337 INFO:teuthology.orchestra.run.smithi178.stderr: 53250K .......... .......... .......... .......... .......... 57% 18.7M 2s 2024-06-28T15:27:04.343 INFO:teuthology.orchestra.run.smithi178.stderr: 53300K .......... .......... .......... .......... .......... 57% 9.36M 2s 2024-06-28T15:27:04.344 INFO:teuthology.orchestra.run.smithi178.stderr: 53350K .......... .......... .......... .......... .......... 57% 34.6M 2s 2024-06-28T15:27:04.345 INFO:teuthology.orchestra.run.smithi178.stderr: 53400K .......... .......... .......... .......... .......... 57% 30.8M 2s 2024-06-28T15:27:04.348 INFO:teuthology.orchestra.run.smithi178.stderr: 53450K .......... .......... .......... .......... .......... 57% 31.4M 2s 2024-06-28T15:27:04.349 INFO:teuthology.orchestra.run.smithi178.stderr: 53500K .......... .......... .......... .......... .......... 57% 32.8M 2s 2024-06-28T15:27:04.354 INFO:teuthology.orchestra.run.smithi178.stderr: 53550K .......... .......... .......... .......... .......... 57% 9.69M 2s 2024-06-28T15:27:04.356 INFO:teuthology.orchestra.run.smithi178.stderr: 53600K .......... .......... .......... .......... .......... 57% 17.5M 2s 2024-06-28T15:27:04.359 INFO:teuthology.orchestra.run.smithi178.stderr: 53650K .......... .......... .......... .......... .......... 57% 21.0M 2s 2024-06-28T15:27:04.363 INFO:teuthology.orchestra.run.smithi178.stderr: 53700K .......... .......... .......... .......... .......... 57% 14.2M 2s 2024-06-28T15:27:04.366 INFO:teuthology.orchestra.run.smithi178.stderr: 53750K .......... .......... .......... .......... .......... 57% 18.2M 2s 2024-06-28T15:27:04.366 INFO:teuthology.orchestra.run.smithi178.stderr: 53800K .......... .......... .......... .......... .......... 57% 59.5M 2s 2024-06-28T15:27:04.368 INFO:teuthology.orchestra.run.smithi178.stderr: 53850K .......... .......... .......... .......... .......... 58% 27.3M 2s 2024-06-28T15:27:04.372 INFO:teuthology.orchestra.run.smithi178.stderr: 53900K .......... .......... .......... .......... .......... 58% 21.6M 2s 2024-06-28T15:27:04.372 INFO:teuthology.orchestra.run.smithi178.stderr: 53950K .......... .......... .......... .......... .......... 58% 51.1M 2s 2024-06-28T15:27:04.373 INFO:teuthology.orchestra.run.smithi178.stderr: 54000K .......... .......... .......... .......... .......... 58% 27.2M 2s 2024-06-28T15:27:04.374 INFO:teuthology.orchestra.run.smithi178.stderr: 54050K .......... .......... .......... .......... .......... 58% 38.1M 2s 2024-06-28T15:27:04.376 INFO:teuthology.orchestra.run.smithi178.stderr: 54100K .......... .......... .......... .......... .......... 58% 21.0M 2s 2024-06-28T15:27:04.377 INFO:teuthology.orchestra.run.smithi178.stderr: 54150K .......... .......... .......... .......... .......... 58% 37.0M 2s 2024-06-28T15:27:04.380 INFO:teuthology.orchestra.run.smithi178.stderr: 54200K .......... .......... .......... .......... .......... 58% 22.3M 2s 2024-06-28T15:27:04.382 INFO:teuthology.orchestra.run.smithi178.stderr: 54250K .......... .......... .......... .......... .......... 58% 24.9M 2s 2024-06-28T15:27:04.383 INFO:teuthology.orchestra.run.smithi178.stderr: 54300K .......... .......... .......... .......... .......... 58% 28.4M 2s 2024-06-28T15:27:04.385 INFO:teuthology.orchestra.run.smithi178.stderr: 54350K .......... .......... .......... .......... .......... 58% 40.6M 2s 2024-06-28T15:27:04.386 INFO:teuthology.orchestra.run.smithi178.stderr: 54400K .......... .......... .......... .......... .......... 58% 49.1M 2s 2024-06-28T15:27:04.387 INFO:teuthology.orchestra.run.smithi178.stderr: 54450K .......... .......... .......... .......... .......... 58% 41.3M 2s 2024-06-28T15:27:04.388 INFO:teuthology.orchestra.run.smithi178.stderr: 54500K .......... .......... .......... .......... .......... 58% 31.6M 2s 2024-06-28T15:27:04.395 INFO:teuthology.orchestra.run.smithi178.stderr: 54550K .......... .......... .......... .......... .......... 58% 7.00M 2s 2024-06-28T15:27:04.397 INFO:teuthology.orchestra.run.smithi178.stderr: 54600K .......... .......... .......... .......... .......... 58% 25.1M 2s 2024-06-28T15:27:04.401 INFO:teuthology.orchestra.run.smithi178.stderr: 54650K .......... .......... .......... .......... .......... 58% 15.3M 2s 2024-06-28T15:27:04.404 INFO:teuthology.orchestra.run.smithi178.stderr: 54700K .......... .......... .......... .......... .......... 58% 14.4M 2s 2024-06-28T15:27:04.405 INFO:teuthology.orchestra.run.smithi178.stderr: 54750K .......... .......... .......... .......... .......... 58% 50.1M 2s 2024-06-28T15:27:04.407 INFO:teuthology.orchestra.run.smithi178.stderr: 54800K .......... .......... .......... .......... .......... 59% 20.0M 2s 2024-06-28T15:27:04.411 INFO:teuthology.orchestra.run.smithi178.stderr: 54850K .......... .......... .......... .......... .......... 59% 16.5M 2s 2024-06-28T15:27:04.411 INFO:teuthology.orchestra.run.smithi178.stderr: 54900K .......... .......... .......... .......... .......... 59% 33.9M 2s 2024-06-28T15:27:04.413 INFO:teuthology.orchestra.run.smithi178.stderr: 54950K .......... .......... .......... .......... .......... 59% 34.8M 2s 2024-06-28T15:27:04.416 INFO:teuthology.orchestra.run.smithi178.stderr: 55000K .......... .......... .......... .......... .......... 59% 16.7M 2s 2024-06-28T15:27:04.418 INFO:teuthology.orchestra.run.smithi178.stderr: 55050K .......... .......... .......... .......... .......... 59% 26.6M 2s 2024-06-28T15:27:04.420 INFO:teuthology.orchestra.run.smithi178.stderr: 55100K .......... .......... .......... .......... .......... 59% 26.3M 2s 2024-06-28T15:27:04.421 INFO:teuthology.orchestra.run.smithi178.stderr: 55150K .......... .......... .......... .......... .......... 59% 33.8M 2s 2024-06-28T15:27:04.423 INFO:teuthology.orchestra.run.smithi178.stderr: 55200K .......... .......... .......... .......... .......... 59% 37.7M 2s 2024-06-28T15:27:04.424 INFO:teuthology.orchestra.run.smithi178.stderr: 55250K .......... .......... .......... .......... .......... 59% 35.9M 2s 2024-06-28T15:27:04.426 INFO:teuthology.orchestra.run.smithi178.stderr: 55300K .......... .......... .......... .......... .......... 59% 33.2M 2s 2024-06-28T15:27:04.428 INFO:teuthology.orchestra.run.smithi178.stderr: 55350K .......... .......... .......... .......... .......... 59% 18.3M 2s 2024-06-28T15:27:04.430 INFO:teuthology.orchestra.run.smithi178.stderr: 55400K .......... .......... .......... .......... .......... 59% 27.2M 2s 2024-06-28T15:27:04.432 INFO:teuthology.orchestra.run.smithi178.stderr: 55450K .......... .......... .......... .......... .......... 59% 25.7M 2s 2024-06-28T15:27:04.435 INFO:teuthology.orchestra.run.smithi178.stderr: 55500K .......... .......... .......... .......... .......... 59% 15.0M 2s 2024-06-28T15:27:04.437 INFO:teuthology.orchestra.run.smithi178.stderr: 55550K .......... .......... .......... .......... .......... 59% 19.2M 2s 2024-06-28T15:27:04.440 INFO:teuthology.orchestra.run.smithi178.stderr: 55600K .......... .......... .......... .......... .......... 59% 21.2M 2s 2024-06-28T15:27:04.441 INFO:teuthology.orchestra.run.smithi178.stderr: 55650K .......... .......... .......... .......... .......... 59% 27.6M 2s 2024-06-28T15:27:04.444 INFO:teuthology.orchestra.run.smithi178.stderr: 55700K .......... .......... .......... .......... .......... 60% 17.9M 2s 2024-06-28T15:27:04.447 INFO:teuthology.orchestra.run.smithi178.stderr: 55750K .......... .......... .......... .......... .......... 60% 35.4M 2s 2024-06-28T15:27:04.448 INFO:teuthology.orchestra.run.smithi178.stderr: 55800K .......... .......... .......... .......... .......... 60% 21.8M 2s 2024-06-28T15:27:04.452 INFO:teuthology.orchestra.run.smithi178.stderr: 55850K .......... .......... .......... .......... .......... 60% 13.7M 2s 2024-06-28T15:27:04.453 INFO:teuthology.orchestra.run.smithi178.stderr: 55900K .......... .......... .......... .......... .......... 60% 40.9M 2s 2024-06-28T15:27:04.457 INFO:teuthology.orchestra.run.smithi178.stderr: 55950K .......... .......... .......... .......... .......... 60% 19.6M 2s 2024-06-28T15:27:04.457 INFO:teuthology.orchestra.run.smithi178.stderr: 56000K .......... .......... .......... .......... .......... 60% 38.5M 2s 2024-06-28T15:27:04.458 INFO:teuthology.orchestra.run.smithi178.stderr: 56050K .......... .......... .......... .......... .......... 60% 28.3M 2s 2024-06-28T15:27:04.460 INFO:teuthology.orchestra.run.smithi178.stderr: 56100K .......... .......... .......... .......... .......... 60% 24.8M 2s 2024-06-28T15:27:04.462 INFO:teuthology.orchestra.run.smithi178.stderr: 56150K .......... .......... .......... .......... .......... 60% 27.2M 2s 2024-06-28T15:27:04.464 INFO:teuthology.orchestra.run.smithi178.stderr: 56200K .......... .......... .......... .......... .......... 60% 28.2M 2s 2024-06-28T15:27:04.469 INFO:teuthology.orchestra.run.smithi178.stderr: 56250K .......... .......... .......... .......... .......... 60% 14.4M 2s 2024-06-28T15:27:04.470 INFO:teuthology.orchestra.run.smithi178.stderr: 56300K .......... .......... .......... .......... .......... 60% 24.5M 2s 2024-06-28T15:27:04.474 INFO:teuthology.orchestra.run.smithi178.stderr: 56350K .......... .......... .......... .......... .......... 60% 18.0M 2s 2024-06-28T15:27:04.474 INFO:teuthology.orchestra.run.smithi178.stderr: 56400K .......... .......... .......... .......... .......... 60% 31.4M 2s 2024-06-28T15:27:04.475 INFO:teuthology.orchestra.run.smithi178.stderr: 56450K .......... .......... .......... .......... .......... 60% 28.4M 2s 2024-06-28T15:27:04.479 INFO:teuthology.orchestra.run.smithi178.stderr: 56500K .......... .......... .......... .......... .......... 60% 13.5M 2s 2024-06-28T15:27:04.483 INFO:teuthology.orchestra.run.smithi178.stderr: 56550K .......... .......... .......... .......... .......... 60% 19.1M 2s 2024-06-28T15:27:04.483 INFO:teuthology.orchestra.run.smithi178.stderr: 56600K .......... .......... .......... .......... .......... 60% 35.2M 2s 2024-06-28T15:27:04.485 INFO:teuthology.orchestra.run.smithi178.stderr: 56650K .......... .......... .......... .......... .......... 61% 19.7M 2s 2024-06-28T15:27:04.488 INFO:teuthology.orchestra.run.smithi178.stderr: 56700K .......... .......... .......... .......... .......... 61% 22.0M 2s 2024-06-28T15:27:04.488 INFO:teuthology.orchestra.run.smithi178.stderr: 56750K .......... .......... .......... .......... .......... 61% 83.9M 2s 2024-06-28T15:27:04.490 INFO:teuthology.orchestra.run.smithi178.stderr: 56800K .......... .......... .......... .......... .......... 61% 28.8M 2s 2024-06-28T15:27:04.491 INFO:teuthology.orchestra.run.smithi178.stderr: 56850K .......... .......... .......... .......... .......... 61% 45.3M 2s 2024-06-28T15:27:04.492 INFO:teuthology.orchestra.run.smithi178.stderr: 56900K .......... .......... .......... .......... .......... 61% 35.4M 2s 2024-06-28T15:27:04.493 INFO:teuthology.orchestra.run.smithi178.stderr: 56950K .......... .......... .......... .......... .......... 61% 50.8M 2s 2024-06-28T15:27:04.495 INFO:teuthology.orchestra.run.smithi178.stderr: 57000K .......... .......... .......... .......... .......... 61% 21.4M 2s 2024-06-28T15:27:04.497 INFO:teuthology.orchestra.run.smithi178.stderr: 57050K .......... .......... .......... .......... .......... 61% 44.2M 2s 2024-06-28T15:27:04.498 INFO:teuthology.orchestra.run.smithi178.stderr: 57100K .......... .......... .......... .......... .......... 61% 36.0M 2s 2024-06-28T15:27:04.505 INFO:teuthology.orchestra.run.smithi178.stderr: 57150K .......... .......... .......... .......... .......... 61% 7.71M 2s 2024-06-28T15:27:04.507 INFO:teuthology.orchestra.run.smithi178.stderr: 57200K .......... .......... .......... .......... .......... 61% 22.3M 2s 2024-06-28T15:27:04.509 INFO:teuthology.orchestra.run.smithi178.stderr: 57250K .......... .......... .......... .......... .......... 61% 18.2M 2s 2024-06-28T15:27:04.512 INFO:teuthology.orchestra.run.smithi178.stderr: 57300K .......... .......... .......... .......... .......... 61% 17.6M 2s 2024-06-28T15:27:04.515 INFO:teuthology.orchestra.run.smithi178.stderr: 57350K .......... .......... .......... .......... .......... 61% 17.2M 2s 2024-06-28T15:27:04.516 INFO:teuthology.orchestra.run.smithi178.stderr: 57400K .......... .......... .......... .......... .......... 61% 27.3M 2s 2024-06-28T15:27:04.520 INFO:teuthology.orchestra.run.smithi178.stderr: 57450K .......... .......... .......... .......... .......... 61% 12.7M 2s 2024-06-28T15:27:04.523 INFO:teuthology.orchestra.run.smithi178.stderr: 57500K .......... .......... .......... .......... .......... 61% 16.7M 2s 2024-06-28T15:27:04.525 INFO:teuthology.orchestra.run.smithi178.stderr: 57550K .......... .......... .......... .......... .......... 62% 30.4M 2s 2024-06-28T15:27:04.526 INFO:teuthology.orchestra.run.smithi178.stderr: 57600K .......... .......... .......... .......... .......... 62% 27.9M 2s 2024-06-28T15:27:04.527 INFO:teuthology.orchestra.run.smithi178.stderr: 57650K .......... .......... .......... .......... .......... 62% 45.3M 2s 2024-06-28T15:27:04.529 INFO:teuthology.orchestra.run.smithi178.stderr: 57700K .......... .......... .......... .......... .......... 62% 35.8M 2s 2024-06-28T15:27:04.531 INFO:teuthology.orchestra.run.smithi178.stderr: 57750K .......... .......... .......... .......... .......... 62% 39.0M 2s 2024-06-28T15:27:04.532 INFO:teuthology.orchestra.run.smithi178.stderr: 57800K .......... .......... .......... .......... .......... 62% 39.0M 2s 2024-06-28T15:27:04.536 INFO:teuthology.orchestra.run.smithi178.stderr: 57850K .......... .......... .......... .......... .......... 62% 24.4M 2s 2024-06-28T15:27:04.536 INFO:teuthology.orchestra.run.smithi178.stderr: 57900K .......... .......... .......... .......... .......... 62% 36.0M 2s 2024-06-28T15:27:04.536 INFO:teuthology.orchestra.run.smithi178.stderr: 57950K .......... .......... .......... .......... .......... 62% 38.8M 2s 2024-06-28T15:27:04.537 INFO:teuthology.orchestra.run.smithi178.stderr: 58000K .......... .......... .......... .......... .......... 62% 39.4M 2s 2024-06-28T15:27:04.542 INFO:teuthology.orchestra.run.smithi178.stderr: 58050K .......... .......... .......... .......... .......... 62% 11.1M 2s 2024-06-28T15:27:04.543 INFO:teuthology.orchestra.run.smithi178.stderr: 58100K .......... .......... .......... .......... .......... 62% 30.9M 2s 2024-06-28T15:27:04.548 INFO:teuthology.orchestra.run.smithi178.stderr: 58150K .......... .......... .......... .......... .......... 62% 9.90M 2s 2024-06-28T15:27:04.549 INFO:teuthology.orchestra.run.smithi178.stderr: 58200K .......... .......... .......... .......... .......... 62% 38.4M 2s 2024-06-28T15:27:04.556 INFO:teuthology.orchestra.run.smithi178.stderr: 58250K .......... .......... .......... .......... .......... 62% 13.1M 2s 2024-06-28T15:27:04.556 INFO:teuthology.orchestra.run.smithi178.stderr: 58300K .......... .......... .......... .......... .......... 62% 17.4M 2s 2024-06-28T15:27:04.558 INFO:teuthology.orchestra.run.smithi178.stderr: 58350K .......... .......... .......... .......... .......... 62% 21.6M 2s 2024-06-28T15:27:04.563 INFO:teuthology.orchestra.run.smithi178.stderr: 58400K .......... .......... .......... .......... .......... 62% 10.2M 2s 2024-06-28T15:27:04.566 INFO:teuthology.orchestra.run.smithi178.stderr: 58450K .......... .......... .......... .......... .......... 62% 25.6M 2s 2024-06-28T15:27:04.566 INFO:teuthology.orchestra.run.smithi178.stderr: 58500K .......... .......... .......... .......... .......... 63% 29.4M 2s 2024-06-28T15:27:04.568 INFO:teuthology.orchestra.run.smithi178.stderr: 58550K .......... .......... .......... .......... .......... 63% 33.2M 2s 2024-06-28T15:27:04.569 INFO:teuthology.orchestra.run.smithi178.stderr: 58600K .......... .......... .......... .......... .......... 63% 48.5M 2s 2024-06-28T15:27:04.571 INFO:teuthology.orchestra.run.smithi178.stderr: 58650K .......... .......... .......... .......... .......... 63% 21.3M 2s 2024-06-28T15:27:04.574 INFO:teuthology.orchestra.run.smithi178.stderr: 58700K .......... .......... .......... .......... .......... 63% 18.0M 2s 2024-06-28T15:27:04.575 INFO:teuthology.orchestra.run.smithi178.stderr: 58750K .......... .......... .......... .......... .......... 63% 32.2M 2s 2024-06-28T15:27:04.577 INFO:teuthology.orchestra.run.smithi178.stderr: 58800K .......... .......... .......... .......... .......... 63% 34.0M 2s 2024-06-28T15:27:04.578 INFO:teuthology.orchestra.run.smithi178.stderr: 58850K .......... .......... .......... .......... .......... 63% 40.9M 2s 2024-06-28T15:27:04.583 INFO:teuthology.orchestra.run.smithi178.stderr: 58900K .......... .......... .......... .......... .......... 63% 10.6M 2s 2024-06-28T15:27:04.585 INFO:teuthology.orchestra.run.smithi178.stderr: 58950K .......... .......... .......... .......... .......... 63% 25.7M 2s 2024-06-28T15:27:04.587 INFO:teuthology.orchestra.run.smithi178.stderr: 59000K .......... .......... .......... .......... .......... 63% 22.2M 2s 2024-06-28T15:27:04.589 INFO:teuthology.orchestra.run.smithi178.stderr: 59050K .......... .......... .......... .......... .......... 63% 33.0M 2s 2024-06-28T15:27:04.593 INFO:teuthology.orchestra.run.smithi178.stderr: 59100K .......... .......... .......... .......... .......... 63% 9.64M 2s 2024-06-28T15:27:04.595 INFO:teuthology.orchestra.run.smithi178.stderr: 59150K .......... .......... .......... .......... .......... 63% 27.0M 2s 2024-06-28T15:27:04.597 INFO:teuthology.orchestra.run.smithi178.stderr: 59200K .......... .......... .......... .......... .......... 63% 22.7M 2s 2024-06-28T15:27:04.600 INFO:teuthology.orchestra.run.smithi178.stderr: 59250K .......... .......... .......... .......... .......... 63% 23.0M 2s 2024-06-28T15:27:04.601 INFO:teuthology.orchestra.run.smithi178.stderr: 59300K .......... .......... .......... .......... .......... 63% 41.0M 2s 2024-06-28T15:27:04.604 INFO:teuthology.orchestra.run.smithi178.stderr: 59350K .......... .......... .......... .......... .......... 63% 14.0M 2s 2024-06-28T15:27:04.605 INFO:teuthology.orchestra.run.smithi178.stderr: 59400K .......... .......... .......... .......... .......... 63% 36.4M 2s 2024-06-28T15:27:04.608 INFO:teuthology.orchestra.run.smithi178.stderr: 59450K .......... .......... .......... .......... .......... 64% 20.6M 2s 2024-06-28T15:27:04.611 INFO:teuthology.orchestra.run.smithi178.stderr: 59500K .......... .......... .......... .......... .......... 64% 21.8M 2s 2024-06-28T15:27:04.611 INFO:teuthology.orchestra.run.smithi178.stderr: 59550K .......... .......... .......... .......... .......... 64% 73.3M 2s 2024-06-28T15:27:04.613 INFO:teuthology.orchestra.run.smithi178.stderr: 59600K .......... .......... .......... .......... .......... 64% 28.9M 2s 2024-06-28T15:27:04.615 INFO:teuthology.orchestra.run.smithi178.stderr: 59650K .......... .......... .......... .......... .......... 64% 17.9M 2s 2024-06-28T15:27:04.618 INFO:teuthology.orchestra.run.smithi178.stderr: 59700K .......... .......... .......... .......... .......... 64% 18.6M 2s 2024-06-28T15:27:04.619 INFO:teuthology.orchestra.run.smithi178.stderr: 59750K .......... .......... .......... .......... .......... 64% 52.5M 2s 2024-06-28T15:27:04.627 INFO:teuthology.orchestra.run.smithi178.stderr: 59800K .......... .......... .......... .......... .......... 64% 5.82M 2s 2024-06-28T15:27:04.630 INFO:teuthology.orchestra.run.smithi178.stderr: 59850K .......... .......... .......... .......... .......... 64% 33.5M 2s 2024-06-28T15:27:04.633 INFO:teuthology.orchestra.run.smithi178.stderr: 59900K .......... .......... .......... .......... .......... 64% 11.7M 2s 2024-06-28T15:27:04.635 INFO:teuthology.orchestra.run.smithi178.stderr: 59950K .......... .......... .......... .......... .......... 64% 26.1M 2s 2024-06-28T15:27:04.637 INFO:teuthology.orchestra.run.smithi178.stderr: 60000K .......... .......... .......... .......... .......... 64% 17.2M 2s 2024-06-28T15:27:04.639 INFO:teuthology.orchestra.run.smithi178.stderr: 60050K .......... .......... .......... .......... .......... 64% 32.7M 2s 2024-06-28T15:27:04.644 INFO:teuthology.orchestra.run.smithi178.stderr: 60100K .......... .......... .......... .......... .......... 64% 9.41M 2s 2024-06-28T15:27:04.646 INFO:teuthology.orchestra.run.smithi178.stderr: 60150K .......... .......... .......... .......... .......... 64% 28.0M 2s 2024-06-28T15:27:04.648 INFO:teuthology.orchestra.run.smithi178.stderr: 60200K .......... .......... .......... .......... .......... 64% 20.9M 2s 2024-06-28T15:27:04.651 INFO:teuthology.orchestra.run.smithi178.stderr: 60250K .......... .......... .......... .......... .......... 64% 30.0M 2s 2024-06-28T15:27:04.652 INFO:teuthology.orchestra.run.smithi178.stderr: 60300K .......... .......... .......... .......... .......... 64% 26.0M 2s 2024-06-28T15:27:04.654 INFO:teuthology.orchestra.run.smithi178.stderr: 60350K .......... .......... .......... .......... .......... 65% 22.5M 2s 2024-06-28T15:27:04.656 INFO:teuthology.orchestra.run.smithi178.stderr: 60400K .......... .......... .......... .......... .......... 65% 27.3M 2s 2024-06-28T15:27:04.659 INFO:teuthology.orchestra.run.smithi178.stderr: 60450K .......... .......... .......... .......... .......... 65% 16.5M 2s 2024-06-28T15:27:04.660 INFO:teuthology.orchestra.run.smithi178.stderr: 60500K .......... .......... .......... .......... .......... 65% 30.4M 2s 2024-06-28T15:27:04.661 INFO:teuthology.orchestra.run.smithi178.stderr: 60550K .......... .......... .......... .......... .......... 65% 61.4M 2s 2024-06-28T15:27:04.663 INFO:teuthology.orchestra.run.smithi178.stderr: 60600K .......... .......... .......... .......... .......... 65% 26.9M 2s 2024-06-28T15:27:04.668 INFO:teuthology.orchestra.run.smithi178.stderr: 60650K .......... .......... .......... .......... .......... 65% 41.5M 2s 2024-06-28T15:27:04.669 INFO:teuthology.orchestra.run.smithi178.stderr: 60700K .......... .......... .......... .......... .......... 65% 9.33M 2s 2024-06-28T15:27:04.673 INFO:teuthology.orchestra.run.smithi178.stderr: 60750K .......... .......... .......... .......... .......... 65% 14.0M 2s 2024-06-28T15:27:04.681 INFO:teuthology.orchestra.run.smithi178.stderr: 60800K .......... .......... .......... .......... .......... 65% 6.29M 2s 2024-06-28T15:27:04.686 INFO:teuthology.orchestra.run.smithi178.stderr: 60850K .......... .......... .......... .......... .......... 65% 14.0M 2s 2024-06-28T15:27:04.687 INFO:teuthology.orchestra.run.smithi178.stderr: 60900K .......... .......... .......... .......... .......... 65% 16.7M 2s 2024-06-28T15:27:04.688 INFO:teuthology.orchestra.run.smithi178.stderr: 60950K .......... .......... .......... .......... .......... 65% 43.5M 2s 2024-06-28T15:27:04.691 INFO:teuthology.orchestra.run.smithi178.stderr: 61000K .......... .......... .......... .......... .......... 65% 18.2M 2s 2024-06-28T15:27:04.695 INFO:teuthology.orchestra.run.smithi178.stderr: 61050K .......... .......... .......... .......... .......... 65% 20.9M 2s 2024-06-28T15:27:04.697 INFO:teuthology.orchestra.run.smithi178.stderr: 61100K .......... .......... .......... .......... .......... 65% 12.3M 2s 2024-06-28T15:27:04.699 INFO:teuthology.orchestra.run.smithi178.stderr: 61150K .......... .......... .......... .......... .......... 65% 28.9M 2s 2024-06-28T15:27:04.700 INFO:teuthology.orchestra.run.smithi178.stderr: 61200K .......... .......... .......... .......... .......... 65% 53.7M 2s 2024-06-28T15:27:04.702 INFO:teuthology.orchestra.run.smithi178.stderr: 61250K .......... .......... .......... .......... .......... 65% 32.8M 2s 2024-06-28T15:27:04.702 INFO:teuthology.orchestra.run.smithi178.stderr: 61300K .......... .......... .......... .......... .......... 66% 59.1M 2s 2024-06-28T15:27:04.703 INFO:teuthology.orchestra.run.smithi178.stderr: 61350K .......... .......... .......... .......... .......... 66% 40.0M 2s 2024-06-28T15:27:04.705 INFO:teuthology.orchestra.run.smithi178.stderr: 61400K .......... .......... .......... .......... .......... 66% 29.9M 2s 2024-06-28T15:27:04.709 INFO:teuthology.orchestra.run.smithi178.stderr: 61450K .......... .......... .......... .......... .......... 66% 11.1M 2s 2024-06-28T15:27:04.712 INFO:teuthology.orchestra.run.smithi178.stderr: 61500K .......... .......... .......... .......... .......... 66% 20.4M 2s 2024-06-28T15:27:04.715 INFO:teuthology.orchestra.run.smithi178.stderr: 61550K .......... .......... .......... .......... .......... 66% 14.9M 2s 2024-06-28T15:27:04.717 INFO:teuthology.orchestra.run.smithi178.stderr: 61600K .......... .......... .......... .......... .......... 66% 19.2M 2s 2024-06-28T15:27:04.721 INFO:teuthology.orchestra.run.smithi178.stderr: 61650K .......... .......... .......... .......... .......... 66% 15.3M 2s 2024-06-28T15:27:04.796 INFO:teuthology.orchestra.run.smithi178.stderr: 61700K .......... .......... .......... .......... .......... 66% 864K 2s 2024-06-28T15:27:04.797 INFO:teuthology.orchestra.run.smithi178.stderr: 61750K .......... .......... .......... .......... .......... 66% 136M 2s 2024-06-28T15:27:04.797 INFO:teuthology.orchestra.run.smithi178.stderr: 61800K .......... .......... .......... .......... .......... 66% 148M 2s 2024-06-28T15:27:04.797 INFO:teuthology.orchestra.run.smithi178.stderr: 61850K .......... .......... .......... .......... .......... 66% 136M 2s 2024-06-28T15:27:04.797 INFO:teuthology.orchestra.run.smithi178.stderr: 61900K .......... .......... .......... .......... .......... 66% 123M 2s 2024-06-28T15:27:04.797 INFO:teuthology.orchestra.run.smithi178.stderr: 61950K .......... .......... .......... .......... .......... 66% 148M 2s 2024-06-28T15:27:04.797 INFO:teuthology.orchestra.run.smithi178.stderr: 62000K .......... .......... .......... .......... .......... 66% 142M 2s 2024-06-28T15:27:04.797 INFO:teuthology.orchestra.run.smithi178.stderr: 62050K .......... .......... .......... .......... .......... 66% 121M 2s 2024-06-28T15:27:04.797 INFO:teuthology.orchestra.run.smithi178.stderr: 62100K .......... .......... .......... .......... .......... 66% 121M 2s 2024-06-28T15:27:04.797 INFO:teuthology.orchestra.run.smithi178.stderr: 62150K .......... .......... .......... .......... .......... 66% 151M 2s 2024-06-28T15:27:04.798 INFO:teuthology.orchestra.run.smithi178.stderr: 62200K .......... .......... .......... .......... .......... 67% 151M 2s 2024-06-28T15:27:04.798 INFO:teuthology.orchestra.run.smithi178.stderr: 62250K .......... .......... .......... .......... .......... 67% 149M 1s 2024-06-28T15:27:04.798 INFO:teuthology.orchestra.run.smithi178.stderr: 62300K .......... .......... .......... .......... .......... 67% 127M 1s 2024-06-28T15:27:04.798 INFO:teuthology.orchestra.run.smithi178.stderr: 62350K .......... .......... .......... .......... .......... 67% 149M 1s 2024-06-28T15:27:04.798 INFO:teuthology.orchestra.run.smithi178.stderr: 62400K .......... .......... .......... .......... .......... 67% 99.4M 1s 2024-06-28T15:27:04.798 INFO:teuthology.orchestra.run.smithi178.stderr: 62450K .......... .......... .......... .......... .......... 67% 149M 1s 2024-06-28T15:27:04.798 INFO:teuthology.orchestra.run.smithi178.stderr: 62500K .......... .......... .......... .......... .......... 67% 133M 1s 2024-06-28T15:27:04.798 INFO:teuthology.orchestra.run.smithi178.stderr: 62550K .......... .......... .......... .......... .......... 67% 151M 1s 2024-06-28T15:27:04.798 INFO:teuthology.orchestra.run.smithi178.stderr: 62600K .......... .......... .......... .......... .......... 67% 152M 1s 2024-06-28T15:27:04.798 INFO:teuthology.orchestra.run.smithi178.stderr: 62650K .......... .......... .......... .......... .......... 67% 139M 1s 2024-06-28T15:27:04.798 INFO:teuthology.orchestra.run.smithi178.stderr: 62700K .......... .......... .......... .......... .......... 67% 121M 1s 2024-06-28T15:27:04.798 INFO:teuthology.orchestra.run.smithi178.stderr: 62750K .......... .......... .......... .......... .......... 67% 150M 1s 2024-06-28T15:27:04.798 INFO:teuthology.orchestra.run.smithi178.stderr: 62800K .......... .......... .......... .......... .......... 67% 145M 1s 2024-06-28T15:27:04.799 INFO:teuthology.orchestra.run.smithi178.stderr: 62850K .......... .......... .......... .......... .......... 67% 15.6M 1s 2024-06-28T15:27:04.799 INFO:teuthology.orchestra.run.smithi178.stderr: 62900K .......... .......... .......... .......... .......... 67% 9.37M 1s 2024-06-28T15:27:04.799 INFO:teuthology.orchestra.run.smithi178.stderr: 62950K .......... .......... .......... .......... .......... 67% 30.4M 1s 2024-06-28T15:27:04.800 INFO:teuthology.orchestra.run.smithi178.stderr: 63000K .......... .......... .......... .......... .......... 67% 28.1M 1s 2024-06-28T15:27:04.801 INFO:teuthology.orchestra.run.smithi178.stderr: 63050K .......... .......... .......... .......... .......... 67% 23.9M 1s 2024-06-28T15:27:04.802 INFO:teuthology.orchestra.run.smithi178.stderr: 63100K .......... .......... .......... .......... .......... 67% 22.9M 1s 2024-06-28T15:27:04.807 INFO:teuthology.orchestra.run.smithi178.stderr: 63150K .......... .......... .......... .......... .......... 68% 9.20M 1s 2024-06-28T15:27:04.813 INFO:teuthology.orchestra.run.smithi178.stderr: 63200K .......... .......... .......... .......... .......... 68% 9.44M 1s 2024-06-28T15:27:04.815 INFO:teuthology.orchestra.run.smithi178.stderr: 63250K .......... .......... .......... .......... .......... 68% 25.5M 1s 2024-06-28T15:27:04.817 INFO:teuthology.orchestra.run.smithi178.stderr: 63300K .......... .......... .......... .......... .......... 68% 16.6M 1s 2024-06-28T15:27:04.881 INFO:teuthology.orchestra.run.smithi178.stderr: 63350K .......... .......... .......... .......... .......... 68% 788K 1s 2024-06-28T15:27:04.898 INFO:teuthology.orchestra.run.smithi178.stderr: 63400K .......... .......... .......... .......... .......... 68% 14.4M 1s 2024-06-28T15:27:04.899 INFO:teuthology.orchestra.run.smithi178.stderr: 63450K .......... .......... .......... .......... .......... 68% 141M 1s 2024-06-28T15:27:04.899 INFO:teuthology.orchestra.run.smithi178.stderr: 63500K .......... .......... .......... .......... .......... 68% 122M 1s 2024-06-28T15:27:04.899 INFO:teuthology.orchestra.run.smithi178.stderr: 63550K .......... .......... .......... .......... .......... 68% 149M 1s 2024-06-28T15:27:04.899 INFO:teuthology.orchestra.run.smithi178.stderr: 63600K .......... .......... .......... .......... .......... 68% 148M 1s 2024-06-28T15:27:04.899 INFO:teuthology.orchestra.run.smithi178.stderr: 63650K .......... .......... .......... .......... .......... 68% 150M 1s 2024-06-28T15:27:04.899 INFO:teuthology.orchestra.run.smithi178.stderr: 63700K .......... .......... .......... .......... .......... 68% 135M 1s 2024-06-28T15:27:04.899 INFO:teuthology.orchestra.run.smithi178.stderr: 63750K .......... .......... .......... .......... .......... 68% 148M 1s 2024-06-28T15:27:04.899 INFO:teuthology.orchestra.run.smithi178.stderr: 63800K .......... .......... .......... .......... .......... 68% 149M 1s 2024-06-28T15:27:04.899 INFO:teuthology.orchestra.run.smithi178.stderr: 63850K .......... .......... .......... .......... .......... 68% 149M 1s 2024-06-28T15:27:04.899 INFO:teuthology.orchestra.run.smithi178.stderr: 63900K .......... .......... .......... .......... .......... 68% 116M 1s 2024-06-28T15:27:04.900 INFO:teuthology.orchestra.run.smithi178.stderr: 63950K .......... .......... .......... .......... .......... 68% 130M 1s 2024-06-28T15:27:04.900 INFO:teuthology.orchestra.run.smithi178.stderr: 64000K .......... .......... .......... .......... .......... 68% 143M 1s 2024-06-28T15:27:04.900 INFO:teuthology.orchestra.run.smithi178.stderr: 64050K .......... .......... .......... .......... .......... 68% 139M 1s 2024-06-28T15:27:04.900 INFO:teuthology.orchestra.run.smithi178.stderr: 64100K .......... .......... .......... .......... .......... 69% 128M 1s 2024-06-28T15:27:04.900 INFO:teuthology.orchestra.run.smithi178.stderr: 64150K .......... .......... .......... .......... .......... 69% 11.7M 1s 2024-06-28T15:27:04.900 INFO:teuthology.orchestra.run.smithi178.stderr: 64200K .......... .......... .......... .......... .......... 69% 15.8M 1s 2024-06-28T15:27:04.900 INFO:teuthology.orchestra.run.smithi178.stderr: 64250K .......... .......... .......... .......... .......... 69% 39.6M 1s 2024-06-28T15:27:04.907 INFO:teuthology.orchestra.run.smithi178.stderr: 64300K .......... .......... .......... .......... .......... 69% 5.56M 1s 2024-06-28T15:27:04.911 INFO:teuthology.orchestra.run.smithi178.stderr: 64350K .......... .......... .......... .......... .......... 69% 11.6M 1s 2024-06-28T15:27:04.912 INFO:teuthology.orchestra.run.smithi178.stderr: 64400K .......... .......... .......... .......... .......... 69% 33.5M 1s 2024-06-28T15:27:04.916 INFO:teuthology.orchestra.run.smithi178.stderr: 64450K .......... .......... .......... .......... .......... 69% 11.6M 1s 2024-06-28T15:27:04.923 INFO:teuthology.orchestra.run.smithi178.stderr: 64500K .......... .......... .......... .......... .......... 69% 7.56M 1s 2024-06-28T15:27:04.925 INFO:teuthology.orchestra.run.smithi178.stderr: 64550K .......... .......... .......... .......... .......... 69% 27.0M 1s 2024-06-28T15:27:04.934 INFO:teuthology.orchestra.run.smithi178.stderr: 64600K .......... .......... .......... .......... .......... 69% 5.22M 1s 2024-06-28T15:27:04.940 INFO:teuthology.orchestra.run.smithi178.stderr: 64650K .......... .......... .......... .......... .......... 69% 12.3M 1s 2024-06-28T15:27:04.946 INFO:teuthology.orchestra.run.smithi178.stderr: 64700K .......... .......... .......... .......... .......... 69% 5.75M 1s 2024-06-28T15:27:04.951 INFO:teuthology.orchestra.run.smithi178.stderr: 64750K .......... .......... .......... .......... .......... 69% 11.1M 1s 2024-06-28T15:27:04.955 INFO:teuthology.orchestra.run.smithi178.stderr: 64800K .......... .......... .......... .......... .......... 69% 12.8M 1s 2024-06-28T15:27:04.960 INFO:teuthology.orchestra.run.smithi178.stderr: 64850K .......... .......... .......... .......... .......... 69% 9.86M 1s 2024-06-28T15:27:04.965 INFO:teuthology.orchestra.run.smithi178.stderr: 64900K .......... .......... .......... .......... .......... 69% 10.3M 1s 2024-06-28T15:27:04.968 INFO:teuthology.orchestra.run.smithi178.stderr: 64950K .......... .......... .......... .......... .......... 69% 16.3M 1s 2024-06-28T15:27:04.974 INFO:teuthology.orchestra.run.smithi178.stderr: 65000K .......... .......... .......... .......... .......... 70% 8.48M 1s 2024-06-28T15:27:04.982 INFO:teuthology.orchestra.run.smithi178.stderr: 65050K .......... .......... .......... .......... .......... 70% 22.8M 1s 2024-06-28T15:27:04.985 INFO:teuthology.orchestra.run.smithi178.stderr: 65100K .......... .......... .......... .......... .......... 70% 5.15M 1s 2024-06-28T15:27:04.990 INFO:teuthology.orchestra.run.smithi178.stderr: 65150K .......... .......... .......... .......... .......... 70% 9.56M 1s 2024-06-28T15:27:05.043 INFO:teuthology.orchestra.run.smithi178.stderr: 65200K .......... .......... .......... .......... .......... 70% 940K 1s 2024-06-28T15:27:05.056 INFO:teuthology.orchestra.run.smithi178.stderr: 65250K .......... .......... .......... .......... .......... 70% 33.2M 1s 2024-06-28T15:27:05.056 INFO:teuthology.orchestra.run.smithi178.stderr: 65300K .......... .......... .......... .......... .......... 70% 130M 1s 2024-06-28T15:27:05.056 INFO:teuthology.orchestra.run.smithi178.stderr: 65350K .......... .......... .......... .......... .......... 70% 150M 1s 2024-06-28T15:27:05.056 INFO:teuthology.orchestra.run.smithi178.stderr: 65400K .......... .......... .......... .......... .......... 70% 151M 1s 2024-06-28T15:27:05.056 INFO:teuthology.orchestra.run.smithi178.stderr: 65450K .......... .......... .......... .......... .......... 70% 113M 1s 2024-06-28T15:27:05.056 INFO:teuthology.orchestra.run.smithi178.stderr: 65500K .......... .......... .......... .......... .......... 70% 104M 1s 2024-06-28T15:27:05.056 INFO:teuthology.orchestra.run.smithi178.stderr: 65550K .......... .......... .......... .......... .......... 70% 119M 1s 2024-06-28T15:27:05.056 INFO:teuthology.orchestra.run.smithi178.stderr: 65600K .......... .......... .......... .......... .......... 70% 102M 1s 2024-06-28T15:27:05.056 INFO:teuthology.orchestra.run.smithi178.stderr: 65650K .......... .......... .......... .......... .......... 70% 87.6M 1s 2024-06-28T15:27:05.056 INFO:teuthology.orchestra.run.smithi178.stderr: 65700K .......... .......... .......... .......... .......... 70% 135M 1s 2024-06-28T15:27:05.057 INFO:teuthology.orchestra.run.smithi178.stderr: 65750K .......... .......... .......... .......... .......... 70% 108M 1s 2024-06-28T15:27:05.057 INFO:teuthology.orchestra.run.smithi178.stderr: 65800K .......... .......... .......... .......... .......... 70% 75.5M 1s 2024-06-28T15:27:05.057 INFO:teuthology.orchestra.run.smithi178.stderr: 65850K .......... .......... .......... .......... .......... 70% 9.93M 1s 2024-06-28T15:27:05.060 INFO:teuthology.orchestra.run.smithi178.stderr: 65900K .......... .......... .......... .......... .......... 70% 8.41M 1s 2024-06-28T15:27:05.068 INFO:teuthology.orchestra.run.smithi178.stderr: 65950K .......... .......... .......... .......... .......... 71% 6.42M 1s 2024-06-28T15:27:05.074 INFO:teuthology.orchestra.run.smithi178.stderr: 66000K .......... .......... .......... .......... .......... 71% 7.88M 1s 2024-06-28T15:27:05.079 INFO:teuthology.orchestra.run.smithi178.stderr: 66050K .......... .......... .......... .......... .......... 71% 10.1M 1s 2024-06-28T15:27:05.082 INFO:teuthology.orchestra.run.smithi178.stderr: 66100K .......... .......... .......... .......... .......... 71% 14.1M 1s 2024-06-28T15:27:05.087 INFO:teuthology.orchestra.run.smithi178.stderr: 66150K .......... .......... .......... .......... .......... 71% 9.25M 1s 2024-06-28T15:27:05.091 INFO:teuthology.orchestra.run.smithi178.stderr: 66200K .......... .......... .......... .......... .......... 71% 14.1M 1s 2024-06-28T15:27:05.098 INFO:teuthology.orchestra.run.smithi178.stderr: 66250K .......... .......... .......... .......... .......... 71% 10.3M 1s 2024-06-28T15:27:05.105 INFO:teuthology.orchestra.run.smithi178.stderr: 66300K .......... .......... .......... .......... .......... 71% 5.48M 1s 2024-06-28T15:27:05.110 INFO:teuthology.orchestra.run.smithi178.stderr: 66350K .......... .......... .......... .......... .......... 71% 8.83M 1s 2024-06-28T15:27:05.115 INFO:teuthology.orchestra.run.smithi178.stderr: 66400K .......... .......... .......... .......... .......... 71% 10.9M 1s 2024-06-28T15:27:05.118 INFO:teuthology.orchestra.run.smithi178.stderr: 66450K .......... .......... .......... .......... .......... 71% 13.2M 1s 2024-06-28T15:27:05.122 INFO:teuthology.orchestra.run.smithi178.stderr: 66500K .......... .......... .......... .......... .......... 71% 12.3M 1s 2024-06-28T15:27:05.132 INFO:teuthology.orchestra.run.smithi178.stderr: 66550K .......... .......... .......... .......... .......... 71% 5.57M 1s 2024-06-28T15:27:05.135 INFO:teuthology.orchestra.run.smithi178.stderr: 66600K .......... .......... .......... .......... .......... 71% 13.0M 1s 2024-06-28T15:27:05.144 INFO:teuthology.orchestra.run.smithi178.stderr: 66650K .......... .......... .......... .......... .......... 71% 5.27M 1s 2024-06-28T15:27:05.148 INFO:teuthology.orchestra.run.smithi178.stderr: 66700K .......... .......... .......... .......... .......... 71% 14.3M 1s 2024-06-28T15:27:05.150 INFO:teuthology.orchestra.run.smithi178.stderr: 66750K .......... .......... .......... .......... .......... 71% 16.8M 1s 2024-06-28T15:27:05.157 INFO:teuthology.orchestra.run.smithi178.stderr: 66800K .......... .......... .......... .......... .......... 71% 7.76M 1s 2024-06-28T15:27:05.161 INFO:teuthology.orchestra.run.smithi178.stderr: 66850K .......... .......... .......... .......... .......... 72% 11.0M 1s 2024-06-28T15:27:05.168 INFO:teuthology.orchestra.run.smithi178.stderr: 66900K .......... .......... .......... .......... .......... 72% 6.83M 1s 2024-06-28T15:27:05.170 INFO:teuthology.orchestra.run.smithi178.stderr: 66950K .......... .......... .......... .......... .......... 72% 24.9M 1s 2024-06-28T15:27:05.179 INFO:teuthology.orchestra.run.smithi178.stderr: 67000K .......... .......... .......... .......... .......... 72% 5.65M 1s 2024-06-28T15:27:05.184 INFO:teuthology.orchestra.run.smithi178.stderr: 67050K .......... .......... .......... .......... .......... 72% 10.9M 1s 2024-06-28T15:27:05.188 INFO:teuthology.orchestra.run.smithi178.stderr: 67100K .......... .......... .......... .......... .......... 72% 10.2M 1s 2024-06-28T15:27:05.195 INFO:teuthology.orchestra.run.smithi178.stderr: 67150K .......... .......... .......... .......... .......... 72% 7.41M 1s 2024-06-28T15:27:05.201 INFO:teuthology.orchestra.run.smithi178.stderr: 67200K .......... .......... .......... .......... .......... 72% 7.70M 1s 2024-06-28T15:27:05.209 INFO:teuthology.orchestra.run.smithi178.stderr: 67250K .......... .......... .......... .......... .......... 72% 6.34M 1s 2024-06-28T15:27:05.218 INFO:teuthology.orchestra.run.smithi178.stderr: 67300K .......... .......... .......... .......... .......... 72% 5.42M 1s 2024-06-28T15:27:05.222 INFO:teuthology.orchestra.run.smithi178.stderr: 67350K .......... .......... .......... .......... .......... 72% 10.5M 1s 2024-06-28T15:27:05.226 INFO:teuthology.orchestra.run.smithi178.stderr: 67400K .......... .......... .......... .......... .......... 72% 14.6M 1s 2024-06-28T15:27:05.230 INFO:teuthology.orchestra.run.smithi178.stderr: 67450K .......... .......... .......... .......... .......... 72% 16.8M 1s 2024-06-28T15:27:05.237 INFO:teuthology.orchestra.run.smithi178.stderr: 67500K .......... .......... .......... .......... .......... 72% 5.67M 1s 2024-06-28T15:27:05.245 INFO:teuthology.orchestra.run.smithi178.stderr: 67550K .......... .......... .......... .......... .......... 72% 6.76M 1s 2024-06-28T15:27:05.250 INFO:teuthology.orchestra.run.smithi178.stderr: 67600K .......... .......... .......... .......... .......... 72% 9.66M 1s 2024-06-28T15:27:05.260 INFO:teuthology.orchestra.run.smithi178.stderr: 67650K .......... .......... .......... .......... .......... 72% 4.59M 1s 2024-06-28T15:27:05.267 INFO:teuthology.orchestra.run.smithi178.stderr: 67700K .......... .......... .......... .......... .......... 72% 8.16M 1s 2024-06-28T15:27:05.271 INFO:teuthology.orchestra.run.smithi178.stderr: 67750K .......... .......... .......... .......... .......... 72% 14.6M 1s 2024-06-28T15:27:05.275 INFO:teuthology.orchestra.run.smithi178.stderr: 67800K .......... .......... .......... .......... .......... 73% 10.1M 1s 2024-06-28T15:27:05.285 INFO:teuthology.orchestra.run.smithi178.stderr: 67850K .......... .......... .......... .......... .......... 73% 6.04M 1s 2024-06-28T15:27:05.295 INFO:teuthology.orchestra.run.smithi178.stderr: 67900K .......... .......... .......... .......... .......... 73% 5.83M 1s 2024-06-28T15:27:05.297 INFO:teuthology.orchestra.run.smithi178.stderr: 67950K .......... .......... .......... .......... .......... 73% 7.85M 1s 2024-06-28T15:27:05.309 INFO:teuthology.orchestra.run.smithi178.stderr: 68000K .......... .......... .......... .......... .......... 73% 5.34M 1s 2024-06-28T15:27:05.313 INFO:teuthology.orchestra.run.smithi178.stderr: 68050K .......... .......... .......... .......... .......... 73% 7.45M 1s 2024-06-28T15:27:05.316 INFO:teuthology.orchestra.run.smithi178.stderr: 68100K .......... .......... .......... .......... .......... 73% 17.1M 1s 2024-06-28T15:27:05.320 INFO:teuthology.orchestra.run.smithi178.stderr: 68150K .......... .......... .......... .......... .......... 73% 10.4M 1s 2024-06-28T15:27:05.327 INFO:teuthology.orchestra.run.smithi178.stderr: 68200K .......... .......... .......... .......... .......... 73% 7.73M 1s 2024-06-28T15:27:05.334 INFO:teuthology.orchestra.run.smithi178.stderr: 68250K .......... .......... .......... .......... .......... 73% 8.72M 1s 2024-06-28T15:27:05.341 INFO:teuthology.orchestra.run.smithi178.stderr: 68300K .......... .......... .......... .......... .......... 73% 6.09M 1s 2024-06-28T15:27:05.350 INFO:teuthology.orchestra.run.smithi178.stderr: 68350K .......... .......... .......... .......... .......... 73% 4.89M 1s 2024-06-28T15:27:05.354 INFO:teuthology.orchestra.run.smithi178.stderr: 68400K .......... .......... .......... .......... .......... 73% 11.9M 1s 2024-06-28T15:27:05.360 INFO:teuthology.orchestra.run.smithi178.stderr: 68450K .......... .......... .......... .......... .......... 73% 8.43M 1s 2024-06-28T15:27:05.365 INFO:teuthology.orchestra.run.smithi178.stderr: 68500K .......... .......... .......... .......... .......... 73% 10.7M 1s 2024-06-28T15:27:05.373 INFO:teuthology.orchestra.run.smithi178.stderr: 68550K .......... .......... .......... .......... .......... 73% 5.78M 1s 2024-06-28T15:27:05.380 INFO:teuthology.orchestra.run.smithi178.stderr: 68600K .......... .......... .......... .......... .......... 73% 6.76M 1s 2024-06-28T15:27:05.386 INFO:teuthology.orchestra.run.smithi178.stderr: 68650K .......... .......... .......... .......... .......... 73% 9.31M 1s 2024-06-28T15:27:05.401 INFO:teuthology.orchestra.run.smithi178.stderr: 68700K .......... .......... .......... .......... .......... 74% 3.25M 1s 2024-06-28T15:27:05.406 INFO:teuthology.orchestra.run.smithi178.stderr: 68750K .......... .......... .......... .......... .......... 74% 9.08M 1s 2024-06-28T15:27:05.413 INFO:teuthology.orchestra.run.smithi178.stderr: 68800K .......... .......... .......... .......... .......... 74% 7.05M 1s 2024-06-28T15:27:05.419 INFO:teuthology.orchestra.run.smithi178.stderr: 68850K .......... .......... .......... .......... .......... 74% 7.96M 1s 2024-06-28T15:27:05.474 INFO:teuthology.orchestra.run.smithi178.stderr: 68900K .......... .......... .......... .......... .......... 74% 912K 1s 2024-06-28T15:27:05.484 INFO:teuthology.orchestra.run.smithi178.stderr: 68950K .......... .......... .......... .......... .......... 74% 11.3M 1s 2024-06-28T15:27:05.485 INFO:teuthology.orchestra.run.smithi178.stderr: 69000K .......... .......... .......... .......... .......... 74% 124M 1s 2024-06-28T15:27:05.485 INFO:teuthology.orchestra.run.smithi178.stderr: 69050K .......... .......... .......... .......... .......... 74% 140M 1s 2024-06-28T15:27:05.485 INFO:teuthology.orchestra.run.smithi178.stderr: 69100K .......... .......... .......... .......... .......... 74% 118M 1s 2024-06-28T15:27:05.485 INFO:teuthology.orchestra.run.smithi178.stderr: 69150K .......... .......... .......... .......... .......... 74% 121M 1s 2024-06-28T15:27:05.485 INFO:teuthology.orchestra.run.smithi178.stderr: 69200K .......... .......... .......... .......... .......... 74% 151M 1s 2024-06-28T15:27:05.485 INFO:teuthology.orchestra.run.smithi178.stderr: 69250K .......... .......... .......... .......... .......... 74% 154M 1s 2024-06-28T15:27:05.485 INFO:teuthology.orchestra.run.smithi178.stderr: 69300K .......... .......... .......... .......... .......... 74% 131M 1s 2024-06-28T15:27:05.494 INFO:teuthology.orchestra.run.smithi178.stderr: 69350K .......... .......... .......... .......... .......... 74% 3.65M 1s 2024-06-28T15:27:05.500 INFO:teuthology.orchestra.run.smithi178.stderr: 69400K .......... .......... .......... .......... .......... 74% 8.48M 1s 2024-06-28T15:27:05.506 INFO:teuthology.orchestra.run.smithi178.stderr: 69450K .......... .......... .......... .......... .......... 74% 8.93M 1s 2024-06-28T15:27:05.513 INFO:teuthology.orchestra.run.smithi178.stderr: 69500K .......... .......... .......... .......... .......... 74% 6.85M 1s 2024-06-28T15:27:05.517 INFO:teuthology.orchestra.run.smithi178.stderr: 69550K .......... .......... .......... .......... .......... 74% 11.0M 1s 2024-06-28T15:27:05.528 INFO:teuthology.orchestra.run.smithi178.stderr: 69600K .......... .......... .......... .......... .......... 74% 4.55M 1s 2024-06-28T15:27:05.534 INFO:teuthology.orchestra.run.smithi178.stderr: 69650K .......... .......... .......... .......... .......... 75% 7.37M 1s 2024-06-28T15:27:05.539 INFO:teuthology.orchestra.run.smithi178.stderr: 69700K .......... .......... .......... .......... .......... 75% 9.95M 1s 2024-06-28T15:27:05.546 INFO:teuthology.orchestra.run.smithi178.stderr: 69750K .......... .......... .......... .......... .......... 75% 7.45M 1s 2024-06-28T15:27:05.553 INFO:teuthology.orchestra.run.smithi178.stderr: 69800K .......... .......... .......... .......... .......... 75% 6.40M 1s 2024-06-28T15:27:05.565 INFO:teuthology.orchestra.run.smithi178.stderr: 69850K .......... .......... .......... .......... .......... 75% 5.32M 1s 2024-06-28T15:27:05.569 INFO:teuthology.orchestra.run.smithi178.stderr: 69900K .......... .......... .......... .......... .......... 75% 7.22M 1s 2024-06-28T15:27:05.574 INFO:teuthology.orchestra.run.smithi178.stderr: 69950K .......... .......... .......... .......... .......... 75% 10.1M 1s 2024-06-28T15:27:05.583 INFO:teuthology.orchestra.run.smithi178.stderr: 70000K .......... .......... .......... .......... .......... 75% 5.51M 1s 2024-06-28T15:27:05.590 INFO:teuthology.orchestra.run.smithi178.stderr: 70050K .......... .......... .......... .......... .......... 75% 6.62M 1s 2024-06-28T15:27:05.606 INFO:teuthology.orchestra.run.smithi178.stderr: 70100K .......... .......... .......... .......... .......... 75% 3.06M 1s 2024-06-28T15:27:05.614 INFO:teuthology.orchestra.run.smithi178.stderr: 70150K .......... .......... .......... .......... .......... 75% 6.36M 1s 2024-06-28T15:27:05.621 INFO:teuthology.orchestra.run.smithi178.stderr: 70200K .......... .......... .......... .......... .......... 75% 7.11M 1s 2024-06-28T15:27:05.630 INFO:teuthology.orchestra.run.smithi178.stderr: 70250K .......... .......... .......... .......... .......... 75% 6.20M 1s 2024-06-28T15:27:05.639 INFO:teuthology.orchestra.run.smithi178.stderr: 70300K .......... .......... .......... .......... .......... 75% 4.65M 1s 2024-06-28T15:27:05.657 INFO:teuthology.orchestra.run.smithi178.stderr: 70350K .......... .......... .......... .......... .......... 75% 2.68M 1s 2024-06-28T15:27:05.667 INFO:teuthology.orchestra.run.smithi178.stderr: 70400K .......... .......... .......... .......... .......... 75% 5.18M 1s 2024-06-28T15:27:05.674 INFO:teuthology.orchestra.run.smithi178.stderr: 70450K .......... .......... .......... .......... .......... 75% 7.40M 1s 2024-06-28T15:27:05.680 INFO:teuthology.orchestra.run.smithi178.stderr: 70500K .......... .......... .......... .......... .......... 75% 7.07M 1s 2024-06-28T15:27:05.688 INFO:teuthology.orchestra.run.smithi178.stderr: 70550K .......... .......... .......... .......... .......... 75% 6.04M 1s 2024-06-28T15:27:05.706 INFO:teuthology.orchestra.run.smithi178.stderr: 70600K .......... .......... .......... .......... .......... 76% 2.83M 1s 2024-06-28T15:27:05.726 INFO:teuthology.orchestra.run.smithi178.stderr: 70650K .......... .......... .......... .......... .......... 76% 2.99M 1s 2024-06-28T15:27:05.797 INFO:teuthology.orchestra.run.smithi178.stderr: 70700K .......... .......... .......... .......... .......... 76% 668K 1s 2024-06-28T15:27:05.804 INFO:teuthology.orchestra.run.smithi178.stderr: 70750K .......... .......... .......... .......... .......... 76% 7.17M 1s 2024-06-28T15:27:05.806 INFO:teuthology.orchestra.run.smithi178.stderr: 70800K .......... .......... .......... .......... .......... 76% 34.3M 1s 2024-06-28T15:27:05.824 INFO:teuthology.orchestra.run.smithi178.stderr: 70850K .......... .......... .......... .......... .......... 76% 2.62M 1s 2024-06-28T15:27:05.836 INFO:teuthology.orchestra.run.smithi178.stderr: 70900K .......... .......... .......... .......... .......... 76% 5.88M 1s 2024-06-28T15:27:05.836 INFO:teuthology.orchestra.run.smithi178.stderr: 70950K .......... .......... .......... .......... .......... 76% 132M 1s 2024-06-28T15:27:05.836 INFO:teuthology.orchestra.run.smithi178.stderr: 71000K .......... .......... .......... .......... .......... 76% 146M 1s 2024-06-28T15:27:05.837 INFO:teuthology.orchestra.run.smithi178.stderr: 71050K .......... .......... .......... .......... .......... 76% 28.1M 1s 2024-06-28T15:27:05.843 INFO:teuthology.orchestra.run.smithi178.stderr: 71100K .......... .......... .......... .......... .......... 76% 6.15M 1s 2024-06-28T15:27:05.862 INFO:teuthology.orchestra.run.smithi178.stderr: 71150K .......... .......... .......... .......... .......... 76% 2.52M 1s 2024-06-28T15:27:05.877 INFO:teuthology.orchestra.run.smithi178.stderr: 71200K .......... .......... .......... .......... .......... 76% 3.34M 1s 2024-06-28T15:27:05.884 INFO:teuthology.orchestra.run.smithi178.stderr: 71250K .......... .......... .......... .......... .......... 76% 6.84M 1s 2024-06-28T15:27:05.891 INFO:teuthology.orchestra.run.smithi178.stderr: 71300K .......... .......... .......... .......... .......... 76% 7.05M 1s 2024-06-28T15:27:05.898 INFO:teuthology.orchestra.run.smithi178.stderr: 71350K .......... .......... .......... .......... .......... 76% 6.39M 1s 2024-06-28T15:27:05.910 INFO:teuthology.orchestra.run.smithi178.stderr: 71400K .......... .......... .......... .......... .......... 76% 4.02M 1s 2024-06-28T15:27:05.924 INFO:teuthology.orchestra.run.smithi178.stderr: 71450K .......... .......... .......... .......... .......... 76% 4.93M 1s 2024-06-28T15:27:05.932 INFO:teuthology.orchestra.run.smithi178.stderr: 71500K .......... .......... .......... .......... .......... 77% 4.07M 1s 2024-06-28T15:27:05.943 INFO:teuthology.orchestra.run.smithi178.stderr: 71550K .......... .......... .......... .......... .......... 77% 4.54M 1s 2024-06-28T15:27:05.955 INFO:teuthology.orchestra.run.smithi178.stderr: 71600K .......... .......... .......... .......... .......... 77% 4.10M 1s 2024-06-28T15:27:05.966 INFO:teuthology.orchestra.run.smithi178.stderr: 71650K .......... .......... .......... .......... .......... 77% 4.34M 1s 2024-06-28T15:27:05.976 INFO:teuthology.orchestra.run.smithi178.stderr: 71700K .......... .......... .......... .......... .......... 77% 4.87M 1s 2024-06-28T15:27:05.993 INFO:teuthology.orchestra.run.smithi178.stderr: 71750K .......... .......... .......... .......... .......... 77% 2.90M 1s 2024-06-28T15:27:06.005 INFO:teuthology.orchestra.run.smithi178.stderr: 71800K .......... .......... .......... .......... .......... 77% 4.34M 1s 2024-06-28T15:27:06.021 INFO:teuthology.orchestra.run.smithi178.stderr: 71850K .......... .......... .......... .......... .......... 77% 4.34M 1s 2024-06-28T15:27:06.031 INFO:teuthology.orchestra.run.smithi178.stderr: 71900K .......... .......... .......... .......... .......... 77% 3.13M 1s 2024-06-28T15:27:06.042 INFO:teuthology.orchestra.run.smithi178.stderr: 71950K .......... .......... .......... .......... .......... 77% 4.48M 1s 2024-06-28T15:27:06.049 INFO:teuthology.orchestra.run.smithi178.stderr: 72000K .......... .......... .......... .......... .......... 77% 7.00M 1s 2024-06-28T15:27:06.057 INFO:teuthology.orchestra.run.smithi178.stderr: 72050K .......... .......... .......... .......... .......... 77% 6.43M 1s 2024-06-28T15:27:06.072 INFO:teuthology.orchestra.run.smithi178.stderr: 72100K .......... .......... .......... .......... .......... 77% 3.08M 1s 2024-06-28T15:27:06.090 INFO:teuthology.orchestra.run.smithi178.stderr: 72150K .......... .......... .......... .......... .......... 77% 2.74M 1s 2024-06-28T15:27:06.106 INFO:teuthology.orchestra.run.smithi178.stderr: 72200K .......... .......... .......... .......... .......... 77% 3.18M 1s 2024-06-28T15:27:06.118 INFO:teuthology.orchestra.run.smithi178.stderr: 72250K .......... .......... .......... .......... .......... 77% 4.62M 1s 2024-06-28T15:27:06.127 INFO:teuthology.orchestra.run.smithi178.stderr: 72300K .......... .......... .......... .......... .......... 77% 4.68M 1s 2024-06-28T15:27:06.138 INFO:teuthology.orchestra.run.smithi178.stderr: 72350K .......... .......... .......... .......... .......... 77% 4.36M 1s 2024-06-28T15:27:06.147 INFO:teuthology.orchestra.run.smithi178.stderr: 72400K .......... .......... .......... .......... .......... 77% 5.17M 1s 2024-06-28T15:27:06.163 INFO:teuthology.orchestra.run.smithi178.stderr: 72450K .......... .......... .......... .......... .......... 78% 3.12M 1s 2024-06-28T15:27:06.182 INFO:teuthology.orchestra.run.smithi178.stderr: 72500K .......... .......... .......... .......... .......... 78% 2.61M 1s 2024-06-28T15:27:06.198 INFO:teuthology.orchestra.run.smithi178.stderr: 72550K .......... .......... .......... .......... .......... 78% 3.05M 1s 2024-06-28T15:27:06.203 INFO:teuthology.orchestra.run.smithi178.stderr: 72600K .......... .......... .......... .......... .......... 78% 8.44M 1s 2024-06-28T15:27:06.224 INFO:teuthology.orchestra.run.smithi178.stderr: 72650K .......... .......... .......... .......... .......... 78% 4.86M 1s 2024-06-28T15:27:06.236 INFO:teuthology.orchestra.run.smithi178.stderr: 72700K .......... .......... .......... .......... .......... 78% 2.18M 1s 2024-06-28T15:27:06.257 INFO:teuthology.orchestra.run.smithi178.stderr: 72750K .......... .......... .......... .......... .......... 78% 2.33M 1s 2024-06-28T15:27:06.263 INFO:teuthology.orchestra.run.smithi178.stderr: 72800K .......... .......... .......... .......... .......... 78% 7.42M 1s 2024-06-28T15:27:06.279 INFO:teuthology.orchestra.run.smithi178.stderr: 72850K .......... .......... .......... .......... .......... 78% 3.11M 1s 2024-06-28T15:27:06.290 INFO:teuthology.orchestra.run.smithi178.stderr: 72900K .......... .......... .......... .......... .......... 78% 4.63M 1s 2024-06-28T15:27:06.313 INFO:teuthology.orchestra.run.smithi178.stderr: 72950K .......... .......... .......... .......... .......... 78% 2.07M 1s 2024-06-28T15:27:06.319 INFO:teuthology.orchestra.run.smithi178.stderr: 73000K .......... .......... .......... .......... .......... 78% 8.90M 1s 2024-06-28T15:27:06.341 INFO:teuthology.orchestra.run.smithi178.stderr: 73050K .......... .......... .......... .......... .......... 78% 11.1M 1s 2024-06-28T15:27:06.349 INFO:teuthology.orchestra.run.smithi178.stderr: 73100K .......... .......... .......... .......... .......... 78% 1.91M 1s 2024-06-28T15:27:06.372 INFO:teuthology.orchestra.run.smithi178.stderr: 73150K .......... .......... .......... .......... .......... 78% 2.13M 1s 2024-06-28T15:27:06.377 INFO:teuthology.orchestra.run.smithi178.stderr: 73200K .......... .......... .......... .......... .......... 78% 9.43M 1s 2024-06-28T15:27:06.386 INFO:teuthology.orchestra.run.smithi178.stderr: 73250K .......... .......... .......... .......... .......... 78% 5.34M 1s 2024-06-28T15:27:06.394 INFO:teuthology.orchestra.run.smithi178.stderr: 73300K .......... .......... .......... .......... .......... 78% 5.97M 1s 2024-06-28T15:27:06.406 INFO:teuthology.orchestra.run.smithi178.stderr: 73350K .......... .......... .......... .......... .......... 79% 4.16M 1s 2024-06-28T15:27:06.411 INFO:teuthology.orchestra.run.smithi178.stderr: 73400K .......... .......... .......... .......... .......... 79% 9.34M 1s 2024-06-28T15:27:06.423 INFO:teuthology.orchestra.run.smithi178.stderr: 73450K .......... .......... .......... .......... .......... 79% 5.29M 1s 2024-06-28T15:27:06.431 INFO:teuthology.orchestra.run.smithi178.stderr: 73500K .......... .......... .......... .......... .......... 79% 4.87M 1s 2024-06-28T15:27:06.438 INFO:teuthology.orchestra.run.smithi178.stderr: 73550K .......... .......... .......... .......... .......... 79% 6.20M 1s 2024-06-28T15:27:06.445 INFO:teuthology.orchestra.run.smithi178.stderr: 73600K .......... .......... .......... .......... .......... 79% 7.67M 1s 2024-06-28T15:27:06.450 INFO:teuthology.orchestra.run.smithi178.stderr: 73650K .......... .......... .......... .......... .......... 79% 9.13M 1s 2024-06-28T15:27:06.456 INFO:teuthology.orchestra.run.smithi178.stderr: 73700K .......... .......... .......... .......... .......... 79% 7.99M 1s 2024-06-28T15:27:06.462 INFO:teuthology.orchestra.run.smithi178.stderr: 73750K .......... .......... .......... .......... .......... 79% 8.15M 1s 2024-06-28T15:27:06.470 INFO:teuthology.orchestra.run.smithi178.stderr: 73800K .......... .......... .......... .......... .......... 79% 6.15M 1s 2024-06-28T15:27:06.477 INFO:teuthology.orchestra.run.smithi178.stderr: 73850K .......... .......... .......... .......... .......... 79% 8.74M 1s 2024-06-28T15:27:06.484 INFO:teuthology.orchestra.run.smithi178.stderr: 73900K .......... .......... .......... .......... .......... 79% 6.12M 1s 2024-06-28T15:27:06.489 INFO:teuthology.orchestra.run.smithi178.stderr: 73950K .......... .......... .......... .......... .......... 79% 9.61M 1s 2024-06-28T15:27:06.499 INFO:teuthology.orchestra.run.smithi178.stderr: 74000K .......... .......... .......... .......... .......... 79% 4.57M 1s 2024-06-28T15:27:06.504 INFO:teuthology.orchestra.run.smithi178.stderr: 74050K .......... .......... .......... .......... .......... 79% 10.7M 1s 2024-06-28T15:27:06.512 INFO:teuthology.orchestra.run.smithi178.stderr: 74100K .......... .......... .......... .......... .......... 79% 6.10M 1s 2024-06-28T15:27:06.518 INFO:teuthology.orchestra.run.smithi178.stderr: 74150K .......... .......... .......... .......... .......... 79% 8.50M 1s 2024-06-28T15:27:06.525 INFO:teuthology.orchestra.run.smithi178.stderr: 74200K .......... .......... .......... .......... .......... 79% 6.64M 1s 2024-06-28T15:27:06.534 INFO:teuthology.orchestra.run.smithi178.stderr: 74250K .......... .......... .......... .......... .......... 79% 9.03M 1s 2024-06-28T15:27:06.539 INFO:teuthology.orchestra.run.smithi178.stderr: 74300K .......... .......... .......... .......... .......... 80% 5.66M 1s 2024-06-28T15:27:06.547 INFO:teuthology.orchestra.run.smithi178.stderr: 74350K .......... .......... .......... .......... .......... 80% 6.17M 1s 2024-06-28T15:27:06.554 INFO:teuthology.orchestra.run.smithi178.stderr: 74400K .......... .......... .......... .......... .......... 80% 6.82M 1s 2024-06-28T15:27:06.560 INFO:teuthology.orchestra.run.smithi178.stderr: 74450K .......... .......... .......... .......... .......... 80% 8.43M 1s 2024-06-28T15:27:06.565 INFO:teuthology.orchestra.run.smithi178.stderr: 74500K .......... .......... .......... .......... .......... 80% 9.36M 1s 2024-06-28T15:27:06.573 INFO:teuthology.orchestra.run.smithi178.stderr: 74550K .......... .......... .......... .......... .......... 80% 5.87M 1s 2024-06-28T15:27:06.580 INFO:teuthology.orchestra.run.smithi178.stderr: 74600K .......... .......... .......... .......... .......... 80% 7.46M 1s 2024-06-28T15:27:06.588 INFO:teuthology.orchestra.run.smithi178.stderr: 74650K .......... .......... .......... .......... .......... 80% 8.37M 1s 2024-06-28T15:27:06.595 INFO:teuthology.orchestra.run.smithi178.stderr: 74700K .......... .......... .......... .......... .......... 80% 5.47M 1s 2024-06-28T15:27:06.604 INFO:teuthology.orchestra.run.smithi178.stderr: 74750K .......... .......... .......... .......... .......... 80% 5.73M 1s 2024-06-28T15:27:06.608 INFO:teuthology.orchestra.run.smithi178.stderr: 74800K .......... .......... .......... .......... .......... 80% 9.39M 1s 2024-06-28T15:27:06.614 INFO:teuthology.orchestra.run.smithi178.stderr: 74850K .......... .......... .......... .......... .......... 80% 8.70M 1s 2024-06-28T15:27:06.622 INFO:teuthology.orchestra.run.smithi178.stderr: 74900K .......... .......... .......... .......... .......... 80% 6.36M 1s 2024-06-28T15:27:06.631 INFO:teuthology.orchestra.run.smithi178.stderr: 74950K .......... .......... .......... .......... .......... 80% 5.48M 1s 2024-06-28T15:27:06.639 INFO:teuthology.orchestra.run.smithi178.stderr: 75000K .......... .......... .......... .......... .......... 80% 5.72M 1s 2024-06-28T15:27:06.647 INFO:teuthology.orchestra.run.smithi178.stderr: 75050K .......... .......... .......... .......... .......... 80% 8.02M 1s 2024-06-28T15:27:06.653 INFO:teuthology.orchestra.run.smithi178.stderr: 75100K .......... .......... .......... .......... .......... 80% 6.65M 1s 2024-06-28T15:27:06.662 INFO:teuthology.orchestra.run.smithi178.stderr: 75150K .......... .......... .......... .......... .......... 80% 5.09M 1s 2024-06-28T15:27:06.669 INFO:teuthology.orchestra.run.smithi178.stderr: 75200K .......... .......... .......... .......... .......... 81% 7.30M 1s 2024-06-28T15:27:06.674 INFO:teuthology.orchestra.run.smithi178.stderr: 75250K .......... .......... .......... .......... .......... 81% 8.87M 1s 2024-06-28T15:27:06.683 INFO:teuthology.orchestra.run.smithi178.stderr: 75300K .......... .......... .......... .......... .......... 81% 5.94M 1s 2024-06-28T15:27:06.690 INFO:teuthology.orchestra.run.smithi178.stderr: 75350K .......... .......... .......... .......... .......... 81% 6.79M 1s 2024-06-28T15:27:06.698 INFO:teuthology.orchestra.run.smithi178.stderr: 75400K .......... .......... .......... .......... .......... 81% 6.00M 1s 2024-06-28T15:27:06.709 INFO:teuthology.orchestra.run.smithi178.stderr: 75450K .......... .......... .......... .......... .......... 81% 6.51M 1s 2024-06-28T15:27:06.717 INFO:teuthology.orchestra.run.smithi178.stderr: 75500K .......... .......... .......... .......... .......... 81% 4.30M 1s 2024-06-28T15:27:06.725 INFO:teuthology.orchestra.run.smithi178.stderr: 75550K .......... .......... .......... .......... .......... 81% 5.65M 1s 2024-06-28T15:27:06.734 INFO:teuthology.orchestra.run.smithi178.stderr: 75600K .......... .......... .......... .......... .......... 81% 5.99M 1s 2024-06-28T15:27:06.739 INFO:teuthology.orchestra.run.smithi178.stderr: 75650K .......... .......... .......... .......... .......... 81% 8.75M 1s 2024-06-28T15:27:06.748 INFO:teuthology.orchestra.run.smithi178.stderr: 75700K .......... .......... .......... .......... .......... 81% 5.75M 1s 2024-06-28T15:27:06.754 INFO:teuthology.orchestra.run.smithi178.stderr: 75750K .......... .......... .......... .......... .......... 81% 7.80M 1s 2024-06-28T15:27:06.759 INFO:teuthology.orchestra.run.smithi178.stderr: 75800K .......... .......... .......... .......... .......... 81% 10.4M 1s 2024-06-28T15:27:06.767 INFO:teuthology.orchestra.run.smithi178.stderr: 75850K .......... .......... .......... .......... .......... 81% 8.62M 1s 2024-06-28T15:27:06.774 INFO:teuthology.orchestra.run.smithi178.stderr: 75900K .......... .......... .......... .......... .......... 81% 5.20M 1s 2024-06-28T15:27:06.781 INFO:teuthology.orchestra.run.smithi178.stderr: 75950K .......... .......... .......... .......... .......... 81% 7.07M 1s 2024-06-28T15:27:06.787 INFO:teuthology.orchestra.run.smithi178.stderr: 76000K .......... .......... .......... .......... .......... 81% 7.97M 1s 2024-06-28T15:27:06.792 INFO:teuthology.orchestra.run.smithi178.stderr: 76050K .......... .......... .......... .......... .......... 81% 8.63M 1s 2024-06-28T15:27:06.798 INFO:teuthology.orchestra.run.smithi178.stderr: 76100K .......... .......... .......... .......... .......... 81% 8.04M 1s 2024-06-28T15:27:06.808 INFO:teuthology.orchestra.run.smithi178.stderr: 76150K .......... .......... .......... .......... .......... 82% 5.13M 1s 2024-06-28T15:27:06.815 INFO:teuthology.orchestra.run.smithi178.stderr: 76200K .......... .......... .......... .......... .......... 82% 7.66M 1s 2024-06-28T15:27:06.820 INFO:teuthology.orchestra.run.smithi178.stderr: 76250K .......... .......... .......... .......... .......... 82% 12.1M 1s 2024-06-28T15:27:06.824 INFO:teuthology.orchestra.run.smithi178.stderr: 76300K .......... .......... .......... .......... .......... 82% 8.25M 1s 2024-06-28T15:27:06.835 INFO:teuthology.orchestra.run.smithi178.stderr: 76350K .......... .......... .......... .......... .......... 82% 4.46M 1s 2024-06-28T15:27:06.842 INFO:teuthology.orchestra.run.smithi178.stderr: 76400K .......... .......... .......... .......... .......... 82% 7.05M 1s 2024-06-28T15:27:06.846 INFO:teuthology.orchestra.run.smithi178.stderr: 76450K .......... .......... .......... .......... .......... 82% 12.1M 1s 2024-06-28T15:27:06.850 INFO:teuthology.orchestra.run.smithi178.stderr: 76500K .......... .......... .......... .......... .......... 82% 14.1M 1s 2024-06-28T15:27:06.858 INFO:teuthology.orchestra.run.smithi178.stderr: 76550K .......... .......... .......... .......... .......... 82% 5.52M 1s 2024-06-28T15:27:06.867 INFO:teuthology.orchestra.run.smithi178.stderr: 76600K .......... .......... .......... .......... .......... 82% 6.03M 1s 2024-06-28T15:27:06.876 INFO:teuthology.orchestra.run.smithi178.stderr: 76650K .......... .......... .......... .......... .......... 82% 7.32M 1s 2024-06-28T15:27:06.887 INFO:teuthology.orchestra.run.smithi178.stderr: 76700K .......... .......... .......... .......... .......... 82% 3.69M 1s 2024-06-28T15:27:06.891 INFO:teuthology.orchestra.run.smithi178.stderr: 76750K .......... .......... .......... .......... .......... 82% 11.2M 1s 2024-06-28T15:27:06.897 INFO:teuthology.orchestra.run.smithi178.stderr: 76800K .......... .......... .......... .......... .......... 82% 8.14M 1s 2024-06-28T15:27:06.903 INFO:teuthology.orchestra.run.smithi178.stderr: 76850K .......... .......... .......... .......... .......... 82% 7.44M 1s 2024-06-28T15:27:06.913 INFO:teuthology.orchestra.run.smithi178.stderr: 76900K .......... .......... .......... .......... .......... 82% 4.88M 1s 2024-06-28T15:27:06.919 INFO:teuthology.orchestra.run.smithi178.stderr: 76950K .......... .......... .......... .......... .......... 82% 8.02M 1s 2024-06-28T15:27:06.924 INFO:teuthology.orchestra.run.smithi178.stderr: 77000K .......... .......... .......... .......... .......... 82% 10.2M 1s 2024-06-28T15:27:06.932 INFO:teuthology.orchestra.run.smithi178.stderr: 77050K .......... .......... .......... .......... .......... 82% 7.97M 1s 2024-06-28T15:27:06.942 INFO:teuthology.orchestra.run.smithi178.stderr: 77100K .......... .......... .......... .......... .......... 83% 5.13M 1s 2024-06-28T15:27:06.948 INFO:teuthology.orchestra.run.smithi178.stderr: 77150K .......... .......... .......... .......... .......... 83% 6.34M 1s 2024-06-28T15:27:06.954 INFO:teuthology.orchestra.run.smithi178.stderr: 77200K .......... .......... .......... .......... .......... 83% 8.15M 1s 2024-06-28T15:27:06.960 INFO:teuthology.orchestra.run.smithi178.stderr: 77250K .......... .......... .......... .......... .......... 83% 7.60M 1s 2024-06-28T15:27:06.973 INFO:teuthology.orchestra.run.smithi178.stderr: 77300K .......... .......... .......... .......... .......... 83% 3.81M 1s 2024-06-28T15:27:06.979 INFO:teuthology.orchestra.run.smithi178.stderr: 77350K .......... .......... .......... .......... .......... 83% 7.91M 1s 2024-06-28T15:27:06.984 INFO:teuthology.orchestra.run.smithi178.stderr: 77400K .......... .......... .......... .......... .......... 83% 9.04M 1s 2024-06-28T15:27:06.992 INFO:teuthology.orchestra.run.smithi178.stderr: 77450K .......... .......... .......... .......... .......... 83% 9.75M 1s 2024-06-28T15:27:07.002 INFO:teuthology.orchestra.run.smithi178.stderr: 77500K .......... .......... .......... .......... .......... 83% 3.91M 1s 2024-06-28T15:27:07.008 INFO:teuthology.orchestra.run.smithi178.stderr: 77550K .......... .......... .......... .......... .......... 83% 8.22M 1s 2024-06-28T15:27:07.014 INFO:teuthology.orchestra.run.smithi178.stderr: 77600K .......... .......... .......... .......... .......... 83% 8.55M 1s 2024-06-28T15:27:07.019 INFO:teuthology.orchestra.run.smithi178.stderr: 77650K .......... .......... .......... .......... .......... 83% 9.74M 1s 2024-06-28T15:27:07.024 INFO:teuthology.orchestra.run.smithi178.stderr: 77700K .......... .......... .......... .......... .......... 83% 9.71M 1s 2024-06-28T15:27:07.030 INFO:teuthology.orchestra.run.smithi178.stderr: 77750K .......... .......... .......... .......... .......... 83% 7.39M 1s 2024-06-28T15:27:07.037 INFO:teuthology.orchestra.run.smithi178.stderr: 77800K .......... .......... .......... .......... .......... 83% 6.88M 1s 2024-06-28T15:27:07.044 INFO:teuthology.orchestra.run.smithi178.stderr: 77850K .......... .......... .......... .......... .......... 83% 8.96M 1s 2024-06-28T15:27:07.057 INFO:teuthology.orchestra.run.smithi178.stderr: 77900K .......... .......... .......... .......... .......... 83% 3.50M 1s 2024-06-28T15:27:07.060 INFO:teuthology.orchestra.run.smithi178.stderr: 77950K .......... .......... .......... .......... .......... 83% 13.5M 1s 2024-06-28T15:27:07.069 INFO:teuthology.orchestra.run.smithi178.stderr: 78000K .......... .......... .......... .......... .......... 84% 5.79M 1s 2024-06-28T15:27:07.074 INFO:teuthology.orchestra.run.smithi178.stderr: 78050K .......... .......... .......... .......... .......... 84% 9.96M 1s 2024-06-28T15:27:07.081 INFO:teuthology.orchestra.run.smithi178.stderr: 78100K .......... .......... .......... .......... .......... 84% 6.73M 1s 2024-06-28T15:27:07.089 INFO:teuthology.orchestra.run.smithi178.stderr: 78150K .......... .......... .......... .......... .......... 84% 5.86M 1s 2024-06-28T15:27:07.094 INFO:teuthology.orchestra.run.smithi178.stderr: 78200K .......... .......... .......... .......... .......... 84% 10.6M 1s 2024-06-28T15:27:07.101 INFO:teuthology.orchestra.run.smithi178.stderr: 78250K .......... .......... .......... .......... .......... 84% 8.51M 1s 2024-06-28T15:27:07.110 INFO:teuthology.orchestra.run.smithi178.stderr: 78300K .......... .......... .......... .......... .......... 84% 4.88M 1s 2024-06-28T15:27:07.117 INFO:teuthology.orchestra.run.smithi178.stderr: 78350K .......... .......... .......... .......... .......... 84% 7.11M 1s 2024-06-28T15:27:07.124 INFO:teuthology.orchestra.run.smithi178.stderr: 78400K .......... .......... .......... .......... .......... 84% 6.73M 1s 2024-06-28T15:27:07.133 INFO:teuthology.orchestra.run.smithi178.stderr: 78450K .......... .......... .......... .......... .......... 84% 5.08M 1s 2024-06-28T15:27:07.139 INFO:teuthology.orchestra.run.smithi178.stderr: 78500K .......... .......... .......... .......... .......... 84% 8.16M 1s 2024-06-28T15:27:07.147 INFO:teuthology.orchestra.run.smithi178.stderr: 78550K .......... .......... .......... .......... .......... 84% 6.39M 1s 2024-06-28T15:27:07.150 INFO:teuthology.orchestra.run.smithi178.stderr: 78600K .......... .......... .......... .......... .......... 84% 13.7M 1s 2024-06-28T15:27:07.163 INFO:teuthology.orchestra.run.smithi178.stderr: 78650K .......... .......... .......... .......... .......... 84% 5.95M 1s 2024-06-28T15:27:07.170 INFO:teuthology.orchestra.run.smithi178.stderr: 78700K .......... .......... .......... .......... .......... 84% 4.35M 1s 2024-06-28T15:27:07.178 INFO:teuthology.orchestra.run.smithi178.stderr: 78750K .......... .......... .......... .......... .......... 84% 6.18M 1s 2024-06-28T15:27:07.183 INFO:teuthology.orchestra.run.smithi178.stderr: 78800K .......... .......... .......... .......... .......... 84% 9.74M 1s 2024-06-28T15:27:07.189 INFO:teuthology.orchestra.run.smithi178.stderr: 78850K .......... .......... .......... .......... .......... 84% 8.58M 1s 2024-06-28T15:27:07.195 INFO:teuthology.orchestra.run.smithi178.stderr: 78900K .......... .......... .......... .......... .......... 84% 7.22M 1s 2024-06-28T15:27:07.206 INFO:teuthology.orchestra.run.smithi178.stderr: 78950K .......... .......... .......... .......... .......... 85% 4.73M 1s 2024-06-28T15:27:07.213 INFO:teuthology.orchestra.run.smithi178.stderr: 79000K .......... .......... .......... .......... .......... 85% 6.30M 1s 2024-06-28T15:27:07.221 INFO:teuthology.orchestra.run.smithi178.stderr: 79050K .......... .......... .......... .......... .......... 85% 7.87M 1s 2024-06-28T15:27:07.227 INFO:teuthology.orchestra.run.smithi178.stderr: 79100K .......... .......... .......... .......... .......... 85% 6.33M 1s 2024-06-28T15:27:07.234 INFO:teuthology.orchestra.run.smithi178.stderr: 79150K .......... .......... .......... .......... .......... 85% 7.76M 1s 2024-06-28T15:27:07.242 INFO:teuthology.orchestra.run.smithi178.stderr: 79200K .......... .......... .......... .......... .......... 85% 5.80M 1s 2024-06-28T15:27:07.250 INFO:teuthology.orchestra.run.smithi178.stderr: 79250K .......... .......... .......... .......... .......... 85% 5.96M 1s 2024-06-28T15:27:07.256 INFO:teuthology.orchestra.run.smithi178.stderr: 79300K .......... .......... .......... .......... .......... 85% 8.08M 1s 2024-06-28T15:27:07.261 INFO:teuthology.orchestra.run.smithi178.stderr: 79350K .......... .......... .......... .......... .......... 85% 9.78M 1s 2024-06-28T15:27:07.267 INFO:teuthology.orchestra.run.smithi178.stderr: 79400K .......... .......... .......... .......... .......... 85% 7.87M 1s 2024-06-28T15:27:07.278 INFO:teuthology.orchestra.run.smithi178.stderr: 79450K .......... .......... .......... .......... .......... 85% 6.27M 1s 2024-06-28T15:27:07.285 INFO:teuthology.orchestra.run.smithi178.stderr: 79500K .......... .......... .......... .......... .......... 85% 5.17M 1s 2024-06-28T15:27:07.290 INFO:teuthology.orchestra.run.smithi178.stderr: 79550K .......... .......... .......... .......... .......... 85% 8.65M 1s 2024-06-28T15:27:07.296 INFO:teuthology.orchestra.run.smithi178.stderr: 79600K .......... .......... .......... .......... .......... 85% 8.54M 1s 2024-06-28T15:27:07.304 INFO:teuthology.orchestra.run.smithi178.stderr: 79650K .......... .......... .......... .......... .......... 85% 6.45M 1s 2024-06-28T15:27:07.309 INFO:teuthology.orchestra.run.smithi178.stderr: 79700K .......... .......... .......... .......... .......... 85% 8.80M 1s 2024-06-28T15:27:07.315 INFO:teuthology.orchestra.run.smithi178.stderr: 79750K .......... .......... .......... .......... .......... 85% 8.90M 1s 2024-06-28T15:27:07.321 INFO:teuthology.orchestra.run.smithi178.stderr: 79800K .......... .......... .......... .......... .......... 85% 8.00M 1s 2024-06-28T15:27:07.331 INFO:teuthology.orchestra.run.smithi178.stderr: 79850K .......... .......... .......... .......... .......... 86% 7.25M 1s 2024-06-28T15:27:07.337 INFO:teuthology.orchestra.run.smithi178.stderr: 79900K .......... .......... .......... .......... .......... 86% 5.36M 1s 2024-06-28T15:27:07.341 INFO:teuthology.orchestra.run.smithi178.stderr: 79950K .......... .......... .......... .......... .......... 86% 11.0M 1s 2024-06-28T15:27:07.347 INFO:teuthology.orchestra.run.smithi178.stderr: 80000K .......... .......... .......... .......... .......... 86% 8.79M 1s 2024-06-28T15:27:07.353 INFO:teuthology.orchestra.run.smithi178.stderr: 80050K .......... .......... .......... .......... .......... 86% 7.16M 1s 2024-06-28T15:27:07.360 INFO:teuthology.orchestra.run.smithi178.stderr: 80100K .......... .......... .......... .......... .......... 86% 7.78M 1s 2024-06-28T15:27:07.365 INFO:teuthology.orchestra.run.smithi178.stderr: 80150K .......... .......... .......... .......... .......... 86% 9.96M 1s 2024-06-28T15:27:07.370 INFO:teuthology.orchestra.run.smithi178.stderr: 80200K .......... .......... .......... .......... .......... 86% 9.44M 1s 2024-06-28T15:27:07.379 INFO:teuthology.orchestra.run.smithi178.stderr: 80250K .......... .......... .......... .......... .......... 86% 7.05M 1s 2024-06-28T15:27:07.385 INFO:teuthology.orchestra.run.smithi178.stderr: 80300K .......... .......... .......... .......... .......... 86% 6.54M 1s 2024-06-28T15:27:07.388 INFO:teuthology.orchestra.run.smithi178.stderr: 80350K .......... .......... .......... .......... .......... 86% 13.3M 1s 2024-06-28T15:27:07.395 INFO:teuthology.orchestra.run.smithi178.stderr: 80400K .......... .......... .......... .......... .......... 86% 7.00M 1s 2024-06-28T15:27:07.402 INFO:teuthology.orchestra.run.smithi178.stderr: 80450K .......... .......... .......... .......... .......... 86% 6.91M 1s 2024-06-28T15:27:07.408 INFO:teuthology.orchestra.run.smithi178.stderr: 80500K .......... .......... .......... .......... .......... 86% 8.40M 1s 2024-06-28T15:27:07.411 INFO:teuthology.orchestra.run.smithi178.stderr: 80550K .......... .......... .......... .......... .......... 86% 12.9M 1s 2024-06-28T15:27:07.418 INFO:teuthology.orchestra.run.smithi178.stderr: 80600K .......... .......... .......... .......... .......... 86% 7.80M 1s 2024-06-28T15:27:07.427 INFO:teuthology.orchestra.run.smithi178.stderr: 80650K .......... .......... .......... .......... .......... 86% 8.50M 1s 2024-06-28T15:27:07.432 INFO:teuthology.orchestra.run.smithi178.stderr: 80700K .......... .......... .......... .......... .......... 86% 5.74M 1s 2024-06-28T15:27:07.439 INFO:teuthology.orchestra.run.smithi178.stderr: 80750K .......... .......... .......... .......... .......... 86% 7.44M 1s 2024-06-28T15:27:07.444 INFO:teuthology.orchestra.run.smithi178.stderr: 80800K .......... .......... .......... .......... .......... 87% 8.56M 1s 2024-06-28T15:27:07.450 INFO:teuthology.orchestra.run.smithi178.stderr: 80850K .......... .......... .......... .......... .......... 87% 8.77M 1s 2024-06-28T15:27:07.456 INFO:teuthology.orchestra.run.smithi178.stderr: 80900K .......... .......... .......... .......... .......... 87% 7.54M 1s 2024-06-28T15:27:07.462 INFO:teuthology.orchestra.run.smithi178.stderr: 80950K .......... .......... .......... .......... .......... 87% 7.88M 1s 2024-06-28T15:27:07.468 INFO:teuthology.orchestra.run.smithi178.stderr: 81000K .......... .......... .......... .......... .......... 87% 9.41M 1s 2024-06-28T15:27:07.475 INFO:teuthology.orchestra.run.smithi178.stderr: 81050K .......... .......... .......... .......... .......... 87% 7.80M 1s 2024-06-28T15:27:07.481 INFO:teuthology.orchestra.run.smithi178.stderr: 81100K .......... .......... .......... .......... .......... 87% 7.08M 1s 2024-06-28T15:27:07.489 INFO:teuthology.orchestra.run.smithi178.stderr: 81150K .......... .......... .......... .......... .......... 87% 6.31M 1s 2024-06-28T15:27:07.494 INFO:teuthology.orchestra.run.smithi178.stderr: 81200K .......... .......... .......... .......... .......... 87% 9.57M 1s 2024-06-28T15:27:07.500 INFO:teuthology.orchestra.run.smithi178.stderr: 81250K .......... .......... .......... .......... .......... 87% 8.42M 1s 2024-06-28T15:27:07.506 INFO:teuthology.orchestra.run.smithi178.stderr: 81300K .......... .......... .......... .......... .......... 87% 8.00M 1s 2024-06-28T15:27:07.515 INFO:teuthology.orchestra.run.smithi178.stderr: 81350K .......... .......... .......... .......... .......... 87% 5.31M 1s 2024-06-28T15:27:07.521 INFO:teuthology.orchestra.run.smithi178.stderr: 81400K .......... .......... .......... .......... .......... 87% 8.22M 1s 2024-06-28T15:27:07.525 INFO:teuthology.orchestra.run.smithi178.stderr: 81450K .......... .......... .......... .......... .......... 87% 16.5M 1s 2024-06-28T15:27:07.530 INFO:teuthology.orchestra.run.smithi178.stderr: 81500K .......... .......... .......... .......... .......... 87% 7.29M 1s 2024-06-28T15:27:07.538 INFO:teuthology.orchestra.run.smithi178.stderr: 81550K .......... .......... .......... .......... .......... 87% 6.07M 1s 2024-06-28T15:27:07.542 INFO:teuthology.orchestra.run.smithi178.stderr: 81600K .......... .......... .......... .......... .......... 87% 13.4M 1s 2024-06-28T15:27:07.545 INFO:teuthology.orchestra.run.smithi178.stderr: 81650K .......... .......... .......... .......... .......... 87% 16.2M 1s 2024-06-28T15:27:07.554 INFO:teuthology.orchestra.run.smithi178.stderr: 81700K .......... .......... .......... .......... .......... 87% 7.48M 1s 2024-06-28T15:27:07.558 INFO:teuthology.orchestra.run.smithi178.stderr: 81750K .......... .......... .......... .......... .......... 88% 7.49M 1s 2024-06-28T15:27:07.565 INFO:teuthology.orchestra.run.smithi178.stderr: 81800K .......... .......... .......... .......... .......... 88% 6.74M 1s 2024-06-28T15:27:07.572 INFO:teuthology.orchestra.run.smithi178.stderr: 81850K .......... .......... .......... .......... .......... 88% 9.21M 1s 2024-06-28T15:27:07.576 INFO:teuthology.orchestra.run.smithi178.stderr: 81900K .......... .......... .......... .......... .......... 88% 9.40M 1s 2024-06-28T15:27:07.584 INFO:teuthology.orchestra.run.smithi178.stderr: 81950K .......... .......... .......... .......... .......... 88% 6.44M 1s 2024-06-28T15:27:07.597 INFO:teuthology.orchestra.run.smithi178.stderr: 82000K .......... .......... .......... .......... .......... 88% 3.59M 1s 2024-06-28T15:27:07.602 INFO:teuthology.orchestra.run.smithi178.stderr: 82050K .......... .......... .......... .......... .......... 88% 10.6M 1s 2024-06-28T15:27:07.606 INFO:teuthology.orchestra.run.smithi178.stderr: 82100K .......... .......... .......... .......... .......... 88% 11.7M 1s 2024-06-28T15:27:07.610 INFO:teuthology.orchestra.run.smithi178.stderr: 82150K .......... .......... .......... .......... .......... 88% 10.8M 1s 2024-06-28T15:27:07.620 INFO:teuthology.orchestra.run.smithi178.stderr: 82200K .......... .......... .......... .......... .......... 88% 5.02M 1s 2024-06-28T15:27:07.628 INFO:teuthology.orchestra.run.smithi178.stderr: 82250K .......... .......... .......... .......... .......... 88% 7.53M 1s 2024-06-28T15:27:07.634 INFO:teuthology.orchestra.run.smithi178.stderr: 82300K .......... .......... .......... .......... .......... 88% 6.84M 1s 2024-06-28T15:27:07.641 INFO:teuthology.orchestra.run.smithi178.stderr: 82350K .......... .......... .......... .......... .......... 88% 6.58M 1s 2024-06-28T15:27:07.648 INFO:teuthology.orchestra.run.smithi178.stderr: 82400K .......... .......... .......... .......... .......... 88% 7.20M 1s 2024-06-28T15:27:07.655 INFO:teuthology.orchestra.run.smithi178.stderr: 82450K .......... .......... .......... .......... .......... 88% 7.60M 1s 2024-06-28T15:27:07.661 INFO:teuthology.orchestra.run.smithi178.stderr: 82500K .......... .......... .......... .......... .......... 88% 8.33M 1s 2024-06-28T15:27:07.671 INFO:teuthology.orchestra.run.smithi178.stderr: 82550K .......... .......... .......... .......... .......... 88% 4.51M 1s 2024-06-28T15:27:07.678 INFO:teuthology.orchestra.run.smithi178.stderr: 82600K .......... .......... .......... .......... .......... 88% 7.21M 1s 2024-06-28T15:27:07.687 INFO:teuthology.orchestra.run.smithi178.stderr: 82650K .......... .......... .......... .......... .......... 89% 7.27M 1s 2024-06-28T15:27:07.696 INFO:teuthology.orchestra.run.smithi178.stderr: 82700K .......... .......... .......... .......... .......... 89% 4.20M 1s 2024-06-28T15:27:07.702 INFO:teuthology.orchestra.run.smithi178.stderr: 82750K .......... .......... .......... .......... .......... 89% 8.79M 1s 2024-06-28T15:27:07.708 INFO:teuthology.orchestra.run.smithi178.stderr: 82800K .......... .......... .......... .......... .......... 89% 7.62M 1s 2024-06-28T15:27:07.719 INFO:teuthology.orchestra.run.smithi178.stderr: 82850K .......... .......... .......... .......... .......... 89% 4.75M 1s 2024-06-28T15:27:07.726 INFO:teuthology.orchestra.run.smithi178.stderr: 82900K .......... .......... .......... .......... .......... 89% 6.15M 1s 2024-06-28T15:27:07.732 INFO:teuthology.orchestra.run.smithi178.stderr: 82950K .......... .......... .......... .......... .......... 89% 9.37M 1s 2024-06-28T15:27:07.740 INFO:teuthology.orchestra.run.smithi178.stderr: 83000K .......... .......... .......... .......... .......... 89% 5.91M 1s 2024-06-28T15:27:07.753 INFO:teuthology.orchestra.run.smithi178.stderr: 83050K .......... .......... .......... .......... .......... 89% 5.92M 1s 2024-06-28T15:27:07.758 INFO:teuthology.orchestra.run.smithi178.stderr: 83100K .......... .......... .......... .......... .......... 89% 4.96M 1s 2024-06-28T15:27:07.764 INFO:teuthology.orchestra.run.smithi178.stderr: 83150K .......... .......... .......... .......... .......... 89% 8.55M 1s 2024-06-28T15:27:07.769 INFO:teuthology.orchestra.run.smithi178.stderr: 83200K .......... .......... .......... .......... .......... 89% 8.65M 1s 2024-06-28T15:27:07.776 INFO:teuthology.orchestra.run.smithi178.stderr: 83250K .......... .......... .......... .......... .......... 89% 7.33M 1s 2024-06-28T15:27:07.783 INFO:teuthology.orchestra.run.smithi178.stderr: 83300K .......... .......... .......... .......... .......... 89% 6.94M 1s 2024-06-28T15:27:07.792 INFO:teuthology.orchestra.run.smithi178.stderr: 83350K .......... .......... .......... .......... .......... 89% 5.27M 1s 2024-06-28T15:27:07.803 INFO:teuthology.orchestra.run.smithi178.stderr: 83400K .......... .......... .......... .......... .......... 89% 4.65M 1s 2024-06-28T15:27:07.808 INFO:teuthology.orchestra.run.smithi178.stderr: 83450K .......... .......... .......... .......... .......... 89% 13.6M 1s 2024-06-28T15:27:07.815 INFO:teuthology.orchestra.run.smithi178.stderr: 83500K .......... .......... .......... .......... .......... 89% 5.94M 1s 2024-06-28T15:27:07.819 INFO:teuthology.orchestra.run.smithi178.stderr: 83550K .......... .......... .......... .......... .......... 89% 10.3M 1s 2024-06-28T15:27:07.831 INFO:teuthology.orchestra.run.smithi178.stderr: 83600K .......... .......... .......... .......... .......... 90% 4.22M 1s 2024-06-28T15:27:07.836 INFO:teuthology.orchestra.run.smithi178.stderr: 83650K .......... .......... .......... .......... .......... 90% 8.80M 1s 2024-06-28T15:27:07.843 INFO:teuthology.orchestra.run.smithi178.stderr: 83700K .......... .......... .......... .......... .......... 90% 7.77M 1s 2024-06-28T15:27:07.847 INFO:teuthology.orchestra.run.smithi178.stderr: 83750K .......... .......... .......... .......... .......... 90% 10.1M 1s 2024-06-28T15:27:07.855 INFO:teuthology.orchestra.run.smithi178.stderr: 83800K .......... .......... .......... .......... .......... 90% 6.54M 1s 2024-06-28T15:27:07.863 INFO:teuthology.orchestra.run.smithi178.stderr: 83850K .......... .......... .......... .......... .......... 90% 7.80M 1s 2024-06-28T15:27:07.873 INFO:teuthology.orchestra.run.smithi178.stderr: 83900K .......... .......... .......... .......... .......... 90% 4.14M 1s 2024-06-28T15:27:07.879 INFO:teuthology.orchestra.run.smithi178.stderr: 83950K .......... .......... .......... .......... .......... 90% 8.07M 1s 2024-06-28T15:27:07.885 INFO:teuthology.orchestra.run.smithi178.stderr: 84000K .......... .......... .......... .......... .......... 90% 7.99M 1s 2024-06-28T15:27:07.890 INFO:teuthology.orchestra.run.smithi178.stderr: 84050K .......... .......... .......... .......... .......... 90% 10.4M 1s 2024-06-28T15:27:07.898 INFO:teuthology.orchestra.run.smithi178.stderr: 84100K .......... .......... .......... .......... .......... 90% 5.94M 1s 2024-06-28T15:27:07.916 INFO:teuthology.orchestra.run.smithi178.stderr: 84150K .......... .......... .......... .......... .......... 90% 2.81M 1s 2024-06-28T15:27:07.925 INFO:teuthology.orchestra.run.smithi178.stderr: 84200K .......... .......... .......... .......... .......... 90% 4.95M 1s 2024-06-28T15:27:07.942 INFO:teuthology.orchestra.run.smithi178.stderr: 84250K .......... .......... .......... .......... .......... 90% 5.25M 1s 2024-06-28T15:27:07.950 INFO:teuthology.orchestra.run.smithi178.stderr: 84300K .......... .......... .......... .......... .......... 90% 3.21M 1s 2024-06-28T15:27:07.958 INFO:teuthology.orchestra.run.smithi178.stderr: 84350K .......... .......... .......... .......... .......... 90% 6.44M 1s 2024-06-28T15:27:07.967 INFO:teuthology.orchestra.run.smithi178.stderr: 84400K .......... .......... .......... .......... .......... 90% 6.38M 1s 2024-06-28T15:27:07.972 INFO:teuthology.orchestra.run.smithi178.stderr: 84450K .......... .......... .......... .......... .......... 90% 6.97M 1s 2024-06-28T15:27:07.981 INFO:teuthology.orchestra.run.smithi178.stderr: 84500K .......... .......... .......... .......... .......... 91% 5.58M 1s 2024-06-28T15:27:07.993 INFO:teuthology.orchestra.run.smithi178.stderr: 84550K .......... .......... .......... .......... .......... 91% 4.20M 1s 2024-06-28T15:27:08.010 INFO:teuthology.orchestra.run.smithi178.stderr: 84600K .......... .......... .......... .......... .......... 91% 2.79M 1s 2024-06-28T15:27:08.028 INFO:teuthology.orchestra.run.smithi178.stderr: 84650K .......... .......... .......... .......... .......... 91% 3.22M 1s 2024-06-28T15:27:08.034 INFO:teuthology.orchestra.run.smithi178.stderr: 84700K .......... .......... .......... .......... .......... 91% 5.24M 1s 2024-06-28T15:27:08.041 INFO:teuthology.orchestra.run.smithi178.stderr: 84750K .......... .......... .......... .......... .......... 91% 7.78M 1s 2024-06-28T15:27:08.057 INFO:teuthology.orchestra.run.smithi178.stderr: 84800K .......... .......... .......... .......... .......... 91% 2.97M 1s 2024-06-28T15:27:08.068 INFO:teuthology.orchestra.run.smithi178.stderr: 84850K .......... .......... .......... .......... .......... 91% 4.42M 1s 2024-06-28T15:27:08.080 INFO:teuthology.orchestra.run.smithi178.stderr: 84900K .......... .......... .......... .......... .......... 91% 4.28M 1s 2024-06-28T15:27:08.085 INFO:teuthology.orchestra.run.smithi178.stderr: 84950K .......... .......... .......... .......... .......... 91% 9.08M 1s 2024-06-28T15:27:08.098 INFO:teuthology.orchestra.run.smithi178.stderr: 85000K .......... .......... .......... .......... .......... 91% 3.63M 1s 2024-06-28T15:27:08.119 INFO:teuthology.orchestra.run.smithi178.stderr: 85050K .......... .......... .......... .......... .......... 91% 2.93M 1s 2024-06-28T15:27:08.133 INFO:teuthology.orchestra.run.smithi178.stderr: 85100K .......... .......... .......... .......... .......... 91% 2.80M 1s 2024-06-28T15:27:08.138 INFO:teuthology.orchestra.run.smithi178.stderr: 85150K .......... .......... .......... .......... .......... 91% 9.75M 1s 2024-06-28T15:27:08.147 INFO:teuthology.orchestra.run.smithi178.stderr: 85200K .......... .......... .......... .......... .......... 91% 5.48M 1s 2024-06-28T15:27:08.162 INFO:teuthology.orchestra.run.smithi178.stderr: 85250K .......... .......... .......... .......... .......... 91% 3.06M 1s 2024-06-28T15:27:08.174 INFO:teuthology.orchestra.run.smithi178.stderr: 85300K .......... .......... .......... .......... .......... 91% 4.28M 1s 2024-06-28T15:27:08.182 INFO:teuthology.orchestra.run.smithi178.stderr: 85350K .......... .......... .......... .......... .......... 91% 5.80M 1s 2024-06-28T15:27:08.197 INFO:teuthology.orchestra.run.smithi178.stderr: 85400K .......... .......... .......... .......... .......... 91% 3.42M 1s 2024-06-28T15:27:08.207 INFO:teuthology.orchestra.run.smithi178.stderr: 85450K .......... .......... .......... .......... .......... 92% 6.85M 1s 2024-06-28T15:27:08.223 INFO:teuthology.orchestra.run.smithi178.stderr: 85500K .......... .......... .......... .......... .......... 92% 2.56M 1s 2024-06-28T15:27:08.236 INFO:teuthology.orchestra.run.smithi178.stderr: 85550K .......... .......... .......... .......... .......... 92% 3.73M 1s 2024-06-28T15:27:08.244 INFO:teuthology.orchestra.run.smithi178.stderr: 85600K .......... .......... .......... .......... .......... 92% 6.30M 1s 2024-06-28T15:27:08.250 INFO:teuthology.orchestra.run.smithi178.stderr: 85650K .......... .......... .......... .......... .......... 92% 7.97M 1s 2024-06-28T15:27:08.264 INFO:teuthology.orchestra.run.smithi178.stderr: 85700K .......... .......... .......... .......... .......... 92% 3.46M 1s 2024-06-28T15:27:08.275 INFO:teuthology.orchestra.run.smithi178.stderr: 85750K .......... .......... .......... .......... .......... 92% 4.36M 1s 2024-06-28T15:27:08.285 INFO:teuthology.orchestra.run.smithi178.stderr: 85800K .......... .......... .......... .......... .......... 92% 5.15M 1s 2024-06-28T15:27:08.294 INFO:teuthology.orchestra.run.smithi178.stderr: 85850K .......... .......... .......... .......... .......... 92% 5.87M 1s 2024-06-28T15:27:08.299 INFO:teuthology.orchestra.run.smithi178.stderr: 85900K .......... .......... .......... .......... .......... 92% 8.01M 1s 2024-06-28T15:27:08.317 INFO:teuthology.orchestra.run.smithi178.stderr: 85950K .......... .......... .......... .......... .......... 92% 2.71M 1s 2024-06-28T15:27:08.328 INFO:teuthology.orchestra.run.smithi178.stderr: 86000K .......... .......... .......... .......... .......... 92% 4.37M 1s 2024-06-28T15:27:08.337 INFO:teuthology.orchestra.run.smithi178.stderr: 86050K .......... .......... .......... .......... .......... 92% 5.69M 1s 2024-06-28T15:27:08.344 INFO:teuthology.orchestra.run.smithi178.stderr: 86100K .......... .......... .......... .......... .......... 92% 6.41M 1s 2024-06-28T15:27:08.348 INFO:teuthology.orchestra.run.smithi178.stderr: 86150K .......... .......... .......... .......... .......... 92% 12.2M 1s 2024-06-28T15:27:08.368 INFO:teuthology.orchestra.run.smithi178.stderr: 86200K .......... .......... .......... .......... .......... 92% 2.46M 1s 2024-06-28T15:27:08.380 INFO:teuthology.orchestra.run.smithi178.stderr: 86250K .......... .......... .......... .......... .......... 92% 5.42M 1s 2024-06-28T15:27:08.390 INFO:teuthology.orchestra.run.smithi178.stderr: 86300K .......... .......... .......... .......... .......... 92% 3.91M 1s 2024-06-28T15:27:08.394 INFO:teuthology.orchestra.run.smithi178.stderr: 86350K .......... .......... .......... .......... .......... 93% 11.3M 1s 2024-06-28T15:27:08.411 INFO:teuthology.orchestra.run.smithi178.stderr: 86400K .......... .......... .......... .......... .......... 93% 2.94M 0s 2024-06-28T15:27:08.422 INFO:teuthology.orchestra.run.smithi178.stderr: 86450K .......... .......... .......... .......... .......... 93% 4.23M 0s 2024-06-28T15:27:08.431 INFO:teuthology.orchestra.run.smithi178.stderr: 86500K .......... .......... .......... .......... .......... 93% 5.50M 0s 2024-06-28T15:27:08.439 INFO:teuthology.orchestra.run.smithi178.stderr: 86550K .......... .......... .......... .......... .......... 93% 6.24M 0s 2024-06-28T15:27:08.445 INFO:teuthology.orchestra.run.smithi178.stderr: 86600K .......... .......... .......... .......... .......... 93% 7.16M 0s 2024-06-28T15:27:08.469 INFO:teuthology.orchestra.run.smithi178.stderr: 86650K .......... .......... .......... .......... .......... 93% 2.30M 0s 2024-06-28T15:27:08.475 INFO:teuthology.orchestra.run.smithi178.stderr: 86700K .......... .......... .......... .......... .......... 93% 5.69M 0s 2024-06-28T15:27:08.484 INFO:teuthology.orchestra.run.smithi178.stderr: 86750K .......... .......... .......... .......... .......... 93% 5.47M 0s 2024-06-28T15:27:08.490 INFO:teuthology.orchestra.run.smithi178.stderr: 86800K .......... .......... .......... .......... .......... 93% 8.49M 0s 2024-06-28T15:27:08.495 INFO:teuthology.orchestra.run.smithi178.stderr: 86850K .......... .......... .......... .......... .......... 93% 9.14M 0s 2024-06-28T15:27:08.519 INFO:teuthology.orchestra.run.smithi178.stderr: 86900K .......... .......... .......... .......... .......... 93% 2.03M 0s 2024-06-28T15:27:08.529 INFO:teuthology.orchestra.run.smithi178.stderr: 86950K .......... .......... .......... .......... .......... 93% 5.24M 0s 2024-06-28T15:27:08.536 INFO:teuthology.orchestra.run.smithi178.stderr: 87000K .......... .......... .......... .......... .......... 93% 6.48M 0s 2024-06-28T15:27:08.545 INFO:teuthology.orchestra.run.smithi178.stderr: 87050K .......... .......... .......... .......... .......... 93% 9.72M 0s 2024-06-28T15:27:08.556 INFO:teuthology.orchestra.run.smithi178.stderr: 87100K .......... .......... .......... .......... .......... 93% 3.24M 0s 2024-06-28T15:27:08.575 INFO:teuthology.orchestra.run.smithi178.stderr: 87150K .......... .......... .......... .......... .......... 93% 2.69M 0s 2024-06-28T15:27:08.582 INFO:teuthology.orchestra.run.smithi178.stderr: 87200K .......... .......... .......... .......... .......... 93% 7.16M 0s 2024-06-28T15:27:08.588 INFO:teuthology.orchestra.run.smithi178.stderr: 87250K .......... .......... .......... .......... .......... 93% 7.43M 0s 2024-06-28T15:27:08.593 INFO:teuthology.orchestra.run.smithi178.stderr: 87300K .......... .......... .......... .......... .......... 94% 9.06M 0s 2024-06-28T15:27:08.613 INFO:teuthology.orchestra.run.smithi178.stderr: 87350K .......... .......... .......... .......... .......... 94% 2.52M 0s 2024-06-28T15:27:08.624 INFO:teuthology.orchestra.run.smithi178.stderr: 87400K .......... .......... .......... .......... .......... 94% 4.22M 0s 2024-06-28T15:27:08.634 INFO:teuthology.orchestra.run.smithi178.stderr: 87450K .......... .......... .......... .......... .......... 94% 6.73M 0s 2024-06-28T15:27:08.640 INFO:teuthology.orchestra.run.smithi178.stderr: 87500K .......... .......... .......... .......... .......... 94% 5.52M 0s 2024-06-28T15:27:08.648 INFO:teuthology.orchestra.run.smithi178.stderr: 87550K .......... .......... .......... .......... .......... 94% 6.36M 0s 2024-06-28T15:27:08.671 INFO:teuthology.orchestra.run.smithi178.stderr: 87600K .......... .......... .......... .......... .......... 94% 2.13M 0s 2024-06-28T15:27:08.679 INFO:teuthology.orchestra.run.smithi178.stderr: 87650K .......... .......... .......... .......... .......... 94% 5.94M 0s 2024-06-28T15:27:08.685 INFO:teuthology.orchestra.run.smithi178.stderr: 87700K .......... .......... .......... .......... .......... 94% 7.84M 0s 2024-06-28T15:27:08.691 INFO:teuthology.orchestra.run.smithi178.stderr: 87750K .......... .......... .......... .......... .......... 94% 8.00M 0s 2024-06-28T15:27:08.704 INFO:teuthology.orchestra.run.smithi178.stderr: 87800K .......... .......... .......... .......... .......... 94% 3.90M 0s 2024-06-28T15:27:08.727 INFO:teuthology.orchestra.run.smithi178.stderr: 87850K .......... .......... .......... .......... .......... 94% 2.54M 0s 2024-06-28T15:27:08.733 INFO:teuthology.orchestra.run.smithi178.stderr: 87900K .......... .......... .......... .......... .......... 94% 4.88M 0s 2024-06-28T15:27:08.741 INFO:teuthology.orchestra.run.smithi178.stderr: 87950K .......... .......... .......... .......... .......... 94% 6.32M 0s 2024-06-28T15:27:08.745 INFO:teuthology.orchestra.run.smithi178.stderr: 88000K .......... .......... .......... .......... .......... 94% 12.8M 0s 2024-06-28T15:27:08.769 INFO:teuthology.orchestra.run.smithi178.stderr: 88050K .......... .......... .......... .......... .......... 94% 1.99M 0s 2024-06-28T15:27:08.780 INFO:teuthology.orchestra.run.smithi178.stderr: 88100K .......... .......... .......... .......... .......... 94% 4.50M 0s 2024-06-28T15:27:08.790 INFO:teuthology.orchestra.run.smithi178.stderr: 88150K .......... .......... .......... .......... .......... 94% 4.74M 0s 2024-06-28T15:27:08.795 INFO:teuthology.orchestra.run.smithi178.stderr: 88200K .......... .......... .......... .......... .......... 94% 9.97M 0s 2024-06-28T15:27:08.808 INFO:teuthology.orchestra.run.smithi178.stderr: 88250K .......... .......... .......... .......... .......... 95% 5.40M 0s 2024-06-28T15:27:08.833 INFO:teuthology.orchestra.run.smithi178.stderr: 88300K .......... .......... .......... .......... .......... 95% 1.75M 0s 2024-06-28T15:27:08.841 INFO:teuthology.orchestra.run.smithi178.stderr: 88350K .......... .......... .......... .......... .......... 95% 5.71M 0s 2024-06-28T15:27:08.846 INFO:teuthology.orchestra.run.smithi178.stderr: 88400K .......... .......... .......... .......... .......... 95% 9.26M 0s 2024-06-28T15:27:08.852 INFO:teuthology.orchestra.run.smithi178.stderr: 88450K .......... .......... .......... .......... .......... 95% 9.20M 0s 2024-06-28T15:27:08.866 INFO:teuthology.orchestra.run.smithi178.stderr: 88500K .......... .......... .......... .......... .......... 95% 3.49M 0s 2024-06-28T15:27:08.891 INFO:teuthology.orchestra.run.smithi178.stderr: 88550K .......... .......... .......... .......... .......... 95% 1.94M 0s 2024-06-28T15:27:08.900 INFO:teuthology.orchestra.run.smithi178.stderr: 88600K .......... .......... .......... .......... .......... 95% 4.99M 0s 2024-06-28T15:27:08.961 INFO:teuthology.orchestra.run.smithi178.stderr: 88650K .......... .......... .......... .......... .......... 95% 838K 0s 2024-06-28T15:27:08.975 INFO:teuthology.orchestra.run.smithi178.stderr: 88700K .......... .......... .......... .......... .......... 95% 3.71M 0s 2024-06-28T15:27:08.975 INFO:teuthology.orchestra.run.smithi178.stderr: 88750K .......... .......... .......... .......... .......... 95% 147M 0s 2024-06-28T15:27:09.007 INFO:teuthology.orchestra.run.smithi178.stderr: 88800K .......... .......... .......... .......... .......... 95% 1.59M 0s 2024-06-28T15:27:09.007 INFO:teuthology.orchestra.run.smithi178.stderr: 88850K .......... .......... .......... .......... .......... 95% 133M 0s 2024-06-28T15:27:09.007 INFO:teuthology.orchestra.run.smithi178.stderr: 88900K .......... .......... .......... .......... .......... 95% 135M 0s 2024-06-28T15:27:09.007 INFO:teuthology.orchestra.run.smithi178.stderr: 88950K .......... .......... .......... .......... .......... 95% 155M 0s 2024-06-28T15:27:09.007 INFO:teuthology.orchestra.run.smithi178.stderr: 89000K .......... .......... .......... .......... .......... 95% 104M 0s 2024-06-28T15:27:09.019 INFO:teuthology.orchestra.run.smithi178.stderr: 89050K .......... .......... .......... .......... .......... 95% 4.04M 0s 2024-06-28T15:27:09.038 INFO:teuthology.orchestra.run.smithi178.stderr: 89100K .......... .......... .......... .......... .......... 95% 2.47M 0s 2024-06-28T15:27:09.051 INFO:teuthology.orchestra.run.smithi178.stderr: 89150K .......... .......... .......... .......... .......... 96% 3.83M 0s 2024-06-28T15:27:09.056 INFO:teuthology.orchestra.run.smithi178.stderr: 89200K .......... .......... .......... .......... .......... 96% 8.67M 0s 2024-06-28T15:27:09.068 INFO:teuthology.orchestra.run.smithi178.stderr: 89250K .......... .......... .......... .......... .......... 96% 4.32M 0s 2024-06-28T15:27:09.081 INFO:teuthology.orchestra.run.smithi178.stderr: 89300K .......... .......... .......... .......... .......... 96% 3.73M 0s 2024-06-28T15:27:09.090 INFO:teuthology.orchestra.run.smithi178.stderr: 89350K .......... .......... .......... .......... .......... 96% 5.22M 0s 2024-06-28T15:27:09.100 INFO:teuthology.orchestra.run.smithi178.stderr: 89400K .......... .......... .......... .......... .......... 96% 4.74M 0s 2024-06-28T15:27:09.113 INFO:teuthology.orchestra.run.smithi178.stderr: 89450K .......... .......... .......... .......... .......... 96% 5.34M 0s 2024-06-28T15:27:09.120 INFO:teuthology.orchestra.run.smithi178.stderr: 89500K .......... .......... .......... .......... .......... 96% 4.65M 0s 2024-06-28T15:27:09.145 INFO:teuthology.orchestra.run.smithi178.stderr: 89550K .......... .......... .......... .......... .......... 96% 1.98M 0s 2024-06-28T15:27:09.155 INFO:teuthology.orchestra.run.smithi178.stderr: 89600K .......... .......... .......... .......... .......... 96% 4.89M 0s 2024-06-28T15:27:09.162 INFO:teuthology.orchestra.run.smithi178.stderr: 89650K .......... .......... .......... .......... .......... 96% 6.36M 0s 2024-06-28T15:27:09.177 INFO:teuthology.orchestra.run.smithi178.stderr: 89700K .......... .......... .......... .......... .......... 96% 3.54M 0s 2024-06-28T15:27:09.184 INFO:teuthology.orchestra.run.smithi178.stderr: 89750K .......... .......... .......... .......... .......... 96% 5.95M 0s 2024-06-28T15:27:09.194 INFO:teuthology.orchestra.run.smithi178.stderr: 89800K .......... .......... .......... .......... .......... 96% 4.79M 0s 2024-06-28T15:27:09.217 INFO:teuthology.orchestra.run.smithi178.stderr: 89850K .......... .......... .......... .......... .......... 96% 2.70M 0s 2024-06-28T15:27:09.233 INFO:teuthology.orchestra.run.smithi178.stderr: 89900K .......... .......... .......... .......... .......... 96% 2.38M 0s 2024-06-28T15:27:09.245 INFO:teuthology.orchestra.run.smithi178.stderr: 89950K .......... .......... .......... .......... .......... 96% 4.13M 0s 2024-06-28T15:27:09.257 INFO:teuthology.orchestra.run.smithi178.stderr: 90000K .......... .......... .......... .......... .......... 96% 4.04M 0s 2024-06-28T15:27:09.267 INFO:teuthology.orchestra.run.smithi178.stderr: 90050K .......... .......... .......... .......... .......... 96% 5.03M 0s 2024-06-28T15:27:09.274 INFO:teuthology.orchestra.run.smithi178.stderr: 90100K .......... .......... .......... .......... .......... 97% 6.19M 0s 2024-06-28T15:27:09.288 INFO:teuthology.orchestra.run.smithi178.stderr: 90150K .......... .......... .......... .......... .......... 97% 3.73M 0s 2024-06-28T15:27:09.300 INFO:teuthology.orchestra.run.smithi178.stderr: 90200K .......... .......... .......... .......... .......... 97% 3.85M 0s 2024-06-28T15:27:09.317 INFO:teuthology.orchestra.run.smithi178.stderr: 90250K .......... .......... .......... .......... .......... 97% 3.62M 0s 2024-06-28T15:27:09.329 INFO:teuthology.orchestra.run.smithi178.stderr: 90300K .......... .......... .......... .......... .......... 97% 3.39M 0s 2024-06-28T15:27:09.348 INFO:teuthology.orchestra.run.smithi178.stderr: 90350K .......... .......... .......... .......... .......... 97% 2.49M 0s 2024-06-28T15:27:09.357 INFO:teuthology.orchestra.run.smithi178.stderr: 90400K .......... .......... .......... .......... .......... 97% 5.21M 0s 2024-06-28T15:27:09.364 INFO:teuthology.orchestra.run.smithi178.stderr: 90450K .......... .......... .......... .......... .......... 97% 6.74M 0s 2024-06-28T15:27:09.376 INFO:teuthology.orchestra.run.smithi178.stderr: 90500K .......... .......... .......... .......... .......... 97% 4.07M 0s 2024-06-28T15:27:09.391 INFO:teuthology.orchestra.run.smithi178.stderr: 90550K .......... .......... .......... .......... .......... 97% 3.38M 0s 2024-06-28T15:27:09.404 INFO:teuthology.orchestra.run.smithi178.stderr: 90600K .......... .......... .......... .......... .......... 97% 3.62M 0s 2024-06-28T15:27:09.421 INFO:teuthology.orchestra.run.smithi178.stderr: 90650K .......... .......... .......... .......... .......... 97% 3.79M 0s 2024-06-28T15:27:09.431 INFO:teuthology.orchestra.run.smithi178.stderr: 90700K .......... .......... .......... .......... .......... 97% 3.64M 0s 2024-06-28T15:27:09.443 INFO:teuthology.orchestra.run.smithi178.stderr: 90750K .......... .......... .......... .......... .......... 97% 3.81M 0s 2024-06-28T15:27:09.461 INFO:teuthology.orchestra.run.smithi178.stderr: 90800K .......... .......... .......... .......... .......... 97% 2.83M 0s 2024-06-28T15:27:09.468 INFO:teuthology.orchestra.run.smithi178.stderr: 90850K .......... .......... .......... .......... .......... 97% 7.19M 0s 2024-06-28T15:27:09.481 INFO:teuthology.orchestra.run.smithi178.stderr: 90900K .......... .......... .......... .......... .......... 97% 3.68M 0s 2024-06-28T15:27:09.490 INFO:teuthology.orchestra.run.smithi178.stderr: 90950K .......... .......... .......... .......... .......... 97% 5.36M 0s 2024-06-28T15:27:09.510 INFO:teuthology.orchestra.run.smithi178.stderr: 91000K .......... .......... .......... .......... .......... 98% 2.46M 0s 2024-06-28T15:27:09.526 INFO:teuthology.orchestra.run.smithi178.stderr: 91050K .......... .......... .......... .......... .......... 98% 3.75M 0s 2024-06-28T15:27:09.534 INFO:teuthology.orchestra.run.smithi178.stderr: 91100K .......... .......... .......... .......... .......... 98% 4.33M 0s 2024-06-28T15:27:09.556 INFO:teuthology.orchestra.run.smithi178.stderr: 91150K .......... .......... .......... .......... .......... 98% 2.24M 0s 2024-06-28T15:27:09.565 INFO:teuthology.orchestra.run.smithi178.stderr: 91200K .......... .......... .......... .......... .......... 98% 5.68M 0s 2024-06-28T15:27:09.576 INFO:teuthology.orchestra.run.smithi178.stderr: 91250K .......... .......... .......... .......... .......... 98% 4.36M 0s 2024-06-28T15:27:09.583 INFO:teuthology.orchestra.run.smithi178.stderr: 91300K .......... .......... .......... .......... .......... 98% 6.24M 0s 2024-06-28T15:27:09.604 INFO:teuthology.orchestra.run.smithi178.stderr: 91350K .......... .......... .......... .......... .......... 98% 2.41M 0s 2024-06-28T15:27:09.615 INFO:teuthology.orchestra.run.smithi178.stderr: 91400K .......... .......... .......... .......... .......... 98% 4.43M 0s 2024-06-28T15:27:09.626 INFO:teuthology.orchestra.run.smithi178.stderr: 91450K .......... .......... .......... .......... .......... 98% 4.75M 0s 2024-06-28T15:27:09.633 INFO:teuthology.orchestra.run.smithi178.stderr: 91500K .......... .......... .......... .......... .......... 98% 5.96M 0s 2024-06-28T15:27:09.657 INFO:teuthology.orchestra.run.smithi178.stderr: 91550K .......... .......... .......... .......... .......... 98% 2.04M 0s 2024-06-28T15:27:09.665 INFO:teuthology.orchestra.run.smithi178.stderr: 91600K .......... .......... .......... .......... .......... 98% 5.98M 0s 2024-06-28T15:27:09.675 INFO:teuthology.orchestra.run.smithi178.stderr: 91650K .......... .......... .......... .......... .......... 98% 5.19M 0s 2024-06-28T15:27:09.680 INFO:teuthology.orchestra.run.smithi178.stderr: 91700K .......... .......... .......... .......... .......... 98% 8.53M 0s 2024-06-28T15:27:09.706 INFO:teuthology.orchestra.run.smithi178.stderr: 91750K .......... .......... .......... .......... .......... 98% 1.93M 0s 2024-06-28T15:27:09.716 INFO:teuthology.orchestra.run.smithi178.stderr: 91800K .......... .......... .......... .......... .......... 98% 4.79M 0s 2024-06-28T15:27:09.726 INFO:teuthology.orchestra.run.smithi178.stderr: 91850K .......... .......... .......... .......... .......... 98% 6.36M 0s 2024-06-28T15:27:09.737 INFO:teuthology.orchestra.run.smithi178.stderr: 91900K .......... .......... .......... .......... .......... 98% 3.59M 0s 2024-06-28T15:27:09.756 INFO:teuthology.orchestra.run.smithi178.stderr: 91950K .......... .......... .......... .......... .......... 99% 2.69M 0s 2024-06-28T15:27:09.764 INFO:teuthology.orchestra.run.smithi178.stderr: 92000K .......... .......... .......... .......... .......... 99% 5.53M 0s 2024-06-28T15:27:09.774 INFO:teuthology.orchestra.run.smithi178.stderr: 92050K .......... .......... .......... .......... .......... 99% 5.25M 0s 2024-06-28T15:27:09.779 INFO:teuthology.orchestra.run.smithi178.stderr: 92100K .......... .......... .......... .......... .......... 99% 8.56M 0s 2024-06-28T15:27:09.800 INFO:teuthology.orchestra.run.smithi178.stderr: 92150K .......... .......... .......... .......... .......... 99% 2.30M 0s 2024-06-28T15:27:09.811 INFO:teuthology.orchestra.run.smithi178.stderr: 92200K .......... .......... .......... .......... .......... 99% 5.68M 0s 2024-06-28T15:27:09.820 INFO:teuthology.orchestra.run.smithi178.stderr: 92250K .......... .......... .......... .......... .......... 99% 5.19M 0s 2024-06-28T15:27:09.836 INFO:teuthology.orchestra.run.smithi178.stderr: 92300K .......... .......... .......... .......... .......... 99% 2.74M 0s 2024-06-28T15:27:09.845 INFO:teuthology.orchestra.run.smithi178.stderr: 92350K .......... .......... .......... .......... .......... 99% 5.85M 0s 2024-06-28T15:27:09.856 INFO:teuthology.orchestra.run.smithi178.stderr: 92400K .......... .......... .......... .......... .......... 99% 4.29M 0s 2024-06-28T15:27:09.861 INFO:teuthology.orchestra.run.smithi178.stderr: 92450K .......... .......... .......... .......... .......... 99% 9.39M 0s 2024-06-28T15:27:09.866 INFO:teuthology.orchestra.run.smithi178.stderr: 92500K .......... .......... .......... .......... .......... 99% 10.4M 0s 2024-06-28T15:27:09.879 INFO:teuthology.orchestra.run.smithi178.stderr: 92550K .......... .......... .......... .......... .......... 99% 3.66M 0s 2024-06-28T15:27:09.888 INFO:teuthology.orchestra.run.smithi178.stderr: 92600K .......... .......... .......... .......... .......... 99% 5.86M 0s 2024-06-28T15:27:09.897 INFO:teuthology.orchestra.run.smithi178.stderr: 92650K .......... .......... .......... .......... .......... 99% 6.08M 0s 2024-06-28T15:27:09.905 INFO:teuthology.orchestra.run.smithi178.stderr: 92700K .......... .......... .......... .......... .......... 99% 5.01M 0s 2024-06-28T15:27:09.913 INFO:teuthology.orchestra.run.smithi178.stderr: 92750K .......... .......... .......... .......... .......... 99% 6.54M 0s 2024-06-28T15:27:09.921 INFO:teuthology.orchestra.run.smithi178.stderr: 92800K .......... .......... .......... .......... .......... 99% 5.84M 0s 2024-06-28T15:27:09.930 INFO:teuthology.orchestra.run.smithi178.stderr: 92850K .......... .......... .......... .......... .......... 99% 6.01M 0s 2024-06-28T15:27:09.930 INFO:teuthology.orchestra.run.smithi178.stderr: 92900K 100% 134G=8.2s 2024-06-28T15:27:09.930 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-06-28T15:27:09.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28 15:27:09 (11.1 MB/s) - 'gradle-6.0.1-bin.zip' saved [95129672/95129672] 2024-06-28T15:27:09.930 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-06-28T15:27:09.943 INFO:teuthology.orchestra.run.smithi178.stdout:Archive: gradle-6.0.1-bin.zip 2024-06-28T15:27:09.943 INFO:teuthology.orchestra.run.smithi178.stdout: creating: /opt/gradle/gradle-6.0.1/ 2024-06-28T15:27:09.944 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/getting-started.html 2024-06-28T15:27:09.945 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/LICENSE 2024-06-28T15:27:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/NOTICE 2024-06-28T15:27:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: creating: /opt/gradle/gradle-6.0.1/init.d/ 2024-06-28T15:27:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/init.d/readme.txt 2024-06-28T15:27:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: creating: /opt/gradle/gradle-6.0.1/bin/ 2024-06-28T15:27:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/bin/gradle 2024-06-28T15:27:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/bin/gradle.bat 2024-06-28T15:27:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: creating: /opt/gradle/gradle-6.0.1/lib/ 2024-06-28T15:27:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-api-metadata-6.0.1.jar 2024-06-28T15:27:09.949 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-launcher-6.0.1.jar 2024-06-28T15:27:09.950 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-runtime-api-info-6.0.1.jar 2024-06-28T15:27:09.964 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-kotlin-dsl-6.0.1.jar 2024-06-28T15:27:09.966 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-tooling-api-6.0.1.jar 2024-06-28T15:27:09.967 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-wrapper-6.0.1.jar 2024-06-28T15:27:09.967 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-installation-beacon-6.0.1.jar 2024-06-28T15:27:09.967 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-bootstrap-6.0.1.jar 2024-06-28T15:27:09.985 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-core-6.0.1.jar 2024-06-28T15:27:09.986 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-file-collections-6.0.1.jar 2024-06-28T15:27:09.987 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-model-groovy-6.0.1.jar 2024-06-28T15:27:09.988 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-execution-6.0.1.jar 2024-06-28T15:27:09.995 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-model-core-6.0.1.jar 2024-06-28T15:27:09.995 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-cache-packaging-6.0.1.jar 2024-06-28T15:27:09.995 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-cache-6.0.1.jar 2024-06-28T15:27:09.997 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-core-api-6.0.1.jar 2024-06-28T15:27:09.999 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-persistent-cache-6.0.1.jar 2024-06-28T15:27:09.999 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-worker-processes-6.0.1.jar 2024-06-28T15:27:10.002 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-logging-6.0.1.jar 2024-06-28T15:27:10.002 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-jvm-services-6.0.1.jar 2024-06-28T15:27:10.002 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-process-services-6.0.1.jar 2024-06-28T15:27:10.002 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-resources-6.0.1.jar 2024-06-28T15:27:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-messaging-6.0.1.jar 2024-06-28T15:27:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-native-6.0.1.jar 2024-06-28T15:27:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-base-services-groovy-6.0.1.jar 2024-06-28T15:27:10.008 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-base-services-6.0.1.jar 2024-06-28T15:27:10.008 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-option-6.0.1.jar 2024-06-28T15:27:10.008 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-cli-6.0.1.jar 2024-06-28T15:27:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-snapshots-6.0.1.jar 2024-06-28T15:27:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-files-6.0.1.jar 2024-06-28T15:27:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-kotlin-dsl-tooling-models-6.0.1.jar 2024-06-28T15:27:10.254 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-compiler-embeddable-1.3.50-patched-for-gradle-6.0.1.jar 2024-06-28T15:27:10.255 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-hashing-6.0.1.jar 2024-06-28T15:27:10.255 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-pineapple-6.0.1.jar 2024-06-28T15:27:10.255 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-docs-6.0.1.jar 2024-06-28T15:27:10.310 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/groovy-all-1.3-2.5.8.jar 2024-06-28T15:27:10.311 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jul-to-slf4j-1.7.28.jar 2024-06-28T15:27:10.311 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/log4j-over-slf4j-1.7.28.jar 2024-06-28T15:27:10.311 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jcl-over-slf4j-1.7.28.jar 2024-06-28T15:27:10.311 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/slf4j-api-1.7.28.jar 2024-06-28T15:27:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/guava-27.1-android.jar 2024-06-28T15:27:10.329 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-io-2.6.jar 2024-06-28T15:27:10.331 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-lang-2.6.jar 2024-06-28T15:27:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-commons-7.1.jar 2024-06-28T15:27:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-analysis-7.1.jar 2024-06-28T15:27:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-tree-7.1.jar 2024-06-28T15:27:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-7.1.jar 2024-06-28T15:27:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/ant-1.10.7.jar 2024-06-28T15:27:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-jdk8-1.3.50.jar 2024-06-28T15:27:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/javax.inject-1.jar 2024-06-28T15:27:10.349 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-common-1.3.50.jar 2024-06-28T15:27:10.350 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-jvm-1.3.50.jar 2024-06-28T15:27:10.350 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-jvm-host-embeddable-1.3.50.jar 2024-06-28T15:27:10.352 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-compiler-embeddable-1.3.50.jar 2024-06-28T15:27:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-compiler-impl-embeddable-1.3.50.jar 2024-06-28T15:27:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-sam-with-receiver-compiler-plugin-1.3.50.jar 2024-06-28T15:27:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlinx-metadata-jvm-0.1.0.jar 2024-06-28T15:27:10.372 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-compress-1.19.jar 2024-06-28T15:27:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/fastutil-8.3.0-min.jar 2024-06-28T15:27:10.378 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jansi-1.18.jar 2024-06-28T15:27:10.378 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jsr305-3.0.2.jar 2024-06-28T15:27:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kryo-2.24.0.jar 2024-06-28T15:27:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-0.18.jar 2024-06-28T15:27:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/objenesis-2.6.jar 2024-06-28T15:27:10.384 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/trove4j-1.0.20181211.jar 2024-06-28T15:27:10.386 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/xml-apis-1.4.01.jar 2024-06-28T15:27:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-reflect-1.3.50.jar 2024-06-28T15:27:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-jdk7-1.3.50.jar 2024-06-28T15:27:10.412 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-1.3.50.jar 2024-06-28T15:27:10.412 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/annotations-13.0.jar 2024-06-28T15:27:10.412 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/ant-launcher-1.10.7.jar 2024-06-28T15:27:10.412 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/minlog-1.2.jar 2024-06-28T15:27:10.412 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-script-runtime-1.3.50.jar 2024-06-28T15:27:10.420 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-daemon-embeddable-1.3.50.jar 2024-06-28T15:27:10.420 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/failureaccess-1.0.1.jar 2024-06-28T15:27:10.422 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-common-1.3.50.jar 2024-06-28T15:27:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-amd64-libcpp-0.18.jar 2024-06-28T15:27:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-amd64-libstdcpp-0.18.jar 2024-06-28T15:27:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-i386-libcpp-0.18.jar 2024-06-28T15:27:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-i386-libstdcpp-0.18.jar 2024-06-28T15:27:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-aarch64-0.18.jar 2024-06-28T15:27:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-0.18.jar 2024-06-28T15:27:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-0.18.jar 2024-06-28T15:27:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-osx-amd64-0.18.jar 2024-06-28T15:27:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-amd64-0.18.jar 2024-06-28T15:27:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-amd64-min-0.18.jar 2024-06-28T15:27:10.425 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-i386-0.18.jar 2024-06-28T15:27:10.425 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-i386-min-0.18.jar 2024-06-28T15:27:10.425 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-aarch64-ncurses5-0.18.jar 2024-06-28T15:27:10.425 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-ncurses5-0.18.jar 2024-06-28T15:27:10.426 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-ncurses6-0.18.jar 2024-06-28T15:27:10.426 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-ncurses5-0.18.jar 2024-06-28T15:27:10.426 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-ncurses6-0.18.jar 2024-06-28T15:27:10.426 INFO:teuthology.orchestra.run.smithi178.stdout: creating: /opt/gradle/gradle-6.0.1/lib/plugins/ 2024-06-28T15:27:10.426 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-antlr-6.0.1.jar 2024-06-28T15:27:10.426 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-code-quality-6.0.1.jar 2024-06-28T15:27:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-signing-6.0.1.jar 2024-06-28T15:27:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-tooling-native-6.0.1.jar 2024-06-28T15:27:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-native-6.0.1.jar 2024-06-28T15:27:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-native-6.0.1.jar 2024-06-28T15:27:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-native-6.0.1.jar 2024-06-28T15:27:10.431 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-kotlin-dsl-provider-plugins-6.0.1.jar 2024-06-28T15:27:10.432 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugin-development-6.0.1.jar 2024-06-28T15:27:10.433 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-maven-6.0.1.jar 2024-06-28T15:27:10.434 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-play-6.0.1.jar 2024-06-28T15:27:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-6.0.1.jar 2024-06-28T15:27:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-scala-6.0.1.jar 2024-06-28T15:27:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ear-6.0.1.jar 2024-06-28T15:27:10.438 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-play-6.0.1.jar 2024-06-28T15:27:10.439 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-javascript-6.0.1.jar 2024-06-28T15:27:10.440 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ivy-6.0.1.jar 2024-06-28T15:27:10.440 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-jacoco-6.0.1.jar 2024-06-28T15:27:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-init-6.0.1.jar 2024-06-28T15:27:10.443 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-kotlin-dsl-tooling-builders-6.0.1.jar 2024-06-28T15:27:10.444 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugins-6.0.1.jar 2024-06-28T15:27:10.445 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-test-kit-6.0.1.jar 2024-06-28T15:27:10.445 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-groovy-6.0.1.jar 2024-06-28T15:27:10.446 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-scala-6.0.1.jar 2024-06-28T15:27:10.446 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-junit-platform-6.0.1.jar 2024-06-28T15:27:10.446 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-tooling-api-builders-6.0.1.jar 2024-06-28T15:27:10.447 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-jvm-6.0.1.jar 2024-06-28T15:27:10.449 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-java-6.0.1.jar 2024-06-28T15:27:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-jvm-6.0.1.jar 2024-06-28T15:27:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-jvm-6.0.1.jar 2024-06-28T15:27:10.454 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-native-6.0.1.jar 2024-06-28T15:27:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-diagnostics-6.0.1.jar 2024-06-28T15:27:10.456 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-publish-6.0.1.jar 2024-06-28T15:27:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-base-6.0.1.jar 2024-06-28T15:27:10.459 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-base-6.0.1.jar 2024-06-28T15:27:10.459 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugin-use-6.0.1.jar 2024-06-28T15:27:10.465 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-instant-execution-6.0.1.jar 2024-06-28T15:27:10.465 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-version-control-6.0.1.jar 2024-06-28T15:27:10.479 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-dependency-management-6.0.1.jar 2024-06-28T15:27:10.479 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-composite-builds-6.0.1.jar 2024-06-28T15:27:10.480 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-reporting-6.0.1.jar 2024-06-28T15:27:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-s3-6.0.1.jar 2024-06-28T15:27:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-gcs-6.0.1.jar 2024-06-28T15:27:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-cache-http-6.0.1.jar 2024-06-28T15:27:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-http-6.0.1.jar 2024-06-28T15:27:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-sftp-6.0.1.jar 2024-06-28T15:27:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-profile-6.0.1.jar 2024-06-28T15:27:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-workers-6.0.1.jar 2024-06-28T15:27:10.500 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/org.eclipse.jgit-5.5.0.201909110433-r.jar 2024-06-28T15:27:10.500 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-http-3.0.0.jar 2024-06-28T15:27:10.500 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-http-shared-3.0.0.jar 2024-06-28T15:27:10.501 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-api-services-storage-v1-rev136-1.25.0.jar 2024-06-28T15:27:10.502 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-api-client-1.25.0.jar 2024-06-28T15:27:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/asm-util-7.1.jar 2024-06-28T15:27:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/ivy-2.3.0.jar 2024-06-28T15:27:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-compat-3.0.4.jar 2024-06-28T15:27:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-core-3.0.4.jar 2024-06-28T15:27:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/pmaven-common-0.8-20100325.jar 2024-06-28T15:27:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/pmaven-groovy-0.8-20100325.jar 2024-06-28T15:27:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-file-3.0.0.jar 2024-06-28T15:27:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-container-default-1.7.1.jar 2024-06-28T15:27:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-connector-wagon-1.13.1.jar 2024-06-28T15:27:10.520 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcpg-jdk15on-1.63.jar 2024-06-28T15:27:10.528 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/rhino-1.7.10.jar 2024-06-28T15:27:10.529 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gson-2.8.5.jar 2024-06-28T15:27:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/simple-4.1.21.jar 2024-06-28T15:27:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jatl-0.2.3.jar 2024-06-28T15:27:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-plugin-api-3.0.4.jar 2024-06-28T15:27:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-oauth-client-1.25.0.jar 2024-06-28T15:27:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-http-client-jackson2-1.25.0.jar 2024-06-28T15:27:10.534 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-http-client-1.25.0.jar 2024-06-28T15:27:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/testng-6.3.1.jar 2024-06-28T15:27:10.540 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/snakeyaml-1.17.jar 2024-06-28T15:27:10.540 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/dd-plist-1.21.jar 2024-06-28T15:27:10.542 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-4.12.jar 2024-06-28T15:27:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bsh-2.0b6.jar 2024-06-28T15:27:10.546 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-launcher-1.3.1.jar 2024-06-28T15:27:10.552 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-s3-1.11.633.jar 2024-06-28T15:27:10.555 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-kms-1.11.633.jar 2024-06-28T15:27:10.561 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-core-1.11.633.jar 2024-06-28T15:27:10.566 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/httpclient-4.5.10.jar 2024-06-28T15:27:10.568 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jcifs-1.3.17.jar 2024-06-28T15:27:10.569 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/nekohtml-1.9.22.jar 2024-06-28T15:27:10.579 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/xercesImpl-2.12.0.jar 2024-06-28T15:27:10.581 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jsch-0.1.55.jar 2024-06-28T15:27:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jaxb-impl-2.3.2.jar 2024-06-28T15:27:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jmespath-java-1.11.633.jar 2024-06-28T15:27:10.596 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-databind-2.9.10.jar 2024-06-28T15:27:10.599 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-core-2.9.10.jar 2024-06-28T15:27:10.599 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-annotations-2.9.10.jar 2024-06-28T15:27:10.603 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/joda-time-2.10.4.jar 2024-06-28T15:27:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcpkix-jdk15on-1.61.jar 2024-06-28T15:27:10.637 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcprov-jdk15on-1.63.jar 2024-06-28T15:27:10.639 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/commons-codec-1.13.jar 2024-06-28T15:27:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-aether-provider-3.0.4.jar 2024-06-28T15:27:10.641 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-impl-1.13.1.jar 2024-06-28T15:27:10.641 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-util-1.13.1.jar 2024-06-28T15:27:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-spi-1.13.1.jar 2024-06-28T15:27:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-api-1.13.1.jar 2024-06-28T15:27:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/hamcrest-core-1.3.jar 2024-06-28T15:27:10.644 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/httpcore-4.4.12.jar 2024-06-28T15:27:10.645 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jcommander-1.72.jar 2024-06-28T15:27:10.646 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jzlib-1.1.3.jar 2024-06-28T15:27:10.646 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-artifact-3.0.4.jar 2024-06-28T15:27:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-model-builder-3.0.4.jar 2024-06-28T15:27:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-model-3.0.4.jar 2024-06-28T15:27:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-repository-metadata-3.0.4.jar 2024-06-28T15:27:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-settings-builder-3.0.4.jar 2024-06-28T15:27:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-settings-3.0.4.jar 2024-06-28T15:27:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-sec-dispatcher-1.3.jar 2024-06-28T15:27:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-cipher-1.7.jar 2024-06-28T15:27:10.651 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-classworlds-2.5.1.jar 2024-06-28T15:27:10.651 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-component-annotations-1.5.5.jar 2024-06-28T15:27:10.651 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-interpolation-1.14.jar 2024-06-28T15:27:10.651 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-provider-api-3.0.0.jar 2024-06-28T15:27:10.652 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-utils-3.1.0.jar 2024-06-28T15:27:10.653 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/xbean-reflect-3.7.jar 2024-06-28T15:27:10.657 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/ion-java-1.0.2.jar 2024-06-28T15:27:10.657 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-engine-1.3.1.jar 2024-06-28T15:27:10.658 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-commons-1.3.1.jar 2024-06-28T15:27:10.658 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/apiguardian-api-1.0.0.jar 2024-06-28T15:27:10.658 INFO:teuthology.orchestra.run.smithi178.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/opentest4j-1.1.1.jar 2024-06-28T15:27:10.688 INFO:teuthology.orchestra.run.smithi178.stdout:export PATH=/opt/gradle/gradle-6.0.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin 2024-06-28T15:27:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:27:11.152 INFO:teuthology.orchestra.run.smithi178.stdout:Welcome to Gradle 6.0.1! 2024-06-28T15:27:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:27:11.152 INFO:teuthology.orchestra.run.smithi178.stdout:Here are the highlights of this release: 2024-06-28T15:27:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: - Substantial improvements in dependency management, including 2024-06-28T15:27:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: - Publishing Gradle Module Metadata in addition to pom.xml 2024-06-28T15:27:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: - Advanced control of transitive versions 2024-06-28T15:27:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: - Support for optional features and dependencies 2024-06-28T15:27:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: - Rules to tweak published metadata 2024-06-28T15:27:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: - Support for Java 13 2024-06-28T15:27:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: - Faster incremental Java and Groovy compilation 2024-06-28T15:27:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: - New Zinc compiler for Scala 2024-06-28T15:27:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: - VS2019 support 2024-06-28T15:27:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: - Support for Gradle Enterprise plugin 3.0 2024-06-28T15:27:11.155 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:27:11.155 INFO:teuthology.orchestra.run.smithi178.stdout:For more details see https://docs.gradle.org/6.0.1/release-notes.html 2024-06-28T15:27:11.156 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:27:11.156 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:27:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:------------------------------------------------------------ 2024-06-28T15:27:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:Gradle 6.0.1 2024-06-28T15:27:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:------------------------------------------------------------ 2024-06-28T15:27:11.156 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:27:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:Build time: 2019-11-18 20:25:01 UTC 2024-06-28T15:27:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:Revision: fad121066a68c4701acd362daf4287a7c309a0f5 2024-06-28T15:27:11.156 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:27:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:Kotlin: 1.3.50 2024-06-28T15:27:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:Groovy: 2.5.8 2024-06-28T15:27:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:Ant: Apache Ant(TM) version 1.10.7 compiled on September 1 2019 2024-06-28T15:27:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:JVM: 11.0.17 (Ubuntu 11.0.17+8-post-Ubuntu-1ubuntu222.04) 2024-06-28T15:27:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:OS: Linux 5.15.0-113-generic amd64 2024-06-28T15:27:11.157 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:27:11.172 DEBUG:tasks.s3tests_java:S3 Tests Java: BEGIN 2024-06-28T15:27:11.172 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-06-28T15:27:11.172 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi178.front.sepia.ceph.com 2024-06-28T15:27:11.172 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-06-28T15:27:11.172 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi195.front.sepia.ceph.com 2024-06-28T15:27:11.172 INFO:tasks.s3tests_java:S3 Tests Java: Creating S3 users... 2024-06-28T15:27:11.363 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf is {'bucket_prefix': 'test-', 's3main': {'access_key': None, 'access_secret': None, 'region': 'mexico', 'endpoint': None, 'port': None, 'display_name': None, 'email': None, 'is_secure': None, 'SSE': 'AES256', 's3dir': 'folder', 'dir': '../data'}} 2024-06-28T15:27:11.363 DEBUG:tasks.s3tests_java:S3 Tests Java: Creating user tester.client.0 2024-06-28T15:27:11.364 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf[s3main] is {'access_key': 'LBUUKGXBLQEGBRHYUGIQ', 'access_secret': 'ykPVuASUFvy4arKMsQIo11917Vw7AFOSSjcvSYligmvu3QJC1CexHw==', 'region': 'mexico', 'endpoint': 'smithi178.front.sepia.ceph.com:80', 'port': 80, 'display_name': 'Ms. tester.client.0', 'email': 'tester.client.0_test@test.test', 'is_secure': False, 'SSE': 'AES256', 's3dir': 'folder', 'dir': '../data', 'user_id': 'tester.client.0', 'host': 'smithi178.front.sepia.ceph.com'} 2024-06-28T15:27:11.364 DEBUG:tasks.s3tests_java:S3 Tests Java: Setion, User = s3main, tester.client.0 2024-06-28T15:27:11.364 INFO:tasks.s3tests_java:['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '-n', 'client.0', 'user', 'create', '--uid', 'tester.client.0', '--display-name', 'Ms. tester.client.0', '--access-key', 'LBUUKGXBLQEGBRHYUGIQ', '--secret', 'ykPVuASUFvy4arKMsQIo11917Vw7AFOSSjcvSYligmvu3QJC1CexHw==', '--email', 'tester.client.0_test@test.test', '--cluster', 'ceph'] 2024-06-28T15:27:11.364 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid tester.client.0 --display-name 'Ms. tester.client.0' --access-key LBUUKGXBLQEGBRHYUGIQ --secret ykPVuASUFvy4arKMsQIo11917Vw7AFOSSjcvSYligmvu3QJC1CexHw== --email tester.client.0_test@test.test --cluster ceph 2024-06-28T15:27:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:27:11.406 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:27:11.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.467+0000 7f834583a980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.467+0000 7f834583a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:11.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.467+0000 7f834583a980 20 realm 2024-06-28T15:27:11.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.467+0000 7f834583a980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.467+0000 7f834583a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:11.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.467+0000 7f834583a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:27:11.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.467+0000 7f834583a980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.467+0000 7f834583a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:11.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.467+0000 7f834583a980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.471+0000 7f834583a980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:27:11.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.471+0000 7f834583a980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.471+0000 7f834583a980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:27:11.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.471+0000 7f834583a980 20 searching for the correct realm 2024-06-28T15:27:11.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.475+0000 7f8343b4c640 0 -- 172.21.15.178:0/3017507636 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f83380f6040 msgr2=0x7f8338123570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:27:11.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.695+0000 7f834334b640 0 -- 172.21.15.178:0/3017507636 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55de5a7845b0 msgr2=0x55de5a7869a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:27:11.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.703+0000 7f834583a980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:27:11.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.703+0000 7f834583a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:27:11.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.703+0000 7f834583a980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:27:11.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.703+0000 7f834583a980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:27:11.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.703+0000 7f834583a980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:27:11.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.703+0000 7f834583a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:27:11.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.703+0000 7f834583a980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.703+0000 7f834583a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:11.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.703+0000 7f834583a980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.703+0000 7f834583a980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:27:11.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.703+0000 7f834583a980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:27:11.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 20 zone default found 2024-06-28T15:27:11.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 4 Realm: () 2024-06-28T15:27:11.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:27:11.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:27:11.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:27:11.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 20 zonegroup default 2024-06-28T15:27:11.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:11.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:11.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:11.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 20 started sync module instance, tier type = 2024-06-28T15:27:11.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.707+0000 7f834583a980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:27:11.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.735+0000 7f834583a980 20 add_watcher() i=0 2024-06-28T15:27:11.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.735+0000 7f834583a980 20 add_watcher() i=1 2024-06-28T15:27:11.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.735+0000 7f834583a980 20 add_watcher() i=2 2024-06-28T15:27:11.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.735+0000 7f834583a980 20 add_watcher() i=3 2024-06-28T15:27:11.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.735+0000 7f834583a980 20 add_watcher() i=4 2024-06-28T15:27:11.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.735+0000 7f834583a980 20 add_watcher() i=5 2024-06-28T15:27:11.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.735+0000 7f834583a980 20 add_watcher() i=6 2024-06-28T15:27:11.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.735+0000 7f834583a980 20 add_watcher() i=7 2024-06-28T15:27:11.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.735+0000 7f834583a980 2 all 8 watchers are set, enabling cache 2024-06-28T15:27:11.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.739+0000 7f81f9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:27:11.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.739+0000 7f834583a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:27:11.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.739+0000 7f834583a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:27:11.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.739+0000 7f834583a980 5 note: GC not initialized 2024-06-28T15:27:11.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f81f8fe1640 20 reqs_thread_entry: start 2024-06-28T15:27:11.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 20 init_complete bucket index max shards: 1 2024-06-28T15:27:11.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 20 Filter name: none 2024-06-28T15:27:11.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f81727fc640 20 reqs_thread_entry: start 2024-06-28T15:27:11.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2024-06-28T15:27:11.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:11.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x0 2024-06-28T15:27:11.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-06-28T15:27:11.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 10 cache get: name=default.rgw.meta+users.email+tester.client.0_test@test.test : miss 2024-06-28T15:27:11.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:11.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x0 2024-06-28T15:27:11.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 10 adding default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-06-28T15:27:11.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 10 cache get: name=default.rgw.meta+users.keys+LBUUKGXBLQEGBRHYUGIQ : miss 2024-06-28T15:27:11.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:11.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 10 cache put: name=default.rgw.meta+users.keys+LBUUKGXBLQEGBRHYUGIQ info.flags=0x0 2024-06-28T15:27:11.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 10 adding default.rgw.meta+users.keys+LBUUKGXBLQEGBRHYUGIQ to cache LRU end 2024-06-28T15:27:11.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 10 cache get: name=default.rgw.meta+users.keys+LBUUKGXBLQEGBRHYUGIQ : hit (negative entry) 2024-06-28T15:27:11.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.743+0000 7f834583a980 10 cache get: name=default.rgw.meta+users.keys+LBUUKGXBLQEGBRHYUGIQ : hit (negative entry) 2024-06-28T15:27:11.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.747+0000 7f834583a980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2024-06-28T15:27:11.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.747+0000 7f834583a980 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-06-28T15:27:11.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.747+0000 7f834583a980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.uid:tester.client.0, ofs0, ns] 2024-06-28T15:27:11.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.747+0000 7f8226ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595103 cookie 94413488980160 notifier 7533 bl.length()=670 2024-06-28T15:27:11.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.747+0000 7f8226ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2024-06-28T15:27:11.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.747+0000 7f8226ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-06-28T15:27:11.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.751+0000 7f834583a980 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2024-06-28T15:27:11.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.751+0000 7f834583a980 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-06-28T15:27:11.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.751+0000 7f834583a980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:tester.client.0_test@test.test, ofs0, ns] 2024-06-28T15:27:11.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.751+0000 7f8226ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595125 cookie 94413488968512 notifier 7533 bl.length()=200 2024-06-28T15:27:11.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.751+0000 7f8226ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2024-06-28T15:27:11.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.751+0000 7f8226ffd640 10 rgw watcher librados: moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-06-28T15:27:11.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.755+0000 7f834583a980 10 cache put: name=default.rgw.meta+users.keys+LBUUKGXBLQEGBRHYUGIQ info.flags=0x7 2024-06-28T15:27:11.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.755+0000 7f834583a980 10 moving default.rgw.meta+users.keys+LBUUKGXBLQEGBRHYUGIQ to cache LRU end 2024-06-28T15:27:11.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.755+0000 7f834583a980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.keys:LBUUKGXBLQEGBRHYUGIQ, ofs0, ns] 2024-06-28T15:27:11.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.755+0000 7f8226ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595095 cookie 94413488973232 notifier 7533 bl.length()=189 2024-06-28T15:27:11.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.755+0000 7f8226ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+LBUUKGXBLQEGBRHYUGIQ info.flags=0x7 2024-06-28T15:27:11.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.755+0000 7f8226ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+LBUUKGXBLQEGBRHYUGIQ to cache LRU end 2024-06-28T15:27:11.741 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T15:27:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: "user_id": "tester.client.0", 2024-06-28T15:27:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: "display_name": "Ms. tester.client.0", 2024-06-28T15:27:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: "email": "tester.client.0_test@test.test", 2024-06-28T15:27:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: "suspended": 0, 2024-06-28T15:27:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:27:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: "subusers": [], 2024-06-28T15:27:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: "keys": [ 2024-06-28T15:27:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T15:27:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: "user": "tester.client.0", 2024-06-28T15:27:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: "access_key": "LBUUKGXBLQEGBRHYUGIQ", 2024-06-28T15:27:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: "secret_key": "ykPVuASUFvy4arKMsQIo11917Vw7AFOSSjcvSYligmvu3QJC1CexHw==", 2024-06-28T15:27:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: "active": true, 2024-06-28T15:27:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:27:11.747980Z" 2024-06-28T15:27:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T15:27:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-06-28T15:27:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: "swift_keys": [], 2024-06-28T15:27:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": [], 2024-06-28T15:27:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: "op_mask": "read, write, delete", 2024-06-28T15:27:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: "default_placement": "", 2024-06-28T15:27:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: "default_storage_class": "", 2024-06-28T15:27:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: "placement_tags": [], 2024-06-28T15:27:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: "bucket_quota": { 2024-06-28T15:27:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:27:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:27:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:27:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:27:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:27:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:27:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: "user_quota": { 2024-06-28T15:27:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:27:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:27:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:27:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:27:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:27:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:27:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: "temp_url_keys": [], 2024-06-28T15:27:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "rgw", 2024-06-28T15:27:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: "mfa_ids": [], 2024-06-28T15:27:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: "account_id": "", 2024-06-28T15:27:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2024-06-28T15:27:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:27:11.747942Z", 2024-06-28T15:27:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2024-06-28T15:27:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: "group_ids": [] 2024-06-28T15:27:11.745 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T15:27:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:27:11.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.759+0000 7f81f9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:27:11.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.763+0000 7f834583a980 20 remove_watcher() i=0 2024-06-28T15:27:11.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.763+0000 7f834583a980 2 removed watcher, disabling cache 2024-06-28T15:27:11.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.767+0000 7f834583a980 20 remove_watcher() i=1 2024-06-28T15:27:11.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.771+0000 7f834583a980 20 remove_watcher() i=2 2024-06-28T15:27:11.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.771+0000 7f834583a980 20 remove_watcher() i=3 2024-06-28T15:27:11.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.771+0000 7f834583a980 20 remove_watcher() i=4 2024-06-28T15:27:11.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.775+0000 7f834583a980 20 remove_watcher() i=5 2024-06-28T15:27:11.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.775+0000 7f834583a980 20 remove_watcher() i=6 2024-06-28T15:27:11.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:11.779+0000 7f834583a980 20 remove_watcher() i=7 2024-06-28T15:27:11.779 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T15:27:11.779 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf 2024-06-28T15:27:11.827 INFO:tasks.s3tests_java:S3 Tests Java: Running tests... 2024-06-28T15:27:11.827 DEBUG:teuthology.orchestra.run.smithi178:> cp /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf /home/ubuntu/cephtest/s3-tests-java/config.properties 2024-06-28T15:27:11.872 DEBUG:teuthology.orchestra.run.smithi178:> radosgw-admin gc process --include-all 2024-06-28T15:27:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:27:11.947 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:27:11.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.011+0000 7fed92427980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.015+0000 7fed92427980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:11.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.015+0000 7fed92427980 20 realm 2024-06-28T15:27:11.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.015+0000 7fed92427980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.015+0000 7fed92427980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:11.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.015+0000 7fed92427980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:27:11.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.015+0000 7fed92427980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.015+0000 7fed92427980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:11.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.015+0000 7fed92427980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.015+0000 7fed92427980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:27:11.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.015+0000 7fed92427980 20 rados->read ofs=0 len=0 2024-06-28T15:27:11.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.015+0000 7fed92427980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:27:11.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.015+0000 7fed92427980 20 searching for the correct realm 2024-06-28T15:27:12.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.019+0000 7fed8f737640 0 -- 172.21.15.178:0/1202830280 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fed80018050 msgr2=0x7fed8005c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:27:12.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.235+0000 7fed8ff38640 0 -- 172.21.15.178:0/1202830280 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fed84047440 msgr2=0x7fed840478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:27:12.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.451+0000 7fed92427980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:27:12.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.451+0000 7fed92427980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:27:12.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.451+0000 7fed92427980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:27:12.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.451+0000 7fed92427980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:27:12.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.451+0000 7fed92427980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:27:12.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.451+0000 7fed92427980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:27:12.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.451+0000 7fed92427980 20 rados->read ofs=0 len=0 2024-06-28T15:27:12.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.451+0000 7fed92427980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:12.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.451+0000 7fed92427980 20 rados->read ofs=0 len=0 2024-06-28T15:27:12.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:27:12.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 20 rados->read ofs=0 len=0 2024-06-28T15:27:12.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:27:12.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 20 zone default found 2024-06-28T15:27:12.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 4 Realm: () 2024-06-28T15:27:12.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:27:12.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:27:12.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:27:12.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 20 zonegroup default 2024-06-28T15:27:12.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 20 rados->read ofs=0 len=0 2024-06-28T15:27:12.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:12.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 20 rados->read ofs=0 len=0 2024-06-28T15:27:12.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:12.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 20 rados->read ofs=0 len=0 2024-06-28T15:27:12.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:12.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 20 started sync module instance, tier type = 2024-06-28T15:27:12.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.455+0000 7fed92427980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:27:12.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.483+0000 7fed92427980 20 add_watcher() i=0 2024-06-28T15:27:12.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.483+0000 7fed92427980 20 add_watcher() i=1 2024-06-28T15:27:12.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.483+0000 7fed92427980 20 add_watcher() i=2 2024-06-28T15:27:12.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.483+0000 7fed92427980 20 add_watcher() i=3 2024-06-28T15:27:12.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.483+0000 7fed92427980 20 add_watcher() i=4 2024-06-28T15:27:12.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.483+0000 7fed92427980 20 add_watcher() i=5 2024-06-28T15:27:12.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.483+0000 7fed92427980 20 add_watcher() i=6 2024-06-28T15:27:12.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.483+0000 7fed92427980 20 add_watcher() i=7 2024-06-28T15:27:12.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.483+0000 7fed92427980 2 all 8 watchers are set, enabling cache 2024-06-28T15:27:12.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.487+0000 7fec45fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:27:12.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.487+0000 7fed92427980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:27:12.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.487+0000 7fed92427980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:27:12.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.583+0000 7fec44fe1640 20 reqs_thread_entry: start 2024-06-28T15:27:12.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.583+0000 7fed92427980 20 init_complete bucket index max shards: 1 2024-06-28T15:27:12.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.583+0000 7fed92427980 20 Filter name: none 2024-06-28T15:27:12.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.583+0000 7febbe7fc640 20 reqs_thread_entry: start 2024-06-28T15:27:12.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.583+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-06-28T15:27:12.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=10, truncated=0, next_marker='' 2024-06-28T15:27:12.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.3h8nBofYqfTXfnIUkfD5984-BBtf2Gz_1_d827886f1ba02dcee0a9d6d5a18d293b', time=2024-06-28T17:21:54.490668+0000, chain.objs.size()=1 2024-06-28T15:27:12.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.3h8nBofYqfTXfnIUkfD5984-BBtf2Gz_0 2024-06-28T15:27:12.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12760458337667342165', time=2024-06-28T17:23:52.397832+0000, chain.objs.size()=1 2024-06-28T15:27:12.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.Hb0SL0VVhrqf-3hyHJzvyJQOZCKbZqU_0 2024-06-28T15:27:12.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3818338248210262966', time=2024-06-28T17:23:52.514677+0000, chain.objs.size()=1 2024-06-28T15:27:12.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.gsnKglM0mbfmWgQJl9VyaSBF4d9fcvo_0 2024-06-28T15:27:12.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17712614253410365269', time=2024-06-28T17:23:56.953333+0000, chain.objs.size()=1 2024-06-28T15:27:12.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.EsET2VkXwq5g5SQRtp_HekIn13KRITf_0 2024-06-28T15:27:12.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3638414344448065957', time=2024-06-28T17:23:57.029569+0000, chain.objs.size()=1 2024-06-28T15:27:12.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.wEzfE-UrHPL2S-6dY0cOUub1hk103uV_0 2024-06-28T15:27:12.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14381773949804202797', time=2024-06-28T17:24:31.758710+0000, chain.objs.size()=1 2024-06-28T15:27:12.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.GGgVqTTHoI0jETazFTqfAPUKIKTkEuM_0 2024-06-28T15:27:12.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11323336910321249762', time=2024-06-28T17:24:35.599290+0000, chain.objs.size()=1 2024-06-28T15:27:12.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.rijTX4YGoEq-urTdl_ToPltPJkYwVvz_0 2024-06-28T15:27:12.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6733647000295062282', time=2024-06-28T17:24:35.790918+0000, chain.objs.size()=1 2024-06-28T15:27:12.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.5unKRo_2WSuC_5_C6l00qPc1M7UQqpC_0 2024-06-28T15:27:12.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12115478195456358416', time=2024-06-28T17:25:14.502158+0000, chain.objs.size()=1 2024-06-28T15:27:12.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.L7L74grSl7X7rMKSyV2ldebP2gAF_AW_0 2024-06-28T15:27:12.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10737053281295459131', time=2024-06-28T17:25:53.965864+0000, chain.objs.size()=1 2024-06-28T15:27:12.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.587+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.a73Qdp7JC9lABN33XmDqVYZeLfsVgvg_0 2024-06-28T15:27:12.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.595+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.599+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-06-28T15:27:12.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.603+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.603+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.607+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-06-28T15:27:12.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.607+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5730256288860443778', time=2024-06-28T17:23:52.391342+0000, chain.objs.size()=1 2024-06-28T15:27:12.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.607+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.ok0t5N8DwcaQA8jnCl0UcKUHjJjQ-kX_0 2024-06-28T15:27:12.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.607+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14470716029023849168', time=2024-06-28T17:24:31.623217+0000, chain.objs.size()=1 2024-06-28T15:27:12.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.607+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.ZfQCcwerz1l65BebYmSuEJ8z8y2KAcd_0 2024-06-28T15:27:12.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.607+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7429984005952571907', time=2024-06-28T17:24:35.659790+0000, chain.objs.size()=1 2024-06-28T15:27:12.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.607+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.8ZDlXbQlwvRRMdgOQRULD_CvL3aXzvk_0 2024-06-28T15:27:12.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.607+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14462084485962565350', time=2024-06-28T17:25:13.296674+0000, chain.objs.size()=1 2024-06-28T15:27:12.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.607+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.ED1WZp-O7RSgYfg1Eh9jrU7tV30X8f-_0 2024-06-28T15:27:12.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.607+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7447355719090570448', time=2024-06-28T17:25:55.622330+0000, chain.objs.size()=1 2024-06-28T15:27:12.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.607+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.18__shadow_.JoT_5Pjc1xy3pzu0MZM_yHqMQHB3Jrv_0 2024-06-28T15:27:12.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.607+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13955351532272299599', time=2024-06-28T17:26:03.313874+0000, chain.objs.size()=1 2024-06-28T15:27:12.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.607+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.19__shadow:o.mHuksCRuaqPVTspFtNfeGbm-fgkRF_.A4po8mN1RyBy7gISDTzMbfSQ8O_jCoe_0 2024-06-28T15:27:12.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.615+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.619+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-06-28T15:27:12.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.623+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.623+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.623+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-06-28T15:27:12.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.623+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.05oqeZkcZBXD7L5ZKW3Yg7ODJxNZ8DC_1_042f62947b5bb2063ce9c3578e63f830', time=2024-06-28T17:21:54.541578+0000, chain.objs.size()=1 2024-06-28T15:27:12.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.623+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.05oqeZkcZBXD7L5ZKW3Yg7ODJxNZ8DC_0 2024-06-28T15:27:12.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.623+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11047743510652385363', time=2024-06-28T17:23:17.682863+0000, chain.objs.size()=1 2024-06-28T15:27:12.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.623+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.pdi2P8M-tN0cCn8h9oKr9wsmPptJvPt_0 2024-06-28T15:27:12.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.623+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15107356144917559296', time=2024-06-28T17:23:52.514677+0000, chain.objs.size()=1 2024-06-28T15:27:12.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.623+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.HRL308rrFjr0qhT_sPXKrFey9RcVNv3_0 2024-06-28T15:27:12.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.623+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6312315166014784049', time=2024-06-28T17:25:14.431393+0000, chain.objs.size()=1 2024-06-28T15:27:12.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.623+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.v1gV1CpcmNnWcyGPPY31AF05MZ0gUcB_0 2024-06-28T15:27:12.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.627+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.631+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-06-28T15:27:12.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=12, truncated=0, next_marker='' 2024-06-28T15:27:12.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.265532173213398166', time=2024-06-28T17:23:17.894948+0000, chain.objs.size()=1 2024-06-28T15:27:12.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.VJRnKUeXj_2bk8LO96rKzrMU66HaJrj_0 2024-06-28T15:27:12.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7496078355401239914', time=2024-06-28T17:23:52.509277+0000, chain.objs.size()=1 2024-06-28T15:27:12.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.Ywdz3U9bJFWfg2mSK3GYkNBpbnnviCw_0 2024-06-28T15:27:12.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5528445079807447172', time=2024-06-28T17:23:56.195653+0000, chain.objs.size()=1 2024-06-28T15:27:12.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.nNZGsSCeoknhmn-50Cij3bA_WSW2JvU_0 2024-06-28T15:27:12.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1642173863128021293', time=2024-06-28T17:23:56.944477+0000, chain.objs.size()=1 2024-06-28T15:27:12.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_._Fdrl_vy1fqGZHDkxKQ0BSoFdiMFKFe_0 2024-06-28T15:27:12.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10947564826439267763', time=2024-06-28T17:23:57.031548+0000, chain.objs.size()=1 2024-06-28T15:27:12.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.7ZWwbZuIvckNbWZ1T-sFTjh0g6SqerT_0 2024-06-28T15:27:12.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15171628879254386476', time=2024-06-28T17:23:57.126499+0000, chain.objs.size()=1 2024-06-28T15:27:12.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.73B_a8Ow0AEF1ID5zF_RnYf1JhBgS30_0 2024-06-28T15:27:12.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.248606201711959276', time=2024-06-28T17:24:31.607199+0000, chain.objs.size()=1 2024-06-28T15:27:12.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.RLi1rV9cKbYQcEgarMXVOkjl9kPQXQL_0 2024-06-28T15:27:12.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9743208660350108967', time=2024-06-28T17:24:31.608994+0000, chain.objs.size()=1 2024-06-28T15:27:12.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.sr8zKmokRZvHXSni47rKR8KwQYfUDXK_0 2024-06-28T15:27:12.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15312688713543597030', time=2024-06-28T17:24:31.610640+0000, chain.objs.size()=1 2024-06-28T15:27:12.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.igtym9ITT5LiyqHol8pumsxf7IcmeSP_0 2024-06-28T15:27:12.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17511145272177076002', time=2024-06-28T17:24:35.847914+0000, chain.objs.size()=1 2024-06-28T15:27:12.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.xzKBV8G9-7JotabR5qVxiKTD13I_ZY9_0 2024-06-28T15:27:12.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10853879938781373865', time=2024-06-28T17:25:14.429798+0000, chain.objs.size()=1 2024-06-28T15:27:12.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.S4Fu1anBFakzyV0PE7YixC4wT6e46uZ_0 2024-06-28T15:27:12.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4565138389903205780', time=2024-06-28T17:25:14.501331+0000, chain.objs.size()=1 2024-06-28T15:27:12.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.635+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.Yl-mUVRLeHFV_4iHw3CkDCSNDrgTeS__0 2024-06-28T15:27:12.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.655+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.659+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-06-28T15:27:12.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.659+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-06-28T15:27:12.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='_KQFYKaTh-9doPnysAYflcKpbO6SbcgV', time=2024-06-28T17:21:54.532450+0000, chain.objs.size()=1 2024-06-28T15:27:12.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.NIcKd001J0p3V_9x5kkKjpF57U6PSFl_0 2024-06-28T15:27:12.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1332505300669175676', time=2024-06-28T17:21:54.813663+0000, chain.objs.size()=3 2024-06-28T15:27:12.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-STANDARD.2~HW--NWmyhMyiauJwopSuQ8FEdWTm3fm.1 2024-06-28T15:27:12.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-STANDARD.2~HW--NWmyhMyiauJwopSuQ8FEdWTm3fm.2 2024-06-28T15:27:12.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-STANDARD.2~HW--NWmyhMyiauJwopSuQ8FEdWTm3fm.3 2024-06-28T15:27:12.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13332544484712400686', time=2024-06-28T17:23:13.515448+0000, chain.objs.size()=1 2024-06-28T15:27:12.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.u3BeKPXoWK16evQ1yS5lbz0AgoH864s_0 2024-06-28T15:27:12.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16874510436986979801', time=2024-06-28T17:23:17.682203+0000, chain.objs.size()=1 2024-06-28T15:27:12.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.UPQK7nklQKh5Jj-tKMS4P2HIz9HYnwO_0 2024-06-28T15:27:12.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3548375611909521117', time=2024-06-28T17:25:14.243605+0000, chain.objs.size()=1 2024-06-28T15:27:12.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.O04NjIBf69H9KKsXdjhZnP5zKOpbCzg_0 2024-06-28T15:27:12.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.503225548575185023', time=2024-06-28T17:25:14.431421+0000, chain.objs.size()=1 2024-06-28T15:27:12.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.KK7rsec-NfWm_fL7hgOD9tGvixII9iE_0 2024-06-28T15:27:12.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.773113911425259414', time=2024-06-28T17:25:18.877585+0000, chain.objs.size()=1 2024-06-28T15:27:12.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.663+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.iKE6HOQbeSgkmasD74u6VcBI8HNOCFR_0 2024-06-28T15:27:12.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.675+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.679+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-06-28T15:27:12.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-06-28T15:27:12.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='_vwMLj2567_mZeHpWnpIRgpi7ztcAeld', time=2024-06-28T17:21:54.582736+0000, chain.objs.size()=1 2024-06-28T15:27:12.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.AsKF9D4X18jdffSCO2aq_O7jMedmfRM_0 2024-06-28T15:27:12.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11117367891280923712', time=2024-06-28T17:23:13.565711+0000, chain.objs.size()=1 2024-06-28T15:27:12.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.HJEGSMR6G0JtOVT6SpL3xjzeKJrM5EC_0 2024-06-28T15:27:12.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3669551209070582748', time=2024-06-28T17:23:17.881114+0000, chain.objs.size()=1 2024-06-28T15:27:12.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.fFFiDxzacXNyNYrGQK3zM9i-2pb3_tA_0 2024-06-28T15:27:12.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18332168230776333602', time=2024-06-28T17:24:35.662881+0000, chain.objs.size()=1 2024-06-28T15:27:12.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.N83aVDMtwv8UZP72lvPxNDcJWGrb4yr_0 2024-06-28T15:27:12.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11960726095848021238', time=2024-06-28T17:25:18.870240+0000, chain.objs.size()=1 2024-06-28T15:27:12.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.M8SF0JzKQI2-9hUClR3x2rSQTqt9J-L_0 2024-06-28T15:27:12.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1166105631065748', time=2024-06-28T17:25:53.902349+0000, chain.objs.size()=1 2024-06-28T15:27:12.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.683+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.msp3wO4CjdJek5TX2uQAjONU-AFSpRr_0 2024-06-28T15:27:12.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.691+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.695+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-06-28T15:27:12.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-06-28T15:27:12.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='_-qSAQ1IQaDM8KYlGdteAQi16c6bS2_g', time=2024-06-28T17:21:54.571508+0000, chain.objs.size()=1 2024-06-28T15:27:12.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.0I-_usiTRLpI1DB-RKeIDUyxHE4rJsB_0 2024-06-28T15:27:12.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13476551789143544042', time=2024-06-28T17:23:17.659117+0000, chain.objs.size()=1 2024-06-28T15:27:12.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.lybOFihDtpB5DmDFa6e30NENiEK2Bbo_0 2024-06-28T15:27:12.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9965827643396135276', time=2024-06-28T17:23:57.020184+0000, chain.objs.size()=1 2024-06-28T15:27:12.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.6WwNB3W691Q00DfRuMMCMBQeLtw6XyH_0 2024-06-28T15:27:12.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12352006914749902106', time=2024-06-28T17:24:35.752085+0000, chain.objs.size()=1 2024-06-28T15:27:12.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.LTNptOos4HIatUu01qre_QBvXeYMQyi_0 2024-06-28T15:27:12.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3459113314602310204', time=2024-06-28T17:24:35.754381+0000, chain.objs.size()=1 2024-06-28T15:27:12.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.U-8RVVFiInaO7znOftE0W-knomMWscc_0 2024-06-28T15:27:12.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16524743776132254044', time=2024-06-28T17:25:14.324047+0000, chain.objs.size()=1 2024-06-28T15:27:12.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.TbNAqaQ2iHj97v0wwFXwy7zI_zDt8_u_0 2024-06-28T15:27:12.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1019176242082939061', time=2024-06-28T17:25:53.798630+0000, chain.objs.size()=1 2024-06-28T15:27:12.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.H3rCInPg5QqwTMjlopo8X04s5_gM1l7_0 2024-06-28T15:27:12.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16882812227058687757', time=2024-06-28T17:25:53.913349+0000, chain.objs.size()=1 2024-06-28T15:27:12.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.699+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.BnbBH4rfXVVEXixGswHRhz_HTBN2sSF_0 2024-06-28T15:27:12.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.707+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.711+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-06-28T15:27:12.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-06-28T15:27:12.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6827930523031767619', time=2024-06-28T17:23:17.701044+0000, chain.objs.size()=1 2024-06-28T15:27:12.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.xULUEChm9x5xTwebrGi6w6jFLwZcJ3o_0 2024-06-28T15:27:12.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10197818479482175770', time=2024-06-28T17:23:17.704442+0000, chain.objs.size()=1 2024-06-28T15:27:12.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.kn4R_MyQ7IS_QRz5kBBwSAzsV92jLzd_0 2024-06-28T15:27:12.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6091405208926865489', time=2024-06-28T17:24:31.608768+0000, chain.objs.size()=1 2024-06-28T15:27:12.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.PvNHuVzWjamfeFdHwYn3mwRxZYUnSDr_0 2024-06-28T15:27:12.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4822807438667463007', time=2024-06-28T17:25:17.859820+0000, chain.objs.size()=1 2024-06-28T15:27:12.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.KCb9312QFW7GWxDHaOZ-eluvgmvosFU_0 2024-06-28T15:27:12.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1598369290620085037', time=2024-06-28T17:25:18.906252+0000, chain.objs.size()=1 2024-06-28T15:27:12.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.zRzjupSO9LdD4X8FydxhkWmCzabLzZy_0 2024-06-28T15:27:12.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17655234660518034377', time=2024-06-28T17:25:53.900519+0000, chain.objs.size()=1 2024-06-28T15:27:12.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.715+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.SCd9G5qfaLGaUEiu0rUeHXDJYM3DZKf_0 2024-06-28T15:27:12.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.719+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-06-28T15:27:12.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-06-28T15:27:12.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11204990728759701358', time=2024-06-28T17:21:54.481405+0000, chain.objs.size()=1 2024-06-28T15:27:12.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.3h8nBofYqfTXfnIUkfD5984-BBtf2Gz_0 2024-06-28T15:27:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17424450910814204230', time=2024-06-28T17:23:52.509046+0000, chain.objs.size()=1 2024-06-28T15:27:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.o1oy6lO5CX34Oq4pX6ReDv2b7JkIJ1g_0 2024-06-28T15:27:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10689845992172025444', time=2024-06-28T17:23:52.511285+0000, chain.objs.size()=1 2024-06-28T15:27:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.yOy33eqLz6PsfczUWXEJvtrv5jPE96R_0 2024-06-28T15:27:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15436730482921925436', time=2024-06-28T17:24:31.699839+0000, chain.objs.size()=1 2024-06-28T15:27:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.ijWa0Ueg1a9uxzg5jRFq0VhqX0Rsu73_0 2024-06-28T15:27:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15317793901945802848', time=2024-06-28T17:24:35.791246+0000, chain.objs.size()=1 2024-06-28T15:27:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.mmFDPrsJ8XZY29fQYRhy1my27mPn-s8_0 2024-06-28T15:27:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17379215340641808323', time=2024-06-28T17:24:39.141001+0000, chain.objs.size()=1 2024-06-28T15:27:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.iQk04nPlIkfBnKJx86HdDnj-QF6aP5K_0 2024-06-28T15:27:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15633300680611548527', time=2024-06-28T17:25:19.006438+0000, chain.objs.size()=1 2024-06-28T15:27:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.IvzN5j6AiYbvZ6MIeOjV8IRg8iha_Xc_0 2024-06-28T15:27:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5926691857599167846', time=2024-06-28T17:25:53.827262+0000, chain.objs.size()=1 2024-06-28T15:27:12.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.727+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.J4sZOxlMkLXxsjpeKPOuJ3f2QyTDt5W_0 2024-06-28T15:27:12.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.735+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.739+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-06-28T15:27:12.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=15, truncated=0, next_marker='' 2024-06-28T15:27:12.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13889709452861551557', time=2024-06-28T17:21:54.445279+0000, chain.objs.size()=4 2024-06-28T15:27:12.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.10__multipart_foo.2~ciuZ3mMrUjxgn1vBVm8kyCvgCYx1c42.1 2024-06-28T15:27:12.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.10__multipart_foo.2~ciuZ3mMrUjxgn1vBVm8kyCvgCYx1c42.2 2024-06-28T15:27:12.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.10__multipart_foo.2~ciuZ3mMrUjxgn1vBVm8kyCvgCYx1c42.3 2024-06-28T15:27:12.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.10__multipart_foo.2~ciuZ3mMrUjxgn1vBVm8kyCvgCYx1c42.4 2024-06-28T15:27:12.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2163557565277149311', time=2024-06-28T17:21:54.977221+0000, chain.objs.size()=1 2024-06-28T15:27:12.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.8__shadow_.97ZlBFmz2V3ll9O3l4nMwEiPBkrCtnW_0 2024-06-28T15:27:12.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2834129547698130132', time=2024-06-28T17:23:13.694278+0000, chain.objs.size()=1 2024-06-28T15:27:12.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.YXVUSiLJC_exMh4K7npyUwA4XtfcuI2_0 2024-06-28T15:27:12.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1121088794539404714', time=2024-06-28T17:23:17.765696+0000, chain.objs.size()=1 2024-06-28T15:27:12.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.duEaSvE9Hu8Qj-UGH64HtgZ9YlKvPx7_0 2024-06-28T15:27:12.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5541009593853598628', time=2024-06-28T17:23:56.950405+0000, chain.objs.size()=1 2024-06-28T15:27:12.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.mx4JfKw5XKJa3zZV0vOUUAKRIvbD06t_0 2024-06-28T15:27:12.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3790194692126865068', time=2024-06-28T17:23:57.022735+0000, chain.objs.size()=1 2024-06-28T15:27:12.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.Z33EvfXhZMN7nmCVuVavIH9zIpGif8b_0 2024-06-28T15:27:12.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3366815093757007872', time=2024-06-28T17:24:00.198008+0000, chain.objs.size()=1 2024-06-28T15:27:12.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.xsLQe3xTWctrOc1diqahHycYVIfASPk_0 2024-06-28T15:27:12.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17063900993049171421', time=2024-06-28T17:24:35.549741+0000, chain.objs.size()=1 2024-06-28T15:27:12.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.nOawaU6b-GA0tz3KorQaxPdOUW8cWiW_0 2024-06-28T15:27:12.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7394287439517949440', time=2024-06-28T17:24:35.748100+0000, chain.objs.size()=1 2024-06-28T15:27:12.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.743+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.BV_s8y0iKqrXawAF4w8IqMx0T5WtWLt_0 2024-06-28T15:27:12.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.751+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6789872266596328937', time=2024-06-28T17:24:35.750236+0000, chain.objs.size()=1 2024-06-28T15:27:12.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.751+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.FaBczXvB0EOBlS7cv7mYZTbJDNW_two_0 2024-06-28T15:27:12.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.751+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3667494597279120619', time=2024-06-28T17:24:39.058138+0000, chain.objs.size()=1 2024-06-28T15:27:12.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.751+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.BpWRuLLBDV9zrXiwyHztwoUsAqNpy7D_0 2024-06-28T15:27:12.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.751+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18383701071147480563', time=2024-06-28T17:25:14.358541+0000, chain.objs.size()=1 2024-06-28T15:27:12.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.751+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.1iV5XM_WzGWA0c6551DIJfuEtFl8LZf_0 2024-06-28T15:27:12.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.755+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4991359962455242154', time=2024-06-28T17:25:19.004703+0000, chain.objs.size()=1 2024-06-28T15:27:12.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.755+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.qvx_VoOVm78aoxBIyXHbUJniBG0NDD__0 2024-06-28T15:27:12.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.755+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2767906180717229985', time=2024-06-28T17:25:19.063162+0000, chain.objs.size()=1 2024-06-28T15:27:12.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.755+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.hKnisJyE-anZ_gTran6DEwtHEan7UEZ_0 2024-06-28T15:27:12.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.755+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15157868226783317261', time=2024-06-28T17:26:09.773283+0000, chain.objs.size()=1 2024-06-28T15:27:12.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.755+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.19__shadow:C96XzHCwwnwar1U1P7A6llj1R1EiRF6_.UpdGl8uyhjk3AXGszg7GMnp5W1uwlQi_0 2024-06-28T15:27:12.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.759+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.763+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-06-28T15:27:12.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.767+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.767+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.767+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-06-28T15:27:12.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.767+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5764869312466433341', time=2024-06-28T17:21:54.257971+0000, chain.objs.size()=1 2024-06-28T15:27:12.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.767+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.M7EjadUhwoSAtTf53ukLTRQF3iozu5A_0 2024-06-28T15:27:12.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.767+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.707320261432521826', time=2024-06-28T17:23:57.136096+0000, chain.objs.size()=1 2024-06-28T15:27:12.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.767+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.yTgwNhWyCBwRHwCXAaBl8BKRhL_AoRs_0 2024-06-28T15:27:12.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.767+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6505636214846337351', time=2024-06-28T17:25:18.893681+0000, chain.objs.size()=1 2024-06-28T15:27:12.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.767+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.G0tAzUGwFLkiNH4jQmXBvtoZwTslKfl_0 2024-06-28T15:27:12.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.771+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.775+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-06-28T15:27:12.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.775+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.775+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.775+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-06-28T15:27:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.775+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17043748717799448144', time=2024-06-28T17:21:54.996711+0000, chain.objs.size()=1 2024-06-28T15:27:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.775+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.8__shadow_.GXaGwg-JTGEkPfW_ypUGwT7Do7wSvf0_0 2024-06-28T15:27:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.775+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5708589557636172717', time=2024-06-28T17:21:55.067569+0000, chain.objs.size()=1 2024-06-28T15:27:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.775+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.9__shadow_.xMwOjFVEFVekjC9hn-Y-lDanvSRb0wf_0 2024-06-28T15:27:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.775+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9933587204060776803', time=2024-06-28T17:23:13.656969+0000, chain.objs.size()=1 2024-06-28T15:27:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.775+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.RKsGdVFmjX2rnw8_vG1JHOsXnW1VC7G_0 2024-06-28T15:27:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.775+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13507466219964123193', time=2024-06-28T17:23:17.730458+0000, chain.objs.size()=1 2024-06-28T15:27:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.775+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.50aKZSAl-ExY93UW-lW6gJyS69v1coB_0 2024-06-28T15:27:12.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.775+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17744363138482017048', time=2024-06-28T17:24:38.996880+0000, chain.objs.size()=1 2024-06-28T15:27:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.775+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.B2sIOgH0yVPZM2fDFufA_u4o0ear2iz_0 2024-06-28T15:27:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.779+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3289053617014002590', time=2024-06-28T17:25:14.262739+0000, chain.objs.size()=1 2024-06-28T15:27:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.779+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.bdKv4c9aDXAORcYtLBOEGxz_-0dVcFh_0 2024-06-28T15:27:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.779+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5567509749445248810', time=2024-06-28T17:25:53.966846+0000, chain.objs.size()=1 2024-06-28T15:27:12.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.779+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.ADj9qLYt43FCJM9znokcq9iFG6sqm7T_0 2024-06-28T15:27:12.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.783+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.787+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-06-28T15:27:12.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.787+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.787+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-06-28T15:27:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='_vlahhlCU5vSdYyZi3QWYG_byviuBrE9', time=2024-06-28T17:21:54.500605+0000, chain.objs.size()=1 2024-06-28T15:27:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.M57Kjtb2oYS-KjgBnBasqr2QasddNuR_0 2024-06-28T15:27:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12299554465292202902', time=2024-06-28T17:23:52.627542+0000, chain.objs.size()=1 2024-06-28T15:27:12.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.ckfpmzmDZX1v1ZFaOCAvk9cwGr6ejje_0 2024-06-28T15:27:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2399985324131896112', time=2024-06-28T17:24:31.696289+0000, chain.objs.size()=1 2024-06-28T15:27:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.boHHU-TH3Rf6-l-T4GYB9v47pZ_MyOn_0 2024-06-28T15:27:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16846433071162527463', time=2024-06-28T17:25:14.364867+0000, chain.objs.size()=1 2024-06-28T15:27:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.1rsc3xhosC9ljO-WENLjM1B7IrKNvau_0 2024-06-28T15:27:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5112841975718456896', time=2024-06-28T17:25:18.880478+0000, chain.objs.size()=1 2024-06-28T15:27:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.fHtRGKeDd1Jn7tlacCRprT__h-9qd7Y_0 2024-06-28T15:27:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15875806572276304097', time=2024-06-28T17:25:53.831838+0000, chain.objs.size()=1 2024-06-28T15:27:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.3xWOISVB_6e6s7kAJa_mUpKloBwd19n_0 2024-06-28T15:27:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9189168127117737169', time=2024-06-28T17:25:53.965862+0000, chain.objs.size()=1 2024-06-28T15:27:12.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.791+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.R2vWcoIJB-i2JCKYtwaDN48VR7qdqeH_0 2024-06-28T15:27:12.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.795+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.799+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-06-28T15:27:12.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.799+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-06-28T15:27:12.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='_4CudkS5lgEZ8zloP_ToCgshC19S6yZY', time=2024-06-28T17:21:54.552094+0000, chain.objs.size()=1 2024-06-28T15:27:12.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.xxk4_Dc-vtTqwKoHhv8M6X0Wl-DWaYh_0 2024-06-28T15:27:12.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16802073543734993788', time=2024-06-28T17:23:13.687136+0000, chain.objs.size()=1 2024-06-28T15:27:12.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.CgKzC8d8DizTvCguEO-WpYf_9sqZoV2_0 2024-06-28T15:27:12.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7818644086168958464', time=2024-06-28T17:23:13.689292+0000, chain.objs.size()=1 2024-06-28T15:27:12.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.qlo1m8wAdNxVP3Q51cw9uEz2aCICd2D_0 2024-06-28T15:27:12.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3704054744513476559', time=2024-06-28T17:23:13.691179+0000, chain.objs.size()=1 2024-06-28T15:27:12.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.JgMv1OpnsS2rkofYMkdgoj5EjkshBOF_0 2024-06-28T15:27:12.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7881143650856630432', time=2024-06-28T17:23:57.111636+0000, chain.objs.size()=1 2024-06-28T15:27:12.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.b3VtdFhBL5r3i3Oe4npo-KtQ1oE7W2L_0 2024-06-28T15:27:12.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9367472978748322577', time=2024-06-28T17:24:39.114680+0000, chain.objs.size()=1 2024-06-28T15:27:12.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.98weQyuLO5pY7BUH1afidJLXyR8N_kl_0 2024-06-28T15:27:12.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1051212241522672387', time=2024-06-28T17:25:14.268986+0000, chain.objs.size()=1 2024-06-28T15:27:12.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.m_zWyBfEZqHjLMwrc3WmmHV2LTVcugE_0 2024-06-28T15:27:12.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12714408205540569769', time=2024-06-28T17:25:18.869393+0000, chain.objs.size()=1 2024-06-28T15:27:12.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.24b7R8Qul_iB3wMp2U_a1Y4iBvFxQYB_0 2024-06-28T15:27:12.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5624776836936347485', time=2024-06-28T17:25:55.572248+0000, chain.objs.size()=1 2024-06-28T15:27:12.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.803+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.18__shadow_.moP0Z-jeOdpDadA8ptYlIxDlF3ifxEv_0 2024-06-28T15:27:12.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.811+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.811+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-06-28T15:27:12.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.815+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.815+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.815+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-06-28T15:27:12.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.815+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12695288585948761914', time=2024-06-28T17:21:54.281684+0000, chain.objs.size()=1 2024-06-28T15:27:12.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.815+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.DCdAK61jqs8FF8IgYo6F8mlU4Wn11Un_0 2024-06-28T15:27:12.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.815+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10454808907365715507', time=2024-06-28T17:23:17.673603+0000, chain.objs.size()=1 2024-06-28T15:27:12.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.815+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.YobcBQL3RErQZco2Y8DiEAUuj_VXjBQ_0 2024-06-28T15:27:12.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.815+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15258130889001666232', time=2024-06-28T17:23:17.865611+0000, chain.objs.size()=1 2024-06-28T15:27:12.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.815+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.NIHAQ9eWrVc71hJGiOG-C-wxnDrTXs0_0 2024-06-28T15:27:12.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.815+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2786377650312281580', time=2024-06-28T17:25:18.945605+0000, chain.objs.size()=1 2024-06-28T15:27:12.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.815+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.bj3vECgw3mjL5nRq0xqF43hNPx4RCVS_0 2024-06-28T15:27:12.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.819+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.823+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-06-28T15:27:12.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=12, truncated=0, next_marker='' 2024-06-28T15:27:12.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='_2pL3HWtQ7O3f6zEI9QjB7PvdTs8Wcq6', time=2024-06-28T17:21:54.666044+0000, chain.objs.size()=2 2024-06-28T15:27:12.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.8Yroa_OsiFNVbq-v8nKUsVhPyWEjPpP_0 2024-06-28T15:27:12.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.8Yroa_OsiFNVbq-v8nKUsVhPyWEjPpP_1 2024-06-28T15:27:12.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.2_16a036810e9433481d99fdfb9e1f2f8d', time=2024-06-28T17:21:54.753908+0000, chain.objs.size()=3 2024-06-28T15:27:12.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.1 2024-06-28T15:27:12.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.2 2024-06-28T15:27:12.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.3 2024-06-28T15:27:12.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.555452763890355875', time=2024-06-28T17:23:13.656802+0000, chain.objs.size()=1 2024-06-28T15:27:12.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.Sus8kMuDJ7DVtwwKe2d-u5yxXvChv2l_0 2024-06-28T15:27:12.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8908934681202319005', time=2024-06-28T17:23:17.730632+0000, chain.objs.size()=1 2024-06-28T15:27:12.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.fEbiXeIXc3P6OtiklJSMWQxcyP6N3tk_0 2024-06-28T15:27:12.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.400802928253590557', time=2024-06-28T17:23:52.341662+0000, chain.objs.size()=1 2024-06-28T15:27:12.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.ep_X2Y5_Vw9iuv_q_elf53zTsJ8dzsT_0 2024-06-28T15:27:12.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14979463913669267965', time=2024-06-28T17:23:56.930873+0000, chain.objs.size()=1 2024-06-28T15:27:12.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.kC4gU_oqLTD1paAAMUsN08mZIufiNvZ_0 2024-06-28T15:27:12.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15657370136275308533', time=2024-06-28T17:24:31.784491+0000, chain.objs.size()=1 2024-06-28T15:27:12.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.FHtK9X37Gn7FWyDyYeDaw0njleuYEF0_0 2024-06-28T15:27:12.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13148438869282849149', time=2024-06-28T17:24:39.035542+0000, chain.objs.size()=1 2024-06-28T15:27:12.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.fwts7nctX8hnO_HeD1lgP5tVl7L9qum_0 2024-06-28T15:27:12.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10621155170792982026', time=2024-06-28T17:25:14.341229+0000, chain.objs.size()=1 2024-06-28T15:27:12.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.827+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_._jP2uprkS4zbcok8IvvSf9ugZ2m6a0R_0 2024-06-28T15:27:12.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.831+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2195532618654026497', time=2024-06-28T17:25:19.006752+0000, chain.objs.size()=1 2024-06-28T15:27:12.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.831+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.2P0iBaW17OnFfGZbd3NrwARvhnHGV2B_0 2024-06-28T15:27:12.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.835+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17328010863827444133', time=2024-06-28T17:25:53.825548+0000, chain.objs.size()=1 2024-06-28T15:27:12.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.835+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.k__6EJ_urydFTVvO0HlaeKFPEtDLyoN_0 2024-06-28T15:27:12.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.835+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12421608495366114148', time=2024-06-28T17:26:09.759492+0000, chain.objs.size()=1 2024-06-28T15:27:12.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.835+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.19__shadow:wWclF-yNVIJclWECK08hB.IzQuLG-0f_.QPPvacTGmlEqPcO9zKdVbT4GOjaD-fC_0 2024-06-28T15:27:12.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.839+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.843+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-06-28T15:27:12.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.843+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.843+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=9, truncated=0, next_marker='' 2024-06-28T15:27:12.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12149661540729176768', time=2024-06-28T17:21:54.232107+0000, chain.objs.size()=1 2024-06-28T15:27:12.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.OilZA6weLTPfLV0jLZG8MbHosg_dIhk_0 2024-06-28T15:27:12.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10061548790983698222', time=2024-06-28T17:21:54.265959+0000, chain.objs.size()=1 2024-06-28T15:27:12.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.j5xlxUvls8Q_iPS9jMuhRV65Wtrg-lm_0 2024-06-28T15:27:12.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1881584416359652293', time=2024-06-28T17:21:54.522162+0000, chain.objs.size()=1 2024-06-28T15:27:12.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.05oqeZkcZBXD7L5ZKW3Yg7ODJxNZ8DC_0 2024-06-28T15:27:12.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1680904577117860548', time=2024-06-28T17:21:54.865379+0000, chain.objs.size()=2 2024-06-28T15:27:12.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.14__shadow_.-1KUIHHeqH8ixt3tVlNOgPh1nuKG74M_0 2024-06-28T15:27:12.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.14__shadow_.-1KUIHHeqH8ixt3tVlNOgPh1nuKG74M_1 2024-06-28T15:27:12.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15913898295358819137', time=2024-06-28T17:23:17.699042+0000, chain.objs.size()=1 2024-06-28T15:27:12.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.4kXbUZsU_5JX0S1E8K_k1D407oB4Xzy_0 2024-06-28T15:27:12.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5326375127014836796', time=2024-06-28T17:23:52.615444+0000, chain.objs.size()=1 2024-06-28T15:27:12.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.Mt-q7gk1cnwyCrr1LZJ4CaswGgG6C_F_0 2024-06-28T15:27:12.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5958339994834550777', time=2024-06-28T17:23:56.883330+0000, chain.objs.size()=1 2024-06-28T15:27:12.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.iC7YF3PuPDRyW-Isl1J_dvwOnr6uoyA_0 2024-06-28T15:27:12.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10542183450842117293', time=2024-06-28T17:24:35.765705+0000, chain.objs.size()=1 2024-06-28T15:27:12.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.rKxJ5XVim66MVRTHJVoEoZeeRx1cAj__0 2024-06-28T15:27:12.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15838339679483126190', time=2024-06-28T17:25:54.025514+0000, chain.objs.size()=1 2024-06-28T15:27:12.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.847+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.9Sco-M7zor_Z5laRgiaJm43S8ub01CZ_0 2024-06-28T15:27:12.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.851+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-06-28T15:27:12.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-06-28T15:27:12.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7614661601685312247', time=2024-06-28T17:23:13.687272+0000, chain.objs.size()=1 2024-06-28T15:27:12.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.UsbYfINiQQ6XRXWtRdpGkAzPIxxbB5J_0 2024-06-28T15:27:12.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9282671293287315014', time=2024-06-28T17:23:17.906915+0000, chain.objs.size()=1 2024-06-28T15:27:12.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.OHY86TDoLfz-pWZcaw0Eb6xOv2UtYWU_0 2024-06-28T15:27:12.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5333488728708337611', time=2024-06-28T17:23:56.913099+0000, chain.objs.size()=1 2024-06-28T15:27:12.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.O51EUlDBZt6tsqC5YZpQe1O2KU4bpIX_0 2024-06-28T15:27:12.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8977568619589363359', time=2024-06-28T17:24:31.627322+0000, chain.objs.size()=1 2024-06-28T15:27:12.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.g1ws62ZMkDRwMM_x_9qYqXouBUtJQyM_0 2024-06-28T15:27:12.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16135048931412524025', time=2024-06-28T17:24:39.112998+0000, chain.objs.size()=1 2024-06-28T15:27:12.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.k-VVG-6Go_iUMNiMReoStNP4At83JaK_0 2024-06-28T15:27:12.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9126935891876473259', time=2024-06-28T17:25:55.613503+0000, chain.objs.size()=1 2024-06-28T15:27:12.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.855+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.18__shadow_.lGL12n6n4q2g1p8sauwCIsANC9TGBqI_0 2024-06-28T15:27:12.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.863+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.863+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-06-28T15:27:12.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=10, truncated=0, next_marker='' 2024-06-28T15:27:12.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='_XLf-MakSHGTjxFKDyWrzLBCYGLStajN', time=2024-06-28T17:21:54.691906+0000, chain.objs.size()=2 2024-06-28T15:27:12.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.IWQLiw9c7uiRkdJmU5Eqz_8U3xfa2BA_0 2024-06-28T15:27:12.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.IWQLiw9c7uiRkdJmU5Eqz_8U3xfa2BA_1 2024-06-28T15:27:12.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.175067232702430969', time=2024-06-28T17:21:55.007360+0000, chain.objs.size()=1 2024-06-28T15:27:12.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.8__shadow_.AZ0NaE_t4-pu0BYAuqLXpV6kCj-rjHL_0 2024-06-28T15:27:12.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.480262186642331680', time=2024-06-28T17:23:17.670325+0000, chain.objs.size()=1 2024-06-28T15:27:12.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.orl63OumJZ-riiMlzU-6Kxe1nGDyTRu_0 2024-06-28T15:27:12.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3746323871377437409', time=2024-06-28T17:24:34.715769+0000, chain.objs.size()=1 2024-06-28T15:27:12.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.GWylQmIDnP1RdiQjseaNDr26CSjCl0P_0 2024-06-28T15:27:12.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3557478215637722262', time=2024-06-28T17:24:39.259202+0000, chain.objs.size()=1 2024-06-28T15:27:12.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.5CYkHuJnr-lxkrCePD5s97oGUPJJ6eh_0 2024-06-28T15:27:12.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15897739056681650714', time=2024-06-28T17:24:39.281042+0000, chain.objs.size()=1 2024-06-28T15:27:12.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_._73LQ3WhvXQMFyCYXtIv6f3zSlfeDqf_0 2024-06-28T15:27:12.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17366584808136713935', time=2024-06-28T17:25:14.345420+0000, chain.objs.size()=1 2024-06-28T15:27:12.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.vIUm-euHrn7ko0ufcAov49z0P4dPWXt_0 2024-06-28T15:27:12.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14413553453937313450', time=2024-06-28T17:25:14.410604+0000, chain.objs.size()=1 2024-06-28T15:27:12.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.jRD_NGHfDSueev824DqZ6wdDBXeYfRz_0 2024-06-28T15:27:12.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15809023770689844767', time=2024-06-28T17:25:18.920601+0000, chain.objs.size()=1 2024-06-28T15:27:12.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.87fVN5-dz3FlDqHBbU9081aBkkLjw_8_0 2024-06-28T15:27:12.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4805857724400998943', time=2024-06-28T17:25:19.039627+0000, chain.objs.size()=1 2024-06-28T15:27:12.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.867+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.sb_zuWrcWS8FMO0vnFLakMnZkVGQ3uQ_0 2024-06-28T15:27:12.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.875+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.879+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-06-28T15:27:12.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.879+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.879+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=10, truncated=0, next_marker='' 2024-06-28T15:27:12.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12560414832874831080', time=2024-06-28T17:21:54.222270+0000, chain.objs.size()=1 2024-06-28T15:27:12.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.kMtjaixVfqqQVLy5VOIi9lWcG6uo-8i_0 2024-06-28T15:27:12.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14468871253119589124', time=2024-06-28T17:21:54.355624+0000, chain.objs.size()=4 2024-06-28T15:27:12.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.13__multipart_foo.2~rKM6r06SyAbjSQW5B-r_P28bJ4olQt5.1 2024-06-28T15:27:12.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.13__multipart_foo.2~rKM6r06SyAbjSQW5B-r_P28bJ4olQt5.2 2024-06-28T15:27:12.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.13__multipart_foo.2~rKM6r06SyAbjSQW5B-r_P28bJ4olQt5.3 2024-06-28T15:27:12.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.13__multipart_foo.2~rKM6r06SyAbjSQW5B-r_P28bJ4olQt5.4 2024-06-28T15:27:12.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7561639209902354137', time=2024-06-28T17:23:17.700011+0000, chain.objs.size()=1 2024-06-28T15:27:12.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.RQT1KtyAqlLHLURUhB5r5m49vGtnGjF_0 2024-06-28T15:27:12.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15048690332398842266', time=2024-06-28T17:23:17.703511+0000, chain.objs.size()=1 2024-06-28T15:27:12.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.qv7zsfPXGE5Pv917ypZwmdiiT7WRW7E_0 2024-06-28T15:27:12.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4036030175869176160', time=2024-06-28T17:23:52.520258+0000, chain.objs.size()=1 2024-06-28T15:27:12.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.zY4O09zmBu39rDOBUUk7tH75d7dXhuv_0 2024-06-28T15:27:12.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4972330108382369699', time=2024-06-28T17:24:31.598579+0000, chain.objs.size()=1 2024-06-28T15:27:12.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.W9MSiBErB2Dfdi-bpz2pWK9r0ProJWr_0 2024-06-28T15:27:12.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12653289956094377297', time=2024-06-28T17:24:31.623135+0000, chain.objs.size()=1 2024-06-28T15:27:12.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.KHfpOfPUR7Sk1cLrpELrR_bZDqVhpan_0 2024-06-28T15:27:12.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6535265682328278563', time=2024-06-28T17:24:31.789834+0000, chain.objs.size()=1 2024-06-28T15:27:12.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.NfA5IiNOVDqj657g74vCx0kRdnI79w__0 2024-06-28T15:27:12.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5349756315225861243', time=2024-06-28T17:25:18.862367+0000, chain.objs.size()=1 2024-06-28T15:27:12.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.OrkGcWugHiOYSMXdJVIFyR0ZSXnpsv5_0 2024-06-28T15:27:12.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3547234861816320746', time=2024-06-28T17:25:53.806055+0000, chain.objs.size()=1 2024-06-28T15:27:12.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.883+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.E5ydGb7KdrWu0VetN4J6SleyHWVrgOE_0 2024-06-28T15:27:12.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.891+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-06-28T15:27:12.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-06-28T15:27:12.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='_GkAN5wu5sM0-Wse6E7A9jzdb6QWLzPh', time=2024-06-28T17:21:54.712747+0000, chain.objs.size()=2 2024-06-28T15:27:12.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.UwlAV0X9NBMdkaVOr0E601-Niw-N0LA_0 2024-06-28T15:27:12.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.UwlAV0X9NBMdkaVOr0E601-Niw-N0LA_1 2024-06-28T15:27:12.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3382539424937402029', time=2024-06-28T17:21:54.724115+0000, chain.objs.size()=3 2024-06-28T15:27:12.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.1 2024-06-28T15:27:12.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.2 2024-06-28T15:27:12.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-STANDARD.2~68xLxCvO2NP7SA5wx5rQcToRdov7rZp.3 2024-06-28T15:27:12.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.36165453831064552', time=2024-06-28T17:21:55.057210+0000, chain.objs.size()=1 2024-06-28T15:27:12.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.9__shadow_.nj_8oRrkOL8CYVoJxq_pwy74Eg8CcvO_0 2024-06-28T15:27:12.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10257703035015562031', time=2024-06-28T17:24:31.595336+0000, chain.objs.size()=1 2024-06-28T15:27:12.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.crbl6vcQ9-GSBYohhrtUyhd5Deu4ch6_0 2024-06-28T15:27:12.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15122538296658567737', time=2024-06-28T17:25:18.862185+0000, chain.objs.size()=1 2024-06-28T15:27:12.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.xWaDwgb2qAo4vrtHQUIJuBTTnazE660_0 2024-06-28T15:27:12.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5341445915824837210', time=2024-06-28T17:25:18.985522+0000, chain.objs.size()=1 2024-06-28T15:27:12.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.VEs2_XCajNbEahGqdeJfGjfu6xsZFoo_0 2024-06-28T15:27:12.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.26734190191034928', time=2024-06-28T17:25:55.632528+0000, chain.objs.size()=1 2024-06-28T15:27:12.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.895+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.18__shadow_.6SWVDicJlCKqDmTlzfgHMMgwLOjUD3l_0 2024-06-28T15:27:12.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.903+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-06-28T15:27:12.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=12, truncated=0, next_marker='' 2024-06-28T15:27:12.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2743304086958580136', time=2024-06-28T17:21:54.635047+0000, chain.objs.size()=3 2024-06-28T15:27:12.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.1 2024-06-28T15:27:12.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.2 2024-06-28T15:27:12.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.3 2024-06-28T15:27:12.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10817757572399398305', time=2024-06-28T17:23:13.590582+0000, chain.objs.size()=1 2024-06-28T15:27:12.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_._261w5Auc0veauh8eKMi-hrxGAKNGLx_0 2024-06-28T15:27:12.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5811631798411022144', time=2024-06-28T17:23:13.591970+0000, chain.objs.size()=1 2024-06-28T15:27:12.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.KDmWlpaipdfoV7300FRDSO5Q5-C5lP__0 2024-06-28T15:27:12.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5627155363340715220', time=2024-06-28T17:23:17.607847+0000, chain.objs.size()=1 2024-06-28T15:27:12.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.M-ysdDiCKfGG6M84f9KIe1dhynCAWBt_0 2024-06-28T15:27:12.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11215902816369701139', time=2024-06-28T17:23:20.995008+0000, chain.objs.size()=1 2024-06-28T15:27:12.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.Fbvfu2FXe40cE1GF87SJz2-xYxnf0kl_0 2024-06-28T15:27:12.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2036854748629840590', time=2024-06-28T17:23:52.514715+0000, chain.objs.size()=1 2024-06-28T15:27:12.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.H1_F3pc8Hz3ByfmuPRSeF7w-XaYmK-x_0 2024-06-28T15:27:12.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13044355696383372050', time=2024-06-28T17:23:56.953590+0000, chain.objs.size()=1 2024-06-28T15:27:12.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.y639LkttYOfq0-iEVe-lI-c2m4OFfVt_0 2024-06-28T15:27:12.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1242580033464565544', time=2024-06-28T17:23:57.029543+0000, chain.objs.size()=1 2024-06-28T15:27:12.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.asymFC1gHGzRoYnq-4M1wz4BVUKifg5_0 2024-06-28T15:27:12.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13212856866992818608', time=2024-06-28T17:24:31.696306+0000, chain.objs.size()=1 2024-06-28T15:27:12.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.nt8wbkZYRuiJdcSLHB5BA0UZ_0l6X-V_0 2024-06-28T15:27:12.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15725606737686003856', time=2024-06-28T17:24:39.045469+0000, chain.objs.size()=1 2024-06-28T15:27:12.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.907+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.mUmtDB8JfwnNNt4NfoxNQstuTEzbwrP_0 2024-06-28T15:27:12.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.911+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4802712879639931364', time=2024-06-28T17:24:39.123762+0000, chain.objs.size()=1 2024-06-28T15:27:12.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.911+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.x3upZBIFSIirPBm97cVHVNfJNxZq5bm_0 2024-06-28T15:27:12.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.911+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10478356913411945800', time=2024-06-28T17:25:19.055439+0000, chain.objs.size()=1 2024-06-28T15:27:12.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.911+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.xnjbyGQQ7j_p7SYAjsX4QAH93Q2TiYV_0 2024-06-28T15:27:12.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.915+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.919+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-06-28T15:27:12.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.919+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.919+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.919+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-06-28T15:27:12.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.919+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16075280366673605200', time=2024-06-28T17:21:54.316902+0000, chain.objs.size()=4 2024-06-28T15:27:12.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.12__multipart_foo.2~xTHSyet_mXJEGmeACs0Br1TaRSbwvDn.1 2024-06-28T15:27:12.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.12__multipart_foo.2~xTHSyet_mXJEGmeACs0Br1TaRSbwvDn.2 2024-06-28T15:27:12.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.12__multipart_foo.2~xTHSyet_mXJEGmeACs0Br1TaRSbwvDn.3 2024-06-28T15:27:12.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.12__multipart_foo.2~xTHSyet_mXJEGmeACs0Br1TaRSbwvDn.4 2024-06-28T15:27:12.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18391893484726116323', time=2024-06-28T17:21:55.047514+0000, chain.objs.size()=1 2024-06-28T15:27:12.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.9__shadow_.YeRwfleGbdAziA7fF6xOcgncV56Lqrq_0 2024-06-28T15:27:12.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10640296016279267333', time=2024-06-28T17:23:13.692887+0000, chain.objs.size()=1 2024-06-28T15:27:12.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.n_rm9tVruWD3ltip58k0YAWb_KC8gqj_0 2024-06-28T15:27:12.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2624402555337367767', time=2024-06-28T17:23:52.467896+0000, chain.objs.size()=1 2024-06-28T15:27:12.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.OPbrIbTDWTOJOLwCM-HlUt0JguUyWT3_0 2024-06-28T15:27:12.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3980019410503156957', time=2024-06-28T17:24:31.653149+0000, chain.objs.size()=1 2024-06-28T15:27:12.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.N6G1giXnFRy87A_o3gtHVFRFoyaqWzw_0 2024-06-28T15:27:12.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11140206927970051526', time=2024-06-28T17:24:39.213578+0000, chain.objs.size()=1 2024-06-28T15:27:12.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.i8xgar2FhsolQyTV4YyyHJwe-Ki8R59_0 2024-06-28T15:27:12.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15855083839301213194', time=2024-06-28T17:25:54.046617+0000, chain.objs.size()=1 2024-06-28T15:27:12.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.923+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.UKvdL2ZSuU2s1KYFBe1DaNznrRqMhva_0 2024-06-28T15:27:12.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.927+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.931+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-06-28T15:27:12.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.931+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.931+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.931+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=12, truncated=0, next_marker='' 2024-06-28T15:27:12.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.931+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.2_59e5b182d51295a5faf47e285dd68abe', time=2024-06-28T17:21:54.701331+0000, chain.objs.size()=3 2024-06-28T15:27:12.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.931+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.1 2024-06-28T15:27:12.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.931+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.2 2024-06-28T15:27:12.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.3 2024-06-28T15:27:12.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8489236377501723306', time=2024-06-28T17:21:54.875848+0000, chain.objs.size()=1 2024-06-28T15:27:12.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.14__shadow_.HRrjuKiJfl1-hn79QLqwfDgeRPybSoP_0 2024-06-28T15:27:12.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16806047856251082129', time=2024-06-28T17:23:52.599620+0000, chain.objs.size()=1 2024-06-28T15:27:12.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.dtigxh_s1i0OpB9djb67MIpGy29geUY_0 2024-06-28T15:27:12.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3003351032909572368', time=2024-06-28T17:23:56.923509+0000, chain.objs.size()=1 2024-06-28T15:27:12.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.hSZHXv43lc8MGiI4V25G9GgStKeCmM-_0 2024-06-28T15:27:12.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4565859679470390314', time=2024-06-28T17:24:31.697964+0000, chain.objs.size()=1 2024-06-28T15:27:12.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.lZ3LwridSQ6M0ULnAOluq8s66_Peuti_0 2024-06-28T15:27:12.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12659251146767201145', time=2024-06-28T17:24:35.789636+0000, chain.objs.size()=1 2024-06-28T15:27:12.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.oAUANuxesHQ_ntIO5KaElssdIy8-1T3_0 2024-06-28T15:27:12.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4111887515500382766', time=2024-06-28T17:25:14.340836+0000, chain.objs.size()=1 2024-06-28T15:27:12.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.d7L24U0z-lH-1-Rpy8QZdlihBBPqfrG_0 2024-06-28T15:27:12.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17288977985786348707', time=2024-06-28T17:25:14.479906+0000, chain.objs.size()=1 2024-06-28T15:27:12.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.dhwoSw6wzaBttiAdsIzow5b-jO8BvPZ_0 2024-06-28T15:27:12.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2101153785552199308', time=2024-06-28T17:25:53.825396+0000, chain.objs.size()=1 2024-06-28T15:27:12.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.r4r-u32PgsI1J6NWmQu4wSmZ5feUch8_0 2024-06-28T15:27:12.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4280190358000476609', time=2024-06-28T17:25:53.895888+0000, chain.objs.size()=1 2024-06-28T15:27:12.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.kCMzlW23hUKa9BjBToFtqFfqIJl7--v_0 2024-06-28T15:27:12.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6823119731171679471', time=2024-06-28T17:25:55.591787+0000, chain.objs.size()=1 2024-06-28T15:27:12.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.935+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.18__shadow_.-ws4Y5ji1XJEfpu-ZSMRUXM6YBljErC_0 2024-06-28T15:27:12.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.939+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2668860927915768932', time=2024-06-28T17:26:09.782126+0000, chain.objs.size()=1 2024-06-28T15:27:12.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.939+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.19__shadow:jV.e7v8RhX3FCIQqOG0fvdTct09MF.5_.ykUjsB1zucalcuNby21_jpnPQkjnRLD_0 2024-06-28T15:27:12.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.939+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.943+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-06-28T15:27:12.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.943+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.943+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-06-28T15:27:12.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4187850353238860256', time=2024-06-28T17:21:54.240121+0000, chain.objs.size()=1 2024-06-28T15:27:12.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.xLxxFeFKU69d8QatRUyjVrtraYDc29Y_0 2024-06-28T15:27:12.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9725363603752228366', time=2024-06-28T17:21:54.273724+0000, chain.objs.size()=1 2024-06-28T15:27:12.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.T4mH8dfZRqRVP1yOxCm9zYyoNNRWRua_0 2024-06-28T15:27:12.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='_dnTwCM1lBSW-j3HTwp92fx5BeGNM0-X', time=2024-06-28T17:21:54.734223+0000, chain.objs.size()=2 2024-06-28T15:27:12.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.rjPary_F-RFuLp1X7o4lYwIO8gv0gPg_0 2024-06-28T15:27:12.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.rjPary_F-RFuLp1X7o4lYwIO8gv0gPg_1 2024-06-28T15:27:12.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12565385435276146935', time=2024-06-28T17:21:54.854243+0000, chain.objs.size()=2 2024-06-28T15:27:12.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.14__shadow_.u9cb5U65wT55S3n5QR45N7N_TbXC4EC_0 2024-06-28T15:27:12.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.14__shadow_.u9cb5U65wT55S3n5QR45N7N_TbXC4EC_1 2024-06-28T15:27:12.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16025556445391654542', time=2024-06-28T17:23:52.470356+0000, chain.objs.size()=1 2024-06-28T15:27:12.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.UYBjgEI2zHM-XDN4DqoM18GzmbU4fKi_0 2024-06-28T15:27:12.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3122471321811946841', time=2024-06-28T17:24:39.113109+0000, chain.objs.size()=1 2024-06-28T15:27:12.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.XpUNcETiHcD2-hBYETNgXNcDS9KqBIe_0 2024-06-28T15:27:12.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10174733420846134564', time=2024-06-28T17:24:39.116451+0000, chain.objs.size()=1 2024-06-28T15:27:12.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.oJKllp_z5fFzQYRNSb8GnOq04RkFWxp_0 2024-06-28T15:27:12.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16502472998128029984', time=2024-06-28T17:25:14.266815+0000, chain.objs.size()=1 2024-06-28T15:27:12.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.947+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.IpCxyobuE_C7qRSGAW5C2NF9Cplz4Sg_0 2024-06-28T15:27:12.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.951+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-06-28T15:27:12.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-06-28T15:27:12.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1783670138229329555', time=2024-06-28T17:21:54.987191+0000, chain.objs.size()=1 2024-06-28T15:27:12.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.8__shadow_.sfhDN1Hh3xzn35Xuq_7hRU225YRVCQ5_0 2024-06-28T15:27:12.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13561022630684745619', time=2024-06-28T17:23:13.690846+0000, chain.objs.size()=1 2024-06-28T15:27:12.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.5DOUZSX9x65T-t9tLh_t5fBPrsqVmz1_0 2024-06-28T15:27:12.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9538647083041773611', time=2024-06-28T17:23:17.782601+0000, chain.objs.size()=1 2024-06-28T15:27:12.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.DHX1Kb988vmr770m1xawP3w1Z2ncizG_0 2024-06-28T15:27:12.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9537121320477323976', time=2024-06-28T17:23:52.570122+0000, chain.objs.size()=1 2024-06-28T15:27:12.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.VZweZD7c5HWCpVHRr8kZlQW9X7Tt_O5_0 2024-06-28T15:27:12.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16707452322257019683', time=2024-06-28T17:24:35.610357+0000, chain.objs.size()=1 2024-06-28T15:27:12.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.i3R0VjmGsR9bVYofx1okwu7NFrrFcz-_0 2024-06-28T15:27:12.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15077734101344684105', time=2024-06-28T17:24:39.000158+0000, chain.objs.size()=1 2024-06-28T15:27:12.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.m7hpSZobdMl0uE-8rLli7Dvg0h35WPZ_0 2024-06-28T15:27:12.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14286881078108909198', time=2024-06-28T17:25:54.024326+0000, chain.objs.size()=1 2024-06-28T15:27:12.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.LQMY4tkSMHUGcjWAuZTCsqjcs4vEJ-o_0 2024-06-28T15:27:12.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10316802713493333622', time=2024-06-28T17:25:55.631977+0000, chain.objs.size()=1 2024-06-28T15:27:12.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.955+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.18__shadow_.RGyn-WvTpKddKz9gDZc7PxDfFgdscws_0 2024-06-28T15:27:12.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.959+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.963+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-06-28T15:27:12.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-06-28T15:27:12.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17870332517152078614', time=2024-06-28T17:21:54.401163+0000, chain.objs.size()=4 2024-06-28T15:27:12.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.11__multipart_foo.2~Rlj2fxm8bYt9LC8GIdaJSFx-t7whYsg.1 2024-06-28T15:27:12.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.11__multipart_foo.2~Rlj2fxm8bYt9LC8GIdaJSFx-t7whYsg.2 2024-06-28T15:27:12.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.11__multipart_foo.2~Rlj2fxm8bYt9LC8GIdaJSFx-t7whYsg.3 2024-06-28T15:27:12.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.11__multipart_foo.2~Rlj2fxm8bYt9LC8GIdaJSFx-t7whYsg.4 2024-06-28T15:27:12.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='_XdGhjbaiCXQ-Jjz24bkSGLEwTMQoFQ4', time=2024-06-28T17:21:54.511424+0000, chain.objs.size()=1 2024-06-28T15:27:12.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.8ycHKyODPBp9Ax5XUWBD8C5AqbYFdtg_0 2024-06-28T15:27:12.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.2_724698bb6f712eb05c7aa6a9df77043f', time=2024-06-28T17:21:54.645725+0000, chain.objs.size()=3 2024-06-28T15:27:12.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.1 2024-06-28T15:27:12.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.2 2024-06-28T15:27:12.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-FROZEN.2~eHKplF7qdyt6I1T6pDfMPH-LJXRgod2.3 2024-06-28T15:27:12.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14317980836071081796', time=2024-06-28T17:21:54.793543+0000, chain.objs.size()=3 2024-06-28T15:27:12.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-FROZEN.2~rT2zkyXXyEwXPN-6nXcB-u7-as7cHJB.1 2024-06-28T15:27:12.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-FROZEN.2~rT2zkyXXyEwXPN-6nXcB-u7-as7cHJB.2 2024-06-28T15:27:12.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-FROZEN.2~rT2zkyXXyEwXPN-6nXcB-u7-as7cHJB.3 2024-06-28T15:27:12.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10259449070996476782', time=2024-06-28T17:23:13.691789+0000, chain.objs.size()=1 2024-06-28T15:27:12.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.967+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.VemkYQcyaSecq6Xt2cbkxPcgLFqG2sC_0 2024-06-28T15:27:12.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.971+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4414984245532560373', time=2024-06-28T17:23:52.467824+0000, chain.objs.size()=1 2024-06-28T15:27:12.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.971+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_._xKtbLsFg8gGs32sNhyaZ8W0y_g1zBz_0 2024-06-28T15:27:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.971+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13243089444944515311', time=2024-06-28T17:24:39.028133+0000, chain.objs.size()=1 2024-06-28T15:27:12.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.971+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.dhOZSKmkwIZo_DdroFrvbrp302oLKY-_0 2024-06-28T15:27:12.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.975+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-06-28T15:27:12.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=11, truncated=0, next_marker='' 2024-06-28T15:27:12.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3043720328400159613', time=2024-06-28T17:21:54.214105+0000, chain.objs.size()=1 2024-06-28T15:27:12.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.lP5DhIrL25P6OG_3BLMGtP_hqdIBdeI_0 2024-06-28T15:27:12.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3611825967390322888', time=2024-06-28T17:21:54.249200+0000, chain.objs.size()=1 2024-06-28T15:27:12.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.5__shadow_.ZBcakGelu7-Kf8eg6D68hQOzlkoMwP7_0 2024-06-28T15:27:12.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2114521308000306919', time=2024-06-28T17:21:54.561973+0000, chain.objs.size()=1 2024-06-28T15:27:12.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.0_BMyaWYD_CRK7ZjgmtPTdbTcrC3iWs_0 2024-06-28T15:27:12.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3602318093819116136', time=2024-06-28T17:23:13.773015+0000, chain.objs.size()=1 2024-06-28T15:27:12.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.TEB1yioplJcG7G0TM34A_VffA-3iR0I_0 2024-06-28T15:27:12.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1547553638090257380', time=2024-06-28T17:23:56.923509+0000, chain.objs.size()=1 2024-06-28T15:27:12.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.0PCa5SDPpfSyfpqpL5ZNzBpDs5CqDkJ_0 2024-06-28T15:27:12.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5864489232094149900', time=2024-06-28T17:23:56.925645+0000, chain.objs.size()=1 2024-06-28T15:27:12.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.59cFeUQc8eagh6afXEUU4MI8ZGnykZk_0 2024-06-28T15:27:12.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4093044865484499680', time=2024-06-28T17:24:35.832798+0000, chain.objs.size()=1 2024-06-28T15:27:12.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.QhJmLJK5Sn3iW3Lw2y6D94uM2mvqvsI_0 2024-06-28T15:27:12.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17487641581950193817', time=2024-06-28T17:24:39.035517+0000, chain.objs.size()=1 2024-06-28T15:27:12.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.Rnw9-V9FzYVnZ7Ucrrh4w1SEufOpAKb_0 2024-06-28T15:27:12.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12088925389596102033', time=2024-06-28T17:24:39.140993+0000, chain.objs.size()=1 2024-06-28T15:27:12.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.ujcc-6j8RXGPJH6_wa2_9QsRzc8XpEE_0 2024-06-28T15:27:12.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14407732439209058724', time=2024-06-28T17:25:52.414251+0000, chain.objs.size()=1 2024-06-28T15:27:12.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.2Xh7UvDBxqdudj639cfsUYtJE2W54Xn_0 2024-06-28T15:27:12.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17958869062386146853', time=2024-06-28T17:25:53.825141+0000, chain.objs.size()=1 2024-06-28T15:27:12.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.979+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.sEkF_G6Rfi4LXscrRJsb4rkfCb9_uhM_0 2024-06-28T15:27:12.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.987+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.987+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-06-28T15:27:12.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-06-28T15:27:12.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8424947470339001389', time=2024-06-28T17:23:13.641041+0000, chain.objs.size()=1 2024-06-28T15:27:12.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.jNeOGtvVT73BqM6H51GysS5ymmbDNuP_0 2024-06-28T15:27:12.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6283824749953201691', time=2024-06-28T17:23:17.782763+0000, chain.objs.size()=1 2024-06-28T15:27:12.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.21__shadow_.jUyyrWACCjKg6Xi56NS1bNkUJKd5jhX_0 2024-06-28T15:27:12.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10973661984243374806', time=2024-06-28T17:24:35.659166+0000, chain.objs.size()=1 2024-06-28T15:27:12.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.elew4KnFF37IWKenZBkM4D4JHJtgUY7_0 2024-06-28T15:27:12.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9756845140049968010', time=2024-06-28T17:24:35.749392+0000, chain.objs.size()=1 2024-06-28T15:27:12.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.fJwqdC9CTjtQUcAUlMYctXFU60oQKef_0 2024-06-28T15:27:12.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4726336858519262156', time=2024-06-28T17:24:35.828996+0000, chain.objs.size()=1 2024-06-28T15:27:12.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.NcQUykpGsnoZ7P_hT08XJMApuoMaEjv_0 2024-06-28T15:27:12.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6534274503494129189', time=2024-06-28T17:25:14.344442+0000, chain.objs.size()=1 2024-06-28T15:27:12.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.991+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.JOlfwY7IoTCWOSaaCJJckBbZneeWj4W_0 2024-06-28T15:27:12.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.995+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-06-28T15:27:12.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-06-28T15:27:12.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8666825984778865762', time=2024-06-28T17:21:54.678704+0000, chain.objs.size()=3 2024-06-28T15:27:12.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.1 2024-06-28T15:27:12.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.2 2024-06-28T15:27:12.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.17__multipart_foo-LUKEWARM.2~kaNC_3RNE3LmSpE9jIj5pRxYGLcPSHB.3 2024-06-28T15:27:12.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5965116731016975394', time=2024-06-28T17:23:12.766001+0000, chain.objs.size()=1 2024-06-28T15:27:12.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.75AV89IXy0D1-iJGF788ATXOcA2Zmmw_0 2024-06-28T15:27:12.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14887352999960993552', time=2024-06-28T17:23:52.472511+0000, chain.objs.size()=1 2024-06-28T15:27:12.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.LG4B7n6u9D9vNFnL0RjnpS6mTmXgMgv_0 2024-06-28T15:27:12.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14434988843527767827', time=2024-06-28T17:23:57.020116+0000, chain.objs.size()=1 2024-06-28T15:27:12.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.23__shadow_.XkL4rRxljjdIvzGxgFvkdGfELg9K-Dp_0 2024-06-28T15:27:12.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18202059617997877848', time=2024-06-28T17:24:31.536983+0000, chain.objs.size()=1 2024-06-28T15:27:12.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.c1gjLE3Yr10KE0AUsv1QOdyrMbq-IVa_0 2024-06-28T15:27:12.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6312629658866140893', time=2024-06-28T17:24:31.627328+0000, chain.objs.size()=1 2024-06-28T15:27:12.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.24__shadow_.fYLFLIDji0MoYao-F5l4AGPkLhblbZ3_0 2024-06-28T15:27:12.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4073252326625693857', time=2024-06-28T17:24:35.661337+0000, chain.objs.size()=1 2024-06-28T15:27:12.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.25__shadow_.NYaPMq_4GGdSjLP-iJLpNIEEfbL79pL_0 2024-06-28T15:27:12.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12984308280858114822', time=2024-06-28T17:25:53.913206+0000, chain.objs.size()=1 2024-06-28T15:27:12.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:12.999+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.z5TMBSwrZMJIdVp4VtpLafCIJRuKtHE_0 2024-06-28T15:27:12.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.003+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:12.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.007+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-06-28T15:27:12.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.007+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:12.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:12.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=10, truncated=0, next_marker='' 2024-06-28T15:27:12.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.0_BMyaWYD_CRK7ZjgmtPTdbTcrC3iWs_1_73aac645e1e440535c1c09f0714b79ce', time=2024-06-28T17:21:54.593475+0000, chain.objs.size()=1 2024-06-28T15:27:12.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.16__shadow_.0_BMyaWYD_CRK7ZjgmtPTdbTcrC3iWs_0 2024-06-28T15:27:12.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3749436943869302853', time=2024-06-28T17:21:54.804170+0000, chain.objs.size()=3 2024-06-28T15:27:12.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-LUKEWARM.2~0yRDU9th93HT14K4ALzXZT0n3impj1A.1 2024-06-28T15:27:12.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-LUKEWARM.2~0yRDU9th93HT14K4ALzXZT0n3impj1A.2 2024-06-28T15:27:12.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.15__multipart_foo-LUKEWARM.2~0yRDU9th93HT14K4ALzXZT0n3impj1A.3 2024-06-28T15:27:12.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11184735148000703322', time=2024-06-28T17:23:13.641001+0000, chain.objs.size()=1 2024-06-28T15:27:12.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.wGGYD8-YjWEgY9t-7_NqGWvxrEoPDUg_0 2024-06-28T15:27:12.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5014287695208096580', time=2024-06-28T17:23:13.793040+0000, chain.objs.size()=1 2024-06-28T15:27:12.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.R4_E7s4jKIJwlVM60g5itfplEDiN2_O_0 2024-06-28T15:27:12.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10509041516586527632', time=2024-06-28T17:23:13.794607+0000, chain.objs.size()=1 2024-06-28T15:27:12.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.20__shadow_.J5NiYRxGLIe6hm7h8m7kSdH8SeuRYDu_0 2024-06-28T15:27:12.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3901515231465950278', time=2024-06-28T17:23:52.464894+0000, chain.objs.size()=1 2024-06-28T15:27:12.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.22__shadow_.prXLWMIixNRwqIzRINIcEpJTBDkxDGf_0 2024-06-28T15:27:12.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10260126373311899323', time=2024-06-28T17:24:39.272192+0000, chain.objs.size()=1 2024-06-28T15:27:12.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.HeBHjpBeaQJSf0tZmDj4WAcLg8DkOjt_0 2024-06-28T15:27:12.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17619088499831039577', time=2024-06-28T17:25:14.273038+0000, chain.objs.size()=1 2024-06-28T15:27:12.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.27__shadow_.i0Xyil3iwuck86Ztjl_4pMKsNnzKRmG_0 2024-06-28T15:27:12.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16479729138658979130', time=2024-06-28T17:25:18.808928+0000, chain.objs.size()=1 2024-06-28T15:27:12.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.zEoF60IUfm3uFtITBVRNozfewvXLNuA_0 2024-06-28T15:27:12.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18411284005057585575', time=2024-06-28T17:25:55.634301+0000, chain.objs.size()=1 2024-06-28T15:27:12.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.011+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.18__shadow_.S5nAgQi5rlpGrK5KLTQxFmr5mMXDovA_0 2024-06-28T15:27:13.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.019+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:13.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.019+0000 7fed92427980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-06-28T15:27:13.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-06-28T15:27:13.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='_T11QvOyYfRM1_IJbyrkT5fpPOifhl8U', time=2024-06-28T17:21:54.656854+0000, chain.objs.size()=2 2024-06-28T15:27:13.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.DfNtUAZMT2iMHx7mS9oU5rVOV2zY7BZ_0 2024-06-28T15:27:13.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.DfNtUAZMT2iMHx7mS9oU5rVOV2zY7BZ_1 2024-06-28T15:27:13.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='__HbLmKkyqGHBakVveXe42oUiQiLzywV', time=2024-06-28T17:21:54.744582+0000, chain.objs.size()=2 2024-06-28T15:27:13.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.H0s5D5UYzconMghB09AAKEVcheSCumD_0 2024-06-28T15:27:13.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.17__shadow_.H0s5D5UYzconMghB09AAKEVcheSCumD_1 2024-06-28T15:27:13.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9911405742747893466', time=2024-06-28T17:21:55.079989+0000, chain.objs.size()=1 2024-06-28T15:27:13.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.9__shadow_.PCm6GjSJvqSjZyWzLIvOlNXVaJKxgrK_0 2024-06-28T15:27:13.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2197312273754532963', time=2024-06-28T17:24:39.179760+0000, chain.objs.size()=1 2024-06-28T15:27:13.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.aOLV4hS2ocNt0tzJvdKjltCpCRXjVnK_0 2024-06-28T15:27:13.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10434107509807641965', time=2024-06-28T17:24:39.181776+0000, chain.objs.size()=1 2024-06-28T15:27:13.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.26__shadow_.jXz5kz2DkwZm-XyERA6Y7--BWQpFg5D_0 2024-06-28T15:27:13.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17682092085022564710', time=2024-06-28T17:25:18.893587+0000, chain.objs.size()=1 2024-06-28T15:27:13.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.28__shadow_.u8YupxCXlkliYtdV2FgeQgEsxYcLn4E_0 2024-06-28T15:27:13.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.767572088935915118', time=2024-06-28T17:25:53.975695+0000, chain.objs.size()=1 2024-06-28T15:27:13.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.29__shadow_.0KX7aH5myrDYLOFvPTHO4fGG3J3KHns_0 2024-06-28T15:27:13.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4744878673786669478', time=2024-06-28T17:25:55.634312+0000, chain.objs.size()=1 2024-06-28T15:27:13.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.023+0000 7fed92427980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.18__shadow_.swya3Jy4ZYZbJVBQkYEEPHQr8hi5K1N_0 2024-06-28T15:27:13.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.027+0000 7fed92427980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:27:13.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.031+0000 7fec45fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:27:13.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.035+0000 7fed92427980 20 remove_watcher() i=0 2024-06-28T15:27:13.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.035+0000 7fed92427980 2 removed watcher, disabling cache 2024-06-28T15:27:13.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.035+0000 7fed92427980 20 remove_watcher() i=1 2024-06-28T15:27:13.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.039+0000 7fed92427980 20 remove_watcher() i=2 2024-06-28T15:27:13.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.039+0000 7fed92427980 20 remove_watcher() i=3 2024-06-28T15:27:13.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.039+0000 7fed92427980 20 remove_watcher() i=4 2024-06-28T15:27:13.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.043+0000 7fed92427980 20 remove_watcher() i=5 2024-06-28T15:27:13.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.043+0000 7fed92427980 20 remove_watcher() i=6 2024-06-28T15:27:13.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.043+0000 7fed92427980 20 remove_watcher() i=7 2024-06-28T15:27:13.040 DEBUG:teuthology.orchestra.run.smithi178:> radosgw-admin gc process --include-all 2024-06-28T15:27:13.113 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:27:13.114 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:27:13.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.175+0000 7fe8fad5d980 20 rados->read ofs=0 len=0 2024-06-28T15:27:13.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.179+0000 7fe8fad5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:13.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.179+0000 7fe8fad5d980 20 realm 2024-06-28T15:27:13.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.179+0000 7fe8fad5d980 20 rados->read ofs=0 len=0 2024-06-28T15:27:13.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.179+0000 7fe8fad5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:13.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.179+0000 7fe8fad5d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:27:13.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.179+0000 7fe8fad5d980 20 rados->read ofs=0 len=0 2024-06-28T15:27:13.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.179+0000 7fe8fad5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:13.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.179+0000 7fe8fad5d980 20 rados->read ofs=0 len=0 2024-06-28T15:27:13.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.179+0000 7fe8fad5d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:27:13.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.179+0000 7fe8fad5d980 20 rados->read ofs=0 len=0 2024-06-28T15:27:13.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.179+0000 7fe8fad5d980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:27:13.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.179+0000 7fe8fad5d980 20 searching for the correct realm 2024-06-28T15:27:13.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.183+0000 7fe8f886e640 0 -- 172.21.15.178:0/1459082001 >> [v2:172.21.15.178:6823/1880098663,v1:172.21.15.178:6825/1880098663] conn(0x7fe8f0018030 msgr2=0x7fe8f0057a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:27:13.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.191+0000 7fe8f906f640 0 -- 172.21.15.178:0/1459082001 >> [v2:172.21.15.195:6824/1278553753,v1:172.21.15.195:6825/1278553753] conn(0x7fe8ec02a040 msgr2=0x7fe8ec06c040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:27:13.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.199+0000 7fe8fad5d980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:27:13.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.199+0000 7fe8fad5d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:27:13.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.199+0000 7fe8fad5d980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:27:13.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.199+0000 7fe8fad5d980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:27:13.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.199+0000 7fe8fad5d980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:27:13.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.199+0000 7fe8fad5d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:27:13.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.199+0000 7fe8fad5d980 20 rados->read ofs=0 len=0 2024-06-28T15:27:13.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:13.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 rados->read ofs=0 len=0 2024-06-28T15:27:13.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:27:13.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 rados->read ofs=0 len=0 2024-06-28T15:27:13.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:27:13.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 zone default found 2024-06-28T15:27:13.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 4 Realm: () 2024-06-28T15:27:13.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:27:13.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:27:13.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:27:13.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 zonegroup default 2024-06-28T15:27:13.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 rados->read ofs=0 len=0 2024-06-28T15:27:13.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:13.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 rados->read ofs=0 len=0 2024-06-28T15:27:13.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:13.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 rados->read ofs=0 len=0 2024-06-28T15:27:13.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:27:13.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 started sync module instance, tier type = 2024-06-28T15:27:13.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.203+0000 7fe8fad5d980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:27:13.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.219+0000 7fe8fad5d980 20 add_watcher() i=0 2024-06-28T15:27:13.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.219+0000 7fe8fad5d980 20 add_watcher() i=1 2024-06-28T15:27:13.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.219+0000 7fe8fad5d980 20 add_watcher() i=2 2024-06-28T15:27:13.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.219+0000 7fe8fad5d980 20 add_watcher() i=3 2024-06-28T15:27:13.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.219+0000 7fe8fad5d980 20 add_watcher() i=4 2024-06-28T15:27:13.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.219+0000 7fe8fad5d980 20 add_watcher() i=5 2024-06-28T15:27:13.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.219+0000 7fe8fad5d980 20 add_watcher() i=6 2024-06-28T15:27:13.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.223+0000 7fe8fad5d980 20 add_watcher() i=7 2024-06-28T15:27:13.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.223+0000 7fe8fad5d980 2 all 8 watchers are set, enabling cache 2024-06-28T15:27:13.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.223+0000 7fe7aefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:27:13.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.223+0000 7fe8fad5d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:27:13.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.223+0000 7fe8fad5d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:27:13.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.295+0000 7fe7adfe3640 20 reqs_thread_entry: start 2024-06-28T15:27:13.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.295+0000 7fe8fad5d980 20 init_complete bucket index max shards: 1 2024-06-28T15:27:13.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.295+0000 7fe8fad5d980 20 Filter name: none 2024-06-28T15:27:13.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.295+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-06-28T15:27:13.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.295+0000 7fe71f7fe640 20 reqs_thread_entry: start 2024-06-28T15:27:13.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.299+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.299+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.299+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.303+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-06-28T15:27:13.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.307+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.307+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.311+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.311+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-06-28T15:27:13.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.315+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.315+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.315+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.319+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-06-28T15:27:13.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.319+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.323+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.323+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.327+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-06-28T15:27:13.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.331+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.331+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.331+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.331+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-06-28T15:27:13.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.335+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.335+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.339+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.339+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-06-28T15:27:13.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.343+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.347+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.347+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.351+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-06-28T15:27:13.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.351+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.355+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.355+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.355+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-06-28T15:27:13.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.359+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.363+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.363+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.367+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-06-28T15:27:13.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.367+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.371+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.371+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.375+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-06-28T15:27:13.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.375+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.379+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.379+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.383+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-06-28T15:27:13.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.387+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.387+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.387+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.391+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-06-28T15:27:13.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.395+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.395+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.395+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.399+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-06-28T15:27:13.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.403+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.403+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.403+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.407+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-06-28T15:27:13.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.411+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.411+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.411+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.415+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-06-28T15:27:13.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.419+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.419+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.419+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.423+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-06-28T15:27:13.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.427+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.427+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.427+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.427+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-06-28T15:27:13.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.431+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.435+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.435+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.435+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-06-28T15:27:13.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.439+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.439+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.439+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.443+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-06-28T15:27:13.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.447+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.447+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.447+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.451+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-06-28T15:27:13.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.455+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.455+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.455+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.459+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-06-28T15:27:13.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.463+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.463+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.463+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.467+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-06-28T15:27:13.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.471+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.471+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.471+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.475+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-06-28T15:27:13.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.479+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.479+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.479+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.483+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-06-28T15:27:13.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.487+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.487+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.487+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.491+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-06-28T15:27:13.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.495+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.495+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.499+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.499+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-06-28T15:27:13.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.503+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.503+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.507+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.507+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-06-28T15:27:13.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.511+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.515+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.515+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.519+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-06-28T15:27:13.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.523+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.523+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.523+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.527+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-06-28T15:27:13.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.531+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.531+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.535+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.535+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-06-28T15:27:13.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.539+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.543+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.543+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.543+0000 7fe8fad5d980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-06-28T15:27:13.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.547+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.551+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:27:13.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.551+0000 7fe8fad5d980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:27:13.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.555+0000 7fe7aefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:27:13.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.563+0000 7fe8fad5d980 20 remove_watcher() i=0 2024-06-28T15:27:13.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.563+0000 7fe8fad5d980 2 removed watcher, disabling cache 2024-06-28T15:27:13.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.563+0000 7fe8fad5d980 20 remove_watcher() i=1 2024-06-28T15:27:13.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.567+0000 7fe8fad5d980 20 remove_watcher() i=2 2024-06-28T15:27:13.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.571+0000 7fe8fad5d980 20 remove_watcher() i=3 2024-06-28T15:27:13.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.571+0000 7fe8fad5d980 20 remove_watcher() i=4 2024-06-28T15:27:13.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.575+0000 7fe8fad5d980 20 remove_watcher() i=5 2024-06-28T15:27:13.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.575+0000 7fe8fad5d980 20 remove_watcher() i=6 2024-06-28T15:27:13.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:27:13.579+0000 7fe8fad5d980 20 remove_watcher() i=7 2024-06-28T15:27:13.585 DEBUG:teuthology.orchestra.run.smithi178:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test 2024-06-28T15:27:14.181 INFO:teuthology.orchestra.run.smithi178.stdout:Starting a Gradle Daemon (subsequent builds will be faster) 2024-06-28T15:27:19.481 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :clean UP-TO-DATE 2024-06-28T15:27:49.480 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :compileJava 2024-06-28T15:27:56.980 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :processResources 2024-06-28T15:27:56.980 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :classes 2024-06-28T15:27:57.680 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:27:57.680 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :compileTestJava 2024-06-28T15:27:57.681 INFO:teuthology.orchestra.run.smithi178.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-06-28T15:27:57.681 INFO:teuthology.orchestra.run.smithi178.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-06-28T15:27:57.682 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:27:57.682 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :processTestResources NO-SOURCE 2024-06-28T15:27:57.682 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :testClasses 2024-06-28T15:28:03.780 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:03.780 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :test 2024-06-28T15:28:03.780 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:03.781 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 STARTED 2024-06-28T15:28:11.780 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:11.780 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 PASSED 2024-06-28T15:28:12.280 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:12.280 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 STARTED 2024-06-28T15:28:12.580 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:12.580 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 PASSED 2024-06-28T15:28:13.080 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:13.080 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 STARTED 2024-06-28T15:28:23.680 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:23.681 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 PASSED 2024-06-28T15:28:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:24.381 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 STARTED 2024-06-28T15:28:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:24.381 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 PASSED 2024-06-28T15:28:24.580 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:24.580 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 STARTED 2024-06-28T15:28:24.780 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:24.780 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 PASSED 2024-06-28T15:28:25.280 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:25.280 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 STARTED 2024-06-28T15:28:33.681 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:33.681 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 PASSED 2024-06-28T15:28:34.780 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:34.781 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STARTED 2024-06-28T15:28:35.180 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:28:35.181 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STANDARD_OUT 2024-06-28T15:28:35.181 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28 15:28:35,184 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-06-28T15:29:13.183 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:29:13.183 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 PASSED 2024-06-28T15:29:14.181 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:29:14.181 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 STARTED 2024-06-28T15:29:14.381 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:29:14.381 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 PASSED 2024-06-28T15:29:14.981 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:29:14.981 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 STARTED 2024-06-28T15:29:15.281 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:29:15.281 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 PASSED 2024-06-28T15:29:16.181 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:29:16.181 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 STARTED 2024-06-28T15:29:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:29:23.782 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 PASSED 2024-06-28T15:29:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:29:24.381 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 STARTED 2024-06-28T15:29:24.581 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:29:24.581 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 PASSED 2024-06-28T15:29:24.681 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:29:24.681 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STARTED 2024-06-28T15:29:35.681 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:29:35.682 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 PASSED 2024-06-28T15:29:36.481 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:29:36.481 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 STARTED 2024-06-28T15:29:53.982 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:29:53.982 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 PASSED 2024-06-28T15:29:54.981 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:29:54.982 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 STARTED 2024-06-28T15:30:03.182 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:30:03.182 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 PASSED 2024-06-28T15:30:03.781 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:30:03.782 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 STARTED 2024-06-28T15:32:05.183 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:05.184 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 PASSED 2024-06-28T15:32:05.782 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:05.783 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 STARTED 2024-06-28T15:32:24.683 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:24.683 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 PASSED 2024-06-28T15:32:25.283 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:25.283 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 STARTED 2024-06-28T15:32:33.183 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:33.183 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 PASSED 2024-06-28T15:32:33.783 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:33.783 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 STARTED 2024-06-28T15:32:37.083 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:37.083 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 PASSED 2024-06-28T15:32:48.683 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:48.683 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 STARTED 2024-06-28T15:32:49.084 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:49.084 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 PASSED 2024-06-28T15:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 STARTED 2024-06-28T15:32:52.383 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:52.384 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 PASSED 2024-06-28T15:32:52.983 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:52.983 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 STARTED 2024-06-28T15:32:53.183 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:53.183 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 PASSED 2024-06-28T15:32:53.883 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:53.883 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 STARTED 2024-06-28T15:32:54.083 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:54.083 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 PASSED 2024-06-28T15:32:54.683 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:54.683 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 STARTED 2024-06-28T15:32:54.983 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:54.983 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 PASSED 2024-06-28T15:32:55.683 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:55.683 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 STARTED 2024-06-28T15:32:55.883 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:55.883 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 PASSED 2024-06-28T15:32:56.483 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:56.483 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 STARTED 2024-06-28T15:32:56.783 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:56.783 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 PASSED 2024-06-28T15:32:57.383 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:32:57.383 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 STARTED 2024-06-28T15:33:37.385 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:37.385 INFO:teuthology.orchestra.run.smithi178.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 PASSED 2024-06-28T15:33:38.061 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:38.061 INFO:teuthology.orchestra.run.smithi178.stdout:---------------------------------------------------------------------- 2024-06-28T15:33:38.061 INFO:teuthology.orchestra.run.smithi178.stdout:| Results: SUCCESS (26 tests, 26 successes, 0 failures, 0 skipped) | 2024-06-28T15:33:38.061 INFO:teuthology.orchestra.run.smithi178.stdout:---------------------------------------------------------------------- 2024-06-28T15:33:38.061 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:38.061 INFO:teuthology.orchestra.run.smithi178.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-06-28T15:33:38.061 INFO:teuthology.orchestra.run.smithi178.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-06-28T15:33:38.061 INFO:teuthology.orchestra.run.smithi178.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-06-28T15:33:38.061 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:38.061 INFO:teuthology.orchestra.run.smithi178.stdout:BUILD SUCCESSFUL in 6m 24s 2024-06-28T15:33:38.062 INFO:teuthology.orchestra.run.smithi178.stdout:5 actionable tasks: 4 executed, 1 up-to-date 2024-06-28T15:33:38.383 DEBUG:teuthology.orchestra.run.smithi178:> radosgw-admin gc process --include-all 2024-06-28T15:33:38.414 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:33:38.418 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:33:38.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.469+0000 7f49e2bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:33:38.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.469+0000 7f49e2bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:38.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.469+0000 7f49e2bc5980 20 realm 2024-06-28T15:33:38.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.469+0000 7f49e2bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:33:38.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.469+0000 7f49e2bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:38.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.469+0000 7f49e2bc5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:33:38.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.469+0000 7f49e2bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:33:38.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.469+0000 7f49e2bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:38.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.469+0000 7f49e2bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:33:38.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.473+0000 7f49e2bc5980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:38.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.473+0000 7f49e2bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:33:38.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.473+0000 7f49e2bc5980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:33:38.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.473+0000 7f49e2bc5980 20 searching for the correct realm 2024-06-28T15:33:38.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.477+0000 7f49dfed5640 0 -- 172.21.15.178:0/1608197402 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x560e90fe40c0 msgr2=0x560e90fe64d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:33:38.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.485+0000 7f49e06d6640 0 -- 172.21.15.178:0/1608197402 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f49d400c910 msgr2=0x7f49d400d3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:33:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:33:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:33:38.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:33:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:33:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:33:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:33:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:33:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:33:38.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:33:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:33:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 zone default found 2024-06-28T15:33:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 4 Realm: () 2024-06-28T15:33:38.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:33:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:33:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:33:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 zonegroup default 2024-06-28T15:33:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:33:38.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.493+0000 7f49e2bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:33:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.497+0000 7f49e2bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.497+0000 7f49e2bc5980 20 rados->read ofs=0 len=0 2024-06-28T15:33:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.497+0000 7f49e2bc5980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.497+0000 7f49e2bc5980 20 started sync module instance, tier type = 2024-06-28T15:33:38.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.497+0000 7f49e2bc5980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:33:38.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.513+0000 7f49e2bc5980 20 add_watcher() i=0 2024-06-28T15:33:38.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.513+0000 7f49e2bc5980 20 add_watcher() i=1 2024-06-28T15:33:38.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.513+0000 7f49e2bc5980 20 add_watcher() i=2 2024-06-28T15:33:38.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.513+0000 7f49e2bc5980 20 add_watcher() i=3 2024-06-28T15:33:38.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.513+0000 7f49e2bc5980 20 add_watcher() i=4 2024-06-28T15:33:38.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.513+0000 7f49e2bc5980 20 add_watcher() i=5 2024-06-28T15:33:38.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.513+0000 7f49e2bc5980 20 add_watcher() i=6 2024-06-28T15:33:38.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.513+0000 7f49e2bc5980 20 add_watcher() i=7 2024-06-28T15:33:38.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.513+0000 7f49e2bc5980 2 all 8 watchers are set, enabling cache 2024-06-28T15:33:38.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.517+0000 7f4896fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:38.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.517+0000 7f49e2bc5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:33:38.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.517+0000 7f49e2bc5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:33:38.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.589+0000 7f4895fdb640 20 reqs_thread_entry: start 2024-06-28T15:33:38.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.589+0000 7f49e2bc5980 20 init_complete bucket index max shards: 1 2024-06-28T15:33:38.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.589+0000 7f49e2bc5980 20 Filter name: none 2024-06-28T15:33:38.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.589+0000 7f48077fe640 20 reqs_thread_entry: start 2024-06-28T15:33:38.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.593+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-06-28T15:33:38.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.593+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.597+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.597+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.597+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2343478123233640546', time=2024-06-28T17:29:13.030527+0000, chain.objs.size()=5 2024-06-28T15:33:38.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.597+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~wN9MUYNmvAxOTdMdZaA2PxIjdyRuk4I.1 2024-06-28T15:33:38.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.597+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__shadow_key1.2~wN9MUYNmvAxOTdMdZaA2PxIjdyRuk4I.1_1 2024-06-28T15:33:38.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.597+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~wN9MUYNmvAxOTdMdZaA2PxIjdyRuk4I.2 2024-06-28T15:33:38.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.597+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__shadow_key1.2~wN9MUYNmvAxOTdMdZaA2PxIjdyRuk4I.2_1 2024-06-28T15:33:38.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.597+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~wN9MUYNmvAxOTdMdZaA2PxIjdyRuk4I.3 2024-06-28T15:33:38.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.605+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.613+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-06-28T15:33:38.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-06-28T15:33:38.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13426349069750560704', time=2024-06-28T17:29:36.049363+0000, chain.objs.size()=5 2024-06-28T15:33:38.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.42__multipart_key1.2~HJdX44afghVsUauWacL4f2XZQ4KdkPF.1 2024-06-28T15:33:38.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.42__multipart_key1.2~HJdX44afghVsUauWacL4f2XZQ4KdkPF.2 2024-06-28T15:33:38.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.42__multipart_key1.2~HJdX44afghVsUauWacL4f2XZQ4KdkPF.3 2024-06-28T15:33:38.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.42__multipart_key1.2~HJdX44afghVsUauWacL4f2XZQ4KdkPF.4 2024-06-28T15:33:38.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.42__multipart_key1.2~HJdX44afghVsUauWacL4f2XZQ4KdkPF.5 2024-06-28T15:33:38.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8261751336671434088', time=2024-06-28T17:31:47.243749+0000, chain.objs.size()=11 2024-06-28T15:33:38.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~vW4nYFMMwrmy1L95rn77wUDkX7tVNGw.1 2024-06-28T15:33:38.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~vW4nYFMMwrmy1L95rn77wUDkX7tVNGw.1_1 2024-06-28T15:33:38.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~vW4nYFMMwrmy1L95rn77wUDkX7tVNGw.2 2024-06-28T15:33:38.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~vW4nYFMMwrmy1L95rn77wUDkX7tVNGw.2_1 2024-06-28T15:33:38.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~vW4nYFMMwrmy1L95rn77wUDkX7tVNGw.3 2024-06-28T15:33:38.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~vW4nYFMMwrmy1L95rn77wUDkX7tVNGw.3_1 2024-06-28T15:33:38.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.617+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~vW4nYFMMwrmy1L95rn77wUDkX7tVNGw.4 2024-06-28T15:33:38.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.629+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~vW4nYFMMwrmy1L95rn77wUDkX7tVNGw.4_1 2024-06-28T15:33:38.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.629+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~vW4nYFMMwrmy1L95rn77wUDkX7tVNGw.5 2024-06-28T15:33:38.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.629+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~vW4nYFMMwrmy1L95rn77wUDkX7tVNGw.5_1 2024-06-28T15:33:38.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.629+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~vW4nYFMMwrmy1L95rn77wUDkX7tVNGw.6 2024-06-28T15:33:38.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.649+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.653+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-06-28T15:33:38.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.657+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.657+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.657+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.657+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4847031065536244336', time=2024-06-28T17:32:53.458427+0000, chain.objs.size()=1 2024-06-28T15:33:38.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.657+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.51__shadow_.QzJ7cg2aygGkRRTp_ny6PnGgtjCXQIu_0 2024-06-28T15:33:38.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.657+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.661+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-06-28T15:33:38.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-06-28T15:33:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='2~EZHxVitBY7mqrQa9xtPc_EaAx6DDLDc', time=2024-06-28T17:28:11.663059+0000, chain.objs.size()=5 2024-06-28T15:33:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.30__multipart_key1.2~EZHxVitBY7mqrQa9xtPc_EaAx6DDLDc.1 2024-06-28T15:33:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.30__multipart_key1.2~EZHxVitBY7mqrQa9xtPc_EaAx6DDLDc.2 2024-06-28T15:33:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.30__multipart_key1.2~EZHxVitBY7mqrQa9xtPc_EaAx6DDLDc.3 2024-06-28T15:33:38.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.30__multipart_key1.2~EZHxVitBY7mqrQa9xtPc_EaAx6DDLDc.4 2024-06-28T15:33:38.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.30__multipart_key1.2~EZHxVitBY7mqrQa9xtPc_EaAx6DDLDc.5 2024-06-28T15:33:38.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10558508038464164904', time=2024-06-28T17:30:58.377063+0000, chain.objs.size()=11 2024-06-28T15:33:38.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~uUuTnMU03j5OsERmnZK514qrMBhoWzM.1 2024-06-28T15:33:38.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~uUuTnMU03j5OsERmnZK514qrMBhoWzM.2 2024-06-28T15:33:38.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~uUuTnMU03j5OsERmnZK514qrMBhoWzM.3 2024-06-28T15:33:38.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~uUuTnMU03j5OsERmnZK514qrMBhoWzM.4 2024-06-28T15:33:38.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~uUuTnMU03j5OsERmnZK514qrMBhoWzM.5 2024-06-28T15:33:38.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~uUuTnMU03j5OsERmnZK514qrMBhoWzM.6 2024-06-28T15:33:38.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.665+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~uUuTnMU03j5OsERmnZK514qrMBhoWzM.7 2024-06-28T15:33:38.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.673+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~uUuTnMU03j5OsERmnZK514qrMBhoWzM.8 2024-06-28T15:33:38.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.673+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~uUuTnMU03j5OsERmnZK514qrMBhoWzM.9 2024-06-28T15:33:38.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.673+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~uUuTnMU03j5OsERmnZK514qrMBhoWzM.10 2024-06-28T15:33:38.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.677+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~uUuTnMU03j5OsERmnZK514qrMBhoWzM.11 2024-06-28T15:33:38.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.677+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11950192410702859581', time=2024-06-28T17:32:05.393355+0000, chain.objs.size()=11 2024-06-28T15:33:38.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.677+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~RnrAyWj_GiFRUswT-98imYD4rI8p_s1.1 2024-06-28T15:33:38.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.677+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~RnrAyWj_GiFRUswT-98imYD4rI8p_s1.1_1 2024-06-28T15:33:38.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.677+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~RnrAyWj_GiFRUswT-98imYD4rI8p_s1.2 2024-06-28T15:33:38.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.677+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~RnrAyWj_GiFRUswT-98imYD4rI8p_s1.2_1 2024-06-28T15:33:38.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.681+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~RnrAyWj_GiFRUswT-98imYD4rI8p_s1.3 2024-06-28T15:33:38.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.681+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~RnrAyWj_GiFRUswT-98imYD4rI8p_s1.3_1 2024-06-28T15:33:38.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.681+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~RnrAyWj_GiFRUswT-98imYD4rI8p_s1.4 2024-06-28T15:33:38.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.685+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~RnrAyWj_GiFRUswT-98imYD4rI8p_s1.4_1 2024-06-28T15:33:38.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.685+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~RnrAyWj_GiFRUswT-98imYD4rI8p_s1.5 2024-06-28T15:33:38.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.685+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~RnrAyWj_GiFRUswT-98imYD4rI8p_s1.5_1 2024-06-28T15:33:38.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.689+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~RnrAyWj_GiFRUswT-98imYD4rI8p_s1.6 2024-06-28T15:33:38.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.697+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.701+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-06-28T15:33:38.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.705+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.705+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.705+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.705+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15746891782917244156', time=2024-06-28T17:32:22.896900+0000, chain.objs.size()=1 2024-06-28T15:33:38.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.705+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.46__shadow_.zCdqG6Srob59_8bRSJW4Pt94WfC664X_0 2024-06-28T15:33:38.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.709+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.709+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-06-28T15:33:38.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.713+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.713+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.713+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.713+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18296049536620132672', time=2024-06-28T17:32:55.228065+0000, chain.objs.size()=1 2024-06-28T15:33:38.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.713+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.53__shadow_.jnUyxgD8FrWPY9EybslDZSCZM8Zewww_0 2024-06-28T15:33:38.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.717+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-06-28T15:33:38.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-06-28T15:33:38.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12518708510044623364', time=2024-06-28T17:28:23.932131+0000, chain.objs.size()=5 2024-06-28T15:33:38.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.32__multipart_key1.2~-56D7_E46yEksFK-r_szUS5hSUiyJVL.1 2024-06-28T15:33:38.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.32__multipart_key1.2~-56D7_E46yEksFK-r_szUS5hSUiyJVL.2 2024-06-28T15:33:38.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.32__multipart_key1.2~-56D7_E46yEksFK-r_szUS5hSUiyJVL.3 2024-06-28T15:33:38.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.32__multipart_key1.2~-56D7_E46yEksFK-r_szUS5hSUiyJVL.4 2024-06-28T15:33:38.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.32__multipart_key1.2~-56D7_E46yEksFK-r_szUS5hSUiyJVL.5 2024-06-28T15:33:38.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14051910535751652725', time=2024-06-28T17:29:54.160932+0000, chain.objs.size()=5 2024-06-28T15:33:38.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.43__multipart_file.mpg.2~pIy-kLEssfzwd9C1ZzJqeYCiYLcl-xx.1 2024-06-28T15:33:38.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.43__multipart_file.mpg.2~pIy-kLEssfzwd9C1ZzJqeYCiYLcl-xx.2 2024-06-28T15:33:38.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.43__multipart_file.mpg.2~pIy-kLEssfzwd9C1ZzJqeYCiYLcl-xx.3 2024-06-28T15:33:38.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.43__multipart_file.mpg.2~pIy-kLEssfzwd9C1ZzJqeYCiYLcl-xx.4 2024-06-28T15:33:38.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.721+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.43__multipart_file.mpg.2~pIy-kLEssfzwd9C1ZzJqeYCiYLcl-xx.5 2024-06-28T15:33:38.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.733+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.737+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-06-28T15:33:38.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.737+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.737+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.737+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-06-28T15:33:38.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.737+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.34__multipart_key1.2~oKX-O3W1arIbBze_K1WtOxq9R3qJDlu.2_9688acb05e0d8ef0d663600db23f2137', time=2024-06-28T17:28:33.883093+0000, chain.objs.size()=5 2024-06-28T15:33:38.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.737+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.34__multipart_key1.2~oKX-O3W1arIbBze_K1WtOxq9R3qJDlu.1 2024-06-28T15:33:38.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.737+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.34__multipart_key1.2~oKX-O3W1arIbBze_K1WtOxq9R3qJDlu.2 2024-06-28T15:33:38.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.737+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.34__multipart_key1.2~oKX-O3W1arIbBze_K1WtOxq9R3qJDlu.3 2024-06-28T15:33:38.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.737+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.34__multipart_key1.2~oKX-O3W1arIbBze_K1WtOxq9R3qJDlu.4 2024-06-28T15:33:38.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.737+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.34__multipart_key1.2~oKX-O3W1arIbBze_K1WtOxq9R3qJDlu.5 2024-06-28T15:33:38.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.737+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1020046570329079054', time=2024-06-28T17:32:52.591755+0000, chain.objs.size()=1 2024-06-28T15:33:38.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.737+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.50__shadow_.5yWuV3vKoD5R0wmmDbCepDDs4uF11tu_0 2024-06-28T15:33:38.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.745+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.749+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-06-28T15:33:38.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.749+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.749+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.749+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.753+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-06-28T15:33:38.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.753+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.753+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.753+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.753+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12635893454358632971', time=2024-06-28T17:29:13.380969+0000, chain.objs.size()=5 2024-06-28T15:33:38.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.753+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~tQ8zLh2Vu1rTfd6qHIU6qJv1Msceu6F.1 2024-06-28T15:33:38.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.753+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__shadow_key1.2~tQ8zLh2Vu1rTfd6qHIU6qJv1Msceu6F.1_1 2024-06-28T15:33:38.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.753+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~tQ8zLh2Vu1rTfd6qHIU6qJv1Msceu6F.2 2024-06-28T15:33:38.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.753+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__shadow_key1.2~tQ8zLh2Vu1rTfd6qHIU6qJv1Msceu6F.2_1 2024-06-28T15:33:38.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.753+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~tQ8zLh2Vu1rTfd6qHIU6qJv1Msceu6F.3 2024-06-28T15:33:38.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.761+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.765+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-06-28T15:33:38.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.769+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.769+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.769+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.769+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12934838157622217869', time=2024-06-28T17:29:54.260041+0000, chain.objs.size()=1 2024-06-28T15:33:38.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.769+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.43__shadow_.rn6lIHiV9-tV0ftPTweAL4hgOOvyq99_0 2024-06-28T15:33:38.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.769+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.773+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-06-28T15:33:38.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.773+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.773+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.777+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.777+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-06-28T15:33:38.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.777+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.781+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.781+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.781+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-06-28T15:33:38.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.785+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.785+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.785+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.785+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3484373975882413816', time=2024-06-28T17:29:54.338428+0000, chain.objs.size()=5 2024-06-28T15:33:38.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.785+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.43__multipart_file2.mpg.2~eSuzFGGlMjAC4IJnJMV8o0aif2zSAgN.1 2024-06-28T15:33:38.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.785+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.43__multipart_file2.mpg.2~eSuzFGGlMjAC4IJnJMV8o0aif2zSAgN.2 2024-06-28T15:33:38.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.785+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.43__multipart_file2.mpg.2~eSuzFGGlMjAC4IJnJMV8o0aif2zSAgN.3 2024-06-28T15:33:38.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.785+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.43__multipart_file2.mpg.2~eSuzFGGlMjAC4IJnJMV8o0aif2zSAgN.4 2024-06-28T15:33:38.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.785+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.43__multipart_file2.mpg.2~eSuzFGGlMjAC4IJnJMV8o0aif2zSAgN.5 2024-06-28T15:33:38.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.789+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.797+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-06-28T15:33:38.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.797+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.797+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.797+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.797+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5032388662992476638', time=2024-06-28T17:29:54.542661+0000, chain.objs.size()=1 2024-06-28T15:33:38.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.797+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.43__shadow_.Lua9XKs3OpyI8q1HPhooTSuWrdVDvii_0 2024-06-28T15:33:38.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.801+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.805+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-06-28T15:33:38.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.805+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.809+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.809+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-06-28T15:33:38.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.809+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1784376247115828638', time=2024-06-28T17:29:13.796730+0000, chain.objs.size()=4 2024-06-28T15:33:38.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.809+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.36__shadow_.QbmCnb-woB2BNbqrtXY8Wcih5VouuLs_0 2024-06-28T15:33:38.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.809+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.36__shadow_.QbmCnb-woB2BNbqrtXY8Wcih5VouuLs_1 2024-06-28T15:33:38.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.809+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.36__shadow_.QbmCnb-woB2BNbqrtXY8Wcih5VouuLs_2 2024-06-28T15:33:38.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.809+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.36__shadow_.QbmCnb-woB2BNbqrtXY8Wcih5VouuLs_3 2024-06-28T15:33:38.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.809+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5025497370545887189', time=2024-06-28T17:29:54.441075+0000, chain.objs.size()=1 2024-06-28T15:33:38.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.809+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.43__shadow_.4lllYsZbArMb5oUHAQKnuPqFraTs22__0 2024-06-28T15:33:38.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.813+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.821+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-06-28T15:33:38.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.821+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.821+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.821+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.821+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13087390892493980196', time=2024-06-28T17:31:31.667815+0000, chain.objs.size()=10 2024-06-28T15:33:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.821+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~hV4XKa6XrWtr1ipuzyc2hOa22dutT_G.1 2024-06-28T15:33:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.821+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~hV4XKa6XrWtr1ipuzyc2hOa22dutT_G.2 2024-06-28T15:33:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.821+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~hV4XKa6XrWtr1ipuzyc2hOa22dutT_G.3 2024-06-28T15:33:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.821+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~hV4XKa6XrWtr1ipuzyc2hOa22dutT_G.4 2024-06-28T15:33:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.821+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~hV4XKa6XrWtr1ipuzyc2hOa22dutT_G.5 2024-06-28T15:33:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.821+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~hV4XKa6XrWtr1ipuzyc2hOa22dutT_G.6 2024-06-28T15:33:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.821+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~hV4XKa6XrWtr1ipuzyc2hOa22dutT_G.7 2024-06-28T15:33:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.821+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~hV4XKa6XrWtr1ipuzyc2hOa22dutT_G.8 2024-06-28T15:33:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.821+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~hV4XKa6XrWtr1ipuzyc2hOa22dutT_G.9 2024-06-28T15:33:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.825+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~hV4XKa6XrWtr1ipuzyc2hOa22dutT_G.10 2024-06-28T15:33:38.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.833+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.837+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-06-28T15:33:38.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.841+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.841+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.841+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.845+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-06-28T15:33:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.845+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.845+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.845+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.845+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-06-28T15:33:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.849+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.849+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.849+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-06-28T15:33:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.849+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18123855663819018281', time=2024-06-28T17:28:34.346500+0000, chain.objs.size()=5 2024-06-28T15:33:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.849+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.34__multipart_key1.2~oKX-O3W1arIbBze_K1WtOxq9R3qJDlu.1 2024-06-28T15:33:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.849+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.34__multipart_key1.2~oKX-O3W1arIbBze_K1WtOxq9R3qJDlu.2 2024-06-28T15:33:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.849+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.34__multipart_key1.2~oKX-O3W1arIbBze_K1WtOxq9R3qJDlu.3 2024-06-28T15:33:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.849+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.34__multipart_key1.2~oKX-O3W1arIbBze_K1WtOxq9R3qJDlu.4 2024-06-28T15:33:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.849+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.34__multipart_key1.2~oKX-O3W1arIbBze_K1WtOxq9R3qJDlu.5 2024-06-28T15:33:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.849+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8141379933332561862', time=2024-06-28T17:32:49.479121+0000, chain.objs.size()=1 2024-06-28T15:33:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.849+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.49__shadow_.CSTbsIekdVOMT3J4TuQYLN8HfDXE1Um_0 2024-06-28T15:33:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.849+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18236364717956396294', time=2024-06-28T17:32:56.128653+0000, chain.objs.size()=1 2024-06-28T15:33:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.849+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.54__shadow_.O87R8NJGFvgtSZMjfyNCs5Gruv3GDBn_0 2024-06-28T15:33:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.853+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.857+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-06-28T15:33:38.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.857+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-06-28T15:33:38.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11803709323866816220', time=2024-06-28T17:29:03.727348+0000, chain.objs.size()=5 2024-06-28T15:33:38.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~l7oeWk2GTyR_RyuPRSG1lRH52Jfx016.1 2024-06-28T15:33:38.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~l7oeWk2GTyR_RyuPRSG1lRH52Jfx016.2 2024-06-28T15:33:38.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~l7oeWk2GTyR_RyuPRSG1lRH52Jfx016.3 2024-06-28T15:33:38.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~l7oeWk2GTyR_RyuPRSG1lRH52Jfx016.4 2024-06-28T15:33:38.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~l7oeWk2GTyR_RyuPRSG1lRH52Jfx016.5 2024-06-28T15:33:38.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2162530130192868744', time=2024-06-28T17:31:15.544290+0000, chain.objs.size()=11 2024-06-28T15:33:38.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~aRq4SuwDQc-FOD6C3FV6hAqVuQzKYtH.1 2024-06-28T15:33:38.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~aRq4SuwDQc-FOD6C3FV6hAqVuQzKYtH.2 2024-06-28T15:33:38.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~aRq4SuwDQc-FOD6C3FV6hAqVuQzKYtH.3 2024-06-28T15:33:38.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~aRq4SuwDQc-FOD6C3FV6hAqVuQzKYtH.4 2024-06-28T15:33:38.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~aRq4SuwDQc-FOD6C3FV6hAqVuQzKYtH.5 2024-06-28T15:33:38.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~aRq4SuwDQc-FOD6C3FV6hAqVuQzKYtH.6 2024-06-28T15:33:38.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.861+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~aRq4SuwDQc-FOD6C3FV6hAqVuQzKYtH.7 2024-06-28T15:33:38.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.869+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~aRq4SuwDQc-FOD6C3FV6hAqVuQzKYtH.8 2024-06-28T15:33:38.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.869+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~aRq4SuwDQc-FOD6C3FV6hAqVuQzKYtH.9 2024-06-28T15:33:38.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.869+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~aRq4SuwDQc-FOD6C3FV6hAqVuQzKYtH.10 2024-06-28T15:33:38.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.869+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~aRq4SuwDQc-FOD6C3FV6hAqVuQzKYtH.11 2024-06-28T15:33:38.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.869+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13857210138789051681', time=2024-06-28T17:32:54.305287+0000, chain.objs.size()=1 2024-06-28T15:33:38.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.869+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.52__shadow_.i-FvOHVHbRH08W8qB0dFshd4UvfVOLr_0 2024-06-28T15:33:38.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.881+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-06-28T15:33:38.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14742235573088110511', time=2024-06-28T17:32:05.037142+0000, chain.objs.size()=11 2024-06-28T15:33:38.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~JKnVedJbCEdKhngGaZjrclreWCxDddv.1 2024-06-28T15:33:38.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~JKnVedJbCEdKhngGaZjrclreWCxDddv.1_1 2024-06-28T15:33:38.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~JKnVedJbCEdKhngGaZjrclreWCxDddv.2 2024-06-28T15:33:38.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~JKnVedJbCEdKhngGaZjrclreWCxDddv.2_1 2024-06-28T15:33:38.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~JKnVedJbCEdKhngGaZjrclreWCxDddv.3 2024-06-28T15:33:38.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~JKnVedJbCEdKhngGaZjrclreWCxDddv.3_1 2024-06-28T15:33:38.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~JKnVedJbCEdKhngGaZjrclreWCxDddv.4 2024-06-28T15:33:38.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~JKnVedJbCEdKhngGaZjrclreWCxDddv.4_1 2024-06-28T15:33:38.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~JKnVedJbCEdKhngGaZjrclreWCxDddv.5 2024-06-28T15:33:38.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__shadow_key1.2~JKnVedJbCEdKhngGaZjrclreWCxDddv.5_1 2024-06-28T15:33:38.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.885+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.45__multipart_key1.2~JKnVedJbCEdKhngGaZjrclreWCxDddv.6 2024-06-28T15:33:38.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.897+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.901+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-06-28T15:33:38.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.901+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.901+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.901+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.901+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7935233635982340209', time=2024-06-28T17:28:58.820452+0000, chain.objs.size()=5 2024-06-28T15:33:38.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.901+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~U3dR2Ks9hK9bZ0G2P_5kfbKowlcDAG5.1 2024-06-28T15:33:38.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.901+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~U3dR2Ks9hK9bZ0G2P_5kfbKowlcDAG5.2 2024-06-28T15:33:38.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.901+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~U3dR2Ks9hK9bZ0G2P_5kfbKowlcDAG5.3 2024-06-28T15:33:38.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.901+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~U3dR2Ks9hK9bZ0G2P_5kfbKowlcDAG5.4 2024-06-28T15:33:38.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.901+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~U3dR2Ks9hK9bZ0G2P_5kfbKowlcDAG5.5 2024-06-28T15:33:38.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.905+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.913+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-06-28T15:33:38.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.913+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.913+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.913+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.913+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3742538800896268334', time=2024-06-28T17:29:24.006117+0000, chain.objs.size()=5 2024-06-28T15:33:38.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.913+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.41__multipart_key1.2~tvryHUxzH4k3KUt1ahz8hlSnsUjw13N.1 2024-06-28T15:33:38.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.913+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.41__multipart_key1.2~tvryHUxzH4k3KUt1ahz8hlSnsUjw13N.2 2024-06-28T15:33:38.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.913+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.41__multipart_key1.2~tvryHUxzH4k3KUt1ahz8hlSnsUjw13N.3 2024-06-28T15:33:38.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.913+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.41__multipart_key1.2~tvryHUxzH4k3KUt1ahz8hlSnsUjw13N.4 2024-06-28T15:33:38.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.913+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.41__multipart_key1.2~tvryHUxzH4k3KUt1ahz8hlSnsUjw13N.5 2024-06-28T15:33:38.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.917+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.921+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-06-28T15:33:38.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.925+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.925+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.925+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.925+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-06-28T15:33:38.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.929+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.929+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.929+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.929+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-06-28T15:33:38.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16943046228160803890', time=2024-06-28T17:33:37.566592+0000, chain.objs.size()=11 2024-06-28T15:33:38.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.56__multipart_key1.2~nU5bkir1x_Y8x4_kMHlNxsk25TrR6X8.1 2024-06-28T15:33:38.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.56__multipart_key1.2~nU5bkir1x_Y8x4_kMHlNxsk25TrR6X8.2 2024-06-28T15:33:38.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.56__multipart_key1.2~nU5bkir1x_Y8x4_kMHlNxsk25TrR6X8.3 2024-06-28T15:33:38.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.56__multipart_key1.2~nU5bkir1x_Y8x4_kMHlNxsk25TrR6X8.4 2024-06-28T15:33:38.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.56__multipart_key1.2~nU5bkir1x_Y8x4_kMHlNxsk25TrR6X8.5 2024-06-28T15:33:38.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.56__multipart_key1.2~nU5bkir1x_Y8x4_kMHlNxsk25TrR6X8.6 2024-06-28T15:33:38.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.56__multipart_key1.2~nU5bkir1x_Y8x4_kMHlNxsk25TrR6X8.7 2024-06-28T15:33:38.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.56__multipart_key1.2~nU5bkir1x_Y8x4_kMHlNxsk25TrR6X8.8 2024-06-28T15:33:38.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.56__multipart_key1.2~nU5bkir1x_Y8x4_kMHlNxsk25TrR6X8.9 2024-06-28T15:33:38.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.56__multipart_key1.2~nU5bkir1x_Y8x4_kMHlNxsk25TrR6X8.10 2024-06-28T15:33:38.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.933+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.56__multipart_key1.2~nU5bkir1x_Y8x4_kMHlNxsk25TrR6X8.11 2024-06-28T15:33:38.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.941+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.945+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-06-28T15:33:38.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.945+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.949+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.949+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.949+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14263819158790249067', time=2024-06-28T17:32:24.858561+0000, chain.objs.size()=5 2024-06-28T15:33:38.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.949+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.46__multipart_key1.2~75VRnT9HuEskb-mVSKavtIgsnByXEQM.1 2024-06-28T15:33:38.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.949+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.46__multipart_key1.2~75VRnT9HuEskb-mVSKavtIgsnByXEQM.2 2024-06-28T15:33:38.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.949+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.46__multipart_key1.2~75VRnT9HuEskb-mVSKavtIgsnByXEQM.3 2024-06-28T15:33:38.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.949+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.46__multipart_key1.2~75VRnT9HuEskb-mVSKavtIgsnByXEQM.4 2024-06-28T15:33:38.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.949+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.46__multipart_key1.2~75VRnT9HuEskb-mVSKavtIgsnByXEQM.5 2024-06-28T15:33:38.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.953+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-06-28T15:33:38.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-06-28T15:33:38.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.495439049089707576', time=2024-06-28T17:32:33.391067+0000, chain.objs.size()=5 2024-06-28T15:33:38.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.47__multipart_key1.2~E0uhcDl15B2Fiwjxf8byzeIybepYSBf.1 2024-06-28T15:33:38.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.47__multipart_key1.2~E0uhcDl15B2Fiwjxf8byzeIybepYSBf.2 2024-06-28T15:33:38.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.47__multipart_key1.2~E0uhcDl15B2Fiwjxf8byzeIybepYSBf.3 2024-06-28T15:33:38.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.47__multipart_key1.2~E0uhcDl15B2Fiwjxf8byzeIybepYSBf.4 2024-06-28T15:33:38.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.47__multipart_key1.2~E0uhcDl15B2Fiwjxf8byzeIybepYSBf.5 2024-06-28T15:33:38.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz', time=2024-06-28T17:32:44.879281+0000, chain.objs.size()=1 2024-06-28T15:33:38.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.15 2024-06-28T15:33:38.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz', time=2024-06-28T17:32:45.137187+0000, chain.objs.size()=1 2024-06-28T15:33:38.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.16 2024-06-28T15:33:38.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz', time=2024-06-28T17:32:45.167356+0000, chain.objs.size()=1 2024-06-28T15:33:38.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.17 2024-06-28T15:33:38.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz', time=2024-06-28T17:32:45.419815+0000, chain.objs.size()=1 2024-06-28T15:33:38.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.18 2024-06-28T15:33:38.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz', time=2024-06-28T17:32:45.514743+0000, chain.objs.size()=1 2024-06-28T15:33:38.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.19 2024-06-28T15:33:38.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz', time=2024-06-28T17:32:45.591340+0000, chain.objs.size()=1 2024-06-28T15:33:38.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.20 2024-06-28T15:33:38.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz', time=2024-06-28T17:32:45.641050+0000, chain.objs.size()=34 2024-06-28T15:33:38.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.957+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.1 2024-06-28T15:33:38.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.965+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.2 2024-06-28T15:33:38.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.965+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.3 2024-06-28T15:33:38.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.965+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.4 2024-06-28T15:33:38.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.965+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.5 2024-06-28T15:33:38.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.965+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.6 2024-06-28T15:33:38.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.965+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.7 2024-06-28T15:33:38.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.965+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.8 2024-06-28T15:33:38.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.965+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.9 2024-06-28T15:33:38.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.965+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.10 2024-06-28T15:33:38.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.965+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.11 2024-06-28T15:33:38.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.965+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.12 2024-06-28T15:33:38.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.969+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.13 2024-06-28T15:33:38.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.969+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.14 2024-06-28T15:33:38.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.969+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.7wQqG7COw3V1G7xbmszWtd6dYBePgiy-.15 2024-06-28T15:33:38.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.977+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.dSqp0gNAv9Xr8B1sPk0xbzbamBB76n2k.16 2024-06-28T15:33:38.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.977+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.52er56P8OG-wD3k7_m5TwinptgKDF7-Y.17 2024-06-28T15:33:38.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.977+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.drJZQJ0yagulFuMGKoRqfw9I2yIi-rho.18 2024-06-28T15:33:38.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.977+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.8Dm2QDvyeK_GqUklxJ0yHe61oMNxv-IL.19 2024-06-28T15:33:38.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.977+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.hfRu-CR4qamGE1CBQUHiojtDt1wjWgO6.20 2024-06-28T15:33:38.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.977+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.21 2024-06-28T15:33:38.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.977+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.22 2024-06-28T15:33:38.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.977+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.23 2024-06-28T15:33:38.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.977+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.24 2024-06-28T15:33:38.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.977+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.25 2024-06-28T15:33:38.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.977+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.26 2024-06-28T15:33:38.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.981+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.27 2024-06-28T15:33:38.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.981+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.28 2024-06-28T15:33:38.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.981+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.29 2024-06-28T15:33:38.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.981+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.30 2024-06-28T15:33:38.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.981+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.31 2024-06-28T15:33:38.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.981+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.32 2024-06-28T15:33:38.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.981+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.33 2024-06-28T15:33:38.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.981+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.48__multipart_key1.2~CYbT-SMKRt6suyOmC7c5w4TbQL-QAbz.34 2024-06-28T15:33:38.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.989+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-06-28T15:33:38.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-06-28T15:33:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8608897832223286485', time=2024-06-28T17:28:53.540206+0000, chain.objs.size()=5 2024-06-28T15:33:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~7F6QYXZuCgHOzC_ptMuw5VHJd2UfLOX.1 2024-06-28T15:33:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~7F6QYXZuCgHOzC_ptMuw5VHJd2UfLOX.2 2024-06-28T15:33:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~7F6QYXZuCgHOzC_ptMuw5VHJd2UfLOX.3 2024-06-28T15:33:38.983 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~7F6QYXZuCgHOzC_ptMuw5VHJd2UfLOX.4 2024-06-28T15:33:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~7F6QYXZuCgHOzC_ptMuw5VHJd2UfLOX.5 2024-06-28T15:33:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2262571632077031898', time=2024-06-28T17:29:08.068083+0000, chain.objs.size()=5 2024-06-28T15:33:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~dCUEISn6IjNb0c9VL58eaiCRbH_XaPU.1 2024-06-28T15:33:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__shadow_key1.2~dCUEISn6IjNb0c9VL58eaiCRbH_XaPU.1_1 2024-06-28T15:33:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~dCUEISn6IjNb0c9VL58eaiCRbH_XaPU.2 2024-06-28T15:33:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__shadow_key1.2~dCUEISn6IjNb0c9VL58eaiCRbH_XaPU.2_1 2024-06-28T15:33:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:38.993+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.37__multipart_key1.2~dCUEISn6IjNb0c9VL58eaiCRbH_XaPU.3 2024-06-28T15:33:38.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.001+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:38.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.005+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-06-28T15:33:38.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.005+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.005+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:38.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.005+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.005+0000 7f49e2bc5980 20 garbage collection: RGWGC::process iterating over entry tag='2~1_obqYUh0LR6u9-MrqOSDq8uuoF4CYN', time=2024-06-28T17:30:03.563956+0000, chain.objs.size()=3 2024-06-28T15:33:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.005+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.44__multipart_key1.2~1_obqYUh0LR6u9-MrqOSDq8uuoF4CYN.7 2024-06-28T15:33:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.005+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.44__multipart_key1.2~1_obqYUh0LR6u9-MrqOSDq8uuoF4CYN.10 2024-06-28T15:33:38.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.005+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.44__multipart_key1.2~1_obqYUh0LR6u9-MrqOSDq8uuoF4CYN.13 2024-06-28T15:33:38.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.009+0000 7f49e2bc5980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:39.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.013+0000 7f49e2bc5980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-06-28T15:33:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.013+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.013+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.013+0000 7f49e2bc5980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.017+0000 7f4896fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:39.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.017+0000 7f49e2bc5980 20 remove_watcher() i=0 2024-06-28T15:33:39.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.017+0000 7f49e2bc5980 2 removed watcher, disabling cache 2024-06-28T15:33:39.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.021+0000 7f49e2bc5980 20 remove_watcher() i=1 2024-06-28T15:33:39.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.021+0000 7f49e2bc5980 20 remove_watcher() i=2 2024-06-28T15:33:39.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.021+0000 7f49e2bc5980 20 remove_watcher() i=3 2024-06-28T15:33:39.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.025+0000 7f49e2bc5980 20 remove_watcher() i=4 2024-06-28T15:33:39.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.025+0000 7f49e2bc5980 20 remove_watcher() i=5 2024-06-28T15:33:39.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.025+0000 7f49e2bc5980 20 remove_watcher() i=6 2024-06-28T15:33:39.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.029+0000 7f49e2bc5980 20 remove_watcher() i=7 2024-06-28T15:33:39.031 DEBUG:teuthology.orchestra.run.smithi178:> radosgw-admin gc process --include-all 2024-06-28T15:33:39.097 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:33:39.097 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:33:39.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.153+0000 7f3047bc4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.153+0000 7f3047bc4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.153+0000 7f3047bc4980 20 realm 2024-06-28T15:33:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.153+0000 7f3047bc4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.153+0000 7f3047bc4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.153+0000 7f3047bc4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:33:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.153+0000 7f3047bc4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.153+0000 7f3047bc4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:39.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.153+0000 7f3047bc4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.157+0000 7f3047bc4980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:39.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.157+0000 7f3047bc4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.157+0000 7f3047bc4980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:33:39.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.157+0000 7f3047bc4980 20 searching for the correct realm 2024-06-28T15:33:39.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.161+0000 7f3044ed4640 0 -- 172.21.15.178:0/3475228040 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f304001e040 msgr2=0x7f3040068760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:33:39.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.377+0000 7f30456d5640 0 -- 172.21.15.178:0/3475228040 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f30300fd310 msgr2=0x7f30300b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:33:39.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.593+0000 7f3047bc4980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:33:39.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.593+0000 7f3047bc4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:33:39.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.593+0000 7f3047bc4980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:33:39.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.593+0000 7f3047bc4980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:33:39.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.593+0000 7f3047bc4980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:33:39.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.593+0000 7f3047bc4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:33:39.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.593+0000 7f3047bc4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:39.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:39.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:39.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:39.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:39.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:33:39.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 20 zone default found 2024-06-28T15:33:39.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 4 Realm: () 2024-06-28T15:33:39.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:33:39.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:33:39.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:33:39.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 20 zonegroup default 2024-06-28T15:33:39.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:39.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:39.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:39.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:39.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.597+0000 7f3047bc4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:39.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.601+0000 7f3047bc4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:39.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.601+0000 7f3047bc4980 20 started sync module instance, tier type = 2024-06-28T15:33:39.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.601+0000 7f3047bc4980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:33:39.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.625+0000 7f3047bc4980 20 add_watcher() i=0 2024-06-28T15:33:39.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.625+0000 7f3047bc4980 20 add_watcher() i=1 2024-06-28T15:33:39.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.625+0000 7f3047bc4980 20 add_watcher() i=2 2024-06-28T15:33:39.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.625+0000 7f3047bc4980 20 add_watcher() i=3 2024-06-28T15:33:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.625+0000 7f3047bc4980 20 add_watcher() i=4 2024-06-28T15:33:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.625+0000 7f3047bc4980 20 add_watcher() i=5 2024-06-28T15:33:39.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.625+0000 7f3047bc4980 20 add_watcher() i=6 2024-06-28T15:33:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.625+0000 7f3047bc4980 20 add_watcher() i=7 2024-06-28T15:33:39.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.625+0000 7f3047bc4980 2 all 8 watchers are set, enabling cache 2024-06-28T15:33:39.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.629+0000 7f2efb7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.629+0000 7f3047bc4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:33:39.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.629+0000 7f3047bc4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:33:39.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.717+0000 7f2efa7e4640 20 reqs_thread_entry: start 2024-06-28T15:33:39.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.717+0000 7f3047bc4980 20 init_complete bucket index max shards: 1 2024-06-28T15:33:39.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.717+0000 7f3047bc4980 20 Filter name: none 2024-06-28T15:33:39.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.717+0000 7f2e6bfff640 20 reqs_thread_entry: start 2024-06-28T15:33:39.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.717+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-06-28T15:33:39.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.721+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.725+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.725+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.725+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-06-28T15:33:39.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.729+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.729+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.729+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.729+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-06-28T15:33:39.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.733+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.733+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.733+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.737+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-06-28T15:33:39.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.741+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.741+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.741+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.745+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-06-28T15:33:39.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.745+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.749+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.749+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.753+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-06-28T15:33:39.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.753+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.753+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.753+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.757+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-06-28T15:33:39.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.761+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.761+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.765+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.765+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-06-28T15:33:39.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.769+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.769+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.769+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.773+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-06-28T15:33:39.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.777+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.777+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.777+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.781+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-06-28T15:33:39.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.785+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.785+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.785+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.789+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-06-28T15:33:39.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.793+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.793+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.793+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.793+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-06-28T15:33:39.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.797+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.801+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.801+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.801+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-06-28T15:33:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.805+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.805+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.809+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.809+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-06-28T15:33:39.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.813+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.813+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.804 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.817+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.817+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-06-28T15:33:39.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.821+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.821+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.821+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.825+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-06-28T15:33:39.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.829+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.833+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.833+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.837+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-06-28T15:33:39.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.841+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.841+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.841+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.845+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-06-28T15:33:39.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.849+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.849+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.849+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.853+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-06-28T15:33:39.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.853+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.857+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.857+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.857+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-06-28T15:33:39.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.861+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.861+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.861+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.865+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-06-28T15:33:39.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.869+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.869+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.873+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.873+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-06-28T15:33:39.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.877+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.877+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.881+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.881+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-06-28T15:33:39.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.885+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.885+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.889+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.889+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-06-28T15:33:39.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.893+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.897+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.897+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.897+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-06-28T15:33:39.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.901+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.905+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.905+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.909+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-06-28T15:33:39.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.913+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.902 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.913+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.913+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.917+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-06-28T15:33:39.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.921+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.921+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.925+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.929+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-06-28T15:33:39.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.929+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.933+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.933+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.937+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-06-28T15:33:39.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.941+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.945+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.945+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.949+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-06-28T15:33:39.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.953+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.953+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.953+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.957+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-06-28T15:33:39.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.961+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.961+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.965+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.955 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.965+0000 7f3047bc4980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-06-28T15:33:39.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.973+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.977+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:39.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.977+0000 7f3047bc4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:39.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.981+0000 7f2efb7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:39.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.985+0000 7f3047bc4980 20 remove_watcher() i=0 2024-06-28T15:33:39.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.985+0000 7f3047bc4980 2 removed watcher, disabling cache 2024-06-28T15:33:39.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.989+0000 7f3047bc4980 20 remove_watcher() i=1 2024-06-28T15:33:39.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.989+0000 7f3047bc4980 20 remove_watcher() i=2 2024-06-28T15:33:39.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.993+0000 7f3047bc4980 20 remove_watcher() i=3 2024-06-28T15:33:39.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.997+0000 7f3047bc4980 20 remove_watcher() i=4 2024-06-28T15:33:39.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.997+0000 7f3047bc4980 20 remove_watcher() i=5 2024-06-28T15:33:39.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:39.997+0000 7f3047bc4980 20 remove_watcher() i=6 2024-06-28T15:33:39.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.001+0000 7f3047bc4980 20 remove_watcher() i=7 2024-06-28T15:33:40.010 DEBUG:teuthology.orchestra.run.smithi178:> radosgw-admin gc process --include-all 2024-06-28T15:33:40.082 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:33:40.082 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:33:40.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.137+0000 7faf2c72a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:40.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.141+0000 7faf2c72a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:40.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.141+0000 7faf2c72a980 20 realm 2024-06-28T15:33:40.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.141+0000 7faf2c72a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:40.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.141+0000 7faf2c72a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:40.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.141+0000 7faf2c72a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:33:40.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.141+0000 7faf2c72a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:40.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.141+0000 7faf2c72a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:40.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.141+0000 7faf2c72a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:40.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.145+0000 7faf2c72a980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:40.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.145+0000 7faf2c72a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:40.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.145+0000 7faf2c72a980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:33:40.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.145+0000 7faf2c72a980 20 searching for the correct realm 2024-06-28T15:33:40.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.149+0000 7faf29a3a640 0 -- 172.21.15.178:0/3572507971 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x559faaca4940 msgr2=0x559faaca6d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:33:40.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.365+0000 7faf2a23b640 0 -- 172.21.15.178:0/3572507971 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7faf1406c9c0 msgr2=0x7faf1407d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:33:40.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.581+0000 7faf2c72a980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:33:40.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.581+0000 7faf2c72a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:33:40.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.581+0000 7faf2c72a980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:33:40.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.581+0000 7faf2c72a980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:33:40.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.581+0000 7faf2c72a980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:33:40.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.581+0000 7faf2c72a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:33:40.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.581+0000 7faf2c72a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:40.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:40.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:40.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:40.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:40.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:33:40.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 zone default found 2024-06-28T15:33:40.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 4 Realm: () 2024-06-28T15:33:40.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:33:40.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:33:40.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:33:40.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 zonegroup default 2024-06-28T15:33:40.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:40.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:40.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:40.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:40.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:40.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:40.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 started sync module instance, tier type = 2024-06-28T15:33:40.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.585+0000 7faf2c72a980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:33:40.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.609+0000 7faf2c72a980 20 add_watcher() i=0 2024-06-28T15:33:40.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.609+0000 7faf2c72a980 20 add_watcher() i=1 2024-06-28T15:33:40.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.609+0000 7faf2c72a980 20 add_watcher() i=2 2024-06-28T15:33:40.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.609+0000 7faf2c72a980 20 add_watcher() i=3 2024-06-28T15:33:40.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.609+0000 7faf2c72a980 20 add_watcher() i=4 2024-06-28T15:33:40.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.609+0000 7faf2c72a980 20 add_watcher() i=5 2024-06-28T15:33:40.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.609+0000 7faf2c72a980 20 add_watcher() i=6 2024-06-28T15:33:40.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.609+0000 7faf2c72a980 20 add_watcher() i=7 2024-06-28T15:33:40.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.609+0000 7faf2c72a980 2 all 8 watchers are set, enabling cache 2024-06-28T15:33:40.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.613+0000 7fade0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:40.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.617+0000 7faf2c72a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:33:40.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.617+0000 7faf2c72a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:33:40.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.713+0000 7fad5b7fe640 20 reqs_thread_entry: start 2024-06-28T15:33:40.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.713+0000 7faf2c72a980 20 init_complete bucket index max shards: 1 2024-06-28T15:33:40.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.713+0000 7faf2c72a980 20 Filter name: none 2024-06-28T15:33:40.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.713+0000 7fad597fa640 20 reqs_thread_entry: start 2024-06-28T15:33:40.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.713+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-06-28T15:33:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.717+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.717+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.717+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.721+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-06-28T15:33:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.721+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.725+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.725+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.729+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-06-28T15:33:40.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.729+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.729+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.733+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.733+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-06-28T15:33:40.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.737+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.737+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.741+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.741+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-06-28T15:33:40.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.745+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.749+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.749+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.749+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-06-28T15:33:40.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.753+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.757+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.757+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.761+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-06-28T15:33:40.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.761+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.765+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.765+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.765+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-06-28T15:33:40.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.769+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.769+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.773+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.773+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-06-28T15:33:40.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.777+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.777+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.777+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.781+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-06-28T15:33:40.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.781+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.785+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.785+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.785+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-06-28T15:33:40.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.793+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.793+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.793+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.797+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-06-28T15:33:40.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.801+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.801+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.801+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.805+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-06-28T15:33:40.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.809+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.809+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.809+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.813+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-06-28T15:33:40.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.817+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.821+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.821+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.825+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-06-28T15:33:40.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.829+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.829+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.833+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.833+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-06-28T15:33:40.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.837+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.841+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.841+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.841+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-06-28T15:33:40.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.845+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.849+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.849+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.853+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-06-28T15:33:40.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.857+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.857+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.857+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.861+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-06-28T15:33:40.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.865+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.865+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.869+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.869+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-06-28T15:33:40.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.873+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.873+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.877+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.877+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-06-28T15:33:40.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.881+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.881+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.881+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.885+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-06-28T15:33:40.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.889+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.889+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.889+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.893+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-06-28T15:33:40.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.897+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.901+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.901+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.905+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-06-28T15:33:40.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.909+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.909+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.909+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.913+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-06-28T15:33:40.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.917+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.917+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.917+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.921+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-06-28T15:33:40.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.925+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.925+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.925+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.929+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-06-28T15:33:40.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.933+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.933+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.933+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.937+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-06-28T15:33:40.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.941+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.941+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.945+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.945+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-06-28T15:33:40.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.949+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.949+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.953+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.953+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-06-28T15:33:40.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.957+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.957+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.961+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.961+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-06-28T15:33:40.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.965+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.969+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.969+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.973+0000 7faf2c72a980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-06-28T15:33:40.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.977+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.977+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:40.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.977+0000 7faf2c72a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:40.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.981+0000 7fade0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:40.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.989+0000 7faf2c72a980 20 remove_watcher() i=0 2024-06-28T15:33:40.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.989+0000 7faf2c72a980 2 removed watcher, disabling cache 2024-06-28T15:33:40.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.989+0000 7faf2c72a980 20 remove_watcher() i=1 2024-06-28T15:33:40.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.993+0000 7faf2c72a980 20 remove_watcher() i=2 2024-06-28T15:33:40.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:40.997+0000 7faf2c72a980 20 remove_watcher() i=3 2024-06-28T15:33:40.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.001+0000 7faf2c72a980 20 remove_watcher() i=4 2024-06-28T15:33:40.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.001+0000 7faf2c72a980 20 remove_watcher() i=5 2024-06-28T15:33:40.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.005+0000 7faf2c72a980 20 remove_watcher() i=6 2024-06-28T15:33:40.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.005+0000 7faf2c72a980 20 remove_watcher() i=7 2024-06-28T15:33:41.014 DEBUG:teuthology.orchestra.run.smithi178:> radosgw-admin gc process --include-all 2024-06-28T15:33:41.088 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:33:41.088 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:33:41.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.145+0000 7f5a29267980 20 rados->read ofs=0 len=0 2024-06-28T15:33:41.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.145+0000 7f5a29267980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:41.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.145+0000 7f5a29267980 20 realm 2024-06-28T15:33:41.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.145+0000 7f5a29267980 20 rados->read ofs=0 len=0 2024-06-28T15:33:41.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.145+0000 7f5a29267980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:41.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.145+0000 7f5a29267980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:33:41.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.145+0000 7f5a29267980 20 rados->read ofs=0 len=0 2024-06-28T15:33:41.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.145+0000 7f5a29267980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:41.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.145+0000 7f5a29267980 20 rados->read ofs=0 len=0 2024-06-28T15:33:41.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.149+0000 7f5a29267980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:41.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.149+0000 7f5a29267980 20 rados->read ofs=0 len=0 2024-06-28T15:33:41.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.149+0000 7f5a29267980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:33:41.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.149+0000 7f5a29267980 20 searching for the correct realm 2024-06-28T15:33:41.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.153+0000 7f5a26577640 0 -- 172.21.15.178:0/4199626234 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x55a0c0697450 msgr2=0x55a0c06998a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:33:41.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.161+0000 7f5a26577640 0 -- 172.21.15.178:0/4199626234 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f5a1802c040 msgr2=0x7f5a18052930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:33:41.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:33:41.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:33:41.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:33:41.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:33:41.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:33:41.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:33:41.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 rados->read ofs=0 len=0 2024-06-28T15:33:41.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:41.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 rados->read ofs=0 len=0 2024-06-28T15:33:41.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:41.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 rados->read ofs=0 len=0 2024-06-28T15:33:41.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:33:41.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 zone default found 2024-06-28T15:33:41.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 4 Realm: () 2024-06-28T15:33:41.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:33:41.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:33:41.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:33:41.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 zonegroup default 2024-06-28T15:33:41.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 rados->read ofs=0 len=0 2024-06-28T15:33:41.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:41.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 rados->read ofs=0 len=0 2024-06-28T15:33:41.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:41.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 rados->read ofs=0 len=0 2024-06-28T15:33:41.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:41.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 started sync module instance, tier type = 2024-06-28T15:33:41.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.169+0000 7f5a29267980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:33:41.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.185+0000 7f5a29267980 20 add_watcher() i=0 2024-06-28T15:33:41.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.185+0000 7f5a29267980 20 add_watcher() i=1 2024-06-28T15:33:41.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.185+0000 7f5a29267980 20 add_watcher() i=2 2024-06-28T15:33:41.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.185+0000 7f5a29267980 20 add_watcher() i=3 2024-06-28T15:33:41.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.185+0000 7f5a29267980 20 add_watcher() i=4 2024-06-28T15:33:41.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.185+0000 7f5a29267980 20 add_watcher() i=5 2024-06-28T15:33:41.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.185+0000 7f5a29267980 20 add_watcher() i=6 2024-06-28T15:33:41.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.189+0000 7f5a29267980 20 add_watcher() i=7 2024-06-28T15:33:41.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.189+0000 7f5a29267980 2 all 8 watchers are set, enabling cache 2024-06-28T15:33:41.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.189+0000 7f58dd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:41.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.193+0000 7f5a29267980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:33:41.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.193+0000 7f5a29267980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:33:41.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.269+0000 7f5857fff640 20 reqs_thread_entry: start 2024-06-28T15:33:41.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.269+0000 7f5a29267980 20 init_complete bucket index max shards: 1 2024-06-28T15:33:41.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.269+0000 7f5a29267980 20 Filter name: none 2024-06-28T15:33:41.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.269+0000 7f5855ffb640 20 reqs_thread_entry: start 2024-06-28T15:33:41.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.269+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-06-28T15:33:41.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.273+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.273+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.273+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.277+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-06-28T15:33:41.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.277+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.281+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.281+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.281+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-06-28T15:33:41.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.285+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.285+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.285+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.289+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-06-28T15:33:41.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.289+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.289+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.289+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.293+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-06-28T15:33:41.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.297+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.297+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.297+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.301+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-06-28T15:33:41.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.305+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.305+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.305+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.309+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-06-28T15:33:41.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.309+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.313+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.313+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.317+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-06-28T15:33:41.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.321+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.321+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.321+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.321+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-06-28T15:33:41.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.325+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.329+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.329+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.333+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-06-28T15:33:41.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.337+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.337+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.337+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.341+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-06-28T15:33:41.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.345+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.349+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.349+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.353+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-06-28T15:33:41.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.353+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.357+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.357+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.357+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-06-28T15:33:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.365+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.365+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.365+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.369+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-06-28T15:33:41.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.369+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.373+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.373+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.373+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-06-28T15:33:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.377+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.377+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.381+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.381+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-06-28T15:33:41.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.385+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.385+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.389+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.393+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-06-28T15:33:41.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.397+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.397+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.397+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.401+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-06-28T15:33:41.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.405+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.405+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.405+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.409+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-06-28T15:33:41.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.413+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.413+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.413+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.417+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-06-28T15:33:41.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.421+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.421+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.421+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.425+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-06-28T15:33:41.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.425+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.429+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.429+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.433+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-06-28T15:33:41.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.437+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.437+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.437+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.441+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-06-28T15:33:41.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.441+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.441+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.445+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.445+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-06-28T15:33:41.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.449+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.449+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.453+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.453+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-06-28T15:33:41.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.457+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.461+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.461+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.465+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-06-28T15:33:41.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.465+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.465+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.469+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.469+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-06-28T15:33:41.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.473+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.473+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.477+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.477+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-06-28T15:33:41.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.481+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.485+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.485+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.485+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-06-28T15:33:41.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.489+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.493+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.493+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.493+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-06-28T15:33:41.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.497+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.497+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.501+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.501+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-06-28T15:33:41.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.505+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.505+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.505+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.509+0000 7f5a29267980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-06-28T15:33:41.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.517+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.517+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:41.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.517+0000 7f5a29267980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:41.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.521+0000 7f58dd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:41.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.525+0000 7f5a29267980 20 remove_watcher() i=0 2024-06-28T15:33:41.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.525+0000 7f5a29267980 2 removed watcher, disabling cache 2024-06-28T15:33:41.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.529+0000 7f5a29267980 20 remove_watcher() i=1 2024-06-28T15:33:41.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.529+0000 7f5a29267980 20 remove_watcher() i=2 2024-06-28T15:33:41.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.533+0000 7f5a29267980 20 remove_watcher() i=3 2024-06-28T15:33:41.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.533+0000 7f5a29267980 20 remove_watcher() i=4 2024-06-28T15:33:41.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.537+0000 7f5a29267980 20 remove_watcher() i=5 2024-06-28T15:33:41.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.537+0000 7f5a29267980 20 remove_watcher() i=6 2024-06-28T15:33:41.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:41.541+0000 7f5a29267980 20 remove_watcher() i=7 2024-06-28T15:33:41.554 DEBUG:teuthology.orchestra.run.smithi178:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests BucketTest 2024-06-28T15:33:42.281 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :clean 2024-06-28T15:33:43.079 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :compileJava 2024-06-28T15:33:43.179 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :processResources 2024-06-28T15:33:43.179 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :classes 2024-06-28T15:33:43.679 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:43.679 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :compileTestJava 2024-06-28T15:33:43.680 INFO:teuthology.orchestra.run.smithi178.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-06-28T15:33:43.680 INFO:teuthology.orchestra.run.smithi178.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-06-28T15:33:43.680 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:43.681 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :processTestResources NO-SOURCE 2024-06-28T15:33:43.681 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :testClasses 2024-06-28T15:33:45.680 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:45.681 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :test 2024-06-28T15:33:45.681 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:45.681 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests STANDARD_OUT 2024-06-28T15:33:45.681 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28 15:33:45,016 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-06-28T15:33:45.681 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:45.681 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty STARTED 2024-06-28T15:33:45.681 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:45.681 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty PASSED 2024-06-28T15:33:45.681 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:45.681 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch STARTED 2024-06-28T15:33:45.779 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:45.779 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch PASSED 2024-06-28T15:33:46.179 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:46.179 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable STARTED 2024-06-28T15:33:46.279 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:46.279 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable PASSED 2024-06-28T15:33:46.779 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:46.779 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete STARTED 2024-06-28T15:33:47.179 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:47.179 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete PASSED 2024-06-28T15:33:47.279 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:47.279 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty STARTED 2024-06-28T15:33:47.579 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:47.579 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty PASSED 2024-06-28T15:33:48.079 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:48.079 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist STARTED 2024-06-28T15:33:48.179 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:48.179 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist PASSED 2024-06-28T15:33:48.279 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:48.279 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct STARTED 2024-06-28T15:33:48.679 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:48.679 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct PASSED 2024-06-28T15:33:49.479 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:49.479 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty STARTED 2024-06-28T15:33:49.679 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:49.679 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty PASSED 2024-06-28T15:33:50.179 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:50.179 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketNotExist STARTED 2024-06-28T15:33:50.279 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:50.279 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Bucket tests > BucketTest.testBucketNotExist PASSED 2024-06-28T15:33:50.380 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:50.380 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------- 2024-06-28T15:33:50.380 INFO:teuthology.orchestra.run.smithi178.stdout:| Results: SUCCESS (9 tests, 9 successes, 0 failures, 0 skipped) | 2024-06-28T15:33:50.380 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------- 2024-06-28T15:33:50.381 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:50.381 INFO:teuthology.orchestra.run.smithi178.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-06-28T15:33:50.381 INFO:teuthology.orchestra.run.smithi178.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-06-28T15:33:50.381 INFO:teuthology.orchestra.run.smithi178.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-06-28T15:33:50.381 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:50.381 INFO:teuthology.orchestra.run.smithi178.stdout:BUILD SUCCESSFUL in 8s 2024-06-28T15:33:50.381 INFO:teuthology.orchestra.run.smithi178.stdout:5 actionable tasks: 5 executed 2024-06-28T15:33:50.389 DEBUG:teuthology.orchestra.run.smithi178:> radosgw-admin gc process --include-all 2024-06-28T15:33:50.415 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:33:50.415 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:33:50.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.469+0000 7f54ea594980 20 rados->read ofs=0 len=0 2024-06-28T15:33:50.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.473+0000 7f54ea594980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:50.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.473+0000 7f54ea594980 20 realm 2024-06-28T15:33:50.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.473+0000 7f54ea594980 20 rados->read ofs=0 len=0 2024-06-28T15:33:50.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.473+0000 7f54ea594980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:50.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.473+0000 7f54ea594980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:33:50.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.473+0000 7f54ea594980 20 rados->read ofs=0 len=0 2024-06-28T15:33:50.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.473+0000 7f54ea594980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:50.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.473+0000 7f54ea594980 20 rados->read ofs=0 len=0 2024-06-28T15:33:50.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.473+0000 7f54ea594980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:50.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.473+0000 7f54ea594980 20 rados->read ofs=0 len=0 2024-06-28T15:33:50.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.473+0000 7f54ea594980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:33:50.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.473+0000 7f54ea594980 20 searching for the correct realm 2024-06-28T15:33:50.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.481+0000 7f54e88a6640 0 -- 172.21.15.178:0/3580970760 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f54dc008050 msgr2=0x7f54dc051950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:33:50.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.697+0000 7f54e80a5640 0 -- 172.21.15.178:0/3580970760 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55d5c0fbd740 msgr2=0x55d5c0fbfb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:33:50.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.705+0000 7f54ea594980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:33:50.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.705+0000 7f54ea594980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:33:50.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.705+0000 7f54ea594980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:33:50.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.705+0000 7f54ea594980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:33:50.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.705+0000 7f54ea594980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:33:50.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.705+0000 7f54ea594980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:33:50.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.705+0000 7f54ea594980 20 rados->read ofs=0 len=0 2024-06-28T15:33:50.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.705+0000 7f54ea594980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:50.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.705+0000 7f54ea594980 20 rados->read ofs=0 len=0 2024-06-28T15:33:50.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.705+0000 7f54ea594980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:50.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.705+0000 7f54ea594980 20 rados->read ofs=0 len=0 2024-06-28T15:33:50.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:33:50.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 20 zone default found 2024-06-28T15:33:50.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 4 Realm: () 2024-06-28T15:33:50.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:33:50.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:33:50.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:33:50.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 20 zonegroup default 2024-06-28T15:33:50.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 20 rados->read ofs=0 len=0 2024-06-28T15:33:50.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:50.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 20 rados->read ofs=0 len=0 2024-06-28T15:33:50.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:50.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 20 rados->read ofs=0 len=0 2024-06-28T15:33:50.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:50.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 20 started sync module instance, tier type = 2024-06-28T15:33:50.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.709+0000 7f54ea594980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:33:50.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.737+0000 7f54ea594980 20 add_watcher() i=0 2024-06-28T15:33:50.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.737+0000 7f54ea594980 20 add_watcher() i=1 2024-06-28T15:33:50.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.737+0000 7f54ea594980 20 add_watcher() i=2 2024-06-28T15:33:50.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.737+0000 7f54ea594980 20 add_watcher() i=3 2024-06-28T15:33:50.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.737+0000 7f54ea594980 20 add_watcher() i=4 2024-06-28T15:33:50.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.737+0000 7f54ea594980 20 add_watcher() i=5 2024-06-28T15:33:50.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.741+0000 7f54ea594980 20 add_watcher() i=6 2024-06-28T15:33:50.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.741+0000 7f54ea594980 20 add_watcher() i=7 2024-06-28T15:33:50.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.741+0000 7f54ea594980 2 all 8 watchers are set, enabling cache 2024-06-28T15:33:50.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.745+0000 7f539e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:50.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.745+0000 7f54ea594980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:33:50.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.745+0000 7f54ea594980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:33:50.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.837+0000 7f539d7e2640 20 reqs_thread_entry: start 2024-06-28T15:33:50.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.837+0000 7f54ea594980 20 init_complete bucket index max shards: 1 2024-06-28T15:33:50.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.837+0000 7f54ea594980 20 Filter name: none 2024-06-28T15:33:50.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.837+0000 7f530effd640 20 reqs_thread_entry: start 2024-06-28T15:33:50.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.837+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-06-28T15:33:50.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.841+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.841+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.841+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.845+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-06-28T15:33:50.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.845+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.849+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.849+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.849+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-06-28T15:33:50.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.853+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.853+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.857+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.857+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-06-28T15:33:50.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.861+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.865+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.865+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.869+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-06-28T15:33:50.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.873+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.873+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.873+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.877+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-06-28T15:33:50.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.881+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.881+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.881+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.881+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-06-28T15:33:50.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.885+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.889+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.878 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.889+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.893+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-06-28T15:33:50.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.893+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.897+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.897+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.897+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-06-28T15:33:50.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.901+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.901+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.901+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.905+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-06-28T15:33:50.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.909+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.909+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.909+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.913+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-06-28T15:33:50.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.917+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.917+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.917+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.921+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-06-28T15:33:50.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.925+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.925+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.925+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:50.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.925+0000 7f54ea594980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8962103160998576540', time=2024-06-28T17:33:47.749276+0000, chain.objs.size()=1 2024-06-28T15:33:50.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.925+0000 7f54ea594980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.61__shadow_.LVlyVfRU7DQxB-jJPqf59wtpKv-HWPX_0 2024-06-28T15:33:50.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.929+0000 7f54ea594980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:50.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.933+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-06-28T15:33:50.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.937+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.937+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.937+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.941+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-06-28T15:33:50.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.941+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.941+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.945+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.945+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-06-28T15:33:50.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.949+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.949+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.953+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.953+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-06-28T15:33:50.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.957+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.961+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.961+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.965+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-06-28T15:33:50.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.969+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.969+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.969+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:33:50.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.969+0000 7f54ea594980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4959410683378818680', time=2024-06-28T17:33:48.853497+0000, chain.objs.size()=1 2024-06-28T15:33:50.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.969+0000 7f54ea594980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.62__shadow_.Jq-rMfPwoLxkLWmBjefD_Rjb-Qu8D56_0 2024-06-28T15:33:50.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.973+0000 7f54ea594980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:33:50.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.977+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-06-28T15:33:50.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.977+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.981+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.981+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.985+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-06-28T15:33:50.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.989+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.993+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.993+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.997+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-06-28T15:33:50.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:50.997+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.001+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.001+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.005+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-06-28T15:33:50.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.009+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:50.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.009+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:50.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.009+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.013+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-06-28T15:33:51.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.017+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.017+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.017+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.021+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-06-28T15:33:51.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.025+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.025+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.029+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.029+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-06-28T15:33:51.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.033+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.037+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.037+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.041+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-06-28T15:33:51.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.045+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.045+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.045+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.049+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-06-28T15:33:51.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.053+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.053+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.053+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.057+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-06-28T15:33:51.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.061+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.061+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.065+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.065+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-06-28T15:33:51.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.069+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.073+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.073+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.077+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-06-28T15:33:51.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.081+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.081+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.085+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.085+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-06-28T15:33:51.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.089+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.089+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.089+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.093+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-06-28T15:33:51.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.097+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.101+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.101+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.105+0000 7f54ea594980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-06-28T15:33:51.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.109+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.109+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.109+0000 7f54ea594980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.117+0000 7f539e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:51.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.121+0000 7f54ea594980 20 remove_watcher() i=0 2024-06-28T15:33:51.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.121+0000 7f54ea594980 2 removed watcher, disabling cache 2024-06-28T15:33:51.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.125+0000 7f54ea594980 20 remove_watcher() i=1 2024-06-28T15:33:51.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.125+0000 7f54ea594980 20 remove_watcher() i=2 2024-06-28T15:33:51.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.129+0000 7f54ea594980 20 remove_watcher() i=3 2024-06-28T15:33:51.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.133+0000 7f54ea594980 20 remove_watcher() i=4 2024-06-28T15:33:51.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.133+0000 7f54ea594980 20 remove_watcher() i=5 2024-06-28T15:33:51.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.137+0000 7f54ea594980 20 remove_watcher() i=6 2024-06-28T15:33:51.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.137+0000 7f54ea594980 20 remove_watcher() i=7 2024-06-28T15:33:51.149 DEBUG:teuthology.orchestra.run.smithi178:> radosgw-admin gc process --include-all 2024-06-28T15:33:51.223 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:33:51.223 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:33:51.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.277+0000 7faf9475a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:51.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.281+0000 7faf9475a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:51.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.281+0000 7faf9475a980 20 realm 2024-06-28T15:33:51.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.281+0000 7faf9475a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:51.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.281+0000 7faf9475a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:51.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.281+0000 7faf9475a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:33:51.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.281+0000 7faf9475a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:51.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.281+0000 7faf9475a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:51.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.281+0000 7faf9475a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:51.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.281+0000 7faf9475a980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:51.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.281+0000 7faf9475a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:51.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.285+0000 7faf9475a980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:33:51.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.285+0000 7faf9475a980 20 searching for the correct realm 2024-06-28T15:33:51.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.289+0000 7faf92a6c640 0 -- 172.21.15.178:0/2927308916 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7faf7c0fcb50 msgr2=0x7faf7c0fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:33:51.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.505+0000 7faf9226b640 0 -- 172.21.15.178:0/2927308916 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x55afc8a0eef0 msgr2=0x55afc8a112e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:33:51.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.509+0000 7faf9475a980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:33:51.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.509+0000 7faf9475a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:33:51.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.509+0000 7faf9475a980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:33:51.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.509+0000 7faf9475a980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:33:51.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.509+0000 7faf9475a980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:33:51.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.509+0000 7faf9475a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:33:51.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:51.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:51.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:51.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:33:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 20 zone default found 2024-06-28T15:33:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 4 Realm: () 2024-06-28T15:33:51.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:33:51.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:33:51.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:33:51.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 20 zonegroup default 2024-06-28T15:33:51.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:51.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:51.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:51.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:51.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.513+0000 7faf9475a980 20 rados->read ofs=0 len=0 2024-06-28T15:33:51.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.517+0000 7faf9475a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:51.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.517+0000 7faf9475a980 20 started sync module instance, tier type = 2024-06-28T15:33:51.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.517+0000 7faf9475a980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:33:51.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.537+0000 7faf9475a980 20 add_watcher() i=0 2024-06-28T15:33:51.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.537+0000 7faf9475a980 20 add_watcher() i=1 2024-06-28T15:33:51.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.537+0000 7faf9475a980 20 add_watcher() i=2 2024-06-28T15:33:51.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.537+0000 7faf9475a980 20 add_watcher() i=3 2024-06-28T15:33:51.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.537+0000 7faf9475a980 20 add_watcher() i=4 2024-06-28T15:33:51.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.537+0000 7faf9475a980 20 add_watcher() i=5 2024-06-28T15:33:51.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.537+0000 7faf9475a980 20 add_watcher() i=6 2024-06-28T15:33:51.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.537+0000 7faf9475a980 20 add_watcher() i=7 2024-06-28T15:33:51.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.537+0000 7faf9475a980 2 all 8 watchers are set, enabling cache 2024-06-28T15:33:51.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.541+0000 7fae48fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:51.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.541+0000 7faf9475a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:33:51.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.541+0000 7faf9475a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:33:51.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.625+0000 7fadc37fe640 20 reqs_thread_entry: start 2024-06-28T15:33:51.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.625+0000 7faf9475a980 20 init_complete bucket index max shards: 1 2024-06-28T15:33:51.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.625+0000 7faf9475a980 20 Filter name: none 2024-06-28T15:33:51.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.625+0000 7fadc17fa640 20 reqs_thread_entry: start 2024-06-28T15:33:51.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.625+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-06-28T15:33:51.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.629+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.633+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.633+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.637+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-06-28T15:33:51.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.641+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.641+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.641+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.645+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-06-28T15:33:51.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.649+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.649+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.649+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.653+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-06-28T15:33:51.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.657+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.657+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.657+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.657+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-06-28T15:33:51.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.661+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.665+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.665+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.669+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-06-28T15:33:51.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.669+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.673+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.673+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.673+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-06-28T15:33:51.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.677+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.677+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.677+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.677+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-06-28T15:33:51.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.681+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.685+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.685+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.685+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-06-28T15:33:51.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.689+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.693+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.693+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.697+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-06-28T15:33:51.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.697+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.697+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.697+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.701+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-06-28T15:33:51.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.705+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.705+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.705+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.709+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-06-28T15:33:51.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.713+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.713+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.713+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.713+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-06-28T15:33:51.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.717+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.717+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.721+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.721+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-06-28T15:33:51.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.725+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.716 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.725+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.729+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.729+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-06-28T15:33:51.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.733+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.733+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.737+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.737+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-06-28T15:33:51.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.741+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.741+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.741+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.745+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-06-28T15:33:51.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.749+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.749+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.749+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.749+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-06-28T15:33:51.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.753+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.753+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.753+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.757+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-06-28T15:33:51.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.761+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.761+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.761+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.765+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-06-28T15:33:51.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.769+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.769+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.769+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.773+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-06-28T15:33:51.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.773+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.777+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.777+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.777+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-06-28T15:33:51.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.781+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.781+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.781+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.785+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-06-28T15:33:51.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.785+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.789+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.789+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.789+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-06-28T15:33:51.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.793+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.793+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.793+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.793+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-06-28T15:33:51.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.797+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.801+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.801+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.801+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-06-28T15:33:51.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.805+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.805+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.805+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.809+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-06-28T15:33:51.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.809+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.809+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.813+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.813+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-06-28T15:33:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.817+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.817+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.817+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.817+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-06-28T15:33:51.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.821+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.825+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.825+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.829+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-06-28T15:33:51.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.833+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.833+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.833+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.837+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-06-28T15:33:51.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.841+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.841+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.841+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.845+0000 7faf9475a980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-06-28T15:33:51.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.849+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.849+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:51.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.849+0000 7faf9475a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:51.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.853+0000 7fae48fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:51.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.861+0000 7faf9475a980 20 remove_watcher() i=0 2024-06-28T15:33:51.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.861+0000 7faf9475a980 2 removed watcher, disabling cache 2024-06-28T15:33:51.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.861+0000 7faf9475a980 20 remove_watcher() i=1 2024-06-28T15:33:51.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.865+0000 7faf9475a980 20 remove_watcher() i=2 2024-06-28T15:33:51.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.865+0000 7faf9475a980 20 remove_watcher() i=3 2024-06-28T15:33:51.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.869+0000 7faf9475a980 20 remove_watcher() i=4 2024-06-28T15:33:51.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.873+0000 7faf9475a980 20 remove_watcher() i=5 2024-06-28T15:33:51.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.873+0000 7faf9475a980 20 remove_watcher() i=6 2024-06-28T15:33:51.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:51.877+0000 7faf9475a980 20 remove_watcher() i=7 2024-06-28T15:33:51.882 DEBUG:teuthology.orchestra.run.smithi178:> radosgw-admin gc process --include-all 2024-06-28T15:33:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:33:51.955 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:33:52.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.013+0000 7f2781c29980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.013+0000 7f2781c29980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:52.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.013+0000 7f2781c29980 20 realm 2024-06-28T15:33:52.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.013+0000 7f2781c29980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.013+0000 7f2781c29980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:52.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.013+0000 7f2781c29980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:33:52.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.013+0000 7f2781c29980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.013+0000 7f2781c29980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:52.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.013+0000 7f2781c29980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.017+0000 7f2781c29980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:52.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.017+0000 7f2781c29980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.017+0000 7f2781c29980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:33:52.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.017+0000 7f2781c29980 20 searching for the correct realm 2024-06-28T15:33:52.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.021+0000 7f277ef39640 0 -- 172.21.15.178:0/362603783 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x55c140e96b90 msgr2=0x55c140e98fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:33:52.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.029+0000 7f277f73a640 0 -- 172.21.15.178:0/362603783 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f2774096460 msgr2=0x7f2774065120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:33:52.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:33:52.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:33:52.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:33:52.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:33:52.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:33:52.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:33:52.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:52.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:52.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:33:52.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 zone default found 2024-06-28T15:33:52.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 4 Realm: () 2024-06-28T15:33:52.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:33:52.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:33:52.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:33:52.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 zonegroup default 2024-06-28T15:33:52.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:52.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:52.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:52.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 started sync module instance, tier type = 2024-06-28T15:33:52.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.037+0000 7f2781c29980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:33:52.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.053+0000 7f2781c29980 20 add_watcher() i=0 2024-06-28T15:33:52.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.053+0000 7f2781c29980 20 add_watcher() i=1 2024-06-28T15:33:52.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.053+0000 7f2781c29980 20 add_watcher() i=2 2024-06-28T15:33:52.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.053+0000 7f2781c29980 20 add_watcher() i=3 2024-06-28T15:33:52.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.053+0000 7f2781c29980 20 add_watcher() i=4 2024-06-28T15:33:52.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.053+0000 7f2781c29980 20 add_watcher() i=5 2024-06-28T15:33:52.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.053+0000 7f2781c29980 20 add_watcher() i=6 2024-06-28T15:33:52.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.057+0000 7f2781c29980 20 add_watcher() i=7 2024-06-28T15:33:52.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.057+0000 7f2781c29980 2 all 8 watchers are set, enabling cache 2024-06-28T15:33:52.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.057+0000 7f2635fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:52.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.057+0000 7f2781c29980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:33:52.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.057+0000 7f2781c29980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:33:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.137+0000 7f2634fe1640 20 reqs_thread_entry: start 2024-06-28T15:33:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.137+0000 7f2781c29980 20 init_complete bucket index max shards: 1 2024-06-28T15:33:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.137+0000 7f2781c29980 20 Filter name: none 2024-06-28T15:33:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.137+0000 7f25ae7fc640 20 reqs_thread_entry: start 2024-06-28T15:33:52.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.137+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-06-28T15:33:52.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.141+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.145+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.145+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.149+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-06-28T15:33:52.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.149+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.153+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.153+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.153+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-06-28T15:33:52.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.157+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.157+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.157+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.161+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-06-28T15:33:52.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.165+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.165+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.165+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.169+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-06-28T15:33:52.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.173+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.173+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.173+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.177+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-06-28T15:33:52.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.181+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.181+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.181+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.185+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-06-28T15:33:52.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.189+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.189+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.189+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.193+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-06-28T15:33:52.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.197+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.201+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.201+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.205+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-06-28T15:33:52.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.205+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.209+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.209+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.213+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-06-28T15:33:52.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.213+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.217+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.217+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.221+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-06-28T15:33:52.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.429+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.429+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.429+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.433+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-06-28T15:33:52.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.437+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.441+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.441+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.441+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-06-28T15:33:52.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.445+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.449+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.449+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.453+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-06-28T15:33:52.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.457+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.457+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.457+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.461+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-06-28T15:33:52.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.465+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.469+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.469+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.473+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-06-28T15:33:52.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.477+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.477+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.477+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.485+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-06-28T15:33:52.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.489+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.489+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.489+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.493+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-06-28T15:33:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.497+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.497+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.501+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.501+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-06-28T15:33:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.505+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.509+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.509+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.513+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-06-28T15:33:52.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.513+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.517+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.517+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.521+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-06-28T15:33:52.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.521+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.525+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.525+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.529+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-06-28T15:33:52.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.533+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.533+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.533+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.537+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-06-28T15:33:52.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.541+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.541+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.541+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.545+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-06-28T15:33:52.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.549+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.549+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.549+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.553+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-06-28T15:33:52.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.557+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.557+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.561+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.561+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-06-28T15:33:52.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.565+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.569+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.569+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.573+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-06-28T15:33:52.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.573+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.577+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.577+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.581+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-06-28T15:33:52.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.585+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.585+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.589+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.589+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-06-28T15:33:52.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.593+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.597+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.597+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.601+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-06-28T15:33:52.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.605+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.605+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.609+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.609+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-06-28T15:33:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.613+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.613+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.617+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.617+0000 7f2781c29980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-06-28T15:33:52.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.621+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.625+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:52.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.625+0000 7f2781c29980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:52.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.629+0000 7f2635fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:52.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.637+0000 7f2781c29980 20 remove_watcher() i=0 2024-06-28T15:33:52.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.637+0000 7f2781c29980 2 removed watcher, disabling cache 2024-06-28T15:33:52.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.637+0000 7f2781c29980 20 remove_watcher() i=1 2024-06-28T15:33:52.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.641+0000 7f2781c29980 20 remove_watcher() i=2 2024-06-28T15:33:52.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.645+0000 7f2781c29980 20 remove_watcher() i=3 2024-06-28T15:33:52.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.645+0000 7f2781c29980 20 remove_watcher() i=4 2024-06-28T15:33:52.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.649+0000 7f2781c29980 20 remove_watcher() i=5 2024-06-28T15:33:52.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.649+0000 7f2781c29980 20 remove_watcher() i=6 2024-06-28T15:33:52.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.653+0000 7f2781c29980 20 remove_watcher() i=7 2024-06-28T15:33:52.660 DEBUG:teuthology.orchestra.run.smithi178:> radosgw-admin gc process --include-all 2024-06-28T15:33:52.740 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:33:52.740 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:33:52.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.793+0000 7fa5a9bf4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.797+0000 7fa5a9bf4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.797+0000 7fa5a9bf4980 20 realm 2024-06-28T15:33:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.797+0000 7fa5a9bf4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.797+0000 7fa5a9bf4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:52.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.797+0000 7fa5a9bf4980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:33:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.797+0000 7fa5a9bf4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.797+0000 7fa5a9bf4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:52.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.797+0000 7fa5a9bf4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.797+0000 7fa5a9bf4980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:52.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.797+0000 7fa5a9bf4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.801+0000 7fa5a9bf4980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:33:52.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.801+0000 7fa5a9bf4980 20 searching for the correct realm 2024-06-28T15:33:52.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:52.801+0000 7fa5a7705640 0 -- 172.21.15.178:0/3403081762 >> [v2:172.21.15.178:6823/1880098663,v1:172.21.15.178:6825/1880098663] conn(0x7fa5a001f040 msgr2=0x7fa5a000f430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:33:53.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.013+0000 7fa5a7705640 0 -- 172.21.15.178:0/3403081762 >> [v2:172.21.15.178:6823/1880098663,v1:172.21.15.178:6825/1880098663] conn(0x7fa5a001f040 msgr2=0x7fa5a000f430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:33:53.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:33:53.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:33:53.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:33:53.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:33:53.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:33:53.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:33:53.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:53.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:53.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:53.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:33:53.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:53.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:33:53.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 zone default found 2024-06-28T15:33:53.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 4 Realm: () 2024-06-28T15:33:53.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:33:53.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:33:53.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:33:53.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 zonegroup default 2024-06-28T15:33:53.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:53.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:53.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.033+0000 7fa5a9bf4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:53.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.037+0000 7fa5a9bf4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:53.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.037+0000 7fa5a9bf4980 20 rados->read ofs=0 len=0 2024-06-28T15:33:53.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.037+0000 7fa5a9bf4980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:33:53.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.037+0000 7fa5a9bf4980 20 started sync module instance, tier type = 2024-06-28T15:33:53.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.037+0000 7fa5a9bf4980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:33:53.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.057+0000 7fa5a9bf4980 20 add_watcher() i=0 2024-06-28T15:33:53.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.057+0000 7fa5a9bf4980 20 add_watcher() i=1 2024-06-28T15:33:53.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.057+0000 7fa5a9bf4980 20 add_watcher() i=2 2024-06-28T15:33:53.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.057+0000 7fa5a9bf4980 20 add_watcher() i=3 2024-06-28T15:33:53.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.057+0000 7fa5a9bf4980 20 add_watcher() i=4 2024-06-28T15:33:53.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.057+0000 7fa5a9bf4980 20 add_watcher() i=5 2024-06-28T15:33:53.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.057+0000 7fa5a9bf4980 20 add_watcher() i=6 2024-06-28T15:33:53.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.061+0000 7fa5a9bf4980 20 add_watcher() i=7 2024-06-28T15:33:53.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.061+0000 7fa5a9bf4980 2 all 8 watchers are set, enabling cache 2024-06-28T15:33:53.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.065+0000 7fa45dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:53.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.065+0000 7fa5a9bf4980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:33:53.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.065+0000 7fa5a9bf4980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:33:53.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.157+0000 7fa45cfe1640 20 reqs_thread_entry: start 2024-06-28T15:33:53.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.157+0000 7fa5a9bf4980 20 init_complete bucket index max shards: 1 2024-06-28T15:33:53.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.157+0000 7fa5a9bf4980 20 Filter name: none 2024-06-28T15:33:53.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.157+0000 7fa3d67fc640 20 reqs_thread_entry: start 2024-06-28T15:33:53.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.157+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-06-28T15:33:53.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.161+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.161+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.161+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.161+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-06-28T15:33:53.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.165+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.165+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.169+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.169+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-06-28T15:33:53.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.173+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.177+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.177+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.177+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-06-28T15:33:53.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.181+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.185+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.185+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.189+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-06-28T15:33:53.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.189+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.193+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.193+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.193+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-06-28T15:33:53.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.197+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.197+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.201+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.201+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-06-28T15:33:53.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.205+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.205+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.205+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.209+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-06-28T15:33:53.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.213+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.213+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.213+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.217+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-06-28T15:33:53.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.221+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.221+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.221+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.225+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-06-28T15:33:53.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.229+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.229+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.229+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.233+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-06-28T15:33:53.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.237+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.237+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.237+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.241+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-06-28T15:33:53.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.245+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.245+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.245+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.249+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-06-28T15:33:53.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.249+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.253+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.253+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.253+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-06-28T15:33:53.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.257+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.257+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.261+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.261+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-06-28T15:33:53.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.265+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.269+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.269+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.273+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-06-28T15:33:53.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.277+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.277+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.277+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.281+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-06-28T15:33:53.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.285+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.285+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.285+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.289+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-06-28T15:33:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.293+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.293+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.297+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.297+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-06-28T15:33:53.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.301+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.301+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.305+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.305+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-06-28T15:33:53.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.309+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.313+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.313+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.313+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-06-28T15:33:53.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.317+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.321+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.321+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.325+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-06-28T15:33:53.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.325+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.329+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.329+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.329+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-06-28T15:33:53.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.333+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.337+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.337+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.341+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-06-28T15:33:53.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.345+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.345+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.349+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.349+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-06-28T15:33:53.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.353+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.357+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.357+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.357+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-06-28T15:33:53.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.361+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.365+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.365+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.369+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-06-28T15:33:53.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.373+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.373+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.373+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.377+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-06-28T15:33:53.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.381+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.381+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.381+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.385+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-06-28T15:33:53.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.389+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.389+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.393+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.393+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-06-28T15:33:53.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.397+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.401+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.401+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.401+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-06-28T15:33:53.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.405+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.409+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.409+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.413+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-06-28T15:33:53.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.417+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.417+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:33:53.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.417+0000 7fa5a9bf4980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:33:53.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.425+0000 7fa45dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:33:53.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.429+0000 7fa5a9bf4980 20 remove_watcher() i=0 2024-06-28T15:33:53.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.429+0000 7fa5a9bf4980 2 removed watcher, disabling cache 2024-06-28T15:33:53.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.433+0000 7fa5a9bf4980 20 remove_watcher() i=1 2024-06-28T15:33:53.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.433+0000 7fa5a9bf4980 20 remove_watcher() i=2 2024-06-28T15:33:53.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.437+0000 7fa5a9bf4980 20 remove_watcher() i=3 2024-06-28T15:33:53.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.441+0000 7fa5a9bf4980 20 remove_watcher() i=4 2024-06-28T15:33:53.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.441+0000 7fa5a9bf4980 20 remove_watcher() i=5 2024-06-28T15:33:53.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.445+0000 7fa5a9bf4980 20 remove_watcher() i=6 2024-06-28T15:33:53.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:33:53.445+0000 7fa5a9bf4980 20 remove_watcher() i=7 2024-06-28T15:33:53.454 DEBUG:teuthology.orchestra.run.smithi178:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests ObjectTest 2024-06-28T15:33:54.150 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :clean 2024-06-28T15:33:54.947 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :compileJava 2024-06-28T15:33:54.947 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :processResources 2024-06-28T15:33:54.947 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :classes 2024-06-28T15:33:55.448 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:55.448 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :compileTestJava 2024-06-28T15:33:55.448 INFO:teuthology.orchestra.run.smithi178.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-06-28T15:33:55.448 INFO:teuthology.orchestra.run.smithi178.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-06-28T15:33:55.547 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:55.548 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :processTestResources NO-SOURCE 2024-06-28T15:33:55.548 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :testClasses 2024-06-28T15:33:57.447 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:57.448 INFO:teuthology.orchestra.run.smithi178.stdout:> Task :test 2024-06-28T15:33:57.448 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:57.448 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests STANDARD_OUT 2024-06-28T15:33:57.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28 15:33:56,916 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-06-28T15:33:57.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:57.449 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound STARTED 2024-06-28T15:33:57.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:57.449 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound PASSED 2024-06-28T15:33:58.047 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone STARTED 2024-06-28T15:33:58.247 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:58.247 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone PASSED 2024-06-28T15:33:58.747 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:58.747 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty STARTED 2024-06-28T15:33:58.947 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:58.947 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty PASSED 2024-06-28T15:33:59.447 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:33:59.447 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI STARTED 2024-06-28T15:34:11.948 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:11.948 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI PASSED 2024-06-28T15:34:12.447 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:12.448 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI STARTED 2024-06-28T15:34:12.647 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:12.648 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI PASSED 2024-06-28T15:34:13.047 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:13.048 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI STARTED 2024-06-28T15:34:13.447 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:13.448 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI PASSED 2024-06-28T15:34:13.947 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:13.948 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI STARTED 2024-06-28T15:34:14.048 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:14.048 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI PASSED 2024-06-28T15:34:14.148 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:14.148 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI STARTED 2024-06-28T15:34:14.347 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:14.348 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI PASSED 2024-06-28T15:34:14.747 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:14.747 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB STARTED 2024-06-28T15:34:16.947 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:16.948 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB PASSED 2024-06-28T15:34:17.547 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:17.548 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b STARTED 2024-06-28T15:34:17.647 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:17.647 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b PASSED 2024-06-28T15:34:18.247 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:18.248 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb STARTED 2024-06-28T15:34:18.448 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:18.448 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb PASSED 2024-06-28T15:34:18.947 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:18.948 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC STARTED 2024-06-28T15:34:19.150 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:19.151 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC PASSED 2024-06-28T15:34:19.547 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:19.548 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 STARTED 2024-06-28T15:34:19.748 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:19.748 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 PASSED 2024-06-28T15:34:20.248 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:20.248 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey STARTED 2024-06-28T15:34:20.348 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:20.348 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey PASSED 2024-06-28T15:34:20.847 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:20.848 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 STARTED 2024-06-28T15:34:21.048 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:21.048 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 PASSED 2024-06-28T15:34:21.448 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:21.448 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete STARTED 2024-06-28T15:34:21.848 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:21.848 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete PASSED 2024-06-28T15:34:22.247 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:22.248 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA STARTED 2024-06-28T15:34:25.848 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:25.848 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA PASSED 2024-06-28T15:34:26.748 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:26.748 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI STARTED 2024-06-28T15:34:59.748 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:34:59.749 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI PASSED 2024-06-28T15:35:00.648 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:00.648 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI STARTED 2024-06-28T15:35:00.748 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:00.748 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI PASSED 2024-06-28T15:35:01.248 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:01.248 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI STARTED 2024-06-28T15:35:01.448 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:01.448 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI PASSED 2024-06-28T15:35:02.148 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:02.148 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI STARTED 2024-06-28T15:35:05.448 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:05.448 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI PASSED 2024-06-28T15:35:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:05.949 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI STARTED 2024-06-28T15:35:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:05.949 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI PASSED 2024-06-28T15:35:06.148 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:06.148 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI STARTED 2024-06-28T15:35:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:10.848 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI PASSED 2024-06-28T15:35:11.648 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:11.649 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests STANDARD_OUT 2024-06-28T15:35:11.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28 15:35:11,626 [s3-transfer-manager-worker-10] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-06-28T15:35:11.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28 15:35:11,626 [s3-transfer-manager-worker-9] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-06-28T15:35:12.148 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:12.148 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 STARTED 2024-06-28T15:35:18.048 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:18.049 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 PASSED 2024-06-28T15:35:18.849 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:18.849 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI STARTED 2024-06-28T15:35:30.948 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:30.949 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI PASSED 2024-06-28T15:35:31.448 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:35:31.548 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI STARTED 2024-06-28T15:36:17.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:17.750 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI PASSED 2024-06-28T15:36:18.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:18.249 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI STARTED 2024-06-28T15:36:22.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:22.449 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI PASSED 2024-06-28T15:36:22.849 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:22.849 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI STARTED 2024-06-28T15:36:26.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:26.649 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI PASSED 2024-06-28T15:36:27.149 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:27.149 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause STARTED 2024-06-28T15:36:29.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:29.649 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause PASSED 2024-06-28T15:36:37.849 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:37.849 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound STARTED 2024-06-28T15:36:38.049 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:38.049 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound PASSED 2024-06-28T15:36:38.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:38.249 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch STARTED 2024-06-28T15:36:38.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:38.649 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch PASSED 2024-06-28T15:36:43.049 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:43.050 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort STARTED 2024-06-28T15:36:43.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:43.249 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort PASSED 2024-06-28T15:36:43.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:43.749 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid STARTED 2024-06-28T15:36:43.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:43.949 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid PASSED 2024-06-28T15:36:44.351 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:44.351 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None STARTED 2024-06-28T15:36:44.549 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:44.549 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None PASSED 2024-06-28T15:36:44.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:44.949 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable STARTED 2024-06-28T15:36:45.149 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:45.149 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable PASSED 2024-06-28T15:36:45.549 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:45.549 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes STARTED 2024-06-28T15:36:45.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:45.749 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes PASSED 2024-06-28T15:36:46.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:46.249 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot STARTED 2024-06-28T15:36:46.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:46.749 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot PASSED 2024-06-28T15:36:47.549 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:47.549 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist STARTED 2024-06-28T15:36:48.049 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:48.049 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist PASSED 2024-06-28T15:36:48.849 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:48.849 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage STARTED 2024-06-28T15:36:49.349 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:49.349 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage PASSED 2024-06-28T15:36:50.049 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:50.049 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList STARTED 2024-06-28T15:36:50.549 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:50.550 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList PASSED 2024-06-28T15:36:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:51.249 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList STARTED 2024-06-28T15:36:51.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:51.749 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList PASSED 2024-06-28T15:36:52.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:52.449 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList STARTED 2024-06-28T15:36:52.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:52.949 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList PASSED 2024-06-28T15:36:53.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:53.749 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable STARTED 2024-06-28T15:36:54.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:54.249 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable PASSED 2024-06-28T15:36:54.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:54.949 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative STARTED 2024-06-28T15:36:55.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:55.449 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative PASSED 2024-06-28T15:36:56.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:56.449 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone STARTED 2024-06-28T15:36:56.849 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:56.849 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone PASSED 2024-06-28T15:36:57.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:57.649 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne STARTED 2024-06-28T15:36:58.149 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:58.149 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne PASSED 2024-06-28T15:36:58.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:58.949 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero STARTED 2024-06-28T15:36:59.349 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:36:59.349 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero PASSED 2024-06-28T15:37:00.149 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:00.149 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt STARTED 2024-06-28T15:37:00.549 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:00.549 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt PASSED 2024-06-28T15:37:01.149 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:01.149 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic STARTED 2024-06-28T15:37:01.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:01.649 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic PASSED 2024-06-28T15:37:02.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:02.249 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt STARTED 2024-06-28T15:37:02.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:02.749 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt PASSED 2024-06-28T15:37:03.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:03.449 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic STARTED 2024-06-28T15:37:03.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:03.949 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic PASSED 2024-06-28T15:37:04.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:04.749 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist STARTED 2024-06-28T15:37:05.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:05.249 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist PASSED 2024-06-28T15:37:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:05.949 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist STARTED 2024-06-28T15:37:06.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:06.449 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist PASSED 2024-06-28T15:37:07.149 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:07.149 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist STARTED 2024-06-28T15:37:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:07.649 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist PASSED 2024-06-28T15:37:08.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:08.449 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty STARTED 2024-06-28T15:37:08.849 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:08.849 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty PASSED 2024-06-28T15:37:09.549 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:09.549 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone STARTED 2024-06-28T15:37:09.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:09.949 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone PASSED 2024-06-28T15:37:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:10.649 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist STARTED 2024-06-28T15:37:11.049 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:11.049 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist PASSED 2024-06-28T15:37:11.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:11.649 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable STARTED 2024-06-28T15:37:12.049 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:12.049 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable PASSED 2024-06-28T15:37:12.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:12.749 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket STARTED 2024-06-28T15:37:12.849 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:12.849 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket PASSED 2024-06-28T15:37:13.349 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:13.349 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist STARTED 2024-06-28T15:37:13.549 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:13.549 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist PASSED 2024-06-28T15:37:13.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:13.949 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl STARTED 2024-06-28T15:37:14.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:14.250 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl PASSED 2024-06-28T15:37:14.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:14.749 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag STARTED 2024-06-28T15:37:14.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:14.949 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag PASSED 2024-06-28T15:37:15.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:15.449 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete STARTED 2024-06-28T15:37:16.049 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:16.049 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete PASSED 2024-06-28T15:37:16.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:16.449 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket STARTED 2024-06-28T15:37:16.549 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:16.549 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket PASSED 2024-06-28T15:37:16.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:16.649 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode STARTED 2024-06-28T15:37:16.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:16.949 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode PASSED 2024-06-28T15:37:17.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:17.449 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode STARTED 2024-06-28T15:37:17.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:17.749 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode PASSED 2024-06-28T15:37:18.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:18.249 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey STARTED 2024-06-28T15:37:18.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:18.649 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey PASSED 2024-06-28T15:37:19.149 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:19.149 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared STARTED 2024-06-28T15:37:19.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:19.249 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared PASSED 2024-06-28T15:37:19.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:19.749 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile STARTED 2024-06-28T15:37:23.549 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:23.549 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile PASSED 2024-06-28T15:37:24.049 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:24.049 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testUploadFileList STARTED 2024-06-28T15:37:27.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:27.249 INFO:teuthology.orchestra.run.smithi178.stdout:suite > Object tests > ObjectTest.testUploadFileList PASSED 2024-06-28T15:37:27.895 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:27.895 INFO:teuthology.orchestra.run.smithi178.stdout:---------------------------------------------------------------------- 2024-06-28T15:37:27.895 INFO:teuthology.orchestra.run.smithi178.stdout:| Results: SUCCESS (70 tests, 70 successes, 0 failures, 0 skipped) | 2024-06-28T15:37:27.895 INFO:teuthology.orchestra.run.smithi178.stdout:---------------------------------------------------------------------- 2024-06-28T15:37:27.895 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:27.895 INFO:teuthology.orchestra.run.smithi178.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-06-28T15:37:27.895 INFO:teuthology.orchestra.run.smithi178.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-06-28T15:37:27.895 INFO:teuthology.orchestra.run.smithi178.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-06-28T15:37:27.896 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:27.896 INFO:teuthology.orchestra.run.smithi178.stdout:BUILD SUCCESSFUL in 3m 34s 2024-06-28T15:37:27.896 INFO:teuthology.orchestra.run.smithi178.stdout:5 actionable tasks: 5 executed 2024-06-28T15:37:27.914 DEBUG:teuthology.orchestra.run.smithi178:> radosgw-admin gc process --include-all 2024-06-28T15:37:27.941 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:27.941 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:27.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:27.995+0000 7fdc7cb57980 20 rados->read ofs=0 len=0 2024-06-28T15:37:27.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:27.999+0000 7fdc7cb57980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:27.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:27.999+0000 7fdc7cb57980 20 realm 2024-06-28T15:37:27.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:27.999+0000 7fdc7cb57980 20 rados->read ofs=0 len=0 2024-06-28T15:37:27.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:27.999+0000 7fdc7cb57980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:27.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:27.999+0000 7fdc7cb57980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:27.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:27.999+0000 7fdc7cb57980 20 rados->read ofs=0 len=0 2024-06-28T15:37:27.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:27.999+0000 7fdc7cb57980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:27.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:27.999+0000 7fdc7cb57980 20 rados->read ofs=0 len=0 2024-06-28T15:37:27.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:27.999+0000 7fdc7cb57980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:27.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:27.999+0000 7fdc7cb57980 20 rados->read ofs=0 len=0 2024-06-28T15:37:27.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:27.999+0000 7fdc7cb57980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:27.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:27.999+0000 7fdc7cb57980 20 searching for the correct realm 2024-06-28T15:37:27.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.007+0000 7fdc71e67640 0 -- 172.21.15.178:0/3027587277 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x55b7eca74c90 msgr2=0x55b7eca770a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:37:28.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.011+0000 7fdc7a668640 0 -- 172.21.15.178:0/3027587277 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fdc6406b500 msgr2=0x7fdc64075130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:28.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.227+0000 7fdc7cb57980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:28.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.227+0000 7fdc7cb57980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:28.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.227+0000 7fdc7cb57980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:28.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.227+0000 7fdc7cb57980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:28.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.227+0000 7fdc7cb57980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:28.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.227+0000 7fdc7cb57980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:28.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.227+0000 7fdc7cb57980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.227+0000 7fdc7cb57980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:28.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.227+0000 7fdc7cb57980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:28.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:28.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 20 zone default found 2024-06-28T15:37:28.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 4 Realm: () 2024-06-28T15:37:28.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:28.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:28.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:28.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 20 zonegroup default 2024-06-28T15:37:28.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:28.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:28.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:28.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 20 started sync module instance, tier type = 2024-06-28T15:37:28.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.231+0000 7fdc7cb57980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:28.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.255+0000 7fdc7cb57980 20 add_watcher() i=0 2024-06-28T15:37:28.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.255+0000 7fdc7cb57980 20 add_watcher() i=1 2024-06-28T15:37:28.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.255+0000 7fdc7cb57980 20 add_watcher() i=2 2024-06-28T15:37:28.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.255+0000 7fdc7cb57980 20 add_watcher() i=3 2024-06-28T15:37:28.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.255+0000 7fdc7cb57980 20 add_watcher() i=4 2024-06-28T15:37:28.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.255+0000 7fdc7cb57980 20 add_watcher() i=5 2024-06-28T15:37:28.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.255+0000 7fdc7cb57980 20 add_watcher() i=6 2024-06-28T15:37:28.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.259+0000 7fdc7cb57980 20 add_watcher() i=7 2024-06-28T15:37:28.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.259+0000 7fdc7cb57980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:28.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.263+0000 7fdb30fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:28.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.263+0000 7fdc7cb57980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:28.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.263+0000 7fdc7cb57980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:28.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.355+0000 7fdaab7fe640 20 reqs_thread_entry: start 2024-06-28T15:37:28.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.355+0000 7fdc7cb57980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:28.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.355+0000 7fdc7cb57980 20 Filter name: none 2024-06-28T15:37:28.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.355+0000 7fdaa97fa640 20 reqs_thread_entry: start 2024-06-28T15:37:28.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.355+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-06-28T15:37:28.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.359+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.363+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.363+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.367+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-06-28T15:37:28.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.367+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.367+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.371+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-06-28T15:37:28.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.371+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12843503601835244269', time=2024-06-28T17:34:13.617370+0000, chain.objs.size()=1 2024-06-28T15:37:28.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.371+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.71__shadow_.APKh0VKu-BM0LI2H-aKH8nRYOrfkacn_0 2024-06-28T15:37:28.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.371+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15024445357228040858', time=2024-06-28T17:36:57.056453+0000, chain.objs.size()=1 2024-06-28T15:37:28.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.371+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.110__shadow_.fGvB5sdAuXTXiRtngHKigekX2duvdTJ_0 2024-06-28T15:37:28.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.371+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12841968126126245715', time=2024-06-28T17:37:03.000428+0000, chain.objs.size()=1 2024-06-28T15:37:28.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.371+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.115__shadow_.GRf9vbDWK5whvr6LS9iQ7YjXygwpZmm_0 2024-06-28T15:37:28.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.371+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12866315502708647975', time=2024-06-28T17:37:10.282953+0000, chain.objs.size()=1 2024-06-28T15:37:28.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.371+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.121__shadow_.2OOi6mqW1LRwa3Daaajf2bXBlmOv-AC_0 2024-06-28T15:37:28.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.371+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.754295354216958535', time=2024-06-28T17:37:11.333297+0000, chain.objs.size()=1 2024-06-28T15:37:28.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.371+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.122__shadow_.dO0q8pomCvJVKhYlh212mWQYprMC8qJ_0 2024-06-28T15:37:28.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.379+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.383+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-06-28T15:37:28.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.383+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.383+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-06-28T15:37:28.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2613761793672119077', time=2024-06-28T17:34:17.148410+0000, chain.objs.size()=2 2024-06-28T15:37:28.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.73__shadow_.Afzw2sKudGWLxiLcmYRh8eN6H80kx3M_0 2024-06-28T15:37:28.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.73__shadow_.Afzw2sKudGWLxiLcmYRh8eN6H80kx3M_1 2024-06-28T15:37:28.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17152349518682172549', time=2024-06-28T17:34:18.594218+0000, chain.objs.size()=1 2024-06-28T15:37:28.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.75__shadow_.EeG46gsN17aDvycsubEtsIrCy4C8-2R_0 2024-06-28T15:37:28.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10420057670630570864', time=2024-06-28T17:35:00.337404+0000, chain.objs.size()=4 2024-06-28T15:37:28.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.83__shadow_._9Y_V9Ty6CC-09GJy2jQh-o02U8Bexl_0 2024-06-28T15:37:28.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.83__shadow_._9Y_V9Ty6CC-09GJy2jQh-o02U8Bexl_1 2024-06-28T15:37:28.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.83__shadow_._9Y_V9Ty6CC-09GJy2jQh-o02U8Bexl_2 2024-06-28T15:37:28.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.83__shadow_._9Y_V9Ty6CC-09GJy2jQh-o02U8Bexl_3 2024-06-28T15:37:28.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2556073297872110500', time=2024-06-28T17:35:46.847010+0000, chain.objs.size()=5 2024-06-28T15:37:28.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~PTRGGE7uriPLDkzi7tUhN0ogq_3VIr8.1 2024-06-28T15:37:28.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~PTRGGE7uriPLDkzi7tUhN0ogq_3VIr8.2 2024-06-28T15:37:28.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~PTRGGE7uriPLDkzi7tUhN0ogq_3VIr8.3 2024-06-28T15:37:28.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~PTRGGE7uriPLDkzi7tUhN0ogq_3VIr8.4 2024-06-28T15:37:28.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.387+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~PTRGGE7uriPLDkzi7tUhN0ogq_3VIr8.5 2024-06-28T15:37:28.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.391+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7277648389819109239', time=2024-06-28T17:36:53.169279+0000, chain.objs.size()=1 2024-06-28T15:37:28.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.391+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.107__shadow_.QUa41mEYnkKt-69Gl9XMainnhX6JEzB_0 2024-06-28T15:37:28.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.399+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7393503896060116083', time=2024-06-28T17:36:53.393050+0000, chain.objs.size()=1 2024-06-28T15:37:28.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.399+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.107__shadow_.KIsxW9QWG0H6s8RgZPype2q3c1Y0Ojt_0 2024-06-28T15:37:28.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.399+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1785394413588154057', time=2024-06-28T17:37:00.680223+0000, chain.objs.size()=1 2024-06-28T15:37:28.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.399+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.113__shadow_.C-vqZMvUzaJOh1Ru5h3AkNFc4GFSxcS_0 2024-06-28T15:37:28.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.411+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.415+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-06-28T15:37:28.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-06-28T15:37:28.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5698654348654805783', time=2024-06-28T17:36:02.774487+0000, chain.objs.size()=5 2024-06-28T15:37:28.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~TuWGJi8-4EMgWVADef_sZ3W2F6jDiVO.1 2024-06-28T15:37:28.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~TuWGJi8-4EMgWVADef_sZ3W2F6jDiVO.2 2024-06-28T15:37:28.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~TuWGJi8-4EMgWVADef_sZ3W2F6jDiVO.3 2024-06-28T15:37:28.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~TuWGJi8-4EMgWVADef_sZ3W2F6jDiVO.4 2024-06-28T15:37:28.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~TuWGJi8-4EMgWVADef_sZ3W2F6jDiVO.5 2024-06-28T15:37:28.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16722410234148192705', time=2024-06-28T17:36:49.726491+0000, chain.objs.size()=1 2024-06-28T15:37:28.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.104__shadow_.YJDvDKkb-2HQPyiW4VYxI_w0proqK8H_0 2024-06-28T15:37:28.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2631894203010631508', time=2024-06-28T17:37:00.825501+0000, chain.objs.size()=1 2024-06-28T15:37:28.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.113__shadow_.Qa2ChnmKoxMoh4qh3zRFo-M78pdcz8M_0 2024-06-28T15:37:28.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11535148416946860085', time=2024-06-28T17:37:03.142542+0000, chain.objs.size()=1 2024-06-28T15:37:28.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.115__shadow_.bRmwvAQyG30oHXTr_U1bnVVE5AJdQOc_0 2024-06-28T15:37:28.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13290005030491399390', time=2024-06-28T17:37:23.681596+0000, chain.objs.size()=5 2024-06-28T15:37:28.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.133__multipart_key1.2~kbn4QKj0GXFYnczyma69O2AUveo2_V1.1 2024-06-28T15:37:28.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.133__multipart_key1.2~kbn4QKj0GXFYnczyma69O2AUveo2_V1.2 2024-06-28T15:37:28.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.133__multipart_key1.2~kbn4QKj0GXFYnczyma69O2AUveo2_V1.3 2024-06-28T15:37:28.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.419+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.133__multipart_key1.2~kbn4QKj0GXFYnczyma69O2AUveo2_V1.4 2024-06-28T15:37:28.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.427+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.133__multipart_key1.2~kbn4QKj0GXFYnczyma69O2AUveo2_V1.5 2024-06-28T15:37:28.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.443+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.447+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-06-28T15:37:28.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-06-28T15:37:28.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6414992488480139739', time=2024-06-28T17:35:18.349955+0000, chain.objs.size()=5 2024-06-28T15:37:28.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.90__multipart_file2.mpg.2~6EDQYFkvX01LKhUW9n5t64efOvJuvTw.1 2024-06-28T15:37:28.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.90__multipart_file2.mpg.2~6EDQYFkvX01LKhUW9n5t64efOvJuvTw.2 2024-06-28T15:37:28.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.90__multipart_file2.mpg.2~6EDQYFkvX01LKhUW9n5t64efOvJuvTw.3 2024-06-28T15:37:28.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.90__multipart_file2.mpg.2~6EDQYFkvX01LKhUW9n5t64efOvJuvTw.4 2024-06-28T15:37:28.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.90__multipart_file2.mpg.2~6EDQYFkvX01LKhUW9n5t64efOvJuvTw.5 2024-06-28T15:37:28.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='2~-TTB7Qnk3ylE6eohyUV35wmhftPu-a1', time=2024-06-28T17:35:31.282372+0000, chain.objs.size()=5 2024-06-28T15:37:28.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.91__multipart_key1.2~-TTB7Qnk3ylE6eohyUV35wmhftPu-a1.7 2024-06-28T15:37:28.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.91__multipart_key1.2~-TTB7Qnk3ylE6eohyUV35wmhftPu-a1.8 2024-06-28T15:37:28.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.91__multipart_key1.2~-TTB7Qnk3ylE6eohyUV35wmhftPu-a1.9 2024-06-28T15:37:28.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.91__multipart_key1.2~-TTB7Qnk3ylE6eohyUV35wmhftPu-a1.10 2024-06-28T15:37:28.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.91__multipart_key1.2~-TTB7Qnk3ylE6eohyUV35wmhftPu-a1.11 2024-06-28T15:37:28.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2530292428229739009', time=2024-06-28T17:36:57.131016+0000, chain.objs.size()=1 2024-06-28T15:37:28.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.451+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.110__shadow_.lNtE8Bs5eRkmdNy05h8Nd7bJOFpzurr_0 2024-06-28T15:37:28.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.467+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.471+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-06-28T15:37:28.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.475+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.475+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.475+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-06-28T15:37:28.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.475+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14493271778190702681', time=2024-06-28T17:36:39.027127+0000, chain.objs.size()=1 2024-06-28T15:37:28.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.475+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.96__shadow_.TI3Rv7bHI-lCizdpKib_Q7ff1702TVY_0 2024-06-28T15:37:28.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.475+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10698579983889936886', time=2024-06-28T17:36:52.145969+0000, chain.objs.size()=1 2024-06-28T15:37:28.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.475+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.106__shadow_.aDTiR67MErU7F3vUw1Lw1w98wXNlMHp_0 2024-06-28T15:37:28.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.475+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9233167759239353481', time=2024-06-28T17:36:53.318062+0000, chain.objs.size()=1 2024-06-28T15:37:28.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.475+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.107__shadow_.L3hh_14Xf3wwIxzOGlpMszBnPlTLIBU_0 2024-06-28T15:37:28.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.475+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2996901766923758880', time=2024-06-28T17:37:04.251017+0000, chain.objs.size()=1 2024-06-28T15:37:28.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.475+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.116__shadow_.dLMoHXKnEH30Yd3ep7bGqOeDSOjK0sG_0 2024-06-28T15:37:28.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.479+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.483+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-06-28T15:37:28.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.483+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.487+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.487+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.487+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-06-28T15:37:28.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.491+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.491+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.491+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-06-28T15:37:28.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.491+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14872129961958860444', time=2024-06-28T17:36:48.470425+0000, chain.objs.size()=1 2024-06-28T15:37:28.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.491+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.103__shadow_.3fsWSsULphqx1BlKzFnbSVKgIKvvV6c_0 2024-06-28T15:37:28.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.491+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9933513645727760194', time=2024-06-28T17:37:05.401123+0000, chain.objs.size()=1 2024-06-28T15:37:28.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.491+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.117__shadow_.xhV5An84jQyodPX6Vv6tOKWVZeRM015_0 2024-06-28T15:37:28.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.491+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5463273279373417953', time=2024-06-28T17:37:27.512824+0000, chain.objs.size()=1 2024-06-28T15:37:28.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.491+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.134__shadow_.t3GWv2sweEYi_kuH4T-GrBxVfKlR4QU_0 2024-06-28T15:37:28.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.495+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.499+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-06-28T15:37:28.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.499+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.503+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.503+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-06-28T15:37:28.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.503+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17559606177575018381', time=2024-06-28T17:36:50.868078+0000, chain.objs.size()=1 2024-06-28T15:37:28.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.503+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.105__shadow_.kTbtBPdFgjBmtkiAM4IqNuG-TXlc6aZ_0 2024-06-28T15:37:28.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.503+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10168722985840406167', time=2024-06-28T17:36:56.053975+0000, chain.objs.size()=1 2024-06-28T15:37:28.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.503+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.109__shadow_.ZfgHKrMRn6PhRFS08g-LzMHQ4Tqw5wF_0 2024-06-28T15:37:28.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.503+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11890421264668396250', time=2024-06-28T17:36:58.333582+0000, chain.objs.size()=1 2024-06-28T15:37:28.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.503+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.111__shadow_.O05_kginMnJzkd_EOHtXlkn0X9QgNH-_0 2024-06-28T15:37:28.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.507+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.511+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-06-28T15:37:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-06-28T15:37:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.81__multipart_key1.2~ZKPHlJ1FU36Kf612E5mRd4QEG3fqK8H.2_bac23f98bde8ea110080d581ff47525e', time=2024-06-28T17:34:25.998789+0000, chain.objs.size()=5 2024-06-28T15:37:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.81__multipart_key1.2~ZKPHlJ1FU36Kf612E5mRd4QEG3fqK8H.1 2024-06-28T15:37:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.81__multipart_key1.2~ZKPHlJ1FU36Kf612E5mRd4QEG3fqK8H.2 2024-06-28T15:37:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.81__multipart_key1.2~ZKPHlJ1FU36Kf612E5mRd4QEG3fqK8H.3 2024-06-28T15:37:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.81__multipart_key1.2~ZKPHlJ1FU36Kf612E5mRd4QEG3fqK8H.4 2024-06-28T15:37:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.81__multipart_key1.2~ZKPHlJ1FU36Kf612E5mRd4QEG3fqK8H.5 2024-06-28T15:37:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5568744222900719460', time=2024-06-28T17:36:47.043156+0000, chain.objs.size()=1 2024-06-28T15:37:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.102__shadow_.z-0AwOOnGQ71a66IU5qKOHZkH1yLudd_0 2024-06-28T15:37:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.256826831299030197', time=2024-06-28T17:36:50.722628+0000, chain.objs.size()=1 2024-06-28T15:37:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.105__shadow_.HSwSd1SooD_BHizUTulzGUOwywKyMkl_0 2024-06-28T15:37:28.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12473851514137118627', time=2024-06-28T17:36:50.794934+0000, chain.objs.size()=1 2024-06-28T15:37:28.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.105__shadow_.J4rziUSzu0Jkk_FONBv5uy9DidNADXO_0 2024-06-28T15:37:28.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18296074582917916011', time=2024-06-28T17:37:02.928098+0000, chain.objs.size()=1 2024-06-28T15:37:28.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.115__shadow_.kdATi5LJv9aNwMV3ZryLf4f2aXQPYEe_0 2024-06-28T15:37:28.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.731603399833873497', time=2024-06-28T17:37:09.057457+0000, chain.objs.size()=1 2024-06-28T15:37:28.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.515+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.120__shadow_.Y4yvMT2ZZUScjgXPoxdYWKxHXhh8lBh_0 2024-06-28T15:37:28.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.523+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.531+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-06-28T15:37:28.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.531+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.535+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.535+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-06-28T15:37:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.535+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6232508723375290063', time=2024-06-28T17:35:18.281633+0000, chain.objs.size()=1 2024-06-28T15:37:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.535+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.90__shadow_._6mhrQifXAWgF7WV0KNhSMxz_IH74RI_0 2024-06-28T15:37:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.535+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17141510849817676738', time=2024-06-28T17:36:14.972457+0000, chain.objs.size()=5 2024-06-28T15:37:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.535+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~BZhzx-cPbnKJUA39TVkSmOfP63bD2G_.1 2024-06-28T15:37:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.535+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__shadow_key1.2~BZhzx-cPbnKJUA39TVkSmOfP63bD2G_.1_1 2024-06-28T15:37:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.535+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~BZhzx-cPbnKJUA39TVkSmOfP63bD2G_.2 2024-06-28T15:37:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.535+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__shadow_key1.2~BZhzx-cPbnKJUA39TVkSmOfP63bD2G_.2_1 2024-06-28T15:37:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.535+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~BZhzx-cPbnKJUA39TVkSmOfP63bD2G_.3 2024-06-28T15:37:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.535+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18106164297386057531', time=2024-06-28T17:37:01.815667+0000, chain.objs.size()=1 2024-06-28T15:37:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.535+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.114__shadow_.fMqo9ryCTu3v0FD1UoG05sLuGru3Y9P_0 2024-06-28T15:37:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.535+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14165604973631646261', time=2024-06-28T17:37:17.108450+0000, chain.objs.size()=1 2024-06-28T15:37:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.535+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.129__shadow_.Zxdz21GM6GEbqehp-iLEHNjTvPV6Ro5_0 2024-06-28T15:37:28.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.543+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.547+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-06-28T15:37:28.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.547+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.547+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=6, truncated=0, next_marker='' 2024-06-28T15:37:28.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14915102240601196596', time=2024-06-28T17:34:26.380669+0000, chain.objs.size()=5 2024-06-28T15:37:28.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.81__multipart_key1.2~ZKPHlJ1FU36Kf612E5mRd4QEG3fqK8H.1 2024-06-28T15:37:28.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.81__multipart_key1.2~ZKPHlJ1FU36Kf612E5mRd4QEG3fqK8H.2 2024-06-28T15:37:28.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.81__multipart_key1.2~ZKPHlJ1FU36Kf612E5mRd4QEG3fqK8H.3 2024-06-28T15:37:28.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.81__multipart_key1.2~ZKPHlJ1FU36Kf612E5mRd4QEG3fqK8H.4 2024-06-28T15:37:28.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.81__multipart_key1.2~ZKPHlJ1FU36Kf612E5mRd4QEG3fqK8H.5 2024-06-28T15:37:28.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14800390961256640336', time=2024-06-28T17:35:05.574610+0000, chain.objs.size()=5 2024-06-28T15:37:28.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.88__multipart_key1.2~K27hXUeBBa5EyUBSOH0nq7SGwyE_BO0.1 2024-06-28T15:37:28.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.88__multipart_key1.2~K27hXUeBBa5EyUBSOH0nq7SGwyE_BO0.2 2024-06-28T15:37:28.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.88__multipart_key1.2~K27hXUeBBa5EyUBSOH0nq7SGwyE_BO0.3 2024-06-28T15:37:28.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.88__multipart_key1.2~K27hXUeBBa5EyUBSOH0nq7SGwyE_BO0.4 2024-06-28T15:37:28.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.88__multipart_key1.2~K27hXUeBBa5EyUBSOH0nq7SGwyE_BO0.5 2024-06-28T15:37:28.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14151793178437305657', time=2024-06-28T17:35:18.205436+0000, chain.objs.size()=5 2024-06-28T15:37:28.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.90__multipart_file.mpg.2~pHKXIF1QhJF_0YgmwCDpFBThvkiRKkf.1 2024-06-28T15:37:28.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.551+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.90__multipart_file.mpg.2~pHKXIF1QhJF_0YgmwCDpFBThvkiRKkf.2 2024-06-28T15:37:28.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.555+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.90__multipart_file.mpg.2~pHKXIF1QhJF_0YgmwCDpFBThvkiRKkf.3 2024-06-28T15:37:28.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.559+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.90__multipart_file.mpg.2~pHKXIF1QhJF_0YgmwCDpFBThvkiRKkf.4 2024-06-28T15:37:28.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.559+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.90__multipart_file.mpg.2~pHKXIF1QhJF_0YgmwCDpFBThvkiRKkf.5 2024-06-28T15:37:28.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.559+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17452343288802101083', time=2024-06-28T17:36:52.003191+0000, chain.objs.size()=1 2024-06-28T15:37:28.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.559+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.106__shadow_.eWf1VpXznvrMKrzcN1l9Tm_6Bm3_nxr_0 2024-06-28T15:37:28.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.559+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3057903795792935445', time=2024-06-28T17:37:12.309230+0000, chain.objs.size()=1 2024-06-28T15:37:28.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.559+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.123__shadow_.l9DVhbOPiYk31UUvJC5EMzO3H4dHi00_0 2024-06-28T15:37:28.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.559+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10710205007290359031', time=2024-06-28T17:37:12.380237+0000, chain.objs.size()=1 2024-06-28T15:37:28.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.559+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.123__shadow_.CqrkNuyp2ZMeRFHrzlR5vZvrKpH5moQ_0 2024-06-28T15:37:28.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.567+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.567+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-06-28T15:37:28.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.571+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.571+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.571+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:37:28.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.571+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9213241130452245531', time=2024-06-28T17:37:11.188069+0000, chain.objs.size()=1 2024-06-28T15:37:28.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.571+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.122__shadow_.f_jZw7N5QHSID8s7LTFILKo5F0wfKRz_0 2024-06-28T15:37:28.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.575+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-06-28T15:37:28.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-06-28T15:37:28.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1874875010758930585', time=2024-06-28T17:35:18.416651+0000, chain.objs.size()=1 2024-06-28T15:37:28.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.90__shadow_.rghyxlKPp_CK2wXIUm9FsyngfdVfKa1_0 2024-06-28T15:37:28.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6343505758951134447', time=2024-06-28T17:36:49.509746+0000, chain.objs.size()=1 2024-06-28T15:37:28.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.104__shadow_.o3ADFor8TchwVBizt78AFCzh256e4-0_0 2024-06-28T15:37:28.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15780966868397225872', time=2024-06-28T17:36:58.477775+0000, chain.objs.size()=1 2024-06-28T15:37:28.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.111__shadow_.sPeZGRlKWfkH6hPAY12MPUPdDl7SofR_0 2024-06-28T15:37:28.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2057256664364723758', time=2024-06-28T17:37:05.627062+0000, chain.objs.size()=1 2024-06-28T15:37:28.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.117__shadow_.13ptbwIKJ-Fr-J_wO_z_qJ_uJfLuoqh_0 2024-06-28T15:37:28.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14624843193115327016', time=2024-06-28T17:37:10.224068+0000, chain.objs.size()=1 2024-06-28T15:37:28.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.121__shadow_.Axc5lB64FZKlTPDEVqWhoIQ1XiPk8Pc_0 2024-06-28T15:37:28.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3800704211449412310', time=2024-06-28T17:37:14.371225+0000, chain.objs.size()=1 2024-06-28T15:37:28.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.126__shadow_.fIS00LOwYTTve1c0hCRLYSZovoxHdMt_0 2024-06-28T15:37:28.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9607969172916512256', time=2024-06-28T17:37:17.892544+0000, chain.objs.size()=1 2024-06-28T15:37:28.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.579+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.130__shadow_.JuOtxXh9bl6PU0Xm_QmQjjKu-rkRAP-_0 2024-06-28T15:37:28.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.587+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.587+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-06-28T15:37:28.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.591+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.591+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.591+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-06-28T15:37:28.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.591+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13248490442944153679', time=2024-06-28T17:36:54.412551+0000, chain.objs.size()=1 2024-06-28T15:37:28.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.591+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.108__shadow_.d7F4IZWaDL6r9tjs0yNjcJaC3QtsVS2_0 2024-06-28T15:37:28.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.591+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15838058560936309419', time=2024-06-28T17:36:54.479424+0000, chain.objs.size()=1 2024-06-28T15:37:28.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.591+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.108__shadow_.DO_Qfy2Ik8pTvZVXCGx_c9bwWg8DE7Y_0 2024-06-28T15:37:28.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.591+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14451449060835942365', time=2024-06-28T17:36:55.624677+0000, chain.objs.size()=1 2024-06-28T15:37:28.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.591+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.109__shadow_.-TpUy7vxCyFbBE7mw7_NGJ83C4-AuAS_0 2024-06-28T15:37:28.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.591+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15489004105721868812', time=2024-06-28T17:37:01.755298+0000, chain.objs.size()=1 2024-06-28T15:37:28.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.591+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.114__shadow_.wLzFPX5WfPVtN67STk8IftswNusibhX_0 2024-06-28T15:37:28.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.595+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.599+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-06-28T15:37:28.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.599+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-06-28T15:37:28.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12144791256664022355', time=2024-06-28T17:34:54.760865+0000, chain.objs.size()=5 2024-06-28T15:37:28.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~DCgsgjHyQ-5UXHS3_rKtZsfywKWcWF3.1 2024-06-28T15:37:28.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__shadow_key1.2~DCgsgjHyQ-5UXHS3_rKtZsfywKWcWF3.1_1 2024-06-28T15:37:28.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~DCgsgjHyQ-5UXHS3_rKtZsfywKWcWF3.2 2024-06-28T15:37:28.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__shadow_key1.2~DCgsgjHyQ-5UXHS3_rKtZsfywKWcWF3.2_1 2024-06-28T15:37:28.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~DCgsgjHyQ-5UXHS3_rKtZsfywKWcWF3.3 2024-06-28T15:37:28.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8201888269027580545', time=2024-06-28T17:35:50.595232+0000, chain.objs.size()=5 2024-06-28T15:37:28.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~HM-XUyw20yexwPbb2OIGO72xPsfSyKE.1 2024-06-28T15:37:28.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~HM-XUyw20yexwPbb2OIGO72xPsfSyKE.2 2024-06-28T15:37:28.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~HM-XUyw20yexwPbb2OIGO72xPsfSyKE.3 2024-06-28T15:37:28.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~HM-XUyw20yexwPbb2OIGO72xPsfSyKE.4 2024-06-28T15:37:28.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~HM-XUyw20yexwPbb2OIGO72xPsfSyKE.5 2024-06-28T15:37:28.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10379277405830758944', time=2024-06-28T17:36:52.080599+0000, chain.objs.size()=1 2024-06-28T15:37:28.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.106__shadow_.oA9WcSdkyKqW1QE7UiqXAAjO6rRj9XM_0 2024-06-28T15:37:28.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13404352877596775389', time=2024-06-28T17:37:05.470389+0000, chain.objs.size()=1 2024-06-28T15:37:28.593 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.603+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.117__shadow_.xi0MUegQrOI6Fe8_nQLbVUpqrt8eHH2_0 2024-06-28T15:37:28.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.615+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.619+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-06-28T15:37:28.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.619+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.619+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.623+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-06-28T15:37:28.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.623+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7577739534052094958', time=2024-06-28T17:36:48.248754+0000, chain.objs.size()=1 2024-06-28T15:37:28.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.623+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.103__shadow_.Z7kzgaHF-SF14xynrF4J3729Cd7-i3Q_0 2024-06-28T15:37:28.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.623+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12424279421137195069', time=2024-06-28T17:36:58.406187+0000, chain.objs.size()=1 2024-06-28T15:37:28.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.623+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.111__shadow_.iJRuur-7PAq9IFrUt2eYtYGUl11LRQq_0 2024-06-28T15:37:28.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.623+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12591487019078687611', time=2024-06-28T17:36:59.655534+0000, chain.objs.size()=1 2024-06-28T15:37:28.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.623+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.112__shadow_.gXbIZKHv0gTMVtFnfH9ngqCwJ_MU6Ci_0 2024-06-28T15:37:28.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.623+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2297610747690436162', time=2024-06-28T17:37:01.890284+0000, chain.objs.size()=1 2024-06-28T15:37:28.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.623+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.114__shadow_.qaQZAJSIcE0UCvE528SBNyyhSMayu0t_0 2024-06-28T15:37:28.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.623+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16793061394873390262', time=2024-06-28T17:37:06.628094+0000, chain.objs.size()=1 2024-06-28T15:37:28.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.623+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.118__shadow_.2ZB0_9M2tnZ4wmT36bcWw-FsaAdHUeu_0 2024-06-28T15:37:28.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.627+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.631+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-06-28T15:37:28.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.631+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.631+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.631+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-06-28T15:37:28.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.631+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3574430798964774728', time=2024-06-28T17:36:51.932438+0000, chain.objs.size()=1 2024-06-28T15:37:28.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.631+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.106__shadow_.UoiE_HJLKyohXlbzC5Olr41p9irOI51_0 2024-06-28T15:37:28.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.631+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17257786158037237834', time=2024-06-28T17:37:04.380479+0000, chain.objs.size()=1 2024-06-28T15:37:28.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.631+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.116__shadow_.mCg3wmmZNC5nRyBajgX5Dn6loZwR4Ba_0 2024-06-28T15:37:28.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.635+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.639+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-06-28T15:37:28.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.639+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.639+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.639+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-06-28T15:37:28.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.639+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='2~XI0kVlsvJlUsNCdMH4Jtozo5x22c3MD', time=2024-06-28T17:34:11.925179+0000, chain.objs.size()=5 2024-06-28T15:37:28.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.69__multipart_key1.2~XI0kVlsvJlUsNCdMH4Jtozo5x22c3MD.1 2024-06-28T15:37:28.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.69__multipart_key1.2~XI0kVlsvJlUsNCdMH4Jtozo5x22c3MD.2 2024-06-28T15:37:28.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.69__multipart_key1.2~XI0kVlsvJlUsNCdMH4Jtozo5x22c3MD.3 2024-06-28T15:37:28.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.69__multipart_key1.2~XI0kVlsvJlUsNCdMH4Jtozo5x22c3MD.4 2024-06-28T15:37:28.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.69__multipart_key1.2~XI0kVlsvJlUsNCdMH4Jtozo5x22c3MD.5 2024-06-28T15:37:28.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18268231543752926833', time=2024-06-28T17:34:45.508326+0000, chain.objs.size()=5 2024-06-28T15:37:28.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~_TIUeqpEp5OGTuUWc-1W0LJD2rCmmS6.1 2024-06-28T15:37:28.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~_TIUeqpEp5OGTuUWc-1W0LJD2rCmmS6.2 2024-06-28T15:37:28.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~_TIUeqpEp5OGTuUWc-1W0LJD2rCmmS6.3 2024-06-28T15:37:28.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~_TIUeqpEp5OGTuUWc-1W0LJD2rCmmS6.4 2024-06-28T15:37:28.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~_TIUeqpEp5OGTuUWc-1W0LJD2rCmmS6.5 2024-06-28T15:37:28.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2787710644646051317', time=2024-06-28T17:36:21.206995+0000, chain.objs.size()=1 2024-06-28T15:37:28.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.93__shadow_.c0WPjesagfQHYfiCkLe-7403eHNnCId_0 2024-06-28T15:37:28.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12287457150707120667', time=2024-06-28T17:36:49.578224+0000, chain.objs.size()=1 2024-06-28T15:37:28.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.643+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.104__shadow_.8097xdA1cA_lfgMOlXc2uPhI0HWu9hD_0 2024-06-28T15:37:28.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.651+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.655+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-06-28T15:37:28.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.655+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.659+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.659+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:37:28.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.659+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13177733489904139983', time=2024-06-28T17:36:46.969876+0000, chain.objs.size()=1 2024-06-28T15:37:28.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.659+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.102__shadow_.0KVTxMzBX13RcVGDfbxNRbRH-z8pSl8_0 2024-06-28T15:37:28.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.659+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.663+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-06-28T15:37:28.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.663+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.663+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.667+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-06-28T15:37:28.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.667+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.730269993180016784', time=2024-06-28T17:34:21.693527+0000, chain.objs.size()=1 2024-06-28T15:37:28.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.667+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.80__shadow_.fpBTqcFRxkGgcXNIqKvjXbxxtV86f6w_0 2024-06-28T15:37:28.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.667+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15322188566200595021', time=2024-06-28T17:34:49.700302+0000, chain.objs.size()=5 2024-06-28T15:37:28.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.667+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~8IUQ-_GgLCbfus47sqgdEzgl-XW4wmX.1 2024-06-28T15:37:28.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.667+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~8IUQ-_GgLCbfus47sqgdEzgl-XW4wmX.2 2024-06-28T15:37:28.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.667+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~8IUQ-_GgLCbfus47sqgdEzgl-XW4wmX.3 2024-06-28T15:37:28.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.667+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~8IUQ-_GgLCbfus47sqgdEzgl-XW4wmX.4 2024-06-28T15:37:28.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.667+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~8IUQ-_GgLCbfus47sqgdEzgl-XW4wmX.5 2024-06-28T15:37:28.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.671+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.675+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-06-28T15:37:28.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.675+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.675+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=10, truncated=0, next_marker='' 2024-06-28T15:37:28.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15485149909348618120', time=2024-06-28T17:33:58.420338+0000, chain.objs.size()=1 2024-06-28T15:37:28.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.67__shadow_.GcaWD7sjAQan-3bTXAFOjQiopFVS0Y0_0 2024-06-28T15:37:28.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11460061001325597104', time=2024-06-28T17:36:22.566379+0000, chain.objs.size()=5 2024-06-28T15:37:28.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.93__multipart_key1.2~ctuTDppkgtN_JsziOuHokQ0oLJ03-Sr.1 2024-06-28T15:37:28.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.93__multipart_key1.2~ctuTDppkgtN_JsziOuHokQ0oLJ03-Sr.2 2024-06-28T15:37:28.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.93__multipart_key1.2~ctuTDppkgtN_JsziOuHokQ0oLJ03-Sr.3 2024-06-28T15:37:28.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.93__multipart_key1.2~ctuTDppkgtN_JsziOuHokQ0oLJ03-Sr.4 2024-06-28T15:37:28.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.93__multipart_key1.2~ctuTDppkgtN_JsziOuHokQ0oLJ03-Sr.5 2024-06-28T15:37:28.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9774411403844022439', time=2024-06-28T17:36:54.612043+0000, chain.objs.size()=1 2024-06-28T15:37:28.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.108__shadow_.if_9i1D3hAu3z3gC3uxhpnXbe6apIyQ_0 2024-06-28T15:37:28.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18063680566506810679', time=2024-06-28T17:36:57.221588+0000, chain.objs.size()=1 2024-06-28T15:37:28.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.110__shadow_.3KZjmcjdY7W09M82xgqKpP2y3s4qn0Y_0 2024-06-28T15:37:28.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15173000116276223237', time=2024-06-28T17:37:05.554654+0000, chain.objs.size()=1 2024-06-28T15:37:28.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.117__shadow_.zmrXaI-TSNfMysLnh5Ht6l-tCfrkNPC_0 2024-06-28T15:37:28.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6905231528432606123', time=2024-06-28T17:37:07.839118+0000, chain.objs.size()=1 2024-06-28T15:37:28.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.119__shadow_.gaJ_O03C5T0pxueoTExcv6AceyTd-SL_0 2024-06-28T15:37:28.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16982229317416102113', time=2024-06-28T17:37:08.024861+0000, chain.objs.size()=1 2024-06-28T15:37:28.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.119__shadow_.VeykT2agRe3d7KSi_zex4NjhpjeVb-c_0 2024-06-28T15:37:28.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16001778787794464076', time=2024-06-28T17:37:10.145608+0000, chain.objs.size()=1 2024-06-28T15:37:28.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.679+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.121__shadow_.XrOni04ZBMD7yHjSIcJaolbDmUkuW1v_0 2024-06-28T15:37:28.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.683+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12373082705389468021', time=2024-06-28T17:37:12.236445+0000, chain.objs.size()=1 2024-06-28T15:37:28.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.683+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.123__shadow_.kLhgTr07NHZ2I6fvh9tEp5lOtbiI8gd_0 2024-06-28T15:37:28.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.683+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2372865812229563494', time=2024-06-28T17:37:15.734907+0000, chain.objs.size()=1 2024-06-28T15:37:28.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.683+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.128__shadow_.HDJ-0mr2IqFZ1IHysKPANkdsOazk1Pn_0 2024-06-28T15:37:28.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.687+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.691+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-06-28T15:37:28.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.691+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.691+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-06-28T15:37:28.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA', time=2024-06-28T17:36:35.468127+0000, chain.objs.size()=1 2024-06-28T15:37:28.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.19 2024-06-28T15:37:28.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA', time=2024-06-28T17:36:35.700610+0000, chain.objs.size()=1 2024-06-28T15:37:28.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.20 2024-06-28T15:37:28.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA', time=2024-06-28T17:36:35.993094+0000, chain.objs.size()=44 2024-06-28T15:37:28.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.1 2024-06-28T15:37:28.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.2 2024-06-28T15:37:28.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.3 2024-06-28T15:37:28.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.4 2024-06-28T15:37:28.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.5 2024-06-28T15:37:28.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.6 2024-06-28T15:37:28.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.7 2024-06-28T15:37:28.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.8 2024-06-28T15:37:28.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.9 2024-06-28T15:37:28.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.695+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.10 2024-06-28T15:37:28.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.699+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.11 2024-06-28T15:37:28.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.699+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.12 2024-06-28T15:37:28.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.699+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.13 2024-06-28T15:37:28.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.699+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.14 2024-06-28T15:37:28.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.699+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.15 2024-06-28T15:37:28.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.699+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.16 2024-06-28T15:37:28.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.699+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.17 2024-06-28T15:37:28.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.699+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.18 2024-06-28T15:37:28.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.699+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.VyPRzCDRYAQKRG5AY1CQxk8UsHtn_Y7l.19 2024-06-28T15:37:28.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.699+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.o2HKbQXDdbQlFy90SJjkzZIBUOjen4ce.20 2024-06-28T15:37:28.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.699+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.21 2024-06-28T15:37:28.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.703+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.22 2024-06-28T15:37:28.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.707+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.23 2024-06-28T15:37:28.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.707+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.24 2024-06-28T15:37:28.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.707+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.25 2024-06-28T15:37:28.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.707+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.26 2024-06-28T15:37:28.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.707+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.27 2024-06-28T15:37:28.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.707+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.28 2024-06-28T15:37:28.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.707+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.29 2024-06-28T15:37:28.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.707+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.30 2024-06-28T15:37:28.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.707+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.31 2024-06-28T15:37:28.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.707+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.32 2024-06-28T15:37:28.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.707+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.33 2024-06-28T15:37:28.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.711+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.34 2024-06-28T15:37:28.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.711+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.35 2024-06-28T15:37:28.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.719+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.36 2024-06-28T15:37:28.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.719+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.37 2024-06-28T15:37:28.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.719+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.38 2024-06-28T15:37:28.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.719+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.39 2024-06-28T15:37:28.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.719+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.40 2024-06-28T15:37:28.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.719+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.41 2024-06-28T15:37:28.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.719+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.42 2024-06-28T15:37:28.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.719+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.43 2024-06-28T15:37:28.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.719+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.95__multipart_key1.2~klvYuxdreSynTU9KwDMeEzHT4qCIEsA.44 2024-06-28T15:37:28.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.719+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10229645110512727508', time=2024-06-28T17:36:48.321449+0000, chain.objs.size()=1 2024-06-28T15:37:28.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.719+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.103__shadow_.cfHeCEhW2tJQaTIJ-YyhIZad59Smrox_0 2024-06-28T15:37:28.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.719+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5242512717449530050', time=2024-06-28T17:37:06.844993+0000, chain.objs.size()=1 2024-06-28T15:37:28.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.719+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.118__shadow_.Bb4oCBD_kKk3g8K6yi2qmR2KBM5ZoKz_0 2024-06-28T15:37:28.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.727+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.727+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-06-28T15:37:28.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-06-28T15:37:28.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14384189121156317330', time=2024-06-28T17:36:26.844738+0000, chain.objs.size()=5 2024-06-28T15:37:28.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.94__multipart_key1.2~Jhmeshftzmx_6e-SgQmqi97TZICtG53.1 2024-06-28T15:37:28.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.94__multipart_key1.2~Jhmeshftzmx_6e-SgQmqi97TZICtG53.2 2024-06-28T15:37:28.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.94__multipart_key1.2~Jhmeshftzmx_6e-SgQmqi97TZICtG53.3 2024-06-28T15:37:28.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.94__multipart_key1.2~Jhmeshftzmx_6e-SgQmqi97TZICtG53.4 2024-06-28T15:37:28.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.94__multipart_key1.2~Jhmeshftzmx_6e-SgQmqi97TZICtG53.5 2024-06-28T15:37:28.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2313390986897640527', time=2024-06-28T17:37:04.176389+0000, chain.objs.size()=1 2024-06-28T15:37:28.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.116__shadow_.rmvxjrnO9d9IIoOeDdiGpj7kOxkV5Zl_0 2024-06-28T15:37:28.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1008683503328931561', time=2024-06-28T17:37:06.773318+0000, chain.objs.size()=1 2024-06-28T15:37:28.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.118__shadow_.oW_juyORNiV6qvY-UXr9lFVzhrNngDv_0 2024-06-28T15:37:28.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2632411649687316829', time=2024-06-28T17:37:09.203960+0000, chain.objs.size()=1 2024-06-28T15:37:28.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.731+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.120__shadow_.U5oKnL2sVeljY9Yx2E2z7GMgJOEy7BB_0 2024-06-28T15:37:28.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.735+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-06-28T15:37:28.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-06-28T15:37:28.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11759386187981959083', time=2024-06-28T17:34:59.678042+0000, chain.objs.size()=5 2024-06-28T15:37:28.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~BhIaibkiaEkc45kJQTWD89RTo49R4b-.1 2024-06-28T15:37:28.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__shadow_key1.2~BhIaibkiaEkc45kJQTWD89RTo49R4b-.1_1 2024-06-28T15:37:28.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~BhIaibkiaEkc45kJQTWD89RTo49R4b-.2 2024-06-28T15:37:28.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__shadow_key1.2~BhIaibkiaEkc45kJQTWD89RTo49R4b-.2_1 2024-06-28T15:37:28.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~BhIaibkiaEkc45kJQTWD89RTo49R4b-.3 2024-06-28T15:37:28.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16902307695737311925', time=2024-06-28T17:34:59.968306+0000, chain.objs.size()=5 2024-06-28T15:37:28.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~gcmLGXe9n08oWojn4nSjmFSTtDGN8je.1 2024-06-28T15:37:28.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__shadow_key1.2~gcmLGXe9n08oWojn4nSjmFSTtDGN8je.1_1 2024-06-28T15:37:28.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~gcmLGXe9n08oWojn4nSjmFSTtDGN8je.2 2024-06-28T15:37:28.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__shadow_key1.2~gcmLGXe9n08oWojn4nSjmFSTtDGN8je.2_1 2024-06-28T15:37:28.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~gcmLGXe9n08oWojn4nSjmFSTtDGN8je.3 2024-06-28T15:37:28.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9504298068846474962', time=2024-06-28T17:36:57.295658+0000, chain.objs.size()=1 2024-06-28T15:37:28.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.110__shadow_.gl38AHObWdhkMLkErYtrjOXuCheTWb9_0 2024-06-28T15:37:28.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3030275401445724070', time=2024-06-28T17:37:15.083336+0000, chain.objs.size()=1 2024-06-28T15:37:28.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.739+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.127__shadow_.BXl9zkj5k80P0pfzcCpWdywbs70zMyb_0 2024-06-28T15:37:28.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.747+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-06-28T15:37:28.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=5, truncated=0, next_marker='' 2024-06-28T15:37:28.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11449639482276612896', time=2024-06-28T17:36:17.908575+0000, chain.objs.size()=5 2024-06-28T15:37:28.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~xrZL0lY4cxkgEfopqH7A0MJbJwc4CMo.1 2024-06-28T15:37:28.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__shadow_key1.2~xrZL0lY4cxkgEfopqH7A0MJbJwc4CMo.1_1 2024-06-28T15:37:28.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~xrZL0lY4cxkgEfopqH7A0MJbJwc4CMo.2 2024-06-28T15:37:28.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__shadow_key1.2~xrZL0lY4cxkgEfopqH7A0MJbJwc4CMo.2_1 2024-06-28T15:37:28.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~xrZL0lY4cxkgEfopqH7A0MJbJwc4CMo.3 2024-06-28T15:37:28.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13091992021737324695', time=2024-06-28T17:36:50.939811+0000, chain.objs.size()=1 2024-06-28T15:37:28.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.105__shadow_.Dz_cNdeBu-XL53z4UmaSpYxA7G64me__0 2024-06-28T15:37:28.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3804360322380861476', time=2024-06-28T17:36:59.580026+0000, chain.objs.size()=1 2024-06-28T15:37:28.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.112__shadow_.CyyMaJ8hJrNmdFRtMgoRJGWAYGSuEoi_0 2024-06-28T15:37:28.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10708162766571945184', time=2024-06-28T17:37:06.698908+0000, chain.objs.size()=1 2024-06-28T15:37:28.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.118__shadow_.ZTDWoWsXkKctYtCa_qniVh9ucawOUdF_0 2024-06-28T15:37:28.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16284281469521212705', time=2024-06-28T17:37:07.948228+0000, chain.objs.size()=1 2024-06-28T15:37:28.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.751+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.119__shadow_.REBKSxvjzg7hy9iFVUKpu8UzUqn8SnQ_0 2024-06-28T15:37:28.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.759+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.759+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-06-28T15:37:28.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.763+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.763+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.763+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-06-28T15:37:28.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.763+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6102468972346377013', time=2024-06-28T17:36:59.781568+0000, chain.objs.size()=1 2024-06-28T15:37:28.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.763+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.112__shadow_.aQMYDkMY1OuaKzEJApF39GPNaxXd_w9_0 2024-06-28T15:37:28.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.763+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-06-28T15:37:28.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-06-28T15:37:28.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3625458633129360597', time=2024-06-28T17:34:40.323646+0000, chain.objs.size()=5 2024-06-28T15:37:28.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~1GBvRM_CFtC9-uY2mLZay0-gBbRxmxg.1 2024-06-28T15:37:28.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~1GBvRM_CFtC9-uY2mLZay0-gBbRxmxg.2 2024-06-28T15:37:28.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~1GBvRM_CFtC9-uY2mLZay0-gBbRxmxg.3 2024-06-28T15:37:28.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~1GBvRM_CFtC9-uY2mLZay0-gBbRxmxg.4 2024-06-28T15:37:28.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.84__multipart_key1.2~1GBvRM_CFtC9-uY2mLZay0-gBbRxmxg.5 2024-06-28T15:37:28.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8705375063180390277', time=2024-06-28T17:36:17.619271+0000, chain.objs.size()=5 2024-06-28T15:37:28.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~wfbH3EUB-VGRItNiAsHPwrteFDLXT7h.1 2024-06-28T15:37:28.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__shadow_key1.2~wfbH3EUB-VGRItNiAsHPwrteFDLXT7h.1_1 2024-06-28T15:37:28.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~wfbH3EUB-VGRItNiAsHPwrteFDLXT7h.2 2024-06-28T15:37:28.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__shadow_key1.2~wfbH3EUB-VGRItNiAsHPwrteFDLXT7h.2_1 2024-06-28T15:37:28.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.92__multipart_key1.2~wfbH3EUB-VGRItNiAsHPwrteFDLXT7h.3 2024-06-28T15:37:28.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13723950393368185157', time=2024-06-28T17:37:27.435028+0000, chain.objs.size()=5 2024-06-28T15:37:28.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.134__multipart_key1/data/file.mpg.2~VaDiF9Orbk2OPiucXyI1NnCb3CmF_AP.1 2024-06-28T15:37:28.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.767+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.134__multipart_key1/data/file.mpg.2~VaDiF9Orbk2OPiucXyI1NnCb3CmF_AP.2 2024-06-28T15:37:28.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.775+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.134__multipart_key1/data/file.mpg.2~VaDiF9Orbk2OPiucXyI1NnCb3CmF_AP.3 2024-06-28T15:37:28.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.775+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.134__multipart_key1/data/file.mpg.2~VaDiF9Orbk2OPiucXyI1NnCb3CmF_AP.4 2024-06-28T15:37:28.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.775+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.134__multipart_key1/data/file.mpg.2~VaDiF9Orbk2OPiucXyI1NnCb3CmF_AP.5 2024-06-28T15:37:28.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.779+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.783+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-06-28T15:37:28.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.783+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.783+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-06-28T15:37:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7344626940366877231', time=2024-06-28T17:36:47.117210+0000, chain.objs.size()=1 2024-06-28T15:37:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.102__shadow_.WOChWXuowkKaL4JAuAkl0Exq6gtav_J_0 2024-06-28T15:37:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8138206219722904604', time=2024-06-28T17:36:48.397678+0000, chain.objs.size()=1 2024-06-28T15:37:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.103__shadow_.rLNpwzhK35jSP3foTLnigG0meuIoROy_0 2024-06-28T15:37:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7725183268708330586', time=2024-06-28T17:36:54.553724+0000, chain.objs.size()=1 2024-06-28T15:37:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.108__shadow_.Z-HCgmjAi7cR_ueRyYkovxCvUWoN0JL_0 2024-06-28T15:37:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16361163768168694375', time=2024-06-28T17:36:55.696144+0000, chain.objs.size()=1 2024-06-28T15:37:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.109__shadow_.2kNnxlEv4rLr13z3LxOIZY9QlIOegEx_0 2024-06-28T15:37:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3279679622916053570', time=2024-06-28T17:36:59.723359+0000, chain.objs.size()=1 2024-06-28T15:37:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.112__shadow_.exqFyRFyEsFJzY2lNCITyz9Z0qT80Gc_0 2024-06-28T15:37:28.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8060709035860666838', time=2024-06-28T17:37:04.324839+0000, chain.objs.size()=1 2024-06-28T15:37:28.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.116__shadow_.Q5UBERBPD53vaHfEYPV8LIqye8cqFNL_0 2024-06-28T15:37:28.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10001636968637060220', time=2024-06-28T17:37:08.099763+0000, chain.objs.size()=1 2024-06-28T15:37:28.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.787+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.119__shadow_.C_KhLHsX-qre5Bh0nEzyaieVJ-Om5qT_0 2024-06-28T15:37:28.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.791+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.791+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-06-28T15:37:28.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.795+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.795+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.795+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-06-28T15:37:28.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.795+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2462081693461983265', time=2024-06-28T17:35:18.489997+0000, chain.objs.size()=1 2024-06-28T15:37:28.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.795+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.90__shadow_.QPb9sk20wjVWGbKAIJ_nFNoukJdv7h2_0 2024-06-28T15:37:28.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.795+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6742866039350307828', time=2024-06-28T17:36:55.981156+0000, chain.objs.size()=1 2024-06-28T15:37:28.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.795+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.109__shadow_.S1shnp58SKs6_0mXDLPhAFXWcYp0JNx_0 2024-06-28T15:37:28.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.795+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-06-28T15:37:28.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-06-28T15:37:28.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15124941177865572945', time=2024-06-28T17:35:11.645858+0000, chain.objs.size()=5 2024-06-28T15:37:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.89__multipart_key1.2~y9FQgto8vBsRz7Vq6X_JalwU6e7rEyj.1 2024-06-28T15:37:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.89__multipart_key1.2~y9FQgto8vBsRz7Vq6X_JalwU6e7rEyj.2 2024-06-28T15:37:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.89__multipart_key1.2~y9FQgto8vBsRz7Vq6X_JalwU6e7rEyj.3 2024-06-28T15:37:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.89__multipart_key1.2~y9FQgto8vBsRz7Vq6X_JalwU6e7rEyj.4 2024-06-28T15:37:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.89__multipart_key1.2~y9FQgto8vBsRz7Vq6X_JalwU6e7rEyj.5 2024-06-28T15:37:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13917975019183203456', time=2024-06-28T17:36:53.242223+0000, chain.objs.size()=1 2024-06-28T15:37:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.107__shadow_.gUzNLfV6ngGXFL8lzIToRQNbWJPK0WL_0 2024-06-28T15:37:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7547744774595581187', time=2024-06-28T17:37:03.071130+0000, chain.objs.size()=1 2024-06-28T15:37:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.115__shadow_.LQS8JSGabC4aVJJKj6r6qvZuXVz9tmK_0 2024-06-28T15:37:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3937533731841016040', time=2024-06-28T17:37:11.258529+0000, chain.objs.size()=1 2024-06-28T15:37:28.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.799+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.122__shadow_.dt75mbzcAPzSbgRDe9wiu_IjR2WP0fT_0 2024-06-28T15:37:28.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.807+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-06-28T15:37:28.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:28.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:28.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=7, truncated=0, next_marker='' 2024-06-28T15:37:28.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2721391413244912444', time=2024-06-28T17:34:17.866507+0000, chain.objs.size()=1 2024-06-28T15:37:28.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.74__shadow_.Iz2QgYiLmJWsIz-pMnk9oBT58NCRgAp_0 2024-06-28T15:37:28.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11546544765909028172', time=2024-06-28T17:36:47.178123+0000, chain.objs.size()=1 2024-06-28T15:37:28.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.102__shadow_.MfGPthIklK-NI1Bpr9R9XtRknxw3Snv_0 2024-06-28T15:37:28.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8989260243421334393', time=2024-06-28T17:36:49.651265+0000, chain.objs.size()=1 2024-06-28T15:37:28.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.104__shadow_.NWtf5RdS3ztrShKYRlTjGr1xuOlXFRO_0 2024-06-28T15:37:28.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14580860165467230107', time=2024-06-28T17:36:58.550865+0000, chain.objs.size()=1 2024-06-28T15:37:28.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.111__shadow_.HVz3V8bkKBIeWV4-OYS9OAMmB2S7FDl_0 2024-06-28T15:37:28.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9158500566506750345', time=2024-06-28T17:37:00.752152+0000, chain.objs.size()=1 2024-06-28T15:37:28.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.113__shadow_.XK3Ba9_kGKzagSM5E5bHr2VI8799EZE_0 2024-06-28T15:37:28.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16081281363414842662', time=2024-06-28T17:37:09.129378+0000, chain.objs.size()=1 2024-06-28T15:37:28.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.120__shadow_.Su8dA1Ouq87LKWClbVdJThXK8Xdknor_0 2024-06-28T15:37:28.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.36579577818454684', time=2024-06-28T17:37:15.891529+0000, chain.objs.size()=1 2024-06-28T15:37:28.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.811+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.128__shadow_.NH-PF80b70aFXxI89qk_y3gg7X4xwCv_0 2024-06-28T15:37:28.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.815+0000 7fdc7cb57980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T15:37:28.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.819+0000 7fdb30fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:28.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.823+0000 7fdc7cb57980 20 remove_watcher() i=0 2024-06-28T15:37:28.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.823+0000 7fdc7cb57980 2 removed watcher, disabling cache 2024-06-28T15:37:28.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.823+0000 7fdc7cb57980 20 remove_watcher() i=1 2024-06-28T15:37:28.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.823+0000 7fdc7cb57980 20 remove_watcher() i=2 2024-06-28T15:37:28.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.823+0000 7fdc7cb57980 20 remove_watcher() i=3 2024-06-28T15:37:28.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.827+0000 7fdc7cb57980 20 remove_watcher() i=4 2024-06-28T15:37:28.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.827+0000 7fdc7cb57980 20 remove_watcher() i=5 2024-06-28T15:37:28.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.827+0000 7fdc7cb57980 20 remove_watcher() i=6 2024-06-28T15:37:28.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.831+0000 7fdc7cb57980 20 remove_watcher() i=7 2024-06-28T15:37:28.831 DEBUG:teuthology.orchestra.run.smithi178:> radosgw-admin gc process --include-all 2024-06-28T15:37:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:28.903 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:28.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.955+0000 7f1d6c2c6980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.959+0000 7f1d6c2c6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:28.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.959+0000 7f1d6c2c6980 20 realm 2024-06-28T15:37:28.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.959+0000 7f1d6c2c6980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.959+0000 7f1d6c2c6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:28.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.959+0000 7f1d6c2c6980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:28.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.959+0000 7f1d6c2c6980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.959+0000 7f1d6c2c6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:28.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.959+0000 7f1d6c2c6980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.959+0000 7f1d6c2c6980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:28.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.959+0000 7f1d6c2c6980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.959+0000 7f1d6c2c6980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:28.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.959+0000 7f1d6c2c6980 20 searching for the correct realm 2024-06-28T15:37:28.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.967+0000 7f1d695d6640 0 -- 172.21.15.178:0/2977507100 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f1d5c042400 msgr2=0x7f1d5c042820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:28.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.971+0000 7f1d695d6640 0 -- 172.21.15.178:0/2977507100 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f1c3407fca0 msgr2=0x7f1c340820d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:28.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.979+0000 7f1d6c2c6980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:28.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.979+0000 7f1d6c2c6980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:28.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.979+0000 7f1d6c2c6980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:28.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.979+0000 7f1d6c2c6980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:28.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.979+0000 7f1d6c2c6980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:28.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.979+0000 7f1d6c2c6980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:28.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.979+0000 7f1d6c2c6980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.979+0000 7f1d6c2c6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:28.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.979+0000 7f1d6c2c6980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.979+0000 7f1d6c2c6980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:28.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.979+0000 7f1d6c2c6980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:28.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 20 zone default found 2024-06-28T15:37:28.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 4 Realm: () 2024-06-28T15:37:28.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:28.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:28.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:28.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 20 zonegroup default 2024-06-28T15:37:28.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:28.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:28.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 20 rados->read ofs=0 len=0 2024-06-28T15:37:28.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:28.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 20 started sync module instance, tier type = 2024-06-28T15:37:28.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.983+0000 7f1d6c2c6980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:28.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.999+0000 7f1d6c2c6980 20 add_watcher() i=0 2024-06-28T15:37:28.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.999+0000 7f1d6c2c6980 20 add_watcher() i=1 2024-06-28T15:37:28.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.999+0000 7f1d6c2c6980 20 add_watcher() i=2 2024-06-28T15:37:28.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.999+0000 7f1d6c2c6980 20 add_watcher() i=3 2024-06-28T15:37:28.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.999+0000 7f1d6c2c6980 20 add_watcher() i=4 2024-06-28T15:37:28.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.999+0000 7f1d6c2c6980 20 add_watcher() i=5 2024-06-28T15:37:28.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.999+0000 7f1d6c2c6980 20 add_watcher() i=6 2024-06-28T15:37:28.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.999+0000 7f1d6c2c6980 20 add_watcher() i=7 2024-06-28T15:37:28.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.999+0000 7f1d6c2c6980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:28.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:28.999+0000 7f1c1b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:28.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.003+0000 7f1d6c2c6980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:28.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.003+0000 7f1d6c2c6980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:29.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.075+0000 7f1c1a7fc640 20 reqs_thread_entry: start 2024-06-28T15:37:29.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.075+0000 7f1d6c2c6980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:29.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.075+0000 7f1d6c2c6980 20 Filter name: none 2024-06-28T15:37:29.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.075+0000 7f1b8bfff640 20 reqs_thread_entry: start 2024-06-28T15:37:29.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.075+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-06-28T15:37:29.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.079+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.079+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.079+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.083+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-06-28T15:37:29.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.087+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.087+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.087+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.087+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-06-28T15:37:29.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.091+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.095+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.095+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.099+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-06-28T15:37:29.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.099+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.103+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.103+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.103+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-06-28T15:37:29.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.107+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.107+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.107+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.111+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-06-28T15:37:29.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.115+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.115+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.115+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.119+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-06-28T15:37:29.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.123+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.123+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.123+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.127+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-06-28T15:37:29.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.131+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.131+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.131+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.135+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-06-28T15:37:29.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.139+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.139+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.139+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.143+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-06-28T15:37:29.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.147+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.147+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.147+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.151+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-06-28T15:37:29.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.151+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.155+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.155+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.155+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-06-28T15:37:29.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.163+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.163+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.167+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.167+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-06-28T15:37:29.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.171+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.171+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.171+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.175+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-06-28T15:37:29.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.179+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.179+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.179+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.179+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-06-28T15:37:29.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.183+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.183+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.187+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.187+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-06-28T15:37:29.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.191+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.191+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.191+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.191+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-06-28T15:37:29.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.195+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.195+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.195+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.199+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-06-28T15:37:29.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.203+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.203+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.203+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.207+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-06-28T15:37:29.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.207+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.207+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.207+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.211+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-06-28T15:37:29.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.215+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.215+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.215+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.206 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.219+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-06-28T15:37:29.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.219+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.223+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.223+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.223+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-06-28T15:37:29.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.227+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.227+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.227+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.231+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-06-28T15:37:29.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.235+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.235+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.239+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.239+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-06-28T15:37:29.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.243+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.243+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.243+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.247+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-06-28T15:37:29.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.251+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.251+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.251+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.255+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-06-28T15:37:29.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.259+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.259+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.259+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.259+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-06-28T15:37:29.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.263+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.267+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.267+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.271+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-06-28T15:37:29.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.271+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.275+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.275+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.275+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-06-28T15:37:29.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.279+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.279+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.279+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.283+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-06-28T15:37:29.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.287+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.287+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.287+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.291+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-06-28T15:37:29.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.295+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.295+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.295+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.299+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-06-28T15:37:29.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.303+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.303+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T15:37:29.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.303+0000 7f1d6c2c6980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T15:37:29.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.311+0000 7f1c1b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:29.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.315+0000 7f1d6c2c6980 20 remove_watcher() i=0 2024-06-28T15:37:29.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.315+0000 7f1d6c2c6980 2 removed watcher, disabling cache 2024-06-28T15:37:29.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.319+0000 7f1d6c2c6980 20 remove_watcher() i=1 2024-06-28T15:37:29.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.319+0000 7f1d6c2c6980 20 remove_watcher() i=2 2024-06-28T15:37:29.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.323+0000 7f1d6c2c6980 20 remove_watcher() i=3 2024-06-28T15:37:29.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.323+0000 7f1d6c2c6980 20 remove_watcher() i=4 2024-06-28T15:37:29.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.327+0000 7f1d6c2c6980 20 remove_watcher() i=5 2024-06-28T15:37:29.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.327+0000 7f1d6c2c6980 20 remove_watcher() i=6 2024-06-28T15:37:29.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:29.331+0000 7f1d6c2c6980 20 remove_watcher() i=7 2024-06-28T15:37:29.338 INFO:teuthology.run_tasks:Running task s3tests... 2024-06-28T15:37:29.356 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'accounts': {'iam alt root': 'RGW99999999999999999', 'iam root': 'RGW88888888888888888', 's3 tenant': 'RGW33333333333333333'}, 'force-branch': 'ceph-master', 'slow_backend': True, 'storage classes': 'LUKEWARM, FROZEN'}} 2024-06-28T15:37:29.356 INFO:tasks.s3tests:Downloading s3-tests... 2024-06-28T15:37:29.357 INFO:tasks.s3tests:Using branch 'ceph-master' for s3tests 2024-06-28T15:37:29.357 DEBUG:teuthology.orchestra.run.smithi178:> git clone -b ceph-master https://git.ceph.com/s3-tests.git /home/ubuntu/cephtest/s3-tests-client.0 2024-06-28T15:37:29.387 INFO:teuthology.orchestra.run.smithi178.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-client.0'... 2024-06-28T15:37:30.536 INFO:tasks.s3tests:Creating rgw users... 2024-06-28T15:37:30.537 DEBUG:tasks.s3tests:Creating user foo.client.0 on client.0 2024-06-28T15:37:30.537 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name Mr.foo.client.0 --email foo+test@test.test --access-key NXOLOROICVIOYCVNDNIP --secret 0eOElRVcDN5i1eybuyaLejk4EwHNh+L1NPf9SGNpZSavs1HHMVfMiA== --cluster ceph 2024-06-28T15:37:30.624 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:30.624 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:30.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.675+0000 7fb68aea3980 20 rados->read ofs=0 len=0 2024-06-28T15:37:30.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.679+0000 7fb68aea3980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:30.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.679+0000 7fb68aea3980 20 realm 2024-06-28T15:37:30.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.679+0000 7fb68aea3980 20 rados->read ofs=0 len=0 2024-06-28T15:37:30.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.679+0000 7fb68aea3980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:30.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.679+0000 7fb68aea3980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:30.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.679+0000 7fb68aea3980 20 rados->read ofs=0 len=0 2024-06-28T15:37:30.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.679+0000 7fb68aea3980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:30.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.679+0000 7fb68aea3980 20 rados->read ofs=0 len=0 2024-06-28T15:37:30.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.679+0000 7fb68aea3980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:30.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.679+0000 7fb68aea3980 20 rados->read ofs=0 len=0 2024-06-28T15:37:30.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.679+0000 7fb68aea3980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:30.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.679+0000 7fb68aea3980 20 searching for the correct realm 2024-06-28T15:37:30.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.687+0000 7fb6881b3640 0 -- 172.21.15.178:0/3557466187 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x7fb55407fda0 msgr2=0x7fb5540821d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:30.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:30.899+0000 7fb6889b4640 0 -- 172.21.15.178:0/3557466187 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fb67c050940 msgr2=0x7fb67c05c3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:31.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.115+0000 7fb68aea3980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:31.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.115+0000 7fb68aea3980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:31.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.115+0000 7fb68aea3980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:31.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.115+0000 7fb68aea3980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:31.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.115+0000 7fb68aea3980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:31.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.115+0000 7fb68aea3980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:31.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.115+0000 7fb68aea3980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:31.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:31.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:31.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 20 zone default found 2024-06-28T15:37:31.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 4 Realm: () 2024-06-28T15:37:31.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:31.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:31.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:31.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 20 zonegroup default 2024-06-28T15:37:31.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:31.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:31.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.119+0000 7fb68aea3980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.123+0000 7fb68aea3980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:31.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.123+0000 7fb68aea3980 20 started sync module instance, tier type = 2024-06-28T15:37:31.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.123+0000 7fb68aea3980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:31.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.143+0000 7fb68aea3980 20 add_watcher() i=0 2024-06-28T15:37:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.143+0000 7fb68aea3980 20 add_watcher() i=1 2024-06-28T15:37:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.143+0000 7fb68aea3980 20 add_watcher() i=2 2024-06-28T15:37:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.143+0000 7fb68aea3980 20 add_watcher() i=3 2024-06-28T15:37:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.143+0000 7fb68aea3980 20 add_watcher() i=4 2024-06-28T15:37:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.143+0000 7fb68aea3980 20 add_watcher() i=5 2024-06-28T15:37:31.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.143+0000 7fb68aea3980 20 add_watcher() i=6 2024-06-28T15:37:31.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.147+0000 7fb68aea3980 20 add_watcher() i=7 2024-06-28T15:37:31.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.147+0000 7fb68aea3980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:31.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.151+0000 7fb53f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:31.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.151+0000 7fb68aea3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:31.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.151+0000 7fb68aea3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:31.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.151+0000 7fb68aea3980 5 note: GC not initialized 2024-06-28T15:37:31.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.151+0000 7fb53e7e4640 20 reqs_thread_entry: start 2024-06-28T15:37:31.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.151+0000 7fb68aea3980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:31.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.151+0000 7fb68aea3980 20 Filter name: none 2024-06-28T15:37:31.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.151+0000 7fb5377fe640 20 reqs_thread_entry: start 2024-06-28T15:37:31.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.151+0000 7fb68aea3980 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2024-06-28T15:37:31.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:31.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2024-06-28T15:37:31.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-06-28T15:37:31.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 10 cache get: name=default.rgw.meta+users.email+foo+test@test.test : miss 2024-06-28T15:37:31.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:31.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 10 cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x0 2024-06-28T15:37:31.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 10 adding default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-06-28T15:37:31.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 10 cache get: name=default.rgw.meta+users.keys+NXOLOROICVIOYCVNDNIP : miss 2024-06-28T15:37:31.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 10 cache put: name=default.rgw.meta+users.keys+NXOLOROICVIOYCVNDNIP info.flags=0x0 2024-06-28T15:37:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 10 adding default.rgw.meta+users.keys+NXOLOROICVIOYCVNDNIP to cache LRU end 2024-06-28T15:37:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 10 cache get: name=default.rgw.meta+users.keys+NXOLOROICVIOYCVNDNIP : hit (negative entry) 2024-06-28T15:37:31.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.155+0000 7fb68aea3980 10 cache get: name=default.rgw.meta+users.keys+NXOLOROICVIOYCVNDNIP : hit (negative entry) 2024-06-28T15:37:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.159+0000 7fb68aea3980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-06-28T15:37:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.159+0000 7fb68aea3980 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-06-28T15:37:31.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.159+0000 7fb68aea3980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2024-06-28T15:37:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.159+0000 7fb55ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595255 cookie 94797875385136 notifier 7238 bl.length()=645 2024-06-28T15:37:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.159+0000 7fb55ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-06-28T15:37:31.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.159+0000 7fb55ffff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-06-28T15:37:31.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.163+0000 7fb68aea3980 10 cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x7 2024-06-28T15:37:31.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.163+0000 7fb68aea3980 10 moving default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-06-28T15:37:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.163+0000 7fb68aea3980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.email:foo+test@test.test, ofs0, ns] 2024-06-28T15:37:31.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.163+0000 7fb55ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595256 cookie 94797875401904 notifier 7238 bl.length()=185 2024-06-28T15:37:31.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.163+0000 7fb55ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x7 2024-06-28T15:37:31.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.163+0000 7fb55ffff640 10 rgw watcher librados: moving default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-06-28T15:37:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.167+0000 7fb68aea3980 10 cache put: name=default.rgw.meta+users.keys+NXOLOROICVIOYCVNDNIP info.flags=0x7 2024-06-28T15:37:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.167+0000 7fb68aea3980 10 moving default.rgw.meta+users.keys+NXOLOROICVIOYCVNDNIP to cache LRU end 2024-06-28T15:37:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.167+0000 7fb68aea3980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:NXOLOROICVIOYCVNDNIP, ofs0, ns] 2024-06-28T15:37:31.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.167+0000 7fb55f7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1103806595185 cookie 94797875406352 notifier 7238 bl.length()=186 2024-06-28T15:37:31.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.167+0000 7fb55f7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NXOLOROICVIOYCVNDNIP info.flags=0x7 2024-06-28T15:37:31.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.167+0000 7fb55f7fe640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NXOLOROICVIOYCVNDNIP to cache LRU end 2024-06-28T15:37:31.160 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T15:37:31.160 INFO:teuthology.orchestra.run.smithi178.stdout: "user_id": "foo.client.0", 2024-06-28T15:37:31.160 INFO:teuthology.orchestra.run.smithi178.stdout: "display_name": "Mr.foo.client.0", 2024-06-28T15:37:31.160 INFO:teuthology.orchestra.run.smithi178.stdout: "email": "foo+test@test.test", 2024-06-28T15:37:31.160 INFO:teuthology.orchestra.run.smithi178.stdout: "suspended": 0, 2024-06-28T15:37:31.160 INFO:teuthology.orchestra.run.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:37:31.161 INFO:teuthology.orchestra.run.smithi178.stdout: "subusers": [], 2024-06-28T15:37:31.161 INFO:teuthology.orchestra.run.smithi178.stdout: "keys": [ 2024-06-28T15:37:31.161 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T15:37:31.161 INFO:teuthology.orchestra.run.smithi178.stdout: "user": "foo.client.0", 2024-06-28T15:37:31.161 INFO:teuthology.orchestra.run.smithi178.stdout: "access_key": "NXOLOROICVIOYCVNDNIP", 2024-06-28T15:37:31.161 INFO:teuthology.orchestra.run.smithi178.stdout: "secret_key": "0eOElRVcDN5i1eybuyaLejk4EwHNh+L1NPf9SGNpZSavs1HHMVfMiA==", 2024-06-28T15:37:31.162 INFO:teuthology.orchestra.run.smithi178.stdout: "active": true, 2024-06-28T15:37:31.162 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:37:31.158999Z" 2024-06-28T15:37:31.162 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T15:37:31.162 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-06-28T15:37:31.162 INFO:teuthology.orchestra.run.smithi178.stdout: "swift_keys": [], 2024-06-28T15:37:31.162 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": [], 2024-06-28T15:37:31.163 INFO:teuthology.orchestra.run.smithi178.stdout: "op_mask": "read, write, delete", 2024-06-28T15:37:31.163 INFO:teuthology.orchestra.run.smithi178.stdout: "default_placement": "", 2024-06-28T15:37:31.163 INFO:teuthology.orchestra.run.smithi178.stdout: "default_storage_class": "", 2024-06-28T15:37:31.163 INFO:teuthology.orchestra.run.smithi178.stdout: "placement_tags": [], 2024-06-28T15:37:31.163 INFO:teuthology.orchestra.run.smithi178.stdout: "bucket_quota": { 2024-06-28T15:37:31.163 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:31.163 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:31.163 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:31.163 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:31.163 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:31.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:31.163 INFO:teuthology.orchestra.run.smithi178.stdout: "user_quota": { 2024-06-28T15:37:31.163 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:31.163 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:31.164 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:31.164 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:31.164 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:31.164 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:31.164 INFO:teuthology.orchestra.run.smithi178.stdout: "temp_url_keys": [], 2024-06-28T15:37:31.164 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "rgw", 2024-06-28T15:37:31.164 INFO:teuthology.orchestra.run.smithi178.stdout: "mfa_ids": [], 2024-06-28T15:37:31.164 INFO:teuthology.orchestra.run.smithi178.stdout: "account_id": "", 2024-06-28T15:37:31.164 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2024-06-28T15:37:31.164 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:37:31.158962Z", 2024-06-28T15:37:31.164 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2024-06-28T15:37:31.164 INFO:teuthology.orchestra.run.smithi178.stdout: "group_ids": [] 2024-06-28T15:37:31.164 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T15:37:31.164 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:31.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.171+0000 7fb53f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:31.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.175+0000 7fb68aea3980 20 remove_watcher() i=0 2024-06-28T15:37:31.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.175+0000 7fb68aea3980 2 removed watcher, disabling cache 2024-06-28T15:37:31.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.179+0000 7fb68aea3980 20 remove_watcher() i=1 2024-06-28T15:37:31.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.179+0000 7fb68aea3980 20 remove_watcher() i=2 2024-06-28T15:37:31.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.183+0000 7fb68aea3980 20 remove_watcher() i=3 2024-06-28T15:37:31.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.183+0000 7fb68aea3980 20 remove_watcher() i=4 2024-06-28T15:37:31.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.187+0000 7fb68aea3980 20 remove_watcher() i=5 2024-06-28T15:37:31.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.187+0000 7fb68aea3980 20 remove_watcher() i=6 2024-06-28T15:37:31.179 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.191+0000 7fb68aea3980 20 remove_watcher() i=7 2024-06-28T15:37:31.194 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 2147604732 --totp-seed MQKXZBQHZ67D7GXUC3CQIOCFBGGAZUPS5JWJ2K3IJVZLGJ6BBJXRUKLVKEEF6HGY --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-06-28T15:37:31.234 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:31.234 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:31.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.287+0000 7fb2defed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.287+0000 7fb2defed980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:31.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.287+0000 7fb2defed980 20 realm 2024-06-28T15:37:31.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.287+0000 7fb2defed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.287+0000 7fb2defed980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:31.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.287+0000 7fb2defed980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:31.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.287+0000 7fb2defed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.287+0000 7fb2defed980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:31.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.287+0000 7fb2defed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.291+0000 7fb2defed980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:31.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.291+0000 7fb2defed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.291+0000 7fb2defed980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:31.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.291+0000 7fb2defed980 20 searching for the correct realm 2024-06-28T15:37:31.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.295+0000 7fb2dd2ff640 0 -- 172.21.15.178:0/1841821544 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7fb2c40e1b60 msgr2=0x7fb2c40f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:31.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.511+0000 7fb2dcafe640 0 -- 172.21.15.178:0/1841821544 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x56420b92cc50 msgr2=0x56420b92f0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:31.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.519+0000 7fb2defed980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:31.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.519+0000 7fb2defed980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:31.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.519+0000 7fb2defed980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:31.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.519+0000 7fb2defed980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:31.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.519+0000 7fb2defed980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:31.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.519+0000 7fb2defed980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:31.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.519+0000 7fb2defed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.519+0000 7fb2defed980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:31.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.519+0000 7fb2defed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.519+0000 7fb2defed980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:31.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.519+0000 7fb2defed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:31.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 20 zone default found 2024-06-28T15:37:31.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 4 Realm: () 2024-06-28T15:37:31.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:31.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:31.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:31.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 20 zonegroup default 2024-06-28T15:37:31.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:31.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:31.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:31.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 20 started sync module instance, tier type = 2024-06-28T15:37:31.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.523+0000 7fb2defed980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:31.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.547+0000 7fb2defed980 20 add_watcher() i=0 2024-06-28T15:37:31.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.547+0000 7fb2defed980 20 add_watcher() i=1 2024-06-28T15:37:31.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.547+0000 7fb2defed980 20 add_watcher() i=2 2024-06-28T15:37:31.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.547+0000 7fb2defed980 20 add_watcher() i=3 2024-06-28T15:37:31.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.547+0000 7fb2defed980 20 add_watcher() i=4 2024-06-28T15:37:31.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.547+0000 7fb2defed980 20 add_watcher() i=5 2024-06-28T15:37:31.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.547+0000 7fb2defed980 20 add_watcher() i=6 2024-06-28T15:37:31.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.551+0000 7fb2defed980 20 add_watcher() i=7 2024-06-28T15:37:31.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.551+0000 7fb2defed980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:31.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.555+0000 7fb192fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:31.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.555+0000 7fb2defed980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:31.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.555+0000 7fb2defed980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:31.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.555+0000 7fb2defed980 5 note: GC not initialized 2024-06-28T15:37:31.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.555+0000 7fb191fe3640 20 reqs_thread_entry: start 2024-06-28T15:37:31.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.555+0000 7fb2defed980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:31.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.555+0000 7fb2defed980 20 Filter name: none 2024-06-28T15:37:31.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.555+0000 7fb1037fe640 20 reqs_thread_entry: start 2024-06-28T15:37:31.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.555+0000 7fb2defed980 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2024-06-28T15:37:31.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.555+0000 7fb2defed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:31.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.555+0000 7fb2defed980 20 rados_obj.operate() r=0 bl.length=460 2024-06-28T15:37:31.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.555+0000 7fb2defed980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x13 2024-06-28T15:37:31.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.555+0000 7fb2defed980 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-06-28T15:37:31.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:31.555+0000 7fb2defed980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:37:33.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.351+0000 7fb2defed980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-06-28T15:37:33.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.351+0000 7fb2defed980 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-06-28T15:37:33.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.351+0000 7fb2defed980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2024-06-28T15:37:33.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.355+0000 7fb1b37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529849 cookie 94841662006032 notifier 7791 bl.length()=659 2024-06-28T15:37:33.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.355+0000 7fb1b37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-06-28T15:37:33.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.355+0000 7fb1b37fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-06-28T15:37:33.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.359+0000 7fb192fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:33.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.363+0000 7fb2defed980 20 remove_watcher() i=0 2024-06-28T15:37:33.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.363+0000 7fb2defed980 2 removed watcher, disabling cache 2024-06-28T15:37:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.367+0000 7fb2defed980 20 remove_watcher() i=1 2024-06-28T15:37:33.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.367+0000 7fb2defed980 20 remove_watcher() i=2 2024-06-28T15:37:33.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.371+0000 7fb2defed980 20 remove_watcher() i=3 2024-06-28T15:37:33.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.371+0000 7fb2defed980 20 remove_watcher() i=4 2024-06-28T15:37:33.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.375+0000 7fb2defed980 20 remove_watcher() i=5 2024-06-28T15:37:33.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.375+0000 7fb2defed980 20 remove_watcher() i=6 2024-06-28T15:37:33.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.379+0000 7fb2defed980 20 remove_watcher() i=7 2024-06-28T15:37:33.387 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2024-06-28T15:37:33.387 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name Mr.bar.client.0 --email bar+test@test.test --access-key FSWBFVQZTYOXXOKESLAT --secret 4FVp29UHx3cvz3mISVbHIReDboSlZ+cTOr55gVLTIEHhVYuhQH/T2Q== --cluster ceph 2024-06-28T15:37:33.474 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:33.474 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:33.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.527+0000 7fb93d6fe980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.527+0000 7fb93d6fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.527+0000 7fb93d6fe980 20 realm 2024-06-28T15:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.527+0000 7fb93d6fe980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.527+0000 7fb93d6fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.527+0000 7fb93d6fe980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.527+0000 7fb93d6fe980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.527+0000 7fb93d6fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.527+0000 7fb93d6fe980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.531+0000 7fb93d6fe980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:33.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.531+0000 7fb93d6fe980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.531+0000 7fb93d6fe980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:33.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.531+0000 7fb93d6fe980 20 searching for the correct realm 2024-06-28T15:37:33.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.535+0000 7fb93aa0e640 0 -- 172.21.15.178:0/2130816149 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fb92c120150 msgr2=0x7fb92c1193b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:33.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.543+0000 7fb93aa0e640 0 -- 172.21.15.178:0/2130816149 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fb804080a80 msgr2=0x7fb804082eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:33.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.547+0000 7fb93d6fe980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:33.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.547+0000 7fb93d6fe980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:33.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.547+0000 7fb93d6fe980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:33.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.547+0000 7fb93d6fe980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:33.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.547+0000 7fb93d6fe980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:33.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.547+0000 7fb93d6fe980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:33.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.547+0000 7fb93d6fe980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:33.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:33.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 zone default found 2024-06-28T15:37:33.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 4 Realm: () 2024-06-28T15:37:33.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:33.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:33.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:33.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 zonegroup default 2024-06-28T15:37:33.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 started sync module instance, tier type = 2024-06-28T15:37:33.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.551+0000 7fb93d6fe980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:33.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.567+0000 7fb93d6fe980 20 add_watcher() i=0 2024-06-28T15:37:33.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.567+0000 7fb93d6fe980 20 add_watcher() i=1 2024-06-28T15:37:33.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.567+0000 7fb93d6fe980 20 add_watcher() i=2 2024-06-28T15:37:33.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.567+0000 7fb93d6fe980 20 add_watcher() i=3 2024-06-28T15:37:33.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.567+0000 7fb93d6fe980 20 add_watcher() i=4 2024-06-28T15:37:33.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.567+0000 7fb93d6fe980 20 add_watcher() i=5 2024-06-28T15:37:33.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.567+0000 7fb93d6fe980 20 add_watcher() i=6 2024-06-28T15:37:33.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.567+0000 7fb93d6fe980 20 add_watcher() i=7 2024-06-28T15:37:33.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.567+0000 7fb93d6fe980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:33.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb7f27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:33.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:33.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:33.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 5 note: GC not initialized 2024-06-28T15:37:33.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb7f17e2640 20 reqs_thread_entry: start 2024-06-28T15:37:33.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:33.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 20 Filter name: none 2024-06-28T15:37:33.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb7ea7fc640 20 reqs_thread_entry: start 2024-06-28T15:37:33.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2024-06-28T15:37:33.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2024-06-28T15:37:33.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-06-28T15:37:33.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 10 cache get: name=default.rgw.meta+users.email+bar+test@test.test : miss 2024-06-28T15:37:33.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 10 cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x0 2024-06-28T15:37:33.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 10 adding default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-06-28T15:37:33.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 10 cache get: name=default.rgw.meta+users.keys+FSWBFVQZTYOXXOKESLAT : miss 2024-06-28T15:37:33.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 10 cache put: name=default.rgw.meta+users.keys+FSWBFVQZTYOXXOKESLAT info.flags=0x0 2024-06-28T15:37:33.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 10 adding default.rgw.meta+users.keys+FSWBFVQZTYOXXOKESLAT to cache LRU end 2024-06-28T15:37:33.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 10 cache get: name=default.rgw.meta+users.keys+FSWBFVQZTYOXXOKESLAT : hit (negative entry) 2024-06-28T15:37:33.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.571+0000 7fb93d6fe980 10 cache get: name=default.rgw.meta+users.keys+FSWBFVQZTYOXXOKESLAT : hit (negative entry) 2024-06-28T15:37:33.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.575+0000 7fb93d6fe980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-06-28T15:37:33.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.575+0000 7fb93d6fe980 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-06-28T15:37:33.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.575+0000 7fb93d6fe980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2024-06-28T15:37:33.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.575+0000 7fb81effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529816 cookie 93862503644736 notifier 7812 bl.length()=645 2024-06-28T15:37:33.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.575+0000 7fb81effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-06-28T15:37:33.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.575+0000 7fb81effd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-06-28T15:37:33.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.579+0000 7fb93d6fe980 10 cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x7 2024-06-28T15:37:33.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.579+0000 7fb93d6fe980 10 moving default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-06-28T15:37:33.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.579+0000 7fb93d6fe980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:bar+test@test.test, ofs0, ns] 2024-06-28T15:37:33.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.579+0000 7fb81effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529802 cookie 93862503656080 notifier 7812 bl.length()=185 2024-06-28T15:37:33.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.579+0000 7fb81effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x7 2024-06-28T15:37:33.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.579+0000 7fb81effd640 10 rgw watcher librados: moving default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-06-28T15:37:33.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.583+0000 7fb93d6fe980 10 cache put: name=default.rgw.meta+users.keys+FSWBFVQZTYOXXOKESLAT info.flags=0x7 2024-06-28T15:37:33.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.583+0000 7fb93d6fe980 10 moving default.rgw.meta+users.keys+FSWBFVQZTYOXXOKESLAT to cache LRU end 2024-06-28T15:37:33.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.583+0000 7fb93d6fe980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:FSWBFVQZTYOXXOKESLAT, ofs0, ns] 2024-06-28T15:37:33.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.583+0000 7fb81e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529850 cookie 93862503652384 notifier 7812 bl.length()=186 2024-06-28T15:37:33.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.583+0000 7fb81e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+FSWBFVQZTYOXXOKESLAT info.flags=0x7 2024-06-28T15:37:33.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.583+0000 7fb81e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+FSWBFVQZTYOXXOKESLAT to cache LRU end 2024-06-28T15:37:33.575 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T15:37:33.575 INFO:teuthology.orchestra.run.smithi178.stdout: "user_id": "bar.client.0", 2024-06-28T15:37:33.575 INFO:teuthology.orchestra.run.smithi178.stdout: "display_name": "Mr.bar.client.0", 2024-06-28T15:37:33.575 INFO:teuthology.orchestra.run.smithi178.stdout: "email": "bar+test@test.test", 2024-06-28T15:37:33.575 INFO:teuthology.orchestra.run.smithi178.stdout: "suspended": 0, 2024-06-28T15:37:33.575 INFO:teuthology.orchestra.run.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:37:33.575 INFO:teuthology.orchestra.run.smithi178.stdout: "subusers": [], 2024-06-28T15:37:33.575 INFO:teuthology.orchestra.run.smithi178.stdout: "keys": [ 2024-06-28T15:37:33.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T15:37:33.576 INFO:teuthology.orchestra.run.smithi178.stdout: "user": "bar.client.0", 2024-06-28T15:37:33.576 INFO:teuthology.orchestra.run.smithi178.stdout: "access_key": "FSWBFVQZTYOXXOKESLAT", 2024-06-28T15:37:33.576 INFO:teuthology.orchestra.run.smithi178.stdout: "secret_key": "4FVp29UHx3cvz3mISVbHIReDboSlZ+cTOr55gVLTIEHhVYuhQH/T2Q==", 2024-06-28T15:37:33.576 INFO:teuthology.orchestra.run.smithi178.stdout: "active": true, 2024-06-28T15:37:33.576 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:37:33.576875Z" 2024-06-28T15:37:33.576 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T15:37:33.576 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-06-28T15:37:33.576 INFO:teuthology.orchestra.run.smithi178.stdout: "swift_keys": [], 2024-06-28T15:37:33.576 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": [], 2024-06-28T15:37:33.576 INFO:teuthology.orchestra.run.smithi178.stdout: "op_mask": "read, write, delete", 2024-06-28T15:37:33.577 INFO:teuthology.orchestra.run.smithi178.stdout: "default_placement": "", 2024-06-28T15:37:33.577 INFO:teuthology.orchestra.run.smithi178.stdout: "default_storage_class": "", 2024-06-28T15:37:33.577 INFO:teuthology.orchestra.run.smithi178.stdout: "placement_tags": [], 2024-06-28T15:37:33.577 INFO:teuthology.orchestra.run.smithi178.stdout: "bucket_quota": { 2024-06-28T15:37:33.577 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:33.577 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:33.577 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:33.577 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:33.577 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:33.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:33.577 INFO:teuthology.orchestra.run.smithi178.stdout: "user_quota": { 2024-06-28T15:37:33.577 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:33.577 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:33.577 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:33.578 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:33.578 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:33.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:33.578 INFO:teuthology.orchestra.run.smithi178.stdout: "temp_url_keys": [], 2024-06-28T15:37:33.578 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "rgw", 2024-06-28T15:37:33.578 INFO:teuthology.orchestra.run.smithi178.stdout: "mfa_ids": [], 2024-06-28T15:37:33.578 INFO:teuthology.orchestra.run.smithi178.stdout: "account_id": "", 2024-06-28T15:37:33.578 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2024-06-28T15:37:33.578 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:37:33.576861Z", 2024-06-28T15:37:33.578 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2024-06-28T15:37:33.578 INFO:teuthology.orchestra.run.smithi178.stdout: "group_ids": [] 2024-06-28T15:37:33.578 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T15:37:33.578 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:33.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.587+0000 7fb7f27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:33.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.591+0000 7fb93d6fe980 20 remove_watcher() i=0 2024-06-28T15:37:33.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.591+0000 7fb93d6fe980 2 removed watcher, disabling cache 2024-06-28T15:37:33.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.595+0000 7fb93d6fe980 20 remove_watcher() i=1 2024-06-28T15:37:33.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.595+0000 7fb93d6fe980 20 remove_watcher() i=2 2024-06-28T15:37:33.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.599+0000 7fb93d6fe980 20 remove_watcher() i=3 2024-06-28T15:37:33.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.599+0000 7fb93d6fe980 20 remove_watcher() i=4 2024-06-28T15:37:33.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.599+0000 7fb93d6fe980 20 remove_watcher() i=5 2024-06-28T15:37:33.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.603+0000 7fb93d6fe980 20 remove_watcher() i=6 2024-06-28T15:37:33.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.603+0000 7fb93d6fe980 20 remove_watcher() i=7 2024-06-28T15:37:33.606 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 1625075473 --totp-seed UFA27LCOQ7GP35BAQXHAWO74UFS5723VMTVAMRPGQYG5KWNB3IAQ7GNLHQUK47SS --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-06-28T15:37:33.691 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:33.691 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:33.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.743+0000 7f1aa59ed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.747+0000 7f1aa59ed980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.747+0000 7f1aa59ed980 20 realm 2024-06-28T15:37:33.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.747+0000 7f1aa59ed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.747+0000 7f1aa59ed980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.747+0000 7f1aa59ed980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:33.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.747+0000 7f1aa59ed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.747+0000 7f1aa59ed980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.747+0000 7f1aa59ed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.747+0000 7f1aa59ed980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:33.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.747+0000 7f1aa59ed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.747+0000 7f1aa59ed980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:33.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.747+0000 7f1aa59ed980 20 searching for the correct realm 2024-06-28T15:37:33.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.755+0000 7f1aa3cff640 0 -- 172.21.15.178:0/1191264084 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x558863abf850 msgr2=0x558863ac1cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:33.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.759+0000 7f1aa2cfd640 0 -- 172.21.15.178:0/1191264084 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f1a940e8020 msgr2=0x7f1a941359c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:37:33.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.767+0000 7f1aa59ed980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:33.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.767+0000 7f1aa59ed980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:33.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.767+0000 7f1aa59ed980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:33.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.767+0000 7f1aa59ed980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:33.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.767+0000 7f1aa59ed980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:33.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.767+0000 7f1aa59ed980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:33.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.767+0000 7f1aa59ed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.767+0000 7f1aa59ed980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.767+0000 7f1aa59ed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:33.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:33.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 20 zone default found 2024-06-28T15:37:33.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 4 Realm: () 2024-06-28T15:37:33.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:33.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:33.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:33.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 20 zonegroup default 2024-06-28T15:37:33.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 20 started sync module instance, tier type = 2024-06-28T15:37:33.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.771+0000 7f1aa59ed980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:33.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.787+0000 7f1aa59ed980 20 add_watcher() i=0 2024-06-28T15:37:33.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.787+0000 7f1aa59ed980 20 add_watcher() i=1 2024-06-28T15:37:33.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.787+0000 7f1aa59ed980 20 add_watcher() i=2 2024-06-28T15:37:33.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.787+0000 7f1aa59ed980 20 add_watcher() i=3 2024-06-28T15:37:33.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.787+0000 7f1aa59ed980 20 add_watcher() i=4 2024-06-28T15:37:33.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.787+0000 7f1aa59ed980 20 add_watcher() i=5 2024-06-28T15:37:33.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.787+0000 7f1aa59ed980 20 add_watcher() i=6 2024-06-28T15:37:33.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.787+0000 7f1aa59ed980 20 add_watcher() i=7 2024-06-28T15:37:33.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.787+0000 7f1aa59ed980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.791+0000 7f1959fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.791+0000 7f1aa59ed980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.791+0000 7f1aa59ed980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.791+0000 7f1aa59ed980 5 note: GC not initialized 2024-06-28T15:37:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.791+0000 7f1958fe1640 20 reqs_thread_entry: start 2024-06-28T15:37:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.791+0000 7f1aa59ed980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.791+0000 7f1aa59ed980 20 Filter name: none 2024-06-28T15:37:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.791+0000 7f18d27fc640 20 reqs_thread_entry: start 2024-06-28T15:37:33.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.791+0000 7f1aa59ed980 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2024-06-28T15:37:33.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.791+0000 7f1aa59ed980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.791+0000 7f1aa59ed980 20 rados_obj.operate() r=0 bl.length=460 2024-06-28T15:37:33.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.791+0000 7f1aa59ed980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x13 2024-06-28T15:37:33.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.791+0000 7f1aa59ed980 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-06-28T15:37:33.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.791+0000 7f1aa59ed980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:37:33.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.799+0000 7f1aa59ed980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-06-28T15:37:33.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.799+0000 7f1aa59ed980 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-06-28T15:37:33.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.799+0000 7f1aa59ed980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2024-06-28T15:37:33.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.799+0000 7f1986ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529817 cookie 94044276180816 notifier 7283 bl.length()=659 2024-06-28T15:37:33.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.799+0000 7f1986ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-06-28T15:37:33.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.799+0000 7f1986ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-06-28T15:37:33.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.803+0000 7f1959fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:33.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.807+0000 7f1aa59ed980 20 remove_watcher() i=0 2024-06-28T15:37:33.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.807+0000 7f1aa59ed980 2 removed watcher, disabling cache 2024-06-28T15:37:33.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.807+0000 7f1aa59ed980 20 remove_watcher() i=1 2024-06-28T15:37:33.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.811+0000 7f1aa59ed980 20 remove_watcher() i=2 2024-06-28T15:37:33.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.811+0000 7f1aa59ed980 20 remove_watcher() i=3 2024-06-28T15:37:33.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.815+0000 7f1aa59ed980 20 remove_watcher() i=4 2024-06-28T15:37:33.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.815+0000 7f1aa59ed980 20 remove_watcher() i=5 2024-06-28T15:37:33.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.815+0000 7f1aa59ed980 20 remove_watcher() i=6 2024-06-28T15:37:33.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.819+0000 7f1aa59ed980 20 remove_watcher() i=7 2024-06-28T15:37:33.821 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW33333333333333333 --account-name Mr.RGW33333333333333333 --email RGW33333333333333333+test@test.test --tenant testx 2024-06-28T15:37:33.900 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:33.901 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:33.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.951+0000 7f6e594a1980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.955+0000 7f6e594a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.955+0000 7f6e594a1980 20 realm 2024-06-28T15:37:33.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.955+0000 7f6e594a1980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.955+0000 7f6e594a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.955+0000 7f6e594a1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:33.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.955+0000 7f6e594a1980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.955+0000 7f6e594a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.955+0000 7f6e594a1980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.955+0000 7f6e594a1980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:33.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.955+0000 7f6e594a1980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.959+0000 7f6e594a1980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:33.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.959+0000 7f6e594a1980 20 searching for the correct realm 2024-06-28T15:37:33.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.963+0000 7f6e567b1640 0 -- 172.21.15.178:0/3513516372 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x55fb8a279700 msgr2=0x55fb8a27bb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:33.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.967+0000 7f6e56fb2640 0 -- 172.21.15.178:0/3513516372 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f6e4001b950 msgr2=0x7f6e40046400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:37:33.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.975+0000 7f6e594a1980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:33.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.975+0000 7f6e594a1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:33.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.975+0000 7f6e594a1980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:33.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.975+0000 7f6e594a1980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:33.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.975+0000 7f6e594a1980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:33.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.975+0000 7f6e594a1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:33.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.975+0000 7f6e594a1980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.975+0000 7f6e594a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.975+0000 7f6e594a1980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:33.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 20 zone default found 2024-06-28T15:37:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 4 Realm: () 2024-06-28T15:37:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 20 zonegroup default 2024-06-28T15:37:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 20 started sync module instance, tier type = 2024-06-28T15:37:33.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.979+0000 7f6e594a1980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.995+0000 7f6e594a1980 20 add_watcher() i=0 2024-06-28T15:37:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.995+0000 7f6e594a1980 20 add_watcher() i=1 2024-06-28T15:37:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.995+0000 7f6e594a1980 20 add_watcher() i=2 2024-06-28T15:37:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.995+0000 7f6e594a1980 20 add_watcher() i=3 2024-06-28T15:37:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.995+0000 7f6e594a1980 20 add_watcher() i=4 2024-06-28T15:37:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.995+0000 7f6e594a1980 20 add_watcher() i=5 2024-06-28T15:37:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.995+0000 7f6e594a1980 20 add_watcher() i=6 2024-06-28T15:37:33.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.995+0000 7f6e594a1980 20 add_watcher() i=7 2024-06-28T15:37:33.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.995+0000 7f6e594a1980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:33.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.999+0000 7f6d0d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:33.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.999+0000 7f6e594a1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:33.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.999+0000 7f6e594a1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:33.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.999+0000 7f6e594a1980 5 note: GC not initialized 2024-06-28T15:37:33.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.999+0000 7f6c87fff640 20 reqs_thread_entry: start 2024-06-28T15:37:33.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.999+0000 7f6e594a1980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:33.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.999+0000 7f6e594a1980 20 Filter name: none 2024-06-28T15:37:33.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.999+0000 7f6c85ffb640 20 reqs_thread_entry: start 2024-06-28T15:37:33.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.999+0000 7f6e594a1980 10 cache get: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 : miss 2024-06-28T15:37:33.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:33.999+0000 7f6e594a1980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.003+0000 7f6e594a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.003+0000 7f6e594a1980 10 cache put: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 info.flags=0x0 2024-06-28T15:37:33.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.003+0000 7f6e594a1980 10 adding default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 to cache LRU end 2024-06-28T15:37:33.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.003+0000 7f6e594a1980 20 failed to read name.testx$Mr.RGW33333333333333333 with: (2) No such file or directory 2024-06-28T15:37:33.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.003+0000 7f6e594a1980 10 cache get: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test : miss 2024-06-28T15:37:33.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.003+0000 7f6e594a1980 20 rados->read ofs=0 len=0 2024-06-28T15:37:33.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.003+0000 7f6e594a1980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:33.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.003+0000 7f6e594a1980 10 cache put: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test info.flags=0x0 2024-06-28T15:37:33.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.003+0000 7f6e594a1980 10 adding default.rgw.meta+users.email+rgw33333333333333333+test@test.test to cache LRU end 2024-06-28T15:37:33.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.003+0000 7f6e594a1980 20 failed to read rgw33333333333333333+test@test.test with: (2) No such file or directory 2024-06-28T15:37:33.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.003+0000 7f6e594a1980 10 cache put: name=default.rgw.meta+accounts+account.RGW33333333333333333 info.flags=0x17 2024-06-28T15:37:33.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.003+0000 7f6e594a1980 10 adding default.rgw.meta+accounts+account.RGW33333333333333333 to cache LRU end 2024-06-28T15:37:33.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.003+0000 7f6e594a1980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW33333333333333333, ofs0, ns] 2024-06-28T15:37:33.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.007+0000 7f6d3a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529818 cookie 94538843022768 notifier 7845 bl.length()=389 2024-06-28T15:37:33.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.007+0000 7f6d3a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW33333333333333333 info.flags=0x17 2024-06-28T15:37:33.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.007+0000 7f6d3a7fc640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW33333333333333333 to cache LRU end 2024-06-28T15:37:33.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.011+0000 7f6e594a1980 10 cache put: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 info.flags=0x17 2024-06-28T15:37:33.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.011+0000 7f6e594a1980 10 moving default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 to cache LRU end 2024-06-28T15:37:33.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.011+0000 7f6e594a1980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.testx$Mr.RGW33333333333333333, ofs0, ns] 2024-06-28T15:37:34.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.011+0000 7f6d3a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529851 cookie 94538843029808 notifier 7845 bl.length()=230 2024-06-28T15:37:34.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.011+0000 7f6d3a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 info.flags=0x17 2024-06-28T15:37:34.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.011+0000 7f6d3a7fc640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.testx$Mr.RGW33333333333333333 to cache LRU end 2024-06-28T15:37:34.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.015+0000 7f6e594a1980 10 cache put: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test info.flags=0x17 2024-06-28T15:37:34.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.015+0000 7f6e594a1980 10 moving default.rgw.meta+users.email+rgw33333333333333333+test@test.test to cache LRU end 2024-06-28T15:37:34.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.015+0000 7f6e594a1980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw33333333333333333+test@test.test, ofs0, ns] 2024-06-28T15:37:34.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.015+0000 7f6d3a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1112396529778 cookie 94538843010448 notifier 7845 bl.length()=234 2024-06-28T15:37:34.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.015+0000 7f6d3a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw33333333333333333+test@test.test info.flags=0x17 2024-06-28T15:37:34.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.015+0000 7f6d3a7fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw33333333333333333+test@test.test to cache LRU end 2024-06-28T15:37:34.008 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.019+0000 7f6d0d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:34.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.023+0000 7f6e594a1980 20 remove_watcher() i=0 2024-06-28T15:37:34.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.023+0000 7f6e594a1980 2 removed watcher, disabling cache 2024-06-28T15:37:34.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.027+0000 7f6e594a1980 20 remove_watcher() i=1 2024-06-28T15:37:34.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.027+0000 7f6e594a1980 20 remove_watcher() i=2 2024-06-28T15:37:34.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.031+0000 7f6e594a1980 20 remove_watcher() i=3 2024-06-28T15:37:34.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.031+0000 7f6e594a1980 20 remove_watcher() i=4 2024-06-28T15:37:34.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.035+0000 7f6e594a1980 20 remove_watcher() i=5 2024-06-28T15:37:34.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.035+0000 7f6e594a1980 20 remove_watcher() i=6 2024-06-28T15:37:34.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.035+0000 7f6e594a1980 20 remove_watcher() i=7 2024-06-28T15:37:34.038 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T15:37:34.038 INFO:teuthology.orchestra.run.smithi178.stdout: "id": "RGW33333333333333333", 2024-06-28T15:37:34.038 INFO:teuthology.orchestra.run.smithi178.stdout: "tenant": "testx", 2024-06-28T15:37:34.038 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "Mr.RGW33333333333333333", 2024-06-28T15:37:34.038 INFO:teuthology.orchestra.run.smithi178.stdout: "email": "RGW33333333333333333+test@test.test", 2024-06-28T15:37:34.038 INFO:teuthology.orchestra.run.smithi178.stdout: "quota": { 2024-06-28T15:37:34.038 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:34.039 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:34.039 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:34.039 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:34.039 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:34.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:34.039 INFO:teuthology.orchestra.run.smithi178.stdout: "bucket_quota": { 2024-06-28T15:37:34.039 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:34.039 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:34.039 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:34.039 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:34.039 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:34.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:34.039 INFO:teuthology.orchestra.run.smithi178.stdout: "max_users": 1000, 2024-06-28T15:37:34.039 INFO:teuthology.orchestra.run.smithi178.stdout: "max_roles": 1000, 2024-06-28T15:37:34.040 INFO:teuthology.orchestra.run.smithi178.stdout: "max_groups": 1000, 2024-06-28T15:37:34.040 INFO:teuthology.orchestra.run.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:37:34.040 INFO:teuthology.orchestra.run.smithi178.stdout: "max_access_keys": 4 2024-06-28T15:37:34.040 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T15:37:34.041 DEBUG:tasks.s3tests:Creating user RGW33333333333333333 on client.0 2024-06-28T15:37:34.041 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name Mr.RGW33333333333333333 --email 'testx$tenanteduser+test@test.test' --access-key BGKYQOOFCOKGCJYUTYHY --secret 2w1ByA3aS+2fdZD/JWRLB8ODzhdrTYMgqE7MwsyGUO4fst+dsoYL+w== --cluster ceph --account-id RGW33333333333333333 --account-root 2024-06-28T15:37:34.130 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:34.131 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:34.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.183+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.183+0000 7f13ceadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.183+0000 7f13ceadf980 20 realm 2024-06-28T15:37:34.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.183+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.183+0000 7f13ceadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.183+0000 7f13ceadf980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:34.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.183+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.183+0000 7f13ceadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.183+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.187+0000 7f13ceadf980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:34.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.187+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.187+0000 7f13ceadf980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:34.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.187+0000 7f13ceadf980 20 searching for the correct realm 2024-06-28T15:37:34.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.195+0000 7f13ccdf1640 0 -- 172.21.15.178:0/647398455 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f13bc1037f0 msgr2=0x7f13bc0fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:34.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.447+0000 7f13cc5f0640 0 -- 172.21.15.178:0/647398455 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f13bc1175d0 msgr2=0x7f13bc0d0700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:34.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.455+0000 7f13ceadf980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:34.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.455+0000 7f13ceadf980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:34.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.455+0000 7f13ceadf980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:34.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.455+0000 7f13ceadf980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:34.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.455+0000 7f13ceadf980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:34.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.455+0000 7f13ceadf980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:34.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.455+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.455+0000 7f13ceadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.455+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:34.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:34.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 20 zone default found 2024-06-28T15:37:34.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 4 Realm: () 2024-06-28T15:37:34.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:34.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:34.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:34.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 20 zonegroup default 2024-06-28T15:37:34.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.459+0000 7f13ceadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.463+0000 7f13ceadf980 20 started sync module instance, tier type = 2024-06-28T15:37:34.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.463+0000 7f13ceadf980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:34.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.487+0000 7f13ceadf980 20 add_watcher() i=0 2024-06-28T15:37:34.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.487+0000 7f13ceadf980 20 add_watcher() i=1 2024-06-28T15:37:34.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.487+0000 7f13ceadf980 20 add_watcher() i=2 2024-06-28T15:37:34.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.487+0000 7f13ceadf980 20 add_watcher() i=3 2024-06-28T15:37:34.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.487+0000 7f13ceadf980 20 add_watcher() i=4 2024-06-28T15:37:34.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.487+0000 7f13ceadf980 20 add_watcher() i=5 2024-06-28T15:37:34.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.487+0000 7f13ceadf980 20 add_watcher() i=6 2024-06-28T15:37:34.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.487+0000 7f13ceadf980 20 add_watcher() i=7 2024-06-28T15:37:34.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.487+0000 7f13ceadf980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:34.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f12837de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:34.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:34.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:34.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 5 note: GC not initialized 2024-06-28T15:37:34.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f12827dc640 20 reqs_thread_entry: start 2024-06-28T15:37:34.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:34.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 20 Filter name: none 2024-06-28T15:37:34.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f127b7fe640 20 reqs_thread_entry: start 2024-06-28T15:37:34.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2024-06-28T15:37:34.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2024-06-28T15:37:34.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-06-28T15:37:34.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test : miss 2024-06-28T15:37:34.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x0 2024-06-28T15:37:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 10 adding default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-06-28T15:37:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 10 cache get: name=default.rgw.meta+users.keys+BGKYQOOFCOKGCJYUTYHY : miss 2024-06-28T15:37:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 10 cache put: name=default.rgw.meta+users.keys+BGKYQOOFCOKGCJYUTYHY info.flags=0x0 2024-06-28T15:37:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 10 adding default.rgw.meta+users.keys+BGKYQOOFCOKGCJYUTYHY to cache LRU end 2024-06-28T15:37:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 10 cache get: name=default.rgw.meta+accounts+account.RGW33333333333333333 : miss 2024-06-28T15:37:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.495+0000 7f13ceadf980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.499+0000 7f13ceadf980 20 rados_obj.operate() r=0 bl.length=189 2024-06-28T15:37:34.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.499+0000 7f13ceadf980 10 cache put: name=default.rgw.meta+accounts+account.RGW33333333333333333 info.flags=0x17 2024-06-28T15:37:34.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.499+0000 7f13ceadf980 10 adding default.rgw.meta+accounts+account.RGW33333333333333333 to cache LRU end 2024-06-28T15:37:34.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.499+0000 7f13ceadf980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:37:34.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.499+0000 7f13ceadf980 10 cache get: name=default.rgw.meta+users.keys+BGKYQOOFCOKGCJYUTYHY : hit (negative entry) 2024-06-28T15:37:34.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.499+0000 7f13ceadf980 10 cache get: name=default.rgw.meta+users.keys+BGKYQOOFCOKGCJYUTYHY : hit (negative entry) 2024-06-28T15:37:34.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.499+0000 7f13ceadf980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-06-28T15:37:34.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.499+0000 7f13ceadf980 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-06-28T15:37:34.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.499+0000 7f13ceadf980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2024-06-28T15:37:34.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.503+0000 7f12a37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497099 cookie 93899808176272 notifier 7860 bl.length()=732 2024-06-28T15:37:34.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.503+0000 7f12a37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-06-28T15:37:34.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.503+0000 7f12a37fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-06-28T15:37:34.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.507+0000 7f13ceadf980 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x7 2024-06-28T15:37:34.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.507+0000 7f13ceadf980 10 moving default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-06-28T15:37:34.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.507+0000 7f13ceadf980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.email:testx$tenanteduser+test@test.test, ofs0, ns] 2024-06-28T15:37:34.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.507+0000 7f12a37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497028 cookie 93899808182256 notifier 7860 bl.length()=215 2024-06-28T15:37:34.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.507+0000 7f12a37fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x7 2024-06-28T15:37:34.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.507+0000 7f12a37fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-06-28T15:37:34.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.511+0000 7f13ceadf980 10 cache put: name=default.rgw.meta+users.keys+BGKYQOOFCOKGCJYUTYHY info.flags=0x7 2024-06-28T15:37:34.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.511+0000 7f13ceadf980 10 moving default.rgw.meta+users.keys+BGKYQOOFCOKGCJYUTYHY to cache LRU end 2024-06-28T15:37:34.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.511+0000 7f13ceadf980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:BGKYQOOFCOKGCJYUTYHY, ofs0, ns] 2024-06-28T15:37:34.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.511+0000 7f12a2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497148 cookie 93899808167472 notifier 7860 bl.length()=201 2024-06-28T15:37:34.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.511+0000 7f12a2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+BGKYQOOFCOKGCJYUTYHY info.flags=0x7 2024-06-28T15:37:34.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.511+0000 7f12a2ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+BGKYQOOFCOKGCJYUTYHY to cache LRU end 2024-06-28T15:37:34.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.515+0000 7f13ceadf980 20 linked user testx$tenanteduser.client.0 to account RGW33333333333333333 2024-06-28T15:37:34.504 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T15:37:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: "user_id": "testx$tenanteduser.client.0", 2024-06-28T15:37:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: "display_name": "Mr.RGW33333333333333333", 2024-06-28T15:37:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "email": "testx$tenanteduser+test@test.test", 2024-06-28T15:37:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "suspended": 0, 2024-06-28T15:37:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:37:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "subusers": [], 2024-06-28T15:37:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "keys": [ 2024-06-28T15:37:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T15:37:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "user": "testx$tenanteduser.client.0", 2024-06-28T15:37:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "access_key": "BGKYQOOFCOKGCJYUTYHY", 2024-06-28T15:37:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "secret_key": "2w1ByA3aS+2fdZD/JWRLB8ODzhdrTYMgqE7MwsyGUO4fst+dsoYL+w==", 2024-06-28T15:37:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "active": true, 2024-06-28T15:37:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:37:34.502432Z" 2024-06-28T15:37:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T15:37:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-06-28T15:37:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "swift_keys": [], 2024-06-28T15:37:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": [], 2024-06-28T15:37:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "op_mask": "read, write, delete", 2024-06-28T15:37:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "default_placement": "", 2024-06-28T15:37:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "default_storage_class": "", 2024-06-28T15:37:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "placement_tags": [], 2024-06-28T15:37:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "bucket_quota": { 2024-06-28T15:37:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "user_quota": { 2024-06-28T15:37:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "temp_url_keys": [], 2024-06-28T15:37:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "root", 2024-06-28T15:37:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "mfa_ids": [], 2024-06-28T15:37:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "account_id": "RGW33333333333333333", 2024-06-28T15:37:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2024-06-28T15:37:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:37:34.502397Z", 2024-06-28T15:37:34.508 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2024-06-28T15:37:34.508 INFO:teuthology.orchestra.run.smithi178.stdout: "group_ids": [] 2024-06-28T15:37:34.508 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T15:37:34.508 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:34.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.515+0000 7f12837de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:34.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.523+0000 7f13ceadf980 20 remove_watcher() i=0 2024-06-28T15:37:34.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.523+0000 7f13ceadf980 2 removed watcher, disabling cache 2024-06-28T15:37:34.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.523+0000 7f13ceadf980 20 remove_watcher() i=1 2024-06-28T15:37:34.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.527+0000 7f13ceadf980 20 remove_watcher() i=2 2024-06-28T15:37:34.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.527+0000 7f13ceadf980 20 remove_watcher() i=3 2024-06-28T15:37:34.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.531+0000 7f13ceadf980 20 remove_watcher() i=4 2024-06-28T15:37:34.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.531+0000 7f13ceadf980 20 remove_watcher() i=5 2024-06-28T15:37:34.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.535+0000 7f13ceadf980 20 remove_watcher() i=6 2024-06-28T15:37:34.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.535+0000 7f13ceadf980 20 remove_watcher() i=7 2024-06-28T15:37:34.542 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 3626483834 --totp-seed P3SSHART6V2ZDTWAIYEFUM5P74RTSEXBKTCFZXU3RHTWXL23ENC5MSBQIPZPB7RQ --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-06-28T15:37:34.587 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:34.588 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:34.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.639+0000 7f084237c980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.639+0000 7f084237c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.639+0000 7f084237c980 20 realm 2024-06-28T15:37:34.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.639+0000 7f084237c980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.639+0000 7f084237c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.639+0000 7f084237c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:34.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.639+0000 7f084237c980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.639+0000 7f084237c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.639+0000 7f084237c980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.643+0000 7f084237c980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:34.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.643+0000 7f084237c980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.643+0000 7f084237c980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:34.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.643+0000 7f084237c980 20 searching for the correct realm 2024-06-28T15:37:34.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.651+0000 7f083f68c640 0 -- 172.21.15.178:0/3047708269 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f083804a060 msgr2=0x7f083804c0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:34.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.655+0000 7f083f68c640 0 -- 172.21.15.178:0/3047708269 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f0708080b00 msgr2=0x7f0708082f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:34.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:34.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:34.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:34.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:34.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:34.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:34.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:34.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:34.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 zone default found 2024-06-28T15:37:34.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 4 Realm: () 2024-06-28T15:37:34.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:34.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:34.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:34.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 zonegroup default 2024-06-28T15:37:34.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.663+0000 7f084237c980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.667+0000 7f084237c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.667+0000 7f084237c980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.667+0000 7f084237c980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.667+0000 7f084237c980 20 started sync module instance, tier type = 2024-06-28T15:37:34.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.667+0000 7f084237c980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:34.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.683+0000 7f084237c980 20 add_watcher() i=0 2024-06-28T15:37:34.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.683+0000 7f084237c980 20 add_watcher() i=1 2024-06-28T15:37:34.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.683+0000 7f084237c980 20 add_watcher() i=2 2024-06-28T15:37:34.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.683+0000 7f084237c980 20 add_watcher() i=3 2024-06-28T15:37:34.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.683+0000 7f084237c980 20 add_watcher() i=4 2024-06-28T15:37:34.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.683+0000 7f084237c980 20 add_watcher() i=5 2024-06-28T15:37:34.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.683+0000 7f084237c980 20 add_watcher() i=6 2024-06-28T15:37:34.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.683+0000 7f084237c980 20 add_watcher() i=7 2024-06-28T15:37:34.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.683+0000 7f084237c980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:34.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.683+0000 7f06f67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:34.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.683+0000 7f084237c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:34.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.683+0000 7f084237c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:34.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.683+0000 7f084237c980 5 note: GC not initialized 2024-06-28T15:37:34.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.687+0000 7f06f57e2640 20 reqs_thread_entry: start 2024-06-28T15:37:34.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.687+0000 7f084237c980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:34.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.687+0000 7f084237c980 20 Filter name: none 2024-06-28T15:37:34.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.687+0000 7f06ee7fc640 20 reqs_thread_entry: start 2024-06-28T15:37:34.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.687+0000 7f084237c980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2024-06-28T15:37:34.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.687+0000 7f084237c980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.687+0000 7f084237c980 20 rados_obj.operate() r=0 bl.length=532 2024-06-28T15:37:34.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.687+0000 7f084237c980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x13 2024-06-28T15:37:34.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.687+0000 7f084237c980 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-06-28T15:37:34.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.687+0000 7f084237c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:37:34.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.691+0000 7f084237c980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-06-28T15:37:34.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.691+0000 7f084237c980 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-06-28T15:37:34.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.691+0000 7f084237c980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2024-06-28T15:37:34.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.691+0000 7f071effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497100 cookie 94578478795456 notifier 7337 bl.length()=746 2024-06-28T15:37:34.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.691+0000 7f071effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-06-28T15:37:34.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.691+0000 7f071effd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-06-28T15:37:34.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.695+0000 7f06f67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:34.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.699+0000 7f084237c980 20 remove_watcher() i=0 2024-06-28T15:37:34.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.699+0000 7f084237c980 2 removed watcher, disabling cache 2024-06-28T15:37:34.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.699+0000 7f084237c980 20 remove_watcher() i=1 2024-06-28T15:37:34.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.703+0000 7f084237c980 20 remove_watcher() i=2 2024-06-28T15:37:34.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.703+0000 7f084237c980 20 remove_watcher() i=3 2024-06-28T15:37:34.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.707+0000 7f084237c980 20 remove_watcher() i=4 2024-06-28T15:37:34.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.707+0000 7f084237c980 20 remove_watcher() i=5 2024-06-28T15:37:34.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.711+0000 7f084237c980 20 remove_watcher() i=6 2024-06-28T15:37:34.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.711+0000 7f084237c980 20 remove_watcher() i=7 2024-06-28T15:37:34.714 DEBUG:tasks.s3tests:Creating user foobar.client.0 on client.0 2024-06-28T15:37:34.714 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foobar.client.0 --display-name Mr.foobar.client.0 --email foobar+test@test.test --access-key EIFQPLUKQCPWQTPVZVEI --secret IOv4LG7enko1gUMD7DVL1hvjeBc5s5iErpp/5Y8KDanLbYxI0jc24w== --cluster ceph 2024-06-28T15:37:34.753 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:34.753 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:34.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.803+0000 7f951c872980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.803+0000 7f951c872980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.803+0000 7f951c872980 20 realm 2024-06-28T15:37:34.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.803+0000 7f951c872980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.807+0000 7f951c872980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.807+0000 7f951c872980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:34.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.807+0000 7f951c872980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.807+0000 7f951c872980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:34.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.807+0000 7f951c872980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.807+0000 7f951c872980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:34.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.807+0000 7f951c872980 20 rados->read ofs=0 len=0 2024-06-28T15:37:34.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.807+0000 7f951c872980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:34.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.807+0000 7f951c872980 20 searching for the correct realm 2024-06-28T15:37:34.803 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:34.815+0000 7f951ab84640 0 -- 172.21.15.178:0/2496896490 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f950411ebd0 msgr2=0x7f95040c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:37:35.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.031+0000 7f951a383640 0 -- 172.21.15.178:0/2496896490 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x5565de889120 msgr2=0x5565de88b510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:35.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.039+0000 7f951c872980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:35.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.039+0000 7f951c872980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:35.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.039+0000 7f951c872980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:35.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.039+0000 7f951c872980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:35.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.039+0000 7f951c872980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:35.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.039+0000 7f951c872980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:35.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.039+0000 7f951c872980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.039+0000 7f951c872980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.039+0000 7f951c872980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.039+0000 7f951c872980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:35.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.039+0000 7f951c872980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:35.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 20 zone default found 2024-06-28T15:37:35.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 4 Realm: () 2024-06-28T15:37:35.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:35.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:35.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:35.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 20 zonegroup default 2024-06-28T15:37:35.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 20 started sync module instance, tier type = 2024-06-28T15:37:35.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.043+0000 7f951c872980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:35.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.063+0000 7f951c872980 20 add_watcher() i=0 2024-06-28T15:37:35.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.063+0000 7f951c872980 20 add_watcher() i=1 2024-06-28T15:37:35.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.063+0000 7f951c872980 20 add_watcher() i=2 2024-06-28T15:37:35.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.063+0000 7f951c872980 20 add_watcher() i=3 2024-06-28T15:37:35.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.063+0000 7f951c872980 20 add_watcher() i=4 2024-06-28T15:37:35.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.063+0000 7f951c872980 20 add_watcher() i=5 2024-06-28T15:37:35.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.063+0000 7f951c872980 20 add_watcher() i=6 2024-06-28T15:37:35.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.067+0000 7f951c872980 20 add_watcher() i=7 2024-06-28T15:37:35.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.067+0000 7f951c872980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:35.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f93d0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:35.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f951c872980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:35.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f951c872980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:35.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f951c872980 5 note: GC not initialized 2024-06-28T15:37:35.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f934b7fe640 20 reqs_thread_entry: start 2024-06-28T15:37:35.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f951c872980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:35.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f951c872980 20 Filter name: none 2024-06-28T15:37:35.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f93497fa640 20 reqs_thread_entry: start 2024-06-28T15:37:35.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f951c872980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-06-28T15:37:35.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f951c872980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f951c872980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f951c872980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x0 2024-06-28T15:37:35.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f951c872980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-06-28T15:37:35.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f951c872980 10 cache get: name=default.rgw.meta+users.email+foobar+test@test.test : miss 2024-06-28T15:37:35.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.071+0000 7f951c872980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f951c872980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f951c872980 10 cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x0 2024-06-28T15:37:35.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f951c872980 10 adding default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-06-28T15:37:35.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f951c872980 10 cache get: name=default.rgw.meta+users.keys+EIFQPLUKQCPWQTPVZVEI : miss 2024-06-28T15:37:35.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f951c872980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f951c872980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f951c872980 10 cache put: name=default.rgw.meta+users.keys+EIFQPLUKQCPWQTPVZVEI info.flags=0x0 2024-06-28T15:37:35.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f951c872980 10 adding default.rgw.meta+users.keys+EIFQPLUKQCPWQTPVZVEI to cache LRU end 2024-06-28T15:37:35.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f951c872980 10 cache get: name=default.rgw.meta+users.keys+EIFQPLUKQCPWQTPVZVEI : hit (negative entry) 2024-06-28T15:37:35.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f951c872980 10 cache get: name=default.rgw.meta+users.keys+EIFQPLUKQCPWQTPVZVEI : hit (negative entry) 2024-06-28T15:37:35.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f951c872980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-06-28T15:37:35.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f951c872980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-06-28T15:37:35.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f951c872980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-06-28T15:37:35.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f93fdffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497029 cookie 93896013602704 notifier 7896 bl.length()=660 2024-06-28T15:37:35.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f93fdffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-06-28T15:37:35.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.075+0000 7f93fdffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-06-28T15:37:35.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.083+0000 7f951c872980 10 cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x7 2024-06-28T15:37:35.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.083+0000 7f951c872980 10 moving default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-06-28T15:37:35.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.083+0000 7f951c872980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:foobar+test@test.test, ofs0, ns] 2024-06-28T15:37:35.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.083+0000 7f93fdffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497115 cookie 93896013585152 notifier 7896 bl.length()=191 2024-06-28T15:37:35.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.083+0000 7f93fdffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x7 2024-06-28T15:37:35.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.083+0000 7f93fdffb640 10 rgw watcher librados: moving default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-06-28T15:37:35.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.087+0000 7f951c872980 10 cache put: name=default.rgw.meta+users.keys+EIFQPLUKQCPWQTPVZVEI info.flags=0x7 2024-06-28T15:37:35.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.087+0000 7f951c872980 10 moving default.rgw.meta+users.keys+EIFQPLUKQCPWQTPVZVEI to cache LRU end 2024-06-28T15:37:35.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.087+0000 7f951c872980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:EIFQPLUKQCPWQTPVZVEI, ofs0, ns] 2024-06-28T15:37:35.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.087+0000 7f93fdffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497075 cookie 93896013586208 notifier 7896 bl.length()=189 2024-06-28T15:37:35.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.087+0000 7f93fdffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+EIFQPLUKQCPWQTPVZVEI info.flags=0x7 2024-06-28T15:37:35.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.087+0000 7f93fdffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+EIFQPLUKQCPWQTPVZVEI to cache LRU end 2024-06-28T15:37:35.079 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T15:37:35.079 INFO:teuthology.orchestra.run.smithi178.stdout: "user_id": "foobar.client.0", 2024-06-28T15:37:35.079 INFO:teuthology.orchestra.run.smithi178.stdout: "display_name": "Mr.foobar.client.0", 2024-06-28T15:37:35.079 INFO:teuthology.orchestra.run.smithi178.stdout: "email": "foobar+test@test.test", 2024-06-28T15:37:35.079 INFO:teuthology.orchestra.run.smithi178.stdout: "suspended": 0, 2024-06-28T15:37:35.079 INFO:teuthology.orchestra.run.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:37:35.079 INFO:teuthology.orchestra.run.smithi178.stdout: "subusers": [], 2024-06-28T15:37:35.079 INFO:teuthology.orchestra.run.smithi178.stdout: "keys": [ 2024-06-28T15:37:35.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T15:37:35.080 INFO:teuthology.orchestra.run.smithi178.stdout: "user": "foobar.client.0", 2024-06-28T15:37:35.080 INFO:teuthology.orchestra.run.smithi178.stdout: "access_key": "EIFQPLUKQCPWQTPVZVEI", 2024-06-28T15:37:35.080 INFO:teuthology.orchestra.run.smithi178.stdout: "secret_key": "IOv4LG7enko1gUMD7DVL1hvjeBc5s5iErpp/5Y8KDanLbYxI0jc24w==", 2024-06-28T15:37:35.080 INFO:teuthology.orchestra.run.smithi178.stdout: "active": true, 2024-06-28T15:37:35.080 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:37:35.077872Z" 2024-06-28T15:37:35.080 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T15:37:35.080 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-06-28T15:37:35.080 INFO:teuthology.orchestra.run.smithi178.stdout: "swift_keys": [], 2024-06-28T15:37:35.080 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": [], 2024-06-28T15:37:35.080 INFO:teuthology.orchestra.run.smithi178.stdout: "op_mask": "read, write, delete", 2024-06-28T15:37:35.080 INFO:teuthology.orchestra.run.smithi178.stdout: "default_placement": "", 2024-06-28T15:37:35.080 INFO:teuthology.orchestra.run.smithi178.stdout: "default_storage_class": "", 2024-06-28T15:37:35.080 INFO:teuthology.orchestra.run.smithi178.stdout: "placement_tags": [], 2024-06-28T15:37:35.081 INFO:teuthology.orchestra.run.smithi178.stdout: "bucket_quota": { 2024-06-28T15:37:35.081 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:35.081 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:35.081 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:35.081 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:35.081 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:35.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:35.081 INFO:teuthology.orchestra.run.smithi178.stdout: "user_quota": { 2024-06-28T15:37:35.081 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:35.081 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:35.081 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:35.081 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:35.081 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:35.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:35.082 INFO:teuthology.orchestra.run.smithi178.stdout: "temp_url_keys": [], 2024-06-28T15:37:35.082 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "rgw", 2024-06-28T15:37:35.082 INFO:teuthology.orchestra.run.smithi178.stdout: "mfa_ids": [], 2024-06-28T15:37:35.082 INFO:teuthology.orchestra.run.smithi178.stdout: "account_id": "", 2024-06-28T15:37:35.082 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2024-06-28T15:37:35.082 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:37:35.077828Z", 2024-06-28T15:37:35.082 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2024-06-28T15:37:35.082 INFO:teuthology.orchestra.run.smithi178.stdout: "group_ids": [] 2024-06-28T15:37:35.082 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T15:37:35.082 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:35.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.091+0000 7f93d0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:35.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.095+0000 7f951c872980 20 remove_watcher() i=0 2024-06-28T15:37:35.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.095+0000 7f951c872980 2 removed watcher, disabling cache 2024-06-28T15:37:35.087 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.099+0000 7f951c872980 20 remove_watcher() i=1 2024-06-28T15:37:35.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.099+0000 7f951c872980 20 remove_watcher() i=2 2024-06-28T15:37:35.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.103+0000 7f951c872980 20 remove_watcher() i=3 2024-06-28T15:37:35.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.103+0000 7f951c872980 20 remove_watcher() i=4 2024-06-28T15:37:35.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.107+0000 7f951c872980 20 remove_watcher() i=5 2024-06-28T15:37:35.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.107+0000 7f951c872980 20 remove_watcher() i=6 2024-06-28T15:37:35.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.111+0000 7f951c872980 20 remove_watcher() i=7 2024-06-28T15:37:35.115 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foobar.client.0 --totp-serial 2346894901 --totp-seed 7SZCTGEHZKUJCFMFS6FVUKQCJP5LBGOKVHCBEFF3ALUTZT7KOWOD3BR4HHBCUZDT --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-06-28T15:37:35.205 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:35.205 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:35.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.255+0000 7ffafb496980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.259+0000 7ffafb496980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.259+0000 7ffafb496980 20 realm 2024-06-28T15:37:35.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.259+0000 7ffafb496980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.259+0000 7ffafb496980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.259+0000 7ffafb496980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:35.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.259+0000 7ffafb496980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.259+0000 7ffafb496980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.259+0000 7ffafb496980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.259+0000 7ffafb496980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:35.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.259+0000 7ffafb496980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.259+0000 7ffafb496980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:35.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.259+0000 7ffafb496980 20 searching for the correct realm 2024-06-28T15:37:35.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.267+0000 7ffaf87a6640 0 -- 172.21.15.178:0/2910989449 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x55f3bd38b210 msgr2=0x55f3bd38d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:35.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.271+0000 7ffaf8fa7640 0 -- 172.21.15.178:0/2910989449 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7ffae0076040 msgr2=0x7ffae006aa40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:35.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.279+0000 7ffafb496980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:35.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.279+0000 7ffafb496980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:35.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.279+0000 7ffafb496980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:35.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.279+0000 7ffafb496980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:35.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.279+0000 7ffafb496980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:35.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.279+0000 7ffafb496980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:35.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.279+0000 7ffafb496980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.279+0000 7ffafb496980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.279+0000 7ffafb496980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:35.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:35.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 20 zone default found 2024-06-28T15:37:35.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 4 Realm: () 2024-06-28T15:37:35.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:35.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:35.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:35.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 20 zonegroup default 2024-06-28T15:37:35.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 20 started sync module instance, tier type = 2024-06-28T15:37:35.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.283+0000 7ffafb496980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:35.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.299+0000 7ffafb496980 20 add_watcher() i=0 2024-06-28T15:37:35.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.299+0000 7ffafb496980 20 add_watcher() i=1 2024-06-28T15:37:35.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.299+0000 7ffafb496980 20 add_watcher() i=2 2024-06-28T15:37:35.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.299+0000 7ffafb496980 20 add_watcher() i=3 2024-06-28T15:37:35.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.299+0000 7ffafb496980 20 add_watcher() i=4 2024-06-28T15:37:35.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.299+0000 7ffafb496980 20 add_watcher() i=5 2024-06-28T15:37:35.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.299+0000 7ffafb496980 20 add_watcher() i=6 2024-06-28T15:37:35.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.299+0000 7ffafb496980 20 add_watcher() i=7 2024-06-28T15:37:35.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.299+0000 7ffafb496980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:35.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.303+0000 7ff9af7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:35.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.303+0000 7ffafb496980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:35.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.303+0000 7ffafb496980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:35.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.303+0000 7ffafb496980 5 note: GC not initialized 2024-06-28T15:37:35.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.303+0000 7ff9ae7e4640 20 reqs_thread_entry: start 2024-06-28T15:37:35.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.303+0000 7ffafb496980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:35.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.303+0000 7ffafb496980 20 Filter name: none 2024-06-28T15:37:35.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.303+0000 7ff9a77fe640 20 reqs_thread_entry: start 2024-06-28T15:37:35.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.303+0000 7ffafb496980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-06-28T15:37:35.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.303+0000 7ffafb496980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.303+0000 7ffafb496980 20 rados_obj.operate() r=0 bl.length=472 2024-06-28T15:37:35.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.303+0000 7ffafb496980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-06-28T15:37:35.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.303+0000 7ffafb496980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-06-28T15:37:35.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.303+0000 7ffafb496980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:37:35.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.307+0000 7ffafb496980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-06-28T15:37:35.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.307+0000 7ffafb496980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-06-28T15:37:35.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.307+0000 7ffafb496980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-06-28T15:37:35.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.307+0000 7ff9cffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497030 cookie 94505340060016 notifier 7914 bl.length()=674 2024-06-28T15:37:35.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.307+0000 7ff9cffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-06-28T15:37:35.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.307+0000 7ff9cffff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-06-28T15:37:35.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.311+0000 7ff9af7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:35.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.315+0000 7ffafb496980 20 remove_watcher() i=0 2024-06-28T15:37:35.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.315+0000 7ffafb496980 2 removed watcher, disabling cache 2024-06-28T15:37:35.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.319+0000 7ffafb496980 20 remove_watcher() i=1 2024-06-28T15:37:35.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.319+0000 7ffafb496980 20 remove_watcher() i=2 2024-06-28T15:37:35.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.323+0000 7ffafb496980 20 remove_watcher() i=3 2024-06-28T15:37:35.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.323+0000 7ffafb496980 20 remove_watcher() i=4 2024-06-28T15:37:35.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.327+0000 7ffafb496980 20 remove_watcher() i=5 2024-06-28T15:37:35.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.327+0000 7ffafb496980 20 remove_watcher() i=6 2024-06-28T15:37:35.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.327+0000 7ffafb496980 20 remove_watcher() i=7 2024-06-28T15:37:35.335 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 caps add --uid foobar.client.0 --caps 'oidc-provider=*;roles=*;user-policy=*' --cluster ceph 2024-06-28T15:37:35.371 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:35.371 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:35.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.423+0000 7fdb3d57a980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.423+0000 7fdb3d57a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.423+0000 7fdb3d57a980 20 realm 2024-06-28T15:37:35.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.423+0000 7fdb3d57a980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.423+0000 7fdb3d57a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.423+0000 7fdb3d57a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:35.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.423+0000 7fdb3d57a980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.423+0000 7fdb3d57a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.423+0000 7fdb3d57a980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.427+0000 7fdb3d57a980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:35.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.427+0000 7fdb3d57a980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.427+0000 7fdb3d57a980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:35.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.427+0000 7fdb3d57a980 20 searching for the correct realm 2024-06-28T15:37:35.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.435+0000 7fdb3a88a640 0 -- 172.21.15.178:0/3121946095 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fdb34024040 msgr2=0x7fdb340263d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:35.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.439+0000 7fdb3a88a640 0 -- 172.21.15.178:0/3121946095 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fda04080b00 msgr2=0x7fda04082f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:35.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.447+0000 7fdb3d57a980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:35.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.447+0000 7fdb3d57a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:35.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.447+0000 7fdb3d57a980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:35.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.447+0000 7fdb3d57a980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:35.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.447+0000 7fdb3d57a980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:35.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.447+0000 7fdb3d57a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:35.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.447+0000 7fdb3d57a980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.447+0000 7fdb3d57a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.447+0000 7fdb3d57a980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.447+0000 7fdb3d57a980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:35.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.447+0000 7fdb3d57a980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:35.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 20 zone default found 2024-06-28T15:37:35.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 4 Realm: () 2024-06-28T15:37:35.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:35.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:35.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:35.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 20 zonegroup default 2024-06-28T15:37:35.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 20 started sync module instance, tier type = 2024-06-28T15:37:35.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.451+0000 7fdb3d57a980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:35.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.471+0000 7fdb3d57a980 20 add_watcher() i=0 2024-06-28T15:37:35.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.471+0000 7fdb3d57a980 20 add_watcher() i=1 2024-06-28T15:37:35.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.471+0000 7fdb3d57a980 20 add_watcher() i=2 2024-06-28T15:37:35.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.471+0000 7fdb3d57a980 20 add_watcher() i=3 2024-06-28T15:37:35.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.471+0000 7fdb3d57a980 20 add_watcher() i=4 2024-06-28T15:37:35.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.471+0000 7fdb3d57a980 20 add_watcher() i=5 2024-06-28T15:37:35.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.471+0000 7fdb3d57a980 20 add_watcher() i=6 2024-06-28T15:37:35.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.471+0000 7fdb3d57a980 20 add_watcher() i=7 2024-06-28T15:37:35.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.471+0000 7fdb3d57a980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:35.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fd9f1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:35.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fdb3d57a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:35.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fdb3d57a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:35.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fdb3d57a980 5 note: GC not initialized 2024-06-28T15:37:35.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fd9f0fe1640 20 reqs_thread_entry: start 2024-06-28T15:37:35.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fdb3d57a980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:35.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fdb3d57a980 20 Filter name: none 2024-06-28T15:37:35.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fd96a7fc640 20 reqs_thread_entry: start 2024-06-28T15:37:35.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fdb3d57a980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-06-28T15:37:35.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fdb3d57a980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fdb3d57a980 20 rados_obj.operate() r=0 bl.length=486 2024-06-28T15:37:35.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fdb3d57a980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-06-28T15:37:35.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fdb3d57a980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-06-28T15:37:35.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fdb3d57a980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:37:35.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fdb3d57a980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-06-28T15:37:35.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fdb3d57a980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-06-28T15:37:35.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.475+0000 7fdb3d57a980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-06-28T15:37:35.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.479+0000 7fda1effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497031 cookie 94487802690336 notifier 7382 bl.length()=727 2024-06-28T15:37:35.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.479+0000 7fda1effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-06-28T15:37:35.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.479+0000 7fda1effd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-06-28T15:37:35.468 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T15:37:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: "user_id": "foobar.client.0", 2024-06-28T15:37:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: "display_name": "Mr.foobar.client.0", 2024-06-28T15:37:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: "email": "foobar+test@test.test", 2024-06-28T15:37:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: "suspended": 0, 2024-06-28T15:37:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:37:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "subusers": [], 2024-06-28T15:37:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "keys": [ 2024-06-28T15:37:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T15:37:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "user": "foobar.client.0", 2024-06-28T15:37:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "access_key": "EIFQPLUKQCPWQTPVZVEI", 2024-06-28T15:37:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "secret_key": "IOv4LG7enko1gUMD7DVL1hvjeBc5s5iErpp/5Y8KDanLbYxI0jc24w==", 2024-06-28T15:37:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "active": true, 2024-06-28T15:37:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:37:35.077872Z" 2024-06-28T15:37:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T15:37:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-06-28T15:37:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "swift_keys": [], 2024-06-28T15:37:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": [ 2024-06-28T15:37:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T15:37:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "oidc-provider", 2024-06-28T15:37:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: "perm": "*" 2024-06-28T15:37:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T15:37:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "roles", 2024-06-28T15:37:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: "perm": "*" 2024-06-28T15:37:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T15:37:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "user-policy", 2024-06-28T15:37:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: "perm": "*" 2024-06-28T15:37:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T15:37:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-06-28T15:37:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "op_mask": "read, write, delete", 2024-06-28T15:37:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "default_placement": "", 2024-06-28T15:37:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "default_storage_class": "", 2024-06-28T15:37:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "placement_tags": [], 2024-06-28T15:37:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "bucket_quota": { 2024-06-28T15:37:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "user_quota": { 2024-06-28T15:37:35.472 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:35.472 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:35.472 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:35.472 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:35.472 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:35.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:35.472 INFO:teuthology.orchestra.run.smithi178.stdout: "temp_url_keys": [], 2024-06-28T15:37:35.472 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "rgw", 2024-06-28T15:37:35.472 INFO:teuthology.orchestra.run.smithi178.stdout: "mfa_ids": [ 2024-06-28T15:37:35.472 INFO:teuthology.orchestra.run.smithi178.stdout: "2346894901" 2024-06-28T15:37:35.472 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-06-28T15:37:35.472 INFO:teuthology.orchestra.run.smithi178.stdout: "account_id": "", 2024-06-28T15:37:35.472 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2024-06-28T15:37:35.473 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:37:35.077828Z", 2024-06-28T15:37:35.473 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2024-06-28T15:37:35.473 INFO:teuthology.orchestra.run.smithi178.stdout: "group_ids": [] 2024-06-28T15:37:35.473 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T15:37:35.473 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:35.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.479+0000 7fd9f1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:35.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.483+0000 7fdb3d57a980 20 remove_watcher() i=0 2024-06-28T15:37:35.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.483+0000 7fdb3d57a980 2 removed watcher, disabling cache 2024-06-28T15:37:35.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.487+0000 7fdb3d57a980 20 remove_watcher() i=1 2024-06-28T15:37:35.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.487+0000 7fdb3d57a980 20 remove_watcher() i=2 2024-06-28T15:37:35.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.491+0000 7fdb3d57a980 20 remove_watcher() i=3 2024-06-28T15:37:35.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.491+0000 7fdb3d57a980 20 remove_watcher() i=4 2024-06-28T15:37:35.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.495+0000 7fdb3d57a980 20 remove_watcher() i=5 2024-06-28T15:37:35.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.495+0000 7fdb3d57a980 20 remove_watcher() i=6 2024-06-28T15:37:35.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.495+0000 7fdb3d57a980 20 remove_watcher() i=7 2024-06-28T15:37:35.499 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW88888888888888888 --account-name Mr.RGW88888888888888888 --email RGW88888888888888888+test@test.test 2024-06-28T15:37:35.579 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:35.580 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:35.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.631+0000 7f127cb72980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.635+0000 7f127cb72980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.635+0000 7f127cb72980 20 realm 2024-06-28T15:37:35.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.635+0000 7f127cb72980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.635+0000 7f127cb72980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.635+0000 7f127cb72980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:35.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.635+0000 7f127cb72980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.635+0000 7f127cb72980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.635+0000 7f127cb72980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.635+0000 7f127cb72980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:35.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.635+0000 7f127cb72980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.635+0000 7f127cb72980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:35.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.635+0000 7f127cb72980 20 searching for the correct realm 2024-06-28T15:37:35.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.643+0000 7f1279e82640 0 -- 172.21.15.178:0/1752486002 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f126c0380b0 msgr2=0x7f126c070700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:35.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.647+0000 7f1279e82640 0 -- 172.21.15.178:0/1752486002 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7f1144080b60 msgr2=0x7f1144082f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:35.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.655+0000 7f127cb72980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:35.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.655+0000 7f127cb72980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:35.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.655+0000 7f127cb72980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:35.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.655+0000 7f127cb72980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:35.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.655+0000 7f127cb72980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:35.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.655+0000 7f127cb72980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.655+0000 7f127cb72980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.655+0000 7f127cb72980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.655+0000 7f127cb72980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.655+0000 7f127cb72980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:35.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.655+0000 7f127cb72980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:35.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 20 zone default found 2024-06-28T15:37:35.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 4 Realm: () 2024-06-28T15:37:35.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:35.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:35.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:35.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 20 zonegroup default 2024-06-28T15:37:35.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 20 started sync module instance, tier type = 2024-06-28T15:37:35.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.659+0000 7f127cb72980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:35.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.675+0000 7f127cb72980 20 add_watcher() i=0 2024-06-28T15:37:35.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.675+0000 7f127cb72980 20 add_watcher() i=1 2024-06-28T15:37:35.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.675+0000 7f127cb72980 20 add_watcher() i=2 2024-06-28T15:37:35.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.675+0000 7f127cb72980 20 add_watcher() i=3 2024-06-28T15:37:35.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.675+0000 7f127cb72980 20 add_watcher() i=4 2024-06-28T15:37:35.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.675+0000 7f127cb72980 20 add_watcher() i=5 2024-06-28T15:37:35.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.675+0000 7f127cb72980 20 add_watcher() i=6 2024-06-28T15:37:35.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.675+0000 7f127cb72980 20 add_watcher() i=7 2024-06-28T15:37:35.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.675+0000 7f127cb72980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:35.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f1130fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:35.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:35.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:35.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 5 note: GC not initialized 2024-06-28T15:37:35.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f10ab7fe640 20 reqs_thread_entry: start 2024-06-28T15:37:35.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:35.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 20 Filter name: none 2024-06-28T15:37:35.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f10a97fa640 20 reqs_thread_entry: start 2024-06-28T15:37:35.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 : miss 2024-06-28T15:37:35.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x0 2024-06-28T15:37:35.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 10 adding default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-06-28T15:37:35.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 20 failed to read name.$Mr.RGW88888888888888888 with: (2) No such file or directory 2024-06-28T15:37:35.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 10 cache get: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test : miss 2024-06-28T15:37:35.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 10 cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x0 2024-06-28T15:37:35.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 10 adding default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-06-28T15:37:35.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.679+0000 7f127cb72980 20 failed to read rgw88888888888888888+test@test.test with: (2) No such file or directory 2024-06-28T15:37:35.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.683+0000 7f127cb72980 10 cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-06-28T15:37:35.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.683+0000 7f127cb72980 10 adding default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-06-28T15:37:35.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.683+0000 7f127cb72980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW88888888888888888, ofs0, ns] 2024-06-28T15:37:35.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.683+0000 7f115dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497116 cookie 94609557733280 notifier 7400 bl.length()=384 2024-06-28T15:37:35.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.683+0000 7f115dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-06-28T15:37:35.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.683+0000 7f115dffb640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-06-28T15:37:35.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.687+0000 7f127cb72980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x17 2024-06-28T15:37:35.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.687+0000 7f127cb72980 10 moving default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-06-28T15:37:35.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.687+0000 7f127cb72980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW88888888888888888, ofs0, ns] 2024-06-28T15:37:35.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.687+0000 7f115d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497149 cookie 94609557741088 notifier 7400 bl.length()=225 2024-06-28T15:37:35.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.687+0000 7f115d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x17 2024-06-28T15:37:35.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.687+0000 7f115d7fa640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-06-28T15:37:35.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.691+0000 7f127cb72980 10 cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x17 2024-06-28T15:37:35.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.691+0000 7f127cb72980 10 moving default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-06-28T15:37:35.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.691+0000 7f127cb72980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw88888888888888888+test@test.test, ofs0, ns] 2024-06-28T15:37:35.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.691+0000 7f115d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497076 cookie 94609557737184 notifier 7400 bl.length()=234 2024-06-28T15:37:35.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.691+0000 7f115d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x17 2024-06-28T15:37:35.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.691+0000 7f115d7fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-06-28T15:37:35.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.695+0000 7f1130fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:35.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.699+0000 7f127cb72980 20 remove_watcher() i=0 2024-06-28T15:37:35.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.699+0000 7f127cb72980 2 removed watcher, disabling cache 2024-06-28T15:37:35.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.703+0000 7f127cb72980 20 remove_watcher() i=1 2024-06-28T15:37:35.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.703+0000 7f127cb72980 20 remove_watcher() i=2 2024-06-28T15:37:35.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.707+0000 7f127cb72980 20 remove_watcher() i=3 2024-06-28T15:37:35.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.707+0000 7f127cb72980 20 remove_watcher() i=4 2024-06-28T15:37:35.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.707+0000 7f127cb72980 20 remove_watcher() i=5 2024-06-28T15:37:35.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.711+0000 7f127cb72980 20 remove_watcher() i=6 2024-06-28T15:37:35.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.711+0000 7f127cb72980 20 remove_watcher() i=7 2024-06-28T15:37:35.712 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T15:37:35.712 INFO:teuthology.orchestra.run.smithi178.stdout: "id": "RGW88888888888888888", 2024-06-28T15:37:35.712 INFO:teuthology.orchestra.run.smithi178.stdout: "tenant": "", 2024-06-28T15:37:35.712 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "Mr.RGW88888888888888888", 2024-06-28T15:37:35.712 INFO:teuthology.orchestra.run.smithi178.stdout: "email": "RGW88888888888888888+test@test.test", 2024-06-28T15:37:35.713 INFO:teuthology.orchestra.run.smithi178.stdout: "quota": { 2024-06-28T15:37:35.713 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:35.713 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:35.713 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:35.713 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:35.713 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:35.713 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:35.713 INFO:teuthology.orchestra.run.smithi178.stdout: "bucket_quota": { 2024-06-28T15:37:35.713 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:35.713 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:35.713 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:35.713 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:35.713 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:35.713 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:35.714 INFO:teuthology.orchestra.run.smithi178.stdout: "max_users": 1000, 2024-06-28T15:37:35.714 INFO:teuthology.orchestra.run.smithi178.stdout: "max_roles": 1000, 2024-06-28T15:37:35.714 INFO:teuthology.orchestra.run.smithi178.stdout: "max_groups": 1000, 2024-06-28T15:37:35.714 INFO:teuthology.orchestra.run.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:37:35.714 INFO:teuthology.orchestra.run.smithi178.stdout: "max_access_keys": 4 2024-06-28T15:37:35.714 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T15:37:35.715 DEBUG:tasks.s3tests:Creating user RGW88888888888888888 on client.0 2024-06-28T15:37:35.715 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid root1.client.0 --display-name Mr.RGW88888888888888888 --email root1+test@test.test --access-key YINFFCUZYLTKIDZITFDQ --secret dJqO52E7xF5VFg+EVHb3ddGisO15XLx63NpWHUz5QlpRXZvMrFTZkA== --cluster ceph --account-id RGW88888888888888888 --account-root 2024-06-28T15:37:35.810 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:35.810 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:35.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.863+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.863+0000 7faf1fe5f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.863+0000 7faf1fe5f980 20 realm 2024-06-28T15:37:35.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.863+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.863+0000 7faf1fe5f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.863+0000 7faf1fe5f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:35.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.863+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.863+0000 7faf1fe5f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:35.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.863+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.867+0000 7faf1fe5f980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:35.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.867+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:35.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.867+0000 7faf1fe5f980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:35.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.867+0000 7faf1fe5f980 20 searching for the correct realm 2024-06-28T15:37:35.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:35.871+0000 7faf1e171640 0 -- 172.21.15.178:0/782363690 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x55dced1f04d0 msgr2=0x55dced1f2940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:36.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.079+0000 7faf1d970640 0 -- 172.21.15.178:0/782363690 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7faf14067400 msgr2=0x7faf14065570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:36.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:36.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:36.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:36.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 zone default found 2024-06-28T15:37:36.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 4 Realm: () 2024-06-28T15:37:36.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:36.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:36.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:36.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 zonegroup default 2024-06-28T15:37:36.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.295+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.299+0000 7faf1fe5f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.299+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.299+0000 7faf1fe5f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.299+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.299+0000 7faf1fe5f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.299+0000 7faf1fe5f980 20 started sync module instance, tier type = 2024-06-28T15:37:36.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.299+0000 7faf1fe5f980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:36.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.323+0000 7faf1fe5f980 20 add_watcher() i=0 2024-06-28T15:37:36.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.323+0000 7faf1fe5f980 20 add_watcher() i=1 2024-06-28T15:37:36.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.323+0000 7faf1fe5f980 20 add_watcher() i=2 2024-06-28T15:37:36.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.323+0000 7faf1fe5f980 20 add_watcher() i=3 2024-06-28T15:37:36.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.323+0000 7faf1fe5f980 20 add_watcher() i=4 2024-06-28T15:37:36.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.323+0000 7faf1fe5f980 20 add_watcher() i=5 2024-06-28T15:37:36.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.323+0000 7faf1fe5f980 20 add_watcher() i=6 2024-06-28T15:37:36.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.323+0000 7faf1fe5f980 20 add_watcher() i=7 2024-06-28T15:37:36.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.323+0000 7faf1fe5f980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:36.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.327+0000 7fadcf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:36.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.327+0000 7faf1fe5f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:36.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.327+0000 7faf1fe5f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:36.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 5 note: GC not initialized 2024-06-28T15:37:36.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7fadce7fc640 20 reqs_thread_entry: start 2024-06-28T15:37:36.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:36.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 20 Filter name: none 2024-06-28T15:37:36.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7fad3ffff640 20 reqs_thread_entry: start 2024-06-28T15:37:36.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 10 cache get: name=default.rgw.meta+users.uid+root1.client.0 : miss 2024-06-28T15:37:36.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x0 2024-06-28T15:37:36.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 10 adding default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-06-28T15:37:36.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 10 cache get: name=default.rgw.meta+users.email+root1+test@test.test : miss 2024-06-28T15:37:36.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 10 cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x0 2024-06-28T15:37:36.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 10 adding default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-06-28T15:37:36.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 10 cache get: name=default.rgw.meta+users.keys+YINFFCUZYLTKIDZITFDQ : miss 2024-06-28T15:37:36.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 10 cache put: name=default.rgw.meta+users.keys+YINFFCUZYLTKIDZITFDQ info.flags=0x0 2024-06-28T15:37:36.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 10 adding default.rgw.meta+users.keys+YINFFCUZYLTKIDZITFDQ to cache LRU end 2024-06-28T15:37:36.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 10 cache get: name=default.rgw.meta+accounts+account.RGW88888888888888888 : miss 2024-06-28T15:37:36.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.331+0000 7faf1fe5f980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.335+0000 7faf1fe5f980 20 rados_obj.operate() r=0 bl.length=184 2024-06-28T15:37:36.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.335+0000 7faf1fe5f980 10 cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-06-28T15:37:36.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.335+0000 7faf1fe5f980 10 adding default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-06-28T15:37:36.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.335+0000 7faf1fe5f980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:37:36.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.335+0000 7faf1fe5f980 10 cache get: name=default.rgw.meta+users.keys+YINFFCUZYLTKIDZITFDQ : hit (negative entry) 2024-06-28T15:37:36.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.335+0000 7faf1fe5f980 10 cache get: name=default.rgw.meta+users.keys+YINFFCUZYLTKIDZITFDQ : hit (negative entry) 2024-06-28T15:37:36.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.335+0000 7faf1fe5f980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-06-28T15:37:36.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.335+0000 7faf1fe5f980 10 moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-06-28T15:37:36.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.335+0000 7faf1fe5f980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root1.client.0, ofs0, ns] 2024-06-28T15:37:36.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.339+0000 7fae017fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497117 cookie 94407359452240 notifier 7974 bl.length()=681 2024-06-28T15:37:36.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.339+0000 7fae017fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-06-28T15:37:36.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.339+0000 7fae017fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-06-28T15:37:36.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.343+0000 7faf1fe5f980 10 cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x7 2024-06-28T15:37:36.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.343+0000 7faf1fe5f980 10 moving default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-06-28T15:37:36.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.343+0000 7faf1fe5f980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:root1+test@test.test, ofs0, ns] 2024-06-28T15:37:36.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.343+0000 7fae017fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497101 cookie 94407359462640 notifier 7974 bl.length()=189 2024-06-28T15:37:36.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.343+0000 7fae017fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x7 2024-06-28T15:37:36.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.343+0000 7fae017fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-06-28T15:37:36.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.347+0000 7faf1fe5f980 10 cache put: name=default.rgw.meta+users.keys+YINFFCUZYLTKIDZITFDQ info.flags=0x7 2024-06-28T15:37:36.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.347+0000 7faf1fe5f980 10 moving default.rgw.meta+users.keys+YINFFCUZYLTKIDZITFDQ to cache LRU end 2024-06-28T15:37:36.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.347+0000 7faf1fe5f980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.keys:YINFFCUZYLTKIDZITFDQ, ofs0, ns] 2024-06-28T15:37:36.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.347+0000 7fae017fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497118 cookie 94407359452240 notifier 7974 bl.length()=188 2024-06-28T15:37:36.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.347+0000 7fae017fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+YINFFCUZYLTKIDZITFDQ info.flags=0x7 2024-06-28T15:37:36.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.347+0000 7fae017fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+YINFFCUZYLTKIDZITFDQ to cache LRU end 2024-06-28T15:37:36.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.351+0000 7faf1fe5f980 20 linked user root1.client.0 to account RGW88888888888888888 2024-06-28T15:37:36.341 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T15:37:36.341 INFO:teuthology.orchestra.run.smithi178.stdout: "user_id": "root1.client.0", 2024-06-28T15:37:36.341 INFO:teuthology.orchestra.run.smithi178.stdout: "display_name": "Mr.RGW88888888888888888", 2024-06-28T15:37:36.341 INFO:teuthology.orchestra.run.smithi178.stdout: "email": "root1+test@test.test", 2024-06-28T15:37:36.341 INFO:teuthology.orchestra.run.smithi178.stdout: "suspended": 0, 2024-06-28T15:37:36.341 INFO:teuthology.orchestra.run.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:37:36.341 INFO:teuthology.orchestra.run.smithi178.stdout: "subusers": [], 2024-06-28T15:37:36.341 INFO:teuthology.orchestra.run.smithi178.stdout: "keys": [ 2024-06-28T15:37:36.341 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T15:37:36.341 INFO:teuthology.orchestra.run.smithi178.stdout: "user": "root1.client.0", 2024-06-28T15:37:36.341 INFO:teuthology.orchestra.run.smithi178.stdout: "access_key": "YINFFCUZYLTKIDZITFDQ", 2024-06-28T15:37:36.341 INFO:teuthology.orchestra.run.smithi178.stdout: "secret_key": "dJqO52E7xF5VFg+EVHb3ddGisO15XLx63NpWHUz5QlpRXZvMrFTZkA==", 2024-06-28T15:37:36.342 INFO:teuthology.orchestra.run.smithi178.stdout: "active": true, 2024-06-28T15:37:36.342 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:37:36.337368Z" 2024-06-28T15:37:36.342 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T15:37:36.342 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-06-28T15:37:36.342 INFO:teuthology.orchestra.run.smithi178.stdout: "swift_keys": [], 2024-06-28T15:37:36.342 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": [], 2024-06-28T15:37:36.342 INFO:teuthology.orchestra.run.smithi178.stdout: "op_mask": "read, write, delete", 2024-06-28T15:37:36.342 INFO:teuthology.orchestra.run.smithi178.stdout: "default_placement": "", 2024-06-28T15:37:36.342 INFO:teuthology.orchestra.run.smithi178.stdout: "default_storage_class": "", 2024-06-28T15:37:36.342 INFO:teuthology.orchestra.run.smithi178.stdout: "placement_tags": [], 2024-06-28T15:37:36.342 INFO:teuthology.orchestra.run.smithi178.stdout: "bucket_quota": { 2024-06-28T15:37:36.342 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:36.342 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:36.342 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: "user_quota": { 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: "temp_url_keys": [], 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "root", 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: "mfa_ids": [], 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: "account_id": "RGW88888888888888888", 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2024-06-28T15:37:36.343 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:37:36.337338Z", 2024-06-28T15:37:36.344 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2024-06-28T15:37:36.344 INFO:teuthology.orchestra.run.smithi178.stdout: "group_ids": [] 2024-06-28T15:37:36.344 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T15:37:36.344 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:36.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.351+0000 7fadcf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:36.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.359+0000 7faf1fe5f980 20 remove_watcher() i=0 2024-06-28T15:37:36.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.359+0000 7faf1fe5f980 2 removed watcher, disabling cache 2024-06-28T15:37:36.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.359+0000 7faf1fe5f980 20 remove_watcher() i=1 2024-06-28T15:37:36.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.363+0000 7faf1fe5f980 20 remove_watcher() i=2 2024-06-28T15:37:36.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.363+0000 7faf1fe5f980 20 remove_watcher() i=3 2024-06-28T15:37:36.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.367+0000 7faf1fe5f980 20 remove_watcher() i=4 2024-06-28T15:37:36.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.367+0000 7faf1fe5f980 20 remove_watcher() i=5 2024-06-28T15:37:36.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.371+0000 7faf1fe5f980 20 remove_watcher() i=6 2024-06-28T15:37:36.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.371+0000 7faf1fe5f980 20 remove_watcher() i=7 2024-06-28T15:37:36.378 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid root1.client.0 --totp-serial 0368966611 --totp-seed ORDOTC6HTIZGRM55SOGW6NQ6AJENJXFIHKL2BP3A4S2VPEMM32V5RI2G23SRHRZO --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-06-28T15:37:36.424 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:36.424 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:36.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.475+0000 7f5857184980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.479+0000 7f5857184980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.479+0000 7f5857184980 20 realm 2024-06-28T15:37:36.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.479+0000 7f5857184980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.479+0000 7f5857184980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.479+0000 7f5857184980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:36.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.479+0000 7f5857184980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.479+0000 7f5857184980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.479+0000 7f5857184980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.479+0000 7f5857184980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:36.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.479+0000 7f5857184980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.479+0000 7f5857184980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:36.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.479+0000 7f5857184980 20 searching for the correct realm 2024-06-28T15:37:36.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.487+0000 7f5854494640 0 -- 172.21.15.178:0/1105488685 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x55bf584d3360 msgr2=0x55bf584d5750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:36.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.491+0000 7f5854c95640 0 -- 172.21.15.178:0/1105488685 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f5848060230 msgr2=0x7f5848063350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T15:37:36.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.499+0000 7f5857184980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:36.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.499+0000 7f5857184980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:36.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.499+0000 7f5857184980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:36.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.499+0000 7f5857184980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:36.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.499+0000 7f5857184980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:36.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.499+0000 7f5857184980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:36.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.499+0000 7f5857184980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.499+0000 7f5857184980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.499+0000 7f5857184980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.499+0000 7f5857184980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:36.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.499+0000 7f5857184980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:36.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 20 zone default found 2024-06-28T15:37:36.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 4 Realm: () 2024-06-28T15:37:36.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:36.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:36.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:36.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 20 zonegroup default 2024-06-28T15:37:36.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 20 started sync module instance, tier type = 2024-06-28T15:37:36.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.503+0000 7f5857184980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:36.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.519+0000 7f5857184980 20 add_watcher() i=0 2024-06-28T15:37:36.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.519+0000 7f5857184980 20 add_watcher() i=1 2024-06-28T15:37:36.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.519+0000 7f5857184980 20 add_watcher() i=2 2024-06-28T15:37:36.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.519+0000 7f5857184980 20 add_watcher() i=3 2024-06-28T15:37:36.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.519+0000 7f5857184980 20 add_watcher() i=4 2024-06-28T15:37:36.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.519+0000 7f5857184980 20 add_watcher() i=5 2024-06-28T15:37:36.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.519+0000 7f5857184980 20 add_watcher() i=6 2024-06-28T15:37:36.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.519+0000 7f5857184980 20 add_watcher() i=7 2024-06-28T15:37:36.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.519+0000 7f5857184980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:36.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.523+0000 7f570afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:36.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.523+0000 7f5857184980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:36.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.523+0000 7f5857184980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:36.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.523+0000 7f5857184980 5 note: GC not initialized 2024-06-28T15:37:36.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.523+0000 7f5709fe3640 20 reqs_thread_entry: start 2024-06-28T15:37:36.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.523+0000 7f5857184980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:36.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.523+0000 7f5857184980 20 Filter name: none 2024-06-28T15:37:36.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.523+0000 7f567b7fe640 20 reqs_thread_entry: start 2024-06-28T15:37:36.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.523+0000 7f5857184980 10 cache get: name=default.rgw.meta+users.uid+root1.client.0 : miss 2024-06-28T15:37:36.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.523+0000 7f5857184980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.523+0000 7f5857184980 20 rados_obj.operate() r=0 bl.length=494 2024-06-28T15:37:36.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.523+0000 7f5857184980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x13 2024-06-28T15:37:36.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.523+0000 7f5857184980 10 adding default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-06-28T15:37:36.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.523+0000 7f5857184980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:37:36.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.527+0000 7f5857184980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-06-28T15:37:36.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.527+0000 7f5857184980 10 moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-06-28T15:37:36.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.527+0000 7f5857184980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root1.client.0, ofs0, ns] 2024-06-28T15:37:36.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.527+0000 7f572bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497119 cookie 94280308588448 notifier 7995 bl.length()=695 2024-06-28T15:37:36.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.527+0000 7f572bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-06-28T15:37:36.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.527+0000 7f572bfff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-06-28T15:37:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.531+0000 7f570afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:36.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.535+0000 7f5857184980 20 remove_watcher() i=0 2024-06-28T15:37:36.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.535+0000 7f5857184980 2 removed watcher, disabling cache 2024-06-28T15:37:36.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.539+0000 7f5857184980 20 remove_watcher() i=1 2024-06-28T15:37:36.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.539+0000 7f5857184980 20 remove_watcher() i=2 2024-06-28T15:37:36.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.543+0000 7f5857184980 20 remove_watcher() i=3 2024-06-28T15:37:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.543+0000 7f5857184980 20 remove_watcher() i=4 2024-06-28T15:37:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.547+0000 7f5857184980 20 remove_watcher() i=5 2024-06-28T15:37:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.547+0000 7f5857184980 20 remove_watcher() i=6 2024-06-28T15:37:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.547+0000 7f5857184980 20 remove_watcher() i=7 2024-06-28T15:37:36.552 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW99999999999999999 --account-name Mr.RGW99999999999999999 --email RGW99999999999999999+test@test.test 2024-06-28T15:37:36.586 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:36.586 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:36.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.635+0000 7fa3d43c7980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.639+0000 7fa3d43c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.639+0000 7fa3d43c7980 20 realm 2024-06-28T15:37:36.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.639+0000 7fa3d43c7980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.639+0000 7fa3d43c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.639+0000 7fa3d43c7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:36.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.639+0000 7fa3d43c7980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.639+0000 7fa3d43c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.639+0000 7fa3d43c7980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.639+0000 7fa3d43c7980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:36.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.639+0000 7fa3d43c7980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.643+0000 7fa3d43c7980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:36.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.643+0000 7fa3d43c7980 20 searching for the correct realm 2024-06-28T15:37:36.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.647+0000 7fa3d16d7640 0 -- 172.21.15.178:0/3817682041 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x556aac1bb690 msgr2=0x556aac1bda80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:36.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.651+0000 7fa3d1ed8640 0 -- 172.21.15.178:0/3817682041 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7fa3c812bb10 msgr2=0x7fa3c8118210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:36.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.659+0000 7fa3d43c7980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:36.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.659+0000 7fa3d43c7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:36.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.659+0000 7fa3d43c7980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:36.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.659+0000 7fa3d43c7980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:36.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.659+0000 7fa3d43c7980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:36.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.659+0000 7fa3d43c7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:36.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.659+0000 7fa3d43c7980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.659+0000 7fa3d43c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.659+0000 7fa3d43c7980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.659+0000 7fa3d43c7980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:36.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.659+0000 7fa3d43c7980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:36.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 20 zone default found 2024-06-28T15:37:36.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 4 Realm: () 2024-06-28T15:37:36.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:36.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:36.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:36.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 20 zonegroup default 2024-06-28T15:37:36.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 20 started sync module instance, tier type = 2024-06-28T15:37:36.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.663+0000 7fa3d43c7980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:36.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.679+0000 7fa3d43c7980 20 add_watcher() i=0 2024-06-28T15:37:36.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.679+0000 7fa3d43c7980 20 add_watcher() i=1 2024-06-28T15:37:36.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.679+0000 7fa3d43c7980 20 add_watcher() i=2 2024-06-28T15:37:36.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.679+0000 7fa3d43c7980 20 add_watcher() i=3 2024-06-28T15:37:36.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.679+0000 7fa3d43c7980 20 add_watcher() i=4 2024-06-28T15:37:36.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.679+0000 7fa3d43c7980 20 add_watcher() i=5 2024-06-28T15:37:36.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.679+0000 7fa3d43c7980 20 add_watcher() i=6 2024-06-28T15:37:36.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.679+0000 7fa3d43c7980 20 add_watcher() i=7 2024-06-28T15:37:36.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.679+0000 7fa3d43c7980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:36.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa2837fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:36.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:36.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:36.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 5 note: GC not initialized 2024-06-28T15:37:36.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa2827fc640 20 reqs_thread_entry: start 2024-06-28T15:37:36.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:36.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 20 Filter name: none 2024-06-28T15:37:36.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa1f3fff640 20 reqs_thread_entry: start 2024-06-28T15:37:36.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 : miss 2024-06-28T15:37:36.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x0 2024-06-28T15:37:36.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 10 adding default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-06-28T15:37:36.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 20 failed to read name.$Mr.RGW99999999999999999 with: (2) No such file or directory 2024-06-28T15:37:36.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 10 cache get: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test : miss 2024-06-28T15:37:36.674 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 10 cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x0 2024-06-28T15:37:36.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 10 adding default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-06-28T15:37:36.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.683+0000 7fa3d43c7980 20 failed to read rgw99999999999999999+test@test.test with: (2) No such file or directory 2024-06-28T15:37:36.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.687+0000 7fa3d43c7980 10 cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-06-28T15:37:36.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.687+0000 7fa3d43c7980 10 adding default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-06-28T15:37:36.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.687+0000 7fa3d43c7980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW99999999999999999, ofs0, ns] 2024-06-28T15:37:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.691+0000 7fa2b57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497120 cookie 93916642414048 notifier 7445 bl.length()=384 2024-06-28T15:37:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.691+0000 7fa2b57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-06-28T15:37:36.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.691+0000 7fa2b57fa640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-06-28T15:37:36.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.691+0000 7fa3d43c7980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x17 2024-06-28T15:37:36.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.691+0000 7fa3d43c7980 10 moving default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-06-28T15:37:36.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.691+0000 7fa3d43c7980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW99999999999999999, ofs0, ns] 2024-06-28T15:37:36.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.695+0000 7fa2b57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497150 cookie 93916642421472 notifier 7445 bl.length()=225 2024-06-28T15:37:36.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.695+0000 7fa2b57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x17 2024-06-28T15:37:36.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.695+0000 7fa2b57fa640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-06-28T15:37:36.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.699+0000 7fa3d43c7980 10 cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x17 2024-06-28T15:37:36.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.699+0000 7fa3d43c7980 10 moving default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-06-28T15:37:36.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.699+0000 7fa3d43c7980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw99999999999999999+test@test.test, ofs0, ns] 2024-06-28T15:37:36.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.699+0000 7fa2b57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497077 cookie 93916642420416 notifier 7445 bl.length()=234 2024-06-28T15:37:36.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.699+0000 7fa2b57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x17 2024-06-28T15:37:36.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.699+0000 7fa2b57fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-06-28T15:37:36.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.703+0000 7fa2837fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.707+0000 7fa3d43c7980 20 remove_watcher() i=0 2024-06-28T15:37:36.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.707+0000 7fa3d43c7980 2 removed watcher, disabling cache 2024-06-28T15:37:36.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.711+0000 7fa3d43c7980 20 remove_watcher() i=1 2024-06-28T15:37:36.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.711+0000 7fa3d43c7980 20 remove_watcher() i=2 2024-06-28T15:37:36.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.715+0000 7fa3d43c7980 20 remove_watcher() i=3 2024-06-28T15:37:36.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.715+0000 7fa3d43c7980 20 remove_watcher() i=4 2024-06-28T15:37:36.706 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.715+0000 7fa3d43c7980 20 remove_watcher() i=5 2024-06-28T15:37:36.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.719+0000 7fa3d43c7980 20 remove_watcher() i=6 2024-06-28T15:37:36.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.719+0000 7fa3d43c7980 20 remove_watcher() i=7 2024-06-28T15:37:36.720 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T15:37:36.721 INFO:teuthology.orchestra.run.smithi178.stdout: "id": "RGW99999999999999999", 2024-06-28T15:37:36.721 INFO:teuthology.orchestra.run.smithi178.stdout: "tenant": "", 2024-06-28T15:37:36.721 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "Mr.RGW99999999999999999", 2024-06-28T15:37:36.721 INFO:teuthology.orchestra.run.smithi178.stdout: "email": "RGW99999999999999999+test@test.test", 2024-06-28T15:37:36.721 INFO:teuthology.orchestra.run.smithi178.stdout: "quota": { 2024-06-28T15:37:36.721 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:36.721 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:36.721 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:36.721 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:36.721 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:36.721 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:36.721 INFO:teuthology.orchestra.run.smithi178.stdout: "bucket_quota": { 2024-06-28T15:37:36.722 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:36.722 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:36.722 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:36.722 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:36.722 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:36.722 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:36.722 INFO:teuthology.orchestra.run.smithi178.stdout: "max_users": 1000, 2024-06-28T15:37:36.722 INFO:teuthology.orchestra.run.smithi178.stdout: "max_roles": 1000, 2024-06-28T15:37:36.722 INFO:teuthology.orchestra.run.smithi178.stdout: "max_groups": 1000, 2024-06-28T15:37:36.722 INFO:teuthology.orchestra.run.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:37:36.722 INFO:teuthology.orchestra.run.smithi178.stdout: "max_access_keys": 4 2024-06-28T15:37:36.722 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T15:37:36.723 DEBUG:tasks.s3tests:Creating user RGW99999999999999999 on client.0 2024-06-28T15:37:36.724 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid root2.client.0 --display-name Mr.RGW99999999999999999 --email root2+test@test.test --access-key GOMLQAYSDHGPJAVKRXIR --secret 5NB9fm7YVLGi4/dEdGHIX8j1kY4OcYejBoaW2zhpp64nBzgcpq7j8A== --cluster ceph --account-id RGW99999999999999999 --account-root 2024-06-28T15:37:36.809 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:36.809 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:36.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.859+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.863+0000 7fb39d855980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.863+0000 7fb39d855980 20 realm 2024-06-28T15:37:36.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.863+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.863+0000 7fb39d855980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.863+0000 7fb39d855980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:36.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.863+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.863+0000 7fb39d855980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.863+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.863+0000 7fb39d855980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:36.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.863+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.863+0000 7fb39d855980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:36.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.863+0000 7fb39d855980 20 searching for the correct realm 2024-06-28T15:37:36.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.871+0000 7fb39bb67640 0 -- 172.21.15.178:0/1240080881 >> [v2:172.21.15.178:6810/1880801124,v1:172.21.15.178:6812/1880801124] conn(0x5576b44f8230 msgr2=0x5576b44fa6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:36.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.875+0000 7fb39ab65640 0 -- 172.21.15.178:0/1240080881 >> [v2:172.21.15.195:6816/2928422746,v1:172.21.15.195:6817/2928422746] conn(0x7fb39401d060 msgr2=0x7fb394061840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:36.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:36.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:36.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:36.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:36.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:36.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:36.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 zone default found 2024-06-28T15:37:36.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 4 Realm: () 2024-06-28T15:37:36.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:36.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:36.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:36.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 zonegroup default 2024-06-28T15:37:36.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.883+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.887+0000 7fb39d855980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.887+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.887+0000 7fb39d855980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.887+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.887+0000 7fb39d855980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.887+0000 7fb39d855980 20 started sync module instance, tier type = 2024-06-28T15:37:36.876 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.887+0000 7fb39d855980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:36.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 20 add_watcher() i=0 2024-06-28T15:37:36.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 20 add_watcher() i=1 2024-06-28T15:37:36.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 20 add_watcher() i=2 2024-06-28T15:37:36.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 20 add_watcher() i=3 2024-06-28T15:37:36.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 20 add_watcher() i=4 2024-06-28T15:37:36.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 20 add_watcher() i=5 2024-06-28T15:37:36.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 20 add_watcher() i=6 2024-06-28T15:37:36.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 20 add_watcher() i=7 2024-06-28T15:37:36.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:36.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb251fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:36.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:36.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:36.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 5 note: GC not initialized 2024-06-28T15:37:36.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb250fe1640 20 reqs_thread_entry: start 2024-06-28T15:37:36.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:36.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 20 Filter name: none 2024-06-28T15:37:36.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb1ca7fc640 20 reqs_thread_entry: start 2024-06-28T15:37:36.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 10 cache get: name=default.rgw.meta+users.uid+root2.client.0 : miss 2024-06-28T15:37:36.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.903+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x0 2024-06-28T15:37:36.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 10 adding default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-06-28T15:37:36.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 10 cache get: name=default.rgw.meta+users.email+root2+test@test.test : miss 2024-06-28T15:37:36.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 10 cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x0 2024-06-28T15:37:36.896 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 10 adding default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-06-28T15:37:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 10 cache get: name=default.rgw.meta+users.keys+GOMLQAYSDHGPJAVKRXIR : miss 2024-06-28T15:37:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 10 cache put: name=default.rgw.meta+users.keys+GOMLQAYSDHGPJAVKRXIR info.flags=0x0 2024-06-28T15:37:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 10 adding default.rgw.meta+users.keys+GOMLQAYSDHGPJAVKRXIR to cache LRU end 2024-06-28T15:37:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 10 cache get: name=default.rgw.meta+accounts+account.RGW99999999999999999 : miss 2024-06-28T15:37:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 20 rados->read ofs=0 len=0 2024-06-28T15:37:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 20 rados_obj.operate() r=0 bl.length=184 2024-06-28T15:37:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 10 cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-06-28T15:37:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 10 adding default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-06-28T15:37:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:37:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 10 cache get: name=default.rgw.meta+users.keys+GOMLQAYSDHGPJAVKRXIR : hit (negative entry) 2024-06-28T15:37:36.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.907+0000 7fb39d855980 10 cache get: name=default.rgw.meta+users.keys+GOMLQAYSDHGPJAVKRXIR : hit (negative entry) 2024-06-28T15:37:36.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.911+0000 7fb39d855980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-06-28T15:37:36.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.911+0000 7fb39d855980 10 moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-06-28T15:37:36.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.911+0000 7fb39d855980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root2.client.0, ofs0, ns] 2024-06-28T15:37:36.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.911+0000 7fb27effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497121 cookie 93968319665712 notifier 7463 bl.length()=681 2024-06-28T15:37:36.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.911+0000 7fb27effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-06-28T15:37:36.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.911+0000 7fb27effd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-06-28T15:37:36.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.915+0000 7fb39d855980 10 cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x7 2024-06-28T15:37:36.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.915+0000 7fb39d855980 10 moving default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-06-28T15:37:36.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.915+0000 7fb39d855980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:root2+test@test.test, ofs0, ns] 2024-06-28T15:37:36.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.915+0000 7fb27effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497102 cookie 93968319676928 notifier 7463 bl.length()=189 2024-06-28T15:37:36.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.915+0000 7fb27effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x7 2024-06-28T15:37:36.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.915+0000 7fb27effd640 10 rgw watcher librados: moving default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-06-28T15:37:36.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.919+0000 7fb39d855980 10 cache put: name=default.rgw.meta+users.keys+GOMLQAYSDHGPJAVKRXIR info.flags=0x7 2024-06-28T15:37:36.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.919+0000 7fb39d855980 10 moving default.rgw.meta+users.keys+GOMLQAYSDHGPJAVKRXIR to cache LRU end 2024-06-28T15:37:36.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.919+0000 7fb39d855980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:GOMLQAYSDHGPJAVKRXIR, ofs0, ns] 2024-06-28T15:37:36.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.919+0000 7fb27e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497103 cookie 93968319676928 notifier 7463 bl.length()=188 2024-06-28T15:37:36.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.919+0000 7fb27e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+GOMLQAYSDHGPJAVKRXIR info.flags=0x7 2024-06-28T15:37:36.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.919+0000 7fb27e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+GOMLQAYSDHGPJAVKRXIR to cache LRU end 2024-06-28T15:37:36.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.923+0000 7fb39d855980 20 linked user root2.client.0 to account RGW99999999999999999 2024-06-28T15:37:36.911 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T15:37:36.912 INFO:teuthology.orchestra.run.smithi178.stdout: "user_id": "root2.client.0", 2024-06-28T15:37:36.912 INFO:teuthology.orchestra.run.smithi178.stdout: "display_name": "Mr.RGW99999999999999999", 2024-06-28T15:37:36.912 INFO:teuthology.orchestra.run.smithi178.stdout: "email": "root2+test@test.test", 2024-06-28T15:37:36.912 INFO:teuthology.orchestra.run.smithi178.stdout: "suspended": 0, 2024-06-28T15:37:36.912 INFO:teuthology.orchestra.run.smithi178.stdout: "max_buckets": 1000, 2024-06-28T15:37:36.912 INFO:teuthology.orchestra.run.smithi178.stdout: "subusers": [], 2024-06-28T15:37:36.912 INFO:teuthology.orchestra.run.smithi178.stdout: "keys": [ 2024-06-28T15:37:36.912 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T15:37:36.912 INFO:teuthology.orchestra.run.smithi178.stdout: "user": "root2.client.0", 2024-06-28T15:37:36.912 INFO:teuthology.orchestra.run.smithi178.stdout: "access_key": "GOMLQAYSDHGPJAVKRXIR", 2024-06-28T15:37:36.912 INFO:teuthology.orchestra.run.smithi178.stdout: "secret_key": "5NB9fm7YVLGi4/dEdGHIX8j1kY4OcYejBoaW2zhpp64nBzgcpq7j8A==", 2024-06-28T15:37:36.912 INFO:teuthology.orchestra.run.smithi178.stdout: "active": true, 2024-06-28T15:37:36.912 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:37:36.911059Z" 2024-06-28T15:37:36.912 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T15:37:36.913 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-06-28T15:37:36.913 INFO:teuthology.orchestra.run.smithi178.stdout: "swift_keys": [], 2024-06-28T15:37:36.913 INFO:teuthology.orchestra.run.smithi178.stdout: "caps": [], 2024-06-28T15:37:36.913 INFO:teuthology.orchestra.run.smithi178.stdout: "op_mask": "read, write, delete", 2024-06-28T15:37:36.913 INFO:teuthology.orchestra.run.smithi178.stdout: "default_placement": "", 2024-06-28T15:37:36.913 INFO:teuthology.orchestra.run.smithi178.stdout: "default_storage_class": "", 2024-06-28T15:37:36.913 INFO:teuthology.orchestra.run.smithi178.stdout: "placement_tags": [], 2024-06-28T15:37:36.913 INFO:teuthology.orchestra.run.smithi178.stdout: "bucket_quota": { 2024-06-28T15:37:36.913 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:36.913 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:36.913 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:36.913 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:36.913 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:36.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:36.914 INFO:teuthology.orchestra.run.smithi178.stdout: "user_quota": { 2024-06-28T15:37:36.914 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": false, 2024-06-28T15:37:36.914 INFO:teuthology.orchestra.run.smithi178.stdout: "check_on_raw": false, 2024-06-28T15:37:36.914 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size": -1, 2024-06-28T15:37:36.914 INFO:teuthology.orchestra.run.smithi178.stdout: "max_size_kb": 0, 2024-06-28T15:37:36.914 INFO:teuthology.orchestra.run.smithi178.stdout: "max_objects": -1 2024-06-28T15:37:36.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T15:37:36.914 INFO:teuthology.orchestra.run.smithi178.stdout: "temp_url_keys": [], 2024-06-28T15:37:36.914 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "root", 2024-06-28T15:37:36.914 INFO:teuthology.orchestra.run.smithi178.stdout: "mfa_ids": [], 2024-06-28T15:37:36.914 INFO:teuthology.orchestra.run.smithi178.stdout: "account_id": "RGW99999999999999999", 2024-06-28T15:37:36.914 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2024-06-28T15:37:36.915 INFO:teuthology.orchestra.run.smithi178.stdout: "create_date": "2024-06-28T15:37:36.911042Z", 2024-06-28T15:37:36.915 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2024-06-28T15:37:36.915 INFO:teuthology.orchestra.run.smithi178.stdout: "group_ids": [] 2024-06-28T15:37:36.915 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T15:37:36.915 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:37:36.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.923+0000 7fb251fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:36.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.927+0000 7fb39d855980 20 remove_watcher() i=0 2024-06-28T15:37:36.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.927+0000 7fb39d855980 2 removed watcher, disabling cache 2024-06-28T15:37:36.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.931+0000 7fb39d855980 20 remove_watcher() i=1 2024-06-28T15:37:36.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.931+0000 7fb39d855980 20 remove_watcher() i=2 2024-06-28T15:37:36.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.935+0000 7fb39d855980 20 remove_watcher() i=3 2024-06-28T15:37:36.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.935+0000 7fb39d855980 20 remove_watcher() i=4 2024-06-28T15:37:36.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.939+0000 7fb39d855980 20 remove_watcher() i=5 2024-06-28T15:37:36.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.939+0000 7fb39d855980 20 remove_watcher() i=6 2024-06-28T15:37:36.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:36.939+0000 7fb39d855980 20 remove_watcher() i=7 2024-06-28T15:37:36.940 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid root2.client.0 --totp-serial 7225967354 --totp-seed 3FZV7QB4CJUAB4FMQPMMGBV77PXRZCII6BR5KLDE6GHRF5G2DHWNT7N4AY5WM5SV --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-06-28T15:37:37.021 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T15:37:37.021 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T15:37:37.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.067+0000 7f9edc4ad980 20 rados->read ofs=0 len=0 2024-06-28T15:37:37.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.071+0000 7f9edc4ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:37.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.071+0000 7f9edc4ad980 20 realm 2024-06-28T15:37:37.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.071+0000 7f9edc4ad980 20 rados->read ofs=0 len=0 2024-06-28T15:37:37.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.071+0000 7f9edc4ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:37.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.071+0000 7f9edc4ad980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T15:37:37.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.071+0000 7f9edc4ad980 20 rados->read ofs=0 len=0 2024-06-28T15:37:37.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.071+0000 7f9edc4ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:37.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.071+0000 7f9edc4ad980 20 rados->read ofs=0 len=0 2024-06-28T15:37:37.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.071+0000 7f9edc4ad980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:37.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.071+0000 7f9edc4ad980 20 rados->read ofs=0 len=0 2024-06-28T15:37:37.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.071+0000 7f9edc4ad980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T15:37:37.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.071+0000 7f9edc4ad980 20 searching for the correct realm 2024-06-28T15:37:37.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.079+0000 7f9ed97bd640 0 -- 172.21.15.178:0/3841467007 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x556ae5f94490 msgr2=0x556ae5f968a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:37.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.083+0000 7f9ed9fbe640 0 -- 172.21.15.178:0/3841467007 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f9ecc061e00 msgr2=0x7f9ecc095850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T15:37:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T15:37:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T15:37:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T15:37:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T15:37:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T15:37:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T15:37:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 rados->read ofs=0 len=0 2024-06-28T15:37:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:37.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 rados->read ofs=0 len=0 2024-06-28T15:37:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T15:37:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 rados->read ofs=0 len=0 2024-06-28T15:37:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T15:37:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 zone default found 2024-06-28T15:37:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 4 Realm: () 2024-06-28T15:37:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T15:37:37.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T15:37:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T15:37:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 zonegroup default 2024-06-28T15:37:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 rados->read ofs=0 len=0 2024-06-28T15:37:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 rados->read ofs=0 len=0 2024-06-28T15:37:37.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 rados->read ofs=0 len=0 2024-06-28T15:37:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T15:37:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 started sync module instance, tier type = 2024-06-28T15:37:37.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.091+0000 7f9edc4ad980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T15:37:37.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.107+0000 7f9edc4ad980 20 add_watcher() i=0 2024-06-28T15:37:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.107+0000 7f9edc4ad980 20 add_watcher() i=1 2024-06-28T15:37:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.107+0000 7f9edc4ad980 20 add_watcher() i=2 2024-06-28T15:37:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.107+0000 7f9edc4ad980 20 add_watcher() i=3 2024-06-28T15:37:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.107+0000 7f9edc4ad980 20 add_watcher() i=4 2024-06-28T15:37:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.107+0000 7f9edc4ad980 20 add_watcher() i=5 2024-06-28T15:37:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.107+0000 7f9edc4ad980 20 add_watcher() i=6 2024-06-28T15:37:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.107+0000 7f9edc4ad980 20 add_watcher() i=7 2024-06-28T15:37:37.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.107+0000 7f9edc4ad980 2 all 8 watchers are set, enabling cache 2024-06-28T15:37:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.111+0000 7f9d8b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.111+0000 7f9edc4ad980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T15:37:37.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.111+0000 7f9edc4ad980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T15:37:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.111+0000 7f9edc4ad980 5 note: GC not initialized 2024-06-28T15:37:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.111+0000 7f9d8a7fc640 20 reqs_thread_entry: start 2024-06-28T15:37:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.111+0000 7f9edc4ad980 20 init_complete bucket index max shards: 1 2024-06-28T15:37:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.111+0000 7f9edc4ad980 20 Filter name: none 2024-06-28T15:37:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.111+0000 7f9cfbfff640 20 reqs_thread_entry: start 2024-06-28T15:37:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.111+0000 7f9edc4ad980 10 cache get: name=default.rgw.meta+users.uid+root2.client.0 : miss 2024-06-28T15:37:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.111+0000 7f9edc4ad980 20 rados->read ofs=0 len=0 2024-06-28T15:37:37.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.111+0000 7f9edc4ad980 20 rados_obj.operate() r=0 bl.length=494 2024-06-28T15:37:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.111+0000 7f9edc4ad980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x13 2024-06-28T15:37:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.111+0000 7f9edc4ad980 10 adding default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-06-28T15:37:37.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.111+0000 7f9edc4ad980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T15:37:37.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.115+0000 7f9edc4ad980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-06-28T15:37:37.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.115+0000 7f9edc4ad980 10 moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-06-28T15:37:37.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.115+0000 7f9edc4ad980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root2.client.0, ofs0, ns] 2024-06-28T15:37:37.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.115+0000 7f9dbd7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497122 cookie 93917613234032 notifier 8049 bl.length()=695 2024-06-28T15:37:37.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.115+0000 7f9dbd7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-06-28T15:37:37.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.115+0000 7f9dbd7fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-06-28T15:37:37.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.119+0000 7f9d8b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T15:37:37.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.123+0000 7f9edc4ad980 20 remove_watcher() i=0 2024-06-28T15:37:37.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.123+0000 7f9edc4ad980 2 removed watcher, disabling cache 2024-06-28T15:37:37.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.127+0000 7f9edc4ad980 20 remove_watcher() i=1 2024-06-28T15:37:37.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.127+0000 7f9edc4ad980 20 remove_watcher() i=2 2024-06-28T15:37:37.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.131+0000 7f9edc4ad980 20 remove_watcher() i=3 2024-06-28T15:37:37.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.131+0000 7f9edc4ad980 20 remove_watcher() i=4 2024-06-28T15:37:37.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.135+0000 7f9edc4ad980 20 remove_watcher() i=5 2024-06-28T15:37:37.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.135+0000 7f9edc4ad980 20 remove_watcher() i=6 2024-06-28T15:37:37.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T15:37:37.135+0000 7f9edc4ad980 20 remove_watcher() i=7 2024-06-28T15:37:37.136 INFO:tasks.s3tests:Configuring s3-tests... 2024-06-28T15:37:37.137 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T15:37:37.137 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2024-06-28T15:37:37.182 INFO:tasks.s3tests:Configuring boto... 2024-06-28T15:37:37.183 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-28T15:37:37.183 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/boto-client.0.cfg 2024-06-28T15:37:37.230 DEBUG:teuthology.orchestra.run.smithi178:s3 tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' 2024-06-28T15:37:37.402 INFO:teuthology.orchestra.run.smithi178.stdout:GLOB sdist-make: /home/ubuntu/cephtest/s3-tests-client.0/setup.py 2024-06-28T15:37:37.598 INFO:teuthology.orchestra.run.smithi178.stdout:py create: /home/ubuntu/cephtest/s3-tests-client.0/.tox/py 2024-06-28T15:37:37.878 INFO:teuthology.orchestra.run.smithi178.stdout:py installdeps: -rrequirements.txt 2024-06-28T15:39:45.803 INFO:teuthology.orchestra.run.smithi178.stdout:py inst: /home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.zip 2024-06-28T15:39:47.532 INFO:teuthology.orchestra.run.smithi178.stdout:py installed: boto==2.49.0,boto3==1.24.96,botocore==1.27.96,cachetools==5.3.3,certifi==2024.6.2,chardet==5.2.0,charset-normalizer==3.3.2,colorama==0.4.6,distlib==0.3.8,exceptiongroup==1.2.1,filelock==3.15.4,gevent==24.2.1,greenlet==3.0.3,httplib2==0.22.0,idna==3.7,iniconfig==2.0.0,isodate==0.6.1,jmespath==1.0.1,lxml==5.2.2,munch==4.0.0,packaging==24.1,platformdirs==4.2.2,pluggy==1.5.0,pyparsing==3.1.2,pyproject-api==1.7.1,pytest==8.2.2,python-dateutil==2.9.0.post0,pytz==2024.1,PyYAML==6.0.1,requests==2.32.3,s3tests @ file:///home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.zip#sha256=20f84905ddd0044c13ce8458a4c38e7a901b7831b6c4437fba0f29af8c854827,s3transfer==0.6.2,six==1.16.0,tomli==2.0.1,tox==4.15.1,urllib3==1.26.19,virtualenv==20.26.3,zope.event==5.0,zope.interface==6.4.post2 2024-06-28T15:39:47.533 INFO:teuthology.orchestra.run.smithi178.stdout:py run-test-pre: PYTHONHASHSEED='2925931158' 2024-06-28T15:39:47.533 INFO:teuthology.orchestra.run.smithi178.stdout:py run-test: commands[0] | pytest -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' 2024-06-28T15:39:47.725 INFO:teuthology.orchestra.run.smithi178.stdout:============================= test session starts ============================== 2024-06-28T15:39:47.725 INFO:teuthology.orchestra.run.smithi178.stdout:platform linux -- Python 3.10.6, pytest-8.2.2, pluggy-1.5.0 -- /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/bin/python 2024-06-28T15:39:47.725 INFO:teuthology.orchestra.run.smithi178.stdout:cachedir: .tox/py/.pytest_cache 2024-06-28T15:39:47.725 INFO:teuthology.orchestra.run.smithi178.stdout:rootdir: /home/ubuntu/cephtest/s3-tests-client.0 2024-06-28T15:39:47.725 INFO:teuthology.orchestra.run.smithi178.stdout:configfile: pytest.ini 2024-06-28T15:39:49.488 INFO:teuthology.orchestra.run.smithi178.stdout:collecting ... collected 859 items / 127 deselected / 732 selected 2024-06-28T15:39:49.488 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T15:39:50.117 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_none PASSED [ 0%] 2024-06-28T15:39:50.611 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_empty PASSED [ 0%] 2024-06-28T15:39:51.203 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_date_and_amz_date PASSED [ 0%] 2024-06-28T15:39:51.821 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_amz_date_and_no_date PASSED [ 0%] 2024-06-28T15:39:52.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_none PASSED [ 0%] 2024-06-28T15:39:52.788 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_contentlength_none PASSED [ 0%] 2024-06-28T15:39:53.584 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_acl_create_contentlength_none PASSED [ 0%] 2024-06-28T15:39:54.041 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_contentlength_none PASSED [ 1%] 2024-06-28T15:39:54.203 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_empty PASSED [ 1%] 2024-06-28T15:39:54.369 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_none PASSED [ 1%] 2024-06-28T15:39:54.856 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_mismatch_below_aws2 PASSED [ 1%] 2024-06-28T15:39:55.357 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_incorrect_aws2 PASSED [ 1%] 2024-06-28T15:39:55.842 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_invalid_aws2 PASSED [ 1%] 2024-06-28T15:39:56.341 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_none_aws2 PASSED [ 1%] 2024-06-28T15:39:56.510 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_invalid_aws2 PASSED [ 2%] 2024-06-28T15:39:56.679 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_none_aws2 PASSED [ 2%] 2024-06-28T15:39:56.793 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws4 SKIPPED [ 2%] 2024-06-28T15:39:56.904 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_mismatch_below_aws4 SKIPPED [ 2%] 2024-06-28T15:39:57.015 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_incorrect_aws4 SKIPPED [ 2%] 2024-06-28T15:39:57.127 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_invalid_aws4 SKIPPED [ 2%] 2024-06-28T15:39:57.245 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_empty_aws4 SKIPPED [ 2%] 2024-06-28T15:39:57.362 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_none_aws4 SKIPPED [ 3%] 2024-06-28T15:39:57.473 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_invalid_aws4 SKIPPED [ 3%] 2024-06-28T15:39:57.584 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_invalid_aws4 SKIPPED [ 3%] 2024-06-28T15:39:57.698 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_empty_aws4 SKIPPED [ 3%] 2024-06-28T15:39:57.813 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_empty_aws4 SKIPPED [ 3%] 2024-06-28T15:39:57.928 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_none_aws4 SKIPPED [ 3%] 2024-06-28T15:39:58.040 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_none_aws4 SKIPPED [ 3%] 2024-06-28T15:39:58.168 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_today_aws4 SKIPPED [ 3%] 2024-06-28T15:39:58.280 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_before_today_aws4 SKIPPED [ 4%] 2024-06-28T15:39:58.397 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_today_aws4 SKIPPED [ 4%] 2024-06-28T15:39:58.508 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_after_today_aws4 SKIPPED [ 4%] 2024-06-28T15:39:58.644 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws4 SKIPPED [ 4%] 2024-06-28T15:39:58.754 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_before_epoch_aws4 SKIPPED [ 4%] 2024-06-28T15:39:58.867 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_end_aws4 SKIPPED [ 4%] 2024-06-28T15:39:58.979 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_after_end_aws4 SKIPPED [ 4%] 2024-06-28T15:39:59.094 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_missing_signed_custom_header_aws4 SKIPPED [ 5%] 2024-06-28T15:39:59.209 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_object_create_missing_signed_header_aws4 SKIPPED [ 5%] 2024-06-28T15:39:59.322 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_invalid_aws4 SKIPPED [ 5%] 2024-06-28T15:39:59.436 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_ua_empty_aws4 SKIPPED [ 5%] 2024-06-28T15:39:59.554 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_ua_none_aws4 SKIPPED [ 5%] 2024-06-28T15:39:59.666 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws4 SKIPPED [ 5%] 2024-06-28T15:39:59.779 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_invalid_aws4 SKIPPED [ 5%] 2024-06-28T15:39:59.893 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_empty_aws4 SKIPPED [ 6%] 2024-06-28T15:40:00.005 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_empty_aws4 SKIPPED [ 6%] 2024-06-28T15:40:00.125 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_none_aws4 SKIPPED [ 6%] 2024-06-28T15:40:00.235 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_none_aws4 SKIPPED [ 6%] 2024-06-28T15:40:00.348 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws4 SKIPPED [ 6%] 2024-06-28T15:40:00.459 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_today_aws4 SKIPPED [ 6%] 2024-06-28T15:40:00.571 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws4 SKIPPED [ 6%] 2024-06-28T15:40:00.687 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_after_today_aws4 SKIPPED [ 6%] 2024-06-28T15:40:00.798 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws4 SKIPPED [ 7%] 2024-06-28T15:40:00.913 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_epoch_aws4 SKIPPED [ 7%] 2024-06-28T15:40:01.986 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3.py::test_versioning_obj_read_not_exist_null PASSED [ 7%] 2024-06-28T15:40:06.647 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3.py::test_object_storage_class PASSED [ 7%] 2024-06-28T15:40:13.254 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3.py::test_object_storage_class_multipart PASSED [ 7%] 2024-06-28T15:40:22.200 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3.py::test_object_modify_storage_class PASSED [ 7%] 2024-06-28T15:40:35.208 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3.py::test_object_modify_storage_class_multipart PASSED [ 7%] 2024-06-28T15:40:37.943 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3.py::test_object_storage_class_copy PASSED [ 8%] 2024-06-28T15:40:41.708 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3.py::test_object_storage_class_copy_multipart PASSED [ 8%] 2024-06-28T15:40:43.135 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3.py::test_atomic_dual_conditional_write_1mb PASSED [ 8%] 2024-06-28T15:40:43.585 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3.py::test_atomic_write_bucket_gone PASSED [ 8%] 2024-06-28T15:40:44.685 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_invalid_chunks_1 PASSED [ 8%] 2024-06-28T15:40:45.826 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_invalid_chunks_2 PASSED [ 8%] 2024-06-28T15:40:47.503 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3.py::test_bucket_policy_set_condition_operator_end_with_IfExists PASSED [ 8%] 2024-06-28T15:40:48.543 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3.py::test_bucket_policy_put_obj_request_obj_tag PASSED [ 9%] 2024-06-28T15:40:49.138 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_nonexistant_bucket_rgw SKIPPED [ 9%] 2024-06-28T15:40:49.143 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_public_index SKIPPED [ 9%] 2024-06-28T15:40:49.145 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_public_index SKIPPED [ 9%] 2024-06-28T15:40:49.148 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty SKIPPED [ 9%] 2024-06-28T15:40:49.150 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty SKIPPED [ 9%] 2024-06-28T15:40:49.153 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index SKIPPED [ 9%] 2024-06-28T15:40:49.155 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index SKIPPED [ 9%] 2024-06-28T15:40:49.158 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_missingerrordoc SKIPPED [ 10%] 2024-06-28T15:40:49.160 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_missingerrordoc SKIPPED [ 10%] 2024-06-28T15:40:49.163 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_missingerrordoc SKIPPED [ 10%] 2024-06-28T15:40:49.165 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_missingerrordoc SKIPPED [ 10%] 2024-06-28T15:40:49.169 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_blockederrordoc SKIPPED [ 10%] 2024-06-28T15:40:49.171 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_pubilc_errordoc SKIPPED [ 10%] 2024-06-28T15:40:49.174 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_blockederrordoc SKIPPED [ 10%] 2024-06-28T15:40:49.176 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_blockederrordoc SKIPPED [ 11%] 2024-06-28T15:40:49.179 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_blockederrordoc SKIPPED [ 11%] 2024-06-28T15:40:49.182 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_gooderrordoc SKIPPED [ 11%] 2024-06-28T15:40:49.184 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_gooderrordoc SKIPPED [ 11%] 2024-06-28T15:40:49.186 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_gooderrordoc SKIPPED [ 11%] 2024-06-28T15:40:49.190 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_gooderrordoc SKIPPED [ 11%] 2024-06-28T15:40:49.228 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_base SKIPPED [ 11%] 2024-06-28T15:40:49.230 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_path SKIPPED [ 12%] 2024-06-28T15:40:49.235 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_path_upgrade SKIPPED [ 12%] 2024-06-28T15:40:49.235 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_nonwebsite SKIPPED [ 12%] 2024-06-28T15:40:49.236 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_public_relative SKIPPED [ 12%] 2024-06-28T15:40:49.236 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_public_abs SKIPPED [ 12%] 2024-06-28T15:40:49.237 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_private_relative SKIPPED [ 12%] 2024-06-28T15:40:49.238 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_private_abs SKIPPED [ 12%] 2024-06-28T15:40:49.334 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py::test_routing_generator XFAIL [ 12%] 2024-06-28T15:40:49.342 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_utils.py::test_generate PASSED [ 13%] 2024-06-28T15:40:51.048 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_invalid_short PASSED [ 13%] 2024-06-28T15:41:03.995 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_bad PASSED [ 13%] 2024-06-28T15:41:05.056 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_empty PASSED [ 13%] 2024-06-28T15:41:06.280 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_none PASSED [ 13%] 2024-06-28T15:41:07.507 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_mismatch PASSED [ 13%] 2024-06-28T15:41:08.744 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_empty PASSED [ 13%] 2024-06-28T15:41:09.995 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_none PASSED [ 14%] 2024-06-28T15:41:10.924 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contentlength_negative PASSED [ 14%] 2024-06-28T15:41:12.136 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_invalid PASSED [ 14%] 2024-06-28T15:41:13.251 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_empty PASSED [ 14%] 2024-06-28T15:41:14.479 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_none PASSED [ 14%] 2024-06-28T15:41:15.468 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_put_bad_canned_acl PASSED [ 14%] 2024-06-28T15:41:16.360 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_expect_mismatch PASSED [ 14%] 2024-06-28T15:41:17.251 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_expect_empty PASSED [ 15%] 2024-06-28T15:41:17.788 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_contentlength_negative PASSED [ 15%] 2024-06-28T15:41:18.792 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws2 PASSED [ 15%] 2024-06-28T15:41:20.011 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_ua_empty_aws2 PASSED [ 15%] 2024-06-28T15:41:21.164 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_ua_none_aws2 PASSED [ 15%] 2024-06-28T15:41:22.120 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_invalid_aws2 PASSED [ 15%] 2024-06-28T15:41:23.124 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_empty_aws2 PASSED [ 15%] 2024-06-28T15:41:24.092 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_before_today_aws2 PASSED [ 15%] 2024-06-28T15:41:25.076 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws2 PASSED [ 16%] 2024-06-28T15:41:26.036 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_after_end_aws2 PASSED [ 16%] 2024-06-28T15:41:26.919 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_ua_empty_aws2 PASSED [ 16%] 2024-06-28T15:41:27.804 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_ua_none_aws2 PASSED [ 16%] 2024-06-28T15:41:28.363 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws2 PASSED [ 16%] 2024-06-28T15:41:28.947 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_empty_aws2 PASSED [ 16%] 2024-06-28T15:41:29.508 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws2 PASSED [ 16%] 2024-06-28T15:41:30.140 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws2 PASSED [ 17%] 2024-06-28T15:41:30.716 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws2 PASSED [ 17%] 2024-06-28T15:41:31.804 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy PASSED [ 17%] 2024-06-28T15:41:32.408 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy_invalid_user PASSED [ 17%] 2024-06-28T15:41:33.068 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy_parameter_limit PASSED [ 17%] 2024-06-28T15:41:34.060 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_put_existing_user_policy PASSED [ 17%] 2024-06-28T15:41:34.984 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_list_user_policy PASSED [ 17%] 2024-06-28T15:41:35.584 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_list_user_policy_invalid_user PASSED [ 18%] 2024-06-28T15:41:36.488 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy PASSED [ 18%] 2024-06-28T15:41:37.420 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy_invalid_user PASSED [ 18%] 2024-06-28T15:41:38.632 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy_from_multiple_policies PASSED [ 18%] 2024-06-28T15:41:39.452 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy PASSED [ 18%] 2024-06-28T15:41:40.324 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_invalid_user PASSED [ 18%] 2024-06-28T15:41:41.216 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_invalid_policy_name PASSED [ 18%] 2024-06-28T15:41:42.740 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_from_multiple_policies PASSED [ 18%] 2024-06-28T15:41:44.164 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_allow_bucket_actions_in_user_policy PASSED [ 19%] 2024-06-28T15:41:45.400 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_deny_bucket_actions_in_user_policy PASSED [ 19%] 2024-06-28T15:41:47.032 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_allow_object_actions_in_user_policy PASSED [ 19%] 2024-06-28T15:41:48.708 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_deny_object_actions_in_user_policy PASSED [ 19%] 2024-06-28T15:41:51.296 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_allow_multipart_actions_in_user_policy PASSED [ 19%] 2024-06-28T15:41:53.916 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_deny_multipart_actions_in_user_policy PASSED [ 19%] 2024-06-28T15:41:55.712 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_allow_tagging_actions_in_user_policy PASSED [ 19%] 2024-06-28T15:41:57.404 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_deny_tagging_actions_in_user_policy PASSED [ 20%] 2024-06-28T15:41:58.620 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_verify_conflicting_user_policy_statements PASSED [ 20%] 2024-06-28T15:42:00.200 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_verify_conflicting_user_policies PASSED [ 20%] 2024-06-28T15:42:01.220 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_verify_allow_iam_actions PASSED [ 20%] 2024-06-28T15:42:03.808 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_create PASSED [ 20%] 2024-06-28T15:42:05.444 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_case_insensitive_name PASSED [ 20%] 2024-06-28T15:42:07.341 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_delete PASSED [ 20%] 2024-06-28T15:42:11.184 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_list PASSED [ 21%] 2024-06-28T15:42:16.212 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_list_path_prefix PASSED [ 21%] 2024-06-28T15:42:19.056 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_update_name PASSED [ 21%] 2024-06-28T15:42:21.708 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_update_path PASSED [ 21%] 2024-06-28T15:42:23.728 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_create PASSED [ 21%] 2024-06-28T15:42:25.080 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_create PASSED [ 21%] 2024-06-28T15:42:27.512 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_update PASSED [ 21%] 2024-06-28T15:42:29.176 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_update PASSED [ 21%] 2024-06-28T15:42:31.444 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_delete PASSED [ 22%] 2024-06-28T15:42:32.989 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_delete PASSED [ 22%] 2024-06-28T15:42:35.908 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_list PASSED [ 22%] 2024-06-28T15:42:38.284 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_bucket_policy_allow PASSED [ 22%] 2024-06-28T15:42:41.524 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy PASSED [ 22%] 2024-06-28T15:42:44.780 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy_managed PASSED [ 22%] 2024-06-28T15:42:47.029 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy_allow PASSED [ 22%] 2024-06-28T15:42:49.092 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_create PASSED [ 23%] 2024-06-28T15:42:50.725 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_case_insensitive_name PASSED [ 23%] 2024-06-28T15:42:54.605 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_list PASSED [ 23%] 2024-06-28T15:42:57.460 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_update PASSED [ 23%] 2024-06-28T15:43:00.676 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_inline_group_policy PASSED [ 23%] 2024-06-28T15:43:03.676 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_managed_group_policy PASSED [ 23%] 2024-06-28T15:43:06.640 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_inline_group_policy_allow PASSED [ 23%] 2024-06-28T15:43:09.541 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_managed_group_policy_allow PASSED [ 24%] 2024-06-28T15:43:12.104 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_create PASSED [ 24%] 2024-06-28T15:43:13.828 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_case_insensitive_name PASSED [ 24%] 2024-06-28T15:43:15.801 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_delete PASSED [ 24%] 2024-06-28T15:43:19.373 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_list PASSED [ 24%] 2024-06-28T15:43:23.928 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_list_path_prefix PASSED [ 24%] 2024-06-28T15:43:25.876 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_update PASSED [ 24%] 2024-06-28T15:43:28.845 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy PASSED [ 25%] 2024-06-28T15:43:31.749 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_managed PASSED [ 25%] 2024-06-28T15:43:34.637 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow PASSED [ 25%] 2024-06-28T15:43:37.681 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_same_account_role_policy_allow PASSED [ 25%] 2024-06-28T15:43:40.933 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_role_policy_allow PASSED [ 25%] 2024-06-28T15:43:43.988 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow_create_bucket PASSED [ 25%] 2024-06-28T15:43:46.957 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow_get_role PASSED [ 25%] 2024-06-28T15:43:48.928 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_account_oidc_provider PASSED [ 25%] 2024-06-28T15:43:51.737 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_user_arn PASSED [ 26%] 2024-06-28T15:43:54.573 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_user_arn PASSED [ 26%] 2024-06-28T15:43:57.430 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_account_arn PASSED [ 26%] 2024-06-28T15:44:00.233 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_account_arn PASSED [ 26%] 2024-06-28T15:44:03.161 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_account_id PASSED [ 26%] 2024-06-28T15:44:06.005 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_account_id PASSED [ 26%] 2024-06-28T15:44:08.861 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_user_id PASSED [ 26%] 2024-06-28T15:44:11.745 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_user_id PASSED [ 27%] 2024-06-28T15:44:14.549 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_canonical_id PASSED [ 27%] 2024-06-28T15:44:17.389 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_canonical_id PASSED [ 27%] 2024-06-28T15:44:20.165 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_account_email PASSED [ 27%] 2024-06-28T15:44:22.993 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_account_email PASSED [ 27%] 2024-06-28T15:44:24.821 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_policy_allow_account_arn PASSED [ 27%] 2024-06-28T15:44:26.665 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_policy_allow_account_id PASSED [ 27%] 2024-06-28T15:44:28.417 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_acl_grant_canonical_id PASSED [ 28%] 2024-06-28T15:44:30.153 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_acl_grant_account_email PASSED [ 28%] 2024-06-28T15:44:31.420 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_empty PASSED [ 28%] 2024-06-28T15:44:33.017 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_distinct PASSED [ 28%] 2024-06-28T15:44:34.808 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_many PASSED [ 28%] 2024-06-28T15:44:36.417 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_many PASSED [ 28%] 2024-06-28T15:44:38.032 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_basic_key_count PASSED [ 28%] 2024-06-28T15:44:39.652 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_basic PASSED [ 28%] 2024-06-28T15:44:41.221 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_basic PASSED [ 29%] 2024-06-28T15:44:42.837 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_encoding_basic PASSED [ 29%] 2024-06-28T15:44:44.428 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_encoding_basic PASSED [ 29%] 2024-06-28T15:44:46.733 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix PASSED [ 29%] 2024-06-28T15:44:49.088 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix PASSED [ 29%] 2024-06-28T15:44:50.297 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_ends_with_delimiter PASSED [ 29%] 2024-06-28T15:44:51.529 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix_ends_with_delimiter PASSED [ 29%] 2024-06-28T15:44:53.128 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_alt PASSED [ 30%] 2024-06-28T15:44:54.676 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_alt PASSED [ 30%] 2024-06-28T15:44:56.989 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix_underscore PASSED [ 30%] 2024-06-28T15:44:59.309 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_underscore PASSED [ 30%] 2024-06-28T15:45:00.848 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_percentage PASSED [ 30%] 2024-06-28T15:45:02.380 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_percentage PASSED [ 30%] 2024-06-28T15:45:04.008 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_whitespace PASSED [ 30%] 2024-06-28T15:45:05.597 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_whitespace PASSED [ 31%] 2024-06-28T15:45:07.273 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_dot PASSED [ 31%] 2024-06-28T15:45:08.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_dot PASSED [ 31%] 2024-06-28T15:45:10.464 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_unreadable PASSED [ 31%] 2024-06-28T15:45:12.096 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_unreadable PASSED [ 31%] 2024-06-28T15:45:13.677 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_empty PASSED [ 31%] 2024-06-28T15:45:15.261 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_empty PASSED [ 31%] 2024-06-28T15:45:16.868 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_none PASSED [ 31%] 2024-06-28T15:45:18.493 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_none PASSED [ 32%] 2024-06-28T15:45:19.961 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_notempty PASSED [ 32%] 2024-06-28T15:45:21.425 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_defaultempty PASSED [ 32%] 2024-06-28T15:45:22.909 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_empty PASSED [ 32%] 2024-06-28T15:45:24.512 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_not_exist PASSED [ 32%] 2024-06-28T15:45:26.141 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_not_exist PASSED [ 32%] 2024-06-28T15:47:40.285 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_not_skip_special PASSED [ 32%] 2024-06-28T15:47:41.733 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_basic PASSED [ 33%] 2024-06-28T15:47:43.209 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_basic PASSED [ 33%] 2024-06-28T15:47:44.701 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_alt PASSED [ 33%] 2024-06-28T15:47:46.161 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_alt PASSED [ 33%] 2024-06-28T15:47:47.601 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_empty PASSED [ 33%] 2024-06-28T15:47:49.089 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_empty PASSED [ 33%] 2024-06-28T15:47:50.573 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_none PASSED [ 33%] 2024-06-28T15:47:52.041 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_none PASSED [ 34%] 2024-06-28T15:47:53.537 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_not_exist PASSED [ 34%] 2024-06-28T15:47:55.057 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_not_exist PASSED [ 34%] 2024-06-28T15:47:56.497 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_unreadable PASSED [ 34%] 2024-06-28T15:47:57.929 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_unreadable PASSED [ 34%] 2024-06-28T15:47:59.561 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_basic PASSED [ 34%] 2024-06-28T15:48:01.125 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_basic PASSED [ 34%] 2024-06-28T15:48:02.685 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_alt PASSED [ 34%] 2024-06-28T15:48:04.341 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_alt PASSED [ 35%] 2024-06-28T15:48:05.969 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_not_exist PASSED [ 35%] 2024-06-28T15:48:07.545 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_not_exist PASSED [ 35%] 2024-06-28T15:48:09.125 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_delimiter_not_exist PASSED [ 35%] 2024-06-28T15:48:10.725 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_delimiter_not_exist PASSED [ 35%] 2024-06-28T15:48:12.289 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 35%] 2024-06-28T15:48:13.897 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 35%] 2024-06-28T15:48:15.617 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_one PASSED [ 36%] 2024-06-28T15:48:17.269 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_one PASSED [ 36%] 2024-06-28T15:48:18.845 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_zero PASSED [ 36%] 2024-06-28T15:48:20.441 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_zero PASSED [ 36%] 2024-06-28T15:48:22.021 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_none PASSED [ 36%] 2024-06-28T15:48:23.657 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_none PASSED [ 36%] 2024-06-28T15:48:24.318 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_account_usage PASSED [ 36%] 2024-06-28T15:48:25.525 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_head_bucket_usage PASSED [ 37%] 2024-06-28T15:48:30.642 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_unordered PASSED [ 37%] 2024-06-28T15:48:35.541 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_unordered PASSED [ 37%] 2024-06-28T15:48:37.109 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_invalid PASSED [ 37%] 2024-06-28T15:48:38.717 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_none PASSED [ 37%] 2024-06-28T15:48:40.373 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_empty PASSED [ 37%] 2024-06-28T15:48:41.957 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_continuationtoken_empty PASSED [ 37%] 2024-06-28T15:48:43.637 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_continuationtoken PASSED [ 37%] 2024-06-28T15:48:45.345 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_both_continuationtoken_startafter PASSED [ 38%] 2024-06-28T15:48:46.917 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_unreadable PASSED [ 38%] 2024-06-28T15:48:48.513 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_unreadable PASSED [ 38%] 2024-06-28T15:48:50.149 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_not_in_list PASSED [ 38%] 2024-06-28T15:48:51.789 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_not_in_list PASSED [ 38%] 2024-06-28T15:48:53.357 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_after_list PASSED [ 38%] 2024-06-28T15:48:54.957 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_after_list PASSED [ 38%] 2024-06-28T15:48:56.966 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data PASSED [ 39%] 2024-06-28T15:48:59.605 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data_versioning PASSED [ 39%] 2024-06-28T15:49:00.665 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_objects_anonymous PASSED [ 39%] 2024-06-28T15:49:01.728 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_objects_anonymous PASSED [ 39%] 2024-06-28T15:49:02.690 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_objects_anonymous_fail PASSED [ 39%] 2024-06-28T15:49:03.641 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_objects_anonymous_fail PASSED [ 39%] 2024-06-28T15:49:04.269 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_notexist PASSED [ 39%] 2024-06-28T15:49:04.869 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_notexist PASSED [ 40%] 2024-06-28T15:49:05.481 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_delete_notexist PASSED [ 40%] 2024-06-28T15:49:06.689 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_delete_nonempty PASSED [ 40%] 2024-06-28T15:49:10.618 INFO:tasks.rgw.client.0.smithi178.stdout:2024-06-28T15:49:10.616+0000 e95a5640 -1 req 2528609968999735323 0.107997879s s3:delete_bucket ERROR: could not remove bucket test-yy0of5l43hi8iei4ttc0dgli-156 2024-06-28T15:49:11.045 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_concurrent_set_canned_acl PASSED [ 40%] 2024-06-28T15:49:11.452 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket PASSED [ 40%] 2024-06-28T15:49:11.453 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket ERROR [ 40%] 2024-06-28T15:49:12.265 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding PASSED [ 40%] 2024-06-28T15:49:12.266 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding ERROR [ 40%] 2024-06-28T15:49:12.873 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_delete PASSED [ 40%] 2024-06-28T15:49:12.873 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_delete ERROR [ 40%] 2024-06-28T15:49:13.538 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_read_not_exist PASSED [ 40%] 2024-06-28T15:49:13.539 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_read_not_exist ERROR [ 40%] 2024-06-28T15:49:14.280 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error PASSED [ 41%] 2024-06-28T15:49:14.280 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error ERROR [ 41%] 2024-06-28T15:49:18.738 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete PASSED [ 41%] 2024-06-28T15:49:18.738 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete ERROR [ 41%] 2024-06-28T15:49:20.241 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete PASSED [ 41%] 2024-06-28T15:49:20.241 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete ERROR [ 41%] 2024-06-28T15:49:21.662 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete PASSED [ 41%] 2024-06-28T15:49:21.663 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete ERROR [ 41%] 2024-06-28T15:51:38.717 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit PASSED [ 41%] 2024-06-28T15:51:38.717 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit ERROR [ 41%] 2024-06-28T15:53:52.542 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit PASSED [ 41%] 2024-06-28T15:53:52.542 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit ERROR [ 41%] 2024-06-28T15:53:53.400 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes PASSED [ 41%] 2024-06-28T15:53:53.400 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes ERROR [ 41%] 2024-06-28T15:53:54.216 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_check_etag PASSED [ 42%] 2024-06-28T15:53:54.216 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_check_etag ERROR [ 42%] 2024-06-28T15:53:55.144 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_cache_control PASSED [ 42%] 2024-06-28T15:53:55.144 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_cache_control ERROR [ 42%] 2024-06-28T15:53:56.013 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_expires PASSED [ 42%] 2024-06-28T15:53:56.013 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_expires ERROR [ 42%] 2024-06-28T15:53:57.183 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete PASSED [ 42%] 2024-06-28T15:53:57.183 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete ERROR [ 42%] 2024-06-28T15:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good PASSED [ 42%] 2024-06-28T15:53:58.091 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good ERROR [ 42%] 2024-06-28T15:53:59.119 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty PASSED [ 42%] 2024-06-28T15:53:59.120 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty ERROR [ 42%] 2024-06-28T15:54:00.221 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty PASSED [ 42%] 2024-06-28T15:54:00.221 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty ERROR [ 42%] 2024-06-28T15:54:01.209 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put PASSED [ 43%] 2024-06-28T15:54:01.209 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put ERROR [ 43%] 2024-06-28T15:54:02.144 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_file PASSED [ 43%] 2024-06-28T15:54:02.144 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_file ERROR [ 43%] 2024-06-28T15:54:03.201 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request PASSED [ 43%] 2024-06-28T15:54:03.201 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request ERROR [ 43%] 2024-06-28T15:54:04.279 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request PASSED [ 43%] 2024-06-28T15:54:04.279 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request ERROR [ 43%] 2024-06-28T15:54:05.115 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type PASSED [ 43%] 2024-06-28T15:54:05.115 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type ERROR [ 43%] 2024-06-28T15:54:05.801 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key PASSED [ 43%] 2024-06-28T15:54:05.802 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key ERROR [ 43%] 2024-06-28T15:54:06.649 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code PASSED [ 43%] 2024-06-28T15:54:06.649 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code ERROR [ 43%] 2024-06-28T15:54:07.441 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code PASSED [ 43%] 2024-06-28T15:54:07.441 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code ERROR [ 43%] 2024-06-28T15:54:08.847 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk PASSED [ 44%] 2024-06-28T15:54:08.847 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk ERROR [ 44%] 2024-06-28T15:54:09.777 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename PASSED [ 44%] 2024-06-28T15:54:09.777 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename ERROR [ 44%] 2024-06-28T15:54:10.644 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header PASSED [ 44%] 2024-06-28T15:54:10.644 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header ERROR [ 44%] 2024-06-28T15:54:11.460 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields PASSED [ 44%] 2024-06-28T15:54:11.460 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields ERROR [ 44%] 2024-06-28T15:54:12.322 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values PASSED [ 44%] 2024-06-28T15:54:12.322 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values ERROR [ 44%] 2024-06-28T15:54:13.332 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action PASSED [ 44%] 2024-06-28T15:54:13.332 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action ERROR [ 44%] 2024-06-28T15:54:14.024 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature PASSED [ 44%] 2024-06-28T15:54:14.025 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature ERROR [ 44%] 2024-06-28T15:54:14.670 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key PASSED [ 45%] 2024-06-28T15:54:14.671 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key ERROR [ 45%] 2024-06-28T15:54:15.327 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format PASSED [ 45%] 2024-06-28T15:54:15.327 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format ERROR [ 45%] 2024-06-28T15:54:15.987 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified PASSED [ 45%] 2024-06-28T15:54:15.988 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified ERROR [ 45%] 2024-06-28T15:54:16.665 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature PASSED [ 45%] 2024-06-28T15:54:16.666 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature ERROR [ 45%] 2024-06-28T15:54:17.332 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition PASSED [ 45%] 2024-06-28T15:54:17.333 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition ERROR [ 45%] 2024-06-28T15:54:18.262 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header PASSED [ 45%] 2024-06-28T15:54:18.262 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header ERROR [ 45%] 2024-06-28T15:54:18.909 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field PASSED [ 45%] 2024-06-28T15:54:18.909 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field ERROR [ 45%] 2024-06-28T15:54:19.647 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive PASSED [ 46%] 2024-06-28T15:54:19.647 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive ERROR [ 46%] 2024-06-28T15:54:20.310 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive PASSED [ 46%] 2024-06-28T15:54:20.311 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive ERROR [ 46%] 2024-06-28T15:54:20.947 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy PASSED [ 46%] 2024-06-28T15:54:20.948 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy ERROR [ 46%] 2024-06-28T15:54:21.987 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket PASSED [ 46%] 2024-06-28T15:54:21.987 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket ERROR [ 46%] 2024-06-28T15:54:22.674 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value PASSED [ 46%] 2024-06-28T15:54:22.674 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value ERROR [ 46%] 2024-06-28T15:54:23.319 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition PASSED [ 46%] 2024-06-28T15:54:23.319 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition ERROR [ 46%] 2024-06-28T15:54:24.021 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list PASSED [ 46%] 2024-06-28T15:54:24.021 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list ERROR [ 46%] 2024-06-28T15:54:24.698 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded PASSED [ 46%] 2024-06-28T15:54:24.698 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded ERROR [ 46%] 2024-06-28T15:54:25.323 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument PASSED [ 47%] 2024-06-28T15:54:25.323 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument ERROR [ 47%] 2024-06-28T15:54:25.998 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument PASSED [ 47%] 2024-06-28T15:54:25.998 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument ERROR [ 47%] 2024-06-28T15:54:26.696 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum PASSED [ 47%] 2024-06-28T15:54:26.696 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum ERROR [ 47%] 2024-06-28T15:54:28.320 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug PASSED [ 47%] 2024-06-28T15:54:28.320 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug ERROR [ 47%] 2024-06-28T15:54:29.003 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions PASSED [ 47%] 2024-06-28T15:54:29.003 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions ERROR [ 47%] 2024-06-28T15:54:30.004 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good PASSED [ 47%] 2024-06-28T15:54:30.005 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good ERROR [ 47%] 2024-06-28T15:54:30.916 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed PASSED [ 47%] 2024-06-28T15:54:30.916 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed ERROR [ 47%] 2024-06-28T15:54:31.829 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good PASSED [ 48%] 2024-06-28T15:54:31.829 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good ERROR [ 48%] 2024-06-28T15:54:32.782 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed PASSED [ 48%] 2024-06-28T15:54:32.782 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed ERROR [ 48%] 2024-06-28T15:54:33.805 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good PASSED [ 48%] 2024-06-28T15:54:33.805 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good ERROR [ 48%] 2024-06-28T15:54:35.829 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed PASSED [ 48%] 2024-06-28T15:54:35.829 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed ERROR [ 48%] 2024-06-28T15:54:36.788 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good PASSED [ 48%] 2024-06-28T15:54:36.788 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good ERROR [ 48%] 2024-06-28T15:54:37.708 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed PASSED [ 48%] 2024-06-28T15:54:37.708 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed ERROR [ 48%] 2024-06-28T15:54:38.957 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good PASSED [ 48%] 2024-06-28T15:54:38.958 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good ERROR [ 48%] 2024-06-28T15:54:40.164 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed PASSED [ 49%] 2024-06-28T15:54:40.164 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed ERROR [ 49%] 2024-06-28T15:54:41.363 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good PASSED [ 49%] 2024-06-28T15:54:41.363 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good ERROR [ 49%] 2024-06-28T15:54:42.186 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed PASSED [ 49%] 2024-06-28T15:54:42.187 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed ERROR [ 49%] 2024-06-28T15:54:43.400 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good PASSED [ 49%] 2024-06-28T15:54:43.400 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good ERROR [ 49%] 2024-06-28T15:54:44.633 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed PASSED [ 49%] 2024-06-28T15:54:44.633 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed ERROR [ 49%] 2024-06-28T15:54:45.600 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good PASSED [ 49%] 2024-06-28T15:54:45.600 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good ERROR [ 49%] 2024-06-28T15:54:46.815 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed PASSED [ 49%] 2024-06-28T15:54:46.816 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed ERROR [ 49%] 2024-06-28T15:54:47.694 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get PASSED [ 50%] 2024-06-28T15:54:47.695 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get ERROR [ 50%] 2024-06-28T15:54:48.432 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone PASSED [ 50%] 2024-06-28T15:54:48.432 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone ERROR [ 50%] 2024-06-28T15:54:49.174 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone PASSED [ 50%] 2024-06-28T15:54:49.174 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone ERROR [ 50%] 2024-06-28T15:54:50.041 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone PASSED [ 50%] 2024-06-28T15:54:50.041 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone ERROR [ 50%] 2024-06-28T15:54:50.710 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head PASSED [ 50%] 2024-06-28T15:54:50.711 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head ERROR [ 50%] 2024-06-28T15:54:51.009 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist PASSED [ 50%] 2024-06-28T15:54:51.009 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist ERROR [ 50%] 2024-06-28T15:54:52.360 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_extended PASSED [ 50%] 2024-06-28T15:54:52.360 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_extended ERROR [ 50%] 2024-06-28T15:54:53.224 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl PASSED [ 50%] 2024-06-28T15:54:53.225 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl ERROR [ 50%] 2024-06-28T15:54:54.085 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl PASSED [ 51%] 2024-06-28T15:54:54.085 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl ERROR [ 51%] 2024-06-28T15:54:57.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime PASSED [ 51%] 2024-06-28T15:54:57.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime ERROR [ 51%] 2024-06-28T15:54:58.736 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated PASSED [ 51%] 2024-06-28T15:54:58.736 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated ERROR [ 51%] 2024-06-28T15:54:59.632 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers PASSED [ 51%] 2024-06-28T15:54:59.632 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers ERROR [ 51%] 2024-06-28T15:55:00.462 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl PASSED [ 51%] 2024-06-28T15:55:00.462 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl ERROR [ 51%] 2024-06-28T15:55:01.310 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl PASSED [ 51%] 2024-06-28T15:55:01.310 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl ERROR [ 51%] 2024-06-28T15:55:02.025 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone PASSED [ 51%] 2024-06-28T15:55:02.025 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone ERROR [ 51%] 2024-06-28T15:55:02.969 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone PASSED [ 52%] 2024-06-28T15:55:02.969 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone ERROR [ 52%] 2024-06-28T15:55:03.858 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired PASSED [ 52%] 2024-06-28T15:55:03.858 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired ERROR [ 52%] 2024-06-28T15:55:04.529 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant PASSED [ 52%] 2024-06-28T15:55:04.529 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant ERROR [ 52%] 2024-06-28T15:55:05.363 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero PASSED [ 52%] 2024-06-28T15:55:05.363 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero ERROR [ 52%] 2024-06-28T15:55:06.189 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range PASSED [ 52%] 2024-06-28T15:55:06.189 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range ERROR [ 52%] 2024-06-28T15:55:07.024 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range PASSED [ 52%] 2024-06-28T15:55:07.024 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range ERROR [ 52%] 2024-06-28T15:55:07.831 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put PASSED [ 52%] 2024-06-28T15:55:07.832 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put ERROR [ 52%] 2024-06-28T15:55:08.690 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access PASSED [ 53%] 2024-06-28T15:55:08.691 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access ERROR [ 53%] 2024-06-28T15:55:09.627 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_authenticated PASSED [ 53%] 2024-06-28T15:55:09.628 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_authenticated ERROR [ 53%] 2024-06-28T15:55:10.535 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl PASSED [ 53%] 2024-06-28T15:55:10.535 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl ERROR [ 53%] 2024-06-28T15:55:11.044 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant PASSED [ 53%] 2024-06-28T15:55:11.044 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant ERROR [ 53%] 2024-06-28T15:55:11.897 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired PASSED [ 53%] 2024-06-28T15:55:11.897 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired ERROR [ 53%] 2024-06-28T15:55:12.223 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha PASSED [ 53%] 2024-06-28T15:55:12.223 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha ERROR [ 53%] 2024-06-28T15:55:12.522 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one PASSED [ 53%] 2024-06-28T15:55:12.522 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one ERROR [ 53%] 2024-06-28T15:55:12.830 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two PASSED [ 53%] 2024-06-28T15:55:12.830 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two ERROR [ 53%] 2024-06-28T15:55:13.397 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 PASSED [ 54%] 2024-06-28T15:55:13.397 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 ERROR [ 54%] 2024-06-28T15:55:14.035 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 PASSED [ 54%] 2024-06-28T15:55:14.035 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 ERROR [ 54%] 2024-06-28T15:55:14.700 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 PASSED [ 54%] 2024-06-28T15:55:14.700 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 ERROR [ 54%] 2024-06-28T15:55:15.330 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 PASSED [ 54%] 2024-06-28T15:55:15.330 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 ERROR [ 54%] 2024-06-28T15:55:16.012 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name PASSED [ 54%] 2024-06-28T15:55:16.012 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name ERROR [ 54%] 2024-06-28T15:55:16.316 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip PASSED [ 54%] 2024-06-28T15:55:16.316 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip ERROR [ 54%] 2024-06-28T15:55:16.657 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore PASSED [ 54%] 2024-06-28T15:55:16.657 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore ERROR [ 54%] 2024-06-28T15:55:17.247 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long PASSED [ 55%] 2024-06-28T15:55:17.248 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long ERROR [ 55%] 2024-06-28T15:55:17.545 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end PASSED [ 55%] 2024-06-28T15:55:17.545 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end ERROR [ 55%] 2024-06-28T15:55:17.832 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot PASSED [ 55%] 2024-06-28T15:55:17.832 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot ERROR [ 55%] 2024-06-28T15:55:18.158 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash PASSED [ 55%] 2024-06-28T15:55:18.158 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash ERROR [ 55%] 2024-06-28T15:55:18.459 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot PASSED [ 55%] 2024-06-28T15:55:18.459 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot ERROR [ 55%] 2024-06-28T15:55:19.173 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists PASSED [ 55%] 2024-06-28T15:55:19.173 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists ERROR [ 55%] 2024-06-28T15:55:19.940 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_get_location PASSED [ 55%] 2024-06-28T15:55:19.940 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_get_location ERROR [ 55%] 2024-06-28T15:55:20.661 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner PASSED [ 56%] 2024-06-28T15:55:20.661 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner ERROR [ 56%] 2024-06-28T15:55:21.330 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl PASSED [ 56%] 2024-06-28T15:55:21.330 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl ERROR [ 56%] 2024-06-28T15:55:22.030 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl PASSED [ 56%] 2024-06-28T15:55:22.030 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl ERROR [ 56%] 2024-06-28T15:55:22.686 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_default PASSED [ 56%] 2024-06-28T15:55:22.686 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_default ERROR [ 56%] 2024-06-28T15:55:23.389 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create PASSED [ 56%] 2024-06-28T15:55:23.389 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create ERROR [ 56%] 2024-06-28T15:55:24.319 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned PASSED [ 56%] 2024-06-28T15:55:24.319 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned ERROR [ 56%] 2024-06-28T15:55:25.017 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite PASSED [ 56%] 2024-06-28T15:55:25.017 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite ERROR [ 56%] 2024-06-28T15:55:25.738 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread PASSED [ 56%] 2024-06-28T15:55:25.738 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread ERROR [ 56%] 2024-06-28T15:55:26.643 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_default PASSED [ 57%] 2024-06-28T15:55:26.643 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_default ERROR [ 57%] 2024-06-28T15:55:27.668 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create PASSED [ 57%] 2024-06-28T15:55:27.669 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create ERROR [ 57%] 2024-06-28T15:55:28.785 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned PASSED [ 57%] 2024-06-28T15:55:28.785 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned ERROR [ 57%] 2024-06-28T15:55:29.760 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite PASSED [ 57%] 2024-06-28T15:55:29.760 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite ERROR [ 57%] 2024-06-28T15:55:30.666 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread PASSED [ 57%] 2024-06-28T15:55:30.666 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread ERROR [ 57%] 2024-06-28T15:55:31.792 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread PASSED [ 57%] 2024-06-28T15:55:31.792 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread ERROR [ 57%] 2024-06-28T15:55:32.869 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol PASSED [ 57%] 2024-06-28T15:55:32.869 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol ERROR [ 57%] 2024-06-28T15:55:34.067 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner PASSED [ 58%] 2024-06-28T15:55:34.067 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner ERROR [ 58%] 2024-06-28T15:55:35.295 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes PASSED [ 58%] 2024-06-28T15:55:35.295 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes ERROR [ 58%] 2024-06-28T15:55:36.026 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private PASSED [ 58%] 2024-06-28T15:55:36.026 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private ERROR [ 58%] 2024-06-28T15:55:37.147 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl PASSED [ 58%] 2024-06-28T15:55:37.148 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl ERROR [ 58%] 2024-06-28T15:55:38.270 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_write PASSED [ 58%] 2024-06-28T15:55:38.270 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_write ERROR [ 58%] 2024-06-28T15:55:39.382 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp PASSED [ 58%] 2024-06-28T15:55:39.382 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp ERROR [ 58%] 2024-06-28T15:55:40.609 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_read PASSED [ 58%] 2024-06-28T15:55:40.609 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_read ERROR [ 58%] 2024-06-28T15:55:41.721 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_readacp PASSED [ 59%] 2024-06-28T15:55:41.721 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_readacp ERROR [ 59%] 2024-06-28T15:55:43.033 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol PASSED [ 59%] 2024-06-28T15:55:43.033 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol ERROR [ 59%] 2024-06-28T15:55:44.191 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read PASSED [ 59%] 2024-06-28T15:55:44.192 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read ERROR [ 59%] 2024-06-28T15:55:45.364 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp PASSED [ 59%] 2024-06-28T15:55:45.365 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp ERROR [ 59%] 2024-06-28T15:55:46.644 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write PASSED [ 59%] 2024-06-28T15:55:46.644 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write ERROR [ 59%] 2024-06-28T15:55:47.843 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp PASSED [ 59%] 2024-06-28T15:55:47.843 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp ERROR [ 59%] 2024-06-28T15:55:48.586 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user PASSED [ 59%] 2024-06-28T15:55:48.586 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user ERROR [ 59%] 2024-06-28T15:55:49.566 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants PASSED [ 59%] 2024-06-28T15:55:49.566 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants ERROR [ 59%] 2024-06-28T15:55:50.616 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants PASSED [ 60%] 2024-06-28T15:55:50.616 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants ERROR [ 60%] 2024-06-28T15:55:51.580 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email PASSED [ 60%] 2024-06-28T15:55:51.580 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email ERROR [ 60%] 2024-06-28T15:55:52.323 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist PASSED [ 60%] 2024-06-28T15:55:52.323 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist ERROR [ 60%] 2024-06-28T15:55:53.483 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all PASSED [ 60%] 2024-06-28T15:55:53.484 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all ERROR [ 60%] 2024-06-28T15:55:55.128 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private PASSED [ 60%] 2024-06-28T15:55:55.129 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private ERROR [ 60%] 2024-06-28T15:55:56.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private PASSED [ 60%] 2024-06-28T15:55:56.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private ERROR [ 60%] 2024-06-28T15:55:58.475 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread PASSED [ 60%] 2024-06-28T15:55:58.475 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread ERROR [ 60%] 2024-06-28T15:56:00.185 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread PASSED [ 61%] 2024-06-28T15:56:00.185 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread ERROR [ 61%] 2024-06-28T15:56:01.880 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite PASSED [ 61%] 2024-06-28T15:56:01.880 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite ERROR [ 61%] 2024-06-28T15:56:03.519 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite PASSED [ 61%] 2024-06-28T15:56:03.595 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite ERROR [ 61%] 2024-06-28T15:56:05.191 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private PASSED [ 61%] 2024-06-28T15:56:05.191 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private ERROR [ 61%] 2024-06-28T15:56:06.924 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread PASSED [ 61%] 2024-06-28T15:56:06.924 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread ERROR [ 61%] 2024-06-28T15:56:08.591 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite PASSED [ 61%] 2024-06-28T15:56:08.592 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite ERROR [ 61%] 2024-06-28T15:56:10.437 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private PASSED [ 61%] 2024-06-28T15:56:10.437 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private ERROR [ 61%] 2024-06-28T15:56:12.307 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread PASSED [ 62%] 2024-06-28T15:56:12.307 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread ERROR [ 62%] 2024-06-28T15:56:14.056 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite PASSED [ 62%] 2024-06-28T15:56:14.056 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite ERROR [ 62%] 2024-06-28T15:56:16.375 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list PASSED [ 62%] 2024-06-28T15:56:16.375 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list ERROR [ 62%] 2024-06-28T15:56:18.600 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime PASSED [ 62%] 2024-06-28T15:56:18.601 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime ERROR [ 62%] 2024-06-28T15:56:18.888 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous PASSED [ 62%] 2024-06-28T15:56:18.888 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous ERROR [ 62%] 2024-06-28T15:56:19.159 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth PASSED [ 62%] 2024-06-28T15:56:19.160 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth ERROR [ 62%] 2024-06-28T15:56:19.395 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth PASSED [ 62%] 2024-06-28T15:56:19.395 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth ERROR [ 62%] 2024-06-28T15:56:20.149 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha PASSED [ 62%] 2024-06-28T15:56:20.149 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha ERROR [ 62%] 2024-06-28T15:56:21.018 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit PASSED [ 63%] 2024-06-28T15:56:21.018 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit ERROR [ 63%] 2024-06-28T15:56:21.584 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period PASSED [ 63%] 2024-06-28T15:56:21.584 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period ERROR [ 63%] 2024-06-28T15:56:22.206 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen PASSED [ 63%] 2024-06-28T15:56:22.206 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen ERROR [ 63%] 2024-06-28T15:56:23.476 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding PASSED [ 63%] 2024-06-28T15:56:23.477 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding ERROR [ 63%] 2024-06-28T15:56:28.403 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names PASSED [ 63%] 2024-06-28T15:56:28.403 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names ERROR [ 63%] 2024-06-28T15:56:29.924 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix PASSED [ 63%] 2024-06-28T15:56:29.924 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix ERROR [ 63%] 2024-06-28T15:56:31.209 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size PASSED [ 63%] 2024-06-28T15:56:31.209 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size ERROR [ 63%] 2024-06-28T15:56:35.925 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_16m PASSED [ 64%] 2024-06-28T15:56:35.926 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_16m ERROR [ 64%] 2024-06-28T15:56:37.113 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket PASSED [ 64%] 2024-06-28T15:56:37.113 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket ERROR [ 64%] 2024-06-28T15:56:38.337 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype PASSED [ 64%] 2024-06-28T15:56:38.337 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype ERROR [ 64%] 2024-06-28T15:56:39.345 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself PASSED [ 64%] 2024-06-28T15:56:39.345 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself ERROR [ 64%] 2024-06-28T15:56:40.470 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata PASSED [ 64%] 2024-06-28T15:56:40.470 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata ERROR [ 64%] 2024-06-28T15:56:42.061 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket PASSED [ 64%] 2024-06-28T15:56:42.062 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket ERROR [ 64%] 2024-06-28T15:56:43.078 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket PASSED [ 64%] 2024-06-28T15:56:43.078 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket ERROR [ 64%] 2024-06-28T15:56:44.547 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket PASSED [ 65%] 2024-06-28T15:56:44.547 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket ERROR [ 65%] 2024-06-28T15:56:45.983 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl PASSED [ 65%] 2024-06-28T15:56:45.983 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl ERROR [ 65%] 2024-06-28T15:56:48.240 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata PASSED [ 65%] 2024-06-28T15:56:48.240 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata ERROR [ 65%] 2024-06-28T15:56:50.553 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata PASSED [ 65%] 2024-06-28T15:56:50.553 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata ERROR [ 65%] 2024-06-28T15:56:51.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found PASSED [ 65%] 2024-06-28T15:56:51.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found ERROR [ 65%] 2024-06-28T15:56:51.951 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found PASSED [ 65%] 2024-06-28T15:56:51.951 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found ERROR [ 65%] 2024-06-28T15:56:56.095 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket PASSED [ 65%] 2024-06-28T15:56:56.096 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket ERROR [ 65%] 2024-06-28T15:56:57.608 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding PASSED [ 65%] 2024-06-28T15:56:57.608 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding ERROR [ 65%] 2024-06-28T15:57:10.156 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload PASSED [ 66%] 2024-06-28T15:57:10.156 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload ERROR [ 66%] 2024-06-28T15:57:10.984 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty PASSED [ 66%] 2024-06-28T15:57:10.984 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty ERROR [ 66%] 2024-06-28T15:57:12.190 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_small PASSED [ 66%] 2024-06-28T15:57:12.190 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_small ERROR [ 66%] 2024-06-28T15:57:16.195 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_small PASSED [ 66%] 2024-06-28T15:57:16.196 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_small ERROR [ 66%] 2024-06-28T15:57:17.293 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range PASSED [ 66%] 2024-06-28T15:57:17.294 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range ERROR [ 66%] 2024-06-28T15:57:19.423 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range PASSED [ 66%] 2024-06-28T15:57:19.424 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range ERROR [ 66%] 2024-06-28T15:57:29.222 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names PASSED [ 66%] 2024-06-28T15:57:29.222 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names ERROR [ 66%] 2024-06-28T15:57:47.727 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload PASSED [ 67%] 2024-06-28T15:57:47.727 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload ERROR [ 67%] 2024-06-28T15:58:10.767 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned PASSED [ 67%] 2024-06-28T15:58:10.767 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned ERROR [ 67%] 2024-06-28T15:59:33.873 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part PASSED [ 67%] 2024-06-28T15:59:33.874 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part ERROR [ 67%] 2024-06-28T15:59:45.375 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes PASSED [ 67%] 2024-06-28T15:59:45.375 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes ERROR [ 67%] 2024-06-28T16:00:02.484 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes PASSED [ 67%] 2024-06-28T16:00:02.484 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes ERROR [ 67%] 2024-06-28T16:00:04.573 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small PASSED [ 67%] 2024-06-28T16:00:04.574 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small ERROR [ 67%] 2024-06-28T16:00:09.689 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents PASSED [ 67%] 2024-06-28T16:00:09.689 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents ERROR [ 67%] 2024-06-28T16:00:14.450 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object PASSED [ 68%] 2024-06-28T16:00:14.450 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object ERROR [ 68%] 2024-06-28T16:00:17.061 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload PASSED [ 68%] 2024-06-28T16:00:17.061 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload ERROR [ 68%] 2024-06-28T16:00:17.929 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found PASSED [ 68%] 2024-06-28T16:00:17.930 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found ERROR [ 68%] 2024-06-28T16:00:22.279 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload PASSED [ 68%] 2024-06-28T16:00:22.280 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload ERROR [ 68%] 2024-06-28T16:00:23.570 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner PASSED [ 68%] 2024-06-28T16:00:23.570 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner ERROR [ 68%] 2024-06-28T16:00:24.571 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part PASSED [ 68%] 2024-06-28T16:00:24.571 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part ERROR [ 68%] 2024-06-28T16:00:25.575 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag PASSED [ 68%] 2024-06-28T16:00:25.575 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag ERROR [ 68%] 2024-06-28T16:00:46.424 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_get_part PASSED [ 68%] 2024-06-28T16:00:46.424 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_get_part ERROR [ 68%] 2024-06-28T16:00:47.462 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part PASSED [ 69%] 2024-06-28T16:00:47.462 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part ERROR [ 69%] 2024-06-28T16:00:48.164 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_100_continue PASSED [ 69%] 2024-06-28T16:00:48.164 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_100_continue ERROR [ 69%] 2024-06-28T16:00:49.240 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_set_cors PASSED [ 69%] 2024-06-28T16:00:49.240 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_set_cors ERROR [ 69%] 2024-06-28T16:00:54.406 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_response PASSED [ 69%] 2024-06-28T16:00:54.407 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_response ERROR [ 69%] 2024-06-28T16:00:58.295 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_wildcard PASSED [ 69%] 2024-06-28T16:00:58.295 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_wildcard ERROR [ 69%] 2024-06-28T16:01:02.098 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_header_option PASSED [ 69%] 2024-06-28T16:01:02.098 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_header_option ERROR [ 69%] 2024-06-28T16:01:03.064 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object PASSED [ 69%] 2024-06-28T16:01:03.065 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object ERROR [ 69%] 2024-06-28T16:01:03.726 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object_tenant PASSED [ 70%] 2024-06-28T16:01:03.726 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object_tenant ERROR [ 70%] 2024-06-28T16:01:04.763 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object PASSED [ 70%] 2024-06-28T16:01:04.763 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object ERROR [ 70%] 2024-06-28T16:01:05.757 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_with_acl PASSED [ 70%] 2024-06-28T16:01:05.757 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_with_acl ERROR [ 70%] 2024-06-28T16:01:06.397 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant PASSED [ 70%] 2024-06-28T16:01:06.397 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant ERROR [ 70%] 2024-06-28T16:01:07.078 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant_with_acl PASSED [ 70%] 2024-06-28T16:01:07.078 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant_with_acl ERROR [ 70%] 2024-06-28T16:01:08.047 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_set_bucket_tagging PASSED [ 70%] 2024-06-28T16:01:08.047 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_set_bucket_tagging ERROR [ 70%] 2024-06-28T16:01:09.843 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_1mb PASSED [ 70%] 2024-06-28T16:01:09.843 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_1mb ERROR [ 70%] 2024-06-28T16:01:12.234 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_4mb PASSED [ 71%] 2024-06-28T16:01:12.234 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_4mb ERROR [ 71%] 2024-06-28T16:01:15.870 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_8mb PASSED [ 71%] 2024-06-28T16:01:15.871 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_read_8mb ERROR [ 71%] 2024-06-28T16:01:17.947 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_1mb PASSED [ 71%] 2024-06-28T16:01:17.947 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_1mb ERROR [ 71%] 2024-06-28T16:01:21.422 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_4mb PASSED [ 71%] 2024-06-28T16:01:21.423 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_4mb ERROR [ 71%] 2024-06-28T16:01:26.299 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_8mb PASSED [ 71%] 2024-06-28T16:01:26.300 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_write_8mb ERROR [ 71%] 2024-06-28T16:01:28.519 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_1mb PASSED [ 71%] 2024-06-28T16:01:28.519 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_1mb ERROR [ 71%] 2024-06-28T16:01:32.487 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_4mb PASSED [ 71%] 2024-06-28T16:01:32.487 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_4mb ERROR [ 71%] 2024-06-28T16:01:38.958 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_8mb PASSED [ 71%] 2024-06-28T16:01:38.959 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_8mb ERROR [ 71%] 2024-06-28T16:01:40.968 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_conditional_write_1mb PASSED [ 72%] 2024-06-28T16:01:40.968 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_conditional_write_1mb ERROR [ 72%] 2024-06-28T16:01:42.605 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_multipart_upload_write PASSED [ 72%] 2024-06-28T16:01:42.605 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_atomic_multipart_upload_write ERROR [ 72%] 2024-06-28T16:01:44.610 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_resend_first_finishes_last PASSED [ 72%] 2024-06-28T16:01:44.610 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_resend_first_finishes_last ERROR [ 72%] 2024-06-28T16:01:45.560 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_response_code PASSED [ 72%] 2024-06-28T16:01:45.560 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_response_code ERROR [ 72%] 2024-06-28T16:01:52.412 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_big_request_response_code PASSED [ 72%] 2024-06-28T16:01:52.412 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_big_request_response_code ERROR [ 72%] 2024-06-28T16:01:53.327 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code PASSED [ 72%] 2024-06-28T16:01:53.327 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code ERROR [ 72%] 2024-06-28T16:01:54.324 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code PASSED [ 72%] 2024-06-28T16:01:54.324 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code ERROR [ 72%] 2024-06-28T16:01:55.293 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_invalid_range PASSED [ 73%] 2024-06-28T16:01:55.294 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_invalid_range ERROR [ 73%] 2024-06-28T16:01:56.235 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_empty_object PASSED [ 73%] 2024-06-28T16:01:56.235 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_ranged_request_empty_object ERROR [ 73%] 2024-06-28T16:01:57.719 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_create_suspend PASSED [ 73%] 2024-06-28T16:01:57.720 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_create_suspend ERROR [ 73%] 2024-06-28T16:02:20.744 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove PASSED [ 73%] 2024-06-28T16:02:20.745 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove ERROR [ 73%] 2024-06-28T16:02:23.218 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove_head PASSED [ 73%] 2024-06-28T16:02:23.218 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove_head ERROR [ 73%] 2024-06-28T16:02:24.517 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_removal PASSED [ 73%] 2024-06-28T16:02:24.517 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_removal ERROR [ 73%] 2024-06-28T16:02:26.446 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite PASSED [ 73%] 2024-06-28T16:02:26.446 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite ERROR [ 73%] 2024-06-28T16:02:28.318 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended PASSED [ 74%] 2024-06-28T16:02:28.318 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended ERROR [ 74%] 2024-06-28T16:02:37.878 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspend_versions PASSED [ 74%] 2024-06-28T16:02:37.879 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspend_versions ERROR [ 74%] 2024-06-28T16:02:40.076 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspended_copy PASSED [ 74%] 2024-06-28T16:02:40.076 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspended_copy ERROR [ 74%] 2024-06-28T16:02:51.152 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_all PASSED [ 74%] 2024-06-28T16:02:51.152 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_all ERROR [ 74%] 2024-06-28T16:03:33.282 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names PASSED [ 74%] 2024-06-28T16:03:33.283 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names ERROR [ 74%] 2024-06-28T16:03:51.977 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart PASSED [ 74%] 2024-06-28T16:03:51.978 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart ERROR [ 74%] 2024-06-28T16:03:56.985 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_list_marker PASSED [ 74%] 2024-06-28T16:03:56.985 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_obj_list_marker ERROR [ 74%] 2024-06-28T16:04:01.495 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_copy_obj_version PASSED [ 75%] 2024-06-28T16:04:01.495 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_copy_obj_version ERROR [ 75%] 2024-06-28T16:04:03.216 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete PASSED [ 75%] 2024-06-28T16:04:03.216 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete ERROR [ 75%] 2024-06-28T16:04:05.123 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker PASSED [ 75%] 2024-06-28T16:04:05.124 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker ERROR [ 75%] 2024-06-28T16:04:06.295 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create PASSED [ 75%] 2024-06-28T16:04:06.295 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create ERROR [ 75%] 2024-06-28T16:04:08.734 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl PASSED [ 75%] 2024-06-28T16:04:08.734 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl ERROR [ 75%] 2024-06-28T16:04:10.761 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl_no_version_specified PASSED [ 75%] 2024-06-28T16:04:10.761 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_object_acl_no_version_specified ERROR [ 75%] 2024-06-28T16:04:14.792 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove PASSED [ 75%] 2024-06-28T16:04:14.792 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove ERROR [ 75%] 2024-06-28T16:04:15.572 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set PASSED [ 75%] 2024-06-28T16:04:15.572 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set ERROR [ 75%] 2024-06-28T16:04:16.384 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get PASSED [ 76%] 2024-06-28T16:04:16.385 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get ERROR [ 76%] 2024-06-28T16:04:17.230 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get_no_id PASSED [ 76%] 2024-06-28T16:04:17.230 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_get_no_id ERROR [ 76%] 2024-06-28T16:04:17.924 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_id_too_long PASSED [ 76%] 2024-06-28T16:04:17.924 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_id_too_long ERROR [ 76%] 2024-06-28T16:04:18.521 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_same_id PASSED [ 76%] 2024-06-28T16:04:18.521 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_same_id ERROR [ 76%] 2024-06-28T16:04:19.411 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_invalid_status PASSED [ 76%] 2024-06-28T16:04:19.411 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_invalid_status ERROR [ 76%] 2024-06-28T16:04:20.095 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_date PASSED [ 76%] 2024-06-28T16:04:20.096 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_date ERROR [ 76%] 2024-06-28T16:04:20.753 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_invalid_date PASSED [ 76%] 2024-06-28T16:04:20.753 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_invalid_date ERROR [ 76%] 2024-06-28T16:04:21.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_noncurrent PASSED [ 77%] 2024-06-28T16:04:21.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_noncurrent ERROR [ 77%] 2024-06-28T16:04:22.535 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_deletemarker PASSED [ 77%] 2024-06-28T16:04:22.535 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_deletemarker ERROR [ 77%] 2024-06-28T16:04:23.293 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_filter PASSED [ 77%] 2024-06-28T16:04:23.294 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_filter ERROR [ 77%] 2024-06-28T16:04:24.080 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_empty_filter PASSED [ 77%] 2024-06-28T16:04:24.081 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_empty_filter ERROR [ 77%] 2024-06-28T16:04:24.784 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_multipart PASSED [ 77%] 2024-06-28T16:04:24.784 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_set_multipart ERROR [ 77%] 2024-06-28T16:04:25.488 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_transition_set_invalid_date PASSED [ 77%] 2024-06-28T16:04:25.489 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_lifecycle_transition_set_invalid_date ERROR [ 77%] 2024-06-28T16:04:26.502 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1b PASSED [ 77%] 2024-06-28T16:04:26.502 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1b ERROR [ 77%] 2024-06-28T16:04:27.539 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1kb PASSED [ 78%] 2024-06-28T16:04:27.539 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1kb ERROR [ 78%] 2024-06-28T16:04:28.984 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1MB PASSED [ 78%] 2024-06-28T16:04:28.984 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1MB ERROR [ 78%] 2024-06-28T16:04:29.930 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_13b PASSED [ 78%] 2024-06-28T16:04:29.930 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_13b ERROR [ 78%] 2024-06-28T16:04:31.000 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_method_head PASSED [ 78%] 2024-06-28T16:04:31.000 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_method_head ERROR [ 78%] 2024-06-28T16:04:32.018 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_present PASSED [ 78%] 2024-06-28T16:04:32.018 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_present ERROR [ 78%] 2024-06-28T16:04:32.872 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_other_key PASSED [ 78%] 2024-06-28T16:04:32.872 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_other_key ERROR [ 78%] 2024-06-28T16:04:33.603 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_invalid_md5 PASSED [ 78%] 2024-06-28T16:04:33.603 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_invalid_md5 ERROR [ 78%] 2024-06-28T16:04:34.298 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_md5 PASSED [ 78%] 2024-06-28T16:04:34.298 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_md5 ERROR [ 78%] 2024-06-28T16:04:35.084 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_key PASSED [ 79%] 2024-06-28T16:04:35.084 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_key ERROR [ 79%] 2024-06-28T16:04:35.816 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_key_no_sse_c PASSED [ 79%] 2024-06-28T16:04:35.816 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_key_no_sse_c ERROR [ 79%] 2024-06-28T16:05:13.767 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_upload PASSED [ 79%] 2024-06-28T16:05:13.768 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_upload ERROR [ 79%] 2024-06-28T16:05:51.007 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload PASSED [ 79%] 2024-06-28T16:05:51.008 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload ERROR [ 79%] 2024-06-28T16:06:00.570 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download PASSED [ 79%] 2024-06-28T16:06:00.571 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download ERROR [ 79%] 2024-06-28T16:06:01.443 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request PASSED [ 79%] 2024-06-28T16:06:01.444 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request ERROR [ 79%] 2024-06-28T16:06:02.454 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_method_head PASSED [ 79%] 2024-06-28T16:06:02.455 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_method_head ERROR [ 79%] 2024-06-28T16:06:03.372 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_present PASSED [ 80%] 2024-06-28T16:06:03.372 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_present ERROR [ 80%] 2024-06-28T16:06:04.162 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_no_key PASSED [ 80%] 2024-06-28T16:06:04.162 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_no_key ERROR [ 80%] 2024-06-28T16:06:04.834 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_not_declared PASSED [ 80%] 2024-06-28T16:06:04.834 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_not_declared ERROR [ 80%] 2024-06-28T16:06:31.050 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_upload PASSED [ 80%] 2024-06-28T16:06:31.051 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_upload ERROR [ 80%] 2024-06-28T16:06:39.442 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 PASSED [ 80%] 2024-06-28T16:06:39.443 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 ERROR [ 80%] 2024-06-28T16:06:48.023 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 PASSED [ 80%] 2024-06-28T16:06:48.023 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 ERROR [ 80%] 2024-06-28T16:06:48.913 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_post_object_authenticated_request PASSED [ 80%] 2024-06-28T16:06:48.914 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_post_object_authenticated_request ERROR [ 80%] 2024-06-28T16:06:49.895 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1b PASSED [ 81%] 2024-06-28T16:06:49.896 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1b ERROR [ 81%] 2024-06-28T16:06:50.831 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1kb PASSED [ 81%] 2024-06-28T16:06:50.831 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1kb ERROR [ 81%] 2024-06-28T16:06:52.283 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1MB PASSED [ 81%] 2024-06-28T16:06:52.283 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1MB ERROR [ 81%] 2024-06-28T16:06:53.239 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_13b PASSED [ 81%] 2024-06-28T16:06:53.239 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_13b ERROR [ 81%] 2024-06-28T16:06:54.178 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_read_declare PASSED [ 81%] 2024-06-28T16:06:54.178 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_read_declare ERROR [ 81%] 2024-06-28T16:06:55.148 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy PASSED [ 81%] 2024-06-28T16:06:55.148 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy ERROR [ 81%] 2024-06-28T16:06:56.144 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy PASSED [ 81%] 2024-06-28T16:06:56.144 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy ERROR [ 81%] 2024-06-28T16:06:57.414 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_acl PASSED [ 81%] 2024-06-28T16:06:57.414 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_acl ERROR [ 81%] 2024-06-28T16:06:58.656 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_acl PASSED [ 82%] 2024-06-28T16:06:58.656 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_acl ERROR [ 82%] 2024-06-28T16:07:00.564 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_another_bucket PASSED [ 82%] 2024-06-28T16:07:00.564 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_another_bucket ERROR [ 82%] 2024-06-28T16:07:02.579 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_another_bucket PASSED [ 82%] 2024-06-28T16:07:02.579 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_another_bucket ERROR [ 82%] 2024-06-28T16:07:04.816 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_tagging PASSED [ 82%] 2024-06-28T16:07:04.816 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_tagging ERROR [ 82%] 2024-06-28T16:07:06.663 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_head_tagging PASSED [ 82%] 2024-06-28T16:07:06.663 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_obj_head_tagging ERROR [ 82%] 2024-06-28T16:07:08.651 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_tags PASSED [ 82%] 2024-06-28T16:07:08.651 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_tags ERROR [ 82%] 2024-06-28T16:07:10.570 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_tags PASSED [ 82%] 2024-06-28T16:07:10.571 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_tags ERROR [ 82%] 2024-06-28T16:07:13.277 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_kvsize_tags PASSED [ 83%] 2024-06-28T16:07:13.277 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_max_kvsize_tags ERROR [ 83%] 2024-06-28T16:07:15.422 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_key_tags PASSED [ 83%] 2024-06-28T16:07:15.422 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_key_tags ERROR [ 83%] 2024-06-28T16:07:17.489 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_val_tags PASSED [ 83%] 2024-06-28T16:07:17.489 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_excess_val_tags ERROR [ 83%] 2024-06-28T16:07:19.680 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_modify_tags PASSED [ 83%] 2024-06-28T16:07:19.680 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_modify_tags ERROR [ 83%] 2024-06-28T16:07:22.416 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_delete_tags PASSED [ 83%] 2024-06-28T16:07:22.416 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_delete_tags ERROR [ 83%] 2024-06-28T16:07:23.470 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_anonymous_request PASSED [ 83%] 2024-06-28T16:07:23.470 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_anonymous_request ERROR [ 83%] 2024-06-28T16:07:24.343 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_authenticated_request PASSED [ 83%] 2024-06-28T16:07:24.343 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_tags_authenticated_request ERROR [ 83%] 2024-06-28T16:07:25.394 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_with_tags PASSED [ 84%] 2024-06-28T16:07:25.394 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_with_tags ERROR [ 84%] 2024-06-28T16:07:27.541 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_tags_acl_public PASSED [ 84%] 2024-06-28T16:07:27.541 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_tags_acl_public ERROR [ 84%] 2024-06-28T16:07:29.651 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_tags_acl_public PASSED [ 84%] 2024-06-28T16:07:29.651 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_tags_acl_public ERROR [ 84%] 2024-06-28T16:07:31.889 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_delete_tags_obj_public PASSED [ 84%] 2024-06-28T16:07:31.889 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_delete_tags_obj_public ERROR [ 84%] 2024-06-28T16:07:34.470 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id PASSED [ 84%] 2024-06-28T16:07:34.471 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id ERROR [ 84%] 2024-06-28T16:07:57.752 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id PASSED [ 84%] 2024-06-28T16:07:57.753 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id ERROR [ 84%] 2024-06-28T16:07:59.483 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag PASSED [ 84%] 2024-06-28T16:07:59.483 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag ERROR [ 84%] 2024-06-28T16:08:01.423 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag PASSED [ 84%] 2024-06-28T16:08:01.423 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag ERROR [ 84%] 2024-06-28T16:08:03.289 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag PASSED [ 85%] 2024-06-28T16:08:03.290 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag ERROR [ 85%] 2024-06-28T16:08:05.838 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source PASSED [ 85%] 2024-06-28T16:08:05.839 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source ERROR [ 85%] 2024-06-28T16:08:07.946 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta PASSED [ 85%] 2024-06-28T16:08:07.947 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta ERROR [ 85%] 2024-06-28T16:08:08.980 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_acl PASSED [ 85%] 2024-06-28T16:08:08.980 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_acl ERROR [ 85%] 2024-06-28T16:08:10.899 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_grant PASSED [ 85%] 2024-06-28T16:08:10.899 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_grant ERROR [ 85%] 2024-06-28T16:08:11.537 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 PASSED [ 85%] 2024-06-28T16:08:11.537 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 ERROR [ 85%] 2024-06-28T16:08:12.136 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_kms PASSED [ 85%] 2024-06-28T16:08:12.136 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_kms ERROR [ 85%] 2024-06-28T16:08:12.740 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms PASSED [ 86%] 2024-06-28T16:08:12.741 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms ERROR [ 86%] 2024-06-28T16:08:13.472 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms PASSED [ 86%] 2024-06-28T16:08:13.472 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms ERROR [ 86%] 2024-06-28T16:08:14.477 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc PASSED [ 86%] 2024-06-28T16:08:14.477 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc ERROR [ 86%] 2024-06-28T16:08:15.292 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 PASSED [ 86%] 2024-06-28T16:08:15.292 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 ERROR [ 86%] 2024-06-28T16:08:17.221 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag PASSED [ 86%] 2024-06-28T16:08:17.222 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag ERROR [ 86%] 2024-06-28T16:08:18.248 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock PASSED [ 86%] 2024-06-28T16:08:18.248 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock ERROR [ 86%] 2024-06-28T16:08:18.951 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket PASSED [ 86%] 2024-06-28T16:08:18.951 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket ERROR [ 86%] 2024-06-28T16:08:19.680 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years PASSED [ 87%] 2024-06-28T16:08:19.680 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years ERROR [ 87%] 2024-06-28T16:08:20.386 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days PASSED [ 87%] 2024-06-28T16:08:20.386 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days ERROR [ 87%] 2024-06-28T16:08:21.165 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years PASSED [ 87%] 2024-06-28T16:08:21.166 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years ERROR [ 87%] 2024-06-28T16:08:21.952 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode PASSED [ 87%] 2024-06-28T16:08:21.953 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode ERROR [ 87%] 2024-06-28T16:08:22.676 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status PASSED [ 87%] 2024-06-28T16:08:22.676 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status ERROR [ 87%] 2024-06-28T16:08:23.455 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_suspend_versioning PASSED [ 87%] 2024-06-28T16:08:23.455 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_suspend_versioning ERROR [ 87%] 2024-06-28T16:08:24.312 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock PASSED [ 87%] 2024-06-28T16:08:24.313 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock ERROR [ 87%] 2024-06-28T16:08:24.991 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket PASSED [ 87%] 2024-06-28T16:08:24.991 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket ERROR [ 87%] 2024-06-28T16:08:26.171 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention PASSED [ 88%] 2024-06-28T16:08:26.171 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention ERROR [ 88%] 2024-06-28T16:08:27.109 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket PASSED [ 88%] 2024-06-28T16:08:27.109 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket ERROR [ 88%] 2024-06-28T16:08:28.272 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode PASSED [ 88%] 2024-06-28T16:08:28.272 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode ERROR [ 88%] 2024-06-28T16:08:29.473 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention PASSED [ 88%] 2024-06-28T16:08:29.473 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention ERROR [ 88%] 2024-06-28T16:08:30.564 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 PASSED [ 88%] 2024-06-28T16:08:30.564 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 ERROR [ 88%] 2024-06-28T16:08:31.498 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket PASSED [ 88%] 2024-06-28T16:08:31.498 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket ERROR [ 88%] 2024-06-28T16:08:32.902 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_versionid PASSED [ 88%] 2024-06-28T16:08:32.902 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_versionid ERROR [ 88%] 2024-06-28T16:08:34.235 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention PASSED [ 89%] 2024-06-28T16:08:34.235 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention ERROR [ 89%] 2024-06-28T16:08:35.493 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period PASSED [ 89%] 2024-06-28T16:08:35.493 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period ERROR [ 89%] 2024-06-28T16:08:36.580 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period PASSED [ 89%] 2024-06-28T16:08:36.580 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period ERROR [ 89%] 2024-06-28T16:08:37.849 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass PASSED [ 89%] 2024-06-28T16:08:37.849 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass ERROR [ 89%] 2024-06-28T16:08:39.128 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention PASSED [ 89%] 2024-06-28T16:08:39.129 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention ERROR [ 89%] 2024-06-28T16:08:40.488 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_retention PASSED [ 89%] 2024-06-28T16:08:40.488 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_retention ERROR [ 89%] 2024-06-28T16:08:41.948 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker PASSED [ 89%] 2024-06-28T16:08:41.948 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker ERROR [ 89%] 2024-06-28T16:08:43.411 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention PASSED [ 90%] 2024-06-28T16:08:43.411 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention ERROR [ 90%] 2024-06-28T16:08:44.660 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold PASSED [ 90%] 2024-06-28T16:08:44.660 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold ERROR [ 90%] 2024-06-28T16:08:45.609 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket PASSED [ 90%] 2024-06-28T16:08:45.609 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket ERROR [ 90%] 2024-06-28T16:08:46.705 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status PASSED [ 90%] 2024-06-28T16:08:46.705 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status ERROR [ 90%] 2024-06-28T16:08:48.042 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold PASSED [ 90%] 2024-06-28T16:08:48.042 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold ERROR [ 90%] 2024-06-28T16:08:49.049 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket PASSED [ 90%] 2024-06-28T16:08:49.049 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket ERROR [ 90%] 2024-06-28T16:08:50.274 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on PASSED [ 90%] 2024-06-28T16:08:50.274 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on ERROR [ 90%] 2024-06-28T16:08:51.763 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_legal_hold_on PASSED [ 90%] 2024-06-28T16:08:51.763 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_legal_hold_on ERROR [ 90%] 2024-06-28T16:08:52.790 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off PASSED [ 91%] 2024-06-28T16:08:52.790 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off ERROR [ 91%] 2024-06-28T16:08:54.179 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_metadata PASSED [ 91%] 2024-06-28T16:08:54.179 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_metadata ERROR [ 91%] 2024-06-28T16:08:55.392 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_uploading_obj PASSED [ 91%] 2024-06-28T16:08:55.392 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_uploading_obj ERROR [ 91%] 2024-06-28T16:09:05.985 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass PASSED [ 91%] 2024-06-28T16:09:05.985 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass ERROR [ 91%] 2024-06-28T16:09:07.077 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass PASSED [ 91%] 2024-06-28T16:09:07.077 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass ERROR [ 91%] 2024-06-28T16:09:17.514 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_compliance PASSED [ 91%] 2024-06-28T16:09:17.515 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_compliance ERROR [ 91%] 2024-06-28T16:09:18.772 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifmatch_good PASSED [ 91%] 2024-06-28T16:09:18.772 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifmatch_good ERROR [ 91%] 2024-06-28T16:09:19.943 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifnonematch_failed PASSED [ 92%] 2024-06-28T16:09:19.943 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_copy_object_ifnonematch_failed ERROR [ 92%] 2024-06-28T16:09:20.608 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_policy_status PASSED [ 92%] 2024-06-28T16:09:20.608 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_policy_status ERROR [ 92%] 2024-06-28T16:09:21.410 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_acl_bucket_policy_status PASSED [ 92%] 2024-06-28T16:09:21.410 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_acl_bucket_policy_status ERROR [ 92%] 2024-06-28T16:09:22.235 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status PASSED [ 92%] 2024-06-28T16:09:22.236 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status ERROR [ 92%] 2024-06-28T16:09:23.115 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status PASSED [ 92%] 2024-06-28T16:09:23.115 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status ERROR [ 92%] 2024-06-28T16:09:24.023 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status PASSED [ 92%] 2024-06-28T16:09:24.024 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status ERROR [ 92%] 2024-06-28T16:09:24.897 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_deny_bucket_policy_status PASSED [ 92%] 2024-06-28T16:09:24.898 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_deny_bucket_policy_status ERROR [ 92%] 2024-06-28T16:09:25.752 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_undefined_public_block PASSED [ 93%] 2024-06-28T16:09:25.752 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_undefined_public_block ERROR [ 93%] 2024-06-28T16:09:26.773 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_block_deny_bucket_policy PASSED [ 93%] 2024-06-28T16:09:26.773 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_public_block_deny_bucket_policy ERROR [ 93%] 2024-06-28T16:09:27.559 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_public_block PASSED [ 93%] 2024-06-28T16:09:27.560 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_public_block ERROR [ 93%] 2024-06-28T16:09:28.617 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_put_bucket_acls PASSED [ 93%] 2024-06-28T16:09:28.617 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_put_bucket_acls ERROR [ 93%] 2024-06-28T16:09:29.633 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_object_canned_acls PASSED [ 93%] 2024-06-28T16:09:29.633 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_object_canned_acls ERROR [ 93%] 2024-06-28T16:09:30.423 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_policy PASSED [ 93%] 2024-06-28T16:09:30.424 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_block_public_policy ERROR [ 93%] 2024-06-28T16:09:31.797 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_ignore_public_acls PASSED [ 93%] 2024-06-28T16:09:31.797 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_ignore_public_acls ERROR [ 93%] 2024-06-28T16:09:41.736 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy PASSED [ 93%] 2024-06-28T16:09:41.736 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy ERROR [ 93%] 2024-06-28T16:09:42.479 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_bucket_encryption_kms PASSED [ 94%] 2024-06-28T16:09:42.479 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_put_bucket_encryption_kms ERROR [ 94%] 2024-06-28T16:09:43.334 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_encryption_kms PASSED [ 94%] 2024-06-28T16:09:43.335 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_bucket_encryption_kms ERROR [ 94%] 2024-06-28T16:09:44.349 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_delete_bucket_encryption_kms PASSED [ 94%] 2024-06-28T16:09:44.350 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_delete_bucket_encryption_kms ERROR [ 94%] 2024-06-28T16:09:45.354 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request PASSED [ 94%] 2024-06-28T16:09:45.354 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request ERROR [ 94%] 2024-06-28T16:09:47.232 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_torrent PASSED [ 94%] 2024-06-28T16:09:47.233 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_torrent ERROR [ 94%] 2024-06-28T16:10:00.419 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_generate_where_clause PASSED [ 94%] 2024-06-28T16:10:11.704 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_generate_projection PASSED [ 94%] 2024-06-28T16:10:12.851 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_count_operation PASSED [ 95%] 2024-06-28T16:10:14.515 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_count_json_operation PASSED [ 95%] 2024-06-28T16:10:29.823 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_json_column_sum_min_max PASSED [ 95%] 2024-06-28T16:10:58.240 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_json_nullif_expressions PASSED [ 95%] 2024-06-28T16:11:03.569 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_column_sum_min_max PASSED [ 95%] 2024-06-28T16:11:10.660 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_nullif_expressions PASSED [ 95%] 2024-06-28T16:11:13.720 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_nulliftrue_expressions PASSED [ 95%] 2024-06-28T16:11:16.180 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_is_not_null_expressions PASSED [ 96%] 2024-06-28T16:11:17.228 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_lowerupper_expressions PASSED [ 96%] 2024-06-28T16:11:22.664 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_in_expressions PASSED [ 96%] 2024-06-28T16:11:28.792 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_true_false_in_expressions PASSED [ 96%] 2024-06-28T16:11:46.968 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_like_expressions PASSED [ 96%] 2024-06-28T16:11:51.140 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_truefalselike_expressions PASSED [ 96%] 2024-06-28T16:11:54.005 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_complex_expressions PASSED [ 96%] 2024-06-28T16:11:56.393 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_alias PASSED [ 96%] 2024-06-28T16:11:57.797 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_alias_cyclic_refernce PASSED [ 97%] 2024-06-28T16:12:04.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_datetime PASSED [ 97%] 2024-06-28T16:12:08.769 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_true_false_datetime PASSED [ 97%] 2024-06-28T16:12:10.413 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_csv_parser PASSED [ 97%] 2024-06-28T16:12:12.245 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_csv_definition PASSED [ 97%] 2024-06-28T16:12:20.629 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_schema_definition PASSED [ 97%] 2024-06-28T16:12:30.373 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_when_then_else_expressions PASSED [ 97%] 2024-06-28T16:12:43.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_coalesce_expressions PASSED [ 98%] 2024-06-28T16:12:45.849 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_cast_expressions PASSED [ 98%] 2024-06-28T16:12:46.377 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_version PASSED [ 98%] 2024-06-28T16:13:01.889 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_trim_expressions PASSED [ 98%] 2024-06-28T16:13:06.953 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_truefalse_trim_expressions PASSED [ 98%] 2024-06-28T16:13:14.405 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_escape_expressions PASSED [ 98%] 2024-06-28T16:13:19.881 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_case_value_expressions PASSED [ 98%] 2024-06-28T16:13:23.649 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_bool_cast_expressions PASSED [ 99%] 2024-06-28T16:14:02.062 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_progress_expressions PASSED [ 99%] 2024-06-28T16:14:02.598 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3select.py::test_output_serial_expressions PASSED [ 99%] 2024-06-28T16:14:04.501 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_sns.py::test_account_topic PASSED [ 99%] 2024-06-28T16:14:05.901 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_sns.py::test_cross_account_topic PASSED [ 99%] 2024-06-28T16:14:07.341 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_sns.py::test_account_topic_publish PASSED [ 99%] 2024-06-28T16:14:09.162 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_sns.py::test_cross_account_topic_publish PASSED [ 99%] 2024-06-28T16:14:09.203 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_utils.py::test_generate PASSED [100%] 2024-06-28T16:14:09.204 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.204 INFO:teuthology.orchestra.run.smithi178.stdout:==================================== ERRORS ==================================== 2024-06-28T16:14:09.204 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_object_write_to_nonexist_bucket ___________ 2024-06-28T16:14:09.204 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.204 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.204 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.204 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.204 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.205 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.205 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.205 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.205 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.205 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.205 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.205 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.205 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.206 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.206 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.206 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.206 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.206 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.206 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.206 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.206 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.206 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.207 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.207 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.207 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.207 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.207 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.207 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.207 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.207 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.207 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.208 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.208 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.208 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.208 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.208 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.208 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.208 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.208 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.209 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.209 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.209 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.209 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.209 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.209 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.209 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.209 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.209 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.209 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.209 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.210 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.210 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.210 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.210 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.210 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.210 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.210 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.210 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.210 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.211 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.211 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.211 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.211 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.211 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.211 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.211 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.211 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.211 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.211 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.211 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.211 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.212 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.213 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.213 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.213 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.213 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.213 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_object_write_with_chunked_transfer_encoding _____ 2024-06-28T16:14:09.213 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.213 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.213 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.213 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.213 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.213 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.214 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.214 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.214 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.214 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.214 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.214 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.214 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.214 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.214 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.214 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.215 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.215 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.215 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.215 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.215 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.215 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.215 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.215 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.215 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.216 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.216 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.216 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.216 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.216 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.216 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.216 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.216 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.217 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.217 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.217 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.217 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.217 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.217 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.217 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.217 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.217 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.218 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.218 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.218 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.218 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.218 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.218 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.218 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.219 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.219 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.219 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.219 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.219 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.219 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.219 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.219 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.220 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.220 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.220 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.220 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.220 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.220 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.220 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.221 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.221 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.221 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.221 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.221 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.221 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.221 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.222 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.222 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.222 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.222 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.222 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.222 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.222 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.223 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.223 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.223 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.223 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.223 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.223 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.224 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.224 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.224 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.224 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.224 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.224 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.224 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_bucket_create_delete ________________ 2024-06-28T16:14:09.224 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.224 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.224 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.225 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.225 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.225 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.225 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.225 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.225 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.225 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.225 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.225 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.229 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.230 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.230 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.230 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.230 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.230 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.230 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.230 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.230 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.230 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.230 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.230 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.230 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.230 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.231 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.231 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.231 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.231 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.231 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.231 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.231 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.232 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.232 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.232 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.232 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.232 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.232 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.232 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.232 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.232 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.232 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.232 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.232 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.232 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.232 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.233 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.233 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.233 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.233 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.233 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.233 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.233 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.233 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.233 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.233 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.233 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.233 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.233 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.233 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.234 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.234 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.234 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.234 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.234 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.234 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.234 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.234 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.234 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.234 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.234 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.234 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.234 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.235 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.235 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.235 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.235 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.235 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.235 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.235 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.235 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.235 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.235 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.235 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.236 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.236 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.236 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_object_read_not_exist ________________ 2024-06-28T16:14:09.236 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.236 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.236 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.236 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.236 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.236 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.236 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.236 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.236 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.236 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.237 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.237 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.237 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.237 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.237 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.237 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.237 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.237 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.237 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.237 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.237 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.237 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.237 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.237 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.238 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.238 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.238 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.238 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.238 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.238 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.238 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.238 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.239 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.239 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.239 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.239 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.239 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.239 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.239 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.239 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.239 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.240 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.241 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.242 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.242 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.242 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.243 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.243 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.243 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_object_requestid_matches_header_on_error ______ 2024-06-28T16:14:09.243 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.243 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.243 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.243 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.243 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.243 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.243 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.243 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.243 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.243 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.243 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.244 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.244 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.244 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.244 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.244 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.244 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.244 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.244 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.244 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.244 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.244 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.244 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.244 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.245 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.245 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.245 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.246 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.247 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.247 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.247 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.247 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.247 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.247 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.247 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.247 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.247 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.247 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.247 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.247 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.247 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.247 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.248 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.248 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.248 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.248 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.248 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.248 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.248 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.248 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.248 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.248 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.248 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.248 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.248 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.248 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.249 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.249 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.249 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.249 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.249 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.249 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.249 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.249 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.249 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_versioning_concurrent_multi_object_delete ______ 2024-06-28T16:14:09.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.249 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.250 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.250 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.250 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.250 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.250 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.250 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.250 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.250 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.251 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.251 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.251 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.251 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.251 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.251 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.251 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.251 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.251 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.251 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.251 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.251 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.251 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.251 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.252 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.252 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.252 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.252 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.252 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.252 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.252 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.252 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.252 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.252 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.252 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.253 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.253 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.253 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.253 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.253 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.254 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.254 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.254 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.254 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.254 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.254 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.254 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.254 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.254 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.254 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.254 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.254 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.254 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.255 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.255 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.255 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.255 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.255 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.255 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.255 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.255 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.255 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.255 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.255 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.255 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.255 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.255 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.256 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.257 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.257 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.257 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.257 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.257 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.257 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_multi_object_delete _________________ 2024-06-28T16:14:09.257 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.257 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.257 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.257 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.257 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.257 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.257 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.257 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.258 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.258 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.258 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.258 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.258 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.258 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.258 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.259 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.259 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.259 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.259 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.259 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.259 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.259 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.259 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.259 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.259 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.259 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.259 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.259 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.260 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.260 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.260 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.260 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.260 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.260 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.260 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.260 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.261 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.261 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.261 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.261 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.261 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.261 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.261 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.261 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.261 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.261 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.261 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.261 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.261 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.262 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.262 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.262 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.262 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.262 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.262 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.262 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.262 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.262 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.262 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.262 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.262 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.262 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.263 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.263 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.263 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.263 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.263 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.263 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.263 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.263 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.263 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.263 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.263 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.264 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.264 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.264 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.264 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.264 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.264 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.264 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.264 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.264 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.264 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.264 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.264 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.264 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.264 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.265 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.265 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_multi_objectv2_delete ________________ 2024-06-28T16:14:09.265 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.265 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.265 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.265 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.265 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.265 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.265 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.265 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.265 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.265 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.266 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.266 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.266 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.266 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.266 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.266 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.266 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.266 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.266 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.266 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.266 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.266 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.267 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.267 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.267 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.267 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.267 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.267 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.267 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.267 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.267 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.267 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.268 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.268 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.268 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.268 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.268 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.268 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.268 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.268 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.268 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.268 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.268 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.268 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.268 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.268 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.269 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.269 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.269 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.269 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.269 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.269 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.269 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.269 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.269 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.269 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.269 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.269 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.270 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.270 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.270 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.270 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.270 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.271 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.271 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.271 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.271 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.271 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.271 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.271 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.271 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.271 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.271 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.271 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.271 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.271 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.272 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.272 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.272 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.272 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.272 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.272 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.272 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.272 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.272 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.272 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.272 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.272 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.272 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.272 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.273 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_multi_object_delete_key_limit ____________ 2024-06-28T16:14:09.273 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.273 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.273 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.273 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.273 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.273 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.273 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.273 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.273 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.273 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.273 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.273 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.274 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.274 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.274 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.274 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.274 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.274 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.274 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.274 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.274 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.274 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.274 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.274 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.274 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.275 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.275 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.275 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.275 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.275 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.275 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.275 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.275 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.275 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.275 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.275 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.275 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.275 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.276 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.276 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.276 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.276 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.276 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.276 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.276 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.276 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.276 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.276 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.276 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.276 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.276 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.277 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.277 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.277 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.277 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.277 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.277 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.277 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.277 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.277 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.277 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.277 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.277 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.277 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.278 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.278 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.278 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.278 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.278 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.278 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.278 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.278 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.278 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.278 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.278 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.278 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.278 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.279 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.279 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.279 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.279 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.279 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_multi_objectv2_delete_key_limit ___________ 2024-06-28T16:14:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.280 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.280 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.280 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.280 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.280 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.280 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.280 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.281 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.281 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.281 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.281 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.281 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.281 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.281 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.281 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.281 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.282 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.283 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.284 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.285 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.286 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.286 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.286 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.286 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.286 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_object_head_zero_bytes _______________ 2024-06-28T16:14:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.287 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.287 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.287 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.287 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.287 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.287 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.287 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.288 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.288 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.288 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.288 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.288 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.288 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.288 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.288 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.288 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.289 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.290 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.291 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.292 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.293 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.293 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.293 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.293 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_object_write_check_etag _______________ 2024-06-28T16:14:09.293 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.294 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.294 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.294 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.294 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.294 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.294 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.294 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.295 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.295 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.295 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.295 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.295 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.295 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.295 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.295 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.296 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.296 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.297 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.298 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.299 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.300 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.300 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.300 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.300 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_object_write_cache_control _____________ 2024-06-28T16:14:09.300 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.301 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.301 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.301 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.301 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.301 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.301 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.301 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.301 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.301 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.301 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.301 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.301 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.301 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.301 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.302 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.302 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.302 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.302 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.302 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.302 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.302 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.302 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.302 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.302 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.302 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.302 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.302 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.303 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.303 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.303 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.303 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.303 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.303 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.303 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.303 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.303 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.303 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.303 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.303 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.303 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.304 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.304 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.304 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.304 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.304 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.304 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.304 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.304 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.304 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.304 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.304 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.304 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.304 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.304 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.305 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.305 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.305 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.305 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.305 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.305 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.305 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.305 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.305 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.305 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.305 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.305 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.305 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.306 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.306 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.306 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.306 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.306 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.306 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.306 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.306 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.306 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.306 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.306 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.306 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.306 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.307 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.307 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.307 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.307 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.307 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.307 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.307 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.307 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.307 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.307 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_object_write_expires ________________ 2024-06-28T16:14:09.307 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.307 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.307 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.308 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.308 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.308 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.308 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.308 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.308 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.308 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.308 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.308 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.308 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.308 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.308 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.308 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.309 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.309 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.309 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.309 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.309 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.309 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.309 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.309 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.309 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.309 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.309 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.309 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.309 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.309 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.310 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.310 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.310 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.310 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.310 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.310 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.310 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.310 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.310 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.310 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.310 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.311 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.311 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.311 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.311 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.311 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.311 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.311 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.311 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.311 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.311 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.311 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.311 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.311 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.312 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.312 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.312 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.312 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.312 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.312 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.312 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.312 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.312 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.312 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.312 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.312 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.312 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.312 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.313 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.313 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.313 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.313 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.313 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.313 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.313 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.313 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.313 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.313 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.313 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.313 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.313 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.314 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.314 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.314 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.314 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.314 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.314 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.314 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.314 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_object_write_read_update_read_delete ________ 2024-06-28T16:14:09.314 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.314 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.314 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.314 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.314 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.315 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.315 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.315 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.315 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.315 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.315 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.315 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.315 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.315 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.315 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.315 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.315 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.315 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.315 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.316 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.316 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.316 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.316 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.316 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.316 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.316 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.316 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.316 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.316 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.316 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.316 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.317 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.317 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.317 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.317 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.317 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.317 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.317 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.317 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.317 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.317 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.317 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.318 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.318 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.318 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.318 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.318 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.318 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.318 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.318 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.318 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.318 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.318 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.318 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.318 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.318 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.319 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.319 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.319 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.319 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.319 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.319 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.319 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.319 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.319 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.319 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.319 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.319 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.320 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.320 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.320 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.320 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.320 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.320 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.320 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.320 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.320 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.320 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.320 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.320 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.321 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.321 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.321 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.321 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.321 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.321 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.321 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_object_set_get_metadata_none_to_good ________ 2024-06-28T16:14:09.321 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.321 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.321 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.321 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.321 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.321 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.322 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.322 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.322 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.322 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.322 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.322 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.322 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.322 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.322 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.322 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.322 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.322 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.322 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.323 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.323 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.323 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.323 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.323 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.323 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.323 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.323 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.323 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.323 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.323 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.324 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.324 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.324 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.324 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.324 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.324 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.324 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.324 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.324 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.324 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.324 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.324 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.324 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.325 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.325 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.325 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.325 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.325 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.325 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.325 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.325 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.325 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.325 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.325 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.325 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.325 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.326 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.326 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.326 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.326 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.326 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.326 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.326 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.326 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.326 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.326 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.326 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.326 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.326 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.327 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.327 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.327 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.327 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.327 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.327 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.327 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.327 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.327 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.327 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.327 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.327 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.328 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.328 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.328 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.328 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.328 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.328 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.328 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.328 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_object_set_get_metadata_none_to_empty ________ 2024-06-28T16:14:09.328 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.328 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.328 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.328 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.328 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.328 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.329 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.329 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.329 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.329 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.329 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.329 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.329 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.329 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.329 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.329 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.329 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.329 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.329 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.329 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.330 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.330 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.330 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.330 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.330 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.330 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.330 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.330 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.330 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.330 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.330 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.330 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.331 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.331 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.331 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.331 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.331 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.331 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.331 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.331 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.331 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.331 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.331 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.331 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.331 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.332 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.332 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.332 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.332 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.332 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.332 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.332 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.332 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.332 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.332 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.332 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.333 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.333 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.333 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.333 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.333 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.333 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.333 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.333 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.333 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.333 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.333 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.333 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.333 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.333 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.334 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.334 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.334 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.334 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.334 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.334 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.334 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.334 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.334 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.334 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.334 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.334 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.335 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.335 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.335 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.335 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.335 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.335 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.335 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.335 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.335 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_object_set_get_metadata_overwrite_to_empty _____ 2024-06-28T16:14:09.335 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.335 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.335 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.335 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.336 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.336 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.336 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.336 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.336 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.336 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.336 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.336 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.336 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.336 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.336 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.336 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.336 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.337 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.337 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.337 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.337 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.337 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.337 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.337 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.337 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.337 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.337 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.337 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.337 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.337 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.338 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.338 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.338 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.338 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.338 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.338 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.338 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.338 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.338 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.338 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.338 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.338 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.338 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.339 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.339 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.339 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.339 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.339 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.339 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.339 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.339 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.339 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.339 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.339 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.339 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.339 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.340 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.340 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.340 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.340 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.340 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.340 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.340 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.340 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.340 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.340 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.340 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.340 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.340 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.340 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.341 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.341 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.341 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.341 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.341 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.341 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.341 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.341 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.341 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.341 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.341 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.341 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.341 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.342 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.342 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.342 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.342 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.342 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.342 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.342 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.342 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_object_metadata_replaced_on_put ___________ 2024-06-28T16:14:09.342 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.342 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.342 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.342 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.345 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.345 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.345 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.345 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.345 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.345 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.345 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.346 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.346 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.346 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.346 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.346 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.346 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.346 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.346 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.346 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.346 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.346 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.346 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.347 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.347 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.347 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.347 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.347 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.347 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.347 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.347 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.347 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.347 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.347 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.347 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.347 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.347 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.348 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.348 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.348 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.348 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.348 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.348 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.348 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.348 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.348 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.348 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.348 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.348 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.348 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.349 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.349 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.349 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.349 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.349 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.349 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.349 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.349 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.349 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.349 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.349 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.349 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.349 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.350 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.350 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.350 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.350 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.350 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.350 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.350 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.350 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.350 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.350 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.350 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.350 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.350 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.351 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.351 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.351 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.351 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.351 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.351 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.351 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.351 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.351 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.351 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.351 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.351 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.351 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.352 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.352 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_object_write_file __________________ 2024-06-28T16:14:09.352 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.352 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.352 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.352 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.352 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.352 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.352 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.352 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.352 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.352 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.352 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.353 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.353 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.353 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.353 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.353 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.353 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.353 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.353 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.353 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.353 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.353 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.353 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.353 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.354 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.354 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.354 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.354 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.354 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.354 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.354 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.354 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.354 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.354 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.354 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.354 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.355 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.355 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.355 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.355 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.355 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.355 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.355 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.355 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.355 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.355 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.355 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.355 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.355 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.355 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.356 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.356 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.356 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.356 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.356 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.356 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.356 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.356 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.356 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.356 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.356 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.356 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.356 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.357 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.357 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.357 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.357 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.357 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.357 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.357 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.357 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.357 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.357 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.357 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.357 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.357 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.358 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.358 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.358 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.358 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.358 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.358 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.358 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.358 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.358 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.358 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.358 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.358 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.358 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.359 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.359 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_post_object_anonymous_request ____________ 2024-06-28T16:14:09.359 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.359 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.359 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.359 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.359 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.359 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.359 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.359 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.359 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.359 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.359 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.360 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.360 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.360 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.360 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.360 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.360 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.360 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.360 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.360 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.360 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.360 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.360 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.360 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.360 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.361 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.361 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.361 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.361 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.361 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.361 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.361 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.361 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.361 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.361 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.361 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.361 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.361 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.362 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.362 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.362 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.362 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.362 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.362 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.362 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.362 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.362 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.362 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.362 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.362 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.362 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.363 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.363 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.363 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.363 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.363 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.363 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.363 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.363 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.363 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.363 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.363 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.363 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.363 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.364 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.364 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.364 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.364 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.364 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.364 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.364 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.364 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.364 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.364 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.364 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.364 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.364 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.364 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.365 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.365 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.365 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.365 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.365 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.365 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.365 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.365 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.365 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.365 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.365 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.365 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.365 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_post_object_authenticated_request __________ 2024-06-28T16:14:09.365 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.366 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.366 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.366 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.366 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.366 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.366 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.366 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.366 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.366 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.366 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.366 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.366 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.366 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.367 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.367 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.367 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.367 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.367 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.367 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.367 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.367 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.367 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.367 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.367 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.367 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.367 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.367 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.368 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.368 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.368 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.368 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.368 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.368 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.368 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.368 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.368 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.368 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.368 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.368 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.368 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.369 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.369 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.369 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.369 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.369 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.369 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.369 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.369 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.369 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.369 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.369 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.369 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.369 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.370 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.370 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.370 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.370 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.370 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.370 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.370 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.370 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.370 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.370 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.370 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.370 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.370 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.371 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.371 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.371 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.371 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.371 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.371 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.371 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.371 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.371 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.371 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.371 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.371 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.371 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.371 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.372 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.372 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.372 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.372 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.372 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.372 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.372 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.372 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.372 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.372 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_post_object_authenticated_no_content_type ______ 2024-06-28T16:14:09.372 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.372 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.372 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.373 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.373 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.373 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.373 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.373 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.373 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.373 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.373 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.373 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.373 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.373 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.373 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.373 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.374 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.374 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.374 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.374 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.374 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.374 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.374 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.374 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.374 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.374 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.374 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.374 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.374 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.375 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.375 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.375 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.375 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.375 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.375 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.375 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.375 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.375 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.375 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.375 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.375 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.375 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.375 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.376 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.376 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.376 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.376 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.376 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.376 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.376 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.376 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.376 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.376 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.376 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.376 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.376 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.377 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.377 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.377 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.377 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.377 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.377 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.377 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.377 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.377 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.377 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.377 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.377 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.377 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.378 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.378 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.378 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.378 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.378 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.378 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.378 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.378 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.378 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.378 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.378 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.378 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.378 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.379 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.379 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.379 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.379 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.379 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.379 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.379 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.379 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.379 INFO:teuthology.orchestra.run.smithi178.stdout:__ ERROR at teardown of test_post_object_authenticated_request_bad_access_key __ 2024-06-28T16:14:09.379 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.379 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.379 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.379 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.379 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.380 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.380 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.380 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.380 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.380 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.380 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.380 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.380 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.380 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.380 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.380 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.380 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.380 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.381 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.381 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.381 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.381 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.381 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.381 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.381 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.381 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.381 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.381 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.381 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.381 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.381 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.381 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.382 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.382 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.382 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.382 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.382 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.382 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.382 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.382 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.382 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.382 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.382 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.382 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.382 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.383 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.383 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.383 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.383 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.383 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.383 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.383 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.383 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.383 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.383 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.383 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.383 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.384 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.384 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.384 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.384 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.384 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.384 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.384 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.384 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.384 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.384 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.384 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.384 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.384 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.384 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.385 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.385 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.385 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.385 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.385 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.385 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.385 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.385 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.385 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.385 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.385 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.385 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.385 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.385 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.386 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.386 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.386 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.386 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.386 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.386 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_post_object_set_success_code ____________ 2024-06-28T16:14:09.386 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.386 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.386 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.386 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.386 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.386 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.387 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.387 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.387 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.387 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.387 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.387 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.387 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.387 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.387 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.387 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.387 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.387 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.387 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.387 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.388 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.388 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.388 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.388 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.388 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.388 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.388 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.388 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.388 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.388 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.388 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.388 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.388 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.389 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.389 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.389 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.389 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.389 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.389 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.389 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.389 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.389 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.389 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.389 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.389 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.389 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.390 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.390 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.390 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.390 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.390 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.390 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.390 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.390 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.390 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.390 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.390 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.390 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.390 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.391 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.391 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.391 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.391 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.391 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.391 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.391 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.391 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.391 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.391 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.391 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.391 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.391 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.392 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.392 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.392 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.392 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.392 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.392 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.392 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.392 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.392 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.392 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.392 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.392 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.392 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.393 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.393 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.393 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.393 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.393 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.393 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_post_object_set_invalid_success_code ________ 2024-06-28T16:14:09.393 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.393 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.393 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.393 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.393 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.393 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.393 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.393 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.394 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.394 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.394 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.394 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.394 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.394 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.394 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.394 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.394 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.394 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.394 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.394 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.394 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.395 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.395 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.395 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.395 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.395 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.395 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.395 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.395 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.395 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.395 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.395 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.395 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.395 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.396 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.396 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.396 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.396 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.396 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.396 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.396 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.396 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.396 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.396 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.396 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.397 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.397 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.397 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.397 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.397 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.397 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.397 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.397 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.397 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.397 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.397 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.397 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.397 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.398 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.398 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.398 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.398 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.398 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.398 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.398 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.398 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.398 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.398 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.398 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.398 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.399 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.399 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.399 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.399 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.399 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.399 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.399 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.399 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.399 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.399 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.399 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.399 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.399 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.399 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.400 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.400 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.400 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.400 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.400 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_post_object_upload_larger_than_chunk ________ 2024-06-28T16:14:09.400 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.400 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.400 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.400 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.400 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.400 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.400 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.401 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.401 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.401 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.401 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.401 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.401 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.401 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.401 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.401 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.401 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.401 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.401 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.401 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.401 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.402 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.402 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.402 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.402 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.402 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.402 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.402 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.402 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.402 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.402 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.402 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.402 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.403 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.403 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.403 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.403 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.403 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.403 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.403 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.403 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.403 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.403 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.403 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.403 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.404 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.404 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.404 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.404 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.404 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.404 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.404 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.404 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.404 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.404 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.404 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.404 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.405 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.405 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.405 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.405 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.405 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.405 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.405 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.405 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.405 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.405 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.405 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.405 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.405 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.405 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.406 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.406 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.406 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.406 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.406 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.406 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.406 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.406 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.406 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.406 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.406 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.406 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.406 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.407 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.407 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.407 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.407 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.407 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_post_object_set_key_from_filename __________ 2024-06-28T16:14:09.407 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.407 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.407 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.407 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.407 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.407 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.407 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.407 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.408 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.408 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.408 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.408 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.408 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.408 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.408 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.408 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.408 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.408 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.408 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.408 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.408 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.409 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.409 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.409 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.409 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.409 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.409 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.409 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.409 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.409 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.409 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.409 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.409 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.409 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.410 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.410 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.410 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.410 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.410 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.410 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.410 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.410 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.410 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.410 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.410 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.410 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.411 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.411 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.411 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.411 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.411 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.411 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.411 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.411 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.411 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.411 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.411 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.411 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.411 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.412 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.412 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.412 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.412 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.412 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.412 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.412 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.412 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.412 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.412 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.412 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.412 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.412 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.412 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.413 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.413 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.413 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.413 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.413 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.413 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.413 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.413 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.413 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.414 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.414 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.414 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.414 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.414 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.414 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.414 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_post_object_ignored_header _____________ 2024-06-28T16:14:09.414 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.414 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.414 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.414 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.414 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.414 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.415 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.415 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.415 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.415 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.415 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.415 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.415 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.415 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.415 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.415 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.415 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.415 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.415 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.415 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.416 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.416 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.416 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.416 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.416 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.416 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.416 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.416 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.416 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.416 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.416 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.416 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.416 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.417 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.417 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.417 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.417 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.417 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.417 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.417 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.417 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.417 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.417 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.417 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.417 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.418 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.418 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.418 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.418 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.418 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.418 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.418 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.418 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.418 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.418 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.418 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.418 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.419 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.419 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.419 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.419 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.419 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.419 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.419 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.419 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.419 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.419 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.419 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.419 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.419 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.419 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.420 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.420 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.420 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.420 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.420 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.420 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.420 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.420 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.420 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.420 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.420 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.420 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.420 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.421 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.421 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.421 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.421 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.421 INFO:teuthology.orchestra.run.smithi178.stdout:___ ERROR at teardown of test_post_object_case_insensitive_condition_fields ____ 2024-06-28T16:14:09.421 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.421 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.421 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.421 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.421 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.421 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.421 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.421 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.422 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.422 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.422 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.422 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.422 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.422 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.422 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.422 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.422 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.422 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.422 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.422 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.423 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.423 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.423 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.423 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.423 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.423 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.423 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.423 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.423 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.423 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.423 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.423 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.423 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.424 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.424 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.424 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.424 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.424 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.424 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.424 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.424 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.424 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.424 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.424 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.424 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.424 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.425 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.425 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.425 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.425 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.425 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.425 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.425 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.425 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.425 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.425 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.425 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.425 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.425 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.426 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.426 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.426 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.426 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.426 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.426 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.426 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.426 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.426 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.426 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.426 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.426 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.426 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.427 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.427 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.427 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.427 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.427 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.427 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.427 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.427 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.427 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.427 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.427 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.427 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.427 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.428 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.428 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.428 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.428 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_post_object_escaped_field_values __________ 2024-06-28T16:14:09.428 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.428 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.428 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.428 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.428 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.428 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.428 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.428 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.428 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.429 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.429 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.429 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.429 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.429 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.429 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.429 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.429 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.429 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.429 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.429 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.429 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.429 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.429 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.430 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.430 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.430 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.434 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.434 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.434 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.434 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.434 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.434 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.434 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.434 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.434 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.434 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.434 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.434 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.434 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.435 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_post_object_success_redirect_action _________ 2024-06-28T16:14:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.435 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.435 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.435 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.435 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.435 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.435 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.436 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.436 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.436 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.436 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.436 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.436 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.436 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.437 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.437 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.437 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.437 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.438 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.439 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.440 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.441 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.441 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.441 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.441 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.442 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_post_object_invalid_signature ____________ 2024-06-28T16:14:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.442 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.442 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.442 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.442 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.442 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.442 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.443 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.443 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.443 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.443 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.443 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.443 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.443 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.444 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.444 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.447 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.448 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.448 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.448 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.448 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.448 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_post_object_invalid_access_key ___________ 2024-06-28T16:14:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.449 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.449 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.449 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.449 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.449 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.449 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.449 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.450 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.450 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.450 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.450 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.450 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.450 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.451 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.451 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.451 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.452 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.453 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.454 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.455 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.455 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.455 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.455 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.455 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_post_object_invalid_date_format ___________ 2024-06-28T16:14:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.456 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.456 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.456 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.456 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.456 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.456 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.456 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.457 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.457 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.457 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.457 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.457 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.457 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.457 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.457 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.457 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.457 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.457 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.457 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.457 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.458 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.458 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.458 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.458 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.458 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.458 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.458 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.458 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.458 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.458 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.458 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.458 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.458 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.459 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.459 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.459 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.459 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.459 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.459 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.459 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.459 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.459 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.459 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.459 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.459 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.459 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.459 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.460 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.460 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.460 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.460 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.460 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.460 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.460 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.460 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.460 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.460 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.460 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.460 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.460 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.461 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.461 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.461 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.461 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.461 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.461 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.461 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.461 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.461 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.461 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.461 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.461 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.461 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.462 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.462 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.462 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.462 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.462 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.462 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.462 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.462 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.462 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.462 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.462 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_post_object_no_key_specified ____________ 2024-06-28T16:14:09.462 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.462 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.463 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.463 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.463 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.463 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.463 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.463 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.463 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.463 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.463 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.463 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.463 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.463 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.463 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.463 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.464 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.464 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.464 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.464 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.464 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.464 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.464 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.464 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.464 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.464 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.464 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.464 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.465 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.465 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.465 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.465 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.465 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.465 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.465 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.465 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.465 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.465 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.465 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.465 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.466 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.466 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.466 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.466 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.466 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.466 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.466 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.466 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.466 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.466 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.466 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.466 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.466 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.467 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.467 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.467 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.467 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.467 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.467 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.467 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.467 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.467 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.467 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.467 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.467 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.467 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.467 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.468 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.468 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.468 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.468 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.468 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.468 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.468 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.468 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.468 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.468 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.468 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.468 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.468 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.469 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.469 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.469 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.469 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.469 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.469 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.469 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.469 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.469 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.469 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_post_object_missing_signature ____________ 2024-06-28T16:14:09.469 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.469 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.469 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.469 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.470 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.470 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.470 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.470 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.470 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.470 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.470 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.470 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.470 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.470 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.470 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.470 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.470 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.471 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.471 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.471 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.471 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.471 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.471 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.471 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.471 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.471 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.471 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.471 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.471 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.471 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.471 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.472 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.472 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.472 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.472 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.472 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.472 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.472 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.472 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.472 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.472 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.472 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.472 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.473 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.473 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.473 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.473 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.473 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.473 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.473 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.473 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.473 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.473 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.473 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.473 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.473 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.474 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.474 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.474 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.474 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.474 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.474 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.474 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.474 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.474 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.474 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.474 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.474 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.475 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.475 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.475 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.475 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.475 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.475 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.475 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.475 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.475 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.475 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.475 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.475 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.475 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.475 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.476 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.476 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.476 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.476 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.476 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.476 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.476 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.476 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_post_object_missing_policy_condition ________ 2024-06-28T16:14:09.476 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.476 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.476 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.476 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.476 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.477 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.477 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.477 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.477 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.477 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.477 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.477 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.477 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.477 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.477 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.477 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.477 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.477 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.478 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.478 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.478 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.478 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.478 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.478 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.478 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.478 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.478 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.478 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.478 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.478 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.479 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.479 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.479 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.479 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.479 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.479 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.479 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.479 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.479 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.479 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.479 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.479 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.479 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.480 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.480 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.480 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.480 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.480 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.480 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.480 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.480 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.480 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.480 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.480 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.480 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.480 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.480 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.481 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.481 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.481 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.481 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.481 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.481 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.481 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.481 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.481 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.481 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.481 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.481 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.481 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.482 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.482 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.482 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.482 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.482 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.482 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.482 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.482 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.482 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.482 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.482 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.482 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.482 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.483 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.483 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.483 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.483 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.483 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.483 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.483 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_post_object_user_specified_header __________ 2024-06-28T16:14:09.483 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.483 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.483 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.483 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.483 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.484 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.484 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.484 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.484 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.484 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.484 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.484 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.484 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.484 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.484 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.484 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.484 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.484 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.484 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.485 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.485 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.485 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.485 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.485 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.485 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.485 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.485 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.485 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.485 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.485 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.485 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.485 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.485 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.486 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.486 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.486 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.486 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.486 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.486 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.486 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.486 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.486 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.486 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.486 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.486 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.486 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.487 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.487 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.487 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.487 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.487 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.487 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.487 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.487 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.487 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.487 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.487 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.487 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.488 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.488 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.488 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.488 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.488 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.488 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.488 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.488 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.488 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.488 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.488 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.488 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.488 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.489 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.489 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.489 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.489 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.489 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.489 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.489 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.489 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.489 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.489 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.489 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.489 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.489 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.490 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.490 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.490 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.490 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.490 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.490 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.490 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_post_object_request_missing_policy_specified_field _ 2024-06-28T16:14:09.490 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.490 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.490 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.490 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.490 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.490 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.491 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.491 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.491 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.491 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.491 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.491 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.491 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.491 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.491 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.491 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.491 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.491 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.491 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.492 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.492 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.492 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.492 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.492 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.492 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.492 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.492 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.492 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.492 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.492 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.492 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.492 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.493 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.493 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.493 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.493 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.493 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.493 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.493 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.493 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.493 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.493 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.493 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.493 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.493 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.493 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.494 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.494 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.494 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.494 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.494 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.494 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.494 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.494 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.494 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.494 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.494 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.494 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.495 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.495 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.495 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.495 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.495 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.495 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.495 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.495 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.495 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.495 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.495 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.495 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.495 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.495 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.496 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.496 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.496 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.496 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.496 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.496 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.496 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.496 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.496 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.496 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.496 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.496 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.496 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.497 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.497 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.497 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.497 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.497 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.497 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_post_object_condition_is_case_sensitive _______ 2024-06-28T16:14:09.497 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.497 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.497 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.497 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.497 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.497 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.497 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.497 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.498 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.498 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.498 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.498 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.498 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.498 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.498 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.498 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.498 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.498 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.498 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.498 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.498 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.499 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.499 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.499 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.499 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.499 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.499 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.499 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.499 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.499 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.499 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.499 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.499 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.499 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.500 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.501 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.501 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.501 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.501 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.501 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.501 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.501 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.501 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.501 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.501 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.501 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.501 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.501 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.502 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.502 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.502 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.502 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.502 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.502 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.502 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.502 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.502 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.502 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.502 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.502 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.502 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.503 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.503 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.503 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.503 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.503 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.503 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.503 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.503 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.503 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.503 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.503 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.503 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.503 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.504 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.504 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.504 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_post_object_expires_is_case_sensitive ________ 2024-06-28T16:14:09.504 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.504 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.504 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.504 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.504 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.504 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.504 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.504 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.504 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.504 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.505 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.505 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.505 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.505 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.505 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.505 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.505 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.505 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.505 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.505 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.505 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.505 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.506 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.506 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.506 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.506 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.506 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.506 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.506 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.506 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.506 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.506 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.506 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.506 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.506 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.507 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.507 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.507 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.507 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.507 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.507 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.507 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.507 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.507 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.507 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.507 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.507 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.508 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.508 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.508 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.508 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.508 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.508 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.508 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.508 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.508 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.508 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.508 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.508 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.508 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.508 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.509 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.509 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.509 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.509 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.509 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.509 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.509 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.509 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.509 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.509 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.509 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.509 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.509 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.510 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.510 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.510 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.510 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.510 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.510 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.510 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.510 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.510 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.510 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.510 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.510 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.511 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.511 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.511 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.511 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.511 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_post_object_expired_policy _____________ 2024-06-28T16:14:09.511 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.511 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.511 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.511 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.511 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.511 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.511 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.511 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.512 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.512 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.512 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.512 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.512 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.512 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.512 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.512 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.512 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.512 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.512 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.512 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.512 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.513 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.513 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.513 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.513 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.513 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.513 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.513 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.513 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.513 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.513 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.513 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.513 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.513 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.514 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.514 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.514 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.514 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.514 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.514 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.514 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.514 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.514 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.514 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.514 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.514 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.514 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.515 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.515 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.515 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.515 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.515 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.515 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.515 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.515 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.515 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.515 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.515 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.515 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.516 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.516 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.516 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.516 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.516 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.516 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.516 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.516 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.516 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.516 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.516 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.516 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.516 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.517 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.517 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.517 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.517 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.517 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.517 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.517 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.517 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.517 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.517 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.517 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.517 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.518 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.518 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.518 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.518 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.518 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.518 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.518 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_post_object_wrong_bucket ______________ 2024-06-28T16:14:09.518 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.518 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.518 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.518 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.518 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.518 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.519 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.519 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.519 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.519 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.519 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.519 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.519 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.519 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.519 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.519 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.519 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.519 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.520 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.520 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.520 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.520 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.520 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.520 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.520 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.520 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.520 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.520 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.520 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.520 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.520 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.520 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.521 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.521 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.521 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.521 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.521 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.521 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.521 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.521 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.521 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.521 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.521 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.521 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.521 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.522 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.522 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.522 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.522 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.522 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.522 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.522 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.522 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.522 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.522 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.522 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.522 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.523 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.523 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.523 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.523 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.523 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.523 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.523 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.523 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.523 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.523 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.523 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.523 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.523 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.524 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.524 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.524 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.524 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.524 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.524 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.524 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.524 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.524 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.524 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.524 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.524 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.524 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.525 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.525 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.525 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.525 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.525 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.525 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.525 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.525 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_post_object_invalid_request_field_value _______ 2024-06-28T16:14:09.525 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.525 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.525 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.525 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.525 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.526 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.526 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.526 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.526 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.526 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.526 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.526 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.526 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.526 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.526 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.526 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.526 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.526 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.527 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.527 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.527 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.527 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.527 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.527 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.527 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.527 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.527 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.527 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.527 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.527 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.527 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.528 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.528 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.528 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.528 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.528 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.528 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.528 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.528 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.528 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.528 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.528 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.528 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.528 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.529 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.530 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.531 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.532 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.549 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.549 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.549 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.549 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.550 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.550 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.550 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_post_object_missing_expires_condition ________ 2024-06-28T16:14:09.550 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.550 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.550 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.550 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.550 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.550 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.550 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.550 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.550 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.551 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.551 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.551 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.551 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.551 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.551 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.551 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.551 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.551 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.551 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.551 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.551 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.551 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.552 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.553 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.553 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.553 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.553 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.553 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.553 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.554 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.554 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.554 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.554 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.554 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.554 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.554 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.554 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.554 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.554 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.554 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.555 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.555 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.555 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.555 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.555 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.555 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.555 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.555 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.555 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.555 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.555 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.555 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.555 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.556 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.556 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.556 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.556 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.556 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.556 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.556 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.556 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.556 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.556 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.556 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.556 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.556 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.557 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.557 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.557 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.557 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.557 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.557 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.557 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.557 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.557 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.557 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.557 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.557 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.557 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.558 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.558 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.558 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.558 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.558 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.558 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.558 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.558 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.558 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.558 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.558 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.558 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_post_object_missing_conditions_list _________ 2024-06-28T16:14:09.558 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.558 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.559 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.559 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.559 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.559 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.559 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.559 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.559 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.559 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.559 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.559 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.559 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.559 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.559 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.559 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.560 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.560 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.560 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.560 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.560 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.560 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.560 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.560 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.560 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.560 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.560 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.560 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.560 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.560 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.561 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.561 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.561 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.561 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.561 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.561 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.561 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.561 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.561 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.561 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.561 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.561 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.562 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.562 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.562 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.562 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.562 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.562 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.562 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.562 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.562 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.562 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.562 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.562 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.562 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.563 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.563 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.563 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.563 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.563 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.563 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.563 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.563 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.563 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.563 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.563 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.563 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.563 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.563 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.564 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.564 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.564 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.564 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.564 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.564 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.564 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.564 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.564 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.564 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.564 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.564 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.564 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.564 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.565 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.565 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.565 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.565 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.565 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.565 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_post_object_upload_size_limit_exceeded _______ 2024-06-28T16:14:09.565 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.565 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.565 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.565 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.565 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.566 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.566 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.566 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.566 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.566 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.566 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.566 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.566 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.566 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.566 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.566 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.566 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.566 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.567 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.567 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.567 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.567 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.567 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.567 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.567 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.567 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.567 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.567 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.567 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.567 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.567 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.568 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.568 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.568 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.568 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.568 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.568 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.568 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.568 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.568 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.568 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.568 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.568 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.569 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.569 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.569 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.569 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.569 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.569 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.569 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.569 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.569 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.569 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.569 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.569 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.569 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.569 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.570 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.570 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.570 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.570 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.570 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.570 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.570 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.570 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.570 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.570 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.570 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.570 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.570 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.571 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.571 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.571 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.571 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.571 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.571 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.571 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.571 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.571 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.571 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.571 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.571 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.571 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.572 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.572 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.572 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.572 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.572 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.572 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.572 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_post_object_missing_content_length_argument _____ 2024-06-28T16:14:09.572 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.572 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.572 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.572 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.572 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.572 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.573 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.573 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.573 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.573 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.573 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.573 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.573 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.573 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.573 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.573 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.573 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.573 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.573 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.573 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.574 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.574 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.574 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.574 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.574 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.574 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.574 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.574 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.574 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.574 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.574 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.574 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.575 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.575 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.575 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.575 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.575 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.575 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.575 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.575 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.575 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.575 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.575 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.575 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.576 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.577 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.578 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.578 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.578 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.578 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.578 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.578 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.578 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.578 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.578 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.578 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.578 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.578 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.578 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.579 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.579 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.579 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.579 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.579 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_post_object_invalid_content_length_argument _____ 2024-06-28T16:14:09.579 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.579 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.579 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.579 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.579 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.579 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.579 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.579 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.579 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.580 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.580 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.580 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.580 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.580 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.580 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.580 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.580 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.580 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.580 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.581 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.581 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.581 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.581 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.581 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.581 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.581 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.581 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.581 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.581 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.581 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.581 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.581 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.582 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.582 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.582 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.582 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.582 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.582 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.582 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.582 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.582 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.582 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.582 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.583 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.583 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.583 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.583 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.583 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.583 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.583 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.583 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.583 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.583 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.583 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.583 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.583 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.584 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.584 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.584 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.584 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.584 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.584 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.584 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.584 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.584 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.584 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.584 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.584 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.584 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.585 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.585 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.585 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.585 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.585 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.585 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.585 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.585 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.585 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.585 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.585 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.585 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.585 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.586 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.586 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.586 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.586 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.586 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.586 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.586 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_post_object_upload_size_below_minimum ________ 2024-06-28T16:14:09.586 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.586 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.586 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.586 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.586 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.586 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.587 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.587 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.587 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.587 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.587 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.587 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.587 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.587 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.587 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.587 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.587 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.587 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.587 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.588 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.588 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.588 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.588 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.588 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.588 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.588 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.588 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.588 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.588 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.588 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.588 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.588 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.589 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.589 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.589 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.589 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.589 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.589 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.589 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.589 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.589 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.589 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.589 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.589 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.590 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.590 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.590 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.590 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.590 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.590 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.590 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.590 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.590 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.590 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.590 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.590 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.590 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.591 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.591 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.591 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.591 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.591 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.591 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.591 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.591 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.591 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.591 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.591 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.591 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.591 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.592 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.592 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.592 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.592 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.592 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.592 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.592 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.592 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.592 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.592 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.592 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.592 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.592 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.592 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.593 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.593 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.593 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.593 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.593 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_post_object_upload_size_rgw_chunk_size_bug _____ 2024-06-28T16:14:09.593 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.593 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.593 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.593 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.593 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.593 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.593 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.593 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.594 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.594 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.594 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.594 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.594 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.594 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.594 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.594 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.594 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.594 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.594 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.594 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.594 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.595 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.595 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.595 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.595 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.595 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.595 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.595 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.595 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.595 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.595 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.595 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.595 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.596 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.596 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.596 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.596 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.596 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.596 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.596 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.596 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.597 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.597 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.597 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.597 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.597 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.597 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.597 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.597 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.597 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.597 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.597 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.597 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.597 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.598 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.598 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.598 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.598 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.598 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.598 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.598 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.598 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.598 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.598 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.598 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.598 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.598 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.599 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.600 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.600 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.600 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.600 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_post_object_empty_conditions ____________ 2024-06-28T16:14:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.600 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.600 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.601 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.601 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.601 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.601 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.601 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.601 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.601 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.601 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.602 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.602 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.602 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.602 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.602 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.602 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.602 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.603 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.604 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.605 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.606 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.607 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.607 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.607 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.607 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_get_object_ifmatch_good _______________ 2024-06-28T16:14:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.607 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.607 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.608 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.608 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.608 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.608 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.608 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.608 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.608 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.608 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.608 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.609 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.609 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.609 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.609 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.609 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.610 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.611 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.612 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.613 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.614 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.614 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.614 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.614 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_get_object_ifmatch_failed ______________ 2024-06-28T16:14:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.614 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.614 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.615 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.615 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.615 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.615 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.615 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.615 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.615 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.615 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.616 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.616 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.616 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.616 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.616 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.617 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.618 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.619 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.620 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.620 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.620 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.621 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.621 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_get_object_ifnonematch_good _____________ 2024-06-28T16:14:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.621 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.621 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.621 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.621 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.621 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.621 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.622 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.622 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.622 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.622 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.622 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.622 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.622 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.622 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.623 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.623 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.623 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.624 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.624 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.624 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.624 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.624 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.624 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.624 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.624 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.624 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.624 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.624 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.624 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.624 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.625 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.625 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.625 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.625 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.625 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.625 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.625 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.625 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.625 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.625 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.625 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.626 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.626 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.626 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.626 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.626 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.626 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.626 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.626 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.626 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.626 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.626 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.626 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.626 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.627 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.627 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.627 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.627 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.627 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.627 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.627 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.627 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.627 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.627 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.627 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.627 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.627 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.627 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.628 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_get_object_ifnonematch_failed ____________ 2024-06-28T16:14:09.628 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.628 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.628 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.628 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.628 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.628 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.628 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.628 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.628 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.628 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.628 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.628 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.628 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.629 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.629 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.629 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.629 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.629 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.629 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.629 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.629 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.629 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.629 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.629 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.629 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.629 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.629 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.630 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.630 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.630 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.630 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.630 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.630 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.630 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.630 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.630 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.630 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.630 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.630 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.630 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.631 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.631 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.631 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.631 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.631 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.631 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.631 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.631 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.631 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.631 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.631 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.631 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.631 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.631 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.632 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.632 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.632 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.632 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.632 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.632 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.632 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.632 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.632 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.632 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.632 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.632 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.632 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.633 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.633 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.633 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.633 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.633 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.633 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.633 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.633 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.633 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.633 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.633 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.633 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.633 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.633 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.634 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.634 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.634 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.634 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.634 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.634 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.634 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.634 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.634 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.634 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_get_object_ifmodifiedsince_good ___________ 2024-06-28T16:14:09.634 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.634 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.634 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.635 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.635 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.635 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.635 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.635 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.635 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.635 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.635 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.635 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.635 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.635 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.635 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.635 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.635 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.636 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.636 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.636 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.636 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.636 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.636 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.636 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.636 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.636 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.636 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.636 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.636 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.636 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.636 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.637 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.638 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.638 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.638 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.638 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.638 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.638 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.638 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.638 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.638 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.638 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.638 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.638 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.638 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.639 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.639 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.639 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.639 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.639 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.639 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.639 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.639 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.639 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.639 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.639 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.640 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.640 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.640 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.640 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.640 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.640 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.640 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.640 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.640 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.640 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.640 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.640 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.640 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.640 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.641 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.641 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.641 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.641 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.641 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.641 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.641 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_get_object_ifmodifiedsince_failed __________ 2024-06-28T16:14:09.641 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.641 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.641 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.641 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.641 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.641 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.641 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.642 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.642 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.642 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.642 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.642 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.642 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.642 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.642 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.642 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.642 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.642 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.642 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.643 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.643 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.643 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.643 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.643 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.643 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.643 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.643 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.643 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.643 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.643 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.643 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.643 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.644 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.644 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.644 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.644 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.644 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.644 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.644 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.644 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.644 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.644 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.644 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.644 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.644 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.644 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.645 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.646 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.646 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.646 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.646 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.646 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.646 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.646 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.646 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.646 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.646 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.646 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.646 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.646 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.647 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.647 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.647 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.647 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.647 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.647 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.647 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.647 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.647 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.647 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.647 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.647 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.647 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.647 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_get_object_ifunmodifiedsince_good __________ 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.648 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.649 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.649 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.649 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.649 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.649 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.649 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.649 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.649 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.649 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.649 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.649 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.649 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.649 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.650 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.650 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.650 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.650 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.650 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.650 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.650 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.650 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.650 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.650 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.650 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.651 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.651 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.651 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.651 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.651 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.651 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.651 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.651 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.651 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.651 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.651 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.651 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.651 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.651 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.652 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.652 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.652 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.652 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.652 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.652 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.652 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.652 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.652 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.652 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.652 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.652 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.652 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.652 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.653 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.653 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.653 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.653 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.653 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.653 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.653 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.653 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.653 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.653 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.653 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.653 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.653 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.653 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.654 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.654 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.654 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.654 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.654 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.654 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.654 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.654 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.654 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.654 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_get_object_ifunmodifiedsince_failed _________ 2024-06-28T16:14:09.654 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.655 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.655 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.655 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.655 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.655 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.655 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.655 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.655 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.655 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.655 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.655 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.655 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.655 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.656 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.656 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.656 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.656 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.656 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.656 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.656 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.656 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.656 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.656 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.656 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.656 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.657 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.657 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.657 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.657 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.657 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.657 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.657 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.657 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.657 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.657 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.657 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.657 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.657 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.657 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.658 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.658 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.658 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.658 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.658 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.658 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.658 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.658 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.658 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.658 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.658 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.658 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.658 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.659 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.659 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.659 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.659 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.659 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.659 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.659 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.659 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.659 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.659 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.659 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.659 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.659 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.659 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.660 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.660 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.660 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.660 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.660 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.660 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.660 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.660 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.660 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.660 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.660 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.660 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.660 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_put_object_ifmatch_good _______________ 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.661 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.662 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.662 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.662 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.662 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.662 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.662 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.662 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.662 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.662 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.662 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.662 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.662 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.663 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.663 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.663 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.663 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.663 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.663 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.663 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.663 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.663 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.663 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.663 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.663 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.663 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.663 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.664 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.664 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.664 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.664 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.664 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.664 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.664 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.664 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.664 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.664 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.664 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.665 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.665 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.665 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.665 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.665 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.665 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.665 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.665 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.665 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.665 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.665 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.665 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.665 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.665 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.666 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.666 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.666 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.666 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.666 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.666 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.666 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.666 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.666 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.666 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.666 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.666 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.667 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.667 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.667 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.667 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.667 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.667 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.667 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.667 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.667 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.667 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.667 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.667 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.667 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.668 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.668 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.668 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.668 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.668 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.668 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.668 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.668 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.668 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.668 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_put_object_ifmatch_failed ______________ 2024-06-28T16:14:09.668 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.668 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.668 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.669 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.669 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.669 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.669 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.669 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.669 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.669 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.669 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.669 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.669 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.669 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.670 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.670 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.670 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.670 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.670 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.670 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.670 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.670 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.670 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.670 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.670 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.671 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.671 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.671 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.671 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.671 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.671 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.671 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.671 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.671 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.671 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.671 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.671 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.672 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.672 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.672 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.672 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.672 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.672 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.672 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.672 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.672 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.672 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.672 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.672 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.672 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.672 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.673 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.673 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.673 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.673 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.673 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.673 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.673 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.673 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.673 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.673 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.673 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.673 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.673 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.674 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.674 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.674 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.674 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.674 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.674 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.674 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.674 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.674 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.674 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.674 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.674 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.674 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.675 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.675 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.675 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.675 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.675 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.675 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.675 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.675 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.675 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.675 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.675 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.675 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.675 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_put_object_ifmatch_overwrite_existed_good ______ 2024-06-28T16:14:09.676 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.676 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.676 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.676 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.676 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.676 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.676 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.676 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.676 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.676 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.676 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.676 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.676 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.676 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.677 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.677 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.677 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.677 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.677 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.677 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.677 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.677 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.677 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.677 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.677 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.677 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.677 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.678 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.678 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.678 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.678 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.678 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.678 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.678 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.678 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.678 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.678 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.678 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.678 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.678 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.679 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.679 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.679 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.679 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.679 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.679 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.679 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.679 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.679 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.679 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.679 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.679 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.679 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.679 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.680 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.680 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.680 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.680 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.680 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.680 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.680 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.680 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.680 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.680 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.680 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.680 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.680 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.680 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.681 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.681 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.681 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.681 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.681 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.681 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.681 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.681 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.681 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.681 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.681 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.681 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.681 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.682 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.682 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.682 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.682 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.682 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.682 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.682 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.682 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.682 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_put_object_ifmatch_nonexisted_failed ________ 2024-06-28T16:14:09.682 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.682 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.682 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.682 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.683 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.683 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.683 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.683 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.683 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.683 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.683 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.683 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.683 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.683 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.683 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.683 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.683 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.683 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.684 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.684 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.684 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.684 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.684 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.684 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.684 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.684 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.684 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.684 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.684 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.685 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.685 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.685 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.685 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.685 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.685 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.685 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.685 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.685 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.685 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.685 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.685 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.685 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.685 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.686 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.686 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.686 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.686 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.686 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.686 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.686 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.686 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.686 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.686 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.686 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.686 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.686 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.687 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.687 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.687 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.687 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.687 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.687 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.687 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.687 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.687 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.687 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.687 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.687 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.687 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.687 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.688 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.688 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.688 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.688 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.688 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.688 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.688 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.688 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.688 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.688 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.688 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.688 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.688 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.689 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.689 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.689 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.689 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.689 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.689 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.689 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.689 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_put_object_ifnonmatch_good _____________ 2024-06-28T16:14:09.689 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.689 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.689 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.689 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.689 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.690 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.690 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.690 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.690 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.690 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.690 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.690 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.690 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.690 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.690 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.690 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.690 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.690 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.691 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.691 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.691 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.691 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.691 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.691 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.691 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.691 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.691 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.691 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.691 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.691 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.691 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.691 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.692 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.692 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.692 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.692 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.692 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.692 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.692 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.692 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.692 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.692 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.692 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.692 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.692 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.692 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.693 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.693 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.693 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.693 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.693 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.693 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.693 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.693 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.693 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.693 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.693 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.693 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.693 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.694 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.694 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.694 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.694 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.694 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.694 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.694 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.694 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.694 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.694 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.694 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.694 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.694 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.695 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.695 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.695 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.695 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.695 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.695 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.695 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.695 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.695 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.695 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.695 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.695 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.695 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.695 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.696 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.696 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.696 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.696 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.696 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_put_object_ifnonmatch_failed ____________ 2024-06-28T16:14:09.696 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.696 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.696 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.696 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.696 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.696 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.696 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.696 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.697 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.697 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.697 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.697 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.697 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.697 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.697 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.697 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.697 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.697 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.697 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.697 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.697 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.697 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.698 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.698 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.698 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.698 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.698 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.698 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.698 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.698 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.698 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.698 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.698 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.699 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.700 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.700 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.700 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.700 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.700 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.700 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.700 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.700 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.700 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.700 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.700 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.700 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.700 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.701 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.701 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.701 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.701 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.701 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.701 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.701 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.701 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.701 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.701 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.701 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.701 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.701 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.701 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.702 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.702 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.702 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.702 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.702 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.702 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.702 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.702 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.702 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.702 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.702 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.702 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.702 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.703 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.703 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.703 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_put_object_ifnonmatch_nonexisted_good ________ 2024-06-28T16:14:09.703 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.703 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.703 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.703 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.703 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.703 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.703 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.703 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.703 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.703 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.703 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.704 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.704 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.704 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.704 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.704 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.704 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.704 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.704 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.704 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.704 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.704 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.704 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.704 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.704 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.705 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.705 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.705 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.705 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.705 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.705 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.705 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.705 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.705 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.705 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.705 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.705 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.705 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.705 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.706 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.706 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.706 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.706 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.706 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.706 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.706 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.706 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.706 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.706 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.706 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.706 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.707 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.708 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.708 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.708 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.708 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.708 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.708 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.708 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.708 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.708 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.708 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.708 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.708 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.708 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.708 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.709 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.709 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.709 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.709 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.709 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.709 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.709 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.709 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.709 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.709 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.709 INFO:teuthology.orchestra.run.smithi178.stdout:___ ERROR at teardown of test_put_object_ifnonmatch_overwrite_existed_failed ___ 2024-06-28T16:14:09.709 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.709 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.709 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.710 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.710 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.710 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.710 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.710 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.710 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.710 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.710 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.710 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.710 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.710 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.710 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.710 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.710 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.711 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.711 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.711 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.711 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.711 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.711 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.711 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.711 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.711 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.711 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.711 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.711 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.711 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.712 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.713 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.713 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.713 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.713 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.713 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.713 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.713 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.713 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.713 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.713 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.713 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.713 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.714 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.714 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.714 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.714 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.714 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.714 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.714 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.714 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.714 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.714 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.714 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.714 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.715 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.715 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.715 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.715 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.715 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.715 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.715 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.715 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.715 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.715 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.715 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.715 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.715 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.716 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.716 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.716 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.716 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.716 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.716 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.716 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.716 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.716 INFO:teuthology.orchestra.run.smithi178.stdout:___________________ ERROR at teardown of test_object_raw_get ___________________ 2024-06-28T16:14:09.716 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.716 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.716 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.716 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.716 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.717 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.717 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.717 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.717 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.717 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.717 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.717 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.717 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.717 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.717 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.717 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.717 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.717 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.717 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.718 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.718 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.718 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.718 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.718 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.718 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.718 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.718 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.718 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.718 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.718 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.718 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.718 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.719 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.719 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.719 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.719 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.719 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.719 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.719 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.719 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.719 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.719 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.719 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.719 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.719 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.719 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.720 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.721 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.721 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.721 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.721 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.721 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.721 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.721 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.721 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.721 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.721 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.721 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.721 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.721 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.721 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.722 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.722 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.722 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.722 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.722 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.722 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.722 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.722 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.722 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.722 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.722 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.722 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.722 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.723 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.723 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.723 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_object_raw_get_bucket_gone _____________ 2024-06-28T16:14:09.723 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.723 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.723 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.723 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.723 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.723 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.723 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.723 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.723 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.723 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.723 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.724 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.725 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.725 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.725 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.725 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.725 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.725 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.725 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.725 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.725 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.725 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.725 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.725 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.725 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.725 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.726 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.726 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.726 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.726 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.726 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.726 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.726 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.726 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.726 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.726 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.726 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.726 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.726 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.727 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.727 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.727 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.727 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.727 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.727 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.727 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.727 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.727 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.727 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.727 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.728 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.728 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.728 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.728 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.728 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.728 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.728 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.728 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.728 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.728 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.728 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.728 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.728 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.729 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.729 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.729 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.729 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.729 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.729 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.729 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.729 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.729 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.729 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.729 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.729 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.729 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.729 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_object_delete_key_bucket_gone ____________ 2024-06-28T16:14:09.730 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.730 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.730 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.730 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.730 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.730 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.730 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.730 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.730 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.730 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.730 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.730 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.730 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.731 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.731 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.731 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.731 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.731 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.731 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.731 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.731 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.731 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.731 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.731 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.731 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.731 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.732 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.732 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.732 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.732 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.732 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.732 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.732 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.732 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.732 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.732 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.732 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.732 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.732 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.732 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.733 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.734 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.734 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.734 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.734 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.734 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.734 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.734 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.734 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.734 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.734 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.734 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.734 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.734 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.735 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.735 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.735 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.735 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.735 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.735 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.735 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.735 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.735 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.735 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.735 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.735 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.735 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.735 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_object_raw_get_object_gone _____________ 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.736 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.737 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.738 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.738 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.738 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.738 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.738 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.738 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.738 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.738 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.738 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.738 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.738 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.738 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.738 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.739 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.739 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.739 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.739 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.739 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.739 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.740 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.740 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.740 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.740 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.740 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.740 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.740 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.740 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.740 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.740 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.740 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.740 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.740 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.741 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.741 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.741 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.741 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.741 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.741 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.741 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.741 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.741 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.741 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.741 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.741 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.741 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.742 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.742 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.742 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.742 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.742 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.742 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.742 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.742 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.742 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.742 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.742 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.742 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.742 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.743 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.743 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.743 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.743 INFO:teuthology.orchestra.run.smithi178.stdout:____________________ ERROR at teardown of test_bucket_head _____________________ 2024-06-28T16:14:09.743 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.743 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.743 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.743 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.743 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.743 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.743 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.743 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.743 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.744 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.745 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.745 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.745 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.745 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.745 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.745 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.745 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.745 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.745 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.745 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.745 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.745 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.745 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.745 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.746 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.746 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.746 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.746 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.746 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.746 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.746 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.746 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.746 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.746 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.746 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.746 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.746 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.747 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.748 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.748 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.748 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.748 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.748 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.748 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.748 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.748 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.748 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.748 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.748 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.748 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.748 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.748 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.749 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.749 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.749 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.749 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.749 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.749 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.749 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.749 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.749 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_bucket_head_notexist ________________ 2024-06-28T16:14:09.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.749 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.750 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.750 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.750 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.750 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.750 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.750 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.750 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.750 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.750 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.750 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.750 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.750 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.750 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.751 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.751 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.751 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.751 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.751 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.751 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.751 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.751 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.751 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.751 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.751 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.751 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.751 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.752 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.752 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.752 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.752 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.752 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.752 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.752 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.752 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.752 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.752 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.752 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.752 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.752 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.753 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.754 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.754 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.754 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.754 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.754 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.754 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.754 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.754 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.754 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.754 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.754 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.754 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.754 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.754 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.755 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.756 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.756 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.756 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.756 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.756 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_bucket_head_extended ________________ 2024-06-28T16:14:09.756 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.756 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.756 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.756 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.756 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.756 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.756 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.757 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.757 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.757 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.757 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.757 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.757 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.757 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.757 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.758 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.758 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.758 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.758 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.758 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.758 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.759 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.760 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.761 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.762 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_object_raw_get_bucket_acl ______________ 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.763 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.764 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.764 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.764 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.764 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.764 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.764 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.764 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.764 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.764 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.765 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.766 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.767 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.768 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_object_raw_get_object_acl ______________ 2024-06-28T16:14:09.769 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.770 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.770 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.770 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.770 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.770 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.770 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.771 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.771 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.771 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.771 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.771 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.771 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.771 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.771 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.772 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.772 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.772 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.773 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.774 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.775 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_object_put_acl_mtime ________________ 2024-06-28T16:14:09.776 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.777 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.777 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.777 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.777 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.777 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.777 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.777 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.777 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.778 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.778 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.778 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.778 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.778 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.778 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.778 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.778 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.779 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.780 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.781 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.781 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.781 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.781 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.781 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.781 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.781 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.781 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.781 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.781 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.781 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.781 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.781 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.781 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.782 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.782 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.782 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.782 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.782 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.782 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.782 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.782 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.782 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.782 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.782 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.782 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.782 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.783 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.783 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.783 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.783 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.783 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.783 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.783 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.783 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.783 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_object_raw_authenticated ______________ 2024-06-28T16:14:09.783 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.783 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.783 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.783 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.783 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.784 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.784 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.784 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.784 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.784 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.784 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.784 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.784 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.784 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.784 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.784 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.784 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.784 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.785 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.785 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.785 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.785 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.785 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.785 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.785 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.785 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.785 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.785 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.785 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.785 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.786 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.786 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.786 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.786 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.786 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.786 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.787 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.787 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.787 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.787 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.787 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.787 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.787 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.787 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.787 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.787 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.787 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.787 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.787 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.788 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.788 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.788 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.788 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.788 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.788 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.788 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.788 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.788 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.788 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.788 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.788 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.788 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.788 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.789 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.789 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.789 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.789 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.789 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.789 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.789 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.789 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.789 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.789 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.789 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.789 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.789 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.790 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.790 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.790 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.790 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.790 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.790 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.790 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.790 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.790 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.790 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_object_raw_response_headers _____________ 2024-06-28T16:14:09.790 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.790 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.791 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.791 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.791 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.791 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.791 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.791 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.791 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.791 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.791 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.791 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.791 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.791 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.791 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.791 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.792 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.792 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.792 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.792 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.792 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.792 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.792 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.792 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.792 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.792 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.792 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.792 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.792 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.793 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.793 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.793 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.793 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.793 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.793 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.794 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.794 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.794 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.794 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.794 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.794 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.794 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.794 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.794 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.794 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.794 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.794 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.794 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.794 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.795 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.795 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.795 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.795 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.795 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.795 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.795 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.795 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.795 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.795 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.795 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.795 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.795 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.796 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.797 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.797 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.797 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.797 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.797 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.797 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.797 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_object_raw_authenticated_bucket_acl _________ 2024-06-28T16:14:09.797 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.797 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.797 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.797 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.797 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.797 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.797 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.798 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.798 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.798 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.798 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.798 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.798 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.798 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.798 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.798 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.798 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.798 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.798 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.798 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.799 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.799 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.799 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.799 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.799 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.799 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.799 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.799 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.799 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.799 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.799 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.800 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.800 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.800 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.800 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.800 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.800 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.800 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.800 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.800 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.800 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.800 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.801 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.801 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.801 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.801 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.801 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.801 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.801 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.801 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.801 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.801 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.801 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.801 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.801 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.802 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.802 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.802 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.802 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.802 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.802 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.802 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.802 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.802 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.802 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.802 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.802 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.802 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.803 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.803 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.803 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.803 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.803 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.803 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.803 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.803 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.803 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.803 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.803 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.803 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.803 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.804 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.804 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.804 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.804 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.804 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.804 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.804 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.804 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.804 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_object_raw_authenticated_object_acl _________ 2024-06-28T16:14:09.804 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.804 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.804 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.804 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.804 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.805 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.805 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.805 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.805 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.805 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.805 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.805 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.805 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.805 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.805 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.805 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.805 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.805 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.806 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.806 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.806 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.806 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.806 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.806 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.806 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.806 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.806 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.806 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.806 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.806 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.806 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.807 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.807 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.807 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.807 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.807 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.807 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.807 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.807 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.807 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.807 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.807 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.807 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.807 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.808 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.808 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.808 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.808 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.808 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.808 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.808 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.808 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.808 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.808 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.808 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.808 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.808 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.808 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.809 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.809 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.809 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.809 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.809 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.809 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.809 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.809 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.809 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.809 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.809 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.809 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.809 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.809 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.810 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.810 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.810 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.810 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.810 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.810 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.810 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.810 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.810 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.810 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.810 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.810 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.810 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.811 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.811 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.811 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.811 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_object_raw_authenticated_bucket_gone ________ 2024-06-28T16:14:09.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.811 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.811 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.811 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.811 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.811 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.811 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.812 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.812 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.812 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.812 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.812 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.812 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.812 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.812 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.812 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.813 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.813 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.813 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.813 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.813 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.813 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.813 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.813 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.813 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.813 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.813 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.813 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.813 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.813 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.814 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.814 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.814 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.814 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.814 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.814 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.814 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.814 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.814 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.815 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.815 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.815 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.815 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.815 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.815 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.815 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.815 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.815 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.815 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.815 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.815 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.815 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.817 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.817 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.817 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.817 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.817 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.817 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.817 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.817 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.817 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.817 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.817 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.817 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.817 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.817 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.818 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.818 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.818 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_object_raw_authenticated_object_gone ________ 2024-06-28T16:14:09.818 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.818 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.818 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.818 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.818 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.818 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.818 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.818 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.818 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.818 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.819 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.819 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.819 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.819 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.819 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.819 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.819 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.819 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.819 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.820 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.820 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.820 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.820 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.820 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.820 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.820 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.820 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.820 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.820 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.820 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.820 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.821 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.821 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.821 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.821 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.821 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.821 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.821 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.821 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.822 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.822 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.822 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.822 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.822 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.822 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.822 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.822 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.822 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.822 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.822 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.822 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.823 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.823 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.823 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.823 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.823 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.823 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.823 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.823 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.823 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.823 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.823 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.823 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.823 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.823 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.824 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.824 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.824 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.824 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.824 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.824 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.824 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.824 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.824 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.824 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.824 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.824 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.824 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.824 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_object_raw_get_x_amz_expires_not_expired ______ 2024-06-28T16:14:09.825 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.825 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.825 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.825 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.825 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.825 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.825 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.825 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.825 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.825 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.825 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.825 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.825 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.826 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.826 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.826 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.826 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.826 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.826 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.826 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.826 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.826 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.826 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.826 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.826 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.826 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.827 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.827 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.827 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.827 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.827 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.827 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.827 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.827 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.827 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.827 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.827 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.828 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.828 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.828 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.828 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.828 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.828 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.828 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.828 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.829 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.829 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.829 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.829 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.829 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.829 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.829 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.829 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.829 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.829 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.829 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.829 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.830 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.830 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.830 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.830 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.830 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.830 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.830 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.830 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.830 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.830 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.830 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.830 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.831 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.831 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.831 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.831 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.831 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.831 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.831 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.831 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.831 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.831 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.831 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.831 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.831 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.832 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.832 INFO:teuthology.orchestra.run.smithi178.stdout:__ ERROR at teardown of test_object_raw_get_x_amz_expires_not_expired_tenant ___ 2024-06-28T16:14:09.832 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.832 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.832 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.832 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.832 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.832 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.832 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.832 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.832 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.832 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.832 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.833 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.833 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.833 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.833 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.833 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.833 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.833 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.833 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.833 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.833 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.833 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.833 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.833 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.833 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.834 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.834 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.834 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.834 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.834 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.834 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.834 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.834 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.834 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.834 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.834 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.835 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.835 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.835 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.835 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.835 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.835 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.835 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.835 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.835 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.836 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.836 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.836 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.836 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.836 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.836 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.836 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.836 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.836 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.836 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.836 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.836 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.836 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.836 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.837 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.837 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.837 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.837 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.837 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.837 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.837 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.837 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.837 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.837 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.837 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.837 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.837 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.837 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.838 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.838 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.838 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.838 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.838 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.838 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.838 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.838 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.838 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.838 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.838 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_object_raw_get_x_amz_expires_out_range_zero _____ 2024-06-28T16:14:09.838 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.838 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.839 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.839 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.839 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.839 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.839 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.839 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.839 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.839 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.839 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.839 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.839 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.839 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.839 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.840 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.841 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.841 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.841 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.841 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.841 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.841 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.841 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.841 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.841 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.841 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.841 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.842 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.842 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.842 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.842 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.842 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.842 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.842 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.842 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.842 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.842 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.842 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.842 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.843 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.843 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.843 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.843 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.843 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.843 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.843 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.843 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.843 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.843 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.843 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.843 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.843 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.843 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.844 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.844 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.844 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.844 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.844 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.844 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.844 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.844 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.844 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.844 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.844 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.844 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.845 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.845 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.845 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.845 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.845 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.845 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.845 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.845 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.845 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.845 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_object_raw_get_x_amz_expires_out_max_range _____ 2024-06-28T16:14:09.845 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.845 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.845 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.846 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.846 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.846 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.846 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.846 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.846 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.846 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.846 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.846 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.846 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.846 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.846 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.846 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.847 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.847 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.847 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.847 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.847 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.847 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.847 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.847 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.847 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.847 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.847 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.847 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.847 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.848 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.848 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.848 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.848 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.848 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.848 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.848 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.848 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.848 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.849 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.849 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.849 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.849 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.849 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.849 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.849 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.849 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.849 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.849 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.849 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.849 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.850 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.850 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.850 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.850 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.850 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.850 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.850 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.850 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.850 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.850 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.850 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.850 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.850 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.850 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.852 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.852 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.852 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.852 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.852 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.852 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.852 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.852 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.852 INFO:teuthology.orchestra.run.smithi178.stdout:__ ERROR at teardown of test_object_raw_get_x_amz_expires_out_positive_range ___ 2024-06-28T16:14:09.852 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.852 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.852 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.853 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.853 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.853 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.853 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.853 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.853 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.853 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.853 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.853 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.853 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.853 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.853 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.853 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.854 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.854 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.854 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.854 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.854 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.854 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.854 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.854 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.854 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.854 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.854 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.854 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.854 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.855 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.855 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.855 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.855 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.855 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.855 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.855 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.855 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.856 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.856 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.856 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.856 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.856 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.856 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.856 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.856 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.856 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.856 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.856 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.856 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.856 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.857 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.857 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.857 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.857 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.857 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.857 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.857 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.857 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.857 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.857 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.857 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.857 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.857 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.858 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.858 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.858 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.858 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.858 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.858 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.858 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.858 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.858 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.858 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.858 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.858 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.858 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.859 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.859 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.859 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.859 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.859 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.859 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.859 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.859 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.859 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_object_anon_put ___________________ 2024-06-28T16:14:09.859 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.859 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.859 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.860 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.860 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.860 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.860 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.860 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.860 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.860 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.860 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.860 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.861 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.861 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.861 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.861 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.861 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.861 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.861 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.861 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.861 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.861 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.861 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.861 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.861 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.861 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.862 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.862 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.862 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.862 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.862 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.862 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.862 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.862 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.862 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.862 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.863 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.863 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.863 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.863 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.863 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.863 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.863 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.863 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.863 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.863 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.863 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.863 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.863 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.863 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.864 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.864 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.864 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.864 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.864 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.864 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.864 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.864 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.864 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.864 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.864 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.864 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.865 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.865 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.865 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.865 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.865 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.865 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.865 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.865 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.865 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.865 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.865 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.865 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.865 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.865 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.866 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.866 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.866 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.866 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.866 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.866 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.866 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.866 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.866 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_object_anon_put_write_access ____________ 2024-06-28T16:14:09.866 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.866 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.866 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.867 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.867 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.867 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.867 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.867 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.867 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.867 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.867 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.867 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.867 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.867 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.867 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.867 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.868 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.868 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.868 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.868 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.868 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.868 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.868 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.868 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.868 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.868 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.868 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.868 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.868 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.868 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.869 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.869 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.869 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.869 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.869 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.869 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.869 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.869 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.869 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.869 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.869 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.870 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.870 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.870 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.870 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.870 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.870 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.870 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.870 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.870 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.870 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.870 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.870 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.870 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.871 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.871 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.871 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.871 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.871 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.871 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.871 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.871 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.871 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.871 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.871 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.871 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.872 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.872 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.872 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.872 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.872 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.872 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.872 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.872 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.872 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.872 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.872 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.872 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.872 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.872 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.873 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.873 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.873 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.873 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.873 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.873 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.873 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.873 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.873 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_object_put_authenticated ______________ 2024-06-28T16:14:09.873 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.873 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.873 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.873 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.874 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.874 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.874 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.874 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.874 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.874 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.874 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.874 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.874 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.874 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.875 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.875 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.875 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.875 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.875 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.875 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.875 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.875 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.875 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.875 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.875 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.875 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.875 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.876 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.876 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.876 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.876 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.876 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.876 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.876 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.876 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.876 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.876 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.876 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.877 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.877 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.877 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.877 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.877 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.877 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.877 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.877 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.877 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.877 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.877 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.877 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.878 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.878 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.878 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.878 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.878 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.878 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.878 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.878 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.878 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.878 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.878 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.878 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.878 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.879 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.879 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.879 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.879 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.879 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.879 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.879 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.879 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.879 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.879 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.879 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.879 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.879 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.880 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.880 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.880 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.880 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.880 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.880 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.880 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.880 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.880 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.880 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.880 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.880 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.881 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_object_presigned_put_object_with_acl ________ 2024-06-28T16:14:09.881 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.881 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.881 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.881 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.881 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.881 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.881 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.881 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.881 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.881 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.881 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.881 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.881 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.882 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.882 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.882 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.882 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.882 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.882 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.882 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.882 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.882 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.882 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.882 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.882 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.882 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.883 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.883 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.883 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.883 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.883 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.883 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.883 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.883 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.883 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.883 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.883 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.883 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.883 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.884 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.884 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.884 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.884 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.884 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.884 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.884 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.884 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.884 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.884 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.884 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.884 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.884 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.885 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.885 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.885 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.885 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.885 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.885 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.885 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.885 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.885 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.885 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.885 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.885 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.885 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.886 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.886 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.886 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.886 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.886 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.886 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.886 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.886 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.886 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.886 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.886 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.886 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.886 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.887 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.887 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.887 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.887 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.887 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.887 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.887 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.887 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.887 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.887 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.887 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.887 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_object_presigned_put_object_with_acl_tenant _____ 2024-06-28T16:14:09.887 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.887 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.888 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.888 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.888 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.888 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.888 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.888 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.888 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.888 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.888 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.888 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.888 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.888 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.888 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.889 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.889 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.889 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.889 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.889 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.889 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.889 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.889 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.889 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.889 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.889 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.889 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.889 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.890 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.890 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.890 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.890 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.890 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.890 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.890 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.890 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.890 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.890 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.891 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.891 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.891 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.891 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.891 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.891 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.891 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.891 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.891 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.891 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.891 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.891 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.891 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.892 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.892 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.892 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.892 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.892 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.892 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.892 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.892 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.892 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.892 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.892 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.892 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.892 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.892 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.893 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.893 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.893 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.893 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.893 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.893 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.893 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.893 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.893 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.893 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.893 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.893 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.893 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.893 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.894 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.894 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.894 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.894 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.894 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.894 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.894 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.894 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.894 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.894 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_object_raw_put_authenticated_expired ________ 2024-06-28T16:14:09.894 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.894 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.895 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.895 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.895 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.895 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.895 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.895 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.895 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.895 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.895 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.895 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.895 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.895 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.895 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.896 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.897 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.897 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.897 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.897 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.897 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.897 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.897 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.897 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.897 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.897 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.897 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.897 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.897 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.898 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.898 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.898 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.898 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.898 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.898 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.898 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.898 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.898 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.898 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.898 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.898 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.898 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.899 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.899 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.899 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.899 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.899 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.899 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.899 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.899 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.899 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.899 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.899 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.899 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.899 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.900 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.900 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.900 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.900 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.900 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.900 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.900 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.900 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.900 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.900 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.900 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.900 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.900 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_bucket_create_naming_bad_starts_nonalpha ______ 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.901 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.902 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.902 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.902 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.902 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.902 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.902 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.902 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.902 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.902 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.902 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.902 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.902 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.902 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.903 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.903 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.903 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.903 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.903 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.903 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.903 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.903 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.903 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.903 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.903 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.903 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.903 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.903 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.904 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.904 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.904 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.904 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.904 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.904 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.904 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.904 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.904 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.904 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.905 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.905 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.905 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.905 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.905 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.905 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.905 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.905 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.905 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.905 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.905 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.905 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.905 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.906 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.906 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.906 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.906 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.906 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.906 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.906 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.906 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.906 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.906 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.906 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.906 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.906 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.907 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.907 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.907 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.907 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.907 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.907 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.907 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.907 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.907 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.907 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.907 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.907 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.907 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.908 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.908 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.908 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.908 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.908 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.908 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.908 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_bucket_create_naming_bad_short_one _________ 2024-06-28T16:14:09.908 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.908 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.908 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.908 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.908 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.908 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.908 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.909 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.909 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.909 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.909 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.909 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.909 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.909 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.909 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.909 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.909 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.909 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.909 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.909 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.909 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.910 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.910 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.910 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.910 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.910 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.910 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.910 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.910 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.910 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.910 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.910 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.910 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.910 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.911 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.911 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.911 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.911 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.911 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.911 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.911 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.911 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.911 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.911 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.911 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.911 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.912 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.912 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.912 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.912 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.912 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.912 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.912 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.912 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.912 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.912 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.912 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.912 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.912 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.912 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.913 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.913 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.913 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.913 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.913 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.913 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.913 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.913 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.913 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.913 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.913 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.913 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.913 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.914 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.914 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.914 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.914 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.914 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.914 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.914 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.914 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.914 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.914 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.914 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.914 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.914 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.915 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.915 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.915 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.915 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_bucket_create_naming_bad_short_two _________ 2024-06-28T16:14:09.915 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.915 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.915 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.915 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.915 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.915 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.915 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.915 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.915 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.916 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.916 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.916 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.916 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.916 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.916 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.916 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.916 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.917 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.918 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.919 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.920 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.921 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.922 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.922 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_60 __________ 2024-06-28T16:14:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.922 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.922 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.922 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.922 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.922 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.923 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.924 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.924 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.924 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.925 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.926 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.927 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.928 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.928 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.928 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.928 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.928 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_61 __________ 2024-06-28T16:14:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.929 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.929 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.929 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.929 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.929 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.929 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.929 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.930 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.930 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.930 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.930 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.930 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.930 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.930 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.930 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.931 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.931 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.932 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.933 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.934 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.935 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.935 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.935 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.935 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.936 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_62 __________ 2024-06-28T16:14:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.936 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.936 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.936 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.936 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.936 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.936 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.937 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.937 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.937 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.937 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.937 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.937 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.937 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.937 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.937 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.938 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.938 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.939 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.939 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.939 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.939 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.939 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.939 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.939 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.939 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.939 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.939 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.939 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.939 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.939 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.939 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.940 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.941 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.941 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.941 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.941 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.941 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.941 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.941 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.941 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.941 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.941 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.941 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.941 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.941 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.941 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.942 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.942 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.942 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.942 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.942 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.942 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.942 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.942 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.942 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.942 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_bucket_create_naming_good_long_63 __________ 2024-06-28T16:14:09.942 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.942 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.943 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.943 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.943 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.943 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.943 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.943 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.943 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.943 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.943 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.943 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.943 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.943 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.943 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.943 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.944 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.945 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.945 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.945 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.945 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.945 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.945 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.945 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.945 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.945 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.945 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.945 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.945 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.945 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.946 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.947 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.948 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.948 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.948 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.948 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.948 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.948 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.948 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.948 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.948 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.948 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.948 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.948 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.949 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.949 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.949 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.949 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.949 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.949 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_bucket_list_long_name ________________ 2024-06-28T16:14:09.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.949 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.949 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.949 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.949 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.950 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.950 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.950 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.950 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.950 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.950 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.950 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.950 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.950 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.950 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.950 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.950 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.951 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.951 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.951 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.951 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.951 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.951 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.951 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.951 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.951 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.951 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.951 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.951 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.952 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.952 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.952 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.952 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.952 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.952 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.952 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.952 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.952 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.952 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.952 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.953 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.953 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.953 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.953 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.953 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.953 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.953 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.953 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.953 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.953 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.953 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.953 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.953 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.954 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.954 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.954 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.954 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.954 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.954 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.954 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.954 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.954 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.954 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.954 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.954 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.955 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.955 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.955 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.955 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.955 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.955 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.955 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.955 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.955 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.955 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.955 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.955 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.955 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.955 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.956 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.956 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.956 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.956 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.956 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.956 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.956 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.956 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.956 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_bucket_create_naming_bad_ip _____________ 2024-06-28T16:14:09.956 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.956 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.956 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.956 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.957 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.957 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.957 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.957 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.957 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.957 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.957 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.957 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.957 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.957 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.957 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.957 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.957 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.957 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.958 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.958 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.958 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.958 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.958 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.958 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.958 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.958 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.958 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.958 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.958 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.958 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.958 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.959 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.959 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.959 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.959 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.959 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.959 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.959 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.959 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.959 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.959 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.959 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.959 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.959 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.961 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.961 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.961 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.961 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.961 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.961 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.961 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.961 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.961 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.961 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.961 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.961 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.961 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.962 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.962 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.962 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.962 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.962 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.962 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.962 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.962 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.962 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.962 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.962 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.962 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.963 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.963 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.963 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.963 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.963 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.963 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.963 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.965 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_bucket_create_naming_dns_underscore _________ 2024-06-28T16:14:09.965 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.965 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.965 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.965 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.965 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.965 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.965 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.966 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.966 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.966 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.966 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.966 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.966 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.966 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.966 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.966 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.966 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.966 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.966 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.967 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.967 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.967 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.967 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.967 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.967 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.967 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.967 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.967 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.967 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.967 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.967 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.967 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.968 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.968 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.968 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.968 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.968 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.968 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.968 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.968 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.968 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.968 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.968 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.968 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.968 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.969 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.969 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.969 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.969 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.969 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.969 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.969 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.969 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.969 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.969 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.969 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.969 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.970 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.970 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.970 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.970 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.970 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.970 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.970 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.970 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.970 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.970 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.970 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.970 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.970 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.971 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.971 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.971 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.971 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.971 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.971 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.971 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.971 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.971 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.971 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.971 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.971 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.971 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.972 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.972 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.972 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.972 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.972 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.972 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.972 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_bucket_create_naming_dns_long ____________ 2024-06-28T16:14:09.972 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.972 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.972 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.972 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.972 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.972 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.973 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.973 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.973 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.973 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.973 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.973 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.973 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.973 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.973 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.973 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.973 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.973 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.973 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.974 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.974 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.974 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.974 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.974 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.974 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.974 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.974 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.974 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.974 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.974 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.974 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.974 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.974 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.975 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.975 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.975 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.975 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.975 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.975 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.975 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.975 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.975 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.975 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.975 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.975 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.975 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.976 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.976 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.976 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.976 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.976 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.976 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.976 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.976 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.976 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.976 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.976 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.976 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.976 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.976 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.977 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.977 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.977 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.977 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.977 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.977 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.977 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.977 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.977 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.977 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.977 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.977 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.977 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.978 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.978 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.978 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.978 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.978 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.978 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.978 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.978 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.978 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.978 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.978 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.978 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.978 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.979 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.979 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.979 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.979 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_bucket_create_naming_dns_dash_at_end ________ 2024-06-28T16:14:09.979 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.979 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.979 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.979 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.979 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.979 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.979 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.979 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.979 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.979 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.980 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.980 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.980 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.980 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.980 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.980 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.980 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.980 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.980 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.980 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.980 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.981 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.981 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.981 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.981 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.981 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.981 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.981 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.981 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.981 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.981 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.981 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.981 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.981 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.982 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.982 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.982 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.982 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.982 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.982 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.982 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.982 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.982 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.982 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.982 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.982 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.982 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.983 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.983 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.983 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.983 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.983 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.983 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.983 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.983 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.983 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.983 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.983 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.983 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.983 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.984 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.984 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.984 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.984 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.984 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.984 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.984 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.984 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.984 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.984 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.984 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.984 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.984 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.984 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.985 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.985 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.985 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.985 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.985 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.985 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.985 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.985 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.985 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.985 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.985 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.985 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.985 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.985 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.986 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.986 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.986 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_bucket_create_naming_dns_dot_dot __________ 2024-06-28T16:14:09.986 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.986 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.986 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.986 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.986 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.986 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.986 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.986 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.986 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.986 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.987 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.987 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.987 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.987 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.987 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.987 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.987 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.987 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.987 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.987 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.987 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.987 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.987 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.988 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.988 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.988 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.988 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.988 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.988 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.988 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.988 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.988 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.988 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.988 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.988 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.988 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.988 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.989 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.989 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.989 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.989 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.989 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.989 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.989 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.989 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.989 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.989 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.989 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.989 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.990 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.990 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.990 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.990 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.990 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.990 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.990 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.990 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.990 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.990 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.990 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.990 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.991 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.991 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.991 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.991 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.991 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.991 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.991 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.991 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.991 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.991 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.991 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.991 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.992 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.992 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.992 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.992 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.992 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.992 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.992 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.992 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.992 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.992 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.992 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.992 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.992 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:09.992 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:09.993 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.993 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:09.993 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_bucket_create_naming_dns_dot_dash __________ 2024-06-28T16:14:09.993 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.993 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:09.993 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.993 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:09.993 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:09.993 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:09.993 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:09.993 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.993 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:09.993 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.994 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:09.994 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:09.994 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:09.994 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:09.994 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:09.994 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:09.994 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:09.994 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:09.994 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:09.994 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:09.994 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:09.994 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:09.994 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:09.995 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.995 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:09.995 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:09.995 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:09.995 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.995 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:09.995 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:09.995 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:09.995 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:09.995 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:09.995 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:09.995 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:09.995 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:09.996 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:09.996 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:09.996 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.996 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:09.996 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:09.996 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:09.996 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.996 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:09.996 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:09.996 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:09.996 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:09.996 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:09.996 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:09.997 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:09.997 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:09.997 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.997 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:09.997 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.997 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:09.997 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:09.997 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.997 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.997 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.997 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.997 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:09.997 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:09.998 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.998 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.998 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.998 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:09.998 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:09.998 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:09.998 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:09.998 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:09.998 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:09.998 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.998 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.998 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:09.999 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:09.999 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:09.999 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:09.999 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:09.999 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:09.999 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:09.999 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:09.999 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:09.999 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:09.999 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:09.999 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:09.999 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:09.999 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.000 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.000 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.000 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.000 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_bucket_create_naming_dns_dash_dot __________ 2024-06-28T16:14:10.000 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.000 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.000 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.000 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.000 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.000 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.000 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.000 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.000 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.001 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.001 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.001 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.001 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.001 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.001 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.001 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.001 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.001 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.001 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.001 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.001 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.001 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.001 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.002 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.002 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.002 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.002 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.002 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.002 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.002 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.002 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.002 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.002 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.002 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.002 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.002 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.003 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.003 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.003 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.003 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.003 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.003 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.003 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.003 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.003 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.003 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.003 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.003 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.004 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.005 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.006 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.006 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.006 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.006 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.006 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.006 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.006 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.006 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.006 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.006 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.006 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.006 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.006 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.007 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_bucket_create_exists ________________ 2024-06-28T16:14:10.007 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.007 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.007 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.007 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.007 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.007 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.007 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.007 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.007 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.007 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.007 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.007 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.007 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.008 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.008 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.008 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.008 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.008 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.008 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.008 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.008 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.008 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.008 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.008 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.008 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.008 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.009 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.009 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.010 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.010 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.010 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.010 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.010 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.010 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.010 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.010 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.010 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.010 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.010 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.010 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.010 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.011 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.011 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.011 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.011 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.011 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.011 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.011 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.011 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.011 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.011 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.011 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.011 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.011 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.012 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.012 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.012 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.012 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.012 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.012 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.012 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.012 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.012 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.012 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.012 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.012 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.012 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.013 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.013 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.013 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.013 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.013 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.013 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.013 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.013 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.013 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.013 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.013 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.013 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.013 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_bucket_get_location _________________ 2024-06-28T16:14:10.014 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.014 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.014 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.014 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.014 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.014 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.014 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.014 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.014 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.014 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.014 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.014 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.015 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.015 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.015 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.015 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.015 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.015 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.015 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.015 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.015 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.015 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.015 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.015 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.015 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.015 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.016 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.016 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.016 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.016 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.016 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.016 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.016 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.016 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.016 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.016 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.016 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.016 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.016 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.016 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.017 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.017 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.017 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.017 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.017 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.017 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.017 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.017 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.017 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.017 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.017 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.017 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.017 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.018 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.018 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.018 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.018 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.018 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.018 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.018 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.018 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.018 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.018 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.018 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.018 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.018 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.019 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.019 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.019 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.019 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.019 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.019 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.019 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.019 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.019 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.019 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.019 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.019 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.019 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.019 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.020 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.020 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.020 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.020 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.020 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.020 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.020 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.020 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.020 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.020 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.020 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_bucket_create_exists_nonowner ____________ 2024-06-28T16:14:10.020 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.020 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.021 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.021 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.021 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.021 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.021 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.021 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.021 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.021 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.021 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.021 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.021 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.021 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.021 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.022 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.022 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.022 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.022 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.022 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.022 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.022 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.022 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.022 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.022 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.022 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.022 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.022 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.023 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.023 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.023 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.023 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.023 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.023 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.023 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.023 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.023 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.023 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.023 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.023 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.023 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.024 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.024 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.024 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.024 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.024 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.024 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.024 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.024 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.024 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.024 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.024 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.024 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.025 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.025 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.025 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.025 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.025 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.025 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.025 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.025 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.025 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.025 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.025 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.025 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.026 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.026 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.026 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.026 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.026 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.026 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.026 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.026 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.026 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.026 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.026 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.026 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.026 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.027 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.027 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.027 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.027 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.027 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.027 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.027 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.027 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.027 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.027 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.027 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.027 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_bucket_recreate_overwrite_acl ____________ 2024-06-28T16:14:10.027 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.028 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.028 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.028 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.028 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.028 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.028 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.028 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.028 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.028 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.028 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.028 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.028 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.028 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.029 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.029 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.029 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.029 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.029 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.029 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.029 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.029 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.029 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.029 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.029 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.029 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.029 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.029 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.030 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.030 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.030 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.030 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.030 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.030 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.030 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.030 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.030 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.030 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.030 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.030 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.030 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.031 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.031 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.031 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.031 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.031 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.031 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.031 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.031 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.031 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.031 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.031 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.031 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.032 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.032 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.032 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.032 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.032 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.032 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.032 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.032 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.032 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.032 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.032 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.032 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.032 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.033 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.033 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.033 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.033 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.033 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.033 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.033 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.033 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.033 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.033 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.033 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.033 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.033 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.033 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.034 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.034 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.034 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.034 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.034 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.034 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.034 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.034 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.034 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.034 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.034 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_bucket_recreate_new_acl _______________ 2024-06-28T16:14:10.034 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.035 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.036 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.036 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.036 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.036 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.036 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.036 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.036 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.036 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.036 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.036 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.036 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.036 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.036 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.036 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.037 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.038 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.038 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.038 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.038 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.038 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.038 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.038 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.038 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.038 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.038 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.038 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.038 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.038 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.039 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.039 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.039 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.039 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.039 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.039 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.039 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.039 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.039 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.039 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.039 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.039 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.040 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.040 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.040 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.040 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.040 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.040 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.040 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.040 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.040 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.040 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.040 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.040 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.040 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.041 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.041 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.041 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.041 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.041 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.041 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.041 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.041 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_bucket_acl_default _________________ 2024-06-28T16:14:10.041 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.041 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.041 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.041 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.041 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.042 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.042 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.042 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.042 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.042 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.042 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.042 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.042 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.042 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.042 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.042 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.042 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.043 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.043 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.043 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.043 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.043 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.043 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.043 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.043 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.043 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.043 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.043 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.043 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.043 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.043 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.044 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.044 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.044 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.044 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.044 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.044 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.044 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.044 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.044 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.044 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.044 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.045 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.045 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.045 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.045 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.045 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.045 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.045 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.045 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.045 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.045 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.045 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.045 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.046 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.046 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.046 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.046 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.046 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.046 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.046 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.046 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.046 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.046 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.046 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.046 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.046 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.047 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.047 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.047 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.047 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.047 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.047 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.047 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.047 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.047 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.047 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.047 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.047 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.047 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_bucket_acl_canned_during_create ___________ 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.048 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.049 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.049 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.049 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.049 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.049 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.049 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.049 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.049 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.049 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.049 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.049 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.049 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.049 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.050 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.050 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.050 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.050 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.050 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.050 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.050 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.050 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.050 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.050 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.050 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.050 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.050 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.051 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.051 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.051 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.051 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.051 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.051 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.051 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.051 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.051 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.051 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.051 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.051 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.052 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.052 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.052 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.052 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.052 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.052 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.052 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.052 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.052 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.052 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.052 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.052 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.053 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.053 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.053 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.053 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.053 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.053 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.053 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.053 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.053 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.053 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.053 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.053 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.053 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.053 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.054 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.054 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.054 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.054 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.054 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.054 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.054 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.054 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.054 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.054 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.054 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.055 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.055 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.055 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.055 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.055 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.055 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.055 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.055 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.055 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_bucket_acl_canned __________________ 2024-06-28T16:14:10.055 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.055 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.055 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.055 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.056 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.056 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.056 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.056 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.056 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.056 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.056 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.056 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.056 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.056 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.056 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.056 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.056 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.056 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.057 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.057 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.057 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.057 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.057 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.057 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.057 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.057 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.057 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.057 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.057 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.057 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.057 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.058 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.058 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.058 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.058 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.058 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.058 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.058 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.058 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.058 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.058 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.058 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.058 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.059 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.059 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.059 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.059 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.059 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.059 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.059 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.059 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.059 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.059 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.059 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.059 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.060 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.060 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.060 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.060 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.060 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.060 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.060 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.060 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.060 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.060 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.060 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.060 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.060 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.061 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.061 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.061 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.061 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.061 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.061 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.061 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.061 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.061 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.061 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.061 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.061 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.061 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.062 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.062 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.062 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.062 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.062 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.062 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.062 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.062 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.062 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_bucket_acl_canned_publicreadwrite __________ 2024-06-28T16:14:10.062 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.062 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.062 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.062 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.063 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.063 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.063 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.063 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.063 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.063 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.063 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.063 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.063 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.063 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.063 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.063 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.063 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.064 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.065 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.065 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.065 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.065 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.065 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.065 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.065 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.065 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.065 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.065 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.065 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.065 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.065 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.065 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.066 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.066 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.066 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.066 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.066 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.066 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.066 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.066 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.066 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.066 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.066 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.066 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.066 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.067 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.067 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.067 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.067 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.067 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.067 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.067 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.067 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.067 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.067 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.067 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.067 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.067 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.067 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.068 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.068 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.068 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.068 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.068 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.068 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.068 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.068 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.068 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.068 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.068 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.068 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.068 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.069 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.069 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.069 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.069 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.069 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_bucket_acl_canned_authenticatedread _________ 2024-06-28T16:14:10.069 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.069 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.069 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.069 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.069 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.069 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.070 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.070 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.070 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.070 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.070 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.070 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.070 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.070 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.070 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.070 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.070 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.070 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.071 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.071 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.071 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.071 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.071 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.071 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.071 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.071 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.071 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.071 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.071 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.071 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.071 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.072 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.072 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.072 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.072 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.072 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.072 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.072 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.072 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.072 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.072 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.072 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.072 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.073 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.073 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.073 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.073 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.073 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.073 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.073 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.073 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.073 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.073 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.073 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.073 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.073 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.074 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.074 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.074 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.074 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.074 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.074 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.074 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.074 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.074 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.074 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.074 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.074 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.074 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.075 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.075 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.075 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.075 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.075 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.075 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.075 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.075 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.075 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.075 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.075 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.075 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.075 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.076 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.076 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.076 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.076 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.076 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.076 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.076 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.076 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_object_acl_default _________________ 2024-06-28T16:14:10.076 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.076 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.076 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.076 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.076 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.076 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.077 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.077 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.077 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.077 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.077 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.077 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.077 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.077 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.077 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.077 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.077 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.077 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.077 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.077 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.078 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.078 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.078 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.078 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.078 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.078 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.078 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.078 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.078 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.078 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.078 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.078 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.079 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.079 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.079 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.079 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.079 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.079 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.079 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.079 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.079 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.079 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.079 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.079 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.080 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.080 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.080 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.080 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.080 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.080 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.080 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.080 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.080 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.080 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.080 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.080 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.080 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.081 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.082 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.083 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.083 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.083 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.083 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_object_acl_canned_during_create ___________ 2024-06-28T16:14:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.083 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.083 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.083 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.084 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.084 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.084 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.084 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.084 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.084 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.084 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.084 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.084 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.084 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.084 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.084 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.085 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.085 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.085 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.085 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.085 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.085 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.085 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.085 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.085 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.085 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.085 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.085 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.085 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.086 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.086 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.086 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.086 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.086 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.086 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.086 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.086 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.086 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.086 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.086 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.087 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.087 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.087 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.087 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.087 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.087 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.087 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.087 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.087 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.087 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.087 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.087 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.087 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.087 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.088 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.088 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.088 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.088 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.088 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.088 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.088 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.088 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.088 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.088 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.088 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.088 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.088 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.088 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.089 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.089 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.089 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.089 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.089 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.089 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.089 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.089 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.089 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.089 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.089 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.089 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.089 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.090 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.090 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.090 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.090 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_object_acl_canned __________________ 2024-06-28T16:14:10.090 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.090 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.090 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.090 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.090 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.090 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.090 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.090 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.090 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.091 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.091 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.091 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.091 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.091 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.091 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.091 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.091 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.091 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.091 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.091 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.091 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.091 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.092 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.092 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.092 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.092 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.092 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.092 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.092 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.092 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.092 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.092 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.092 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.092 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.093 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.093 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.093 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.093 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.093 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.093 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.093 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.093 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.093 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.093 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.093 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.093 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.093 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.094 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.094 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.094 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.094 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.094 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.094 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.094 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.094 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.094 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.094 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.094 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.094 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.095 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.096 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.097 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_object_acl_canned_publicreadwrite __________ 2024-06-28T16:14:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.097 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.097 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.097 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.097 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.097 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.097 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.097 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.098 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.098 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.098 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.098 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.098 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.098 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.098 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.099 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.099 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.100 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.101 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.102 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.103 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.103 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.103 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.103 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_object_acl_canned_authenticatedread _________ 2024-06-28T16:14:10.103 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.104 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.104 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.104 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.104 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.104 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.104 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.104 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.105 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.105 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.105 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.105 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.105 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.105 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.105 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.105 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.105 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.106 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.106 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.106 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.106 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.106 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.106 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.106 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.106 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.106 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.106 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.106 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.107 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.108 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.108 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.108 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.108 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.108 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.108 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.108 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.108 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.108 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.108 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.108 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.108 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.108 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.108 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.109 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.109 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.109 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.109 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.109 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.109 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.109 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.109 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.109 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.109 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.109 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.109 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.109 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.109 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.110 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.110 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.110 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.110 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.110 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.110 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.110 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.110 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_object_acl_canned_bucketownerread __________ 2024-06-28T16:14:10.110 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.110 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.110 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.110 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.110 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.110 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.111 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.111 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.111 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.111 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.111 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.111 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.111 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.111 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.111 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.111 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.111 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.111 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.111 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.112 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.112 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.112 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.112 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.112 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.112 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.112 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.112 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.112 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.112 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.112 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.112 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.112 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.113 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.113 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.113 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.113 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.113 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.113 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.113 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.113 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.113 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.113 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.114 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.114 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.114 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.114 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.114 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.114 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.114 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.114 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.114 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.114 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.114 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.114 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.114 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.115 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.115 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.115 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.115 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.115 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.115 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.115 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.115 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.115 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.115 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.115 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.115 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.115 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.115 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.116 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.116 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.116 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.116 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.116 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.116 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.116 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.116 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.116 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.116 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.116 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.116 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.116 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.116 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_object_acl_canned_bucketownerfullcontrol ______ 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.117 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.118 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.118 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.118 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.118 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.118 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.118 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.118 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.118 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.118 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.118 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.118 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.118 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.119 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.119 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.119 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.119 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.119 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.119 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.119 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.119 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.119 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.119 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.119 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.119 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.120 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.120 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.120 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.120 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.120 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.120 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.120 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.120 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.120 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.120 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.120 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.120 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.120 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.121 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.121 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.121 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.121 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.121 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.121 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.121 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.121 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.121 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.121 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.121 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.121 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.121 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.122 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.122 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.122 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.122 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.122 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.122 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.122 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.122 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.122 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.122 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.122 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.122 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.122 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.123 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.123 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.123 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.123 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.123 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.123 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.123 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.123 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.123 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.123 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.123 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.123 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.123 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.123 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.124 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.124 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.124 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_object_acl_full_control_verify_owner ________ 2024-06-28T16:14:10.124 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.124 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.124 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.124 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.124 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.124 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.124 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.124 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.124 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.124 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.124 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.125 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.125 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.125 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.125 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.125 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.125 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.125 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.125 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.125 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.125 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.125 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.125 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.125 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.125 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.126 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.126 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.126 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.126 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.126 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.126 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.126 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.126 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.126 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.126 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.126 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.127 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.127 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.127 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.127 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.127 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.127 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.127 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.127 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.127 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.127 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.127 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.127 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.127 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.128 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.128 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.128 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.128 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.128 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.128 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.128 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.128 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.128 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.128 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.128 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.129 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.129 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.129 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.129 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.129 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.129 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.129 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.129 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.129 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.129 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.129 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.129 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.129 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.129 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.130 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.130 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.130 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.130 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.130 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.130 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.130 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.130 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.130 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.130 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.130 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.130 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.130 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.131 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.131 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_object_acl_full_control_verify_attributes ______ 2024-06-28T16:14:10.131 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.131 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.131 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.131 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.131 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.131 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.131 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.131 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.131 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.131 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.131 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.131 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.132 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.132 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.132 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.132 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.132 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.132 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.132 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.132 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.132 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.132 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.132 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.132 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.132 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.132 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.133 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.133 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.133 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.133 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.133 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.133 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.133 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.133 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.133 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.133 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.133 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.134 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.134 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.134 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.134 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.134 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.134 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.134 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.134 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.134 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.134 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.134 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.134 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.134 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.135 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.135 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.135 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.135 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.135 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.135 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.135 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.135 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.135 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.135 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.135 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.135 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.135 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.136 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.136 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.136 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.136 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.136 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.136 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.136 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.136 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.136 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.136 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.136 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.136 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.136 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_bucket_acl_canned_private_to_private ________ 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.137 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.138 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.138 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.138 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.138 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.138 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.138 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.138 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.138 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.138 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.138 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.138 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.138 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.138 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.139 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.139 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.139 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.139 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.139 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.139 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.139 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.139 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.139 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.139 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.139 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.139 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.139 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.139 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.140 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.140 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.140 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.140 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.140 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.140 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.140 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.140 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.140 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.140 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.140 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.140 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.141 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.141 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.141 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.141 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.141 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.141 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.141 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.141 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.141 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.141 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.141 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.141 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.141 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.141 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.142 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.142 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.142 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.142 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.142 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.142 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.142 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.142 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.142 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.142 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.142 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.142 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.143 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.143 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.143 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.143 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.143 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.143 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.143 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.143 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.143 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.143 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.143 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.143 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.143 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.144 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.144 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.144 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.144 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.144 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.144 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.144 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.144 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.144 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.144 INFO:teuthology.orchestra.run.smithi178.stdout:_____________________ ERROR at teardown of test_object_acl _____________________ 2024-06-28T16:14:10.144 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.144 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.144 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.144 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.145 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.145 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.145 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.145 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.145 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.145 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.145 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.145 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.145 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.145 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.145 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.145 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.145 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.145 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.146 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.146 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.146 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.146 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.146 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.146 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.146 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.146 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.146 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.146 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.146 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.146 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.146 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.147 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.147 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.147 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.147 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.147 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.147 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.147 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.147 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.147 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.147 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.147 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.147 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.148 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.148 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.148 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.148 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.148 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.148 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.148 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.148 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.148 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.148 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.148 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.148 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.148 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.148 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.149 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.149 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.149 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.149 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.149 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.149 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.149 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.149 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.149 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.149 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.149 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.149 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.149 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.149 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.150 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.150 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.150 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.150 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.150 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.150 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.150 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.150 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.150 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.150 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.150 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.150 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.151 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.151 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.151 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.151 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.151 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.151 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_object_acl_write __________________ 2024-06-28T16:14:10.151 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.151 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.151 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.151 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.151 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.151 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.151 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.152 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.152 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.152 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.152 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.152 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.152 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.152 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.152 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.152 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.152 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.152 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.152 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.152 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.153 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.153 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.153 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.153 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.153 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.153 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.153 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.153 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.153 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.153 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.153 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.153 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.153 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.154 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.154 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.154 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.154 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.154 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.154 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.154 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.154 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.154 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.154 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.154 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.154 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.155 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.156 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.156 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.156 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.156 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.156 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.156 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.156 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.156 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.156 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.156 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.156 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.156 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.156 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.157 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.157 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.157 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.157 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.157 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.157 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.157 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.157 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.157 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.157 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.157 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.157 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.157 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.158 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.158 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.158 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_object_acl_writeacp _________________ 2024-06-28T16:14:10.158 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.158 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.158 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.158 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.158 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.158 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.158 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.158 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.158 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.158 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.159 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.159 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.159 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.159 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.159 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.159 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.159 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.159 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.159 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.159 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.159 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.159 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.159 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.160 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.160 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.160 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.160 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.160 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.160 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.160 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.160 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.160 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.160 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.160 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.160 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.160 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.160 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.161 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.161 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.161 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.161 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.161 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.161 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.161 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.161 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.161 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.161 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.161 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.161 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.161 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.162 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.162 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.162 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.162 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.162 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.162 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.162 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.162 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.162 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.162 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.162 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.162 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.163 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.163 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.163 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.163 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.163 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.163 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.163 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.163 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.163 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.163 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.163 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.163 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.163 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.164 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.164 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.164 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.164 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.164 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.164 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.164 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.164 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.164 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.164 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.164 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.164 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.164 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.165 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.165 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_object_acl_read ___________________ 2024-06-28T16:14:10.165 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.165 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.165 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.165 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.165 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.165 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.165 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.165 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.165 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.165 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.165 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.165 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.166 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.166 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.166 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.166 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.166 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.166 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.166 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.166 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.166 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.166 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.166 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.166 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.166 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.167 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.167 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.167 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.167 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.167 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.167 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.167 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.167 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.167 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.167 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.167 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.167 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.167 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.167 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.169 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.169 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.169 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.169 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.169 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.169 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.169 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.170 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.170 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.170 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.170 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.170 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.170 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.170 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.170 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.170 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.170 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.170 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.170 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.170 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.171 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.172 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.172 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.172 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.172 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.172 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.172 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.172 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.172 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.172 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.172 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.172 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.172 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.172 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.173 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.173 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.173 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.173 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_object_acl_readacp _________________ 2024-06-28T16:14:10.173 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.173 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.173 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.173 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.173 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.173 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.173 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.174 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.174 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.174 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.174 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.174 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.174 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.174 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.174 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.174 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.174 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.174 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.174 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.174 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.175 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.175 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.175 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.175 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.175 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.175 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.175 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.175 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.175 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.175 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.175 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.175 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.175 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.176 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.176 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.176 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.176 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.176 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.176 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.176 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.176 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.176 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.176 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.176 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.176 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.177 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.177 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.177 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.177 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.177 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.177 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.177 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.177 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.177 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.177 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.177 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.177 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.177 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.178 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.178 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.178 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.178 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.178 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.178 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.178 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.178 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.178 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.178 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.178 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.178 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.179 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.179 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.179 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.179 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.179 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.179 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.179 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.179 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.179 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.179 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.179 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.179 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.179 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.179 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.180 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.180 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.180 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.180 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.180 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.180 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_bucket_acl_grant_userid_fullcontrol _________ 2024-06-28T16:14:10.180 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.180 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.180 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.180 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.180 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.180 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.180 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.180 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.181 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.181 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.181 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.181 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.181 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.181 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.181 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.181 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.181 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.181 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.181 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.181 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.181 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.181 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.182 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.182 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.182 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.182 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.182 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.182 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.182 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.182 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.182 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.182 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.182 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.182 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.183 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.183 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.183 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.183 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.183 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.183 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.183 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.183 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.183 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.183 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.183 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.183 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.183 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.184 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.184 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.184 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.184 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.184 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.184 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.184 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.184 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.184 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.184 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.184 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.184 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.184 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.185 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.185 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.185 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.185 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.185 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.185 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.185 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.185 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.185 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.185 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.185 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.185 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.185 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.186 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.186 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.186 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.186 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.186 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.186 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.186 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.186 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.186 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.186 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.186 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.186 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.186 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.187 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.187 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.187 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.187 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.187 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_bucket_acl_grant_userid_read ____________ 2024-06-28T16:14:10.187 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.187 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.187 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.187 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.187 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.187 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.187 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.187 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.187 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.188 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.188 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.188 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.188 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.188 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.188 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.188 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.188 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.188 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.188 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.188 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.188 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.189 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.189 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.189 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.189 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.189 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.189 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.189 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.189 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.189 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.189 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.189 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.189 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.189 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.190 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.190 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.190 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.190 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.190 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.190 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.190 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.190 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.190 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.190 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.190 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.190 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.191 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.191 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.191 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.191 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.191 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.191 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.191 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.191 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.191 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.191 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.191 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.191 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.191 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.192 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.192 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.192 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.192 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.192 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.192 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.192 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.192 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.192 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.192 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.192 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.192 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.192 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.192 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.193 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.193 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.193 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.193 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.193 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.193 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.193 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.193 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.193 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.193 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.193 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.193 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.193 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.194 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.194 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.194 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.194 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_bucket_acl_grant_userid_readacp ___________ 2024-06-28T16:14:10.194 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.194 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.194 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.194 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.194 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.194 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.194 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.194 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.194 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.195 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.195 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.195 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.195 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.195 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.195 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.195 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.195 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.195 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.195 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.195 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.195 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.195 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.196 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.196 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.196 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.196 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.196 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.196 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.196 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.196 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.196 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.196 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.196 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.196 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.196 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.197 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.197 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.197 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.197 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.197 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.197 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.197 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.197 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.197 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.197 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.197 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.197 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.197 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.198 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.198 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.198 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.198 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.198 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.198 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.198 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.198 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.198 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.198 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.198 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.198 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.198 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.199 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.199 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.199 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.199 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.199 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.199 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.199 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.199 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.199 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.199 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.199 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.199 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.199 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.200 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.200 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.200 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.200 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.200 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.200 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.200 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.200 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.200 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.200 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.200 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.200 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.200 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.200 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.201 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.201 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_bucket_acl_grant_userid_write ____________ 2024-06-28T16:14:10.201 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.201 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.201 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.201 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.201 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.201 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.201 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.201 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.201 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.201 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.201 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.202 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.202 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.202 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.202 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.202 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.202 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.202 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.202 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.202 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.202 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.202 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.202 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.202 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.203 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.203 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.203 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.203 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.203 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.203 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.203 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.203 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.203 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.203 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.204 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.204 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.204 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.204 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.204 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.204 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.204 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.204 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.204 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.204 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.204 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.204 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.204 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.204 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.205 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.205 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.205 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.205 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.205 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.205 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.205 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.205 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.205 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.205 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.205 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.205 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.206 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.206 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.206 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.206 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.206 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.206 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.206 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.206 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.206 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.206 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.206 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.206 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.206 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.207 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.207 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.207 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.207 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.207 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.207 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.207 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.207 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.207 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.207 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.207 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.207 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.207 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.207 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.208 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.208 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.208 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.208 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_bucket_acl_grant_userid_writeacp __________ 2024-06-28T16:14:10.208 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.208 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.208 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.208 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.208 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.208 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.208 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.208 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.208 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.208 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.209 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.209 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.209 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.209 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.209 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.209 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.209 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.209 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.209 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.209 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.209 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.209 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.209 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.210 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.210 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.210 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.210 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.210 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.210 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.210 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.210 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.210 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.210 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.210 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.210 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.210 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.210 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.211 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.211 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.211 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.211 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.211 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.211 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.211 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.211 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.211 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.211 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.211 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.211 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.211 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.211 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.212 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.212 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.212 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.212 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.212 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.212 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.212 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.212 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.212 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.212 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.212 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.212 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.212 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.212 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.213 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.213 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.213 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.213 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.213 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.213 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.213 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.213 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.213 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.213 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.213 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.213 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.213 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.214 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.214 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.214 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.214 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.214 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.214 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.214 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.214 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.214 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.214 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.214 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.214 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.214 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_bucket_acl_grant_nonexist_user ___________ 2024-06-28T16:14:10.215 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.215 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.215 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.215 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.215 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.215 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.215 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.215 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.215 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.215 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.215 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.215 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.215 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.215 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.216 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.216 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.216 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.216 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.216 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.216 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.216 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.216 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.216 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.216 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.216 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.216 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.216 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.216 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.217 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.217 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.217 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.217 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.217 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.217 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.217 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.217 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.217 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.217 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.217 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.217 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.217 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.218 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.218 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.218 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.218 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.218 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.218 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.218 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.218 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.218 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.218 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.218 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.218 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.219 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.219 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.219 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.219 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.219 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.219 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.219 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.219 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.219 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.219 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.219 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.219 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.219 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.220 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.220 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.220 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.220 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.220 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.220 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.220 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.220 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.220 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.220 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.220 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.220 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.220 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.220 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.221 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.221 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.221 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.221 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.221 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.221 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.221 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.221 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.221 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.221 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.221 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_object_header_acl_grants ______________ 2024-06-28T16:14:10.221 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.221 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.222 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.222 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.222 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.222 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.222 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.222 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.222 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.222 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.222 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.222 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.222 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.222 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.222 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.222 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.223 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.223 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.223 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.223 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.223 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.223 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.223 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.223 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.223 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.223 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.223 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.223 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.223 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.224 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.224 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.224 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.224 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.224 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.224 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.224 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.224 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.224 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.224 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.224 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.224 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.224 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.224 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.225 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.225 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.225 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.225 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.225 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.225 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.225 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.225 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.225 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.225 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.225 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.225 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.225 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.225 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.226 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.226 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.226 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.226 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.226 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.226 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.226 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.226 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.226 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.226 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.226 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.226 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.226 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.227 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.227 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.227 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.227 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.227 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.227 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.227 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.227 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.227 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.227 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.227 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.227 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.227 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.227 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.228 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.228 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.228 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.228 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.228 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.228 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.228 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_bucket_header_acl_grants ______________ 2024-06-28T16:14:10.228 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.228 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.228 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.228 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.228 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.228 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.229 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.229 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.229 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.229 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.229 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.229 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.229 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.229 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.229 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.229 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.229 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.229 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.229 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.229 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.230 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.230 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.230 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.230 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.230 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.230 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.230 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.230 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.230 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.230 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.230 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.230 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.230 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.231 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.231 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.231 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.231 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.231 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.231 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.231 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.232 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.232 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.232 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.232 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.232 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.232 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.232 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.232 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.232 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.232 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.232 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.232 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.233 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.233 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.233 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.233 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.233 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.233 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.233 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.233 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.233 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.233 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.233 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.233 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.233 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.234 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.234 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.234 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.234 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.234 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.234 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.234 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.234 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.234 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.234 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.234 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.234 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.234 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.235 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.235 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.235 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.235 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.235 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.235 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_bucket_acl_grant_email _______________ 2024-06-28T16:14:10.235 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.235 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.235 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.235 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.235 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.235 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.235 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.235 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.236 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.236 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.236 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.236 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.236 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.236 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.236 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.236 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.236 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.236 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.236 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.236 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.236 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.236 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.237 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.237 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.237 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.237 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.237 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.237 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.237 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.237 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.237 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.237 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.237 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.237 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.237 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.238 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.238 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.238 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.238 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.238 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.238 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.239 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.239 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.239 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.239 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.239 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.239 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.239 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.239 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.239 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.239 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.239 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.239 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.239 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.239 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.240 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.240 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.240 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.240 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.240 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.240 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.240 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.240 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.240 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.240 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.240 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.240 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.240 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.240 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.241 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.241 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.241 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.241 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.241 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.241 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.241 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.241 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.241 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.241 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.241 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.241 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.241 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.241 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_bucket_acl_grant_email_not_exist __________ 2024-06-28T16:14:10.242 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.242 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.242 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.242 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.242 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.242 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.242 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.242 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.242 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.242 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.242 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.242 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.242 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.243 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.243 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.243 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.243 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.243 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.243 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.243 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.243 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.243 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.243 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.243 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.243 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.243 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.243 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.244 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.244 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.244 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.244 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.244 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.244 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.244 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.244 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.244 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.244 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.244 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.244 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.245 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.245 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.245 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.245 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.245 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.245 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.245 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.245 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.245 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.245 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.245 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.245 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.245 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.245 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.246 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.246 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.246 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.246 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.246 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.246 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.246 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.246 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.246 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.246 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.246 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.246 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.246 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.247 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.247 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.247 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.247 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.247 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.247 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.247 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.247 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.247 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.247 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.247 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.247 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.248 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.248 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.248 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.248 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.248 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.248 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.248 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.248 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.248 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.248 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.248 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_bucket_acl_revoke_all ________________ 2024-06-28T16:14:10.248 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.248 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.248 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.249 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.249 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.249 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.249 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.249 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.249 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.249 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.249 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.249 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.249 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.249 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.249 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.249 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.250 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.250 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.250 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.250 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.250 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.250 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.250 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.250 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.250 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.250 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.250 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.250 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.250 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.251 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.251 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.251 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.251 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.251 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.251 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.251 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.251 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.251 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.251 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.251 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.251 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.252 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.252 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.252 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.252 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.252 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.252 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.252 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.252 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.252 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.252 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.252 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.252 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.252 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.253 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.253 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.253 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.253 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.253 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.253 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.253 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.253 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.253 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.253 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.253 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.253 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.253 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.253 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.254 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.254 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.254 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.254 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.254 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.254 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.254 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.254 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.254 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.254 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.254 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.254 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.254 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.255 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.255 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.255 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.255 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.255 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.255 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.255 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_access_bucket_private_object_private ________ 2024-06-28T16:14:10.255 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.255 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.255 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.255 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.255 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.255 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.256 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.256 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.256 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.256 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.256 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.256 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.256 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.256 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.256 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.256 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.256 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.256 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.256 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.256 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.257 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.257 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.257 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.257 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.257 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.257 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.257 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.257 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.257 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.257 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.257 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.257 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.257 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.258 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.258 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.258 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.258 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.258 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.258 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.258 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.258 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.258 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.258 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.258 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.258 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.259 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.259 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.259 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.259 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.259 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.259 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.259 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.259 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.259 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.259 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.259 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.259 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.259 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.259 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.260 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.260 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.260 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.260 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.260 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.260 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.260 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.260 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.260 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.260 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.260 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.260 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.260 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.260 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.261 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.261 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.261 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.261 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.261 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.261 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.261 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.261 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.261 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.261 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.261 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.261 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.262 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.262 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.262 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.262 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_access_bucket_private_objectv2_private _______ 2024-06-28T16:14:10.262 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.262 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.262 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.262 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.262 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.262 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.262 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.262 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.262 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.263 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.263 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.263 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.263 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.263 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.263 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.263 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.263 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.263 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.263 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.263 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.263 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.263 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.263 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.264 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.264 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.264 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.264 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.264 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.264 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.264 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.264 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.264 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.264 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.264 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.264 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.265 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.265 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.265 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.265 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.265 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.265 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.265 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.265 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.265 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.265 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.265 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.265 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.266 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.266 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.266 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.266 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.266 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.266 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.266 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.266 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.266 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.266 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.266 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.266 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.266 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.266 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.267 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.267 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.267 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.267 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.267 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.267 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.267 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.267 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.267 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.267 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.267 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.267 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.267 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.267 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.268 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.268 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.268 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.268 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.268 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.268 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.268 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.268 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.268 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.268 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.268 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.268 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.268 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.269 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.269 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_access_bucket_private_object_publicread _______ 2024-06-28T16:14:10.269 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.269 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.269 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.269 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.269 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.269 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.269 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.269 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.269 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.269 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.269 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.269 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.270 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.270 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.270 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.270 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.270 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.270 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.270 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.270 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.270 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.270 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.270 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.270 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.270 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.271 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.271 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.271 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.271 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.271 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.271 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.271 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.271 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.271 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.271 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.271 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.271 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.272 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.272 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.272 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.272 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.272 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.272 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.272 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.272 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.272 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.272 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.272 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.272 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.272 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.272 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.273 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.273 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.273 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.273 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.273 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.273 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.273 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.273 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.273 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.273 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.273 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.273 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.273 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.274 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.274 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.274 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.274 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.274 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.274 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.274 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.274 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.274 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.274 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.274 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.274 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.274 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.274 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.275 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.275 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.275 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.275 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.275 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.275 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.275 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.275 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.275 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.275 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.275 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_access_bucket_private_objectv2_publicread ______ 2024-06-28T16:14:10.275 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.275 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.275 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.276 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.276 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.276 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.276 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.276 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.276 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.276 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.276 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.276 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.276 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.276 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.276 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.277 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.277 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.277 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.277 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.277 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.277 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.277 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.277 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.277 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.277 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.277 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.277 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.277 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.277 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.278 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.278 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.278 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.278 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.278 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.278 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.278 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.278 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.278 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.278 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.278 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.278 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.278 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.279 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.279 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.279 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.279 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.279 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.279 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.279 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.279 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.279 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.279 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.279 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.279 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.279 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.279 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.280 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.280 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.280 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.280 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.280 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.280 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.280 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.280 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.280 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.280 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.280 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.280 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.280 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.281 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.281 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.281 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.281 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.281 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.281 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.281 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.281 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.281 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.281 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.281 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.281 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.281 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.281 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.282 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.282 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.282 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.282 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.282 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.282 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.282 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.282 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_access_bucket_private_object_publicreadwrite ____ 2024-06-28T16:14:10.282 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.282 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.282 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.282 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.282 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.283 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.283 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.283 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.283 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.283 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.283 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.283 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.283 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.283 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.283 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.283 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.283 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.283 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.283 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.284 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.284 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.284 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.284 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.284 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.284 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.284 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.284 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.284 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.284 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.284 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.284 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.284 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.285 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.285 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.285 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.285 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.285 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.285 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.285 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.285 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.285 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.285 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.285 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.285 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.285 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.285 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.286 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.286 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.286 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.286 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.286 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.286 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.286 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.286 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.286 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.286 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.286 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.286 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.286 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.287 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.287 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.287 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.287 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.287 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.287 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.287 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.287 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.287 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.287 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.287 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.287 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.287 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.287 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.288 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.288 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.288 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.288 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.288 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.288 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.288 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.288 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.288 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.288 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.288 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.288 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.288 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.289 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.289 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.289 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.289 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.289 INFO:teuthology.orchestra.run.smithi178.stdout:___ ERROR at teardown of test_access_bucket_private_objectv2_publicreadwrite ___ 2024-06-28T16:14:10.289 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.289 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.289 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.289 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.289 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.289 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.289 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.289 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.290 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.290 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.290 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.290 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.290 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.290 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.290 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.290 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.290 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.290 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.290 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.290 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.291 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.291 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.291 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.291 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.291 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.291 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.291 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.291 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.291 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.291 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.291 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.291 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.291 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.291 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.292 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.292 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.292 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.292 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.292 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.292 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.292 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.292 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.292 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.292 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.292 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.292 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.293 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.293 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.293 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.293 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.293 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.293 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.293 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.293 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.293 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.293 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.293 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.293 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.293 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.294 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.294 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.294 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.294 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.294 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.294 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.294 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.294 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.294 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.294 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.294 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.294 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.294 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.295 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.295 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.295 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.295 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.295 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.295 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.295 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.295 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.295 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.295 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.295 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.295 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.295 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.296 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.296 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.296 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.296 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.296 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_access_bucket_publicread_object_private _______ 2024-06-28T16:14:10.296 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.296 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.296 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.296 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.296 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.296 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.296 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.296 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.296 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.297 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.297 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.297 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.297 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.297 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.297 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.297 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.297 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.297 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.297 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.297 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.297 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.297 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.297 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.298 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.298 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.298 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.298 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.298 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.298 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.298 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.298 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.298 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.298 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.298 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.298 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.299 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.299 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.299 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.299 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.299 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.299 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.300 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.300 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.300 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.300 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.300 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.300 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.300 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.300 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.300 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.300 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.300 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.300 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.300 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.300 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.301 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.301 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.301 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.301 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.301 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.301 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.301 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.301 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.301 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.301 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.301 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.301 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.301 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.301 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.302 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.302 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.302 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.302 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.302 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.302 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.302 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.302 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.302 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.302 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.302 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.302 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.302 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.303 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_access_bucket_publicread_object_publicread _____ 2024-06-28T16:14:10.303 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.303 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.303 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.303 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.303 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.303 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.303 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.303 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.303 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.303 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.303 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.303 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.304 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.304 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.304 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.304 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.304 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.304 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.304 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.304 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.304 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.304 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.304 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.304 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.304 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.305 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.305 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.305 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.305 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.305 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.305 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.305 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.305 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.305 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.305 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.305 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.306 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.306 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.306 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.306 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.306 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.306 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.306 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.306 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.306 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.306 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.306 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.306 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.306 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.307 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.307 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.307 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.307 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.307 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.307 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.307 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.307 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.307 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.307 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.307 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.307 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.307 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.308 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.308 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.308 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.308 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.308 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.308 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.308 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.308 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.308 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.308 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.308 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.308 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.308 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.308 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.309 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.309 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.309 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.309 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.309 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.309 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.309 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.309 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.309 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.309 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.309 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.309 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.309 INFO:teuthology.orchestra.run.smithi178.stdout:__ ERROR at teardown of test_access_bucket_publicread_object_publicreadwrite ___ 2024-06-28T16:14:10.310 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.310 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.310 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.310 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.310 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.310 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.310 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.310 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.310 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.310 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.310 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.310 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.310 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.311 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.311 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.311 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.311 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.311 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.311 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.311 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.311 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.311 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.311 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.311 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.311 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.311 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.311 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.312 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.312 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.312 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.312 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.312 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.312 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.312 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.312 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.312 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.312 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.312 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.315 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.315 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.315 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.315 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.315 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.315 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.315 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.315 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.315 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.315 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.315 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.315 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.315 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.316 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.316 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.316 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.316 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.316 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.316 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.316 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.316 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_access_bucket_publicreadwrite_object_private ____ 2024-06-28T16:14:10.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.316 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.316 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.317 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.317 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.317 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.317 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.317 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.317 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.317 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.317 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.317 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.318 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.318 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.318 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.318 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.318 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.318 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.318 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.318 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.318 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.318 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.318 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.318 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.318 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.319 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.319 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.319 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.319 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.319 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.319 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.319 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.319 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.319 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.319 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.319 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.319 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.320 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.320 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.320 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.320 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.320 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.320 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.320 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.320 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.320 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.320 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.320 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.320 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.320 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.320 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.321 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.322 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.322 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.322 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.322 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.322 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.322 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.322 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.322 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.322 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.322 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.322 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.322 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.322 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.323 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.323 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.323 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.323 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.323 INFO:teuthology.orchestra.run.smithi178.stdout:__ ERROR at teardown of test_access_bucket_publicreadwrite_object_publicread ___ 2024-06-28T16:14:10.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.323 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.323 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.323 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.323 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.323 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.324 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.324 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.324 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.324 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.324 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.324 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.324 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.324 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.324 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.324 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.324 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.324 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.324 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.325 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.325 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.325 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.325 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.325 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.325 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.325 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.325 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.325 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.325 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.325 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.325 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.325 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.325 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.326 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.326 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.326 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.326 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.326 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.326 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.326 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.326 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.326 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.326 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.326 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.326 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.326 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.327 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.328 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.329 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.329 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.329 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.329 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.329 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.329 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.329 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.329 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.329 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.329 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.329 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.329 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.329 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.329 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.330 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_access_bucket_publicreadwrite_object_publicreadwrite _ 2024-06-28T16:14:10.330 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.330 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.330 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.330 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.330 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.330 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.330 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.330 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.330 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.330 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.330 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.330 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.330 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.331 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.331 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.331 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.331 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.331 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.331 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.331 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.331 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.331 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.331 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.331 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.331 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.331 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.331 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.332 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.333 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.334 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.334 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.334 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.334 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.334 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.334 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.334 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.334 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.334 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.334 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.334 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.335 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.335 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.335 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.335 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.335 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.335 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.335 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.335 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.335 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.335 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.335 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.335 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.335 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.335 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.336 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.336 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.336 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.336 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.336 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.336 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.336 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.336 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.336 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.336 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.336 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_buckets_create_then_list ______________ 2024-06-28T16:14:10.336 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.336 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.336 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.337 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.337 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.337 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.337 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.337 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.337 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.337 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.337 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.337 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.337 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.337 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.337 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.337 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.337 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.338 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.338 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.338 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.338 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.338 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.338 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.338 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.338 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.338 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.338 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.338 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.338 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.338 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.339 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.339 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.339 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.339 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.339 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.339 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.339 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.339 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.339 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.339 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.339 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.339 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.339 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.339 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.340 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.341 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.341 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.341 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.341 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.341 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.341 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.341 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.341 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.341 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.341 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.341 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.341 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.341 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.341 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.342 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.342 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.342 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.342 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.342 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.342 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.342 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.342 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.342 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.342 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.342 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.342 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.342 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.343 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.343 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.343 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.343 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.343 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.343 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_buckets_list_ctime _________________ 2024-06-28T16:14:10.343 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.343 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.343 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.343 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.343 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.343 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.343 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.343 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.344 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.344 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.344 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.344 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.344 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.344 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.344 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.344 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.344 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.344 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.344 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.344 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.344 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.344 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.345 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.345 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.345 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.345 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.345 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.345 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.345 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.345 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.345 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.345 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.345 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.345 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.345 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.345 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.346 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.346 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.346 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.346 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.346 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.346 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.346 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.346 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.346 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.346 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.346 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.346 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.346 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.347 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.347 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.347 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.347 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.347 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.347 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.347 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.347 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.347 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.347 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.347 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.347 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.347 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.347 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.348 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.349 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.349 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.349 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.349 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.349 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.349 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.349 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.349 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.349 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.349 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.349 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.349 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.349 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.349 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.350 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.350 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_list_buckets_anonymous _______________ 2024-06-28T16:14:10.350 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.350 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.350 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.350 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.350 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.350 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.350 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.350 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.350 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.350 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.350 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.351 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.351 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.351 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.351 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.351 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.351 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.351 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.351 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.351 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.351 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.351 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.351 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.351 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.351 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.352 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.352 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.352 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.352 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.352 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.352 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.352 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.352 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.352 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.352 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.352 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.352 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.352 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.353 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.353 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.353 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.353 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.353 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.353 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.353 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.353 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.353 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.353 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.353 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.353 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.353 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.354 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.355 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.355 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.355 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.355 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.355 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.355 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.355 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.355 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.355 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.355 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.355 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.355 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.355 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.356 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.356 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.356 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.356 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.356 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.356 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.356 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.356 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.356 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.356 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.356 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.356 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_list_buckets_invalid_auth ______________ 2024-06-28T16:14:10.356 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.356 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.357 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.357 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.357 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.357 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.357 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.357 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.357 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.357 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.357 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.357 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.357 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.357 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.357 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.357 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.358 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.358 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.358 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.358 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.358 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.358 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.358 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.358 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.358 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.358 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.358 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.358 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.358 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.358 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.359 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.359 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.359 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.359 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.359 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.359 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.359 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.359 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.359 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.359 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.359 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.359 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.359 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.360 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.360 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.360 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.360 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.360 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.360 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.360 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.360 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.360 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.360 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.360 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.360 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.360 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.361 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.362 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.362 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.362 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.362 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.362 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.362 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.362 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.362 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.362 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.362 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.362 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.362 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.362 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.363 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.363 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.363 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.363 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.363 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.363 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.363 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.363 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_list_buckets_bad_auth ________________ 2024-06-28T16:14:10.363 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.363 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.363 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.363 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.364 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.364 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.364 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.364 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.364 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.364 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.364 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.364 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.364 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.364 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.364 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.364 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.364 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.364 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.365 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.365 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.365 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.365 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.365 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.365 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.365 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.365 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.365 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.365 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.365 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.365 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.365 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.366 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.366 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.366 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.366 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.366 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.366 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.366 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.366 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.366 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.366 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.366 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.366 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.366 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.367 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.367 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.367 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.367 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.367 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.367 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.367 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.367 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.367 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.367 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.367 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.367 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.367 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.368 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.368 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.368 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.368 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.368 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.368 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.368 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.368 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.368 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.368 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.368 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.368 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.368 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.369 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.370 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.370 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.370 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.370 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.370 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.370 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.370 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_bucket_create_naming_good_starts_alpha _______ 2024-06-28T16:14:10.370 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.370 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.370 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.370 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.370 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.370 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.371 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.371 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.371 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.371 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.371 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.371 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.371 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.371 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.371 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.371 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.371 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.371 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.371 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.371 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.372 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.372 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.372 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.372 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.372 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.372 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.372 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.372 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.372 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.372 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.372 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.372 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.372 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.373 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.373 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.373 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.373 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.373 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.373 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.373 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.373 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.373 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.373 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.373 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.373 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.373 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.373 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.374 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.374 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.374 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.374 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.374 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.374 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.374 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.374 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.374 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.374 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.374 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.374 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.375 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.375 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.375 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.375 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.375 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.375 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.375 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.375 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.375 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.375 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.375 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.375 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.375 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.376 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.377 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.377 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.377 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.377 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.377 INFO:teuthology.orchestra.run.smithi178.stdout:---------------------------- Captured stdout setup ----------------------------- 2024-06-28T16:14:10.377 INFO:teuthology.orchestra.run.smithi178.stdout:Done with cleanup of buckets in tests. 2024-06-28T16:14:10.377 INFO:teuthology.orchestra.run.smithi178.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-06-28T16:14:10.377 INFO:teuthology.orchestra.run.smithi178.stdout:Done with cleanup of buckets in tests. 2024-06-28T16:14:10.377 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_bucket_create_naming_good_starts_digit _______ 2024-06-28T16:14:10.377 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.377 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.378 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.378 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.378 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.378 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.378 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.378 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.378 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.378 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.378 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.378 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.378 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.378 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.378 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.379 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.379 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.379 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.379 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.379 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.379 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.379 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.379 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.379 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.379 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.379 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.379 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.379 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.380 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.381 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.382 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.382 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.382 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.382 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.382 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.382 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.382 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.382 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.382 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.382 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.382 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.382 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.382 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.383 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.384 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.384 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.384 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.384 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.384 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.384 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.384 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.384 INFO:teuthology.orchestra.run.smithi178.stdout:---------------------------- Captured stdout setup ----------------------------- 2024-06-28T16:14:10.384 INFO:teuthology.orchestra.run.smithi178.stdout:Done with cleanup of buckets in tests. 2024-06-28T16:14:10.384 INFO:teuthology.orchestra.run.smithi178.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-06-28T16:14:10.384 INFO:teuthology.orchestra.run.smithi178.stdout:Done with cleanup of buckets in tests. 2024-06-28T16:14:10.384 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_bucket_create_naming_good_contains_period ______ 2024-06-28T16:14:10.384 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.384 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.385 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.386 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.386 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.386 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.386 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.386 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.386 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.386 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.386 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.386 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.386 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.386 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.386 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.386 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.387 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.387 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.387 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.387 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.387 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.387 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.387 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.387 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.387 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.387 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.387 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.387 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.387 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.387 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.388 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.388 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.388 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.388 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.388 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.388 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.388 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.388 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.388 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.388 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.388 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.388 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.388 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.388 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.389 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.389 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.389 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.389 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.389 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.389 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.389 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.389 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.389 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.389 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.389 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.389 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.389 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.390 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.390 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.390 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.390 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.390 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.390 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.390 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.390 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.390 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.390 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.390 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.390 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.390 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.390 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.391 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.391 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.391 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.391 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.391 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.391 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_bucket_create_naming_good_contains_hyphen ______ 2024-06-28T16:14:10.391 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.391 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.391 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.391 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.391 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.391 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.391 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.391 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.392 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.392 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.392 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.392 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.392 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.392 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.392 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.392 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.392 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.392 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.392 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.392 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.392 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.393 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.393 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.393 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.393 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.393 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.393 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.393 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.393 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.393 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.393 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.393 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.393 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.393 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.393 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.394 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.394 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.394 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.394 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.394 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.394 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.394 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.394 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.394 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.394 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.394 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.394 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.395 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.395 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.395 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.395 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.395 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.395 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.395 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.395 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.395 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.395 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.395 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.395 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.395 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.395 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.396 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.396 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.396 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.396 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.396 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.396 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.396 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.396 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.396 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.396 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.396 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.396 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.396 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.396 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.397 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.397 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.397 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.397 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.397 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.397 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.397 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.397 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.397 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.397 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.397 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.397 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.397 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.398 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.398 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_bucket_recreate_not_overriding ___________ 2024-06-28T16:14:10.398 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.398 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.398 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.398 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.398 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.398 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.398 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.398 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.398 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.398 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.398 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.398 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.399 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.399 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.399 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.399 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.399 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.399 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.399 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.399 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.399 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.399 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.399 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.399 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.399 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.399 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.400 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.400 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.400 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.400 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.400 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.400 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.400 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.400 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.400 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.400 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.400 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.400 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.401 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.401 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.401 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.401 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.401 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.401 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.401 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.401 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.401 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.401 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.401 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.401 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.401 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.401 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.402 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.402 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.402 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.402 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.402 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.402 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.402 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.402 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.402 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.402 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.402 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.402 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.402 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.403 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.404 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.404 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.404 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.404 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.404 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.404 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.404 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.404 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.404 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.404 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_bucket_create_special_key_names ___________ 2024-06-28T16:14:10.404 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.404 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.404 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.404 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.405 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.405 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.405 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.405 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.405 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.405 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.405 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.405 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.405 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.405 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.405 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.405 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.405 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.405 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.406 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.406 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.406 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.406 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.406 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.406 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.406 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.406 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.406 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.406 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.406 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.407 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.407 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.407 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.407 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.407 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.407 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.407 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.407 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.407 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.407 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.407 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.407 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.408 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.409 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.409 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.409 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.409 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.409 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.409 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.409 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.409 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.409 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.409 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.409 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.409 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.409 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.409 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.410 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.410 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.410 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.410 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.410 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.410 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.410 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.410 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.410 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.410 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.410 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.410 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.410 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.411 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.411 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.411 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.411 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.411 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.411 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.411 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_bucket_list_special_prefix _____________ 2024-06-28T16:14:10.411 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.411 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.411 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.411 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.411 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.411 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.411 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.412 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.412 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.412 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.412 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.412 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.412 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.412 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.412 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.412 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.412 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.412 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.412 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.412 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.412 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.413 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.413 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.413 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.413 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.413 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.413 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.413 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.413 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.413 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.413 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.413 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.413 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.413 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.414 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.414 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.414 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.414 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.414 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.414 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.414 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.414 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.414 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.414 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.414 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.414 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.414 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.415 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.415 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.415 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.415 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.415 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.415 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.415 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.415 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.415 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.415 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.415 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.415 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.415 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.416 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.416 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.416 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.416 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.416 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.416 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.416 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.416 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.416 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.416 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.416 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.416 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.416 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.416 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.417 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.417 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.417 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.417 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.417 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.417 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.417 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.417 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.417 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.417 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.417 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.417 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.417 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.417 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.418 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_object_copy_zero_size ________________ 2024-06-28T16:14:10.418 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.418 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.418 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.418 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.418 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.418 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.418 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.418 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.418 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.418 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.418 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.418 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.418 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.419 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.419 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.419 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.419 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.419 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.419 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.419 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.419 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.419 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.419 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.419 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.419 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.419 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.419 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.420 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.420 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.420 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.420 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.420 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.420 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.420 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.420 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.420 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.420 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.421 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.421 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.421 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.421 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.421 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.421 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.421 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.421 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.421 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.421 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.421 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.421 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.421 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.421 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.422 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.422 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.422 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.422 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.422 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.422 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.422 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.422 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.422 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.422 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.422 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.422 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.422 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.423 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.424 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.424 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.424 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.424 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.424 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.424 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.424 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.424 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.424 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.424 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.424 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_object_copy_16m ___________________ 2024-06-28T16:14:10.424 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.424 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.424 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.425 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.425 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.425 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.425 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.425 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.425 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.425 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.425 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.425 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.425 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.425 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.425 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.425 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.425 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.426 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.426 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.426 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.426 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.426 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.426 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.426 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.426 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.426 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.426 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.426 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.426 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.426 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.427 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.427 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.427 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.427 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.427 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.427 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.427 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.427 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.427 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.427 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.427 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.427 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.428 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.428 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.428 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.428 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.428 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.428 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.428 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.428 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.428 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.428 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.428 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.428 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.428 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.429 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.430 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.431 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.431 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.431 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.431 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.431 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.431 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.431 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_object_copy_same_bucket _______________ 2024-06-28T16:14:10.431 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.431 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.431 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.431 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.431 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.431 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.431 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.432 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.432 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.432 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.432 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.432 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.432 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.432 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.432 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.432 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.432 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.432 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.432 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.432 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.433 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.433 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.433 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.433 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.433 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.433 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.433 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.433 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.433 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.433 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.433 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.433 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.433 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.434 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.434 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.434 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.434 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.434 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.434 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.434 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.434 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.434 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.434 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.434 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.434 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.435 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.435 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.435 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.435 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.435 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.435 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.435 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.435 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.435 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.435 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.435 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.435 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.436 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.436 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.436 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.436 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.436 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.436 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.436 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.436 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.436 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.436 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.436 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.436 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.436 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.436 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.437 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.438 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.438 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.438 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.438 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.438 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_object_copy_verify_contenttype ___________ 2024-06-28T16:14:10.438 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.438 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.438 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.438 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.438 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.438 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.438 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.439 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.440 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.440 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.440 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.440 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.440 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.440 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.440 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.440 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.440 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.440 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.440 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.440 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.441 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.441 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.441 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.441 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.441 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.441 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.441 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.441 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.441 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.441 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.441 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.441 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.442 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.443 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.443 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.443 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.443 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.443 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.443 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.443 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.443 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.443 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.443 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.443 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.443 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.443 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.444 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.444 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.444 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.444 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.444 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.444 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.444 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.444 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.444 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.444 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.444 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.444 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.444 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.444 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.445 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.445 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_object_copy_to_itself ________________ 2024-06-28T16:14:10.445 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.445 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.445 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.445 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.445 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.445 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.445 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.445 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.445 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.445 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.445 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.445 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.446 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.446 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.446 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.446 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.446 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.446 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.446 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.446 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.446 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.446 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.446 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.446 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.446 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.446 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.447 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.447 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.447 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.447 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.447 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.447 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.447 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.447 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.447 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.447 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.447 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.447 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.448 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.448 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.448 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.448 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.448 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.448 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.448 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.448 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.448 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.448 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.448 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.448 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.448 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.448 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.449 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.449 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.449 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.449 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.449 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.449 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.449 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.449 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.449 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.449 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.449 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.450 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.451 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.451 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.451 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.451 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.451 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.451 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.451 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.451 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.451 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.451 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.451 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.451 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.451 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_object_copy_to_itself_with_metadata _________ 2024-06-28T16:14:10.452 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.452 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.452 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.452 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.452 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.452 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.452 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.452 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.452 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.452 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.452 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.452 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.452 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.452 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.453 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.453 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.453 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.453 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.453 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.453 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.453 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.453 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.453 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.453 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.453 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.453 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.453 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.453 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.454 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.454 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.454 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.454 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.454 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.454 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.454 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.454 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.454 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.454 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.454 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.454 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.454 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.455 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.456 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.456 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.456 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.456 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.456 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.456 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.456 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.456 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.456 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.456 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.456 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.456 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.456 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.457 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.458 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.458 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.458 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.458 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.458 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.458 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.458 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.458 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.458 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_object_copy_diff_bucket _______________ 2024-06-28T16:14:10.458 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.458 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.458 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.458 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.458 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.459 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.459 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.459 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.459 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.459 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.459 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.459 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.459 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.459 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.459 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.459 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.459 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.459 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.460 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.460 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.460 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.460 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.460 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.460 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.460 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.460 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.460 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.460 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.460 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.460 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.460 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.460 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.461 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.461 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.461 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.461 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.461 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.461 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.461 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.461 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.461 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.461 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.461 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.461 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.461 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.462 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.462 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.462 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.462 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.462 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.462 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.462 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.462 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.462 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.462 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.462 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.462 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.463 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.463 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.463 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.463 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.463 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.463 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.463 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.463 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.463 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.463 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.463 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.463 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.463 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.463 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.464 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.464 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.464 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.464 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.464 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.464 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.464 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.464 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.464 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.464 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.464 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.465 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.465 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.465 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.465 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.465 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.465 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.465 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.465 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_object_copy_not_owned_bucket ____________ 2024-06-28T16:14:10.465 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.465 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.465 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.465 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.465 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.465 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.466 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.466 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.466 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.466 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.466 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.466 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.466 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.466 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.466 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.466 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.466 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.466 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.467 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.467 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.467 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.467 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.467 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.467 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.467 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.467 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.467 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.467 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.467 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.467 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.467 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.467 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.468 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.468 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.468 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.468 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.468 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.468 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.468 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.468 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.468 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.468 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.468 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.468 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.468 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.469 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.469 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.469 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.469 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.469 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.469 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.469 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.469 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.469 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.469 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.469 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.469 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.469 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.469 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.470 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.470 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.470 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.470 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.470 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.470 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.470 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.470 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.470 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.470 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.470 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.470 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.470 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.471 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.471 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.471 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.471 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.471 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.471 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.471 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.471 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.471 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.471 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.471 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.471 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.471 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.471 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.472 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.472 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.472 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.472 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_object_copy_not_owned_object_bucket _________ 2024-06-28T16:14:10.472 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.472 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.472 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.472 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.472 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.472 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.472 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.472 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.472 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.473 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.473 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.473 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.473 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.473 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.473 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.473 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.473 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.473 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.473 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.473 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.473 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.473 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.473 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.474 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.474 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.474 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.474 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.474 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.474 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.474 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.474 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.474 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.474 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.474 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.474 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.474 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.475 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.475 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.475 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.475 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.475 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.475 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.475 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.475 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.475 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.475 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.475 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.475 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.475 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.475 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.476 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.476 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.476 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.476 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.476 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.476 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.476 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.476 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.476 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.476 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.476 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.476 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.476 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.477 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.477 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.477 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.477 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.477 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.477 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.477 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.477 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.477 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.477 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.477 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.477 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.477 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.478 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.478 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.478 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.478 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.478 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.478 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.478 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.478 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.478 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.478 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.478 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.478 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.479 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.479 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.479 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_object_copy_canned_acl _______________ 2024-06-28T16:14:10.479 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.479 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.479 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.479 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.479 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.479 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.479 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.479 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.479 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.479 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.480 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.480 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.480 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.480 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.480 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.480 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.480 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.480 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.480 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.480 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.480 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.480 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.480 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.481 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.481 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.481 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.481 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.483 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.483 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.483 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.483 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.483 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.483 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.483 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.483 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.483 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.483 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.483 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.483 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.483 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.484 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.484 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.484 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.484 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.484 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.484 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.484 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.484 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.484 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.484 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.484 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.484 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.484 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.484 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.485 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.485 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.485 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.485 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.485 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.485 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.485 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.485 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.485 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.485 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.485 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.485 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.485 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.485 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_object_copy_retaining_metadata ___________ 2024-06-28T16:14:10.486 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.486 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.486 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.486 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.486 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.486 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.486 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.486 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.486 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.486 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.486 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.486 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.486 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.487 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.487 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.487 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.487 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.487 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.487 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.487 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.487 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.487 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.487 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.487 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.487 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.487 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.488 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.488 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.488 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.488 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.488 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.488 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.488 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.488 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.488 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.488 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.488 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.488 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.488 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.488 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.489 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.489 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.489 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.489 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.489 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.489 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.489 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.489 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.489 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.489 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.489 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.489 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.489 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.489 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.490 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.490 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.490 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.490 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.490 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.490 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.490 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.490 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.490 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.490 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.490 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.490 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.490 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.491 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.491 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.491 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.491 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.491 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.491 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.491 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.491 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.491 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.491 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.491 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.491 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.491 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.492 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.492 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.492 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.492 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.492 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.492 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.492 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.492 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.492 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.492 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.492 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_object_copy_replacing_metadata ___________ 2024-06-28T16:14:10.492 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.492 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.492 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.493 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.493 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.493 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.493 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.493 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.493 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.493 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.493 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.493 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.493 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.493 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.493 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.494 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.494 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.494 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.494 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.494 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.494 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.494 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.494 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.494 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.494 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.494 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.494 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.495 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.495 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.495 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.495 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.495 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.495 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.495 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.495 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.495 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.495 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.495 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.495 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.495 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.495 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.496 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.496 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.496 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.496 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.496 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.496 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.496 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.496 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.496 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.496 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.496 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.496 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.496 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.497 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.497 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.497 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.497 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.497 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.497 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.497 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.497 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.497 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.497 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.497 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.497 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.497 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.498 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.498 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.498 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.498 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.498 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.498 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.498 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.498 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.498 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.498 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.498 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.498 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.498 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.499 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.499 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.499 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.499 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.499 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.499 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.499 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.499 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.499 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.499 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.499 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_object_copy_bucket_not_found ____________ 2024-06-28T16:14:10.499 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.499 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.499 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.500 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.500 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.500 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.500 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.500 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.500 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.500 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.500 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.500 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.500 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.500 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.500 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.500 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.501 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.501 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.501 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.501 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.501 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.501 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.501 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.501 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.501 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.501 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.501 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.501 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.501 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.502 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.502 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.502 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.502 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.502 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.502 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.502 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.502 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.502 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.502 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.502 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.502 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.502 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.503 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.504 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.504 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.504 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.504 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.504 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.504 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.504 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.504 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.504 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.504 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.504 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.504 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.504 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.505 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.505 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.505 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.505 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.505 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.505 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.505 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.505 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.505 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.505 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.505 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.505 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.505 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.505 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.506 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.506 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.506 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.506 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.506 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.506 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.506 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.506 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_object_copy_key_not_found ______________ 2024-06-28T16:14:10.506 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.506 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.506 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.506 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.507 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.507 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.507 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.507 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.507 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.507 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.507 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.507 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.507 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.507 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.507 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.507 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.507 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.507 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.508 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.508 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.508 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.508 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.508 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.508 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.508 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.508 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.508 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.508 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.508 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.509 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.509 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.509 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.509 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.509 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.509 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.509 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.509 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.509 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.509 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.509 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.509 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.509 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.511 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.511 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.511 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.511 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.511 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.511 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.511 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.511 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.511 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.511 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.511 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.511 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.511 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.512 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.513 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.513 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.513 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.513 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.513 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.513 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.513 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.513 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.513 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_object_copy_versioned_bucket ____________ 2024-06-28T16:14:10.513 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.513 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.513 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.513 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.513 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.514 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.514 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.514 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.514 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.514 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.514 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.514 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.514 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.514 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.514 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.514 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.514 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.514 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.515 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.515 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.515 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.515 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.515 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.515 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.515 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.515 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.515 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.515 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.515 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.515 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.516 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.517 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.518 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.519 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.519 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.519 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.519 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.519 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.519 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.519 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.519 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.519 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.519 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.519 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.519 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.519 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.520 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.520 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.520 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.520 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.520 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.520 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_object_copy_versioned_url_encoding _________ 2024-06-28T16:14:10.520 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.520 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.520 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.520 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.520 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.520 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.520 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.520 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.521 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.521 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.521 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.521 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.521 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.521 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.521 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.521 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.521 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.521 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.521 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.521 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.521 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.521 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.522 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.522 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.522 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.522 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.522 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.522 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.522 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.522 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.522 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.522 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.522 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.522 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.522 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.523 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.523 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.523 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.523 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.523 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.523 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.523 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.523 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.523 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.523 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.523 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.524 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.524 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.524 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.524 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.524 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.524 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.524 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.524 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.524 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.524 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.524 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.524 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.524 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.524 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.525 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.525 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.525 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.525 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.525 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.525 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.525 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.525 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.525 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.525 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.525 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.525 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.525 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.526 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.526 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.526 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.526 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.526 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.526 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.526 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.526 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.526 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.526 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.526 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.526 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.526 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.527 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.527 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.527 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.527 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.527 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_object_copy_versioning_multipart_upload _______ 2024-06-28T16:14:10.527 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.527 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.527 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.527 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.527 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.527 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.527 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.527 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.527 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.528 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.528 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.528 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.528 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.528 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.528 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.528 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.528 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.528 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.528 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.528 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.528 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.528 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.529 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.529 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.529 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.529 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.529 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.529 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.529 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.529 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.529 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.529 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.529 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.529 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.529 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.529 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.530 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.530 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.530 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.530 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.530 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.530 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.530 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.530 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.530 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.530 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.530 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.530 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.530 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.531 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.532 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.533 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.533 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.533 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.533 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.533 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.533 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.533 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.533 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.533 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.533 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.533 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.533 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.533 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.534 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_multipart_upload_empty _______________ 2024-06-28T16:14:10.534 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.534 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.534 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.534 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.534 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.534 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.534 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.534 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.534 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.534 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.534 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.534 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.534 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.535 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.535 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.535 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.535 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.535 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.535 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.535 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.535 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.535 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.535 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.535 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.535 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.535 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.535 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.536 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.536 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.536 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.536 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.536 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.536 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.536 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.536 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.536 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.536 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.536 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.536 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.536 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.536 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.537 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.537 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.537 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.537 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.537 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.537 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.537 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.537 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.537 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.537 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.537 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.537 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.538 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.539 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.539 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.539 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.539 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.539 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.539 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.539 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.539 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.539 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.539 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.539 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.539 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.539 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.540 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.540 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.540 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.540 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.540 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.540 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.540 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.540 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.540 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.540 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.540 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.540 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_multipart_upload_small _______________ 2024-06-28T16:14:10.540 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.540 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.541 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.541 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.541 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.541 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.541 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.541 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.541 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.541 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.541 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.541 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.541 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.541 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.541 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.541 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.542 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.542 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.542 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.542 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.542 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.542 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.542 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.542 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.542 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.542 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.542 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.542 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.542 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.543 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.543 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.543 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.543 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.543 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.543 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.543 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.543 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.543 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.543 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.543 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.543 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.543 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.543 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.544 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.544 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.544 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.544 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.544 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.544 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.544 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.544 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.544 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.544 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.544 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.544 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.544 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.545 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.546 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.546 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.546 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.546 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.546 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.546 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.546 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.546 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.546 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.546 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.546 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.546 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.546 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.547 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.547 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.547 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.547 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.547 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.547 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.547 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.547 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_multipart_copy_small ________________ 2024-06-28T16:14:10.547 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.547 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.547 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.547 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.547 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.547 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.548 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.548 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.548 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.548 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.548 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.548 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.548 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.548 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.548 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.548 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.548 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.548 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.548 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.548 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.549 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.549 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.549 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.549 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.549 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.549 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.549 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.549 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.549 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.549 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.549 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.549 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.549 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.550 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.550 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.550 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.550 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.550 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.550 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.550 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.550 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.550 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.550 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.550 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.550 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.550 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.550 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.551 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.551 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.551 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.551 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.551 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.551 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.551 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.551 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.551 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.551 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.551 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.551 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.551 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.551 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.552 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.552 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.552 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.552 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.552 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.552 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.552 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.552 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.552 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.552 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.552 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.552 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.553 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.553 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.553 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.553 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.553 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.553 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.553 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.553 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.553 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.553 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.553 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.553 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.553 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.553 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.554 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.554 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.554 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.554 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_multipart_copy_invalid_range ____________ 2024-06-28T16:14:10.554 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.554 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.554 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.554 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.554 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.554 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.554 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.554 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.554 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.555 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.555 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.555 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.555 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.555 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.555 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.555 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.555 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.555 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.555 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.555 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.555 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.555 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.556 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.556 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.556 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.556 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.556 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.556 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.556 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.556 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.556 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.556 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.556 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.556 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.556 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.556 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.557 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.557 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.557 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.557 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.557 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.557 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.557 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.557 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.557 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.557 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.557 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.557 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.557 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.557 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.558 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.558 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.558 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.558 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.558 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.558 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.558 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.558 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.558 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.558 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.558 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.558 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.558 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.559 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.559 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.559 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.559 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.559 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.559 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.559 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.559 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.559 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.559 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.559 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.559 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.559 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.559 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.560 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.560 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.560 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.560 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.560 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.560 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.560 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.560 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.560 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.560 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.560 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.560 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.560 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.560 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_multipart_copy_without_range ____________ 2024-06-28T16:14:10.561 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.561 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.561 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.561 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.561 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.561 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.561 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.561 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.561 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.561 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.561 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.561 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.561 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.562 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.562 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.562 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.562 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.562 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.562 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.562 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.562 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.562 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.562 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.562 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.562 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.562 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.563 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.563 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.563 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.563 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.563 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.563 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.563 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.563 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.563 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.563 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.563 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.563 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.563 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.564 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.564 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.564 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.564 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.564 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.564 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.564 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.564 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.564 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.564 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.564 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.564 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.564 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.564 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.565 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.565 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.565 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.565 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.565 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.565 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.565 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.565 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.565 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.565 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.565 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.565 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.565 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.565 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.566 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.566 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.566 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.566 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.566 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.566 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.566 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.566 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.566 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.566 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.566 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.566 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.567 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.567 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.567 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.567 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.567 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.567 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.567 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.567 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.567 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.567 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.567 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.567 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_multipart_copy_special_names ____________ 2024-06-28T16:14:10.567 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.568 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.568 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.568 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.568 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.568 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.568 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.568 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.568 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.568 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.568 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.568 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.568 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.568 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.569 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.569 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.569 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.569 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.569 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.569 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.569 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.569 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.569 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.569 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.569 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.569 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.569 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.569 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.570 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.570 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.570 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.570 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.570 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.570 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.570 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.570 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.570 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.570 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.570 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.572 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.572 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.572 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.572 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.572 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.572 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.572 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.572 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.572 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.572 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.572 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.572 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.572 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.573 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.573 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.573 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.573 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.573 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.573 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.573 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.573 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.573 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.573 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.573 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.573 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.573 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.574 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.574 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.574 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.574 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.574 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.574 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.574 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.574 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.574 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.574 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_multipart_upload __________________ 2024-06-28T16:14:10.574 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.574 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.574 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.575 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.575 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.575 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.575 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.575 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.575 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.575 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.575 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.575 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.575 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.575 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.575 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.575 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.577 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.577 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.577 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.577 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.577 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.577 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.577 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.577 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.578 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.578 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.578 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.578 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.578 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.578 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.578 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.578 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.578 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.578 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.578 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.578 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.579 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.579 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.579 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.579 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.579 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.579 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.579 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.579 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.579 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.579 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.579 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.579 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.579 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.580 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.580 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.580 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.580 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.580 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.580 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.580 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.580 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.580 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.580 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.580 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.580 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.580 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.580 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.581 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.581 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.581 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.581 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.581 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.581 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.581 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.581 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.581 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.581 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.581 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.581 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.581 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.582 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.582 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.582 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.582 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.582 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.582 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.582 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.582 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.582 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.582 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.582 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.582 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.583 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_multipart_copy_versioned ______________ 2024-06-28T16:14:10.583 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.583 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.583 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.583 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.583 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.583 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.583 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.583 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.583 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.583 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.584 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.584 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.584 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.584 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.584 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.584 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.584 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.584 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.584 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.584 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.584 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.584 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.584 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.584 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.585 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.585 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.585 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.585 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.585 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.585 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.585 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.585 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.585 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.585 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.585 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.585 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.586 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.586 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.586 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.586 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.586 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.586 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.586 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.586 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.586 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.586 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.586 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.586 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.587 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.587 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.587 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.587 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.587 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.587 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.587 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.587 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.587 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.587 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.587 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.587 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.587 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.587 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.588 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.589 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.589 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.589 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.589 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.589 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.589 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.589 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.589 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.589 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.589 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.589 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.589 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.589 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.590 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_multipart_upload_resend_part ____________ 2024-06-28T16:14:10.590 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.590 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.590 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.590 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.590 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.590 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.590 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.590 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.590 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.590 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.590 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.590 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.591 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.591 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.591 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.591 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.591 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.591 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.591 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.591 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.591 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.591 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.591 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.591 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.591 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.592 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.592 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.592 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.592 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.592 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.592 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.592 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.592 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.592 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.592 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.592 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.592 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.593 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.593 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.593 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.593 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.593 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.593 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.593 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.593 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.593 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.593 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.593 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.593 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.593 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.593 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.594 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.594 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.594 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.594 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.594 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.594 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.594 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.594 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.594 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.594 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.594 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.594 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.594 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.595 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.595 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.595 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.595 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.595 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.595 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.595 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.595 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.595 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.595 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.595 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.595 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.595 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.595 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.596 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.596 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.596 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.596 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.596 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.596 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.596 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.596 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.596 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.596 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.596 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_multipart_upload_multiple_sizes ___________ 2024-06-28T16:14:10.596 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.596 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.596 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.597 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.597 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.597 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.597 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.597 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.597 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.597 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.597 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.597 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.597 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.597 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.597 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.598 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.598 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.598 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.598 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.598 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.598 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.598 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.598 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.598 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.598 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.598 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.598 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.598 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.599 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.599 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.599 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.599 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.599 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.599 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.599 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.599 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.599 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.599 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.600 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.600 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.600 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.600 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.600 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.600 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.600 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.600 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.600 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.600 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.600 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.600 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.600 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.600 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.601 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.601 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.601 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.601 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.601 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.601 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.601 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.601 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.601 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.601 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.601 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.601 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.601 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.601 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.602 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.602 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.602 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.602 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.602 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.602 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.602 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.602 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.602 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.602 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.602 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.602 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.602 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.603 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.603 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.603 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.603 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.603 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.603 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.603 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.603 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.603 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.603 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_multipart_copy_multiple_sizes ____________ 2024-06-28T16:14:10.603 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.603 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.603 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.603 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.604 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.604 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.604 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.604 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.604 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.604 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.604 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.604 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.604 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.604 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.604 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.604 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.604 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.605 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.605 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.605 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.605 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.605 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.605 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.605 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.605 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.605 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.605 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.605 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.605 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.605 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.605 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.606 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.606 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.606 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.606 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.606 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.606 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.606 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.606 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.606 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.606 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.606 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.607 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.607 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.607 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.607 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.607 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.607 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.607 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.607 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.607 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.607 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.607 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.607 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.607 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.607 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.608 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.608 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.608 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.608 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.608 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.608 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.608 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.608 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.608 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.608 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.608 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.608 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.608 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.609 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.610 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.610 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.610 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.610 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.610 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.610 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_multipart_upload_size_too_small ___________ 2024-06-28T16:14:10.610 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.610 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.610 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.610 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.610 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.610 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.610 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.611 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.611 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.611 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.611 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.611 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.611 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.611 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.611 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.611 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.611 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.611 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.611 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.611 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.611 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.612 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.612 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.612 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.612 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.612 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.612 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.612 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.612 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.612 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.612 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.612 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.612 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.613 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.613 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.613 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.613 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.613 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.613 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.613 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.613 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.613 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.613 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.613 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.613 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.614 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.614 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.614 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.614 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.614 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.614 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.614 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.614 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.614 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.614 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.614 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.614 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.614 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.615 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.615 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.615 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.615 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.615 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.615 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.615 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.615 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.615 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.615 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.615 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.615 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.615 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.615 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.616 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.616 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.616 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.616 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.616 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.616 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.616 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.616 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.616 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.616 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.616 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.616 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.616 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.616 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.617 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.617 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.617 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_multipart_upload_contents ______________ 2024-06-28T16:14:10.617 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.617 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.617 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.617 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.617 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.617 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.617 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.617 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.617 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.617 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.618 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.618 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.618 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.618 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.618 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.618 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.618 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.618 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.618 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.618 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.618 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.618 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.618 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.618 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.619 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.619 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.619 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.619 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.619 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.619 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.619 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.619 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.619 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.619 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.619 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.619 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.620 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.620 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.620 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.620 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.620 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.620 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.620 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.620 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.620 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.620 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.620 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.620 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.620 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.620 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.621 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.621 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.621 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.621 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.621 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.621 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.621 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.621 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.621 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.621 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.621 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.621 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.621 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.621 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.622 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.622 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.622 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.622 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.622 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.622 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.622 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.622 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.622 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.622 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.622 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.622 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.622 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.623 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.623 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.623 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.623 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.623 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.623 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.623 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.623 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.623 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.623 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.623 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.623 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.623 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_multipart_upload_overwrite_existing_object _____ 2024-06-28T16:14:10.623 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.624 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.624 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.624 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.624 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.624 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.624 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.624 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.624 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.624 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.624 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.624 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.624 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.624 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.624 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.625 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.625 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.625 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.625 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.625 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.625 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.625 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.625 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.625 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.625 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.625 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.625 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.625 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.625 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.626 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.626 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.626 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.626 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.626 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.626 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.626 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.626 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.626 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.626 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.627 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.627 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.627 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.627 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.627 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.627 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.627 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.627 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.627 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.627 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.627 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.627 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.627 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.627 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.628 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.628 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.628 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.628 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.628 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.628 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.628 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.628 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.628 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.628 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.628 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.628 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.628 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.628 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.629 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.629 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.629 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.629 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.629 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.629 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.629 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.629 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.629 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.629 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.629 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.629 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.629 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.630 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.630 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.630 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.630 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.630 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.630 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.630 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.630 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.630 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.630 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_abort_multipart_upload _______________ 2024-06-28T16:14:10.630 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.630 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.630 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.630 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.631 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.631 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.631 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.631 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.631 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.631 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.631 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.631 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.631 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.631 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.631 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.631 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.631 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.631 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.632 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.632 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.632 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.632 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.632 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.632 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.632 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.632 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.632 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.632 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.632 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.632 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.632 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.633 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.633 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.633 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.633 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.633 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.633 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.633 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.633 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.633 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.633 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.633 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.633 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.633 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.633 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.634 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.634 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.634 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.634 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.634 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.634 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.634 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.634 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.634 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.634 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.634 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.634 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.634 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.634 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.635 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.635 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.635 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.635 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.635 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.635 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.635 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.635 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.635 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.635 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.635 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.635 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.635 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.636 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.636 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.636 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.636 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.636 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.636 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.636 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.636 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.636 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.636 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.636 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.636 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.636 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.636 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.637 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.637 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.637 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.637 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.637 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_abort_multipart_upload_not_found __________ 2024-06-28T16:14:10.637 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.637 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.637 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.637 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.637 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.637 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.637 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.637 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.637 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.638 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.638 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.638 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.638 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.638 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.638 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.638 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.638 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.638 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.638 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.638 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.638 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.638 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.639 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.639 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.639 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.639 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.639 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.639 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.639 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.639 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.639 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.639 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.639 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.639 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.639 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.640 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.641 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.641 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.641 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.641 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.641 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.641 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.641 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.641 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.641 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.641 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.641 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.642 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.643 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.643 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.643 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.643 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.643 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.643 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.643 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.643 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.643 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.643 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.643 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.643 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.643 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.644 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.644 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.644 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.644 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_list_multipart_upload ________________ 2024-06-28T16:14:10.644 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.644 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.644 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.644 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.644 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.644 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.644 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.644 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.644 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.644 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.645 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.645 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.645 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.645 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.645 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.645 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.645 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.645 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.645 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.645 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.645 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.645 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.645 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.646 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.646 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.646 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.646 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.646 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.646 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.646 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.646 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.646 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.646 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.646 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.646 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.646 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.647 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.648 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.649 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.650 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.650 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.650 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.650 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.650 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.650 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.650 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.650 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.650 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.650 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.650 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.650 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.650 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.651 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.651 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_list_multipart_upload_owner _____________ 2024-06-28T16:14:10.651 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.651 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.651 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.651 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.651 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.651 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.651 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.651 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.651 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.651 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.651 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.652 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.652 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.652 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.652 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.652 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.652 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.652 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.652 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.652 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.652 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.652 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.652 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.652 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.652 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.653 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.653 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.653 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.653 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.653 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.653 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.653 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.653 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.653 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.653 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.653 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.653 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.653 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.653 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.654 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.654 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.654 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.654 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.654 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.654 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.654 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.654 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.654 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.654 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.654 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.654 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.655 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.655 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.655 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.655 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.655 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.655 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.655 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.655 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.655 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.655 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.655 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.655 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.655 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.655 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.656 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.656 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.656 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.656 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.656 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.656 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.656 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.656 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.656 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.656 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.656 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.657 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.657 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.657 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.657 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.657 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.657 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.657 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.657 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.657 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.657 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.657 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.657 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.657 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.658 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.658 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_multipart_upload_missing_part ____________ 2024-06-28T16:14:10.658 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.658 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.658 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.658 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.658 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.658 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.658 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.658 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.658 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.658 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.658 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.659 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.659 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.659 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.659 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.659 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.659 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.659 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.659 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.659 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.659 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.659 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.659 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.659 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.660 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.660 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.660 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.660 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.660 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.660 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.660 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.660 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.660 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.660 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.660 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.660 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.660 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.661 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.661 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.661 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.661 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.661 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.661 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.661 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.661 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.661 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.661 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.661 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.661 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.661 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.662 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.662 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.662 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.662 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.662 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.662 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.662 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.662 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.662 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.662 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.662 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.662 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.663 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.663 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.663 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.663 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.663 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.663 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.663 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.663 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.663 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.663 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.663 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.663 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.663 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.663 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_multipart_upload_incorrect_etag ___________ 2024-06-28T16:14:10.664 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.665 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.665 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.665 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.665 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.665 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.665 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.665 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.665 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.665 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.665 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.665 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.665 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.665 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.666 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.666 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.666 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.666 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.666 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.666 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.666 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.666 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.666 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.666 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.666 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.666 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.666 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.667 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.667 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.667 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.667 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.667 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.667 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.667 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.667 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.667 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.667 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.667 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.667 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.668 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.668 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.668 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.668 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.668 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.668 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.668 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.668 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.668 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.668 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.668 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.668 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.668 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.668 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.669 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.669 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.669 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.669 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.669 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.669 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.669 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.669 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.669 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.669 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.669 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.669 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.669 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.669 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.670 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.670 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.670 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.670 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.670 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.670 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.670 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.670 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.670 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.670 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.670 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.670 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.671 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.671 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.671 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.671 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.671 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.671 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.671 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.671 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.671 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.671 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_multipart_get_part _________________ 2024-06-28T16:14:10.671 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.671 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.671 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.672 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.672 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.672 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.672 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.672 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.672 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.672 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.672 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.672 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.672 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.672 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.672 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.672 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.673 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.673 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.673 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.673 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.673 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.673 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.673 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.673 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.673 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.673 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.673 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.673 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.673 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.673 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.674 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.674 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.674 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.674 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.674 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.674 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.674 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.674 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.674 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.674 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.674 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.674 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.675 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.675 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.675 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.675 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.675 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.675 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.675 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.675 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.675 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.675 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.675 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.675 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.675 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.676 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.676 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.676 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.676 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.676 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.676 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.676 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.676 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.676 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.676 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.676 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.676 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.676 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.676 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.677 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.677 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.677 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.677 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.677 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.677 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.677 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.677 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.677 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.677 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.677 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.677 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.677 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.678 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.678 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.678 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.678 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.678 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.678 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.678 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.678 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_non_multipart_get_part _______________ 2024-06-28T16:14:10.678 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.678 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.678 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.678 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.679 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.679 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.679 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.679 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.679 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.679 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.679 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.679 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.679 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.679 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.679 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.679 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.679 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.679 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.680 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.680 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.680 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.680 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.680 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.680 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.680 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.680 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.680 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.680 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.680 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.680 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.680 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.680 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.681 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.681 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.681 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.681 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.681 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.681 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.681 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.681 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.681 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.681 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.681 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.681 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.681 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.681 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.682 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.682 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.682 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.682 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.682 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.682 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.682 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.682 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.682 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.682 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.682 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.682 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.682 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.683 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.683 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.683 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.683 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.683 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.683 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.683 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.683 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.683 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.683 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.683 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.683 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.683 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.683 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.684 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.684 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.684 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.684 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.684 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.684 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.684 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.684 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.684 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.684 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.684 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.684 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.684 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.685 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.685 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.685 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.685 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.685 INFO:teuthology.orchestra.run.smithi178.stdout:____________________ ERROR at teardown of test_100_continue ____________________ 2024-06-28T16:14:10.685 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.685 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.685 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.685 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.685 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.685 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.685 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.686 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.686 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.686 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.686 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.686 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.686 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.686 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.686 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.686 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.686 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.686 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.686 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.686 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.687 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.687 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.687 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.687 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.687 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.687 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.687 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.687 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.687 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.687 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.687 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.687 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.687 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.688 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.688 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.688 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.688 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.688 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.688 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.688 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.688 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.688 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.688 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.688 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.688 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.688 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.688 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.689 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.689 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.689 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.689 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.689 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.689 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.689 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.689 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.689 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.689 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.689 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.689 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.689 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.689 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.690 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.690 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.690 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.690 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.690 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.690 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.690 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.690 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.690 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.690 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.690 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.690 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.690 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.691 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.691 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.691 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.691 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.691 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.691 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.691 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.691 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.691 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.691 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.691 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.691 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.691 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.692 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.692 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.692 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.692 INFO:teuthology.orchestra.run.smithi178.stdout:______________________ ERROR at teardown of test_set_cors ______________________ 2024-06-28T16:14:10.692 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.692 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.692 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.692 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.692 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.692 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.692 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.692 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.692 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.693 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.693 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.693 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.693 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.693 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.693 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.693 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.693 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.693 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.693 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.693 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.693 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.693 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.693 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.694 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.694 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.694 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.694 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.694 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.694 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.694 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.694 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.694 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.694 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.694 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.694 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.694 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.694 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.695 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.695 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.695 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.695 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.695 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.695 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.695 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.695 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.695 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.695 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.695 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.695 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.695 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.696 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.696 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.696 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.696 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.696 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.696 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.696 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.696 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.696 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.696 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.696 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.696 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.696 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.697 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.697 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.697 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.697 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.697 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.697 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.697 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.697 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.697 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.697 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.697 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.697 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.697 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.697 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.698 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.698 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.698 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.698 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.698 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.698 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.698 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.698 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.698 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.698 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.698 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.698 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.698 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.699 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_cors_origin_response ________________ 2024-06-28T16:14:10.699 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.699 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.699 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.699 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.699 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.699 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.699 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.699 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.699 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.699 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.699 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.699 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.700 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.700 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.700 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.700 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.700 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.700 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.700 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.700 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.700 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.700 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.700 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.700 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.701 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.701 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.701 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.701 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.701 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.701 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.701 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.701 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.701 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.701 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.701 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.701 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.701 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.701 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.702 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.702 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.702 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.702 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.702 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.702 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.702 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.702 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.702 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.702 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.702 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.702 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.702 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.703 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.704 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.705 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.705 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.705 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.705 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.705 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.705 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.705 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.705 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.705 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.705 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.705 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.705 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.705 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_cors_origin_wildcard ________________ 2024-06-28T16:14:10.705 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.706 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.706 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.706 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.706 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.706 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.706 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.706 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.706 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.706 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.706 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.706 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.706 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.706 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.707 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.707 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.707 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.707 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.707 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.707 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.707 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.707 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.707 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.707 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.707 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.707 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.707 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.708 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.708 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.708 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.708 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.708 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.708 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.708 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.708 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.708 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.708 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.708 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.708 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.708 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.708 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.709 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.709 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.709 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.709 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.709 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.709 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.709 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.709 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.709 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.709 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.709 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.709 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.709 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.710 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.710 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.710 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.710 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.710 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.710 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.710 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.710 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.710 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.710 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.710 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.710 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.710 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.711 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.711 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.711 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.711 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.711 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.711 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.711 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.711 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.711 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.711 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.711 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.711 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.711 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.711 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.712 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.712 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.712 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.712 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.712 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.712 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.712 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.712 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.712 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.712 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_cors_header_option _________________ 2024-06-28T16:14:10.712 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.712 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.712 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.713 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.713 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.713 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.713 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.713 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.713 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.713 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.713 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.713 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.713 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.713 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.713 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.713 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.714 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.714 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.714 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.714 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.714 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.714 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.714 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.714 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.714 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.714 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.714 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.715 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.715 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.715 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.715 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.715 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.715 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.715 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.715 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.715 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.715 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.715 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.715 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.715 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.715 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.716 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.716 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.716 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.716 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.716 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.716 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.716 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.716 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.716 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.716 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.716 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.716 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.716 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.717 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.717 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.717 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.717 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.717 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.717 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.717 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.717 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.717 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.717 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.717 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.717 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.717 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.717 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.718 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.718 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.718 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.718 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.718 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.718 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.718 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.718 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.718 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.718 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.718 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.718 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.719 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.719 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.719 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.719 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.719 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.719 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.719 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.719 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.719 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.719 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.719 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_cors_presigned_get_object ______________ 2024-06-28T16:14:10.719 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.719 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.720 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.720 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.720 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.720 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.720 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.720 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.720 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.720 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.720 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.720 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.720 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.720 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.720 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.720 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.721 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.721 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.721 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.721 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.721 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.721 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.721 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.721 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.721 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.721 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.721 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.721 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.721 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.722 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.722 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.722 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.722 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.722 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.722 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.722 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.722 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.722 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.722 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.722 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.722 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.722 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.722 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.723 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.723 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.723 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.723 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.723 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.723 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.723 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.723 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.723 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.723 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.723 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.723 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.723 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.724 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.724 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.724 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.724 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.724 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.724 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.724 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.724 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.724 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.724 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.724 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.724 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.724 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.724 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.725 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.725 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.725 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.725 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.725 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.725 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.725 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.725 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.725 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.725 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.725 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.725 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.725 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.726 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.726 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.726 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.726 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.726 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.726 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.726 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.726 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_cors_presigned_get_object_tenant __________ 2024-06-28T16:14:10.726 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.726 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.726 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.726 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.726 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.726 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.727 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.727 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.727 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.727 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.727 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.727 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.727 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.727 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.727 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.727 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.727 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.727 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.727 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.727 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.728 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.728 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.728 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.728 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.728 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.728 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.728 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.728 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.728 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.728 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.728 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.728 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.728 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.729 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.729 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.729 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.729 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.729 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.729 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.729 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.729 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.729 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.729 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.729 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.729 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.729 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.730 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.730 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.730 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.730 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.730 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.730 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.730 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.730 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.730 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.730 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.730 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.730 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.730 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.731 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.731 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.731 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.731 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.731 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.731 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.731 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.731 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.731 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.731 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.731 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.731 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.731 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.731 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.732 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.732 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.732 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.732 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.732 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.732 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.732 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.732 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.732 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.732 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.732 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.732 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.732 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.733 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.733 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.733 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.733 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.733 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_cors_presigned_put_object ______________ 2024-06-28T16:14:10.733 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.733 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.733 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.733 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.733 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.733 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.733 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.733 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.734 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.734 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.734 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.734 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.734 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.734 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.734 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.734 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.734 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.734 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.734 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.734 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.734 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.735 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.735 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.735 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.735 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.735 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.735 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.735 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.735 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.735 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.735 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.735 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.735 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.735 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.736 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.736 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.736 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.736 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.736 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.736 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.736 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.736 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.736 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.736 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.736 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.736 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.736 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.737 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.737 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.737 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.737 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.737 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.737 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.737 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.737 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.737 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.737 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.737 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.737 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.737 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.737 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.738 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.738 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.738 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.738 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.738 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.738 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.738 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.738 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.738 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.738 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.738 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.738 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.738 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.739 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.739 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.739 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.739 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.739 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.739 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.739 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.739 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.739 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.739 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.739 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.739 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.739 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.740 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.740 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.740 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_cors_presigned_put_object_with_acl _________ 2024-06-28T16:14:10.740 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.740 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.740 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.740 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.740 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.740 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.740 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.740 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.740 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.740 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.741 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.741 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.741 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.741 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.741 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.741 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.741 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.741 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.741 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.741 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.741 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.741 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.741 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.742 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.742 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.742 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.742 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.742 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.742 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.742 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.742 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.742 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.742 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.742 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.742 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.742 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.743 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.743 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.743 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.743 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.743 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.743 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.743 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.743 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.743 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.743 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.743 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.743 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.743 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.743 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.744 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.744 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.744 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.744 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.744 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.744 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.744 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.744 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.744 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.744 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.744 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.744 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.745 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.745 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.745 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.745 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.745 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.745 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.745 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.745 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.745 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.745 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.745 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.745 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.745 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.746 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.746 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.746 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.746 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.746 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.746 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.746 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.746 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.746 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.746 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.746 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.746 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.746 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.747 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.747 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.747 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_cors_presigned_put_object_tenant __________ 2024-06-28T16:14:10.747 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.747 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.747 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.747 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.747 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.747 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.747 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.747 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.747 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.747 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.748 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.748 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.748 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.748 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.748 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.748 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.748 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.748 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.748 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.748 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.748 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.748 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.748 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.748 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.749 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.749 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.749 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.749 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.749 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.749 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.749 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.749 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.749 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.749 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.749 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.749 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.749 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.750 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.750 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.750 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.750 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.750 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.750 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.750 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.750 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.750 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.750 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.750 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.750 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.751 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.751 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.751 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.751 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.751 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.751 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.751 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.751 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.751 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.751 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.751 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.751 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.751 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.752 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.752 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.752 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.752 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.752 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.752 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.752 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.752 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.752 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.752 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.752 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.752 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.752 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.752 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.753 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.753 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.753 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.753 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.753 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.753 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.753 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.753 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.753 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.753 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.753 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.753 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.753 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.754 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_cors_presigned_put_object_tenant_with_acl ______ 2024-06-28T16:14:10.754 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.754 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.754 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.754 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.754 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.754 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.754 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.754 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.754 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.754 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.754 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.754 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.755 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.755 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.755 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.755 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.755 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.755 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.755 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.755 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.755 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.755 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.755 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.755 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.755 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.756 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.756 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.756 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.756 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.756 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.756 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.756 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.756 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.756 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.756 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.756 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.756 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.756 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.757 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.757 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.757 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.757 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.757 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.757 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.757 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.757 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.757 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.757 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.757 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.757 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.757 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.757 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.758 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.758 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.758 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.758 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.758 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.758 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.758 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.758 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.758 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.758 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.758 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.758 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.758 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.759 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.759 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.759 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.759 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.759 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.759 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.759 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.759 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.759 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.759 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.759 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.760 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.760 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.760 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.760 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.760 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.760 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.760 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.760 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.760 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.760 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.760 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.760 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.760 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.761 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.761 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_set_bucket_tagging _________________ 2024-06-28T16:14:10.761 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.761 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.761 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.761 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.761 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.761 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.761 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.761 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.761 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.761 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.761 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.762 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.762 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.762 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.762 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.762 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.762 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.762 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.762 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.762 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.762 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.762 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.762 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.762 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.763 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.763 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.763 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.763 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.763 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.763 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.763 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.763 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.763 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.763 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.763 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.763 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.764 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.764 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.764 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.764 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.764 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.764 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.764 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.764 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.764 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.764 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.764 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.764 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.765 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.765 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.765 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.765 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.765 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.765 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.765 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.765 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.765 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.765 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.765 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.765 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.765 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.766 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.766 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.766 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.766 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.766 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.766 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.766 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.766 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.766 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.766 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.766 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.766 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.767 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.767 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.767 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.767 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.767 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.767 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.767 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.767 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.767 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.767 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.767 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.767 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.768 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.768 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.768 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.768 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_atomic_read_1mb ___________________ 2024-06-28T16:14:10.768 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.768 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.768 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.768 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.768 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.768 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.768 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.768 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.768 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.768 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.769 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.769 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.769 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.769 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.769 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.769 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.769 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.769 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.769 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.769 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.769 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.769 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.769 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.770 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.770 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.770 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.770 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.770 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.770 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.770 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.770 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.770 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.770 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.770 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.770 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.771 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.771 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.771 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.771 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.771 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.771 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.771 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.771 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.771 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.771 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.771 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.771 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.771 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.772 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.772 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.772 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.772 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.772 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.772 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.772 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.772 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.772 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.772 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.772 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.772 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.772 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.773 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.773 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.773 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.773 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.773 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.773 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.773 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.773 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.773 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.773 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.773 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.773 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.773 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.774 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.774 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.774 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.774 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.774 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.774 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.774 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.774 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.774 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.774 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.774 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.775 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.775 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.775 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.775 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.775 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_atomic_read_4mb ___________________ 2024-06-28T16:14:10.775 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.775 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.775 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.775 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.775 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.775 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.775 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.775 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.776 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.776 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.776 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.776 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.776 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.776 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.776 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.776 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.776 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.776 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.776 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.776 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.776 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.777 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.777 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.777 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.777 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.777 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.777 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.777 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.777 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.777 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.777 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.777 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.777 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.778 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.778 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.778 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.778 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.778 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.778 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.778 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.778 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.778 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.778 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.778 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.779 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.779 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.779 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.779 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.779 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.779 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.779 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.779 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.779 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.779 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.779 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.779 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.779 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.779 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.780 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.780 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.780 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.780 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.780 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.780 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.780 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.781 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.781 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.781 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.781 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.781 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.781 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.781 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.781 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.782 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.782 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.782 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.782 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.782 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.782 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.782 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.782 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.782 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.782 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.782 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.782 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.783 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.783 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.783 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.783 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_atomic_read_8mb ___________________ 2024-06-28T16:14:10.783 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.783 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.783 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.783 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.783 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.783 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.783 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.783 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.783 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.784 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.784 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.784 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.784 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.784 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.784 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.784 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.784 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.784 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.784 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.784 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.784 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.785 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.785 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.785 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.785 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.785 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.785 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.785 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.785 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.785 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.785 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.785 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.785 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.785 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.786 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.786 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.786 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.786 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.786 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.787 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.787 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.787 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.787 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.787 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.787 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.787 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.787 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.787 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.787 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.787 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.787 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.787 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.788 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.788 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.788 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.788 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.788 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.788 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.788 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.788 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.788 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.788 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.788 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.788 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.788 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.789 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.789 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.789 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.789 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.789 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.789 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.789 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.789 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.789 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.789 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.789 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.789 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.789 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.790 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.790 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.790 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.790 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.790 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_atomic_write_1mb __________________ 2024-06-28T16:14:10.790 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.790 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.790 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.790 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.790 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.790 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.791 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.791 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.791 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.791 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.791 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.791 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.791 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.791 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.791 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.791 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.791 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.791 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.791 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.792 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.792 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.792 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.792 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.792 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.792 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.792 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.792 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.792 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.792 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.792 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.792 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.792 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.793 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.793 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.793 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.793 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.793 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.794 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.794 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.794 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.794 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.794 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.794 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.794 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.794 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.794 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.794 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.794 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.794 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.795 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.795 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.795 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.795 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.795 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.795 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.795 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.795 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.795 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.795 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.795 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.795 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.795 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.796 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.796 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.796 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.796 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.796 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.796 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.796 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.796 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.796 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.796 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.796 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.796 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.796 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.797 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.797 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.797 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.797 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.797 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.797 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.797 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.797 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_atomic_write_4mb __________________ 2024-06-28T16:14:10.797 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.797 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.797 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.797 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.797 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.798 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.798 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.798 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.798 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.798 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.798 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.798 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.798 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.798 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.798 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.798 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.799 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.799 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.799 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.799 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.799 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.799 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.799 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.799 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.799 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.799 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.799 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.799 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.799 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.799 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.800 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.800 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.800 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.800 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.800 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.800 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.800 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.800 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.800 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.800 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.800 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.800 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.801 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.801 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.801 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.801 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.801 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.801 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.801 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.801 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.801 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.801 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.801 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.801 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.801 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.802 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.802 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.802 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.802 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.802 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.802 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.802 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.802 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.802 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.802 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.802 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.802 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.802 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.803 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.803 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.803 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.803 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.803 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.803 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.803 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.803 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.803 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.803 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.803 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.803 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.803 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.804 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.804 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.804 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.804 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.804 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.804 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.804 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.804 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.804 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_atomic_write_8mb __________________ 2024-06-28T16:14:10.804 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.804 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.804 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.804 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.805 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.805 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.805 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.805 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.805 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.805 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.805 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.805 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.805 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.805 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.805 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.805 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.805 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.806 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.806 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.806 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.806 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.806 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.806 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.806 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.806 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.806 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.806 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.806 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.807 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.807 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.807 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.807 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.807 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.807 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.807 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.807 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.807 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.807 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.807 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.807 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.808 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.808 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.808 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.808 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.808 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.808 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.808 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.808 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.808 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.808 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.808 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.808 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.808 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.808 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.809 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.809 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.809 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.809 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.809 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.809 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.809 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.809 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.809 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.809 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.809 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.809 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.809 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.810 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.810 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.810 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.810 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.810 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.810 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.810 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.810 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.810 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.810 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.810 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.810 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.810 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.811 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.811 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.811 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.811 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.811 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.811 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.811 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.811 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_atomic_dual_write_1mb ________________ 2024-06-28T16:14:10.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.811 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.812 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.812 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.812 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.812 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.812 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.812 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.812 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.812 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.812 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.812 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.813 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.813 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.813 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.813 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.813 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.813 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.813 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.813 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.813 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.813 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.813 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.813 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.813 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.813 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.814 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.814 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.814 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.814 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.814 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.814 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.814 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.814 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.814 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.814 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.814 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.815 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.815 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.815 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.815 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.815 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.815 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.815 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.815 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.815 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.815 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.815 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.815 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.815 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.815 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.816 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.816 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.816 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.816 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.816 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.816 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.816 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.816 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.816 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.816 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.816 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.816 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.816 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.817 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.817 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.817 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.817 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.817 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.817 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.817 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.817 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.817 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.817 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.817 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.817 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.817 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.817 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.818 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.818 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.818 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.818 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.818 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.818 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.818 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.818 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_atomic_dual_write_4mb ________________ 2024-06-28T16:14:10.818 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.818 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.818 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.818 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.819 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.819 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.819 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.819 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.819 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.819 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.819 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.819 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.819 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.820 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.820 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.820 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.820 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.820 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.820 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.820 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.820 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.820 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.820 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.820 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.820 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.821 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.821 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.821 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.821 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.821 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.821 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.821 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.822 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.822 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.822 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.822 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.822 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.822 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.822 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.822 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.822 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.822 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.822 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.822 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.822 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.822 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.823 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.823 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.823 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.823 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.823 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.823 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.823 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.823 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.823 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.823 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.823 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.823 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.823 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.823 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.824 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.824 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.824 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.824 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.824 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.824 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.824 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.824 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.824 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.824 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.824 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.824 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.824 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.825 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.825 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.825 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.825 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.825 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.825 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.825 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_atomic_dual_write_8mb ________________ 2024-06-28T16:14:10.825 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.825 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.825 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.825 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.825 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.825 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.825 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.826 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.826 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.826 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.826 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.826 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.826 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.826 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.826 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.826 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.826 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.826 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.826 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.826 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.827 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.827 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.827 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.827 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.827 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.827 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.827 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.827 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.827 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.827 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.827 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.827 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.827 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.828 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.828 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.828 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.828 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.828 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.829 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.829 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.829 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.829 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.829 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.829 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.829 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.829 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.829 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.829 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.829 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.829 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.829 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.830 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.830 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.830 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.830 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.830 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.830 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.830 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.830 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.830 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.830 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.830 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.830 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.830 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.831 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.831 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.831 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.831 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.831 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.831 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.831 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.831 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.831 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.831 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.831 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.831 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.831 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.831 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.832 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.832 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.832 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.832 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_atomic_conditional_write_1mb ____________ 2024-06-28T16:14:10.832 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.832 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.832 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.832 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.832 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.832 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.832 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.832 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.832 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.832 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.833 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.833 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.833 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.833 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.833 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.833 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.833 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.833 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.833 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.833 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.833 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.833 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.833 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.834 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.834 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.834 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.834 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.834 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.834 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.834 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.834 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.834 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.834 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.834 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.834 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.835 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.835 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.835 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.835 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.835 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.836 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.836 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.836 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.836 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.836 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.836 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.836 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.836 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.836 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.836 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.836 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.836 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.836 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.836 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.838 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.838 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.838 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.838 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.838 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.838 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.838 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.838 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.838 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.838 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.838 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.838 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.838 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.839 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.839 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.839 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_atomic_multipart_upload_write ____________ 2024-06-28T16:14:10.839 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.839 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.839 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.839 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.839 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.839 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.839 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.839 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.839 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.839 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.839 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.840 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.840 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.840 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.840 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.840 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.840 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.840 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.840 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.840 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.840 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.840 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.840 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.840 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.840 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.841 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.841 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.841 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.841 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.841 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.841 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.841 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.841 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.841 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.841 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.841 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.842 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.842 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.842 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.842 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.842 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.842 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.842 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.842 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.842 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.842 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.842 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.842 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.842 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.843 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.843 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.843 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.843 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.843 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.843 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.843 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.843 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.843 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.843 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.843 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.843 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.843 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.843 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.844 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.844 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.844 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.844 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.844 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.844 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.844 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.844 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.844 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.844 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.844 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.844 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.844 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.844 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.845 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.845 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.845 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.845 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.845 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.845 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.845 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.845 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.845 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.845 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.845 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.845 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_multipart_resend_first_finishes_last ________ 2024-06-28T16:14:10.845 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.846 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.846 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.846 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.846 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.846 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.846 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.846 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.846 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.846 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.846 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.846 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.846 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.846 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.847 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.847 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.847 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.847 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.847 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.847 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.847 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.847 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.847 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.847 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.847 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.847 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.847 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.848 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.849 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.849 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.849 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.849 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.849 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.849 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.849 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.849 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.849 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.849 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.849 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.849 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.849 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.850 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.850 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.850 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.850 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.850 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.850 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.850 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.850 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.850 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.850 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.850 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.850 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.851 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.851 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.851 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.851 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.851 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.851 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.851 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.851 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.851 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.851 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.851 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.851 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.851 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.852 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.852 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.852 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.852 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.852 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.852 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.852 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.852 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.852 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.852 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.852 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.852 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_ranged_request_response_code ____________ 2024-06-28T16:14:10.852 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.853 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.853 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.853 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.853 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.853 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.853 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.853 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.853 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.853 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.853 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.853 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.853 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.853 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.854 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.854 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.854 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.854 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.854 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.854 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.854 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.854 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.854 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.854 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.854 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.854 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.855 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.855 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.855 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.855 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.855 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.855 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.855 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.855 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.856 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.856 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.856 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.856 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.856 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.856 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.856 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.856 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.856 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.856 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.856 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.856 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.856 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.857 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.857 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.857 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.857 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.857 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.857 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.857 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.857 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.857 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.857 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.857 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.857 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.857 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.857 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.858 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.858 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.858 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.858 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.858 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.858 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.858 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.858 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.858 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.858 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.858 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.858 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.859 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.859 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.859 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.859 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.859 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.859 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.859 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.859 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.859 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.859 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.859 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.859 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.859 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_ranged_big_request_response_code __________ 2024-06-28T16:14:10.859 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.860 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.860 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.860 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.860 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.860 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.860 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.860 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.860 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.860 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.860 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.861 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.861 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.861 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.861 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.861 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.861 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.861 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.861 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.861 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.861 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.861 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.861 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.861 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.861 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.862 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.862 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.862 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.862 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.862 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.862 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.862 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.862 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.862 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.862 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.862 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.863 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.863 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.863 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.863 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.863 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.863 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.863 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.863 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.863 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.863 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.863 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.863 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.863 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.864 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.864 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.864 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.864 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.864 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.864 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.864 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.864 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.864 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.864 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.864 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.864 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.865 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.865 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.865 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.865 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.865 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.865 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.865 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.865 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.865 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.865 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.865 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.865 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.866 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.866 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.866 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.866 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.866 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.866 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.866 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.866 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.866 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.866 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.866 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.866 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.866 INFO:teuthology.orchestra.run.smithi178.stdout:__ ERROR at teardown of test_ranged_request_skip_leading_bytes_response_code ___ 2024-06-28T16:14:10.867 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.867 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.867 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.867 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.867 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.867 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.867 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.867 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.867 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.867 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.867 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.867 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.867 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.867 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.868 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.868 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.868 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.868 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.868 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.868 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.868 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.868 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.868 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.868 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.868 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.868 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.868 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.869 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.869 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.869 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.869 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.869 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.869 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.869 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.869 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.869 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.869 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.869 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.870 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.870 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.870 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.870 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.870 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.870 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.870 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.870 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.870 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.870 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.870 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.870 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.871 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.871 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.871 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.871 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.871 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.871 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.871 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.871 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.871 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.871 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.871 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.871 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.871 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.872 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.872 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.872 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.872 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.872 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.872 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.872 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.872 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.872 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.872 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.872 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.872 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.872 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.873 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.873 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.873 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.873 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.873 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.873 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.873 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.873 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.873 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.873 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.873 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.873 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.873 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_ranged_request_return_trailing_bytes_response_code _ 2024-06-28T16:14:10.874 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.874 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.874 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.874 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.874 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.874 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.874 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.874 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.874 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.874 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.874 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.874 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.875 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.875 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.875 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.875 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.875 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.875 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.875 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.875 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.875 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.875 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.875 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.875 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.875 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.875 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.876 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.876 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.876 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.876 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.876 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.876 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.876 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.876 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.876 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.876 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.876 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.877 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.877 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.877 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.877 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.877 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.877 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.877 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.877 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.877 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.877 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.877 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.877 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.877 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.878 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.878 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.878 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.878 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.878 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.878 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.878 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.878 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.878 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.878 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.878 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.878 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.878 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.879 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.879 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.879 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.879 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.879 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.879 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.879 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.879 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.879 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.879 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.879 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.879 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.879 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.879 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.880 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.880 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.880 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.880 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.880 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.880 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.880 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.880 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.880 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.880 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.880 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.880 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_ranged_request_invalid_range ____________ 2024-06-28T16:14:10.881 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.881 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.881 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.881 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.881 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.881 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.881 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.881 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.881 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.881 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.881 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.881 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.881 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.881 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.882 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.882 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.882 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.882 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.882 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.882 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.882 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.882 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.882 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.882 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.882 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.882 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.883 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.883 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.883 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.883 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.883 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.883 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.883 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.883 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.883 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.883 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.883 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.883 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.883 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.884 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.884 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.884 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.884 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.884 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.884 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.884 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.884 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.884 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.884 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.884 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.884 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.884 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.885 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.885 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.885 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.885 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.885 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.885 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.885 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.885 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.885 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.885 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.885 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.885 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.885 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.885 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.886 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.886 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.886 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.886 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.886 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.886 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.886 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.886 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.886 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.886 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.886 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.886 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.886 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.887 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.887 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.887 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.887 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.887 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.887 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.887 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.887 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.887 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.887 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.887 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_ranged_request_empty_object _____________ 2024-06-28T16:14:10.887 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.887 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.888 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.888 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.888 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.888 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.888 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.888 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.888 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.888 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.888 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.888 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.888 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.888 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.888 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.888 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.889 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.889 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.889 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.889 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.889 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.889 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.889 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.889 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.889 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.889 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.889 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.889 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.889 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.890 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.891 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.891 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.891 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.891 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.891 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.891 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.891 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.891 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.891 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.891 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.891 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.891 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.891 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.892 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.892 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.892 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.892 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.892 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.892 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.892 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.892 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.892 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.892 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.892 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.892 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.892 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.892 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.893 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.893 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.893 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.893 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.893 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.893 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.893 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.893 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.893 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.893 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.893 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.893 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.893 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.894 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.894 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.894 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.894 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.894 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.894 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.894 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.894 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_versioning_bucket_create_suspend __________ 2024-06-28T16:14:10.894 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.894 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.894 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.894 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.894 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.895 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.895 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.895 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.895 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.895 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.895 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.895 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.895 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.895 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.895 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.895 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.895 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.896 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.896 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.896 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.896 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.896 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.896 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.896 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.896 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.896 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.896 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.896 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.896 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.896 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.896 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.897 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.897 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.897 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.897 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.897 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.897 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.897 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.897 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.897 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.897 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.897 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.897 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.897 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.898 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.898 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.898 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.898 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.898 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.898 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.898 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.898 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.898 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.898 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.898 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.898 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.898 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.899 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.899 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.899 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.899 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.899 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.899 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.899 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.899 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.899 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.899 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.899 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.899 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.899 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.899 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.900 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.900 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.900 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.900 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.900 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.900 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.900 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.900 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.900 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.900 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.900 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.900 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.900 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.900 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.901 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.901 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.901 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.901 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.901 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_versioning_obj_create_read_remove __________ 2024-06-28T16:14:10.901 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.901 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.901 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.901 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.901 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.901 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.901 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.901 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.901 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.902 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.902 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.902 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.902 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.902 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.902 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.902 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.902 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.902 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.902 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.902 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.902 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.902 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.903 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.903 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.903 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.903 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.903 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.903 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.903 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.903 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.903 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.903 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.903 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.903 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.903 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.903 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.904 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.904 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.904 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.904 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.904 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.904 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.904 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.904 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.904 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.904 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.904 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.904 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.904 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.905 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.905 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.905 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.905 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.905 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.905 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.905 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.905 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.905 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.905 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.905 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.905 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.905 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.905 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.906 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.906 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.906 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.906 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.906 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.906 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.906 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.906 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.906 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.906 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.906 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.906 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.906 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.906 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.907 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.907 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.907 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.907 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.907 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.907 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.907 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.907 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.907 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.907 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.907 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.907 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.907 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_versioning_obj_create_read_remove_head _______ 2024-06-28T16:14:10.907 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.908 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.908 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.908 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.908 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.908 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.908 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.908 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.908 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.908 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.908 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.908 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.908 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.908 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.908 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.909 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.909 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.909 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.909 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.909 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.909 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.909 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.909 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.909 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.909 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.909 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.909 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.909 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.910 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.910 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.910 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.910 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.910 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.910 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.910 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.910 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.910 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.910 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.910 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.910 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.911 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.911 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.911 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.911 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.911 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.911 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.911 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.911 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.911 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.911 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.911 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.911 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.911 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.911 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.912 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.912 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.912 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.912 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.912 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.912 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.912 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.912 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.912 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.912 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.912 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.912 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.912 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.912 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.913 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.913 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.913 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.913 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.913 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.913 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.913 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.913 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.913 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.913 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.913 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.913 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.913 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.914 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.914 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.914 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.914 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.914 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.914 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.914 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.914 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.914 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.914 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_versioning_obj_plain_null_version_removal ______ 2024-06-28T16:14:10.914 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.914 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.914 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.914 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.915 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.915 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.915 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.915 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.915 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.915 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.915 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.915 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.915 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.915 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.915 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.915 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.915 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.916 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.916 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.916 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.916 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.916 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.916 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.916 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.916 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.916 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.916 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.916 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.916 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.917 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.917 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.917 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.917 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.917 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.917 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.917 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.917 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.917 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.917 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.917 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.918 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.918 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.918 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.918 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.918 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.918 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.918 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.918 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.918 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.918 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.918 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.918 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.918 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.918 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.919 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.919 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.919 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.919 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.919 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.919 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.919 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.919 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.919 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.919 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.919 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.919 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.919 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.919 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.920 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.920 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.920 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.920 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.920 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.920 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.920 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.920 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.920 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.920 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.920 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.920 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.921 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.921 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.921 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.921 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.921 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_versioning_obj_plain_null_version_overwrite _____ 2024-06-28T16:14:10.921 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.921 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.921 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.921 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.921 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.921 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.921 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.921 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.921 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.922 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.922 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.922 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.922 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.922 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.922 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.922 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.922 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.922 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.922 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.922 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.922 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.922 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.923 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.923 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.923 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.923 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.923 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.923 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.923 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.923 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.923 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.923 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.923 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.923 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.924 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.924 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.924 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.924 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.924 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.925 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.925 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.925 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.925 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.925 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.925 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.925 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.925 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.925 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.925 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.925 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.925 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.925 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.925 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.926 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.926 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.926 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.926 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.926 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.926 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.926 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.926 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.926 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.926 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.926 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.926 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.927 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.927 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.927 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.927 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.927 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.927 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.927 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.927 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.927 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.927 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.927 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.927 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.927 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.927 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.928 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.928 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.928 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_versioning_obj_plain_null_version_overwrite_suspended _ 2024-06-28T16:14:10.928 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.928 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.928 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.928 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.928 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.928 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.928 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.928 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.928 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.928 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.928 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.929 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.929 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.929 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.929 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.929 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.929 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.929 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.929 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.929 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.929 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.929 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.929 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.929 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.929 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.930 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.930 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.930 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.930 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.930 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.930 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.930 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.930 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.930 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.930 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.930 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.931 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.931 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.931 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.931 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.931 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.931 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.931 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.931 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.931 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.931 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.931 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.931 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.931 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.932 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.932 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.932 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.932 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.932 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.932 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.932 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.932 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.932 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.932 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.932 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.932 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.932 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.932 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.933 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.933 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.933 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.933 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.933 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.933 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.933 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.933 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.933 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.933 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.933 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.933 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.933 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.933 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.934 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.934 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.934 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.934 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.934 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.934 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.934 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.934 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.934 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.934 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.934 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.934 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_versioning_obj_suspend_versions ___________ 2024-06-28T16:14:10.934 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.935 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.935 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.935 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.935 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.935 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.935 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.935 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.935 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.935 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.935 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.935 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.935 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.935 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.935 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.936 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.936 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.936 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.936 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.936 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.936 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.936 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.936 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.936 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.936 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.936 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.936 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.936 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.937 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.937 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.937 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.937 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.937 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.937 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.937 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.937 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.937 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.937 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.937 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.937 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.938 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.938 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.938 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.938 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.938 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.938 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.938 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.938 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.938 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.938 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.938 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.938 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.939 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.939 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.939 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.939 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.939 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.939 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.939 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.939 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.939 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.939 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.939 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.939 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.939 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.939 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.940 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.940 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.940 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.940 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.940 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.940 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.940 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.940 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.940 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.940 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.940 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.940 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.940 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.940 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.941 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.941 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.941 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.941 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.941 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.941 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.941 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.941 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.941 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_versioning_obj_suspended_copy ____________ 2024-06-28T16:14:10.941 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.941 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.941 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.941 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.942 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.942 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.942 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.942 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.942 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.942 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.942 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.942 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.942 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.942 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.942 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.942 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.942 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.943 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.943 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.943 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.943 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.943 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.943 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.943 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.943 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.943 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.943 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.943 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.943 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.943 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.944 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.944 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.944 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.944 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.944 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.944 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.944 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.944 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.944 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.944 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.944 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.944 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.944 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.945 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.945 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.945 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.945 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.945 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.945 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.945 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.945 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.945 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.945 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.945 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.945 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.945 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.946 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.946 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.946 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.946 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.946 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.946 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.946 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.946 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.946 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.946 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.946 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.946 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.946 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.947 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.947 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.947 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.947 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.947 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.947 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.947 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.947 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.947 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.947 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.947 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.947 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.947 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.948 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.948 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.948 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.948 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.948 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.948 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.948 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.948 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_versioning_obj_create_versions_remove_all ______ 2024-06-28T16:14:10.948 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.948 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.948 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.948 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.948 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.948 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.949 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.949 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.949 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.949 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.949 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.949 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.949 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.949 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.949 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.949 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.949 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.949 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.949 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.950 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.950 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.950 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.950 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.950 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.950 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.950 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.950 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.950 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.950 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.950 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.950 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.950 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.951 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.951 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.951 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.951 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.951 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.951 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.951 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.951 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.951 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.951 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.951 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.951 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.951 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.952 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.952 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.952 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.952 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.952 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.952 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.952 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.952 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.952 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.952 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.952 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.952 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.953 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.953 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.953 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.953 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.953 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.953 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.953 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.953 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.953 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.953 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.953 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.953 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.953 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.954 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.954 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.954 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.954 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.954 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.954 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.954 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.954 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.954 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.954 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.954 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.954 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.954 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.955 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.955 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.955 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.955 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.955 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.955 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.955 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_versioning_obj_create_versions_remove_special_names _ 2024-06-28T16:14:10.955 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.955 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.955 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.955 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.955 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.955 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.956 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.956 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.956 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.956 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.956 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.956 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.956 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.956 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.956 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.956 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.956 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.956 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.956 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.956 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.957 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.957 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.957 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.957 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.957 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.957 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.957 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.957 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.957 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.957 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.957 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.957 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.958 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.958 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.958 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.958 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.958 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.958 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.958 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.958 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.958 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.958 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.958 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.958 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.958 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.959 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.959 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.959 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.959 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.959 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.959 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.959 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.959 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.959 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.959 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.959 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.959 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.959 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.959 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.960 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.960 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.960 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.960 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.960 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.960 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.960 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.960 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.960 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.960 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.960 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.960 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.960 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.961 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.961 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.961 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.961 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.961 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.961 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.961 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.961 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.961 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.961 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.961 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.961 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.961 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.961 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.962 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.962 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.962 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.962 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.962 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_versioning_obj_create_overwrite_multipart ______ 2024-06-28T16:14:10.962 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.962 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.962 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.962 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.962 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.962 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.962 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.962 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.963 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.963 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.963 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.963 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.963 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.963 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.963 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.963 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.963 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.963 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.963 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.963 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.963 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.964 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.964 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.964 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.964 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.964 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.964 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.964 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.964 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.964 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.964 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.964 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.964 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.964 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.965 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.965 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.965 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.965 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.965 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.965 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.965 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.965 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.965 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.965 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.965 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.965 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.965 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.966 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.966 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.966 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.966 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.966 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.966 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.966 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.966 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.966 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.966 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.966 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.966 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.966 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.967 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.967 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.967 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.967 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.967 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.967 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.967 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.967 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.967 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.967 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.967 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.968 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.968 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.968 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.968 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.968 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.968 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.968 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.968 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.968 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.968 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.968 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.968 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.968 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.969 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.969 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.969 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.969 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_versioning_obj_list_marker _____________ 2024-06-28T16:14:10.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.969 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.969 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.969 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.969 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.969 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.970 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.970 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.970 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.970 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.970 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.970 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.970 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.970 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.970 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.970 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.970 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.970 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.970 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.971 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.971 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.971 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.971 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.971 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.971 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.971 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.971 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.971 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.971 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.971 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.971 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.971 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.971 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.972 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.972 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.972 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.972 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.972 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.972 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.972 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.972 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.972 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.972 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.972 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.972 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.972 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.973 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.973 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.973 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.973 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.973 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.973 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.973 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.973 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.973 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.973 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.973 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.973 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.973 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.974 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.974 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.974 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.974 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.974 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.974 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.974 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.974 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.974 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.974 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.974 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.974 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.975 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.975 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.975 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.975 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.975 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.975 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.975 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.975 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.975 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.975 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.975 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.975 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.975 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.975 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.976 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.976 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.976 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.976 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_versioning_copy_obj_version _____________ 2024-06-28T16:14:10.976 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.976 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.976 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.976 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.976 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.976 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.976 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.976 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.976 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.976 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.977 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.977 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.977 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.977 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.977 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.977 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.977 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.977 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.977 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.977 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.977 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.977 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.977 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.978 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.978 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.978 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.978 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.978 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.978 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.978 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.978 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.978 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.978 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.978 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.978 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.979 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.979 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.979 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.979 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.979 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.979 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.979 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.979 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.979 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.979 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.979 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.979 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.979 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.980 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.980 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.980 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.980 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.980 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.980 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.980 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.980 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.980 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.980 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.980 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.980 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.980 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.981 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.981 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.981 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.981 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.981 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.981 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.981 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.981 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.981 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.981 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.981 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.981 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.981 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.982 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.982 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.982 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.982 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.982 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.982 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.982 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.982 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.982 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.982 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.982 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.982 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.982 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.985 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.985 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.985 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.985 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_versioning_multi_object_delete ___________ 2024-06-28T16:14:10.985 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.985 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.985 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.986 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.986 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.986 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.986 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.986 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.986 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.986 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.986 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.986 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.986 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.986 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.986 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.986 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.987 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.987 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.987 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.987 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.987 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.987 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.987 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.987 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.987 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.987 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.987 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.987 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.987 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.988 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.988 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.988 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.988 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.988 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.988 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.988 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.988 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.988 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.988 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.988 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.988 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.989 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.989 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.989 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.989 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.989 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.989 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.989 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.989 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.989 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.989 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.989 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.989 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.989 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.990 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.990 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.990 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.990 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.990 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.990 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.990 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.990 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.990 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.990 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.990 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.990 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.990 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.991 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.991 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.991 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.991 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.991 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.991 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.991 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.991 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.991 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.991 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.991 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.991 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.991 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.992 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.992 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.992 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.992 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.992 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.992 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.992 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.992 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.992 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.992 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_versioning_multi_object_delete_with_marker _____ 2024-06-28T16:14:10.992 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.992 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.993 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.993 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:10.993 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:10.993 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:10.993 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:10.993 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.993 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:10.993 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.993 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:10.993 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:10.993 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:10.993 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:10.993 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:10.994 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:10.994 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:10.994 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:10.994 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:10.994 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:10.994 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:10.994 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:10.994 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:10.994 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.994 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:10.994 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:10.994 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:10.995 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.995 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:10.995 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:10.995 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:10.995 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:10.995 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:10.995 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:10.995 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:10.995 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:10.995 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:10.995 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:10.995 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.995 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:10.996 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:10.996 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:10.996 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.996 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:10.996 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:10.996 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:10.996 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:10.996 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:10.996 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:10.996 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:10.996 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:10.996 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.996 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:10.996 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.997 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:10.997 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:10.997 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.997 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.997 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.997 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.997 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:10.997 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:10.997 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.997 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.997 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.997 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:10.997 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:10.998 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:10.998 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:10.998 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:10.998 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:10.998 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.998 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.998 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:10.998 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:10.998 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:10.998 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:10.998 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:10.998 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:10.998 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:10.999 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:10.999 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:10.999 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.999 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:10.999 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:10.999 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:10.999 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:10.999 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:10.999 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.999 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:10.999 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_versioning_multi_object_delete_with_marker_create __ 2024-06-28T16:14:10.999 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:10.999 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:10.999 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.000 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.000 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.000 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.000 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.000 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.000 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.000 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.000 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.000 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.000 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.000 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.001 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.001 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.001 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.001 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.001 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.001 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.001 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.001 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.001 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.001 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.001 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.001 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.001 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.001 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.002 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.002 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.002 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.002 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.002 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.002 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.002 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.002 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.002 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.002 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.002 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.002 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.003 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.003 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.003 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.003 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.003 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.003 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.003 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.003 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.003 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.003 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.003 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.003 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.003 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.003 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.004 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.005 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.005 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.005 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.005 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.005 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.005 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.005 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.005 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.005 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.005 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.005 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.005 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.005 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.005 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.006 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.006 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.006 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.006 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.006 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.006 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.006 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.006 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_versioned_object_acl ________________ 2024-06-28T16:14:11.006 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.006 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.006 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.006 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.006 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.007 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.007 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.007 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.007 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.007 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.007 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.007 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.007 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.007 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.007 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.007 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.007 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.007 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.008 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.008 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.008 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.008 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.008 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.008 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.008 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.008 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.008 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.008 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.008 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.008 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.008 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.008 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.009 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.009 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.009 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.009 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.009 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.009 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.009 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.009 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.009 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.009 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.009 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.009 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.009 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.010 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.010 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.010 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.010 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.010 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.010 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.010 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.010 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.010 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.010 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.010 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.010 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.010 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.011 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.011 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.011 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.011 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.011 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.011 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.011 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.011 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.011 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.011 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.011 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.011 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.011 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.011 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.012 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.012 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.012 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.012 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.012 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.012 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.012 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.012 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.012 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.012 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.012 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.012 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.012 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.013 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.013 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.013 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.013 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.013 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.013 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_versioned_object_acl_no_version_specified ______ 2024-06-28T16:14:11.013 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.013 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.013 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.013 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.013 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.013 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.013 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.014 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.014 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.014 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.014 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.014 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.014 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.014 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.014 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.014 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.014 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.014 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.014 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.014 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.015 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.015 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.015 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.015 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.015 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.015 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.015 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.015 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.015 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.015 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.015 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.015 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.015 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.016 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.016 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.016 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.016 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.016 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.016 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.016 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.016 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.016 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.016 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.016 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.016 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.016 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.017 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.017 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.017 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.017 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.017 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.017 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.017 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.017 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.017 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.017 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.017 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.017 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.017 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.018 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.018 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.018 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.018 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.018 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.018 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.018 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.018 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.018 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.018 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.018 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.018 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.019 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.019 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.019 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.019 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.019 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.019 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.019 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.019 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.019 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.019 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.019 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.019 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.019 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.019 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.020 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.020 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.020 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.020 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.020 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.020 INFO:teuthology.orchestra.run.smithi178.stdout:___ ERROR at teardown of test_versioned_concurrent_object_create_and_remove ____ 2024-06-28T16:14:11.020 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.020 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.020 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.020 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.020 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.020 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.020 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.021 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.021 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.021 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.021 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.021 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.021 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.021 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.021 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.021 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.021 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.021 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.021 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.021 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.022 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.022 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.022 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.022 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.022 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.022 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.022 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.022 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.022 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.022 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.022 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.022 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.022 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.023 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.023 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.023 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.023 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.023 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.023 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.023 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.023 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.023 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.023 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.023 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.023 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.024 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.024 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.024 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.024 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.024 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.024 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.024 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.024 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.024 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.024 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.024 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.024 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.024 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.025 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.025 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.025 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.025 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.025 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.025 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.025 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.025 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.025 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.025 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.025 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.026 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.026 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.026 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.026 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.026 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.026 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.026 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.026 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.026 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.026 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.026 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.026 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.026 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.027 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.027 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.027 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.027 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.027 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.027 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.027 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.027 INFO:teuthology.orchestra.run.smithi178.stdout:___________________ ERROR at teardown of test_lifecycle_set ____________________ 2024-06-28T16:14:11.027 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.027 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.027 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.027 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.027 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.027 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.028 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.029 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.029 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.029 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.029 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.029 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.029 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.029 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.029 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.029 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.029 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.029 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.029 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.029 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.029 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.030 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.030 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.030 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.030 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.030 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.030 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.030 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.030 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.030 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.030 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.030 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.030 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.030 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.031 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.031 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.031 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.031 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.031 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.031 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.031 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.031 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.031 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.031 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.031 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.031 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.031 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.031 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.032 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.032 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.032 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.032 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.032 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.032 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.032 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.032 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.032 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.032 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.032 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.032 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.032 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.033 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.033 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.033 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.033 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.033 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.033 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.033 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.033 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.033 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.033 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.033 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.033 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.034 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.034 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.034 INFO:teuthology.orchestra.run.smithi178.stdout:___________________ ERROR at teardown of test_lifecycle_get ____________________ 2024-06-28T16:14:11.034 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.034 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.034 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.034 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.034 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.034 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.034 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.034 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.034 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.034 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.035 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.035 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.035 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.035 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.035 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.035 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.035 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.035 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.035 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.035 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.035 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.035 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.035 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.036 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.036 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.036 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.036 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.036 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.036 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.036 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.036 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.036 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.036 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.036 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.036 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.036 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.036 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.037 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.038 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.038 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.038 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.038 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.038 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.038 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.038 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.038 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.038 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.038 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.038 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.038 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.038 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.039 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.039 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.039 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.039 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.039 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.039 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.039 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.039 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.039 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.039 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.039 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.039 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.039 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.039 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.040 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.040 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.040 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.040 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.040 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.040 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.040 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.040 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.040 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.040 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.040 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.040 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.040 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_lifecycle_get_no_id _________________ 2024-06-28T16:14:11.041 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.041 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.041 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.041 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.041 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.041 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.041 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.041 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.041 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.041 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.041 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.041 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.041 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.041 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.042 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.042 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.042 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.042 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.042 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.042 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.042 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.042 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.042 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.042 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.042 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.042 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.042 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.042 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.043 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.043 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.043 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.043 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.043 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.043 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.043 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.043 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.043 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.043 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.043 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.043 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.044 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.044 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.044 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.044 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.044 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.044 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.044 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.044 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.044 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.044 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.044 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.044 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.044 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.045 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.045 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.045 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.045 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.045 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.045 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.045 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.045 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.045 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.045 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.045 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.045 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.045 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.046 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.046 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.046 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.046 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.046 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.046 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.046 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.046 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.046 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.046 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.046 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.046 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.046 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_lifecycle_id_too_long ________________ 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.047 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.048 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.048 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.048 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.048 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.048 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.048 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.048 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.048 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.048 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.048 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.048 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.048 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.048 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.048 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.049 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.050 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.050 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.050 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.050 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.050 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.050 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.050 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.050 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.050 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.050 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.050 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.050 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.050 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.050 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.051 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.051 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.051 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.051 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.051 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.051 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.051 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.051 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.051 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.051 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.051 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.051 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.051 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.052 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.052 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.052 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.052 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.052 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.052 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.052 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.052 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.052 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.052 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.052 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.052 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.052 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.052 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.053 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.053 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.053 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.053 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.053 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.053 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.053 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.053 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.053 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.053 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.053 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.053 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.053 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.053 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.054 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.054 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.054 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.054 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_lifecycle_same_id __________________ 2024-06-28T16:14:11.054 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.054 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.054 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.054 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.054 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.054 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.054 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.054 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.054 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.055 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.055 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.055 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.055 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.055 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.055 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.055 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.055 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.055 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.055 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.055 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.055 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.055 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.056 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.056 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.056 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.056 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.056 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.056 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.056 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.056 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.056 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.056 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.056 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.056 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.056 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.057 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.057 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.057 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.057 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.057 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.057 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.057 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.057 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.057 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.057 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.057 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.057 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.057 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.057 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.058 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.058 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.058 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.058 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.058 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.058 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.058 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.058 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.058 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.058 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.058 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.058 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.058 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.059 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.059 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.059 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.059 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.059 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.059 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.059 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.059 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.059 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.059 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.059 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.059 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.059 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.060 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.060 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.060 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.060 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.060 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.060 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.060 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.060 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.060 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.060 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.060 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.060 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.060 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.061 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.061 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.061 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_lifecycle_invalid_status ______________ 2024-06-28T16:14:11.061 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.061 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.061 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.061 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.061 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.061 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.061 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.061 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.061 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.061 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.062 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.062 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.062 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.062 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.062 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.062 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.062 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.062 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.062 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.062 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.062 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.062 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.063 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.063 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.063 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.063 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.063 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.063 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.063 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.063 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.063 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.063 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.063 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.063 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.064 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.064 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.064 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.064 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.064 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.064 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.064 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.064 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.064 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.064 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.064 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.064 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.065 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.065 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.065 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.065 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.065 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.065 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.065 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.065 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.065 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.065 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.065 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.065 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.065 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.066 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.066 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.066 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.066 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.066 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.066 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.066 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.066 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.066 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.066 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.066 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.066 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.066 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.067 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.067 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.068 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.068 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.068 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_lifecycle_set_date _________________ 2024-06-28T16:14:11.068 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.068 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.068 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.068 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.068 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.068 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.068 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.068 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.068 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.068 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.069 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.069 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.069 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.069 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.069 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.069 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.069 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.069 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.069 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.069 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.069 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.069 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.069 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.069 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.070 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.070 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.070 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.070 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.070 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.070 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.070 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.070 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.070 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.070 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.070 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.071 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.071 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.071 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.071 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.071 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.071 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.071 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.071 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.071 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.071 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.071 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.071 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.071 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.072 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.072 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.072 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.072 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.072 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.072 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.072 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.072 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.072 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.072 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.072 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.072 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.072 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.072 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.073 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.073 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.073 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.073 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.073 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.073 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.073 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.073 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.073 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.073 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.073 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.073 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.074 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.074 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.074 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.074 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.074 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.074 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.074 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.074 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.074 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.074 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.074 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.074 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.075 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.075 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.075 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_lifecycle_set_invalid_date _____________ 2024-06-28T16:14:11.075 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.075 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.075 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.075 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.075 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.075 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.075 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.075 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.075 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.075 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.075 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.076 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.076 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.076 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.076 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.076 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.076 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.076 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.076 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.076 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.076 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.076 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.076 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.077 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.077 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.077 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.077 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.077 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.077 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.077 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.077 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.077 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.077 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.077 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.077 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.078 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.078 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.078 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.078 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.078 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.078 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.078 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.078 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.078 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.078 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.078 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.078 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.079 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.079 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.079 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.079 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.079 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.079 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.079 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.079 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.079 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.079 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.079 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.079 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.079 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.080 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.080 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.080 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.080 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.080 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.080 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.080 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.080 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.080 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.080 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.080 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.080 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.080 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.081 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.081 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.081 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.081 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.081 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.081 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.081 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.081 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.081 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.081 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.081 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.081 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.081 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.081 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.082 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.082 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_lifecycle_set_noncurrent ______________ 2024-06-28T16:14:11.082 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.082 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.082 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.082 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.082 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.082 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.082 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.082 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.082 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.082 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.082 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.083 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.083 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.083 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.083 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.083 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.083 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.083 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.083 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.083 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.083 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.083 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.083 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.083 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.084 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.084 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.084 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.084 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.084 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.084 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.084 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.084 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.084 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.084 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.084 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.084 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.085 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.085 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.085 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.085 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.085 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.085 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.085 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.085 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.085 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.085 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.085 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.085 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.086 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.086 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.086 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.086 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.086 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.086 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.086 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.086 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.086 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.086 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.086 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.086 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.087 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.087 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.087 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.087 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.087 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.087 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.087 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.087 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.087 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.087 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.087 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.087 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.088 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.088 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.088 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.088 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.088 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.088 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.088 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.088 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.088 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.088 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.088 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.088 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.089 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.089 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.089 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.089 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.089 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_lifecycle_set_deletemarker _____________ 2024-06-28T16:14:11.089 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.089 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.089 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.089 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.089 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.089 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.089 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.090 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.090 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.090 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.090 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.090 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.090 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.090 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.090 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.090 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.090 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.090 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.090 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.090 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.091 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.091 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.091 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.091 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.091 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.091 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.091 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.091 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.091 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.091 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.091 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.091 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.091 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.092 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.092 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.092 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.092 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.092 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.092 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.092 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.092 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.092 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.092 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.092 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.093 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.093 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.093 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.093 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.093 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.093 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.093 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.093 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.093 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.093 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.093 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.093 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.093 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.094 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.094 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.094 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.094 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.094 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.094 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.094 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.094 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.094 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.094 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.094 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.094 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.094 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.095 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.095 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.095 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.095 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.095 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.095 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.095 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.095 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.095 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.095 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.095 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.095 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.095 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.095 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.096 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.096 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.096 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.096 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.096 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_lifecycle_set_filter ________________ 2024-06-28T16:14:11.096 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.096 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.096 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.096 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.096 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.096 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.096 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.096 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.096 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.097 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.097 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.097 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.097 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.097 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.097 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.097 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.097 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.097 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.097 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.097 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.097 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.097 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.098 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.098 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.098 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.098 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.098 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.098 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.098 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.098 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.098 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.098 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.098 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.098 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.099 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.100 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.100 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.100 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.100 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.100 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.100 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.100 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.100 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.100 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.100 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.100 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.100 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.100 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.100 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.101 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.101 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.101 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.101 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.101 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.101 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.101 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.101 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.101 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.101 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.101 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.101 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.101 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.102 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.102 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.102 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.102 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.102 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.102 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.102 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.102 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.102 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.102 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.102 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.102 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.102 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.103 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_lifecycle_set_empty_filter _____________ 2024-06-28T16:14:11.103 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.103 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.103 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.103 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.103 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.103 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.103 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.103 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.103 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.103 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.103 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.103 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.103 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.104 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.105 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.105 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.105 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.105 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.105 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.105 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.105 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.105 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.105 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.105 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.105 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.105 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.106 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.106 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.106 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.106 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.106 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.106 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.106 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.106 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.106 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.106 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.106 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.106 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.106 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.107 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.107 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.107 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.107 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.107 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.107 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.107 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.107 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.107 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.107 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.107 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.107 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.107 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.108 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_lifecycle_set_multipart _______________ 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.109 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.110 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.110 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.110 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.110 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.110 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.110 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.110 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.110 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.110 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.110 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.110 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.110 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.110 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.111 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.111 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.111 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.111 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.111 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.111 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.111 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.111 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.111 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.111 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.111 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.111 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.111 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.112 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.112 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.112 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.112 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.112 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.112 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.112 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.112 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.112 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.112 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.112 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.112 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.113 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.113 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.113 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.113 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.113 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.113 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.113 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.113 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.113 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.113 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.113 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.113 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.113 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.113 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.114 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.114 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.114 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.114 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.114 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.114 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.114 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.114 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.114 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.114 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.114 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.114 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.114 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.115 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.116 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.116 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_lifecycle_transition_set_invalid_date ________ 2024-06-28T16:14:11.116 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.116 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.116 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.116 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.116 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.116 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.116 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.116 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.116 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.116 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.116 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.117 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.117 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.117 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.117 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.117 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.117 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.117 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.117 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.117 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.117 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.117 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.117 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.118 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.118 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.118 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.118 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.118 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.118 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.118 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.118 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.118 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.118 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.118 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.119 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.119 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.119 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.119 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.119 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.119 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.119 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.119 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.119 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.119 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.119 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.119 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.119 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.120 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.120 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.120 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.120 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.120 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.120 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.120 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.120 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.120 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.120 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.120 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.120 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.120 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.121 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.121 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.121 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.121 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.121 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.121 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.121 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.121 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.121 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.121 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.121 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.121 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.121 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.121 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.122 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.122 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.122 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.122 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.122 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.122 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.122 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.122 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.122 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.122 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.122 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.122 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.122 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.122 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.123 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_encrypted_transfer_1b ________________ 2024-06-28T16:14:11.123 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.123 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.123 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.123 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.123 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.123 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.123 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.123 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.123 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.123 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.123 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.123 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.124 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.124 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.124 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.124 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.124 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.124 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.124 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.124 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.124 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.124 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.124 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.124 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.124 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.125 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.125 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.125 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.125 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.125 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.125 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.125 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.125 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.125 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.125 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.125 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.126 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.126 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.126 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.126 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.126 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.126 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.126 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.126 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.126 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.126 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.126 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.126 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.126 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.127 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.127 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.127 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.127 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.127 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.127 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.127 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.127 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.127 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.127 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.127 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.127 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.127 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.127 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.128 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_encrypted_transfer_1kb _______________ 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.129 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.130 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.130 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.130 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.130 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.130 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.130 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.130 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.130 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.130 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.130 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.130 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.130 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.130 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.130 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.131 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.131 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.131 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.131 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.131 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.131 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.131 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.131 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.131 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.131 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.131 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.131 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.131 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.131 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.132 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.132 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.132 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.132 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.132 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.132 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.132 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.132 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.132 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.132 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.132 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.133 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.133 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.133 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.133 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.133 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.133 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.133 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.133 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.133 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.133 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.133 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.133 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.133 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.133 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.134 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.134 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.134 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.134 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.134 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.134 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.134 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.134 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.134 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.134 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.134 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.134 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.134 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.135 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.135 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.135 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.135 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.135 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.135 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.135 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.135 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.135 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.135 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.135 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.135 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.135 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.136 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.136 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.136 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.136 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.136 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.136 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.136 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_encrypted_transfer_1MB _______________ 2024-06-28T16:14:11.136 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.136 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.136 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.136 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.136 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.136 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.137 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.138 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.139 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.139 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.139 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.139 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.139 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.139 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.139 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.139 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.139 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.139 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.139 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.139 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.139 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.139 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.140 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.140 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.140 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.140 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.140 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.140 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.140 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.140 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.140 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.140 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.140 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.140 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.140 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.141 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.141 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.141 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.141 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.141 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.141 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.141 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.141 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.141 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.141 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.141 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.141 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.141 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.142 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.142 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.142 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.142 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.142 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.142 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.142 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.142 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.142 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.142 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.142 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.142 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.142 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.143 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.143 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_encrypted_transfer_13b _______________ 2024-06-28T16:14:11.143 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.143 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.143 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.143 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.143 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.143 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.143 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.143 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.143 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.143 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.143 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.144 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.144 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.144 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.144 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.144 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.144 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.144 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.144 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.144 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.144 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.144 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.144 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.144 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.144 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.145 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.145 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.145 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.145 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.145 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.145 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.145 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.145 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.145 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.145 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.145 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.145 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.146 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.146 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.146 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.146 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.146 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.146 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.146 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.146 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.146 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.146 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.146 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.146 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.146 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.147 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.147 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.147 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.147 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.147 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.147 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.147 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.147 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.147 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.147 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.147 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.147 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.147 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.148 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.148 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.148 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.148 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.148 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.148 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.148 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.148 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.148 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.148 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.148 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.148 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.148 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.148 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.149 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.149 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.149 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.149 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.149 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.149 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.149 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.149 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.149 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.149 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.149 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.149 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_encryption_sse_c_method_head ____________ 2024-06-28T16:14:11.149 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.150 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.150 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.150 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.150 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.150 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.150 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.150 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.150 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.150 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.150 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.150 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.150 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.150 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.151 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.151 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.151 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.151 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.151 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.151 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.151 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.151 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.151 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.151 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.151 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.151 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.151 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.151 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.152 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.153 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.154 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.154 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.154 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.154 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.154 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.154 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.154 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.154 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.154 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.154 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.154 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.154 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.155 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.155 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.155 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.155 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.155 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.155 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.155 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.155 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.155 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.155 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.155 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.155 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.155 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.156 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.156 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.156 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.156 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.156 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.156 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.156 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_encryption_sse_c_present ______________ 2024-06-28T16:14:11.156 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.156 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.157 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.157 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.157 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.157 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.157 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.157 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.157 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.157 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.157 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.157 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.157 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.157 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.157 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.157 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.158 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.158 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.158 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.158 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.158 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.158 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.158 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.158 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.158 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.158 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.158 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.158 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.158 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.159 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.159 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.159 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.159 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.159 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.159 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.159 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.159 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.159 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.159 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.159 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.159 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.159 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.159 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.160 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.160 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.160 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.160 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.160 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.160 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.160 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.160 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.160 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.160 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.160 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.160 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.160 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.161 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.161 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.161 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.161 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.161 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.161 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.161 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.161 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.161 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.161 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.161 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.161 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.162 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.162 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.162 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.162 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.162 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.162 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.162 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.162 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.162 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.162 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.162 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.162 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.162 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.163 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.163 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.163 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.163 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.163 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.163 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.163 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.163 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.163 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.163 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_encryption_sse_c_other_key _____________ 2024-06-28T16:14:11.163 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.163 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.163 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.164 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.164 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.164 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.164 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.164 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.164 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.164 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.164 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.164 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.164 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.164 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.164 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.164 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.164 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.165 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.165 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.165 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.165 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.165 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.165 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.165 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.165 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.165 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.165 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.165 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.165 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.166 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.166 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.166 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.166 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.166 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.166 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.166 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.166 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.166 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.166 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.166 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.166 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.166 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.167 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.167 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.167 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.167 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.167 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.167 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.167 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.167 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.167 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.167 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.167 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.167 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.167 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.168 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.168 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.168 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.168 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.168 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.168 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.168 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.168 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.168 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.168 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.168 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.168 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.169 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.169 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.169 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.169 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.169 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.169 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.169 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.169 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.169 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.169 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.169 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.169 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.169 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.169 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.170 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.170 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.170 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.170 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.170 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.170 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.170 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.170 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.170 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.170 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_encryption_sse_c_invalid_md5 ____________ 2024-06-28T16:14:11.170 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.170 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.170 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.171 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.171 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.171 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.171 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.171 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.171 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.171 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.171 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.171 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.171 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.171 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.171 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.171 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.171 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.172 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.172 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.172 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.172 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.172 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.172 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.172 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.172 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.172 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.172 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.172 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.172 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.172 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.172 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.173 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.173 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.173 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.173 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.173 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.173 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.173 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.173 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.173 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.173 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.173 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.173 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.173 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.173 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.174 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.174 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.174 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.174 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.174 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.174 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.174 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.174 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.174 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.174 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.174 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.174 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.174 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.175 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.175 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.175 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.175 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.175 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.175 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.175 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.175 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.175 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.175 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.175 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.175 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.175 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.175 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.176 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.176 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.176 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.176 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.176 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.176 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.176 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.176 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.176 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.176 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.176 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.176 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.176 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.177 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.177 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.177 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.177 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.177 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.177 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_encryption_sse_c_no_md5 _______________ 2024-06-28T16:14:11.177 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.177 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.177 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.177 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.177 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.177 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.177 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.177 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.178 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.178 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.178 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.178 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.178 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.178 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.178 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.178 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.178 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.178 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.178 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.178 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.178 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.179 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.179 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.179 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.179 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.179 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.179 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.179 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.179 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.179 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.179 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.179 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.179 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.179 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.179 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.180 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.180 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.180 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.180 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.180 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.180 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.180 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.180 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.180 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.180 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.180 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.180 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.180 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.181 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.181 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.181 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.181 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.181 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.181 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.181 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.181 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.181 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.181 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.181 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.181 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.181 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.181 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.182 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.182 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.182 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.182 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.182 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.182 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.182 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.182 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.182 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.182 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.182 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.182 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.182 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.183 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.183 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.183 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.183 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.183 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.183 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.183 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.183 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.183 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.183 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.183 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.183 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.183 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.183 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.184 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.184 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_encryption_sse_c_no_key _______________ 2024-06-28T16:14:11.184 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.184 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.184 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.184 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.184 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.184 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.184 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.184 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.184 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.184 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.184 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.184 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.185 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.185 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.185 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.185 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.185 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.185 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.185 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.185 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.185 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.185 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.185 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.185 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.185 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.185 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.186 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.186 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.186 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.186 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.186 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.186 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.186 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.186 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.186 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.186 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.186 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.186 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.187 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.187 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.187 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.187 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.187 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.187 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.187 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.187 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.187 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.187 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.187 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.187 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.187 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.187 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.188 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.189 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.189 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.189 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.189 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.190 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.190 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.190 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.190 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.190 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.190 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.190 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.190 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.190 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.190 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.190 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.190 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.191 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.192 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.192 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.192 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.192 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.192 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.192 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.192 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_encryption_key_no_sse_c _______________ 2024-06-28T16:14:11.192 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.192 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.192 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.192 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.192 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.193 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.193 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.193 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.193 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.193 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.193 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.193 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.193 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.193 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.193 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.193 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.193 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.193 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.193 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.194 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.194 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.194 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.194 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.194 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.194 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.194 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.194 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.194 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.194 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.194 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.194 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.195 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.195 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.195 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.195 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.195 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.195 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.195 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.195 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.195 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.195 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.195 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.195 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.196 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.196 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.196 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.196 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.196 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.196 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.196 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.196 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.196 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.196 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.196 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.196 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.196 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.197 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.197 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.197 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.197 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.197 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.197 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.197 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.197 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.197 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.197 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.197 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.197 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.197 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.198 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.198 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.198 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.198 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.198 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.198 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.198 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.198 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.198 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.198 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.198 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.198 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.199 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.199 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.199 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.199 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.199 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.199 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.199 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.199 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.199 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_encryption_sse_c_multipart_upload __________ 2024-06-28T16:14:11.199 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.199 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.199 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.200 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.200 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.200 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.200 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.200 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.200 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.200 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.200 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.200 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.200 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.200 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.200 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.200 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.201 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.201 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.201 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.201 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.201 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.201 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.201 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.201 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.201 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.201 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.201 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.201 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.201 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.201 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.202 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.202 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.202 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.202 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.202 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.202 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.202 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.202 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.202 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.202 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.202 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.202 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.202 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.203 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.203 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.203 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.203 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.203 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.203 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.203 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.203 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.203 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.203 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.203 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.203 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.203 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.204 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.204 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.204 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.204 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.204 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.204 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.204 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.204 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.204 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.204 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.204 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.204 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.205 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.205 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.205 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.205 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.205 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.205 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.205 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.205 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.205 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.205 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.205 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.205 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.205 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.206 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.206 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.206 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.206 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.206 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.206 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.206 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.206 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.206 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.206 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_encryption_sse_c_unaligned_multipart_upload _____ 2024-06-28T16:14:11.206 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.206 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.207 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.207 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.207 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.207 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.207 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.207 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.207 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.207 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.207 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.207 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.207 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.207 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.207 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.207 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.208 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.208 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.208 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.208 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.208 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.208 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.208 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.208 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.208 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.208 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.208 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.208 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.209 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.210 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.210 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.210 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.210 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.210 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.210 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.210 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.210 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.210 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.210 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.210 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.211 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.211 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.211 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.211 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.211 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.211 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.211 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.211 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.211 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.211 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.211 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.211 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.212 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.212 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.212 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.212 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.212 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.212 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.212 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.212 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.212 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.212 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.212 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.212 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.213 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.213 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.213 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.213 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.213 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.213 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.213 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.213 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.213 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.213 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.213 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.213 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.214 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_encryption_sse_c_multipart_bad_download _______ 2024-06-28T16:14:11.214 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.214 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.214 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.214 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.214 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.214 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.214 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.214 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.214 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.214 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.214 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.215 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.215 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.215 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.215 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.215 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.215 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.215 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.215 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.215 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.215 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.215 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.215 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.215 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.216 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.216 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.216 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.216 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.216 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.216 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.216 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.216 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.216 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.216 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.216 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.216 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.216 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.216 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.217 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.217 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.217 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.217 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.217 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.217 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.217 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.217 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.217 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.217 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.217 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.217 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.217 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.218 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.218 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.218 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.218 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.218 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.218 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.218 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.218 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.218 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.218 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.218 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.218 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.218 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.218 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.219 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.219 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.219 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.219 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.219 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.219 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.219 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.219 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.219 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.219 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.219 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.219 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.219 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.220 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.220 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.220 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.220 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.220 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.220 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.220 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.220 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.220 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.220 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.220 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.220 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.220 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_encryption_sse_c_post_object_authenticated_request _ 2024-06-28T16:14:11.221 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.221 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.221 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.221 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.221 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.221 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.221 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.221 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.221 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.221 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.221 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.221 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.221 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.221 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.222 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.222 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.222 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.222 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.222 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.222 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.222 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.222 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.222 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.222 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.222 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.222 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.223 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.223 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.223 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.223 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.223 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.223 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.223 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.223 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.223 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.223 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.223 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.223 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.223 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.225 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.225 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.225 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.225 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.225 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.225 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.225 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.225 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.225 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.225 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.225 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.225 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.225 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.226 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.226 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.226 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.226 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.226 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.226 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.226 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.226 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.226 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.226 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.226 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.226 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.227 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.227 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.227 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.227 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.227 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.227 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.227 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.227 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.227 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.227 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.227 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.227 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_sse_kms_method_head _________________ 2024-06-28T16:14:11.227 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.228 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.228 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.228 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.228 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.228 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.228 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.228 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.228 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.228 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.228 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.228 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.228 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.228 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.228 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.229 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.230 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.230 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.230 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.230 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.230 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.230 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.230 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.230 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.230 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.230 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.230 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.230 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.230 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.231 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.231 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.231 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.231 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.231 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.231 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.231 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.231 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.231 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.232 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.232 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.232 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.232 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.232 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.232 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.232 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.232 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.232 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.232 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.232 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.232 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.232 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.233 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.233 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.233 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.233 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.233 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.233 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.233 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.233 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.233 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.233 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.233 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.233 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.233 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.233 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.234 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.234 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.234 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.234 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.234 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.234 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.234 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_sse_kms_present ___________________ 2024-06-28T16:14:11.234 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.234 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.234 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.234 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.234 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.235 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.235 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.235 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.235 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.235 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.235 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.235 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.235 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.235 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.235 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.235 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.235 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.235 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.236 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.236 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.236 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.236 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.236 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.236 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.236 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.236 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.236 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.236 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.236 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.236 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.236 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.237 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.237 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.237 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.237 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.237 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.237 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.237 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.237 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.237 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.237 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.237 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.237 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.237 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.238 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.238 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.238 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.238 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.238 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.238 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.238 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.238 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.238 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.239 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.239 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.239 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.239 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.239 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.239 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.239 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.239 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.239 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.239 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.239 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.239 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.239 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.239 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.240 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.241 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.241 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.241 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.241 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.241 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.241 INFO:teuthology.orchestra.run.smithi178.stdout:___________________ ERROR at teardown of test_sse_kms_no_key ___________________ 2024-06-28T16:14:11.241 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.241 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.241 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.241 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.241 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.241 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.241 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.241 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.242 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.242 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.242 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.242 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.242 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.242 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.242 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.242 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.242 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.242 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.242 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.242 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.242 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.242 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.243 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.243 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.243 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.243 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.243 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.243 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.243 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.243 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.243 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.243 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.243 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.243 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.244 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.244 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.244 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.244 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.244 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.244 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.244 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.244 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.244 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.244 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.244 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.244 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.245 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.245 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.245 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.245 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.245 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.245 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.245 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.245 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.245 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.245 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.245 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.245 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.245 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.245 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.246 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.246 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.246 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.246 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.246 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.246 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.246 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.246 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.246 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.246 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.246 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.246 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.246 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.247 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.247 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.247 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.247 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.247 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.247 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.247 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.247 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.247 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.247 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.247 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.247 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.247 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.248 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.248 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.248 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_sse_kms_not_declared ________________ 2024-06-28T16:14:11.248 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.248 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.248 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.248 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.248 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.248 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.248 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.248 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.248 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.248 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.249 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.249 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.249 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.249 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.249 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.249 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.249 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.249 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.249 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.249 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.249 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.249 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.249 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.250 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.250 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.250 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.250 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.250 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.250 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.250 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.250 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.250 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.250 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.250 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.250 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.251 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.251 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.251 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.251 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.251 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.251 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.251 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.251 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.251 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.251 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.251 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.251 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.251 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.252 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.252 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.252 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.252 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.252 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.252 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.252 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.252 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.252 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.252 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.252 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.252 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.253 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.253 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.253 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.253 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.253 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.253 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.253 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.253 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.253 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.253 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.253 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.253 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.253 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.254 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.254 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.254 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.254 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.254 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.254 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.254 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.254 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.254 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.254 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.254 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.254 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.254 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.255 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.255 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.255 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_sse_kms_multipart_upload ______________ 2024-06-28T16:14:11.255 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.255 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.255 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.255 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.255 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.255 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.255 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.255 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.255 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.255 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.256 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.256 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.256 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.256 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.256 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.256 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.256 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.256 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.256 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.256 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.256 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.256 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.256 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.256 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.257 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.257 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.257 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.257 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.257 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.257 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.257 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.257 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.257 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.257 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.257 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.257 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.258 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.258 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.258 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.258 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.258 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.258 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.258 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.258 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.258 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.258 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.258 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.258 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.259 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.260 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.260 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.260 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.260 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.260 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.260 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.260 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.260 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.260 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.260 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.260 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.260 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.260 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.261 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.261 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.261 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.261 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.261 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.261 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.261 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.261 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.261 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.261 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.261 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.261 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.261 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.262 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.262 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_sse_kms_multipart_invalid_chunks_1 _________ 2024-06-28T16:14:11.262 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.262 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.262 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.262 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.262 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.262 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.262 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.262 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.262 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.262 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.262 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.263 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.263 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.263 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.263 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.263 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.263 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.263 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.263 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.263 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.263 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.263 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.263 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.263 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.264 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.264 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.264 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.264 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.264 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.264 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.264 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.264 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.264 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.264 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.264 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.264 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.265 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.265 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.265 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.265 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.265 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.265 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.265 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.265 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.265 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.265 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.265 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.265 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.265 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.266 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.266 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.266 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.266 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.266 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.266 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.266 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.266 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.266 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.266 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.266 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.266 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.266 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.267 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.267 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.267 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.267 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.267 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.267 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.267 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.267 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.267 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.267 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.267 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.267 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.267 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.268 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.268 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.268 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.268 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.268 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.268 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.268 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.268 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.268 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.268 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.268 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.268 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.268 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.269 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.269 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_sse_kms_multipart_invalid_chunks_2 _________ 2024-06-28T16:14:11.269 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.269 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.269 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.269 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.269 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.269 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.269 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.269 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.269 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.269 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.269 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.269 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.270 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.270 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.270 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.270 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.270 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.270 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.270 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.270 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.270 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.270 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.270 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.270 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.270 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.271 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.271 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.271 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.271 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.271 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.271 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.271 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.271 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.271 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.271 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.271 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.272 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.272 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.272 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.272 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.272 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.272 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.272 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.272 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.272 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.272 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.272 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.272 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.272 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.273 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.273 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.273 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.273 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.273 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.273 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.273 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.273 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.273 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.273 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.273 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.273 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.273 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.274 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.274 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.274 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.274 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.274 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.274 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.274 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.274 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.274 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.274 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.274 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.274 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.275 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.275 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.275 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.275 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.275 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.275 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.275 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.275 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.275 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.275 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.275 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.275 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.275 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.275 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.276 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_sse_kms_post_object_authenticated_request ______ 2024-06-28T16:14:11.276 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.276 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.276 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.276 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.276 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.276 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.276 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.276 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.276 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.276 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.276 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.276 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.276 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.277 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.277 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.277 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.277 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.277 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.277 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.277 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.277 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.277 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.277 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.277 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.277 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.277 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.278 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.278 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.278 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.278 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.278 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.278 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.278 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.278 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.278 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.278 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.278 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.278 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.278 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.279 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.279 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.279 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.279 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.279 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.279 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.279 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.279 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.279 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.279 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.279 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.279 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.279 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.279 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.280 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.280 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.280 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.280 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.280 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.280 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.280 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.280 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.280 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.280 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.280 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.280 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.280 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.281 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.281 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.281 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.281 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.281 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.281 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.281 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.281 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.281 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.281 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.281 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.281 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.281 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.282 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.282 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.282 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.282 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.282 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.282 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.282 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.282 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.282 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.282 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.282 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.282 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_sse_kms_transfer_1b _________________ 2024-06-28T16:14:11.283 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.283 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.283 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.283 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.283 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.283 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.283 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.283 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.283 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.283 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.283 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.283 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.283 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.283 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.284 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.284 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.284 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.284 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.284 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.284 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.284 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.284 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.284 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.284 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.284 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.284 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.285 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.286 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.286 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.286 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.286 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.286 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.286 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.286 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.286 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.286 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.286 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.286 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.286 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.286 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.287 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.287 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.287 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.287 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.287 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.287 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.287 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.287 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.287 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.287 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.287 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.287 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.287 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.288 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.288 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.288 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.288 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.288 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.288 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.288 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.288 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.288 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.288 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.288 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.288 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.289 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.289 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.289 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.289 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.289 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.289 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.289 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.289 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.289 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.289 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.289 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.289 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_sse_kms_transfer_1kb ________________ 2024-06-28T16:14:11.289 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.289 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.290 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.290 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.290 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.290 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.290 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.290 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.290 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.290 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.290 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.290 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.290 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.290 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.290 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.290 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.291 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.291 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.291 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.291 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.291 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.291 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.291 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.291 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.291 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.291 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.291 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.291 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.291 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.292 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.293 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.294 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.295 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.296 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.296 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_sse_kms_transfer_1MB ________________ 2024-06-28T16:14:11.296 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.296 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.296 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.296 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.296 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.296 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.296 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.296 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.296 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.296 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.297 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.297 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.297 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.297 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.297 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.297 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.297 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.297 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.297 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.297 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.297 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.297 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.297 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.298 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.298 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.298 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.298 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.298 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.298 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.298 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.298 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.298 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.298 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.298 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.298 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.299 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.300 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.300 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.300 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.300 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.300 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.300 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.300 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.300 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.300 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.300 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.300 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.300 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.300 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.301 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.301 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.301 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.301 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.301 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.301 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.301 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.301 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.301 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.301 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.301 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.301 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.301 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.301 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.302 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.302 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.302 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.302 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.302 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.302 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.302 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.302 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.302 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.302 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.302 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.302 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.302 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_sse_kms_transfer_13b ________________ 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.303 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.304 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.305 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.305 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.305 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.305 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.305 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.305 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.305 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.305 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.305 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.305 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.305 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.305 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.306 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.306 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.306 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.306 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.306 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.306 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.306 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.306 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.306 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.306 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.306 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.306 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.306 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.307 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.308 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.308 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.308 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.308 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.308 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.308 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.308 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.308 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.308 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.308 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.308 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.308 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.309 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.309 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.309 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.309 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.309 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.309 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.309 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.309 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_sse_kms_read_declare ________________ 2024-06-28T16:14:11.309 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.309 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.309 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.309 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.309 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.310 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.310 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.310 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.310 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.310 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.310 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.310 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.310 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.310 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.310 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.310 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.310 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.311 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.311 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.311 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.311 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.311 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.311 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.311 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.311 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.311 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.311 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.311 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.311 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.311 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.312 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.312 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.312 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.312 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.312 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.312 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.312 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.312 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.312 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.312 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.312 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.313 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.313 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.313 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.313 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.313 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.313 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.313 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.313 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.313 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.313 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.313 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.313 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.313 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.313 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.314 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.314 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.314 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.314 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.314 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.314 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.314 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.314 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.314 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.314 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.314 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.314 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.314 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.315 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.316 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.316 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.316 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.316 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.316 INFO:teuthology.orchestra.run.smithi178.stdout:___________________ ERROR at teardown of test_bucket_policy ____________________ 2024-06-28T16:14:11.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.316 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.316 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.316 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.316 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.316 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.317 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.318 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.318 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.318 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.318 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.318 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.318 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.318 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.318 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.318 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.318 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.318 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.318 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.318 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.319 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.320 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.321 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.321 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.321 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.321 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.321 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.321 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.321 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.321 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.321 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.321 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.321 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.321 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.321 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.322 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.322 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.322 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.322 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.322 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.322 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.322 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.322 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.322 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.322 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.322 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.322 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_bucketv2_policy ___________________ 2024-06-28T16:14:11.322 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.323 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.324 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.325 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.326 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.327 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.327 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.327 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.327 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.327 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.327 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.327 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.327 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.327 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.327 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.327 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.327 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.327 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.327 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.328 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_bucket_policy_acl __________________ 2024-06-28T16:14:11.329 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.329 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.329 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.329 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.329 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.329 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.329 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.329 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.329 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.329 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.329 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.329 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.329 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.330 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.330 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.330 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.330 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.330 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.330 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.330 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.330 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.330 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.330 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.330 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.330 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.330 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.331 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.331 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.331 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.331 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.331 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.331 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.331 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.331 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.331 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.331 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.331 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.331 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.331 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.331 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.332 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.332 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.332 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.332 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.332 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.332 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.332 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.332 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.332 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.332 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.332 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.332 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.332 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.333 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.333 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.333 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.333 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.333 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.333 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.333 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.333 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.333 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.333 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.333 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.333 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.333 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.334 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.334 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.334 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.334 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.334 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.334 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.334 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.334 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.334 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.334 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.334 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.334 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.334 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.335 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.335 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.335 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.335 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.335 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.335 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.335 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.335 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.335 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.335 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.335 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.335 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_bucketv2_policy_acl _________________ 2024-06-28T16:14:11.335 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.336 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.336 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.336 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.336 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.336 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.336 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.336 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.336 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.336 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.336 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.336 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.336 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.336 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.336 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.337 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.340 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.340 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.340 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.340 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.340 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.340 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.340 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.340 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.340 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.340 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.340 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.340 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.340 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.341 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.341 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.341 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.341 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.341 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.341 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.341 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.341 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.341 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.341 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.341 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.341 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.342 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.342 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.342 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.342 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.342 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.342 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.342 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.342 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.342 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_bucket_policy_another_bucket ____________ 2024-06-28T16:14:11.342 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.342 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.342 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.342 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.343 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.343 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.343 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.343 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.343 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.343 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.343 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.343 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.343 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.343 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.343 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.343 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.343 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.343 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.344 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.344 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.344 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.344 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.344 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.344 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.344 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.344 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.344 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.344 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.344 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.344 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.345 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.345 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.345 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.345 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.345 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.345 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.345 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.345 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.345 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.345 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.345 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.345 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.345 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.345 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.346 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.346 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.346 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.346 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.346 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.346 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.346 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.346 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.346 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.346 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.346 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.346 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.346 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.347 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.347 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.347 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.347 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.347 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.347 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.347 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.347 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.347 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.347 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.347 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.347 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.348 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.348 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.348 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.348 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.348 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.348 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.348 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.348 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.348 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.348 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.348 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.348 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.348 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.348 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.349 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.349 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.349 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.349 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.349 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.349 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.349 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.349 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_bucketv2_policy_another_bucket ___________ 2024-06-28T16:14:11.349 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.349 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.349 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.349 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.350 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.350 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.350 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.350 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.350 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.350 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.350 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.350 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.350 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.350 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.350 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.350 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.351 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.351 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.351 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.351 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.351 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.351 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.351 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.351 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.351 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.351 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.351 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.351 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.351 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.351 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.352 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.352 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.352 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.352 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.352 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.352 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.352 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.352 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.352 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.352 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.352 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.352 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.353 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.353 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.353 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.353 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.353 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.353 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.353 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.353 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.353 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.353 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.354 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.354 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.354 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.354 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.354 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.354 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.354 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.354 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.354 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.354 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.354 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.354 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.354 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.355 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.355 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.355 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.355 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.355 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.355 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.355 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.355 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.355 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.355 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.355 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.355 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.355 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.356 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.356 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.356 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.356 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.356 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.356 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.356 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.356 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.356 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.356 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.356 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.356 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_get_obj_tagging ___________________ 2024-06-28T16:14:11.357 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.357 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.357 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.357 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.357 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.357 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.357 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.357 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.357 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.357 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.357 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.357 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.357 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.357 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.358 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.358 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.358 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.358 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.358 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.358 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.358 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.358 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.358 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.358 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.358 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.358 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.359 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.359 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.359 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.359 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.359 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.359 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.359 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.359 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.359 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.359 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.359 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.359 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.359 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.360 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.360 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.360 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.360 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.360 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.360 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.360 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.360 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.360 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.360 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.360 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.360 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.360 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.360 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.361 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.361 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.361 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.361 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.361 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.361 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.361 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.361 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.361 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.361 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.361 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.361 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.362 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.362 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.362 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.362 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.362 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.362 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.362 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.362 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.362 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.362 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.362 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.362 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.363 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.363 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.363 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.363 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.363 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.363 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.363 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.363 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.363 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.363 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.363 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.363 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.363 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.364 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_get_obj_head_tagging ________________ 2024-06-28T16:14:11.364 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.364 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.364 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.364 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.364 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.364 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.364 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.364 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.364 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.364 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.364 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.365 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.365 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.365 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.365 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.365 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.365 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.365 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.365 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.365 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.365 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.365 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.365 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.365 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.366 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.366 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.366 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.366 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.366 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.366 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.366 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.366 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.366 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.366 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.366 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.366 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.366 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.367 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.368 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.368 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.368 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.368 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.368 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.368 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.368 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.368 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.368 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.368 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.368 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.368 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.368 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.368 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.369 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.369 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.369 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.369 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.369 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.369 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.369 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.369 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.369 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.369 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.369 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.370 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.370 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.370 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.370 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.370 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.370 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.370 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.370 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.370 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.370 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.370 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.370 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.370 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.371 INFO:teuthology.orchestra.run.smithi178.stdout:____________________ ERROR at teardown of test_put_max_tags ____________________ 2024-06-28T16:14:11.371 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.371 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.371 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.371 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.371 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.371 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.371 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.371 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.371 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.371 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.371 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.371 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.371 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.372 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.372 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.372 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.372 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.372 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.372 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.372 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.372 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.372 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.372 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.372 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.372 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.372 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.373 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.373 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.373 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.373 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.373 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.373 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.373 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.373 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.373 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.373 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.373 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.373 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.373 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.373 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.374 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.374 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.374 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.374 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.374 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.374 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.374 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.374 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.374 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.374 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.374 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.374 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.374 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.375 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.375 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.375 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.375 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.375 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.375 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.375 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.375 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.375 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.375 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.375 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.375 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.375 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.376 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.376 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.376 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.376 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.376 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.376 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.376 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.376 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.376 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.376 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.376 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.376 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.377 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.377 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.377 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.377 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.377 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.377 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.377 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.377 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.377 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.377 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.377 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.377 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.378 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_put_excess_tags ___________________ 2024-06-28T16:14:11.378 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.378 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.378 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.378 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.378 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.378 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.378 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.378 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.378 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.378 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.378 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.379 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.379 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.379 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.379 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.379 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.379 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.379 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.379 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.379 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.379 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.379 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.379 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.379 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.379 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.380 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.380 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.380 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.380 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.380 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.380 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.380 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.380 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.380 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.380 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.380 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.380 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.380 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.380 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.381 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.381 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.381 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.381 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.381 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.381 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.381 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.381 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.381 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.381 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.381 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.381 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.381 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.382 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.382 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.382 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.382 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.382 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.382 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.382 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.382 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.382 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.382 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.382 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.382 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.383 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.383 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.383 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.383 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.383 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.383 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.383 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.383 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.383 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.383 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.383 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.383 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.383 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.383 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.384 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.384 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.384 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.384 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.384 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.384 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.384 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.384 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.384 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.384 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.384 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.384 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.384 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_put_max_kvsize_tags _________________ 2024-06-28T16:14:11.385 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.385 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.385 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.385 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.385 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.385 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.385 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.385 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.385 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.385 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.385 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.385 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.385 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.386 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.386 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.386 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.386 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.386 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.386 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.386 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.386 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.386 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.386 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.386 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.386 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.387 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.387 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.387 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.387 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.387 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.387 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.387 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.387 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.387 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.387 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.387 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.387 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.387 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.387 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.388 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.388 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.388 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.388 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.388 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.388 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.388 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.388 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.388 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.388 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.388 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.388 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.388 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.388 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.389 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.389 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.389 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.389 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.389 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.389 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.389 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.389 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.389 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.389 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.389 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.389 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.389 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.390 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.390 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.390 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.390 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.390 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.390 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.390 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.390 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.390 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.390 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.390 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.390 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.391 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.391 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.391 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.391 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.391 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.391 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.391 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.391 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.391 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.391 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.391 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.391 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.391 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_put_excess_key_tags _________________ 2024-06-28T16:14:11.391 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.392 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.392 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.392 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.392 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.392 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.392 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.392 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.392 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.392 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.392 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.392 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.392 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.392 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.393 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.393 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.393 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.393 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.393 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.393 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.393 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.393 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.393 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.393 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.393 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.393 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.393 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.393 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.394 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.394 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.394 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.394 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.394 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.394 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.394 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.394 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.394 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.394 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.394 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.395 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.396 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.396 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.396 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.396 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.396 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.396 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.396 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.396 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.396 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.396 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.396 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.396 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.396 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.396 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.397 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.398 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.398 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.398 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.398 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.398 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.398 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_put_excess_val_tags _________________ 2024-06-28T16:14:11.398 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.398 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.398 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.398 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.398 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.398 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.399 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.399 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.399 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.399 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.399 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.399 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.399 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.399 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.399 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.399 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.399 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.399 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.399 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.400 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.400 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.400 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.400 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.400 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.400 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.400 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.400 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.400 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.400 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.400 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.400 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.400 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.401 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.401 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.401 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.401 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.401 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.401 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.401 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.401 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.401 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.401 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.401 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.402 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.402 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.402 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.402 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.402 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.402 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.402 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.402 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.402 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.402 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.402 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.402 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.402 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.403 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.403 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.403 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.403 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.403 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.403 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.403 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.403 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.403 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.403 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.403 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.403 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.403 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.403 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.404 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.404 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.404 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.404 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.404 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.404 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.404 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.404 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.404 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.404 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.404 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.404 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.404 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.405 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.405 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.405 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.405 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.405 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.405 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_put_modify_tags ___________________ 2024-06-28T16:14:11.405 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.405 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.405 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.405 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.405 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.405 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.405 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.406 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.406 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.406 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.406 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.406 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.406 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.406 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.406 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.406 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.406 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.406 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.406 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.407 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.407 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.407 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.407 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.407 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.407 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.407 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.407 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.407 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.407 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.407 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.407 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.407 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.407 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.408 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.408 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.408 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.408 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.408 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.408 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.408 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.408 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.408 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.408 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.408 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.408 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.409 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.409 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.409 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.409 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.409 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.409 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.409 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.409 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.409 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.409 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.409 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.409 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.409 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.409 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.410 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.410 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.410 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.410 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.410 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.410 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.410 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.410 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.410 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.410 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.410 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.410 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.410 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.410 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.411 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.411 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.411 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.411 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.411 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.411 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.411 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.411 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.411 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.411 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.411 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.411 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.411 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.412 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.412 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.412 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_put_delete_tags ___________________ 2024-06-28T16:14:11.412 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.412 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.412 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.412 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.412 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.412 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.412 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.412 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.412 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.412 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.412 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.413 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.413 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.413 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.413 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.413 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.413 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.413 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.413 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.413 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.413 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.413 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.413 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.413 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.414 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.414 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.414 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.414 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.414 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.414 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.414 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.414 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.414 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.414 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.414 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.415 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.415 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.415 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.415 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.415 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.415 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.415 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.415 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.415 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.415 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.415 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.415 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.416 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.416 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.416 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.416 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.416 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.416 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.416 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.416 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.416 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.416 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.416 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.416 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.416 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.416 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.417 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.417 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.417 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.417 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.417 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.417 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.417 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.417 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.417 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.417 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.417 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.417 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.417 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.417 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.418 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.418 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.418 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.418 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.418 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.418 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.418 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.418 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.418 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.418 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.418 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.419 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.419 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.419 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_post_object_tags_anonymous_request _________ 2024-06-28T16:14:11.419 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.419 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.419 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.419 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.419 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.419 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.419 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.419 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.419 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.419 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.420 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.420 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.420 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.420 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.420 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.420 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.420 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.420 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.420 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.420 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.420 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.420 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.420 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.421 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.421 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.421 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.421 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.421 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.421 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.421 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.421 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.421 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.421 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.421 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.421 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.422 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.422 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.422 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.422 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.422 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.422 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.422 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.422 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.422 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.422 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.423 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.423 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.423 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.423 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.423 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.423 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.423 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.423 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.423 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.423 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.423 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.423 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.423 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.423 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.424 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.424 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.424 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.424 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.424 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.424 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.424 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.424 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.424 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.424 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.424 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.424 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.425 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.425 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.425 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.425 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.425 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.425 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.425 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.425 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.425 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.425 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.425 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.425 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.426 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.426 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.426 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.426 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.426 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.426 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_post_object_tags_authenticated_request _______ 2024-06-28T16:14:11.426 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.426 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.426 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.426 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.426 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.426 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.427 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.427 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.427 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.427 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.427 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.427 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.427 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.427 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.427 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.427 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.427 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.427 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.428 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.428 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.428 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.428 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.428 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.428 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.428 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.428 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.428 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.428 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.428 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.429 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.429 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.429 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.429 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.429 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.429 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.429 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.429 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.429 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.429 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.429 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.430 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.430 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.430 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.430 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.430 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.430 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.430 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.430 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.430 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.430 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.430 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.430 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.430 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.431 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.431 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.431 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.431 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.431 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.431 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.431 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.431 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.431 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.431 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.431 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.431 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.431 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.432 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.433 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.433 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.433 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.433 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.433 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.433 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.433 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.433 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.433 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_put_obj_with_tags __________________ 2024-06-28T16:14:11.433 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.433 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.433 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.433 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.434 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.434 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.434 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.434 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.434 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.434 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.434 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.434 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.434 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.434 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.434 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.434 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.434 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.434 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.435 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.435 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.435 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.435 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.435 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.435 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.435 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.435 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.435 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.435 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.435 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.435 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.435 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.436 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.436 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.436 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.436 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.436 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.436 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.436 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.436 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.436 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.436 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.436 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.436 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.437 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.437 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.437 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.437 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.437 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.437 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.437 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.437 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.437 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.437 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.437 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.437 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.437 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.437 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.438 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.439 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.439 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.439 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.439 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.439 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.439 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.439 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.439 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.439 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.439 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.439 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.439 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.439 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.439 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.440 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.440 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.440 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.440 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_get_tags_acl_public _________________ 2024-06-28T16:14:11.440 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.440 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.440 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.440 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.440 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.440 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.440 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.440 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.440 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.441 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.442 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.442 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.442 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.442 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.442 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.442 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.442 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.442 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.442 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.442 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.442 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.442 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.443 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.444 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.445 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.446 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.446 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.446 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.446 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_put_tags_acl_public _________________ 2024-06-28T16:14:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.446 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.446 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.447 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.447 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.447 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.447 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.447 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.447 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.447 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.447 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.447 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.447 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.447 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.447 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.447 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.447 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.448 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.449 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.450 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.450 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.450 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.450 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.450 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.450 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.450 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.450 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.450 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.450 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.450 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.450 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.450 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.451 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.451 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.451 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.451 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.451 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.451 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.451 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.451 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.451 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.451 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.451 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.451 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.452 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.452 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.452 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.452 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.452 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.452 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.452 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.452 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.452 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.452 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.452 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.452 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.452 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.452 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout:_______________ ERROR at teardown of test_delete_tags_obj_public _______________ 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.453 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.454 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.454 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.454 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.454 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.454 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.454 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.454 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.454 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.454 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.454 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.454 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.454 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.454 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.455 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.455 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.455 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.455 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.455 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.455 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.455 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.455 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.455 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.455 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.455 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.455 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.456 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.457 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.457 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.457 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.457 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.457 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.457 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.457 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.457 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.457 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.457 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.457 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.457 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.457 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.457 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.458 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.458 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.458 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.458 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.458 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.458 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.458 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.458 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.458 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.458 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.458 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.458 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.458 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.458 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.459 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.459 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.459 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.459 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.459 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.459 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.459 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.459 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.459 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_versioning_bucket_atomic_upload_return_version_id __ 2024-06-28T16:14:11.459 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.459 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.459 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.459 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.460 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.460 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.460 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.460 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.460 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.460 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.460 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.460 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.461 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.461 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.461 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.461 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.461 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.461 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.461 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.462 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.463 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.464 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.465 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.466 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.466 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.466 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.466 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_versioning_bucket_multipart_upload_return_version_id _ 2024-06-28T16:14:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.466 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.466 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.467 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.467 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.467 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.467 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.467 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.467 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.467 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.467 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.468 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.468 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.468 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.468 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.468 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.468 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.468 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.469 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.470 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.471 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.471 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.471 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.471 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.471 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.471 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.471 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.471 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.471 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.471 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.471 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.471 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.471 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.471 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.472 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.472 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.472 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.472 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.472 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.472 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.472 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.472 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.472 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.472 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.472 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.473 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.473 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.473 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.473 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.473 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.473 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.473 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.473 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.473 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.473 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.473 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_bucket_policy_get_obj_existing_tag _________ 2024-06-28T16:14:11.473 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.473 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.474 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.474 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.474 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.474 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.474 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.474 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.474 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.474 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.474 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.474 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.474 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.474 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.474 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.475 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.476 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.476 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.476 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.476 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.476 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.476 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.476 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.476 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.476 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.476 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.476 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.476 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.476 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.477 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.477 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.477 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.477 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.477 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.477 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.477 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.477 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.477 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.477 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.477 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.477 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.477 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.478 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.478 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.478 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.478 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.478 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.478 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.478 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.478 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.478 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.478 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.478 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.478 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.478 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.479 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.479 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.479 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.479 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.479 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.479 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.479 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.479 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.479 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.479 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.479 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.479 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.479 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.479 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.480 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.480 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.480 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.480 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.480 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.480 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.480 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_bucket_policy_get_obj_tagging_existing_tag _____ 2024-06-28T16:14:11.480 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.480 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.480 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.480 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.480 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.481 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.481 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.481 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.481 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.481 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.481 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.481 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.481 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.481 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.481 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.481 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.481 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.482 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.482 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.482 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.482 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.482 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.482 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.482 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.482 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.482 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.482 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.482 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.482 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.482 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.482 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.483 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.484 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.484 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.484 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.484 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.484 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.484 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.484 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.484 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.484 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.484 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.484 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.484 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.484 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.485 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.485 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.485 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.485 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.485 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.485 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.485 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.485 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.485 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.485 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.485 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.485 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.485 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.486 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.486 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.486 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.486 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.486 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.486 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.486 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.486 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.486 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.486 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.486 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.486 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.486 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.487 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.487 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.487 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.487 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.487 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.487 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.487 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_bucket_policy_put_obj_tagging_existing_tag _____ 2024-06-28T16:14:11.487 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.487 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.487 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.487 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.487 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.487 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.488 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.488 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.488 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.488 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.488 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.488 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.488 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.488 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.488 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.488 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.488 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.488 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.488 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.488 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.489 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.490 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.490 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.490 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.490 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.490 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.490 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.490 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.490 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.490 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.490 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.490 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.490 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.491 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.491 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.491 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.491 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.491 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.491 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.491 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.491 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.491 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.491 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.491 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.491 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.491 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.492 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.492 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.492 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.492 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.492 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.492 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.492 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.492 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.492 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.492 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.492 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.492 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.492 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.493 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.493 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.493 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.493 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.493 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.493 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.493 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.493 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.493 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.493 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.493 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.493 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.493 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.493 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.494 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.494 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.494 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.494 INFO:teuthology.orchestra.run.smithi178.stdout:_________ ERROR at teardown of test_bucket_policy_put_obj_copy_source __________ 2024-06-28T16:14:11.494 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.494 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.494 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.494 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.494 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.494 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.494 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.494 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.495 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.495 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.495 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.495 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.495 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.495 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.495 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.495 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.495 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.495 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.495 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.495 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.495 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.495 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.496 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.496 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.496 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.496 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.496 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.496 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.496 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.496 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.496 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.496 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.496 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.496 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.496 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.497 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.497 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.497 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.497 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.497 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.497 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.497 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.497 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.497 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.497 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.497 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.497 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.497 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.498 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.498 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.498 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.498 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.498 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.498 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.498 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.498 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.498 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.498 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.498 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.498 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.498 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.499 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.499 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.499 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.499 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.499 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.499 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.499 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.499 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.499 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.499 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.499 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.499 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.499 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.500 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.500 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.500 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.500 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.500 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.500 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.500 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.500 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.500 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.500 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.500 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.500 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.500 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.501 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.501 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.501 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.501 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_bucket_policy_put_obj_copy_source_meta _______ 2024-06-28T16:14:11.501 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.501 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.501 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.501 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.501 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.501 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.501 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.501 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.501 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.502 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.502 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.502 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.502 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.502 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.502 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.502 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.502 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.502 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.502 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.502 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.503 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.503 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.503 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.503 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.503 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.503 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.503 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.503 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.503 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.503 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.503 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.503 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.504 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.504 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.504 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.504 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.504 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.504 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.504 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.504 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.504 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.504 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.504 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.504 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.504 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.505 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.506 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.506 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.506 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.506 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.506 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.506 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.506 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.506 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.506 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.506 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.506 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.506 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.507 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.507 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.507 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.507 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.507 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.507 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.507 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.507 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.507 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.507 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.507 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.507 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.507 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_bucket_policy_put_obj_acl ______________ 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.508 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.509 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.509 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.509 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.509 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.509 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.509 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.509 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.509 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.509 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.509 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.509 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.509 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.509 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.510 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.510 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.510 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.510 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.510 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.510 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.510 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.510 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.510 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.510 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.510 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.510 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.511 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.512 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.512 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.512 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.512 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.512 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.512 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.512 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.512 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.512 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.512 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.512 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.512 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.512 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.512 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.513 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.513 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.513 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.513 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.513 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.513 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.513 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.513 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.513 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.513 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.513 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.513 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.513 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.513 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.514 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.514 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.514 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.514 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.514 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.514 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.514 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.514 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.514 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.514 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.514 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.514 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.515 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.515 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_bucket_policy_put_obj_grant _____________ 2024-06-28T16:14:11.515 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.515 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.515 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.515 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.515 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.515 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.515 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.515 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.515 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.515 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.515 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.515 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.516 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.516 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.516 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.516 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.516 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.516 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.516 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.516 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.516 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.516 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.516 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.516 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.516 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.516 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.517 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.517 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.517 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.517 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.517 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.517 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.517 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.517 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.517 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.517 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.517 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.517 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.517 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.517 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.518 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.518 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.518 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.518 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.518 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.518 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.518 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.518 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.518 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.518 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.518 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.518 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.518 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.519 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.519 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.519 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.519 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.519 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.519 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.519 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.519 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.519 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.519 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.519 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.519 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.519 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.520 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.520 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.520 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.520 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.520 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.520 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.520 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.520 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.520 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.520 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.520 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.520 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.520 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.520 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.521 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.521 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.521 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.521 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.521 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.521 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.521 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.521 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.521 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.521 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.521 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_put_obj_enc_conflict_c_s3 ______________ 2024-06-28T16:14:11.521 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.521 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.522 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.522 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.522 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.522 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.522 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.522 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.522 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.522 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.522 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.522 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.522 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.522 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.523 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.523 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.523 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.523 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.523 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.523 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.523 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.523 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.523 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.523 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.523 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.523 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.523 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.524 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.524 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.524 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.524 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.524 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.524 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.524 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.524 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.524 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.524 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.524 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.524 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.524 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.525 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.526 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.526 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.526 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.526 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.526 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.526 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.526 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.526 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.526 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.526 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.526 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.526 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.526 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.527 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.527 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.527 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.527 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.527 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.527 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.527 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.527 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.527 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.527 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.527 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.527 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.527 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.527 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.528 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.528 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.528 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.528 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.528 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.528 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.528 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.528 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.528 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_put_obj_enc_conflict_c_kms _____________ 2024-06-28T16:14:11.528 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.528 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.528 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.528 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.529 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.529 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.529 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.529 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.529 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.529 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.529 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.529 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.529 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.529 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.529 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.529 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.529 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.530 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.530 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.530 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.530 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.530 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.530 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.530 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.530 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.530 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.530 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.530 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.530 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.531 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.531 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.531 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.531 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.531 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.531 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.531 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.531 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.531 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.531 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.531 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.531 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.531 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.532 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.532 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.532 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.532 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.532 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.532 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.532 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.532 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.532 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.532 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.532 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.532 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.532 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.533 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.533 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.533 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.533 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.533 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.533 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.533 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.533 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.533 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.533 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.533 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.533 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.533 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.534 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.534 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.534 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.534 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.534 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.534 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.534 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.534 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.534 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.534 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.534 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.534 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.534 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.535 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.535 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.535 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.535 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.535 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.535 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.535 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.535 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.535 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.535 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_put_obj_enc_conflict_s3_kms _____________ 2024-06-28T16:14:11.535 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.535 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.535 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.536 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.536 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.536 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.536 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.536 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.536 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.536 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.536 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.536 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.536 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.536 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.536 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.536 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.536 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.537 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.537 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.537 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.537 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.537 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.537 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.537 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.537 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.537 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.537 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.537 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.537 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.538 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.538 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.538 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.538 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.538 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.538 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.538 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.538 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.538 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.538 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.538 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.538 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.538 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.538 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.539 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.539 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.539 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.539 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.539 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.539 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.539 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.539 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.539 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.539 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.539 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.539 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.539 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.539 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.540 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.540 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.540 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.540 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.540 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.540 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.540 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.540 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.540 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.540 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.540 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.540 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.540 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.541 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.541 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.541 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.541 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.541 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.541 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.541 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.541 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.541 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.541 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.541 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.541 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.541 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.541 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.542 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.542 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.542 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.542 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.542 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.542 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.542 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_put_obj_enc_conflict_bad_enc_kms __________ 2024-06-28T16:14:11.542 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.542 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.542 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.542 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.542 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.543 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.543 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.543 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.543 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.543 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.543 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.543 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.543 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.543 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.543 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.543 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.543 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.543 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.544 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.544 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.544 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.544 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.544 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.544 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.544 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.544 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.544 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.544 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.544 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.544 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.544 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.545 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.545 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.545 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.545 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.545 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.545 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.545 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.545 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.545 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.545 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.545 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.545 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.546 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.546 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.546 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.546 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.546 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.546 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.546 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.546 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.546 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.546 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.546 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.547 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.547 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.547 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.547 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.547 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.547 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.547 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.547 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.547 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.547 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.547 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.547 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.548 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.548 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.548 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.548 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.548 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.548 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.548 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.548 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.548 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.548 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.548 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.548 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.549 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.549 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.549 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.549 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.549 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.549 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.549 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.549 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.549 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.549 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.549 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.549 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.549 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_bucket_policy_put_obj_kms_noenc ___________ 2024-06-28T16:14:11.550 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.550 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.550 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.550 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.550 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.550 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.550 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.550 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.550 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.550 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.550 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.550 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.550 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.551 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.551 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.551 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.551 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.551 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.551 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.551 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.551 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.551 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.551 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.551 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.551 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.551 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.551 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.552 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.552 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.552 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.552 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.552 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.552 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.552 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.552 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.552 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.552 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.552 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.552 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.552 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.553 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.553 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.553 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.553 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.553 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.553 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.553 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.553 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.553 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.553 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.553 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.553 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.553 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.554 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.554 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.554 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.554 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.554 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.554 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.554 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.554 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.554 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.554 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.554 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.554 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.555 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.555 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.555 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.555 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.555 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.555 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.555 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.555 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.555 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.555 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.555 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.555 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.555 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.556 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.556 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.556 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.556 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.556 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.556 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.556 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.556 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.556 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.556 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.556 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.556 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.556 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_bucket_policy_put_obj_kms_s3 ____________ 2024-06-28T16:14:11.557 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.557 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.557 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.557 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.557 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.557 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.557 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.557 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.557 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.557 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.557 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.557 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.557 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.557 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.558 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.558 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.558 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.558 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.558 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.558 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.558 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.558 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.558 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.558 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.558 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.558 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.558 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.559 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.559 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.559 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.559 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.559 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.559 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.559 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.559 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.559 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.559 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.559 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.559 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.559 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.559 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.560 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.560 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.560 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.560 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.560 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.560 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.560 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.560 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.560 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.560 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.560 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.560 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.560 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.561 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.561 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.561 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.561 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.561 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.561 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.561 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.561 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.561 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.561 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.561 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.561 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.561 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.562 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.562 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.562 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.562 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.562 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.562 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.562 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.562 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.562 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.562 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.562 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.562 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.562 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.562 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.563 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.563 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.563 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.563 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.563 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.563 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.563 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.563 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.563 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.563 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_bucket_policy_get_obj_acl_existing_tag _______ 2024-06-28T16:14:11.563 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.563 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.564 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.564 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.564 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.564 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.564 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.564 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.564 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.564 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.564 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.564 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.564 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.564 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.564 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.564 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.565 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.565 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.565 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.565 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.565 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.565 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.565 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.565 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.565 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.565 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.565 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.565 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.565 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.566 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.566 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.566 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.566 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.566 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.566 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.566 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.566 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.566 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.566 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.566 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.567 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.568 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.568 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.568 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.568 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.568 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.568 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.568 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.568 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.568 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.568 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.568 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.568 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.568 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.568 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.569 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.569 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.569 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.569 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.569 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.569 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.569 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.569 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.569 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.569 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.569 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.569 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.569 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.570 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.570 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.570 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.570 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.570 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.570 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.570 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_object_lock_put_obj_lock ______________ 2024-06-28T16:14:11.570 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.570 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.570 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.570 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.571 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.571 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.571 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.571 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.571 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.571 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.571 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.571 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.571 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.571 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.571 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.571 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.571 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.571 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.572 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.572 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.572 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.572 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.572 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.572 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.572 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.572 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.572 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.572 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.572 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.572 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.572 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.572 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.573 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.573 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.573 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.573 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.573 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.573 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.573 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.573 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.573 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.573 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.573 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.574 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.574 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.574 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.574 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.574 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.574 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.574 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.574 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.574 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.574 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.574 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.574 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.574 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.575 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.575 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.575 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.575 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.575 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.575 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.575 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.575 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.575 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.575 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.575 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.575 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.576 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.576 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.576 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.576 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.576 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.576 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.576 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.576 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.576 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.576 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.576 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.576 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.576 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.577 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.577 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.577 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.577 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.577 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.577 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.577 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.577 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_bucket _______ 2024-06-28T16:14:11.577 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.577 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.577 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.577 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.577 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.578 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.578 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.578 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.578 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.578 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.578 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.578 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.578 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.578 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.578 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.578 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.578 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.578 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.579 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.579 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.579 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.579 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.579 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.579 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.579 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.579 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.579 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.579 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.579 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.579 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.579 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.579 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.580 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.580 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.580 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.580 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.580 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.580 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.580 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.580 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.580 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.580 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.580 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.580 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.581 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.581 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.581 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.581 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.581 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.581 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.581 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.581 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.581 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.581 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.581 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.581 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.581 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.582 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.582 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.582 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.582 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.582 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.582 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.582 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.582 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.582 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.582 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.582 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.582 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.582 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.582 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.583 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.583 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.583 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.583 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.583 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.583 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.583 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.583 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.583 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.583 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.583 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.583 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.583 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.583 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.584 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.584 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.584 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.584 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.584 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_object_lock_put_obj_lock_with_days_and_years ____ 2024-06-28T16:14:11.584 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.584 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.584 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.584 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.584 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.584 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.584 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.584 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.585 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.585 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.585 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.585 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.585 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.585 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.585 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.585 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.585 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.586 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.586 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.586 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.586 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.586 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.587 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.588 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.589 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.590 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.590 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.591 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.591 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.591 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_days ________ 2024-06-28T16:14:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.591 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.591 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.591 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.591 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.592 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.592 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.592 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.592 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.592 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.592 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.592 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.592 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.593 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.593 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.593 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.593 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.593 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.594 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.595 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.596 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.596 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.596 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.597 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.597 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.597 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.597 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.597 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.598 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.598 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.598 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.598 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.598 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.598 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.598 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.598 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.598 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.598 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.598 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.598 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.598 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.599 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.599 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.599 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.599 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.599 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.599 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.599 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.599 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.599 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.599 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.599 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_years _______ 2024-06-28T16:14:11.599 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.599 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.599 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.600 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.600 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.600 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.600 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.600 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.600 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.600 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.600 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.600 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.600 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.600 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.600 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.600 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.601 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.601 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.601 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.601 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.601 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.601 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.601 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.601 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.601 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.601 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.601 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.601 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.601 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.602 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.602 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.602 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.602 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.602 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.602 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.602 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.602 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.602 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.602 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.603 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.603 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.603 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.603 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.603 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.603 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.603 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.603 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.603 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.603 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.603 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.603 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.603 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.603 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.604 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.604 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.604 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.604 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.604 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.604 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.604 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.604 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.604 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.604 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.604 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.604 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.604 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.605 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.605 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.605 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.605 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.605 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.605 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.605 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.605 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.605 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.605 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.605 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.605 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.605 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.605 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.606 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.606 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.606 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.606 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.606 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.606 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.606 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.606 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.606 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_mode ________ 2024-06-28T16:14:11.606 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.606 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.606 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.607 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.607 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.607 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.607 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.607 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.607 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.607 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.607 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.607 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.607 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.607 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.608 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.608 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.608 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.608 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.608 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.608 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.608 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.608 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.608 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.608 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.608 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.608 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.608 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.609 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.609 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.609 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.609 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.609 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.609 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.609 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.609 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.609 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.609 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.609 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.610 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.610 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.610 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.610 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.610 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.610 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.610 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.610 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.610 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.610 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.610 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.610 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.610 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.611 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.612 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.612 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.612 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.612 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.612 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.612 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.612 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.612 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.612 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.612 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.612 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.612 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.612 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.613 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.613 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.613 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.613 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.613 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.613 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.613 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.613 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.613 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.613 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.613 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_object_lock_put_obj_lock_invalid_status _______ 2024-06-28T16:14:11.613 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.613 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.614 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.614 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.614 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.614 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.614 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.614 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.614 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.614 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.614 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.614 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.614 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.614 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.614 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.615 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.615 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.615 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.615 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.615 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.615 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.615 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.615 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.615 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.615 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.615 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.615 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.615 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.615 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.616 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.616 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.616 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.616 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.616 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.616 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.616 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.616 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.616 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.616 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.616 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.617 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.617 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.617 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.617 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.617 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.617 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.617 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.617 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.617 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.617 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.617 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.617 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.617 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.617 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.618 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.618 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.618 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.618 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.618 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.618 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.618 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.618 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.618 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.618 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.618 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.618 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.618 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.619 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.619 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.619 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.619 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.619 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.619 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.619 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.619 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.619 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.619 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.619 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.619 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.619 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.620 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.620 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.620 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.620 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.620 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.620 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.620 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.620 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.620 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_object_lock_suspend_versioning ___________ 2024-06-28T16:14:11.620 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.620 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.620 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.620 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.620 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.621 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.621 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.621 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.621 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.621 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.621 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.621 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.621 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.621 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.621 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.621 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.621 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.622 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.622 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.622 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.622 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.622 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.622 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.622 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.622 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.622 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.622 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.622 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.622 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.623 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.623 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.623 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.623 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.623 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.623 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.623 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.623 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.623 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.623 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.623 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.624 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.624 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.624 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.624 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.624 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.624 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.624 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.624 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.624 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.624 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.624 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.624 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.625 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.625 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.625 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.625 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.625 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.625 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.625 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.625 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.625 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.625 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.625 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.625 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.625 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.626 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.626 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.626 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.626 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.626 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.626 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.626 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.626 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.626 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.626 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.626 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.626 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.627 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.627 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.627 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.627 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.627 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.627 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.627 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.627 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.627 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.627 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.627 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.627 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_object_lock_get_obj_lock ______________ 2024-06-28T16:14:11.627 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.628 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.628 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.628 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.628 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.628 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.628 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.628 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.628 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.628 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.628 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.628 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.628 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.628 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.629 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.629 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.629 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.629 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.629 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.629 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.629 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.629 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.629 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.629 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.629 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.629 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.629 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.629 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.630 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.630 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.630 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.630 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.630 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.630 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.630 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.630 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.630 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.630 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.630 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.630 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.630 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.631 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.631 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.631 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.631 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.631 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.631 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.631 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.631 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.631 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.631 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.631 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.631 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.631 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.632 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.632 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.632 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.632 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.632 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.632 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.632 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.632 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.632 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.632 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.632 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.632 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.632 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.632 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.633 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.633 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.633 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.633 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.633 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.633 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.633 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.633 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.633 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.633 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.633 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.633 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.633 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.633 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.634 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.634 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.634 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.634 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.634 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.634 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.634 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.634 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.634 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_object_lock_get_obj_lock_invalid_bucket _______ 2024-06-28T16:14:11.634 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.634 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.634 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.634 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.635 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.635 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.635 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.635 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.635 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.635 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.635 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.635 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.635 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.635 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.635 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.635 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.635 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.636 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.636 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.636 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.636 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.636 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.636 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.636 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.636 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.636 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.636 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.636 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.636 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.636 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.637 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.637 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.637 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.637 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.637 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.637 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.637 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.637 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.637 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.637 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.637 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.637 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.638 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.638 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.638 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.638 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.638 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.638 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.638 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.638 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.638 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.638 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.638 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.638 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.638 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.639 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.639 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.639 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.639 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.639 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.639 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.639 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.639 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.639 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.639 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.639 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.639 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.639 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.640 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.640 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.640 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.640 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.640 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.640 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.640 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.640 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.640 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.640 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.640 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.640 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.640 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.641 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.641 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.641 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.641 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.641 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.641 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.641 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.641 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.641 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_object_lock_put_obj_retention ____________ 2024-06-28T16:14:11.641 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.641 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.641 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.641 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.641 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.642 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.642 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.642 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.642 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.642 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.642 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.642 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.642 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.642 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.642 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.642 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.643 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.644 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.645 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.645 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.645 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.645 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.645 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.645 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.645 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.645 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.645 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.645 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.645 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.645 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.645 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.646 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.646 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.646 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.646 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.646 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.646 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.646 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.646 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.646 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.646 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.646 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.646 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.647 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.647 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.647 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.647 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.647 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.647 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.647 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.647 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.647 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.647 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.647 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.647 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.647 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.648 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.648 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.648 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.648 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.648 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.648 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.648 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.648 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_object_lock_put_obj_retention_invalid_bucket ____ 2024-06-28T16:14:11.648 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.648 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.648 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.648 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.648 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.648 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.649 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.649 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.649 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.649 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.649 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.649 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.649 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.649 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.649 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.649 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.649 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.649 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.649 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.650 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.650 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.650 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.650 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.650 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.650 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.650 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.650 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.650 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.650 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.650 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.650 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.650 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.651 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.651 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.651 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.651 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.651 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.651 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.651 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.651 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.651 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.651 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.651 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.651 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.651 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.651 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.652 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.652 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.652 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.652 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.652 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.652 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.652 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.652 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.652 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.652 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.652 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.652 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.652 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.653 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.654 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.654 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.654 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.654 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.654 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.654 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.654 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.654 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.654 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.654 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.654 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.654 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.655 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.655 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.655 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.655 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.655 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_object_lock_put_obj_retention_invalid_mode _____ 2024-06-28T16:14:11.655 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.655 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.655 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.655 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.655 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.655 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.655 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.656 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.656 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.656 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.656 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.656 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.656 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.656 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.656 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.656 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.656 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.656 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.656 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.656 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.656 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.657 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.657 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.657 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.657 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.657 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.657 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.657 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.657 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.657 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.657 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.657 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.657 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.657 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.658 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.659 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.659 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.659 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.659 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.659 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.659 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.659 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.659 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.659 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.659 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.659 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.660 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.660 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.660 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.660 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.660 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.660 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.660 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.660 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.660 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.660 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.660 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.660 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.661 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.661 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.661 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.661 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.661 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.661 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.661 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.661 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.661 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.661 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.661 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.661 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.661 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.661 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.662 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.662 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.662 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.662 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.662 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_object_lock_get_obj_retention ____________ 2024-06-28T16:14:11.662 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.662 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.662 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.662 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.662 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.662 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.662 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.662 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.662 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.663 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.664 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.665 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.666 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.666 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.666 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.666 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.666 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.666 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.666 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.666 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.666 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.666 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.666 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.666 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.667 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout:_______ ERROR at teardown of test_object_lock_get_obj_retention_iso8601 ________ 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.668 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.669 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.669 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.669 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.669 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.669 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.669 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.669 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.669 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.669 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.669 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.669 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.669 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.669 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.670 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.670 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.670 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.670 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.670 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.670 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.670 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.670 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.670 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.670 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.670 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.670 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.670 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.671 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.671 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.671 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.671 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.671 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.671 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.671 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.671 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.671 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.671 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.671 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.671 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.672 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.672 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.672 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.672 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.672 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.672 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.672 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.672 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.672 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.672 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.672 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.672 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.672 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.672 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.673 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.674 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_object_lock_get_obj_retention_invalid_bucket ____ 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.675 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.676 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.677 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.678 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.678 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.678 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.678 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.678 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.678 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.678 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.678 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.678 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.678 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.678 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.678 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.679 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.679 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.679 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.679 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.679 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.679 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.679 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.679 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.679 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.679 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.679 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.679 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.680 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.680 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.680 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.680 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.680 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.680 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.680 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.680 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.680 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.680 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.680 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.680 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.681 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.681 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.681 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.681 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.681 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.681 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.681 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.681 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.681 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.681 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.681 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.681 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_object_lock_put_obj_retention_versionid _______ 2024-06-28T16:14:11.681 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.682 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.682 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.682 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.682 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.682 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.682 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.682 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.682 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.682 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.682 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.682 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.682 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.682 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.683 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.683 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.683 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.683 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.683 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.683 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.683 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.683 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.683 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.683 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.683 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.683 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.683 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.683 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.684 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.684 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.684 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.684 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.684 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.684 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.684 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.684 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.684 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.684 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.684 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.684 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.684 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.685 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.686 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.686 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.686 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.686 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.686 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.686 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.686 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.686 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.686 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.686 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.686 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.686 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.686 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.686 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.687 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_object_lock_put_obj_retention_override_default_retention _ 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.688 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.689 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.690 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.690 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.690 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.690 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.690 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.690 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.690 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.690 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.690 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.690 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.690 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.691 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.691 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.691 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.691 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.691 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.691 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.691 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.691 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.691 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.691 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.691 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.691 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.691 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.691 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.692 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.692 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.692 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.692 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.692 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.692 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.692 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.692 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.692 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.692 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.692 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.692 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.693 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.693 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.693 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.693 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.693 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.693 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.693 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.693 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.693 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.693 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.693 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.693 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.693 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.694 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.694 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.694 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.694 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.694 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.694 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.694 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.694 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.694 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.694 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.694 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.694 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.694 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.695 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.695 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.695 INFO:teuthology.orchestra.run.smithi178.stdout:___ ERROR at teardown of test_object_lock_put_obj_retention_increase_period ____ 2024-06-28T16:14:11.695 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.695 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.695 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.695 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.695 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.695 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.695 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.695 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.695 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.696 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.697 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.697 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.697 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.697 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.697 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.697 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.697 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.697 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.697 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.697 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.697 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.697 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.697 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.697 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.698 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.698 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.698 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.698 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.698 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.698 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.698 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.698 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.698 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.698 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.698 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.698 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.698 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.699 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.699 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.699 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.699 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.699 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.699 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.699 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.699 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.699 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.699 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.699 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.699 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.699 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.699 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.700 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.700 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.700 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.700 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.700 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.700 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.700 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.700 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.700 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.700 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.700 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.700 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.701 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.701 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.701 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.701 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.701 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.701 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.701 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.701 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.701 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.701 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.701 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.701 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.701 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.701 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_object_lock_put_obj_retention_shorten_period ____ 2024-06-28T16:14:11.702 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.702 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.702 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.702 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.702 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.702 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.702 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.702 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.702 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.702 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.702 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.702 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.702 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.702 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.703 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.704 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.705 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.705 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.705 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.705 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.705 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.705 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.705 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.705 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.705 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.705 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.705 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.705 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.705 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.706 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.706 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.706 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.706 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.706 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.706 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.706 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.706 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.706 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.706 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.706 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.706 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.706 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.707 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.707 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.707 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.707 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.707 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.707 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.707 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.707 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.707 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.707 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.707 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.707 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.707 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.707 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.708 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.708 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.708 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.708 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.708 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.708 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.708 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_object_lock_put_obj_retention_shorten_period_bypass _ 2024-06-28T16:14:11.708 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.708 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.708 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.708 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.708 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.708 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.708 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.709 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.709 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.709 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.709 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.709 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.709 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.709 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.709 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.709 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.709 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.709 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.709 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.709 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.709 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.710 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.710 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.710 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.710 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.710 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.710 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.710 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.710 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.710 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.710 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.710 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.710 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.710 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.711 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.711 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.711 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.711 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.711 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.711 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.711 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.711 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.711 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.711 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.711 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.711 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.711 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.711 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.712 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.712 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.712 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.712 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.712 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.712 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.712 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.712 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.712 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.712 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.712 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.712 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.712 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.713 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.713 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.713 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.713 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.713 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.713 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.713 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.713 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.713 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.713 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.713 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.713 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.713 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.714 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.714 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.714 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.714 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.714 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.714 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.714 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.714 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.714 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.714 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.714 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.714 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.714 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.714 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.715 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.715 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.715 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_object_lock_delete_object_with_retention ______ 2024-06-28T16:14:11.715 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.715 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.715 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.715 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.715 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.715 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.715 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.715 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.715 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.715 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.716 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.716 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.716 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.716 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.716 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.716 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.716 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.716 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.716 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.716 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.716 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.716 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.717 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.718 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.718 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.718 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.718 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.718 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.718 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.718 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.718 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.718 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.718 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.718 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.718 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.718 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.718 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.719 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.719 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.719 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.719 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.719 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.719 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.719 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.719 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.719 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.719 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.719 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.719 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.719 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.719 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.720 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.720 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.720 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.720 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.720 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.720 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.720 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.720 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.720 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.720 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.720 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.720 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.720 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.721 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.721 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.721 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.721 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.721 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.721 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.721 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.721 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.721 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.721 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.721 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.721 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.721 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_object_lock_delete_multipart_object_with_retention _ 2024-06-28T16:14:11.722 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.722 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.722 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.722 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.722 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.722 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.722 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.722 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.722 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.722 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.722 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.722 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.722 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.723 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.723 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.723 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.723 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.723 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.723 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.723 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.723 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.723 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.723 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.723 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.724 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.724 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.724 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.724 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.724 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.724 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.724 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.724 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.724 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.724 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.724 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.724 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.725 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.725 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.725 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.725 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.725 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.725 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.725 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.725 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.725 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.725 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.725 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.725 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.726 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.726 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.726 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.726 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.726 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.726 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.726 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.726 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.726 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.726 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.726 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.726 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.726 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.727 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.727 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.727 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.727 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.727 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.727 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.727 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.727 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.727 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.727 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.727 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.727 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.727 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.728 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.728 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.728 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.728 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.728 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.728 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.728 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.728 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.728 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.728 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.728 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.728 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.728 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.729 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.729 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.729 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.729 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_object_lock_delete_object_with_retention_and_marker _ 2024-06-28T16:14:11.729 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.729 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.729 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.729 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.729 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.729 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.729 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.729 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.729 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.729 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.730 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.730 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.730 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.730 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.730 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.730 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.730 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.730 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.730 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.730 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.730 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.730 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.731 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.731 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.731 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.731 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.731 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.731 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.731 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.731 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.731 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.731 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.731 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.731 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.731 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.732 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.732 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.732 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.732 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.732 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.732 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.732 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.732 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.732 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.732 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.732 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.732 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.732 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.733 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.733 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.733 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.733 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.733 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.733 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.733 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.733 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.733 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.733 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.733 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.733 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.733 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.734 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.734 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.734 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.734 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.734 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.734 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.734 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.734 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.734 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.734 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.734 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.734 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.734 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.735 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.735 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.735 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.735 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.735 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.735 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.735 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.735 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.735 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.735 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.735 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.735 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.735 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.735 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.736 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.736 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.736 INFO:teuthology.orchestra.run.smithi178.stdout:___ ERROR at teardown of test_object_lock_multi_delete_object_with_retention ___ 2024-06-28T16:14:11.736 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.736 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.736 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.736 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.736 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.736 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.736 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.736 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.736 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.736 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.736 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.737 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.738 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.738 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.738 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.738 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.738 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.738 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.738 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.738 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.738 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.741 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.742 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.742 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.742 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.743 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.743 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_object_lock_put_legal_hold _____________ 2024-06-28T16:14:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.743 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.743 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.743 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.743 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.743 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.743 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.744 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.745 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.746 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.746 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.746 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.746 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.746 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.746 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.746 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.746 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.746 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.746 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.746 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.746 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.747 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.748 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.748 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.748 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.748 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.748 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.748 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.748 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.748 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.748 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.748 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.748 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.748 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.748 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.748 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.749 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.749 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.749 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.749 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.749 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.749 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.749 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_object_lock_put_legal_hold_invalid_bucket ______ 2024-06-28T16:14:11.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.749 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.749 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.749 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.750 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.750 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.750 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.750 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.750 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.750 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.750 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.750 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.750 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.750 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.750 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.750 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.750 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.751 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.751 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.751 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.751 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.751 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.751 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.751 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.751 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.751 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.751 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.751 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.751 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.751 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.752 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.752 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.752 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.752 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.752 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.752 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.752 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.752 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.752 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.752 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.752 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.752 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.752 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.753 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.754 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.754 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.754 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.754 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.754 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.754 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.754 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.754 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.754 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.754 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.754 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.754 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.754 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.755 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.755 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.755 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.755 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.755 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.755 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.755 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.755 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.755 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.755 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.755 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.755 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.755 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.755 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_object_lock_put_legal_hold_invalid_status ______ 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.756 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.757 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.757 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.757 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.757 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.757 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.757 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.757 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.757 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.757 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.757 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.757 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.757 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.757 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.758 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.758 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.758 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.758 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.758 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.758 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.758 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.758 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.758 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.758 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.758 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.758 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.758 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.759 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.759 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.759 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.759 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.759 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.759 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.759 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.759 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.759 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.759 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.759 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.759 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.759 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.760 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.760 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.760 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.760 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.760 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.760 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.760 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.760 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.760 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.760 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.760 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.761 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.761 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.761 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.761 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.761 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.761 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.761 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.761 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.761 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.761 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.761 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.761 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.761 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.761 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.762 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.762 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.762 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.762 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.762 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.762 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.762 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.762 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.762 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.762 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.762 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.762 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.762 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.763 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.763 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.763 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.763 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_object_lock_get_legal_hold _____________ 2024-06-28T16:14:11.763 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.763 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.763 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.763 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.763 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.763 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.763 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.763 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.763 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.764 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.764 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.764 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.764 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.764 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.764 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.764 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.764 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.764 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.764 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.764 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.764 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.764 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.764 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.765 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.765 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.765 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.765 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.765 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.765 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.765 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.765 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.765 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.765 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.765 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.766 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.766 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.766 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.766 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.766 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.766 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.766 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.766 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.766 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.766 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.766 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.767 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.767 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.767 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.767 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.767 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.767 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.767 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.767 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.767 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.767 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.767 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.767 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.767 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.768 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.768 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.768 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.768 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.768 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.768 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.768 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.768 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.768 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.768 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.768 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.768 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.768 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.769 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.769 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.769 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.769 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.769 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.769 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.769 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.769 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.769 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.769 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.769 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.769 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.769 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.770 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.770 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.770 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.770 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.770 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_object_lock_get_legal_hold_invalid_bucket ______ 2024-06-28T16:14:11.770 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.770 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.770 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.770 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.770 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.770 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.770 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.770 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.771 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.771 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.771 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.771 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.771 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.771 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.771 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.771 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.771 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.771 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.771 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.771 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.771 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.771 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.772 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.772 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.772 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.772 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.772 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.772 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.772 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.772 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.772 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.772 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.772 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.772 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.772 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.772 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.773 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.773 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.773 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.773 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.773 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.773 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.773 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.773 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.773 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.773 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.773 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.773 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.773 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.774 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.774 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.774 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.774 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.774 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.774 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.774 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.774 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.774 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.774 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.774 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.774 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.774 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.775 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.775 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.775 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.775 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.775 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.775 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.775 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.775 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.775 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.775 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.775 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.775 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.775 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.775 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.776 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.776 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.776 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.776 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.776 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.776 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.776 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.776 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.776 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.776 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.776 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.776 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.777 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.777 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_object_lock_delete_object_with_legal_hold_on ____ 2024-06-28T16:14:11.777 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.777 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.777 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.777 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.777 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.777 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.777 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.777 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.777 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.777 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.777 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.777 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.778 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.778 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.778 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.778 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.778 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.778 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.778 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.778 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.778 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.778 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.778 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.778 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.778 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.779 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.779 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.779 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.779 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.779 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.779 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.779 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.779 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.779 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.779 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.779 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.780 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.780 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.780 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.780 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.780 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.780 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.780 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.780 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.780 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.780 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.780 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.780 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.780 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.781 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.781 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.781 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.781 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.781 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.781 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.781 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.781 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.781 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.781 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.781 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.781 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.781 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.781 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.782 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.782 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.782 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.782 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.782 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.782 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.782 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.782 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.782 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.782 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.782 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.783 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.783 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.783 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.783 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.783 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.783 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.783 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.783 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.783 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.783 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.783 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.783 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.783 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.783 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.784 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_object_lock_delete_multipart_object_with_legal_hold_on _ 2024-06-28T16:14:11.784 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.784 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.784 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.784 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.784 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.784 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.784 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.784 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.784 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.784 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.784 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.784 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.785 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.785 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.785 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.785 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.785 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.785 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.785 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.785 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.785 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.785 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.785 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.785 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.786 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.786 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.786 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.786 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.786 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.786 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.786 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.786 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.786 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.786 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.787 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.787 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.787 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.787 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.787 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.787 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.787 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.787 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.787 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.787 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.787 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.787 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.787 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.788 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.788 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.788 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.788 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.788 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.788 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.788 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.788 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.788 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.788 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.788 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.788 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.789 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.789 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.789 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.789 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.789 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.789 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.789 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.789 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.789 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.789 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.789 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.789 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.789 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.789 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.790 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.790 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.790 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.790 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.790 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.790 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.790 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.790 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.790 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.790 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.790 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.790 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.791 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.791 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.791 INFO:teuthology.orchestra.run.smithi178.stdout:___ ERROR at teardown of test_object_lock_delete_object_with_legal_hold_off ____ 2024-06-28T16:14:11.791 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.791 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.791 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.791 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.791 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.791 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.791 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.791 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.791 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.791 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.791 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.792 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.792 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.792 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.792 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.792 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.792 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.792 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.792 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.792 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.792 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.792 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.792 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.792 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.792 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.793 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.793 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.793 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.793 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.793 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.793 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.793 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.793 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.794 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.794 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.794 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.794 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.794 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.794 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.794 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.794 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.794 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.794 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.794 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.795 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.795 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.795 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.795 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.795 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.795 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.795 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.795 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.795 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.795 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.795 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.795 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.795 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.796 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.796 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.796 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.796 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.796 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.796 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.796 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.796 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.796 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.796 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.796 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.796 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.796 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.797 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.797 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.797 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.797 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.797 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.797 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.797 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.797 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.797 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.797 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.797 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.797 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.797 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.797 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.798 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.798 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_object_lock_get_obj_metadata ____________ 2024-06-28T16:14:11.798 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.798 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.798 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.798 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.798 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.798 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.798 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.798 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.798 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.798 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.799 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.799 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.799 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.799 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.799 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.799 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.799 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.799 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.799 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.799 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.799 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.799 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.799 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.800 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.800 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.800 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.800 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.800 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.800 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.800 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.800 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.800 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.800 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.800 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.800 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.800 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.801 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.801 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.801 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.801 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.801 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.801 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.801 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.801 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.801 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.801 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.801 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.802 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.802 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.802 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.802 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.802 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.802 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.802 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.802 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.802 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.802 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.802 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.802 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.802 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.803 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.803 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.803 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.803 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.803 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.803 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.803 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.803 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.803 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.803 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.803 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.803 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.803 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.804 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.804 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.804 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.804 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.804 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.804 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.804 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.804 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.804 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.804 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.804 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.804 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.804 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.804 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.805 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.805 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.805 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_object_lock_uploading_obj ______________ 2024-06-28T16:14:11.805 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.805 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.805 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.805 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.805 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.805 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.805 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.805 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.805 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.805 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.805 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.806 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.806 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.806 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.806 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.806 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.806 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.806 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.806 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.806 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.806 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.806 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.806 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.808 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.809 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.810 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.810 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.810 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.810 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.810 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.810 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.810 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.810 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.810 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.810 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.810 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.810 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.810 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_object_lock_changing_mode_from_governance_with_bypass _ 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.811 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.812 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.812 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.812 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.812 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.812 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.812 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.812 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.812 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.812 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.813 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.813 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.813 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.813 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.813 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.813 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.813 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.813 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.813 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.813 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.813 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.813 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.813 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.814 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.814 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.814 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.814 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.814 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.814 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.814 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.814 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.814 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.814 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.814 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.814 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.815 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.816 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.816 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.816 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.816 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.816 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.816 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.816 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.816 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.816 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.816 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.816 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.816 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.816 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.816 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.817 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.817 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.817 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.817 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.817 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.817 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.817 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.817 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.817 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.817 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.817 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.817 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.817 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.818 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.818 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.818 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.818 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.818 INFO:teuthology.orchestra.run.smithi178.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-06-28T16:14:11.818 INFO:teuthology.orchestra.run.smithi178.stdout:nuke_bucket test-yy0of5l43hi8iei4ttc0dgli-550 waiting 9.261322 seconds for object locks to expire 2024-06-28T16:14:11.818 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_object_lock_changing_mode_from_governance_without_bypass _ 2024-06-28T16:14:11.818 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.818 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.818 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.818 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.818 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.818 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.818 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.819 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.819 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.819 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.819 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.819 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.819 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.819 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.819 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.819 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.820 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.820 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.820 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.820 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.820 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.820 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.820 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.820 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.820 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.820 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.820 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.820 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.820 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.821 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.821 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.821 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.821 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.821 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.821 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.822 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.822 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.822 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.822 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.822 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.822 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.822 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.822 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.822 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.822 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.822 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.822 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.822 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.822 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.823 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.824 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.824 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.824 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.825 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.825 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_object_lock_changing_mode_from_compliance ______ 2024-06-28T16:14:11.825 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.825 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.825 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.825 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.825 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.825 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.825 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.825 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.825 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.825 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.825 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.825 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.826 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.826 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.826 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.826 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.826 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.826 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.826 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.826 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.826 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.826 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.826 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.826 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.826 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.827 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.827 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.827 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.827 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.827 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.827 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.827 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.827 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.827 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.827 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.827 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.828 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.828 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.828 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.828 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.828 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.828 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.828 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.828 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.828 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.829 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.829 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.829 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.829 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.829 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.829 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.829 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.829 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.829 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.829 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.829 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.829 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.829 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.830 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.830 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.830 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.830 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.830 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.830 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.830 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.830 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.830 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.830 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.830 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.830 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.830 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.831 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.831 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.831 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.831 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.831 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.831 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.831 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.831 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.831 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.831 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.831 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.831 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.831 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.832 INFO:teuthology.orchestra.run.smithi178.stdout:--------------------------- Captured stdout teardown --------------------------- 2024-06-28T16:14:11.832 INFO:teuthology.orchestra.run.smithi178.stdout:nuke_bucket test-yy0of5l43hi8iei4ttc0dgli-552 waiting 9.073323 seconds for object locks to expire 2024-06-28T16:14:11.832 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_copy_object_ifmatch_good ______________ 2024-06-28T16:14:11.832 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.832 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.832 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.832 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.832 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.832 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.832 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.832 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.832 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.832 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.832 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.833 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.833 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.833 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.833 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.833 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.833 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.833 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.833 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.833 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.833 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.833 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.833 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.833 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.833 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.834 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.834 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.834 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.834 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.834 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.834 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.834 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.834 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.834 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.834 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.834 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.838 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.838 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.838 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.838 INFO:teuthology.orchestra.run.smithi178.stdout:___________ ERROR at teardown of test_copy_object_ifnonematch_failed ___________ 2024-06-28T16:14:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.838 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.839 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.839 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.839 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.839 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.839 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.839 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.839 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.839 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.839 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.839 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.839 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.839 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.839 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.840 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.840 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.840 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.840 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.840 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.840 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.840 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.840 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.840 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.840 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.840 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.840 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.840 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.845 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.845 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.845 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.845 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.845 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.845 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.845 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.845 INFO:teuthology.orchestra.run.smithi178.stdout:______________ ERROR at teardown of test_get_bucket_policy_status ______________ 2024-06-28T16:14:11.845 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.845 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.845 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.845 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.845 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.845 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.846 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.846 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.846 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.846 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.846 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.846 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.846 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.846 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.846 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.846 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.846 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.846 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.847 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.847 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.847 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.847 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.847 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.847 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.847 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.847 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.847 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.847 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.847 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.847 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.847 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.847 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.852 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.852 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.852 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.852 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.852 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_get_public_acl_bucket_policy_status _________ 2024-06-28T16:14:11.852 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.852 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.852 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.852 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.852 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.852 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.852 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.852 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.852 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.853 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.853 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.853 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.853 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.853 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.853 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.853 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.853 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.853 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.853 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.853 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.853 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.853 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.854 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.854 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.854 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.854 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.858 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.859 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.859 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.859 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.859 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_get_authpublic_acl_bucket_policy_status _______ 2024-06-28T16:14:11.859 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.859 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.859 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.859 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.859 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.859 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.859 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.859 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.859 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.859 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.860 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.860 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.860 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.860 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.860 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.860 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.860 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.860 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.860 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.861 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.861 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.861 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.865 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.865 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.865 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.865 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.865 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.865 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.865 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.865 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.865 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.865 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.865 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.865 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.865 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.865 INFO:teuthology.orchestra.run.smithi178.stdout:_____ ERROR at teardown of test_get_publicpolicy_acl_bucket_policy_status ______ 2024-06-28T16:14:11.866 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.866 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.866 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.866 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.866 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.866 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.866 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.866 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.866 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.866 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.866 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.866 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.866 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.867 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.867 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.867 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.867 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.867 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.867 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.867 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.867 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.867 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.867 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.867 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.867 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.867 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.868 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.869 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.869 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.869 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.869 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.869 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.869 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.869 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.869 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.869 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.869 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.869 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.869 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.869 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.870 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.870 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.870 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.870 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.870 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.870 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.870 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.870 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.870 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.870 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.870 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.870 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.871 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.871 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.871 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.871 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.871 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.871 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.871 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.871 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.871 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.871 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.871 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.871 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.871 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.871 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.872 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.872 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.872 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.872 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.872 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.872 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.872 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.872 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.872 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.872 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.872 INFO:teuthology.orchestra.run.smithi178.stdout:____ ERROR at teardown of test_get_nonpublicpolicy_acl_bucket_policy_status ____ 2024-06-28T16:14:11.872 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.872 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.873 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.873 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.873 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.873 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.873 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.873 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.873 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.873 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.873 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.873 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.873 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.873 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.873 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.873 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.874 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.874 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.874 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.874 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.874 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.874 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.874 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.874 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.874 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.874 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.874 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.874 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.874 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.875 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.875 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.875 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.875 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.875 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.875 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.875 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.875 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.875 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.875 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.876 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.876 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.876 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.876 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.876 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.876 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.876 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.876 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.876 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.876 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.876 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.876 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.877 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.877 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.877 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.877 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.877 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.877 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.877 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.877 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.877 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.877 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.877 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.877 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.877 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.878 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.878 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.878 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.878 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.878 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.878 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.878 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.878 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.878 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.878 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.878 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.878 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.878 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.879 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.879 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.879 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.879 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.879 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.879 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.879 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.879 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.879 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.879 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.879 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.879 INFO:teuthology.orchestra.run.smithi178.stdout:___ ERROR at teardown of test_get_nonpublicpolicy_deny_bucket_policy_status ____ 2024-06-28T16:14:11.879 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.880 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.880 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.880 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.880 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.880 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.880 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.880 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.880 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.880 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.880 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.880 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.880 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.880 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.880 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.881 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.881 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.881 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.881 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.881 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.881 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.881 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.881 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.881 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.881 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.881 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.881 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.881 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.881 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.882 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.882 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.882 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.882 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.882 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.882 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.882 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.882 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.882 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.882 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.882 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.882 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.883 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.883 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.883 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.883 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.883 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.883 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.883 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.883 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.883 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.883 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.883 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.883 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.883 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.883 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.884 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.885 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.885 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.885 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.885 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.885 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.885 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.885 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.885 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.885 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.885 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.885 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.885 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.885 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.886 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.886 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.886 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.886 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.886 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.886 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.886 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.886 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_get_undefined_public_block _____________ 2024-06-28T16:14:11.886 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.886 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.886 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.886 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.886 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.887 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.887 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.887 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.887 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.887 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.887 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.887 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.887 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.887 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.887 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.887 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.887 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.887 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.887 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.888 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.888 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.888 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.888 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.888 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.888 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.888 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.888 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.888 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.888 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.888 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.888 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.888 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.888 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.889 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.889 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.889 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.889 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.889 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.889 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.889 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.889 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.889 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.889 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.889 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.889 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.890 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.890 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.890 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.890 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.890 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.890 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.890 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.890 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.890 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.890 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.890 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.890 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.890 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.891 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.891 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.891 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.891 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.891 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.891 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.891 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.891 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.891 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.891 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.891 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.891 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.891 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.892 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.892 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.892 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.892 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.892 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.892 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.892 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.892 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.892 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.892 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.892 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.892 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.892 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.892 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.893 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.893 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.893 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.893 INFO:teuthology.orchestra.run.smithi178.stdout:________ ERROR at teardown of test_get_public_block_deny_bucket_policy _________ 2024-06-28T16:14:11.893 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.893 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.893 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.893 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.893 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.893 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.893 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.893 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.893 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.894 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.894 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.894 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.894 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.894 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.894 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.894 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.894 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.894 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.894 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.894 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.894 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.895 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.895 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.895 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.895 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.895 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.895 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.895 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.895 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.895 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.895 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.895 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.895 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.895 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.895 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.896 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.896 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.896 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.896 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.896 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.896 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.896 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.896 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.896 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.896 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.896 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.896 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.896 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.897 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.897 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.897 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.897 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.897 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.897 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.897 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.897 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.897 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.897 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.897 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.897 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.897 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.898 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.898 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.898 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.898 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.898 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.898 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.898 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.898 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.898 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.898 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.898 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.898 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.898 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.899 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.899 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.899 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.899 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.899 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.899 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.899 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.899 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.899 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.899 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.899 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.899 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.899 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.900 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.900 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.900 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.900 INFO:teuthology.orchestra.run.smithi178.stdout:__________________ ERROR at teardown of test_put_public_block __________________ 2024-06-28T16:14:11.900 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.900 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.900 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.900 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.900 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.900 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.900 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.900 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.900 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.901 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.901 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.901 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.901 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.901 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.901 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.901 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.901 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.901 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.901 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.901 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.901 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.901 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.902 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.902 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.902 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.902 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.902 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.902 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.902 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.902 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.902 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.902 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.902 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.902 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.902 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.902 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.903 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.903 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.903 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.903 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.903 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.903 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.903 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.903 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.903 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.903 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.903 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.903 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.903 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.904 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.904 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.904 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.904 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.904 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.904 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.904 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.904 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.904 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.904 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.904 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.904 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.904 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.905 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.905 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.905 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.905 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.905 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.905 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.905 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.905 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.905 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.905 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.905 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.905 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.905 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.905 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.906 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.906 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.906 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.906 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.906 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.906 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.906 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.906 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.906 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.906 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.906 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.906 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.907 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.907 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.907 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_block_public_put_bucket_acls ____________ 2024-06-28T16:14:11.907 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.907 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.907 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.907 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.907 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.907 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.907 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.907 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.907 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.907 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.907 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.908 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.908 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.908 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.908 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.908 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.908 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.908 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.908 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.908 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.908 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.908 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.908 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.908 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.909 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.909 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.909 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.909 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.909 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.909 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.909 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.909 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.909 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.909 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.909 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.909 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.909 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.909 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.910 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.910 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.910 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.910 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.910 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.910 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.910 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.910 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.910 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.910 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.910 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.910 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.911 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.911 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.911 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.911 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.911 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.911 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.911 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.911 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.911 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.911 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.911 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.911 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.911 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.911 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.912 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.912 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.912 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.912 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.912 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.912 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.912 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.912 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.912 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.912 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.912 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.912 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.912 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.913 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.913 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.913 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.913 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.913 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.913 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.913 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.913 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.913 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.913 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.913 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.913 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.913 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.913 INFO:teuthology.orchestra.run.smithi178.stdout:__________ ERROR at teardown of test_block_public_object_canned_acls ___________ 2024-06-28T16:14:11.914 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.914 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.914 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.914 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.914 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.914 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.914 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.914 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.914 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.914 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.914 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.914 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.914 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.915 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.915 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.915 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.915 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.915 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.915 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.915 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.915 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.915 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.915 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.915 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.915 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.915 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.915 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.916 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.916 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.916 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.916 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.916 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.916 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.916 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.916 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.916 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.916 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.916 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.917 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.917 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.917 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.917 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.917 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.917 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.917 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.917 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.917 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.917 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.917 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.917 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.917 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.917 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.918 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.918 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.918 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.918 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.918 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.918 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.918 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.918 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.918 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.918 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.918 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.918 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.918 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.919 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.919 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.919 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.919 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.919 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.919 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.919 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.919 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.919 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.919 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.919 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.919 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.919 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.920 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.920 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.920 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.920 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.920 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.920 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.920 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.920 INFO:teuthology.orchestra.run.smithi178.stdout:________________ ERROR at teardown of test_block_public_policy _________________ 2024-06-28T16:14:11.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.920 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.920 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.921 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.921 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.921 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.921 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.921 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.921 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.921 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.921 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.921 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.921 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.921 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.921 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.921 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.922 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.922 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.922 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.922 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.922 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.922 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.922 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.922 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.922 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.922 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.922 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.922 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.922 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.922 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.923 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.923 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.923 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.923 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.923 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.923 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.923 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.923 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.923 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.923 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.923 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.924 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.924 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.924 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.924 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.924 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.924 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.924 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.924 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.924 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.924 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.925 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.925 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.925 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.925 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.925 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.925 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.925 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.925 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.925 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.925 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.925 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.925 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.925 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.926 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.926 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.926 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.926 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.926 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.926 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.926 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.926 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.926 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.926 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.926 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.926 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.926 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.927 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.927 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.927 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.927 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.927 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.927 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.927 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.927 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.927 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_ignore_public_acls _________________ 2024-06-28T16:14:11.927 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.927 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.927 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.927 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.927 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.928 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.928 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.928 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.928 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.928 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.928 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.928 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.928 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.928 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.928 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.928 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.928 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.928 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.928 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.929 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.929 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.929 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.929 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.929 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.929 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.929 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.929 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.929 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.929 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.929 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.929 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.929 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.930 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.930 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.930 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.930 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.930 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.930 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.930 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.930 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.930 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.930 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.930 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.930 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.931 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.931 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.931 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.931 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.931 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.931 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.931 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.931 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.931 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.931 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.931 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.931 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.931 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.932 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.932 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.932 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.932 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.932 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.932 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.932 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.932 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.932 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.932 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.932 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.932 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.932 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.932 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.933 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.933 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.933 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.933 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.933 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.933 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.933 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.933 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.933 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.933 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.933 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.933 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.933 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.934 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.934 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.934 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.934 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.934 INFO:teuthology.orchestra.run.smithi178.stdout:______ ERROR at teardown of test_multipart_upload_on_a_bucket_with_policy ______ 2024-06-28T16:14:11.934 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.934 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.934 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.934 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.934 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.934 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.934 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.934 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.935 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.935 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.935 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.935 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.935 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.935 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.935 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.935 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.935 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.935 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.935 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.935 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.936 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.936 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.936 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.936 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.936 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.936 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.936 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.936 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.936 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.936 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.936 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.936 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.936 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.937 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.937 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.937 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.937 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.937 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.937 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.937 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.937 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.937 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.937 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.937 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.938 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.938 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.938 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.938 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.938 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.938 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.938 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.938 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.938 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.938 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.938 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.938 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.938 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.939 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.939 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.939 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.939 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.939 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.939 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.939 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.939 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.939 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.939 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.939 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.939 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.940 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.941 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.941 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.941 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.941 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.941 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.941 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_put_bucket_encryption_kms ______________ 2024-06-28T16:14:11.941 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.941 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.941 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.941 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.941 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.941 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.942 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.942 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.942 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.942 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.942 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.942 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.942 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.942 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.942 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.942 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.942 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.942 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.942 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.943 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.943 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.943 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.943 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.943 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.943 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.943 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.943 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.943 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.943 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.943 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.943 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.943 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.943 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.944 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.945 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.946 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.946 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.946 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.946 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.946 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.946 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.946 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.946 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.946 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.946 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.946 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.946 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.946 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.947 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.947 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.947 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.947 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.947 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.947 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.947 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.947 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.947 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.947 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.947 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.947 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.947 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.948 INFO:teuthology.orchestra.run.smithi178.stdout:_____________ ERROR at teardown of test_get_bucket_encryption_kms ______________ 2024-06-28T16:14:11.948 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.948 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.948 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.948 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.948 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.948 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.948 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.948 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.948 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.948 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.948 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.948 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.949 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.949 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.949 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.949 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.949 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.949 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.949 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.949 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.949 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.949 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.949 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.949 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.949 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.950 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.950 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.950 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.950 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.950 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.950 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.950 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.950 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.950 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.950 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.950 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.950 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.951 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.951 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.951 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.951 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.951 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.951 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.951 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.951 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.951 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.951 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.951 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.951 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.951 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.952 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.952 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.952 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.952 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.952 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.952 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.952 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.952 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.952 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.952 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.952 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.952 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.952 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.953 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.953 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.953 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.953 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.953 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.953 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.953 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.953 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.953 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.953 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.953 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.953 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.953 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.953 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.954 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.954 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.954 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.954 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.954 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.954 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.954 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.954 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.954 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.954 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.954 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.954 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.954 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.955 INFO:teuthology.orchestra.run.smithi178.stdout:____________ ERROR at teardown of test_delete_bucket_encryption_kms ____________ 2024-06-28T16:14:11.955 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.955 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.955 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.955 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.955 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.955 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.955 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.955 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.955 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.955 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.955 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.955 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.956 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.956 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.956 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.956 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.956 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.956 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.956 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.956 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.956 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.956 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.956 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.956 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.956 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.957 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.957 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.957 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.957 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.957 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.957 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.957 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.957 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.957 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.957 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.957 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.957 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.957 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.958 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.959 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.959 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.959 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.959 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.959 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.959 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.959 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.959 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.959 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.959 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.959 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.959 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.959 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.960 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.960 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.960 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.960 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.960 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.960 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.960 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.960 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.960 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.960 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.960 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.960 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.960 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.960 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.961 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.961 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.961 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.961 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.961 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.961 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.961 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.961 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.961 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.961 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.961 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.961 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.961 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.962 INFO:teuthology.orchestra.run.smithi178.stdout:_ ERROR at teardown of test_sse_kms_default_post_object_authenticated_request __ 2024-06-28T16:14:11.962 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.962 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.962 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.962 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.962 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.962 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.962 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.962 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.962 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.962 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.962 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.962 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.963 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.963 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.963 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.963 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.963 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.963 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.963 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.963 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.963 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.963 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.963 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.963 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.963 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.964 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.964 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.964 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.964 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.964 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.964 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.964 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.964 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.964 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.964 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.964 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.964 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.965 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.965 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.965 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.965 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.965 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.965 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.965 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.965 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.965 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.965 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.965 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.965 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.965 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.965 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.966 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.966 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.966 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.966 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.966 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.966 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.966 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.966 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.966 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.966 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.966 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.966 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.966 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.967 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.967 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.967 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.967 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.967 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.967 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.967 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.967 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.967 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.967 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.967 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.967 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.967 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.967 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.968 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.968 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.968 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.968 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.968 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.968 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.968 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.968 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.968 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.968 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.968 INFO:teuthology.orchestra.run.smithi178.stdout:_________________ ERROR at teardown of test_get_object_torrent _________________ 2024-06-28T16:14:11.968 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.968 INFO:teuthology.orchestra.run.smithi178.stdout:configfile = 2024-06-28T16:14:11.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.969 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.fixture(autouse=True) 2024-06-28T16:14:11.969 INFO:teuthology.orchestra.run.smithi178.stdout: def setup_teardown(configfile): 2024-06-28T16:14:11.969 INFO:teuthology.orchestra.run.smithi178.stdout: yield 2024-06-28T16:14:11.969 INFO:teuthology.orchestra.run.smithi178.stdout:> teardown() 2024-06-28T16:14:11.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.969 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:332: 2024-06-28T16:14:11.969 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.969 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:302: in teardown 2024-06-28T16:14:11.969 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_prefixed_buckets(prefix=prefix) 2024-06-28T16:14:11.969 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:159: in nuke_prefixed_buckets 2024-06-28T16:14:11.969 INFO:teuthology.orchestra.run.smithi178.stdout: raise err 2024-06-28T16:14:11.969 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:150: in nuke_prefixed_buckets 2024-06-28T16:14:11.970 INFO:teuthology.orchestra.run.smithi178.stdout: nuke_bucket(client, bucket_name) 2024-06-28T16:14:11.970 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:103: in nuke_bucket 2024-06-28T16:14:11.970 INFO:teuthology.orchestra.run.smithi178.stdout: for objects in list_versions(client, bucket, batch_size): 2024-06-28T16:14:11.970 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/__init__.py:88: in list_versions 2024-06-28T16:14:11.970 INFO:teuthology.orchestra.run.smithi178.stdout: listing = client.list_object_versions(**kwargs) 2024-06-28T16:14:11.970 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:514: in _api_call 2024-06-28T16:14:11.970 INFO:teuthology.orchestra.run.smithi178.stdout: return self._make_api_call(operation_name, kwargs) 2024-06-28T16:14:11.970 INFO:teuthology.orchestra.run.smithi178.stdout:_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 2024-06-28T16:14:11.970 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.970 INFO:teuthology.orchestra.run.smithi178.stdout:self = 2024-06-28T16:14:11.970 INFO:teuthology.orchestra.run.smithi178.stdout:operation_name = 'ListObjectVersions' 2024-06-28T16:14:11.970 INFO:teuthology.orchestra.run.smithi178.stdout:api_params = {'Bucket': 'test-yy0of5l43hi8iei4ttc0dgli-156', 'EncodingType': 'url', 'MaxKeys': 128} 2024-06-28T16:14:11.971 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.971 INFO:teuthology.orchestra.run.smithi178.stdout: def _make_api_call(self, operation_name, api_params): 2024-06-28T16:14:11.971 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model = self._service_model.operation_model(operation_name) 2024-06-28T16:14:11.971 INFO:teuthology.orchestra.run.smithi178.stdout: service_name = self._service_model.service_name 2024-06-28T16:14:11.971 INFO:teuthology.orchestra.run.smithi178.stdout: history_recorder.record( 2024-06-28T16:14:11.971 INFO:teuthology.orchestra.run.smithi178.stdout: 'API_CALL', 2024-06-28T16:14:11.971 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-06-28T16:14:11.971 INFO:teuthology.orchestra.run.smithi178.stdout: 'service': service_name, 2024-06-28T16:14:11.971 INFO:teuthology.orchestra.run.smithi178.stdout: 'operation': operation_name, 2024-06-28T16:14:11.971 INFO:teuthology.orchestra.run.smithi178.stdout: 'params': api_params, 2024-06-28T16:14:11.971 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-06-28T16:14:11.971 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.971 INFO:teuthology.orchestra.run.smithi178.stdout: if operation_model.deprecated: 2024-06-28T16:14:11.971 INFO:teuthology.orchestra.run.smithi178.stdout: logger.debug( 2024-06-28T16:14:11.972 INFO:teuthology.orchestra.run.smithi178.stdout: 'Warning: %s.%s() is deprecated', service_name, operation_name 2024-06-28T16:14:11.972 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.972 INFO:teuthology.orchestra.run.smithi178.stdout: request_context = { 2024-06-28T16:14:11.972 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_region': self.meta.region_name, 2024-06-28T16:14:11.972 INFO:teuthology.orchestra.run.smithi178.stdout: 'client_config': self.meta.config, 2024-06-28T16:14:11.972 INFO:teuthology.orchestra.run.smithi178.stdout: 'has_streaming_input': operation_model.has_streaming_input, 2024-06-28T16:14:11.972 INFO:teuthology.orchestra.run.smithi178.stdout: 'auth_type': operation_model.auth_type, 2024-06-28T16:14:11.972 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-06-28T16:14:11.972 INFO:teuthology.orchestra.run.smithi178.stdout: request_dict = self._convert_to_request_dict( 2024-06-28T16:14:11.972 INFO:teuthology.orchestra.run.smithi178.stdout: api_params, operation_model, context=request_context 2024-06-28T16:14:11.972 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.972 INFO:teuthology.orchestra.run.smithi178.stdout: resolve_checksum_context(request_dict, operation_model, api_params) 2024-06-28T16:14:11.972 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.972 INFO:teuthology.orchestra.run.smithi178.stdout: service_id = self._service_model.service_id.hyphenize() 2024-06-28T16:14:11.973 INFO:teuthology.orchestra.run.smithi178.stdout: handler, event_response = self.meta.events.emit_until_response( 2024-06-28T16:14:11.973 INFO:teuthology.orchestra.run.smithi178.stdout: 'before-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.973 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.973 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.973 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.973 INFO:teuthology.orchestra.run.smithi178.stdout: params=request_dict, 2024-06-28T16:14:11.973 INFO:teuthology.orchestra.run.smithi178.stdout: request_signer=self._request_signer, 2024-06-28T16:14:11.973 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.973 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.973 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.973 INFO:teuthology.orchestra.run.smithi178.stdout: if event_response is not None: 2024-06-28T16:14:11.973 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = event_response 2024-06-28T16:14:11.973 INFO:teuthology.orchestra.run.smithi178.stdout: else: 2024-06-28T16:14:11.974 INFO:teuthology.orchestra.run.smithi178.stdout: apply_request_checksum(request_dict) 2024-06-28T16:14:11.974 INFO:teuthology.orchestra.run.smithi178.stdout: http, parsed_response = self._make_request( 2024-06-28T16:14:11.974 INFO:teuthology.orchestra.run.smithi178.stdout: operation_model, request_dict, request_context 2024-06-28T16:14:11.974 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.974 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.974 INFO:teuthology.orchestra.run.smithi178.stdout: self.meta.events.emit( 2024-06-28T16:14:11.974 INFO:teuthology.orchestra.run.smithi178.stdout: 'after-call.{service_id}.{operation_name}'.format( 2024-06-28T16:14:11.974 INFO:teuthology.orchestra.run.smithi178.stdout: service_id=service_id, operation_name=operation_name 2024-06-28T16:14:11.974 INFO:teuthology.orchestra.run.smithi178.stdout: ), 2024-06-28T16:14:11.974 INFO:teuthology.orchestra.run.smithi178.stdout: http_response=http, 2024-06-28T16:14:11.974 INFO:teuthology.orchestra.run.smithi178.stdout: parsed=parsed_response, 2024-06-28T16:14:11.974 INFO:teuthology.orchestra.run.smithi178.stdout: model=operation_model, 2024-06-28T16:14:11.974 INFO:teuthology.orchestra.run.smithi178.stdout: context=request_context, 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout: ) 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout: if http.status_code >= 300: 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout: error_code = parsed_response.get("Error", {}).get("Code") 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout: error_class = self.exceptions.from_code(error_code) 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout:> raise error_class(parsed_response, operation_name) 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout:E botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectVersions operation: None 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/botocore/client.py:938: NoSuchBucket 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout:=============================== warnings summary =============================== 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout:.tox/py/lib/python3.10/site-packages/boto/plugin.py:40 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/lib/python3.10/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout: import imp 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.975 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py:120 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_headers.py:120: SyntaxWarning: "is" with a literal. Did you mean "=="? 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout: if boto_type is 'S3Connection': 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_headers.py:124 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_headers.py:124: SyntaxWarning: "is" with a literal. Did you mean "=="? 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout: elif boto_type is 'HTTPRequest': 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3.py:359 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3.py:359: DeprecationWarning: invalid escape sequence '\W' 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout: extra_sc = re.split('\W+', config['main']['storage_classes']) 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py:176 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3_website.py:176: SyntaxWarning: "is" with a literal. Did you mean "=="? 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout: if k is 'BucketName': 2024-06-28T16:14:11.976 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.977 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py:253 2024-06-28T16:14:11.977 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests/functional/test_s3_website.py:253: PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html 2024-06-28T16:14:11.977 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.mark.timeout(10) 2024-06-28T16:14:11.977 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.977 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests/functional/test_s3_website.py:1077 2024-06-28T16:14:11.977 INFO:teuthology.orchestra.run.smithi178.stdout: s3tests/functional/test_s3_website.py:1077: PytestCollectionWarning: yield tests were removed in pytest 4.0 - test_routing_generator will be ignored 2024-06-28T16:14:11.977 INFO:teuthology.orchestra.run.smithi178.stdout: @pytest.mark.s3website_routing_rules 2024-06-28T16:14:11.977 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.977 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py:2213 2024-06-28T16:14:11.977 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2213: DeprecationWarning: invalid escape sequence '\$' 2024-06-28T16:14:11.977 INFO:teuthology.orchestra.run.smithi178.stdout: ["starts-with", "$key", "\$foo"],\ 2024-06-28T16:14:11.977 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.977 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py:2228 2024-06-28T16:14:11.977 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2228: DeprecationWarning: invalid escape sequence '\$' 2024-06-28T16:14:11.978 INFO:teuthology.orchestra.run.smithi178.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\ 2024-06-28T16:14:11.978 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.978 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py:2234 2024-06-28T16:14:11.978 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2234: DeprecationWarning: invalid escape sequence '\$' 2024-06-28T16:14:11.978 INFO:teuthology.orchestra.run.smithi178.stdout: response = client.get_object(Bucket=bucket_name, Key='\$foo.txt') 2024-06-28T16:14:11.978 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.978 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py:2291 2024-06-28T16:14:11.978 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2291: DeprecationWarning: invalid escape sequence '\$' 2024-06-28T16:14:11.978 INFO:teuthology.orchestra.run.smithi178.stdout: ["starts-with", "$key", "\$foo"],\ 2024-06-28T16:14:11.978 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.978 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py:2306 2024-06-28T16:14:11.978 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2306: DeprecationWarning: invalid escape sequence '\$' 2024-06-28T16:14:11.979 INFO:teuthology.orchestra.run.smithi178.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\ 2024-06-28T16:14:11.979 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.979 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py:2324 2024-06-28T16:14:11.979 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2324: DeprecationWarning: invalid escape sequence '\$' 2024-06-28T16:14:11.979 INFO:teuthology.orchestra.run.smithi178.stdout: ["starts-with", "$key", "\$foo"],\ 2024-06-28T16:14:11.979 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.979 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py:2339 2024-06-28T16:14:11.979 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2339: DeprecationWarning: invalid escape sequence '\$' 2024-06-28T16:14:11.979 INFO:teuthology.orchestra.run.smithi178.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id[::-1]),\ 2024-06-28T16:14:11.979 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.979 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py:2357 2024-06-28T16:14:11.979 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2357: DeprecationWarning: invalid escape sequence '\$' 2024-06-28T16:14:11.979 INFO:teuthology.orchestra.run.smithi178.stdout: ["starts-with", "$key", "\$foo"],\ 2024-06-28T16:14:11.979 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.980 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py:2372 2024-06-28T16:14:11.980 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2372: DeprecationWarning: invalid escape sequence '\$' 2024-06-28T16:14:11.980 INFO:teuthology.orchestra.run.smithi178.stdout: payload = OrderedDict([ ("key" , "\$foo.txt"),("AWSAccessKeyId" , aws_access_key_id),\ 2024-06-28T16:14:11.980 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.980 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py:2422 2024-06-28T16:14:11.980 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2422: DeprecationWarning: invalid escape sequence '\$' 2024-06-28T16:14:11.980 INFO:teuthology.orchestra.run.smithi178.stdout: ["starts-with", "$key", "\$foo"],\ 2024-06-28T16:14:11.980 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.980 INFO:teuthology.orchestra.run.smithi178.stdout:s3tests_boto3/functional/test_s3.py:2454 2024-06-28T16:14:11.980 INFO:teuthology.orchestra.run.smithi178.stdout: /home/ubuntu/cephtest/s3-tests-client.0/s3tests_boto3/functional/test_s3.py:2454: DeprecationWarning: invalid escape sequence '\$' 2024-06-28T16:14:11.980 INFO:teuthology.orchestra.run.smithi178.stdout: ["starts-with", "$key", "\$foo"],\ 2024-06-28T16:14:11.980 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:11.981 INFO:teuthology.orchestra.run.smithi178.stdout:-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html 2024-06-28T16:14:11.981 INFO:teuthology.orchestra.run.smithi178.stdout:=========================== short test summary info ============================ 2024-06-28T16:14:11.981 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket 2024-06-28T16:14:11.981 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding 2024-06-28T16:14:11.981 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_delete - botoco... 2024-06-28T16:14:11.981 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_read_not_exist - botoc... 2024-06-28T16:14:11.981 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error 2024-06-28T16:14:11.981 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete 2024-06-28T16:14:11.981 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_object_delete - botocor... 2024-06-28T16:14:11.981 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete - botoc... 2024-06-28T16:14:11.981 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit 2024-06-28T16:14:11.981 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit 2024-06-28T16:14:11.981 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes - boto... 2024-06-28T16:14:11.981 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_check_etag - bot... 2024-06-28T16:14:11.982 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_cache_control - ... 2024-06-28T16:14:11.982 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_expires - botoco... 2024-06-28T16:14:11.982 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete 2024-06-28T16:14:11.982 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good 2024-06-28T16:14:11.982 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty 2024-06-28T16:14:11.982 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty 2024-06-28T16:14:11.982 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put 2024-06-28T16:14:11.982 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_write_file - botocore.... 2024-06-28T16:14:11.982 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request 2024-06-28T16:14:11.982 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request 2024-06-28T16:14:11.982 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type 2024-06-28T16:14:11.982 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key 2024-06-28T16:14:11.982 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code 2024-06-28T16:14:11.983 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code 2024-06-28T16:14:11.983 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk 2024-06-28T16:14:11.983 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename 2024-06-28T16:14:11.983 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header - ... 2024-06-28T16:14:11.983 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields 2024-06-28T16:14:11.983 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values 2024-06-28T16:14:11.983 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action 2024-06-28T16:14:11.983 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature 2024-06-28T16:14:11.983 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key 2024-06-28T16:14:11.983 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format 2024-06-28T16:14:11.983 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified 2024-06-28T16:14:11.983 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature 2024-06-28T16:14:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition 2024-06-28T16:14:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header 2024-06-28T16:14:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field 2024-06-28T16:14:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive 2024-06-28T16:14:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive 2024-06-28T16:14:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy - ... 2024-06-28T16:14:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket - bo... 2024-06-28T16:14:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value 2024-06-28T16:14:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition 2024-06-28T16:14:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list 2024-06-28T16:14:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded 2024-06-28T16:14:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument 2024-06-28T16:14:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument 2024-06-28T16:14:11.984 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum 2024-06-28T16:14:11.985 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug 2024-06-28T16:14:11.985 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions 2024-06-28T16:14:11.985 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good - bot... 2024-06-28T16:14:11.985 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed - b... 2024-06-28T16:14:11.985 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good 2024-06-28T16:14:11.985 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed 2024-06-28T16:14:11.985 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good 2024-06-28T16:14:11.985 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed 2024-06-28T16:14:11.985 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good 2024-06-28T16:14:11.985 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed 2024-06-28T16:14:11.985 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good - bot... 2024-06-28T16:14:11.985 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed - b... 2024-06-28T16:14:11.986 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good 2024-06-28T16:14:11.986 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed 2024-06-28T16:14:11.986 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good - ... 2024-06-28T16:14:11.986 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed 2024-06-28T16:14:11.986 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good 2024-06-28T16:14:11.986 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed 2024-06-28T16:14:11.986 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get - botocore.err... 2024-06-28T16:14:11.986 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone - ... 2024-06-28T16:14:11.986 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone 2024-06-28T16:14:11.986 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone - ... 2024-06-28T16:14:11.986 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_head - botocore.errorf... 2024-06-28T16:14:11.986 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist - botoco... 2024-06-28T16:14:11.987 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_head_extended - botoco... 2024-06-28T16:14:11.987 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl - b... 2024-06-28T16:14:11.987 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl - b... 2024-06-28T16:14:11.987 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime - botoco... 2024-06-28T16:14:11.987 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated - bo... 2024-06-28T16:14:11.987 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers 2024-06-28T16:14:11.987 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl 2024-06-28T16:14:11.987 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl 2024-06-28T16:14:11.987 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone 2024-06-28T16:14:11.987 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone 2024-06-28T16:14:11.987 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired 2024-06-28T16:14:11.987 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant 2024-06-28T16:14:11.987 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero 2024-06-28T16:14:11.988 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range 2024-06-28T16:14:11.988 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range 2024-06-28T16:14:11.988 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_anon_put - botocore.er... 2024-06-28T16:14:11.988 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access 2024-06-28T16:14:11.988 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_put_authenticated - bo... 2024-06-28T16:14:11.988 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl 2024-06-28T16:14:11.988 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant 2024-06-28T16:14:11.988 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired 2024-06-28T16:14:11.988 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha 2024-06-28T16:14:11.988 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one 2024-06-28T16:14:11.988 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two 2024-06-28T16:14:11.988 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 2024-06-28T16:14:11.988 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 2024-06-28T16:14:11.988 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 2024-06-28T16:14:11.989 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 2024-06-28T16:14:11.989 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name - botoc... 2024-06-28T16:14:11.989 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip 2024-06-28T16:14:11.989 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore 2024-06-28T16:14:11.989 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long 2024-06-28T16:14:11.989 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end 2024-06-28T16:14:11.989 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot 2024-06-28T16:14:11.989 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash 2024-06-28T16:14:11.989 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot 2024-06-28T16:14:11.989 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_exists - botoco... 2024-06-28T16:14:11.989 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_get_location - botocor... 2024-06-28T16:14:11.989 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner 2024-06-28T16:14:11.989 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl 2024-06-28T16:14:11.989 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl - bot... 2024-06-28T16:14:11.990 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_default - botocore... 2024-06-28T16:14:11.990 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create 2024-06-28T16:14:11.990 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned - botocore.... 2024-06-28T16:14:11.990 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite 2024-06-28T16:14:11.990 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread 2024-06-28T16:14:11.990 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_default - botocore... 2024-06-28T16:14:11.990 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create 2024-06-28T16:14:11.990 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned - botocore.... 2024-06-28T16:14:11.990 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite 2024-06-28T16:14:11.990 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread 2024-06-28T16:14:11.990 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread 2024-06-28T16:14:11.990 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol 2024-06-28T16:14:11.991 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner 2024-06-28T16:14:11.991 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes 2024-06-28T16:14:11.991 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private 2024-06-28T16:14:11.991 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl - botocore.errorfa... 2024-06-28T16:14:11.991 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_write - botocore.e... 2024-06-28T16:14:11.991 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp - botocor... 2024-06-28T16:14:11.991 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_read - botocore.er... 2024-06-28T16:14:11.991 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_acl_readacp - botocore... 2024-06-28T16:14:11.991 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol 2024-06-28T16:14:11.991 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read 2024-06-28T16:14:11.991 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp 2024-06-28T16:14:11.991 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write 2024-06-28T16:14:11.991 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp 2024-06-28T16:14:11.992 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user 2024-06-28T16:14:11.992 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants - bo... 2024-06-28T16:14:11.992 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants - bo... 2024-06-28T16:14:11.992 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email - boto... 2024-06-28T16:14:11.992 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist 2024-06-28T16:14:11.992 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all - botoc... 2024-06-28T16:14:11.992 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private 2024-06-28T16:14:11.992 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private 2024-06-28T16:14:11.992 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread 2024-06-28T16:14:11.992 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread 2024-06-28T16:14:11.992 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite 2024-06-28T16:14:11.992 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite 2024-06-28T16:14:11.992 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private 2024-06-28T16:14:11.992 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread 2024-06-28T16:14:11.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite 2024-06-28T16:14:11.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private 2024-06-28T16:14:11.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread 2024-06-28T16:14:11.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite 2024-06-28T16:14:11.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list - bo... 2024-06-28T16:14:11.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime - botocore... 2024-06-28T16:14:11.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous - boto... 2024-06-28T16:14:11.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth - b... 2024-06-28T16:14:11.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth - botoc... 2024-06-28T16:14:11.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha 2024-06-28T16:14:11.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit 2024-06-28T16:14:11.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period 2024-06-28T16:14:11.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen 2024-06-28T16:14:11.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding 2024-06-28T16:14:11.994 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names 2024-06-28T16:14:11.994 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix - ... 2024-06-28T16:14:11.994 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size - botoc... 2024-06-28T16:14:11.994 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_16m - botocore.er... 2024-06-28T16:14:11.994 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket - bot... 2024-06-28T16:14:11.994 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype 2024-06-28T16:14:11.994 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself - botoc... 2024-06-28T16:14:11.994 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata 2024-06-28T16:14:11.994 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket - bot... 2024-06-28T16:14:11.994 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket 2024-06-28T16:14:11.994 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket 2024-06-28T16:14:11.994 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl - boto... 2024-06-28T16:14:11.994 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata 2024-06-28T16:14:11.995 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata 2024-06-28T16:14:11.995 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found 2024-06-28T16:14:11.995 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found - b... 2024-06-28T16:14:11.995 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket 2024-06-28T16:14:11.995 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding 2024-06-28T16:14:11.995 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload 2024-06-28T16:14:11.995 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty - boto... 2024-06-28T16:14:11.995 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_small - boto... 2024-06-28T16:14:11.995 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_small - botoco... 2024-06-28T16:14:11.995 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range 2024-06-28T16:14:11.995 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range 2024-06-28T16:14:11.995 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names 2024-06-28T16:14:11.995 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload - botocore.e... 2024-06-28T16:14:11.995 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned - bo... 2024-06-28T16:14:11.996 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part 2024-06-28T16:14:11.996 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes 2024-06-28T16:14:11.996 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes 2024-06-28T16:14:11.996 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small 2024-06-28T16:14:11.996 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents - b... 2024-06-28T16:14:11.996 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object 2024-06-28T16:14:11.996 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload - boto... 2024-06-28T16:14:11.996 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found 2024-06-28T16:14:11.996 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_multipart_upload - botoc... 2024-06-28T16:14:11.996 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner 2024-06-28T16:14:11.996 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part 2024-06-28T16:14:11.996 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag 2024-06-28T16:14:11.996 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_get_part - botocore... 2024-06-28T16:14:11.996 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part - boto... 2024-06-28T16:14:11.997 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_100_continue - botocore.error... 2024-06-28T16:14:11.997 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_set_cors - botocore.errorfact... 2024-06-28T16:14:11.997 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_origin_response - botoco... 2024-06-28T16:14:11.997 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_origin_wildcard - botoco... 2024-06-28T16:14:11.997 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_header_option - botocore... 2024-06-28T16:14:11.997 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object - b... 2024-06-28T16:14:11.997 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_get_object_tenant 2024-06-28T16:14:11.997 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object - b... 2024-06-28T16:14:11.997 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_with_acl 2024-06-28T16:14:11.997 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant 2024-06-28T16:14:11.997 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_cors_presigned_put_object_tenant_with_acl 2024-06-28T16:14:11.997 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_set_bucket_tagging - botocore... 2024-06-28T16:14:11.997 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_read_1mb - botocore.er... 2024-06-28T16:14:11.998 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_read_4mb - botocore.er... 2024-06-28T16:14:11.998 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_read_8mb - botocore.er... 2024-06-28T16:14:11.998 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_write_1mb - botocore.e... 2024-06-28T16:14:11.998 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_write_4mb - botocore.e... 2024-06-28T16:14:11.998 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_write_8mb - botocore.e... 2024-06-28T16:14:11.998 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_1mb - botoc... 2024-06-28T16:14:11.998 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_4mb - botoc... 2024-06-28T16:14:11.998 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_dual_write_8mb - botoc... 2024-06-28T16:14:11.998 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_conditional_write_1mb 2024-06-28T16:14:11.998 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_atomic_multipart_upload_write 2024-06-28T16:14:11.999 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_resend_first_finishes_last 2024-06-28T16:14:11.999 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_response_code 2024-06-28T16:14:11.999 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_big_request_response_code 2024-06-28T16:14:11.999 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_skip_leading_bytes_response_code 2024-06-28T16:14:11.999 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_return_trailing_bytes_response_code 2024-06-28T16:14:11.999 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_invalid_range 2024-06-28T16:14:11.999 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ranged_request_empty_object 2024-06-28T16:14:11.999 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_bucket_create_suspend 2024-06-28T16:14:11.999 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove 2024-06-28T16:14:11.999 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_read_remove_head 2024-06-28T16:14:11.999 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_removal 2024-06-28T16:14:11.999 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite 2024-06-28T16:14:11.999 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_plain_null_version_overwrite_suspended 2024-06-28T16:14:12.000 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspend_versions 2024-06-28T16:14:12.000 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_suspended_copy 2024-06-28T16:14:12.000 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_all 2024-06-28T16:14:12.000 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_versions_remove_special_names 2024-06-28T16:14:12.000 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_create_overwrite_multipart 2024-06-28T16:14:12.000 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_obj_list_marker - ... 2024-06-28T16:14:12.000 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_copy_obj_version 2024-06-28T16:14:12.000 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete 2024-06-28T16:14:12.000 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker 2024-06-28T16:14:12.000 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete_with_marker_create 2024-06-28T16:14:12.000 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioned_object_acl - botoco... 2024-06-28T16:14:12.000 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioned_object_acl_no_version_specified 2024-06-28T16:14:12.001 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioned_concurrent_object_create_and_remove 2024-06-28T16:14:12.001 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set - botocore.erro... 2024-06-28T16:14:12.001 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_get - botocore.erro... 2024-06-28T16:14:12.001 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_get_no_id - botocor... 2024-06-28T16:14:12.001 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_id_too_long - botoc... 2024-06-28T16:14:12.001 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_same_id - botocore.... 2024-06-28T16:14:12.001 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_invalid_status - bo... 2024-06-28T16:14:12.001 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_date - botocore... 2024-06-28T16:14:12.001 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_invalid_date - ... 2024-06-28T16:14:12.001 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_noncurrent - bo... 2024-06-28T16:14:12.001 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_deletemarker - ... 2024-06-28T16:14:12.001 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_filter - botoco... 2024-06-28T16:14:12.002 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_empty_filter - ... 2024-06-28T16:14:12.002 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_set_multipart - bot... 2024-06-28T16:14:12.002 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_lifecycle_transition_set_invalid_date 2024-06-28T16:14:12.002 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1b - botoc... 2024-06-28T16:14:12.002 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1kb - boto... 2024-06-28T16:14:12.002 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_1MB - boto... 2024-06-28T16:14:12.002 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encrypted_transfer_13b - boto... 2024-06-28T16:14:12.002 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_method_head 2024-06-28T16:14:12.002 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_present - bo... 2024-06-28T16:14:12.002 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_other_key - ... 2024-06-28T16:14:12.002 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_invalid_md5 2024-06-28T16:14:12.002 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_md5 - bot... 2024-06-28T16:14:12.003 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_no_key - bot... 2024-06-28T16:14:12.003 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_key_no_sse_c - bot... 2024-06-28T16:14:12.003 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_upload 2024-06-28T16:14:12.003 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_unaligned_multipart_upload 2024-06-28T16:14:12.003 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_multipart_bad_download 2024-06-28T16:14:12.003 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_encryption_sse_c_post_object_authenticated_request 2024-06-28T16:14:12.003 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_method_head - botocor... 2024-06-28T16:14:12.003 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_present - botocore.er... 2024-06-28T16:14:12.003 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_no_key - botocore.err... 2024-06-28T16:14:12.003 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_not_declared - botoco... 2024-06-28T16:14:12.003 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_upload - bo... 2024-06-28T16:14:12.003 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_1 2024-06-28T16:14:12.004 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_multipart_invalid_chunks_2 2024-06-28T16:14:12.004 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_post_object_authenticated_request 2024-06-28T16:14:12.004 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1b - botocor... 2024-06-28T16:14:12.004 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1kb - botoco... 2024-06-28T16:14:12.004 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_1MB - botoco... 2024-06-28T16:14:12.005 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_transfer_13b - botoco... 2024-06-28T16:14:12.005 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_read_declare - botoco... 2024-06-28T16:14:12.005 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy - botocore.erro... 2024-06-28T16:14:12.005 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucketv2_policy - botocore.er... 2024-06-28T16:14:12.006 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_acl - botocore.... 2024-06-28T16:14:12.006 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_acl - botocor... 2024-06-28T16:14:12.006 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_another_bucket 2024-06-28T16:14:12.006 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucketv2_policy_another_bucket 2024-06-28T16:14:12.006 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_obj_tagging - botocore.er... 2024-06-28T16:14:12.006 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_obj_head_tagging - botoco... 2024-06-28T16:14:12.006 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_max_tags - botocore.error... 2024-06-28T16:14:12.006 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_excess_tags - botocore.er... 2024-06-28T16:14:12.006 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_max_kvsize_tags - botocor... 2024-06-28T16:14:12.006 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_excess_key_tags - botocor... 2024-06-28T16:14:12.006 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_excess_val_tags - botocor... 2024-06-28T16:14:12.006 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_modify_tags - botocore.er... 2024-06-28T16:14:12.006 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_delete_tags - botocore.er... 2024-06-28T16:14:12.007 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_tags_anonymous_request 2024-06-28T16:14:12.007 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_post_object_tags_authenticated_request 2024-06-28T16:14:12.007 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_with_tags - botocore.... 2024-06-28T16:14:12.007 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_tags_acl_public - botocor... 2024-06-28T16:14:12.007 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_tags_acl_public - botocor... 2024-06-28T16:14:12.007 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_delete_tags_obj_public - boto... 2024-06-28T16:14:12.007 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_bucket_atomic_upload_return_version_id 2024-06-28T16:14:12.007 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_versioning_bucket_multipart_upload_return_version_id 2024-06-28T16:14:12.007 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_existing_tag 2024-06-28T16:14:12.007 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_tagging_existing_tag 2024-06-28T16:14:12.007 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_tagging_existing_tag 2024-06-28T16:14:12.007 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source 2024-06-28T16:14:12.007 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_copy_source_meta 2024-06-28T16:14:12.007 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_acl - b... 2024-06-28T16:14:12.008 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_grant 2024-06-28T16:14:12.008 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_s3 - b... 2024-06-28T16:14:12.008 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_c_kms - ... 2024-06-28T16:14:12.008 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_s3_kms 2024-06-28T16:14:12.008 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_obj_enc_conflict_bad_enc_kms 2024-06-28T16:14:12.008 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_noenc 2024-06-28T16:14:12.008 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_put_obj_kms_s3 2024-06-28T16:14:12.008 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_bucket_policy_get_obj_acl_existing_tag 2024-06-28T16:14:12.008 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock - bo... 2024-06-28T16:14:12.008 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_bucket 2024-06-28T16:14:12.008 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_with_days_and_years 2024-06-28T16:14:12.008 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_days 2024-06-28T16:14:12.008 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_years 2024-06-28T16:14:12.008 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_mode 2024-06-28T16:14:12.009 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_lock_invalid_status 2024-06-28T16:14:12.009 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_suspend_versioning 2024-06-28T16:14:12.009 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock - bo... 2024-06-28T16:14:12.009 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_lock_invalid_bucket 2024-06-28T16:14:12.009 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention 2024-06-28T16:14:12.009 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_bucket 2024-06-28T16:14:12.009 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_invalid_mode 2024-06-28T16:14:12.009 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention 2024-06-28T16:14:12.009 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_iso8601 2024-06-28T16:14:12.009 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_retention_invalid_bucket 2024-06-28T16:14:12.009 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_versionid 2024-06-28T16:14:12.009 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_override_default_retention 2024-06-28T16:14:12.009 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_increase_period 2024-06-28T16:14:12.010 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period 2024-06-28T16:14:12.010 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_obj_retention_shorten_period_bypass 2024-06-28T16:14:12.010 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention 2024-06-28T16:14:12.010 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_retention 2024-06-28T16:14:12.010 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_retention_and_marker 2024-06-28T16:14:12.010 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_multi_delete_object_with_retention 2024-06-28T16:14:12.010 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold - ... 2024-06-28T16:14:12.010 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_bucket 2024-06-28T16:14:12.010 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_put_legal_hold_invalid_status 2024-06-28T16:14:12.010 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold - ... 2024-06-28T16:14:12.010 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_legal_hold_invalid_bucket 2024-06-28T16:14:12.010 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_on 2024-06-28T16:14:12.011 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_multipart_object_with_legal_hold_on 2024-06-28T16:14:12.011 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_delete_object_with_legal_hold_off 2024-06-28T16:14:12.011 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_get_obj_metadata 2024-06-28T16:14:12.011 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_uploading_obj - b... 2024-06-28T16:14:12.011 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_with_bypass 2024-06-28T16:14:12.011 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_governance_without_bypass 2024-06-28T16:14:12.011 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_object_lock_changing_mode_from_compliance 2024-06-28T16:14:12.011 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_copy_object_ifmatch_good - bo... 2024-06-28T16:14:12.011 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_copy_object_ifnonematch_failed 2024-06-28T16:14:12.011 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_bucket_policy_status - bo... 2024-06-28T16:14:12.011 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_public_acl_bucket_policy_status 2024-06-28T16:14:12.011 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_authpublic_acl_bucket_policy_status 2024-06-28T16:14:12.011 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_publicpolicy_acl_bucket_policy_status 2024-06-28T16:14:12.011 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_acl_bucket_policy_status 2024-06-28T16:14:12.012 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_nonpublicpolicy_deny_bucket_policy_status 2024-06-28T16:14:12.012 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_undefined_public_block - ... 2024-06-28T16:14:12.012 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_public_block_deny_bucket_policy 2024-06-28T16:14:12.012 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_public_block - botocore.e... 2024-06-28T16:14:12.012 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_block_public_put_bucket_acls 2024-06-28T16:14:12.012 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_block_public_object_canned_acls 2024-06-28T16:14:12.012 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_block_public_policy - botocor... 2024-06-28T16:14:12.012 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_ignore_public_acls - botocore... 2024-06-28T16:14:12.012 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_multipart_upload_on_a_bucket_with_policy 2024-06-28T16:14:12.012 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_put_bucket_encryption_kms - b... 2024-06-28T16:14:12.012 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_bucket_encryption_kms - b... 2024-06-28T16:14:12.012 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_delete_bucket_encryption_kms 2024-06-28T16:14:12.012 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_sse_kms_default_post_object_authenticated_request 2024-06-28T16:14:12.013 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR s3tests_boto3/functional/test_s3.py::test_get_object_torrent - botocore... 2024-06-28T16:14:12.013 INFO:teuthology.orchestra.run.smithi178.stdout:= 666 passed, 65 skipped, 127 deselected, 1 xfailed, 18 warnings, 397 errors in 2061.62s (0:34:21) = 2024-06-28T16:14:12.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T16:14:12.014 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: InvocationError for command /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/bin/pytest -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' (exited with code 1) 2024-06-28T16:14:12.014 INFO:teuthology.orchestra.run.smithi178.stdout:___________________________________ summary ____________________________________ 2024-06-28T16:14:12.014 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: py: commands failed 2024-06-28T16:14:12.015 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (s3 tests against rgw) on smithi178 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-06-28T16:14:12.017 INFO:tasks.s3tests:Cleaning up boto... 2024-06-28T16:14:12.017 DEBUG:teuthology.orchestra.run.smithi178:> rm /home/ubuntu/cephtest/boto-client.0.cfg 2024-06-28T16:14:12.023 INFO:tasks.s3tests:Removing s3-tests... 2024-06-28T16:14:12.024 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/s3-tests-client.0 2024-06-28T16:14:12.217 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/s3tests.py", line 736, in task with contextutil.nested( File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/s3tests.py", line 61, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (s3 tests against rgw) on smithi178 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-06-28T16:14:12.451 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b33761ec7fb144c6b333ea70d845b0c7 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/s3tests.py", line 736, in task with contextutil.nested( File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/s3tests.py", line 61, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (s3 tests against rgw) on smithi178 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-06-28T16:14:12.455 DEBUG:teuthology.run_tasks:Unwinding manager s3tests 2024-06-28T16:14:12.464 DEBUG:teuthology.run_tasks:Unwinding manager s3tests-java 2024-06-28T16:14:12.498 DEBUG:tasks.s3tests_java:S3 Tests Java: END 2024-06-28T16:14:12.498 INFO:tasks.s3tests_java:S3 Tests Java: Cleaning up s3-tests-java... 2024-06-28T16:14:12.499 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/s3-tests-java 2024-06-28T16:14:12.555 INFO:tasks.s3tests_java:S3 Tests Java: Deleting S3 users... 2024-06-28T16:14:12.556 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid tester.client.0 --purge-data --cluster ceph 2024-06-28T16:14:12.592 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T16:14:12.593 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T16:14:12.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.624+0000 7f0137916980 20 rados->read ofs=0 len=0 2024-06-28T16:14:12.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.624+0000 7f0137916980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T16:14:12.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.624+0000 7f0137916980 20 realm 2024-06-28T16:14:12.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.624+0000 7f0137916980 20 rados->read ofs=0 len=0 2024-06-28T16:14:12.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.624+0000 7f0137916980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T16:14:12.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.624+0000 7f0137916980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T16:14:12.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.624+0000 7f0137916980 20 rados->read ofs=0 len=0 2024-06-28T16:14:12.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.624+0000 7f0137916980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T16:14:12.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.624+0000 7f0137916980 20 rados->read ofs=0 len=0 2024-06-28T16:14:12.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.628+0000 7f0137916980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T16:14:12.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.628+0000 7f0137916980 20 rados->read ofs=0 len=0 2024-06-28T16:14:12.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.628+0000 7f0137916980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T16:14:12.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.628+0000 7f0137916980 20 searching for the correct realm 2024-06-28T16:14:12.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.636+0000 7f0135c28640 0 -- 172.21.15.178:0/1120550751 >> [v2:172.21.15.195:6802/1595562688,v1:172.21.15.195:6804/1595562688] conn(0x7f0120008040 msgr2=0x7f0120062830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T16:14:12.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.852+0000 7f0135427640 0 -- 172.21.15.178:0/1120550751 >> [v2:172.21.15.195:6800/2644688681,v1:172.21.15.195:6801/2644688681] conn(0x558e67d82cb0 msgr2=0x558e67d850a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T16:14:12.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.860+0000 7f0137916980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T16:14:12.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.860+0000 7f0137916980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T16:14:12.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.860+0000 7f0137916980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T16:14:12.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.860+0000 7f0137916980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T16:14:12.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.860+0000 7f0137916980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T16:14:12.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.860+0000 7f0137916980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T16:14:12.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.860+0000 7f0137916980 20 rados->read ofs=0 len=0 2024-06-28T16:14:12.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.860+0000 7f0137916980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T16:14:12.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.860+0000 7f0137916980 20 rados->read ofs=0 len=0 2024-06-28T16:14:12.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.860+0000 7f0137916980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T16:14:12.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.860+0000 7f0137916980 20 rados->read ofs=0 len=0 2024-06-28T16:14:12.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T16:14:12.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 20 zone default found 2024-06-28T16:14:12.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 4 Realm: () 2024-06-28T16:14:12.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T16:14:12.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T16:14:12.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T16:14:12.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 20 zonegroup default 2024-06-28T16:14:12.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 20 rados->read ofs=0 len=0 2024-06-28T16:14:12.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T16:14:12.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 20 rados->read ofs=0 len=0 2024-06-28T16:14:12.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T16:14:12.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 20 rados->read ofs=0 len=0 2024-06-28T16:14:12.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T16:14:12.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 20 started sync module instance, tier type = 2024-06-28T16:14:12.874 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.864+0000 7f0137916980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T16:14:12.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.888+0000 7f0137916980 20 add_watcher() i=0 2024-06-28T16:14:12.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.888+0000 7f0137916980 20 add_watcher() i=1 2024-06-28T16:14:12.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.888+0000 7f0137916980 20 add_watcher() i=2 2024-06-28T16:14:12.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.888+0000 7f0137916980 20 add_watcher() i=3 2024-06-28T16:14:12.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.888+0000 7f0137916980 20 add_watcher() i=4 2024-06-28T16:14:12.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.888+0000 7f0137916980 20 add_watcher() i=5 2024-06-28T16:14:12.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.888+0000 7f0137916980 20 add_watcher() i=6 2024-06-28T16:14:12.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.892+0000 7f0137916980 20 add_watcher() i=7 2024-06-28T16:14:12.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.892+0000 7f0137916980 2 all 8 watchers are set, enabling cache 2024-06-28T16:14:12.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.900+0000 7effebfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T16:14:12.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.900+0000 7f0137916980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T16:14:12.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.900+0000 7f0137916980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T16:14:13.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.992+0000 7effeafe5640 20 reqs_thread_entry: start 2024-06-28T16:14:13.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.992+0000 7f0137916980 20 init_complete bucket index max shards: 1 2024-06-28T16:14:13.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.992+0000 7f0137916980 20 Filter name: none 2024-06-28T16:14:13.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.992+0000 7effe8fe1640 20 reqs_thread_entry: start 2024-06-28T16:14:13.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.992+0000 7f0137916980 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2024-06-28T16:14:13.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.992+0000 7f0137916980 20 rados->read ofs=0 len=0 2024-06-28T16:14:13.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.996+0000 7f0137916980 20 rados_obj.operate() r=0 bl.length=482 2024-06-28T16:14:13.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.996+0000 7f0137916980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x13 2024-06-28T16:14:13.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.996+0000 7f0137916980 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-06-28T16:14:13.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.996+0000 7f0137916980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-06-28T16:14:13.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:12.996+0000 7f0137916980 10 removing key index: LBUUKGXBLQEGBRHYUGIQ 2024-06-28T16:14:13.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.000+0000 7f0137916980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.keys:LBUUKGXBLQEGBRHYUGIQ, ofs0, ns] 2024-06-28T16:14:13.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.000+0000 7f0018ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497125 cookie 94070115989792 notifier 7499 bl.length()=170 2024-06-28T16:14:13.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.000+0000 7f0137916980 10 removing email index: tester.client.0_test@test.test 2024-06-28T16:14:13.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.004+0000 7f0137916980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:tester.client.0_test@test.test, ofs0, ns] 2024-06-28T16:14:13.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.004+0000 7f0018ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497157 cookie 94070115979056 notifier 7499 bl.length()=181 2024-06-28T16:14:13.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.004+0000 7f0137916980 10 removing user buckets index 2024-06-28T16:14:13.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.008+0000 7f0137916980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.uid:tester.client.0.buckets, ofs0, ns] 2024-06-28T16:14:13.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.008+0000 7f0007fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497126 cookie 94070115989792 notifier 7499 bl.length()=172 2024-06-28T16:14:13.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.012+0000 7f0137916980 10 removing user index: tester.client.0 2024-06-28T16:14:13.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.016+0000 7f0137916980 10 removing default.rgw.meta+users.uid+tester.client.0 from cache 2024-06-28T16:14:13.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.016+0000 7f0137916980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:tester.client.0, ofs0, ns] 2024-06-28T16:14:13.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.016+0000 7f0007fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1116691497447 cookie 94070115943168 notifier 7499 bl.length()=164 2024-06-28T16:14:13.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.016+0000 7effebfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T16:14:13.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.228+0000 7f0137916980 20 remove_watcher() i=0 2024-06-28T16:14:13.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.228+0000 7f0137916980 2 removed watcher, disabling cache 2024-06-28T16:14:13.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.232+0000 7f0137916980 20 remove_watcher() i=1 2024-06-28T16:14:13.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.236+0000 7f0137916980 20 remove_watcher() i=2 2024-06-28T16:14:13.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.240+0000 7f0137916980 20 remove_watcher() i=3 2024-06-28T16:14:13.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.244+0000 7f0137916980 20 remove_watcher() i=4 2024-06-28T16:14:13.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.244+0000 7f0137916980 20 remove_watcher() i=5 2024-06-28T16:14:13.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.248+0000 7f0137916980 20 remove_watcher() i=6 2024-06-28T16:14:13.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:13.252+0000 7f0137916980 20 remove_watcher() i=7 2024-06-28T16:14:13.285 DEBUG:teuthology.run_tasks:Unwinding manager ragweed 2024-06-28T16:14:13.296 DEBUG:teuthology.run_tasks:Unwinding manager tox 2024-06-28T16:14:13.345 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/tox-venv 2024-06-28T16:14:13.392 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2024-06-28T16:14:13.402 DEBUG:tasks.rgw.client.0:waiting for process to exit 2024-06-28T16:14:13.402 INFO:teuthology.orchestra.run:waiting for 300 2024-06-28T16:14:13.419 INFO:tasks.rgw.client.0.smithi178.stdout:2024-06-28T16:14:13.404+0000 e53d640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends beast port=80 -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground (PID: 37084) UID: 0 2024-06-28T16:14:13.421 INFO:tasks.rgw.client.0.smithi178.stdout:2024-06-28T16:14:13.412+0000 8ea5f80 -1 shutting down 2024-06-28T16:14:28.199 INFO:tasks.rgw.client.0.smithi178.stderr:==00:01:13:59.865 37087== could not unlink /tmp/vgdb-pipe-from-vgdb-to-37087-by-root-on-??? 2024-06-28T16:14:28.199 INFO:tasks.rgw.client.0.smithi178.stderr:==00:01:13:59.865 37087== could not unlink /tmp/vgdb-pipe-to-vgdb-from-37087-by-root-on-??? 2024-06-28T16:14:28.200 INFO:tasks.rgw.client.0.smithi178.stderr:==00:01:13:59.865 37087== could not unlink /tmp/vgdb-pipe-shared-mem-vgdb-37087-by-root-on-??? 2024-06-28T16:14:31.708 INFO:tasks.rgw.client.0:Stopped 2024-06-28T16:14:31.709 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2024-06-28T16:14:31.715 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/vault-root-token 2024-06-28T16:14:31.771 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /home/ubuntu/cephtest/url_file 2024-06-28T16:14:31.828 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process', '--include-all'] 2024-06-28T16:14:31.828 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process', '--include-all'] 2024-06-28T16:14:31.828 DEBUG:teuthology.orchestra.run.smithi178:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all 2024-06-28T16:14:31.906 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setuser ceph since I am not root 2024-06-28T16:14:31.906 INFO:teuthology.orchestra.run.smithi178.stderr:ignoring --setgroup ceph since I am not root 2024-06-28T16:14:31.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.939+0000 7f00c3c02980 20 rados->read ofs=0 len=0 2024-06-28T16:14:31.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.939+0000 7f00c3c02980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T16:14:31.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.939+0000 7f00c3c02980 20 realm 2024-06-28T16:14:31.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.939+0000 7f00c3c02980 20 rados->read ofs=0 len=0 2024-06-28T16:14:31.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.939+0000 7f00c3c02980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T16:14:31.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.939+0000 7f00c3c02980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-06-28T16:14:31.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.939+0000 7f00c3c02980 20 rados->read ofs=0 len=0 2024-06-28T16:14:31.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.939+0000 7f00c3c02980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T16:14:31.951 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.939+0000 7f00c3c02980 20 rados->read ofs=0 len=0 2024-06-28T16:14:31.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.943+0000 7f00c3c02980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T16:14:31.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.943+0000 7f00c3c02980 20 rados->read ofs=0 len=0 2024-06-28T16:14:31.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.943+0000 7f00c3c02980 20 rados_obj.operate() r=0 bl.length=1139 2024-06-28T16:14:31.953 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.943+0000 7f00c3c02980 20 searching for the correct realm 2024-06-28T16:14:31.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.947+0000 7f00c0f12640 0 -- 172.21.15.178:0/890311433 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x55bf84659410 msgr2=0x55bf8465b820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T16:14:31.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:31.955+0000 7f00c1713640 0 -- 172.21.15.178:0/890311433 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f00b80343c0 msgr2=0x7f00b8034840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T16:14:32.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.171+0000 7f00c3c02980 20 RGWRados::pool_iterate: got zone_info.843e24ce-085a-4043-8f55-38e05370e790 2024-06-28T16:14:32.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.171+0000 7f00c3c02980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-06-28T16:14:32.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.171+0000 7f00c3c02980 20 RGWRados::pool_iterate: got zonegroup_info.41a11cae-4b2b-4a4f-9ee7-3e2f73545553 2024-06-28T16:14:32.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.171+0000 7f00c3c02980 20 RGWRados::pool_iterate: got default.zone. 2024-06-28T16:14:32.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.171+0000 7f00c3c02980 20 RGWRados::pool_iterate: got zone_names.default 2024-06-28T16:14:32.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.171+0000 7f00c3c02980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-06-28T16:14:32.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.171+0000 7f00c3c02980 20 rados->read ofs=0 len=0 2024-06-28T16:14:32.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.171+0000 7f00c3c02980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T16:14:32.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.171+0000 7f00c3c02980 20 rados->read ofs=0 len=0 2024-06-28T16:14:32.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.175+0000 7f00c3c02980 20 rados_obj.operate() r=0 bl.length=46 2024-06-28T16:14:32.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.175+0000 7f00c3c02980 20 rados->read ofs=0 len=0 2024-06-28T16:14:32.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.175+0000 7f00c3c02980 20 rados_obj.operate() r=0 bl.length=470 2024-06-28T16:14:32.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.175+0000 7f00c3c02980 20 zone default found 2024-06-28T16:14:32.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.175+0000 7f00c3c02980 4 Realm: () 2024-06-28T16:14:32.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.175+0000 7f00c3c02980 4 ZoneGroup: default (41a11cae-4b2b-4a4f-9ee7-3e2f73545553) 2024-06-28T16:14:32.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.175+0000 7f00c3c02980 4 Zone: default (843e24ce-085a-4043-8f55-38e05370e790) 2024-06-28T16:14:32.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.175+0000 7f00c3c02980 10 cannot find current period zonegroup using local zonegroup configuration 2024-06-28T16:14:32.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.175+0000 7f00c3c02980 20 zonegroup default 2024-06-28T16:14:32.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.175+0000 7f00c3c02980 20 rados->read ofs=0 len=0 2024-06-28T16:14:32.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.175+0000 7f00c3c02980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T16:14:32.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.175+0000 7f00c3c02980 20 rados->read ofs=0 len=0 2024-06-28T16:14:32.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.175+0000 7f00c3c02980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T16:14:32.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.175+0000 7f00c3c02980 20 rados->read ofs=0 len=0 2024-06-28T16:14:32.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.179+0000 7f00c3c02980 20 rados_obj.operate() r=-2 bl.length=0 2024-06-28T16:14:32.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.179+0000 7f00c3c02980 20 started sync module instance, tier type = 2024-06-28T16:14:32.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.179+0000 7f00c3c02980 20 started zone id=843e24ce-085a-4043-8f55-38e05370e790 (name=default) with tier type = 2024-06-28T16:14:32.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.199+0000 7f00c3c02980 20 add_watcher() i=0 2024-06-28T16:14:32.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.199+0000 7f00c3c02980 20 add_watcher() i=1 2024-06-28T16:14:32.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.199+0000 7f00c3c02980 20 add_watcher() i=2 2024-06-28T16:14:32.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.199+0000 7f00c3c02980 20 add_watcher() i=3 2024-06-28T16:14:32.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.199+0000 7f00c3c02980 20 add_watcher() i=4 2024-06-28T16:14:32.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.199+0000 7f00c3c02980 20 add_watcher() i=5 2024-06-28T16:14:32.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.199+0000 7f00c3c02980 20 add_watcher() i=6 2024-06-28T16:14:32.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.199+0000 7f00c3c02980 20 add_watcher() i=7 2024-06-28T16:14:32.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.199+0000 7f00c3c02980 2 all 8 watchers are set, enabling cache 2024-06-28T16:14:32.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.203+0000 7eff777e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T16:14:32.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.203+0000 7f00c3c02980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-06-28T16:14:32.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.203+0000 7f00c3c02980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-06-28T16:14:32.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.291+0000 7eff767e4640 20 reqs_thread_entry: start 2024-06-28T16:14:32.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.295+0000 7f00c3c02980 20 init_complete bucket index max shards: 1 2024-06-28T16:14:32.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.295+0000 7f00c3c02980 20 Filter name: none 2024-06-28T16:14:32.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.295+0000 7efee7fff640 20 reqs_thread_entry: start 2024-06-28T16:14:32.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-06-28T16:14:32.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:32.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:32.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/44354' 2024-06-28T16:14:32.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7048410780311527244', time=2024-06-28T17:40:20.426248+0000, chain.objs.size()=2 2024-06-28T16:14:32.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.DHMteOz4oM0jpT6ikZl6wSaFoIljB3-_0 2024-06-28T16:14:32.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.DHMteOz4oM0jpT6ikZl6wSaFoIljB3-_1 2024-06-28T16:14:32.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_Sgz91TzXN_GNLVUsqW1TSKC-tfJk2n_', time=2024-06-28T17:40:34.996862+0000, chain.objs.size()=1 2024-06-28T16:14:32.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.151__shadow_.pBJDiNf9-MeYXpiEaA0KWNiwStUXcOl_0 2024-06-28T16:14:32.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6912344868131051672', time=2024-06-28T17:41:06.916037+0000, chain.objs.size()=1 2024-06-28T16:14:32.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.167__shadow_.yAdRPpc35x12P4NWH8kSwIBVoS8Bnp__0 2024-06-28T16:14:32.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9462084665324268556', time=2024-06-28T17:41:12.670129+0000, chain.objs.size()=1 2024-06-28T16:14:32.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.172__shadow_.T83BG0lRp7o_SLhgKy5tyKaIqPTFr0F_0 2024-06-28T16:14:32.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16408169705502137523', time=2024-06-28T17:44:39.014983+0000, chain.objs.size()=1 2024-06-28T16:14:32.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.223__shadow_.krD4e_gmS0wWzPxh4jR1KU9o6OAGlVy_0 2024-06-28T16:14:32.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14486833221789648304', time=2024-06-28T17:44:43.814007+0000, chain.objs.size()=1 2024-06-28T16:14:32.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.226__shadow_.7YkhYeGoS1Xk6S-IXTcHNeYf2-YKSAD_0 2024-06-28T16:14:32.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13439788067456245514', time=2024-06-28T17:45:00.235034+0000, chain.objs.size()=1 2024-06-28T16:14:32.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.235__shadow_.2VtiR9E9-yelCI1UnmNW5HyOm-jBdNt_0 2024-06-28T16:14:32.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6771407849392748188', time=2024-06-28T17:45:04.992888+0000, chain.objs.size()=1 2024-06-28T16:14:32.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.238__shadow_.NlLX-IAXMOUz1rSQHZqF8lz7ZIdsMWJ_0 2024-06-28T16:14:32.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17365459698289312763', time=2024-06-28T17:45:16.232608+0000, chain.objs.size()=1 2024-06-28T16:14:32.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.245__shadow_.tLGqGg_fQ-rGItifn1ipYb8hhW47NSK_0 2024-06-28T16:14:32.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7934252692135349833', time=2024-06-28T17:45:25.534269+0000, chain.objs.size()=1 2024-06-28T16:14:32.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.251__shadow_.8O9q27GnfTe_cM2sEwv_zYKVdf28hNj_0 2024-06-28T16:14:32.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2802828674171016825', time=2024-06-28T17:47:08.944291+0000, chain.objs.size()=1 2024-06-28T16:14:32.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.kCOsvEj7-2CEZC8OxgEzPpSJ8EcfkGP_0 2024-06-28T16:14:32.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5199938813026244922', time=2024-06-28T17:47:09.801872+0000, chain.objs.size()=1 2024-06-28T16:14:32.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.T10GOMBJVUXQi8YSHNh2QSgj7fC9p8T_0 2024-06-28T16:14:32.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12030759766440413210', time=2024-06-28T17:47:12.359415+0000, chain.objs.size()=1 2024-06-28T16:14:32.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.n04tBA-1ivf4q7bGFtWcNKmRncfJXuk_0 2024-06-28T16:14:32.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18130144593126009883', time=2024-06-28T17:47:13.262143+0000, chain.objs.size()=1 2024-06-28T16:14:32.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KNVn5I7RHR5V-joKOyHc_X38i8i_qGu_0 2024-06-28T16:14:32.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2884794733461201085', time=2024-06-28T17:47:13.626839+0000, chain.objs.size()=1 2024-06-28T16:14:32.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.c7yL-YIBf7oM_pIK9UVRIdhIS1KMYUu_0 2024-06-28T16:14:32.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12698246535892888550', time=2024-06-28T17:47:13.841457+0000, chain.objs.size()=1 2024-06-28T16:14:32.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9RAx-ZW4shNa8jZNNmdMhL2Ey4DGHIJ_0 2024-06-28T16:14:32.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1541721340394622592', time=2024-06-28T17:47:14.183119+0000, chain.objs.size()=1 2024-06-28T16:14:32.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.DQ1bkUKHJIvjWBc8xJ_6I94SSJHuUVl_0 2024-06-28T16:14:32.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.311+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7397411208709571277', time=2024-06-28T17:47:14.545342+0000, chain.objs.size()=1 2024-06-28T16:14:32.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.311+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VoBT0lGd0Sjc1EvzuqSPE9OJd5aFLjK_0 2024-06-28T16:14:32.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.311+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7837626003736719053', time=2024-06-28T17:47:14.950866+0000, chain.objs.size()=1 2024-06-28T16:14:32.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.311+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Z3J6PTvTymK49u62LJeRH99jiBxdhHW_0 2024-06-28T16:14:32.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.311+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12806293060466158684', time=2024-06-28T17:47:15.363040+0000, chain.objs.size()=1 2024-06-28T16:14:32.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.311+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.QIzMr-foicwKlCDtzGQ-h3JHIOI6zhr_0 2024-06-28T16:14:32.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.311+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15599230457072592403', time=2024-06-28T17:47:18.756168+0000, chain.objs.size()=1 2024-06-28T16:14:32.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.311+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ls9oM5j5yvLajUYhqPFWDOuUdI1Coug_0 2024-06-28T16:14:32.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.311+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15454889379531025169', time=2024-06-28T17:47:18.855399+0000, chain.objs.size()=1 2024-06-28T16:14:32.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.311+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.6VHpsHlgJz6VEPw0MG2soFabGDbJO4w_0 2024-06-28T16:14:32.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8126776865826257554', time=2024-06-28T17:47:18.995741+0000, chain.objs.size()=1 2024-06-28T16:14:32.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.NH9pujVqj3dTO3bQukxIG-kqHwLtbkI_0 2024-06-28T16:14:32.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12431065357211560338', time=2024-06-28T17:47:21.837640+0000, chain.objs.size()=1 2024-06-28T16:14:32.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dzGqQgt3GJ2fzVhLOTO376fbQxuG3rm_0 2024-06-28T16:14:32.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.319+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.581314055277191763', time=2024-06-28T17:47:22.263819+0000, chain.objs.size()=1 2024-06-28T16:14:32.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.cgYwaT0WxUYHQmrkvl3wPKbxU05ubSn_0 2024-06-28T16:14:32.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13502559381146257814', time=2024-06-28T17:47:22.354327+0000, chain.objs.size()=1 2024-06-28T16:14:32.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oywIjtSt1gKbVaRMh2lTbbj7ftG2hDR_0 2024-06-28T16:14:32.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10420392779486276508', time=2024-06-28T17:47:23.773271+0000, chain.objs.size()=1 2024-06-28T16:14:32.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.kjMZ8ogLJcUB7Bl9O1auLbdKAyxxmIC_0 2024-06-28T16:14:32.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5338472104442347910', time=2024-06-28T17:47:25.519944+0000, chain.objs.size()=1 2024-06-28T16:14:32.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.B792WhKzVc6w6aE_iEB8fV7C4V2qV8Q_0 2024-06-28T16:14:32.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6813724062351658322', time=2024-06-28T17:47:26.624636+0000, chain.objs.size()=1 2024-06-28T16:14:32.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.3IXQEWsgZgzMtLBobKBBz3TTNo_0L1V_0 2024-06-28T16:14:32.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12017054753558597578', time=2024-06-28T17:47:26.766475+0000, chain.objs.size()=1 2024-06-28T16:14:32.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.1wFHQu0_xWSFdKVfErDkMtpqKCQC6dR_0 2024-06-28T16:14:32.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8559233355901375320', time=2024-06-28T17:47:27.243819+0000, chain.objs.size()=1 2024-06-28T16:14:32.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.gUPZ3mJW5Pfrg1f5uhfZc4mA-i2lltW_0 2024-06-28T16:14:32.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14975105909394708955', time=2024-06-28T17:47:29.931419+0000, chain.objs.size()=1 2024-06-28T16:14:32.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.kZHNLXwJ2IUPzPWH1_ahfZN9Ipl9pLL_0 2024-06-28T16:14:32.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6808779672340871785', time=2024-06-28T17:47:29.932607+0000, chain.objs.size()=1 2024-06-28T16:14:32.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.lQz70Fqg0TcgTeYgl3kgaK4lKgJd1mp_0 2024-06-28T16:14:32.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15669632718200252742', time=2024-06-28T17:47:30.404898+0000, chain.objs.size()=1 2024-06-28T16:14:32.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.gbBm0li-r3KBuA94o_MkTWyaRUK3Zee_0 2024-06-28T16:14:32.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1093496796029887589', time=2024-06-28T17:47:33.072623+0000, chain.objs.size()=1 2024-06-28T16:14:32.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.SW0L5_WPxwxhReBTI9raUBZ6I2fA59F_0 2024-06-28T16:14:32.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9003582765763442881', time=2024-06-28T17:47:34.129584+0000, chain.objs.size()=1 2024-06-28T16:14:32.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ABtUW7fcbYItfqrV2irZSxsnEciQ_zI_0 2024-06-28T16:14:32.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.335+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1182250571672295849', time=2024-06-28T17:47:34.131688+0000, chain.objs.size()=1 2024-06-28T16:14:32.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.IsnIKoynzQZwob-NjYu0qogBtxW4Jeg_0 2024-06-28T16:14:32.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18204687340354828509', time=2024-06-28T17:47:34.181422+0000, chain.objs.size()=1 2024-06-28T16:14:32.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.0J8hchYEMKgVoZrCHBp2KZjiS6lACS8_0 2024-06-28T16:14:32.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7843926432908636470', time=2024-06-28T17:47:34.905115+0000, chain.objs.size()=1 2024-06-28T16:14:32.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.S_B4o4UddY-Ls_BuVc0wDYGD3VMyXNA_0 2024-06-28T16:14:32.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1030843229789713279', time=2024-06-28T17:47:35.722868+0000, chain.objs.size()=1 2024-06-28T16:14:32.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.O6rR1btwHg897NRwi3NXu3eNg2ty09p_0 2024-06-28T16:14:32.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11894667766424434594', time=2024-06-28T17:47:37.071618+0000, chain.objs.size()=1 2024-06-28T16:14:32.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.tHq8ks7-_NqN44F1-djC1M2kk02r_ZP_0 2024-06-28T16:14:32.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10047558128329109989', time=2024-06-28T17:47:37.867907+0000, chain.objs.size()=1 2024-06-28T16:14:32.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.gmlqwMo5v8Jcdfd0mT45w4Os8Vps0a4_0 2024-06-28T16:14:32.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3135354166292781458', time=2024-06-28T17:47:38.731861+0000, chain.objs.size()=1 2024-06-28T16:14:32.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.QSyJraiEbwUR1UwpVUpzd4Z9Rq0F9ab_0 2024-06-28T16:14:32.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9305090401295328237', time=2024-06-28T17:47:39.246380+0000, chain.objs.size()=1 2024-06-28T16:14:32.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.djRNBE49HXsQhgra2KRsRg_qbs74Iov_0 2024-06-28T16:14:32.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17891391692580656474', time=2024-06-28T17:47:42.574726+0000, chain.objs.size()=1 2024-06-28T16:14:32.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.254__shadow_.LS7ATjFA5ZgUa17h-JLlcVR-xlO2AHl_0 2024-06-28T16:14:32.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10425509759813486555', time=2024-06-28T17:48:03.699029+0000, chain.objs.size()=1 2024-06-28T16:14:32.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.268__shadow_.SXQE1xdwCsej_6RHet4wBei5UWXjRBh_0 2024-06-28T16:14:32.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15527504236145568098', time=2024-06-28T17:48:03.720116+0000, chain.objs.size()=1 2024-06-28T16:14:32.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.268__shadow_.ktB6DNCXqhFnrlzwHE4mv-D1dKG9u8p_0 2024-06-28T16:14:32.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.347+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3893124268585431950', time=2024-06-28T17:48:08.538614+0000, chain.objs.size()=1 2024-06-28T16:14:32.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.347+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.271__shadow_.Fmh39xbSGNYIOUzSg07sGfOPNA-I5WI_0 2024-06-28T16:14:32.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.355+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8250732709416460524', time=2024-06-28T17:48:29.579425+0000, chain.objs.size()=1 2024-06-28T16:14:32.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.355+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.307fAN8K0hzowpfqb57w0whAWKem1iz_0 2024-06-28T16:14:32.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.355+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.262505155359685338', time=2024-06-28T17:48:34.632349+0000, chain.objs.size()=1 2024-06-28T16:14:32.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.355+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.DAQTWhBWSHYOKwrp73o5hF6st8afUGh_0 2024-06-28T16:14:32.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.355+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11325343300667101227', time=2024-06-28T17:48:38.071045+0000, chain.objs.size()=1 2024-06-28T16:14:32.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.355+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.285__shadow_.EaFy6jMqPVV24Am6EXqhKd_GBtz73w3_0 2024-06-28T16:14:32.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.355+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6523740305096376071', time=2024-06-28T17:48:54.354777+0000, chain.objs.size()=1 2024-06-28T16:14:32.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.355+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.295__shadow_.Td89CFRFSHGE1hLGjL-BiyI3w6gtruu_0 2024-06-28T16:14:32.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.355+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12883623213890645757', time=2024-06-28T17:51:07.415733+0000, chain.objs.size()=1 2024-06-28T16:14:32.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.355+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ciXAjaROTQHdpdSES8FoNcJSwybQNdG_0 2024-06-28T16:14:32.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.355+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14122084051573560863', time=2024-06-28T17:51:08.206508+0000, chain.objs.size()=1 2024-06-28T16:14:32.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.355+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.oKAbQXmtgBt4Ylak6BWpa6PYUzteD4L_0 2024-06-28T16:14:32.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.355+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.391446762704402343', time=2024-06-28T17:51:09.074356+0000, chain.objs.size()=1 2024-06-28T16:14:32.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.355+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.g51ShgtutDhg0hJ4tUiTDPW8KpLy5YF_0 2024-06-28T16:14:32.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15124817402381963384', time=2024-06-28T17:51:11.210952+0000, chain.objs.size()=1 2024-06-28T16:14:32.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GlmC0lD3_1bnC0Z8Q0P9mbEE6JWsaXy_0 2024-06-28T16:14:32.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17784249531988205930', time=2024-06-28T17:51:12.245566+0000, chain.objs.size()=1 2024-06-28T16:14:32.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Ke3iPHRJ2sFbogsdCdICC63WLkAbfTW_0 2024-06-28T16:14:32.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13012163043228093733', time=2024-06-28T17:51:12.460929+0000, chain.objs.size()=1 2024-06-28T16:14:32.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.e048vN3OFCU6venc6FFaeJ5Jf0QHq2O_0 2024-06-28T16:14:32.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8485666316913425850', time=2024-06-28T17:51:13.349257+0000, chain.objs.size()=1 2024-06-28T16:14:32.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ITetJ86FiT7CZfpAXE4QK0XQVcd1q6M_0 2024-06-28T16:14:32.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11252678954904618913', time=2024-06-28T17:51:16.297412+0000, chain.objs.size()=1 2024-06-28T16:14:32.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.81xZ6QX5VhJ2rJZjX__UQnWXBLoport_0 2024-06-28T16:14:32.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18403500567907754106', time=2024-06-28T17:51:16.639143+0000, chain.objs.size()=1 2024-06-28T16:14:32.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.HuwzIwSnx8LGkMEtSv6HfB_zDAjJwwY_0 2024-06-28T16:14:32.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13248074560153892540', time=2024-06-28T17:51:16.825112+0000, chain.objs.size()=1 2024-06-28T16:14:32.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.rrX36t425vbFvKCDj7lUXqJ6SMVir-M_0 2024-06-28T16:14:32.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1155461714709746159', time=2024-06-28T17:51:19.416223+0000, chain.objs.size()=1 2024-06-28T16:14:32.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.LKWB2aD2Zg5SDTqdO0ixwdkxcEjg5xL_0 2024-06-28T16:14:32.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7686152746475232101', time=2024-06-28T17:51:19.909749+0000, chain.objs.size()=1 2024-06-28T16:14:32.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nMuQZY0HSbeZjWywNKf2wW3szkj0UyW_0 2024-06-28T16:14:32.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17805106415021594837', time=2024-06-28T17:51:22.263095+0000, chain.objs.size()=1 2024-06-28T16:14:32.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.8L4Ps1ah9CFHqPwVSlEZGfFS6qwMsfI_0 2024-06-28T16:14:32.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13323717660844906960', time=2024-06-28T17:51:23.642464+0000, chain.objs.size()=1 2024-06-28T16:14:32.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PHV8TuST_N3CBnX1D3UN21U6NMkAer1_0 2024-06-28T16:14:32.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17204766351844841942', time=2024-06-28T17:51:24.216394+0000, chain.objs.size()=1 2024-06-28T16:14:32.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.MyFR_89spA82Whq9uNB737AllvCcpd__0 2024-06-28T16:14:32.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.367+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13111646422629491293', time=2024-06-28T17:51:24.352939+0000, chain.objs.size()=1 2024-06-28T16:14:32.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.W30wreSBMGRh4nLk2RZRAR6ysuisvto_0 2024-06-28T16:14:32.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.367+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16301280655755595724', time=2024-06-28T17:51:24.637928+0000, chain.objs.size()=1 2024-06-28T16:14:32.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.JcY78OgDTjcEFJUbJD_tCLaB7IBkLdJ_0 2024-06-28T16:14:32.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.367+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8094354410794987354', time=2024-06-28T17:51:26.541722+0000, chain.objs.size()=1 2024-06-28T16:14:32.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.uZvENdzdlvjgeKftD46UyHueVhPa9nV_0 2024-06-28T16:14:32.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.371+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17137838316950246675', time=2024-06-28T17:51:30.316403+0000, chain.objs.size()=1 2024-06-28T16:14:32.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YE4Q8upcEG6xgvvQ5rG1MwfrHeAKdwT_0 2024-06-28T16:14:32.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.371+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1227552100173605198', time=2024-06-28T17:51:31.892244+0000, chain.objs.size()=1 2024-06-28T16:14:32.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.8nwpMQltb6ofPS5OnYncWjB1UbeiKYt_0 2024-06-28T16:14:32.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17265273658887143638', time=2024-06-28T17:51:32.320288+0000, chain.objs.size()=1 2024-06-28T16:14:32.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.NQmKI4uGlOWU9tcAN-wtNWYq0deCox5_0 2024-06-28T16:14:32.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6003000773625664904', time=2024-06-28T17:51:33.378634+0000, chain.objs.size()=1 2024-06-28T16:14:32.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.M9H5jHIz0r16JLQsnGmaFiFRFdV3AEg_0 2024-06-28T16:14:32.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8752682801637513039', time=2024-06-28T17:51:34.088278+0000, chain.objs.size()=1 2024-06-28T16:14:32.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.N0JWC-hJwb-nphckFTIKIeUtttHRUww_0 2024-06-28T16:14:32.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17899938108723983372', time=2024-06-28T17:51:34.197450+0000, chain.objs.size()=1 2024-06-28T16:14:32.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_._Cjim99XokV7eHBCd9_Ru4aDda8FBW-_0 2024-06-28T16:14:32.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16785000517845001205', time=2024-06-28T17:51:35.797576+0000, chain.objs.size()=1 2024-06-28T16:14:32.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Kn36ButpvHqj_dLiz-oJBjZfSxtsZAc_0 2024-06-28T16:14:32.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1533297432709654791', time=2024-06-28T17:51:35.875972+0000, chain.objs.size()=1 2024-06-28T16:14:32.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.RkNCTpBe6Q0aH5xnNe8jGKT-2eUs6FB_0 2024-06-28T16:14:32.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8834749771973983769', time=2024-06-28T17:51:36.878909+0000, chain.objs.size()=1 2024-06-28T16:14:32.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.95AXOXHrWKUh0A-O-dH6oHMNhv1_T1K_0 2024-06-28T16:14:32.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2942322997809909973', time=2024-06-28T17:51:38.426892+0000, chain.objs.size()=1 2024-06-28T16:14:32.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.SLTysh4ZGbtxSYc0KPceON1vnA75K4f_0 2024-06-28T16:14:32.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3346541705924236754', time=2024-06-28T17:53:20.983031+0000, chain.objs.size()=1 2024-06-28T16:14:32.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.yuNjcaqw5at79JEibrv5P_DzqiSrAxd_0 2024-06-28T16:14:32.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.379+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11536851142138638732', time=2024-06-28T17:53:21.488407+0000, chain.objs.size()=1 2024-06-28T16:14:32.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.379+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.dyPpGU4MWhOmLHb4lFVoMMJXkuy_nCw_0 2024-06-28T16:14:32.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.379+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18358224727806863543', time=2024-06-28T17:53:21.490953+0000, chain.objs.size()=1 2024-06-28T16:14:32.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.379+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.HRlJH9UCcJR72023e1y_YQywgrrpHJn_0 2024-06-28T16:14:32.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11879081186660598122', time=2024-06-28T17:53:21.493170+0000, chain.objs.size()=1 2024-06-28T16:14:32.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.5PAreKWG0O9VsFpyuRWV_cF-TXEtiu3_0 2024-06-28T16:14:32.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16756397804689713765', time=2024-06-28T17:53:22.224132+0000, chain.objs.size()=1 2024-06-28T16:14:32.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.DdsiHPj_xTW7ZqgWv26w06UBHgdraaN_0 2024-06-28T16:14:32.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15361327981739180583', time=2024-06-28T17:53:23.122684+0000, chain.objs.size()=1 2024-06-28T16:14:32.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bQm5_copn4xHMYQTTWltGw-z7m8blwI_0 2024-06-28T16:14:32.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2005132088833720847', time=2024-06-28T17:53:23.223054+0000, chain.objs.size()=1 2024-06-28T16:14:32.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.XNwKtXkjLUoLDwYGqE6rBln83wtZank_0 2024-06-28T16:14:32.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13375697834150432439', time=2024-06-28T17:53:23.225912+0000, chain.objs.size()=1 2024-06-28T16:14:32.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.dNZDv802qH6mIPZh7U3G-Inng6f29D9_0 2024-06-28T16:14:32.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3892547426341217555', time=2024-06-28T17:53:24.130562+0000, chain.objs.size()=1 2024-06-28T16:14:32.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Aqt01FZ0tU2LoJ6v10Mi-72AqoZ05Pp_0 2024-06-28T16:14:32.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6482759197464197779', time=2024-06-28T17:53:25.620758+0000, chain.objs.size()=1 2024-06-28T16:14:32.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.at0Qaccy0cdMF5dhYLjqKrKoBWDmI8m_0 2024-06-28T16:14:32.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.387+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8731449648607235837', time=2024-06-28T17:53:26.908899+0000, chain.objs.size()=1 2024-06-28T16:14:32.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.387+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.sa2lmQQGS6aS-F9iz1gxSLYV64XPw-e_0 2024-06-28T16:14:32.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.387+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17018995843738992259', time=2024-06-28T17:53:27.415384+0000, chain.objs.size()=1 2024-06-28T16:14:32.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.387+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.xAy5dEOH_F2ozr82D12C-WdlQSyOSxA_0 2024-06-28T16:14:32.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.387+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9921224050394600892', time=2024-06-28T17:53:27.417050+0000, chain.objs.size()=1 2024-06-28T16:14:32.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.387+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.AojMlvx5vzx6jYJFvG9hWsAh5mNaTSH_0 2024-06-28T16:14:32.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.387+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.626535587239126409', time=2024-06-28T17:53:28.309333+0000, chain.objs.size()=1 2024-06-28T16:14:32.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.387+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.K5RoitdD79L_cWcLU-GlC2mqdjy7D1l_0 2024-06-28T16:14:32.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.387+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14985699376657481226', time=2024-06-28T17:53:29.598396+0000, chain.objs.size()=1 2024-06-28T16:14:32.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.387+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.eOJCZn4Pf2jvV2646xGolC6SUmbfLIW_0 2024-06-28T16:14:32.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4802994159249282110', time=2024-06-28T17:53:30.053871+0000, chain.objs.size()=1 2024-06-28T16:14:32.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wYBholwsbwyqwlqMwxrRthEXTwz3m9I_0 2024-06-28T16:14:32.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7285921721214830845', time=2024-06-28T17:53:30.055808+0000, chain.objs.size()=1 2024-06-28T16:14:32.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Fh1p5jVy3c6nqgonh7rg7YgXMuvIXUQ_0 2024-06-28T16:14:32.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5954857416972223988', time=2024-06-28T17:53:30.289821+0000, chain.objs.size()=1 2024-06-28T16:14:32.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-M_pbDOcLLYgISkmKx_ABZRPdCDyvxr_0 2024-06-28T16:14:32.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12271947711307073214', time=2024-06-28T17:53:30.536118+0000, chain.objs.size()=1 2024-06-28T16:14:32.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.FaVSfjx6fH-JAnksEQAx5be37Vy-i6v_0 2024-06-28T16:14:32.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1781041284139623357', time=2024-06-28T17:53:31.157034+0000, chain.objs.size()=1 2024-06-28T16:14:32.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.l2eq-KhhJNaVGub9tIdzo9tuMbKzOaU_0 2024-06-28T16:14:32.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/44354 2024-06-28T16:14:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=34, truncated=0, next_marker='' 2024-06-28T16:14:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10700573695393113839', time=2024-06-28T17:53:31.903741+0000, chain.objs.size()=1 2024-06-28T16:14:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.tljKVKcGYLArPbp58qqm0Cg7Zb4HoQ3_0 2024-06-28T16:14:32.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14368528834881932992', time=2024-06-28T17:53:32.686330+0000, chain.objs.size()=1 2024-06-28T16:14:32.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.MaNN0HFXRBHxFNHmy6kpnXa5wqXysXm_0 2024-06-28T16:14:32.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12588562606649135476', time=2024-06-28T17:53:34.885980+0000, chain.objs.size()=1 2024-06-28T16:14:32.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vPL6seNdzqktOWg7Lm17pNZaDi1LjAo_0 2024-06-28T16:14:32.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8148332002189488470', time=2024-06-28T17:53:35.449391+0000, chain.objs.size()=1 2024-06-28T16:14:32.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.mjxKhMrq-BWwfHo4lej461I1behY2H9_0 2024-06-28T16:14:32.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14329769509743541612', time=2024-06-28T17:53:38.266423+0000, chain.objs.size()=1 2024-06-28T16:14:32.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.mKa8MdFVsy4bLNoSlYHbjmNN1W-IKs5_0 2024-06-28T16:14:32.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3631851488573291487', time=2024-06-28T17:53:39.267510+0000, chain.objs.size()=1 2024-06-28T16:14:32.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4fvAKB4kpJT1kjeme21K8r040BebA93_0 2024-06-28T16:14:32.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11661591635149391548', time=2024-06-28T17:53:40.696687+0000, chain.objs.size()=1 2024-06-28T16:14:32.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.2FS1d94UDLTxBpQq-vC5r9ggHsCG0mQ_0 2024-06-28T16:14:32.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17679221671491886454', time=2024-06-28T17:53:42.307819+0000, chain.objs.size()=1 2024-06-28T16:14:32.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Ns1Eij4FXCeqedn2Ane8dns_RouS8-M_0 2024-06-28T16:14:32.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.860557879296305784', time=2024-06-28T17:53:42.659786+0000, chain.objs.size()=1 2024-06-28T16:14:32.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.dk1a7v9-vw-_u0W7JIk78sCNF51Nz22_0 2024-06-28T16:14:32.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2324757182805948041', time=2024-06-28T17:53:42.929281+0000, chain.objs.size()=1 2024-06-28T16:14:32.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.V1MTSj-nLfdKf9f4pI1oS7JeuB61Vi7_0 2024-06-28T16:14:32.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10184549312554815559', time=2024-06-28T17:53:43.346457+0000, chain.objs.size()=1 2024-06-28T16:14:32.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vw2AK4g2_BOfqBTSS9PGHbYHCq3exax_0 2024-06-28T16:14:32.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10186549709226919507', time=2024-06-28T17:53:44.225058+0000, chain.objs.size()=1 2024-06-28T16:14:32.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.KZLKInQXhd5y41KTD0i3fPf1__EDsI__0 2024-06-28T16:14:32.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.403+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6672518610351713938', time=2024-06-28T17:53:45.019708+0000, chain.objs.size()=1 2024-06-28T16:14:32.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.403+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.c6GamagNmP2KcZCisQKeP5a7khqqRdI_0 2024-06-28T16:14:32.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.403+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6745056994475108731', time=2024-06-28T17:53:47.376558+0000, chain.objs.size()=1 2024-06-28T16:14:32.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.403+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.h3mLe91GqwJQ9cG3JEckYGYrbjX6Kcy_0 2024-06-28T16:14:32.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.403+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8754014305605968394', time=2024-06-28T17:53:47.730556+0000, chain.objs.size()=1 2024-06-28T16:14:32.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.403+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.hgruDgi7qZ-Rc1-TOYipOdMtTN2cpBO_0 2024-06-28T16:14:32.415 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14614622484686219191', time=2024-06-28T17:53:48.938222+0000, chain.objs.size()=1 2024-06-28T16:14:32.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.0xRoObydMCvwZwj1zuU4X9hxR3r5v4__0 2024-06-28T16:14:32.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9574991687803128782', time=2024-06-28T17:53:49.018814+0000, chain.objs.size()=1 2024-06-28T16:14:32.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.iuzq-WSxrKvNEzB9YyUxqlq2xzZp_BR_0 2024-06-28T16:14:32.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6310769197618588943', time=2024-06-28T17:53:49.713907+0000, chain.objs.size()=1 2024-06-28T16:14:32.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vHLxJQq6S335qER6ioml5OhiI7jw2hv_0 2024-06-28T16:14:32.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15698865250288446440', time=2024-06-28T17:53:50.232056+0000, chain.objs.size()=1 2024-06-28T16:14:32.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.s5PcHEGQoNTddlwXxk2y-nZRqMwDWQZ_0 2024-06-28T16:14:32.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1238823632114444019', time=2024-06-28T17:53:50.959980+0000, chain.objs.size()=1 2024-06-28T16:14:32.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZjiIC-NIi9bbs1lt8mzrPyEfCpdmjkG_0 2024-06-28T16:14:32.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10538830820379701987', time=2024-06-28T17:53:52.152225+0000, chain.objs.size()=1 2024-06-28T16:14:32.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.q--UVsNXCiXvqTixNil4oQq0HCootuU_0 2024-06-28T16:14:32.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17754577576209802113', time=2024-06-28T17:55:42.787495+0000, chain.objs.size()=1 2024-06-28T16:14:32.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.434__shadow_.3b6k_9uIapnVeu2b2biDG4bTOlNZGhG_0 2024-06-28T16:14:32.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1081019494124616110', time=2024-06-28T17:55:59.875364+0000, chain.objs.size()=1 2024-06-28T16:14:32.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.448__shadow_.RcJVOZ-mmGk57SC8NiATRLxFdgKlD7y_0 2024-06-28T16:14:32.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12195844650378999830', time=2024-06-28T17:56:01.600286+0000, chain.objs.size()=1 2024-06-28T16:14:32.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.449__shadow_.Bvc2EZHZURJD8hmbWi-47Y1U2fT1465_0 2024-06-28T16:14:32.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8669323903618435291', time=2024-06-28T17:56:11.991801+0000, chain.objs.size()=1 2024-06-28T16:14:32.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.455__shadow_.hHu0FmqQslQ59mLGG3ozclYjyFm8P-H_0 2024-06-28T16:14:32.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2661311707902843908', time=2024-06-28T17:56:36.859529+0000, chain.objs.size()=1 2024-06-28T16:14:32.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.477__shadow_.kojZS1O31-LCsDRsPT55FWpbanAp2LT_0 2024-06-28T16:14:32.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5626143922839808597', time=2024-06-28T18:01:32.169463+0000, chain.objs.size()=1 2024-06-28T16:14:32.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.546__shadow_.8vvns3Xtr8c55SMGboFzH0d0XNe38Kr_0 2024-06-28T16:14:32.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14569123431240726549', time=2024-06-28T18:02:50.124107+0000, chain.objs.size()=1 2024-06-28T16:14:32.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.565__shadow:skCQZSeDSmaqAc2WNXyrlTX5wriNwSG_.X8JRd8B4fX9DBAmF_aMVHSWmDHJKzMp_0 2024-06-28T16:14:32.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4813829188100510086', time=2024-06-28T18:03:21.945139+0000, chain.objs.size()=1 2024-06-28T16:14:32.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:pN-PrGIPzqZ2oCbordlpNOz9bBJb8tD_.c5gIPpM64YXfoDVtBp2pveZyHLnHT3L_0 2024-06-28T16:14:32.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11996142684058567477', time=2024-06-28T18:03:48.745538+0000, chain.objs.size()=4 2024-06-28T16:14:32.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.567__multipart_testobj.2~mGKFRFuS0VUAMIhFUPvDsVlKztkzThY.1 2024-06-28T16:14:32.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.567__multipart_testobj.2~mGKFRFuS0VUAMIhFUPvDsVlKztkzThY.2 2024-06-28T16:14:32.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.567__multipart_testobj.2~mGKFRFuS0VUAMIhFUPvDsVlKztkzThY.3 2024-06-28T16:14:32.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.567__multipart_testobj.2~mGKFRFuS0VUAMIhFUPvDsVlKztkzThY.4 2024-06-28T16:14:32.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7696474900678255999', time=2024-06-28T18:04:21.530191+0000, chain.objs.size()=1 2024-06-28T16:14:32.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.585__shadow_.zygDWtQPyDtM0S1kpariWdmO5MtvYwv_0 2024-06-28T16:14:32.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.415+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~TCgDd9pZgyQkv1eHy4QrZO1fY4JR-Jz', time=2024-06-28T18:06:47.823198+0000, chain.objs.size()=6 2024-06-28T16:14:32.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.415+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.612__multipart_multipart_enc.2~TCgDd9pZgyQkv1eHy4QrZO1fY4JR-Jz.1 2024-06-28T16:14:32.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.415+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.612__multipart_multipart_enc.2~TCgDd9pZgyQkv1eHy4QrZO1fY4JR-Jz.2 2024-06-28T16:14:32.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.612__multipart_multipart_enc.2~TCgDd9pZgyQkv1eHy4QrZO1fY4JR-Jz.3 2024-06-28T16:14:32.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.612__multipart_multipart_enc.2~TCgDd9pZgyQkv1eHy4QrZO1fY4JR-Jz.4 2024-06-28T16:14:32.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.612__multipart_multipart_enc.2~TCgDd9pZgyQkv1eHy4QrZO1fY4JR-Jz.5 2024-06-28T16:14:32.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.612__multipart_multipart_enc.2~TCgDd9pZgyQkv1eHy4QrZO1fY4JR-Jz.6 2024-06-28T16:14:32.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.419+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15977202225668699944', time=2024-06-28T18:07:56.935121+0000, chain.objs.size()=6 2024-06-28T16:14:32.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.646__multipart_baz.2~f1uBLvoX7ifFP93NiA0L6QuFILWYjkg.1 2024-06-28T16:14:32.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.646__multipart_baz.2~f1uBLvoX7ifFP93NiA0L6QuFILWYjkg.2 2024-06-28T16:14:32.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.646__multipart_baz.2~f1uBLvoX7ifFP93NiA0L6QuFILWYjkg.3 2024-06-28T16:14:32.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.646__multipart_baz.2~f1uBLvoX7ifFP93NiA0L6QuFILWYjkg.4 2024-06-28T16:14:32.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.646__multipart_baz.2~f1uBLvoX7ifFP93NiA0L6QuFILWYjkg.5 2024-06-28T16:14:32.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.423+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.646__multipart_baz.2~f1uBLvoX7ifFP93NiA0L6QuFILWYjkg.6 2024-06-28T16:14:32.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.423+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12901809731650571976', time=2024-06-28T18:07:59.206193+0000, chain.objs.size()=1 2024-06-28T16:14:32.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.423+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.648__shadow_.zDR6uAdxMBHKsmIim2d6_8iijR7LYw5_0 2024-06-28T16:14:32.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:32.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-06-28T16:14:32.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:32.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:32.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/51205' 2024-06-28T16:14:32.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6004109027742010479', time=2024-06-28T17:40:25.481806+0000, chain.objs.size()=3 2024-06-28T16:14:32.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.151__multipart_foo-STANDARD.2~tR0QIWhnAkeKfIrFwgng34m97-b5tBJ.1 2024-06-28T16:14:32.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.151__multipart_foo-STANDARD.2~tR0QIWhnAkeKfIrFwgng34m97-b5tBJ.2 2024-06-28T16:14:32.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.151__multipart_foo-STANDARD.2~tR0QIWhnAkeKfIrFwgng34m97-b5tBJ.3 2024-06-28T16:14:32.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10369708418691660718', time=2024-06-28T17:40:47.165993+0000, chain.objs.size()=1 2024-06-28T16:14:32.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.160__shadow_.7eTofdNdDR3t17uaf_xrJRfCJyGqF1P_0 2024-06-28T16:14:32.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5698087380617674282', time=2024-06-28T17:44:43.796608+0000, chain.objs.size()=1 2024-06-28T16:14:32.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.226__shadow_.24IG9hsfYzfY62h8dUF8ilH_tGEAjdL_0 2024-06-28T16:14:32.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3447454848177333490', time=2024-06-28T17:45:01.759749+0000, chain.objs.size()=1 2024-06-28T16:14:32.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.236__shadow_.RB9apKDlnRIySuFSvTMjQo3DuZNaSjY_0 2024-06-28T16:14:32.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13967314696799083830', time=2024-06-28T17:45:08.266038+0000, chain.objs.size()=1 2024-06-28T16:14:32.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.240__shadow_.OcllT2_8XmwjJVr6yBG8gj2LUgS51Xj_0 2024-06-28T16:14:32.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9156617535029420559', time=2024-06-28T17:47:10.291410+0000, chain.objs.size()=1 2024-06-28T16:14:32.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.1uTmkM-oD2qDxBCarYQ4QgRpwSwnUxg_0 2024-06-28T16:14:32.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1782673972899128315', time=2024-06-28T17:47:10.492874+0000, chain.objs.size()=1 2024-06-28T16:14:32.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.G8DT14nSJyUW4T7aP5CQLBYLKPyB_Uv_0 2024-06-28T16:14:32.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17074335964040993221', time=2024-06-28T17:47:10.494262+0000, chain.objs.size()=1 2024-06-28T16:14:32.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.o1nrSrnygApBrSvol0JGYk6UTyhXibC_0 2024-06-28T16:14:32.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3606381521161519424', time=2024-06-28T17:47:12.361365+0000, chain.objs.size()=1 2024-06-28T16:14:32.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.okKbRF6QtbM5FNj1rBfGtV4NEX8RZGf_0 2024-06-28T16:14:32.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1268700480911085372', time=2024-06-28T17:47:12.588595+0000, chain.objs.size()=1 2024-06-28T16:14:32.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VEmP8l3BFAu2Yshokpf4SzjHdUSpY4M_0 2024-06-28T16:14:32.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18015823190341212126', time=2024-06-28T17:47:13.484797+0000, chain.objs.size()=1 2024-06-28T16:14:32.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ng9g1S93aejJiiRDjwlPv8LPLz2pezW_0 2024-06-28T16:14:32.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6962702523239234407', time=2024-06-28T17:47:14.792660+0000, chain.objs.size()=1 2024-06-28T16:14:32.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.jgrh-CQp8qoA2SEhJMATccO9vbgKJej_0 2024-06-28T16:14:32.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4942592638807629405', time=2024-06-28T17:47:15.561549+0000, chain.objs.size()=1 2024-06-28T16:14:32.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.lFN_tMvgVqAbkX_VoTeNFO8VBKDRVnF_0 2024-06-28T16:14:32.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4567035715143550734', time=2024-06-28T17:47:15.622299+0000, chain.objs.size()=1 2024-06-28T16:14:32.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.NDLaHTaQdfZuOJ0EphGple7BcAqpydN_0 2024-06-28T16:14:32.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6280096549579028962', time=2024-06-28T17:47:18.111934+0000, chain.objs.size()=1 2024-06-28T16:14:32.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.IJHsVGTBRt1LhI4GTgsbgg8Iv3fcbTn_0 2024-06-28T16:14:32.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3734934948471818990', time=2024-06-28T17:47:22.023752+0000, chain.objs.size()=1 2024-06-28T16:14:32.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-4TgMYycS84SMQ5ExginnnN7QH4o33y_0 2024-06-28T16:14:32.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2096143024892412353', time=2024-06-28T17:47:22.912419+0000, chain.objs.size()=1 2024-06-28T16:14:32.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.sAAjMbjSEtl9E58sIq-KFZMaNi4Ro8p_0 2024-06-28T16:14:32.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14284794547341152679', time=2024-06-28T17:47:23.763744+0000, chain.objs.size()=1 2024-06-28T16:14:32.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.60GWbmP9pLj-F4vE41_jLqCHHZfU-l-_0 2024-06-28T16:14:32.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3249033668307124379', time=2024-06-28T17:47:23.765565+0000, chain.objs.size()=1 2024-06-28T16:14:32.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.54JiIFb7500Rzx1jXMJ4jNNRLIW6v1n_0 2024-06-28T16:14:32.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8505571378353484320', time=2024-06-28T17:47:31.252827+0000, chain.objs.size()=1 2024-06-28T16:14:32.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.r9IQhbbN_wSy5xjCUx1SD8yM7EC-iKR_0 2024-06-28T16:14:32.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8577571231640746238', time=2024-06-28T17:47:33.062546+0000, chain.objs.size()=1 2024-06-28T16:14:32.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.jKXGJs-cJ-qEOuGc25D3vpN0bj_Q7kp_0 2024-06-28T16:14:32.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.443+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12188352603270651831', time=2024-06-28T17:47:34.090329+0000, chain.objs.size()=1 2024-06-28T16:14:32.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.443+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.PkRrEkdTMRPSHEPi_iiJEFaIqE48BKi_0 2024-06-28T16:14:32.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.443+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15303366478501566216', time=2024-06-28T17:47:35.435019+0000, chain.objs.size()=1 2024-06-28T16:14:32.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.443+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_._XtsJxS8hKy_UVnIvN0q8rVbRYHFiNP_0 2024-06-28T16:14:32.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.443+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13134207358009938801', time=2024-06-28T17:47:37.074638+0000, chain.objs.size()=1 2024-06-28T16:14:32.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.443+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dPb3ops2j-qffFwGCVfqZLEWmkiNk_r_0 2024-06-28T16:14:32.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.443+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17573827103818195892', time=2024-06-28T17:47:37.076695+0000, chain.objs.size()=1 2024-06-28T16:14:32.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.443+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.XdOxHw1stpmlfcAVFuX258F_s7S1J73_0 2024-06-28T16:14:32.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.443+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5134671661424602050', time=2024-06-28T17:47:38.520376+0000, chain.objs.size()=1 2024-06-28T16:14:32.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.443+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.8mdXa--4XvfiQ-JSsgLtmX0DaUBQq7r_0 2024-06-28T16:14:32.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.443+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2056500679780203750', time=2024-06-28T17:47:38.728693+0000, chain.objs.size()=1 2024-06-28T16:14:32.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.443+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.j5-eiOosKxBUkbvijpy6Dxgtd3AMnrQ_0 2024-06-28T16:14:32.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.443+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3363358330997609790', time=2024-06-28T17:48:00.524676+0000, chain.objs.size()=1 2024-06-28T16:14:32.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.443+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.266__shadow_.kESvOBurjR1gC9tKpwl1aAD55BxyFu8_0 2024-06-28T16:14:32.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7221237079111097835', time=2024-06-28T17:48:08.535391+0000, chain.objs.size()=1 2024-06-28T16:14:32.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.271__shadow_.xeJRNNPyelfu2JKfe42d67hiu3skzcV_0 2024-06-28T16:14:32.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16245742471486814572', time=2024-06-28T17:48:34.641598+0000, chain.objs.size()=1 2024-06-28T16:14:32.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.IMbKoIxD7rjpUMLyDop4Aen3C2axng1_0 2024-06-28T16:14:32.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2293536118985670026', time=2024-06-28T17:48:58.969831+0000, chain.objs.size()=1 2024-06-28T16:14:32.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.297__shadow:SFP7lPhejzEc9ALK7auxxTx3cRNui9G_.sYl37IE3awvw8SN4s5_S7564CggclJw_0 2024-06-28T16:14:32.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.958817544219502224', time=2024-06-28T17:51:08.758441+0000, chain.objs.size()=1 2024-06-28T16:14:32.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ZJRbEKQO2_5weRdp53-ywsK85MgfxIT_0 2024-06-28T16:14:32.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14243469362891360209', time=2024-06-28T17:51:09.914409+0000, chain.objs.size()=1 2024-06-28T16:14:32.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.D_G2Zb9b7_fa9Zybey8sOUyvOGUL89O_0 2024-06-28T16:14:32.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4835487479648000857', time=2024-06-28T17:51:13.535750+0000, chain.objs.size()=1 2024-06-28T16:14:32.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.lgsuG4uZeuXVPXS0JKtZW1g4Z3ULx2F_0 2024-06-28T16:14:32.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13063904706543290414', time=2024-06-28T17:51:15.921127+0000, chain.objs.size()=1 2024-06-28T16:14:32.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.jvRe9XdVbgDxo6cS3ZrqYPzn6Nm5ri__0 2024-06-28T16:14:32.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8985436380491982326', time=2024-06-28T17:51:17.680624+0000, chain.objs.size()=1 2024-06-28T16:14:32.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fqyv7QnCJ1UnhnW7AAM34_PCG15A3Zc_0 2024-06-28T16:14:32.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3160658662352131706', time=2024-06-28T17:51:18.251407+0000, chain.objs.size()=1 2024-06-28T16:14:32.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.3fkQrM6Z5QudRKajA6KEvaRSsPZEf4a_0 2024-06-28T16:14:32.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.446811188416530359', time=2024-06-28T17:51:18.253379+0000, chain.objs.size()=1 2024-06-28T16:14:32.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.JAF3muqSlgzzliU5zTZ72bwVP6VIubq_0 2024-06-28T16:14:32.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1305100555609158339', time=2024-06-28T17:51:19.454013+0000, chain.objs.size()=1 2024-06-28T16:14:32.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.QjsVtDXljadL0IlFk1Cq6kJ_C9bjCR-_0 2024-06-28T16:14:32.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3582602124088962028', time=2024-06-28T17:51:20.023969+0000, chain.objs.size()=1 2024-06-28T16:14:32.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.VXWTAZPW3TFgJn7m63iiC-xbsncNRk4_0 2024-06-28T16:14:32.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2357418373315094604', time=2024-06-28T17:51:21.475339+0000, chain.objs.size()=1 2024-06-28T16:14:32.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.WFSPDvaxZn2hRkvXkcZ4iV3LWJH1VNn_0 2024-06-28T16:14:32.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5445676632742954480', time=2024-06-28T17:51:22.201910+0000, chain.objs.size()=1 2024-06-28T16:14:32.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.w7qSDrN2MUidGhGy0J92uU2bpaz4HlA_0 2024-06-28T16:14:32.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9684548186517501289', time=2024-06-28T17:51:22.338474+0000, chain.objs.size()=1 2024-06-28T16:14:32.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yNQ5rbaHvxHkLsWNUbwV8xW8ML_YmUb_0 2024-06-28T16:14:32.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7881915456076987691', time=2024-06-28T17:51:22.454977+0000, chain.objs.size()=1 2024-06-28T16:14:32.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YHtB5TPW5aNLYR26nVYcahU1YiO3WDo_0 2024-06-28T16:14:32.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18335209660505586513', time=2024-06-28T17:51:22.556668+0000, chain.objs.size()=1 2024-06-28T16:14:32.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hTO0rIhrt5Z0rXCi0bt88J9bhuzNjq3_0 2024-06-28T16:14:32.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10570578584456100494', time=2024-06-28T17:51:24.324209+0000, chain.objs.size()=1 2024-06-28T16:14:32.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Zw0MBJEIvTbUubbyiQ7kFcIyh0NL001_0 2024-06-28T16:14:32.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7863854859153623580', time=2024-06-28T17:51:24.915293+0000, chain.objs.size()=1 2024-06-28T16:14:32.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1VwqX6d-3JRPBOBpOxmtBDCWafGVi1H_0 2024-06-28T16:14:32.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5150439202461627826', time=2024-06-28T17:51:25.807771+0000, chain.objs.size()=1 2024-06-28T16:14:32.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.FiuzcFvnZFGoMvuUUqYxCwjUEFpL-As_0 2024-06-28T16:14:32.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1431553744656309887', time=2024-06-28T17:51:25.809852+0000, chain.objs.size()=1 2024-06-28T16:14:32.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.0Hc9VQNDhFuS3OsfNfGy-fccX2LYoKd_0 2024-06-28T16:14:32.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.459+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16738430716378591439', time=2024-06-28T17:51:25.812237+0000, chain.objs.size()=1 2024-06-28T16:14:32.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.459+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Wfay8JtD9EIZQ1AX9F8EgXRO45a6Zao_0 2024-06-28T16:14:32.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.459+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13390067507012503801', time=2024-06-28T17:51:26.690924+0000, chain.objs.size()=1 2024-06-28T16:14:32.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.459+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.rrk60rp5E1cZSTr3jTcyo97vQ9OHqr8_0 2024-06-28T16:14:32.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.459+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6710449693957826413', time=2024-06-28T17:51:26.789805+0000, chain.objs.size()=1 2024-06-28T16:14:32.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.459+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xcINlbVYFIfSVEiwc7YXL3XYNbrB_Fq_0 2024-06-28T16:14:32.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13761749897474025063', time=2024-06-28T17:51:28.197207+0000, chain.objs.size()=1 2024-06-28T16:14:32.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.K2W4GWbfCg1vxGq_Q_fY4BssbENP3jO_0 2024-06-28T16:14:32.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8530331596865489465', time=2024-06-28T17:51:28.781403+0000, chain.objs.size()=1 2024-06-28T16:14:32.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.62IRqOCl4Wq-KzKEPP4uRi-4g3hIGF6_0 2024-06-28T16:14:32.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4581655582545898861', time=2024-06-28T17:51:29.717478+0000, chain.objs.size()=1 2024-06-28T16:14:32.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.J7NXTVTpL3FPZ2wr_yo89Fd9q7KlZ0q_0 2024-06-28T16:14:32.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13117568283979035705', time=2024-06-28T17:51:29.821493+0000, chain.objs.size()=1 2024-06-28T16:14:32.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.gStBfp_l7XBLS5K89C39a5fZTmKTMjZ_0 2024-06-28T16:14:32.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3136676775314360487', time=2024-06-28T17:51:30.045857+0000, chain.objs.size()=1 2024-06-28T16:14:32.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.rHHCro5JZLYfxFkCZXo-S_1oz55niHo_0 2024-06-28T16:14:32.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16243875024420272207', time=2024-06-28T17:51:30.329055+0000, chain.objs.size()=1 2024-06-28T16:14:32.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.5zerfd4RgTTpTCMwmevYTKOofRSrGz__0 2024-06-28T16:14:32.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18056018576950114525', time=2024-06-28T17:51:30.722696+0000, chain.objs.size()=1 2024-06-28T16:14:32.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.b8oFee23TnH7-5v-ykcwViJYIeApsH0_0 2024-06-28T16:14:32.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5090529450305426516', time=2024-06-28T17:51:30.724566+0000, chain.objs.size()=1 2024-06-28T16:14:32.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.aC74KzvSxAdII4hOUXK1RYCPvCRt6Vl_0 2024-06-28T16:14:32.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11084125913569230593', time=2024-06-28T17:51:32.279726+0000, chain.objs.size()=1 2024-06-28T16:14:32.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.QZY2ir7680bwbPBxLEF9HUtUPjzGzV-_0 2024-06-28T16:14:32.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12756800650470560497', time=2024-06-28T17:51:33.523569+0000, chain.objs.size()=1 2024-06-28T16:14:32.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.zHeXDmnhn2GMc6UzzxBqwljrvzYLaQj_0 2024-06-28T16:14:32.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14810828343891470855', time=2024-06-28T17:51:33.680559+0000, chain.objs.size()=1 2024-06-28T16:14:32.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xLvjzXAtu6OTTBVvEUEOXZdUiBWzz2t_0 2024-06-28T16:14:32.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14010753590782365987', time=2024-06-28T17:51:34.823028+0000, chain.objs.size()=1 2024-06-28T16:14:32.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.V3Y-GUbHMccixPqeGHqrL-JsQaLBMeV_0 2024-06-28T16:14:32.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6601999645459418961', time=2024-06-28T17:51:34.944242+0000, chain.objs.size()=1 2024-06-28T16:14:32.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.4oX0Fv1C5Rfgcjd1nb390VOjz1E8X8P_0 2024-06-28T16:14:32.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5853594397502806327', time=2024-06-28T17:51:36.385437+0000, chain.objs.size()=1 2024-06-28T16:14:32.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yktb9Aou36COMU1CCn7PGMsvNtEvbLk_0 2024-06-28T16:14:32.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8711421902613797930', time=2024-06-28T17:51:37.475626+0000, chain.objs.size()=1 2024-06-28T16:14:32.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.gTHMqUUxRN9HbFPrsZzp5MtFLOXjVv4_0 2024-06-28T16:14:32.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7285205383783674123', time=2024-06-28T17:51:37.649929+0000, chain.objs.size()=1 2024-06-28T16:14:32.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.0cSbZJkE1d-u8lEUl6i6wZ7AzEyZzfj_0 2024-06-28T16:14:32.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6677233303714030789', time=2024-06-28T17:51:37.784396+0000, chain.objs.size()=1 2024-06-28T16:14:32.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.UahOAR4vGCKTyq4UQGgr06OpGbWjxGm_0 2024-06-28T16:14:32.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.202377464119994517', time=2024-06-28T17:53:21.853597+0000, chain.objs.size()=1 2024-06-28T16:14:32.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.xTFn6vrSpWJcBAGsA98hrqEmzX8nmZ0_0 2024-06-28T16:14:32.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5118329919568935875', time=2024-06-28T17:53:22.388671+0000, chain.objs.size()=1 2024-06-28T16:14:32.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.40fM3aAVPuoETml6L__mX-BsZN6pBc8_0 2024-06-28T16:14:32.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12504561251657002371', time=2024-06-28T17:53:23.827983+0000, chain.objs.size()=1 2024-06-28T16:14:32.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CIe-1p0FA4LSJnqbMI0vXA1ELN12CbQ_0 2024-06-28T16:14:32.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3294378992932689016', time=2024-06-28T17:53:24.127166+0000, chain.objs.size()=1 2024-06-28T16:14:32.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Oj223O_IfXfyoGdLIJujjW0iMcUpSHm_0 2024-06-28T16:14:32.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10142049271409644978', time=2024-06-28T17:53:27.688694+0000, chain.objs.size()=1 2024-06-28T16:14:32.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.PuXSmiNt8PG95te0Whs1y35N8h9jkrg_0 2024-06-28T16:14:32.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.471+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15180921113715072074', time=2024-06-28T17:53:30.372290+0000, chain.objs.size()=1 2024-06-28T16:14:32.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.471+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.SQ8yaBIFerQ0hgPAEIqKE48xxJ-KVhO_0 2024-06-28T16:14:32.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.471+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13796615114943229940', time=2024-06-28T17:53:30.531627+0000, chain.objs.size()=1 2024-06-28T16:14:32.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.471+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.NeF6bpXZBxzRGCyjx0YbhHvD4r0Wkwu_0 2024-06-28T16:14:32.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13723098207205855302', time=2024-06-28T17:53:30.533806+0000, chain.objs.size()=1 2024-06-28T16:14:32.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.INtlz8N3gypCmK_AfS2y9Sp0zuTGN57_0 2024-06-28T16:14:32.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3541158019654084230', time=2024-06-28T17:53:30.989581+0000, chain.objs.size()=1 2024-06-28T16:14:32.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wK2_jWhDby_ePM-tUHaDeNpjcW7aQZX_0 2024-06-28T16:14:32.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8378705605076767441', time=2024-06-28T17:53:32.018720+0000, chain.objs.size()=1 2024-06-28T16:14:32.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wEREOFwkI7n30eMHOfHOEE-0zCxCgi7_0 2024-06-28T16:14:32.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13833800378084483262', time=2024-06-28T17:53:34.414960+0000, chain.objs.size()=1 2024-06-28T16:14:32.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.hQA-IeNBw3OFw3kz6bwOgjeW3xbuyqX_0 2024-06-28T16:14:32.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14784531679000262400', time=2024-06-28T17:53:35.643138+0000, chain.objs.size()=1 2024-06-28T16:14:32.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vOzVRiyAy3hJwuZ4lWdCbJUjd-eVfAv_0 2024-06-28T16:14:32.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7623676699065037742', time=2024-06-28T17:53:36.559686+0000, chain.objs.size()=1 2024-06-28T16:14:32.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.UaJVxqGpmYdHyGsR5WqYjA_SBi65Asm_0 2024-06-28T16:14:32.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15916429582060427380', time=2024-06-28T17:53:36.715574+0000, chain.objs.size()=1 2024-06-28T16:14:32.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ry2kJDGD7PwirHjr0hHB84KLd9sbE2__0 2024-06-28T16:14:32.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18025654838260733588', time=2024-06-28T17:53:38.147803+0000, chain.objs.size()=1 2024-06-28T16:14:32.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.fJgwhkbBtK8lsE2aj6FCI82ioa78qTT_0 2024-06-28T16:14:32.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10510739366952277969', time=2024-06-28T17:53:38.665165+0000, chain.objs.size()=1 2024-06-28T16:14:32.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.sJeY0rjlARWZEOuGoj2me43CIP4AcDu_0 2024-06-28T16:14:32.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4224953944790368729', time=2024-06-28T17:53:38.846825+0000, chain.objs.size()=1 2024-06-28T16:14:32.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.l5ndXS1n3P9Q2ga6dplYB1_x72Iqblu_0 2024-06-28T16:14:32.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4531038219470432571', time=2024-06-28T17:53:39.908018+0000, chain.objs.size()=1 2024-06-28T16:14:32.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.dvJK5ANSiWba4K-s3oPdwag4FkHQwAu_0 2024-06-28T16:14:32.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.479+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.247437882007926485', time=2024-06-28T17:53:40.238720+0000, chain.objs.size()=1 2024-06-28T16:14:32.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.479+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.q5LYr5bqgM_bFzqiiZ4SJCTM2BsYjap_0 2024-06-28T16:14:32.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.479+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14859718472352488409', time=2024-06-28T17:53:40.696496+0000, chain.objs.size()=1 2024-06-28T16:14:32.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.479+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.hNfZs59XhpBjky259K_jjfKhjrHCzOL_0 2024-06-28T16:14:32.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.479+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7983624842105580020', time=2024-06-28T17:53:40.808536+0000, chain.objs.size()=1 2024-06-28T16:14:32.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.479+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GLZrhaMoeVolcRo9JwEEDRAzfAa5jbx_0 2024-06-28T16:14:32.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.479+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7819058334711374870', time=2024-06-28T17:53:41.956805+0000, chain.objs.size()=1 2024-06-28T16:14:32.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.479+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.J8Ebm9DaLDUSkvsnCjPxYNmi_0kJTwO_0 2024-06-28T16:14:32.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.479+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.862793455185519121', time=2024-06-28T17:53:41.958253+0000, chain.objs.size()=1 2024-06-28T16:14:32.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.479+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.rAGD3_eDpn426QFaKshArARiNtGYfdt_0 2024-06-28T16:14:32.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.479+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2727132723493854075', time=2024-06-28T17:53:43.344866+0000, chain.objs.size()=1 2024-06-28T16:14:32.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.479+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.aZsIq4kEjGntDEn0U_5q7Hwmq7AboJD_0 2024-06-28T16:14:32.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1073246584022005247', time=2024-06-28T17:53:43.812098+0000, chain.objs.size()=1 2024-06-28T16:14:32.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.oL9kZp_L7bhs37Z8AkaWKh-YVfLl6A8_0 2024-06-28T16:14:32.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16951733155258787384', time=2024-06-28T17:53:44.229557+0000, chain.objs.size()=1 2024-06-28T16:14:32.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.PBWkUAoPOLR8c-37GSXa7A6OX53_VL-_0 2024-06-28T16:14:32.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13319839066719009735', time=2024-06-28T17:53:45.021146+0000, chain.objs.size()=1 2024-06-28T16:14:32.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.yazIjhl-Jtu7ffMj-ObT1zrgFuhWheR_0 2024-06-28T16:14:32.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15019634945475532891', time=2024-06-28T17:53:47.012304+0000, chain.objs.size()=1 2024-06-28T16:14:32.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.chXeBr6v2L9xSArdinIX4zKW99V_Ck9_0 2024-06-28T16:14:32.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8212802830558102182', time=2024-06-28T17:53:48.939093+0000, chain.objs.size()=1 2024-06-28T16:14:32.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.cCTj0CxL5MXq8SmNgTDPVgBjqUN2DyF_0 2024-06-28T16:14:32.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13155413553467046300', time=2024-06-28T17:53:49.712043+0000, chain.objs.size()=1 2024-06-28T16:14:32.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.dgp_vC1Jbfw_QBezV7vtOw6aZ1Vewh6_0 2024-06-28T16:14:32.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14107217077736553773', time=2024-06-28T17:53:49.714764+0000, chain.objs.size()=1 2024-06-28T16:14:32.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.zFT9PzJzAp2uprPa9DGFk6e35pS0KRc_0 2024-06-28T16:14:32.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/51205 2024-06-28T16:14:32.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=16, truncated=0, next_marker='' 2024-06-28T16:14:32.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3890688792616534302', time=2024-06-28T17:53:50.717601+0000, chain.objs.size()=1 2024-06-28T16:14:32.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZJWCfQ51ZssqGlvPTYFoTtJFYWVVEPM_0 2024-06-28T16:14:32.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15081102956173038632', time=2024-06-28T17:53:50.719257+0000, chain.objs.size()=1 2024-06-28T16:14:32.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qYB74dCpBbYVzOq1Fqszdx7iTNr8y4Q_0 2024-06-28T16:14:32.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13179482556008193917', time=2024-06-28T17:53:52.166596+0000, chain.objs.size()=1 2024-06-28T16:14:32.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.S1oUgfSOfkXWxR8QtY16cR3aiKlsWOg_0 2024-06-28T16:14:32.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17953145971531065280', time=2024-06-28T17:54:32.536047+0000, chain.objs.size()=1 2024-06-28T16:14:32.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.362__shadow_.DVQQ_hU0Cyl6bAKZ5ThWtULIFFP2V9F_0 2024-06-28T16:14:32.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13169244828637996067', time=2024-06-28T17:55:08.438485+0000, chain.objs.size()=1 2024-06-28T16:14:32.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.397__shadow_.CLF-CD2TJLB2WyxnqOZirQFdRhjzPjq_0 2024-06-28T16:14:32.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2107469562909854316', time=2024-06-28T17:55:41.473357+0000, chain.objs.size()=1 2024-06-28T16:14:32.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.433__shadow_.YY02XsEwc7277Ix0Mg6NaLkUW2Aqqbj_0 2024-06-28T16:14:32.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.486__shadow_.dZl9bSO4f54rrBWnT7sho6L_3EycXVD_1_90e69b6e3cf6c306db7954a69f016b9c', time=2024-06-28T17:56:45.727236+0000, chain.objs.size()=1 2024-06-28T16:14:32.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.486__shadow_.dZl9bSO4f54rrBWnT7sho6L_3EycXVD_0 2024-06-28T16:14:32.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10573150544816305363', time=2024-06-28T18:00:01.859771+0000, chain.objs.size()=1 2024-06-28T16:14:32.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.514__shadow_.hWr_EneibTotOc5L14MB9qPfSM3ld7V_0 2024-06-28T16:14:32.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13044421062002160464', time=2024-06-28T18:01:17.191277+0000, chain.objs.size()=1 2024-06-28T16:14:32.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.542__shadow_.dD3NnHXJJXaiaz4OecuO1LAcbgVBKJd_0 2024-06-28T16:14:32.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13389385634816117276', time=2024-06-28T18:02:08.389208+0000, chain.objs.size()=1 2024-06-28T16:14:32.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:LzK6uKzmnWGY4ucJDgOxjVmsMpES.JJ_.4UkbWll0VMWkLaNW6tpQgth52B0shPS_0 2024-06-28T16:14:32.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15301696792604256974', time=2024-06-28T18:03:17.738431+0000, chain.objs.size()=1 2024-06-28T16:14:32.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:ndb-6ruljsWMwdHq8YWGOb8Mms.Eg5i_.U1Q-PIQHjiJzO-N-313x8kI8hNk2i8i_0 2024-06-28T16:14:32.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16449127729651946518', time=2024-06-28T18:03:55.857872+0000, chain.objs.size()=1 2024-06-28T16:14:32.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.568__shadow:fDjOCMBKSX7kGwlGRzLOJuXnvIOX6nQ_.vN6cCvYqqEUkFEQYxXKlj4vrVTk8oVN_0 2024-06-28T16:14:32.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2612843385038336867', time=2024-06-28T18:06:48.655558+0000, chain.objs.size()=1 2024-06-28T16:14:32.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.613__shadow_.XnLdXGEtBAJN9_KW76hOkOqrIH8xWCH_0 2024-06-28T16:14:32.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7070687442466992524', time=2024-06-28T18:06:55.868709+0000, chain.objs.size()=1 2024-06-28T16:14:32.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.620__shadow_.d1qRd-whbHeAUZRBQyO0qpwukJEEM2__0 2024-06-28T16:14:32.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1204649362178540045', time=2024-06-28T18:07:15.166482+0000, chain.objs.size()=2 2024-06-28T16:14:32.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.632__shadow_.bSnZPAfa8wqxxpHO-pbV9-pbsM34O6u_0 2024-06-28T16:14:32.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.632__shadow_.bSnZPAfa8wqxxpHO-pbV9-pbsM34O6u_1 2024-06-28T16:14:32.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7016465809121012488', time=2024-06-28T18:08:03.013391+0000, chain.objs.size()=1 2024-06-28T16:14:32.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.650__shadow_.4FLVcF9A-Y_8aocrgB82J7-kEzOh43l_0 2024-06-28T16:14:32.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:32.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.503+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-06-28T16:14:32.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.503+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:32.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.503+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:32.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/47826' 2024-06-28T16:14:32.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2162795015783761288', time=2024-06-28T17:40:12.966515+0000, chain.objs.size()=3 2024-06-28T16:14:32.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.149__multipart_mymultipart-LUKEWARM.2~DhO2bJL12n6Hf44aAZzw7AgGebDnrJV.1 2024-06-28T16:14:32.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.149__multipart_mymultipart-LUKEWARM.2~DhO2bJL12n6Hf44aAZzw7AgGebDnrJV.2 2024-06-28T16:14:32.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.149__multipart_mymultipart-LUKEWARM.2~DhO2bJL12n6Hf44aAZzw7AgGebDnrJV.3 2024-06-28T16:14:32.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_hKNz05AV06C7yqSNPaRpMUt9q4UECeU', time=2024-06-28T17:40:18.171035+0000, chain.objs.size()=2 2024-06-28T16:14:32.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.Dr9lZPboHm1h_NdmazbSiCGPXV6n4Wj_0 2024-06-28T16:14:32.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.Dr9lZPboHm1h_NdmazbSiCGPXV6n4Wj_1 2024-06-28T16:14:32.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_J-SyglVCQ4Q8l1zhx_Ps6YB9TV1peo3', time=2024-06-28T17:40:21.847607+0000, chain.objs.size()=2 2024-06-28T16:14:32.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.ZBzOOFgt3w3ATpWH_9t592MW4EKSuAR_0 2024-06-28T16:14:32.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.ZBzOOFgt3w3ATpWH_9t592MW4EKSuAR_1 2024-06-28T16:14:32.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12569779214760752472', time=2024-06-28T17:41:13.887136+0000, chain.objs.size()=1 2024-06-28T16:14:32.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.173__shadow_.8Ln7DMSRRDdYpzougBcPUE5sPULojHN_0 2024-06-28T16:14:32.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9734849901386383827', time=2024-06-28T17:45:03.343973+0000, chain.objs.size()=1 2024-06-28T16:14:32.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.237__shadow_.sdkwY7LP_z3iH5NvUxtMwtEoWZdqLk__0 2024-06-28T16:14:32.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4334102158571099617', time=2024-06-28T17:45:06.611047+0000, chain.objs.size()=1 2024-06-28T16:14:32.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.239__shadow_.orf2uV40zxwgFzW7_s7HQqy4_XsUM2r_0 2024-06-28T16:14:32.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3534097733954330046', time=2024-06-28T17:45:11.450697+0000, chain.objs.size()=1 2024-06-28T16:14:32.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.242__shadow_.wHrq4a3KdVf25CeV9GszcwOPIYGGdpx_0 2024-06-28T16:14:32.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6659340104450994834', time=2024-06-28T17:45:13.054480+0000, chain.objs.size()=1 2024-06-28T16:14:32.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.243__shadow_.XQI8ucBWcfCfCwrEeZWH5udjxkP_XXM_0 2024-06-28T16:14:32.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7558547736970044967', time=2024-06-28T17:45:17.882743+0000, chain.objs.size()=1 2024-06-28T16:14:32.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.246__shadow_.ElPUXV1AvUdFP7iAjSeLlM6HYDi9av4_0 2024-06-28T16:14:32.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5708426128556698511', time=2024-06-28T17:45:23.892777+0000, chain.objs.size()=1 2024-06-28T16:14:32.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.250__shadow_.hSBlJTzcIme_JjDOtF8DeP6l03p6TvI_0 2024-06-28T16:14:32.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16191953759199672971', time=2024-06-28T17:47:08.725702+0000, chain.objs.size()=1 2024-06-28T16:14:32.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.QUu9sADp-F8N5_wlod72TYzxQ64kSNl_0 2024-06-28T16:14:32.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10837362396096106945', time=2024-06-28T17:47:09.458002+0000, chain.objs.size()=1 2024-06-28T16:14:32.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.8hMjqpGH_0DPKNW9PP9GXw7lSzujovP_0 2024-06-28T16:14:32.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6460598307058153729', time=2024-06-28T17:47:09.763848+0000, chain.objs.size()=1 2024-06-28T16:14:32.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.NHqbNjLwW_eMwKOaejYS8X23UKZB4Wb_0 2024-06-28T16:14:32.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15658422742212415519', time=2024-06-28T17:47:10.943799+0000, chain.objs.size()=1 2024-06-28T16:14:32.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.m8lU6V1EnGjhkLVp2ynSKrYljK-Znxp_0 2024-06-28T16:14:32.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11837595236031893166', time=2024-06-28T17:47:11.136256+0000, chain.objs.size()=1 2024-06-28T16:14:32.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.OjNuHBdkO34-HIRicON0b_dlcC3OOp0_0 2024-06-28T16:14:32.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14243152488462000988', time=2024-06-28T17:47:15.614061+0000, chain.objs.size()=1 2024-06-28T16:14:32.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Z0OcSJevu2-sasEp8Ql3u4pSbX-tsBT_0 2024-06-28T16:14:32.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10444666494655623609', time=2024-06-28T17:47:16.588744+0000, chain.objs.size()=1 2024-06-28T16:14:32.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.j4oDuon7GrOn9G11JaGwU_OjeJnvhIF_0 2024-06-28T16:14:32.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16334207416908279338', time=2024-06-28T17:47:17.547851+0000, chain.objs.size()=1 2024-06-28T16:14:32.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.XQ212ni2I-GQ9wg0-DYE3quyJHCd3L2_0 2024-06-28T16:14:32.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18333798507892829841', time=2024-06-28T17:47:18.970556+0000, chain.objs.size()=1 2024-06-28T16:14:32.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.JhiPpYJK4CQlqf9x9WHFNUtJ0wQU8-U_0 2024-06-28T16:14:32.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8765389621948699799', time=2024-06-28T17:47:19.775285+0000, chain.objs.size()=1 2024-06-28T16:14:32.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.p6rMM1zx0-BWflCEgJ46OQ3svaBtrlJ_0 2024-06-28T16:14:32.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3787207265591503748', time=2024-06-28T17:47:22.723576+0000, chain.objs.size()=1 2024-06-28T16:14:32.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.JTWpG57UYTSX8AvAI8J2haA1srr5xAN_0 2024-06-28T16:14:32.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15575185568814014717', time=2024-06-28T17:47:22.865301+0000, chain.objs.size()=1 2024-06-28T16:14:32.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ozdmwo8nnYE4gSzjWeWM2vwKC7Bp816_0 2024-06-28T16:14:32.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18417753010867579845', time=2024-06-28T17:47:22.866821+0000, chain.objs.size()=1 2024-06-28T16:14:32.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.iyQ4qhxu8oHNWClvBe_835hUIZvuvOs_0 2024-06-28T16:14:32.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4934258838094169031', time=2024-06-28T17:47:23.918711+0000, chain.objs.size()=1 2024-06-28T16:14:32.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-f1Lo6CZfIuvUSsYiXTodJa0AYgTfJW_0 2024-06-28T16:14:32.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16856768870583429723', time=2024-06-28T17:47:24.970821+0000, chain.objs.size()=1 2024-06-28T16:14:32.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Kt84hqcy9sDxIuBs-_bLBXnwztod8EN_0 2024-06-28T16:14:32.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17068002413969589185', time=2024-06-28T17:47:25.487209+0000, chain.objs.size()=1 2024-06-28T16:14:32.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.PPmdlimzf-g9Lle6MgPi_t6d0VpcxIm_0 2024-06-28T16:14:32.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12744611629026116544', time=2024-06-28T17:47:26.126161+0000, chain.objs.size()=1 2024-06-28T16:14:32.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.184r1Cyl3noFNy7s5i-sqRYBhSFHRfy_0 2024-06-28T16:14:32.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12567947383989239918', time=2024-06-28T17:47:26.248851+0000, chain.objs.size()=1 2024-06-28T16:14:32.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.gxh6Pu4F9WKZ8dCVaV7gi_tS44Ob8iN_0 2024-06-28T16:14:32.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14598425439263575703', time=2024-06-28T17:47:26.735216+0000, chain.objs.size()=1 2024-06-28T16:14:32.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.BSErt_tysl-YZdfMEU31F9OZNzjnRgE_0 2024-06-28T16:14:32.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18313898969823936487', time=2024-06-28T17:47:26.891315+0000, chain.objs.size()=1 2024-06-28T16:14:32.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.l3CRvaK-yh7nJWF8f5Dq-pd5b33UZyT_0 2024-06-28T16:14:32.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2996227488288974163', time=2024-06-28T17:47:27.047191+0000, chain.objs.size()=1 2024-06-28T16:14:32.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.YdITnpFZx6xQL-791KSBmkO1tYaZZcO_0 2024-06-28T16:14:32.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16433405243174441488', time=2024-06-28T17:47:27.372138+0000, chain.objs.size()=1 2024-06-28T16:14:32.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Nlf7lGWTmBaNrstlKpFWfY1fX__q8_X_0 2024-06-28T16:14:32.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10921319025096305419', time=2024-06-28T17:47:27.447502+0000, chain.objs.size()=1 2024-06-28T16:14:32.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Q1BPt_fbK_7QI_w-_lqbePVG3HDFoHY_0 2024-06-28T16:14:32.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10981396561462916425', time=2024-06-28T17:47:27.495869+0000, chain.objs.size()=1 2024-06-28T16:14:32.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.qb9D-oGZNf_DTzq-o2_mQNNx_5Pk9k5_0 2024-06-28T16:14:32.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12726217531724886086', time=2024-06-28T17:47:28.115864+0000, chain.objs.size()=1 2024-06-28T16:14:32.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VjOFihTlBSopYlL7xv6EcJxtdmDLe1c_0 2024-06-28T16:14:32.532 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1421433983780996233', time=2024-06-28T17:47:31.039745+0000, chain.objs.size()=1 2024-06-28T16:14:32.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.l_xnNrINrIzr7mOjvvEPIJO8UXQLAEn_0 2024-06-28T16:14:32.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13693118639167546053', time=2024-06-28T17:47:31.162330+0000, chain.objs.size()=1 2024-06-28T16:14:32.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.38Ao7DPx0bQJYwI-X5_FCkKejAqTNN5_0 2024-06-28T16:14:32.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10160427017642426294', time=2024-06-28T17:47:31.997661+0000, chain.objs.size()=1 2024-06-28T16:14:32.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.cF5AFwAZuIHo2oXQSOJqOJ6tHH_GlWW_0 2024-06-28T16:14:32.533 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1041254176693967368', time=2024-06-28T17:47:33.474938+0000, chain.objs.size()=1 2024-06-28T16:14:32.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Vh9QKR4FrXWf-ocGVjJiDZVuTqKajvd_0 2024-06-28T16:14:32.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16032849178687941899', time=2024-06-28T17:47:34.419828+0000, chain.objs.size()=1 2024-06-28T16:14:32.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.NtK-dO9pdb6tDlFquiDOTARuj5FiDzS_0 2024-06-28T16:14:32.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7984397544981126633', time=2024-06-28T17:47:34.421806+0000, chain.objs.size()=1 2024-06-28T16:14:32.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.paWmvLJ9cEAirU9XX9u0t0zVV2P8j8D_0 2024-06-28T16:14:32.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6340547562044438910', time=2024-06-28T17:47:39.348592+0000, chain.objs.size()=1 2024-06-28T16:14:32.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.DgJvMp_DYfgylrhxiA7eRYczQGfwEao_0 2024-06-28T16:14:32.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15178749389456827786', time=2024-06-28T17:47:39.638014+0000, chain.objs.size()=1 2024-06-28T16:14:32.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.OWXh84EK9gee4OC5iDjTTv3x5oq5i-A_0 2024-06-28T16:14:32.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7831530553483057292', time=2024-06-28T17:47:45.569374+0000, chain.objs.size()=1 2024-06-28T16:14:32.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.256__shadow_.pNDGORdw337BKHqNXeGkIwCZyWJG4F-_0 2024-06-28T16:14:32.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11677027402398915189', time=2024-06-28T17:48:00.503108+0000, chain.objs.size()=1 2024-06-28T16:14:32.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.266__shadow_.BRVHBgrycLX1aP6kqB30qbORXnw1BVY_0 2024-06-28T16:14:32.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10931475134564286551', time=2024-06-28T17:48:02.088566+0000, chain.objs.size()=1 2024-06-28T16:14:32.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.267__shadow_.btYiF1Tr1BfQMYHESSlx_YXWsSaVRma_0 2024-06-28T16:14:32.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11457240648177995537', time=2024-06-28T17:48:05.341352+0000, chain.objs.size()=1 2024-06-28T16:14:32.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.269__shadow_.2egrZGqmzHXWnbmCeXvOsjgWjOMn2cI_0 2024-06-28T16:14:32.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.774763979905930501', time=2024-06-28T17:48:08.513277+0000, chain.objs.size()=1 2024-06-28T16:14:32.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.271__shadow_.SHOP_bjWfbztxAZT0AuGe0wdKLQqcIY_0 2024-06-28T16:14:32.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16958735424086303320', time=2024-06-28T17:48:19.786855+0000, chain.objs.size()=1 2024-06-28T16:14:32.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.278__shadow_.MfbQTjbLW9N1GiXe6F_bjuH0UY8wTY7_0 2024-06-28T16:14:32.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14268768466577072974', time=2024-06-28T17:48:21.388746+0000, chain.objs.size()=1 2024-06-28T16:14:32.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.279__shadow_.YaicXBL7rug-JbTCIuUops2t5Q1vC7__0 2024-06-28T16:14:32.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2994316842103319250', time=2024-06-28T17:48:34.823140+0000, chain.objs.size()=1 2024-06-28T16:14:32.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.Nkwq6dYhHQnTC5fLLYWESXcDcF9uyFa_0 2024-06-28T16:14:32.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16650915266807466511', time=2024-06-28T17:48:44.707622+0000, chain.objs.size()=1 2024-06-28T16:14:32.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.289__shadow_.3Dy0rOOsMo8y3X_7vKlDR5A_wbqD_8Z_0 2024-06-28T16:14:32.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5791659631876808330', time=2024-06-28T17:51:07.131981+0000, chain.objs.size()=1 2024-06-28T16:14:32.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.NsJ38cec3DDlXImuPjrP7_fod8n3dQf_0 2024-06-28T16:14:32.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9476350437819406066', time=2024-06-28T17:51:09.022402+0000, chain.objs.size()=1 2024-06-28T16:14:32.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.9oOkxrID-FcP5pvOgmOvIXjy8_qbSK1_0 2024-06-28T16:14:32.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12968297788799840250', time=2024-06-28T17:51:09.407602+0000, chain.objs.size()=1 2024-06-28T16:14:32.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.F5oQ-Zp3Uo7-gcneXsOhsdxpOk_zTh8_0 2024-06-28T16:14:32.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13199420106982605824', time=2024-06-28T17:51:09.409790+0000, chain.objs.size()=1 2024-06-28T16:14:32.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.gNCYaNZcimJ2QrQPLhM9sIm0bZHCTKs_0 2024-06-28T16:14:32.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15102086713533190779', time=2024-06-28T17:51:11.520232+0000, chain.objs.size()=1 2024-06-28T16:14:32.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.IuHsdh443QC3dFx3nqxlZNAWjBuG2hz_0 2024-06-28T16:14:32.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10736349921209539888', time=2024-06-28T17:51:12.030241+0000, chain.objs.size()=1 2024-06-28T16:14:32.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.9v3-X2f-wJUY2sj5MrRyu7MYIg8NXq1_0 2024-06-28T16:14:32.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14332074209377352919', time=2024-06-28T17:51:12.106260+0000, chain.objs.size()=1 2024-06-28T16:14:32.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.CsW25yNR2pjeI2fTlubJw0m6lpw6kWH_0 2024-06-28T16:14:32.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8847666871234544189', time=2024-06-28T17:51:12.200294+0000, chain.objs.size()=1 2024-06-28T16:14:32.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ibZDsr0I6C6TnW1g6AEPfWi4yGavekA_0 2024-06-28T16:14:32.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6751226804079472876', time=2024-06-28T17:51:12.294086+0000, chain.objs.size()=1 2024-06-28T16:14:32.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.2aiYOJGGqYU2hmK_acyZ4UtEpDEzofd_0 2024-06-28T16:14:32.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18441332428402936169', time=2024-06-28T17:51:12.448323+0000, chain.objs.size()=1 2024-06-28T16:14:32.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.NU_GrsZ9XfWxGvtSSXAyY3SjdiABu_T_0 2024-06-28T16:14:32.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8606950480959888896', time=2024-06-28T17:51:12.828207+0000, chain.objs.size()=1 2024-06-28T16:14:32.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.NS95BpWat_-cPcEUJWjcQiOBSYfodWg_0 2024-06-28T16:14:32.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16267710986654761623', time=2024-06-28T17:51:14.131840+0000, chain.objs.size()=1 2024-06-28T16:14:32.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.OqGQheCowstLEehvzNm0AbO6U-8Izch_0 2024-06-28T16:14:32.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7907437088054658028', time=2024-06-28T17:51:14.311721+0000, chain.objs.size()=1 2024-06-28T16:14:32.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.i3hXdrFR8y0C5H4-x5wL9L4hkfYVqA0_0 2024-06-28T16:14:32.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.539+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15974441920928765163', time=2024-06-28T17:51:15.444667+0000, chain.objs.size()=1 2024-06-28T16:14:32.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.539+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1gS_U7HVINNzDGgP5lEifS0T0MNujL-_0 2024-06-28T16:14:32.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.539+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6473873946551862476', time=2024-06-28T17:51:16.050141+0000, chain.objs.size()=1 2024-06-28T16:14:32.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.539+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.10WmFViDN576e6UIEgC3hl1d85f5d5N_0 2024-06-28T16:14:32.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.539+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1370320685540882323', time=2024-06-28T17:51:16.309836+0000, chain.objs.size()=1 2024-06-28T16:14:32.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.539+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.n8rFxyKWYqxb2NEHCc6kMwIZLK9lTQA_0 2024-06-28T16:14:32.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.539+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17672360145549076038', time=2024-06-28T17:51:19.870771+0000, chain.objs.size()=1 2024-06-28T16:14:32.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.539+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.X5091huFyMcvXG1ZIixcE3sEZvLGln8_0 2024-06-28T16:14:32.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.539+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4250825801879433377', time=2024-06-28T17:51:21.437506+0000, chain.objs.size()=1 2024-06-28T16:14:32.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.539+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PxcWMtTrEIU5L5cMvWbxEO5l2-DVUKS_0 2024-06-28T16:14:32.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.539+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18390632889075118575', time=2024-06-28T17:51:24.120994+0000, chain.objs.size()=1 2024-06-28T16:14:32.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.539+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.eXZyAyx17ZEUkYy6B_EJsROJTESuq9X_0 2024-06-28T16:14:32.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3646651159746067945', time=2024-06-28T17:51:24.510365+0000, chain.objs.size()=1 2024-06-28T16:14:32.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Rm62QqCjjS1QzM0J5YDjFUbbN7jN3ze_0 2024-06-28T16:14:32.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16739742525620375525', time=2024-06-28T17:51:25.098634+0000, chain.objs.size()=1 2024-06-28T16:14:32.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Fi-RQ2RDV0DGXe3cF1VDRk1MBnIREme_0 2024-06-28T16:14:32.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11419872980643265301', time=2024-06-28T17:51:25.940815+0000, chain.objs.size()=1 2024-06-28T16:14:32.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.qP2ODpWupM2VJ_aI4GZbvYEqz3yyapZ_0 2024-06-28T16:14:32.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.727649847204647501', time=2024-06-28T17:51:27.699764+0000, chain.objs.size()=1 2024-06-28T16:14:32.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.q3XrI3ZYGrEKj_iXENCuuAW2O4TdBzP_0 2024-06-28T16:14:32.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12865351708800275257', time=2024-06-28T17:51:28.604833+0000, chain.objs.size()=1 2024-06-28T16:14:32.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.THftOy6VJr_RstY7ZfUZnZTBgNiAukn_0 2024-06-28T16:14:32.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16214908939155817190', time=2024-06-28T17:51:28.606143+0000, chain.objs.size()=1 2024-06-28T16:14:32.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PXDhkGl-RHbuxY5hmbsvhOZAQYBO18T_0 2024-06-28T16:14:32.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6011392513901364024', time=2024-06-28T17:51:28.690676+0000, chain.objs.size()=1 2024-06-28T16:14:32.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Ox3eZFKcTSH-2s_0ID9KUKJOTinnV5r_0 2024-06-28T16:14:32.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15924936763370477968', time=2024-06-28T17:51:29.549405+0000, chain.objs.size()=1 2024-06-28T16:14:32.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.NbCgW26r9_n8AxECVY0a_ZgqMKV7Poz_0 2024-06-28T16:14:32.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.103032248455241618', time=2024-06-28T17:51:29.718076+0000, chain.objs.size()=1 2024-06-28T16:14:32.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GYcfouPNHBtCyiQSdv9HJ0TckqJOjmZ_0 2024-06-28T16:14:32.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9463540449438956733', time=2024-06-28T17:51:31.881172+0000, chain.objs.size()=1 2024-06-28T16:14:32.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.uiYZ4JU91PmhteCPrzz8mD9CVIbQhNY_0 2024-06-28T16:14:32.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11266634913951886666', time=2024-06-28T17:51:34.039641+0000, chain.objs.size()=1 2024-06-28T16:14:32.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.sXEnHm3Z21F6JmU2siw4yzMjhE4MNf3_0 2024-06-28T16:14:32.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4467386965322252650', time=2024-06-28T17:51:34.532996+0000, chain.objs.size()=1 2024-06-28T16:14:32.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GsNmwhPV0YR6-rMUZ2HlZr4bcXUVcFt_0 2024-06-28T16:14:32.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9752668243078612302', time=2024-06-28T17:51:34.534739+0000, chain.objs.size()=1 2024-06-28T16:14:32.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.D8aofwA0M6pN-MpnwqaZL9VEgv6L_ti_0 2024-06-28T16:14:32.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4170069944994616767', time=2024-06-28T17:51:36.307761+0000, chain.objs.size()=1 2024-06-28T16:14:32.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PZBRuJ2x8W3d69rYQmb4fvdYol7QWN6_0 2024-06-28T16:14:32.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3952954296934816088', time=2024-06-28T17:51:37.196600+0000, chain.objs.size()=1 2024-06-28T16:14:32.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.dRHo5wZapjN3S3tQW2SAniF__EcIqxh_0 2024-06-28T16:14:32.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5859151011975270887', time=2024-06-28T17:51:37.198688+0000, chain.objs.size()=1 2024-06-28T16:14:32.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.c3wq2Pvy-YNHhchk42jLUb_5ZBvipgu_0 2024-06-28T16:14:32.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14002520540052658108', time=2024-06-28T17:51:37.269196+0000, chain.objs.size()=1 2024-06-28T16:14:32.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.NwVx-qrhfmuCUVUnX2-WalbQ9EZb4TV_0 2024-06-28T16:14:32.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17520957040684904043', time=2024-06-28T17:53:20.910697+0000, chain.objs.size()=1 2024-06-28T16:14:32.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.LMgOOu0khT427c0cniT5ECvAkp0Iiip_0 2024-06-28T16:14:32.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13479574329458812365', time=2024-06-28T17:53:21.242598+0000, chain.objs.size()=1 2024-06-28T16:14:32.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.YOfdSmlfbLOg8Ynab_026fx24mcejkh_0 2024-06-28T16:14:32.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.400432063515342463', time=2024-06-28T17:53:21.432727+0000, chain.objs.size()=1 2024-06-28T16:14:32.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BZ922Rs__9bQPTVCBXkMmTxStZf3r_s_0 2024-06-28T16:14:32.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.551+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17209615485178034758', time=2024-06-28T17:53:21.434734+0000, chain.objs.size()=1 2024-06-28T16:14:32.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.551+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.8WwKvPO66UxcZre8f8J68aE48LcOXYK_0 2024-06-28T16:14:32.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.551+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17716698139989584288', time=2024-06-28T17:53:21.956435+0000, chain.objs.size()=1 2024-06-28T16:14:32.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.551+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.EQ8Y18uXy1GDS18GSnB3uKyZA_GCK2S_0 2024-06-28T16:14:32.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.551+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7688187037146985748', time=2024-06-28T17:53:22.267172+0000, chain.objs.size()=1 2024-06-28T16:14:32.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.551+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.y2L-ETnXtCuz2u5fA2yJy1XMfVApR2H_0 2024-06-28T16:14:32.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.551+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5078569440456807742', time=2024-06-28T17:53:25.668506+0000, chain.objs.size()=1 2024-06-28T16:14:32.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.551+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4RE2Wnio-XcrWwSxAq64yjY88dx_wWh_0 2024-06-28T16:14:32.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14915790853801488282', time=2024-06-28T17:53:26.775170+0000, chain.objs.size()=1 2024-06-28T16:14:32.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.MkTRUp1tuI_kwFSGXWekR8whh1Bs0bJ_0 2024-06-28T16:14:32.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7122805072981530433', time=2024-06-28T17:53:27.136812+0000, chain.objs.size()=1 2024-06-28T16:14:32.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ie8My2TCiGxnWqVG2xrWPAbH-GSCNPp_0 2024-06-28T16:14:32.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6276128660236778738', time=2024-06-28T17:53:27.527175+0000, chain.objs.size()=1 2024-06-28T16:14:32.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.6cqlgsAWydptamMcIKGaxzWCQxi5YMz_0 2024-06-28T16:14:32.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11495395823456395368', time=2024-06-28T17:53:30.784123+0000, chain.objs.size()=1 2024-06-28T16:14:32.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1LNLulQbG2ts4kBwKnWoiUyNOef172J_0 2024-06-28T16:14:32.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14582395349116150247', time=2024-06-28T17:53:31.379062+0000, chain.objs.size()=1 2024-06-28T16:14:32.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.b5C9nyOBpQe7SMZRMFBSCFEC1PLStFq_0 2024-06-28T16:14:32.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/47826 2024-06-28T16:14:32.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=35, truncated=0, next_marker='' 2024-06-28T16:14:32.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13052495330843357623', time=2024-06-28T17:53:31.686442+0000, chain.objs.size()=1 2024-06-28T16:14:32.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.RZ36rKxJgKsYu3V8Az4uJuLCFtJmv1i_0 2024-06-28T16:14:32.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8984427209899054524', time=2024-06-28T17:53:33.727395+0000, chain.objs.size()=1 2024-06-28T16:14:32.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.81kQXZPZaZu_MCmqDMjP9cLHOuaQAM8_0 2024-06-28T16:14:32.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15935522931279210492', time=2024-06-28T17:53:33.991173+0000, chain.objs.size()=1 2024-06-28T16:14:32.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.hspk_rg14wlOOWb7rzrBTx15-nZYpms_0 2024-06-28T16:14:32.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10156853012095796677', time=2024-06-28T17:53:34.343733+0000, chain.objs.size()=1 2024-06-28T16:14:32.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.jIjzaFNjfmes0BaXwYkG0CWhPFuVNw2_0 2024-06-28T16:14:32.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13259311781479993780', time=2024-06-28T17:53:35.492880+0000, chain.objs.size()=1 2024-06-28T16:14:32.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.xrjxxjfm-qYwqk2HQiWQh2ilNhkV0fP_0 2024-06-28T16:14:32.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15538472914687488529', time=2024-06-28T17:53:36.098560+0000, chain.objs.size()=1 2024-06-28T16:14:32.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GDlPc2pMCgWzG7pmbd6VKKfZcHdEE1k_0 2024-06-28T16:14:32.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3192586408441529755', time=2024-06-28T17:53:36.478417+0000, chain.objs.size()=1 2024-06-28T16:14:32.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.3j8PBzxhKaD6S57gF2rxLDa3kPhkGGt_0 2024-06-28T16:14:32.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10145776451721085726', time=2024-06-28T17:53:40.503616+0000, chain.objs.size()=1 2024-06-28T16:14:32.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wnxkSKYDl3ZJMlE__w-KFg6SOBuVQH4_0 2024-06-28T16:14:32.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11524800761175956812', time=2024-06-28T17:53:42.493418+0000, chain.objs.size()=1 2024-06-28T16:14:32.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.HRPkAYoJYGfv91AJ-pO1cz_LohQ6wig_0 2024-06-28T16:14:32.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12872135420064171218', time=2024-06-28T17:53:43.281630+0000, chain.objs.size()=1 2024-06-28T16:14:32.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GxtdYV30mJpeKnQxQilH39OfOCLf9fQ_0 2024-06-28T16:14:32.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.271541130738814899', time=2024-06-28T17:53:45.801336+0000, chain.objs.size()=1 2024-06-28T16:14:32.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Z3_6mmG_10nit6UqMUodMIq8_i8JEIp_0 2024-06-28T16:14:32.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16163768127139882638', time=2024-06-28T17:53:47.302334+0000, chain.objs.size()=1 2024-06-28T16:14:32.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.2fDMurZuYXAasUE22Bb8iL6xRZ1XDN__0 2024-06-28T16:14:32.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5674115507374498596', time=2024-06-28T17:53:47.706126+0000, chain.objs.size()=1 2024-06-28T16:14:32.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.KAnJNMHJM3qykWQ8Ebi3iOG-MKSGxk9_0 2024-06-28T16:14:32.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8459868827765048592', time=2024-06-28T17:53:50.739639+0000, chain.objs.size()=1 2024-06-28T16:14:32.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.IBIVg5of5tN48g27cHVsXd-SpUVLRM7_0 2024-06-28T16:14:32.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.974995191270553763', time=2024-06-28T17:53:51.413337+0000, chain.objs.size()=1 2024-06-28T16:14:32.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.AUdWF6JdyZMB3ZzJIysVX_zF1cqvgzH_0 2024-06-28T16:14:32.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1810654374783109991', time=2024-06-28T17:53:51.801542+0000, chain.objs.size()=1 2024-06-28T16:14:32.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.NJUjeJdwBFo-K5BuDWUVlEPvMgTrRC7_0 2024-06-28T16:14:32.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10183050221946008990', time=2024-06-28T17:53:51.942559+0000, chain.objs.size()=1 2024-06-28T16:14:32.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.i3KKtXGvFy4AOgV9DuJDlZ-x7vRcrkh_0 2024-06-28T16:14:32.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3760234425822610577', time=2024-06-28T17:53:54.889625+0000, chain.objs.size()=1 2024-06-28T16:14:32.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.317__shadow_.s8SS2Q-0Kg9EAPqyDsvKdP-j4_kS0rm_0 2024-06-28T16:14:32.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15907146120139075481', time=2024-06-28T17:54:18.013683+0000, chain.objs.size()=1 2024-06-28T16:14:32.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.343__shadow_.QHoM9v1PWfoUC22yYwz3j0IcGMp-Jtn_0 2024-06-28T16:14:32.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17154366547745284982', time=2024-06-28T17:56:09.259452+0000, chain.objs.size()=1 2024-06-28T16:14:32.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.454__shadow_.7nK26nqbxTKVUbrxkR8npxNj4RGn4JQ_0 2024-06-28T16:14:32.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6585426218369863622', time=2024-06-28T17:56:10.170823+0000, chain.objs.size()=1 2024-06-28T16:14:32.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.454__shadow_.aailm5DRsV9C1z1Z5rY8Q-ox3Y7ygjO_0 2024-06-28T16:14:32.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15452728813685134669', time=2024-06-28T17:56:29.665283+0000, chain.objs.size()=1 2024-06-28T16:14:32.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.474__shadow_.krWC3RzL4vgWguAOAhx4lN_3BS_j-Us_0 2024-06-28T16:14:32.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.476__shadow_.FIZWtkSAP1ynDLVqO7xV4i1CqgW6oMx_1_431ece137e3478ee419b4580478d075e', time=2024-06-28T17:56:35.609687+0000, chain.objs.size()=1 2024-06-28T16:14:32.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.476__shadow_.FIZWtkSAP1ynDLVqO7xV4i1CqgW6oMx_0 2024-06-28T16:14:32.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2250948086156921417', time=2024-06-28T17:56:38.076229+0000, chain.objs.size()=1 2024-06-28T16:14:32.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.478__shadow_.2P51eHUNvnd8vt80wRKUqvwqpbgjfOB_0 2024-06-28T16:14:32.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.990709235923778050', time=2024-06-28T17:57:19.173321+0000, chain.objs.size()=1 2024-06-28T16:14:32.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.506__multipart_mymultipartcopy.2~vHDzRbSwmVUBknQpVgzjKBz17jD7CRA.1 2024-06-28T16:14:32.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14199449930815584044', time=2024-06-28T18:00:47.207885+0000, chain.objs.size()=1 2024-06-28T16:14:32.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.526__shadow_.PLoq3gahGH5yOfNQFz1DxzWmwWcV_-G_0 2024-06-28T16:14:32.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15494074712924296777', time=2024-06-28T18:02:16.365702+0000, chain.objs.size()=1 2024-06-28T16:14:32.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:WIyGOZKO8b6lYu4DhaZbtYJywkHQGLw_.vuU7B7n1nkniqVkdKwYQQzw6B1abypD_0 2024-06-28T16:14:32.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16439536337640622143', time=2024-06-28T18:02:18.302775+0000, chain.objs.size()=1 2024-06-28T16:14:32.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:4IANX8p7ePYx9Ym6uFfF7GTMVqBmiQo_.LMvQd5_L7M2XhMiJ7hwcebdcp_6DPQv_0 2024-06-28T16:14:32.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7523200167276557695', time=2024-06-28T18:02:25.324847+0000, chain.objs.size()=1 2024-06-28T16:14:32.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.561__shadow:Y5Mmfhejx0RYvnkA9wlTXL2mgLZ3Kpa_.k3z9alCed4J5GhDKd8mqvu_PKwWXLUC_0 2024-06-28T16:14:32.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9689543936149121774', time=2024-06-28T18:02:31.384006+0000, chain.objs.size()=1 2024-06-28T16:14:32.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.563__shadow:Cw4w5nZA-k8luJHeT0kLzpdT23p1zDl_.mzDgTq19OgabcK8DVUup7ahjByz_zJr_0 2024-06-28T16:14:32.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.147400874842211460', time=2024-06-28T18:02:36.097470+0000, chain.objs.size()=1 2024-06-28T16:14:32.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.563__shadow:3it41ku3Eg235swqPF2X3E8nZw7vIee_._1lSX1kEJDZKU_qNyIH-nfev1QBBE3j_0 2024-06-28T16:14:32.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12077158770753294527', time=2024-06-28T18:02:50.601214+0000, chain.objs.size()=1 2024-06-28T16:14:32.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.565__shadow:F.yznoM8n3ZFPtZF9jO9-zTtvMdlt6._.5zl0GwyES4DoeIblJz4YXs7t64e3raL_0 2024-06-28T16:14:32.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1799580518111486173', time=2024-06-28T18:03:01.787784+0000, chain.objs.size()=1 2024-06-28T16:14:32.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:OU-YKuyBaIzutC4VRvl2A4LIjEj5qsB_.mZcKmsvhO3qGP7hiW7vCY2I7fmF-9NT_0 2024-06-28T16:14:32.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14566844639052380234', time=2024-06-28T18:04:30.743779+0000, chain.objs.size()=1 2024-06-28T16:14:32.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.595__shadow_.NY5OtWAUyhCX9tHxUpbCT4Nz6pT3udd_0 2024-06-28T16:14:32.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1363986583561845060', time=2024-06-28T18:07:27.269777+0000, chain.objs.size()=1 2024-06-28T16:14:32.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.639__shadow_.4Eow75r2n_j08OTmoZgL5FW3PBN7gvI_0 2024-06-28T16:14:32.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:32.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-06-28T16:14:32.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:32.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:32.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/42298' 2024-06-28T16:14:32.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14521790940693365703', time=2024-06-28T17:40:41.136904+0000, chain.objs.size()=2 2024-06-28T16:14:32.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.154__multipart_foo.2~gCwA_5PVdRmXqcUlR2IymKF5onxZ3oi.1 2024-06-28T16:14:32.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.154__multipart_foo.2~gCwA_5PVdRmXqcUlR2IymKF5onxZ3oi.2 2024-06-28T16:14:32.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12615323649185817910', time=2024-06-28T17:41:05.697850+0000, chain.objs.size()=1 2024-06-28T16:14:32.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.166__shadow_.dqAMs5A_bb72YW8tUa9BTw5--2y46tY_0 2024-06-28T16:14:32.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10556618704073005563', time=2024-06-28T17:41:11.558593+0000, chain.objs.size()=1 2024-06-28T16:14:32.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.171__shadow_.NUoJgcTXLWZrcn1P5Sc3goV6ZS1bIG-_0 2024-06-28T16:14:32.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3701336200067350816', time=2024-06-28T17:45:06.612365+0000, chain.objs.size()=1 2024-06-28T16:14:32.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.239__shadow_.fYVZC6oMhZn10ir3v_uTq0w3MotXVPe_0 2024-06-28T16:14:32.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4349851057156676094', time=2024-06-28T17:45:23.874784+0000, chain.objs.size()=1 2024-06-28T16:14:32.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.250__shadow_.MBeGdvCrrcvnD7fX7VB88IBu-uJDCIw_0 2024-06-28T16:14:32.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7533535973139961314', time=2024-06-28T17:45:25.515148+0000, chain.objs.size()=1 2024-06-28T16:14:32.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.251__shadow_.OkqyQtMdujus8ZcAfYG9j1EluCbOXL5_0 2024-06-28T16:14:32.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13883706553434912737', time=2024-06-28T17:47:08.315549+0000, chain.objs.size()=1 2024-06-28T16:14:32.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_._WQMGvucH-B0PzXpXrqvjwrBUGhzQqN_0 2024-06-28T16:14:32.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17260135832773036320', time=2024-06-28T17:47:09.061990+0000, chain.objs.size()=1 2024-06-28T16:14:32.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.nv71EvD_DS1cVTpSsOxEYECj-VvPxAo_0 2024-06-28T16:14:32.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17490594193208211101', time=2024-06-28T17:47:10.193858+0000, chain.objs.size()=1 2024-06-28T16:14:32.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ZMjUbtB0y6Y3SXRP3hvwHg1G7i7ssGc_0 2024-06-28T16:14:32.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5534218948310756736', time=2024-06-28T17:47:10.590204+0000, chain.objs.size()=1 2024-06-28T16:14:32.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VGHtBHFZTr-teqICmPZw6MukuUXhL-i_0 2024-06-28T16:14:32.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5577820869790945194', time=2024-06-28T17:47:12.404064+0000, chain.objs.size()=1 2024-06-28T16:14:32.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.FIMkHQGt1TRScPU29MBZEqRXEP0ZQlE_0 2024-06-28T16:14:32.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14857376421243298073', time=2024-06-28T17:47:13.213803+0000, chain.objs.size()=1 2024-06-28T16:14:32.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.67IWMxTMMOmvI5MW_f-oXhRQs4mDvfK_0 2024-06-28T16:14:32.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16527162016262515914', time=2024-06-28T17:47:14.056433+0000, chain.objs.size()=1 2024-06-28T16:14:32.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.8J0l6GqqeJCqkEZE6vVPtj6394FPdrz_0 2024-06-28T16:14:32.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8114244358126840804', time=2024-06-28T17:47:18.030177+0000, chain.objs.size()=1 2024-06-28T16:14:32.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.uIOv3aZXPwk_nAeGb6XE0_snuBZy5Et_0 2024-06-28T16:14:32.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8730864098248684527', time=2024-06-28T17:47:18.332322+0000, chain.objs.size()=1 2024-06-28T16:14:32.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.R4jdjLFJC2KVMimJY3jKDlnGqw8tHPr_0 2024-06-28T16:14:32.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12336124942003737772', time=2024-06-28T17:47:19.215274+0000, chain.objs.size()=1 2024-06-28T16:14:32.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.UauNmtGgsI3de0E3xbPYyebOZkz_NM6_0 2024-06-28T16:14:32.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11842769902516271699', time=2024-06-28T17:47:19.875998+0000, chain.objs.size()=1 2024-06-28T16:14:32.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dwLCxJld7QSdS2TU9EVfyALyLnDcygp_0 2024-06-28T16:14:32.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2454125620898218773', time=2024-06-28T17:47:22.190141+0000, chain.objs.size()=1 2024-06-28T16:14:32.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ESr4wYG_-PWACX0B9yqKdcRrDrXXbo1_0 2024-06-28T16:14:32.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17900778938903900957', time=2024-06-28T17:47:22.511603+0000, chain.objs.size()=1 2024-06-28T16:14:32.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.yRiStiLmQiB-egtNEwZi2mLuqEhPiwd_0 2024-06-28T16:14:32.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16846164470395851229', time=2024-06-28T17:47:25.801449+0000, chain.objs.size()=1 2024-06-28T16:14:32.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.LrWwoZoR16XeoyFMhB-1a7_0iBcYewM_0 2024-06-28T16:14:32.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17029022740252216834', time=2024-06-28T17:47:26.028087+0000, chain.objs.size()=1 2024-06-28T16:14:32.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.MzImTH0rmEcjl8UsxK0TBErv6ZBRrcQ_0 2024-06-28T16:14:32.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10264481173515782457', time=2024-06-28T17:47:26.732939+0000, chain.objs.size()=1 2024-06-28T16:14:32.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.AVduaKKX99oqzbf-53W_Oi6-Hk6hQsh_0 2024-06-28T16:14:32.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.595+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17468803899102858520', time=2024-06-28T17:47:27.605880+0000, chain.objs.size()=1 2024-06-28T16:14:32.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.595+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Wcjch8DpTtqWz9kvdck_KHCseZqNfVo_0 2024-06-28T16:14:32.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.595+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6514308502433715938', time=2024-06-28T17:47:27.715599+0000, chain.objs.size()=1 2024-06-28T16:14:32.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.595+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5wjA1S1g9sVsvf-9-sMqY4Qrhmwu3FF_0 2024-06-28T16:14:32.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.595+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8616436355536455116', time=2024-06-28T17:47:29.435547+0000, chain.objs.size()=1 2024-06-28T16:14:32.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.595+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.sJzTHTriDtSkCZ8sGw-Q2jRejS9_x8U_0 2024-06-28T16:14:32.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8395249596377613509', time=2024-06-28T17:47:30.901993+0000, chain.objs.size()=1 2024-06-28T16:14:32.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.uHCYBapJtfMUrMB2AwSUghNNDsuWr3C_0 2024-06-28T16:14:32.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4979585657708313361', time=2024-06-28T17:47:34.880371+0000, chain.objs.size()=1 2024-06-28T16:14:32.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.e8Qkme7Z3V4g70I6AxdPIzGLcfDNZqg_0 2024-06-28T16:14:32.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8869024364541066897', time=2024-06-28T17:47:36.972029+0000, chain.objs.size()=1 2024-06-28T16:14:32.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4dSvxI9Ba3Xy6hNwxZfWSaTF_C_epML_0 2024-06-28T16:14:32.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9409055764467454209', time=2024-06-28T17:47:37.475157+0000, chain.objs.size()=1 2024-06-28T16:14:32.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.pabrkVzJHaM9UiYMJVIc6sO8Qe6-8R2_0 2024-06-28T16:14:32.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15674443544420364944', time=2024-06-28T17:47:37.578045+0000, chain.objs.size()=1 2024-06-28T16:14:32.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7yZCJ9pfj8fNWemnbpOBh6fxRDqE_qe_0 2024-06-28T16:14:32.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4650634836858318775', time=2024-06-28T17:48:02.065475+0000, chain.objs.size()=1 2024-06-28T16:14:32.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.267__shadow_.x1XKcaWXeMsLr_AT1mdpWzQIoBApUYJ_0 2024-06-28T16:14:32.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6810488343961404789', time=2024-06-28T17:48:06.952519+0000, chain.objs.size()=1 2024-06-28T16:14:32.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.270__shadow_.hkuntLHPhCEEqYDxSXIv9fAj8s6UAsg_0 2024-06-28T16:14:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13882604752557428427', time=2024-06-28T17:48:16.676593+0000, chain.objs.size()=1 2024-06-28T16:14:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.276__shadow_.DdPJU9M1EwPMXQ22p4LbWGbxKDlECcd_0 2024-06-28T16:14:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12282625027725032870', time=2024-06-28T17:48:29.577115+0000, chain.objs.size()=1 2024-06-28T16:14:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.-A3Cnhh05qOTrmYL6tWXUYIqepbFfx__0 2024-06-28T16:14:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17460742072879286043', time=2024-06-28T17:48:30.024005+0000, chain.objs.size()=1 2024-06-28T16:14:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.6LCjYakSCDYUwW4rLX5RPsHTzWrJd8W_0 2024-06-28T16:14:32.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17992763613945546218', time=2024-06-28T17:48:36.501180+0000, chain.objs.size()=1 2024-06-28T16:14:32.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.284__shadow_.SY_pXdZz8tf8lasmbmiUXjQsZdtgfMl_0 2024-06-28T16:14:32.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16646417151013335163', time=2024-06-28T17:48:47.885821+0000, chain.objs.size()=1 2024-06-28T16:14:32.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.291__shadow_.0zDKz4UzodQwK9Bg9DYPAnHG1jG9i4m_0 2024-06-28T16:14:32.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1177041152188086417', time=2024-06-28T17:48:54.346132+0000, chain.objs.size()=1 2024-06-28T16:14:32.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.295__shadow_.gnDuR6HtT2nBPFUSb6EUqwaEymrxo1h_0 2024-06-28T16:14:32.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18344225350595437036', time=2024-06-28T17:51:06.760770+0000, chain.objs.size()=1 2024-06-28T16:14:32.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.UdrYKtT9bTaeM8DSyi9ztCQpXmNfAeu_0 2024-06-28T16:14:32.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7242839005214198715', time=2024-06-28T17:51:08.632731+0000, chain.objs.size()=1 2024-06-28T16:14:32.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.DMq7S9F2EOgKxarUc8e2CCWss95NKfy_0 2024-06-28T16:14:32.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.956366460217312172', time=2024-06-28T17:51:09.934536+0000, chain.objs.size()=1 2024-06-28T16:14:32.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Kp6tPWA95GZZ1RDPEPLu-YuPZRRhPts_0 2024-06-28T16:14:32.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7571784953385648638', time=2024-06-28T17:51:11.915609+0000, chain.objs.size()=1 2024-06-28T16:14:32.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.zWgTi6qfY0gRiqnnSOMXJ4BYE0rVPTv_0 2024-06-28T16:14:32.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1594358775541343216', time=2024-06-28T17:51:12.754283+0000, chain.objs.size()=1 2024-06-28T16:14:32.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.tcTz3XswvJ-0QEywYDkurYybWDv17Ad_0 2024-06-28T16:14:32.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8950005223638362935', time=2024-06-28T17:51:16.382217+0000, chain.objs.size()=1 2024-06-28T16:14:32.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1SVyqiGgd2PE9isNAoqZ-c3rwqpVlsw_0 2024-06-28T16:14:32.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1717609191002690704', time=2024-06-28T17:51:16.707049+0000, chain.objs.size()=1 2024-06-28T16:14:32.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.g_xXdli9VXFs1krozcvbdiapqwGM_FC_0 2024-06-28T16:14:32.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8393451870060795048', time=2024-06-28T17:51:17.199490+0000, chain.objs.size()=1 2024-06-28T16:14:32.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.WcenTG8eRhD7VzG2k9HOD3a7g9wJxKa_0 2024-06-28T16:14:32.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9535919169051171429', time=2024-06-28T17:51:17.483772+0000, chain.objs.size()=1 2024-06-28T16:14:32.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.AZd1hlnU_d9NZDrjbSEdbDUhUBcd0G__0 2024-06-28T16:14:32.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13342855817660641917', time=2024-06-28T17:51:17.666660+0000, chain.objs.size()=1 2024-06-28T16:14:32.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.607+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.WiPfhGwapC1AsJ3nUzT4x9teY4dVQLB_0 2024-06-28T16:14:32.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11237900093011215219', time=2024-06-28T17:51:18.503875+0000, chain.objs.size()=1 2024-06-28T16:14:32.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.MGydvlf047t657f6uDlb6omC5JLE8Ma_0 2024-06-28T16:14:32.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7173562006406454905', time=2024-06-28T17:51:19.623676+0000, chain.objs.size()=1 2024-06-28T16:14:32.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.RW9VVyMtJ-Lp1OV7y_-hSVap7PkzrLz_0 2024-06-28T16:14:32.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9280961781994088081', time=2024-06-28T17:51:20.941079+0000, chain.objs.size()=1 2024-06-28T16:14:32.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ZwxSFtKYKUDcjNVc-xfBfGiFyq18J1O_0 2024-06-28T16:14:32.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14273574207516112472', time=2024-06-28T17:51:21.583832+0000, chain.objs.size()=1 2024-06-28T16:14:32.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PJKB6ZhdqY8bsarVc6DrJpUqkQSD80O_0 2024-06-28T16:14:32.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4966909020515993516', time=2024-06-28T17:51:22.077649+0000, chain.objs.size()=1 2024-06-28T16:14:32.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.mh2ooj9GMlb0uzoSBD3BHqaBpbB02o4_0 2024-06-28T16:14:32.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.551930996292245483', time=2024-06-28T17:51:22.160027+0000, chain.objs.size()=1 2024-06-28T16:14:32.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.EvNyLYpz_2ws6SlxWq6U_bnPvYTeTF6_0 2024-06-28T16:14:32.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16374076267483284469', time=2024-06-28T17:51:22.596343+0000, chain.objs.size()=1 2024-06-28T16:14:32.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.s1sE40sbcZOU5dwlLFb9J44eFL3JnmB_0 2024-06-28T16:14:32.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11514731170206226144', time=2024-06-28T17:51:26.208728+0000, chain.objs.size()=1 2024-06-28T16:14:32.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.B3VaIVuTlMeXraW31qPIdcrp1OOFP1W_0 2024-06-28T16:14:32.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2855467996970635636', time=2024-06-28T17:51:26.430537+0000, chain.objs.size()=1 2024-06-28T16:14:32.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.9gctazDa5TugyRzA_qw98NC3nBFjZdd_0 2024-06-28T16:14:32.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10050862772893903010', time=2024-06-28T17:51:27.716638+0000, chain.objs.size()=1 2024-06-28T16:14:32.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6Q0soCK-gCFvSzbsNsTzlUCDkD1QfIe_0 2024-06-28T16:14:32.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5999647058675520833', time=2024-06-28T17:51:28.616049+0000, chain.objs.size()=1 2024-06-28T16:14:32.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.7WSMjifmrJr-FH5x5oEbs3fp1sxlHwW_0 2024-06-28T16:14:32.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18361918488716254705', time=2024-06-28T17:51:30.668098+0000, chain.objs.size()=1 2024-06-28T16:14:32.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.CGTP8avO4FS1gFJXQiy8fwSdSvVBDWw_0 2024-06-28T16:14:32.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6300595751379450742', time=2024-06-28T17:51:30.907907+0000, chain.objs.size()=1 2024-06-28T16:14:32.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.81Pt-jRhf0J0fynXFwjQkCIfR6fjOQr_0 2024-06-28T16:14:32.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6905035025698638458', time=2024-06-28T17:51:32.273092+0000, chain.objs.size()=1 2024-06-28T16:14:32.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.2okaKjMZZWQ2CzPqvQluHAA1Mb4Prkk_0 2024-06-28T16:14:32.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1388992242048157640', time=2024-06-28T17:51:34.365216+0000, chain.objs.size()=1 2024-06-28T16:14:32.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.CC3gxlKaKDPc3Eps2cKWVJbrQCKRJLh_0 2024-06-28T16:14:32.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5082427899618457639', time=2024-06-28T17:51:35.100875+0000, chain.objs.size()=1 2024-06-28T16:14:32.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.2PTtFnDatwHZtctlqLl0DMLeE1vNsJ3_0 2024-06-28T16:14:32.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9756601896968640800', time=2024-06-28T17:53:21.849386+0000, chain.objs.size()=1 2024-06-28T16:14:32.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-K7y5yrsApG8hicQb2666tXly_up1MP_0 2024-06-28T16:14:32.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9587914540217402789', time=2024-06-28T17:53:21.880975+0000, chain.objs.size()=1 2024-06-28T16:14:32.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.WLR9sD50FMRSPjmqB3jMU_Zf4FC9IB__0 2024-06-28T16:14:32.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14114170125853686949', time=2024-06-28T17:53:21.958095+0000, chain.objs.size()=1 2024-06-28T16:14:32.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gu4MtW1gJDffjEd6pNk0sejgWfonjMj_0 2024-06-28T16:14:32.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14476373034847507378', time=2024-06-28T17:53:22.658053+0000, chain.objs.size()=1 2024-06-28T16:14:32.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4qa9B1eyWPYhaGTHcOePfzWOa8aazJh_0 2024-06-28T16:14:32.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6317790978635204031', time=2024-06-28T17:53:23.125073+0000, chain.objs.size()=1 2024-06-28T16:14:32.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vTmAXQpwisp1M2ti2SUSA22VdOcufBw_0 2024-06-28T16:14:32.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11633258688853526321', time=2024-06-28T17:53:23.278762+0000, chain.objs.size()=1 2024-06-28T16:14:32.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.G3Tdx--FZ-HRLpFSusUzVqKEa5FVLpn_0 2024-06-28T16:14:32.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10928508721320857836', time=2024-06-28T17:53:25.534197+0000, chain.objs.size()=1 2024-06-28T16:14:32.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.cfroIgd9cfuW6ttHX4Ix6pmhXDEhBqV_0 2024-06-28T16:14:32.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10153877710347992995', time=2024-06-28T17:53:26.053084+0000, chain.objs.size()=1 2024-06-28T16:14:32.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vVvexhdnUf6Q1EVmYB5Riaypp235hh8_0 2024-06-28T16:14:32.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3579851362841040379', time=2024-06-28T17:53:26.210366+0000, chain.objs.size()=1 2024-06-28T16:14:32.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.R73Sbv0sbtwGbDpSvNXLEvr8X3D3-Ze_0 2024-06-28T16:14:32.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15902914221386134343', time=2024-06-28T17:53:26.516882+0000, chain.objs.size()=1 2024-06-28T16:14:32.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.UbGlekfTjvl0C1uHDWwDKa9BXMxLBd0_0 2024-06-28T16:14:32.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16782588604251090679', time=2024-06-28T17:53:27.088108+0000, chain.objs.size()=1 2024-06-28T16:14:32.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ROpS2xUG6AjvaadpX2d-GXLG6XA7_IH_0 2024-06-28T16:14:32.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14988839391690030008', time=2024-06-28T17:53:31.493690+0000, chain.objs.size()=1 2024-06-28T16:14:32.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.JpqI4cdYxXPRMzmz8gXh5KLqfOiXE8-_0 2024-06-28T16:14:32.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13951920719908375569', time=2024-06-28T17:53:31.748368+0000, chain.objs.size()=1 2024-06-28T16:14:32.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CfjCLA3R73XRilLuv8AayhJKAhebMYz_0 2024-06-28T16:14:32.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16432809569498754736', time=2024-06-28T17:53:31.969217+0000, chain.objs.size()=1 2024-06-28T16:14:32.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.623+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.uDDiTzUyPqS7ckxwOaLAeo6wrJT14AS_0 2024-06-28T16:14:32.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3188620537749705351', time=2024-06-28T17:53:32.144282+0000, chain.objs.size()=1 2024-06-28T16:14:32.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.F1epqUmAyPdHNCJI86fCRhyHXcNXdZB_0 2024-06-28T16:14:32.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16948281093377880919', time=2024-06-28T17:53:33.734155+0000, chain.objs.size()=1 2024-06-28T16:14:32.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Xyu2pQar-AbZFqQAdYqqtsQjEKigECO_0 2024-06-28T16:14:32.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4604394687448946773', time=2024-06-28T17:53:34.189535+0000, chain.objs.size()=1 2024-06-28T16:14:32.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.VEYQHmmW8fN_FdYB2nvs2QMK-pCAVjc_0 2024-06-28T16:14:32.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18365828960338893068', time=2024-06-28T17:53:34.349373+0000, chain.objs.size()=1 2024-06-28T16:14:32.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Ds1VMRAKTFtDvJlt_Q_MgmTGqDpPVcZ_0 2024-06-28T16:14:32.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11425484077202222471', time=2024-06-28T17:53:35.984726+0000, chain.objs.size()=1 2024-06-28T16:14:32.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ALaVwCTTQ1F6nUb_AzmBhcLTNdim-i2_0 2024-06-28T16:14:32.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9204038225567886568', time=2024-06-28T17:53:36.221785+0000, chain.objs.size()=1 2024-06-28T16:14:32.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.jwQqlUAQ-7lHuUFF8lwvXD7dhbBN9SQ_0 2024-06-28T16:14:32.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2385436309211993814', time=2024-06-28T17:53:36.227502+0000, chain.objs.size()=1 2024-06-28T16:14:32.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.POX762CKRK-3IZg3RtkqQugYUM7qA6p_0 2024-06-28T16:14:32.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13573489431312785122', time=2024-06-28T17:53:36.763588+0000, chain.objs.size()=1 2024-06-28T16:14:32.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.S65i6NLfC5NUjA_OAIkQv5_80VGFBVO_0 2024-06-28T16:14:32.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13022564380067803841', time=2024-06-28T17:53:38.790894+0000, chain.objs.size()=1 2024-06-28T16:14:32.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.U7W0c2ANDlwRvsHBPwxEX_oHF-5kGOh_0 2024-06-28T16:14:32.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17138753663248399713', time=2024-06-28T17:53:39.482492+0000, chain.objs.size()=1 2024-06-28T16:14:32.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.PrY-t4oJ0XklEuPrMRUHgLu-6dv66YU_0 2024-06-28T16:14:32.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10575001911399438858', time=2024-06-28T17:53:39.874819+0000, chain.objs.size()=1 2024-06-28T16:14:32.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.EnPQpfm83nYGl6_uttwzpK0E19mTGjo_0 2024-06-28T16:14:32.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8522599903739073031', time=2024-06-28T17:53:40.443269+0000, chain.objs.size()=1 2024-06-28T16:14:32.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1b69bEVORGv4xQCSISP46bDjhowqGRE_0 2024-06-28T16:14:32.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4211205683396930130', time=2024-06-28T17:53:41.863766+0000, chain.objs.size()=1 2024-06-28T16:14:32.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.FdLWDyi5cyuonTwL7G0RK5c3O2La-xc_0 2024-06-28T16:14:32.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17314764765349966854', time=2024-06-28T17:53:43.861947+0000, chain.objs.size()=1 2024-06-28T16:14:32.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.LqsbhUlhl8IpipEsCUQmLoq3yirsB7__0 2024-06-28T16:14:32.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15875194236144602518', time=2024-06-28T17:53:44.236563+0000, chain.objs.size()=1 2024-06-28T16:14:32.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.w0D05XXzISgp9uigmFFELGOgUx7_Qr1_0 2024-06-28T16:14:32.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10533750712635269950', time=2024-06-28T17:53:45.142999+0000, chain.objs.size()=1 2024-06-28T16:14:32.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.zb2XCZUVaKVz_zwM1zbE_acVCmNAavX_0 2024-06-28T16:14:32.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10574172881734510508', time=2024-06-28T17:53:45.858818+0000, chain.objs.size()=1 2024-06-28T16:14:32.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.K2gFq_RTGBo6Rc_0h32iQEYQtcsqs_X_0 2024-06-28T16:14:32.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18251436810299171679', time=2024-06-28T17:53:48.419590+0000, chain.objs.size()=1 2024-06-28T16:14:32.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.PkAwQTUhhyLpuAALPCCdtYY7L-sefg0_0 2024-06-28T16:14:32.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17895898102021520502', time=2024-06-28T17:53:51.165609+0000, chain.objs.size()=1 2024-06-28T16:14:32.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Ru4XgfoWDC1AI5_bEDm9Oa2hSNkTBCZ_0 2024-06-28T16:14:32.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11101029864601041227', time=2024-06-28T17:54:37.461843+0000, chain.objs.size()=1 2024-06-28T16:14:32.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.366__shadow_.0SmWe_25Q24-YKDuyjGec9WqlMohcDh_0 2024-06-28T16:14:32.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.639+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10425206960102096296', time=2024-06-28T17:55:09.377978+0000, chain.objs.size()=1 2024-06-28T16:14:32.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.639+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.398__shadow_.ki6m4x6AF4y7fjMMmu8f6_JAy9nBoZ0_0 2024-06-28T16:14:32.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.639+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8953299918833557207', time=2024-06-28T17:55:35.037961+0000, chain.objs.size()=1 2024-06-28T16:14:32.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.639+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.427__shadow_.a4yhIBHcn3yY8XNqRfu_6O5zV470VtM_0 2024-06-28T16:14:32.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/42298 2024-06-28T16:14:32.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=19, truncated=0, next_marker='' 2024-06-28T16:14:32.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2633856071377091377', time=2024-06-28T17:55:49.307524+0000, chain.objs.size()=1 2024-06-28T16:14:32.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.440__shadow_.e0cQE0IJm-VT7BayaF7PZod6YtJxbUr_0 2024-06-28T16:14:32.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14140356269634064953', time=2024-06-28T17:55:58.218518+0000, chain.objs.size()=1 2024-06-28T16:14:32.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.447__shadow_.NuNR8qf_PnetRnELgARcfEAiPKpSqOb_0 2024-06-28T16:14:32.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10060456320995682197', time=2024-06-28T17:56:28.119227+0000, chain.objs.size()=1 2024-06-28T16:14:32.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.473__shadow_.5suUtQJ0YZL0P63ZRclt3KFRgmHcaUc_0 2024-06-28T16:14:32.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.2_a5dccbbfd0fd098e0a50a9bc231e9534', time=2024-06-28T17:57:09.020220+0000, chain.objs.size()=6 2024-06-28T16:14:32.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.1 2024-06-28T16:14:32.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.2 2024-06-28T16:14:32.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.3 2024-06-28T16:14:32.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.4 2024-06-28T16:14:32.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.5 2024-06-28T16:14:32.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.6 2024-06-28T16:14:32.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.2_0f706f030bdf9f4d0cd493f7d1bb12d5', time=2024-06-28T17:57:09.909616+0000, chain.objs.size()=6 2024-06-28T16:14:32.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.1 2024-06-28T16:14:32.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.2 2024-06-28T16:14:32.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.3 2024-06-28T16:14:32.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.647+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.4 2024-06-28T16:14:32.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.647+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.5 2024-06-28T16:14:32.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.647+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.6 2024-06-28T16:14:32.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.647+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18229964064938320561', time=2024-06-28T17:57:23.163936+0000, chain.objs.size()=1 2024-06-28T16:14:32.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.647+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.508__multipart_mymultipart.2~7M0odjRUrhYDQ7dJ3yAv-4tonTytWGt.1 2024-06-28T16:14:32.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.647+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~LK1ola-s2eSbSATq5mwpuRxonJvwdws', time=2024-06-28T18:00:04.382993+0000, chain.objs.size()=10 2024-06-28T16:14:32.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.647+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.516__multipart_mymultipart.2~LK1ola-s2eSbSATq5mwpuRxonJvwdws.1 2024-06-28T16:14:32.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.516__multipart_mymultipart.2~LK1ola-s2eSbSATq5mwpuRxonJvwdws.2 2024-06-28T16:14:32.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.516__multipart_mymultipart.2~LK1ola-s2eSbSATq5mwpuRxonJvwdws.3 2024-06-28T16:14:32.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.516__multipart_mymultipart.2~LK1ola-s2eSbSATq5mwpuRxonJvwdws.4 2024-06-28T16:14:32.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.516__multipart_mymultipart.2~LK1ola-s2eSbSATq5mwpuRxonJvwdws.5 2024-06-28T16:14:32.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.516__multipart_mymultipart.2~LK1ola-s2eSbSATq5mwpuRxonJvwdws.6 2024-06-28T16:14:32.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.516__multipart_mymultipart.2~LK1ola-s2eSbSATq5mwpuRxonJvwdws.7 2024-06-28T16:14:32.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.516__multipart_mymultipart.2~LK1ola-s2eSbSATq5mwpuRxonJvwdws.8 2024-06-28T16:14:32.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.516__multipart_mymultipart.2~LK1ola-s2eSbSATq5mwpuRxonJvwdws.9 2024-06-28T16:14:32.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.516__multipart_mymultipart.2~LK1ola-s2eSbSATq5mwpuRxonJvwdws.10 2024-06-28T16:14:32.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12199004592068268805', time=2024-06-28T18:01:30.285893+0000, chain.objs.size()=1 2024-06-28T16:14:32.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.546__shadow_.dQmt5KCbzxH0otalsLNgr5J-MQb_zRN_0 2024-06-28T16:14:32.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8800808261022178653', time=2024-06-28T18:01:30.998547+0000, chain.objs.size()=1 2024-06-28T16:14:32.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.546__shadow_.J4m7UGaVWpLJh8Y9VETzmPW3HdTQIsY_0 2024-06-28T16:14:32.664 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11477531510221952716', time=2024-06-28T18:02:07.086239+0000, chain.objs.size()=1 2024-06-28T16:14:32.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:XaoRx1Sqw8swQumzEF9.cu7Q3L6xRvM_.1VGo_z4PoCEZLiATIIru7DKqz5iysfD_0 2024-06-28T16:14:32.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3369854328740797817', time=2024-06-28T18:02:07.803257+0000, chain.objs.size()=1 2024-06-28T16:14:32.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:wwUOh4vBebMWKruqDBw4XLLFWKSOrsm_.wdTvwzSH4n-ythW5g2uhF5H2tU4EOa2_0 2024-06-28T16:14:32.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7943537700257027892', time=2024-06-28T18:02:22.662396+0000, chain.objs.size()=1 2024-06-28T16:14:32.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.559__shadow:ZEXbTzvNVEdbtbFSZgbpqATOWiWKB0F_.8XPRX0fLQv8x2pCZf14ipZkRUeOPAuX_0 2024-06-28T16:14:32.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3302144010971135104', time=2024-06-28T18:02:43.521006+0000, chain.objs.size()=1 2024-06-28T16:14:32.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.565__shadow:fV.s1uuY2PGKGd812SgFeHV50f7HSNb_.vL5C6-fLu9l41lkpB6_winKPEfuV-3M_0 2024-06-28T16:14:32.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4575556292561427136', time=2024-06-28T18:02:48.010525+0000, chain.objs.size()=1 2024-06-28T16:14:32.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.565__shadow:M4AuKg2q3v9i13dziRJis7m6lICvsyy_.jqTdMUUIWGLXtNM-kIK1cGCXw6DvlBq_0 2024-06-28T16:14:32.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6734342058144213421', time=2024-06-28T18:04:13.177389+0000, chain.objs.size()=1 2024-06-28T16:14:32.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.576__shadow:7JKwlwtp.owZa73-0LAF4vaRqfgjqrz_.R9UOTXYUgliG-UrmvY94YFTRjazzeRL_0 2024-06-28T16:14:32.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4750897499076667746', time=2024-06-28T18:06:57.157231+0000, chain.objs.size()=1 2024-06-28T16:14:32.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.621__shadow_.CfyYE08wezAgAOm-JoQOrCm6q3kvgcx_0 2024-06-28T16:14:32.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.651__shadow_.jeii3PvZHO_mnarpdhVEIypPVX1wVqn_1_0677d7c113242752353e01199f0954da', time=2024-06-28T18:08:05.561726+0000, chain.objs.size()=1 2024-06-28T16:14:32.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.651__shadow_.jeii3PvZHO_mnarpdhVEIypPVX1wVqn_0 2024-06-28T16:14:32.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7337186710296942863', time=2024-06-28T18:08:16.950488+0000, chain.objs.size()=1 2024-06-28T16:14:32.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.664__shadow_.GddjY1XPGAKjU-3J4fGxVnTADTAZ-gF_0 2024-06-28T16:14:32.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4315465548338695969', time=2024-06-28T18:08:31.233900+0000, chain.objs.size()=1 2024-06-28T16:14:32.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.680__shadow_.fpb3bNqiAmXkt78eZfXQ1bax57mnRKU_0 2024-06-28T16:14:32.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.663+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:32.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-06-28T16:14:32.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:32.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:32.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/46459' 2024-06-28T16:14:32.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14891848276759509228', time=2024-06-28T17:40:01.760259+0000, chain.objs.size()=1 2024-06-28T16:14:32.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.147__shadow:6mFbOsj-61AJytpDUblEZV-6crbSZuU_.VyVAQ2RNf8R0zYwtyyQxwt2Nv3HlAF8_0 2024-06-28T16:14:32.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12062309911082625298', time=2024-06-28T17:44:56.346440+0000, chain.objs.size()=1 2024-06-28T16:14:32.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.233__shadow_.gNnumt2rjdEiSKJSdJDU-wKKW_06fN2_0 2024-06-28T16:14:32.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2345825216673894526', time=2024-06-28T17:45:20.800429+0000, chain.objs.size()=1 2024-06-28T16:14:32.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.248__shadow_.YZoylRYmqoLlHc5yEELPWar_dMQ8oQj_0 2024-06-28T16:14:32.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2469600642308522801', time=2024-06-28T17:47:09.587225+0000, chain.objs.size()=1 2024-06-28T16:14:32.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.EjH8S-UGM8oCNLgkr8bZwaTXBHCJoUv_0 2024-06-28T16:14:32.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16769589826958284051', time=2024-06-28T17:47:12.184466+0000, chain.objs.size()=1 2024-06-28T16:14:32.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7rvlms8mOZIfOdrDLSIBHNLytyFL3pY_0 2024-06-28T16:14:32.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17699485679042678868', time=2024-06-28T17:47:16.785723+0000, chain.objs.size()=1 2024-06-28T16:14:32.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.SAwrSnDpiCNgANEzmv95PuaJfu2I3uf_0 2024-06-28T16:14:32.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3554917632734917387', time=2024-06-28T17:47:17.839868+0000, chain.objs.size()=1 2024-06-28T16:14:32.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9hP3oaxNiqYTclM96__M7AiuZntFnc1_0 2024-06-28T16:14:32.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2676265522363169412', time=2024-06-28T17:47:19.178160+0000, chain.objs.size()=1 2024-06-28T16:14:32.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.WY0pj1FxiuVG-XfGrn7e9HR-ombnPHL_0 2024-06-28T16:14:32.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9016079143915238359', time=2024-06-28T17:47:20.767851+0000, chain.objs.size()=1 2024-06-28T16:14:32.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.R_Tk6kkDDzwYEmgNsU3dA5e-NOEH97d_0 2024-06-28T16:14:32.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11126717910683815440', time=2024-06-28T17:47:20.839469+0000, chain.objs.size()=1 2024-06-28T16:14:32.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.meNMagdLvXLUXKrOqeAu2cvGZlUDbAI_0 2024-06-28T16:14:32.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16101559727006163381', time=2024-06-28T17:47:21.297932+0000, chain.objs.size()=1 2024-06-28T16:14:32.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-fTB7RsBHoxNymZffVrVLusmQyB2Ty8_0 2024-06-28T16:14:32.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16841910087313167181', time=2024-06-28T17:47:22.518750+0000, chain.objs.size()=1 2024-06-28T16:14:32.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7ZuurHTqO8RUvN5xPVWeVN45nyKt9az_0 2024-06-28T16:14:32.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8324368638071441781', time=2024-06-28T17:47:23.774469+0000, chain.objs.size()=1 2024-06-28T16:14:32.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.MJ2cuX2rMzXsIMeSkDJRfvV-cj-ZPOo_0 2024-06-28T16:14:32.684 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11339171814152468436', time=2024-06-28T17:47:24.017199+0000, chain.objs.size()=1 2024-06-28T16:14:32.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5mvRH3CLC2zRo7wAlzcq3Rn_rVUrg9V_0 2024-06-28T16:14:32.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14936141266033007151', time=2024-06-28T17:47:24.037584+0000, chain.objs.size()=1 2024-06-28T16:14:32.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.pINIfElLCYib9oubZuziWP02YHs_Au7_0 2024-06-28T16:14:32.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8012957475632389407', time=2024-06-28T17:47:26.041287+0000, chain.objs.size()=1 2024-06-28T16:14:32.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.W0jgfqFlhoX0Rx1IvepRfCKu38tMwzt_0 2024-06-28T16:14:32.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10970715978696767756', time=2024-06-28T17:47:27.243803+0000, chain.objs.size()=1 2024-06-28T16:14:32.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.eYeI_9PcW6CEpMHn1QhHD2YNWFx0cwT_0 2024-06-28T16:14:32.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4855867183785698646', time=2024-06-28T17:47:29.125432+0000, chain.objs.size()=1 2024-06-28T16:14:32.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.LcHaDN8s7H58vs5AHdnHIJVgAFtm7df_0 2024-06-28T16:14:32.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4673575978604001733', time=2024-06-28T17:47:30.405132+0000, chain.objs.size()=1 2024-06-28T16:14:32.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.kyHkkHUzYkp7T76a-xRBJ1Cc6Zq_i7t_0 2024-06-28T16:14:32.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10645682102035760069', time=2024-06-28T17:47:30.666231+0000, chain.objs.size()=1 2024-06-28T16:14:32.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_._sY7raKs-xZV36E0e_x94mPnUhBbUCE_0 2024-06-28T16:14:32.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.352957270997964025', time=2024-06-28T17:47:31.089799+0000, chain.objs.size()=1 2024-06-28T16:14:32.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.H54rY2W5Z0iU7Cq8rZQd5o5Kw85Nqvc_0 2024-06-28T16:14:32.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3254853390240123205', time=2024-06-28T17:47:31.091264+0000, chain.objs.size()=1 2024-06-28T16:14:32.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KXAQ-Gt5_Wjtuguq-dBNHdenSjD9FbL_0 2024-06-28T16:14:32.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2760231246027786341', time=2024-06-28T17:47:31.093171+0000, chain.objs.size()=1 2024-06-28T16:14:32.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Zyll-o1fM38uOmBM-6RVnER3Axw66aI_0 2024-06-28T16:14:32.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.679+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7416170162042782335', time=2024-06-28T17:47:33.437189+0000, chain.objs.size()=1 2024-06-28T16:14:32.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.679+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.eZ7Xzp3ZPa4TxuSwHB6jMsy7muuoh5F_0 2024-06-28T16:14:32.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.679+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12201617805912175804', time=2024-06-28T17:47:34.680887+0000, chain.objs.size()=1 2024-06-28T16:14:32.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.679+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.aiCnF3oQXuOcjNvSyUwsrQeKlUcKN56_0 2024-06-28T16:14:32.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.679+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15818984285816566659', time=2024-06-28T17:47:35.228659+0000, chain.objs.size()=1 2024-06-28T16:14:32.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.679+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.FS5Eb7x_KVaoH1YhXL74aPR1hpozAXL_0 2024-06-28T16:14:32.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18348633578463090357', time=2024-06-28T17:47:35.421809+0000, chain.objs.size()=1 2024-06-28T16:14:32.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.SWtyOMT619J-IbvK4UVPMNtLxy7Yfb1_0 2024-06-28T16:14:32.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.253985164209432605', time=2024-06-28T17:47:35.940160+0000, chain.objs.size()=1 2024-06-28T16:14:32.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.kfqJ7v1evOdl3UB92J72X_JniizmtSY_0 2024-06-28T16:14:32.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5198697519109335544', time=2024-06-28T17:47:36.039059+0000, chain.objs.size()=1 2024-06-28T16:14:32.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.1ejJY0-p3th_SWar44EU6EZ9VRN4I9w_0 2024-06-28T16:14:32.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18218125528452842538', time=2024-06-28T17:47:36.042224+0000, chain.objs.size()=1 2024-06-28T16:14:32.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.3c1zdT-MbC2aUvgpm64aQGtNynRhk7b_0 2024-06-28T16:14:32.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12798782617868930287', time=2024-06-28T17:47:37.071397+0000, chain.objs.size()=1 2024-06-28T16:14:32.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.N9fhD156amrYKSbMxB7NWU8sXvw7PA2_0 2024-06-28T16:14:32.692 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6542186640346125754', time=2024-06-28T17:47:38.399222+0000, chain.objs.size()=1 2024-06-28T16:14:32.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.t9G8-6ExXhS5tnz1t9y5dGrVqCVCb8__0 2024-06-28T16:14:32.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15614048298246718500', time=2024-06-28T17:47:38.648048+0000, chain.objs.size()=1 2024-06-28T16:14:32.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.o7r9HvEESQz2cB9zm4JI1yX1VkQPmc3_0 2024-06-28T16:14:32.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15690169097528516858', time=2024-06-28T17:47:38.891461+0000, chain.objs.size()=1 2024-06-28T16:14:32.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.InRT4ZcINotm4kI6Ej7fLHgdznhdfOH_0 2024-06-28T16:14:32.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12969454061026314905', time=2024-06-28T17:47:38.984346+0000, chain.objs.size()=1 2024-06-28T16:14:32.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.k6B0GYYRK5y3aCUOplW7U34xcsCGzWo_0 2024-06-28T16:14:32.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2537993656914729185', time=2024-06-28T17:47:39.601935+0000, chain.objs.size()=1 2024-06-28T16:14:32.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9i882MFEZt7dBuW3w5H-WaVqetHGX0R_0 2024-06-28T16:14:32.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14488677093596217635', time=2024-06-28T17:48:23.011329+0000, chain.objs.size()=1 2024-06-28T16:14:32.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.280__shadow_.GWSSKUoGOaixcq1k9ZkeGt2AmB9tfEl_0 2024-06-28T16:14:32.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2616164990561707682', time=2024-06-28T17:48:34.841670+0000, chain.objs.size()=1 2024-06-28T16:14:32.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.8vtmoXzSzdLMzcxzvVCQHmnHSlK1Get_0 2024-06-28T16:14:32.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14398530955969999647', time=2024-06-28T17:48:39.761249+0000, chain.objs.size()=1 2024-06-28T16:14:32.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.286__shadow_.pVhkpzu-N-c_ywvtfXNIgcMjK3atosz_0 2024-06-28T16:14:32.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1400858307341942670', time=2024-06-28T17:48:44.700037+0000, chain.objs.size()=1 2024-06-28T16:14:32.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.289__shadow_.zdWsTXjW7QaHUb_SR_ucse2BOjb3B9m_0 2024-06-28T16:14:32.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6550368943492628790', time=2024-06-28T17:48:46.317207+0000, chain.objs.size()=1 2024-06-28T16:14:32.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.290__shadow_.oBW4wF9IvDZYECFRMOeIjent6navfDO_0 2024-06-28T16:14:32.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10055336222497831200', time=2024-06-28T17:49:20.841268+0000, chain.objs.size()=1 2024-06-28T16:14:32.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.312__shadow_.27NaQFZNEPs3GrX_2x1jtHTYkPKxfy7_0 2024-06-28T16:14:32.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11226497866340307271', time=2024-06-28T17:51:07.969374+0000, chain.objs.size()=1 2024-06-28T16:14:32.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.kKxWM3GwvEOgBzQ2fdevtX2k1IjYu3J_0 2024-06-28T16:14:32.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5248015429660624310', time=2024-06-28T17:51:09.455243+0000, chain.objs.size()=1 2024-06-28T16:14:32.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hHyY64_JqWq4qXfs6z9MdMqNRq58-pf_0 2024-06-28T16:14:32.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4915255340294356245', time=2024-06-28T17:51:11.545250+0000, chain.objs.size()=1 2024-06-28T16:14:32.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.FG3_bsIbmENAxl6w_t99LGqPI0FbkqO_0 2024-06-28T16:14:32.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.602228529977729224', time=2024-06-28T17:51:11.546902+0000, chain.objs.size()=1 2024-06-28T16:14:32.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.lwCRpQ-i0s_GpNMRUxe43QcZBG2ntOq_0 2024-06-28T16:14:32.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13421746052854301212', time=2024-06-28T17:51:11.637032+0000, chain.objs.size()=1 2024-06-28T16:14:32.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.5_60qM5mrivjtuhgJaXdW595hpyia27_0 2024-06-28T16:14:32.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2162164964920806106', time=2024-06-28T17:51:13.000094+0000, chain.objs.size()=1 2024-06-28T16:14:32.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.gsbeAXpIzxBJ9lfvIBHD7xzna4vbNZh_0 2024-06-28T16:14:32.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11867036616834142020', time=2024-06-28T17:51:14.316836+0000, chain.objs.size()=1 2024-06-28T16:14:32.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xxyIOjTOe99jTZDuhY5LF_2AD7xyFeL_0 2024-06-28T16:14:32.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10849434245620817506', time=2024-06-28T17:51:15.478494+0000, chain.objs.size()=1 2024-06-28T16:14:32.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.3eEZEUCWTDAfcyCdlWO7-3xT6S4-oGj_0 2024-06-28T16:14:32.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5225187690971362923', time=2024-06-28T17:51:16.823561+0000, chain.objs.size()=1 2024-06-28T16:14:32.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.dneEZ2EccM8ZrAdayufHGdrp_DJ6NEa_0 2024-06-28T16:14:32.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8477552323608832291', time=2024-06-28T17:51:17.217373+0000, chain.objs.size()=1 2024-06-28T16:14:32.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6XTYwgyBVgiFapM4NsKuC3XWf7Ffm4d_0 2024-06-28T16:14:32.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15181621093635278295', time=2024-06-28T17:51:18.397482+0000, chain.objs.size()=1 2024-06-28T16:14:32.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.w1PryHb4q-uNYIt106Y_sLDQ_DxDDtL_0 2024-06-28T16:14:32.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2603970817263693788', time=2024-06-28T17:51:25.104236+0000, chain.objs.size()=1 2024-06-28T16:14:32.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6WYnR1YorSQBo7lOc6U2X0EQQzByFsE_0 2024-06-28T16:14:32.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16264747826874519203', time=2024-06-28T17:51:25.175052+0000, chain.objs.size()=1 2024-06-28T16:14:32.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hyU5lA_Q3YvwZrStZESnXjBJ8BNEl3y_0 2024-06-28T16:14:32.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12335360973012533510', time=2024-06-28T17:51:25.648133+0000, chain.objs.size()=1 2024-06-28T16:14:32.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.g1DNc5SGK4Xv99CTB55AvVqdI2NkIeV_0 2024-06-28T16:14:32.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3165900764645735345', time=2024-06-28T17:51:25.771279+0000, chain.objs.size()=1 2024-06-28T16:14:32.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.wqAKPp1FT63wSEgzL2z7Zf97E5280Gh_0 2024-06-28T16:14:32.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17150766394233122831', time=2024-06-28T17:51:28.459391+0000, chain.objs.size()=1 2024-06-28T16:14:32.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fJDS8xxzn5KPI8fuBwL9ZqAitIpsuiZ_0 2024-06-28T16:14:32.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8241681070169026781', time=2024-06-28T17:51:28.929682+0000, chain.objs.size()=1 2024-06-28T16:14:32.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.iLERWlP-oBfaDDS1_4SKOs52Wj3kq72_0 2024-06-28T16:14:32.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8454118959726801756', time=2024-06-28T17:51:31.085333+0000, chain.objs.size()=1 2024-06-28T16:14:32.707 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Ys_owg-1QHhABaEJLI3ebYghjjvS0lY_0 2024-06-28T16:14:32.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.214467771411098613', time=2024-06-28T17:51:32.021871+0000, chain.objs.size()=1 2024-06-28T16:14:32.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.eP1MOEMVf_ucYoVfkeobJXz4GHiM1Ww_0 2024-06-28T16:14:32.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3062595336952868503', time=2024-06-28T17:51:32.925306+0000, chain.objs.size()=1 2024-06-28T16:14:32.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.2f_mFl4xswdzEJ9o4aWKDpfadp-C7CR_0 2024-06-28T16:14:32.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11807323934352027967', time=2024-06-28T17:51:35.013707+0000, chain.objs.size()=1 2024-06-28T16:14:32.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.2ls1BfIHRCj4zAplX3uw5M6vgpKucQq_0 2024-06-28T16:14:32.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11291618794636088567', time=2024-06-28T17:51:35.021797+0000, chain.objs.size()=1 2024-06-28T16:14:32.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.jtPZ8v5EuPELaB3Z7QYR1D1DUaycu4V_0 2024-06-28T16:14:32.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16268368471836003880', time=2024-06-28T17:51:35.139425+0000, chain.objs.size()=1 2024-06-28T16:14:32.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GooJsciNZfLNjoCfy0XdJqZ1QsMUDPU_0 2024-06-28T16:14:32.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9598524837441225221', time=2024-06-28T17:51:35.876060+0000, chain.objs.size()=1 2024-06-28T16:14:32.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6SRGZPY0osVL4uukh4VoLTdwnkgZ7iJ_0 2024-06-28T16:14:32.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8415336909738633536', time=2024-06-28T17:51:36.796606+0000, chain.objs.size()=1 2024-06-28T16:14:32.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.owJKemU5EkhGE25ywJIyHjFGWvA2VMI_0 2024-06-28T16:14:32.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4896088434890056977', time=2024-06-28T17:51:38.321360+0000, chain.objs.size()=1 2024-06-28T16:14:32.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Qm6CxLaA557LnZXNUHvUNvE6w1zSJND_0 2024-06-28T16:14:32.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14733600490946063343', time=2024-06-28T17:53:20.837794+0000, chain.objs.size()=1 2024-06-28T16:14:32.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.t4W2rp3o9wjXrHP4lHV8EKhGLdfmEiI_0 2024-06-28T16:14:32.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1172385523678228878', time=2024-06-28T17:53:21.762549+0000, chain.objs.size()=1 2024-06-28T16:14:32.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.2qjdwlKopl7lRyxAGg1cp2eCV_ZbeVs_0 2024-06-28T16:14:32.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4800807580730404421', time=2024-06-28T17:53:22.390452+0000, chain.objs.size()=1 2024-06-28T16:14:32.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CRUu62jfXj9oY7X-YLuLtrUtxHsJ7jE_0 2024-06-28T16:14:32.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14701807023765007284', time=2024-06-28T17:53:22.760099+0000, chain.objs.size()=1 2024-06-28T16:14:32.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.0mFpSe9Mqe2MnZMszPVRHZ8UMIbrGB1_0 2024-06-28T16:14:32.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1546561185788356183', time=2024-06-28T17:53:26.429834+0000, chain.objs.size()=1 2024-06-28T16:14:32.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZsUPdRJtiioPAH02kYT-y0LayJhssrS_0 2024-06-28T16:14:32.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.424334065683762535', time=2024-06-28T17:53:27.665283+0000, chain.objs.size()=1 2024-06-28T16:14:32.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.G86SLulgp7DrQ8TyGfe_1oKA0-MPSs3_0 2024-06-28T16:14:32.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16226365978628287510', time=2024-06-28T17:53:27.852060+0000, chain.objs.size()=1 2024-06-28T16:14:32.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BGr6Z2uAtJOzKxtM4yMx_1QaldITVpC_0 2024-06-28T16:14:32.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6641911763873551136', time=2024-06-28T17:53:27.933951+0000, chain.objs.size()=1 2024-06-28T16:14:32.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ghR23MJsOZQCP-2pKH28PADtOWF4z6K_0 2024-06-28T16:14:32.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9403620127285508951', time=2024-06-28T17:53:30.054007+0000, chain.objs.size()=1 2024-06-28T16:14:32.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vBpjgB2JOy8zR0oXXn6GmIaK6yDEPdw_0 2024-06-28T16:14:32.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7838784494331161657', time=2024-06-28T17:53:31.447553+0000, chain.objs.size()=1 2024-06-28T16:14:32.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.6KPamI6FTE5EqikCTz2gVxnIK49gjz9_0 2024-06-28T16:14:32.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16803281654474903410', time=2024-06-28T17:53:32.686178+0000, chain.objs.size()=1 2024-06-28T16:14:32.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.yvJABWOyehjoPKxhVdnSO2H0NFp7gNB_0 2024-06-28T16:14:32.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1330163174397469485', time=2024-06-28T17:53:33.824933+0000, chain.objs.size()=1 2024-06-28T16:14:32.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.eixQIpDCYbxYip1RROsfmt7k7-FjjHT_0 2024-06-28T16:14:32.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12315938948849401942', time=2024-06-28T17:53:34.886292+0000, chain.objs.size()=1 2024-06-28T16:14:32.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.h2lZKNVxeAkPJWaTEno2a1HHEYbCpjQ_0 2024-06-28T16:14:32.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1097852032176362682', time=2024-06-28T17:53:34.888270+0000, chain.objs.size()=1 2024-06-28T16:14:32.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.89ugQvPO2-00TVIw4DzPwnXeM-wmCNk_0 2024-06-28T16:14:32.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9099917600260509149', time=2024-06-28T17:53:36.530091+0000, chain.objs.size()=1 2024-06-28T16:14:32.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.P9Oac9SWuentjykI1Z15qBAdNyi809K_0 2024-06-28T16:14:32.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.590727644988493197', time=2024-06-28T17:53:38.265981+0000, chain.objs.size()=1 2024-06-28T16:14:32.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.9oigrdz7MofFKXuFTDlRycHPID3OW99_0 2024-06-28T16:14:32.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2539415836942312012', time=2024-06-28T17:53:38.268978+0000, chain.objs.size()=1 2024-06-28T16:14:32.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.711+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vNBcWhdzRCQSvQYnQnLl30Dy8NefJaW_0 2024-06-28T16:14:32.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.711+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1427165969582970938', time=2024-06-28T17:53:38.271181+0000, chain.objs.size()=1 2024-06-28T16:14:32.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.711+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Ifsj9dsA_jwgIBc8SauA7lk0npTb9eA_0 2024-06-28T16:14:32.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15144349855313389118', time=2024-06-28T17:53:38.409039+0000, chain.objs.size()=1 2024-06-28T16:14:32.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.MPZ6wgSTvYtnYlzqD5Jzf3qeDyUJr4t_0 2024-06-28T16:14:32.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7510114858600514691', time=2024-06-28T17:53:38.790702+0000, chain.objs.size()=1 2024-06-28T16:14:32.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.RfP1Q02VyM5IzZQdMwirGAJYfIW_rMp_0 2024-06-28T16:14:32.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9768498600651388781', time=2024-06-28T17:53:39.271893+0000, chain.objs.size()=1 2024-06-28T16:14:32.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.kJn6GVdNEfNzhbdS4ZsVqG2s279A-3P_0 2024-06-28T16:14:32.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9267053969906993394', time=2024-06-28T17:53:40.904255+0000, chain.objs.size()=1 2024-06-28T16:14:32.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-I5hhwVW_KrQ6EAwxAs18lIJqHiJmji_0 2024-06-28T16:14:32.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11471893715937347044', time=2024-06-28T17:53:42.929609+0000, chain.objs.size()=1 2024-06-28T16:14:32.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GCV2FoZTNy55FsJhM3bmhLF7Rd5lgmA_0 2024-06-28T16:14:32.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5220159271415350949', time=2024-06-28T17:53:43.013911+0000, chain.objs.size()=1 2024-06-28T16:14:32.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.chW-k4HA7RDiEXJEk3Ak7fGeQ1WkBq5_0 2024-06-28T16:14:32.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17973303907591965859', time=2024-06-28T17:53:44.615422+0000, chain.objs.size()=1 2024-06-28T16:14:32.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.K6ykvIlv-82SjA6mbyMb-AbZxAPjz7t_0 2024-06-28T16:14:32.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3090040925324001447', time=2024-06-28T17:53:45.907809+0000, chain.objs.size()=1 2024-06-28T16:14:32.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-Do6tK1yFqFLxkWAExiJfDVmGErZNYW_0 2024-06-28T16:14:32.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10074029185289622021', time=2024-06-28T17:53:46.341079+0000, chain.objs.size()=1 2024-06-28T16:14:32.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bzWFj6X7pJbpTal-hBy2TPMvCOIgGT9_0 2024-06-28T16:14:32.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.814215719320415377', time=2024-06-28T17:53:46.343018+0000, chain.objs.size()=1 2024-06-28T16:14:32.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-gcjGDp8MuIy83IzIpFoQtGDrskMDlL_0 2024-06-28T16:14:32.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9840467310887081293', time=2024-06-28T17:53:46.348195+0000, chain.objs.size()=1 2024-06-28T16:14:32.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.MSYOGvLo8be0ajpnMnbiKRMJeGWB8i4_0 2024-06-28T16:14:32.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13193007520304529576', time=2024-06-28T17:53:46.570792+0000, chain.objs.size()=1 2024-06-28T16:14:32.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GJ8fkVG5qtTNkZOfnU9UEBzLgdFUkps_0 2024-06-28T16:14:32.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11590771927851140286', time=2024-06-28T17:53:51.679780+0000, chain.objs.size()=1 2024-06-28T16:14:32.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.8jRmGuU5z4zQOl-0fW4Vp_5lJB0jkG__0 2024-06-28T16:14:32.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.723+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3283855555659689075', time=2024-06-28T17:53:52.063014+0000, chain.objs.size()=1 2024-06-28T16:14:32.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.723+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.0hqb5d8vS-9QLXnRNuQeUdH81QRe9qH_0 2024-06-28T16:14:32.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.723+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/46459 2024-06-28T16:14:32.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=18, truncated=0, next_marker='' 2024-06-28T16:14:32.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1257226961520121924', time=2024-06-28T17:54:09.522938+0000, chain.objs.size()=1 2024-06-28T16:14:32.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.332__shadow_.FLI53pC-Wxa0UAxYgAozTyAfEIoKAf3_0 2024-06-28T16:14:32.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13053861463256617603', time=2024-06-28T17:54:11.215397+0000, chain.objs.size()=1 2024-06-28T16:14:32.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.334__shadow_.NTsoBJIp7iXmEXNClFvl5-NPgtUciS6_0 2024-06-28T16:14:32.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6598961694680899910', time=2024-06-28T17:54:12.068623+0000, chain.objs.size()=1 2024-06-28T16:14:32.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.335__shadow_.CAmU5Zz_j2qW3jTcCphBJWfY2w9XYxk_0 2024-06-28T16:14:32.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5065444162502316696', time=2024-06-28T17:54:44.366867+0000, chain.objs.size()=1 2024-06-28T16:14:32.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.372__shadow_.tjESXJRE_fi7YfDFNR619lBphMNQMi6_0 2024-06-28T16:14:32.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8820506453743582827', time=2024-06-28T17:55:54.815577+0000, chain.objs.size()=1 2024-06-28T16:14:32.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.445__shadow_.KoIuFMvTJYyjZMKOnb6hZpdRJ4B3yqu_0 2024-06-28T16:14:32.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13395882523771993298', time=2024-06-28T17:56:09.492447+0000, chain.objs.size()=1 2024-06-28T16:14:32.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.454__shadow_.dtoS1gomIQnwJmfzkzQN_rD4vOItB9K_0 2024-06-28T16:14:32.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.486__shadow_.dZl9bSO4f54rrBWnT7sho6L_3EycXVD_1_1e26b31129a27a4a38844525159b7e60', time=2024-06-28T17:56:45.737602+0000, chain.objs.size()=1 2024-06-28T16:14:32.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.486__shadow_.dZl9bSO4f54rrBWnT7sho6L_3EycXVD_0 2024-06-28T16:14:32.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.488__shadow_.bR9TQ5lPZzjoZKkOpHeUZq_U-s7jNij_1_99ab11b0aa859fb60997aa46ea8869c8', time=2024-06-28T17:56:47.977892+0000, chain.objs.size()=1 2024-06-28T16:14:32.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.488__shadow_.bR9TQ5lPZzjoZKkOpHeUZq_U-s7jNij_0 2024-06-28T16:14:32.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11688534016919359820', time=2024-06-28T17:57:11.937909+0000, chain.objs.size()=1 2024-06-28T16:14:32.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.501__multipart_mymultipart.2~HS6olheyzPzI6FaimSrfbc_ayIM256C.1 2024-06-28T16:14:32.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18002984623024717463', time=2024-06-28T18:02:19.612696+0000, chain.objs.size()=1 2024-06-28T16:14:32.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:d-4iZpK6M2U0IAoq5hMCT7sZlvXZwU6_.XgxOVimY1fZzDFPCoOfGk7lYVHj7SQP_0 2024-06-28T16:14:32.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.190814228602606088', time=2024-06-28T18:02:21.751813+0000, chain.objs.size()=1 2024-06-28T16:14:32.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.559__shadow:DEOpeZs0oxcMLrXufQ20vWJbM42RbAP_.alfOZzLa-erRcG7JC68Y7dcd0vjmifc_0 2024-06-28T16:14:32.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9923113122790704967', time=2024-06-28T18:02:22.518075+0000, chain.objs.size()=1 2024-06-28T16:14:32.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.559__shadow:WesKM.CKCN5nSSd9C2gigONRZYPWqI0_.z3ORpDqe2f_36eHVsLR1isW347zmqTI_0 2024-06-28T16:14:32.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.731+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15854431924736162825', time=2024-06-28T18:02:48.826915+0000, chain.objs.size()=1 2024-06-28T16:14:32.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.731+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.565__shadow:bTn.AvbKCpIrtbY0si0AFWaf6GsrHSU_.Utt-aGf4WSUZWHUGEkTvrRHcrV6BnKW_0 2024-06-28T16:14:32.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4240973239453000148', time=2024-06-28T18:03:56.599484+0000, chain.objs.size()=1 2024-06-28T16:14:32.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.568__shadow:BvZro1gJLPeUnWlPQAhMF7rCf76aJr9_.FoE2wHGbkvvc9LwIAv3qKEHRV5ur38a_0 2024-06-28T16:14:32.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9665089820204203602', time=2024-06-28T18:04:31.707798+0000, chain.objs.size()=1 2024-06-28T16:14:32.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.596__shadow_.0SlMwmmqD3QwFcLgKwdi2II0nUwKO1j_0 2024-06-28T16:14:32.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11571619874559990772', time=2024-06-28T18:07:01.923288+0000, chain.objs.size()=1 2024-06-28T16:14:32.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.625__shadow_.iTGRjeWtMvs9lhR6ovOHQ5sD1mwZH1Y_0 2024-06-28T16:14:32.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2554552092303622131', time=2024-06-28T18:08:03.014138+0000, chain.objs.size()=1 2024-06-28T16:14:32.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.650__shadow_.KpdTNdWDrCRhzcfp98mRNKJhkbZePnG_0 2024-06-28T16:14:32.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7033838875184296419', time=2024-06-28T18:08:54.771250+0000, chain.objs.size()=1 2024-06-28T16:14:32.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.699__shadow:xfA0JcQvXW6yIBScs7YPIzhxvknK5Db_.oc5WMGiPHQnEQqbGHk46BBwUjCeX4Q4_0 2024-06-28T16:14:32.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:32.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-06-28T16:14:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:32.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/44037' 2024-06-28T16:14:32.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17445519573899783444', time=2024-06-28T17:44:46.062199+0000, chain.objs.size()=1 2024-06-28T16:14:32.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.227__shadow_.-tqlcmXF2LQ8N0O-3C8ow0vRflpIQMY_0 2024-06-28T16:14:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8070614291505715328', time=2024-06-28T17:44:48.495210+0000, chain.objs.size()=1 2024-06-28T16:14:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.228__shadow_.lkfoMB_bydx0OmzO3YBaw3n4_xugjHw_0 2024-06-28T16:14:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13648264336225609378', time=2024-06-28T17:45:09.838323+0000, chain.objs.size()=1 2024-06-28T16:14:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.241__shadow_.XvZ9l48WwN4fCgrOpfzYw4isYAgDRRJ_0 2024-06-28T16:14:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3069758902391908816', time=2024-06-28T17:45:13.085675+0000, chain.objs.size()=1 2024-06-28T16:14:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.243__shadow_.q61qptsrUpKvA6SZ_S-wUBBFd6YZ2Kl_0 2024-06-28T16:14:32.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7374849762224015955', time=2024-06-28T17:47:07.920297+0000, chain.objs.size()=1 2024-06-28T16:14:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.d7nPv47BJ_x736FnjoNKMMlqbsuUQM__0 2024-06-28T16:14:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9349262812713391116', time=2024-06-28T17:47:07.922235+0000, chain.objs.size()=1 2024-06-28T16:14:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.0znorsfo_giyDE9bOnQ8VTAu8v5HATO_0 2024-06-28T16:14:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6690287306128486765', time=2024-06-28T17:47:08.438791+0000, chain.objs.size()=1 2024-06-28T16:14:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.L_YeHab73FDSg1XE3pBwYevEBQlky90_0 2024-06-28T16:14:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5110950004196741271', time=2024-06-28T17:47:08.440568+0000, chain.objs.size()=1 2024-06-28T16:14:32.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.0oCof2bJFE8b598D73dFEaxefUpPjK3_0 2024-06-28T16:14:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9878888282296565740', time=2024-06-28T17:47:09.066582+0000, chain.objs.size()=1 2024-06-28T16:14:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.YxLbazZR0H7h_uX_BV-OP7LvqPU7QPK_0 2024-06-28T16:14:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14987682097253339725', time=2024-06-28T17:47:09.439470+0000, chain.objs.size()=1 2024-06-28T16:14:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.eJCGjHiOXah3EiJL4iUfz98zgrzJOhy_0 2024-06-28T16:14:32.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15895680555089789945', time=2024-06-28T17:47:10.404817+0000, chain.objs.size()=1 2024-06-28T16:14:32.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.hJwuUzLnVkn8ytLIEhd-10P6MDsXX51_0 2024-06-28T16:14:32.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9284550755618875108', time=2024-06-28T17:47:11.057750+0000, chain.objs.size()=1 2024-06-28T16:14:32.758 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9nXyik73T0T9MIk4jYk2vLxkIkfi6w1_0 2024-06-28T16:14:32.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4394090592996398394', time=2024-06-28T17:47:12.333975+0000, chain.objs.size()=1 2024-06-28T16:14:32.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9hmkDWqXoHerLiYX0wiZslynuLdKNxB_0 2024-06-28T16:14:32.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1198902659555960931', time=2024-06-28T17:47:12.695656+0000, chain.objs.size()=1 2024-06-28T16:14:32.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.WXm2hHITOnk7gaDeZXJRhkYLjtd6Rhk_0 2024-06-28T16:14:32.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.820442261591746876', time=2024-06-28T17:47:14.162687+0000, chain.objs.size()=1 2024-06-28T16:14:32.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.xlfQChrUNhFWizklBw0oJf-hB4q05j0_0 2024-06-28T16:14:32.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12913854084579482325', time=2024-06-28T17:47:14.163988+0000, chain.objs.size()=1 2024-06-28T16:14:32.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.HmilDHCdpeDfFoEke9bUbn40dd6LqFi_0 2024-06-28T16:14:32.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12570681781283709707', time=2024-06-28T17:47:14.688037+0000, chain.objs.size()=1 2024-06-28T16:14:32.760 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.jE76V22Zkm_O17aMHJE90RoCtKI1h15_0 2024-06-28T16:14:32.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8146881859558490324', time=2024-06-28T17:47:15.324679+0000, chain.objs.size()=1 2024-06-28T16:14:32.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9KwKMfskGDrdHFbWHyjl3mnXcVZ0wf2_0 2024-06-28T16:14:32.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15436523178260679181', time=2024-06-28T17:47:15.649726+0000, chain.objs.size()=1 2024-06-28T16:14:32.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.HLPVEyO44766snDNxf2IRUQbYPp0yAI_0 2024-06-28T16:14:32.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10497073552396230946', time=2024-06-28T17:47:16.594062+0000, chain.objs.size()=1 2024-06-28T16:14:32.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Vj0at-QVWJpRnGp7jixuI7_7gaFa-xS_0 2024-06-28T16:14:32.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10898431149951858673', time=2024-06-28T17:47:16.599122+0000, chain.objs.size()=1 2024-06-28T16:14:32.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.OHe2NV1lyx6Jdk5RfhlYuXNtFsDUSrg_0 2024-06-28T16:14:32.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17827960318856358770', time=2024-06-28T17:47:16.610216+0000, chain.objs.size()=1 2024-06-28T16:14:32.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.AVyLbQX6KwnCSpITpxr0_GwzH_oagGX_0 2024-06-28T16:14:32.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4662510456381636008', time=2024-06-28T17:47:16.753566+0000, chain.objs.size()=1 2024-06-28T16:14:32.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.FlmKXoAqgNcKGAVoWzQxtx2RfPSVWrj_0 2024-06-28T16:14:32.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6951833630453346268', time=2024-06-28T17:47:17.087258+0000, chain.objs.size()=1 2024-06-28T16:14:32.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.k3qolJD2DTU7ZZXfotGI49tFmA_UK5M_0 2024-06-28T16:14:32.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18333731265358645505', time=2024-06-28T17:47:17.550101+0000, chain.objs.size()=1 2024-06-28T16:14:32.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.kQQPZsE1tmPrdJhkgL5DdyS3wFtX1Qn_0 2024-06-28T16:14:32.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12607776477888341854', time=2024-06-28T17:47:18.468930+0000, chain.objs.size()=1 2024-06-28T16:14:32.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.cV0XtJMurB_WFpxWfcKWuIQYKOsRcjC_0 2024-06-28T16:14:32.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13057790124208092876', time=2024-06-28T17:47:18.958635+0000, chain.objs.size()=1 2024-06-28T16:14:32.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.McRC1QjZyq-9DVLQ6AMpRQYJX5IcXcn_0 2024-06-28T16:14:32.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16983620309294606021', time=2024-06-28T17:47:19.171228+0000, chain.objs.size()=1 2024-06-28T16:14:32.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.B3GwantKrY7hDCoYdcErsUrMY75AruS_0 2024-06-28T16:14:32.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8211303810816793756', time=2024-06-28T17:47:21.341277+0000, chain.objs.size()=1 2024-06-28T16:14:32.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dQI7i2dwPsaJKl5ppzyoHk11g_mBVbp_0 2024-06-28T16:14:32.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16681489368198721969', time=2024-06-28T17:47:23.375387+0000, chain.objs.size()=1 2024-06-28T16:14:32.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.qdTo-7OzW76GVAUgQzPj0A9U59BpXuZ_0 2024-06-28T16:14:32.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15434768614555465567', time=2024-06-28T17:47:24.105161+0000, chain.objs.size()=1 2024-06-28T16:14:32.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ton5rha5ogTtFZJm68P-3x_53CyaElo_0 2024-06-28T16:14:32.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15010368214730210315', time=2024-06-28T17:47:24.962871+0000, chain.objs.size()=1 2024-06-28T16:14:32.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ySu_TvMR9WsJQcS_lkQPdolAb8FE78I_0 2024-06-28T16:14:32.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10719466214664516491', time=2024-06-28T17:47:25.395987+0000, chain.objs.size()=1 2024-06-28T16:14:32.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ru1er5gWYwg_DfQVOEEkVWxLdK76Xn8_0 2024-06-28T16:14:32.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.759+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7411903959117604975', time=2024-06-28T17:47:25.442925+0000, chain.objs.size()=1 2024-06-28T16:14:32.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.759+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ybf6rdar7TJHGALXhM4o8E9ABPc3USk_0 2024-06-28T16:14:32.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.759+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10902205019070664900', time=2024-06-28T17:47:25.444689+0000, chain.objs.size()=1 2024-06-28T16:14:32.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.759+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.S60nZvY0p8wJ8bmGn6R6vR-suoCCzrP_0 2024-06-28T16:14:32.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.759+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9495956994548980277', time=2024-06-28T17:47:27.649354+0000, chain.objs.size()=1 2024-06-28T16:14:32.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.759+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4GBCABcs9uj7O3IbDj-FtT1SFU0PhRQ_0 2024-06-28T16:14:32.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.759+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11174262222085858394', time=2024-06-28T17:47:29.476345+0000, chain.objs.size()=1 2024-06-28T16:14:32.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.759+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.iko3s87vxAzv-0TPrP7vpfuCwA91nBo_0 2024-06-28T16:14:32.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2404530019210853030', time=2024-06-28T17:47:29.809665+0000, chain.objs.size()=1 2024-06-28T16:14:32.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.uUpOnPNwJJgmi5toSX4Yxdt1bFr0IxM_0 2024-06-28T16:14:32.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1816134340669406075', time=2024-06-28T17:47:30.978992+0000, chain.objs.size()=1 2024-06-28T16:14:32.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.WVhqtEN4FgNeYhlaf5gS4GjkLF5fgnn_0 2024-06-28T16:14:32.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5216317524102226298', time=2024-06-28T17:47:34.801331+0000, chain.objs.size()=1 2024-06-28T16:14:32.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.GQTae2G-cZzkNHNYSxHYOlmLBQnwCiR_0 2024-06-28T16:14:32.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5028265485744969531', time=2024-06-28T17:47:35.675501+0000, chain.objs.size()=1 2024-06-28T16:14:32.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.K41E2gYhrw0bRmLnsraMrA-6fXtw6iN_0 2024-06-28T16:14:32.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8512492673528902632', time=2024-06-28T17:47:36.051388+0000, chain.objs.size()=1 2024-06-28T16:14:32.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.TQbv80tot4MH-pT9NCjhHFgI9uGsphr_0 2024-06-28T16:14:32.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14023703634847375880', time=2024-06-28T17:47:39.234562+0000, chain.objs.size()=1 2024-06-28T16:14:32.774 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.FC_RGvUUi9hibqc1SDl8QgoX_QOv_tO_0 2024-06-28T16:14:32.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13941298800412118307', time=2024-06-28T17:48:03.720115+0000, chain.objs.size()=1 2024-06-28T16:14:32.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.268__shadow_.2cxz4I4i9FW6GtbPRZIAl9NVF8HtDy8_0 2024-06-28T16:14:32.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13798275487131074521', time=2024-06-28T17:48:10.128457+0000, chain.objs.size()=1 2024-06-28T16:14:32.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.272__shadow_.hF7jRux3uA9dt92u-ggjTarCtz8IjE5_0 2024-06-28T16:14:32.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12672344993905062083', time=2024-06-28T17:48:13.266345+0000, chain.objs.size()=1 2024-06-28T16:14:32.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.274__shadow_.3Lp47e-IGdnSEQ9ytHSd9sYiCVfDXzr_0 2024-06-28T16:14:32.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18320633614997482304', time=2024-06-28T17:51:08.758562+0000, chain.objs.size()=1 2024-06-28T16:14:32.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.avKKDfHSV-b1FZSCffEC4K3fk440hkN_0 2024-06-28T16:14:32.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2608255920340358001', time=2024-06-28T17:51:08.879688+0000, chain.objs.size()=1 2024-06-28T16:14:32.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nYJOWHLhoIbsIqebeg6-sjjT753rIMX_0 2024-06-28T16:14:32.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14657564582357172075', time=2024-06-28T17:51:09.309701+0000, chain.objs.size()=1 2024-06-28T16:14:32.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.DHBWN-qJrXrJdZ9zgkbSsZGuuYF6pfI_0 2024-06-28T16:14:32.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10222573436618071065', time=2024-06-28T17:51:09.730657+0000, chain.objs.size()=1 2024-06-28T16:14:32.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.KHsyr9bmuCB0LbNJKfC1kJW9dcyD0Y8_0 2024-06-28T16:14:32.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15264074994160886012', time=2024-06-28T17:51:11.260114+0000, chain.objs.size()=1 2024-06-28T16:14:32.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hP-22rFbSRIkX3rEgEcH_8UtA9ly34Q_0 2024-06-28T16:14:32.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12533684398662105255', time=2024-06-28T17:51:13.077716+0000, chain.objs.size()=1 2024-06-28T16:14:32.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.qbdWjHadUfik94w-X1hoQwBoa1EHRfb_0 2024-06-28T16:14:32.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9870757061743355021', time=2024-06-28T17:51:13.444801+0000, chain.objs.size()=1 2024-06-28T16:14:32.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.41MjZ0q84WcjhZO_T96aJ0oFz_Z0O74_0 2024-06-28T16:14:32.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.771+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11408766292984699213', time=2024-06-28T17:51:15.446205+0000, chain.objs.size()=1 2024-06-28T16:14:32.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.771+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.E1McadIOj0NkTXxaaYAwq_42Y-rldxM_0 2024-06-28T16:14:32.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.771+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5483826845147807428', time=2024-06-28T17:51:15.719768+0000, chain.objs.size()=1 2024-06-28T16:14:32.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.771+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.eZuwR4Olrzhfu9z-Rjqq-i0SDN0XK-L_0 2024-06-28T16:14:32.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.771+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10735477286547913373', time=2024-06-28T17:51:16.248770+0000, chain.objs.size()=1 2024-06-28T16:14:32.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.771+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fuXiCT1UfVZeHrCUYSu_LCu55TWyTDK_0 2024-06-28T16:14:32.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.771+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4871904590467167473', time=2024-06-28T17:51:18.132297+0000, chain.objs.size()=1 2024-06-28T16:14:32.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.771+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Wg0Yx3ukp3NB796NDQCUGcs7oSPkQKr_0 2024-06-28T16:14:32.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.771+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2748115430726301511', time=2024-06-28T17:51:18.133848+0000, chain.objs.size()=1 2024-06-28T16:14:32.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.771+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.OURzWdZ7eL8dzVD4udh92c6_-ioP2ZR_0 2024-06-28T16:14:32.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.771+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18318787400267918750', time=2024-06-28T17:51:18.474866+0000, chain.objs.size()=1 2024-06-28T16:14:32.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.771+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.NQf-hzZz-uviEsqz5uoXZ6EhTgR4S6h_0 2024-06-28T16:14:32.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9045829107647597401', time=2024-06-28T17:51:23.489943+0000, chain.objs.size()=1 2024-06-28T16:14:32.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.74W3GE8dJC5z_NGq-aN0Jj3pgjQlOoq_0 2024-06-28T16:14:32.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2133821944700315856', time=2024-06-28T17:51:24.007690+0000, chain.objs.size()=1 2024-06-28T16:14:32.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.LSrKA7Hok-KNSsGJYMtjOcN2CVlPFlS_0 2024-06-28T16:14:32.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18367879228414000229', time=2024-06-28T17:51:24.121018+0000, chain.objs.size()=1 2024-06-28T16:14:32.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Jhqu--1vvxWK1Oa5kQnsmJt7EJUx49K_0 2024-06-28T16:14:32.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3637501502135457923', time=2024-06-28T17:51:25.312562+0000, chain.objs.size()=1 2024-06-28T16:14:32.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.l7pk2R1TQyXS1wUU-5Vc6dL3f_hDCpD_0 2024-06-28T16:14:32.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8441060812636471490', time=2024-06-28T17:51:25.978818+0000, chain.objs.size()=1 2024-06-28T16:14:32.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.X0dsB0RWyeP4L0VMktTwgZCgLAqvLPJ_0 2024-06-28T16:14:32.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15933427662332093910', time=2024-06-28T17:51:29.845796+0000, chain.objs.size()=1 2024-06-28T16:14:32.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.J6gguiF6-xa8AtBeY5cvZ7WZucGj1T2_0 2024-06-28T16:14:32.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10308232715647532331', time=2024-06-28T17:51:30.632137+0000, chain.objs.size()=1 2024-06-28T16:14:32.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yYaNJG3f4nmpyEmwFv1JOLcaDyJSGtY_0 2024-06-28T16:14:32.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14273112557196196997', time=2024-06-28T17:51:32.794586+0000, chain.objs.size()=1 2024-06-28T16:14:32.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.G2gcP9cHDZjsc1esvPVn5tp1fr2n4h9_0 2024-06-28T16:14:32.786 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.918284798174419437', time=2024-06-28T17:51:33.365480+0000, chain.objs.size()=1 2024-06-28T16:14:32.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.VgpbbpF41ZWiVspzAy7DhRuAixLYsmU_0 2024-06-28T16:14:32.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17024403335780000499', time=2024-06-28T17:51:33.604129+0000, chain.objs.size()=1 2024-06-28T16:14:32.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.JuFTroM6AcndKDQN8FB1obpKJxcNUND_0 2024-06-28T16:14:32.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10881365908751025021', time=2024-06-28T17:51:34.523586+0000, chain.objs.size()=1 2024-06-28T16:14:32.787 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.7Kc5jm7bOJx-L-Ropb7wj3DTYBTwB7S_0 2024-06-28T16:14:32.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12166441806472380061', time=2024-06-28T17:51:34.527472+0000, chain.objs.size()=1 2024-06-28T16:14:32.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ghg6MXHcXQdXyTVFXzPVk7qtzqOaARz_0 2024-06-28T16:14:32.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14794798885109178581', time=2024-06-28T17:51:36.494450+0000, chain.objs.size()=1 2024-06-28T16:14:32.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.7h0bD2KSfy0BkNsxGlzQTzhsMoGxv1M_0 2024-06-28T16:14:32.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10516926487557793747', time=2024-06-28T17:51:36.956821+0000, chain.objs.size()=1 2024-06-28T16:14:32.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.W6INrBNOIRqjfRux6C3o-3oAOmioK26_0 2024-06-28T16:14:32.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14622330508062642659', time=2024-06-28T17:51:37.323395+0000, chain.objs.size()=1 2024-06-28T16:14:32.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.rG9AJNYsz-E7CmnRrzSCwBIkKyydKzm_0 2024-06-28T16:14:32.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9195666555831437992', time=2024-06-28T17:51:37.567428+0000, chain.objs.size()=1 2024-06-28T16:14:32.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.MvAFn63ALaeOqTrATIq9uTfX5RsFq2a_0 2024-06-28T16:14:32.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5044014876108149745', time=2024-06-28T17:53:23.789797+0000, chain.objs.size()=1 2024-06-28T16:14:32.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.U71KhQca-hlOfNeBryOiLAGgixonzCs_0 2024-06-28T16:14:32.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2792646139709961528', time=2024-06-28T17:53:25.657506+0000, chain.objs.size()=1 2024-06-28T16:14:32.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Q2eKrcyzjnjRrzmuRaqhq12EDiBFydn_0 2024-06-28T16:14:32.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6154743716141556855', time=2024-06-28T17:53:25.804404+0000, chain.objs.size()=1 2024-06-28T16:14:32.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1eros5-EliM4fW-dV4TMXRwm5t2RYkR_0 2024-06-28T16:14:32.791 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7529322096210821752', time=2024-06-28T17:53:26.304661+0000, chain.objs.size()=1 2024-06-28T16:14:32.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1JQjysWNbYeK35OhVJgg1pyCjB9TTBl_0 2024-06-28T16:14:32.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17487842753350146304', time=2024-06-28T17:53:26.653868+0000, chain.objs.size()=1 2024-06-28T16:14:32.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qqqobPsw8FSSVOSOvTDeIQK2dVDvlPP_0 2024-06-28T16:14:32.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13848953267984319126', time=2024-06-28T17:53:27.850874+0000, chain.objs.size()=1 2024-06-28T16:14:32.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qer16BLj4VBi2q5ruHPjWb5CwqpDnqP_0 2024-06-28T16:14:32.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.783+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6269199505209702446', time=2024-06-28T17:53:28.206012+0000, chain.objs.size()=1 2024-06-28T16:14:32.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.783+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.pcrJKsxo6rKLTNPNBjQblKuaXISC8Vc_0 2024-06-28T16:14:32.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.783+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11955110506718099586', time=2024-06-28T17:53:32.639185+0000, chain.objs.size()=1 2024-06-28T16:14:32.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.783+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Qoyrlz1121DtIU-7n8Bv6OXmnCqjrt5_0 2024-06-28T16:14:32.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.783+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15214583544097990316', time=2024-06-28T17:53:34.032104+0000, chain.objs.size()=1 2024-06-28T16:14:32.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.783+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.tm0GXr92E_3XJIoxi-VNtxhnwtyT2PC_0 2024-06-28T16:14:32.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1817993459729191706', time=2024-06-28T17:53:34.993007+0000, chain.objs.size()=1 2024-06-28T16:14:32.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.iSwOoB-X8_Rm096aUuamYWqGnJwfgD9_0 2024-06-28T16:14:32.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16213922654475200203', time=2024-06-28T17:53:35.975072+0000, chain.objs.size()=1 2024-06-28T16:14:32.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.h-YIwPZOSO64S08XjBX4Xh8FMF_R6ho_0 2024-06-28T16:14:32.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16427718372061983407', time=2024-06-28T17:53:36.217641+0000, chain.objs.size()=1 2024-06-28T16:14:32.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.j0DfS6SDJnFJjsjCBqhXUvKkLYcxBjS_0 2024-06-28T16:14:32.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16432424275704946129', time=2024-06-28T17:53:36.849967+0000, chain.objs.size()=1 2024-06-28T16:14:32.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.PMno7AMiPOGpcswiGRLS3TiQGfeS0kq_0 2024-06-28T16:14:32.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16243486760633353907', time=2024-06-28T17:53:37.802982+0000, chain.objs.size()=1 2024-06-28T16:14:32.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wIBXIdcTL3Ef3oRhULTyfDumsZbos39_0 2024-06-28T16:14:32.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13015708040338587420', time=2024-06-28T17:53:37.804808+0000, chain.objs.size()=1 2024-06-28T16:14:32.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.eJr4JcxEoflWxpQLkrNzlEDTvGMaImI_0 2024-06-28T16:14:32.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14315807769625071189', time=2024-06-28T17:53:38.387254+0000, chain.objs.size()=1 2024-06-28T16:14:32.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GfYVwuUUg05lUeEOktTiyvgWXjyUUoY_0 2024-06-28T16:14:32.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2124027886628723706', time=2024-06-28T17:53:39.020656+0000, chain.objs.size()=1 2024-06-28T16:14:32.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.QYCuIZQZdLRPJWeLXfjPZLqpW66gG9g_0 2024-06-28T16:14:32.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4153434657006995504', time=2024-06-28T17:53:39.095212+0000, chain.objs.size()=1 2024-06-28T16:14:32.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.0KtIctBptWKiAPxAbqNXsMOXKbqs-dF_0 2024-06-28T16:14:32.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12430484226408941302', time=2024-06-28T17:53:39.213347+0000, chain.objs.size()=1 2024-06-28T16:14:32.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.UVmB6LI-7xGghSGWake99Wt0xakgEa4_0 2024-06-28T16:14:32.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13669905933104578527', time=2024-06-28T17:53:39.690616+0000, chain.objs.size()=1 2024-06-28T16:14:32.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.LjSav4QxGlF4cOMWII_Zg1ZM_-Wcxe7_0 2024-06-28T16:14:32.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15862566645715128740', time=2024-06-28T17:53:42.651024+0000, chain.objs.size()=1 2024-06-28T16:14:32.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.48IrmUBEXqvcHf5iL5HKSaJ593Zwp8E_0 2024-06-28T16:14:32.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10722596922430101971', time=2024-06-28T17:53:43.087790+0000, chain.objs.size()=1 2024-06-28T16:14:32.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.F2La1pm2dkT7AmCDz3MY6s52p_vCgie_0 2024-06-28T16:14:32.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9223602448621030791', time=2024-06-28T17:53:44.038153+0000, chain.objs.size()=1 2024-06-28T16:14:32.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4Xov1ctf1bJbV69HHLJFRW-EWY_0xzx_0 2024-06-28T16:14:32.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14479955063455829403', time=2024-06-28T17:53:45.858398+0000, chain.objs.size()=1 2024-06-28T16:14:32.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ivADeaeAwzga5nh9weWcskXB8-cc6K-_0 2024-06-28T16:14:32.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10211728565772266990', time=2024-06-28T17:53:46.309761+0000, chain.objs.size()=1 2024-06-28T16:14:32.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.791+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.JfHZEe9Lzd5LEDJlmq_wvC4CiI5x-NG_0 2024-06-28T16:14:32.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.795+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/44037 2024-06-28T16:14:32.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=16, truncated=0, next_marker='' 2024-06-28T16:14:32.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14938828012439925323', time=2024-06-28T17:53:46.473691+0000, chain.objs.size()=1 2024-06-28T16:14:32.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.VJ9e-uGcL-PXBDyCAn0us9XePWJUw2n_0 2024-06-28T16:14:32.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.107388500369430980', time=2024-06-28T17:53:46.563949+0000, chain.objs.size()=1 2024-06-28T16:14:32.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wo-1upUE8i0VNtoWIDZ_nyJoj_nS-SH_0 2024-06-28T16:14:32.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11118650324864028017', time=2024-06-28T17:53:47.588830+0000, chain.objs.size()=1 2024-06-28T16:14:32.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Y11_VahJC30Xi8hI3N5-7Bk0qAXBIL3_0 2024-06-28T16:14:32.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16369666854195381092', time=2024-06-28T17:53:48.376307+0000, chain.objs.size()=1 2024-06-28T16:14:32.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.X-kjwRt0pWDEtKAqE7tPUUox_JLeeI6_0 2024-06-28T16:14:32.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14938827694609858279', time=2024-06-28T17:53:48.708033+0000, chain.objs.size()=1 2024-06-28T16:14:32.810 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.DG-KaV8GQ5UnAD6JvE1GoZOQUXuMNvg_0 2024-06-28T16:14:32.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4024419212154203000', time=2024-06-28T17:54:06.362653+0000, chain.objs.size()=1 2024-06-28T16:14:32.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.329__shadow_.OUaydxGyxfdgkuWbMKOZz2pS47fKwON_0 2024-06-28T16:14:32.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1330247339235240267', time=2024-06-28T17:54:29.759319+0000, chain.objs.size()=1 2024-06-28T16:14:32.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.359__shadow_.1eTUAdRutw-srvdzpysss-n_qbMrFYG_0 2024-06-28T16:14:32.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7119546408548936595', time=2024-06-28T17:59:36.588609+0000, chain.objs.size()=1 2024-06-28T16:14:32.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.513__multipart_mymultipart.2~TBYxfzmcrUvefg4Lzd9A1iac9kZCR-X.1 2024-06-28T16:14:32.811 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16949247367649276852', time=2024-06-28T18:01:59.677029+0000, chain.objs.size()=1 2024-06-28T16:14:32.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:wxQD99zaSNrQD-VtHSflwk0nPn-ANN0_.57wpPU2Z4pZgc1xsdsdKg7p0f842hG-_0 2024-06-28T16:14:32.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8577480319356568936', time=2024-06-28T18:02:15.243751+0000, chain.objs.size()=1 2024-06-28T16:14:32.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:ASFDetooX99DybWh3MNMbUvnWtNMF7P_.ujnBBp1X4b4dLsX_x1guSY2LaQzCOB0_0 2024-06-28T16:14:32.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17520045317220015216', time=2024-06-28T18:02:54.575763+0000, chain.objs.size()=1 2024-06-28T16:14:32.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:d4f6NaP5fRP1b0NG40DDoe19WdSwtxp_.xdYnFXgj16_iqg4BUFO-rM7cmvU_Ofa_0 2024-06-28T16:14:32.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:b9JE41FCZ0Y5wSk5sOO07Df6tJ9KFRg_.W4vEkcA5YdpwymBiS2unbImypMQZuyR_1_c325492235aabfb4c7c6229963f2b434', time=2024-06-28T18:04:00.541324+0000, chain.objs.size()=1 2024-06-28T16:14:32.812 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.799+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:b9JE41FCZ0Y5wSk5sOO07Df6tJ9KFRg_.W4vEkcA5YdpwymBiS2unbImypMQZuyR_0 2024-06-28T16:14:32.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4410790168557580665', time=2024-06-28T18:04:13.402200+0000, chain.objs.size()=1 2024-06-28T16:14:32.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.576__shadow:tLGX7OznU6CkfXVPyhkhwBW1qgaosqm_.hVkF6f64qRrwQK_I4ExDnogqN1bSpj6_0 2024-06-28T16:14:32.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16146333273193564991', time=2024-06-28T18:08:01.152718+0000, chain.objs.size()=1 2024-06-28T16:14:32.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.649__shadow_.ob2hO39vNSwqxB-2DSElfxfVrvshiYG_0 2024-06-28T16:14:32.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13413505801555574424', time=2024-06-28T18:08:37.303873+0000, chain.objs.size()=1 2024-06-28T16:14:32.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.685__shadow:vUgBioS4k-8h85Zk4rstFev10fev1Fc_.iwgvSgCbuyALB14lUVE_WtxqUlGHMEA_0 2024-06-28T16:14:32.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4938134545979457424', time=2024-06-28T18:08:45.342333+0000, chain.objs.size()=1 2024-06-28T16:14:32.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.691__shadow_.tPXarLVUT_8i1jBO4YpkkHgtd49T1qR_0 2024-06-28T16:14:32.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.807+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:32.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-06-28T16:14:32.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:32.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:32.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/41659' 2024-06-28T16:14:32.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13356958504676360895', time=2024-06-28T17:44:40.630456+0000, chain.objs.size()=1 2024-06-28T16:14:32.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.224__shadow_.hTRf-z3c4GuG28oStMoSHnC6nj9USyh_0 2024-06-28T16:14:32.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12174044692521625555', time=2024-06-28T17:44:46.086552+0000, chain.objs.size()=1 2024-06-28T16:14:32.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.227__shadow_.lfz19AVCWRIO3a26lFiycUQ7iBfW0Mc_0 2024-06-28T16:14:32.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16440247695018228358', time=2024-06-28T17:44:48.447906+0000, chain.objs.size()=1 2024-06-28T16:14:32.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.228__shadow_.tvCseVZxQX3DW--OWNNxFdIqfz8_LYy_0 2024-06-28T16:14:32.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11118967317405912153', time=2024-06-28T17:44:48.472548+0000, chain.objs.size()=1 2024-06-28T16:14:32.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.228__shadow_.p-QT0YP-f7jRSp-HVGRHMUrobRls409_0 2024-06-28T16:14:32.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.333685683536582819', time=2024-06-28T17:44:56.317784+0000, chain.objs.size()=1 2024-06-28T16:14:32.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.233__shadow_.eaDx_u5G0RBiPNwfOHq3maVIsGVBRUc_0 2024-06-28T16:14:32.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4669197547305133787', time=2024-06-28T17:44:58.700350+0000, chain.objs.size()=1 2024-06-28T16:14:32.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.234__shadow_.pQwjBjckZMcvgqQ7UO68_3uZNJI3g-9_0 2024-06-28T16:14:32.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13112782815444272527', time=2024-06-28T17:45:04.993634+0000, chain.objs.size()=1 2024-06-28T16:14:32.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.238__shadow_.5paJql1uPc_NwVaAEt4x3Y_p7MQZhVc_0 2024-06-28T16:14:32.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3584349991887763188', time=2024-06-28T17:47:07.965795+0000, chain.objs.size()=1 2024-06-28T16:14:32.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.WWPn_heF8_-pC76KRGUFrU1hO2Jj78s_0 2024-06-28T16:14:32.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6342919809157498879', time=2024-06-28T17:47:10.856071+0000, chain.objs.size()=1 2024-06-28T16:14:32.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.H1e0CCoYuWjuiOC0GAc6BPfb5N3wgcN_0 2024-06-28T16:14:32.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11178649816769159291', time=2024-06-28T17:47:12.448437+0000, chain.objs.size()=1 2024-06-28T16:14:32.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4I3fPdomclHGeXz7Zwt3qrA1HYQ7uEE_0 2024-06-28T16:14:32.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6491136439964321890', time=2024-06-28T17:47:12.838314+0000, chain.objs.size()=1 2024-06-28T16:14:32.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.48x_yjRvg_mAgo53JHPuBRvhZGLSF0d_0 2024-06-28T16:14:32.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15866281763516524987', time=2024-06-28T17:47:13.156324+0000, chain.objs.size()=1 2024-06-28T16:14:32.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.3DfUDLkuM91CWNplQWNPEOq9HGPohlR_0 2024-06-28T16:14:32.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.815+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16320751164036454673', time=2024-06-28T17:47:13.713890+0000, chain.objs.size()=1 2024-06-28T16:14:32.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.815+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Itx51yP_1JLEILxWUSQEOERUJpGDax3_0 2024-06-28T16:14:32.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17917676896261930462', time=2024-06-28T17:47:14.312512+0000, chain.objs.size()=1 2024-06-28T16:14:32.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.mfGpT5hi6a6qR3CCzqDk4bvecqVTEYG_0 2024-06-28T16:14:32.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8105771916931618397', time=2024-06-28T17:47:14.866098+0000, chain.objs.size()=1 2024-06-28T16:14:32.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9lE8w1VuUoZgCwHOeddJiUTQokpe8_G_0 2024-06-28T16:14:32.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3789872116083786326', time=2024-06-28T17:47:18.285789+0000, chain.objs.size()=1 2024-06-28T16:14:32.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.RPd4kcNK58eBVguxqhDdu0sxK-moyoT_0 2024-06-28T16:14:32.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9781558218779529352', time=2024-06-28T17:47:19.775266+0000, chain.objs.size()=1 2024-06-28T16:14:32.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.QWYQ9rsuA9NbvUuqrmSYstOomiTn0kK_0 2024-06-28T16:14:32.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14583228718879494110', time=2024-06-28T17:47:21.071038+0000, chain.objs.size()=1 2024-06-28T16:14:32.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4-LDc_h_jpi7PR6DlEoUoHhjHkQTDtR_0 2024-06-28T16:14:32.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14156786247988360798', time=2024-06-28T17:47:21.880574+0000, chain.objs.size()=1 2024-06-28T16:14:32.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.0b1xjTiEFrhHOYDZEh1fofdnkOEZ2l5_0 2024-06-28T16:14:32.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12663549656689210243', time=2024-06-28T17:47:22.219195+0000, chain.objs.size()=1 2024-06-28T16:14:32.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.llE72dMl69n2V_8DdWNZIQDI1H-beOl_0 2024-06-28T16:14:32.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14029670628066440540', time=2024-06-28T17:47:22.321805+0000, chain.objs.size()=1 2024-06-28T16:14:32.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.aDE7sLTPdnPVZr91yjy78F1ZhdIIS8F_0 2024-06-28T16:14:32.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11770823489777782686', time=2024-06-28T17:47:22.723144+0000, chain.objs.size()=1 2024-06-28T16:14:32.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.819+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-oAMXeJccOjh1SlABfUz4X-ZFfrb7G5_0 2024-06-28T16:14:32.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1900834030596398403', time=2024-06-28T17:47:24.105052+0000, chain.objs.size()=1 2024-06-28T16:14:32.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.WCnMT9ikzO575YOlvmlL8fNgdGQew8C_0 2024-06-28T16:14:32.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11971863230767993154', time=2024-06-28T17:47:24.132818+0000, chain.objs.size()=1 2024-06-28T16:14:32.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.j67C9G4x6HkWefXjx7iVkybM8yVH3o__0 2024-06-28T16:14:32.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12204712620355189842', time=2024-06-28T17:47:25.391956+0000, chain.objs.size()=1 2024-06-28T16:14:32.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.j83TctoTvLophSvJDukSaGhVcA3wV80_0 2024-06-28T16:14:32.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12376158136493410087', time=2024-06-28T17:47:26.426103+0000, chain.objs.size()=1 2024-06-28T16:14:32.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oN7u2jZ4TnHdIIOEp1N0VZqq8rl5i2X_0 2024-06-28T16:14:32.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10590140690073798058', time=2024-06-28T17:47:29.450059+0000, chain.objs.size()=1 2024-06-28T16:14:32.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.apnVJvBZBFfsn8xsaixEtjmnCVJP_1c_0 2024-06-28T16:14:32.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13361072056826837656', time=2024-06-28T17:47:29.944175+0000, chain.objs.size()=1 2024-06-28T16:14:32.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dhyGn3e0NFPSBhyZC46sZXlU5BEal1E_0 2024-06-28T16:14:32.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11700449247088515832', time=2024-06-28T17:47:31.039799+0000, chain.objs.size()=1 2024-06-28T16:14:32.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.h9zUtswWdEfFuflwFh5agk8EBB9o4A6_0 2024-06-28T16:14:32.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15131454823767483726', time=2024-06-28T17:47:32.172855+0000, chain.objs.size()=1 2024-06-28T16:14:32.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VPvx-qgohyEkUcelJg84HcSQ-8ZUC6O_0 2024-06-28T16:14:32.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.724231344313336873', time=2024-06-28T17:47:33.236957+0000, chain.objs.size()=1 2024-06-28T16:14:32.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.DmBMKqExr6WhY4MrCMgUE9WLCarxfLW_0 2024-06-28T16:14:32.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10160568280222224634', time=2024-06-28T17:47:35.697067+0000, chain.objs.size()=1 2024-06-28T16:14:32.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.GMcWtjtKEnSL2IX7XxErBPqkN40ftw4_0 2024-06-28T16:14:32.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4629509670888320678', time=2024-06-28T17:47:38.458016+0000, chain.objs.size()=1 2024-06-28T16:14:32.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.HIGb29zQY-TdBLo_wX6OV47WdTllatI_0 2024-06-28T16:14:32.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11149145688151174230', time=2024-06-28T17:47:38.974394+0000, chain.objs.size()=1 2024-06-28T16:14:32.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Jx1I9s3DFP2tt3vl320u7iufwvxxo_c_0 2024-06-28T16:14:32.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1971435249051044186', time=2024-06-28T17:47:39.348634+0000, chain.objs.size()=1 2024-06-28T16:14:32.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.LImukrqHl6WOow7tvcZVbSdhOn5jH_q_0 2024-06-28T16:14:32.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3436479789411550390', time=2024-06-28T17:47:39.588842+0000, chain.objs.size()=1 2024-06-28T16:14:32.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.C1cPN0hv1Adh2gHpLf-ictwoBlFJzXN_0 2024-06-28T16:14:32.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14858280402969053835', time=2024-06-28T17:47:55.896525+0000, chain.objs.size()=1 2024-06-28T16:14:32.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.263__shadow_.AAatjduUeQftXKX-yQhjUnpvvAG0aSO_0 2024-06-28T16:14:32.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17461867543506763319', time=2024-06-28T17:48:18.260142+0000, chain.objs.size()=1 2024-06-28T16:14:32.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.277__shadow_.qKxEXSdr_MwaPZd45YypyjWY1cyEk1R_0 2024-06-28T16:14:32.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11823909591456663141', time=2024-06-28T17:48:29.448301+0000, chain.objs.size()=1 2024-06-28T16:14:32.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.rcbuUUl1gsMxF5ZYmY6UxBz2tmQthbU_0 2024-06-28T16:14:32.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5653937148584289987', time=2024-06-28T17:48:30.019799+0000, chain.objs.size()=1 2024-06-28T16:14:32.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.FGdzkoiarz_NdAKjQDFXXT2Xx6AmX-2_0 2024-06-28T16:14:32.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.831+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16093565227069276101', time=2024-06-28T17:48:43.040144+0000, chain.objs.size()=1 2024-06-28T16:14:32.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.831+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.288__shadow_.ZrRyMBnS7uy2BzdXbVvh7b1aeQRhR_-_0 2024-06-28T16:14:32.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.831+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3370956479660855966', time=2024-06-28T17:51:07.132157+0000, chain.objs.size()=1 2024-06-28T16:14:32.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.831+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.MHZsBWZDep4VhrdZwUjBv-mvFMY7rKR_0 2024-06-28T16:14:32.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.831+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2723946728329106110', time=2024-06-28T17:51:09.407654+0000, chain.objs.size()=1 2024-06-28T16:14:32.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.831+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.wHlHc2Ob1M1bdOc6ufdEZdTp_P4kTvQ_0 2024-06-28T16:14:32.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.831+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15031129310653393610', time=2024-06-28T17:51:11.120199+0000, chain.objs.size()=1 2024-06-28T16:14:32.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.831+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.BeUi5nBIlIZI13k7E9pLbG0C4xYiFwX_0 2024-06-28T16:14:32.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.831+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4466591254126852595', time=2024-06-28T17:51:12.030102+0000, chain.objs.size()=1 2024-06-28T16:14:32.841 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.831+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ypAUt-C6lCfPTdvG6_ZHByKNpp1Gkqx_0 2024-06-28T16:14:32.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3089271993459250302', time=2024-06-28T17:51:12.642717+0000, chain.objs.size()=1 2024-06-28T16:14:32.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.DYLQMahJVovVBBny5vRBZVmEmnnrujr_0 2024-06-28T16:14:32.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17052905661879112001', time=2024-06-28T17:51:15.828270+0000, chain.objs.size()=1 2024-06-28T16:14:32.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.K644XyvwccPUTrGHj_RcxaScSRgnjT2_0 2024-06-28T16:14:32.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.711407987742550809', time=2024-06-28T17:51:16.310110+0000, chain.objs.size()=1 2024-06-28T16:14:32.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.tiRCU6zlE_d8Ze2HFDC8mC5cSWZA6EU_0 2024-06-28T16:14:32.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15911588671506469101', time=2024-06-28T17:51:16.815383+0000, chain.objs.size()=1 2024-06-28T16:14:32.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.zUwD7PrWzascqZ5bSDOsMFqmxctMo1q_0 2024-06-28T16:14:32.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.516356878185952623', time=2024-06-28T17:51:17.740294+0000, chain.objs.size()=1 2024-06-28T16:14:32.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.BU-IRz4WDDAaHpw9i4QBDo7LaYpnf1e_0 2024-06-28T16:14:32.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12083640648681004079', time=2024-06-28T17:51:18.538027+0000, chain.objs.size()=1 2024-06-28T16:14:32.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.SbSTnZ1XNxN7pB16yT6xyydbgcTcExI_0 2024-06-28T16:14:32.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14395597768853579748', time=2024-06-28T17:51:20.852496+0000, chain.objs.size()=1 2024-06-28T16:14:32.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.e7PKf8oo8j55K9wHiZ-sVOOfPjrjLl3_0 2024-06-28T16:14:32.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12924744815297477021', time=2024-06-28T17:51:21.829717+0000, chain.objs.size()=1 2024-06-28T16:14:32.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.S3nZjxB4oB8aCsYXhUrlBJFvoJrc_P5_0 2024-06-28T16:14:32.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.839+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10800777675014067326', time=2024-06-28T17:51:22.096804+0000, chain.objs.size()=1 2024-06-28T16:14:32.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.839+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.FDsn1BtwjhIi2fnsL03DwtgmRSX82FS_0 2024-06-28T16:14:32.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.839+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18248195388147714978', time=2024-06-28T17:51:22.390695+0000, chain.objs.size()=1 2024-06-28T16:14:32.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.839+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.y0ABEI9BhkPrUPSRXYD997pTCW8gEuZ_0 2024-06-28T16:14:32.848 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.839+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17923008624486956862', time=2024-06-28T17:51:23.649279+0000, chain.objs.size()=1 2024-06-28T16:14:32.849 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.839+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.SklWLQ5dM-VVC3UMLwrSsvczrJzLQxp_0 2024-06-28T16:14:32.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.839+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7293004860614389049', time=2024-06-28T17:51:25.017978+0000, chain.objs.size()=1 2024-06-28T16:14:32.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.839+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PKs3wpv71hIJiNNfzQu4pXbY87mQ16z_0 2024-06-28T16:14:32.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.839+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4074038206564020078', time=2024-06-28T17:51:25.590785+0000, chain.objs.size()=1 2024-06-28T16:14:32.850 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.839+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GTa5EO64QxQTUYkkbmoIHUom3vFYE_d_0 2024-06-28T16:14:32.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.839+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11298316343886809262', time=2024-06-28T17:51:26.850817+0000, chain.objs.size()=1 2024-06-28T16:14:32.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.839+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.qeIw-bq_NNS_uNp84sfs91I25_CovS__0 2024-06-28T16:14:32.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.839+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6310479233517406265', time=2024-06-28T17:51:27.664670+0000, chain.objs.size()=1 2024-06-28T16:14:32.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.839+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.flTHwY0QD9V_XVHtwGsVu74JQ55u_RV_0 2024-06-28T16:14:32.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15869534406357848613', time=2024-06-28T17:51:28.136807+0000, chain.objs.size()=1 2024-06-28T16:14:32.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.2OYoMjzxg94i3BSQWMNEuObsGHE-UPs_0 2024-06-28T16:14:32.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.938440326887329009', time=2024-06-28T17:51:29.549516+0000, chain.objs.size()=1 2024-06-28T16:14:32.852 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.3rGbwGJfRxq6_M06gwPPqnh3_DKZYbV_0 2024-06-28T16:14:32.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8993984727238233277', time=2024-06-28T17:51:30.293524+0000, chain.objs.size()=1 2024-06-28T16:14:32.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1q4I_EtkfUBhKx9QDwbzkn1tqVB0YJl_0 2024-06-28T16:14:32.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2096046305766993333', time=2024-06-28T17:51:33.185711+0000, chain.objs.size()=1 2024-06-28T16:14:32.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.zeJFomUMzINWlKf_XGMlnTlcRC00gou_0 2024-06-28T16:14:32.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.329753225804306642', time=2024-06-28T17:51:34.981820+0000, chain.objs.size()=1 2024-06-28T16:14:32.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hx2E9rXGkcXpAnpmiRnoeAiW6jCOHax_0 2024-06-28T16:14:32.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7500527194649976724', time=2024-06-28T17:51:36.267107+0000, chain.objs.size()=1 2024-06-28T16:14:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.4FNJMg2qEkrsxVVmBpQsG58eaVPLVuw_0 2024-06-28T16:14:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13829862551601091391', time=2024-06-28T17:51:36.981757+0000, chain.objs.size()=1 2024-06-28T16:14:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.y_ixnPLeIF5WqXUk6dVrOFg2mpxt5yL_0 2024-06-28T16:14:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8867003480714658135', time=2024-06-28T17:53:20.855479+0000, chain.objs.size()=1 2024-06-28T16:14:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.dooEGACXPatc5pfROXJvu6buz7iXw5b_0 2024-06-28T16:14:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6670392342235558301', time=2024-06-28T17:53:22.315080+0000, chain.objs.size()=1 2024-06-28T16:14:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.QN8mU5-xFObJ7y3zXw0FRecPr7OB7Vh_0 2024-06-28T16:14:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10262853920079924042', time=2024-06-28T17:53:22.872013+0000, chain.objs.size()=1 2024-06-28T16:14:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.mL19Vp7ZWBVadyxRIrvjOIDDnhCLgkh_0 2024-06-28T16:14:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2132182598689406598', time=2024-06-28T17:53:23.125605+0000, chain.objs.size()=1 2024-06-28T16:14:32.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.I5Ns5ZuGDtp0G9urE4XJdXpz_BNHraH_0 2024-06-28T16:14:32.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2790773624542350083', time=2024-06-28T17:53:23.371241+0000, chain.objs.size()=1 2024-06-28T16:14:32.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BoJY1eurDqFAuuLEBgTmzEmfjljg3qA_0 2024-06-28T16:14:32.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17415118726402574667', time=2024-06-28T17:53:23.500358+0000, chain.objs.size()=1 2024-06-28T16:14:32.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.f09jFjsgrEkBfp58-5YGF3XPiKgzvcv_0 2024-06-28T16:14:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8626203305709812241', time=2024-06-28T17:53:23.701215+0000, chain.objs.size()=1 2024-06-28T16:14:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.sD0WmDKn4_XdkuFSTmSllajtvCDmv_g_0 2024-06-28T16:14:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2350584170020547707', time=2024-06-28T17:53:23.865121+0000, chain.objs.size()=1 2024-06-28T16:14:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.00QPnrzVQ1f6FaHg1kETMQDaTERm88L_0 2024-06-28T16:14:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9136886210671027501', time=2024-06-28T17:53:25.212295+0000, chain.objs.size()=1 2024-06-28T16:14:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.847+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1aMX2UHGOsFlJz7szcY0tRwwr1K7gzv_0 2024-06-28T16:14:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.851+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10756140888098030921', time=2024-06-28T17:53:25.680251+0000, chain.objs.size()=1 2024-06-28T16:14:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.E51MCRm4rmkZT-NOGltq3Ii5BgJVi18_0 2024-06-28T16:14:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.851+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15654921158971062874', time=2024-06-28T17:53:25.802626+0000, chain.objs.size()=1 2024-06-28T16:14:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.8QZHvsVQe487uYTadsvaw9BFK52OImP_0 2024-06-28T16:14:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.851+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16852755419871568651', time=2024-06-28T17:53:25.806467+0000, chain.objs.size()=1 2024-06-28T16:14:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.s2GfQpADhw_ihr-asW6ZweS76hnBSjA_0 2024-06-28T16:14:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.851+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16660117407181647400', time=2024-06-28T17:53:27.879848+0000, chain.objs.size()=1 2024-06-28T16:14:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gi5C3fgx-UItWJvTbqVbJzk93T7_xGh_0 2024-06-28T16:14:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.851+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12019546657249398254', time=2024-06-28T17:53:29.951322+0000, chain.objs.size()=1 2024-06-28T16:14:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-1rc0Ez9B2M-cUsaKYAgb1P4fbHHd7Y_0 2024-06-28T16:14:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.851+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10928641973723966486', time=2024-06-28T17:53:33.727376+0000, chain.objs.size()=1 2024-06-28T16:14:32.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.fijbOGYdb_qG0yyrfMkm6VeczBPvLyA_0 2024-06-28T16:14:32.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.855+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9033098119366312579', time=2024-06-28T17:53:33.729255+0000, chain.objs.size()=1 2024-06-28T16:14:32.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.855+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.v9iCFsSYCdzQqciyls5M3991QCT9wng_0 2024-06-28T16:14:32.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.855+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1492598094316913723', time=2024-06-28T17:53:34.343521+0000, chain.objs.size()=1 2024-06-28T16:14:32.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.855+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.REw0MtLG2V2NK6al_lPdOztJPnm83OX_0 2024-06-28T16:14:32.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.855+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16836351851477860606', time=2024-06-28T17:53:34.564697+0000, chain.objs.size()=1 2024-06-28T16:14:32.864 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.855+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.iCwwpINHudoYaZK1H8eJ_AUR9_MCNWc_0 2024-06-28T16:14:32.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.855+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13887847677343705565', time=2024-06-28T17:53:35.494743+0000, chain.objs.size()=1 2024-06-28T16:14:32.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.855+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.sqdaZZYNbr562w_JQUtkycPx3LzS2Mx_0 2024-06-28T16:14:32.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.855+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9296746204532174890', time=2024-06-28T17:53:36.212008+0000, chain.objs.size()=1 2024-06-28T16:14:32.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.855+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.8hhPqvLn_gu6lHWphmq43gkqlVafEiB_0 2024-06-28T16:14:32.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.855+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16699929127802084566', time=2024-06-28T17:53:38.146367+0000, chain.objs.size()=1 2024-06-28T16:14:32.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.855+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Aj325UUSyr9yx9mpTLfqxMCDYF2L2xk_0 2024-06-28T16:14:32.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.855+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16928235542458187189', time=2024-06-28T17:53:39.101363+0000, chain.objs.size()=1 2024-06-28T16:14:32.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.855+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.5rpFlzhRabiTMp8MXsEcFszgxkYyOFU_0 2024-06-28T16:14:32.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9279366487659988018', time=2024-06-28T17:53:39.621044+0000, chain.objs.size()=1 2024-06-28T16:14:32.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.AGOeMq_vEnwm3d8OBLnYOGYeGc_uZLm_0 2024-06-28T16:14:32.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1317655025795707920', time=2024-06-28T17:53:41.840396+0000, chain.objs.size()=1 2024-06-28T16:14:32.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.70S_bVvZcsHwckLlxK1v2P1oU5oraIX_0 2024-06-28T16:14:32.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4196137817335046249', time=2024-06-28T17:53:42.242381+0000, chain.objs.size()=1 2024-06-28T16:14:32.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.uxFwzyh5hsezNG80uGPGSqClr9sCogM_0 2024-06-28T16:14:32.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5090177569471166136', time=2024-06-28T17:53:42.270665+0000, chain.objs.size()=1 2024-06-28T16:14:32.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZJsvPBsnN4fXsiqbaXGjDFy9m8KSnNo_0 2024-06-28T16:14:32.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.489035544191047065', time=2024-06-28T17:53:43.083055+0000, chain.objs.size()=1 2024-06-28T16:14:32.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZOFVlV18BejtvZxclS6NkF0QZXbVr1r_0 2024-06-28T16:14:32.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16433329790746025333', time=2024-06-28T17:53:44.127571+0000, chain.objs.size()=1 2024-06-28T16:14:32.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.rIlZHMEtMD5JGEbEGjVxv25-j_-5Sir_0 2024-06-28T16:14:32.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.975690236234702184', time=2024-06-28T17:53:44.824331+0000, chain.objs.size()=1 2024-06-28T16:14:32.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_._d6aL--gZA0v8ZCHPRI3mTyOBGEBK1K_0 2024-06-28T16:14:32.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4996411338791113504', time=2024-06-28T17:53:46.501269+0000, chain.objs.size()=1 2024-06-28T16:14:32.871 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.msxzU6PSOK3EWbwwj_jwHUZq6ZCmVRL_0 2024-06-28T16:14:32.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.863+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7965026223303707201', time=2024-06-28T17:53:47.706240+0000, chain.objs.size()=1 2024-06-28T16:14:32.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.863+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.t6Jq2Ef7KpyS38CBfCKJLVoJ1YJ5RaW_0 2024-06-28T16:14:32.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.863+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.910042240460206127', time=2024-06-28T17:53:48.722775+0000, chain.objs.size()=1 2024-06-28T16:14:32.872 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.863+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.zK_LI_MjnCwz_BqHq-Dz3AAzcCM0y9M_0 2024-06-28T16:14:32.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.863+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13894418180467609974', time=2024-06-28T17:53:49.708419+0000, chain.objs.size()=1 2024-06-28T16:14:32.873 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.863+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4YL7LXMyxfdsz7xWxptPx-DosKAcTdZ_0 2024-06-28T16:14:32.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.867+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/41659 2024-06-28T16:14:32.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=20, truncated=0, next_marker='' 2024-06-28T16:14:32.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4656203318418303419', time=2024-06-28T17:53:51.622287+0000, chain.objs.size()=1 2024-06-28T16:14:32.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.NjwN79BfKbmGEX00W_8tdSLrHjzcSsx_0 2024-06-28T16:14:32.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3787789936151947585', time=2024-06-28T17:53:52.196936+0000, chain.objs.size()=1 2024-06-28T16:14:32.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.NjThIsXP_hw8KluKU6NPBd8WTYeVEXv_0 2024-06-28T16:14:32.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14464501031714952997', time=2024-06-28T17:53:55.763869+0000, chain.objs.size()=1 2024-06-28T16:14:32.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.318__shadow_.lr1Mn02bOOX9vdhiJ0i4KiBPwNEXP_M_0 2024-06-28T16:14:32.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1713871638264061966', time=2024-06-28T17:53:59.964107+0000, chain.objs.size()=1 2024-06-28T16:14:32.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.322__shadow_.j1yeUz0cWMWideL8lVHdttj4WTTThsi_0 2024-06-28T16:14:32.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14644295675277383483', time=2024-06-28T17:54:52.101709+0000, chain.objs.size()=1 2024-06-28T16:14:32.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.380__shadow_.g1Owpk92C8-F36eq_0JSOhhySia8etn_0 2024-06-28T16:14:32.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2179829519810611151', time=2024-06-28T17:55:33.819409+0000, chain.objs.size()=1 2024-06-28T16:14:32.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.426__shadow_.iOY9w4dPHacMhsuwzaeqjRW0qdMuqdb_0 2024-06-28T16:14:32.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.23970782312945562', time=2024-06-28T17:56:28.078975+0000, chain.objs.size()=1 2024-06-28T16:14:32.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.473__shadow_.qbvWVAiQqlwzc238jauvNgCDyNn26-k_0 2024-06-28T16:14:32.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.487__shadow_.GweOq1OxYOFRv_11Y02969l2p84muei_1_eb1c26ea550bd0b19db504399c05ffe5', time=2024-06-28T17:56:47.592961+0000, chain.objs.size()=1 2024-06-28T16:14:32.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.487__shadow_.GweOq1OxYOFRv_11Y02969l2p84muei_0 2024-06-28T16:14:32.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.493__shadow:rinwpm3WJlZ1AcGvTYYbpbEzG8SjNkf_.0OEFPT4L0L8Hzv3rUAZWjoQj5ExOwB1_1_c21b8d73e6e195f615b7adde40c98b87', time=2024-06-28T17:56:55.424504+0000, chain.objs.size()=1 2024-06-28T16:14:32.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.493__shadow:rinwpm3WJlZ1AcGvTYYbpbEzG8SjNkf_.0OEFPT4L0L8Hzv3rUAZWjoQj5ExOwB1_0 2024-06-28T16:14:32.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10403196414055266323', time=2024-06-28T18:02:08.867385+0000, chain.objs.size()=1 2024-06-28T16:14:32.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:ElgTfw48P3USMMdaQ0-W2SlisWvlRcg_.0UXM_osbTsGSEqjuhR0F4U43HzWpJYS_0 2024-06-28T16:14:32.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8323625731764680543', time=2024-06-28T18:02:10.803621+0000, chain.objs.size()=1 2024-06-28T16:14:32.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:TQjJkrZAUDZs6pqQ8Zwu1CoWQfv0b7x_.QzgXOJ05ZqK0quh4l8MVIWkT2-2Mqsn_0 2024-06-28T16:14:32.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12128690812254272972', time=2024-06-28T18:03:04.905344+0000, chain.objs.size()=1 2024-06-28T16:14:32.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:ACyXYFpZlgBujZtOk5UuVr2-FjO6lLF_.VMxw69fafLW6oR-C-zDkVDHvZe0cZNz_0 2024-06-28T16:14:32.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.875+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4326104059423469915', time=2024-06-28T18:03:11.641752+0000, chain.objs.size()=1 2024-06-28T16:14:32.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:QRs22.ZKknqysw3pMXx2A0OUOZSHufk_.sxRTSNsAQEEmecdF36qDogqsuuwmUJl_0 2024-06-28T16:14:32.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.875+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8196728289298800079', time=2024-06-28T18:03:31.659778+0000, chain.objs.size()=1 2024-06-28T16:14:32.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:kFD52.DFA2Xwhb-q32DRpqMe4NhJIUY_.uq5HG_HO8UqQwbav2Q01uYUKU3omjwe_0 2024-06-28T16:14:32.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.875+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17324596157716934409', time=2024-06-28T18:03:50.195553+0000, chain.objs.size()=4 2024-06-28T16:14:32.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.567__multipart_testobj.2~cExi3Qx3pFaSojpOad-fordiugtRCM8.1 2024-06-28T16:14:32.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.567__multipart_testobj.2~cExi3Qx3pFaSojpOad-fordiugtRCM8.2 2024-06-28T16:14:32.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.567__multipart_testobj.2~cExi3Qx3pFaSojpOad-fordiugtRCM8.3 2024-06-28T16:14:32.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.567__multipart_testobj.2~cExi3Qx3pFaSojpOad-fordiugtRCM8.4 2024-06-28T16:14:32.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.879+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8864155938063804329', time=2024-06-28T18:03:51.416424+0000, chain.objs.size()=4 2024-06-28T16:14:32.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.879+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.567__multipart_testobj.2~--NqzUCWio_Kb40FmXZHILDqb6dnGia.1 2024-06-28T16:14:32.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.879+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.567__multipart_testobj.2~--NqzUCWio_Kb40FmXZHILDqb6dnGia.2 2024-06-28T16:14:32.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.879+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.567__multipart_testobj.2~--NqzUCWio_Kb40FmXZHILDqb6dnGia.3 2024-06-28T16:14:32.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.879+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.567__multipart_testobj.2~--NqzUCWio_Kb40FmXZHILDqb6dnGia.4 2024-06-28T16:14:32.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.879+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11300445257723506189', time=2024-06-28T18:04:02.237811+0000, chain.objs.size()=1 2024-06-28T16:14:32.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.879+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.571__shadow:RkQjl00bWmKeZ8Tn0Y6Iz1WjbBs2wKz_.8z-t6K8KTDw0uR2e_bRzDLCscDVyHl9_0 2024-06-28T16:14:32.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.879+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12768331675628654379', time=2024-06-28T18:04:12.243128+0000, chain.objs.size()=1 2024-06-28T16:14:32.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.879+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.576__shadow:mdmbLKf0ejT1jOAlZu7xqh0sL8SLnNw_.Xu09MULrtMEnxWJVWoWK1uKzO5U7pSd_0 2024-06-28T16:14:32.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.879+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8616089729635803227', time=2024-06-28T18:07:56.534258+0000, chain.objs.size()=6 2024-06-28T16:14:32.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.879+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.645__multipart_bar.2~cpl5HZLZUnr20XPxhitq8Az_2HEnHrw.1 2024-06-28T16:14:32.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.645__multipart_bar.2~cpl5HZLZUnr20XPxhitq8Az_2HEnHrw.2 2024-06-28T16:14:32.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.645__multipart_bar.2~cpl5HZLZUnr20XPxhitq8Az_2HEnHrw.3 2024-06-28T16:14:32.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.645__multipart_bar.2~cpl5HZLZUnr20XPxhitq8Az_2HEnHrw.4 2024-06-28T16:14:32.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.645__multipart_bar.2~cpl5HZLZUnr20XPxhitq8Az_2HEnHrw.5 2024-06-28T16:14:32.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.645__multipart_bar.2~cpl5HZLZUnr20XPxhitq8Az_2HEnHrw.6 2024-06-28T16:14:32.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.883+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16250541896627021797', time=2024-06-28T18:08:27.983673+0000, chain.objs.size()=1 2024-06-28T16:14:32.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.677__shadow:h866i2FjFYTyJOwRQq0ZXVPbL6Zojpr_.RaYOvI1iRDfwlYAOIVJLs6Yjy9HkGJw_0 2024-06-28T16:14:32.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.887+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:32.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.891+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-06-28T16:14:32.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.891+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:32.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.891+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:32.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/53697' 2024-06-28T16:14:32.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_mySbh69wObQPuTzCO6ekSKjVCLAuMyf', time=2024-06-28T17:40:21.993405+0000, chain.objs.size()=1 2024-06-28T16:14:32.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.dXBh_OmO3GGsvWbGfjoVT2jUvBZINVo_0 2024-06-28T16:14:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6975254214560181000', time=2024-06-28T17:41:54.883006+0000, chain.objs.size()=1 2024-06-28T16:14:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.193__shadow_.zuJZMOG0ptqIhNA2vaCZRxH9-M_9i6q_0 2024-06-28T16:14:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9937688992203772030', time=2024-06-28T17:44:39.022081+0000, chain.objs.size()=1 2024-06-28T16:14:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.223__shadow_.yvisO9ndK7ryo-hJ1Altt7mILapqnJo_0 2024-06-28T16:14:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7959935126799457163', time=2024-06-28T17:44:56.358112+0000, chain.objs.size()=1 2024-06-28T16:14:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.233__shadow_.F-shEayVL8KYL_6D0SQzG8zAnlsbyTg_0 2024-06-28T16:14:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1711296473833181221', time=2024-06-28T17:45:16.241475+0000, chain.objs.size()=1 2024-06-28T16:14:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.245__shadow_.40TMghZUxZRY4Lr16D2AMSGprNn5R39_0 2024-06-28T16:14:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4912921224199339744', time=2024-06-28T17:45:22.315614+0000, chain.objs.size()=1 2024-06-28T16:14:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.249__shadow_.9-toEDylcaD3SxL-tsBNdFEAxMqXFGi_0 2024-06-28T16:14:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7763708907142410606', time=2024-06-28T17:47:07.806820+0000, chain.objs.size()=1 2024-06-28T16:14:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KhbN11yAIcdC_cYb7rsjS1vI9C31hnk_0 2024-06-28T16:14:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2719883058319151633', time=2024-06-28T17:47:09.200379+0000, chain.objs.size()=1 2024-06-28T16:14:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.M3dP_boU7MfsOxrd5hUg985TFH4Ofhc_0 2024-06-28T16:14:32.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3824984272482598026', time=2024-06-28T17:47:13.621979+0000, chain.objs.size()=1 2024-06-28T16:14:32.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ZULpJMTvtoENCd2-ogzznKcDzABOncb_0 2024-06-28T16:14:32.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6235702857566771704', time=2024-06-28T17:47:13.741956+0000, chain.objs.size()=1 2024-06-28T16:14:32.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.1nZmULDQlpBDWJT515yqXPzkM2wpzI9_0 2024-06-28T16:14:32.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7175467741258573807', time=2024-06-28T17:47:13.743893+0000, chain.objs.size()=1 2024-06-28T16:14:32.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-TdnZL21E_iq-UYtov2UNNDCyUqEtdZ_0 2024-06-28T16:14:32.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6408956545464030608', time=2024-06-28T17:47:13.774160+0000, chain.objs.size()=1 2024-06-28T16:14:32.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.6hG6mh54UeOmVb2yePfaWZ6PMN88GPe_0 2024-06-28T16:14:32.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17730820214639392950', time=2024-06-28T17:47:14.190912+0000, chain.objs.size()=1 2024-06-28T16:14:32.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.pwebSCxWLjOFGQqG-vmuc4Y1O14WR93_0 2024-06-28T16:14:32.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3189862003466667775', time=2024-06-28T17:47:14.595732+0000, chain.objs.size()=1 2024-06-28T16:14:32.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Np2qfM3ZtLbBD6ljFeu32_Y27UTvgz8_0 2024-06-28T16:14:32.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15726598788405337148', time=2024-06-28T17:47:14.688702+0000, chain.objs.size()=1 2024-06-28T16:14:32.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.GtLRI-V8US-z_Z6hDIUnlIEktkuONFC_0 2024-06-28T16:14:32.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11223680113815737481', time=2024-06-28T17:47:15.220999+0000, chain.objs.size()=1 2024-06-28T16:14:32.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.FidZu0ka4liXcxGigAucHDsyrEt4OPG_0 2024-06-28T16:14:32.908 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13158416551528192323', time=2024-06-28T17:47:15.683883+0000, chain.objs.size()=1 2024-06-28T16:14:32.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dSO-5mdbJfm0UvTIjT53xOBGp92kfmB_0 2024-06-28T16:14:32.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9959684866527490301', time=2024-06-28T17:47:17.199439+0000, chain.objs.size()=1 2024-06-28T16:14:32.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KZA1NMhWUrPegEhHXn8Olo7pfO4c9M1_0 2024-06-28T16:14:32.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18340110239359631397', time=2024-06-28T17:47:17.497866+0000, chain.objs.size()=1 2024-06-28T16:14:32.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.UFQ-0rtZt_aJc26H3MPbHTBZmMj-R9g_0 2024-06-28T16:14:32.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16451864570801931821', time=2024-06-28T17:47:17.930361+0000, chain.objs.size()=1 2024-06-28T16:14:32.909 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.HTTaWKWnvqYlPJxj6rhQXREyoN5T5yw_0 2024-06-28T16:14:32.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17522948632605133885', time=2024-06-28T17:47:18.214532+0000, chain.objs.size()=1 2024-06-28T16:14:32.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_._HQlBkM4mF_AgJYsDDyT1L5Yb0gk11U_0 2024-06-28T16:14:32.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17295900090569437800', time=2024-06-28T17:47:18.469723+0000, chain.objs.size()=1 2024-06-28T16:14:32.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.z5EV6zmF3mGJojnoGmNE26JQB_Ryfqm_0 2024-06-28T16:14:32.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9243159230585105368', time=2024-06-28T17:47:18.612791+0000, chain.objs.size()=1 2024-06-28T16:14:32.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KHrN74l_twzwp4gAZsk7RsV7LXqSARC_0 2024-06-28T16:14:32.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11732008129910071177', time=2024-06-28T17:47:19.807684+0000, chain.objs.size()=1 2024-06-28T16:14:32.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.EBZztiGdLW7q_-3vYQjgwLukzI_xODQ_0 2024-06-28T16:14:32.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.903+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16661879094145880421', time=2024-06-28T17:47:20.837961+0000, chain.objs.size()=1 2024-06-28T16:14:32.914 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.903+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.UiOy8fXnOqQahFiszA1FEiion_M-Lw0_0 2024-06-28T16:14:32.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.64208551990788369', time=2024-06-28T17:47:21.341926+0000, chain.objs.size()=1 2024-06-28T16:14:32.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.xg8ZNsPu2GDqOi9ndMJQZYv_wxsML_1_0 2024-06-28T16:14:32.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2343505744384467032', time=2024-06-28T17:47:26.855330+0000, chain.objs.size()=1 2024-06-28T16:14:32.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.hVYXSiUhnLcAUUNuw7-dkgm_eN1xiat_0 2024-06-28T16:14:32.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9901985575242704493', time=2024-06-28T17:47:28.117616+0000, chain.objs.size()=1 2024-06-28T16:14:32.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.gqx6I8whiKyP7rSb2NyxA5tmz2jQAXs_0 2024-06-28T16:14:32.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3986915674413311907', time=2024-06-28T17:47:29.502341+0000, chain.objs.size()=1 2024-06-28T16:14:32.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.TyLn3YaxD0bgPGPZDcW0tvFeRewDZbh_0 2024-06-28T16:14:32.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3134835428968323884', time=2024-06-28T17:47:29.809299+0000, chain.objs.size()=1 2024-06-28T16:14:32.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.IAULRqSBeutYbZesB_BvLSzhDmzD3dJ_0 2024-06-28T16:14:32.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7329061736822549921', time=2024-06-28T17:47:30.835433+0000, chain.objs.size()=1 2024-06-28T16:14:32.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.bjr9mRebelYlLHOUSs1jbZFiE8Pv0kr_0 2024-06-28T16:14:32.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4258083655473379914', time=2024-06-28T17:47:32.179646+0000, chain.objs.size()=1 2024-06-28T16:14:32.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.d4fhQVGoy4JW5JBumUtfdMdh-GE2YwH_0 2024-06-28T16:14:32.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4465567940095924529', time=2024-06-28T17:47:33.876042+0000, chain.objs.size()=1 2024-06-28T16:14:32.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.1FbE01AFIw-L459Db3o6x4Hvc-5b6aX_0 2024-06-28T16:14:32.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4912531004039508761', time=2024-06-28T17:47:35.406262+0000, chain.objs.size()=1 2024-06-28T16:14:32.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.PnVlx0cXKaiZj5jh2vvnUSkbZzs6yz0_0 2024-06-28T16:14:32.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12533440071093049280', time=2024-06-28T17:47:35.410168+0000, chain.objs.size()=1 2024-06-28T16:14:32.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4GEz35dU-JG_U4eix4eWPmFOgHrB3xx_0 2024-06-28T16:14:32.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10119361470292419236', time=2024-06-28T17:47:37.751613+0000, chain.objs.size()=1 2024-06-28T16:14:32.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ynXO5P0IJPXTkvtRABJnXs7KOFgjbct_0 2024-06-28T16:14:32.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3685767255577234380', time=2024-06-28T17:47:38.324821+0000, chain.objs.size()=1 2024-06-28T16:14:32.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ses4-p5Pn8tntxOJRPn-hrB0Xt4LfJQ_0 2024-06-28T16:14:32.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5529173462568627965', time=2024-06-28T17:47:39.053878+0000, chain.objs.size()=1 2024-06-28T16:14:32.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.w42nq6LGgq3h9Qh9dWLM61ztNSt41s-_0 2024-06-28T16:14:32.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8410107686005246538', time=2024-06-28T17:47:39.269050+0000, chain.objs.size()=1 2024-06-28T16:14:32.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.vDOQ4PXKH-VWwbwhHwtPGB2ECcLG_JY_0 2024-06-28T16:14:32.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8820708479848715608', time=2024-06-28T17:47:39.499434+0000, chain.objs.size()=1 2024-06-28T16:14:32.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KZRFdyq1xFWR5ExUDrQS06_hq0U1UA3_0 2024-06-28T16:14:32.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2297041398075302284', time=2024-06-28T17:47:39.589430+0000, chain.objs.size()=1 2024-06-28T16:14:32.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.MmI37bbJxdZTy_6pggYr907o2IHPqhK_0 2024-06-28T16:14:32.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14721658475003757062', time=2024-06-28T17:48:06.935592+0000, chain.objs.size()=1 2024-06-28T16:14:32.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.270__shadow_.dUMj-4gjB3VRBWXLQ0xGl8SWuBk2kCN_0 2024-06-28T16:14:32.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13238409648464559710', time=2024-06-28T17:48:34.634117+0000, chain.objs.size()=1 2024-06-28T16:14:32.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.CwvX5ne6YdHIH4Rcn5UbkMPUAG7wIiG_0 2024-06-28T16:14:32.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17779425855125399782', time=2024-06-28T17:48:34.887890+0000, chain.objs.size()=1 2024-06-28T16:14:32.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.mTLaeb1EYsN-4r09yjsutPzY8hi7NAS_0 2024-06-28T16:14:32.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8914712389436438533', time=2024-06-28T17:48:51.180323+0000, chain.objs.size()=1 2024-06-28T16:14:32.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.293__shadow_.ZwZFr4TwTkNqkAEJTXn4AlRVf8mP7L__0 2024-06-28T16:14:32.923 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4262479777880979674', time=2024-06-28T17:49:16.720460+0000, chain.objs.size()=1 2024-06-28T16:14:32.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.309__shadow:zPNMNzjHSLZgnRkEOuEWqbD3osksTaM_.GRL-Hd4awiHAKgPvRb6dsetGsCIH5cy_0 2024-06-28T16:14:32.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8049539110756278065', time=2024-06-28T17:51:06.762942+0000, chain.objs.size()=1 2024-06-28T16:14:32.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YyS43yHtLOlA-zOjVtDWJlZ5f-ZJ04j_0 2024-06-28T16:14:32.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7483555595206396137', time=2024-06-28T17:51:07.136354+0000, chain.objs.size()=1 2024-06-28T16:14:32.924 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.lMZ2jJkPL8V4hd4MsVbtTgrwGxVbmRP_0 2024-06-28T16:14:32.925 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15194876138599642772', time=2024-06-28T17:51:07.563380+0000, chain.objs.size()=1 2024-06-28T16:14:32.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xcxYcchOQM2JN2m7y5I36ub5nTNYnLX_0 2024-06-28T16:14:32.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2032086466541202771', time=2024-06-28T17:51:08.003801+0000, chain.objs.size()=1 2024-06-28T16:14:32.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-ttsXhKhXD0NmBoHeS6VHgLvfAqEFox_0 2024-06-28T16:14:32.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14518768842763190698', time=2024-06-28T17:51:08.064044+0000, chain.objs.size()=1 2024-06-28T16:14:32.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.DJVugUQ1jBvFaT9aLNZFyTNLCMB97vP_0 2024-06-28T16:14:32.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11943359596841105775', time=2024-06-28T17:51:08.552354+0000, chain.objs.size()=1 2024-06-28T16:14:32.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YhCGeUhiWbEZxMjBtGBO0cfwq3M6Xz1_0 2024-06-28T16:14:32.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12599991900415753036', time=2024-06-28T17:51:08.668138+0000, chain.objs.size()=1 2024-06-28T16:14:32.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.LdR47qZBwIHYzS-y42fOIRcNhiLE06f_0 2024-06-28T16:14:32.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4992340320034608861', time=2024-06-28T17:51:14.136849+0000, chain.objs.size()=1 2024-06-28T16:14:32.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.SIjWAjGf2oaEXBx7MCRvaKHNeRPVlAA_0 2024-06-28T16:14:32.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5600861303367556254', time=2024-06-28T17:51:15.406189+0000, chain.objs.size()=1 2024-06-28T16:14:32.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.q1j_no3sXNBfVal70oMOkStisfrxAZQ_0 2024-06-28T16:14:32.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8730202691731615092', time=2024-06-28T17:51:17.783581+0000, chain.objs.size()=1 2024-06-28T16:14:32.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YwWR7iwpaF3lhK0I73Da4P5t_gjRfiZ_0 2024-06-28T16:14:32.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.919+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9994145651678866241', time=2024-06-28T17:51:18.588320+0000, chain.objs.size()=1 2024-06-28T16:14:32.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.919+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.4rydBXT0k3mUx7FAFbL5aX_9Y4_07Q5_0 2024-06-28T16:14:32.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.919+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12704046964681817624', time=2024-06-28T17:51:19.860006+0000, chain.objs.size()=1 2024-06-28T16:14:32.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.919+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.CuysYacKDO1l-zvXirFUU3-uNpU3Wry_0 2024-06-28T16:14:32.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.919+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10991207683573091307', time=2024-06-28T17:51:21.278595+0000, chain.objs.size()=1 2024-06-28T16:14:32.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.919+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-_VNk1EaK0yBOBIzryzSCHVZILCUCXK_0 2024-06-28T16:14:32.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16551982538734741752', time=2024-06-28T17:51:21.825274+0000, chain.objs.size()=1 2024-06-28T16:14:32.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.WeZQq_0Ed9l2Mb8HFaY1wUWZKU_gC3o_0 2024-06-28T16:14:32.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17901559770068662175', time=2024-06-28T17:51:22.283769+0000, chain.objs.size()=1 2024-06-28T16:14:32.931 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nceHpj0dh1jNkSu0VNVyJNOJJgLjUNg_0 2024-06-28T16:14:32.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13881762419127193742', time=2024-06-28T17:51:23.467548+0000, chain.objs.size()=1 2024-06-28T16:14:32.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GtC49y92sxSUuqOw9sMD4rgQjNrQuHt_0 2024-06-28T16:14:32.932 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17211691735861122766', time=2024-06-28T17:51:24.125443+0000, chain.objs.size()=1 2024-06-28T16:14:32.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.a8F-6QyMENVwqTXxDbI3moBRfRhyMNH_0 2024-06-28T16:14:32.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16458002749872123729', time=2024-06-28T17:51:24.960881+0000, chain.objs.size()=1 2024-06-28T16:14:32.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.UspXLcXV5cZWyVQcCUgqqDFaEHkcUEY_0 2024-06-28T16:14:32.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15502601767203539427', time=2024-06-28T17:51:26.208877+0000, chain.objs.size()=1 2024-06-28T16:14:32.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hEWbL--ml6JsrlzAc6cMwEhEwKcXEYs_0 2024-06-28T16:14:32.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14114237623940425192', time=2024-06-28T17:51:28.283366+0000, chain.objs.size()=1 2024-06-28T16:14:32.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.gHsojVZfkepGkDFApOxJJKEDE6_6Ca__0 2024-06-28T16:14:32.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16205995431432521313', time=2024-06-28T17:51:29.557547+0000, chain.objs.size()=1 2024-06-28T16:14:32.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.LPyxdR_GhxauRn6_2Sq-KEruMv1x-bH_0 2024-06-28T16:14:32.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10724093281418349130', time=2024-06-28T17:51:30.201346+0000, chain.objs.size()=1 2024-06-28T16:14:32.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.jIa22lNNak5_31KwK8khBSY8Gq2WQE9_0 2024-06-28T16:14:32.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11869952535899419346', time=2024-06-28T17:51:30.891362+0000, chain.objs.size()=1 2024-06-28T16:14:32.934 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.923+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.pqofhZBbEbAtmaNTxr3ioHGyoQlMlXd_0 2024-06-28T16:14:32.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9303676250771939177', time=2024-06-28T17:51:31.068671+0000, chain.objs.size()=1 2024-06-28T16:14:32.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.eb6zxw5POXjpN4LAbFVKV43hqXvqQkg_0 2024-06-28T16:14:32.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9212436161382392593', time=2024-06-28T17:51:31.874435+0000, chain.objs.size()=1 2024-06-28T16:14:32.935 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.BDU3cagEdrnp-0gCmyg1xDbpygwmoSK_0 2024-06-28T16:14:32.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9093870580437063444', time=2024-06-28T17:51:34.008379+0000, chain.objs.size()=1 2024-06-28T16:14:32.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yS9aTU-5jQdH4tseNmKfoulnFKS8gbb_0 2024-06-28T16:14:32.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2733305612613867822', time=2024-06-28T17:51:34.938541+0000, chain.objs.size()=1 2024-06-28T16:14:32.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.IXROO3SSKCXU-s8YC0kFBR9gq1PXYkN_0 2024-06-28T16:14:32.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3859004620323533225', time=2024-06-28T17:51:35.855142+0000, chain.objs.size()=1 2024-06-28T16:14:32.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.5dx8NqMENJbyms0-tT8WKLvOxxcdR5F_0 2024-06-28T16:14:32.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11666248230034452937', time=2024-06-28T17:51:37.986834+0000, chain.objs.size()=1 2024-06-28T16:14:32.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6MO2VlVT0IEfy3yKzD3VCapWsbgnEDz_0 2024-06-28T16:14:32.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13695346971537731421', time=2024-06-28T17:51:37.988441+0000, chain.objs.size()=1 2024-06-28T16:14:32.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.0PQg0ZtPctND9md-vnOmZTFbpCmXEAB_0 2024-06-28T16:14:32.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1030879335714829568', time=2024-06-28T17:51:38.319555+0000, chain.objs.size()=1 2024-06-28T16:14:32.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nE2n21adF0L4Ky3VC3ug87uYeCMzWtM_0 2024-06-28T16:14:32.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2864938523361464265', time=2024-06-28T17:53:22.936541+0000, chain.objs.size()=1 2024-06-28T16:14:32.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.3EmYPe1fs_oMX1b7q0Il3mdU9OR1Ja5_0 2024-06-28T16:14:32.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13502506445260091364', time=2024-06-28T17:53:23.557466+0000, chain.objs.size()=1 2024-06-28T16:14:32.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.nv4kbLtUToyMbSSYTNrh2wGnjI7xJoC_0 2024-06-28T16:14:32.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.506189161478023026', time=2024-06-28T17:53:23.790792+0000, chain.objs.size()=1 2024-06-28T16:14:32.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.hfzAw_nSRpe2RttCFH8G_CvZiUXJBgn_0 2024-06-28T16:14:32.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.931+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.793935592631222745', time=2024-06-28T17:53:24.003091+0000, chain.objs.size()=1 2024-06-28T16:14:32.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.XwLYUUmY51lv5B_wC5retspuy7fYym0_0 2024-06-28T16:14:32.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.931+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4552358737114138719', time=2024-06-28T17:53:26.147741+0000, chain.objs.size()=1 2024-06-28T16:14:32.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.pcoN6O8_8MMx0mpO9MhlbDd15hADPLG_0 2024-06-28T16:14:32.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.931+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15161476091969606135', time=2024-06-28T17:53:27.838222+0000, chain.objs.size()=1 2024-06-28T16:14:32.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vuOgHW35oepyMnwBJTX67e8wAcHrtoI_0 2024-06-28T16:14:32.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.931+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3398661990749334571', time=2024-06-28T17:53:30.005940+0000, chain.objs.size()=1 2024-06-28T16:14:32.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qRtPPXcBAn2qFamfU9K3urtBUh3Sjwy_0 2024-06-28T16:14:32.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.931+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6636824618770134618', time=2024-06-28T17:53:30.761410+0000, chain.objs.size()=1 2024-06-28T16:14:32.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.dvI1pWc7rMoh-Sqe1GjNVlcCyEpKhOC_0 2024-06-28T16:14:32.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.931+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8263392117212645950', time=2024-06-28T17:53:31.125102+0000, chain.objs.size()=1 2024-06-28T16:14:32.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.aES3saHyYXjMMYecoW_NsF1hdcxf2R8_0 2024-06-28T16:14:32.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17036984978289781889', time=2024-06-28T17:53:32.148515+0000, chain.objs.size()=1 2024-06-28T16:14:32.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-SzPV22M63PYYY1rNVGcSvvv_3nzMuF_0 2024-06-28T16:14:32.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14165037756484462212', time=2024-06-28T17:53:32.428651+0000, chain.objs.size()=1 2024-06-28T16:14:32.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gtasRbUKxNMgtzJVEYlIM7ILAYuHGDU_0 2024-06-28T16:14:32.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3811602447681490241', time=2024-06-28T17:53:34.419096+0000, chain.objs.size()=1 2024-06-28T16:14:32.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.cce_P9TkLQLKkHVC2B41-uNJIAuQUBM_0 2024-06-28T16:14:32.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12447722914778862405', time=2024-06-28T17:53:35.988347+0000, chain.objs.size()=1 2024-06-28T16:14:32.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.0uhhAIfc-8MhNKWlV8b_i9dpmdV3nOi_0 2024-06-28T16:14:32.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11706771756860090995', time=2024-06-28T17:53:36.855983+0000, chain.objs.size()=1 2024-06-28T16:14:32.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.PqdpCpAB2C9pmYNLLG57p5nuPbKGPhJ_0 2024-06-28T16:14:32.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6112521224397433353', time=2024-06-28T17:53:37.728851+0000, chain.objs.size()=1 2024-06-28T16:14:32.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ycz0aQ9ruqjYQ7mdNGX4lV9hfh8zLHq_0 2024-06-28T16:14:32.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16891416191919154149', time=2024-06-28T17:53:37.817096+0000, chain.objs.size()=1 2024-06-28T16:14:32.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.RFIv6kGeakLEfQyOdpaJm-ZbLaDszQE_0 2024-06-28T16:14:32.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17095871721360119459', time=2024-06-28T17:53:38.478099+0000, chain.objs.size()=1 2024-06-28T16:14:32.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Hgw44d736P3boHOmnSXyfGWtKNYAzNb_0 2024-06-28T16:14:32.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.935+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5319181683401993894', time=2024-06-28T17:53:38.660294+0000, chain.objs.size()=1 2024-06-28T16:14:32.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.939+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.6ERbto1ItGEi6dOcSYfTEJN8FvctNCf_0 2024-06-28T16:14:32.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.939+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9650986022930750423', time=2024-06-28T17:53:38.664964+0000, chain.objs.size()=1 2024-06-28T16:14:32.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.939+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.v4-I0sAacRmofhLBQx6WCo3yz-REUiJ_0 2024-06-28T16:14:32.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.939+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1604250871303615722', time=2024-06-28T17:53:40.115159+0000, chain.objs.size()=1 2024-06-28T16:14:32.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.939+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.XvQK88VfNQrg09og59js7gLyqO0_2Zb_0 2024-06-28T16:14:32.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.939+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2768779015096398629', time=2024-06-28T17:53:40.495377+0000, chain.objs.size()=1 2024-06-28T16:14:32.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.939+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CZ9vAG7BYlrIMC5gnw9e3Ukxku4rJPF_0 2024-06-28T16:14:32.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.939+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14300726763007915401', time=2024-06-28T17:53:41.830841+0000, chain.objs.size()=1 2024-06-28T16:14:32.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.939+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Lt_m8XFf6pIg5dfSEztw0EIIVy6o-iJ_0 2024-06-28T16:14:32.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.939+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17380694994224014789', time=2024-06-28T17:53:42.420967+0000, chain.objs.size()=1 2024-06-28T16:14:32.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.939+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qZBZuU707G5DOTnp9QRh14iuHdJtxl4_0 2024-06-28T16:14:32.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.943+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/53697 2024-06-28T16:14:32.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=35, truncated=0, next_marker='' 2024-06-28T16:14:32.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3453316942441753653', time=2024-06-28T17:53:42.423052+0000, chain.objs.size()=1 2024-06-28T16:14:32.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Gdmv_I5A5h9lcHiF6Y0VBydCTqN-EwZ_0 2024-06-28T16:14:32.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6786775684760425274', time=2024-06-28T17:53:42.500590+0000, chain.objs.size()=1 2024-06-28T16:14:32.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.0mcYSXC5LpotCN-40C0GrLDnRrgHGi3_0 2024-06-28T16:14:32.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.151637274756102086', time=2024-06-28T17:53:43.110995+0000, chain.objs.size()=1 2024-06-28T16:14:32.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.a9Dc2t_tZ-T8Y0C6bvQppNTi-irX8Qg_0 2024-06-28T16:14:32.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12898298409494681782', time=2024-06-28T17:53:43.376556+0000, chain.objs.size()=1 2024-06-28T16:14:32.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.v2cdfpZ8Jiow8vIT-SqZlD58wFYtR_w_0 2024-06-28T16:14:32.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10197160502523329776', time=2024-06-28T17:53:44.476477+0000, chain.objs.size()=1 2024-06-28T16:14:32.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.yMDzQf2630fCI989iLcxzwl4iFycCj7_0 2024-06-28T16:14:32.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17066346510730187297', time=2024-06-28T17:53:47.002456+0000, chain.objs.size()=1 2024-06-28T16:14:32.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.IVN_aW90-a2z2N2y-PljTX7GmtS2oEs_0 2024-06-28T16:14:32.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15405069007561212116', time=2024-06-28T17:53:48.419722+0000, chain.objs.size()=1 2024-06-28T16:14:32.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.SSXgWL2NHHDZVKrUAREwf9C7AFxhRVd_0 2024-06-28T16:14:32.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15832991679647184215', time=2024-06-28T17:53:51.842947+0000, chain.objs.size()=1 2024-06-28T16:14:32.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qMvtg07Bf4Id1ZCQ_0YtLBrtZxhNUWg_0 2024-06-28T16:14:32.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3196004387342729851', time=2024-06-28T17:53:52.007414+0000, chain.objs.size()=1 2024-06-28T16:14:32.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.DihVqtLuno-ZOPi0PPU527EDYBgb_i3_0 2024-06-28T16:14:32.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5159591067702703532', time=2024-06-28T17:53:56.665879+0000, chain.objs.size()=1 2024-06-28T16:14:32.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.319__shadow_.UyLCem4tWI8oTWXlfvmdGTwrgs_yOOu_0 2024-06-28T16:14:32.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12361892851480875408', time=2024-06-28T17:54:13.085520+0000, chain.objs.size()=1 2024-06-28T16:14:32.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.336__shadow_.c6ntVzqBz7OXbhFBvGqCKiret8GeFBb_0 2024-06-28T16:14:32.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7280942043476583806', time=2024-06-28T17:54:33.557909+0000, chain.objs.size()=1 2024-06-28T16:14:32.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.363__shadow_.LEkA6F6gBbgjXdomJxg69XbuajlLSp4_0 2024-06-28T16:14:32.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.951+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15779537182855559810', time=2024-06-28T17:54:43.153453+0000, chain.objs.size()=1 2024-06-28T16:14:32.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.951+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.371__shadow_.NZjpWRgOEbnSIQc50KShoe5JiGG8L0M_0 2024-06-28T16:14:32.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.951+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14069638364594788156', time=2024-06-28T17:55:40.343201+0000, chain.objs.size()=1 2024-06-28T16:14:32.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.951+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.432__shadow_.fA_shqam5d4P8hCzFl4eiKAI0UQgNoH_0 2024-06-28T16:14:32.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.951+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8975447013696691185', time=2024-06-28T17:55:54.828303+0000, chain.objs.size()=1 2024-06-28T16:14:32.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.951+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.445__shadow_.a5gVU8Tbga1obHookZiA2E2_U8fyeui_0 2024-06-28T16:14:32.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.951+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6040986126715725142', time=2024-06-28T17:56:23.218955+0000, chain.objs.size()=1 2024-06-28T16:14:32.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.951+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.471__shadow_.JGzsPvn1NxvplSqJpHNZ3lPlGFkW-Yy_0 2024-06-28T16:14:32.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.951+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~TBGerqkZshVo8n23G05y_XoR1ruNatp', time=2024-06-28T17:59:25.083180+0000, chain.objs.size()=1 2024-06-28T16:14:32.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.951+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~TBGerqkZshVo8n23G05y_XoR1ruNatp.1 2024-06-28T16:14:32.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.951+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~TBGerqkZshVo8n23G05y_XoR1ruNatp', time=2024-06-28T17:59:25.092553+0000, chain.objs.size()=1 2024-06-28T16:14:32.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.951+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~TBGerqkZshVo8n23G05y_XoR1ruNatp.2 2024-06-28T16:14:32.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.955+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~TBGerqkZshVo8n23G05y_XoR1ruNatp', time=2024-06-28T17:59:25.101757+0000, chain.objs.size()=1 2024-06-28T16:14:32.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.955+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~TBGerqkZshVo8n23G05y_XoR1ruNatp.3 2024-06-28T16:14:32.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.955+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~TBGerqkZshVo8n23G05y_XoR1ruNatp', time=2024-06-28T17:59:25.110866+0000, chain.objs.size()=1 2024-06-28T16:14:32.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.955+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~TBGerqkZshVo8n23G05y_XoR1ruNatp.4 2024-06-28T16:14:32.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.955+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~TBGerqkZshVo8n23G05y_XoR1ruNatp', time=2024-06-28T17:59:25.119878+0000, chain.objs.size()=1 2024-06-28T16:14:32.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.955+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~TBGerqkZshVo8n23G05y_XoR1ruNatp.5 2024-06-28T16:14:32.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.955+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~TBGerqkZshVo8n23G05y_XoR1ruNatp', time=2024-06-28T17:59:25.128950+0000, chain.objs.size()=1 2024-06-28T16:14:32.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.955+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~TBGerqkZshVo8n23G05y_XoR1ruNatp.6 2024-06-28T16:14:32.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.955+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~sFlpCMEE4Hd16BJpgPigUsjOFBgyRCi', time=2024-06-28T18:00:25.378308+0000, chain.objs.size()=1 2024-06-28T16:14:32.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.955+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.524__multipart_mymultipart.2~sFlpCMEE4Hd16BJpgPigUsjOFBgyRCi.1 2024-06-28T16:14:32.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.955+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11196923329772453346', time=2024-06-28T18:01:26.053270+0000, chain.objs.size()=2 2024-06-28T16:14:32.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.955+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.544__shadow_.OnjR5EKaEQ65j0YJUMU2eImMZo3hRkw_0 2024-06-28T16:14:32.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.955+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.544__shadow_.OnjR5EKaEQ65j0YJUMU2eImMZo3hRkw_1 2024-06-28T16:14:32.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.959+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16270502827438848709', time=2024-06-28T18:01:42.344791+0000, chain.objs.size()=1 2024-06-28T16:14:32.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.959+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.549__shadow_.0Su7uE7gqZ0e8s9-N3Y3a40QlZND8T2_0 2024-06-28T16:14:32.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.959+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5606583981902567657', time=2024-06-28T18:02:12.122829+0000, chain.objs.size()=1 2024-06-28T16:14:32.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.959+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:ji8nXfvjlNmEkg7PurG6YqSDBimC3Ph_.5XeJvZk5tI6KHpyyi_J4JzBAwtBTIDE_0 2024-06-28T16:14:32.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.959+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2594637662181842663', time=2024-06-28T18:02:13.706364+0000, chain.objs.size()=1 2024-06-28T16:14:32.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.959+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:7ZXwXXRVRjxAt8dRY3Ffx-33NbUZQu7_.fSRd-zvCIyXPyFNCpjtAicFcta5oXCJ_0 2024-06-28T16:14:32.969 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.959+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4289958029734002748', time=2024-06-28T18:02:27.224245+0000, chain.objs.size()=1 2024-06-28T16:14:32.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.959+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.562__shadow_.VxbRUa6u0xXqcG9LXp_CN60afoMMSBq_0 2024-06-28T16:14:32.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.959+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12214130213007335668', time=2024-06-28T18:02:30.175812+0000, chain.objs.size()=1 2024-06-28T16:14:32.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.959+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.563__shadow_.xXrRJ3UPWAZVnIkGoDdP5hXmlfg0-Ic_0 2024-06-28T16:14:32.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.959+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2058220720268151742', time=2024-06-28T18:02:57.118522+0000, chain.objs.size()=1 2024-06-28T16:14:32.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.959+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:SVZ9GBMLxoC3ICEwHFL.fugaW.nCR.m_.RFDzwFQdRmRBzBp5BblNcjJIeKVVSXk_0 2024-06-28T16:14:32.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.959+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7060207657435717520', time=2024-06-28T18:03:26.902032+0000, chain.objs.size()=1 2024-06-28T16:14:32.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.959+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:Q9RsZk7gkZX6ceiuwYkguJqcBw64x0i_.pKPsNnkvruQ3KCzVGpeI3lC9WGPNCCE_0 2024-06-28T16:14:32.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.963+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3379451985010943593', time=2024-06-28T18:04:08.317111+0000, chain.objs.size()=1 2024-06-28T16:14:32.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.963+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.574__shadow:wqn8dT1cjaUi9py1uxv6mA0trT-tzIh_.4Bi5eHhFhYeacn4kxbMHratvUAOVrFz_0 2024-06-28T16:14:32.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.967+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9725767282297398300', time=2024-06-28T18:04:13.965341+0000, chain.objs.size()=1 2024-06-28T16:14:32.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.967+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.576__shadow:6k9HgrHVPr1HoR6e7VoLvcRHFiAWxMS_.ffHBlErfsTHB42dHBhCKco563jpJ6d5_0 2024-06-28T16:14:32.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.967+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3825461949598394240', time=2024-06-28T18:07:10.321693+0000, chain.objs.size()=1 2024-06-28T16:14:32.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.967+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.630__shadow_.KecdcPO053fy-oTfdR8SZfOSHdw8AJO_0 2024-06-28T16:14:32.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.967+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6522799145842777973', time=2024-06-28T18:08:52.239221+0000, chain.objs.size()=1 2024-06-28T16:14:32.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.967+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.697__shadow:DNSwLEg3dGZJ9otsYMggNLLb7-sfbKF_.XiCI_p5GR1-9sWqHcsiwbzYqH2S53cm_0 2024-06-28T16:14:32.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.971+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:32.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.971+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-06-28T16:14:32.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:32.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:32.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/47708' 2024-06-28T16:14:32.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_macBfH-iQ5gxaflNhT1C1DybdEhFjgM', time=2024-06-28T17:40:34.181017+0000, chain.objs.size()=2 2024-06-28T16:14:32.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.151__shadow_.V_98qtF8lTjt7pp4DE2rR7Yy1Cbs9kA_0 2024-06-28T16:14:32.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.151__shadow_.V_98qtF8lTjt7pp4DE2rR7Yy1Cbs9kA_1 2024-06-28T16:14:32.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_FKYSz9XRlKfBd6KAxm1ncKWw2TcIUuO', time=2024-06-28T17:40:37.661870+0000, chain.objs.size()=1 2024-06-28T16:14:32.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.153__shadow_.WvWo8pZcnyeKqIcOvm3JHxmiuSH9in5_0 2024-06-28T16:14:32.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9956771895980334944', time=2024-06-28T17:44:43.822313+0000, chain.objs.size()=1 2024-06-28T16:14:32.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.226__shadow_.nplRvI5zlh1qrDotPbF1MQs2l1Ls2AW_0 2024-06-28T16:14:32.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.622139930839124950', time=2024-06-28T17:44:46.055438+0000, chain.objs.size()=1 2024-06-28T16:14:32.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.227__shadow_.wV5J2hgY2A49SWgzDduhx07JMbBeUiE_0 2024-06-28T16:14:32.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13715749570177775449', time=2024-06-28T17:44:48.464533+0000, chain.objs.size()=1 2024-06-28T16:14:32.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.228__shadow_.jwHFxMiZuKjhtiHep5hihHlOOFAXT7g_0 2024-06-28T16:14:32.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17581655335679993294', time=2024-06-28T17:44:52.513947+0000, chain.objs.size()=1 2024-06-28T16:14:32.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.231__shadow_.TVSZx7ACfDQZSuiykPkbGG-HqTGSgBv_0 2024-06-28T16:14:32.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4607209480717875523', time=2024-06-28T17:45:00.227769+0000, chain.objs.size()=1 2024-06-28T16:14:32.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.235__shadow_.3pfzqjMon5B0J0z8mXId0v8K0LrHcN-_0 2024-06-28T16:14:32.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1678821987013435284', time=2024-06-28T17:45:14.649518+0000, chain.objs.size()=1 2024-06-28T16:14:32.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.244__shadow_.mvoo99hOib2t-uGJ-N7ZqO2sJqcreO6_0 2024-06-28T16:14:32.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5682280598304870837', time=2024-06-28T17:45:14.664168+0000, chain.objs.size()=1 2024-06-28T16:14:32.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.244__shadow_.IVxAs7Sz2SgHGqdCPfwW2ClU0RsXOib_0 2024-06-28T16:14:32.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2903868123810757209', time=2024-06-28T17:45:23.893978+0000, chain.objs.size()=1 2024-06-28T16:14:32.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.250__shadow_.ChkiGyhkoM7imvO3w--xp_VhKNE3ri4_0 2024-06-28T16:14:32.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7192311633145198584', time=2024-06-28T17:47:07.964432+0000, chain.objs.size()=1 2024-06-28T16:14:32.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.w2lbEfUaueYck6yGYmd8y4N0uli-9wV_0 2024-06-28T16:14:32.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.979+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7844586754937977140', time=2024-06-28T17:47:08.728235+0000, chain.objs.size()=1 2024-06-28T16:14:32.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.979+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.DugYCs8IV7ZfKJEahpnAX9eNNN0GJI8_0 2024-06-28T16:14:32.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.979+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12893573532685199999', time=2024-06-28T17:47:08.729745+0000, chain.objs.size()=1 2024-06-28T16:14:32.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.979+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.RzW_8x4vEqpje0zGqOWVrSb6J_xKacv_0 2024-06-28T16:14:32.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.979+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12319720759424883363', time=2024-06-28T17:47:08.802391+0000, chain.objs.size()=1 2024-06-28T16:14:32.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.979+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7aqyBfuw_WGWav82jiJ01snZZ-uabjV_0 2024-06-28T16:14:32.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3534867681586795460', time=2024-06-28T17:47:10.831324+0000, chain.objs.size()=1 2024-06-28T16:14:32.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.QfHQKyWp2gAEXAg4QOxSikGRqHpUrG5_0 2024-06-28T16:14:32.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13087718083515515629', time=2024-06-28T17:47:13.876843+0000, chain.objs.size()=1 2024-06-28T16:14:32.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.183bPnaK3omMaYi1KOZYPTX3EkOZONI_0 2024-06-28T16:14:32.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14320082899709631670', time=2024-06-28T17:47:14.688796+0000, chain.objs.size()=1 2024-06-28T16:14:32.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.tDCrmreBabvLHaFP2Vasi23wFKhFGlb_0 2024-06-28T16:14:32.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10601257004976878139', time=2024-06-28T17:47:15.216318+0000, chain.objs.size()=1 2024-06-28T16:14:32.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.a7X9tNPlrjYXymvYByagxBqX_TWEsKL_0 2024-06-28T16:14:32.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5326863157230316923', time=2024-06-28T17:47:16.957329+0000, chain.objs.size()=1 2024-06-28T16:14:32.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.p2HZEOqVMO6aH2FMEvYHEqzd5haaLSR_0 2024-06-28T16:14:32.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12326315467026393535', time=2024-06-28T17:47:18.029135+0000, chain.objs.size()=1 2024-06-28T16:14:32.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VZF3U64vcETcrwK_T9HRkK8ahV825Xq_0 2024-06-28T16:14:32.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18344400945264994293', time=2024-06-28T17:47:18.031133+0000, chain.objs.size()=1 2024-06-28T16:14:32.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ljqDVkosPJjgdORKnOUmkYUs22qlG2k_0 2024-06-28T16:14:32.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4204107590951700881', time=2024-06-28T17:47:18.606292+0000, chain.objs.size()=1 2024-06-28T16:14:32.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.DnRq23CLnvTce0M5fH711JdP_4tNY82_0 2024-06-28T16:14:32.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13256966019490037466', time=2024-06-28T17:47:18.726828+0000, chain.objs.size()=1 2024-06-28T16:14:32.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.LiiU_LoClL-WSeLE3GoW5o-oD7WAfFo_0 2024-06-28T16:14:32.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4318307245493299903', time=2024-06-28T17:47:18.988363+0000, chain.objs.size()=1 2024-06-28T16:14:32.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.yTtirVd_jYtAW_Kv_hpYZwJxiyuMBZM_0 2024-06-28T16:14:32.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10541470598804195790', time=2024-06-28T17:47:18.990068+0000, chain.objs.size()=1 2024-06-28T16:14:32.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.zHccPp5S5DGYXAxUyzADZRinrhGVxCN_0 2024-06-28T16:14:32.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4139450222103720971', time=2024-06-28T17:47:19.843368+0000, chain.objs.size()=1 2024-06-28T16:14:32.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.sQrO-ocZHcULvNvFyWYN7BCMnsS5l1g_0 2024-06-28T16:14:32.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10193038665316105341', time=2024-06-28T17:47:19.890868+0000, chain.objs.size()=1 2024-06-28T16:14:32.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.8ri8AYGYeLhxHR1eDjGI2aNYwoTAYej_0 2024-06-28T16:14:32.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11609168088900306077', time=2024-06-28T17:47:20.837195+0000, chain.objs.size()=1 2024-06-28T16:14:32.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.JSqZ4-pYRVoeo7AkNm9H33S2qXdm_SN_0 2024-06-28T16:14:32.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15841023595484447308', time=2024-06-28T17:47:21.433606+0000, chain.objs.size()=1 2024-06-28T16:14:32.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.EZAyKj6xR6KzlmfhkVXlI8gK4Q9IVW__0 2024-06-28T16:14:32.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.193533385086360936', time=2024-06-28T17:47:22.511638+0000, chain.objs.size()=1 2024-06-28T16:14:33.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.abjmR2JpDvfRw_5f-zNK4ZHeE0SRqW-_0 2024-06-28T16:14:33.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2192848359052632899', time=2024-06-28T17:47:22.745205+0000, chain.objs.size()=1 2024-06-28T16:14:33.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.brw3KTsEzR-me0iMU_1Km-Cf6C4iLd__0 2024-06-28T16:14:33.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2502277666338509333', time=2024-06-28T17:47:22.747403+0000, chain.objs.size()=1 2024-06-28T16:14:33.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.nleMEsubtI4kJsAwE3CdhkM-0P2vg2G_0 2024-06-28T16:14:33.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9136027420944821477', time=2024-06-28T17:47:22.882646+0000, chain.objs.size()=1 2024-06-28T16:14:33.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.tJOf3AoDycrp0cGQNxtbR4pRwslBn30_0 2024-06-28T16:14:33.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17792429846549289488', time=2024-06-28T17:47:23.635508+0000, chain.objs.size()=1 2024-06-28T16:14:33.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.PL4moIhBz2eS_g2KTLu1nmadcOKEAyg_0 2024-06-28T16:14:33.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3129440295792439465', time=2024-06-28T17:47:27.368733+0000, chain.objs.size()=1 2024-06-28T16:14:33.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.GIGzwE781dGn2ub8wWXqViMGh9wWIB9_0 2024-06-28T16:14:33.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11486645074792449167', time=2024-06-28T17:47:27.715598+0000, chain.objs.size()=1 2024-06-28T16:14:33.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.q7AIpDJiq_uEe24VUiMELZxZNzaSHtk_0 2024-06-28T16:14:33.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13442625750932502791', time=2024-06-28T17:47:28.193676+0000, chain.objs.size()=1 2024-06-28T16:14:33.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.z0jC7YFVr2v8ds-xDyTqSeIv8qp8LEv_0 2024-06-28T16:14:33.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2563557777188862127', time=2024-06-28T17:47:32.976663+0000, chain.objs.size()=1 2024-06-28T16:14:33.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VGj3hRX-hdBFYPhjZ3qnu9vn3ty4_kz_0 2024-06-28T16:14:33.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7568943990676199630', time=2024-06-28T17:47:33.171437+0000, chain.objs.size()=1 2024-06-28T16:14:33.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Asa7SOLgpCdzpr1CGNsJQbUxASeEYB4_0 2024-06-28T16:14:33.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7015323210029905278', time=2024-06-28T17:47:33.909883+0000, chain.objs.size()=1 2024-06-28T16:14:33.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.2_CTZWNUBJnmnAM6ouOP0vsjSlkLkxt_0 2024-06-28T16:14:33.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.415368644284515836', time=2024-06-28T17:47:34.720204+0000, chain.objs.size()=1 2024-06-28T16:14:33.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.P69EPNl4OkWB_n2K4aeS4BTWZrMD6Lm_0 2024-06-28T16:14:33.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4124397813750114590', time=2024-06-28T17:47:34.799969+0000, chain.objs.size()=1 2024-06-28T16:14:33.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.n_Z2Th2TScVp1EPLSew6SoLnjUQih6n_0 2024-06-28T16:14:33.005 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12137440207992933783', time=2024-06-28T17:47:34.943429+0000, chain.objs.size()=1 2024-06-28T16:14:33.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-GOyDf0p4UotrGWY85B96SMvkcNjEwJ_0 2024-06-28T16:14:33.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18338744169223884554', time=2024-06-28T17:47:35.631257+0000, chain.objs.size()=1 2024-06-28T16:14:33.006 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.0VbsDRImUfiHmiTHVAPvEafX8upNTzT_0 2024-06-28T16:14:33.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7012356863359007412', time=2024-06-28T17:47:39.640003+0000, chain.objs.size()=1 2024-06-28T16:14:33.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Je9SRaKRnFLHtEtNgfnLW79Zohve3bL_0 2024-06-28T16:14:33.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6336493903681596319', time=2024-06-28T17:47:55.899346+0000, chain.objs.size()=1 2024-06-28T16:14:33.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.263__shadow_.WcFb6Vzr7CNCYTJ0ciJabB4M7CunHKP_0 2024-06-28T16:14:33.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15908575556570688995', time=2024-06-28T17:48:05.362765+0000, chain.objs.size()=1 2024-06-28T16:14:33.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.269__shadow_.vctiChgyfmGK2ingID0leO_45P2rzeu_0 2024-06-28T16:14:33.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10057751844825024057', time=2024-06-28T17:48:19.833826+0000, chain.objs.size()=1 2024-06-28T16:14:33.007 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.278__shadow_.1ZrMlwuAx3uJp2rSaiYc5BjEW3dWx4w_0 2024-06-28T16:14:33.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.999+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14005348585546882321', time=2024-06-28T17:48:29.431579+0000, chain.objs.size()=1 2024-06-28T16:14:33.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.999+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.61G6nVNv_vXpCdTQyeqfZXBoRAuhGnM_0 2024-06-28T16:14:33.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.999+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9191972921624556751', time=2024-06-28T17:48:29.487821+0000, chain.objs.size()=1 2024-06-28T16:14:33.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.999+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.ofV2EJDx4Y7nwANi2DJ3K-TaP2G8q_8_0 2024-06-28T16:14:33.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.999+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7653735484100057255', time=2024-06-28T17:48:34.446486+0000, chain.objs.size()=1 2024-06-28T16:14:33.010 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.999+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.nLAmXZuDGj0vPMBcZ6WMeiPkDTS7Zqu_0 2024-06-28T16:14:33.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.999+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5004409008438652606', time=2024-06-28T17:48:34.540426+0000, chain.objs.size()=1 2024-06-28T16:14:33.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.999+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.TzocsmGotZeB0F6RxeX4G1aedcKAL3D_0 2024-06-28T16:14:33.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.999+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4592535996671717345', time=2024-06-28T17:48:49.524650+0000, chain.objs.size()=1 2024-06-28T16:14:33.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.999+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.292__shadow_.Zm6na5cMKK-Xu-LDLc2W7G6bQ63ReXA_0 2024-06-28T16:14:33.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.999+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5935895305801124612', time=2024-06-28T17:48:51.178829+0000, chain.objs.size()=1 2024-06-28T16:14:33.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.999+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.293__shadow_.J6M0fQ3spDQhl_cLMHc3RbDnprD1ylh_0 2024-06-28T16:14:33.011 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.999+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1511277605165866952', time=2024-06-28T17:48:52.730038+0000, chain.objs.size()=1 2024-06-28T16:14:33.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:32.999+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.294__shadow_.brtNXaiupTNsl1f4VpkXCh8rj4QYc9-_0 2024-06-28T16:14:33.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12240987706231880497', time=2024-06-28T17:49:16.720018+0000, chain.objs.size()=1 2024-06-28T16:14:33.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.309__shadow:Kis-JHaHDMXHD7i.tvVJ.xCMrdrHsFq_.aGKwUzztf_bJbNDsy9c8Ef7bYOxMI_7_0 2024-06-28T16:14:33.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8049134929695985323', time=2024-06-28T17:49:19.388286+0000, chain.objs.size()=1 2024-06-28T16:14:33.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.311__shadow_.Jxg8AwMSN7_pZvySV8tSyRi3jgHuFMg_0 2024-06-28T16:14:33.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17588885656170615112', time=2024-06-28T17:51:08.515994+0000, chain.objs.size()=1 2024-06-28T16:14:33.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.X8B82z8e8DIsxBhczTv95pmEOkwi6kg_0 2024-06-28T16:14:33.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17080186277117928226', time=2024-06-28T17:51:08.881895+0000, chain.objs.size()=1 2024-06-28T16:14:33.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_._c2gkX4m2ku-e8gpKq0cr_ET6uxdEbX_0 2024-06-28T16:14:33.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14750234563448428208', time=2024-06-28T17:51:13.486390+0000, chain.objs.size()=1 2024-06-28T16:14:33.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.B1el0rgpapTIukwuzCYu5JWP84HCXlc_0 2024-06-28T16:14:33.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9208836654267883601', time=2024-06-28T17:51:13.835802+0000, chain.objs.size()=1 2024-06-28T16:14:33.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yfmt1FJcWG1enHDPwsL5LuloiGf2LgM_0 2024-06-28T16:14:33.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7816270118908035642', time=2024-06-28T17:51:16.233252+0000, chain.objs.size()=1 2024-06-28T16:14:33.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.K4uPdjI4xOs7kRwn-m-t2nHv_JhXr39_0 2024-06-28T16:14:33.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12154428695380076762', time=2024-06-28T17:51:16.822660+0000, chain.objs.size()=1 2024-06-28T16:14:33.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.vhzaPwM3S5Vlq-STTCPgJhPq1fgv0mn_0 2024-06-28T16:14:33.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13299393661952436058', time=2024-06-28T17:51:16.954315+0000, chain.objs.size()=1 2024-06-28T16:14:33.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.QWbEIaAFS1NsvzFkcXjMu-MEd_auwp7_0 2024-06-28T16:14:33.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.662441522082930464', time=2024-06-28T17:51:17.528835+0000, chain.objs.size()=1 2024-06-28T16:14:33.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hjD-iHNeE2m9hc2dJV07DAODcNdx_gf_0 2024-06-28T16:14:33.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11795060338806853136', time=2024-06-28T17:51:17.783462+0000, chain.objs.size()=1 2024-06-28T16:14:33.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.HDxi3b32YMeR66FYkn8yHiv4oUCmvuK_0 2024-06-28T16:14:33.017 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1748747221717953474', time=2024-06-28T17:51:18.202558+0000, chain.objs.size()=1 2024-06-28T16:14:33.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.BOFbHZWy7TgSDxhY1kuzAyw8YG30bb6_0 2024-06-28T16:14:33.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7454896718471744582', time=2024-06-28T17:51:18.402422+0000, chain.objs.size()=1 2024-06-28T16:14:33.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.37W3h9-ayj2THqk6FNoYErUMYGawMSm_0 2024-06-28T16:14:33.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8706423928171708137', time=2024-06-28T17:51:21.278595+0000, chain.objs.size()=1 2024-06-28T16:14:33.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YnL7SvfxYO9ibIj7zjgYQxVwEk6O3OZ_0 2024-06-28T16:14:33.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14173070421799383402', time=2024-06-28T17:51:22.372203+0000, chain.objs.size()=1 2024-06-28T16:14:33.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.tZpHA7fAimfLB7M-Hi40lM7IwH1SaSX_0 2024-06-28T16:14:33.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16929168972857654940', time=2024-06-28T17:51:22.651044+0000, chain.objs.size()=1 2024-06-28T16:14:33.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ITMQH5BhEeiUfmDlcORb-K7a3QvVbDY_0 2024-06-28T16:14:33.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11113361533763058899', time=2024-06-28T17:51:23.656418+0000, chain.objs.size()=1 2024-06-28T16:14:33.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.sUYTds8yud7unOCD8q1oD1ldnWUBFJT_0 2024-06-28T16:14:33.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4653325012823460388', time=2024-06-28T17:51:23.871487+0000, chain.objs.size()=1 2024-06-28T16:14:33.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YEWskGcVZQEFU8osoQmCp6udROjm9lQ_0 2024-06-28T16:14:33.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8205421968138207805', time=2024-06-28T17:51:26.693638+0000, chain.objs.size()=1 2024-06-28T16:14:33.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.IJE8ElcusNpabiP-nq4UGw9olBYP95O_0 2024-06-28T16:14:33.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2818234939059300346', time=2024-06-28T17:51:26.790099+0000, chain.objs.size()=1 2024-06-28T16:14:33.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-cg43QdKgMd23Iz-XL5DbHLLO8HqeAg_0 2024-06-28T16:14:33.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1309617631684772418', time=2024-06-28T17:51:26.861319+0000, chain.objs.size()=1 2024-06-28T16:14:33.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.8kMDNQY0crQ_CsPMl-J_3lGDSVLEGaZ_0 2024-06-28T16:14:33.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12543615529540111942', time=2024-06-28T17:51:28.065671+0000, chain.objs.size()=1 2024-06-28T16:14:33.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.3ARuYqLC8kQVKxlMsaR00BAjwhkKwph_0 2024-06-28T16:14:33.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13249608738772617482', time=2024-06-28T17:51:30.443582+0000, chain.objs.size()=1 2024-06-28T16:14:33.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.VLyQYJOWYOmEoeilHB5bzXLg1_3wmWR_0 2024-06-28T16:14:33.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13718718794499297141', time=2024-06-28T17:51:30.605480+0000, chain.objs.size()=1 2024-06-28T16:14:33.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.k3Efj-gfalIj_HHntWL3UdMfNavedWQ_0 2024-06-28T16:14:33.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3775919352517209443', time=2024-06-28T17:51:31.033057+0000, chain.objs.size()=1 2024-06-28T16:14:33.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.9KZuvXxR23Et5zPDStXsJwXaL52TYBQ_0 2024-06-28T16:14:33.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17037143762325776256', time=2024-06-28T17:51:33.231351+0000, chain.objs.size()=1 2024-06-28T16:14:33.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.p78k8GMnMNk1AZs3zW4u8uXOoMua7Xo_0 2024-06-28T16:14:33.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7478430293314930715', time=2024-06-28T17:51:34.531841+0000, chain.objs.size()=1 2024-06-28T16:14:33.025 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.W1QJnrXgWw3l9ib7y2j5Dci3wMSQzXI_0 2024-06-28T16:14:33.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10334973900247542680', time=2024-06-28T17:51:38.383757+0000, chain.objs.size()=1 2024-06-28T16:14:33.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.2mukmmSMdRn-VQTt6vcbtUjpLOC3xkx_0 2024-06-28T16:14:33.026 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2293957106384507946', time=2024-06-28T17:53:22.926269+0000, chain.objs.size()=1 2024-06-28T16:14:33.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GwXq8sPh0aUrP2XBgcZx4ixeeLBklYC_0 2024-06-28T16:14:33.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11868910429092027077', time=2024-06-28T17:53:24.002940+0000, chain.objs.size()=1 2024-06-28T16:14:33.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZHPp6cCREPwijUs-68kREAbjpBViJJc_0 2024-06-28T16:14:33.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9614117687906384453', time=2024-06-28T17:53:25.206475+0000, chain.objs.size()=1 2024-06-28T16:14:33.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.T4MTSJpye2tx5aYKl8hmL6TEEsCBgyY_0 2024-06-28T16:14:33.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8508104715918025178', time=2024-06-28T17:53:26.038200+0000, chain.objs.size()=1 2024-06-28T16:14:33.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.baCixl31amikkmp7gzNews5-GVw9a5i_0 2024-06-28T16:14:33.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8464895802053255697', time=2024-06-28T17:53:26.084419+0000, chain.objs.size()=1 2024-06-28T16:14:33.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ArIXGi0yNO395NzYBkasABxZ-I6Qv5a_0 2024-06-28T16:14:33.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17853208524633834982', time=2024-06-28T17:53:26.147830+0000, chain.objs.size()=1 2024-06-28T16:14:33.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.015+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qiv7BbTH5CMURuRgD1Tc6cjk24g217Q_0 2024-06-28T16:14:33.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.019+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2962265183090282648', time=2024-06-28T17:53:26.210436+0000, chain.objs.size()=1 2024-06-28T16:14:33.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.019+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.AoV1Bd0bZULRQG7WlFQaK7RTcsc_Anh_0 2024-06-28T16:14:33.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.019+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.867264076531333904', time=2024-06-28T17:53:26.297173+0000, chain.objs.size()=1 2024-06-28T16:14:33.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.019+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Ys0Fwy9qBHfDGjnOyGMT7JjsLla_32c_0 2024-06-28T16:14:33.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.019+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8286048708705771437', time=2024-06-28T17:53:28.341973+0000, chain.objs.size()=1 2024-06-28T16:14:33.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.019+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.TYw4YseQM2aIBh6WB9rcB--6DD1WEF0_0 2024-06-28T16:14:33.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.019+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15174909579189128547', time=2024-06-28T17:53:29.467990+0000, chain.objs.size()=1 2024-06-28T16:14:33.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.019+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.fTWQb9yW5EdRAQL_temzVTX56ROd6PU_0 2024-06-28T16:14:33.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14501895495760352573', time=2024-06-28T17:53:30.528727+0000, chain.objs.size()=1 2024-06-28T16:14:33.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.WO1NFRF5abY0R8P1KTWAChiP2e-uz3r_0 2024-06-28T16:14:33.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18208963658007156305', time=2024-06-28T17:53:30.669915+0000, chain.objs.size()=1 2024-06-28T16:14:33.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.jFkkGOx2v0Rv66g7bXWHlMSzfY4qxvG_0 2024-06-28T16:14:33.033 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6305296163863755143', time=2024-06-28T17:53:34.054060+0000, chain.objs.size()=1 2024-06-28T16:14:33.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wB6mWr9wCxrBJVUd5rHgkqMKnEmCccF_0 2024-06-28T16:14:33.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14176420419426260136', time=2024-06-28T17:53:34.787112+0000, chain.objs.size()=1 2024-06-28T16:14:33.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.5nr21dkh3YQQHx7DlpV98EvZ4zCObqq_0 2024-06-28T16:14:33.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11159062407153640834', time=2024-06-28T17:53:38.148662+0000, chain.objs.size()=1 2024-06-28T16:14:33.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZMD4-7mz9-ptun8TI7yep5QWpht_zQp_0 2024-06-28T16:14:33.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.749203774525123378', time=2024-06-28T17:53:39.617036+0000, chain.objs.size()=1 2024-06-28T16:14:33.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bqtFwh4IhhTSTpqYGZBDRa5sipwOlLQ_0 2024-06-28T16:14:33.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11441543966065916949', time=2024-06-28T17:53:39.707940+0000, chain.objs.size()=1 2024-06-28T16:14:33.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Hy23TN5_YH314Ijx_icUdXJcP0nCx5P_0 2024-06-28T16:14:33.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.027+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/47708 2024-06-28T16:14:33.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=32, truncated=0, next_marker='' 2024-06-28T16:14:33.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10001519403724743929', time=2024-06-28T17:53:40.253016+0000, chain.objs.size()=1 2024-06-28T16:14:33.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.o_09kg-YMlgOSTKggTmC7qsjp4C1qbL_0 2024-06-28T16:14:33.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18244814294714295610', time=2024-06-28T17:53:41.830841+0000, chain.objs.size()=1 2024-06-28T16:14:33.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.zs6VsSRFufbvlrLZLUbIhNSh2KYd_Hm_0 2024-06-28T16:14:33.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2688881063167601407', time=2024-06-28T17:53:43.542738+0000, chain.objs.size()=1 2024-06-28T16:14:33.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.VTkYnY6dCkwd-FvX9LCk04k8X4kBZrm_0 2024-06-28T16:14:33.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8787017779769834074', time=2024-06-28T17:53:43.687899+0000, chain.objs.size()=1 2024-06-28T16:14:33.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.sO7FYbSzlcJWkktH2EeJMZvNCk8Y0-K_0 2024-06-28T16:14:33.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5414606795904593531', time=2024-06-28T17:53:44.037450+0000, chain.objs.size()=1 2024-06-28T16:14:33.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.8eD3bzNRdzzB39QPkfZ28XMQg0JMtQL_0 2024-06-28T16:14:33.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.701534281026657342', time=2024-06-28T17:53:44.237396+0000, chain.objs.size()=1 2024-06-28T16:14:33.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.KQv1Yuc-Xu0RXUpOiMup1g_18xLfPZA_0 2024-06-28T16:14:33.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8600395076911669530', time=2024-06-28T17:53:44.589879+0000, chain.objs.size()=1 2024-06-28T16:14:33.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.dpJ-2VbEyo4oImv_enFeiDaZ11cz0lJ_0 2024-06-28T16:14:33.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6227488382715982275', time=2024-06-28T17:53:47.189850+0000, chain.objs.size()=1 2024-06-28T16:14:33.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ed5nv3ZB4AHEYhGChk0XTvsasZAtup3_0 2024-06-28T16:14:33.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13302194956033534056', time=2024-06-28T17:53:48.044442+0000, chain.objs.size()=1 2024-06-28T16:14:33.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.XIh_cQ82CNYeklLTPrngW7hHy0Dl7Px_0 2024-06-28T16:14:33.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.740608099016405566', time=2024-06-28T17:53:48.393862+0000, chain.objs.size()=1 2024-06-28T16:14:33.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.PSHBsbNTXUt_xEi4BYS8iYW5e6yHpVQ_0 2024-06-28T16:14:33.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8791980496549486513', time=2024-06-28T17:53:48.418753+0000, chain.objs.size()=1 2024-06-28T16:14:33.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.3sFMYLfvye-z25xkLdvOyiSa7SL7mNA_0 2024-06-28T16:14:33.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6502203937887104375', time=2024-06-28T17:53:48.442993+0000, chain.objs.size()=1 2024-06-28T16:14:33.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.7U85BPNMt5-BAWeSwBiG7MfRKLO0xB3_0 2024-06-28T16:14:33.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6550905721968392017', time=2024-06-28T17:53:48.534854+0000, chain.objs.size()=1 2024-06-28T16:14:33.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.sgGmZYQCpdCUA6vLyNMvPyn9H0_0dId_0 2024-06-28T16:14:33.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7483420637687910999', time=2024-06-28T17:53:50.016268+0000, chain.objs.size()=1 2024-06-28T16:14:33.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.76vf1D5JBYLFUgxTCUzA0x3iX2bitg3_0 2024-06-28T16:14:33.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.874658347944056813', time=2024-06-28T17:53:50.604321+0000, chain.objs.size()=1 2024-06-28T16:14:33.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.sOKzrnQaWs0Ufbvqu3RcGNynm-_63e6_0 2024-06-28T16:14:33.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3418096618592624796', time=2024-06-28T17:53:51.165858+0000, chain.objs.size()=1 2024-06-28T16:14:33.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.y7bML1LpDc48YkVPP8xSbJ3KvEdfXzX_0 2024-06-28T16:14:33.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7720732887554285328', time=2024-06-28T17:53:51.617705+0000, chain.objs.size()=1 2024-06-28T16:14:33.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-8_iPAmZ9NZiWz2V44X2RLCx8GN-Q3u_0 2024-06-28T16:14:33.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1354365781531816850', time=2024-06-28T17:53:52.057829+0000, chain.objs.size()=1 2024-06-28T16:14:33.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.w3lwOEVL03z0QCaybv3_DVv-mt4FfR-_0 2024-06-28T16:14:33.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1159341474928389363', time=2024-06-28T17:54:31.579320+0000, chain.objs.size()=1 2024-06-28T16:14:33.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.361__shadow_.l953ohhOHpPoy2FHPDDqJzhCORo8o9Y_0 2024-06-28T16:14:33.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2241020700519704762', time=2024-06-28T17:54:52.094959+0000, chain.objs.size()=1 2024-06-28T16:14:33.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.380__shadow_.f7TcvYoUirSfM4XR8wXZTJjzQ4H69Dp_0 2024-06-28T16:14:33.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12140009831872628982', time=2024-06-28T17:56:08.342856+0000, chain.objs.size()=1 2024-06-28T16:14:33.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.453__shadow_.fJ_qhuoaXvS1pPtWjf8NmM0TcjEREgE_0 2024-06-28T16:14:33.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.633999351766508398', time=2024-06-28T17:56:40.178349+0000, chain.objs.size()=1 2024-06-28T16:14:33.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.480__shadow_.F5BmsfH_Xs0y6ozgOHjJeHNo8Q4wy-Z_0 2024-06-28T16:14:33.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.749198175991442407', time=2024-06-28T17:56:44.298085+0000, chain.objs.size()=1 2024-06-28T16:14:33.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.485__shadow_.JbpT2OSJT2NyHKZ0dhfYRV89uhwWES2_0 2024-06-28T16:14:33.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.039+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4276791852633384317', time=2024-06-28T17:59:57.868751+0000, chain.objs.size()=3 2024-06-28T16:14:33.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.039+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.515__multipart_mymultipart.2~ymxHKCOBv4L_IzBxfeAmPsnskXVblGJ.1 2024-06-28T16:14:33.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.039+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.515__multipart_mymultipart.2~ymxHKCOBv4L_IzBxfeAmPsnskXVblGJ.2 2024-06-28T16:14:33.049 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.039+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.515__multipart_mymultipart.2~ymxHKCOBv4L_IzBxfeAmPsnskXVblGJ.3 2024-06-28T16:14:33.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4023621387818357193', time=2024-06-28T18:01:53.067395+0000, chain.objs.size()=1 2024-06-28T16:14:33.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.553__shadow_.ieiTtKVswUVfHztRZpMoLj6YftdfNrb_0 2024-06-28T16:14:33.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15741506706485964330', time=2024-06-28T18:02:14.531227+0000, chain.objs.size()=1 2024-06-28T16:14:33.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:rYp2uUfWq9Wk74UaZNTLAsRwC-GGOcR_.fBXAU2EnB9kcQ8txES0YFVj1PSe-WOw_0 2024-06-28T16:14:33.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15911740547033522973', time=2024-06-28T18:02:30.469552+0000, chain.objs.size()=1 2024-06-28T16:14:33.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.563__shadow_.gONG7iAXMBIJ7VgfraoQegVNc_tZKYz_0 2024-06-28T16:14:33.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10332331830440048720', time=2024-06-28T18:02:44.797905+0000, chain.objs.size()=1 2024-06-28T16:14:33.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.565__shadow:JTaUohs88gvdKaHa7I15j.VuQgQeJWs_._HKrBxg1P-n-O4_OP7ShpvBOSxgOpF8_0 2024-06-28T16:14:33.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12563059028707773510', time=2024-06-28T18:03:30.892349+0000, chain.objs.size()=1 2024-06-28T16:14:33.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:lQ-cu5mrO289UmBnKw7c-emO43GXIq5_.dS8spoNKhovOjIrv4XQq9fgMIZl27y__0 2024-06-28T16:14:33.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2351253127149193084', time=2024-06-28T18:03:56.276850+0000, chain.objs.size()=1 2024-06-28T16:14:33.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.568__shadow:j4u880kUFWC8gjfkmaE9POv7Gc-3XT6_.DrDwGf0jFXDasykPLrYIM5yOMou2hrf_0 2024-06-28T16:14:33.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16725281446549039477', time=2024-06-28T18:04:02.285666+0000, chain.objs.size()=1 2024-06-28T16:14:33.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.571__shadow:wQ94CpYSwbR4AQT5WLGbKczM.wXx6XJ_.HwwyGkumcdN5aJ4uKoL5Z3K9mk1AoEQ_0 2024-06-28T16:14:33.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12938765860788474298', time=2024-06-28T18:08:08.709139+0000, chain.objs.size()=1 2024-06-28T16:14:33.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.655__shadow_.2m_xiO6iYnN3kcIk-nUoWLtTZy9NVFB_0 2024-06-28T16:14:33.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.047+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:33.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-06-28T16:14:33.060 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:33.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:33.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/43948' 2024-06-28T16:14:33.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18038972755595860546', time=2024-06-28T17:41:20.582065+0000, chain.objs.size()=1 2024-06-28T16:14:33.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.179__shadow_.hI01nmNS5l1hAqiFrNuNSrEooETKKYI_0 2024-06-28T16:14:33.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15366319174374839802', time=2024-06-28T17:41:56.608659+0000, chain.objs.size()=1 2024-06-28T16:14:33.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.194__shadow_.8oGyHCtHmctm8Bung2XYVuQWSyDlxcu_0 2024-06-28T16:14:33.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12120053318469217085', time=2024-06-28T17:45:14.664102+0000, chain.objs.size()=1 2024-06-28T16:14:33.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.244__shadow_.VcAdo53gilcNTbcpEDaZhBcSPQRulJ5_0 2024-06-28T16:14:33.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2686796929010333630', time=2024-06-28T17:45:16.241578+0000, chain.objs.size()=1 2024-06-28T16:14:33.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.245__shadow_.ninjlmraiHsFYcqU-wgm90-WcJHwIxZ_0 2024-06-28T16:14:33.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13618705633178595422', time=2024-06-28T17:47:07.856011+0000, chain.objs.size()=1 2024-06-28T16:14:33.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.qdV2W_oGMAJb7xr9vtjQDfRVD2CvF9T_0 2024-06-28T16:14:33.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6305024775641490915', time=2024-06-28T17:47:08.448766+0000, chain.objs.size()=1 2024-06-28T16:14:33.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dZVVKqShuww9lb6q_nqwbuhc6GZZm9E_0 2024-06-28T16:14:33.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7120505183748750652', time=2024-06-28T17:47:10.584663+0000, chain.objs.size()=1 2024-06-28T16:14:33.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.URWeNfvCGIE4ja8CLbw7ZuN6a68i-0n_0 2024-06-28T16:14:33.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3505033245924100857', time=2024-06-28T17:47:11.174425+0000, chain.objs.size()=1 2024-06-28T16:14:33.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.8hR2dZQdimI9sRfs25GRHJk63186eU5_0 2024-06-28T16:14:33.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5850581843064644454', time=2024-06-28T17:47:12.333003+0000, chain.objs.size()=1 2024-06-28T16:14:33.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.fXqgZFTo-hjat7XPVzrm6nOv_4_AiLG_0 2024-06-28T16:14:33.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15923885093743568490', time=2024-06-28T17:47:12.405649+0000, chain.objs.size()=1 2024-06-28T16:14:33.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.jFc0JEJE2eeb9v9eK3xLN0oR773xtZ9_0 2024-06-28T16:14:33.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11826152417307405806', time=2024-06-28T17:47:12.901892+0000, chain.objs.size()=1 2024-06-28T16:14:33.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.aeeTx116YfHdAAEWSyLt_hjHXxvShP5_0 2024-06-28T16:14:33.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17544313923485801436', time=2024-06-28T17:47:13.380919+0000, chain.objs.size()=1 2024-06-28T16:14:33.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.lfclzdGbiU5Z3oz_sB7kje1-B3-uhWb_0 2024-06-28T16:14:33.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.055+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11998780854005221983', time=2024-06-28T17:47:15.185972+0000, chain.objs.size()=1 2024-06-28T16:14:33.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.055+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.hkihMEqvvLodQO8yh9XTSC26vmHxrXN_0 2024-06-28T16:14:33.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.055+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10292991378794056016', time=2024-06-28T17:47:15.221729+0000, chain.objs.size()=1 2024-06-28T16:14:33.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.055+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KEIeAGLnVblo6SeaB92MJNrlNiZ62ib_0 2024-06-28T16:14:33.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.055+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1327099086390936971', time=2024-06-28T17:47:21.137383+0000, chain.objs.size()=1 2024-06-28T16:14:33.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.055+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KadHvP10jRJoXtHhdTUUgMg5B-w00Nm_0 2024-06-28T16:14:33.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.055+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12833416841599391866', time=2024-06-28T17:47:22.744520+0000, chain.objs.size()=1 2024-06-28T16:14:33.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.055+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.b-hhIElOdCWukJLa4ik0lHsynfXBN5z_0 2024-06-28T16:14:33.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.857441922711315155', time=2024-06-28T17:47:22.882583+0000, chain.objs.size()=1 2024-06-28T16:14:33.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.E38HOGj7YiZ_eQcxhH4Lu2B5pBOzWo7_0 2024-06-28T16:14:33.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6677686295312119768', time=2024-06-28T17:47:22.998020+0000, chain.objs.size()=1 2024-06-28T16:14:33.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9nVwlsPlCnGtxyTmJgRrlJ_TL06_iMN_0 2024-06-28T16:14:33.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6106597247759959542', time=2024-06-28T17:47:25.398582+0000, chain.objs.size()=1 2024-06-28T16:14:33.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.QMDBu28zaRNDkPXGcUAqR5kDAAtHYHK_0 2024-06-28T16:14:33.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9195130694338368196', time=2024-06-28T17:47:25.848365+0000, chain.objs.size()=1 2024-06-28T16:14:33.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.D55GWVdjAKeE47ZIXMowRFcliA43F0y_0 2024-06-28T16:14:33.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8993142179057936486', time=2024-06-28T17:47:25.891515+0000, chain.objs.size()=1 2024-06-28T16:14:33.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.tf9sV1iDMQnYom2VtdwrAiISKKsA4Iz_0 2024-06-28T16:14:33.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2003716467272521482', time=2024-06-28T17:47:27.091565+0000, chain.objs.size()=1 2024-06-28T16:14:33.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.XuVW9ypTrm2nsYZoy6nGPndoN1DA28X_0 2024-06-28T16:14:33.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6595016300405911667', time=2024-06-28T17:47:27.249636+0000, chain.objs.size()=1 2024-06-28T16:14:33.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.GIJS6WeoDWwgT9rExhpK1dmr-3-1B11_0 2024-06-28T16:14:33.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9422075698934723793', time=2024-06-28T17:47:27.870482+0000, chain.objs.size()=1 2024-06-28T16:14:33.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.arPoeEOOiVKnr75xHYKaXK_LBEZLQpv_0 2024-06-28T16:14:33.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6926616692282465736', time=2024-06-28T17:47:29.070966+0000, chain.objs.size()=1 2024-06-28T16:14:33.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.tjNVnZToUhos6ap-uwVwtyQbaG9wNGv_0 2024-06-28T16:14:33.072 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14877049797449467379', time=2024-06-28T17:47:30.168740+0000, chain.objs.size()=1 2024-06-28T16:14:33.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Cs7Lc7tc3CKyNZcdggn1vhX53Ys-lxF_0 2024-06-28T16:14:33.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6411341063394662213', time=2024-06-28T17:47:30.595098+0000, chain.objs.size()=1 2024-06-28T16:14:33.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.zfAeDYQbOcPhNkabl0CWeTVfUUcZkrn_0 2024-06-28T16:14:33.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12652827096731706783', time=2024-06-28T17:47:31.504888+0000, chain.objs.size()=1 2024-06-28T16:14:33.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Auc6kDQEtJZbKHvRIOBjvjB0HbjT3DE_0 2024-06-28T16:14:33.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4684905576809589221', time=2024-06-28T17:47:34.082276+0000, chain.objs.size()=1 2024-06-28T16:14:33.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ZraQymQCxLJA5HdXKs-o4VzqaWwItOg_0 2024-06-28T16:14:33.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13222476971085199593', time=2024-06-28T17:47:36.207136+0000, chain.objs.size()=1 2024-06-28T16:14:33.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.OaeNyx0E70dzO0lV9NI4zRyhlpLWVoy_0 2024-06-28T16:14:33.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12871083262954658909', time=2024-06-28T17:47:37.818819+0000, chain.objs.size()=1 2024-06-28T16:14:33.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Yl2KTZ1trK8tlfWPfc_kCQTWfN8Xu6Q_0 2024-06-28T16:14:33.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4815627839990040596', time=2024-06-28T17:47:38.528355+0000, chain.objs.size()=1 2024-06-28T16:14:33.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.nIC8MnTDFUR8T4fLYpfO94NnqT0bsR6_0 2024-06-28T16:14:33.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6170302786424590397', time=2024-06-28T17:47:47.006328+0000, chain.objs.size()=1 2024-06-28T16:14:33.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.257__shadow_.N3oS6Zu5ZuU8NliCWKGNlZUenDF1G8P_0 2024-06-28T16:14:33.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9805198182942657558', time=2024-06-28T17:47:57.327931+0000, chain.objs.size()=1 2024-06-28T16:14:33.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.264__shadow_.o1k0jc4EO3-yG6H38XagTnqUxlEF7Ph_0 2024-06-28T16:14:33.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8376096995473848480', time=2024-06-28T17:48:08.519413+0000, chain.objs.size()=1 2024-06-28T16:14:33.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.271__shadow_.rPHUjtrboKnn9jQuG_Ma14rABYCsd8F_0 2024-06-28T16:14:33.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.067+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9552293930873802932', time=2024-06-28T17:48:16.667017+0000, chain.objs.size()=1 2024-06-28T16:14:33.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.067+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.276__shadow_.DHsfrBb9eLvj2a_I_LjgvjL5-BtPXfm_0 2024-06-28T16:14:33.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.067+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17448503824281957971', time=2024-06-28T17:48:23.008970+0000, chain.objs.size()=1 2024-06-28T16:14:33.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.067+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.280__shadow_.1s9c_kup34dc3dWduBSaB-QbnLkb57T_0 2024-06-28T16:14:33.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.067+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12558925839813154517', time=2024-06-28T17:48:29.575369+0000, chain.objs.size()=1 2024-06-28T16:14:33.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.067+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.i78jcJJnzwScVJPxCO7kp5S-4LNuzh4_0 2024-06-28T16:14:33.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10426048451712153405', time=2024-06-28T17:48:29.755688+0000, chain.objs.size()=1 2024-06-28T16:14:33.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.iPC59jG4JQiVyZc_zwzCgGhkXdx9SYu_0 2024-06-28T16:14:33.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17726993291521895358', time=2024-06-28T17:48:34.637094+0000, chain.objs.size()=1 2024-06-28T16:14:33.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.t4ZmkB-SXUu0bnLH495s_V8ytu14COC_0 2024-06-28T16:14:33.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6204683862801635530', time=2024-06-28T17:48:34.638978+0000, chain.objs.size()=1 2024-06-28T16:14:33.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.c5_xsOtYi4mZu5rLPnUX0W5tVSt86R6_0 2024-06-28T16:14:33.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17321915675568837638', time=2024-06-28T17:48:41.364432+0000, chain.objs.size()=1 2024-06-28T16:14:33.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.287__shadow_.IIKnLm-j90PQAleA-3w8yQsZj81rSuu_0 2024-06-28T16:14:33.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6348556963486952751', time=2024-06-28T17:48:51.180467+0000, chain.objs.size()=1 2024-06-28T16:14:33.083 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.293__shadow_.e3sNH5kPs-VMcPTJni7-HcDppvjC1lo_0 2024-06-28T16:14:33.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12213426676826396085', time=2024-06-28T17:51:06.765062+0000, chain.objs.size()=1 2024-06-28T16:14:33.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.72-JcJUe3EZcbXVZTaT3oLBzCKPu4T5_0 2024-06-28T16:14:33.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6037079200616351488', time=2024-06-28T17:51:07.342413+0000, chain.objs.size()=1 2024-06-28T16:14:33.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1Q2NDM2NVlT_VyDCtKSC-VUkgtz07aX_0 2024-06-28T16:14:33.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13994091304419049608', time=2024-06-28T17:51:07.701797+0000, chain.objs.size()=1 2024-06-28T16:14:33.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.T8Su7kjoHOI00Cg_-6qJJIZYvrDePwk_0 2024-06-28T16:14:33.084 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7490125679567712211', time=2024-06-28T17:51:08.487386+0000, chain.objs.size()=1 2024-06-28T16:14:33.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.uLBN9U7UlyBIe6YQFN4pCbWuwof0P4o_0 2024-06-28T16:14:33.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3316391080884827437', time=2024-06-28T17:51:09.026071+0000, chain.objs.size()=1 2024-06-28T16:14:33.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.eRP0jaHtG9dp630D806vPQc6jLLA0VS_0 2024-06-28T16:14:33.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11128095326064455482', time=2024-06-28T17:51:11.198703+0000, chain.objs.size()=1 2024-06-28T16:14:33.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.qUiZ8XaXqLauSNbUWNyUtVv7CGTJR9Y_0 2024-06-28T16:14:33.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.075+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10744287922556992385', time=2024-06-28T17:51:11.874556+0000, chain.objs.size()=1 2024-06-28T16:14:33.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.075+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_._n9IhcqDtrSGJNrkxFuO2GUsuRXMVS8_0 2024-06-28T16:14:33.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.075+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11547540357856608461', time=2024-06-28T17:51:12.296817+0000, chain.objs.size()=1 2024-06-28T16:14:33.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.075+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.XBNu-9Ej1JhkTMoXy0QNlS2HdPaeYYb_0 2024-06-28T16:14:33.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.075+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11783752284325810144', time=2024-06-28T17:51:12.598978+0000, chain.objs.size()=1 2024-06-28T16:14:33.086 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.075+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GUJkYieqC1wDzNoeC22MeBPcRueNENf_0 2024-06-28T16:14:33.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.353337367840659566', time=2024-06-28T17:51:12.759020+0000, chain.objs.size()=1 2024-06-28T16:14:33.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.XedVLK15zj627YI3p2_13JDUapPi2Yl_0 2024-06-28T16:14:33.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11791663506342596925', time=2024-06-28T17:51:13.134616+0000, chain.objs.size()=1 2024-06-28T16:14:33.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.IJbtNRRLV1fGA8i8KofOb8aL20svJYV_0 2024-06-28T16:14:33.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12611290372749791548', time=2024-06-28T17:51:13.475786+0000, chain.objs.size()=1 2024-06-28T16:14:33.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.G1qYdCpu7nYD1p3jHYZOxV6aVkPJokb_0 2024-06-28T16:14:33.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8506910458703521053', time=2024-06-28T17:51:13.880124+0000, chain.objs.size()=1 2024-06-28T16:14:33.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.HpN0UeIGkW_lEDNL9BAB2MbxQ4Ip507_0 2024-06-28T16:14:33.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7272825174668900486', time=2024-06-28T17:51:15.984716+0000, chain.objs.size()=1 2024-06-28T16:14:33.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.CKRsaSm_p_umMhRTIdRGC3WxToMR_MV_0 2024-06-28T16:14:33.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.559225578586439679', time=2024-06-28T17:51:16.286191+0000, chain.objs.size()=1 2024-06-28T16:14:33.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1xRnGQynhOWCwC0aFn8_-iLa_QqAa0E_0 2024-06-28T16:14:33.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14233298959660363909', time=2024-06-28T17:51:16.378890+0000, chain.objs.size()=1 2024-06-28T16:14:33.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.lj_J5e1uxMByZevtuQPlqOidrC5Eutw_0 2024-06-28T16:14:33.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6138028929834665985', time=2024-06-28T17:51:17.329545+0000, chain.objs.size()=1 2024-06-28T16:14:33.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.LamIdTw0Dnrt45B3aaUqL16sDho7NFG_0 2024-06-28T16:14:33.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18103223422218032698', time=2024-06-28T17:51:19.621483+0000, chain.objs.size()=1 2024-06-28T16:14:33.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fnAHbjvrFjcHuT-bADB7OIcdbXvOk8g_0 2024-06-28T16:14:33.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4983129074258490446', time=2024-06-28T17:51:21.176199+0000, chain.objs.size()=1 2024-06-28T16:14:33.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_._aqMRYzwE-mmKTcnLeqvVYBUc8h9bp0_0 2024-06-28T16:14:33.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17214565042074911213', time=2024-06-28T17:51:21.825301+0000, chain.objs.size()=1 2024-06-28T16:14:33.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.rg01JUAdqUX3mDcibfMeazjTM28_Qu3_0 2024-06-28T16:14:33.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6539035630111257612', time=2024-06-28T17:51:23.656329+0000, chain.objs.size()=1 2024-06-28T16:14:33.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xN4mi24xGhA-FUC2JOnVZzgTk1Nof59_0 2024-06-28T16:14:33.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2215987854677674008', time=2024-06-28T17:51:25.220302+0000, chain.objs.size()=1 2024-06-28T16:14:33.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.2ahsyQzP1p4Z293hzdNMEDntT_vRQ7L_0 2024-06-28T16:14:33.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16706688920205935013', time=2024-06-28T17:51:27.718277+0000, chain.objs.size()=1 2024-06-28T16:14:33.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.FUkFCRk1exraLiJlaojxDPiCoMkMTkF_0 2024-06-28T16:14:33.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11485003796031132209', time=2024-06-28T17:51:28.085479+0000, chain.objs.size()=1 2024-06-28T16:14:33.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.vu5YO1PITyKWlFFHVxdx11J1peYRzE2_0 2024-06-28T16:14:33.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16162433081943180286', time=2024-06-28T17:51:29.664024+0000, chain.objs.size()=1 2024-06-28T16:14:33.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.TLwx4O37OOxaI_2Ba11WG0fPyK_lXWY_0 2024-06-28T16:14:33.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10854742574680891212', time=2024-06-28T17:51:30.044275+0000, chain.objs.size()=1 2024-06-28T16:14:33.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.i0HTj7iKDhXySKSmEK5ygnVIBkDJG4g_0 2024-06-28T16:14:33.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13153271688055100524', time=2024-06-28T17:51:30.666469+0000, chain.objs.size()=1 2024-06-28T16:14:33.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fFFnNmIiTjz12sr0cGw2lZ4OkRjWOnu_0 2024-06-28T16:14:33.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6885515706879983272', time=2024-06-28T17:51:32.389618+0000, chain.objs.size()=1 2024-06-28T16:14:33.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.p2TWD17OdsU9xEOm4RtKnBDMWxZqjCU_0 2024-06-28T16:14:33.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.087+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1745788582426164619', time=2024-06-28T17:51:32.417517+0000, chain.objs.size()=1 2024-06-28T16:14:33.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.087+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PKd7MqVaEQopBJm4QHEVxVWNVNzjie-_0 2024-06-28T16:14:33.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.087+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4641330290777536128', time=2024-06-28T17:51:32.901786+0000, chain.objs.size()=1 2024-06-28T16:14:33.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.087+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.80mqy57bTCOu21WoNM1jEYT87U-2VHf_0 2024-06-28T16:14:33.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.087+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17689610812676540781', time=2024-06-28T17:51:33.042758+0000, chain.objs.size()=1 2024-06-28T16:14:33.097 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.087+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.HPt5AL98ynVEN3Z0Hw5F_V9JbnZ4SfR_0 2024-06-28T16:14:33.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4538625903363954472', time=2024-06-28T17:51:33.598946+0000, chain.objs.size()=1 2024-06-28T16:14:33.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.m62dvmiwNnaWvW1STIDFubRxWH6z8Wm_0 2024-06-28T16:14:33.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5683207034295363167', time=2024-06-28T17:51:33.747779+0000, chain.objs.size()=1 2024-06-28T16:14:33.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PnI83tD8Tcllm7bov2Ja9HkZaz1MZ8k_0 2024-06-28T16:14:33.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18409491834324439025', time=2024-06-28T17:51:34.008354+0000, chain.objs.size()=1 2024-06-28T16:14:33.099 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hcJIjAV4ZzmITHtFlf3Ri0YMOQsutwB_0 2024-06-28T16:14:33.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2246539763799248164', time=2024-06-28T17:51:35.856809+0000, chain.objs.size()=1 2024-06-28T16:14:33.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.2fkJpaXnQIXjgg4a06fxtRYfDnaX57C_0 2024-06-28T16:14:33.100 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1580991233765731528', time=2024-06-28T17:51:36.385998+0000, chain.objs.size()=1 2024-06-28T16:14:33.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-awZAX_qkEJdm5Gyh3-hIT0yzsBjF7C_0 2024-06-28T16:14:33.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.298507931736417428', time=2024-06-28T17:51:36.670920+0000, chain.objs.size()=1 2024-06-28T16:14:33.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ESN-nEdVrX-s1gbnFV0WEC_zHMCEvim_0 2024-06-28T16:14:33.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8021624823968742900', time=2024-06-28T17:51:37.447355+0000, chain.objs.size()=1 2024-06-28T16:14:33.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.V8-9GMLC117AuF-TrWhCsAzfd9gmu2R_0 2024-06-28T16:14:33.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9764356473859884402', time=2024-06-28T17:51:38.298590+0000, chain.objs.size()=1 2024-06-28T16:14:33.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.IE3KzRdO9DNMkBwm2MfrN_U7PCcezyj_0 2024-06-28T16:14:33.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16310164783525194108', time=2024-06-28T17:53:20.904766+0000, chain.objs.size()=1 2024-06-28T16:14:33.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.6b7C1my1HqhMfX7OlN7vwom-3PMJpn2_0 2024-06-28T16:14:33.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8400390495564869070', time=2024-06-28T17:53:22.377117+0000, chain.objs.size()=1 2024-06-28T16:14:33.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-6l-lFEy-YGfB3S-MJa_1O-uZgkP9m4_0 2024-06-28T16:14:33.102 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3664104750633854823', time=2024-06-28T17:53:23.457417+0000, chain.objs.size()=1 2024-06-28T16:14:33.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.091+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.3ItmO-thZKydlwEETftyEHzg_iYLPul_0 2024-06-28T16:14:33.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2295464625932592540', time=2024-06-28T17:53:23.704138+0000, chain.objs.size()=1 2024-06-28T16:14:33.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-QNj-qGoF00jCUL8gNb0KWzM07GsR6o_0 2024-06-28T16:14:33.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16184651140958783649', time=2024-06-28T17:53:25.431268+0000, chain.objs.size()=1 2024-06-28T16:14:33.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bqLu4tARpbUIlzdZIH6_Kfn5AoKzlyy_0 2024-06-28T16:14:33.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4303384335712010328', time=2024-06-28T17:53:26.605959+0000, chain.objs.size()=1 2024-06-28T16:14:33.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.OBAtazr6xVuI9Cr2sVW280v1nsZy6ju_0 2024-06-28T16:14:33.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.966717936391521332', time=2024-06-28T17:53:27.086036+0000, chain.objs.size()=1 2024-06-28T16:14:33.105 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gVkXWWt4l61cxp-o8h99SkNoJfDRlsp_0 2024-06-28T16:14:33.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2682089030339450071', time=2024-06-28T17:53:27.396932+0000, chain.objs.size()=1 2024-06-28T16:14:33.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.jznKxZFd4fsBL83hl6cgVikxgNlho3l_0 2024-06-28T16:14:33.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12086801309782908003', time=2024-06-28T17:53:27.601232+0000, chain.objs.size()=1 2024-06-28T16:14:33.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Ztgh2ObOC2UFzaw1XsLHtU0IXpkf15v_0 2024-06-28T16:14:33.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15523226683680479827', time=2024-06-28T17:53:29.453765+0000, chain.objs.size()=1 2024-06-28T16:14:33.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.MfM8ESOk27VsOQr34518E3b3epFeGVf_0 2024-06-28T16:14:33.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10632837332737957142', time=2024-06-28T17:53:30.528759+0000, chain.objs.size()=1 2024-06-28T16:14:33.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.HrBSJgXJ1c117-bA6hq77PMgZT0RA2d_0 2024-06-28T16:14:33.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7136410682362377601', time=2024-06-28T17:53:30.767710+0000, chain.objs.size()=1 2024-06-28T16:14:33.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.uZBO4H5pFh3QhyFdNPbft0lJ4VmS2xL_0 2024-06-28T16:14:33.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4413950329291895846', time=2024-06-28T17:53:30.894586+0000, chain.objs.size()=1 2024-06-28T16:14:33.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.log5HdIRQc60ks23nZtVy2B7g47xzgb_0 2024-06-28T16:14:33.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13630317073638652342', time=2024-06-28T17:53:30.922863+0000, chain.objs.size()=1 2024-06-28T16:14:33.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Y33d7esvPyMA-n5p1OOnhUELeYy4_Bo_0 2024-06-28T16:14:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.099+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17390875028994542794', time=2024-06-28T17:53:31.970906+0000, chain.objs.size()=1 2024-06-28T16:14:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.099+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.rKJYKxs7h5PnS9yu9of2ykMMNmdKg6J_0 2024-06-28T16:14:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.099+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3812560202818572317', time=2024-06-28T17:53:32.386318+0000, chain.objs.size()=1 2024-06-28T16:14:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.099+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.PesTeN-0a82HoG3deWn2DsI4WtkprRi_0 2024-06-28T16:14:33.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.103+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1395314502709680255', time=2024-06-28T17:53:32.565438+0000, chain.objs.size()=1 2024-06-28T16:14:33.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.103+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.IIMZc2E75Z6huzwKllsInVZIoDRkVzi_0 2024-06-28T16:14:33.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.103+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5758960615294682625', time=2024-06-28T17:53:34.054061+0000, chain.objs.size()=1 2024-06-28T16:14:33.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.103+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.RdfwqEG53IBRCVr-pnZmaBLI7UzB5tY_0 2024-06-28T16:14:33.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/43948 2024-06-28T16:14:33.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=32, truncated=0, next_marker='' 2024-06-28T16:14:33.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4749656114964722536', time=2024-06-28T17:53:34.187807+0000, chain.objs.size()=1 2024-06-28T16:14:33.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.nSOeuQYyUJq0rKtPao8AtupS1GY_Mw8_0 2024-06-28T16:14:33.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11699188144627411176', time=2024-06-28T17:53:35.036200+0000, chain.objs.size()=1 2024-06-28T16:14:33.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ecyO_01pBtrWMCA_fZCxVgaBhLC48ve_0 2024-06-28T16:14:33.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2421413070768238262', time=2024-06-28T17:53:35.241733+0000, chain.objs.size()=1 2024-06-28T16:14:33.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Pb2YpjD0IjnPpjD5rkc2kd1_thpn9J__0 2024-06-28T16:14:33.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9234934439142139727', time=2024-06-28T17:53:35.494751+0000, chain.objs.size()=1 2024-06-28T16:14:33.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.19WWegFKmINZmDx-hQ2Z-xX8_yvMkUe_0 2024-06-28T16:14:33.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3210076848289572854', time=2024-06-28T17:53:39.729112+0000, chain.objs.size()=1 2024-06-28T16:14:33.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4D4qwIehALXyGOSRvTxYmF9b4Sey94L_0 2024-06-28T16:14:33.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8335914959875262382', time=2024-06-28T17:53:40.044207+0000, chain.objs.size()=1 2024-06-28T16:14:33.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.MRJ2yzVFe2fLR1oFwnf1duHDNpHXm1n_0 2024-06-28T16:14:33.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8576220277760688062', time=2024-06-28T17:53:40.115159+0000, chain.objs.size()=1 2024-06-28T16:14:33.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CNvVIkl3xKrUltsfQ1oU8wejEelmZkr_0 2024-06-28T16:14:33.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7500490253347184218', time=2024-06-28T17:53:40.909942+0000, chain.objs.size()=1 2024-06-28T16:14:33.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.H9lBKPNptp1yozxZ9iMzyvGUvKkWeCH_0 2024-06-28T16:14:33.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14289906143884710249', time=2024-06-28T17:53:43.499716+0000, chain.objs.size()=1 2024-06-28T16:14:33.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wFcOv-_sMFT7R8ueuRZ2qr1kgvlb7lj_0 2024-06-28T16:14:33.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8939967770012928218', time=2024-06-28T17:53:43.686295+0000, chain.objs.size()=1 2024-06-28T16:14:33.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.5y-XQ3cHtzx_Ng6CroQUR1QDgzaShyi_0 2024-06-28T16:14:33.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14825092626021293983', time=2024-06-28T17:53:45.903569+0000, chain.objs.size()=1 2024-06-28T16:14:33.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.SX3kno8GAMUEw2tRAmfdLnl4_0yP6kF_0 2024-06-28T16:14:33.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.441657947544171407', time=2024-06-28T17:53:46.944196+0000, chain.objs.size()=1 2024-06-28T16:14:33.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_._AEpdZ_uW6ZoHTpI2eQk6_zUaovF-oz_0 2024-06-28T16:14:33.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13836661428513186439', time=2024-06-28T17:53:48.044726+0000, chain.objs.size()=1 2024-06-28T16:14:33.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.AZws6hlWt7DyABtVN47lQEmasuetofb_0 2024-06-28T16:14:33.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.924425970860242827', time=2024-06-28T17:53:48.091241+0000, chain.objs.size()=1 2024-06-28T16:14:33.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.lABlO17-n1doS-uGLqSaUdjMM2NtR9r_0 2024-06-28T16:14:33.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9159166710409819098', time=2024-06-28T17:53:49.028798+0000, chain.objs.size()=1 2024-06-28T16:14:33.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.cBgl8d5cIyq6rVU61-o5CijoECaMI5V_0 2024-06-28T16:14:33.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4742881184830195698', time=2024-06-28T17:53:50.403986+0000, chain.objs.size()=1 2024-06-28T16:14:33.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1pHqsMpLIN7ZxodLxQdBQT3eo02h6lT_0 2024-06-28T16:14:33.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3511007338779663135', time=2024-06-28T17:53:51.309018+0000, chain.objs.size()=1 2024-06-28T16:14:33.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.nOYsfzlinBpOoagPDtqgD1flTAwyHff_0 2024-06-28T16:14:33.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8518947505835890296', time=2024-06-28T17:53:51.310418+0000, chain.objs.size()=1 2024-06-28T16:14:33.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.cWuERXn0H7lwuFaMH_fYsZVrgr5r05B_0 2024-06-28T16:14:33.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14531341420673559975', time=2024-06-28T17:54:41.118550+0000, chain.objs.size()=1 2024-06-28T16:14:33.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.369__shadow_.vOLfPbf0JLsO94jfubJzUL2N3SvLymk_0 2024-06-28T16:14:33.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5216988420423780734', time=2024-06-28T17:55:10.289253+0000, chain.objs.size()=1 2024-06-28T16:14:33.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.399__shadow_.awemOL5mvAwRjstv85gU1FQU4kXtDIP_0 2024-06-28T16:14:33.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1920841929390649684', time=2024-06-28T17:56:08.342592+0000, chain.objs.size()=1 2024-06-28T16:14:33.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.453__shadow_.DwFAYQEo_fQlBZNZQMOXocZIFyBdJU0_0 2024-06-28T16:14:33.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1474939627590909739', time=2024-06-28T17:56:29.670592+0000, chain.objs.size()=1 2024-06-28T16:14:33.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.474__shadow_.RN9qz-0LresANG56GiouOPUw0-wycZF_0 2024-06-28T16:14:33.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.115+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.493__shadow:rinwpm3WJlZ1AcGvTYYbpbEzG8SjNkf_.0OEFPT4L0L8Hzv3rUAZWjoQj5ExOwB1_1_a00ed3902b5c3a9ce9513a1c63c16bd2', time=2024-06-28T17:56:54.975721+0000, chain.objs.size()=1 2024-06-28T16:14:33.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.115+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.493__shadow:rinwpm3WJlZ1AcGvTYYbpbEzG8SjNkf_.0OEFPT4L0L8Hzv3rUAZWjoQj5ExOwB1_0 2024-06-28T16:14:33.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~kbjjgkIYgARYwv9XihPWtoRREJk-50u', time=2024-06-28T18:00:21.620767+0000, chain.objs.size()=2 2024-06-28T16:14:33.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.521__multipart_mymultipart.2~kbjjgkIYgARYwv9XihPWtoRREJk-50u.1 2024-06-28T16:14:33.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.521__multipart_mymultipart.2~kbjjgkIYgARYwv9XihPWtoRREJk-50u.2 2024-06-28T16:14:33.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10591491057686232565', time=2024-06-28T18:01:20.501038+0000, chain.objs.size()=1 2024-06-28T16:14:33.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.543__shadow_.l9a4vQrPQt0mHWnDOA4Ra9T1PwoPbjg_0 2024-06-28T16:14:33.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4846893848847299448', time=2024-06-28T18:01:52.158266+0000, chain.objs.size()=1 2024-06-28T16:14:33.128 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.552__shadow_.zb5ZIijI-vavoP_rHZyDG7CKBc1d-5B_0 2024-06-28T16:14:33.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3841002212072460408', time=2024-06-28T18:02:22.238070+0000, chain.objs.size()=1 2024-06-28T16:14:33.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.559__shadow:z7gIgTuEXhUN8DGpkFIeL7TRHSLwgfC_.TDLuX1D8udoNwemaRg-RkbNBMUN5jXJ_0 2024-06-28T16:14:33.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2001055559591053806', time=2024-06-28T18:03:01.297531+0000, chain.objs.size()=1 2024-06-28T16:14:33.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:YpNof0-0eg6jaMtOFuq8beWbAAe4OMu_.ARtzD5CRf4YKgq6S-oljDp8J7-o5jtW_0 2024-06-28T16:14:33.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:b9JE41FCZ0Y5wSk5sOO07Df6tJ9KFRg_.W4vEkcA5YdpwymBiS2unbImypMQZuyR_1_5d2997b6dfe18dad1d6c1b835802e8f7', time=2024-06-28T18:04:01.225650+0000, chain.objs.size()=1 2024-06-28T16:14:33.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:b9JE41FCZ0Y5wSk5sOO07Df6tJ9KFRg_.W4vEkcA5YdpwymBiS2unbImypMQZuyR_0 2024-06-28T16:14:33.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.315122891484750593', time=2024-06-28T18:06:54.885164+0000, chain.objs.size()=1 2024-06-28T16:14:33.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.619__shadow_.eYNjZuTiK5UFep-452i-Jqv7zJJUTqI_0 2024-06-28T16:14:33.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8942214461364486283', time=2024-06-28T18:08:33.615616+0000, chain.objs.size()=1 2024-06-28T16:14:33.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.682__shadow:GRICTzNpjQPdgSbeYhYntBAZGAhTSDX_.-vcWaNcNa9R70mHOyDbsBTMTBNqune__0 2024-06-28T16:14:33.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12849123150955984447', time=2024-06-28T18:08:34.942346+0000, chain.objs.size()=1 2024-06-28T16:14:33.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.683__shadow:GzRZWmSXpy0nVNl7EjIVhdfWKqIxWSY_.pPUAxwUsV0ajvejBznJ2E1xHOSMZ7jf_0 2024-06-28T16:14:33.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.123+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:33.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-06-28T16:14:33.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:33.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:33.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/48456' 2024-06-28T16:14:33.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2412008762553894694', time=2024-06-28T17:40:17.385751+0000, chain.objs.size()=2 2024-06-28T16:14:33.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.9klf2cCUcwdDOhyxLcaRwrxuoZpzYCk_0 2024-06-28T16:14:33.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.9klf2cCUcwdDOhyxLcaRwrxuoZpzYCk_1 2024-06-28T16:14:33.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_AVgg4HxKE0GzluGFaSOHyY5iSpyf-OT', time=2024-06-28T17:40:21.918533+0000, chain.objs.size()=2 2024-06-28T16:14:33.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.LjOcCUl9E6ed7M-jueM6d2O8xWTT0qY_0 2024-06-28T16:14:33.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.LjOcCUl9E6ed7M-jueM6d2O8xWTT0qY_1 2024-06-28T16:14:33.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3005365046882031983', time=2024-06-28T17:41:43.292462+0000, chain.objs.size()=1 2024-06-28T16:14:33.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.187__shadow_.bJcMN0AvfZLgJ4t_U-nngDgpr2XKZm-_0 2024-06-28T16:14:33.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1402139381557260226', time=2024-06-28T17:45:09.880268+0000, chain.objs.size()=1 2024-06-28T16:14:33.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.241__shadow_.JmJgMQReYICldWA18D0PNifo5yW5xss_0 2024-06-28T16:14:33.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1641596827608766128', time=2024-06-28T17:47:07.813802+0000, chain.objs.size()=1 2024-06-28T16:14:33.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4sg9eMAaoWOtviulx_c5_Wx032o6nJF_0 2024-06-28T16:14:33.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1282310378850337181', time=2024-06-28T17:47:09.495259+0000, chain.objs.size()=1 2024-06-28T16:14:33.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.bauHW910mFctuCHU_hEsN9Ri2RlpCyZ_0 2024-06-28T16:14:33.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15023118079540477007', time=2024-06-28T17:47:10.614372+0000, chain.objs.size()=1 2024-06-28T16:14:33.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7qeqxeAJvA1y4mjoHFlBrvZvzAek8u0_0 2024-06-28T16:14:33.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6328034139181564777', time=2024-06-28T17:47:10.962061+0000, chain.objs.size()=1 2024-06-28T16:14:33.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.bFap_GvGDO4rKyFlYbUNa25aXdM8gs0_0 2024-06-28T16:14:33.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5750922484591091744', time=2024-06-28T17:47:12.361258+0000, chain.objs.size()=1 2024-06-28T16:14:33.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ih5oJ6yRAESQBwO2fSgP4PPEmtU5DAe_0 2024-06-28T16:14:33.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2661928618210527183', time=2024-06-28T17:47:13.263244+0000, chain.objs.size()=1 2024-06-28T16:14:33.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.BLpIK7M33BeKfobeBmi6es02Jwt_weI_0 2024-06-28T16:14:33.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9626647006567611692', time=2024-06-28T17:47:14.275579+0000, chain.objs.size()=1 2024-06-28T16:14:33.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.jutd7y9PQ3-y62XsHjIhyapsWazmvlF_0 2024-06-28T16:14:33.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14280676756024781696', time=2024-06-28T17:47:15.484219+0000, chain.objs.size()=1 2024-06-28T16:14:33.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.sAznt-ufAfKBa8B9Y4m8Xmfdsfu__o3_0 2024-06-28T16:14:33.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14638290495205263577', time=2024-06-28T17:47:16.960813+0000, chain.objs.size()=1 2024-06-28T16:14:33.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7-ndvPOo9DWPF9CtK15VGW4gDxRFv5X_0 2024-06-28T16:14:33.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15075529114451923300', time=2024-06-28T17:47:17.020503+0000, chain.objs.size()=1 2024-06-28T16:14:33.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KWnA6uwtzwGv_w9wUEZ3_RsEaFlfPXT_0 2024-06-28T16:14:33.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.135+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17100211854135908603', time=2024-06-28T17:47:18.506410+0000, chain.objs.size()=1 2024-06-28T16:14:33.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.135+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.SUi2KXuX2fsc1HlyMqMGWG5Roz9LImC_0 2024-06-28T16:14:33.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.135+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5309382151351747047', time=2024-06-28T17:47:18.508009+0000, chain.objs.size()=1 2024-06-28T16:14:33.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.135+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.uj4s9i7q7VgxMh4nuWpFipdtpJgPlMW_0 2024-06-28T16:14:33.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.135+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9297417536796321285', time=2024-06-28T17:47:18.764842+0000, chain.objs.size()=1 2024-06-28T16:14:33.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.135+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.EZajPNXq6egarRfX_dHYq7iE12_B49J_0 2024-06-28T16:14:33.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.135+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7960965445976862915', time=2024-06-28T17:47:22.746963+0000, chain.objs.size()=1 2024-06-28T16:14:33.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.135+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.r2iotsLBrrwHXKKTk9ldxVdip0rrp41_0 2024-06-28T16:14:33.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.135+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6081042168462716924', time=2024-06-28T17:47:25.656188+0000, chain.objs.size()=1 2024-06-28T16:14:33.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.135+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.mA5octOT2WSxK5hZRuaJKpM6U10RT0R_0 2024-06-28T16:14:33.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.135+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18361305022999314848', time=2024-06-28T17:47:26.620375+0000, chain.objs.size()=1 2024-06-28T16:14:33.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.135+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.38QLFdrR5dDplbaeTdEcmH6QNSiVJ4u_0 2024-06-28T16:14:33.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.135+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8598498224121424236', time=2024-06-28T17:47:28.136527+0000, chain.objs.size()=1 2024-06-28T16:14:33.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.135+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.eRSQyBlV44uL7c54QdfOrJmG3FQTiVI_0 2024-06-28T16:14:33.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10112389118293714043', time=2024-06-28T17:47:29.856240+0000, chain.objs.size()=1 2024-06-28T16:14:33.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Yo_7kQqZ8uvfypnPlacJKfOuMFInjJ9_0 2024-06-28T16:14:33.147 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2840434304083460344', time=2024-06-28T17:47:31.957114+0000, chain.objs.size()=1 2024-06-28T16:14:33.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9vdjoau2Yfl104j1G0Zm88uij8IDCWX_0 2024-06-28T16:14:33.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8810247270696359054', time=2024-06-28T17:47:32.143182+0000, chain.objs.size()=1 2024-06-28T16:14:33.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.XCIpLt0oAzH8BQZey6e6Itaw4n8xwSQ_0 2024-06-28T16:14:33.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10970765867279791399', time=2024-06-28T17:47:34.090329+0000, chain.objs.size()=1 2024-06-28T16:14:33.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.PC4gEd8Z0IUKanuD4M4uoIB1wmwCaKz_0 2024-06-28T16:14:33.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6741394239177257983', time=2024-06-28T17:47:34.296497+0000, chain.objs.size()=1 2024-06-28T16:14:33.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.nuyQxmDjAxqhf34kS3KrAhPtnMDbK1u_0 2024-06-28T16:14:33.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2582663100870171167', time=2024-06-28T17:47:34.811325+0000, chain.objs.size()=1 2024-06-28T16:14:33.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.YUqK2MXPMBCtH36ac0a3CZ29g4IuuEX_0 2024-06-28T16:14:33.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8287955708435657156', time=2024-06-28T17:47:34.813106+0000, chain.objs.size()=1 2024-06-28T16:14:33.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.1wmn5_VpZhbnjbB30g5ObrIzAG-79m3_0 2024-06-28T16:14:33.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13253355003920600847', time=2024-06-28T17:47:34.815026+0000, chain.objs.size()=1 2024-06-28T16:14:33.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.clUb3fSS4PiauciIVfCWC8rvO_DFkSk_0 2024-06-28T16:14:33.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17234611546484806075', time=2024-06-28T17:47:34.947225+0000, chain.objs.size()=1 2024-06-28T16:14:33.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.6fxh7aa9U3SKtMjZXVTGJOgF6Xyxcyx_0 2024-06-28T16:14:33.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13762844804875055243', time=2024-06-28T17:47:36.169948+0000, chain.objs.size()=1 2024-06-28T16:14:33.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.qmTpmMsnF6Okr31mLTNXMriu9558UnI_0 2024-06-28T16:14:33.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11566416178164488585', time=2024-06-28T17:47:37.074440+0000, chain.objs.size()=1 2024-06-28T16:14:33.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.JhCKT0OqwptHFZCI1t_cXUKhcynDNy2_0 2024-06-28T16:14:33.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4221316785550417288', time=2024-06-28T17:47:37.427708+0000, chain.objs.size()=1 2024-06-28T16:14:33.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ScMwXVGNViQAhLco-Uc65EwT-sLhQlT_0 2024-06-28T16:14:33.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12211839293883948068', time=2024-06-28T17:47:37.573062+0000, chain.objs.size()=1 2024-06-28T16:14:33.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-z6QcqQwPcpQ4t09EUeKsev0yE0r2kP_0 2024-06-28T16:14:33.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13477180358231169848', time=2024-06-28T17:47:37.575091+0000, chain.objs.size()=1 2024-06-28T16:14:33.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.28CaHdPJUFSFJ2o9wQY_PzfeFqmBwhW_0 2024-06-28T16:14:33.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16530998040486933320', time=2024-06-28T17:47:39.122496+0000, chain.objs.size()=1 2024-06-28T16:14:33.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Tdqb3KUDv3qB_8Jg25nM7vhJzgnuAua_0 2024-06-28T16:14:33.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.893728076805357378', time=2024-06-28T17:48:34.853349+0000, chain.objs.size()=1 2024-06-28T16:14:33.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.VGzja-ekyhOIDEPmN2X3OViAPvsdaZj_0 2024-06-28T16:14:33.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10161625086670217688', time=2024-06-28T17:48:56.366129+0000, chain.objs.size()=1 2024-06-28T16:14:33.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.296__shadow_.Rrzf4pRcxsjMdjMJH7p_CcY6zV0myen_0 2024-06-28T16:14:33.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.147+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16130604256167165582', time=2024-06-28T17:51:07.977683+0000, chain.objs.size()=1 2024-06-28T16:14:33.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.147+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.CypYqdjh_Zvr1pEYbCaSS__3QodsOyV_0 2024-06-28T16:14:33.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.147+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1170419371048365274', time=2024-06-28T17:51:08.758072+0000, chain.objs.size()=1 2024-06-28T16:14:33.156 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.147+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Cf9-cozOR-LiG90C4IvqCjpyBKJhxpY_0 2024-06-28T16:14:33.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.147+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16834979048928193608', time=2024-06-28T17:51:08.933669+0000, chain.objs.size()=1 2024-06-28T16:14:33.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.147+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ydznGYEFlp8ejF3izuV1N3eOSAK-DeL_0 2024-06-28T16:14:33.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.147+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4247166319372273001', time=2024-06-28T17:51:09.966155+0000, chain.objs.size()=1 2024-06-28T16:14:33.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.147+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.DgTdj-2IbVxovRyFcY3tdizTwywgbVi_0 2024-06-28T16:14:33.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.147+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5935841197379710824', time=2024-06-28T17:51:12.103834+0000, chain.objs.size()=1 2024-06-28T16:14:33.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.147+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.EmBdj29prqTfxIyD9FLwazPWmjf5Zoa_0 2024-06-28T16:14:33.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.147+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13924694992633637571', time=2024-06-28T17:51:12.697908+0000, chain.objs.size()=1 2024-06-28T16:14:33.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.147+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-KqR4bDZwOWDRKhVO38lDtPt6MHhRqd_0 2024-06-28T16:14:33.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.147+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17961267672214767850', time=2024-06-28T17:51:14.220015+0000, chain.objs.size()=1 2024-06-28T16:14:33.159 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.147+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.I1MfwMVti1u-B6IMHufd3xz3i33PP9T_0 2024-06-28T16:14:33.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.151+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6667211964931995384', time=2024-06-28T17:51:15.480121+0000, chain.objs.size()=1 2024-06-28T16:14:33.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.151+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.avt5NpI5S67-Gdf2j4NnuNO6v0F0_b__0 2024-06-28T16:14:33.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.151+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17484473234944168606', time=2024-06-28T17:51:16.315115+0000, chain.objs.size()=1 2024-06-28T16:14:33.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.151+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.KGY_kmov9G_i5tzrSxTE2gYUzEH7KD1_0 2024-06-28T16:14:33.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.151+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14987621112535475431', time=2024-06-28T17:51:17.358798+0000, chain.objs.size()=1 2024-06-28T16:14:33.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.151+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xvxdYqnYJeTSZSwr7jZ4Uz-pzcTOhwM_0 2024-06-28T16:14:33.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.151+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7213986980141287081', time=2024-06-28T17:51:19.921664+0000, chain.objs.size()=1 2024-06-28T16:14:33.161 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.151+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xVONO7KiaGEIPAhVfus-9AVyaCw3Ftu_0 2024-06-28T16:14:33.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.151+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13326427142839121043', time=2024-06-28T17:51:20.434965+0000, chain.objs.size()=1 2024-06-28T16:14:33.162 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.151+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6scAf6tN0JTq65C7dBXlqdTaRrqKtzG_0 2024-06-28T16:14:33.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11067416025710873016', time=2024-06-28T17:51:20.545354+0000, chain.objs.size()=1 2024-06-28T16:14:33.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.586jtlzSNDDAYpw3CfH6scWVJsPXVuM_0 2024-06-28T16:14:33.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10747533502269326094', time=2024-06-28T17:51:20.652615+0000, chain.objs.size()=1 2024-06-28T16:14:33.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.UDQs715-VGZlzBivBKhicnTouyF_2-c_0 2024-06-28T16:14:33.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10042547835824421788', time=2024-06-28T17:51:20.766976+0000, chain.objs.size()=1 2024-06-28T16:14:33.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ga7OP5ZJeu7ng--KRmWcjowY2Gn23wC_0 2024-06-28T16:14:33.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12426319797825215712', time=2024-06-28T17:51:21.041061+0000, chain.objs.size()=1 2024-06-28T16:14:33.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.4oNQuk3FU_Ipt5aervLOOEy0HO0YJ9b_0 2024-06-28T16:14:33.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15433689098290182038', time=2024-06-28T17:51:21.042892+0000, chain.objs.size()=1 2024-06-28T16:14:33.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Y-LzGuibmoLmjbsnInkMQRuu9NOsTHR_0 2024-06-28T16:14:33.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10143556466685161169', time=2024-06-28T17:51:25.662566+0000, chain.objs.size()=1 2024-06-28T16:14:33.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_._MnbJtILjwSmCkzPdiLMBtQjOBflPN9_0 2024-06-28T16:14:33.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7861074350729426975', time=2024-06-28T17:51:25.913640+0000, chain.objs.size()=1 2024-06-28T16:14:33.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.C6du4DLF1WXDEY4nuY7CEpSlNMfrMMe_0 2024-06-28T16:14:33.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3279466041110983196', time=2024-06-28T17:51:26.342220+0000, chain.objs.size()=1 2024-06-28T16:14:33.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.3mtLh6aSy5T6I3zRH-luB8J9eg9skKE_0 2024-06-28T16:14:33.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18208599139132746826', time=2024-06-28T17:51:27.723451+0000, chain.objs.size()=1 2024-06-28T16:14:33.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.iEdZysj-iTOwsjCVO-Nft8bDYoKNZk8_0 2024-06-28T16:14:33.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16517322862828911217', time=2024-06-28T17:51:27.918552+0000, chain.objs.size()=1 2024-06-28T16:14:33.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.evetj1QM5tKm9qJR06NQ3cPVmCbQ4jN_0 2024-06-28T16:14:33.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15563601532607238530', time=2024-06-28T17:51:28.680983+0000, chain.objs.size()=1 2024-06-28T16:14:33.167 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.80s8qvdWjgwyLlTms95lbfuI55qthnS_0 2024-06-28T16:14:33.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1799849393758220391', time=2024-06-28T17:51:29.078494+0000, chain.objs.size()=1 2024-06-28T16:14:33.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.BpMMIvyAQzRSgSmKE1GIi81NG8fV5ZB_0 2024-06-28T16:14:33.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8373492320091742609', time=2024-06-28T17:51:30.045535+0000, chain.objs.size()=1 2024-06-28T16:14:33.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.d3G6tIqS5AA5Blkg_y1OqBO7WrPsJtp_0 2024-06-28T16:14:33.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5343881784786317757', time=2024-06-28T17:51:31.011043+0000, chain.objs.size()=1 2024-06-28T16:14:33.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Zuh0d1fVNpCKIcwyiCz5X-Ji8r1VgOL_0 2024-06-28T16:14:33.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17312205395784934730', time=2024-06-28T17:51:32.672081+0000, chain.objs.size()=1 2024-06-28T16:14:33.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.c4F-75ea1x8sgYtx09b2hHxPmj3exVj_0 2024-06-28T16:14:33.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16404889664850034523', time=2024-06-28T17:51:33.996440+0000, chain.objs.size()=1 2024-06-28T16:14:33.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ZRtUGQLES-Bpm-TIM80VkfJzPLsgazg_0 2024-06-28T16:14:33.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5305310602162630513', time=2024-06-28T17:51:34.098944+0000, chain.objs.size()=1 2024-06-28T16:14:33.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.qkNZVLQDIDbkwqPdlHDdZIoZ-OOn27G_0 2024-06-28T16:14:33.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17050051062541769637', time=2024-06-28T17:51:34.208063+0000, chain.objs.size()=1 2024-06-28T16:14:33.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YlPEC73n975ldfArKv-CBL_PW-E2v7H_0 2024-06-28T16:14:33.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17632904731761332885', time=2024-06-28T17:51:36.853940+0000, chain.objs.size()=1 2024-06-28T16:14:33.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.159+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.4KHuFaiGm7UuNjHgEbItKEnz4h99Kzi_0 2024-06-28T16:14:33.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.163+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.927982605275691321', time=2024-06-28T17:53:22.161419+0000, chain.objs.size()=1 2024-06-28T16:14:33.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.163+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.0Tw8Os_avUEJFRJWumzrE0CLpAWVcr1_0 2024-06-28T16:14:33.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.163+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12213884483120483884', time=2024-06-28T17:53:24.015794+0000, chain.objs.size()=1 2024-06-28T16:14:33.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.163+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.8KGZAHSuy7PaPi4T_MAJ1fhU_lB4GVl_0 2024-06-28T16:14:33.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.163+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1434332064777544885', time=2024-06-28T17:53:25.433766+0000, chain.objs.size()=1 2024-06-28T16:14:33.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.163+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.jemDP-ICH2RbOSGYzh8zLDg_Zm30zuH_0 2024-06-28T16:14:33.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.163+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6186050450197406399', time=2024-06-28T17:53:27.688947+0000, chain.objs.size()=1 2024-06-28T16:14:33.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.163+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZMf3Ob8wLwGg_LPxx-w30Hq32rLfkfu_0 2024-06-28T16:14:33.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10334806453194704105', time=2024-06-28T17:53:28.176486+0000, chain.objs.size()=1 2024-06-28T16:14:33.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.fJzsHbflYD8Oe_kuuAxt0W2IVd_UN-s_0 2024-06-28T16:14:33.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1528187285728886190', time=2024-06-28T17:53:28.387181+0000, chain.objs.size()=1 2024-06-28T16:14:33.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gaqDYg-jHXh6mKBR6I6uasleUZRTiQV_0 2024-06-28T16:14:33.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12037836053166042697', time=2024-06-28T17:53:29.476350+0000, chain.objs.size()=1 2024-06-28T16:14:33.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ld1Cf67I3MFo1NmPA5cANK51XjOCUs0_0 2024-06-28T16:14:33.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3348855027772350050', time=2024-06-28T17:53:29.995682+0000, chain.objs.size()=1 2024-06-28T16:14:33.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_._z_J1zK1UY5PoTixAj3SW16mfVpxo-S_0 2024-06-28T16:14:33.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17020797968136755270', time=2024-06-28T17:53:30.989517+0000, chain.objs.size()=1 2024-06-28T16:14:33.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.nvCODYiWGOppjH4PkCXkLHyDHM6cR4X_0 2024-06-28T16:14:33.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10785086445405655711', time=2024-06-28T17:53:31.048064+0000, chain.objs.size()=1 2024-06-28T16:14:33.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.fq5R1vZSaB8QAF-sKmNGfFqfucyOIBC_0 2024-06-28T16:14:33.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7340052684944184768', time=2024-06-28T17:53:32.016593+0000, chain.objs.size()=1 2024-06-28T16:14:33.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.WrwWhESNB77_EzxnF3IvIakC7BfRFqD_0 2024-06-28T16:14:33.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5453441631453896491', time=2024-06-28T17:53:32.289305+0000, chain.objs.size()=1 2024-06-28T16:14:33.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.VTip0nGQ3V3rk_2SIAhRHyYjvEWnal1_0 2024-06-28T16:14:33.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13725055722588913746', time=2024-06-28T17:53:33.660994+0000, chain.objs.size()=1 2024-06-28T16:14:33.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.mJ1cHqgkO1ay6b56QbY0yC5s3Mfciyo_0 2024-06-28T16:14:33.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.81284541431389286', time=2024-06-28T17:53:36.367213+0000, chain.objs.size()=1 2024-06-28T16:14:33.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.72MNy0o3XcM7ez2ErWQeIaFuNPYzZUY_0 2024-06-28T16:14:33.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10523245619311156341', time=2024-06-28T17:53:36.570067+0000, chain.objs.size()=1 2024-06-28T16:14:33.178 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bT6U5sSis3rv1HGln_dhAbvdgRqi-yH_0 2024-06-28T16:14:33.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7052861623178169897', time=2024-06-28T17:53:38.398639+0000, chain.objs.size()=1 2024-06-28T16:14:33.181 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.lNjOtPrWJkz1csoc8ouUMXZgUnikb5Q_0 2024-06-28T16:14:33.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.677530525991773224', time=2024-06-28T17:53:40.380835+0000, chain.objs.size()=1 2024-06-28T16:14:33.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.sk2TN5DaELP8tWTVMQ0JFWPl7Tae02x_0 2024-06-28T16:14:33.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7777082165564228180', time=2024-06-28T17:53:42.304411+0000, chain.objs.size()=1 2024-06-28T16:14:33.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.SAfYLSbr5tkfFsx2-RbN_AdrDelv4TF_0 2024-06-28T16:14:33.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10621112491368962298', time=2024-06-28T17:53:42.656652+0000, chain.objs.size()=1 2024-06-28T16:14:33.182 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Z10tD-RF9bOnmAMXcwHiXtJPJBySXOf_0 2024-06-28T16:14:33.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11645268580842667379', time=2024-06-28T17:53:42.931640+0000, chain.objs.size()=1 2024-06-28T16:14:33.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gMUNaruA5-DWbdA990GRBmTKyYKOLZs_0 2024-06-28T16:14:33.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17690856416626876136', time=2024-06-28T17:53:44.548313+0000, chain.objs.size()=1 2024-06-28T16:14:33.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.noFDCm3CM0WcX1-iRUfqeUfrM6v1fLY_0 2024-06-28T16:14:33.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6404482846750049870', time=2024-06-28T17:53:46.333909+0000, chain.objs.size()=1 2024-06-28T16:14:33.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.xwvJITcAQpW_nTpkK9DRO_DNAXPqWx9_0 2024-06-28T16:14:33.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1942943239544108896', time=2024-06-28T17:53:47.011959+0000, chain.objs.size()=1 2024-06-28T16:14:33.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CCHQV9hdDukycFuVfRzE9jrmCpy-QTB_0 2024-06-28T16:14:33.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15034677692224601225', time=2024-06-28T17:53:47.184574+0000, chain.objs.size()=1 2024-06-28T16:14:33.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.fK6a74bTgztzEx80h42knShpJOpcRcL_0 2024-06-28T16:14:33.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14956988193254295950', time=2024-06-28T17:53:47.363203+0000, chain.objs.size()=1 2024-06-28T16:14:33.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.171+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.tdNqMtmVECWbg1BNVAG1Yeb5CbMVYAF_0 2024-06-28T16:14:33.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4893256143787437183', time=2024-06-28T17:53:47.806718+0000, chain.objs.size()=1 2024-06-28T16:14:33.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.8urKS1sumvaB9b3yhyvYwboZ813Akaz_0 2024-06-28T16:14:33.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14661119644802305779', time=2024-06-28T17:53:49.714310+0000, chain.objs.size()=1 2024-06-28T16:14:33.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.adQ-45gRHNorBK-ol8BZlzcofMfHF6f_0 2024-06-28T16:14:33.185 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.918124979043206724', time=2024-06-28T17:53:50.963332+0000, chain.objs.size()=1 2024-06-28T16:14:33.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Pw9PXlhdLx_Y06R2HegB10FtFBO7utL_0 2024-06-28T16:14:33.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.179+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3878031953230890664', time=2024-06-28T17:54:36.525352+0000, chain.objs.size()=1 2024-06-28T16:14:33.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.179+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.365__shadow_.WTo_60-6o2PwRgYA5HNu-6xgzRz5OlK_0 2024-06-28T16:14:33.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.179+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3234681978613863245', time=2024-06-28T17:54:45.342615+0000, chain.objs.size()=1 2024-06-28T16:14:33.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.179+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.373__shadow_.CYROz1QuC60MguqTtOw4PgbkQn89QMU_0 2024-06-28T16:14:33.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.179+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17608828605961165861', time=2024-06-28T17:55:50.369691+0000, chain.objs.size()=1 2024-06-28T16:14:33.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.179+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.441__shadow_.dwCkWrYJXc08XF2sHKCtnBUnzrvDCDs_0 2024-06-28T16:14:33.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/48456 2024-06-28T16:14:33.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=18, truncated=0, next_marker='' 2024-06-28T16:14:33.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18053301316709259676', time=2024-06-28T17:57:09.017844+0000, chain.objs.size()=6 2024-06-28T16:14:33.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.1 2024-06-28T16:14:33.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.2 2024-06-28T16:14:33.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.3 2024-06-28T16:14:33.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.4 2024-06-28T16:14:33.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.5 2024-06-28T16:14:33.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.6 2024-06-28T16:14:33.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12901583492477848426', time=2024-06-28T17:57:15.482597+0000, chain.objs.size()=1 2024-06-28T16:14:33.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.502__shadow_.3Yo2GOW4bST4MH1KP0NpoKVwBekoJJb_0 2024-06-28T16:14:33.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11166456118068745510', time=2024-06-28T17:58:10.133122+0000, chain.objs.size()=1 2024-06-28T16:14:33.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.510__shadow:NuCm8X6TJq0MY6Ok05nJNJqTA5hpheX_.u48yIoWuLopzn5-i16yh1dMyGyYA7ms_0 2024-06-28T16:14:33.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15031674936831006391', time=2024-06-28T17:59:33.617389+0000, chain.objs.size()=6 2024-06-28T16:14:33.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.gCThvg693x38lwKMFyUcUvanJBz5q5fz.1 2024-06-28T16:14:33.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.90SbqrB1Mgll_ZSTeOxWvsKLR7foeV5d.2 2024-06-28T16:14:33.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.n71I6dl45qbjwtEpcS9wVkl-XNmAQ-df.3 2024-06-28T16:14:33.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.bCPYtStRr3ewUihkoTNEbJG2qFkMEqMc.4 2024-06-28T16:14:33.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.YWFoX9ypYYBlAm1lnijRl1T8jnKl0m00.5 2024-06-28T16:14:33.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.1CLgD_cdjgTl7TS4pHSg8j7k_VgisJfK.6 2024-06-28T16:14:33.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.621649014383035991', time=2024-06-28T17:59:42.679144+0000, chain.objs.size()=3 2024-06-28T16:14:33.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.513__multipart_mymultipart.2~9a8o2nDF9ZpShDpJwbdHqkNnrLlWAQI.1 2024-06-28T16:14:33.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.513__multipart_mymultipart.2~9a8o2nDF9ZpShDpJwbdHqkNnrLlWAQI.2 2024-06-28T16:14:33.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.513__multipart_mymultipart.2~9a8o2nDF9ZpShDpJwbdHqkNnrLlWAQI.3 2024-06-28T16:14:33.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13959433267056064712', time=2024-06-28T17:59:44.674652+0000, chain.objs.size()=3 2024-06-28T16:14:33.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.513__multipart_mymultipart.2~_v0tvF780oG6SZ-DDF31A9vm_4cwy-X.1 2024-06-28T16:14:33.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.513__multipart_mymultipart.2~_v0tvF780oG6SZ-DDF31A9vm_4cwy-X.2 2024-06-28T16:14:33.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.513__multipart_mymultipart.2~_v0tvF780oG6SZ-DDF31A9vm_4cwy-X.3 2024-06-28T16:14:33.199 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.66956014678014210', time=2024-06-28T18:00:13.464827+0000, chain.objs.size()=1 2024-06-28T16:14:33.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.518__shadow_.8J6n4nBmesMz_HcIP5SI4GmGYbRHI3C_0 2024-06-28T16:14:33.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.712163946557065381', time=2024-06-28T18:01:27.707897+0000, chain.objs.size()=1 2024-06-28T16:14:33.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.545__shadow_.kB03Ndz-h0NHf7lg9D6fJyIJk9Vp0hI_0 2024-06-28T16:14:33.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18189437488662056049', time=2024-06-28T18:01:38.701731+0000, chain.objs.size()=1 2024-06-28T16:14:33.200 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.547__shadow_.ISMBze9isSQcNfhK5VStBqWDPADw9Km_0 2024-06-28T16:14:33.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.191+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13592745696358434173', time=2024-06-28T18:02:39.442018+0000, chain.objs.size()=1 2024-06-28T16:14:33.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.191+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.564__shadow:Av8JzXwP8XUPpFRgVLIQNOChK6V3CB0_.xpPxS209jE6a4V3yMe3Rs9idqEl5U7Q_0 2024-06-28T16:14:33.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.191+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2904088463799753174', time=2024-06-28T18:03:08.509539+0000, chain.objs.size()=1 2024-06-28T16:14:33.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.191+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:pRxUdFXG7hPEFUB6wqQ2Oh8oY-ldEnn_.-k6yGiAlvxPuKrO3yaAnPL1UiWAdUzv_0 2024-06-28T16:14:33.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.191+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3637860297921160157', time=2024-06-28T18:03:56.084145+0000, chain.objs.size()=1 2024-06-28T16:14:33.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.191+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.568__shadow:KaJ.OCkyPycWC4MI6Ti0jGtgflVi07A_.R-j7zpmD8bhMKaVsDGMlrIg20XO9m6g_0 2024-06-28T16:14:33.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.195+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:T-3PRIPFnQthlN2zlbvRSM--a-Jg49Q_._srzhP6oOXAaOJL6gJYGDQX4134NPi3_1_3ea1536ad3e65c19a315ae9e7782e6fb', time=2024-06-28T18:04:01.226738+0000, chain.objs.size()=1 2024-06-28T16:14:33.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.195+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:T-3PRIPFnQthlN2zlbvRSM--a-Jg49Q_._srzhP6oOXAaOJL6gJYGDQX4134NPi3_0 2024-06-28T16:14:33.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.195+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9920711583771466078', time=2024-06-28T18:04:13.681906+0000, chain.objs.size()=1 2024-06-28T16:14:33.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.195+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.576__shadow:7Nh1z5PknwY3dpfCzj8e1bJyF-mUESk_.as8UcdFT7ICn-W3rD9HthuLOhzXuGpV_0 2024-06-28T16:14:33.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.471063540251371016', time=2024-06-28T18:06:02.193017+0000, chain.objs.size()=1 2024-06-28T16:14:33.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.606__shadow_.D6X66J4pHrkiRuwV1bmPK23N5WcUT76_0 2024-06-28T16:14:33.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4065677826237452923', time=2024-06-28T18:06:59.838341+0000, chain.objs.size()=1 2024-06-28T16:14:33.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.623__shadow_.vngVvhwZkWBGOZJymHHNKD5-s36DfsO_0 2024-06-28T16:14:33.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7740909655046675070', time=2024-06-28T18:08:07.253811+0000, chain.objs.size()=1 2024-06-28T16:14:33.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.653__shadow_.GJdd58GBjttuVKI5ZrAwK3QGgcsAkh2_0 2024-06-28T16:14:33.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.653__shadow_.1Am_woH9iXwK5pxEKgh_w_IW2U0BCPm_1_e1d6708ecb90d8adc37296339185b4c2', time=2024-06-28T18:08:07.696466+0000, chain.objs.size()=1 2024-06-28T16:14:33.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.653__shadow_.1Am_woH9iXwK5pxEKgh_w_IW2U0BCPm_0 2024-06-28T16:14:33.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.203+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:33.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.203+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-06-28T16:14:33.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:33.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:33.216 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=99, truncated=0, next_marker='' 2024-06-28T16:14:33.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_4723miErveWxICIBYDUEZOH7ryZz-hh', time=2024-06-28T17:40:30.001286+0000, chain.objs.size()=2 2024-06-28T16:14:33.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.151__shadow_.AA5tiX6wTTD4DNncCfJx6hvaQWUtstD_0 2024-06-28T16:14:33.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.151__shadow_.AA5tiX6wTTD4DNncCfJx6hvaQWUtstD_1 2024-06-28T16:14:33.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15934791002994764990', time=2024-06-28T17:40:33.274423+0000, chain.objs.size()=3 2024-06-28T16:14:33.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.151__multipart_foo-FROZEN.2~zK0BC_3yS2TjKEQA-Gw7nEhA0Pwip3n.1 2024-06-28T16:14:33.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.151__multipart_foo-FROZEN.2~zK0BC_3yS2TjKEQA-Gw7nEhA0Pwip3n.2 2024-06-28T16:14:33.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.151__multipart_foo-FROZEN.2~zK0BC_3yS2TjKEQA-Gw7nEhA0Pwip3n.3 2024-06-28T16:14:33.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3746064208507646561', time=2024-06-28T17:44:42.231979+0000, chain.objs.size()=1 2024-06-28T16:14:33.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.225__shadow_.fBcH5Z_46Yto0sd4h69tBVtEIaHCboe_0 2024-06-28T16:14:33.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3272806302900755278', time=2024-06-28T17:44:43.800043+0000, chain.objs.size()=1 2024-06-28T16:14:33.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.226__shadow_.GZAQJ0DSwN1TpbJffaLbL8VsVNUpBAd_0 2024-06-28T16:14:33.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11164764944281903550', time=2024-06-28T17:45:03.356181+0000, chain.objs.size()=1 2024-06-28T16:14:33.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.237__shadow_.rp0deJWg6ZqX9T-aSnyp85zz0-6kup9_0 2024-06-28T16:14:33.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8591512549029925437', time=2024-06-28T17:45:11.448158+0000, chain.objs.size()=1 2024-06-28T16:14:33.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.242__shadow_.mWNgMWL2B6J1ZS_gTUu6mg4IJz44zSs_0 2024-06-28T16:14:33.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15843115722638401151', time=2024-06-28T17:47:11.068562+0000, chain.objs.size()=1 2024-06-28T16:14:33.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ySY2wI62ce-qtOT0bOEVxyX60IeGe7W_0 2024-06-28T16:14:33.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13004063592796567890', time=2024-06-28T17:47:12.762546+0000, chain.objs.size()=1 2024-06-28T16:14:33.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.AbYQwV2XGJVjY-Ryjqkb1GskUj3c90M_0 2024-06-28T16:14:33.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16609864799205640419', time=2024-06-28T17:47:13.743900+0000, chain.objs.size()=1 2024-06-28T16:14:33.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.BP6mcY2PGtWQvdXcB4HUjJut8ZK4FY-_0 2024-06-28T16:14:33.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.876607662022868444', time=2024-06-28T17:47:13.967172+0000, chain.objs.size()=1 2024-06-28T16:14:33.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.UiPplkrlpJrIjJx-jUgQH-wQgt5E5CC_0 2024-06-28T16:14:33.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1695568234201962604', time=2024-06-28T17:47:15.489069+0000, chain.objs.size()=1 2024-06-28T16:14:33.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.OiMCPjHZsKQpr2voVlmnNU61FOkWSVI_0 2024-06-28T16:14:33.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3774865911202426240', time=2024-06-28T17:47:17.373903+0000, chain.objs.size()=1 2024-06-28T16:14:33.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.jXf63VqW9XYNAOsa-1YPRgxHUizU_Al_0 2024-06-28T16:14:33.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2030778273690181537', time=2024-06-28T17:47:17.638544+0000, chain.objs.size()=1 2024-06-28T16:14:33.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.UsqlOSUROkHDY8HdJmuCXH-wpLyZsU-_0 2024-06-28T16:14:33.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17680560313751847581', time=2024-06-28T17:47:18.602111+0000, chain.objs.size()=1 2024-06-28T16:14:33.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.O8kdpC6GJ1IEc9LcUAj4fG2ns3GTfMU_0 2024-06-28T16:14:33.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14207369182336564592', time=2024-06-28T17:47:18.867006+0000, chain.objs.size()=1 2024-06-28T16:14:33.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.DiXhYo3B1pNQ4w0UfQdWfn2R9Dz7Cbp_0 2024-06-28T16:14:33.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3000105721634795284', time=2024-06-28T17:47:19.245599+0000, chain.objs.size()=1 2024-06-28T16:14:33.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Hb-Yo-OhReI8bcYHp4-2XhX39SjrcNV_0 2024-06-28T16:14:33.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17207342407488410682', time=2024-06-28T17:47:19.805975+0000, chain.objs.size()=1 2024-06-28T16:14:33.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_._5WaxiwO53HTGtTP0iu5Fmz1Z_PHJ-a_0 2024-06-28T16:14:33.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8257261166628995646', time=2024-06-28T17:47:23.478443+0000, chain.objs.size()=1 2024-06-28T16:14:33.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.wJWLZhI-L4LYJb-6PJLQ1tiBigSvY7v_0 2024-06-28T16:14:33.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3325087727569861209', time=2024-06-28T17:47:24.966245+0000, chain.objs.size()=1 2024-06-28T16:14:33.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Xpr92kUz92KmiM9mGI2Yk4qVjvyJ3k-_0 2024-06-28T16:14:33.224 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12742117756077400467', time=2024-06-28T17:47:24.968031+0000, chain.objs.size()=1 2024-06-28T16:14:33.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ImdHx9qMAkcOsLwIhCXWTIb79yTOCoq_0 2024-06-28T16:14:33.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.215+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.528521725217797155', time=2024-06-28T17:47:26.318511+0000, chain.objs.size()=1 2024-06-28T16:14:33.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.215+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.qeotSZJrU2wjmJuC3ijvLw7-pDFnGcc_0 2024-06-28T16:14:33.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.215+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.599610102002739147', time=2024-06-28T17:47:27.367886+0000, chain.objs.size()=1 2024-06-28T16:14:33.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.215+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.nKO7oOd259452ihVc4q4J2DwyvI5GHX_0 2024-06-28T16:14:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5645804409748242191', time=2024-06-28T17:47:27.522603+0000, chain.objs.size()=1 2024-06-28T16:14:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.IPwKRN8zi3J7djNKTKZdYTPySz7eY3F_0 2024-06-28T16:14:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11771639716045010250', time=2024-06-28T17:47:28.117664+0000, chain.objs.size()=1 2024-06-28T16:14:33.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dT-27ybZgT5O19VHiJIGFow1mS96VDh_0 2024-06-28T16:14:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17889381140378910289', time=2024-06-28T17:47:28.941146+0000, chain.objs.size()=1 2024-06-28T16:14:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ov1BzIj-7DszimGWddbDJl6iKvlEjv4_0 2024-06-28T16:14:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.662908304928911803', time=2024-06-28T17:47:30.595587+0000, chain.objs.size()=1 2024-06-28T16:14:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5gf-GGjb4rvkOy26cTNDQfWUNGjDQJR_0 2024-06-28T16:14:33.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9409679963391658160', time=2024-06-28T17:47:31.504789+0000, chain.objs.size()=1 2024-06-28T16:14:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.50oaTzC7mV3lTUox5C5cpMX2jhtj_sU_0 2024-06-28T16:14:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4874345976357311777', time=2024-06-28T17:47:32.127742+0000, chain.objs.size()=1 2024-06-28T16:14:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.BwNDqAV-QpbZBZN3DthCdSkGOSNkv7l_0 2024-06-28T16:14:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12046798105137061833', time=2024-06-28T17:47:32.862875+0000, chain.objs.size()=1 2024-06-28T16:14:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.zaeLTD6q6FE19SJAr09HBHBquV6NJVL_0 2024-06-28T16:14:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2311253772971675949', time=2024-06-28T17:47:34.256063+0000, chain.objs.size()=1 2024-06-28T16:14:33.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.N5Wq6UFyCVWFzA1HUUMngdTcbKdGstX_0 2024-06-28T16:14:33.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2932934767931441210', time=2024-06-28T17:47:35.200558+0000, chain.objs.size()=1 2024-06-28T16:14:33.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.b9bNZANaA2_9vjMov8LqQZPVdNHWXTo_0 2024-06-28T16:14:33.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.530788519321095753', time=2024-06-28T17:47:35.699982+0000, chain.objs.size()=1 2024-06-28T16:14:33.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.59nRZ6vHGmUW3M6JMEFVaR_fTiDcVGr_0 2024-06-28T16:14:33.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.223+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16001346672048278710', time=2024-06-28T17:47:36.169129+0000, chain.objs.size()=1 2024-06-28T16:14:33.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.223+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.wpOJo1DDmu1XbpA2HfnZS1s-euZW-lu_0 2024-06-28T16:14:33.234 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.223+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18310028441709740730', time=2024-06-28T17:47:38.083899+0000, chain.objs.size()=1 2024-06-28T16:14:33.235 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.223+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.vd5yKlAyG3HBdrnAQQ31ktR34asCh62_0 2024-06-28T16:14:33.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5589554608254300011', time=2024-06-28T17:47:38.324674+0000, chain.objs.size()=1 2024-06-28T16:14:33.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.0yl8Sxs6_zH72Io61VfjvX8Ay_-9OTl_0 2024-06-28T16:14:33.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13962948508400298437', time=2024-06-28T17:47:38.875482+0000, chain.objs.size()=1 2024-06-28T16:14:33.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7wb1A3wzccng6TwD-t2Fj5soSUoQ8U6_0 2024-06-28T16:14:33.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17833700132297484660', time=2024-06-28T17:47:39.499679+0000, chain.objs.size()=1 2024-06-28T16:14:33.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ZJiRwor2zr7p6U5C1c5ZosavARgiKm7_0 2024-06-28T16:14:33.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4827615324959438246', time=2024-06-28T17:48:18.250836+0000, chain.objs.size()=1 2024-06-28T16:14:33.236 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.277__shadow_.ksu4i0j6uSc1tHVS_wK5CCMbexpbt16_0 2024-06-28T16:14:33.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5573213046788671888', time=2024-06-28T17:48:29.576406+0000, chain.objs.size()=1 2024-06-28T16:14:33.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.FXDj5HyjAZVnmauIYZjXipBn_dsBdz6_0 2024-06-28T16:14:33.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11951491523921982337', time=2024-06-28T17:48:38.062860+0000, chain.objs.size()=1 2024-06-28T16:14:33.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.285__shadow_.jJxKQhge34TKdF2p-5IV0HWx2AvWsBv_0 2024-06-28T16:14:33.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14216444147733918821', time=2024-06-28T17:48:41.351030+0000, chain.objs.size()=1 2024-06-28T16:14:33.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.287__shadow_.X877h_pnTLTHyvbkBnzFFCwy7hxs8p0_0 2024-06-28T16:14:33.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17018534144026566365', time=2024-06-28T17:48:43.020758+0000, chain.objs.size()=1 2024-06-28T16:14:33.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.288__shadow_.oP9CfO_ffmXuQjXtICe3XQg-dEL0zbp_0 2024-06-28T16:14:33.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3629798042197497080', time=2024-06-28T17:49:16.720525+0000, chain.objs.size()=1 2024-06-28T16:14:33.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.309__shadow:2RTQOoHZ3nbZq7gqowHJCQOdqxW6jVy_.WzrK8Y2MKUWrruySJk1W0Ez26wOSu9n_0 2024-06-28T16:14:33.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1833253920074218792', time=2024-06-28T17:49:19.406250+0000, chain.objs.size()=1 2024-06-28T16:14:33.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.311__shadow_.-kgzaESXu5k-dC66zcwYy2a51t5-tCq_0 2024-06-28T16:14:33.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13066998984111191202', time=2024-06-28T17:51:09.779913+0000, chain.objs.size()=1 2024-06-28T16:14:33.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.H17PNJX4ATfQLd9Atu70yAXiTl8C2GL_0 2024-06-28T16:14:33.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11521032883583414455', time=2024-06-28T17:51:12.207195+0000, chain.objs.size()=1 2024-06-28T16:14:33.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.227+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.iKnMkRImQ3z3_QdgSf_hg7H2R7Q0G2r_0 2024-06-28T16:14:33.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16572113008264865267', time=2024-06-28T17:51:13.995891+0000, chain.objs.size()=1 2024-06-28T16:14:33.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ethGNqvdqgJVy4Q_YDlpQ603ccbTQGS_0 2024-06-28T16:14:33.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14796404326382935875', time=2024-06-28T17:51:15.939076+0000, chain.objs.size()=1 2024-06-28T16:14:33.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.N4-Evya0--vuQKkUsGEshKV9nZpI-GX_0 2024-06-28T16:14:33.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1433601613973411047', time=2024-06-28T17:51:16.632979+0000, chain.objs.size()=1 2024-06-28T16:14:33.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.37epQiemjYDHiGKnIDa6HlAaJiTdkej_0 2024-06-28T16:14:33.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.854172965309050088', time=2024-06-28T17:51:17.483520+0000, chain.objs.size()=1 2024-06-28T16:14:33.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.3amFtmD_wL0dMkxJkX4s7a45nFTqgMe_0 2024-06-28T16:14:33.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3550830859915089536', time=2024-06-28T17:51:19.480008+0000, chain.objs.size()=1 2024-06-28T16:14:33.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.qQ15IvgHqFOPWj4AwygsTNStcxRBJcf_0 2024-06-28T16:14:33.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5051272192826983032', time=2024-06-28T17:51:20.030300+0000, chain.objs.size()=1 2024-06-28T16:14:33.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.RhYONlEiEKu1shPg-e9vzqr2lUKRmXD_0 2024-06-28T16:14:33.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6166359750105082742', time=2024-06-28T17:51:20.696555+0000, chain.objs.size()=1 2024-06-28T16:14:33.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.CI0tG2s8emBDPN3458rfab7GeSR3UB8_0 2024-06-28T16:14:33.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3617361978743441366', time=2024-06-28T17:51:29.557671+0000, chain.objs.size()=1 2024-06-28T16:14:33.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ziJkeVGbOF5_r6gZjE4YDQl5dJb_30V_0 2024-06-28T16:14:33.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.235+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1166244804115017877', time=2024-06-28T17:51:32.746295+0000, chain.objs.size()=1 2024-06-28T16:14:33.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.HLAzHFyx3_hqVPJFhNetgeYa1I3fwbE_0 2024-06-28T16:14:33.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.235+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3270759062801286847', time=2024-06-28T17:51:33.119271+0000, chain.objs.size()=1 2024-06-28T16:14:33.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ZDQSkeky2dFcCrzA99RCLeURK55MqN8_0 2024-06-28T16:14:33.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.235+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15654213641324266623', time=2024-06-28T17:51:34.857026+0000, chain.objs.size()=1 2024-06-28T16:14:33.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.cZydGaHpI85_TgiZbvzV2K7bYrkBhcG_0 2024-06-28T16:14:33.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.235+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2373399530517181196', time=2024-06-28T17:51:35.100734+0000, chain.objs.size()=1 2024-06-28T16:14:33.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.MdaUSKILia8I4LK-pPurE99asCFTpyF_0 2024-06-28T16:14:33.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.235+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6452428053536692476', time=2024-06-28T17:51:38.216615+0000, chain.objs.size()=1 2024-06-28T16:14:33.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.E6t3NO7WTSCgaZ1X7oqBZjpMII0BL83_0 2024-06-28T16:14:33.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.92043099938986385', time=2024-06-28T17:53:21.439435+0000, chain.objs.size()=1 2024-06-28T16:14:33.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.JgkVsqYfm80Xsw60IX-HVRIjruZzPdT_0 2024-06-28T16:14:33.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13146265474269252427', time=2024-06-28T17:53:22.927200+0000, chain.objs.size()=1 2024-06-28T16:14:33.248 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.uEar2tzdYSzMUNxFxjKszDcMukD2pGm_0 2024-06-28T16:14:33.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4787169823267775843', time=2024-06-28T17:53:23.557501+0000, chain.objs.size()=1 2024-06-28T16:14:33.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.XA5eBFF4dl2zSehSwn7LxkfaCoBa81z_0 2024-06-28T16:14:33.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3329692846869047382', time=2024-06-28T17:53:25.206611+0000, chain.objs.size()=1 2024-06-28T16:14:33.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.sbzG85rRVVbx7aXfpTAuOEp-4a6Mgk2_0 2024-06-28T16:14:33.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.874386972606795303', time=2024-06-28T17:53:26.606814+0000, chain.objs.size()=1 2024-06-28T16:14:33.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.M4fqwXK3r2_m-SQfyogWbDAKHueH4hr_0 2024-06-28T16:14:33.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14176972097260166277', time=2024-06-28T17:53:29.910810+0000, chain.objs.size()=1 2024-06-28T16:14:33.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.FYDFBlP6ptcfX4e_NfQGz05PxPxLfjS_0 2024-06-28T16:14:33.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8969021935008479836', time=2024-06-28T17:53:30.234111+0000, chain.objs.size()=1 2024-06-28T16:14:33.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Sfydbj9PqEsI49JBlE9RhhbP_s0rJkM_0 2024-06-28T16:14:33.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.636629876611567030', time=2024-06-28T17:53:31.125132+0000, chain.objs.size()=1 2024-06-28T16:14:33.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.WfV_TMTLZYi8De6dzp_ZAJhvG77ITWs_0 2024-06-28T16:14:33.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15763407717124671779', time=2024-06-28T17:53:32.680082+0000, chain.objs.size()=1 2024-06-28T16:14:33.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.DdUFmf8h6eRo1nxXDv50vFKWuE5oTL7_0 2024-06-28T16:14:33.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16808240288856194696', time=2024-06-28T17:53:34.419192+0000, chain.objs.size()=1 2024-06-28T16:14:33.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.nABb-9y7ScjImgmkIUfTSW1CpdurJcT_0 2024-06-28T16:14:33.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9176920390863407743', time=2024-06-28T17:53:35.705257+0000, chain.objs.size()=1 2024-06-28T16:14:33.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bxCnhhgQIPnCREGD1uAF2hLuoLz-N4F_0 2024-06-28T16:14:33.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12783407614144741087', time=2024-06-28T17:53:36.116874+0000, chain.objs.size()=1 2024-06-28T16:14:33.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.2XjKWET1zRpwz1IvHGy8fKkBVB0vvMT_0 2024-06-28T16:14:33.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7288860994347517740', time=2024-06-28T17:53:36.218799+0000, chain.objs.size()=1 2024-06-28T16:14:33.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.8WBFo9uHPgMuafW5OJxKTVNEFv2FRZG_0 2024-06-28T16:14:33.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16846549459244595510', time=2024-06-28T17:53:40.044158+0000, chain.objs.size()=1 2024-06-28T16:14:33.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.VktjgxVAq-8pENMPN5Xnr7pG3z_BULF_0 2024-06-28T16:14:33.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10585158476137516026', time=2024-06-28T17:53:40.910196+0000, chain.objs.size()=1 2024-06-28T16:14:33.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gh1vDIElhMnPRjAcssqf6uqySrDQUZS_0 2024-06-28T16:14:33.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5251329804549969874', time=2024-06-28T17:53:43.111047+0000, chain.objs.size()=1 2024-06-28T16:14:33.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BRBrNyQhVnDq-lkKSZmcErHNsxRjXVF_0 2024-06-28T16:14:33.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15159401763299406756', time=2024-06-28T17:53:43.913823+0000, chain.objs.size()=1 2024-06-28T16:14:33.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.243+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.SHTYncoAj5YnXgoi_bQIRnwF7aFnHNY_0 2024-06-28T16:14:33.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.247+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.987457628177672770', time=2024-06-28T17:53:45.142991+0000, chain.objs.size()=1 2024-06-28T16:14:33.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.247+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BJSpxKZ_oNipz2965jRxJaWmVl7ow6X_0 2024-06-28T16:14:33.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.247+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2097818509040944695', time=2024-06-28T17:53:47.050214+0000, chain.objs.size()=1 2024-06-28T16:14:33.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.247+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.LtAqrIpZov0W9JS0iDS2crHE7NBUPV9_0 2024-06-28T16:14:33.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.247+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14789110248751348134', time=2024-06-28T17:53:47.217087+0000, chain.objs.size()=1 2024-06-28T16:14:33.257 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.247+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.q8p-J4B_r74_kaR5suk4iG_h_nCI_24_0 2024-06-28T16:14:33.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.247+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6466411522064763159', time=2024-06-28T17:53:47.334296+0000, chain.objs.size()=1 2024-06-28T16:14:33.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.247+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.nfibBoW_uQS2mgp2sXtxTbrsmArNowQ_0 2024-06-28T16:14:33.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.247+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11658345149191883479', time=2024-06-28T17:53:47.432691+0000, chain.objs.size()=1 2024-06-28T16:14:33.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.247+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gvI3yJHVwiWtKIYayJOTsRmY0p0uRha_0 2024-06-28T16:14:33.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.247+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2911196119064499632', time=2024-06-28T17:53:47.758014+0000, chain.objs.size()=1 2024-06-28T16:14:33.258 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.247+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Shi719rEut8BOs3uWiQ1KEk-kZ6SiMt_0 2024-06-28T16:14:33.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4216116286783089618', time=2024-06-28T17:53:48.910758+0000, chain.objs.size()=1 2024-06-28T16:14:33.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Aty77qdxKGQ3AyPrwzW5RTXoo4qfoh3_0 2024-06-28T16:14:33.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3972660587880428525', time=2024-06-28T17:53:50.235948+0000, chain.objs.size()=1 2024-06-28T16:14:33.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GsOZ0trekCJQ6borjyldkZicZXq_Pob_0 2024-06-28T16:14:33.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8168466805375509333', time=2024-06-28T17:53:50.396128+0000, chain.objs.size()=1 2024-06-28T16:14:33.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.fGo9u_WBD8YXbLbe-tvj47J6VAhMHfn_0 2024-06-28T16:14:33.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11448689799593733647', time=2024-06-28T17:53:50.399521+0000, chain.objs.size()=1 2024-06-28T16:14:33.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.SRyfBz3KOeA3OXpx2xJqB3YAUpSfUgF_0 2024-06-28T16:14:33.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4714668013224596093', time=2024-06-28T17:53:50.410136+0000, chain.objs.size()=1 2024-06-28T16:14:33.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ROMMRRFRIIHnJ6Yo86qWv165BQmaD3o_0 2024-06-28T16:14:33.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9342451804571237072', time=2024-06-28T17:53:51.622012+0000, chain.objs.size()=1 2024-06-28T16:14:33.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.q6BnpkPy2RGiHHTNY1h5pv8tJDTSXJE_0 2024-06-28T16:14:33.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4050801092010886948', time=2024-06-28T17:53:52.057736+0000, chain.objs.size()=1 2024-06-28T16:14:33.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.J070-cn_N2uOLUuTzBemWDqNt5gCJny_0 2024-06-28T16:14:33.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1562637399950494367', time=2024-06-28T17:54:07.172252+0000, chain.objs.size()=1 2024-06-28T16:14:33.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.330__shadow_.QCt76eA8m_WXbVIZW_8zfuUjHR9OUp2_0 2024-06-28T16:14:33.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1976291993125842434', time=2024-06-28T17:56:03.274472+0000, chain.objs.size()=1 2024-06-28T16:14:33.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.450__shadow_.P07DfOneyj100kbcrz2l7wOjsc0zC76_0 2024-06-28T16:14:33.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5214692796126900611', time=2024-06-28T17:56:11.150101+0000, chain.objs.size()=1 2024-06-28T16:14:33.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.455__shadow_.TvLKTF9m6p7RvmrrknpSLSbr-3OfS4A_0 2024-06-28T16:14:33.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17146089183343712642', time=2024-06-28T17:56:12.009211+0000, chain.objs.size()=1 2024-06-28T16:14:33.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.455__shadow_.cADYdyTEWW9wsOn7cmKP2LnIgKqGea8_0 2024-06-28T16:14:33.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17532584626053366617', time=2024-06-28T17:56:23.225296+0000, chain.objs.size()=1 2024-06-28T16:14:33.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.471__shadow_.ECocxtwu7HZa76MuZqzCPW5KkXcZv7S_0 2024-06-28T16:14:33.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9645124658816484280', time=2024-06-28T17:59:25.191643+0000, chain.objs.size()=6 2024-06-28T16:14:33.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~NfT0D8bLVyyZAgfSytxCDSpO4_EjdPN.1 2024-06-28T16:14:33.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.dDAEDC-a04upW1xtNsK25FoRjtUC82HV.2 2024-06-28T16:14:33.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.1BuUfZHPpod965-W5DPO_ETf4MqNeL_O.3 2024-06-28T16:14:33.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~NfT0D8bLVyyZAgfSytxCDSpO4_EjdPN.4 2024-06-28T16:14:33.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.255+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~NfT0D8bLVyyZAgfSytxCDSpO4_EjdPN.5 2024-06-28T16:14:33.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.259+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~NfT0D8bLVyyZAgfSytxCDSpO4_EjdPN.6 2024-06-28T16:14:33.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.259+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.586849132467613186', time=2024-06-28T18:01:25.304273+0000, chain.objs.size()=1 2024-06-28T16:14:33.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.259+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.544__shadow_.bSm5RE83kPf6-m84ZIFjphvQdygIgBz_0 2024-06-28T16:14:33.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.259+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5934130880773830822', time=2024-06-28T18:01:40.194096+0000, chain.objs.size()=1 2024-06-28T16:14:33.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.259+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.548__shadow_.ur3U6r9Wo2U4nZiYPjIfo0NB5dSy_JQ_0 2024-06-28T16:14:33.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.259+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8900735335492220961', time=2024-06-28T18:02:33.613130+0000, chain.objs.size()=1 2024-06-28T16:14:33.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.259+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.563__shadow:4ilt670LvnEVtzG4dyDOGe8gIZ9mmfj_.Jkt4HZVkTyO-yUe2c9Ses3UzaOe9EYQ_0 2024-06-28T16:14:33.270 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.259+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16318745546790766288', time=2024-06-28T18:07:00.216772+0000, chain.objs.size()=1 2024-06-28T16:14:33.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.259+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.624__shadow_.YKrrdfisLGbcNZsoMY4RVrE9LMHN-tL_0 2024-06-28T16:14:33.273 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.263+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:33.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.267+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-06-28T16:14:33.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.267+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:33.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:33.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/49514' 2024-06-28T16:14:33.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4934379937792230960', time=2024-06-28T17:39:53.346234+0000, chain.objs.size()=1 2024-06-28T16:14:33.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.141__shadow_.AKnogG8RoDfljbR1vA15DFv-g5LLuBv_0 2024-06-28T16:14:33.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.163861016987870143', time=2024-06-28T17:40:13.043704+0000, chain.objs.size()=3 2024-06-28T16:14:33.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.149__multipart_mymultipart-STANDARD.2~H4SA4yHkTILuzvhfaLZVY1f-qcnVlBR.1 2024-06-28T16:14:33.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.149__multipart_mymultipart-STANDARD.2~H4SA4yHkTILuzvhfaLZVY1f-qcnVlBR.2 2024-06-28T16:14:33.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.149__multipart_mymultipart-STANDARD.2~H4SA4yHkTILuzvhfaLZVY1f-qcnVlBR.3 2024-06-28T16:14:33.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8149375421119640716', time=2024-06-28T17:44:35.790848+0000, chain.objs.size()=1 2024-06-28T16:14:33.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.221__shadow_.FcUUFakN5krnuTRh_vstVg3kuwKByvt_0 2024-06-28T16:14:33.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6619176543798822701', time=2024-06-28T17:44:40.609885+0000, chain.objs.size()=1 2024-06-28T16:14:33.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.224__shadow_.jUBOGxmwrWE1sSWSW_glhBfk2X5A3S2_0 2024-06-28T16:14:33.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8327830075939326044', time=2024-06-28T17:44:49.703763+0000, chain.objs.size()=1 2024-06-28T16:14:33.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.229__shadow_.2vugdmnHQSsu8sxaofhQoRqpBfa-Xc6_0 2024-06-28T16:14:33.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11695065454862506900', time=2024-06-28T17:44:58.700993+0000, chain.objs.size()=1 2024-06-28T16:14:33.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.234__shadow_.9Av6c4flW6L6DgDZ88bTzgdKITJ5z3u_0 2024-06-28T16:14:33.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3432981036575794799', time=2024-06-28T17:45:03.375648+0000, chain.objs.size()=1 2024-06-28T16:14:33.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.237__shadow_.dQzDmw2w6MN5k-QbLr3Fb4j-loMhWQO_0 2024-06-28T16:14:33.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2598776539805178459', time=2024-06-28T17:45:25.489006+0000, chain.objs.size()=1 2024-06-28T16:14:33.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.251__shadow_.H_Eb-1vQxQxirSC2z_w9kY_MX_S5sRP_0 2024-06-28T16:14:33.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9561805113743207757', time=2024-06-28T17:47:09.077257+0000, chain.objs.size()=1 2024-06-28T16:14:33.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ijq0_Q3gOrm2PG368bTVxJKhPyvsrJb_0 2024-06-28T16:14:33.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11816838447994618589', time=2024-06-28T17:47:09.587416+0000, chain.objs.size()=1 2024-06-28T16:14:33.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.6Pcs-NxzFXyrhJUKuhkx9YeTIl9retn_0 2024-06-28T16:14:33.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4001752959056539466', time=2024-06-28T17:47:09.801862+0000, chain.objs.size()=1 2024-06-28T16:14:33.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.lml_E_GoGeWmSmUhThYocyA-dn3DCLe_0 2024-06-28T16:14:33.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9753918803459993181', time=2024-06-28T17:47:10.067362+0000, chain.objs.size()=1 2024-06-28T16:14:33.284 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.zGfuoYGUmWo6wSLguf2DYos2hDkaQjO_0 2024-06-28T16:14:33.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10606616983570363754', time=2024-06-28T17:47:12.184338+0000, chain.objs.size()=1 2024-06-28T16:14:33.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ei-cBqimsQcGgjcFreu1OqERNVgps8X_0 2024-06-28T16:14:33.286 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1128664364382888048', time=2024-06-28T17:47:12.259239+0000, chain.objs.size()=1 2024-06-28T16:14:33.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.suhhKXX_1XalmnbrCGM2KSTcYqheJqh_0 2024-06-28T16:14:33.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4901503406675143016', time=2024-06-28T17:47:15.145029+0000, chain.objs.size()=1 2024-06-28T16:14:33.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.HCtmk7pfyUsaZzmc8ktv6swhHq7wFN6_0 2024-06-28T16:14:33.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16436945481423948133', time=2024-06-28T17:47:16.783703+0000, chain.objs.size()=1 2024-06-28T16:14:33.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.I3zmO6PKkT8kyAjdMYNTNEa9xNIJObF_0 2024-06-28T16:14:33.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16134850472305643677', time=2024-06-28T17:47:17.886696+0000, chain.objs.size()=1 2024-06-28T16:14:33.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.E5nLj7ItSxk7VXX5T1IDzjwPkPgDZ1N_0 2024-06-28T16:14:33.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14344285481073973599', time=2024-06-28T17:47:18.071948+0000, chain.objs.size()=1 2024-06-28T16:14:33.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.EUSlI0_Ouodd0hWdC4k8CrFP4RfA3fo_0 2024-06-28T16:14:33.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12675996154847967084', time=2024-06-28T17:47:19.487177+0000, chain.objs.size()=1 2024-06-28T16:14:33.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9Nr7C6U8Cu4bptTYNtlxLyY6FAcS1mU_0 2024-06-28T16:14:33.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4873351075904877354', time=2024-06-28T17:47:19.494799+0000, chain.objs.size()=1 2024-06-28T16:14:33.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dr7MEVmKy9HklY1KMNXgqxc02NyfhcU_0 2024-06-28T16:14:33.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2459446626653483344', time=2024-06-28T17:47:21.643735+0000, chain.objs.size()=1 2024-06-28T16:14:33.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KChD3OWxMregiaGAmx-qIRRSGeRlLvK_0 2024-06-28T16:14:33.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.279+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.608372015521540495', time=2024-06-28T17:47:22.026648+0000, chain.objs.size()=1 2024-06-28T16:14:33.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.279+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.sTOQqtbAo-zS25zxuH9g_1Zp5UNiaXp_0 2024-06-28T16:14:33.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.279+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2209115172469993451', time=2024-06-28T17:47:22.263720+0000, chain.objs.size()=1 2024-06-28T16:14:33.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.279+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VKTlbO8J6DJPUKSEsgjtCHsYCEkuOwf_0 2024-06-28T16:14:33.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.279+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8876605038993127012', time=2024-06-28T17:47:23.270875+0000, chain.objs.size()=1 2024-06-28T16:14:33.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.279+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.XmlsZbszZvSsigVyrlcGmekxVLJ8IBQ_0 2024-06-28T16:14:33.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.279+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13857804023596861084', time=2024-06-28T17:47:23.274833+0000, chain.objs.size()=1 2024-06-28T16:14:33.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.279+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.u5WqAGnj3WDzO7PtDnhPLq-PnY-R_T9_0 2024-06-28T16:14:33.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5349948061246325513', time=2024-06-28T17:47:25.107159+0000, chain.objs.size()=1 2024-06-28T16:14:33.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.r5xH7u-z13ubEFPv8Qt5g0Qj-Gh58_z_0 2024-06-28T16:14:33.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5339601657468186339', time=2024-06-28T17:47:25.522123+0000, chain.objs.size()=1 2024-06-28T16:14:33.291 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.zmHoeai8s5FR01VQrvxhCB6BwD5RsJh_0 2024-06-28T16:14:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16141731989330171070', time=2024-06-28T17:47:29.481394+0000, chain.objs.size()=1 2024-06-28T16:14:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.8XQN3nBHoUnMYTJWSmSPi_kZ_rE-ugL_0 2024-06-28T16:14:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9202052176346325919', time=2024-06-28T17:47:29.754360+0000, chain.objs.size()=1 2024-06-28T16:14:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.RrKEUNbcxRh1flREa3GB0UTcyXfBMw9_0 2024-06-28T16:14:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11302271867921434306', time=2024-06-28T17:47:30.667898+0000, chain.objs.size()=1 2024-06-28T16:14:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.e0JqSZkE4a48YapwpA0VtT7NTP_e_Pv_0 2024-06-28T16:14:33.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13503034567161511742', time=2024-06-28T17:47:31.358770+0000, chain.objs.size()=1 2024-06-28T16:14:33.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.doIojYyyfc6FxUrsrx_p1lPJYBGeNPz_0 2024-06-28T16:14:33.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5932491060165321451', time=2024-06-28T17:47:32.089338+0000, chain.objs.size()=1 2024-06-28T16:14:33.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.wp_nbaRWrbpsDxP7DQxaE7tlWC7oR5a_0 2024-06-28T16:14:33.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9918460990411584912', time=2024-06-28T17:47:33.429032+0000, chain.objs.size()=1 2024-06-28T16:14:33.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ClW27FWOx-vSrQIN5VvbmZwudcjBAK6_0 2024-06-28T16:14:33.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.433859486481170750', time=2024-06-28T17:47:35.127318+0000, chain.objs.size()=1 2024-06-28T16:14:33.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.jnyyi_CFubIcAZtOAsRjyoRJFY1o9qG_0 2024-06-28T16:14:33.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11476470851158146980', time=2024-06-28T17:47:37.528261+0000, chain.objs.size()=1 2024-06-28T16:14:33.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.BoJU71UJr8rkty5KF4LWHclP2YGiBbx_0 2024-06-28T16:14:33.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11659415862852810161', time=2024-06-28T17:47:44.093400+0000, chain.objs.size()=1 2024-06-28T16:14:33.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.255__shadow_.qq1ynY_Ku9bRPUTaOjz307E9aujbSSN_0 2024-06-28T16:14:33.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10203631857698819628', time=2024-06-28T17:47:47.001499+0000, chain.objs.size()=1 2024-06-28T16:14:33.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.257__shadow_.hto0dLwiLzXT9lqZGN4FuDhfOtLx_Bt_0 2024-06-28T16:14:33.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.287+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13736824725157207276', time=2024-06-28T17:48:13.256121+0000, chain.objs.size()=1 2024-06-28T16:14:33.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.287+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.274__shadow_.iNNyoRv2RtsGSseUWQQqxS7K5LSKM3J_0 2024-06-28T16:14:33.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10853958533002952759', time=2024-06-28T17:48:34.474087+0000, chain.objs.size()=1 2024-06-28T16:14:33.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.gXlJStP6DHlqxBHzBXGolksWAOg1EIE_0 2024-06-28T16:14:33.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10666735474918113923', time=2024-06-28T17:48:34.570864+0000, chain.objs.size()=1 2024-06-28T16:14:33.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.1aWQyyLS7QFVKHSSuPIlahbCch3JnaS_0 2024-06-28T16:14:33.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9399397578749540175', time=2024-06-28T17:48:34.902683+0000, chain.objs.size()=1 2024-06-28T16:14:33.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.Sr9YTuTut8BLybMoTlTZ2o736jkATpE_0 2024-06-28T16:14:33.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4199313605430157911', time=2024-06-28T17:48:39.761133+0000, chain.objs.size()=1 2024-06-28T16:14:33.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.286__shadow_.17wvLRTtmn1D2uLAItnkyiLvjWQ7JKJ_0 2024-06-28T16:14:33.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15691310277121807829', time=2024-06-28T17:49:16.794036+0000, chain.objs.size()=1 2024-06-28T16:14:33.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.309__shadow:hbtQECHyEWDBJd3AbQQUZDSvDRzWABa_.ExX0QeP8h0CJI2IM1UiiIMDiCeAbFff_0 2024-06-28T16:14:33.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8289738208974280653', time=2024-06-28T17:49:20.831331+0000, chain.objs.size()=1 2024-06-28T16:14:33.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.312__shadow_.WGmhfrtMpNgfvOoqTaOd0zDQ480tX83_0 2024-06-28T16:14:33.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11003712694676707845', time=2024-06-28T17:49:20.841232+0000, chain.objs.size()=1 2024-06-28T16:14:33.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.312__shadow_.TTejf7n3MmyvZRQjae0Z6ahlnexMEMG_0 2024-06-28T16:14:33.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5414275971312274816', time=2024-06-28T17:51:07.256737+0000, chain.objs.size()=1 2024-06-28T16:14:33.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.VJ3FX-r2WCi89GqsFwuKT7JiHIR3QDO_0 2024-06-28T16:14:33.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16610692046445192779', time=2024-06-28T17:51:09.785529+0000, chain.objs.size()=1 2024-06-28T16:14:33.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.efc8aEp3rcCQDx2jCgOtaaaQj0vJFEp_0 2024-06-28T16:14:33.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11633878100999220535', time=2024-06-28T17:51:12.056932+0000, chain.objs.size()=1 2024-06-28T16:14:33.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.5xRPH2NGkGt6hPj5JIQn7-Wi9z0YCqf_0 2024-06-28T16:14:33.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15158851021775698123', time=2024-06-28T17:51:13.170066+0000, chain.objs.size()=1 2024-06-28T16:14:33.304 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.E9Ask-8pxbCsF8SzAkPT51kt6zdjr1c_0 2024-06-28T16:14:33.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9140235460967746591', time=2024-06-28T17:51:14.316777+0000, chain.objs.size()=1 2024-06-28T16:14:33.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.29jWITVpSNvU-nqgeZ7M-1krkGyg-Xz_0 2024-06-28T16:14:33.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7911334575889540747', time=2024-06-28T17:51:16.956863+0000, chain.objs.size()=1 2024-06-28T16:14:33.305 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Rq6b-p3JvGvI_Eu8nzTrkwJ1Y7H77j3_0 2024-06-28T16:14:33.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3592151742921474647', time=2024-06-28T17:51:17.678980+0000, chain.objs.size()=1 2024-06-28T16:14:33.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.XbUGandZmYaileLlbJnKDfy4rzHJeM3_0 2024-06-28T16:14:33.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9781884082645784831', time=2024-06-28T17:51:19.682870+0000, chain.objs.size()=1 2024-06-28T16:14:33.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.QqT9EGjsV2cSQwYvxgkgvKXjCxEjws4_0 2024-06-28T16:14:33.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11486989153757546992', time=2024-06-28T17:51:20.790943+0000, chain.objs.size()=1 2024-06-28T16:14:33.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.7IzpMFf4RvBnBM6p72TP4914YjjTjpt_0 2024-06-28T16:14:33.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5543210161190427159', time=2024-06-28T17:51:24.419395+0000, chain.objs.size()=1 2024-06-28T16:14:33.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.7WrwVnpEiLajywhjbFolr7zytN86up7_0 2024-06-28T16:14:33.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1007853825118866405', time=2024-06-28T17:51:26.597816+0000, chain.objs.size()=1 2024-06-28T16:14:33.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.TC213l19Aw2odatpxpPaD3TybYv7vBX_0 2024-06-28T16:14:33.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12315032098063057778', time=2024-06-28T17:51:29.197215+0000, chain.objs.size()=1 2024-06-28T16:14:33.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_._TIPGU2GQQFWrerN1DCNL0rpfA_tO9g_0 2024-06-28T16:14:33.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12531757687025561449', time=2024-06-28T17:51:30.042285+0000, chain.objs.size()=1 2024-06-28T16:14:33.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.TXriMlwe9aexS4nCMka4ZVL45uKVQr8_0 2024-06-28T16:14:33.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12934074542968588352', time=2024-06-28T17:51:30.095156+0000, chain.objs.size()=1 2024-06-28T16:14:33.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6waWZWzJmMer94YUFaeiMCas8vBQZb7_0 2024-06-28T16:14:33.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7179083494214009744', time=2024-06-28T17:51:31.085347+0000, chain.objs.size()=1 2024-06-28T16:14:33.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6sWMUE27W9llKxTUOohB8KiDvyqK241_0 2024-06-28T16:14:33.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11245834858075485045', time=2024-06-28T17:51:32.535198+0000, chain.objs.size()=1 2024-06-28T16:14:33.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.P6OpJuXx-UIZ5Xo4-0MQ_ukvwib2RSC_0 2024-06-28T16:14:33.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8710059783878884607', time=2024-06-28T17:51:32.924841+0000, chain.objs.size()=1 2024-06-28T16:14:33.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.S44TEohjR6CxGSiOK8VvcbxdbPMVkfn_0 2024-06-28T16:14:33.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15316905047213254824', time=2024-06-28T17:51:33.510052+0000, chain.objs.size()=1 2024-06-28T16:14:33.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.JMmNPj3pczWKEWvN8clzbF_ilb1gdlj_0 2024-06-28T16:14:33.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2006613301501256191', time=2024-06-28T17:51:34.586373+0000, chain.objs.size()=1 2024-06-28T16:14:33.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.9WQ0Pe86ErBAfHOH4iNl0ufwZHu_N8h_0 2024-06-28T16:14:33.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6952501284049216674', time=2024-06-28T17:51:36.796477+0000, chain.objs.size()=1 2024-06-28T16:14:33.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.wQTLijqtjHAb3LrqriTXBKztxwV2vWn_0 2024-06-28T16:14:33.312 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13509949514325607792', time=2024-06-28T17:51:36.985396+0000, chain.objs.size()=1 2024-06-28T16:14:33.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.TummAfTQ0TGMeMulr1Oe8f6U0Pq0h7U_0 2024-06-28T16:14:33.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13409970893402192125', time=2024-06-28T17:51:37.887473+0000, chain.objs.size()=1 2024-06-28T16:14:33.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.4kxlQkDM9IKoQ0wDjIhmvkbpfYSbZru_0 2024-06-28T16:14:33.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10164830677776616084', time=2024-06-28T17:53:20.983024+0000, chain.objs.size()=1 2024-06-28T16:14:33.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.NTOLLvUr9xzlKNdxT7CvS_Skv-ZisyK_0 2024-06-28T16:14:33.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15387547145486523915', time=2024-06-28T17:53:21.840572+0000, chain.objs.size()=1 2024-06-28T16:14:33.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.FCAIfe3bntcJUxkwbz4OjhyvShbxmjm_0 2024-06-28T16:14:33.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3972175274253021443', time=2024-06-28T17:53:22.676257+0000, chain.objs.size()=1 2024-06-28T16:14:33.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.PflHiyUYFeRFy4yCiL2vy_HUu1QWu3F_0 2024-06-28T16:14:33.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1325027686911355484', time=2024-06-28T17:53:23.223159+0000, chain.objs.size()=1 2024-06-28T16:14:33.314 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bUmcsqMpmjqXLFUBI-s_y5tJn2UIp5S_0 2024-06-28T16:14:33.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3115936525178074463', time=2024-06-28T17:53:23.834298+0000, chain.objs.size()=1 2024-06-28T16:14:33.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.303+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZfbIk5S7c7yk2IacjK9LVSINeRUzy-8_0 2024-06-28T16:14:33.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5441506777425600965', time=2024-06-28T17:53:25.283697+0000, chain.objs.size()=1 2024-06-28T16:14:33.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.j3p7fJtlk0oGNxJL-8Wqn1sof2Bo6H7_0 2024-06-28T16:14:33.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.118192609509136662', time=2024-06-28T17:53:25.699774+0000, chain.objs.size()=1 2024-06-28T16:14:33.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.rnVvmtKHt9WWFafJNK3knVahZqnm3Qj_0 2024-06-28T16:14:33.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6423573110010682021', time=2024-06-28T17:53:25.948395+0000, chain.objs.size()=1 2024-06-28T16:14:33.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.HnwSBCkDD8rvVGQB_8soe0NxAO6IrC-_0 2024-06-28T16:14:33.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7024432260554466329', time=2024-06-28T17:53:26.607587+0000, chain.objs.size()=1 2024-06-28T16:14:33.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Ya080ilhQJth30ySarfzLmfsaFqdfNq_0 2024-06-28T16:14:33.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1995050048803864138', time=2024-06-28T17:53:27.085260+0000, chain.objs.size()=1 2024-06-28T16:14:33.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Qw7tIwhwLUs2MnpxeJYzNsjw-CN5-J__0 2024-06-28T16:14:33.318 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9686433599572833452', time=2024-06-28T17:53:27.743139+0000, chain.objs.size()=1 2024-06-28T16:14:33.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.VJOlPJUg5VaaB84s6PU2Vwlk1bUHS0s_0 2024-06-28T16:14:33.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7352191789888082383', time=2024-06-28T17:53:28.237278+0000, chain.objs.size()=1 2024-06-28T16:14:33.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZxFXZOLt3FT-mgV07SQ6MQRe_Xd4eGw_0 2024-06-28T16:14:33.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14782093074123204731', time=2024-06-28T17:53:30.365124+0000, chain.objs.size()=1 2024-06-28T16:14:33.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.EVTb9bxJyXU1AmcGGgoySlZ1hquNMwN_0 2024-06-28T16:14:33.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1964439605262442561', time=2024-06-28T17:53:30.798816+0000, chain.objs.size()=1 2024-06-28T16:14:33.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.JX9xX81arW41WFmkgogEobnUJYd3y_o_0 2024-06-28T16:14:33.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9123467934197213688', time=2024-06-28T17:53:31.157246+0000, chain.objs.size()=1 2024-06-28T16:14:33.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.cLY3bDZnH66mHd2xsqOcDBihJ67mtxW_0 2024-06-28T16:14:33.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6542854586002074574', time=2024-06-28T17:53:31.339107+0000, chain.objs.size()=1 2024-06-28T16:14:33.320 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CKtgZAlw6PeKsA5hJLInUVMCwTJLPVN_0 2024-06-28T16:14:33.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.311+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12356102844511661549', time=2024-06-28T17:53:31.762002+0000, chain.objs.size()=1 2024-06-28T16:14:33.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.311+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.95LJPVHOCKFxYsQf3j726NZQej7EujA_0 2024-06-28T16:14:33.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.311+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17081133339850271621', time=2024-06-28T17:53:32.276231+0000, chain.objs.size()=1 2024-06-28T16:14:33.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.311+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.SwraAne8gCd-0Eq8YFqqTB1nvvIcDKM_0 2024-06-28T16:14:33.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.311+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5027000713205590698', time=2024-06-28T17:53:34.405395+0000, chain.objs.size()=1 2024-06-28T16:14:33.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.311+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.8lpjrLE2WQyHQB-GnssObsUt3rKPNN5_0 2024-06-28T16:14:33.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1364822150218169148', time=2024-06-28T17:53:34.886421+0000, chain.objs.size()=1 2024-06-28T16:14:33.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.KOtvI-JyD731WfXIU0qkLHsiPlGPAup_0 2024-06-28T16:14:33.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11828607055422845573', time=2024-06-28T17:53:35.116682+0000, chain.objs.size()=1 2024-06-28T16:14:33.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.tDx0F7NZ4YyLlrOv8jF2KzamhFApxYc_0 2024-06-28T16:14:33.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14564457722363742628', time=2024-06-28T17:53:35.389031+0000, chain.objs.size()=1 2024-06-28T16:14:33.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4322FV7iJaj8YY-mVcDzm1nbal9JaYB_0 2024-06-28T16:14:33.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9485607107525106337', time=2024-06-28T17:53:37.810411+0000, chain.objs.size()=1 2024-06-28T16:14:33.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Tkfg35MW0-Q0c2OTVYGwT46MflOEFA4_0 2024-06-28T16:14:33.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17393760079888308698', time=2024-06-28T17:53:38.266269+0000, chain.objs.size()=1 2024-06-28T16:14:33.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.WobKGtNVz4k6Mvav7j2VNxbhKmr_KSd_0 2024-06-28T16:14:33.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4369120144927530008', time=2024-06-28T17:53:38.790690+0000, chain.objs.size()=1 2024-06-28T16:14:33.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.yHh91tIrTJpuGVf4evUUNHPkZImjniI_0 2024-06-28T16:14:33.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8318772264775590379', time=2024-06-28T17:53:39.267873+0000, chain.objs.size()=1 2024-06-28T16:14:33.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-sD57Qc3VN2ejYGm6PzEyRl8M4dRUKR_0 2024-06-28T16:14:33.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13966625849521225961', time=2024-06-28T17:53:40.113955+0000, chain.objs.size()=1 2024-06-28T16:14:33.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.61eRC4OLKHg02Z3LoObDO_Rq9y13lrc_0 2024-06-28T16:14:33.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3946882729033692140', time=2024-06-28T17:53:40.835565+0000, chain.objs.size()=1 2024-06-28T16:14:33.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vQ8bizCOlQFEjl3yz1npE6HauUQCgMG_0 2024-06-28T16:14:33.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4813424779405186132', time=2024-06-28T17:53:41.950166+0000, chain.objs.size()=1 2024-06-28T16:14:33.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.9iNhELM3naOuRuCEAyYPgqc6H8C1fHK_0 2024-06-28T16:14:33.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13760394399483008339', time=2024-06-28T17:53:41.952090+0000, chain.objs.size()=1 2024-06-28T16:14:33.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.xu40QLADUx2btYPQIrfZaGa2zPUsWOE_0 2024-06-28T16:14:33.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8967306084293915647', time=2024-06-28T17:53:42.307688+0000, chain.objs.size()=1 2024-06-28T16:14:33.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.RSWsYL1eI4PITgRyjCke1u1F6eJoexX_0 2024-06-28T16:14:33.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.319+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14496098399892143361', time=2024-06-28T17:53:42.781689+0000, chain.objs.size()=1 2024-06-28T16:14:33.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.VA_Q-0MVKMh86YAJaQlmjWk0F8G-p46_0 2024-06-28T16:14:33.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.319+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8563152151235752705', time=2024-06-28T17:53:42.929295+0000, chain.objs.size()=1 2024-06-28T16:14:33.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qPU0Drg2yzXPlZF7LuqVXrq6DyKVkZD_0 2024-06-28T16:14:33.333 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.323+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/49514 2024-06-28T16:14:33.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=26, truncated=0, next_marker='' 2024-06-28T16:14:33.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5282698257768838638', time=2024-06-28T17:53:42.932564+0000, chain.objs.size()=1 2024-06-28T16:14:33.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.VuejoZZOE1CPaD79TpujLs_xqaZEedV_0 2024-06-28T16:14:33.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2569710406789602272', time=2024-06-28T17:53:44.427773+0000, chain.objs.size()=1 2024-06-28T16:14:33.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GmRC5AvBwgfl3wvPLdV9aW3WJCfsrDG_0 2024-06-28T16:14:33.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12993669540278377552', time=2024-06-28T17:53:45.907468+0000, chain.objs.size()=1 2024-06-28T16:14:33.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.9XU9sJxoYwCUrRGwsfYanrtw9LgWiNX_0 2024-06-28T16:14:33.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7165760951162907024', time=2024-06-28T17:53:45.912100+0000, chain.objs.size()=1 2024-06-28T16:14:33.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Fq3tL8_yYr6N36kar-gwDFFGA6yhp7l_0 2024-06-28T16:14:33.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15377752161842253328', time=2024-06-28T17:53:45.914887+0000, chain.objs.size()=1 2024-06-28T16:14:33.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CPD9HnRW40UfUqKR1ARf9398CZeD2p-_0 2024-06-28T16:14:33.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1073410580041599045', time=2024-06-28T17:53:45.916969+0000, chain.objs.size()=1 2024-06-28T16:14:33.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.3QbxoPAVM9BUvZNt8I1Q37hAhozjj7I_0 2024-06-28T16:14:33.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7907822862888551301', time=2024-06-28T17:53:47.584761+0000, chain.objs.size()=1 2024-06-28T16:14:33.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Sa_8Q5kiaJhBRuyvuw7VNXNNkBE8dTD_0 2024-06-28T16:14:33.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1087016202307448931', time=2024-06-28T17:53:48.118051+0000, chain.objs.size()=1 2024-06-28T16:14:33.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.7ISRPYtYt5DIbGqsyAADcPi04LwZz-T_0 2024-06-28T16:14:33.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11323105399034897291', time=2024-06-28T17:53:48.813593+0000, chain.objs.size()=1 2024-06-28T16:14:33.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-Z3yfKWSQI31zccc4qpykt35vMg7-rF_0 2024-06-28T16:14:33.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5689944082521899085', time=2024-06-28T17:53:50.035716+0000, chain.objs.size()=1 2024-06-28T16:14:33.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.svVrhBeUZ9Yh73_WdG_fGGIujSjrLtt_0 2024-06-28T16:14:33.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13092814207318309334', time=2024-06-28T17:53:50.419428+0000, chain.objs.size()=1 2024-06-28T16:14:33.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.tm-zya1StDXNUaRYzCwn58qb34RknkS_0 2024-06-28T16:14:33.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1726272654510724720', time=2024-06-28T17:53:50.421356+0000, chain.objs.size()=1 2024-06-28T16:14:33.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bkxdUkkyqeWoftxSy4B7wavwHXXua5Q_0 2024-06-28T16:14:33.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.331+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13396207958138158177', time=2024-06-28T17:55:27.418230+0000, chain.objs.size()=1 2024-06-28T16:14:33.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.420__shadow_.RqewvDkaMbSK5UOBQUxTWsTP7OrwxyA_0 2024-06-28T16:14:33.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.331+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2012053308846623304', time=2024-06-28T17:56:28.116482+0000, chain.objs.size()=1 2024-06-28T16:14:33.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.473__shadow_.4MBpbuW_HA5IV8QaJ9i-JVY6jul9A_v_0 2024-06-28T16:14:33.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.331+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15265026345451634048', time=2024-06-28T17:58:51.013539+0000, chain.objs.size()=6 2024-06-28T16:14:33.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~zdhOaZ9lQdy2CERMBS9Zz_HWTjHY2T9.1 2024-06-28T16:14:33.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.QBFUb6XmQ1sCr79MnvQnjRno_t5ZTjAU.2 2024-06-28T16:14:33.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~zdhOaZ9lQdy2CERMBS9Zz_HWTjHY2T9.3 2024-06-28T16:14:33.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~zdhOaZ9lQdy2CERMBS9Zz_HWTjHY2T9.4 2024-06-28T16:14:33.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~zdhOaZ9lQdy2CERMBS9Zz_HWTjHY2T9.5 2024-06-28T16:14:33.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~zdhOaZ9lQdy2CERMBS9Zz_HWTjHY2T9.6 2024-06-28T16:14:33.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.335+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~CpPNfZw4JS_VTkV-4WkTNYuZO6RyFlY', time=2024-06-28T18:00:21.519597+0000, chain.objs.size()=1 2024-06-28T16:14:33.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.521__multipart_mymultipart.2~CpPNfZw4JS_VTkV-4WkTNYuZO6RyFlY.1 2024-06-28T16:14:33.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.335+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11935056198291558522', time=2024-06-28T18:01:27.477797+0000, chain.objs.size()=1 2024-06-28T16:14:33.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.545__shadow_.XvaKQGDEB7gLcMcmVHMiN9oaWtgDLMp_0 2024-06-28T16:14:33.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.335+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17372818954707149507', time=2024-06-28T18:01:43.175086+0000, chain.objs.size()=1 2024-06-28T16:14:33.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.550__multipart_mymultipart.2~xmtEP1wMDOFsWUJ9_WwKHgNQosOR0eU.1 2024-06-28T16:14:33.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.335+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11787220443870095491', time=2024-06-28T18:02:35.368331+0000, chain.objs.size()=1 2024-06-28T16:14:33.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.563__shadow:.9vQfya4DdqD-sTySO1u-EDNtXSLyIk_.erNDiVhzxSMwfQQ32fU6OphHYy2410h_0 2024-06-28T16:14:33.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.335+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8019350717626401837', time=2024-06-28T18:03:03.491894+0000, chain.objs.size()=1 2024-06-28T16:14:33.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:bpsnc9-Z5a3iCyCVAsQSjSkJIlb3GZI_.92nMGJbz-OMAo4tnMIM5AB_5vpJmyXX_0 2024-06-28T16:14:33.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7277348158518260768', time=2024-06-28T18:03:20.570835+0000, chain.objs.size()=1 2024-06-28T16:14:33.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:YMtypJWoRr3Bg3Kt.lPrP-L1JozZWAe_.c5vclaZVq56td2Q7SkUGUZDw9uLw9H0_0 2024-06-28T16:14:33.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1905708242439881353', time=2024-06-28T18:04:10.370791+0000, chain.objs.size()=1 2024-06-28T16:14:33.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.575__shadow:U.HDxs-J-qj3FI6Nn6-Xy-qBZOmyyDf_.PEYam7ogUTcTF_CRtD8tminG6fLjS-5_0 2024-06-28T16:14:33.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11113060997517146102', time=2024-06-28T18:06:00.268016+0000, chain.objs.size()=6 2024-06-28T16:14:33.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.604__multipart_multipart_enc.2~kLkCtyaD2ytXfBqO5CxPDD9v0HdtRhr.1 2024-06-28T16:14:33.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.604__multipart_multipart_enc.2~kLkCtyaD2ytXfBqO5CxPDD9v0HdtRhr.2 2024-06-28T16:14:33.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.604__multipart_multipart_enc.2~kLkCtyaD2ytXfBqO5CxPDD9v0HdtRhr.3 2024-06-28T16:14:33.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.604__multipart_multipart_enc.2~kLkCtyaD2ytXfBqO5CxPDD9v0HdtRhr.4 2024-06-28T16:14:33.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.604__multipart_multipart_enc.2~kLkCtyaD2ytXfBqO5CxPDD9v0HdtRhr.5 2024-06-28T16:14:33.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.604__multipart_multipart_enc.2~kLkCtyaD2ytXfBqO5CxPDD9v0HdtRhr.6 2024-06-28T16:14:33.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16826557499198706395', time=2024-06-28T18:07:13.013919+0000, chain.objs.size()=1 2024-06-28T16:14:33.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.631__shadow_.QUBcBpbS-eQmNDNb31EeBi_9WCY1_cQ_0 2024-06-28T16:14:33.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16887446898111624469', time=2024-06-28T18:08:01.150338+0000, chain.objs.size()=1 2024-06-28T16:14:33.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.649__shadow_.NM3jhvcS9UhGoGCYmtssbMj4aIslMcz_0 2024-06-28T16:14:33.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7305030929724442556', time=2024-06-28T18:09:41.478988+0000, chain.objs.size()=10 2024-06-28T16:14:33.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.718__multipart_foo.2~-ZDuwbctGQZNLhUaf4_DpbEh303PWSg.1 2024-06-28T16:14:33.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.718__multipart_foo.2~-ZDuwbctGQZNLhUaf4_DpbEh303PWSg.2 2024-06-28T16:14:33.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.718__multipart_foo.2~-ZDuwbctGQZNLhUaf4_DpbEh303PWSg.3 2024-06-28T16:14:33.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.718__multipart_foo.2~-ZDuwbctGQZNLhUaf4_DpbEh303PWSg.4 2024-06-28T16:14:33.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.718__multipart_foo.2~-ZDuwbctGQZNLhUaf4_DpbEh303PWSg.5 2024-06-28T16:14:33.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.718__multipart_foo.2~-ZDuwbctGQZNLhUaf4_DpbEh303PWSg.6 2024-06-28T16:14:33.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.718__multipart_foo.2~-ZDuwbctGQZNLhUaf4_DpbEh303PWSg.7 2024-06-28T16:14:33.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.347+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.718__multipart_foo.2~-ZDuwbctGQZNLhUaf4_DpbEh303PWSg.8 2024-06-28T16:14:33.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.347+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.718__multipart_foo.2~-ZDuwbctGQZNLhUaf4_DpbEh303PWSg.9 2024-06-28T16:14:33.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.347+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.718__multipart_foo.2~-ZDuwbctGQZNLhUaf4_DpbEh303PWSg.10 2024-06-28T16:14:33.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.351+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:33.364 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.355+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-06-28T16:14:33.365 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.355+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:33.366 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.355+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:33.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/42674' 2024-06-28T16:14:33.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8991536267481596791', time=2024-06-28T17:40:06.297428+0000, chain.objs.size()=2 2024-06-28T16:14:33.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.148__shadow_.7wZdsF-Iy2wm5T3_gcbT4A4NZ7_Cbv2_0 2024-06-28T16:14:33.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.148__shadow_.7wZdsF-Iy2wm5T3_gcbT4A4NZ7_Cbv2_1 2024-06-28T16:14:33.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10006926646655699621', time=2024-06-28T17:40:06.443069+0000, chain.objs.size()=1 2024-06-28T16:14:33.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.148__shadow_.CwrLACL3G8r9OWyHQlK7Qsd4J10CBFp_0 2024-06-28T16:14:33.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3005479246661138423', time=2024-06-28T17:44:40.630461+0000, chain.objs.size()=1 2024-06-28T16:14:33.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.224__shadow_.ewF-6LyES6qvuV4e--sPeyoYp6LAiv5_0 2024-06-28T16:14:33.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16715789724328739738', time=2024-06-28T17:44:52.492940+0000, chain.objs.size()=1 2024-06-28T16:14:33.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.231__shadow_.7HTrjI93SZ2TnYdrO8YPZWEqenL4zeJ_0 2024-06-28T16:14:33.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6323920904861155060', time=2024-06-28T17:44:52.518794+0000, chain.objs.size()=1 2024-06-28T16:14:33.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.231__shadow_.BTWkRe1mIqn1PQji54IoIOGNUJsR2KA_0 2024-06-28T16:14:33.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12330631867774676576', time=2024-06-28T17:45:06.591105+0000, chain.objs.size()=1 2024-06-28T16:14:33.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.239__shadow_.tN1yJHpKrHCjlRmKgH_pw2svgqMSfFn_0 2024-06-28T16:14:33.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8237177830438814575', time=2024-06-28T17:45:09.866024+0000, chain.objs.size()=1 2024-06-28T16:14:33.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.241__shadow_.OXkCUTRCvHddsLP87UbpUc6S_kg99f6_0 2024-06-28T16:14:33.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10113401179422268389', time=2024-06-28T17:47:07.850682+0000, chain.objs.size()=1 2024-06-28T16:14:33.369 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ax39SySddtOuN_ss-Vx_2vlP3UpLar6_0 2024-06-28T16:14:33.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17338032518292062935', time=2024-06-28T17:47:07.965928+0000, chain.objs.size()=1 2024-06-28T16:14:33.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.tFnBoNQVxAxnPoLjec1p2fnIwfcfjhu_0 2024-06-28T16:14:33.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12202951657098609482', time=2024-06-28T17:47:08.727505+0000, chain.objs.size()=1 2024-06-28T16:14:33.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.gdINS7dPdT4cbIrK3fmq1jr3cE-XOnn_0 2024-06-28T16:14:33.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2890331973184578679', time=2024-06-28T17:47:09.061570+0000, chain.objs.size()=1 2024-06-28T16:14:33.370 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.LpSYb_AWYPAkUiM8zxkdjGWBPH5IW46_0 2024-06-28T16:14:33.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5583805861211289641', time=2024-06-28T17:47:10.913324+0000, chain.objs.size()=1 2024-06-28T16:14:33.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.mkNJ6mOnBzrWPLrvKZTO0fhLJDhxFdF_0 2024-06-28T16:14:33.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11753041299108163207', time=2024-06-28T17:47:11.108115+0000, chain.objs.size()=1 2024-06-28T16:14:33.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.c_8ST8_UuDIM2LBxsNRovruGHS39PyX_0 2024-06-28T16:14:33.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7486619090502803623', time=2024-06-28T17:47:14.290382+0000, chain.objs.size()=1 2024-06-28T16:14:33.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5jtizfRYu4CC9qGijlIiJMXgTyfz9gf_0 2024-06-28T16:14:33.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15716109397604535875', time=2024-06-28T17:47:14.988932+0000, chain.objs.size()=1 2024-06-28T16:14:33.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.3EG88_P5agI0g3RzTleZiptRi-DvvNa_0 2024-06-28T16:14:33.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2508174307442716907', time=2024-06-28T17:47:15.621712+0000, chain.objs.size()=1 2024-06-28T16:14:33.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.RcUtB5tJ4tjiT6R6tdRUjeX9PsLnKgV_0 2024-06-28T16:14:33.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3689421601263311180', time=2024-06-28T17:47:17.197990+0000, chain.objs.size()=1 2024-06-28T16:14:33.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_._LkMpIuQgzAiFKK1LjN7bHuek2UMGn3_0 2024-06-28T16:14:33.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16249992520913601272', time=2024-06-28T17:47:17.635698+0000, chain.objs.size()=1 2024-06-28T16:14:33.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.sOoy4km9Hk825O6odFCN6xkLl1UWMI5_0 2024-06-28T16:14:33.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8127536215543987413', time=2024-06-28T17:47:19.496544+0000, chain.objs.size()=1 2024-06-28T16:14:33.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.u_GV4notuKuHF6jIuukanPERPYftiXe_0 2024-06-28T16:14:33.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7503839574511401451', time=2024-06-28T17:47:19.828064+0000, chain.objs.size()=1 2024-06-28T16:14:33.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.hErXr3zLlysreq-QaMcqlLBwAocLp1O_0 2024-06-28T16:14:33.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7964342581566947404', time=2024-06-28T17:47:22.725961+0000, chain.objs.size()=1 2024-06-28T16:14:33.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.fHQTTsjBaJCYND-PG442J-PvWWFcvOk_0 2024-06-28T16:14:33.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7523310102119764477', time=2024-06-28T17:47:23.741414+0000, chain.objs.size()=1 2024-06-28T16:14:33.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-ZdBclrbs79I4AbnYj3jNvy3Hyz0afY_0 2024-06-28T16:14:33.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17204286293151043351', time=2024-06-28T17:47:25.693290+0000, chain.objs.size()=1 2024-06-28T16:14:33.376 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.mLOT5pV1w1tuNg17XqZuIHc5q-8X1Io_0 2024-06-28T16:14:33.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10783258843062518308', time=2024-06-28T17:47:26.059346+0000, chain.objs.size()=1 2024-06-28T16:14:33.378 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.qcl2zs1PtMfetnAzZFpHrji5VGRHEpV_0 2024-06-28T16:14:33.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9561678696834024200', time=2024-06-28T17:47:27.145359+0000, chain.objs.size()=1 2024-06-28T16:14:33.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.EZEKj-2j2SKKkLdpuBBPSozTbU1nPJg_0 2024-06-28T16:14:33.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6408927841972959004', time=2024-06-28T17:47:28.931623+0000, chain.objs.size()=1 2024-06-28T16:14:33.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.jAlcwLfwh4uMPMSH6hfGT5Gtgpstqvf_0 2024-06-28T16:14:33.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14480936776107965147', time=2024-06-28T17:47:29.305691+0000, chain.objs.size()=1 2024-06-28T16:14:33.379 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.OIYe6kL03jYBzUIaBCLHZVJr6na50QD_0 2024-06-28T16:14:33.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12393256616706255273', time=2024-06-28T17:47:29.607555+0000, chain.objs.size()=1 2024-06-28T16:14:33.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.RrCwrZ2qWsXfXXD6GQf621ectNyJ6f8_0 2024-06-28T16:14:33.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7370002112813636162', time=2024-06-28T17:47:30.904262+0000, chain.objs.size()=1 2024-06-28T16:14:33.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KTBgpdIM-likgLmJcYLGB3wNlwqXVUA_0 2024-06-28T16:14:33.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1348182124844548567', time=2024-06-28T17:47:31.565524+0000, chain.objs.size()=1 2024-06-28T16:14:33.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Z3k_sAjSrRJwN00yq1tEJqR1uXZWjwe_0 2024-06-28T16:14:33.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3034469716793237255', time=2024-06-28T17:47:31.931412+0000, chain.objs.size()=1 2024-06-28T16:14:33.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7jHxhTJQCBb7RE1hENVX8P6X4eMzCRM_0 2024-06-28T16:14:33.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7669791567592965823', time=2024-06-28T17:47:32.047869+0000, chain.objs.size()=1 2024-06-28T16:14:33.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.wvTcBI9aHH-wKSMoqvZ-uMC0GB4sli5_0 2024-06-28T16:14:33.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3410690366274874600', time=2024-06-28T17:47:33.540715+0000, chain.objs.size()=1 2024-06-28T16:14:33.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.JVaI17tM7hNPhLfIBPtB2n2jXKOrEIK_0 2024-06-28T16:14:33.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.371+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11941457066059662749', time=2024-06-28T17:47:36.049673+0000, chain.objs.size()=1 2024-06-28T16:14:33.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.8qe_SgTNjMAh8DSpFrZuEQZTpbbpgyJ_0 2024-06-28T16:14:33.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.371+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5747543173583048366', time=2024-06-28T17:47:37.532287+0000, chain.objs.size()=1 2024-06-28T16:14:33.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.K4km6pXABBA4MbvWbacOY1zjU804Lwp_0 2024-06-28T16:14:33.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8551448091228658348', time=2024-06-28T17:47:38.974201+0000, chain.objs.size()=1 2024-06-28T16:14:33.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_._RkAPstdSvwlLspzvyufDfGmMLaQ8Jz_0 2024-06-28T16:14:33.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14139707709464722170', time=2024-06-28T17:48:06.929642+0000, chain.objs.size()=1 2024-06-28T16:14:33.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.270__shadow_.rvAQ-DJdj4syaa2yRZNuz3gkmeL8Qv__0 2024-06-28T16:14:33.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15735235790412789298', time=2024-06-28T17:48:34.860749+0000, chain.objs.size()=1 2024-06-28T16:14:33.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.InYgmrSQPt1rFjeck53JN9a0wPuFShH_0 2024-06-28T16:14:33.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10288894869207174263', time=2024-06-28T17:48:36.516301+0000, chain.objs.size()=1 2024-06-28T16:14:33.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.284__shadow_.ikKf58CpckDACIrTAyioZ1xcHxtllSl_0 2024-06-28T16:14:33.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16422119580680621927', time=2024-06-28T17:51:07.133948+0000, chain.objs.size()=1 2024-06-28T16:14:33.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.MEeUOFFw4ZOcoQgaZfOgdyjl4SAvcS1_0 2024-06-28T16:14:33.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1827496638864408200', time=2024-06-28T17:51:07.589248+0000, chain.objs.size()=1 2024-06-28T16:14:33.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.u5wGjX0Q-1RsOyUnGzuLZq0XanCBwJR_0 2024-06-28T16:14:33.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14710007434635330016', time=2024-06-28T17:51:07.864852+0000, chain.objs.size()=1 2024-06-28T16:14:33.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1puCslfDXILsLSS6bQ6NA2x79-WIGXy_0 2024-06-28T16:14:33.385 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15745257292404859131', time=2024-06-28T17:51:08.370468+0000, chain.objs.size()=1 2024-06-28T16:14:33.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.9YbNkg4R4nxTu0JkoypOCxV0Jdb_MKe_0 2024-06-28T16:14:33.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6495067570541301307', time=2024-06-28T17:51:09.018250+0000, chain.objs.size()=1 2024-06-28T16:14:33.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Sqk_dNC_YOHM4GKKw6Ihw41o9xiBx-h_0 2024-06-28T16:14:33.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1408288596632263295', time=2024-06-28T17:51:09.577218+0000, chain.objs.size()=1 2024-06-28T16:14:33.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.KsxpkDgLH8Ez6y4eQp7jO6g_RDf0a2R_0 2024-06-28T16:14:33.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.379+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6950659113063022925', time=2024-06-28T17:51:09.786567+0000, chain.objs.size()=1 2024-06-28T16:14:33.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.379+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.52xivaFBiK5_UgiHrdwujjh6xuppiHX_0 2024-06-28T16:14:33.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.379+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1497985477163680071', time=2024-06-28T17:51:11.035589+0000, chain.objs.size()=1 2024-06-28T16:14:33.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.379+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.odXhbTZRlrZasad4KKjdqXNrkBblXjl_0 2024-06-28T16:14:33.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.379+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17770746925319103260', time=2024-06-28T17:51:11.518929+0000, chain.objs.size()=1 2024-06-28T16:14:33.389 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.379+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YnnPVhymLGpZrDPvAOXFShocxUR9uHN_0 2024-06-28T16:14:33.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.379+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15982089365464909954', time=2024-06-28T17:51:11.785607+0000, chain.objs.size()=1 2024-06-28T16:14:33.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.379+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hawXiiI4P31M2PGSFmlyRmGaRQVddeZ_0 2024-06-28T16:14:33.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5484348681315418681', time=2024-06-28T17:51:12.104054+0000, chain.objs.size()=1 2024-06-28T16:14:33.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yOPYCaeZv2jQXR3eva7O1eLObdHR6_Y_0 2024-06-28T16:14:33.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11762563380542350201', time=2024-06-28T17:51:13.456800+0000, chain.objs.size()=1 2024-06-28T16:14:33.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nWfwgaBO65TWARA1QSLeRP1QP-7nf61_0 2024-06-28T16:14:33.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6205142270395866975', time=2024-06-28T17:51:13.458534+0000, chain.objs.size()=1 2024-06-28T16:14:33.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.bCSeExW016l3ujePYworbrSp9w4S_lf_0 2024-06-28T16:14:33.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11844768669235515014', time=2024-06-28T17:51:15.368948+0000, chain.objs.size()=1 2024-06-28T16:14:33.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.wxJY6e2sLIbSGVKxvpc9WHC9N44URSv_0 2024-06-28T16:14:33.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.343237729044590522', time=2024-06-28T17:51:15.905185+0000, chain.objs.size()=1 2024-06-28T16:14:33.392 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6gIhSDyUKAr0cl0iMrwyVJpVnNYMwrs_0 2024-06-28T16:14:33.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7303086421751644739', time=2024-06-28T17:51:16.664373+0000, chain.objs.size()=1 2024-06-28T16:14:33.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xwWh-iZHEWduY2Lu0AfDpCNU9Iny29Q_0 2024-06-28T16:14:33.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14968451514165608101', time=2024-06-28T17:51:19.573749+0000, chain.objs.size()=1 2024-06-28T16:14:33.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-BuZQzrB2BKaa9ji8Eq61lKT4xCFMTj_0 2024-06-28T16:14:33.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15806609643022967198', time=2024-06-28T17:51:22.667659+0000, chain.objs.size()=1 2024-06-28T16:14:33.393 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.FhPiVr7w103JRVS9CkYu_tVvIxyPtqy_0 2024-06-28T16:14:33.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.860585257779581081', time=2024-06-28T17:51:23.649317+0000, chain.objs.size()=1 2024-06-28T16:14:33.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-lkM5J0HXeA4ucV_hVI67VbuLgB_DBf_0 2024-06-28T16:14:33.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8664793470090003552', time=2024-06-28T17:51:24.347089+0000, chain.objs.size()=1 2024-06-28T16:14:33.394 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.sB2aKsFIn09l7wbfAdfWwsiP2P5_elT_0 2024-06-28T16:14:33.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8377283693486741239', time=2024-06-28T17:51:24.666128+0000, chain.objs.size()=1 2024-06-28T16:14:33.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.IkKHxF4PdsQiax3vTdSH87a2dgVVqHF_0 2024-06-28T16:14:33.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6099707769951252878', time=2024-06-28T17:51:25.017837+0000, chain.objs.size()=1 2024-06-28T16:14:33.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.383+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.TffKkRZz8FRyDnH0oc4UDh5cRCl-LJ-_0 2024-06-28T16:14:33.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7524352765129774073', time=2024-06-28T17:51:25.027345+0000, chain.objs.size()=1 2024-06-28T16:14:33.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yJxkKx1pf1TJo1Cx9uCRhYCD5RYK-ot_0 2024-06-28T16:14:33.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15189882633105995672', time=2024-06-28T17:51:26.790981+0000, chain.objs.size()=1 2024-06-28T16:14:33.396 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.N8Mb7ROtlr6a3x_7e2kjKbt9eZTbjfZ_0 2024-06-28T16:14:33.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12744820804160257362', time=2024-06-28T17:51:28.607402+0000, chain.objs.size()=1 2024-06-28T16:14:33.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.B1tq9v6SLu_3efSip6F0Cx0H_lcWQNz_0 2024-06-28T16:14:33.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1689687415881075928', time=2024-06-28T17:51:28.783832+0000, chain.objs.size()=1 2024-06-28T16:14:33.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.zK5ZJ1PsAM2rtj4oegzrgj6vJ9Y3hVu_0 2024-06-28T16:14:33.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8652235802275279454', time=2024-06-28T17:51:29.076164+0000, chain.objs.size()=1 2024-06-28T16:14:33.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.CuP23gMMen0fATA7TdhMSAmr5wADc29_0 2024-06-28T16:14:33.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3943402047414248544', time=2024-06-28T17:51:30.651410+0000, chain.objs.size()=1 2024-06-28T16:14:33.397 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ymBGvvcC5bh0nZZ3KCEsa-JPsAh_C21_0 2024-06-28T16:14:33.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13577573279236995987', time=2024-06-28T17:51:30.973969+0000, chain.objs.size()=1 2024-06-28T16:14:33.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.0CDYuFGIew1OGAIHQ7dBooBXMzmEULd_0 2024-06-28T16:14:33.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9841385000108228554', time=2024-06-28T17:51:32.822935+0000, chain.objs.size()=1 2024-06-28T16:14:33.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.387+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.AcjWJO6Bo0oHmu5ToW5ARCr_MQdnVvM_0 2024-06-28T16:14:33.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14072880055183801482', time=2024-06-28T17:51:33.668238+0000, chain.objs.size()=1 2024-06-28T16:14:33.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1rE3RVEEvvHuO6n9e5S2CJnGnZySfRC_0 2024-06-28T16:14:33.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8831211266757342695', time=2024-06-28T17:51:33.749730+0000, chain.objs.size()=1 2024-06-28T16:14:33.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.qKNjfp0q4WDtbFubwbTl5rH1HH8eZf7_0 2024-06-28T16:14:33.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6070611099479218225', time=2024-06-28T17:51:37.648377+0000, chain.objs.size()=1 2024-06-28T16:14:33.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.HznIM0IH-0TBjc8jJqsKKavf6VvWTWz_0 2024-06-28T16:14:33.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12525575391853563562', time=2024-06-28T17:51:37.895846+0000, chain.objs.size()=1 2024-06-28T16:14:33.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-gcQN795xTp6qBrvcoL1buMm2HqnnGD_0 2024-06-28T16:14:33.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13485922510450802690', time=2024-06-28T17:53:21.662739+0000, chain.objs.size()=1 2024-06-28T16:14:33.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.rVvxYZTTadCB679h5n1zxpXvSnVmp1L_0 2024-06-28T16:14:33.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14863906422508658902', time=2024-06-28T17:53:21.958238+0000, chain.objs.size()=1 2024-06-28T16:14:33.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4QneN8j0t3F251or0yIMoeRIte69xfY_0 2024-06-28T16:14:33.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1535474426933993935', time=2024-06-28T17:53:22.067755+0000, chain.objs.size()=1 2024-06-28T16:14:33.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Exxcy2xzbLyMSBtuxCFqi29byiJfnve_0 2024-06-28T16:14:33.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18350694096557859687', time=2024-06-28T17:53:22.551623+0000, chain.objs.size()=1 2024-06-28T16:14:33.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.KycVscU9WSJkSrONDqpGeuVx9rWeYbU_0 2024-06-28T16:14:33.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16905129657362752789', time=2024-06-28T17:53:25.197990+0000, chain.objs.size()=1 2024-06-28T16:14:33.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.lK6yt9C7cRPq1GMGNUHXqX-svCvYNf7_0 2024-06-28T16:14:33.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3503325809116328328', time=2024-06-28T17:53:25.802655+0000, chain.objs.size()=1 2024-06-28T16:14:33.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.G5pmMddwzj299CkLXgy73DbrukiI3Vr_0 2024-06-28T16:14:33.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12640759093380307545', time=2024-06-28T17:53:26.773751+0000, chain.objs.size()=1 2024-06-28T16:14:33.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.P__wJfSaSTKx2iWZqKXqCz88FiuDhov_0 2024-06-28T16:14:33.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12242047074647913980', time=2024-06-28T17:53:29.978016+0000, chain.objs.size()=1 2024-06-28T16:14:33.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.SPlYsLprlgrigqBS3_BwNlfRLdEG2ua_0 2024-06-28T16:14:33.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14150723244319043414', time=2024-06-28T17:53:31.158619+0000, chain.objs.size()=1 2024-06-28T16:14:33.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.zhs0ZpoyvWtS1UJr7Yp43BW0dV25Lfb_0 2024-06-28T16:14:33.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4368948357776292726', time=2024-06-28T17:53:31.437949+0000, chain.objs.size()=1 2024-06-28T16:14:33.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ct8aqfMuKVrPrheTlIKMha9C32UnBIC_0 2024-06-28T16:14:33.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7695171466688221807', time=2024-06-28T17:53:31.584719+0000, chain.objs.size()=1 2024-06-28T16:14:33.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qqCtqgAmPy67pdTT1FH-29ZD18OEbUl_0 2024-06-28T16:14:33.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3311817450384766797', time=2024-06-28T17:53:31.966495+0000, chain.objs.size()=1 2024-06-28T16:14:33.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Te7kVVha_CgUj5nkXOxQcy1MSQ6FtxM_0 2024-06-28T16:14:33.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9431338716651746326', time=2024-06-28T17:53:32.460552+0000, chain.objs.size()=1 2024-06-28T16:14:33.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.lEHOJ15F3HHanLp-VL12PElQnp_U0E-_0 2024-06-28T16:14:33.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5031555237394120997', time=2024-06-28T17:53:33.729546+0000, chain.objs.size()=1 2024-06-28T16:14:33.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wrCAIzGaGsYeHpUiSPeU1v4CUXbccpB_0 2024-06-28T16:14:33.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15653575515859624967', time=2024-06-28T17:53:34.232454+0000, chain.objs.size()=1 2024-06-28T16:14:33.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.jOLVhylF9fK1avbzvyyx_AevENlWIOJ_0 2024-06-28T16:14:33.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1738220118661680262', time=2024-06-28T17:53:37.808793+0000, chain.objs.size()=1 2024-06-28T16:14:33.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.g33hho-xyMgpi-98su7KHZzjCkUnCCv_0 2024-06-28T16:14:33.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7581006614538964103', time=2024-06-28T17:53:40.286163+0000, chain.objs.size()=1 2024-06-28T16:14:33.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.iMxdl8hxSuoumHkuhfQPHsK3AaLMksy_0 2024-06-28T16:14:33.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4543143999502300532', time=2024-06-28T17:53:40.954814+0000, chain.objs.size()=1 2024-06-28T16:14:33.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.L7CO0WNSP5Gu8Y7eIg9GGENFO7JJ2nb_0 2024-06-28T16:14:33.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1426895629340927774', time=2024-06-28T17:53:41.840393+0000, chain.objs.size()=1 2024-06-28T16:14:33.410 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BbCg5A8hd5h88rg08r1pyAgulEpJS8g_0 2024-06-28T16:14:33.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8729669660644142350', time=2024-06-28T17:53:41.842153+0000, chain.objs.size()=1 2024-06-28T16:14:33.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gsUjKHT9LDqPeKPAiN2Cx5laXa0rfWU_0 2024-06-28T16:14:33.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14537628597620181929', time=2024-06-28T17:53:42.215815+0000, chain.objs.size()=1 2024-06-28T16:14:33.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bNUWs6JM6EixW2b6yiFH_szyKsBKiVy_0 2024-06-28T16:14:33.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17231943936454424586', time=2024-06-28T17:53:43.785169+0000, chain.objs.size()=1 2024-06-28T16:14:33.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vIROU90yLaP8w14I9sKjF5N6mpnAbdT_0 2024-06-28T16:14:33.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15752766749299379601', time=2024-06-28T17:53:44.996554+0000, chain.objs.size()=1 2024-06-28T16:14:33.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.8U5wU88EWfCO3A2Aqhp4AiT8cb8e88l_0 2024-06-28T16:14:33.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4067246117900856964', time=2024-06-28T17:53:46.786676+0000, chain.objs.size()=1 2024-06-28T16:14:33.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.399+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.beuAPN9so4ZhBnElFBdA-m1jhc1YMYr_0 2024-06-28T16:14:33.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18404999652074059316', time=2024-06-28T17:53:47.306724+0000, chain.objs.size()=1 2024-06-28T16:14:33.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qsJWxoqT3IROEgjz8P93C5YUC_MrllG_0 2024-06-28T16:14:33.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15692372400780922511', time=2024-06-28T17:53:47.591731+0000, chain.objs.size()=1 2024-06-28T16:14:33.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Zsd5GSuKiKNH_jp0MTv5xI_dcaZdRL2_0 2024-06-28T16:14:33.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10408167740164234839', time=2024-06-28T17:53:49.923572+0000, chain.objs.size()=1 2024-06-28T16:14:33.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Vit05pyH-vv0yGNqWjhZPhitC_pTJjn_0 2024-06-28T16:14:33.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/42674 2024-06-28T16:14:33.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=18, truncated=0, next_marker='' 2024-06-28T16:14:33.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3853704861937767442', time=2024-06-28T17:53:51.415642+0000, chain.objs.size()=1 2024-06-28T16:14:33.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Il1Q9C88UWUIXSk0-QeVfOhn-HwLcWh_0 2024-06-28T16:14:33.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12242033766587606739', time=2024-06-28T17:55:56.551561+0000, chain.objs.size()=1 2024-06-28T16:14:33.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.446__shadow_.osDcwMatKWxb3ePwtVfUyBKrihHwsjt_0 2024-06-28T16:14:33.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9455793646499352991', time=2024-06-28T17:56:11.377189+0000, chain.objs.size()=1 2024-06-28T16:14:33.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.455__shadow_.qVcOQ6tTWxMatZMxbClpxVL_Pu1TvTh_0 2024-06-28T16:14:33.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10938778010505213427', time=2024-06-28T17:56:13.170776+0000, chain.objs.size()=1 2024-06-28T16:14:33.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.456__shadow_.Lhmf0CbG2z9vCDRRK8ZwWf67_XTmhc6_0 2024-06-28T16:14:33.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.485__shadow_.JbpT2OSJT2NyHKZ0dhfYRV89uhwWES2_1_97f6c69c851f3b20d6bf65c0b692b80b', time=2024-06-28T17:56:44.282037+0000, chain.objs.size()=1 2024-06-28T16:14:33.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.485__shadow_.JbpT2OSJT2NyHKZ0dhfYRV89uhwWES2_0 2024-06-28T16:14:33.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10056989357609149171', time=2024-06-28T17:57:17.011120+0000, chain.objs.size()=1 2024-06-28T16:14:33.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.504__shadow_.3nWCYESTBOgmy0ngrMJYN9PDvakBf7D_0 2024-06-28T16:14:33.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4307306335117063589', time=2024-06-28T17:57:25.368568+0000, chain.objs.size()=1 2024-06-28T16:14:33.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.508__multipart_mymultipart.2~Wh5PGYIvH03nG2QuxKImMEZatrxsa2D.1 2024-06-28T16:14:33.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3922265985026951925', time=2024-06-28T17:57:28.615198+0000, chain.objs.size()=2 2024-06-28T16:14:33.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.507__shadow_.ksSZ7hS43Ztu6hY2tR_8j2H6NMMmqXX_0 2024-06-28T16:14:33.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.507__shadow_.ksSZ7hS43Ztu6hY2tR_8j2H6NMMmqXX_1 2024-06-28T16:14:33.427 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~72uj7Oz_zf5a3iRxeLnWsnMGff6IikF', time=2024-06-28T17:58:19.728653+0000, chain.objs.size()=1 2024-06-28T16:14:33.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~72uj7Oz_zf5a3iRxeLnWsnMGff6IikF.1 2024-06-28T16:14:33.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~XcEhOlbqxwx5ILp93LRz5QO7PCoWf8Y', time=2024-06-28T18:00:21.717268+0000, chain.objs.size()=1 2024-06-28T16:14:33.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.521__multipart_mymultipart2.2~XcEhOlbqxwx5ILp93LRz5QO7PCoWf8Y.1 2024-06-28T16:14:33.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~qvx0Xa0sqdRP-IRXU1FJIs2__zyDMMQ', time=2024-06-28T18:00:24.367898+0000, chain.objs.size()=1 2024-06-28T16:14:33.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.415+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.523__multipart_mymultipart.2~qvx0Xa0sqdRP-IRXU1FJIs2__zyDMMQ.1 2024-06-28T16:14:33.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.419+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8536296571434111499', time=2024-06-28T18:02:15.888649+0000, chain.objs.size()=1 2024-06-28T16:14:33.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:8NbhuOtl8BMIjl2Ex8Vg4uKJmcrzTw9_.RAHxqpwGQ844uJg1ExcJGb1wBtxlRnX_0 2024-06-28T16:14:33.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.419+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6923467408637961806', time=2024-06-28T18:03:22.418244+0000, chain.objs.size()=1 2024-06-28T16:14:33.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:Ptr3CN.ZjzdePVNMtEX1eaET9BRKn3i_.rKJoYMcHYw6qvjh345m4LqcvN0Twcx9_0 2024-06-28T16:14:33.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.419+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:T-3PRIPFnQthlN2zlbvRSM--a-Jg49Q_._srzhP6oOXAaOJL6gJYGDQX4134NPi3_1_abe3a32eaa7d64a2d047f49d62006606', time=2024-06-28T18:04:00.531828+0000, chain.objs.size()=1 2024-06-28T16:14:33.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:T-3PRIPFnQthlN2zlbvRSM--a-Jg49Q_._srzhP6oOXAaOJL6gJYGDQX4134NPi3_0 2024-06-28T16:14:33.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.419+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3117885278759267861', time=2024-06-28T18:04:13.290035+0000, chain.objs.size()=1 2024-06-28T16:14:33.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.576__shadow:7WwBWC.Zw2Zqz1zXSAW4kIFo4-wngbb_.FGpVWqDN3A3Nse8zmOqCPGKbKebNERv_0 2024-06-28T16:14:33.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.419+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15574406794954687473', time=2024-06-28T18:07:19.429473+0000, chain.objs.size()=1 2024-06-28T16:14:33.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.634__shadow_.GLhbMRnr7yMwthC8GXZ8zpvzJos8Q5w_0 2024-06-28T16:14:33.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.419+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5063876783526588374', time=2024-06-28T18:07:25.133044+0000, chain.objs.size()=1 2024-06-28T16:14:33.430 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.638__shadow_.-2rzNP8k16kmVRVhAv1mIx4g-5Gfcc7_0 2024-06-28T16:14:33.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.419+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5140574285465088488', time=2024-06-28T18:09:31.532594+0000, chain.objs.size()=1 2024-06-28T16:14:33.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.717__shadow_.gCmqxHOLvWnuXnA1FbZcCtaAACGw1wT_0 2024-06-28T16:14:33.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.423+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:33.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-06-28T16:14:33.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:33.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:33.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/50991' 2024-06-28T16:14:33.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_3TrYJMIZ_StpPDDNtuNKZYvrvupe8jA', time=2024-06-28T17:40:37.734561+0000, chain.objs.size()=1 2024-06-28T16:14:33.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.153__shadow_.IWbukQkyKKFhFE2j9axPHGUKvHK3DH__0 2024-06-28T16:14:33.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1095080996376647454', time=2024-06-28T17:41:19.391599+0000, chain.objs.size()=1 2024-06-28T16:14:33.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.178__shadow_.3vYE7vMiVhpVI6q94BkwXRc6YiDOuug_0 2024-06-28T16:14:33.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11433048183688287691', time=2024-06-28T17:44:40.597576+0000, chain.objs.size()=1 2024-06-28T16:14:33.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.224__shadow_.h2m8e2ZDfK_XyVwx8NlUDZKokradUtE_0 2024-06-28T16:14:33.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16808365801667687540', time=2024-06-28T17:44:42.248637+0000, chain.objs.size()=1 2024-06-28T16:14:33.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.225__shadow_.GwtrRytSx9LRax-ICg_cViCyw5DBfhs_0 2024-06-28T16:14:33.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3724099067062911173', time=2024-06-28T17:44:58.722264+0000, chain.objs.size()=1 2024-06-28T16:14:33.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.234__shadow_.Xlswl5GsVE2ywfZcYI3O6IB0kZ_7H-Y_0 2024-06-28T16:14:33.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12470720402641431177', time=2024-06-28T17:45:09.868949+0000, chain.objs.size()=1 2024-06-28T16:14:33.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.241__shadow_.a7gzbFPELlZIP_7svJuBS8q1Z-2qJ_6_0 2024-06-28T16:14:33.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2732820505146504228', time=2024-06-28T17:45:13.084986+0000, chain.objs.size()=1 2024-06-28T16:14:33.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.243__shadow_.psthotBWMbKlkDYuymIl9qBOYprGWfy_0 2024-06-28T16:14:33.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16279834563022883595', time=2024-06-28T17:45:16.216552+0000, chain.objs.size()=1 2024-06-28T16:14:33.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.245__shadow_.zUiXIkxchbM2LBmUfbPKUddTOPSV9dw_0 2024-06-28T16:14:33.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11352715491767751537', time=2024-06-28T17:47:09.636685+0000, chain.objs.size()=1 2024-06-28T16:14:33.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.1tB6wCyTcUy7zEtHzw47X9MtKmlAEte_0 2024-06-28T16:14:33.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4039902106301167726', time=2024-06-28T17:47:09.644139+0000, chain.objs.size()=1 2024-06-28T16:14:33.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.f7na1sXyDOJ3OQUbFYERw4u--k_NT4O_0 2024-06-28T16:14:33.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2627880842493806268', time=2024-06-28T17:47:10.066279+0000, chain.objs.size()=1 2024-06-28T16:14:33.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.N-9jZuiJmnhW-Jl-4Y52VTnBigev4g5_0 2024-06-28T16:14:33.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12257025072110823911', time=2024-06-28T17:47:12.168854+0000, chain.objs.size()=1 2024-06-28T16:14:33.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.xpbsHdDvmnarsI1fVbDY9PgJGKnLgfR_0 2024-06-28T16:14:33.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11924022310898365940', time=2024-06-28T17:47:12.338981+0000, chain.objs.size()=1 2024-06-28T16:14:33.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ughcPC5IcIwXd9xWCBg0i9G0kpgQ9ZQ_0 2024-06-28T16:14:33.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6305949867296682861', time=2024-06-28T17:47:13.223072+0000, chain.objs.size()=1 2024-06-28T16:14:33.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.bbJIjfaK1qWjap_EGRTkTxDeqwptoi6_0 2024-06-28T16:14:33.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15832945257129891714', time=2024-06-28T17:47:15.350300+0000, chain.objs.size()=1 2024-06-28T16:14:33.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ACrKzm15XNcX2cSFQaq3Tzox0WDWanL_0 2024-06-28T16:14:33.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7462194062451521336', time=2024-06-28T17:47:17.577201+0000, chain.objs.size()=1 2024-06-28T16:14:33.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.cvCjx0B7AWqfbgooUgNE44amT2SS108_0 2024-06-28T16:14:33.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.754690614560369515', time=2024-06-28T17:47:17.587137+0000, chain.objs.size()=1 2024-06-28T16:14:33.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.GRMARDtczzwl4yXkKGH8PewoYLVARHk_0 2024-06-28T16:14:33.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17738212091589110764', time=2024-06-28T17:47:18.228973+0000, chain.objs.size()=1 2024-06-28T16:14:33.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.yzehajJu0aouCJb-YQA5zJQhwhw3q54_0 2024-06-28T16:14:33.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17973775245286424817', time=2024-06-28T17:47:18.853605+0000, chain.objs.size()=1 2024-06-28T16:14:33.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.tJDJInceUK701N9lKyVK9NAjniYaP-8_0 2024-06-28T16:14:33.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.643160049676947293', time=2024-06-28T17:47:19.439112+0000, chain.objs.size()=1 2024-06-28T16:14:33.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.cSYBpCeA4zbbuUt8madm92ADi4Rg65q_0 2024-06-28T16:14:33.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2398902367570886877', time=2024-06-28T17:47:21.732771+0000, chain.objs.size()=1 2024-06-28T16:14:33.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5YFuwY4r5Hte1Loc7g09bF8roKJjDSj_0 2024-06-28T16:14:33.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17544647529091189980', time=2024-06-28T17:47:21.860726+0000, chain.objs.size()=1 2024-06-28T16:14:33.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oGqcS-B9XSBpwI7pSce5vEj2kkFWnco_0 2024-06-28T16:14:33.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2095469850072860478', time=2024-06-28T17:47:23.439030+0000, chain.objs.size()=1 2024-06-28T16:14:33.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.mDzOrgP5cM3nlBCDt-du5WOQ3YI4zxb_0 2024-06-28T16:14:33.446 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5934390704706155645', time=2024-06-28T17:47:23.756504+0000, chain.objs.size()=1 2024-06-28T16:14:33.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.bQnqGAqOYW3_oVkDRJRiMscm69qrP3T_0 2024-06-28T16:14:33.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4541100931996195474', time=2024-06-28T17:47:24.109031+0000, chain.objs.size()=1 2024-06-28T16:14:33.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.LVt35eW1mc1VY3Wq3R6NXwZPT4Bb1E-_0 2024-06-28T16:14:33.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4237023376229569212', time=2024-06-28T17:47:27.145084+0000, chain.objs.size()=1 2024-06-28T16:14:33.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4YLg9MLDdc3xk6g1kALDnmAKYGFKend_0 2024-06-28T16:14:33.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4099195418452301774', time=2024-06-28T17:47:27.776933+0000, chain.objs.size()=1 2024-06-28T16:14:33.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.nyGhW3ZM8lDXFP_WObGXHWD65tzT84L_0 2024-06-28T16:14:33.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17587108588808154511', time=2024-06-28T17:47:27.788244+0000, chain.objs.size()=1 2024-06-28T16:14:33.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9D9C9XZ63P86itPh1Y0ZGN5igNMuRnG_0 2024-06-28T16:14:33.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10652111432774868358', time=2024-06-28T17:47:27.955323+0000, chain.objs.size()=1 2024-06-28T16:14:33.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.18CPfYTwTdWasiIUeFkqzVFuoPOdDgl_0 2024-06-28T16:14:33.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8084222220187051736', time=2024-06-28T17:47:30.235849+0000, chain.objs.size()=1 2024-06-28T16:14:33.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.iDJguFrFdamWofvd5wUsjAFlXx3YBNU_0 2024-06-28T16:14:33.451 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1310665062145787674', time=2024-06-28T17:47:30.244246+0000, chain.objs.size()=1 2024-06-28T16:14:33.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Dz965XPZ0C-R8_JTBFObFLP1OVXxnn4_0 2024-06-28T16:14:33.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14668516403020553262', time=2024-06-28T17:47:30.885105+0000, chain.objs.size()=1 2024-06-28T16:14:33.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.GxgA1Y41wSEg8qVrAph95GPTpjR_sTh_0 2024-06-28T16:14:33.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1624849656995044018', time=2024-06-28T17:47:31.156890+0000, chain.objs.size()=1 2024-06-28T16:14:33.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-GbnonOiCx1MBhPH5DTYH-JP3etSPMV_0 2024-06-28T16:14:33.452 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.353142933149644946', time=2024-06-28T17:47:31.272441+0000, chain.objs.size()=1 2024-06-28T16:14:33.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.h2dJhUxq86yNXRwOEP4SkJVuDvR33SH_0 2024-06-28T16:14:33.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.443+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11023291540060419738', time=2024-06-28T17:47:31.668354+0000, chain.objs.size()=1 2024-06-28T16:14:33.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.443+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.LJDb2_7Pq_M3ZZaurYKqvUl-yKrSIUB_0 2024-06-28T16:14:33.453 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.443+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9896333824352654774', time=2024-06-28T17:47:32.182946+0000, chain.objs.size()=1 2024-06-28T16:14:33.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.443+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.qy4cJnxAryzn4-_hPwWN_epY0ZSxaaI_0 2024-06-28T16:14:33.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.443+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4695884215458627472', time=2024-06-28T17:47:33.002966+0000, chain.objs.size()=1 2024-06-28T16:14:33.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.443+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.jwQecuMxx3jk24pMZ1TKSsCEptMA2uK_0 2024-06-28T16:14:33.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.443+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10218701980783679612', time=2024-06-28T17:47:33.052603+0000, chain.objs.size()=1 2024-06-28T16:14:33.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.443+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Dv4azsh6I3_zXvf-9MtEbSsB2-U8l8P_0 2024-06-28T16:14:33.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.443+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14541448988041301495', time=2024-06-28T17:47:33.529498+0000, chain.objs.size()=1 2024-06-28T16:14:33.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.443+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Obhe1hrzAyDGbdUGbmhBzCuzMfWhbyH_0 2024-06-28T16:14:33.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.443+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13624539395469539156', time=2024-06-28T17:47:34.330724+0000, chain.objs.size()=1 2024-06-28T16:14:33.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.443+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.2NAeAHb17AuYBcnkBZUV_8U_bAYp1Os_0 2024-06-28T16:14:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1601674390454974094', time=2024-06-28T17:47:34.626003+0000, chain.objs.size()=1 2024-06-28T16:14:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.CM9e90QiHSOVDurNGoyts_kA7mQ-Ue9_0 2024-06-28T16:14:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9344592890440518409', time=2024-06-28T17:47:35.972110+0000, chain.objs.size()=1 2024-06-28T16:14:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.AoxeBB7MS1AqCi1fMh6IfT2G8Qko8ER_0 2024-06-28T16:14:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6055189269507651360', time=2024-06-28T17:47:37.296219+0000, chain.objs.size()=1 2024-06-28T16:14:33.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.UcKzISeIzcHypSNylWRlBQwiB3LH-fM_0 2024-06-28T16:14:33.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15373038372577627190', time=2024-06-28T17:47:38.094449+0000, chain.objs.size()=1 2024-06-28T16:14:33.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.W_8np9FwiatDrakxEUqbxIyqBQfG9Op_0 2024-06-28T16:14:33.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11586362377123207917', time=2024-06-28T17:47:38.328325+0000, chain.objs.size()=1 2024-06-28T16:14:33.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.FSjcTvh0J4ql6NjIe5DPB3jz_PBKUQZ_0 2024-06-28T16:14:33.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17541466063004880586', time=2024-06-28T17:47:41.126565+0000, chain.objs.size()=1 2024-06-28T16:14:33.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.253__shadow_.4Xl92j0xjbIdEmpO-0bUfaALnezDEtG_0 2024-06-28T16:14:33.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15901361335890963318', time=2024-06-28T17:47:44.084773+0000, chain.objs.size()=1 2024-06-28T16:14:33.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.255__shadow_.Z1AbODg2tRHj_WB-4dl9LM3r2bB0pMg_0 2024-06-28T16:14:33.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5751648721999336796', time=2024-06-28T17:47:58.947091+0000, chain.objs.size()=1 2024-06-28T16:14:33.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.265__shadow_.UJAGeb0VG8zEX2Roo0KkZvD6SgkQVlt_0 2024-06-28T16:14:33.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14591110434240073477', time=2024-06-28T17:48:00.511720+0000, chain.objs.size()=1 2024-06-28T16:14:33.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.266__shadow_.1W3yPC01VC2KzK6RHr9db0Wsc4959G1_0 2024-06-28T16:14:33.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1582592137371031633', time=2024-06-28T17:48:13.263695+0000, chain.objs.size()=1 2024-06-28T16:14:33.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.274__shadow_.PbONcwifseoZje0IqSPg2x8EqmYsgMj_0 2024-06-28T16:14:33.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13240294108029690741', time=2024-06-28T17:48:38.062308+0000, chain.objs.size()=1 2024-06-28T16:14:33.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.285__shadow_.eJ8YjZZfW8ZlEa0H4yoEfQ2imELkW1t_0 2024-06-28T16:14:33.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15165142294005683948', time=2024-06-28T17:48:43.019703+0000, chain.objs.size()=1 2024-06-28T16:14:33.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.288__shadow_.IE6CEr0ziZ9zdmhYDqYPeMovzmd3ew4_0 2024-06-28T16:14:33.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17095665262814068492', time=2024-06-28T17:51:07.232402+0000, chain.objs.size()=1 2024-06-28T16:14:33.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.kwpC1EBI2psdgWdVkHwxmHVlBO6I5zX_0 2024-06-28T16:14:33.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4444578431729542442', time=2024-06-28T17:51:07.233727+0000, chain.objs.size()=1 2024-06-28T16:14:33.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.lPfZE2G1gBSCt3pjyzgevnGG9z5OTMu_0 2024-06-28T16:14:33.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9110404392585762292', time=2024-06-28T17:51:07.747239+0000, chain.objs.size()=1 2024-06-28T16:14:33.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.BR9BKeSvDxoFmPTNqFSNHvMIQS6y7go_0 2024-06-28T16:14:33.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3241704736743201254', time=2024-06-28T17:51:08.465912+0000, chain.objs.size()=1 2024-06-28T16:14:33.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.pnqSIjy0_saIYCBJs3VWmNxd3yVeRQ-_0 2024-06-28T16:14:33.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7033037110185119357', time=2024-06-28T17:51:10.011148+0000, chain.objs.size()=1 2024-06-28T16:14:33.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yZQ6QxM39U2hapDzYf2Amdv45KFyG7u_0 2024-06-28T16:14:33.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13160282686471941099', time=2024-06-28T17:51:11.068758+0000, chain.objs.size()=1 2024-06-28T16:14:33.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.49RmxS6hXvXiTZFlt5rHboGacU6uxDq_0 2024-06-28T16:14:33.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.455+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4086827527400126545', time=2024-06-28T17:51:13.085205+0000, chain.objs.size()=1 2024-06-28T16:14:33.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Wd1afWcX8rfp98v5qmXyu4MKuSGyqZH_0 2024-06-28T16:14:33.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.455+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12634880903599992147', time=2024-06-28T17:51:13.131602+0000, chain.objs.size()=1 2024-06-28T16:14:33.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Q80fWSouQiuOs4XxHk72Q7FWsz99ZTs_0 2024-06-28T16:14:33.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.455+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17293767225331907353', time=2024-06-28T17:51:13.537508+0000, chain.objs.size()=1 2024-06-28T16:14:33.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.n0IUTWi-TeEU7VKR-Mxsd-zkPx7utnL_0 2024-06-28T16:14:33.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.455+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14161839484059607640', time=2024-06-28T17:51:14.274037+0000, chain.objs.size()=1 2024-06-28T16:14:33.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.kxCDgOPIfqWnE0z3xzWVVGlFZnm3vEh_0 2024-06-28T16:14:33.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4581302690238941924', time=2024-06-28T17:51:14.325917+0000, chain.objs.size()=1 2024-06-28T16:14:33.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.wuwnmC25dFedsZx2zZJUf8KOeoJ9M7__0 2024-06-28T16:14:33.468 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13124676499741400396', time=2024-06-28T17:51:17.702221+0000, chain.objs.size()=1 2024-06-28T16:14:33.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.R4hrcfvLr3oiRsZOZgHax7bQhRenVZc_0 2024-06-28T16:14:33.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1072244141547979888', time=2024-06-28T17:51:17.827558+0000, chain.objs.size()=1 2024-06-28T16:14:33.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.wKcfZkKjtoElcPPNOvW1N1hi0WFR6AK_0 2024-06-28T16:14:33.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10673466170838309867', time=2024-06-28T17:51:17.844434+0000, chain.objs.size()=1 2024-06-28T16:14:33.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.THQFroHM1KrNIyheYMvM68qII9MqPcp_0 2024-06-28T16:14:33.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5294134537354446613', time=2024-06-28T17:51:18.020317+0000, chain.objs.size()=1 2024-06-28T16:14:33.469 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.MvvhZD9C5MjnVkmwzWHPO4w9r-ePHrk_0 2024-06-28T16:14:33.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4233084888316037817', time=2024-06-28T17:51:19.873529+0000, chain.objs.size()=1 2024-06-28T16:14:33.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hVsDL0eMfXgI9HTjLPqGImkVD5rSZEe_0 2024-06-28T16:14:33.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11889439290021205240', time=2024-06-28T17:51:20.207761+0000, chain.objs.size()=1 2024-06-28T16:14:33.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.BikQgDNaGdKbTWHAoQX7XaqTB2w59l__0 2024-06-28T16:14:33.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10722996054716829710', time=2024-06-28T17:51:20.335281+0000, chain.objs.size()=1 2024-06-28T16:14:33.470 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.JeG7YxHvmkNAh_yymV6TGtcLgoRb969_0 2024-06-28T16:14:33.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18402837565797848092', time=2024-06-28T17:51:21.081598+0000, chain.objs.size()=1 2024-06-28T16:14:33.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.459+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.F3KCqTqbM2YecAQbFDXgv2bci_ICCeP_0 2024-06-28T16:14:33.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4917735106312596565', time=2024-06-28T17:51:21.997106+0000, chain.objs.size()=1 2024-06-28T16:14:33.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.F7JUa4fjb9G-epBGQz-H8ZWlW7QxwUB_0 2024-06-28T16:14:33.471 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11060623806050182269', time=2024-06-28T17:51:22.138360+0000, chain.objs.size()=1 2024-06-28T16:14:33.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.vR8T2BiVmwEON8l1Sv-P_h3IIViYLEY_0 2024-06-28T16:14:33.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8125702598052081157', time=2024-06-28T17:51:23.630022+0000, chain.objs.size()=1 2024-06-28T16:14:33.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xEyNM2fGBb0OaB_9TanXBlNcikMnLcO_0 2024-06-28T16:14:33.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3399616000645626615', time=2024-06-28T17:51:24.650353+0000, chain.objs.size()=1 2024-06-28T16:14:33.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.KRFNgUjM1zUqGPw3Ydq_PWpTO0eYeS__0 2024-06-28T16:14:33.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18212726486826133659', time=2024-06-28T17:51:25.990682+0000, chain.objs.size()=1 2024-06-28T16:14:33.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.7YFBcmiSTITF2xdjoQ059VxTi8TJfU6_0 2024-06-28T16:14:33.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15608997007810882076', time=2024-06-28T17:51:26.082372+0000, chain.objs.size()=1 2024-06-28T16:14:33.473 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.zL2JHd76ReGlu-YAlOgacV-Uv3vphY-_0 2024-06-28T16:14:33.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7474401219537274068', time=2024-06-28T17:51:26.357739+0000, chain.objs.size()=1 2024-06-28T16:14:33.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1SCkrdRLwdvmZpxaUV-Rds9juTkQYEe_0 2024-06-28T16:14:33.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2863887463624683439', time=2024-06-28T17:51:26.801621+0000, chain.objs.size()=1 2024-06-28T16:14:33.474 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.W7xrFS-pc2MHGxyyz8DLXhj-EKjC1SQ_0 2024-06-28T16:14:33.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15112890167685258296', time=2024-06-28T17:51:29.725132+0000, chain.objs.size()=1 2024-06-28T16:14:33.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xIXtz8GwN9xZN-x1lLfxk8ZW8kRJ8YF_0 2024-06-28T16:14:33.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15335848863827978287', time=2024-06-28T17:51:30.332094+0000, chain.objs.size()=1 2024-06-28T16:14:33.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.o9-yz61ilg5q8puNURomdSJUfUf5ksQ_0 2024-06-28T16:14:33.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12966467437656123007', time=2024-06-28T17:51:30.657640+0000, chain.objs.size()=1 2024-06-28T16:14:33.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ctIbS0FVv6Nhj_pXIj_aRAUo2y2BgNS_0 2024-06-28T16:14:33.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16932872882854186980', time=2024-06-28T17:51:30.675504+0000, chain.objs.size()=1 2024-06-28T16:14:33.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.oc7taiTzKFxuSijWBIjfTxw26hxwgNS_0 2024-06-28T16:14:33.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17909381409104052061', time=2024-06-28T17:51:31.871993+0000, chain.objs.size()=1 2024-06-28T16:14:33.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.8U4Z_VPMmFoqbyAMcZ_TN2UWqpDqS_r_0 2024-06-28T16:14:33.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17181453814928298999', time=2024-06-28T17:51:31.878133+0000, chain.objs.size()=1 2024-06-28T16:14:33.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.wTuIlI1q1kJLXuchn-7V1L86jxfBIEf_0 2024-06-28T16:14:33.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.504954595680032897', time=2024-06-28T17:51:32.243696+0000, chain.objs.size()=1 2024-06-28T16:14:33.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.UR_UJHXpA8PSUGc5Q8L_59k6L3tJuFf_0 2024-06-28T16:14:33.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17690370823824410164', time=2024-06-28T17:51:32.352114+0000, chain.objs.size()=1 2024-06-28T16:14:33.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Ko9epRI693rgQ1o-xX1QJOnj00E8N6r_0 2024-06-28T16:14:33.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15263381276352017545', time=2024-06-28T17:51:34.300573+0000, chain.objs.size()=1 2024-06-28T16:14:33.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.k6vqKeFcN2jB7GvVfTnlAg8ouvwGhrL_0 2024-06-28T16:14:33.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4486928188645528436', time=2024-06-28T17:51:37.650965+0000, chain.objs.size()=1 2024-06-28T16:14:33.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.i7Hdr4pJ3BdyiKhV0zodee6fx2g2EuN_0 2024-06-28T16:14:33.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14781952923948624239', time=2024-06-28T17:51:37.784056+0000, chain.objs.size()=1 2024-06-28T16:14:33.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.oVGDnjCt8CS3vN7xJCXfjF4n0M-CA7E_0 2024-06-28T16:14:33.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14567755346749554792', time=2024-06-28T17:51:37.988665+0000, chain.objs.size()=1 2024-06-28T16:14:33.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_._xSS017KKcDPUvAT2J8CRK0keFyyZZN_0 2024-06-28T16:14:33.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17012457151539298139', time=2024-06-28T17:53:21.261735+0000, chain.objs.size()=1 2024-06-28T16:14:33.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.6qDWktVIDcQBwIkGZZ20v5lY0pT3nD2_0 2024-06-28T16:14:33.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5006218578220469362', time=2024-06-28T17:53:26.317988+0000, chain.objs.size()=1 2024-06-28T16:14:33.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.KO9hOsPgth7k0duh9r6Q2r3INKQNntT_0 2024-06-28T16:14:33.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8505960962853181215', time=2024-06-28T17:53:29.712755+0000, chain.objs.size()=1 2024-06-28T16:14:33.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.471+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.DgpcGjAL2g5bPJLY5iPVW1Wgo6MN_Tz_0 2024-06-28T16:14:33.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.432179002951583824', time=2024-06-28T17:53:31.581719+0000, chain.objs.size()=1 2024-06-28T16:14:33.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.i2L7BiK6pmmiwJ8thwg9H2oVcUuMKkO_0 2024-06-28T16:14:33.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14296675112557996365', time=2024-06-28T17:53:34.373029+0000, chain.objs.size()=1 2024-06-28T16:14:33.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.fgAS5b_RhnKrZ-Zd-G4TTCTPcIRcFvQ_0 2024-06-28T16:14:33.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2497502570230934145', time=2024-06-28T17:53:35.008546+0000, chain.objs.size()=1 2024-06-28T16:14:33.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.6HDqtfMO9a-k-d1-HenhlnSxoG_zdPF_0 2024-06-28T16:14:33.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14695068603285681899', time=2024-06-28T17:53:35.617075+0000, chain.objs.size()=1 2024-06-28T16:14:33.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-iie26GWIScIemZqBHWcxa9OPc1oll-_0 2024-06-28T16:14:33.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3532462395179266086', time=2024-06-28T17:53:35.804753+0000, chain.objs.size()=1 2024-06-28T16:14:33.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.mOxzLTEeioHy8v7yG7E-X--mUKdhK96_0 2024-06-28T16:14:33.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5134803595262449364', time=2024-06-28T17:53:35.881579+0000, chain.objs.size()=1 2024-06-28T16:14:33.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4cCX0AtP61W7u6ckNKm1RlMGLAKBIvW_0 2024-06-28T16:14:33.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.479+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/50991 2024-06-28T16:14:33.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=26, truncated=0, next_marker='' 2024-06-28T16:14:33.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17623229256517254942', time=2024-06-28T17:53:36.675158+0000, chain.objs.size()=1 2024-06-28T16:14:33.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.0MwLQuQMRg8N5HsslATb79BZHrs0FMD_0 2024-06-28T16:14:33.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.420685997141509854', time=2024-06-28T17:53:36.843224+0000, chain.objs.size()=1 2024-06-28T16:14:33.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.iUO4biOs5IlplY-Su50DwZA1V0HsLZJ_0 2024-06-28T16:14:33.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10917384909493679526', time=2024-06-28T17:53:37.721671+0000, chain.objs.size()=1 2024-06-28T16:14:33.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.FZj9jbY4DsdlObhqyKpKMduefW9fnP5_0 2024-06-28T16:14:33.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7455231868129975639', time=2024-06-28T17:53:37.820262+0000, chain.objs.size()=1 2024-06-28T16:14:33.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ESN7Jq8euMypXSDyOilcLNafOOoxmsB_0 2024-06-28T16:14:33.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5265043194831052659', time=2024-06-28T17:53:38.860094+0000, chain.objs.size()=1 2024-06-28T16:14:33.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.7wHjRC7Yvi1DHo5qEPmRcMjcU_WYaOp_0 2024-06-28T16:14:33.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16768328872674727458', time=2024-06-28T17:53:39.218656+0000, chain.objs.size()=1 2024-06-28T16:14:33.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Mnaa7eyCaJrwl0lro0tK7QvbuuQUfXw_0 2024-06-28T16:14:33.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4407356287415526104', time=2024-06-28T17:53:41.872143+0000, chain.objs.size()=1 2024-06-28T16:14:33.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.cmESfOtgJ1Z4G0g8q9r7Ha50XUqdXWF_0 2024-06-28T16:14:33.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7612154470068799348', time=2024-06-28T17:53:41.874282+0000, chain.objs.size()=1 2024-06-28T16:14:33.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.R33j96rg1wxzyHClLgAIRDJIfkqMvNx_0 2024-06-28T16:14:33.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5521866585834357150', time=2024-06-28T17:53:43.795002+0000, chain.objs.size()=1 2024-06-28T16:14:33.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.dyruv6kZV-oRQmCon2WZhztmkL06KoL_0 2024-06-28T16:14:33.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2230356537542019731', time=2024-06-28T17:53:44.362039+0000, chain.objs.size()=1 2024-06-28T16:14:33.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.kGiKSdx6kRaJoFgeSryn6hAOo6mDgpf_0 2024-06-28T16:14:33.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1222453732232838108', time=2024-06-28T17:53:46.767915+0000, chain.objs.size()=1 2024-06-28T16:14:33.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.lMcuDUjF3CdIcfpss89QgND_8C4TbX0_0 2024-06-28T16:14:33.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13972373207131443240', time=2024-06-28T17:53:47.994105+0000, chain.objs.size()=1 2024-06-28T16:14:33.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.sfS-QNUP5vkEg9jXPCG1cdRCHfbRAVX_0 2024-06-28T16:14:33.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.379379788005720092', time=2024-06-28T17:53:48.095013+0000, chain.objs.size()=1 2024-06-28T16:14:33.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.aqEwQA5b89RH294QafVR848tYN_QJHN_0 2024-06-28T16:14:33.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2137994424263222937', time=2024-06-28T17:53:48.387987+0000, chain.objs.size()=1 2024-06-28T16:14:33.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.g3J-fdRh3qo_NVL0LV3pprWJ4t92USq_0 2024-06-28T16:14:33.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11443071592660759788', time=2024-06-28T17:53:49.004151+0000, chain.objs.size()=1 2024-06-28T16:14:33.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.IRC9KRC6KWZPN06yhEGsajZyzgcEddP_0 2024-06-28T16:14:33.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7835045775499786940', time=2024-06-28T17:53:49.692351+0000, chain.objs.size()=1 2024-06-28T16:14:33.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.dyz-Jzr_ncGLWi6mBjLsII_VoZjQ4yu_0 2024-06-28T16:14:33.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9314972386625468019', time=2024-06-28T17:53:49.945667+0000, chain.objs.size()=1 2024-06-28T16:14:33.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.flTp_X_RQAqwoOfvyKrJg0h_s7JzZuu_0 2024-06-28T16:14:33.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17094870517993872287', time=2024-06-28T17:53:50.015288+0000, chain.objs.size()=1 2024-06-28T16:14:33.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.A-koyyDmsV632kaAGxcAWBHvaRZBBig_0 2024-06-28T16:14:33.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14380207717593716377', time=2024-06-28T17:53:50.232249+0000, chain.objs.size()=1 2024-06-28T16:14:33.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.9hLCRPqqRFcI34tUWiXM_Tuh9N9asLx_0 2024-06-28T16:14:33.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1581347547040926900', time=2024-06-28T17:53:52.015340+0000, chain.objs.size()=1 2024-06-28T16:14:33.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Fty92WmRo1CstVuwJGJruEy7LonUWyA_0 2024-06-28T16:14:33.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17454416820749632399', time=2024-06-28T17:53:57.841356+0000, chain.objs.size()=1 2024-06-28T16:14:33.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.320__shadow_.m4Fhb7_qF5iXOrf35y9i-dwCJASijyc_0 2024-06-28T16:14:33.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1428928156914792702', time=2024-06-28T17:54:00.980456+0000, chain.objs.size()=1 2024-06-28T16:14:33.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.323__shadow_.wpB2y-HEHCv7g5-xqC7MW-MRVTRG8Xv_0 2024-06-28T16:14:33.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.477__shadow_.kojZS1O31-LCsDRsPT55FWpbanAp2LT_1_58450acc4904e539225a582ace371273', time=2024-06-28T17:56:36.860458+0000, chain.objs.size()=1 2024-06-28T16:14:33.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.477__shadow_.kojZS1O31-LCsDRsPT55FWpbanAp2LT_0 2024-06-28T16:14:33.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11149776688209312832', time=2024-06-28T18:03:13.830997+0000, chain.objs.size()=1 2024-06-28T16:14:33.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:ohjU3JFRHeJCqO0OcFMREsl9P1QajtJ_.Ui-fsIF9LJMhfdwf-_TfGvYFVHJMuAz_0 2024-06-28T16:14:33.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6134846226788112469', time=2024-06-28T18:04:21.542073+0000, chain.objs.size()=1 2024-06-28T16:14:33.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.585__shadow_.8jMCAM0OjCWxhdSMdHtkVCOtOqUqTAt_0 2024-06-28T16:14:33.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16330733291607735557', time=2024-06-28T18:06:58.392349+0000, chain.objs.size()=1 2024-06-28T16:14:33.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.622__shadow_.u1ptKjd4XxI6sIBLJk_kxjSPQj20KWj_0 2024-06-28T16:14:33.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:33.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-06-28T16:14:33.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:33.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:33.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45192' 2024-06-28T16:14:33.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_juGIdqZQJWBeYX2dW2qcvIRPGSx7zc8', time=2024-06-28T17:40:15.118978+0000, chain.objs.size()=1 2024-06-28T16:14:33.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.3vFC7Zr6HSM_kbTLnY2sqNdTLASf0R6_0 2024-06-28T16:14:33.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_L7VE6hHpxEk4rjbupd_SsfESVQM7wGD', time=2024-06-28T17:40:25.967479+0000, chain.objs.size()=1 2024-06-28T16:14:33.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.151__shadow_.EDSMw0NN00wiFEDcnl7wVJT2BAt9Q7e_0 2024-06-28T16:14:33.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4623845226123110082', time=2024-06-28T17:40:29.332333+0000, chain.objs.size()=3 2024-06-28T16:14:33.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.151__multipart_foo-LUKEWARM.2~tUuh4yFKeQnpGYronj3Wk2Z-Iff_gvO.1 2024-06-28T16:14:33.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.151__multipart_foo-LUKEWARM.2~tUuh4yFKeQnpGYronj3Wk2Z-Iff_gvO.2 2024-06-28T16:14:33.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.151__multipart_foo-LUKEWARM.2~tUuh4yFKeQnpGYronj3Wk2Z-Iff_gvO.3 2024-06-28T16:14:33.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16289460289271450534', time=2024-06-28T17:44:50.910381+0000, chain.objs.size()=1 2024-06-28T16:14:33.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.230__shadow_.IVrWGg5PANKvsR43kI9--NAC4uEpSGl_0 2024-06-28T16:14:33.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7198024119643060049', time=2024-06-28T17:44:58.703184+0000, chain.objs.size()=1 2024-06-28T16:14:33.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.234__shadow_.DYRA7rxF1OM7wQlBawEA14asCKmY-UL_0 2024-06-28T16:14:33.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7174203909324393388', time=2024-06-28T17:45:00.238173+0000, chain.objs.size()=1 2024-06-28T16:14:33.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.235__shadow_.catLCNaSLPqzIItKaeEBYN-MCCWOjaC_0 2024-06-28T16:14:33.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6536359311966539208', time=2024-06-28T17:45:20.802752+0000, chain.objs.size()=1 2024-06-28T16:14:33.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.248__shadow_.juvcG96-SQvRxzOoqXTrLVAvjKRsrh0_0 2024-06-28T16:14:33.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13720487615460018222', time=2024-06-28T17:47:09.063618+0000, chain.objs.size()=1 2024-06-28T16:14:33.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.h66cOitZmP7ML9CkAEYUSUcrjSC9bDc_0 2024-06-28T16:14:33.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16837799041295719071', time=2024-06-28T17:47:10.196374+0000, chain.objs.size()=1 2024-06-28T16:14:33.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ICD-a6QxV-k8p0hHp3IWV0qJcuacUAx_0 2024-06-28T16:14:33.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15196288281210108277', time=2024-06-28T17:47:10.591737+0000, chain.objs.size()=1 2024-06-28T16:14:33.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VPdYJ_gElGUS09tjS7e4JCKGNq1P6JO_0 2024-06-28T16:14:33.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.503+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13800971897290031193', time=2024-06-28T17:47:10.914992+0000, chain.objs.size()=1 2024-06-28T16:14:33.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.503+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.D_5hSEc6hyETPYKhGQcKHFLQ_QTp2gN_0 2024-06-28T16:14:33.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.503+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4445833100279289910', time=2024-06-28T17:47:12.805675+0000, chain.objs.size()=1 2024-06-28T16:14:33.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.503+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.RvPECBDGSY3hl79j9O8WpepS43tLoNI_0 2024-06-28T16:14:33.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.503+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9935828352713665675', time=2024-06-28T17:47:14.765062+0000, chain.objs.size()=1 2024-06-28T16:14:33.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.503+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.zeygEVVpIx_tWFSvybOWgab7xMuOFqk_0 2024-06-28T16:14:33.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.503+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6820206790689530456', time=2024-06-28T17:47:15.219324+0000, chain.objs.size()=1 2024-06-28T16:14:33.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.503+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.I7rwsIAkvDVySj_LJrP0EqR19Sty_hp_0 2024-06-28T16:14:33.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8243241325885735696', time=2024-06-28T17:47:15.683649+0000, chain.objs.size()=1 2024-06-28T16:14:33.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Vp0bqMjRqil5a9hqRpoYbcugFo719e5_0 2024-06-28T16:14:33.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14266419930016531384', time=2024-06-28T17:47:15.698975+0000, chain.objs.size()=1 2024-06-28T16:14:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.yLssMUFj8tRmBdgDFsti9wKgYQegJ1J_0 2024-06-28T16:14:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3397846937017354345', time=2024-06-28T17:47:17.572114+0000, chain.objs.size()=1 2024-06-28T16:14:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.3eC8hxjGAvKPTzY0wWR2TdPMvZpLOtk_0 2024-06-28T16:14:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12633195504441851498', time=2024-06-28T17:47:19.160631+0000, chain.objs.size()=1 2024-06-28T16:14:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.WNjuKASGh00Bv3p5VdZxzx4B4RzsdrX_0 2024-06-28T16:14:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12509079592610383267', time=2024-06-28T17:47:20.837965+0000, chain.objs.size()=1 2024-06-28T16:14:33.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.XvE1RFCSCJVFuiAVEZ8XXBuXWL07zXu_0 2024-06-28T16:14:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11745633713192631698', time=2024-06-28T17:47:21.147874+0000, chain.objs.size()=1 2024-06-28T16:14:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.p_0yEDhRnUqJ9W3taWa4WrwPG0RxTFR_0 2024-06-28T16:14:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10327759084710174716', time=2024-06-28T17:47:21.303081+0000, chain.objs.size()=1 2024-06-28T16:14:33.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ATZcS01oQmB9zAEouSroZNfH3qloXW4_0 2024-06-28T16:14:33.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16682519457593579250', time=2024-06-28T17:47:21.433601+0000, chain.objs.size()=1 2024-06-28T16:14:33.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.qY5Oo3kHSM0BrgUbkJQIIEJzcJUtsCt_0 2024-06-28T16:14:33.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12909387683072057687', time=2024-06-28T17:47:21.956152+0000, chain.objs.size()=1 2024-06-28T16:14:33.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.OsktHhyCEvrXHhOxQLuC7z4sGSsNiT3_0 2024-06-28T16:14:33.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13792227369934897333', time=2024-06-28T17:47:23.716436+0000, chain.objs.size()=1 2024-06-28T16:14:33.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.BcHAHZHgoFviC49ptDNYB7TuLAP-du8_0 2024-06-28T16:14:33.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2439564530583269256', time=2024-06-28T17:47:25.037082+0000, chain.objs.size()=1 2024-06-28T16:14:33.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.aRG7qcQ2z-eV3Zjjo5u9wkQii51DlJu_0 2024-06-28T16:14:33.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14261379069990378375', time=2024-06-28T17:47:25.433504+0000, chain.objs.size()=1 2024-06-28T16:14:33.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.iG5rYguwLExoWD6g6wJ6fjy888uyPst_0 2024-06-28T16:14:33.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4974994881564652775', time=2024-06-28T17:47:26.425460+0000, chain.objs.size()=1 2024-06-28T16:14:33.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.N83T4_jBLVp7Rs9urmikWCDHWccQyr9_0 2024-06-28T16:14:33.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9587305021606295546', time=2024-06-28T17:47:27.251533+0000, chain.objs.size()=1 2024-06-28T16:14:33.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.CiSfULnyXikAWW-8EW5FBVsTmCUeGZb_0 2024-06-28T16:14:33.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4637608056159621833', time=2024-06-28T17:47:28.005980+0000, chain.objs.size()=1 2024-06-28T16:14:33.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.B7-_RH7Mv34offWNxyAJLcuaO2_Dfba_0 2024-06-28T16:14:33.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11801674172047397726', time=2024-06-28T17:47:29.311726+0000, chain.objs.size()=1 2024-06-28T16:14:33.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.rVdOyzaWWpbVkQITCdzmraYjaXlxkLv_0 2024-06-28T16:14:33.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16075063804137906062', time=2024-06-28T17:47:29.739272+0000, chain.objs.size()=1 2024-06-28T16:14:33.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.51ZaS4YZaz2XbB6E6p87w3ytKV_2gl0_0 2024-06-28T16:14:33.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9468710408602342290', time=2024-06-28T17:47:30.597161+0000, chain.objs.size()=1 2024-06-28T16:14:33.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.s8OFYUQ9v2T0tGN3ZryvDc4ZBgEuT7J_0 2024-06-28T16:14:33.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4034155246911055543', time=2024-06-28T17:47:31.470156+0000, chain.objs.size()=1 2024-06-28T16:14:33.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.yzL8w-MgbAr7hA0AaKdQEBWq417c7hz_0 2024-06-28T16:14:33.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17550104044272539304', time=2024-06-28T17:47:32.975656+0000, chain.objs.size()=1 2024-06-28T16:14:33.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.H_LpsDCBObmwIFbbqPj5e3oUxt8pv7p_0 2024-06-28T16:14:33.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16528401469771537071', time=2024-06-28T17:47:32.977662+0000, chain.objs.size()=1 2024-06-28T16:14:33.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.1G0RX7C5OCe9bLrHFL8W-o7_eFqXOXK_0 2024-06-28T16:14:33.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17795465789414884984', time=2024-06-28T17:47:33.644837+0000, chain.objs.size()=1 2024-06-28T16:14:33.528 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.R5k-JJ9iuDokuWXeLJcMHMp5HcwMthk_0 2024-06-28T16:14:33.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13693644976496106794', time=2024-06-28T17:47:33.801862+0000, chain.objs.size()=1 2024-06-28T16:14:33.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.2QAczIrsGf2OeIb6o-IkKWeEMZyAX9o_0 2024-06-28T16:14:33.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2456322460174260896', time=2024-06-28T17:47:33.876126+0000, chain.objs.size()=1 2024-06-28T16:14:33.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.sLjYll74a7wEKU6o72H2oIIJXkP_LJi_0 2024-06-28T16:14:33.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7050973098394681337', time=2024-06-28T17:47:35.409662+0000, chain.objs.size()=1 2024-06-28T16:14:33.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.p6n-OIGZsa12LAeB3nkERoe0-b58nLe_0 2024-06-28T16:14:33.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7253487199378498432', time=2024-06-28T17:47:36.057930+0000, chain.objs.size()=1 2024-06-28T16:14:33.529 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.T8cXdywoNzHD_SYR4Glx9ISgm8PWqkE_0 2024-06-28T16:14:33.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1428576087100509264', time=2024-06-28T17:47:38.083852+0000, chain.objs.size()=1 2024-06-28T16:14:33.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.awB8fjDkMHR9eqUM-D5h_QXMcAcNiPy_0 2024-06-28T16:14:33.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.742892790388479466', time=2024-06-28T17:47:47.020048+0000, chain.objs.size()=1 2024-06-28T16:14:33.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.257__shadow_._SUse16OvT3sv9XbT91ngPX_qbgz9ZI_0 2024-06-28T16:14:33.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12504410024675085305', time=2024-06-28T17:47:48.496470+0000, chain.objs.size()=1 2024-06-28T16:14:33.530 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.519+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.258__shadow_.CgJdMTpfU4WovyiN_TMRTRLLgAsjU98_0 2024-06-28T16:14:33.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9753135983709812485', time=2024-06-28T17:47:52.884736+0000, chain.objs.size()=1 2024-06-28T16:14:33.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.261__shadow_.m6OWPf25vmdpuaT5rdMUBOJ2XSqWOOv_0 2024-06-28T16:14:33.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7383050510251909808', time=2024-06-28T17:47:54.439171+0000, chain.objs.size()=1 2024-06-28T16:14:33.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.262__shadow_.4MkW-NGvMzi-xvM5adMpeK19JHk2TSe_0 2024-06-28T16:14:33.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9713263147373983392', time=2024-06-28T17:48:14.959900+0000, chain.objs.size()=1 2024-06-28T16:14:33.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.275__shadow_.m1fHwzEbUwueF03K4AH5kCvvo4Dr7e__0 2024-06-28T16:14:33.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14371726493121641200', time=2024-06-28T17:48:46.319814+0000, chain.objs.size()=1 2024-06-28T16:14:33.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.290__shadow_.HydmKyyTrxc-KhxaTkuS5pn1fH80Jo5_0 2024-06-28T16:14:33.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14657727816014121864', time=2024-06-28T17:51:06.762993+0000, chain.objs.size()=1 2024-06-28T16:14:33.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ooqRLVyXL4kOigzawwfrkicU4nzorpo_0 2024-06-28T16:14:33.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18191526817622902827', time=2024-06-28T17:51:08.156004+0000, chain.objs.size()=1 2024-06-28T16:14:33.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.140FW9-tf-mjJ_WYs7iXgkSslqt6ls9_0 2024-06-28T16:14:33.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9631845074364426566', time=2024-06-28T17:51:08.158723+0000, chain.objs.size()=1 2024-06-28T16:14:33.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.DnYDorZpkSVoGNhLFFrDa2T6GcdUlej_0 2024-06-28T16:14:33.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7225100999366399711', time=2024-06-28T17:51:11.200461+0000, chain.objs.size()=1 2024-06-28T16:14:33.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.NlfHCdKs6WxnE_JH5PRJE80iqrVYFBc_0 2024-06-28T16:14:33.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10678574410838676204', time=2024-06-28T17:51:12.756081+0000, chain.objs.size()=1 2024-06-28T16:14:33.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ckLXoh11LZq3nAfCimEZ4-Wq6gk82kH_0 2024-06-28T16:14:33.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12742008559974366323', time=2024-06-28T17:51:15.406200+0000, chain.objs.size()=1 2024-06-28T16:14:33.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.u-kGUAySkNqvcC4xaBsXaQA8smVtxW-_0 2024-06-28T16:14:33.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15823819307924266006', time=2024-06-28T17:51:15.838680+0000, chain.objs.size()=1 2024-06-28T16:14:33.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.mntktCMP3K4qQcQ8qi9fMz3Y4VYN3IB_0 2024-06-28T16:14:33.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2077875496972880809', time=2024-06-28T17:51:19.960545+0000, chain.objs.size()=1 2024-06-28T16:14:33.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.bGYu_XBEm8MeVLeWVu46-twVIlJPGgG_0 2024-06-28T16:14:33.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10047011986751371682', time=2024-06-28T17:51:20.031170+0000, chain.objs.size()=1 2024-06-28T16:14:33.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ymmRj12_O0I5tUJ1roif74oeFQJCr_v_0 2024-06-28T16:14:33.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10741717450414618251', time=2024-06-28T17:51:21.585652+0000, chain.objs.size()=1 2024-06-28T16:14:33.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PfV0y5DaYCN4HII_ouiFmG2ilfatA1W_0 2024-06-28T16:14:33.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4573568834579569319', time=2024-06-28T17:51:22.283525+0000, chain.objs.size()=1 2024-06-28T16:14:33.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.rowd9Id1X-6R1A-x-hj51fhxuswd41M_0 2024-06-28T16:14:33.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18389465975983899305', time=2024-06-28T17:51:22.632973+0000, chain.objs.size()=1 2024-06-28T16:14:33.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.VbVp2XE5uAheDZ18rZbmfZB7WNoOmUE_0 2024-06-28T16:14:33.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12190283865460655544', time=2024-06-28T17:51:23.825063+0000, chain.objs.size()=1 2024-06-28T16:14:33.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.qaFaNiNaOXI7I7DGC6VQrGc2o6S4ZRX_0 2024-06-28T16:14:33.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6958764705345827542', time=2024-06-28T17:51:25.222020+0000, chain.objs.size()=1 2024-06-28T16:14:33.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xn3atiBP70BJWDGNGqg6JNKJk0_PmRk_0 2024-06-28T16:14:33.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10157301262012868626', time=2024-06-28T17:51:25.913373+0000, chain.objs.size()=1 2024-06-28T16:14:33.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.pLocUjUrb6gyeviXJhcP0pPhvlYoJfB_0 2024-06-28T16:14:33.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11767638308515689602', time=2024-06-28T17:51:32.299218+0000, chain.objs.size()=1 2024-06-28T16:14:33.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.VWV2xYoLnR2g4WwCF1Ebvbzu4VumUBC_0 2024-06-28T16:14:33.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9170583698998840968', time=2024-06-28T17:51:33.670407+0000, chain.objs.size()=1 2024-06-28T16:14:33.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.KgRuluLYiT1hvhBzr_vVVwHuljOLg7E_0 2024-06-28T16:14:33.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10461400305669027606', time=2024-06-28T17:51:33.705468+0000, chain.objs.size()=1 2024-06-28T16:14:33.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.U_Jt34n6nuwUtZVpurmhlb83-0rgGIK_0 2024-06-28T16:14:33.542 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15848396162110091746', time=2024-06-28T17:51:34.939196+0000, chain.objs.size()=1 2024-06-28T16:14:33.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.rWc3OwyclFvFRRDZ7KLOAky_G_x5hPG_0 2024-06-28T16:14:33.543 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7218959326312987578', time=2024-06-28T17:51:35.060844+0000, chain.objs.size()=1 2024-06-28T16:14:33.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.941Em7dVJnis4NnBfFV1fd2nkzKL6jk_0 2024-06-28T16:14:33.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16542757871080808893', time=2024-06-28T17:51:35.854648+0000, chain.objs.size()=1 2024-06-28T16:14:33.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.8oTvDRGbazXuL3DzdFQoM80XJnlhiv-_0 2024-06-28T16:14:33.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15697209241821522355', time=2024-06-28T17:51:35.957731+0000, chain.objs.size()=1 2024-06-28T16:14:33.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.VS3WDzxi1u-amOwNOM5oXeBEC8PWQR2_0 2024-06-28T16:14:33.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8300575675238522784', time=2024-06-28T17:51:36.772794+0000, chain.objs.size()=1 2024-06-28T16:14:33.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.zpwZAkPmT88UZvWaJRj0pChDFUBm5t7_0 2024-06-28T16:14:33.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2714584161081067503', time=2024-06-28T17:51:37.037160+0000, chain.objs.size()=1 2024-06-28T16:14:33.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Gl9oYmQP0OkzpDZQYLkYjuSdsEf7lFl_0 2024-06-28T16:14:33.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1617120588114181150', time=2024-06-28T17:53:21.440733+0000, chain.objs.size()=1 2024-06-28T16:14:33.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.NCsrnHp0Bl61oE6W-DqwzLtbn4MfENn_0 2024-06-28T16:14:33.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12529493126322717165', time=2024-06-28T17:53:21.442528+0000, chain.objs.size()=1 2024-06-28T16:14:33.546 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.R7a-GHJ1g75KC_j_gSG1_gSos372NTN_0 2024-06-28T16:14:33.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5619513255343196134', time=2024-06-28T17:53:23.903538+0000, chain.objs.size()=1 2024-06-28T16:14:33.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.kvAAw13GDavnkPf4bbBCeXoEiSStCgF_0 2024-06-28T16:14:33.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1166474713366032410', time=2024-06-28T17:53:24.082470+0000, chain.objs.size()=1 2024-06-28T16:14:33.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.EYu0QqgKza_uqTigGiCUVhf9tfAaLvy_0 2024-06-28T16:14:33.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8635043712517671192', time=2024-06-28T17:53:24.127472+0000, chain.objs.size()=1 2024-06-28T16:14:33.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.zDnDTClwTjV3WPcqu4pq83nyDHp2V_H_0 2024-06-28T16:14:33.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15618979822582291114', time=2024-06-28T17:53:25.283445+0000, chain.objs.size()=1 2024-06-28T16:14:33.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.SolVy_zeElCdrbhYwnI-Fl4d6sChLRe_0 2024-06-28T16:14:33.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.539+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6789605854132614310', time=2024-06-28T17:53:27.299660+0000, chain.objs.size()=1 2024-06-28T16:14:33.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.539+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.D55_UTP4rsXpwqfw_6xpJ2vYWy5ME-8_0 2024-06-28T16:14:33.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11158714563620248900', time=2024-06-28T17:53:28.430009+0000, chain.objs.size()=1 2024-06-28T16:14:33.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.FlPt_AadU6BdNvZ4W1hKM9RrkU7VGH6_0 2024-06-28T16:14:33.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16362686597247508406', time=2024-06-28T17:53:29.705983+0000, chain.objs.size()=1 2024-06-28T16:14:33.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.N5m53uiqffDYX5-qp1mTIRMH2XZ2Lvh_0 2024-06-28T16:14:33.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5474509036104210247', time=2024-06-28T17:53:30.066776+0000, chain.objs.size()=1 2024-06-28T16:14:33.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.xgY6DGeGCXLobhhYSotlOmcKsZEWrtq_0 2024-06-28T16:14:33.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3197248818757817868', time=2024-06-28T17:53:31.155985+0000, chain.objs.size()=1 2024-06-28T16:14:33.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.EszCCOO9kpVYpvlDOwfPW-xFtxH-izj_0 2024-06-28T16:14:33.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5446220304695171837', time=2024-06-28T17:53:32.443725+0000, chain.objs.size()=1 2024-06-28T16:14:33.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.lNbMBlj4vnzmesDgmpTsTVDNbeSC4id_0 2024-06-28T16:14:33.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17357429670104947857', time=2024-06-28T17:53:35.003542+0000, chain.objs.size()=1 2024-06-28T16:14:33.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.IZnaDxTTc7d8scWB4SN7CSbE2SVbix4_0 2024-06-28T16:14:33.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11749894137194090035', time=2024-06-28T17:53:35.240329+0000, chain.objs.size()=1 2024-06-28T16:14:33.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1EHts4VgpnhmIPG8p8cmOxCr-tRB8WJ_0 2024-06-28T16:14:33.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16235820304275698707', time=2024-06-28T17:53:35.704862+0000, chain.objs.size()=1 2024-06-28T16:14:33.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.jeDhSYDcDzh754vsyr3LFK0varLlVy0_0 2024-06-28T16:14:33.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9190233742612588429', time=2024-06-28T17:53:36.115985+0000, chain.objs.size()=1 2024-06-28T16:14:33.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wUDLRs0EpMarp_6Lp2GbhkMcX0TQqUW_0 2024-06-28T16:14:33.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1220843733963119548', time=2024-06-28T17:53:36.219594+0000, chain.objs.size()=1 2024-06-28T16:14:33.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bNlEKwF3PBqKoTLwFBlbzneiR-J81z9_0 2024-06-28T16:14:33.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6241159589645666781', time=2024-06-28T17:53:38.369972+0000, chain.objs.size()=1 2024-06-28T16:14:33.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.50izAMn-07My4JvCXtrG8pj-NJgoaZH_0 2024-06-28T16:14:33.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3708342658668233749', time=2024-06-28T17:53:39.484882+0000, chain.objs.size()=1 2024-06-28T16:14:33.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1u6-9B5WfVwrG-kpi-ku3KKCksOsqwq_0 2024-06-28T16:14:33.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3851557760919717252', time=2024-06-28T17:53:41.868331+0000, chain.objs.size()=1 2024-06-28T16:14:33.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ShZWshXUOmVC09kH1f_Md-m6zRQbDAZ_0 2024-06-28T16:14:33.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4235519260469702623', time=2024-06-28T17:53:42.421782+0000, chain.objs.size()=1 2024-06-28T16:14:33.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.XEd8w8bGuZubSXRJiOnhvgYlegYnDfW_0 2024-06-28T16:14:33.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15377042606217527180', time=2024-06-28T17:53:43.946792+0000, chain.objs.size()=1 2024-06-28T16:14:33.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.PCWN-3u_XnY6kWGr0mghujanSTiXseF_0 2024-06-28T16:14:33.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2013525706024815889', time=2024-06-28T17:53:44.902281+0000, chain.objs.size()=1 2024-06-28T16:14:33.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.9hurtKJ8HJTGQ62G2eojvBmsA7EpDQ6_0 2024-06-28T16:14:33.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1938973241356430632', time=2024-06-28T17:53:45.002735+0000, chain.objs.size()=1 2024-06-28T16:14:33.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.yO4AdRKSuVndJddayw8dX1678kVbZs7_0 2024-06-28T16:14:33.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15859340732811943605', time=2024-06-28T17:53:45.099552+0000, chain.objs.size()=1 2024-06-28T16:14:33.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.6cFivmCV7WgFBSKcENm9LpdXWbeks8E_0 2024-06-28T16:14:33.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2197009592155667619', time=2024-06-28T17:53:46.187269+0000, chain.objs.size()=1 2024-06-28T16:14:33.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.va9gom6DzpLl-10fZ6EY0lOWxKNP9V5_0 2024-06-28T16:14:33.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15648172587257079352', time=2024-06-28T17:53:47.334262+0000, chain.objs.size()=1 2024-06-28T16:14:33.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.a6-PwFENXB6zkDo6S5VSn-tPPAjZl3P_0 2024-06-28T16:14:33.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13259760022807080723', time=2024-06-28T17:53:47.728565+0000, chain.objs.size()=1 2024-06-28T16:14:33.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GGppxKp122HLqmILf7u-Mpl62Hs5yWV_0 2024-06-28T16:14:33.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.551+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.802077069646613798', time=2024-06-28T17:53:47.929246+0000, chain.objs.size()=1 2024-06-28T16:14:33.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.551+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vG8mpAm6j52qFwsmdEMnW5gPos3XAVQ_0 2024-06-28T16:14:33.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/45192 2024-06-28T16:14:33.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=19, truncated=0, next_marker='' 2024-06-28T16:14:33.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.289891872153399740', time=2024-06-28T17:53:50.293297+0000, chain.objs.size()=1 2024-06-28T16:14:33.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.PB57vHEeZu0NDXdUIyXw-IlHbAK3FdP_0 2024-06-28T16:14:33.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14979754494103080654', time=2024-06-28T17:53:50.682001+0000, chain.objs.size()=1 2024-06-28T16:14:33.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZYD6oeUTlXBz6LpXwuYmNLILJeM45dL_0 2024-06-28T16:14:33.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5441074943001974942', time=2024-06-28T17:53:52.234285+0000, chain.objs.size()=1 2024-06-28T16:14:33.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.SVxV9YwClV9aw_IopWxrf0NSlio6aN9_0 2024-06-28T16:14:33.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1969142579714787391', time=2024-06-28T17:54:04.864399+0000, chain.objs.size()=1 2024-06-28T16:14:33.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.327__shadow_.QzxR1yaIdQS11t34irhf5Ttb4PBWlc9_0 2024-06-28T16:14:33.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10251568536761163371', time=2024-06-28T17:56:35.616240+0000, chain.objs.size()=1 2024-06-28T16:14:33.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.476__shadow_.FIZWtkSAP1ynDLVqO7xV4i1CqgW6oMx_0 2024-06-28T16:14:33.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.493__shadow:rinwpm3WJlZ1AcGvTYYbpbEzG8SjNkf_.0OEFPT4L0L8Hzv3rUAZWjoQj5ExOwB1_1_eb17dc5adaaa1e64a0b4cbfe4f991088', time=2024-06-28T17:56:54.973722+0000, chain.objs.size()=1 2024-06-28T16:14:33.568 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.493__shadow:rinwpm3WJlZ1AcGvTYYbpbEzG8SjNkf_.0OEFPT4L0L8Hzv3rUAZWjoQj5ExOwB1_0 2024-06-28T16:14:33.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~l68ArzwDshy_3A9iAKwiZ0J5WYzyAAw', time=2024-06-28T18:00:16.415253+0000, chain.objs.size()=2 2024-06-28T16:14:33.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.519__multipart_mymultipart.2~l68ArzwDshy_3A9iAKwiZ0J5WYzyAAw.1 2024-06-28T16:14:33.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.519__multipart_mymultipart.2~l68ArzwDshy_3A9iAKwiZ0J5WYzyAAw.2 2024-06-28T16:14:33.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8574027084940415133', time=2024-06-28T18:02:01.466954+0000, chain.objs.size()=1 2024-06-28T16:14:33.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:WAqqqTJ4AIYqIxfUrYHiXJe3d9BUrDc_.O-3e-1GbVhZiiSKNAYiqgObx-WBQqWd_0 2024-06-28T16:14:33.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15116845309464819512', time=2024-06-28T18:02:09.932403+0000, chain.objs.size()=1 2024-06-28T16:14:33.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:1XaEPhJ8F69JILxmaGJYKXDg2GwmmuU_.M6nSUG_hJhUrw08-5NmhFqH-PRdaviU_0 2024-06-28T16:14:33.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5170129702836409354', time=2024-06-28T18:02:17.478448+0000, chain.objs.size()=1 2024-06-28T16:14:33.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:zpuLWdempClwO1I2llgPJuXA5lEsQC-_.DmPE0rPOEAOZqQVNqD5jhteS9bXvdim_0 2024-06-28T16:14:33.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17525146045263796959', time=2024-06-28T18:02:38.932221+0000, chain.objs.size()=1 2024-06-28T16:14:33.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.564__shadow_.px_UYFSpyiaCBXG7zMb6eNn1PETwRxE_0 2024-06-28T16:14:33.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17380501450745688164', time=2024-06-28T18:02:59.949067+0000, chain.objs.size()=1 2024-06-28T16:14:33.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:soQi7bL.-npJEpiAqOJCuuu4iUmPrw6_.dXFDsgT-5fROu9Ck0BV1v3Dcxa5K3m6_0 2024-06-28T16:14:33.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15395413632921391059', time=2024-06-28T18:03:10.297720+0000, chain.objs.size()=1 2024-06-28T16:14:33.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:vF4V-ueGJqLQd3VXjBCgBVDF4v4yI.q_.VrgNYQcUGRSnCkJWqEzQtovBfJj11l7_0 2024-06-28T16:14:33.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12493289683208655849', time=2024-06-28T18:03:32.741017+0000, chain.objs.size()=1 2024-06-28T16:14:33.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:o9ElO-bTGVKED0M3Moje1HF5xsURawT_.tHuGir5eJ9CMOPUMT15iuvmdO_vWfA4_0 2024-06-28T16:14:33.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14403291891459570947', time=2024-06-28T18:03:56.388893+0000, chain.objs.size()=1 2024-06-28T16:14:33.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.568__shadow:uPMDNtLCmLbHR2ukR44iPv6pwITQjKQ_.GhKffqpyCiNfogYYN8kFD16u2VNuuol_0 2024-06-28T16:14:33.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.651__shadow_.27YBSpQgHxw5StUYgl3qY6yteCUAuXV_1_a4eae68a69c70901cd7be27a6d3f4fe5', time=2024-06-28T18:08:05.563489+0000, chain.objs.size()=1 2024-06-28T16:14:33.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.651__shadow_.27YBSpQgHxw5StUYgl3qY6yteCUAuXV_0 2024-06-28T16:14:33.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3271006615916006422', time=2024-06-28T18:08:30.021819+0000, chain.objs.size()=1 2024-06-28T16:14:33.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.679__shadow:BjyNgMlbqckf14NLfvrHgJie5i1PklJ_.9PdgtHlmbIdDDjtR_qmLBcFKxetFx3v_0 2024-06-28T16:14:33.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9997748463796572400', time=2024-06-28T18:08:48.789406+0000, chain.objs.size()=1 2024-06-28T16:14:33.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.694__shadow_.6CaDosYTOLzQIgvXecOKuVRieyifAne_0 2024-06-28T16:14:33.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9789478889798326139', time=2024-06-28T18:09:45.082664+0000, chain.objs.size()=1 2024-06-28T16:14:33.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.722__shadow_.mIzuMvJY0L6Bra2ts1TRJe7ILxTQ3J6_0 2024-06-28T16:14:33.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:33.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-06-28T16:14:33.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:33.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:33.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/42630' 2024-06-28T16:14:33.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6673510824281081658', time=2024-06-28T17:40:14.794802+0000, chain.objs.size()=1 2024-06-28T16:14:33.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.GMOk-y9vq9Jvc6HfgBpyLkHOuolTNum_0 2024-06-28T16:14:33.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12189917351558325228', time=2024-06-28T17:41:08.152849+0000, chain.objs.size()=1 2024-06-28T16:14:33.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.168__shadow_.jY9abVCVTNI57U2rXlIPLpWTlIjyLFR_0 2024-06-28T16:14:33.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5285469203373518034', time=2024-06-28T17:44:33.980396+0000, chain.objs.size()=1 2024-06-28T16:14:33.584 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.220__shadow_.CldoTkAUmp5o2emlc5iYRlrgOf9BnWy_0 2024-06-28T16:14:33.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7399579086794504242', time=2024-06-28T17:44:52.510672+0000, chain.objs.size()=1 2024-06-28T16:14:33.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.231__shadow_.kZ7RzTKOfgwY1CJBlAF6mgLXytGYMkI_0 2024-06-28T16:14:33.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5402590843669174107', time=2024-06-28T17:45:04.967976+0000, chain.objs.size()=1 2024-06-28T16:14:33.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.238__shadow_.TMpK8MxwL64Cf4HOphecgyb_PKgoaOc_0 2024-06-28T16:14:33.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7507879802406394174', time=2024-06-28T17:47:08.053857+0000, chain.objs.size()=1 2024-06-28T16:14:33.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ps0pOuAzBsam_Mc9qBej2c97PD4U1d6_0 2024-06-28T16:14:33.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14687577979970117433', time=2024-06-28T17:47:08.788036+0000, chain.objs.size()=1 2024-06-28T16:14:33.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.fFr3i5A0ldA8iqPBL9Q8uLHe8DXddRt_0 2024-06-28T16:14:33.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13724498841361409986', time=2024-06-28T17:47:09.051296+0000, chain.objs.size()=1 2024-06-28T16:14:33.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VCAd9QkPp8jNu5-pzFM85asAVm2eMBr_0 2024-06-28T16:14:33.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18433734878892316380', time=2024-06-28T17:47:10.331996+0000, chain.objs.size()=1 2024-06-28T16:14:33.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.L5WqL_eYOxOwAhzz6yMVBeyT7SByfLZ_0 2024-06-28T16:14:33.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16969214865517505844', time=2024-06-28T17:47:13.080142+0000, chain.objs.size()=1 2024-06-28T16:14:33.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.m_nkaXnbNDoiQaV16w10aLvzm33WzPf_0 2024-06-28T16:14:33.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14665749778186785302', time=2024-06-28T17:47:14.545169+0000, chain.objs.size()=1 2024-06-28T16:14:33.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.hw1ZS6dRaDWlJeQcio0GrON-9s_KEaj_0 2024-06-28T16:14:33.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5981092371290398229', time=2024-06-28T17:47:15.363065+0000, chain.objs.size()=1 2024-06-28T16:14:33.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.cbXQ4aM6Bq1vDPh9M0DeDpZoHIQK0BN_0 2024-06-28T16:14:33.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4658041744163707136', time=2024-06-28T17:47:18.295012+0000, chain.objs.size()=1 2024-06-28T16:14:33.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.UAvO1CXexTrJJQGV0SWwTY9SZtpL_Hb_0 2024-06-28T16:14:33.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3730514763321643365', time=2024-06-28T17:47:18.455791+0000, chain.objs.size()=1 2024-06-28T16:14:33.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.NgpgMynzvTk7s94LIwr4L-dPceGBAYY_0 2024-06-28T16:14:33.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17944727101882143411', time=2024-06-28T17:47:19.117140+0000, chain.objs.size()=1 2024-06-28T16:14:33.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.DZ4WrBKqi0kuDRZig58AGl7C-znXTkU_0 2024-06-28T16:14:33.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16689074893854305411', time=2024-06-28T17:47:19.634673+0000, chain.objs.size()=1 2024-06-28T16:14:33.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.t3xQ8hgTSyN9NyT7_aeAlep9sXXUVRH_0 2024-06-28T16:14:33.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5264699468056012320', time=2024-06-28T17:47:22.263511+0000, chain.objs.size()=1 2024-06-28T16:14:33.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7lNLFMVcXHWYnB17harUR8OFAVya3gq_0 2024-06-28T16:14:33.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2294915074124134869', time=2024-06-28T17:47:22.265638+0000, chain.objs.size()=1 2024-06-28T16:14:33.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VMSUs6VDKxnnT0n-G80-wt0zLLxErF3_0 2024-06-28T16:14:33.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17389345873379394225', time=2024-06-28T17:47:23.270597+0000, chain.objs.size()=1 2024-06-28T16:14:33.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.AaDHr7R4-l32nNzqyEXnR0ab0R-EuMz_0 2024-06-28T16:14:33.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14922565887016053670', time=2024-06-28T17:47:25.657827+0000, chain.objs.size()=1 2024-06-28T16:14:33.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.S2OY4zCn7hcZBixa4FDTk5-TU6xcSpf_0 2024-06-28T16:14:33.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4469684515927039114', time=2024-06-28T17:47:25.921809+0000, chain.objs.size()=1 2024-06-28T16:14:33.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.2-1MX24ZODCBpnCrhauQkGBwzpRHnkp_0 2024-06-28T16:14:33.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13175638197334518418', time=2024-06-28T17:47:26.151535+0000, chain.objs.size()=1 2024-06-28T16:14:33.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.59vacLeys488dM8SkdInFnwy9pt6bHH_0 2024-06-28T16:14:33.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16354462959012286026', time=2024-06-28T17:47:26.154360+0000, chain.objs.size()=1 2024-06-28T16:14:33.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_._gl8KXhnv6mS-c6yAkVZ3T2v5wG6EM8_0 2024-06-28T16:14:33.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2564013828975096631', time=2024-06-28T17:47:28.956102+0000, chain.objs.size()=1 2024-06-28T16:14:33.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.p_cQ7isxDSma5rStM1Nu-tFlsm27EwV_0 2024-06-28T16:14:33.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15309898446877325187', time=2024-06-28T17:47:29.643684+0000, chain.objs.size()=1 2024-06-28T16:14:33.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.6tM3IitLMf9int1jtJ5niyq2TGEIhjK_0 2024-06-28T16:14:33.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4749384067459819237', time=2024-06-28T17:47:30.301152+0000, chain.objs.size()=1 2024-06-28T16:14:33.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.yzdr8eZWftEoQx7gY8QPJ00g0ZdL4G3_0 2024-06-28T16:14:33.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6804502794190925756', time=2024-06-28T17:47:31.090019+0000, chain.objs.size()=1 2024-06-28T16:14:33.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.r0DkTJEQFUtvbPv4T_oTty6UqjZKx6c_0 2024-06-28T16:14:33.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10914694130080910620', time=2024-06-28T17:47:32.181746+0000, chain.objs.size()=1 2024-06-28T16:14:33.594 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.bWiUizlMGL8srXVeUIgsveR9uVjHmBB_0 2024-06-28T16:14:33.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11322822740730255719', time=2024-06-28T17:47:33.232757+0000, chain.objs.size()=1 2024-06-28T16:14:33.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.p7e_ufsFqA87b1nM1OnCZ4zLuHtUV-h_0 2024-06-28T16:14:33.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14933330817157494820', time=2024-06-28T17:47:36.151254+0000, chain.objs.size()=1 2024-06-28T16:14:33.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Daudxr2nQwxS8riKVQ7Xqvedm84kyoe_0 2024-06-28T16:14:33.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16579938025318467722', time=2024-06-28T17:47:36.915047+0000, chain.objs.size()=1 2024-06-28T16:14:33.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.bNbp4VxQUXN41WXtZOcR2a0UpFfDN_Y_0 2024-06-28T16:14:33.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14846016371481287045', time=2024-06-28T17:47:37.071527+0000, chain.objs.size()=1 2024-06-28T16:14:33.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.E5ACWdu1ajF6khvcqsMZzLTAVrhZno4_0 2024-06-28T16:14:33.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15309814309957077100', time=2024-06-28T17:47:39.635882+0000, chain.objs.size()=1 2024-06-28T16:14:33.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.xruXHfl324_uxMvbQbeDVdZyd4wri4k_0 2024-06-28T16:14:33.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6280403942793888188', time=2024-06-28T17:47:41.139525+0000, chain.objs.size()=1 2024-06-28T16:14:33.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.253__shadow_.wD309LuiHR_oKTsEHtexZw4lEok_r2P_0 2024-06-28T16:14:33.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13673758937615077804', time=2024-06-28T17:47:57.333944+0000, chain.objs.size()=1 2024-06-28T16:14:33.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.264__shadow_.KVQ4_0_NH6FT8V_yKERfV5ArIIh-5lc_0 2024-06-28T16:14:33.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5828581033784868383', time=2024-06-28T17:48:03.698822+0000, chain.objs.size()=1 2024-06-28T16:14:33.597 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.268__shadow_.gAhJBbX7HniXEyncF9aZ7ybhX4m0MWV_0 2024-06-28T16:14:33.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9249310739190592241', time=2024-06-28T17:48:19.812023+0000, chain.objs.size()=1 2024-06-28T16:14:33.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.278__shadow_.XORlkYGTG2llZpnYrHgClMDyGeAsnE0_0 2024-06-28T16:14:33.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.587+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8528666399244126993', time=2024-06-28T17:48:29.579253+0000, chain.objs.size()=1 2024-06-28T16:14:33.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.99xO8VXWf8Rs8cJdMhFYPgUct5Z20Kh_0 2024-06-28T16:14:33.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18010199936782190523', time=2024-06-28T17:48:54.354689+0000, chain.objs.size()=1 2024-06-28T16:14:33.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.295__shadow_.6YV7UfzoulUmFncE0vvgwHLI5si4H2c_0 2024-06-28T16:14:33.599 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17436675254346082321', time=2024-06-28T17:51:08.206088+0000, chain.objs.size()=1 2024-06-28T16:14:33.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.2nrJt62nVez0-IOAG-pPz213a_IsKue_0 2024-06-28T16:14:33.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3912903067736339479', time=2024-06-28T17:51:08.459745+0000, chain.objs.size()=1 2024-06-28T16:14:33.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.NHIm4mrIUn0-22TmWqEe-LhbAxRn0Qs_0 2024-06-28T16:14:33.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5793195395350850780', time=2024-06-28T17:51:09.453406+0000, chain.objs.size()=1 2024-06-28T16:14:33.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.pHnXYK3k1m0ESkpclVTo7i8WFBuhFu3_0 2024-06-28T16:14:33.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10270086373038495689', time=2024-06-28T17:51:11.060203+0000, chain.objs.size()=1 2024-06-28T16:14:33.600 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Z2TvnMdNo9voe7US2BN9j0Ue7kjT5h3_0 2024-06-28T16:14:33.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13681151308489899808', time=2024-06-28T17:51:13.169961+0000, chain.objs.size()=1 2024-06-28T16:14:33.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.2Pa5jOg48FmFpBoIQ9Lx5ieWeQhZ2aM_0 2024-06-28T16:14:33.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14620765429059899500', time=2024-06-28T17:51:13.528806+0000, chain.objs.size()=1 2024-06-28T16:14:33.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.QsPU9ywYvr6N3JLOwkWGw9BDl6CyO51_0 2024-06-28T16:14:33.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10402262876280792728', time=2024-06-28T17:51:16.824251+0000, chain.objs.size()=1 2024-06-28T16:14:33.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.pANMK6dt6B33iiS60D4ykNGNoW2kR8q_0 2024-06-28T16:14:33.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.595+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14307620941695113682', time=2024-06-28T17:51:18.174564+0000, chain.objs.size()=1 2024-06-28T16:14:33.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.595+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xmCHuNqaOGE9OGJ0KXdMjKWYFzthKdP_0 2024-06-28T16:14:33.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.595+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17775499513474413749', time=2024-06-28T17:51:19.363119+0000, chain.objs.size()=1 2024-06-28T16:14:33.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.595+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xbZu2Ay5mWzN7mtkuvtK6H2gUwarZqs_0 2024-06-28T16:14:33.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.595+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3393698271086970581', time=2024-06-28T17:51:19.682362+0000, chain.objs.size()=1 2024-06-28T16:14:33.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.595+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.aq4JkARcYyGeN-lB90K0IBEIMOdGYWa_0 2024-06-28T16:14:33.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.595+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11935636251048950152', time=2024-06-28T17:51:20.161388+0000, chain.objs.size()=1 2024-06-28T16:14:33.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.595+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.woRNy7fAaAYDPn9_0jD3A71fSNcIRxe_0 2024-06-28T16:14:33.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.595+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16508779176158660082', time=2024-06-28T17:51:20.731778+0000, chain.objs.size()=1 2024-06-28T16:14:33.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.595+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nIZcb0u_GEJAWg_5bMQk9Yk-mAO1xr6_0 2024-06-28T16:14:33.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.595+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.679439297409532668', time=2024-06-28T17:51:20.791104+0000, chain.objs.size()=1 2024-06-28T16:14:33.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.595+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.SHv54PJ6uV7SWZW5OXNoe38hRAJRmr__0 2024-06-28T16:14:33.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9457859587032766146', time=2024-06-28T17:51:22.063211+0000, chain.objs.size()=1 2024-06-28T16:14:33.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fRcnbEcDFwhmYxBfZeC3mUxEUas64m__0 2024-06-28T16:14:33.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16269026407992324461', time=2024-06-28T17:51:22.649336+0000, chain.objs.size()=1 2024-06-28T16:14:33.607 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Bo200kr7TTf4tdAI6crluc3N_yDFM6X_0 2024-06-28T16:14:33.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1589392399420918865', time=2024-06-28T17:51:24.637485+0000, chain.objs.size()=1 2024-06-28T16:14:33.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fp4xlsnzJHsQ8iCSoj8kdzaDzXBfS5t_0 2024-06-28T16:14:33.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5282629033091736183', time=2024-06-28T17:51:24.828901+0000, chain.objs.size()=1 2024-06-28T16:14:33.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.gs70L9ckY1DzILpC8Gec0IQuTTbjvAK_0 2024-06-28T16:14:33.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16641322678638389815', time=2024-06-28T17:51:25.770922+0000, chain.objs.size()=1 2024-06-28T16:14:33.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.pE72CUSOSrOyTFvIi7giH3sl3sFqIBA_0 2024-06-28T16:14:33.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2732098887011733579', time=2024-06-28T17:51:26.857355+0000, chain.objs.size()=1 2024-06-28T16:14:33.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fFixhOH3UpDn3Ldx-V4CMd-w1Nu4aON_0 2024-06-28T16:14:33.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11455688729058028362', time=2024-06-28T17:51:28.218617+0000, chain.objs.size()=1 2024-06-28T16:14:33.609 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.w_70p-cuvIIkm5_QfcjvU9t2ZkhER7X_0 2024-06-28T16:14:33.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4575340086510803196', time=2024-06-28T17:51:29.122142+0000, chain.objs.size()=1 2024-06-28T16:14:33.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.3b16VS8Y5CSK_k_uwqwjMEalZfkpOAS_0 2024-06-28T16:14:33.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14028310890909529594', time=2024-06-28T17:51:29.444594+0000, chain.objs.size()=1 2024-06-28T16:14:33.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.v1Ccq5O0g81ydRW8AlEm59M51b83bi0_0 2024-06-28T16:14:33.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5343654686613396201', time=2024-06-28T17:51:29.821041+0000, chain.objs.size()=1 2024-06-28T16:14:33.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.qsXo506YDpxqT2F8RCqvmCyvPRAu58K_0 2024-06-28T16:14:33.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15285400794655723346', time=2024-06-28T17:51:31.040235+0000, chain.objs.size()=1 2024-06-28T16:14:33.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.a8rzIMuxgB86kfCE_APErDknjf_yT83_0 2024-06-28T16:14:33.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8952035917246857807', time=2024-06-28T17:51:32.426581+0000, chain.objs.size()=1 2024-06-28T16:14:33.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YeSr_xIVkwFY5mW9n2dW2g-b58rRiSV_0 2024-06-28T16:14:33.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1194212078328730364', time=2024-06-28T17:51:32.752837+0000, chain.objs.size()=1 2024-06-28T16:14:33.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.DTn93_kF7Ezx0-lHJPe6AEJMfCinvPD_0 2024-06-28T16:14:33.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15584980389248267682', time=2024-06-28T17:51:33.240829+0000, chain.objs.size()=1 2024-06-28T16:14:33.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1n32QeT3DWF3zhx9mh99nJ_X1dVOrwo_0 2024-06-28T16:14:33.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14821431605903792453', time=2024-06-28T17:53:20.983210+0000, chain.objs.size()=1 2024-06-28T16:14:33.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.xEY9sgFzUEfu2uLCWzJFPtWVGq9wZOH_0 2024-06-28T16:14:33.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9739963149357789754', time=2024-06-28T17:53:21.488416+0000, chain.objs.size()=1 2024-06-28T16:14:33.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.2si5uBQH9kpyO72_A2KdjIeOKE5E8VX_0 2024-06-28T16:14:33.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18231602670061573534', time=2024-06-28T17:53:21.762411+0000, chain.objs.size()=1 2024-06-28T16:14:33.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.JOPseKnbTu9lMK5edlmbrvVy_ohlosw_0 2024-06-28T16:14:33.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7198516464186950253', time=2024-06-28T17:53:22.390387+0000, chain.objs.size()=1 2024-06-28T16:14:33.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.lXZOes3vxXVEXSIFQ51XgphWmpCj5Ke_0 2024-06-28T16:14:33.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4678489749553329396', time=2024-06-28T17:53:23.122385+0000, chain.objs.size()=1 2024-06-28T16:14:33.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.A6wObugzEcCpNZ82nNtHXJuKnQ7jIZA_0 2024-06-28T16:14:33.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13505338598922682166', time=2024-06-28T17:53:23.124153+0000, chain.objs.size()=1 2024-06-28T16:14:33.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.AZjm1zNlqR2x3Hwnvyi6vdsZB3rqUsf_0 2024-06-28T16:14:33.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4028930069068521997', time=2024-06-28T17:53:23.222975+0000, chain.objs.size()=1 2024-06-28T16:14:33.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.jGqOasXHnQ1LecNoDDCnVLm-D8uDOJ9_0 2024-06-28T16:14:33.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.607+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13728141747312573655', time=2024-06-28T17:53:26.540678+0000, chain.objs.size()=1 2024-06-28T16:14:33.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.607+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.sNSDg3Pss1czHStfEcYNkBDT7MO6BXF_0 2024-06-28T16:14:33.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.607+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2308079118756803471', time=2024-06-28T17:53:26.918806+0000, chain.objs.size()=1 2024-06-28T16:14:33.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.607+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.cAUZSn-zkMTPUvATDtau-1zMsuPjEvx_0 2024-06-28T16:14:33.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.607+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1900233793482249026', time=2024-06-28T17:53:27.665278+0000, chain.objs.size()=1 2024-06-28T16:14:33.618 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.607+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bw3An4Ml8dE2ExfXg47HtcYxGtH1IvU_0 2024-06-28T16:14:33.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2593679376612148696', time=2024-06-28T17:53:28.150529+0000, chain.objs.size()=1 2024-06-28T16:14:33.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.mPvg_s6NKMu2URmDNEz1s8vPHD_TWGz_0 2024-06-28T16:14:33.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14649236219114858697', time=2024-06-28T17:53:28.423578+0000, chain.objs.size()=1 2024-06-28T16:14:33.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vdWic518lVVDJYXJc_hKJ-2oD6cLeIR_0 2024-06-28T16:14:33.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14577301937606428681', time=2024-06-28T17:53:30.536015+0000, chain.objs.size()=1 2024-06-28T16:14:33.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.nmK5qJIaGtI_at-ZGi24wtYcFkGskbY_0 2024-06-28T16:14:33.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6857413299052765307', time=2024-06-28T17:53:30.679461+0000, chain.objs.size()=1 2024-06-28T16:14:33.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.3djLMWvZDO5L_VdmzQGPnAQoFx8c2vh_0 2024-06-28T16:14:33.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9728775342017112044', time=2024-06-28T17:53:32.073054+0000, chain.objs.size()=1 2024-06-28T16:14:33.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.9kfLKya6dJWUqTWcrZiW7t0i5CfErer_0 2024-06-28T16:14:33.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1002013463559013796', time=2024-06-28T17:53:33.824972+0000, chain.objs.size()=1 2024-06-28T16:14:33.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.i3N7r716iuVJqqD6J78uuI7D6YLxQIF_0 2024-06-28T16:14:33.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9355508530016267452', time=2024-06-28T17:53:34.674221+0000, chain.objs.size()=1 2024-06-28T16:14:33.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.7KiG1zxiLZyXZwic1W5znsiwu5oOVcC_0 2024-06-28T16:14:33.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16031349770249068286', time=2024-06-28T17:53:35.251098+0000, chain.objs.size()=1 2024-06-28T16:14:33.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.iNJd5h2Px1lNfo67JvJiREQCEC3DOgw_0 2024-06-28T16:14:33.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7930367578198077300', time=2024-06-28T17:53:35.999580+0000, chain.objs.size()=1 2024-06-28T16:14:33.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.HhLCwLRd3Mmo8r70OKkLQTLB2tYuZCL_0 2024-06-28T16:14:33.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2646332301679636809', time=2024-06-28T17:53:37.810225+0000, chain.objs.size()=1 2024-06-28T16:14:33.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Pt1J_n_vf6Xs19fvkW3Iox3lZi0H99P_0 2024-06-28T16:14:33.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14283838495171913026', time=2024-06-28T17:53:37.812297+0000, chain.objs.size()=1 2024-06-28T16:14:33.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.TdoVF7ad7yHypqmV1QRahIAOpcBujWi_0 2024-06-28T16:14:33.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5951217380081648721', time=2024-06-28T17:53:38.266006+0000, chain.objs.size()=1 2024-06-28T16:14:33.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.slTWOiIgCtYXlmpgwq6aftCrgvTeAO__0 2024-06-28T16:14:33.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15272274526703523467', time=2024-06-28T17:53:40.114046+0000, chain.objs.size()=1 2024-06-28T16:14:33.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.6iSQhNAJN5bq5gsVy8hZYaK9fel-jXU_0 2024-06-28T16:14:33.624 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1303320760921105622', time=2024-06-28T17:53:40.562271+0000, chain.objs.size()=1 2024-06-28T16:14:33.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.W4ckiq-6NSmS0h3kxEKgOTAvRhq2TLT_0 2024-06-28T16:14:33.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7575992351305938305', time=2024-06-28T17:53:42.781686+0000, chain.objs.size()=1 2024-06-28T16:14:33.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4DnmIbCEPNxa2NZwy9eMARzv4ccgYQe_0 2024-06-28T16:14:33.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16480150501870379519', time=2024-06-28T17:53:44.615212+0000, chain.objs.size()=1 2024-06-28T16:14:33.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1jF6NPC9bnQEJlVhFyvxhKDRMuyTTWS_0 2024-06-28T16:14:33.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12113811868249757666', time=2024-06-28T17:53:45.907683+0000, chain.objs.size()=1 2024-06-28T16:14:33.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ozXu5QVjyAYbOIO4sOtjI6OftPQRUuU_0 2024-06-28T16:14:33.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5774484728565325427', time=2024-06-28T17:53:47.445597+0000, chain.objs.size()=1 2024-06-28T16:14:33.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4d78jfyDX3i8l4StRXKfhJ_19CWg8ay_0 2024-06-28T16:14:33.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9587333360022273334', time=2024-06-28T17:53:48.938357+0000, chain.objs.size()=1 2024-06-28T16:14:33.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.eA1_WG3lQGC0fJ6APYZjONoN8ETbTkM_0 2024-06-28T16:14:33.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18011232154349979921', time=2024-06-28T17:53:50.035656+0000, chain.objs.size()=1 2024-06-28T16:14:33.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.VvVlRcK-ub4b56knjykm36bmjIrAlA__0 2024-06-28T16:14:33.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15949478069981288393', time=2024-06-28T17:53:50.603530+0000, chain.objs.size()=1 2024-06-28T16:14:33.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.uveU7ewaKvYS82xcjPgzxAe6WKFOU-u_0 2024-06-28T16:14:33.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17875765792083525572', time=2024-06-28T17:53:53.964095+0000, chain.objs.size()=1 2024-06-28T16:14:33.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.316__shadow_.cb7DAmNosPEmCURldHvoiUewQlzjoeO_0 2024-06-28T16:14:33.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14266835671071745819', time=2024-06-28T17:54:02.948979+0000, chain.objs.size()=1 2024-06-28T16:14:33.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.325__shadow_.9W0GDUC_cXDGUi8CnvZ9wDmcGpibf___0 2024-06-28T16:14:33.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9824197355676558002', time=2024-06-28T17:54:35.577684+0000, chain.objs.size()=1 2024-06-28T16:14:33.629 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.364__shadow_.4x1HwS_ux6sUB1ujgvzRG4rNctKEzBK_0 2024-06-28T16:14:33.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.623+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/42630 2024-06-28T16:14:33.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-06-28T16:14:33.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7928962784334161961', time=2024-06-28T17:56:10.182638+0000, chain.objs.size()=1 2024-06-28T16:14:33.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.454__shadow_.3wfk8_UzNiP5DzzlOl-gPObpp-ULf3S_0 2024-06-28T16:14:33.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3406998533820873426', time=2024-06-28T17:56:13.801072+0000, chain.objs.size()=1 2024-06-28T16:14:33.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.456__shadow_.lwwgvOy0XpBrbMfv3qIfoeRUsLmMt-l_0 2024-06-28T16:14:33.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14894216540259228882', time=2024-06-28T17:56:28.114711+0000, chain.objs.size()=1 2024-06-28T16:14:33.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.473__shadow_.bJfIIwlqHXbdioTAfXPRAHVZ_C3olFI_0 2024-06-28T16:14:33.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11033233517963214528', time=2024-06-28T17:56:41.457911+0000, chain.objs.size()=1 2024-06-28T16:14:33.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.481__shadow_.Pd7f2HEnQIc26it6ObxFEv-EGpgSjmZ_0 2024-06-28T16:14:33.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2214285462348448770', time=2024-06-28T18:01:58.841314+0000, chain.objs.size()=1 2024-06-28T16:14:33.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:xc9bUUe7g6YvHueM2GOUxrVlNQIJ9eK_.0_7FNWehlHBjwxM8lzmXZn_KZbbcYaP_0 2024-06-28T16:14:33.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18226871376520754993', time=2024-06-28T18:04:08.386729+0000, chain.objs.size()=1 2024-06-28T16:14:33.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.574__shadow:.IWBoG5rKVVJfwqLSNeTGXHD0wQLDva_.8vZSLD_lTTY1Yj4Miu-wzVD0Ff_ylAc_0 2024-06-28T16:14:33.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14341554887002806073', time=2024-06-28T18:06:30.785032+0000, chain.objs.size()=6 2024-06-28T16:14:33.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.610__multipart_multipart_enc.2~-MkfeDl8AjW0IQPAr-NBElLcr3FotoI.1 2024-06-28T16:14:33.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.610__multipart_multipart_enc.2~-MkfeDl8AjW0IQPAr-NBElLcr3FotoI.2 2024-06-28T16:14:33.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.610__multipart_multipart_enc.2~-MkfeDl8AjW0IQPAr-NBElLcr3FotoI.3 2024-06-28T16:14:33.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.610__multipart_multipart_enc.2~-MkfeDl8AjW0IQPAr-NBElLcr3FotoI.4 2024-06-28T16:14:33.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.610__multipart_multipart_enc.2~-MkfeDl8AjW0IQPAr-NBElLcr3FotoI.5 2024-06-28T16:14:33.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.610__multipart_multipart_enc.2~-MkfeDl8AjW0IQPAr-NBElLcr3FotoI.6 2024-06-28T16:14:33.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~eNwzGr-Nrv6SS_2j3tBSNdyUZWFrspa', time=2024-06-28T18:06:39.240659+0000, chain.objs.size()=6 2024-06-28T16:14:33.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.611__multipart_multipart_enc.2~eNwzGr-Nrv6SS_2j3tBSNdyUZWFrspa.1 2024-06-28T16:14:33.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.611__multipart_multipart_enc.2~eNwzGr-Nrv6SS_2j3tBSNdyUZWFrspa.2 2024-06-28T16:14:33.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.611__multipart_multipart_enc.2~eNwzGr-Nrv6SS_2j3tBSNdyUZWFrspa.3 2024-06-28T16:14:33.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.611__multipart_multipart_enc.2~eNwzGr-Nrv6SS_2j3tBSNdyUZWFrspa.4 2024-06-28T16:14:33.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.611__multipart_multipart_enc.2~eNwzGr-Nrv6SS_2j3tBSNdyUZWFrspa.5 2024-06-28T16:14:33.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.611__multipart_multipart_enc.2~eNwzGr-Nrv6SS_2j3tBSNdyUZWFrspa.6 2024-06-28T16:14:33.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.639+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:33.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-06-28T16:14:33.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:33.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:33.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/47467' 2024-06-28T16:14:33.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16838009756187684690', time=2024-06-28T17:47:08.462265+0000, chain.objs.size()=1 2024-06-28T16:14:33.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7LbU2Ubmx2qHdcTuEAmdXHVjLvfIfMs_0 2024-06-28T16:14:33.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13449798385265956123', time=2024-06-28T17:47:10.055937+0000, chain.objs.size()=1 2024-06-28T16:14:33.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dFbHjd7hBIpofhG7o1gGLKNPWFz5lWe_0 2024-06-28T16:14:33.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9698855811648967316', time=2024-06-28T17:47:10.058693+0000, chain.objs.size()=1 2024-06-28T16:14:33.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.kI89urg0z3s2FaoGFlaFuXpaL2keHTX_0 2024-06-28T16:14:33.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13315077910036874970', time=2024-06-28T17:47:10.578423+0000, chain.objs.size()=1 2024-06-28T16:14:33.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.UA__gN5R5_HxRENU98CmFQ9Q0UkBiW8_0 2024-06-28T16:14:33.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11260009162083827926', time=2024-06-28T17:47:13.397851+0000, chain.objs.size()=1 2024-06-28T16:14:33.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Kw4xs7KkGx8oOtCufoC6iNrzh4edYR3_0 2024-06-28T16:14:33.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6513799611398852175', time=2024-06-28T17:47:14.518255+0000, chain.objs.size()=1 2024-06-28T16:14:33.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9jYff9RMMm21tqVmjcFKRDnxV_m9HDx_0 2024-06-28T16:14:33.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4725859411186031219', time=2024-06-28T17:47:16.997402+0000, chain.objs.size()=1 2024-06-28T16:14:33.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dfn-ibDxTmiMX3hPgN5xgWguhsEoY0o_0 2024-06-28T16:14:33.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17587721287867432647', time=2024-06-28T17:47:17.662573+0000, chain.objs.size()=1 2024-06-28T16:14:33.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.XCMI5zzW6kVhMxTJ2r6bkGR9UYpTVef_0 2024-06-28T16:14:33.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8697996350408449353', time=2024-06-28T17:47:18.533700+0000, chain.objs.size()=1 2024-06-28T16:14:33.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.hK3rTZtezXNBCcBa3fXyE2cL78FJ7m8_0 2024-06-28T16:14:33.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.353660538741346983', time=2024-06-28T17:47:18.742835+0000, chain.objs.size()=1 2024-06-28T16:14:33.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5MmdWxwdnncSdjdZRcq4dxNEYAxRSQw_0 2024-06-28T16:14:33.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15257415406275689804', time=2024-06-28T17:47:19.584304+0000, chain.objs.size()=1 2024-06-28T16:14:33.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.xJ2v8BpCBU5JJqONsK0icPyP38RrR26_0 2024-06-28T16:14:33.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15662857513185667920', time=2024-06-28T17:47:22.301652+0000, chain.objs.size()=1 2024-06-28T16:14:33.657 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ybcvy2UVlojNwnAF4M1z0So48kJGWf2_0 2024-06-28T16:14:33.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15518279600704125228', time=2024-06-28T17:47:22.626535+0000, chain.objs.size()=1 2024-06-28T16:14:33.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.C5uGhADeaWEpxX_RRByMB3FF6hlGgvk_0 2024-06-28T16:14:33.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14350822502518315285', time=2024-06-28T17:47:22.876246+0000, chain.objs.size()=1 2024-06-28T16:14:33.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.nTIePaL2d9x4qxy4a3N5uWrK-0oHOfa_0 2024-06-28T16:14:33.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11667108308260947567', time=2024-06-28T17:47:24.067796+0000, chain.objs.size()=1 2024-06-28T16:14:33.660 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.6O2jIRKFU6MgysMW0Fd7u_lw_9WXOzY_0 2024-06-28T16:14:33.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7530892150972097706', time=2024-06-28T17:47:25.071519+0000, chain.objs.size()=1 2024-06-28T16:14:33.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.uprJXGkbvmP2qRV4r4j7xc5Ke7ldsF1_0 2024-06-28T16:14:33.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8395298080495357696', time=2024-06-28T17:47:25.459813+0000, chain.objs.size()=1 2024-06-28T16:14:33.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9JORMLPgFZ9kkTQr7SVNf4TSGqx7rS-_0 2024-06-28T16:14:33.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.950749324887064360', time=2024-06-28T17:47:30.074920+0000, chain.objs.size()=1 2024-06-28T16:14:33.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.0DMWgwHqjxucn7piI6ANIXyngQWhoep_0 2024-06-28T16:14:33.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12129973952365009508', time=2024-06-28T17:47:31.672396+0000, chain.objs.size()=1 2024-06-28T16:14:33.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.M8SmetYP0qsdn96hxa6JzxKIOUhT23K_0 2024-06-28T16:14:33.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17820016053250084872', time=2024-06-28T17:47:36.099973+0000, chain.objs.size()=1 2024-06-28T16:14:33.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ns0I6CJuRg7tGtHwjHe0hGkof12wvX3_0 2024-06-28T16:14:33.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15184196217263633627', time=2024-06-28T17:47:36.231722+0000, chain.objs.size()=1 2024-06-28T16:14:33.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.RGJiIjpIta4f9ytBjR5iL7vKICg6YT5_0 2024-06-28T16:14:33.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3739875767189352188', time=2024-06-28T17:47:36.980717+0000, chain.objs.size()=1 2024-06-28T16:14:33.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.o9xhXC4Dh294PmSP-M2JeBM809BbpR2_0 2024-06-28T16:14:33.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3689393151177309836', time=2024-06-28T17:47:37.480378+0000, chain.objs.size()=1 2024-06-28T16:14:33.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.cewvtm_Qb3Y7dSwOJhPJBCeV096br5f_0 2024-06-28T16:14:33.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16573789725078657631', time=2024-06-28T17:47:37.496932+0000, chain.objs.size()=1 2024-06-28T16:14:33.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.OsTgx8kSyATYrQJpiSA1XOxqqnUMT_I_0 2024-06-28T16:14:33.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6306194843410875642', time=2024-06-28T17:47:39.106532+0000, chain.objs.size()=1 2024-06-28T16:14:33.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.6oxi11qVJe4nx3NDcSq4ZpllcPhhfrd_0 2024-06-28T16:14:33.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7721973940951160776', time=2024-06-28T17:47:51.439640+0000, chain.objs.size()=1 2024-06-28T16:14:33.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.260__shadow_.NkDedsOiyDs3jdLJYLqViyOZb8Ve9gT_0 2024-06-28T16:14:33.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13329258952901639365', time=2024-06-28T17:48:19.822780+0000, chain.objs.size()=1 2024-06-28T16:14:33.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.278__shadow_.UyVd-xCVL8EdR5toq_IooXDwF-9kkbR_0 2024-06-28T16:14:33.665 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17124855583633341216', time=2024-06-28T17:48:29.434727+0000, chain.objs.size()=1 2024-06-28T16:14:33.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.S8qCJtxsG9RZxYlprqPvp3baxCgaV37_0 2024-06-28T16:14:33.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2018009324043372228', time=2024-06-28T17:48:29.565443+0000, chain.objs.size()=1 2024-06-28T16:14:33.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.lX5P5aklT99XorL1VRmbrWAbTB473Or_0 2024-06-28T16:14:33.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15870528265272563331', time=2024-06-28T17:48:39.744312+0000, chain.objs.size()=1 2024-06-28T16:14:33.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.286__shadow_.IpgmYQfwKLh1k5weAo8KqUouX46Frrj_0 2024-06-28T16:14:33.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3227038897798638950', time=2024-06-28T17:48:46.323483+0000, chain.objs.size()=1 2024-06-28T16:14:33.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.290__shadow_.fX2udcODSUutQZTMrGR9ZtDmHx4-7Cu_0 2024-06-28T16:14:33.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2147698991695937671', time=2024-06-28T17:48:47.895563+0000, chain.objs.size()=1 2024-06-28T16:14:33.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.291__shadow_.D_rrKm-hubQ7hb9OpdE7A6IHLM3NGFN_0 2024-06-28T16:14:33.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15912979558157450812', time=2024-06-28T17:51:08.223193+0000, chain.objs.size()=1 2024-06-28T16:14:33.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.UcLyvg8BfY6dwrTIeFdPkpiJC47bpCk_0 2024-06-28T16:14:33.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2691818633332445879', time=2024-06-28T17:51:08.341198+0000, chain.objs.size()=1 2024-06-28T16:14:33.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.0uNmWyXw-UDsJlpTaYn_rs3tANbFU3g_0 2024-06-28T16:14:33.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5146625993282023092', time=2024-06-28T17:51:08.674497+0000, chain.objs.size()=1 2024-06-28T16:14:33.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.lOVfAfPftr-B8RIWT2TK-fzddKQzz9S_0 2024-06-28T16:14:33.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2294255555259980427', time=2024-06-28T17:51:11.068855+0000, chain.objs.size()=1 2024-06-28T16:14:33.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6vslshqRewN6fVFTnQEsv4iI8dVFIjT_0 2024-06-28T16:14:33.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11534636536591348940', time=2024-06-28T17:51:11.705920+0000, chain.objs.size()=1 2024-06-28T16:14:33.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.salU2EhoULAJOnipVtrBlUfhf4dAMSo_0 2024-06-28T16:14:33.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16955323344368830809', time=2024-06-28T17:51:13.907246+0000, chain.objs.size()=1 2024-06-28T16:14:33.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.JAUcIFsuKwV9touMQllj5HyaTMaCd3r_0 2024-06-28T16:14:33.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1465785071531749618', time=2024-06-28T17:51:13.908707+0000, chain.objs.size()=1 2024-06-28T16:14:33.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6cHuq3wKdux_OSBDwKZ8o93Tfq4ZxEx_0 2024-06-28T16:14:33.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18003167553555822168', time=2024-06-28T17:51:14.352082+0000, chain.objs.size()=1 2024-06-28T16:14:33.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GQa3bf3snZSQYBf7iDbC7aUPb_vxdMi_0 2024-06-28T16:14:33.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12677866294880077511', time=2024-06-28T17:51:15.447883+0000, chain.objs.size()=1 2024-06-28T16:14:33.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.QS5uySYrXVGrUai-QOURJO3RplaUXil_0 2024-06-28T16:14:33.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1175757129454646892', time=2024-06-28T17:51:15.456356+0000, chain.objs.size()=1 2024-06-28T16:14:33.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.rd1ImqdgFqoleZqD_aduvXdQLstgwbE_0 2024-06-28T16:14:33.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11371766279675039836', time=2024-06-28T17:51:15.778298+0000, chain.objs.size()=1 2024-06-28T16:14:33.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.MpawMSduT_LOSAvXEgzwMVQo1HJBvcO_0 2024-06-28T16:14:33.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16462227100160972572', time=2024-06-28T17:51:15.779827+0000, chain.objs.size()=1 2024-06-28T16:14:33.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.td0cb8ZYU-wtxOrL1yTOQplNjJX2mx__0 2024-06-28T16:14:33.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14496398257527913010', time=2024-06-28T17:51:15.923635+0000, chain.objs.size()=1 2024-06-28T16:14:33.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GqV5Jq2WOsmK3T1Ulb9zv-N18NVK6y2_0 2024-06-28T16:14:33.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14304842385348828854', time=2024-06-28T17:51:17.079916+0000, chain.objs.size()=1 2024-06-28T16:14:33.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.f2Zyp8nvhwUunmfl4yNu0qHcMQcUfPn_0 2024-06-28T16:14:33.675 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5517164124634603781', time=2024-06-28T17:51:17.230781+0000, chain.objs.size()=1 2024-06-28T16:14:33.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.667+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ZSSYtuwmY9ZVWABRXmJrCpcrLrmzOAR_0 2024-06-28T16:14:33.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8775016252294957451', time=2024-06-28T17:51:17.843865+0000, chain.objs.size()=1 2024-06-28T16:14:33.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.667+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.COINydnaOAgWLfBMZYgk0UPNHYWEbHI_0 2024-06-28T16:14:33.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18054387346343417154', time=2024-06-28T17:51:17.845414+0000, chain.objs.size()=1 2024-06-28T16:14:33.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.667+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.0BbMWD-GpxT_vaa8ZWdq9GVhgNGRjMn_0 2024-06-28T16:14:33.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14862784057166836664', time=2024-06-28T17:51:19.483051+0000, chain.objs.size()=1 2024-06-28T16:14:33.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hJQ7129AjSPwMiMK0aeOb7tE8lmyJFy_0 2024-06-28T16:14:33.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8114954199369127201', time=2024-06-28T17:51:20.335214+0000, chain.objs.size()=1 2024-06-28T16:14:33.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.kHelA7z3kwrF4oauc5luhqGzWDsKQ5U_0 2024-06-28T16:14:33.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2530949043673280995', time=2024-06-28T17:51:20.443933+0000, chain.objs.size()=1 2024-06-28T16:14:33.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.13_y9XWGNuQt6NpPp9SvTBTvr1BboM3_0 2024-06-28T16:14:33.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4916255933770474765', time=2024-06-28T17:51:22.269849+0000, chain.objs.size()=1 2024-06-28T16:14:33.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Vs7y9ZpXl2L_kZcg-B8OcET7tyOyaA__0 2024-06-28T16:14:33.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9957401049188100772', time=2024-06-28T17:51:26.872024+0000, chain.objs.size()=1 2024-06-28T16:14:33.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.cz1F9W7CGEBCNKiKHPkFyeWTyv36aY4_0 2024-06-28T16:14:33.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13735134025967965429', time=2024-06-28T17:51:27.752617+0000, chain.objs.size()=1 2024-06-28T16:14:33.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.IzMMMMsLX_0RypQBx62CMUV3RTJdmv2_0 2024-06-28T16:14:33.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9960713959482137390', time=2024-06-28T17:51:27.763874+0000, chain.objs.size()=1 2024-06-28T16:14:33.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.tP0FXg4uv6JtW9tyU8OQUXkKKCUPMsr_0 2024-06-28T16:14:33.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5416069711868269315', time=2024-06-28T17:51:28.937333+0000, chain.objs.size()=1 2024-06-28T16:14:33.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.miBy-q-bvA1NzOdUnnFbVqWiJir2y6m_0 2024-06-28T16:14:33.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16592112673542362940', time=2024-06-28T17:51:29.555251+0000, chain.objs.size()=1 2024-06-28T16:14:33.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.L_odOeMh8VVvAS92Femz5njnxm25RSh_0 2024-06-28T16:14:33.682 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17078761142129722071', time=2024-06-28T17:51:30.653969+0000, chain.objs.size()=1 2024-06-28T16:14:33.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.22uTGteepsHNAUkZeCo6fqEn65NV3YQ_0 2024-06-28T16:14:33.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3761195208935348189', time=2024-06-28T17:51:31.873327+0000, chain.objs.size()=1 2024-06-28T16:14:33.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.5I8tAQCFI_HG_XDymHrOTanordXhtGs_0 2024-06-28T16:14:33.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16437693810169353196', time=2024-06-28T17:51:32.546648+0000, chain.objs.size()=1 2024-06-28T16:14:33.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.21LBRvHjAY_cD9rabObWQ87ZhPTXzxD_0 2024-06-28T16:14:33.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16815808920125714387', time=2024-06-28T17:51:33.251218+0000, chain.objs.size()=1 2024-06-28T16:14:33.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.kGwmI6uph7TlLqlQ_re9HFswZKhnlea_0 2024-06-28T16:14:33.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17958857573258723303', time=2024-06-28T17:51:33.582244+0000, chain.objs.size()=1 2024-06-28T16:14:33.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.vCAVz5MagNYLwRkqgw40Z4BW8j_Xn-O_0 2024-06-28T16:14:33.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2617579577027688701', time=2024-06-28T17:51:34.038658+0000, chain.objs.size()=1 2024-06-28T16:14:33.685 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.DYmM2PYT53ED7cqh-t_lXw0QcA-9IyK_0 2024-06-28T16:14:33.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16840321682160503536', time=2024-06-28T17:51:34.625582+0000, chain.objs.size()=1 2024-06-28T16:14:33.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.I5jys8UiC3L_yWUtjtelE7Upet5-5hx_0 2024-06-28T16:14:33.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1730282342801775330', time=2024-06-28T17:51:34.831873+0000, chain.objs.size()=1 2024-06-28T16:14:33.686 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ahKaT8HiN6I6K48DWLZt-5G1Kc8UtLA_0 2024-06-28T16:14:33.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10920560716916295265', time=2024-06-28T17:51:35.056379+0000, chain.objs.size()=1 2024-06-28T16:14:33.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.VMljRhDBa5I6Vn92qzvGloXwcsa_IeR_0 2024-06-28T16:14:33.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7905658090089273374', time=2024-06-28T17:51:35.881394+0000, chain.objs.size()=1 2024-06-28T16:14:33.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.EMkwSLLKRelaKmPIhK3Nspt1l_7bLlp_0 2024-06-28T16:14:33.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11842187723733431599', time=2024-06-28T17:51:37.783207+0000, chain.objs.size()=1 2024-06-28T16:14:33.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.OmBaPZubXl4N3oWUPSY3w-qVjgleV7u_0 2024-06-28T16:14:33.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17777446347190202165', time=2024-06-28T17:51:38.254572+0000, chain.objs.size()=1 2024-06-28T16:14:33.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xJWggDIotzWQbajg_WXhUdGq4PmOZyb_0 2024-06-28T16:14:33.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6336551974241362698', time=2024-06-28T17:53:20.921131+0000, chain.objs.size()=1 2024-06-28T16:14:33.688 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.sdm8VhyLee1Kkmh0R6oT7rN6EnWdYc8_0 2024-06-28T16:14:33.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.679+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16734698433606057557', time=2024-06-28T17:53:21.408231+0000, chain.objs.size()=1 2024-06-28T16:14:33.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.679+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1CQ5T1mSSEwwspQxvyzloJHzDexmrU3_0 2024-06-28T16:14:33.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.679+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14021923612524291951', time=2024-06-28T17:53:22.429141+0000, chain.objs.size()=1 2024-06-28T16:14:33.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.679+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gQyT0aiAJcEXQ_36z_jf0sq-5JffLUD_0 2024-06-28T16:14:33.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.679+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17828938705030309623', time=2024-06-28T17:53:26.316372+0000, chain.objs.size()=1 2024-06-28T16:14:33.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.679+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BW38Nq2s2pzBl9hHWlXZUNdQgfV5Cnq_0 2024-06-28T16:14:33.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.679+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16202096796832396692', time=2024-06-28T17:53:27.180025+0000, chain.objs.size()=1 2024-06-28T16:14:33.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.679+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Bj1jsY45rd8emfG_b88TXR2hRrdNJfO_0 2024-06-28T16:14:33.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.679+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12370246649718554118', time=2024-06-28T17:53:27.745292+0000, chain.objs.size()=1 2024-06-28T16:14:33.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.679+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1BZ1iAlMgx4fiKo0vUyCs8YMSovGiY9_0 2024-06-28T16:14:33.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11971338246364717803', time=2024-06-28T17:53:27.748328+0000, chain.objs.size()=1 2024-06-28T16:14:33.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.IwD7QlhaHT-4zBq193hDyVjKU9wqnxl_0 2024-06-28T16:14:33.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16260895745789214716', time=2024-06-28T17:53:27.842143+0000, chain.objs.size()=1 2024-06-28T16:14:33.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vYbJYVM8uvZ0mR2C5x0H614nQLFq1V-_0 2024-06-28T16:14:33.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14998882147494099988', time=2024-06-28T17:53:28.306715+0000, chain.objs.size()=1 2024-06-28T16:14:33.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.5NIQ6NBI95T0oyoT5GkYienS_kcR2dr_0 2024-06-28T16:14:33.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6134507093516541097', time=2024-06-28T17:53:28.434444+0000, chain.objs.size()=1 2024-06-28T16:14:33.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.yHSzAp4zb-JB7MBeWem4gnRICLsa7x-_0 2024-06-28T16:14:33.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10560836798693051623', time=2024-06-28T17:53:31.700259+0000, chain.objs.size()=1 2024-06-28T16:14:33.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.q2X2DrDKyh_XSA4alL0KUBic6WArvpe_0 2024-06-28T16:14:33.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6637406466837904956', time=2024-06-28T17:53:32.043801+0000, chain.objs.size()=1 2024-06-28T16:14:33.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.X4RV1_P4ScOjaKeU7f3y0vRSieg_EE6_0 2024-06-28T16:14:33.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13290419940463955320', time=2024-06-28T17:53:32.635504+0000, chain.objs.size()=1 2024-06-28T16:14:33.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wVExVc-sva6OGD3xoWlVrap5zX6WLk4_0 2024-06-28T16:14:33.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3868764475155410444', time=2024-06-28T17:53:36.513828+0000, chain.objs.size()=1 2024-06-28T16:14:33.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.KTg_fm8nYuKA-ILl2Q3_K_02dlmJuCA_0 2024-06-28T16:14:33.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1615365438141343867', time=2024-06-28T17:53:38.290446+0000, chain.objs.size()=1 2024-06-28T16:14:33.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.u0A5V52Rgd_gXFgC-ldnVqUpuLPlGJ0_0 2024-06-28T16:14:33.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17003784129506764626', time=2024-06-28T17:53:39.588383+0000, chain.objs.size()=1 2024-06-28T16:14:33.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.txA9IvPDTtRgPalpIVXjmOEYxO7lrzV_0 2024-06-28T16:14:33.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9882300349129035533', time=2024-06-28T17:53:40.013246+0000, chain.objs.size()=1 2024-06-28T16:14:33.697 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.5ugyy5TK3okKZuQ6ZywT0etivo41mNt_0 2024-06-28T16:14:33.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2111084867037695576', time=2024-06-28T17:53:40.561859+0000, chain.objs.size()=1 2024-06-28T16:14:33.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.NqQMo0jSLi0fU1WiJpIsZHE5PvGpY7d_0 2024-06-28T16:14:33.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6634365311988710029', time=2024-06-28T17:53:41.883325+0000, chain.objs.size()=1 2024-06-28T16:14:33.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.PyJWClr5GQfrh2CVRwBviogzbfjSvuJ_0 2024-06-28T16:14:33.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12858056875248544866', time=2024-06-28T17:53:42.281281+0000, chain.objs.size()=1 2024-06-28T16:14:33.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Io3bmLX2nMW8ZLSHnzzqsgReEpLejia_0 2024-06-28T16:14:33.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15492886513808570565', time=2024-06-28T17:53:43.805477+0000, chain.objs.size()=1 2024-06-28T16:14:33.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vptEKQxfnvf4x2MZcCSaRpWEBHkUhrh_0 2024-06-28T16:14:33.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8900311717405328116', time=2024-06-28T17:53:44.359322+0000, chain.objs.size()=1 2024-06-28T16:14:33.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1mQf8E1toDfGdtvCVdw0eVhugADgk3A_0 2024-06-28T16:14:33.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14186775028803613417', time=2024-06-28T17:53:45.054557+0000, chain.objs.size()=1 2024-06-28T16:14:33.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.E7lkQ6AGEgpKJJhBcKPHlW3KE6AA8NX_0 2024-06-28T16:14:33.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4227072507674899002', time=2024-06-28T17:53:46.213739+0000, chain.objs.size()=1 2024-06-28T16:14:33.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BLNzfCFjCPY91R2urp8MfHMqGSkvjge_0 2024-06-28T16:14:33.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16030472198483900324', time=2024-06-28T17:53:46.325590+0000, chain.objs.size()=1 2024-06-28T16:14:33.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Z2WfXJai1AUKMjgCr7nwzEieT3Plx_K_0 2024-06-28T16:14:33.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10723487639045559552', time=2024-06-28T17:53:46.854333+0000, chain.objs.size()=1 2024-06-28T16:14:33.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.mIz9_xUg2wGjrPggRpmL8to4JQfZd6i_0 2024-06-28T16:14:33.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10323730844162710639', time=2024-06-28T17:53:47.002111+0000, chain.objs.size()=1 2024-06-28T16:14:33.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.QbcimoW4tuUOT9MX4vrsPKOI4au36kC_0 2024-06-28T16:14:33.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1158695322107037587', time=2024-06-28T17:53:47.816079+0000, chain.objs.size()=1 2024-06-28T16:14:33.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.twGZ0Mx_aAfx_vspzBPjBchy6Y5rtKl_0 2024-06-28T16:14:33.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6886748226110058498', time=2024-06-28T17:53:48.094077+0000, chain.objs.size()=1 2024-06-28T16:14:33.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.dD32NtvCg8Oyfcki3AlX-ypaIk9oXQp_0 2024-06-28T16:14:33.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8622796940491061969', time=2024-06-28T17:53:50.962816+0000, chain.objs.size()=1 2024-06-28T16:14:33.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Vb1wfz9Fx4pjLsiQPyudR4exsN-TRAL_0 2024-06-28T16:14:33.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/47467 2024-06-28T16:14:33.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=20, truncated=0, next_marker='' 2024-06-28T16:14:33.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11012837400479554460', time=2024-06-28T17:53:51.276446+0000, chain.objs.size()=1 2024-06-28T16:14:33.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.0S-Djr3xMmhStfvwG0aXod8cMqwE62J_0 2024-06-28T16:14:33.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8503484905568223226', time=2024-06-28T17:53:52.145984+0000, chain.objs.size()=1 2024-06-28T16:14:33.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.DSNYcI7Htm-y5ugEWO98y_NrZXJhL8A_0 2024-06-28T16:14:33.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11156012406750080000', time=2024-06-28T17:54:38.709949+0000, chain.objs.size()=1 2024-06-28T16:14:33.708 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.367__shadow_.VdHQmL34CKu-uVzK3jPv5AW3w7bvuRQ_0 2024-06-28T16:14:33.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18123749742654158156', time=2024-06-28T17:54:42.656316+0000, chain.objs.size()=1 2024-06-28T16:14:33.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.371__shadow_._zpTSz-1XaxJOBYzBGmBywfz8lXSqTN_0 2024-06-28T16:14:33.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12307948872763464648', time=2024-06-28T17:55:38.029080+0000, chain.objs.size()=1 2024-06-28T16:14:33.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.430__shadow_.Iy9ti1k4AeS1n0vuauPu6Uwn0A4-c8Y_0 2024-06-28T16:14:33.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7086137829899583383', time=2024-06-28T17:56:03.264720+0000, chain.objs.size()=1 2024-06-28T16:14:33.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.450__shadow_.ZdfCMpfSgT6QVwX6CQnnSSZNQLhezKg_0 2024-06-28T16:14:33.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12458213001369827256', time=2024-06-28T17:56:06.670820+0000, chain.objs.size()=1 2024-06-28T16:14:33.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.452__shadow_.X3jiDw18biRCGcYZM1SeMz0DLFDE3YK_0 2024-06-28T16:14:33.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14316531041335173842', time=2024-06-28T17:56:28.116431+0000, chain.objs.size()=1 2024-06-28T16:14:33.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.473__shadow_.n_aAn4QQVtfOegrNjuYWEPxEPo2hkIv_0 2024-06-28T16:14:33.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.478__shadow_.2P51eHUNvnd8vt80wRKUqvwqpbgjfOB_1_662c7c587fbf6f4b7842801696c8c719', time=2024-06-28T17:56:38.087962+0000, chain.objs.size()=1 2024-06-28T16:14:33.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.478__shadow_.2P51eHUNvnd8vt80wRKUqvwqpbgjfOB_0 2024-06-28T16:14:33.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.490__shadow_.kGeXBSp5hgeaAc9Qmx89aDaRZegftIR_1_4ffa44eaa828a719524fdd57a6a74cb0', time=2024-06-28T17:56:50.296859+0000, chain.objs.size()=1 2024-06-28T16:14:33.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.490__shadow_.kGeXBSp5hgeaAc9Qmx89aDaRZegftIR_0 2024-06-28T16:14:33.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.493__shadow:rinwpm3WJlZ1AcGvTYYbpbEzG8SjNkf_.0OEFPT4L0L8Hzv3rUAZWjoQj5ExOwB1_1_e8a18ab4e15f06da2ae34ef5426d2b1f', time=2024-06-28T17:56:54.944288+0000, chain.objs.size()=1 2024-06-28T16:14:33.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.493__shadow:rinwpm3WJlZ1AcGvTYYbpbEzG8SjNkf_.0OEFPT4L0L8Hzv3rUAZWjoQj5ExOwB1_0 2024-06-28T16:14:33.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.773011759123711110', time=2024-06-28T17:58:10.094604+0000, chain.objs.size()=1 2024-06-28T16:14:33.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.510__shadow:35s7ga-sFoDXuwvwBt.Z9b7c37aYA8k_.aBJ2IRVlBXaVC29F0jHcLo9QL9jdu2q_0 2024-06-28T16:14:33.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8562650061334823593', time=2024-06-28T18:00:09.399779+0000, chain.objs.size()=4 2024-06-28T16:14:33.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.517__multipart_mymultipart.2~LBObXfsaqiTmM4zCtQsNqXBQIvV6-bk.1 2024-06-28T16:14:33.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.517__multipart_mymultipart.2~LBObXfsaqiTmM4zCtQsNqXBQIvV6-bk.2 2024-06-28T16:14:33.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.517__multipart_mymultipart.2~LBObXfsaqiTmM4zCtQsNqXBQIvV6-bk.3 2024-06-28T16:14:33.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.517__multipart_mymultipart.2~LBObXfsaqiTmM4zCtQsNqXBQIvV6-bk.4 2024-06-28T16:14:33.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5633622236682759661', time=2024-06-28T18:01:11.984993+0000, chain.objs.size()=1 2024-06-28T16:14:33.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.540__shadow_.1mtWbL5kR1_IKDlukwYWbgvKT4z3_hU_0 2024-06-28T16:14:33.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6390933437456074518', time=2024-06-28T18:01:54.071100+0000, chain.objs.size()=1 2024-06-28T16:14:33.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.554__shadow_.QFupfYTOzBis26U-iagugjaileQj2O9_0 2024-06-28T16:14:33.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11080709444360832150', time=2024-06-28T18:02:02.570387+0000, chain.objs.size()=1 2024-06-28T16:14:33.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:M5xmXRNLiUmT.l6wFCXkyijcTwG9TQD_.ArJasOsohsvdufc2QZYz_fzRtJF4aSx_0 2024-06-28T16:14:33.713 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4676290017594177082', time=2024-06-28T18:02:06.253996+0000, chain.objs.size()=1 2024-06-28T16:14:33.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:ZY.RtXZJbNh7n1yNgEbUWM.8.T-5uWO_.zs4UgR3oUuEhgtcvN0aHBtvWlBvnoXp_0 2024-06-28T16:14:33.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5471390551911203243', time=2024-06-28T18:02:58.176509+0000, chain.objs.size()=1 2024-06-28T16:14:33.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:HjN-ITE6mbH3bWR08lQ34Sx03benG3t_.4HQ3eOB7prLLE90yOG1wJzuP5EdfpGE_0 2024-06-28T16:14:33.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3856818396142230281', time=2024-06-28T18:08:03.012104+0000, chain.objs.size()=1 2024-06-28T16:14:33.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.650__shadow_.uv2817ivNcLVXfNZ2I9hkYp0GY3k8LT_0 2024-06-28T16:14:33.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.720074101266143827', time=2024-06-28T18:08:36.024513+0000, chain.objs.size()=1 2024-06-28T16:14:33.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.684__shadow:8KE-lXA48PJVR9jp7snpoK6qvR2.a-q_.FhK-0hOJ_hV9ReuQmJWLeZ3Iq2xS2jR_0 2024-06-28T16:14:33.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.711+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:33.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-06-28T16:14:33.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:33.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:33.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/42940' 2024-06-28T16:14:33.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_RW6PiqhYUobNR3YNxlS_cCg_zK_tfdI', time=2024-06-28T17:40:41.424837+0000, chain.objs.size()=1 2024-06-28T16:14:33.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.155__shadow_.zkSZT2G72oWQHySAeM4OJSzX69Bkgtv_0 2024-06-28T16:14:33.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5912935242428967077', time=2024-06-28T17:40:42.294834+0000, chain.objs.size()=1 2024-06-28T16:14:33.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.156__shadow_.gWpeEQqISsXDByiQm6eFZifhn9V3Zix_0 2024-06-28T16:14:33.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9871444489523396651', time=2024-06-28T17:47:10.203324+0000, chain.objs.size()=1 2024-06-28T16:14:33.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.S8a-PFGw9DVGrM_TLfWr3ZzEjmDueM__0 2024-06-28T16:14:33.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17938118736177752159', time=2024-06-28T17:47:12.578052+0000, chain.objs.size()=1 2024-06-28T16:14:33.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.0cNbNh6nTqN5elrslcymNoFM0UbxN6V_0 2024-06-28T16:14:33.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11149878295045762880', time=2024-06-28T17:47:12.848940+0000, chain.objs.size()=1 2024-06-28T16:14:33.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.C2gPwwHRl6f_J55aQDwWXai_31b6qFr_0 2024-06-28T16:14:33.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3657017112220830778', time=2024-06-28T17:47:14.609175+0000, chain.objs.size()=1 2024-06-28T16:14:33.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5ibGFhhq4hYeHcRSb-w8Z8dH3c2hPb7_0 2024-06-28T16:14:33.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2211441302010825245', time=2024-06-28T17:47:15.541375+0000, chain.objs.size()=1 2024-06-28T16:14:33.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.FOVfgR3rC5ItW64-G-MVjZtyUfXq81y_0 2024-06-28T16:14:33.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17380065901146278451', time=2024-06-28T17:47:16.590573+0000, chain.objs.size()=1 2024-06-28T16:14:33.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.mLFQrqqNJf6TT2izvsojAIPTFd7sOrG_0 2024-06-28T16:14:33.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.76689764494367955', time=2024-06-28T17:47:17.076271+0000, chain.objs.size()=1 2024-06-28T16:14:33.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.LXaBJDndEsO6Ny7T0zeNc7m1bHk0GdT_0 2024-06-28T16:14:33.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10346695982078438613', time=2024-06-28T17:47:17.419519+0000, chain.objs.size()=1 2024-06-28T16:14:33.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.QPwOZVzB8H3G0J47YU1bolK1VMIYPg0_0 2024-06-28T16:14:33.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6809157668928124292', time=2024-06-28T17:47:17.810873+0000, chain.objs.size()=1 2024-06-28T16:14:33.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.wZdk5qypoiJiU-ROgX8y1ZZTn-KiGet_0 2024-06-28T16:14:33.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3188369890086816197', time=2024-06-28T17:47:18.968925+0000, chain.objs.size()=1 2024-06-28T16:14:33.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ol8nDgmEvuCGtIy5SoUdsCbfjZJeMx1_0 2024-06-28T16:14:33.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14375323844145695380', time=2024-06-28T17:47:19.245231+0000, chain.objs.size()=1 2024-06-28T16:14:33.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.8m_Fpd92Tz02s-YlSPuJHKWjG_1Pi-O_0 2024-06-28T16:14:33.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13742320316256816194', time=2024-06-28T17:47:21.912654+0000, chain.objs.size()=1 2024-06-28T16:14:33.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KEIX-MDfj_WnAkrEm627YzN1Z-pgazW_0 2024-06-28T16:14:33.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1669954873889942825', time=2024-06-28T17:47:23.243229+0000, chain.objs.size()=1 2024-06-28T16:14:33.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.tFke4cKn4lDw9MuG2ZL223A0ZaFsD32_0 2024-06-28T16:14:33.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13127200693249012380', time=2024-06-28T17:47:23.506639+0000, chain.objs.size()=1 2024-06-28T16:14:33.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.sQiOsZG28PgPOvtv-6UgyRL3V8xqrWy_0 2024-06-28T16:14:33.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9472043922015724990', time=2024-06-28T17:47:25.030806+0000, chain.objs.size()=1 2024-06-28T16:14:33.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KXxM__oQuUy0-8pWKnBNvI_XIcYvpt4_0 2024-06-28T16:14:33.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6405046529111488281', time=2024-06-28T17:47:26.733386+0000, chain.objs.size()=1 2024-06-28T16:14:33.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ksF2qdQNKT0lRvHbfMiyyKqybsKlVnt_0 2024-06-28T16:14:33.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6304290167276480069', time=2024-06-28T17:47:27.253698+0000, chain.objs.size()=1 2024-06-28T16:14:33.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.0YaXYCfRxJ0NxngaqXmTJeUoXdoBbC4_0 2024-06-28T16:14:33.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13287153920842309957', time=2024-06-28T17:47:27.577627+0000, chain.objs.size()=1 2024-06-28T16:14:33.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.TYRMPIaer8Kdc7OkMq-VgnwXzwL6R2J_0 2024-06-28T16:14:33.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10098425557094859476', time=2024-06-28T17:47:30.970907+0000, chain.objs.size()=1 2024-06-28T16:14:33.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.JaM-_03ING1ZXoA9nehBIS3AZKz0lL3_0 2024-06-28T16:14:33.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3446124877524201577', time=2024-06-28T17:47:31.563802+0000, chain.objs.size()=1 2024-06-28T16:14:33.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.IiBhd9rNug4CgUudRQj7hfXpNk9cJXm_0 2024-06-28T16:14:33.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10205752808603853637', time=2024-06-28T17:47:31.567529+0000, chain.objs.size()=1 2024-06-28T16:14:33.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.avsQNKYpX4PMZk7fUC470fJTY_zBSwc_0 2024-06-28T16:14:33.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.723+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12885848430237673822', time=2024-06-28T17:47:31.872040+0000, chain.objs.size()=1 2024-06-28T16:14:33.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.723+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.MDlV7HlhJviYiGHiwLoNE3bSrGYsKUi_0 2024-06-28T16:14:33.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.723+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13977489545356017840', time=2024-06-28T17:47:31.874217+0000, chain.objs.size()=1 2024-06-28T16:14:33.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.723+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Pv740CTH2y5ZFRQLtsrGLiHz2XbfJMV_0 2024-06-28T16:14:33.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4328154490708611504', time=2024-06-28T17:47:33.164786+0000, chain.objs.size()=1 2024-06-28T16:14:33.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.fYv92BASL19Hxq5NOLvqwCMzQ3t290s_0 2024-06-28T16:14:33.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2580586964004590362', time=2024-06-28T17:47:33.324568+0000, chain.objs.size()=1 2024-06-28T16:14:33.735 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5XCrLMVw--lgoDyOYnMR1XogQ8SNfJp_0 2024-06-28T16:14:33.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12163959278768557183', time=2024-06-28T17:47:33.538846+0000, chain.objs.size()=1 2024-06-28T16:14:33.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.DnTVc3o1y1XxnckgHqXK91BQP3aimrT_0 2024-06-28T16:14:33.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3125515032186680838', time=2024-06-28T17:47:33.775881+0000, chain.objs.size()=1 2024-06-28T16:14:33.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.kS7387hsVsHcyxvKythv-oXhG2ifn4C_0 2024-06-28T16:14:33.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10868053441772030971', time=2024-06-28T17:47:35.743310+0000, chain.objs.size()=1 2024-06-28T16:14:33.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5Ufj_ZGePw4y5gp34MN-bE5L9Cn5qeq_0 2024-06-28T16:14:33.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18044458891250958099', time=2024-06-28T17:47:36.240075+0000, chain.objs.size()=1 2024-06-28T16:14:33.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.HHIi3fpVAQ6fDnktT_uykwA01d6wgWd_0 2024-06-28T16:14:33.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3984135243299114804', time=2024-06-28T17:47:37.293568+0000, chain.objs.size()=1 2024-06-28T16:14:33.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.yBospBZImQXckQFow67TTslUb4D0ks2_0 2024-06-28T16:14:33.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17699148195051472206', time=2024-06-28T17:47:37.956513+0000, chain.objs.size()=1 2024-06-28T16:14:33.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.apmHok0Of1zSkPC-CgBmNyIAu_VFJxs_0 2024-06-28T16:14:33.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9580798301393345690', time=2024-06-28T17:47:38.563112+0000, chain.objs.size()=1 2024-06-28T16:14:33.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.WNXLZkQBOAJYwInpnttET-1IG2X4S74_0 2024-06-28T16:14:33.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5897599997528164719', time=2024-06-28T17:47:39.261641+0000, chain.objs.size()=1 2024-06-28T16:14:33.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.pHtNxi5OlTFYPa-rljYi3KR3Xxba3nW_0 2024-06-28T16:14:33.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7956774386240019672', time=2024-06-28T17:47:41.134989+0000, chain.objs.size()=1 2024-06-28T16:14:33.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.253__shadow_.PGuesfM05yWHebvUwcXL0ISGAYPuWXO_0 2024-06-28T16:14:33.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11770153637141431602', time=2024-06-28T17:47:42.557538+0000, chain.objs.size()=1 2024-06-28T16:14:33.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.254__shadow_.px1iYI3BQ56j5pJlLSQ33A5ufRTvq2v_0 2024-06-28T16:14:33.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2490025085323157939', time=2024-06-28T17:47:54.437735+0000, chain.objs.size()=1 2024-06-28T16:14:33.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.262__shadow_.tE3HR6Hh_uUFjEkUmPUXRmrpC2UZhkO_0 2024-06-28T16:14:33.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1271681767742651565', time=2024-06-28T17:48:00.530726+0000, chain.objs.size()=1 2024-06-28T16:14:33.740 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.266__shadow_.VL1V0McGlEsRUqSLOEk8KS1TcueliFt_0 2024-06-28T16:14:33.741 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14581388313138655921', time=2024-06-28T17:48:06.924584+0000, chain.objs.size()=1 2024-06-28T16:14:33.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.270__shadow_.BDHB0z5qHojBb45K_bsMiYzaSofrwyi_0 2024-06-28T16:14:33.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14939017950265008167', time=2024-06-28T17:48:10.109128+0000, chain.objs.size()=1 2024-06-28T16:14:33.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.272__shadow_.tO8SW4uLdtwER0XFy9uQe7oirr9Kb2r_0 2024-06-28T16:14:33.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5995698063042256103', time=2024-06-28T17:48:13.241242+0000, chain.objs.size()=1 2024-06-28T16:14:33.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.274__shadow_.GbVyaCI7WVV1JZ6K_8m-9HVP8DOPb82_0 2024-06-28T16:14:33.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2643860840045648240', time=2024-06-28T17:48:22.985188+0000, chain.objs.size()=1 2024-06-28T16:14:33.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.280__shadow_.2h43djeEIoojGeQI_FnEbLRanm3-YFB_0 2024-06-28T16:14:33.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12057045560684328579', time=2024-06-28T17:48:34.463638+0000, chain.objs.size()=1 2024-06-28T16:14:33.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.731+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.iDRAnj37ZlXqpzgGTRstR46nWdRBE-s_0 2024-06-28T16:14:33.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6005290408543781610', time=2024-06-28T17:48:34.650855+0000, chain.objs.size()=1 2024-06-28T16:14:33.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.0rIb_JYpCtSONnnPULxAGz9DNrpIk_H_0 2024-06-28T16:14:33.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5633388264360888052', time=2024-06-28T17:48:36.514088+0000, chain.objs.size()=1 2024-06-28T16:14:33.743 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.284__shadow_.OUI3PuQxEO2L1M9C-wXrATCyCxUQSeD_0 2024-06-28T16:14:33.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4325779673037526100', time=2024-06-28T17:48:49.545667+0000, chain.objs.size()=1 2024-06-28T16:14:33.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.292__shadow_.lRQmXG4D3RiL8Ikn7h3iwjlNJ7iw7ol_0 2024-06-28T16:14:33.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13231877233408689709', time=2024-06-28T17:48:52.736486+0000, chain.objs.size()=1 2024-06-28T16:14:33.744 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.294__shadow_.MpBGRuZhdBWzTbQ_h-1tZpX5Z4jeuIK_0 2024-06-28T16:14:33.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12976738807768417555', time=2024-06-28T17:51:09.373141+0000, chain.objs.size()=1 2024-06-28T16:14:33.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yxQkren5mBxnyVmr7MkEZi_pEngX7d9_0 2024-06-28T16:14:33.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8678447271528516493', time=2024-06-28T17:51:09.687295+0000, chain.objs.size()=1 2024-06-28T16:14:33.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YLgdGVUno4mbwJlPSrFanuuri4xI7L6_0 2024-06-28T16:14:33.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10682464612204521894', time=2024-06-28T17:51:10.007125+0000, chain.objs.size()=1 2024-06-28T16:14:33.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hhVS_sM0T9oFyaM5ZNeqSdmofZgNqX-_0 2024-06-28T16:14:33.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3716822336939982766', time=2024-06-28T17:51:11.517334+0000, chain.objs.size()=1 2024-06-28T16:14:33.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fx0u8hhtUVjEG7bm0thwgsl-N5gX4zM_0 2024-06-28T16:14:33.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9794314603401171352', time=2024-06-28T17:51:12.530452+0000, chain.objs.size()=1 2024-06-28T16:14:33.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.VYLsO8In6y8HyZwq9koZQZ-CMyGZHwF_0 2024-06-28T16:14:33.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1821532607048468227', time=2024-06-28T17:51:13.787941+0000, chain.objs.size()=1 2024-06-28T16:14:33.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.VS_KodPZye4fBAbYd_J7KvggpwX81Ll_0 2024-06-28T16:14:33.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17829570192571051056', time=2024-06-28T17:51:14.189689+0000, chain.objs.size()=1 2024-06-28T16:14:33.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.m3gcMj0DoJaKs0eYkrgJ5QdLCnc-dPN_0 2024-06-28T16:14:33.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2549801935118824438', time=2024-06-28T17:51:16.245612+0000, chain.objs.size()=1 2024-06-28T16:14:33.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.3jxiYYmx1xZPnjG-QxjVZSiQuSh1VVQ_0 2024-06-28T16:14:33.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16877021090504235620', time=2024-06-28T17:51:17.258207+0000, chain.objs.size()=1 2024-06-28T16:14:33.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.H2xU3xINVFQjaALS37bZyhDtX5LNRHH_0 2024-06-28T16:14:33.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15965758202196475257', time=2024-06-28T17:51:18.064924+0000, chain.objs.size()=1 2024-06-28T16:14:33.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.dluFMIdck86PREbBWSd4Lzjj6_CSjpB_0 2024-06-28T16:14:33.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12831253346562883474', time=2024-06-28T17:51:19.480229+0000, chain.objs.size()=1 2024-06-28T16:14:33.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.u7G3hazzs_WCDGTaGYryCdrgANeaKbe_0 2024-06-28T16:14:33.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12973270628896628131', time=2024-06-28T17:51:19.576056+0000, chain.objs.size()=1 2024-06-28T16:14:33.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.q5Sc2BbYivLlp4x0NdzSF__0WgnP673_0 2024-06-28T16:14:33.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9724979205466125466', time=2024-06-28T17:51:20.172060+0000, chain.objs.size()=1 2024-06-28T16:14:33.752 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Pa5794wk7J29zoWYH7lB2PgU5lK8exi_0 2024-06-28T16:14:33.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3937148201822157603', time=2024-06-28T17:51:21.588266+0000, chain.objs.size()=1 2024-06-28T16:14:33.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.mFEYNa1Yr8CLNn6uY3u2AvdRc8pTmKK_0 2024-06-28T16:14:33.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16155893309550775113', time=2024-06-28T17:51:22.096676+0000, chain.objs.size()=1 2024-06-28T16:14:33.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1k6_B_KenuWQ5ZT7KNJM93KRcCZHfXK_0 2024-06-28T16:14:33.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2055310378261550857', time=2024-06-28T17:51:22.512524+0000, chain.objs.size()=1 2024-06-28T16:14:33.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yPW8nvIZ-FfAvH88-tTBq6XJI7GhQSU_0 2024-06-28T16:14:33.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.217458084187068765', time=2024-06-28T17:51:22.552570+0000, chain.objs.size()=1 2024-06-28T16:14:33.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.8NpSsAxXW358DeJkyn7J_vtEctIs7dD_0 2024-06-28T16:14:33.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2720329585940877082', time=2024-06-28T17:51:23.646719+0000, chain.objs.size()=1 2024-06-28T16:14:33.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.vNYgsRDnBTlxPUqTAgozia3OaJlxRJt_0 2024-06-28T16:14:33.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3930624832038195638', time=2024-06-28T17:51:23.651635+0000, chain.objs.size()=1 2024-06-28T16:14:33.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.cag6-Zyh6ya6oto8NbqJo0UR0za3G7V_0 2024-06-28T16:14:33.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.727288208128345495', time=2024-06-28T17:51:25.590688+0000, chain.objs.size()=1 2024-06-28T16:14:33.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Qa0uiH9DHEj4MW-I0EQn-ZCR727Yv91_0 2024-06-28T16:14:33.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.906710056349603834', time=2024-06-28T17:51:25.899101+0000, chain.objs.size()=1 2024-06-28T16:14:33.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.A1KfwThqP5D1Suilm7aoPaflliYjvYL_0 2024-06-28T16:14:33.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8184701846523667750', time=2024-06-28T17:51:26.727887+0000, chain.objs.size()=1 2024-06-28T16:14:33.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.tPRP3GL-1lIfBuvda-EHcfzVdy1cqeE_0 2024-06-28T16:14:33.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8643535191304409444', time=2024-06-28T17:51:28.136773+0000, chain.objs.size()=1 2024-06-28T16:14:33.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.SZnSILLB8uT3Somt_EuwlaR5nY4hfqr_0 2024-06-28T16:14:33.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2521231756595399948', time=2024-06-28T17:51:28.138373+0000, chain.objs.size()=1 2024-06-28T16:14:33.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nVM51tzsnLSZ3hGxFsSsPN-Yr2VWyB__0 2024-06-28T16:14:33.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.747+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13877564708581447558', time=2024-06-28T17:51:28.785832+0000, chain.objs.size()=1 2024-06-28T16:14:33.757 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.747+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-6BEVWawUzWNGF4zpRLZKZ_uwo-YYUc_0 2024-06-28T16:14:33.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14915443059660606767', time=2024-06-28T17:51:29.868912+0000, chain.objs.size()=1 2024-06-28T16:14:33.759 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fNqG4f5W-z39Hm4PpDpwID_Un2GKZk__0 2024-06-28T16:14:33.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2833259468562042687', time=2024-06-28T17:51:31.882901+0000, chain.objs.size()=1 2024-06-28T16:14:33.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.rYUcOh2kLe0NiZ1SkmcBVItIjEpsWZE_0 2024-06-28T16:14:33.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14025436032052913628', time=2024-06-28T17:51:32.846323+0000, chain.objs.size()=1 2024-06-28T16:14:33.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.OPZLjFgTGYNG31nJiT_B_oATIis4Epm_0 2024-06-28T16:14:33.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1121343163820578917', time=2024-06-28T17:51:34.009988+0000, chain.objs.size()=1 2024-06-28T16:14:33.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.kjgsEWnKrC5_mV4EZ2CUOlvW_SP0HCJ_0 2024-06-28T16:14:33.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14991553054867626082', time=2024-06-28T17:51:34.037928+0000, chain.objs.size()=1 2024-06-28T16:14:33.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.QpZzAC-cko7H88cnHE49hFhHW1zkenC_0 2024-06-28T16:14:33.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9215102240244934287', time=2024-06-28T17:51:35.786053+0000, chain.objs.size()=1 2024-06-28T16:14:33.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.SxIKhiKquHpCO6xr27lmF0N6UhLB--Y_0 2024-06-28T16:14:33.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17359089612792701838', time=2024-06-28T17:51:36.266990+0000, chain.objs.size()=1 2024-06-28T16:14:33.762 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.SsBFSHdyXyQSAmTQ38CWU-oP1YlxiCR_0 2024-06-28T16:14:33.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2354120611489253448', time=2024-06-28T17:51:36.484317+0000, chain.objs.size()=1 2024-06-28T16:14:33.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.wUL2a2ccVxPYGZMPhahs_DJwNdPS0sJ_0 2024-06-28T16:14:33.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16141696338675150817', time=2024-06-28T17:51:36.774874+0000, chain.objs.size()=1 2024-06-28T16:14:33.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.K2HM93X0AzqIz32d1YNy4xRxNqN_EE8_0 2024-06-28T16:14:33.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.404477786864011219', time=2024-06-28T17:51:37.891672+0000, chain.objs.size()=1 2024-06-28T16:14:33.763 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.wRMEkmhJqsRF5dBiPhwo4y2v4asWHkP_0 2024-06-28T16:14:33.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13660950772462496262', time=2024-06-28T17:51:37.919487+0000, chain.objs.size()=1 2024-06-28T16:14:33.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.7PAUO-WyO3qxRAKkl2smCRZJSOK2KPp_0 2024-06-28T16:14:33.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3728627872785999538', time=2024-06-28T17:53:21.844542+0000, chain.objs.size()=1 2024-06-28T16:14:33.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Os1U1-Nr2Ks3j7LUuyH22BmCcRmw2_X_0 2024-06-28T16:14:33.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8025535720796454151', time=2024-06-28T17:53:22.698728+0000, chain.objs.size()=1 2024-06-28T16:14:33.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ArCc6vbMegcF4QdlQbn5nJq05B9kdhs_0 2024-06-28T16:14:33.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12886272988929765081', time=2024-06-28T17:53:23.371237+0000, chain.objs.size()=1 2024-06-28T16:14:33.765 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.lHgFXApo_mypaxJIs8JpwXcMpr4-OVx_0 2024-06-28T16:14:33.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5940379222322260724', time=2024-06-28T17:53:23.900451+0000, chain.objs.size()=1 2024-06-28T16:14:33.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.l5HhIPKDhzwGpXU6_g1eECh4trFQy5M_0 2024-06-28T16:14:33.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18108928192746749728', time=2024-06-28T17:53:27.428115+0000, chain.objs.size()=1 2024-06-28T16:14:33.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.AZU92KnLRNp3CnUgcdjtwSIEL4ZTZdG_0 2024-06-28T16:14:33.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.755+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2190062565629821461', time=2024-06-28T17:53:30.084705+0000, chain.objs.size()=1 2024-06-28T16:14:33.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.755+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.awGImZWnVmYrIJnfBg0oXFiQrX8d7ck_0 2024-06-28T16:14:33.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.759+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4485046565928495818', time=2024-06-28T17:53:33.795003+0000, chain.objs.size()=1 2024-06-28T16:14:33.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.759+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.RcdIyZ8HIG4-bIeiUCAqft4WiPEeTpb_0 2024-06-28T16:14:33.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.759+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9716046915703867651', time=2024-06-28T17:53:35.228444+0000, chain.objs.size()=1 2024-06-28T16:14:33.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.759+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.yGJqUqRWaxzFNByEmXJ3Gh8SN3n70By_0 2024-06-28T16:14:33.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.759+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16703493733739231402', time=2024-06-28T17:53:35.490677+0000, chain.objs.size()=1 2024-06-28T16:14:33.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.759+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.MMm3IO8M8LRObl2PfP2hXaRt6Vc3_96_0 2024-06-28T16:14:33.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.759+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2217104885628370994', time=2024-06-28T17:53:35.708354+0000, chain.objs.size()=1 2024-06-28T16:14:33.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.759+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.F3KwDUKQAx84WNI6br5lqxjhB2Tl09E_0 2024-06-28T16:14:33.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.759+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.444699488943447874', time=2024-06-28T17:53:35.979303+0000, chain.objs.size()=1 2024-06-28T16:14:33.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.759+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.FjOZp5i0ngJW_Oex3BUQHuF3EwJLVaF_0 2024-06-28T16:14:33.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.759+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5653383726980016010', time=2024-06-28T17:53:36.520642+0000, chain.objs.size()=1 2024-06-28T16:14:33.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.759+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gF3QfoJ13V_tenhap9xgyyFEaKjxYRm_0 2024-06-28T16:14:33.772 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.382620443499515072', time=2024-06-28T17:53:36.678331+0000, chain.objs.size()=1 2024-06-28T16:14:33.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.d1BKXUm0YTJAjkjyjbZImujeOvHjbqo_0 2024-06-28T16:14:33.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6158202467486374415', time=2024-06-28T17:53:42.495167+0000, chain.objs.size()=1 2024-06-28T16:14:33.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.b6TfBZ8fsFmfX4ofcsJpZJ-U7lx0_Ln_0 2024-06-28T16:14:33.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14004535831787887258', time=2024-06-28T17:53:42.533395+0000, chain.objs.size()=1 2024-06-28T16:14:33.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bNTBLPm1k_gextY1eXn0McftBNbTmFu_0 2024-06-28T16:14:33.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.763+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13409255189483685844', time=2024-06-28T17:53:43.787181+0000, chain.objs.size()=1 2024-06-28T16:14:33.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.763+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.FPa2dhp94pjyduFo_UHCmN5AF2r4R3H_0 2024-06-28T16:14:33.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/42940 2024-06-28T16:14:33.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=26, truncated=0, next_marker='' 2024-06-28T16:14:33.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16011556307965067673', time=2024-06-28T17:53:44.549002+0000, chain.objs.size()=1 2024-06-28T16:14:33.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.2MlEYvpDmiRBJyF8D7vkm0Vn-w1Q1ie_0 2024-06-28T16:14:33.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3408214665474991809', time=2024-06-28T17:53:46.785131+0000, chain.objs.size()=1 2024-06-28T16:14:33.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.RrWMxMC7btuK2b9WOxqPJCKm7Z-XQqc_0 2024-06-28T16:14:33.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15338779695136401245', time=2024-06-28T17:53:47.304484+0000, chain.objs.size()=1 2024-06-28T16:14:33.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.MqjYzWdzlaGmnZYrU2xilrEGbi352tR_0 2024-06-28T16:14:33.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9154517185096248985', time=2024-06-28T17:53:48.419063+0000, chain.objs.size()=1 2024-06-28T16:14:33.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.l9B3VVsM-zMsYA7TB1yWTpHhtzUk_Fo_0 2024-06-28T16:14:33.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2517812755483984224', time=2024-06-28T17:53:48.988591+0000, chain.objs.size()=1 2024-06-28T16:14:33.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.7CCjg515HlPMr5jA0SAuokwJn55m1B9_0 2024-06-28T16:14:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1520950443279388611', time=2024-06-28T17:53:49.925763+0000, chain.objs.size()=1 2024-06-28T16:14:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.QgN7PlJKtlghOKnv3YdjeuZt0y_nyo__0 2024-06-28T16:14:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11659843198262970343', time=2024-06-28T17:53:50.588010+0000, chain.objs.size()=1 2024-06-28T16:14:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.jUz3grl0l-vjMNq16bBQ5QEbX8L8hXv_0 2024-06-28T16:14:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5381063453011795951', time=2024-06-28T17:53:50.910381+0000, chain.objs.size()=1 2024-06-28T16:14:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.iOHzH5KQFKmrAJf28H87CUhcwa3YuTY_0 2024-06-28T16:14:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8726815956814505693', time=2024-06-28T17:53:51.531803+0000, chain.objs.size()=1 2024-06-28T16:14:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.H5dCuTDK4Yb5PG3o2EanmvR7h56aWJg_0 2024-06-28T16:14:33.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17425809406715853342', time=2024-06-28T17:53:51.673712+0000, chain.objs.size()=1 2024-06-28T16:14:33.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Wkt-o-V05qAK50sJ4Gp_8lbqHBPQN0C_0 2024-06-28T16:14:33.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9611756929920748034', time=2024-06-28T17:53:52.002387+0000, chain.objs.size()=1 2024-06-28T16:14:33.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.AKG_ypbQPCsdYrr9avHEECMeU1x6DXb_0 2024-06-28T16:14:33.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12317302040402020474', time=2024-06-28T17:53:52.213816+0000, chain.objs.size()=1 2024-06-28T16:14:33.781 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.767+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bUc7AE4eMh9naw8YHq-EWwtRWTxM89N_0 2024-06-28T16:14:33.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.771+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1190342772142292364', time=2024-06-28T17:54:39.918047+0000, chain.objs.size()=1 2024-06-28T16:14:33.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.771+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.368__shadow_.o-HFjSCC8Yfrx1NtEIHeV3v--Inx7RR_0 2024-06-28T16:14:33.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.771+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5675668641979722169', time=2024-06-28T17:56:12.007532+0000, chain.objs.size()=1 2024-06-28T16:14:33.782 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.771+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.455__shadow_.YtJMzBc3Tb4YkINWu_MWtOgQaczOUW__0 2024-06-28T16:14:33.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.771+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6556416493748686386', time=2024-06-28T17:56:28.070137+0000, chain.objs.size()=1 2024-06-28T16:14:33.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.771+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.473__shadow_.m67hfHIDyoaJkd082DEurtuGmrDHebf_0 2024-06-28T16:14:33.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.771+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17391090818430078852', time=2024-06-28T17:56:28.109042+0000, chain.objs.size()=1 2024-06-28T16:14:33.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.771+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.473__shadow_.O7pFgm2guHPwDYvK4NPIOQrOR8giliH_0 2024-06-28T16:14:33.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.771+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18039178769609334548', time=2024-06-28T17:56:30.244706+0000, chain.objs.size()=1 2024-06-28T16:14:33.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.771+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.475__shadow_.fpkLwQm5sGl_A8QNUpSxougljAiBDLQ_0 2024-06-28T16:14:33.783 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.771+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10657047114671317980', time=2024-06-28T17:59:49.932473+0000, chain.objs.size()=1 2024-06-28T16:14:33.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.771+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.515__multipart_mymultipart.2~eFLbQqfVOqlhFoBvEmVQHabconLP7on.1 2024-06-28T16:14:33.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.775+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12895344913466477862', time=2024-06-28T18:00:00.500694+0000, chain.objs.size()=3 2024-06-28T16:14:33.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.515__multipart_mymultipart.2~h8_owjsgEACqLyNtc8U-LQUItDDsZJ5.1 2024-06-28T16:14:33.784 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.515__multipart_mymultipart.2~h8_owjsgEACqLyNtc8U-LQUItDDsZJ5.2 2024-06-28T16:14:33.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.515__multipart_mymultipart.2~h8_owjsgEACqLyNtc8U-LQUItDDsZJ5.3 2024-06-28T16:14:33.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.775+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8933663163952629716', time=2024-06-28T18:01:40.715090+0000, chain.objs.size()=1 2024-06-28T16:14:33.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.548__shadow_.Z250Gxq-cT886p-BUgj_G-ObwiU8KHw_0 2024-06-28T16:14:33.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.775+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3985519010986094820', time=2024-06-28T18:02:03.396072+0000, chain.objs.size()=1 2024-06-28T16:14:33.785 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.775+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:uHPtyDBE2akHBN6Z-QAkdr8IgWjLIw4_.yEag-_ed3WdAUjBQ0xEh62LejkYsbsG_0 2024-06-28T16:14:33.788 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12189100751626018842', time=2024-06-28T18:02:25.589675+0000, chain.objs.size()=1 2024-06-28T16:14:33.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.561__shadow_.sPWytgZFwAwCjy3E4vjSZvloDS4KHQd_0 2024-06-28T16:14:33.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2506908828375705843', time=2024-06-28T18:04:00.713228+0000, chain.objs.size()=1 2024-06-28T16:14:33.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:63C.NZOu-V2ScGWmT27Bj4CNweN3JuO_.wze7dUjVhkbvyWhLsfojeq6MpbpcSqX_0 2024-06-28T16:14:33.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11823014030080200398', time=2024-06-28T18:04:04.153211+0000, chain.objs.size()=1 2024-06-28T16:14:33.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.572__shadow:GQAduFvkauG2866EXkeFRlxOd68WfOa_.n7hD3nyuCjrF4jSTbNybTfDq8McYyL6_0 2024-06-28T16:14:33.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1023710584062195260', time=2024-06-28T18:07:06.404173+0000, chain.objs.size()=1 2024-06-28T16:14:33.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.628__shadow_.UCkBACbpf01AxFefO0VVTEGNMtNH-mc_0 2024-06-28T16:14:33.789 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.779+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.869505424901764905', time=2024-06-28T18:08:46.397986+0000, chain.objs.size()=1 2024-06-28T16:14:33.790 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.779+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.692__shadow:5pk0jUUKtERcu4Jg4XDSKJaJD7qE6SD_.BW5ZSK4mwwko02B_xc4WiclctnzkRNq_0 2024-06-28T16:14:33.792 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.783+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:33.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-06-28T16:14:33.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:33.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:33.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/43351' 2024-06-28T16:14:33.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16913185820627647511', time=2024-06-28T17:41:46.202638+0000, chain.objs.size()=1 2024-06-28T16:14:33.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.189__shadow_.acRoydQNH_yZv9DdcNRBBVxitrCkEsn_0 2024-06-28T16:14:33.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.27628830289803334', time=2024-06-28T17:44:56.353289+0000, chain.objs.size()=1 2024-06-28T16:14:33.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.233__shadow_.2OP6b0x_hKM4wApULlJ8aUAgieQ3WBF_0 2024-06-28T16:14:33.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12167499690965600521', time=2024-06-28T17:45:00.260169+0000, chain.objs.size()=1 2024-06-28T16:14:33.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.235__shadow_.nCyTuXSDZQG38iBEQmV8uPYikZp8jO4_0 2024-06-28T16:14:33.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2286060186720721339', time=2024-06-28T17:45:23.873254+0000, chain.objs.size()=1 2024-06-28T16:14:33.798 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.250__shadow_.oZ2_TQRQn2cTMAIjc6QNTam4TjgTJ2H_0 2024-06-28T16:14:33.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15559355421696299438', time=2024-06-28T17:47:08.544085+0000, chain.objs.size()=1 2024-06-28T16:14:33.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.yzjU2oiNDfKS6F5KtYu8HkhVZwcoPOw_0 2024-06-28T16:14:33.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14703799996528045211', time=2024-06-28T17:47:09.168817+0000, chain.objs.size()=1 2024-06-28T16:14:33.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.TyGwdMZEqkPib8R_DZ2F9MDqjxuy8kU_0 2024-06-28T16:14:33.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17607504476509266108', time=2024-06-28T17:47:09.422043+0000, chain.objs.size()=1 2024-06-28T16:14:33.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dBFUPoxb21crmFqpnvEsX0m71uO21fC_0 2024-06-28T16:14:33.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1730552820483847486', time=2024-06-28T17:47:10.068982+0000, chain.objs.size()=1 2024-06-28T16:14:33.799 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.iiASavKEr08tkNZCi5PYxcvpnrC2oNJ_0 2024-06-28T16:14:33.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10408673068495863881', time=2024-06-28T17:47:10.912705+0000, chain.objs.size()=1 2024-06-28T16:14:33.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.N05eueR9eXhA_zVxLr0TJon0roZ5ltl_0 2024-06-28T16:14:33.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7383430190479517467', time=2024-06-28T17:47:11.003693+0000, chain.objs.size()=1 2024-06-28T16:14:33.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.tWIz58YXF3CCbd29Df5fliY-tcKQ1cE_0 2024-06-28T16:14:33.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5234533533485873566', time=2024-06-28T17:47:12.166462+0000, chain.objs.size()=1 2024-06-28T16:14:33.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.977WRdR-AGWfdYjX_qPhO3xJeGx2qZt_0 2024-06-28T16:14:33.800 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15138863743280796548', time=2024-06-28T17:47:12.225213+0000, chain.objs.size()=1 2024-06-28T16:14:33.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.787+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.mvEEV6-EgGMd-_U5qTrkjLnn030mGBE_0 2024-06-28T16:14:33.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.791+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11788935341094458548', time=2024-06-28T17:47:12.565099+0000, chain.objs.size()=1 2024-06-28T16:14:33.801 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.791+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.l6E_kvdh5_Mfm157yc2LSy66jKBUZj2_0 2024-06-28T16:14:33.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.791+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.668027359741559448', time=2024-06-28T17:47:13.383986+0000, chain.objs.size()=1 2024-06-28T16:14:33.802 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.791+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.rChY2-emQsUjZIvJarl5Amu3O4TjL0U_0 2024-06-28T16:14:33.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12700912331285387254', time=2024-06-28T17:47:13.391633+0000, chain.objs.size()=1 2024-06-28T16:14:33.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.r3VgCEEB5oyzHAe-r8N9hEGkjXwPcc2_0 2024-06-28T16:14:33.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12453924859580005242', time=2024-06-28T17:47:13.593646+0000, chain.objs.size()=1 2024-06-28T16:14:33.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.HQICLjkPH5O4cJ2UB2tFEWaydDxexDm_0 2024-06-28T16:14:33.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8537994711712439797', time=2024-06-28T17:47:14.111759+0000, chain.objs.size()=1 2024-06-28T16:14:33.805 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VOwR0Ocr2UfrRtfTjhTpjbkaVuMaOdk_0 2024-06-28T16:14:33.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15532409704858127657', time=2024-06-28T17:47:14.479430+0000, chain.objs.size()=1 2024-06-28T16:14:33.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.C9gqx86sKjl9B9di6XtGlOf_UgJKRAP_0 2024-06-28T16:14:33.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8076122039605371873', time=2024-06-28T17:47:14.705792+0000, chain.objs.size()=1 2024-06-28T16:14:33.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.i6VzQWef8gOh-HChlm7Jdf8K3dikZmk_0 2024-06-28T16:14:33.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1944322357892608167', time=2024-06-28T17:47:14.994058+0000, chain.objs.size()=1 2024-06-28T16:14:33.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.AiCPZdJubHcEtPctrSnKJV7m8EzDpDh_0 2024-06-28T16:14:33.806 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7564782426858095148', time=2024-06-28T17:47:16.608588+0000, chain.objs.size()=1 2024-06-28T16:14:33.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Hx88jIs3UFbGxc8IsTZiPfJZrdybMPJ_0 2024-06-28T16:14:33.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17869309976221313628', time=2024-06-28T17:47:16.622236+0000, chain.objs.size()=1 2024-06-28T16:14:33.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.m_Y6-dDKHa-eoNNSOYT0T9gJJhP5c6o_0 2024-06-28T16:14:33.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10911327622470568028', time=2024-06-28T17:47:17.226180+0000, chain.objs.size()=1 2024-06-28T16:14:33.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Eu892diyCWEITwakEJs_KWq97V5Ozye_0 2024-06-28T16:14:33.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17539162084931238293', time=2024-06-28T17:47:17.415604+0000, chain.objs.size()=1 2024-06-28T16:14:33.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.HIU4Igt5OVWm8VYUBYJ1tHMdBSJjtVd_0 2024-06-28T16:14:33.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13704469074570045806', time=2024-06-28T17:47:18.165152+0000, chain.objs.size()=1 2024-06-28T16:14:33.807 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.795+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.kKq5p3oOe1UMM1yrP_L53c80wCgrhv4_0 2024-06-28T16:14:33.808 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.799+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5007305668719850622', time=2024-06-28T17:47:19.317012+0000, chain.objs.size()=1 2024-06-28T16:14:33.809 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.799+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Z_NZgAhvz7FX2Ar6DtBxepie1CHijM1_0 2024-06-28T16:14:33.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13308654993881619792', time=2024-06-28T17:47:19.455509+0000, chain.objs.size()=1 2024-06-28T16:14:33.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.mLZHniPUuP3ECcytw5bY2DyQf0i1W2Z_0 2024-06-28T16:14:33.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.351681290653720985', time=2024-06-28T17:47:19.601380+0000, chain.objs.size()=1 2024-06-28T16:14:33.813 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.XC_teBUt1gQEq8X59g2FdFtUUvFR08Q_0 2024-06-28T16:14:33.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.499329512450608532', time=2024-06-28T17:47:19.883868+0000, chain.objs.size()=1 2024-06-28T16:14:33.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.73XQyMxsp4S_RCIlodtvQHI7_ejBGbY_0 2024-06-28T16:14:33.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17697035211373118827', time=2024-06-28T17:47:22.026410+0000, chain.objs.size()=1 2024-06-28T16:14:33.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.uHY8iLpSd1cQDg0QicxOwYeBlcwazeP_0 2024-06-28T16:14:33.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12587997197697332853', time=2024-06-28T17:47:23.223580+0000, chain.objs.size()=1 2024-06-28T16:14:33.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-jkLg_2sWtDxYTa5TLvkH5zW1CRdqh8_0 2024-06-28T16:14:33.814 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18042367442707164210', time=2024-06-28T17:47:25.106425+0000, chain.objs.size()=1 2024-06-28T16:14:33.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.COw0PtW9wydCwoxYOXhMbB_r6Q1aaVg_0 2024-06-28T16:14:33.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10575481832653208457', time=2024-06-28T17:47:26.602172+0000, chain.objs.size()=1 2024-06-28T16:14:33.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.p9iloa6jExdeRHaaF-suL-ciBtdf0Yd_0 2024-06-28T16:14:33.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2988753787276996143', time=2024-06-28T17:47:28.137074+0000, chain.objs.size()=1 2024-06-28T16:14:33.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.kYFojw3_uyaPwLRy2HHLGs82-vNBYCd_0 2024-06-28T16:14:33.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16228966127079438457', time=2024-06-28T17:47:29.964538+0000, chain.objs.size()=1 2024-06-28T16:14:33.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oXo0LsXC5-D7HW_JRGWVo3LKcg_R4u6_0 2024-06-28T16:14:33.815 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.465320129746986351', time=2024-06-28T17:47:30.173154+0000, chain.objs.size()=1 2024-06-28T16:14:33.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.hMGsu-0CZGlxS3dnLcz2EdlkiqE-pbt_0 2024-06-28T16:14:33.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17110996713081553593', time=2024-06-28T17:47:33.427357+0000, chain.objs.size()=1 2024-06-28T16:14:33.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.803+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.rGAIj0wvvhnClRwcM84zeX3_Hou-rt5_0 2024-06-28T16:14:33.816 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.807+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9241671903983763527', time=2024-06-28T17:47:34.419530+0000, chain.objs.size()=1 2024-06-28T16:14:33.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.807+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.FRsOThpdWyBUcHV9GPFvwiB82xrAoLJ_0 2024-06-28T16:14:33.817 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.807+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2425545044177600857', time=2024-06-28T17:47:34.766173+0000, chain.objs.size()=1 2024-06-28T16:14:33.818 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.807+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ijSamzz88l5yl5XJao3C72TRUo-lUZB_0 2024-06-28T16:14:33.819 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5279591642810835365', time=2024-06-28T17:47:35.862678+0000, chain.objs.size()=1 2024-06-28T16:14:33.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5bWCJpN0Cg_0RUtUXTOyv-8skWlPh1R_0 2024-06-28T16:14:33.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12989583995828808817', time=2024-06-28T17:47:36.918300+0000, chain.objs.size()=1 2024-06-28T16:14:33.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.8bq7dsrY9v_zg83EztWEv-GPQGebnq__0 2024-06-28T16:14:33.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12769888201137093239', time=2024-06-28T17:47:37.046537+0000, chain.objs.size()=1 2024-06-28T16:14:33.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.BEJwMznbTOeH-UeKpgKeUBpVgaXTXLe_0 2024-06-28T16:14:33.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17857329992470981501', time=2024-06-28T17:47:37.838143+0000, chain.objs.size()=1 2024-06-28T16:14:33.820 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.csq8LzCcO3ICe-YuGmKuwtHfuhyyrNU_0 2024-06-28T16:14:33.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1492749334183833507', time=2024-06-28T17:47:38.323182+0000, chain.objs.size()=1 2024-06-28T16:14:33.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.GGZHJ45zQSH5bjQx60ScTJPgQ67mADw_0 2024-06-28T16:14:33.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15993115049406372076', time=2024-06-28T17:47:38.534370+0000, chain.objs.size()=1 2024-06-28T16:14:33.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.HmMebRXRfdIUV8BUnv0QYjhRakBHm_l_0 2024-06-28T16:14:33.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4549204552873601043', time=2024-06-28T17:47:38.692325+0000, chain.objs.size()=1 2024-06-28T16:14:33.821 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oVv2omtdo0MjY091-uEbAfPsT30pkgM_0 2024-06-28T16:14:33.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8419941992530312008', time=2024-06-28T17:48:14.932067+0000, chain.objs.size()=1 2024-06-28T16:14:33.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.275__shadow_.gvL5Dn55UaE0w_SD5QDrIQK8u8Tzrbx_0 2024-06-28T16:14:33.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5142209162527555053', time=2024-06-28T17:48:21.413304+0000, chain.objs.size()=1 2024-06-28T16:14:33.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.279__shadow_.7yi3RA5hjd4moKuf8IckiDOi3hybP3d_0 2024-06-28T16:14:33.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.557972226454469900', time=2024-06-28T17:48:24.924905+0000, chain.objs.size()=1 2024-06-28T16:14:33.822 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.281__shadow_.1Dqw0YmnenYOqu2cbCEd8gr77Te66W-_0 2024-06-28T16:14:33.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16265380203480464984', time=2024-06-28T17:48:29.386626+0000, chain.objs.size()=1 2024-06-28T16:14:33.823 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.811+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.nK218JRnwIg2Yeq-YDJPwQYWpDWPx44_0 2024-06-28T16:14:33.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15475584749055327140', time=2024-06-28T17:48:34.481192+0000, chain.objs.size()=1 2024-06-28T16:14:33.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.nX_kgjiil11a30KyaaBMyC-O6uOGGJT_0 2024-06-28T16:14:33.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15381742445319210630', time=2024-06-28T17:49:12.019626+0000, chain.objs.size()=1 2024-06-28T16:14:33.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.305__shadow_.jHj_R_Fp3ZsxW3mRPSrK6yP8DH_50Me_0 2024-06-28T16:14:33.824 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.919515597256631810', time=2024-06-28T17:49:16.811505+0000, chain.objs.size()=1 2024-06-28T16:14:33.825 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.309__shadow:yy35DIgsQvQuBsSwOnx-wH7cCoS2xPl_.1FEXUpksdL0NI6kpZsj6rpGjL5Fefhj_0 2024-06-28T16:14:33.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14285059458702050659', time=2024-06-28T17:51:06.797298+0000, chain.objs.size()=1 2024-06-28T16:14:33.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.d07MkbeslBShkQz67ftXg4oGko2VQRz_0 2024-06-28T16:14:33.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4408092497815874888', time=2024-06-28T17:51:07.165413+0000, chain.objs.size()=1 2024-06-28T16:14:33.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.CTkJiF7mjxGfwbcvihIZzblDPE1ailm_0 2024-06-28T16:14:33.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10977527238096457995', time=2024-06-28T17:51:07.341383+0000, chain.objs.size()=1 2024-06-28T16:14:33.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1YRnG1Cks8pRqWuLuDfkQHLRuhUdtcD_0 2024-06-28T16:14:33.826 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8761202262355153881', time=2024-06-28T17:51:07.769755+0000, chain.objs.size()=1 2024-06-28T16:14:33.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fOq1PvvS_MR2zZjTdSTYpQERlw9LA-8_0 2024-06-28T16:14:33.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.388786147082996275', time=2024-06-28T17:51:07.771161+0000, chain.objs.size()=1 2024-06-28T16:14:33.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Bq-tSH25lU1g-BmDCF67IsStnB7ILDr_0 2024-06-28T16:14:33.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15152727102108530941', time=2024-06-28T17:51:09.016961+0000, chain.objs.size()=1 2024-06-28T16:14:33.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yrDPec0hJ3griP9z3CUfgJgLYDIz1yK_0 2024-06-28T16:14:33.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17803585381253159514', time=2024-06-28T17:51:09.818231+0000, chain.objs.size()=1 2024-06-28T16:14:33.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.AYUx66eztQLjmKW84wuAHa2SGLaA1ue_0 2024-06-28T16:14:33.827 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9523875097444820423', time=2024-06-28T17:51:10.010075+0000, chain.objs.size()=1 2024-06-28T16:14:33.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.815+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.qhOBwEzW0JVBoHyFuhmDPua4zWpPC1j_0 2024-06-28T16:14:33.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.819+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2769533135650525433', time=2024-06-28T17:51:11.580003+0000, chain.objs.size()=1 2024-06-28T16:14:33.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.819+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.2CLSeZIP9zgMXVj-qdffE-xS0mWudcH_0 2024-06-28T16:14:33.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.819+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11398941096156538563', time=2024-06-28T17:51:12.251323+0000, chain.objs.size()=1 2024-06-28T16:14:33.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.819+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.LPBY8Jx3goj2PyV16DdQ3dKlA_1Z6IJ_0 2024-06-28T16:14:33.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.819+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5107828459815588742', time=2024-06-28T17:51:13.535833+0000, chain.objs.size()=1 2024-06-28T16:14:33.829 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.819+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.pRD2KYhIvS985DzqUxCzt5Cw-WLo540_0 2024-06-28T16:14:33.830 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1650046234714129657', time=2024-06-28T17:51:16.830315+0000, chain.objs.size()=1 2024-06-28T16:14:33.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.eWZRK2_HD0OdNyUo7Y5Hk-6tDe4GLep_0 2024-06-28T16:14:33.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15928868886092666671', time=2024-06-28T17:51:18.310405+0000, chain.objs.size()=1 2024-06-28T16:14:33.831 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xQbQ1fWOKGOCYCdLalLvqBkjRdchv7W_0 2024-06-28T16:14:33.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.119434298566146198', time=2024-06-28T17:51:20.071732+0000, chain.objs.size()=1 2024-06-28T16:14:33.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.EfSbpT5Df0Y0yBnm-RduThjppuZuMxP_0 2024-06-28T16:14:33.832 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6920224240568636816', time=2024-06-28T17:51:22.270932+0000, chain.objs.size()=1 2024-06-28T16:14:33.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.RkfGAtwdXxVCNlTRSwnCvOOE4yyLSY__0 2024-06-28T16:14:33.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11271183304879123197', time=2024-06-28T17:51:25.404166+0000, chain.objs.size()=1 2024-06-28T16:14:33.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6iS7NTj5a0eNv3rB4JEmmL9Da_XqLJ2_0 2024-06-28T16:14:33.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5832081644208515819', time=2024-06-28T17:51:25.409209+0000, chain.objs.size()=1 2024-06-28T16:14:33.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.XBS7Hjcd8DDMpZm2szqsk1ROyxq3I6R_0 2024-06-28T16:14:33.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13877750937673248071', time=2024-06-28T17:51:25.411313+0000, chain.objs.size()=1 2024-06-28T16:14:33.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GbS23C7iDcoVXwicqN1uCSaEbFr7UtZ_0 2024-06-28T16:14:33.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10290658055126706672', time=2024-06-28T17:51:26.352316+0000, chain.objs.size()=1 2024-06-28T16:14:33.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.UAFNv5cnxoIZroLNZMACB0gf9-DJm12_0 2024-06-28T16:14:33.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1920033313865027552', time=2024-06-28T17:51:28.717056+0000, chain.objs.size()=1 2024-06-28T16:14:33.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.9LBoNrRphDo7JLvmR32LrUJfeUmM0nE_0 2024-06-28T16:14:33.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8346644258689187356', time=2024-06-28T17:51:30.673920+0000, chain.objs.size()=1 2024-06-28T16:14:33.834 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.XocaBNHds6E_yjDc35-kZC9BrLmoOft_0 2024-06-28T16:14:33.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5155651894736527630', time=2024-06-28T17:51:31.875764+0000, chain.objs.size()=1 2024-06-28T16:14:33.835 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.823+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.WyqzSMRqPvaBVubnwSRroPEg939aF11_0 2024-06-28T16:14:33.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15246849901616332852', time=2024-06-28T17:51:34.296567+0000, chain.objs.size()=1 2024-06-28T16:14:33.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.B10dTqubNJnVAOrRJ_88QW3zpWyjwfP_0 2024-06-28T16:14:33.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9359978017481843051', time=2024-06-28T17:51:35.056394+0000, chain.objs.size()=1 2024-06-28T16:14:33.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.WGgr_xeLSmlfodGxPm9djjvb7UBJJaH_0 2024-06-28T16:14:33.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7737978611135707265', time=2024-06-28T17:51:36.158250+0000, chain.objs.size()=1 2024-06-28T16:14:33.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xXSfIjrU5y6dHC4jjBvPVL-Lin333dB_0 2024-06-28T16:14:33.837 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16853466122261536701', time=2024-06-28T17:51:37.236188+0000, chain.objs.size()=1 2024-06-28T16:14:33.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.MZVqx3YVylMZcvXDju_C09yMMZesVc2_0 2024-06-28T16:14:33.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14429457337630599743', time=2024-06-28T17:53:22.162010+0000, chain.objs.size()=1 2024-06-28T16:14:33.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.0UQst9I-BQnKd8B3kanDQ3OAjgySvFV_0 2024-06-28T16:14:33.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11031172102213255844', time=2024-06-28T17:53:22.488556+0000, chain.objs.size()=1 2024-06-28T16:14:33.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.VW5gdQIoB2ZGAK8Z5-XDGyAmONrRX1K_0 2024-06-28T16:14:33.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.462120414201844169', time=2024-06-28T17:53:23.200093+0000, chain.objs.size()=1 2024-06-28T16:14:33.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.QhDx1B-vfIwzqFKsNPdNvytm3QUKSYH_0 2024-06-28T16:14:33.838 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9049189323081629509', time=2024-06-28T17:53:23.461678+0000, chain.objs.size()=1 2024-06-28T16:14:33.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.NjHZIi9hXi37emVtuuNv-7OZT1C76bq_0 2024-06-28T16:14:33.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16106501043943801675', time=2024-06-28T17:53:26.636217+0000, chain.objs.size()=1 2024-06-28T16:14:33.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.HMvfCrZVnXGRlasp3_EDBXwK32t0Cwq_0 2024-06-28T16:14:33.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15177416972800696712', time=2024-06-28T17:53:27.835094+0000, chain.objs.size()=1 2024-06-28T16:14:33.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.YRVFe-pHJZ2XPOgTpz4cbPUZ0RxuH4b_0 2024-06-28T16:14:33.839 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10744940975297299650', time=2024-06-28T17:53:30.523261+0000, chain.objs.size()=1 2024-06-28T16:14:33.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.827+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.IiTF4r_Vz8Wdq1sLqBJLyIvhA1Djgl2_0 2024-06-28T16:14:33.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.831+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9717678582253156981', time=2024-06-28T17:53:31.750853+0000, chain.objs.size()=1 2024-06-28T16:14:33.840 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.831+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.y4DLyQhjKBLPK4PKBEcEL0iNGKWmrDi_0 2024-06-28T16:14:33.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.831+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13262005354790852039', time=2024-06-28T17:53:32.434512+0000, chain.objs.size()=1 2024-06-28T16:14:33.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.831+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.WQh_4ywpo5vYqWMZEnOI8QgEG5W5rbl_0 2024-06-28T16:14:33.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.831+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4173475300888154129', time=2024-06-28T17:53:34.698364+0000, chain.objs.size()=1 2024-06-28T16:14:33.842 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.831+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.YJgJMPM_VDUliDFkUsuVb1Aiv8FKdQ8_0 2024-06-28T16:14:33.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.831+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4741850597215233321', time=2024-06-28T17:53:35.391193+0000, chain.objs.size()=1 2024-06-28T16:14:33.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.831+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.S6HPcPhml4CQdafeutZZ5Wz7grvYneu_0 2024-06-28T16:14:33.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.831+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13673557723265116257', time=2024-06-28T17:53:35.763339+0000, chain.objs.size()=1 2024-06-28T16:14:33.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.831+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.rYsr1CBK79FqXVKehs36r8Z_hxz97tR_0 2024-06-28T16:14:33.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.831+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1916723384301619698', time=2024-06-28T17:53:36.718402+0000, chain.objs.size()=1 2024-06-28T16:14:33.843 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.831+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Gm5Jj2EwMvXThVC2w1ygBUmtz1wRs3t_0 2024-06-28T16:14:33.844 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15108484295625601916', time=2024-06-28T17:53:36.859235+0000, chain.objs.size()=1 2024-06-28T16:14:33.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-LL5CJgjS85pOzzvaU092m4zu2k5mll_0 2024-06-28T16:14:33.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5651225379502093459', time=2024-06-28T17:53:36.872094+0000, chain.objs.size()=1 2024-06-28T16:14:33.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qMHN81FX0DUHW8Ymjk74pskpVXEbUzG_0 2024-06-28T16:14:33.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4411123099847843773', time=2024-06-28T17:53:38.733943+0000, chain.objs.size()=1 2024-06-28T16:14:33.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vzVDyXNzVSNjg-CMCs0z_qYoXpAPT1P_0 2024-06-28T16:14:33.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4560036538716687118', time=2024-06-28T17:53:39.710143+0000, chain.objs.size()=1 2024-06-28T16:14:33.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.UoIJtDZYF0V479HXEgj7qVw-UdURD8f_0 2024-06-28T16:14:33.845 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14775025570674428014', time=2024-06-28T17:53:40.122360+0000, chain.objs.size()=1 2024-06-28T16:14:33.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qfoSsHoOV7czpzJp8WWVKcORUQ7DFf6_0 2024-06-28T16:14:33.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2179361510741854343', time=2024-06-28T17:53:40.661238+0000, chain.objs.size()=1 2024-06-28T16:14:33.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.B0A6ySTHMsJe5NF_6Xt4a7KN3a2hfNE_0 2024-06-28T16:14:33.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3890127004350495437', time=2024-06-28T17:53:42.244625+0000, chain.objs.size()=1 2024-06-28T16:14:33.846 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Ia31C8hCZCkHAtl4Q0HCPcLg3zmGpP3_0 2024-06-28T16:14:33.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16388475932336665623', time=2024-06-28T17:53:45.940740+0000, chain.objs.size()=1 2024-06-28T16:14:33.847 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.835+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.yqPw2DF7ikh-zS8Ys4hZeOYARioB71v_0 2024-06-28T16:14:33.851 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/43351 2024-06-28T16:14:33.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=32, truncated=0, next_marker='' 2024-06-28T16:14:33.853 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15414646279757058016', time=2024-06-28T17:53:46.294364+0000, chain.objs.size()=1 2024-06-28T16:14:33.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.SNb9XFAmRadfSrVeb8d7IeRpZv9vlFt_0 2024-06-28T16:14:33.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9764863056331777996', time=2024-06-28T17:53:46.491027+0000, chain.objs.size()=1 2024-06-28T16:14:33.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.AEfC5nbyKUpQlnmfzKDA9u_3_UsxN_4_0 2024-06-28T16:14:33.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11070088186294714412', time=2024-06-28T17:53:46.775503+0000, chain.objs.size()=1 2024-06-28T16:14:33.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gTPUmuSZGR0F-i5vTZ4_7RhL5tMo8Hp_0 2024-06-28T16:14:33.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1482287302538803454', time=2024-06-28T17:53:47.263762+0000, chain.objs.size()=1 2024-06-28T16:14:33.854 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bftbGW0Q7P3f5db3tNHIKOzhS0RrE5s_0 2024-06-28T16:14:33.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16071636255974687398', time=2024-06-28T17:53:48.713743+0000, chain.objs.size()=1 2024-06-28T16:14:33.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.p70_Iap8ZaSFxb_ScIMWz-Lwe4ywyPl_0 2024-06-28T16:14:33.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11045623062262474162', time=2024-06-28T17:53:48.831006+0000, chain.objs.size()=1 2024-06-28T16:14:33.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BEHJG9ZSJEoCuQOBhFxuSIyxKa3pep1_0 2024-06-28T16:14:33.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11936917774091862385', time=2024-06-28T17:53:48.975184+0000, chain.objs.size()=1 2024-06-28T16:14:33.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ANSBtdkLai3Px4VEHykLH5BFFWzQSOz_0 2024-06-28T16:14:33.855 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13094210883630529139', time=2024-06-28T17:53:49.947939+0000, chain.objs.size()=1 2024-06-28T16:14:33.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.uHw4Lqc92Qjizf1NhWOVCjrC76JL5_w_0 2024-06-28T16:14:33.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17851835073539031178', time=2024-06-28T17:53:49.957441+0000, chain.objs.size()=1 2024-06-28T16:14:33.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BG-EQipvwcfXpgFMfEdXrEIdoiCpNZO_0 2024-06-28T16:14:33.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2984890378126802081', time=2024-06-28T17:53:50.917668+0000, chain.objs.size()=1 2024-06-28T16:14:33.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.5INFfmWpYPygoHprduIM1aNIxMOBI-V_0 2024-06-28T16:14:33.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11326627671295614138', time=2024-06-28T17:53:51.181312+0000, chain.objs.size()=1 2024-06-28T16:14:33.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.sifJaK1fs_u8bxmV7ev9fWySz-8d1Td_0 2024-06-28T16:14:33.856 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13576548775490816959', time=2024-06-28T17:53:52.208405+0000, chain.objs.size()=1 2024-06-28T16:14:33.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.843+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.JQGYKo-0w1azS3WQseCLKzMj5P9Jge-_0 2024-06-28T16:14:33.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.847+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15207009881234909246', time=2024-06-28T17:53:52.210303+0000, chain.objs.size()=1 2024-06-28T16:14:33.857 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.847+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.3ngQ43kEU1xImBKwurssBlnVM5wi3RA_0 2024-06-28T16:14:33.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.847+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3127197472089180447', time=2024-06-28T17:54:04.031068+0000, chain.objs.size()=1 2024-06-28T16:14:33.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.847+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.326__shadow_.H8HdFcPV2LB9jTkEuoDdZ7a78iGl2fE_0 2024-06-28T16:14:33.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.847+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5203052375148230456', time=2024-06-28T17:54:28.073580+0000, chain.objs.size()=1 2024-06-28T16:14:33.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.847+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.357__shadow_.bBducpJI46WViZ8ORMHH_zl5oCM5HEK_0 2024-06-28T16:14:33.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.847+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11722319456128225902', time=2024-06-28T17:55:59.879774+0000, chain.objs.size()=1 2024-06-28T16:14:33.858 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.847+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.448__shadow_.yeQn1Mfz3zP6yqg4WcJRCJYIParGSUs_0 2024-06-28T16:14:33.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.847+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5948485669310245635', time=2024-06-28T17:56:29.646450+0000, chain.objs.size()=1 2024-06-28T16:14:33.859 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.847+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.474__shadow_.1mRO7kP-g3UJihdL3RJKk3M6JavU82d_0 2024-06-28T16:14:33.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.851+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.728699697898673288', time=2024-06-28T17:56:29.667537+0000, chain.objs.size()=1 2024-06-28T16:14:33.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.474__shadow_.cRwrChwJh7MNNIFefHQ-aKTc3FT6bp2_0 2024-06-28T16:14:33.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.851+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.2_b5c0f02ba0716f16e0e18267386033e7', time=2024-06-28T17:57:09.903919+0000, chain.objs.size()=6 2024-06-28T16:14:33.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.1 2024-06-28T16:14:33.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.2 2024-06-28T16:14:33.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.3 2024-06-28T16:14:33.861 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.4 2024-06-28T16:14:33.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.5 2024-06-28T16:14:33.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.6 2024-06-28T16:14:33.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.851+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4907581303260101955', time=2024-06-28T17:58:10.037030+0000, chain.objs.size()=3 2024-06-28T16:14:33.862 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.510__shadow:kxdzUyHtwty0o9EmuzAaKfmgkph1geu_.zVaJ20Phwpc0PsQv9DdklmcGsHUW6Gr_0 2024-06-28T16:14:33.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.851+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.510__shadow:kxdzUyHtwty0o9EmuzAaKfmgkph1geu_.zVaJ20Phwpc0PsQv9DdklmcGsHUW6Gr_1 2024-06-28T16:14:33.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.855+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.510__shadow:kxdzUyHtwty0o9EmuzAaKfmgkph1geu_.zVaJ20Phwpc0PsQv9DdklmcGsHUW6Gr_2 2024-06-28T16:14:33.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.855+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10189601330993322356', time=2024-06-28T18:01:11.443763+0000, chain.objs.size()=1 2024-06-28T16:14:33.863 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.855+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.540__shadow_.h-hptqdye3gEBHN0AkV0ReXA7nvHS3u_0 2024-06-28T16:14:33.865 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.855+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16546843722622925601', time=2024-06-28T18:02:00.996300+0000, chain.objs.size()=1 2024-06-28T16:14:33.866 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.855+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:bXDXRaCUIGwYCuRl.IdYPoud44U5LXu_.4tZLekMzYOh-gDEwvSOVZXtiNOpD5qr_0 2024-06-28T16:14:33.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17326939813008946330', time=2024-06-28T18:02:04.097685+0000, chain.objs.size()=1 2024-06-28T16:14:33.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:T-Pm1tCBQiGdMlvGQ.kL4B7OpPkPZ95_.Hj8U7uYVY8-x9dnZWyFDHgFh-XejxqE_0 2024-06-28T16:14:33.867 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14319209285787798587', time=2024-06-28T18:02:11.532545+0000, chain.objs.size()=1 2024-06-28T16:14:33.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:CdUiDgbSEc3T8P.IyBIHKf0s6IHD1IK_.Fo5VbPGkIyq4Tu-ZjDP063P0s6oRxSy_0 2024-06-28T16:14:33.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4987401483459993782', time=2024-06-28T18:02:55.876362+0000, chain.objs.size()=1 2024-06-28T16:14:33.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:qHfbZPYX16KPUhr2o1ITFpzWEnVdXcz_.VugoR_MU0EOmqtCuingPrEFkddMoyk3_0 2024-06-28T16:14:33.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13529264962336241070', time=2024-06-28T18:03:06.207500+0000, chain.objs.size()=1 2024-06-28T16:14:33.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:9ADdswURtvR1zi939J2SEYsJA9y8cow_.E-UtOvOathFq-Fg7bdjafWaOeEMLxvm_0 2024-06-28T16:14:33.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11097401048400332885', time=2024-06-28T18:03:11.008920+0000, chain.objs.size()=1 2024-06-28T16:14:33.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:2jTlfn06kTe8n.ht4ojRc83C6LE0xTr_.1EfMzVOdhjd3huVqGB76-VBcwXl5sJD_0 2024-06-28T16:14:33.868 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11207736110518988242', time=2024-06-28T18:03:25.557824+0000, chain.objs.size()=1 2024-06-28T16:14:33.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:RQmKeGCykJISIfMSxFBtMqzofqv7NIa_.sPlQR13kJaWTC-hQeaEzpLvLOmPoMsE_0 2024-06-28T16:14:33.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10534834380018929128', time=2024-06-28T18:03:56.178567+0000, chain.objs.size()=1 2024-06-28T16:14:33.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.568__shadow:OKoIw1lcTe.cRz2TaKaOAGH-CHqPSJX_.m4IIDs0DYx7kKdQInNagsJp71aFvfH6_0 2024-06-28T16:14:33.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:b9JE41FCZ0Y5wSk5sOO07Df6tJ9KFRg_.W4vEkcA5YdpwymBiS2unbImypMQZuyR_1_ec3867e95b7813ce54b9b3474170e7cc', time=2024-06-28T18:04:01.232202+0000, chain.objs.size()=1 2024-06-28T16:14:33.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:b9JE41FCZ0Y5wSk5sOO07Df6tJ9KFRg_.W4vEkcA5YdpwymBiS2unbImypMQZuyR_0 2024-06-28T16:14:33.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15743441179189145278', time=2024-06-28T18:07:04.558926+0000, chain.objs.size()=2 2024-06-28T16:14:33.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.627__shadow_.14dTjsm1m87l-o64cPcTtOnI2IGIH9H_0 2024-06-28T16:14:33.869 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.627__shadow_.14dTjsm1m87l-o64cPcTtOnI2IGIH9H_1 2024-06-28T16:14:33.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13505158928297265115', time=2024-06-28T18:10:13.725833+0000, chain.objs.size()=1 2024-06-28T16:14:33.870 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.859+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.724__shadow_.89eaDPk-U4NdpByhWZQcJMq0mN4gD49_0 2024-06-28T16:14:33.875 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.867+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:33.877 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.867+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-06-28T16:14:33.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:33.879 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:33.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/46076' 2024-06-28T16:14:33.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9284504644489148292', time=2024-06-28T17:40:37.371424+0000, chain.objs.size()=1 2024-06-28T16:14:33.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.152__shadow_.NsZIdz9t1RKJDyLbXtp44CGE2MyBbYG_0 2024-06-28T16:14:33.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_tpMSfF2la29SbLK0-WwbJ-bKruwhCGl', time=2024-06-28T17:40:41.501040+0000, chain.objs.size()=1 2024-06-28T16:14:33.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.155__shadow_.7agVPgWl8acOIhBC7oNykcmcl9kDz7e_0 2024-06-28T16:14:33.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~LR_FuA-aBDvWHp7EoxXlO94V3e2zLeE', time=2024-06-28T17:41:50.491105+0000, chain.objs.size()=1 2024-06-28T16:14:33.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.191__multipart_mymultipart.2~LR_FuA-aBDvWHp7EoxXlO94V3e2zLeE.1 2024-06-28T16:14:33.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11758785963752812424', time=2024-06-28T17:44:42.226704+0000, chain.objs.size()=1 2024-06-28T16:14:33.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.225__shadow_.b-cpjzwJ-hghWxQsTWsm7uKxpP-JRH-_0 2024-06-28T16:14:33.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7151073276433738616', time=2024-06-28T17:44:54.034402+0000, chain.objs.size()=1 2024-06-28T16:14:33.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.232__shadow_.RjbuflYgwy_wQ3eYI3uMH9bH2_HewVG_0 2024-06-28T16:14:33.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10251619073889323820', time=2024-06-28T17:47:08.289878+0000, chain.objs.size()=1 2024-06-28T16:14:33.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Qw-mLBUECB8QRWG5FkK56F4hhusVrGR_0 2024-06-28T16:14:33.882 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16388674971268668281', time=2024-06-28T17:47:08.371078+0000, chain.objs.size()=1 2024-06-28T16:14:33.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.MoQ8mUKy4qgjzZpat4Kc5wEznIAQTor_0 2024-06-28T16:14:33.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2156333004382742439', time=2024-06-28T17:47:10.061156+0000, chain.objs.size()=1 2024-06-28T16:14:33.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.sMYRm-9r5hQL-PxOIAtW52doMJHAt32_0 2024-06-28T16:14:33.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17693346394009770471', time=2024-06-28T17:47:10.067440+0000, chain.objs.size()=1 2024-06-28T16:14:33.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.riNcROsEQHAPL2Mc2hI6z9_pDfTEkpf_0 2024-06-28T16:14:33.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10173928271386243574', time=2024-06-28T17:47:10.069428+0000, chain.objs.size()=1 2024-06-28T16:14:33.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.yDCU6ka6htmW00OfYb8r8raxsgbkHsF_0 2024-06-28T16:14:33.883 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12681512961104534383', time=2024-06-28T17:47:11.095687+0000, chain.objs.size()=1 2024-06-28T16:14:33.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.K_asnSCFIlBwv8j-0jVCfmTN8pNGvao_0 2024-06-28T16:14:33.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14799677795683652695', time=2024-06-28T17:47:12.342738+0000, chain.objs.size()=1 2024-06-28T16:14:33.884 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.871+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.M5_M2V-NAIU15NeGxbLt5NqltaZjzVv_0 2024-06-28T16:14:33.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17781012467115168366', time=2024-06-28T17:47:12.874787+0000, chain.objs.size()=1 2024-06-28T16:14:33.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.6Ck8LiQIxGAAcIeU0xNVZTjTQWbkcp2_0 2024-06-28T16:14:33.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1849942463925557196', time=2024-06-28T17:47:13.215950+0000, chain.objs.size()=1 2024-06-28T16:14:33.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9cNnix3mlTUp_kVMHb7arxpQD--nuyk_0 2024-06-28T16:14:33.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6528073437020688435', time=2024-06-28T17:47:17.984104+0000, chain.objs.size()=1 2024-06-28T16:14:33.885 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.FHFbfL8Q0oBLxWwygKlvFbCV2apN4jf_0 2024-06-28T16:14:33.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7174467723312559885', time=2024-06-28T17:47:21.336982+0000, chain.objs.size()=1 2024-06-28T16:14:33.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7T4-mjZ6TD4kd8F4zr1u5rwTzvDKPHX_0 2024-06-28T16:14:33.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3057777129773699340', time=2024-06-28T17:47:23.817754+0000, chain.objs.size()=1 2024-06-28T16:14:33.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.V5a2RbLAinjQVUyyQbRkmFxUVEgkYSd_0 2024-06-28T16:14:33.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14552105583227390154', time=2024-06-28T17:47:27.131502+0000, chain.objs.size()=1 2024-06-28T16:14:33.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.NXXT0ysAFIceUqPyP4d8WYgky-DKKQa_0 2024-06-28T16:14:33.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11587000015105263548', time=2024-06-28T17:47:27.377940+0000, chain.objs.size()=1 2024-06-28T16:14:33.886 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.skVGScscUji2hD4exFk33NHAweOnr3z_0 2024-06-28T16:14:33.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12337885396302693965', time=2024-06-28T17:47:28.182524+0000, chain.objs.size()=1 2024-06-28T16:14:33.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ONx4j5zCm9gfrxE2WQH1hOHHlheyK5__0 2024-06-28T16:14:33.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10111658222957003240', time=2024-06-28T17:47:28.184113+0000, chain.objs.size()=1 2024-06-28T16:14:33.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.J-x1N7wHIDbIIsk1vToG_wuA9sCKDeH_0 2024-06-28T16:14:33.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13657979035565444560', time=2024-06-28T17:47:28.874225+0000, chain.objs.size()=1 2024-06-28T16:14:33.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.rNIxE-m2Me2uIrn4oPpM4OG3K4u3ubp_0 2024-06-28T16:14:33.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14420700661865403262', time=2024-06-28T17:47:30.442127+0000, chain.objs.size()=1 2024-06-28T16:14:33.887 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.875+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.39SpqgeMFxDZsUMSRcnHno77EF6qkXq_0 2024-06-28T16:14:33.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.879+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17892091382588284734', time=2024-06-28T17:47:31.888959+0000, chain.objs.size()=1 2024-06-28T16:14:33.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.879+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Nk1Kz90_O0reoJlRirb9PIN6qfPN1HK_0 2024-06-28T16:14:33.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.879+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10710002676240161924', time=2024-06-28T17:47:32.100388+0000, chain.objs.size()=1 2024-06-28T16:14:33.889 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.879+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.LajFlytbLBtG9R6RlPXMJq_BDSsXP9y_0 2024-06-28T16:14:33.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.879+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2850075683462829200', time=2024-06-28T17:47:34.593295+0000, chain.objs.size()=1 2024-06-28T16:14:33.890 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.879+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Nf7Xgatd0Zjw_YmclR6yUtVOhUZZBJR_0 2024-06-28T16:14:33.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7531326933913845698', time=2024-06-28T17:47:34.887007+0000, chain.objs.size()=1 2024-06-28T16:14:33.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.95s9fBgrAovkde4sdqD2subSecymhV6_0 2024-06-28T16:14:33.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5262840471403769316', time=2024-06-28T17:47:35.231045+0000, chain.objs.size()=1 2024-06-28T16:14:33.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Vc3eNJDq7DO9YYeZQchn3P_rF7-239E_0 2024-06-28T16:14:33.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.804147399670952885', time=2024-06-28T17:47:35.414360+0000, chain.objs.size()=1 2024-06-28T16:14:33.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.YgDPvttG8YWBnocLfQ8eCcXdFSWm6sc_0 2024-06-28T16:14:33.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9654727550692734470', time=2024-06-28T17:47:36.082382+0000, chain.objs.size()=1 2024-06-28T16:14:33.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.eaU4DSyzkt3qmfP1-9ePk6NybRoDUAR_0 2024-06-28T16:14:33.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11885017852878265516', time=2024-06-28T17:47:37.040346+0000, chain.objs.size()=1 2024-06-28T16:14:33.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.alRllkP2-EjU18Ws9up-cziKkizyaPy_0 2024-06-28T16:14:33.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.527540010418144152', time=2024-06-28T17:47:37.838881+0000, chain.objs.size()=1 2024-06-28T16:14:33.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.WV34BkqvIQyCbLvP_53bOK7wfmVQQNu_0 2024-06-28T16:14:33.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6286052770935821374', time=2024-06-28T17:47:37.850764+0000, chain.objs.size()=1 2024-06-28T16:14:33.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.XOgIsevVT2Q4NR4_IDkfRtwE2_eYra4_0 2024-06-28T16:14:33.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11842172027652620622', time=2024-06-28T17:47:38.142349+0000, chain.objs.size()=1 2024-06-28T16:14:33.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.DwktzouywbJRpYhluYKUBuhg-CpxHsM_0 2024-06-28T16:14:33.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16933141752058419839', time=2024-06-28T17:47:38.423623+0000, chain.objs.size()=1 2024-06-28T16:14:33.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.viO8wnRnF1Vgn_NfIQy1TWcoWertrT__0 2024-06-28T16:14:33.894 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2202107060293761708', time=2024-06-28T17:47:38.722411+0000, chain.objs.size()=1 2024-06-28T16:14:33.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Jj5OPRj3D7U37Eji0Vp23BvSf_k0K9n_0 2024-06-28T16:14:33.897 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10319002658055030783', time=2024-06-28T17:47:45.574105+0000, chain.objs.size()=1 2024-06-28T16:14:33.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.883+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.256__shadow_.qDs-F4BNEqBROVInf39l61DgkpBfP-Y_0 2024-06-28T16:14:33.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15092282630929484025', time=2024-06-28T17:47:48.485341+0000, chain.objs.size()=1 2024-06-28T16:14:33.898 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.258__shadow_.FEnkS0P7WqYPpoKz-zu509JvCzjbwPy_0 2024-06-28T16:14:33.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15939856943678691780', time=2024-06-28T17:47:54.446563+0000, chain.objs.size()=1 2024-06-28T16:14:33.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.262__shadow_.7kpLKHavCU8ClS75Op74Lsp_0mFSDoq_0 2024-06-28T16:14:33.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10478103897056056987', time=2024-06-28T17:48:29.563546+0000, chain.objs.size()=1 2024-06-28T16:14:33.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.AJe9Uf582CYQAsWlj1oMdBHWpjF3pKW_0 2024-06-28T16:14:33.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3739161776976207401', time=2024-06-28T17:48:41.332373+0000, chain.objs.size()=1 2024-06-28T16:14:33.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.287__shadow_.BufE_4TxLdiaGu3UXlJDUtI4B6OyKCm_0 2024-06-28T16:14:33.899 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16662229375092286510', time=2024-06-28T17:48:49.549320+0000, chain.objs.size()=1 2024-06-28T16:14:33.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.292__shadow_.jlxGZBE8Ygv3Ob6GF5CAFfhCMP2DWaF_0 2024-06-28T16:14:33.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8890834306128063975', time=2024-06-28T17:48:51.156298+0000, chain.objs.size()=1 2024-06-28T16:14:33.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.293__shadow_.xNhCEAe3UBTJ4mn1o5XNxCFV5JMXJiB_0 2024-06-28T16:14:33.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10195134095325755898', time=2024-06-28T17:51:06.794010+0000, chain.objs.size()=1 2024-06-28T16:14:33.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.mtScbz88cb2QPT8YS5bU0iwsuTvCduj_0 2024-06-28T16:14:33.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11512434878036705138', time=2024-06-28T17:51:07.405004+0000, chain.objs.size()=1 2024-06-28T16:14:33.900 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.887+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.cEgd04-YKMWdo2oTIcLgEq3S54QWtsP_0 2024-06-28T16:14:33.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.891+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10542769862348350256', time=2024-06-28T17:51:07.881037+0000, chain.objs.size()=1 2024-06-28T16:14:33.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.891+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.w6XhYUFlSaV05p0AWeAmnZDevO7r2Hy_0 2024-06-28T16:14:33.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.891+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3292713670756300270', time=2024-06-28T17:51:09.229898+0000, chain.objs.size()=1 2024-06-28T16:14:33.901 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.891+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.w4OzYPQPZHk_z3wKmpXUIUFLxrVCfju_0 2024-06-28T16:14:33.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14892849160148205709', time=2024-06-28T17:51:09.736834+0000, chain.objs.size()=1 2024-06-28T16:14:33.903 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.5ks-sCEHy0OcYpSI56D3ou2gIB_7gJU_0 2024-06-28T16:14:33.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17458002476486219541', time=2024-06-28T17:51:15.782082+0000, chain.objs.size()=1 2024-06-28T16:14:33.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.E_uRIUvaJ8Bt8ZSevrsgLeoVP6tTTqr_0 2024-06-28T16:14:33.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8867313255507519738', time=2024-06-28T17:51:15.922260+0000, chain.objs.size()=1 2024-06-28T16:14:33.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.bUS6rQ4Ucl-PExgN0ml_RtxfLHo2TG4_0 2024-06-28T16:14:33.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5411903036999086693', time=2024-06-28T17:51:16.245201+0000, chain.objs.size()=1 2024-06-28T16:14:33.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.KCbsrTPr-VMnqgpESkZyfRgDzKd6ZRd_0 2024-06-28T16:14:33.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13412859004484103554', time=2024-06-28T17:51:17.191344+0000, chain.objs.size()=1 2024-06-28T16:14:33.904 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.tK0wUKxvjJ-BXsFNYKuDKUDHd0sOlLr_0 2024-06-28T16:14:33.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10755404845202717471', time=2024-06-28T17:51:17.824477+0000, chain.objs.size()=1 2024-06-28T16:14:33.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.9J07NaJ1CqMOJuNUlfe2BOTIvCUFr7O_0 2024-06-28T16:14:33.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11735444957422764060', time=2024-06-28T17:51:19.751801+0000, chain.objs.size()=1 2024-06-28T16:14:33.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Oh5mX7wqzWbxTqA90Ox38r6-pZai17B_0 2024-06-28T16:14:33.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13449732502099558536', time=2024-06-28T17:51:20.444125+0000, chain.objs.size()=1 2024-06-28T16:14:33.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.aqsgDSJiAggHXNrZ48INlCTwDEecJHB_0 2024-06-28T16:14:33.905 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17431506682468006208', time=2024-06-28T17:51:22.270780+0000, chain.objs.size()=1 2024-06-28T16:14:33.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.seWmymvlsdrKS33nsB2xmGLjdEle_7m_0 2024-06-28T16:14:33.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11696550891663450101', time=2024-06-28T17:51:22.558304+0000, chain.objs.size()=1 2024-06-28T16:14:33.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.30UvK-8jJwKaOan0Sb7G1hHRKo0QQKr_0 2024-06-28T16:14:33.906 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10888862736195381398', time=2024-06-28T17:51:23.628784+0000, chain.objs.size()=1 2024-06-28T16:14:33.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.dE1Lr4slVsc7emcLzaTYjvLeERkzxtx_0 2024-06-28T16:14:33.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11452885031923940263', time=2024-06-28T17:51:23.966518+0000, chain.objs.size()=1 2024-06-28T16:14:33.907 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.895+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.0WwUmOEFiQdsjURW6Weq9CmjQyD0Uu8_0 2024-06-28T16:14:33.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5092957865292817894', time=2024-06-28T17:51:24.654205+0000, chain.objs.size()=1 2024-06-28T16:14:33.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.rU8zh05lNn-TQkYsUWOgJv615S4KRz7_0 2024-06-28T16:14:33.910 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17835971120469184773', time=2024-06-28T17:51:25.407966+0000, chain.objs.size()=1 2024-06-28T16:14:33.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YmZBLb12T4anUhn2nx0DoRYbrgZvi3R_0 2024-06-28T16:14:33.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9497162985705942788', time=2024-06-28T17:51:25.409837+0000, chain.objs.size()=1 2024-06-28T16:14:33.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.60o8AjNW3g6HVu1kuZCL1EPspUSnlbC_0 2024-06-28T16:14:33.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16243086861688964447', time=2024-06-28T17:51:27.756393+0000, chain.objs.size()=1 2024-06-28T16:14:33.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.zVtG-9ulKGjSbop2AYgfQLPDxLuSm4K_0 2024-06-28T16:14:33.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6767526182193702720', time=2024-06-28T17:51:27.763803+0000, chain.objs.size()=1 2024-06-28T16:14:33.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.AhMRfmEmufn5UZZwQTYGM-qeSj2P5_Y_0 2024-06-28T16:14:33.911 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5461155856811628223', time=2024-06-28T17:51:27.922274+0000, chain.objs.size()=1 2024-06-28T16:14:33.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.cBEVhCJZTmwfV6hWoET7ZqG5envEvI0_0 2024-06-28T16:14:33.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10569936163990552838', time=2024-06-28T17:51:28.280622+0000, chain.objs.size()=1 2024-06-28T16:14:33.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.y2BBKP3W8bKIZgAY_6NIGI6ZTZxYy6Q_0 2024-06-28T16:14:33.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9794980878301767738', time=2024-06-28T17:51:28.390287+0000, chain.objs.size()=1 2024-06-28T16:14:33.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.WcWYAWK3wL8ksM_TEA_xdyJJvu8_g_Z_0 2024-06-28T16:14:33.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12803924719656439528', time=2024-06-28T17:51:29.190570+0000, chain.objs.size()=1 2024-06-28T16:14:33.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.gNCh9yXXrk6qSh9y-pdzue_uvsAuc3H_0 2024-06-28T16:14:33.912 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7216358005326589293', time=2024-06-28T17:51:29.192069+0000, chain.objs.size()=1 2024-06-28T16:14:33.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hNL16FcjUJ_CLS1psRwMVcmIyTQr5Cf_0 2024-06-28T16:14:33.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15572340723423746685', time=2024-06-28T17:51:31.049202+0000, chain.objs.size()=1 2024-06-28T16:14:33.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.899+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.k2tfJkcMzCDMDwIKAKl3StNYtWtFzfP_0 2024-06-28T16:14:33.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.903+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8819365031482740667', time=2024-06-28T17:51:31.881413+0000, chain.objs.size()=1 2024-06-28T16:14:33.913 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.903+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yPBd7mQC8TtuZCfEfwDi5Cnc1dQXhLj_0 2024-06-28T16:14:33.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8335976399863222712', time=2024-06-28T17:51:32.044867+0000, chain.objs.size()=1 2024-06-28T16:14:33.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.LNRLsloF0Q-9u2QQTaWTxx1k2rIfBv4_0 2024-06-28T16:14:33.915 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7480558612294018967', time=2024-06-28T17:51:33.247866+0000, chain.objs.size()=1 2024-06-28T16:14:33.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.o_iT-5ZeEzkX7VWIcPNFgTFlx3I-6Vm_0 2024-06-28T16:14:33.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8401987484276843356', time=2024-06-28T17:51:36.310301+0000, chain.objs.size()=1 2024-06-28T16:14:33.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.3lr5q8fKKH4tYksrpMbqohSArJLqsPh_0 2024-06-28T16:14:33.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8300817627672077741', time=2024-06-28T17:51:36.313018+0000, chain.objs.size()=1 2024-06-28T16:14:33.916 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YImXe9V-p4M59rYpsLVFz0CB3nRsJ9X_0 2024-06-28T16:14:33.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8662985753601060929', time=2024-06-28T17:51:36.314686+0000, chain.objs.size()=1 2024-06-28T16:14:33.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nqXgy8QgU_xb5DnFLcXf7InlswAD4cp_0 2024-06-28T16:14:33.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9582315673673525454', time=2024-06-28T17:51:37.484780+0000, chain.objs.size()=1 2024-06-28T16:14:33.917 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.OFwRiuayIFeQ39DZxzRY0V7WaVUowzj_0 2024-06-28T16:14:33.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17783600946048857711', time=2024-06-28T17:51:38.254284+0000, chain.objs.size()=1 2024-06-28T16:14:33.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.S_LtXKF_JPuqRg2oW_K9Gv2IfovkG03_0 2024-06-28T16:14:33.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9003126711829724730', time=2024-06-28T17:53:20.944250+0000, chain.objs.size()=1 2024-06-28T16:14:33.918 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.tbT97RxtcfIXq2ADgP8S88dWVXx3HS5_0 2024-06-28T16:14:33.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16163452405918266678', time=2024-06-28T17:53:23.061215+0000, chain.objs.size()=1 2024-06-28T16:14:33.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.IonZKReBEcFKUxhrrzBNXjfo49nsdfp_0 2024-06-28T16:14:33.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3580913630382685412', time=2024-06-28T17:53:23.463475+0000, chain.objs.size()=1 2024-06-28T16:14:33.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.0btQnIuLtNMEP4AaXNM79AFVZ1llvei_0 2024-06-28T16:14:33.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5513858545907113705', time=2024-06-28T17:53:24.085022+0000, chain.objs.size()=1 2024-06-28T16:14:33.919 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.907+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.JJ1quM6OIATESRDnJ4JWLfrZK6Zr9qa_0 2024-06-28T16:14:33.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17552457734834168094', time=2024-06-28T17:53:25.203619+0000, chain.objs.size()=1 2024-06-28T16:14:33.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.SGXEvziiy-aKSnUYJkQoI4BpRneliVB_0 2024-06-28T16:14:33.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11549698374619461250', time=2024-06-28T17:53:26.647623+0000, chain.objs.size()=1 2024-06-28T16:14:33.920 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.XbPyGfJ7ObvQ6XGL2y0KL8ObAReRuxP_0 2024-06-28T16:14:33.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15707323908825761493', time=2024-06-28T17:53:26.902134+0000, chain.objs.size()=1 2024-06-28T16:14:33.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.5g46V4Lj9gzDWBZZ4j05HAPE_ghSJIo_0 2024-06-28T16:14:33.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17406582508738899066', time=2024-06-28T17:53:27.470147+0000, chain.objs.size()=1 2024-06-28T16:14:33.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.S9RAiiuptf3FPTx1rZvu4ni_xNjz4df_0 2024-06-28T16:14:33.921 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.565223579564528400', time=2024-06-28T17:53:29.560326+0000, chain.objs.size()=1 2024-06-28T16:14:33.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.2RrDfM0CCQ7DLHAiIHXGUXYzhc47lxq_0 2024-06-28T16:14:33.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.911+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8931576137112251748', time=2024-06-28T17:53:31.202506+0000, chain.objs.size()=1 2024-06-28T16:14:33.922 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.911+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CTwPBdnm7NrzjPCi2Xl66dQExduyh8r_0 2024-06-28T16:14:33.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13972221282211878949', time=2024-06-28T17:53:31.580349+0000, chain.objs.size()=1 2024-06-28T16:14:33.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BpYGT1uaIaGYFwEYKQ8QN3SElcAfqpD_0 2024-06-28T16:14:33.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14412849303571227503', time=2024-06-28T17:53:32.272781+0000, chain.objs.size()=1 2024-06-28T16:14:33.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.MLDzLYgpSrKWXFnYlU5OUFVhbPIcEQf_0 2024-06-28T16:14:33.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6900905534799799531', time=2024-06-28T17:53:33.767118+0000, chain.objs.size()=1 2024-06-28T16:14:33.926 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.J6FxMrOZXQ0dgiAO9eyVteNDGo5CIW4_0 2024-06-28T16:14:33.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3034246183471084485', time=2024-06-28T17:53:35.253033+0000, chain.objs.size()=1 2024-06-28T16:14:33.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.G-YUTKb9StfKjeDlF9qFsN3E5QMVG_p_0 2024-06-28T16:14:33.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4582982561730899745', time=2024-06-28T17:53:36.869271+0000, chain.objs.size()=1 2024-06-28T16:14:33.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.px0TzBAouVIhBUDExWsb27j4XDDCA7r_0 2024-06-28T16:14:33.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11687473132368006519', time=2024-06-28T17:53:37.827892+0000, chain.objs.size()=1 2024-06-28T16:14:33.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.hw3ZFvNXii1N4uiNjdL_WS5xoXKOpCT_0 2024-06-28T16:14:33.927 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17730374146582781249', time=2024-06-28T17:53:38.654509+0000, chain.objs.size()=1 2024-06-28T16:14:33.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.TUBWRvqI2mmBMlV6oB6_yNicYXxws8J_0 2024-06-28T16:14:33.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7611159556731362207', time=2024-06-28T17:53:39.387283+0000, chain.objs.size()=1 2024-06-28T16:14:33.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.zOHMw7etlv0YQ-mJjoGK7F0cLEzareO_0 2024-06-28T16:14:33.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4976740606158001841', time=2024-06-28T17:53:40.955821+0000, chain.objs.size()=1 2024-06-28T16:14:33.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Q3DB8bIAbajivgBjdG5ObsbkIvFbwgN_0 2024-06-28T16:14:33.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2779785317846272426', time=2024-06-28T17:53:42.782259+0000, chain.objs.size()=1 2024-06-28T16:14:33.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.6VS549wMCQlaof3b2gcbgX-x4YTXBz0_0 2024-06-28T16:14:33.928 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16353707790477954964', time=2024-06-28T17:53:43.115265+0000, chain.objs.size()=1 2024-06-28T16:14:33.929 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.915+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.kGrbkXyHl76jSJjdBxzmQI31WTzsiBb_0 2024-06-28T16:14:33.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.919+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6123256650030150736', time=2024-06-28T17:53:44.811527+0000, chain.objs.size()=1 2024-06-28T16:14:33.930 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.919+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.MKQKcpojjppUXuVtGH01Q9w2NFyVUr8_0 2024-06-28T16:14:33.933 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.923+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/46076 2024-06-28T16:14:33.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=26, truncated=0, next_marker='' 2024-06-28T16:14:33.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2202422732122867249', time=2024-06-28T17:53:45.855452+0000, chain.objs.size()=1 2024-06-28T16:14:33.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.eH6UygyZ9VFydjSxSnC2cYliiX-SllG_0 2024-06-28T16:14:33.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8664884687319245180', time=2024-06-28T17:53:47.721544+0000, chain.objs.size()=1 2024-06-28T16:14:33.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.94L1jDlyCth3Mvcko2Jqq28Y8_t3cmX_0 2024-06-28T16:14:33.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1010543135641308653', time=2024-06-28T17:53:48.922739+0000, chain.objs.size()=1 2024-06-28T16:14:33.936 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CU4AJThOADPYcwjI9j8kgP1VI5KVHri_0 2024-06-28T16:14:33.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5236288052093594473', time=2024-06-28T17:53:48.939914+0000, chain.objs.size()=1 2024-06-28T16:14:33.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.rWW7Z4nUOukjCwR65Kp7nTJdO5OargB_0 2024-06-28T16:14:33.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3665988250266283363', time=2024-06-28T17:53:50.790064+0000, chain.objs.size()=1 2024-06-28T16:14:33.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.yV0zIK3pdf0cbhAjrieYMI3qNriAemC_0 2024-06-28T16:14:33.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5631086517540216426', time=2024-06-28T17:53:51.675860+0000, chain.objs.size()=1 2024-06-28T16:14:33.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GIvD2_dyfHQgjKS90RXFU5az9VqkxhY_0 2024-06-28T16:14:33.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16696830991175869737', time=2024-06-28T17:53:51.810700+0000, chain.objs.size()=1 2024-06-28T16:14:33.937 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.o4531qfcrqdkzLpJnuk-CrdLhpTFYFO_0 2024-06-28T16:14:33.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13249309903347063461', time=2024-06-28T17:54:10.350774+0000, chain.objs.size()=1 2024-06-28T16:14:33.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.333__shadow_.VQ1vcc1mOZLS18uvpAWEVQLqcUQIB8H_0 2024-06-28T16:14:33.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13245386329824161345', time=2024-06-28T17:55:58.220680+0000, chain.objs.size()=1 2024-06-28T16:14:33.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.447__shadow_.VIblmdR_AqftTgCWU3bpOKsdS4eAFFq_0 2024-06-28T16:14:33.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14822707425185501518', time=2024-06-28T17:57:27.553373+0000, chain.objs.size()=1 2024-06-28T16:14:33.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.508__multipart_mymultipart.2~2XEvek8YKH3_ffV9fGCzHujYryCB5DZ.1 2024-06-28T16:14:33.938 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7348271953019649', time=2024-06-28T17:58:08.380303+0000, chain.objs.size()=3 2024-06-28T16:14:33.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.511__multipart_mymultipart.2~3BsJhT6gW8nMGmqVX-eC95RrLS7p58V.1 2024-06-28T16:14:33.939 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.927+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.511__multipart_mymultipart.2~3BsJhT6gW8nMGmqVX-eC95RrLS7p58V.2 2024-06-28T16:14:33.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.511__multipart_mymultipart.2~3BsJhT6gW8nMGmqVX-eC95RrLS7p58V.3 2024-06-28T16:14:33.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4367932444031925698', time=2024-06-28T17:59:45.055950+0000, chain.objs.size()=2 2024-06-28T16:14:33.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.513__multipart_mymultipart.2~7Bti7399lIUfxAjpEEYBM0pBN1xmckQ.1 2024-06-28T16:14:33.940 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.513__multipart_mymultipart.2~7Bti7399lIUfxAjpEEYBM0pBN1xmckQ.2 2024-06-28T16:14:33.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15355520824831349066', time=2024-06-28T17:59:51.802482+0000, chain.objs.size()=2 2024-06-28T16:14:33.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.515__multipart_mymultipart.2~C21gVB1hYzCOiNwfKmSaMNWVVawVSWn.1 2024-06-28T16:14:33.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.515__multipart_mymultipart.2~C21gVB1hYzCOiNwfKmSaMNWVVawVSWn.2 2024-06-28T16:14:33.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1411034476611655275', time=2024-06-28T18:00:14.201025+0000, chain.objs.size()=2 2024-06-28T16:14:33.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.518__multipart_mymultipart.2~ykn_Vkq2qqqNGbC0Xs7engEtxmtEbgk.1 2024-06-28T16:14:33.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.518__multipart_mymultipart.2~ykn_Vkq2qqqNGbC0Xs7engEtxmtEbgk.2 2024-06-28T16:14:33.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3171394644187466416', time=2024-06-28T18:01:14.992228+0000, chain.objs.size()=1 2024-06-28T16:14:33.941 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.541__shadow_.EjJeFIb1ooYcdCO85YzM2KZfXA4gUxh_0 2024-06-28T16:14:33.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2607651575057842035', time=2024-06-28T18:01:35.769161+0000, chain.objs.size()=1 2024-06-28T16:14:33.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.547__shadow_.WoHpNPXOoY4D_5mmbfAu_Ao8H9vlnQf_0 2024-06-28T16:14:33.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.96106969666482991', time=2024-06-28T18:01:44.357855+0000, chain.objs.size()=1 2024-06-28T16:14:33.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.550__multipart_mymultipart.0T9j0adq4yAEbx4cDGJiZJVOwhSh7iHd.1 2024-06-28T16:14:33.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.23904686544003128', time=2024-06-28T18:02:30.009645+0000, chain.objs.size()=1 2024-06-28T16:14:33.942 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.931+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.563__shadow_.FKAkIT2MB60mfI6xgE1s7qiXifiqi9W_0 2024-06-28T16:14:33.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.935+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1573289120310554716', time=2024-06-28T18:02:47.071374+0000, chain.objs.size()=1 2024-06-28T16:14:33.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.935+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.565__shadow:H7I7gyFr8RwZtnl2oTCyJ9PO6MAqK42_.OzpFywDOEReV1eCymsmJNDkFz6hU4EJ_0 2024-06-28T16:14:33.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.935+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8428673559720212013', time=2024-06-28T18:02:59.123448+0000, chain.objs.size()=1 2024-06-28T16:14:33.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.935+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:LzmnBbHrUvKyh7XzVk3CTmwFApVJVu-_.w-8uAf9yCQp4QLRh56mJU_ipenZc878_0 2024-06-28T16:14:33.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.935+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16178952778571418371', time=2024-06-28T18:04:00.530369+0000, chain.objs.size()=1 2024-06-28T16:14:33.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.935+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:b9JE41FCZ0Y5wSk5sOO07Df6tJ9KFRg_.W4vEkcA5YdpwymBiS2unbImypMQZuyR_0 2024-06-28T16:14:33.948 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.939+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3428061886137832371', time=2024-06-28T18:06:52.023189+0000, chain.objs.size()=1 2024-06-28T16:14:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.939+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.616__shadow_.zq3j0UQZ8qgUn6FfmQKmozq4ib2C5SA_0 2024-06-28T16:14:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.939+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4543898214546430462', time=2024-06-28T18:07:59.210129+0000, chain.objs.size()=1 2024-06-28T16:14:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.939+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.648__shadow_.Bpc9wmA0NviLiqFuZKcudc1goeVcT9s_0 2024-06-28T16:14:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.939+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12097625014864264116', time=2024-06-28T18:08:32.623918+0000, chain.objs.size()=1 2024-06-28T16:14:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.939+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.681__shadow:tzwYMFCZJJ-Jv3sJnU6lMWeTSR76AXb_.fOB3OBEcOKVeXNeqRlHfDNrIWiPLtQN_0 2024-06-28T16:14:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.939+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17213694726713773938', time=2024-06-28T18:08:42.784275+0000, chain.objs.size()=1 2024-06-28T16:14:33.949 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.939+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.689__shadow:KhzF25YYaVyD9H8KsqD-6FSVluV8oyE_.jpzYkYCHOyxCYesqSs6m-GI_yjs_aUQ_0 2024-06-28T16:14:33.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.939+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10196380236151204503', time=2024-06-28T18:09:05.701017+0000, chain.objs.size()=1 2024-06-28T16:14:33.950 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.939+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.700__shadow:1SfKOr6PNtqM0QHpQj5OCbe2HVEwDr4_.XU0TezSZwB9b8orAzeoVq9Pk1zpoIn2_0 2024-06-28T16:14:33.952 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.943+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:33.954 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-06-28T16:14:33.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:33.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:33.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/53726' 2024-06-28T16:14:33.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10322847128398729838', time=2024-06-28T17:39:51.022085+0000, chain.objs.size()=1 2024-06-28T16:14:33.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.137__shadow_.kwTEGL0zouILwiHEhSEfSnq1p8LIY0Z_0 2024-06-28T16:14:33.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6717327807194135801', time=2024-06-28T17:44:48.485565+0000, chain.objs.size()=1 2024-06-28T16:14:33.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.228__shadow_.Sk73Qtw-tUvn_I8OcfpTVKTZ4FCkqg4_0 2024-06-28T16:14:33.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6038107438865818283', time=2024-06-28T17:45:17.892473+0000, chain.objs.size()=1 2024-06-28T16:14:33.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.246__shadow_.gfi6msP582t4T57M8OfX40v80ZfdERi_0 2024-06-28T16:14:33.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7977430282558511856', time=2024-06-28T17:45:20.810584+0000, chain.objs.size()=1 2024-06-28T16:14:33.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.248__shadow_.Byv16x6qWZNhqxVEWjiohBnjWv7617h_0 2024-06-28T16:14:33.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13746550219981836495', time=2024-06-28T17:47:08.575908+0000, chain.objs.size()=1 2024-06-28T16:14:33.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ZfukVCKhzFs_DoARFZrAcFwxAQF6Wse_0 2024-06-28T16:14:33.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4383831521619198177', time=2024-06-28T17:47:08.783669+0000, chain.objs.size()=1 2024-06-28T16:14:33.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.MOrUL8-tM2-lc_27HnzewpEdYj_z0xJ_0 2024-06-28T16:14:33.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8822301852169927042', time=2024-06-28T17:47:09.437571+0000, chain.objs.size()=1 2024-06-28T16:14:33.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.s_ZvL2MuR2gvVOjwVv6FDJC2E9cVAuB_0 2024-06-28T16:14:33.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5530215395428483541', time=2024-06-28T17:47:09.642257+0000, chain.objs.size()=1 2024-06-28T16:14:33.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.TtazVose7nd5c41R0Z0M28vhp2PYjhw_0 2024-06-28T16:14:33.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3218393795419494890', time=2024-06-28T17:47:12.704091+0000, chain.objs.size()=1 2024-06-28T16:14:33.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7puuCydZ8pDrRbUKPMYWy6Ch61Z_JyP_0 2024-06-28T16:14:33.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4815885576551925103', time=2024-06-28T17:47:13.897061+0000, chain.objs.size()=1 2024-06-28T16:14:33.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7yGIQ1tO3Wo2fWyl5gRVePNUNuEXttv_0 2024-06-28T16:14:33.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6795741086832167896', time=2024-06-28T17:47:17.650756+0000, chain.objs.size()=1 2024-06-28T16:14:33.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.liKSFbkRIpxJyoEC7mUrtgi6WRhY3QN_0 2024-06-28T16:14:33.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11029631141788100328', time=2024-06-28T17:47:17.920290+0000, chain.objs.size()=1 2024-06-28T16:14:33.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.947+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oTF3pbiRft6n2ofFoPdE4FkYvtyZyuI_0 2024-06-28T16:14:33.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.951+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.762194062748622491', time=2024-06-28T17:47:18.112120+0000, chain.objs.size()=1 2024-06-28T16:14:33.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.951+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.6XT722MYFzuZQbuRvPBfUYNzdD4dJAw_0 2024-06-28T16:14:33.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.951+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7978157655994479739', time=2024-06-28T17:47:19.542354+0000, chain.objs.size()=1 2024-06-28T16:14:33.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.951+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.h_4bG4aZjpkoPuTicIrP-aDr0Pp1N8p_0 2024-06-28T16:14:33.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.951+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7847515200373380009', time=2024-06-28T17:47:19.810148+0000, chain.objs.size()=1 2024-06-28T16:14:33.962 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.951+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ze1z1wK1n4ca-n8nyH2HKufgK2-v0-n_0 2024-06-28T16:14:33.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.951+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7280971595230334345', time=2024-06-28T17:47:19.885502+0000, chain.objs.size()=1 2024-06-28T16:14:33.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.951+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.nLbcjaazDM0yUb2HCDyQKYIkxbFuorY_0 2024-06-28T16:14:33.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.951+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7116192143838676556', time=2024-06-28T17:47:21.010076+0000, chain.objs.size()=1 2024-06-28T16:14:33.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.951+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.eSvGcvCt1GaHlNbR86mTbmzFfyFBFKd_0 2024-06-28T16:14:33.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.951+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13171999597121763014', time=2024-06-28T17:47:21.440347+0000, chain.objs.size()=1 2024-06-28T16:14:33.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.951+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.tIvOKFgXiuvMUH5jh3HHjpDc4r5Vatu_0 2024-06-28T16:14:33.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.951+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14769528175674455382', time=2024-06-28T17:47:21.873746+0000, chain.objs.size()=1 2024-06-28T16:14:33.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.951+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.F6Bbx8on7J1rsQCguqRRaCpb2_s_n1q_0 2024-06-28T16:14:33.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.955+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6244438245159977596', time=2024-06-28T17:47:23.301726+0000, chain.objs.size()=1 2024-06-28T16:14:33.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.955+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Gb9kAGXHasfmaQurcnbn6J9qkYDsgiy_0 2024-06-28T16:14:33.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.955+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15435212337541582592', time=2024-06-28T17:47:23.763608+0000, chain.objs.size()=1 2024-06-28T16:14:33.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.955+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.L3BiStsCnwwdcNGDlqaB0XQ8CwZLbld_0 2024-06-28T16:14:33.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.955+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9873913889556950340', time=2024-06-28T17:47:24.134678+0000, chain.objs.size()=1 2024-06-28T16:14:33.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.955+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4tKTVtqN9QyAd8sM_199JgNDlxrKbY8_0 2024-06-28T16:14:33.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.955+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4655176121706556329', time=2024-06-28T17:47:25.127338+0000, chain.objs.size()=1 2024-06-28T16:14:33.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.955+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.6NqlJz_prfydHvScNW8k2L-9iFSVhIw_0 2024-06-28T16:14:33.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.955+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4462432363946812098', time=2024-06-28T17:47:25.553580+0000, chain.objs.size()=1 2024-06-28T16:14:33.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.955+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.T4sFL7n7UtPVYrba6c124kaDPTWxCPG_0 2024-06-28T16:14:33.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.959+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.880698517241057448', time=2024-06-28T17:47:26.331788+0000, chain.objs.size()=1 2024-06-28T16:14:33.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.959+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.1gZM9giQgbrezhHKTs85ITlrI2Fic_b_0 2024-06-28T16:14:33.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.959+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12997592231178576679', time=2024-06-28T17:47:26.333188+0000, chain.objs.size()=1 2024-06-28T16:14:33.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.959+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5tJ5vs-Z1zR2Uqg_cEX6crJuvE1neYK_0 2024-06-28T16:14:33.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.959+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2583787529959928738', time=2024-06-28T17:47:26.334379+0000, chain.objs.size()=1 2024-06-28T16:14:33.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.959+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.2uUCuzF3eUn5Rte5RpM5xM6y65x7ent_0 2024-06-28T16:14:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.959+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5416031151350135247', time=2024-06-28T17:47:26.620592+0000, chain.objs.size()=1 2024-06-28T16:14:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.959+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5aL5zl3y6LLhFpfr7nAFn0HaQzwzBNC_0 2024-06-28T16:14:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.959+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11315057312519223108', time=2024-06-28T17:47:29.066806+0000, chain.objs.size()=1 2024-06-28T16:14:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.959+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.nJU_gTItP8sj71Ogyc7Bt8s6Y-QVg-9_0 2024-06-28T16:14:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.959+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16864369699327241243', time=2024-06-28T17:47:31.400834+0000, chain.objs.size()=1 2024-06-28T16:14:33.968 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.959+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.OO3DVlqGdV1HYhhp8Z7XLW_5vNIKLXS_0 2024-06-28T16:14:33.971 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3266882154463228421', time=2024-06-28T17:47:31.820321+0000, chain.objs.size()=1 2024-06-28T16:14:33.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.OiDyu92WmggVFbwT53sTk2fxJc2lirH_0 2024-06-28T16:14:33.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12813061449314322490', time=2024-06-28T17:47:37.319208+0000, chain.objs.size()=1 2024-06-28T16:14:33.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.J53WLtYVrZYRrZPUg7mvknZzkdVbW1V_0 2024-06-28T16:14:33.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.132559183528054345', time=2024-06-28T17:47:37.427528+0000, chain.objs.size()=1 2024-06-28T16:14:33.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ebWMpYm3Ge48K11oGWFJ1mOXeH5sNK8_0 2024-06-28T16:14:33.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17534359773550422236', time=2024-06-28T17:47:38.728781+0000, chain.objs.size()=1 2024-06-28T16:14:33.972 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.smFIO-OyR3ufC0-cPrSZSHpVIAiTDRK_0 2024-06-28T16:14:33.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15669544629879171135', time=2024-06-28T17:47:58.944347+0000, chain.objs.size()=1 2024-06-28T16:14:33.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.265__shadow_.FsjjqnS2Cq1dNSFnqmo1KOrBGRJ7ovP_0 2024-06-28T16:14:33.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8806253267748600350', time=2024-06-28T17:48:10.120694+0000, chain.objs.size()=1 2024-06-28T16:14:33.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.272__shadow_.UCeOhGLBX4k8nZRGV4oiCC655Oaa6Xe_0 2024-06-28T16:14:33.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14310209697656344658', time=2024-06-28T17:48:11.672840+0000, chain.objs.size()=1 2024-06-28T16:14:33.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.273__shadow_.g27bRdsvprpGqajLCEu8wC0jIsi9AQ8_0 2024-06-28T16:14:33.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14118008263218550107', time=2024-06-28T17:48:29.585497+0000, chain.objs.size()=1 2024-06-28T16:14:33.973 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.KSFGUH9-TVEMVwuvUwd8d8XXvZZwBUe_0 2024-06-28T16:14:33.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16174087837487391022', time=2024-06-28T17:48:29.780098+0000, chain.objs.size()=1 2024-06-28T16:14:33.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.6hWHFbKYptmGo1To1_wu9DaaG2UDstB_0 2024-06-28T16:14:33.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5251284606107918777', time=2024-06-28T17:51:06.782355+0000, chain.objs.size()=1 2024-06-28T16:14:33.974 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nXqOu5ApbkaEv61BFQ20bI9CoYVOpfE_0 2024-06-28T16:14:33.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15172004337496103537', time=2024-06-28T17:51:06.784210+0000, chain.objs.size()=1 2024-06-28T16:14:33.975 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.963+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Aa2zFJueVdeL5a4hR49FF_4Q5JkvKB1_0 2024-06-28T16:14:33.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6727479640988715010', time=2024-06-28T17:51:07.617484+0000, chain.objs.size()=1 2024-06-28T16:14:33.976 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.w9LEcVuruPw5AojhOQHAREwyDLtUGCq_0 2024-06-28T16:14:33.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16278727930269380035', time=2024-06-28T17:51:07.619629+0000, chain.objs.size()=1 2024-06-28T16:14:33.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.E-UKAk9EGDN94SC8Pp4JFOPp-TtvmI6_0 2024-06-28T16:14:33.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16717453283407256571', time=2024-06-28T17:51:08.636868+0000, chain.objs.size()=1 2024-06-28T16:14:33.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nUgfgGmxzuU7X_CWb_8DuYoKex-0rjQ_0 2024-06-28T16:14:33.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17727658937204859015', time=2024-06-28T17:51:09.016586+0000, chain.objs.size()=1 2024-06-28T16:14:33.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.5Hrv_7OworpVhhFQOMIc9Q4LjA6OCIx_0 2024-06-28T16:14:33.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4467436235425145829', time=2024-06-28T17:51:09.457022+0000, chain.objs.size()=1 2024-06-28T16:14:33.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.tyggWcQKc4Njg651YTYbJvt9U4qPq6U_0 2024-06-28T16:14:33.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13159071290056338387', time=2024-06-28T17:51:11.040060+0000, chain.objs.size()=1 2024-06-28T16:14:33.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.0ULTyv4b86BsUNZawp8D7gZ_zHG-7g9_0 2024-06-28T16:14:33.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17036627037209243143', time=2024-06-28T17:51:12.270191+0000, chain.objs.size()=1 2024-06-28T16:14:33.978 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.VPztmAKx_b4OYHlT8Bq54lCvU7u-n36_0 2024-06-28T16:14:33.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14824185122394795301', time=2024-06-28T17:51:12.434422+0000, chain.objs.size()=1 2024-06-28T16:14:33.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.TBXH7Rv7_PZR3O3oyD-YhPr6__OZfRP_0 2024-06-28T16:14:33.979 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16405364979715332437', time=2024-06-28T17:51:13.100280+0000, chain.objs.size()=1 2024-06-28T16:14:33.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.OJhcBKRFRVqKSaCHa2QV2yJ9x_o6nFb_0 2024-06-28T16:14:33.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11234519796300463914', time=2024-06-28T17:51:13.354820+0000, chain.objs.size()=1 2024-06-28T16:14:33.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.e7YzdJMjMnsAXUZXDL0FzaJ6grfZW3y_0 2024-06-28T16:14:33.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11859363513962584785', time=2024-06-28T17:51:14.220024+0000, chain.objs.size()=1 2024-06-28T16:14:33.980 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.967+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GWdjKTVudgtUTH_D-zOQu2mxVICaetj_0 2024-06-28T16:14:33.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.971+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14096137336913688735', time=2024-06-28T17:51:15.479895+0000, chain.objs.size()=1 2024-06-28T16:14:33.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.971+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.CjUtc91SMbPlXN5sKHVuXY3B9CxotWB_0 2024-06-28T16:14:33.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.971+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7703831857123107580', time=2024-06-28T17:51:16.490022+0000, chain.objs.size()=1 2024-06-28T16:14:33.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.971+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.iQEVYBpbatoP-Sl4NhMqTALXrTvdN8__0 2024-06-28T16:14:33.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.971+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18284037896565639678', time=2024-06-28T17:51:16.840496+0000, chain.objs.size()=1 2024-06-28T16:14:33.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.971+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.QasXlXohL0lN8qOrvE-p3yqe4Mh0Qza_0 2024-06-28T16:14:33.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4478756371743737399', time=2024-06-28T17:51:17.843402+0000, chain.objs.size()=1 2024-06-28T16:14:33.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.4c2bpWYWEnQFTocySLmMV1tM8hmM6wv_0 2024-06-28T16:14:33.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9363532648279385702', time=2024-06-28T17:51:19.808258+0000, chain.objs.size()=1 2024-06-28T16:14:33.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.qzq97EUSuhru8M5NMPJ7qjXVTw6Drjj_0 2024-06-28T16:14:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12476339408444573552', time=2024-06-28T17:51:20.435314+0000, chain.objs.size()=1 2024-06-28T16:14:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.JVkurRd4PT6YHqH86ybyskko6KwMs2S_0 2024-06-28T16:14:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10420799550432098762', time=2024-06-28T17:51:21.181835+0000, chain.objs.size()=1 2024-06-28T16:14:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.AKSCFvJ9l7NjaErgLLHgleENk2lcjd8_0 2024-06-28T16:14:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4463914693314583002', time=2024-06-28T17:51:21.183810+0000, chain.objs.size()=1 2024-06-28T16:14:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.TLX-l0DblyKnHz47_r92-b46YhNLaaY_0 2024-06-28T16:14:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8995712485842792545', time=2024-06-28T17:51:21.828325+0000, chain.objs.size()=1 2024-06-28T16:14:33.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.7xuw5DoVMnghNSH6qz9slJBniYx5y75_0 2024-06-28T16:14:33.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2257950193921886572', time=2024-06-28T17:51:23.487691+0000, chain.objs.size()=1 2024-06-28T16:14:33.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.g-LbybChGcsRrv2TV1NcaZSmdzesXpS_0 2024-06-28T16:14:33.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10773644075508763028', time=2024-06-28T17:51:23.643187+0000, chain.objs.size()=1 2024-06-28T16:14:33.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.zk5HQUQs6LANB4qxaplUHPJHBynKVE7_0 2024-06-28T16:14:33.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6267755145797441035', time=2024-06-28T17:51:23.645117+0000, chain.objs.size()=1 2024-06-28T16:14:33.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.4PpYr2VOakKpc5ehD2az1CtS3s0qOHg_0 2024-06-28T16:14:33.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3452285077064021741', time=2024-06-28T17:51:24.155662+0000, chain.objs.size()=1 2024-06-28T16:14:33.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.KGt_ps9GdwXaw9WZx071KrfyhtDsqcS_0 2024-06-28T16:14:33.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6104293540271357521', time=2024-06-28T17:51:24.386352+0000, chain.objs.size()=1 2024-06-28T16:14:33.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.975+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Q9EZiAKyd4JrgaCL9CgPYHZ8wiMlB9q_0 2024-06-28T16:14:33.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.979+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1033303640386370710', time=2024-06-28T17:51:24.666908+0000, chain.objs.size()=1 2024-06-28T16:14:33.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.979+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6JhbImN8YVAMrbA5QV-XpSNtpMr5rha_0 2024-06-28T16:14:33.988 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.979+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17691097789235308783', time=2024-06-28T17:51:24.955071+0000, chain.objs.size()=1 2024-06-28T16:14:33.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.979+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Siw4JSbpGijUeDpa0cc0D4jbp2LIYW2_0 2024-06-28T16:14:33.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.979+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3104963835105291022', time=2024-06-28T17:51:25.265563+0000, chain.objs.size()=1 2024-06-28T16:14:33.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.979+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.mzI2n5Oce9pnixUzW2eWYKYK_CvZN-b_0 2024-06-28T16:14:33.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.979+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10665989751090758349', time=2024-06-28T17:51:28.680978+0000, chain.objs.size()=1 2024-06-28T16:14:33.989 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.979+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.R3rZy1cveiDK9JIcLkLP_cEPH0LGTCq_0 2024-06-28T16:14:33.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.979+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18349410632174554748', time=2024-06-28T17:51:28.781336+0000, chain.objs.size()=1 2024-06-28T16:14:33.990 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.979+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.DIvfS5umGUUwBU5sn4M-yrg_0sijlSs_0 2024-06-28T16:14:33.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15974520132895280229', time=2024-06-28T17:51:29.144815+0000, chain.objs.size()=1 2024-06-28T16:14:33.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.G_0fHnNPc-V4nns09k5-yr8yfc0s6oJ_0 2024-06-28T16:14:33.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10698982590832380773', time=2024-06-28T17:51:30.045953+0000, chain.objs.size()=1 2024-06-28T16:14:33.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.vWlEExNoJZDkNGMjIDFaIZUJT8CwaNT_0 2024-06-28T16:14:33.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5737841838138532203', time=2024-06-28T17:51:31.060662+0000, chain.objs.size()=1 2024-06-28T16:14:33.991 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GfLCqlRiqbFhFaEQrxi4e4pYWCvF3h7_0 2024-06-28T16:14:33.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1246807824049140093', time=2024-06-28T17:51:32.429416+0000, chain.objs.size()=1 2024-06-28T16:14:33.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GiVH939dHw-RBd2WUWhSeCaQdaLDiDL_0 2024-06-28T16:14:33.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8189930582774451917', time=2024-06-28T17:51:32.940732+0000, chain.objs.size()=1 2024-06-28T16:14:33.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.cbwX7sQd8ZH4mKTl4ZoGbgaOc0HOWHx_0 2024-06-28T16:14:33.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2447244271378726714', time=2024-06-28T17:51:33.523617+0000, chain.objs.size()=1 2024-06-28T16:14:33.992 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ChMqV9QWDnQfFxsOJPrTTU5zX1CmobN_0 2024-06-28T16:14:33.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7468944940580991380', time=2024-06-28T17:51:34.610616+0000, chain.objs.size()=1 2024-06-28T16:14:33.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.mOgvYww684F78edfl1fQ2CWa1hPj4oK_0 2024-06-28T16:14:33.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7647142004868806199', time=2024-06-28T17:51:35.056239+0000, chain.objs.size()=1 2024-06-28T16:14:33.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.983+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.4KeT6HCVUCQUOz3x3hlwcxJu2W-1Zt3_0 2024-06-28T16:14:33.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3722111583311047920', time=2024-06-28T17:51:35.964078+0000, chain.objs.size()=1 2024-06-28T16:14:33.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.VCM9rA1DU6Nb27Ogwb74_BFl9idfqTi_0 2024-06-28T16:14:33.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14687848781924757992', time=2024-06-28T17:51:36.239039+0000, chain.objs.size()=1 2024-06-28T16:14:33.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.iapBf_OFpkNGqVB-T1sd7BTom-Zl03a_0 2024-06-28T16:14:33.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9037743942389101720', time=2024-06-28T17:51:37.475638+0000, chain.objs.size()=1 2024-06-28T16:14:33.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.g13On4Cm9BEMEnM07JLnk8SpUbe-hWy_0 2024-06-28T16:14:33.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1417866467980735582', time=2024-06-28T17:53:22.751111+0000, chain.objs.size()=1 2024-06-28T16:14:33.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.iFx4r7GKq9SabaxsuVAwcVitRLBMNCh_0 2024-06-28T16:14:33.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13340061981848633994', time=2024-06-28T17:53:24.162435+0000, chain.objs.size()=1 2024-06-28T16:14:33.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.guV8u5UJx5mYrLOJLg2QFKIap-gpHTa_0 2024-06-28T16:14:33.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4016410446371823247', time=2024-06-28T17:53:25.274064+0000, chain.objs.size()=1 2024-06-28T16:14:33.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.3yeK3lBHVb8MNoZvohZSJA3rabXl24X_0 2024-06-28T16:14:33.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4348567549823362095', time=2024-06-28T17:53:26.245476+0000, chain.objs.size()=1 2024-06-28T16:14:33.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.D-QBhLEuNyixZid9xXuvMVfx4KBy6Cz_0 2024-06-28T16:14:33.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16690182087105427204', time=2024-06-28T17:53:26.646256+0000, chain.objs.size()=1 2024-06-28T16:14:33.998 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.y34cZyRhzVpPQiphpmBaDax8ZQ3Gb-H_0 2024-06-28T16:14:33.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8640353330078033504', time=2024-06-28T17:53:27.094409+0000, chain.objs.size()=1 2024-06-28T16:14:33.999 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.987+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.EQtkK6Ry73lYJcGFhfxe0ZaMbabTLvN_0 2024-06-28T16:14:34.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11897602087421093677', time=2024-06-28T17:53:27.852995+0000, chain.objs.size()=1 2024-06-28T16:14:34.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.KYJe8cVtGX6H6hUu4qlgPmljxl3ggZK_0 2024-06-28T16:14:34.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12273881522580114638', time=2024-06-28T17:53:29.593400+0000, chain.objs.size()=1 2024-06-28T16:14:34.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.alKlom5LhDr6qgeCQU1YJwnBZQBY6Qp_0 2024-06-28T16:14:34.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13488830694219448310', time=2024-06-28T17:53:31.589738+0000, chain.objs.size()=1 2024-06-28T16:14:34.001 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.98cF7lnPd77XdmsOJZ5CMEdYfQjpjd7_0 2024-06-28T16:14:34.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7016259812287542110', time=2024-06-28T17:53:32.289066+0000, chain.objs.size()=1 2024-06-28T16:14:34.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-KlzGT_8xsIYjmz11t8kBVFHt2yWmCY_0 2024-06-28T16:14:34.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8907035796876645173', time=2024-06-28T17:53:36.200618+0000, chain.objs.size()=1 2024-06-28T16:14:34.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ErfOSLY24MBNKHRhn-cDYDvISD4x4N4_0 2024-06-28T16:14:34.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13455341683088128436', time=2024-06-28T17:53:39.097408+0000, chain.objs.size()=1 2024-06-28T16:14:34.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.DEC2AWRkApZPiHoN3nQS73Tr-ZyvGbd_0 2024-06-28T16:14:34.002 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.991+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3766263980024872033', time=2024-06-28T17:53:39.908095+0000, chain.objs.size()=1 2024-06-28T16:14:34.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.991+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.YNYkD3L6fb0oTwFZQZryaC-6rqsw1l2_0 2024-06-28T16:14:34.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14722348975856946935', time=2024-06-28T17:53:40.238516+0000, chain.objs.size()=1 2024-06-28T16:14:34.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CWRpuTk_2Bj2aPgMj2roKHHwT9hkmP2_0 2024-06-28T16:14:34.003 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4975623512929791718', time=2024-06-28T17:53:42.931712+0000, chain.objs.size()=1 2024-06-28T16:14:34.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.OMJSz37LWgFR3Wr96yzqn4ZDs70Z7l6_0 2024-06-28T16:14:34.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15166993559656312958', time=2024-06-28T17:53:43.812076+0000, chain.objs.size()=1 2024-06-28T16:14:34.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.JJWRk5Bw5fxsDlQOroY8Ncg78RHoln4_0 2024-06-28T16:14:34.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6619567874216387885', time=2024-06-28T17:53:44.548049+0000, chain.objs.size()=1 2024-06-28T16:14:34.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.zdn0tACFv7mJx13XrSDfjJgS_44YTdD_0 2024-06-28T16:14:34.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.995+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3936413889812226077', time=2024-06-28T17:53:45.021298+0000, chain.objs.size()=1 2024-06-28T16:14:34.004 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.995+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wZaKBLbIeklWbg2WB6k5J9eJ5XDtJ0t_0 2024-06-28T16:14:34.009 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:33.999+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/53726 2024-06-28T16:14:34.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=28, truncated=0, next_marker='' 2024-06-28T16:14:34.012 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2808218786490858936', time=2024-06-28T17:53:46.840671+0000, chain.objs.size()=1 2024-06-28T16:14:34.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gBn0CFTgrMcvSspXerEZnjw3Gnc4YQ8_0 2024-06-28T16:14:34.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12753366945423526276', time=2024-06-28T17:53:47.756062+0000, chain.objs.size()=1 2024-06-28T16:14:34.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BwYcjb54dTuE1s8Pksba3ObSAXCBJOk_0 2024-06-28T16:14:34.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.874278893203996719', time=2024-06-28T17:53:47.960626+0000, chain.objs.size()=1 2024-06-28T16:14:34.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.tJrETFcZ-ze1pjKnzlBtPZvrH70kwaP_0 2024-06-28T16:14:34.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7930154612725636937', time=2024-06-28T17:53:48.737982+0000, chain.objs.size()=1 2024-06-28T16:14:34.013 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BdHG1VIMF5QRAP13AGHjAF3FJ1wFSn4_0 2024-06-28T16:14:34.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11228560072370432659', time=2024-06-28T17:53:49.712144+0000, chain.objs.size()=1 2024-06-28T16:14:34.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.s6eXthSPKQ3NPYY8nunDnwYQlty_4A-_0 2024-06-28T16:14:34.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9894338854414212363', time=2024-06-28T17:53:50.717725+0000, chain.objs.size()=1 2024-06-28T16:14:34.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.DDlyb-uxf4POj2tEAZNejgcduOlgxMS_0 2024-06-28T16:14:34.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2546660075383386529', time=2024-06-28T17:56:12.946030+0000, chain.objs.size()=1 2024-06-28T16:14:34.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.456__shadow_.HjpNbt_Jv8X6lpNYa_DBHEm9wqt4mBq_0 2024-06-28T16:14:34.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14827796623299334298', time=2024-06-28T17:56:28.107873+0000, chain.objs.size()=1 2024-06-28T16:14:34.014 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.473__shadow_.AOR5Of7afu1oxviozqDc9BMot24k9hy_0 2024-06-28T16:14:34.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7230975467528430060', time=2024-06-28T17:57:18.821730+0000, chain.objs.size()=1 2024-06-28T16:14:34.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.505__shadow_.e653Rct7lxeg1uEVegtvBl3E6PI7yJJ_0 2024-06-28T16:14:34.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5920946681849410815', time=2024-06-28T18:01:09.583857+0000, chain.objs.size()=1 2024-06-28T16:14:34.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.539__shadow_.F4FzEuLa2igQcqMb0FWLl3xbYz0DYzq_0 2024-06-28T16:14:34.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13056308559350466434', time=2024-06-28T18:01:28.259943+0000, chain.objs.size()=1 2024-06-28T16:14:34.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.545__shadow_._z4qUE0rijcX4A1ymCDm4SudqnBJUlu_0 2024-06-28T16:14:34.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4552772206113890686', time=2024-06-28T18:01:45.296787+0000, chain.objs.size()=1 2024-06-28T16:14:34.015 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.003+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.551__shadow_.t_Dmq50irySyXzWJXz1tuA2ADEnOGTd_0 2024-06-28T16:14:34.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17150561630064929281', time=2024-06-28T18:02:05.170918+0000, chain.objs.size()=1 2024-06-28T16:14:34.016 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:gmL34VWooEjSMx1rbfsJl9FDXp9Maaq_.cIU0ZVCS4FQu1kHpWf7qiEYllGxefEr_0 2024-06-28T16:14:34.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8625705762067325324', time=2024-06-28T18:02:19.019515+0000, chain.objs.size()=1 2024-06-28T16:14:34.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:q7nm2sQqcsEKhjB4SoITFdDStbgGgtK_.XAZC9voceAG7NqA-1pM9NAIz_xnznfX_0 2024-06-28T16:14:34.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8874719265157718745', time=2024-06-28T18:02:42.104012+0000, chain.objs.size()=1 2024-06-28T16:14:34.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.565__shadow:rWb-ISBykb2BFSNSP0DwJs6W34TA5Nx_.lCZrNoUOaDZoRde2jMQfBcjmy351Q1S_0 2024-06-28T16:14:34.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17983140767495086752', time=2024-06-28T18:02:45.973687+0000, chain.objs.size()=1 2024-06-28T16:14:34.018 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.565__shadow:gTvb0XkLS4-UfItcF77uReahIngm5BX_.HTTvqCkFUDJ_zgmQJz5XShDg0iTQwAv_0 2024-06-28T16:14:34.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4655645919643190486', time=2024-06-28T18:03:00.655495+0000, chain.objs.size()=1 2024-06-28T16:14:34.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:jPOLG3h6ZIb9hvZiua99pOmt3FGwVEE_.OejXhIl7-29WSn2AmkIFy5u-opFplAQ_0 2024-06-28T16:14:34.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4323136112864717758', time=2024-06-28T18:03:09.466470+0000, chain.objs.size()=1 2024-06-28T16:14:34.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:rkbBQm.TMo0qeGfw3M3U.4jyGyoQKps_.vyiCtpriffVsc35VwYyY8iqeo2nkxyk_0 2024-06-28T16:14:34.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3236394215085148229', time=2024-06-28T18:03:16.549218+0000, chain.objs.size()=1 2024-06-28T16:14:34.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:K9DODsCr8g25KchZ-3dBwDGb9s8YxND_.i1gP9V7iDYXb_ygG4r21xFcoyQOWJTx_0 2024-06-28T16:14:34.019 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9966544305099860401', time=2024-06-28T18:03:30.065989+0000, chain.objs.size()=1 2024-06-28T16:14:34.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:UYVCcCbn3elT9qBVSR1NaksD75Vwfzk_.-j8eS-w_LVDH2a6nFCGpNCuIoH9laIe_0 2024-06-28T16:14:34.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12910809551712074829', time=2024-06-28T18:04:00.638290+0000, chain.objs.size()=1 2024-06-28T16:14:34.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:T-3PRIPFnQthlN2zlbvRSM--a-Jg49Q_._srzhP6oOXAaOJL6gJYGDQX4134NPi3_0 2024-06-28T16:14:34.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5147640655033626021', time=2024-06-28T18:04:04.088709+0000, chain.objs.size()=1 2024-06-28T16:14:34.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.572__shadow:tEzCjIhmtOpln87AEU5u71bQzRrT53T_.2XKoh3nQdL6qjSumNHx5xt5s7pyuBsZ_0 2024-06-28T16:14:34.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4255067952799553943', time=2024-06-28T18:04:08.154485+0000, chain.objs.size()=1 2024-06-28T16:14:34.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.007+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.574__shadow:iGXTw-ItvItMRLx3M8H33je9qeTKXzq_.ZMLn7BXQNb0xQ3dwTIEfLY-JQpWNGQx_0 2024-06-28T16:14:34.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.011+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5827505258465373980', time=2024-06-28T18:07:22.156767+0000, chain.objs.size()=1 2024-06-28T16:14:34.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.635__shadow_.-nXZEvks4CEpmFF_ufL2U_KG3W9h_My_0 2024-06-28T16:14:34.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.011+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.855005110997152547', time=2024-06-28T18:07:23.143354+0000, chain.objs.size()=1 2024-06-28T16:14:34.021 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.636__shadow_.0Sd-MS32ggUu95dDvve7itERXBj0nvg_0 2024-06-28T16:14:34.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.011+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7080361835370825349', time=2024-06-28T18:07:57.347336+0000, chain.objs.size()=6 2024-06-28T16:14:34.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.647__multipart_foo.2~re0qD6hj1R5WSgYbisxnK8-jlXWuBig.1 2024-06-28T16:14:34.022 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.647__multipart_foo.2~re0qD6hj1R5WSgYbisxnK8-jlXWuBig.2 2024-06-28T16:14:34.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.647__multipart_foo.2~re0qD6hj1R5WSgYbisxnK8-jlXWuBig.3 2024-06-28T16:14:34.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.647__multipart_foo.2~re0qD6hj1R5WSgYbisxnK8-jlXWuBig.4 2024-06-28T16:14:34.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.647__multipart_foo.2~re0qD6hj1R5WSgYbisxnK8-jlXWuBig.5 2024-06-28T16:14:34.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.647__multipart_foo.2~re0qD6hj1R5WSgYbisxnK8-jlXWuBig.6 2024-06-28T16:14:34.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.011+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11202400920523248412', time=2024-06-28T18:08:41.396428+0000, chain.objs.size()=1 2024-06-28T16:14:34.023 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.011+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.688__shadow:FgpizzjDlUMA63mk77xcbm3CV9VOoND_.ACzifjM9_Yu-od13bA_I08zsep3JY9o_0 2024-06-28T16:14:34.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.015+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16408585032089781335', time=2024-06-28T18:09:19.676959+0000, chain.objs.size()=1 2024-06-28T16:14:34.024 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.015+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.704__shadow_.IRIaciI89yIJtsgIXKk3QeDa0XQKLPY_0 2024-06-28T16:14:34.027 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.019+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:34.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.019+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-06-28T16:14:34.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:34.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:34.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/42645' 2024-06-28T16:14:34.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_IfQuBIyS6Z9tXA6lNBkRRiF9MPdRTuY', time=2024-06-28T17:40:21.282305+0000, chain.objs.size()=2 2024-06-28T16:14:34.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.F6Tk1g8tOvzarNlv9NsV9yTdHxwCHVq_0 2024-06-28T16:14:34.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.150__shadow_.F6Tk1g8tOvzarNlv9NsV9yTdHxwCHVq_1 2024-06-28T16:14:34.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~IARvKivn3uB42ioCliukP994zqcmr80', time=2024-06-28T17:41:53.134668+0000, chain.objs.size()=1 2024-06-28T16:14:34.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.192__multipart_mymultipart.2~IARvKivn3uB42ioCliukP994zqcmr80.1 2024-06-28T16:14:34.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14087666216431244923', time=2024-06-28T17:44:33.982325+0000, chain.objs.size()=1 2024-06-28T16:14:34.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.220__shadow_.rdVxoCSjwjzubHW2sEsqihGy7y4E8yo_0 2024-06-28T16:14:34.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18300082991298713439', time=2024-06-28T17:44:54.066679+0000, chain.objs.size()=1 2024-06-28T16:14:34.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.232__shadow_.q9bf72VHNMerGmMsMFC7Qi6tgeEzPCW_0 2024-06-28T16:14:34.035 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10051135793216511518', time=2024-06-28T17:44:56.331907+0000, chain.objs.size()=1 2024-06-28T16:14:34.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.233__shadow_.V4fuqZwIJbh2UtZ3yuH5T_7pL2Z7nfg_0 2024-06-28T16:14:34.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5244719345032636947', time=2024-06-28T17:45:25.501767+0000, chain.objs.size()=1 2024-06-28T16:14:34.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.251__shadow_.RgL6Gv91oDTcNnKW6zQhsXzQT4_bPrj_0 2024-06-28T16:14:34.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.763684574093136131', time=2024-06-28T17:47:07.964611+0000, chain.objs.size()=1 2024-06-28T16:14:34.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-BbArKPSI7AJc3zJX9JsLf_mYZ_RQcA_0 2024-06-28T16:14:34.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7673160768168876477', time=2024-06-28T17:47:08.448811+0000, chain.objs.size()=1 2024-06-28T16:14:34.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Xc5GFjzCoL5wSz6CvYJob8jzdvGlfLz_0 2024-06-28T16:14:34.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3849856876135826826', time=2024-06-28T17:47:09.297194+0000, chain.objs.size()=1 2024-06-28T16:14:34.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.vAYfWRK84ETlUBa0DnGxXK4u75XvzqK_0 2024-06-28T16:14:34.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8443719787567531466', time=2024-06-28T17:47:09.591668+0000, chain.objs.size()=1 2024-06-28T16:14:34.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.69MsV6rS_fh3zbkkPucVY4Lk4XRVEZh_0 2024-06-28T16:14:34.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16066474942648125619', time=2024-06-28T17:47:10.580363+0000, chain.objs.size()=1 2024-06-28T16:14:34.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.023+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.sy6APjJbjfxse91AR9QwNO_aPUld7YY_0 2024-06-28T16:14:34.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.027+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10149733431761034710', time=2024-06-28T17:47:11.008555+0000, chain.objs.size()=1 2024-06-28T16:14:34.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.027+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.0gfroR1n0rY8Iv8udkrgLKIqFZuLkV9_0 2024-06-28T16:14:34.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.027+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15921388098329193751', time=2024-06-28T17:47:11.052099+0000, chain.objs.size()=1 2024-06-28T16:14:34.038 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.027+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.S4dIa69OxFsQl44Czh3eoeO2PbtlvJD_0 2024-06-28T16:14:34.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.027+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.665161438781845674', time=2024-06-28T17:47:11.174352+0000, chain.objs.size()=1 2024-06-28T16:14:34.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.027+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.X_YYG_W8CEExUxGDFjqcJu2odyn2c7E_0 2024-06-28T16:14:34.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.027+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9061894305132428501', time=2024-06-28T17:47:13.380677+0000, chain.objs.size()=1 2024-06-28T16:14:34.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.027+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_._7zIL9T940Lz20iP2f2e77WW3Wkprac_0 2024-06-28T16:14:34.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17104622843689144482', time=2024-06-28T17:47:15.220739+0000, chain.objs.size()=1 2024-06-28T16:14:34.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.AmnlvBTITTnkBkGWii8VKppvDnafuKd_0 2024-06-28T16:14:34.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7517813319157420950', time=2024-06-28T17:47:18.030554+0000, chain.objs.size()=1 2024-06-28T16:14:34.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VYIeox4I0NamXsT3JuMUqLB8GSgkc4V_0 2024-06-28T16:14:34.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13098166351830142405', time=2024-06-28T17:47:20.903686+0000, chain.objs.size()=1 2024-06-28T16:14:34.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.iNpR7J0WOHYK_7aHhayW0bE6lHI8Umd_0 2024-06-28T16:14:34.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.757819920008159186', time=2024-06-28T17:47:21.570863+0000, chain.objs.size()=1 2024-06-28T16:14:34.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.TtNtEvpsYm_6_jg-QkNkFDkEUW122cq_0 2024-06-28T16:14:34.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5845278148310531516', time=2024-06-28T17:47:21.735036+0000, chain.objs.size()=1 2024-06-28T16:14:34.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.fGDo2-iIcTZH0VkFBq8SRtLvisL7ztf_0 2024-06-28T16:14:34.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3147988190633612424', time=2024-06-28T17:47:22.292338+0000, chain.objs.size()=1 2024-06-28T16:14:34.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.g8573Q7tcCtVhUprt1if7Rg07-8qNMN_0 2024-06-28T16:14:34.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.031+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16643993151286858347', time=2024-06-28T17:47:23.216538+0000, chain.objs.size()=1 2024-06-28T16:14:34.041 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.031+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.otigaDFgPnUC-yAfotzcsPVj-GXtGu1_0 2024-06-28T16:14:34.042 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11186906399808831144', time=2024-06-28T17:47:24.966997+0000, chain.objs.size()=1 2024-06-28T16:14:34.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.C9jjCOQJEGK0ikedc6_QRoDdIOMV04L_0 2024-06-28T16:14:34.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12852087644334008180', time=2024-06-28T17:47:25.037091+0000, chain.objs.size()=1 2024-06-28T16:14:34.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.XkQXKn0808sejj3nsNCMHu3EijqYviz_0 2024-06-28T16:14:34.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1772640204384470946', time=2024-06-28T17:47:26.620933+0000, chain.objs.size()=1 2024-06-28T16:14:34.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.IH6S-pfwW4QC1dxQAulvRjMxJ99WYcM_0 2024-06-28T16:14:34.043 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15226488893617562075', time=2024-06-28T17:47:26.855232+0000, chain.objs.size()=1 2024-06-28T16:14:34.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.aPhd90DSiq2T5_Xt4aKPiyUn5aHWgEa_0 2024-06-28T16:14:34.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4097693138805305694', time=2024-06-28T17:47:28.941143+0000, chain.objs.size()=1 2024-06-28T16:14:34.044 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KICZXpbDE8FggBXckB4RkFrXvBNwoYb_0 2024-06-28T16:14:34.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8951713577428828895', time=2024-06-28T17:47:30.597963+0000, chain.objs.size()=1 2024-06-28T16:14:34.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.stn49KE_SyrdOPrEzwC30tqay17cQQs_0 2024-06-28T16:14:34.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15867279848712131209', time=2024-06-28T17:47:32.901562+0000, chain.objs.size()=1 2024-06-28T16:14:34.045 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ymDcXC7kyKz7S7NCaL9vxSMcbwTGEyM_0 2024-06-28T16:14:34.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8878699255665004818', time=2024-06-28T17:47:32.976888+0000, chain.objs.size()=1 2024-06-28T16:14:34.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Bpie1nyMmwbJuQP6S3bE9mDM43MqQH3_0 2024-06-28T16:14:34.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18299287797650922557', time=2024-06-28T17:47:33.801774+0000, chain.objs.size()=1 2024-06-28T16:14:34.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.DvLSuIysOfNLeNb7nFADEQg7VEFrWZE_0 2024-06-28T16:14:34.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3826194569377985608', time=2024-06-28T17:47:34.433995+0000, chain.objs.size()=1 2024-06-28T16:14:34.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5tZNOOhB0ItY_ffwxItyvDb0ZKixJOv_0 2024-06-28T16:14:34.046 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6029502861967722786', time=2024-06-28T17:47:34.800229+0000, chain.objs.size()=1 2024-06-28T16:14:34.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.035+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.WtxnOC0I29IQa5kRzU8vQn6QaUkUxUI_0 2024-06-28T16:14:34.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14125805384241268092', time=2024-06-28T17:47:36.057827+0000, chain.objs.size()=1 2024-06-28T16:14:34.048 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oo8xaLbYJ825FKHb1q4qwW-z47q_pBx_0 2024-06-28T16:14:34.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5101722377287167372', time=2024-06-28T17:47:37.666735+0000, chain.objs.size()=1 2024-06-28T16:14:34.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4nt5m_wB0Bm_cfl4y9eIiUZqg87fjqU_0 2024-06-28T16:14:34.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1145222750379270575', time=2024-06-28T17:47:39.093550+0000, chain.objs.size()=1 2024-06-28T16:14:34.050 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.gsLREUHHI60WCueQXKCidn0l0yf9UqL_0 2024-06-28T16:14:34.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13936793260708920938', time=2024-06-28T17:47:39.523260+0000, chain.objs.size()=1 2024-06-28T16:14:34.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.wp1XXhxOC6y-DJxq1ZkEymWStDVZAeN_0 2024-06-28T16:14:34.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12154404428624941056', time=2024-06-28T17:47:39.589425+0000, chain.objs.size()=1 2024-06-28T16:14:34.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.iu_GUe5-5fsXy0WS5r7GBytJ7CF-vL9_0 2024-06-28T16:14:34.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5140070015851182775', time=2024-06-28T17:47:45.562223+0000, chain.objs.size()=1 2024-06-28T16:14:34.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.256__shadow_.xRdY37jdO5DtS-qWVVFMAFzE0qs5nhm_0 2024-06-28T16:14:34.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11618239096454325427', time=2024-06-28T17:47:48.476465+0000, chain.objs.size()=1 2024-06-28T16:14:34.051 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.258__shadow_.B-SMZNU4qWMClC77vTa7CgAwDA3ad_A_0 2024-06-28T16:14:34.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7012610953465122725', time=2024-06-28T17:47:57.327776+0000, chain.objs.size()=1 2024-06-28T16:14:34.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.039+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.264__shadow_.H6OqfEbu7xfRJ4j0yczL2zkMupAYDHX_0 2024-06-28T16:14:34.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10503081687428974180', time=2024-06-28T17:48:14.959308+0000, chain.objs.size()=1 2024-06-28T16:14:34.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.275__shadow_.Jdw8U_hgx0vgiU72AhS47hFNx_uU_t3_0 2024-06-28T16:14:34.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9463098092263111010', time=2024-06-28T17:48:14.960842+0000, chain.objs.size()=1 2024-06-28T16:14:34.052 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.275__shadow_.QpowLL32D4HEfQivAkj0cjLcfwrPLoV_0 2024-06-28T16:14:34.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5542440604403060997', time=2024-06-28T17:48:29.789204+0000, chain.objs.size()=1 2024-06-28T16:14:34.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.m1zCS15Hmt0F6qKokLoiWLSlvhNyy6r_0 2024-06-28T16:14:34.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16100346376365131633', time=2024-06-28T17:48:47.885825+0000, chain.objs.size()=1 2024-06-28T16:14:34.053 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.291__shadow_.C-Ob7aCOcL6NSk1P7PBaP4GzIEcWRoK_0 2024-06-28T16:14:34.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9847995285678024788', time=2024-06-28T17:51:06.703510+0000, chain.objs.size()=1 2024-06-28T16:14:34.054 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.XkA3reYYrN7KMV5UUGWYpxnYNdXHbru_0 2024-06-28T16:14:34.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7825575216896451516', time=2024-06-28T17:51:06.705192+0000, chain.objs.size()=1 2024-06-28T16:14:34.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.5BMF3UNCiZwesLDDZOTaaoyd6Wdia2n_0 2024-06-28T16:14:34.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.138088881229841003', time=2024-06-28T17:51:07.592074+0000, chain.objs.size()=1 2024-06-28T16:14:34.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.MippIRrM0zgGul8_J3JMfyzoShC-wcG_0 2024-06-28T16:14:34.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17640141984383416987', time=2024-06-28T17:51:07.701303+0000, chain.objs.size()=1 2024-06-28T16:14:34.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6hV-67t0weIgnzsJKjXLGVlJukYKprV_0 2024-06-28T16:14:34.055 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1155126620886526880', time=2024-06-28T17:51:09.411697+0000, chain.objs.size()=1 2024-06-28T16:14:34.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.043+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.laaQ7A1RhEarqnUwMA3JO-W7jy0J-a1_0 2024-06-28T16:14:34.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.047+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11965127990736838662', time=2024-06-28T17:51:09.914932+0000, chain.objs.size()=1 2024-06-28T16:14:34.056 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.047+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.si4NoZa5WkdoIbURpfTQsReSr2BgSBy_0 2024-06-28T16:14:34.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.047+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16534524657141066586', time=2024-06-28T17:51:11.040582+0000, chain.objs.size()=1 2024-06-28T16:14:34.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.047+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yPNWb6ncvzbSVTQiQXaelDCvnkeqI6W_0 2024-06-28T16:14:34.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.047+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5192706153907629787', time=2024-06-28T17:51:11.200474+0000, chain.objs.size()=1 2024-06-28T16:14:34.057 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.047+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.a1D5oRdZiyFxQ3NS2hKNxBFFE5t1caw_0 2024-06-28T16:14:34.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.047+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5665443183279612308', time=2024-06-28T17:51:12.444578+0000, chain.objs.size()=1 2024-06-28T16:14:34.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.047+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.u1Bm52z4aXuDwWckUAkHwOpDdiNOtrM_0 2024-06-28T16:14:34.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.047+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12857210964082555483', time=2024-06-28T17:51:13.835839+0000, chain.objs.size()=1 2024-06-28T16:14:34.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.047+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Ss5MPQnIh3nWDs31q-MJjlflnQ3pbcA_0 2024-06-28T16:14:34.058 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1879909862078816100', time=2024-06-28T17:51:13.996778+0000, chain.objs.size()=1 2024-06-28T16:14:34.059 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.vDlbTcnFPagCSGJ9MG_H61BgmXKZmRi_0 2024-06-28T16:14:34.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5452290708651050259', time=2024-06-28T17:51:14.375046+0000, chain.objs.size()=1 2024-06-28T16:14:34.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.zRWeG5dV92Q64R-eNmRH3SwrXHRS-p8_0 2024-06-28T16:14:34.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2255431412115416148', time=2024-06-28T17:51:15.361650+0000, chain.objs.size()=1 2024-06-28T16:14:34.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.IUdqG9nwkYwJzvKiw-tBDh167t2P2Xx_0 2024-06-28T16:14:34.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15495195121893206332', time=2024-06-28T17:51:16.452938+0000, chain.objs.size()=1 2024-06-28T16:14:34.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.q4_ryG7JzWYhvzkEeYgvrxu2N403pvL_0 2024-06-28T16:14:34.061 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16223841591845634756', time=2024-06-28T17:51:16.824667+0000, chain.objs.size()=1 2024-06-28T16:14:34.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.c0Pjqe0h9wudD1W6y22zcME0YjnMFuf_0 2024-06-28T16:14:34.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6334584496603702787', time=2024-06-28T17:51:18.474484+0000, chain.objs.size()=1 2024-06-28T16:14:34.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.LHuG32neiNU_bFG6Z4JARDPKxlawWT8_0 2024-06-28T16:14:34.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10783839342500020968', time=2024-06-28T17:51:19.481032+0000, chain.objs.size()=1 2024-06-28T16:14:34.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.XOKxZUic-cRWeo5-NuFUv7kjcrarvAd_0 2024-06-28T16:14:34.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13685020664068848227', time=2024-06-28T17:51:21.419578+0000, chain.objs.size()=1 2024-06-28T16:14:34.062 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.O8jmXYN4qP2YGuA22_DxQVK32ICu_VX_0 2024-06-28T16:14:34.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12266941327253209473', time=2024-06-28T17:51:24.050159+0000, chain.objs.size()=1 2024-06-28T16:14:34.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6fXusuhp-ufupcoQiEbEpoH8i4R7y9Z_0 2024-06-28T16:14:34.063 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.051+0000 7f00c1f14640 0 -- 172.21.15.178:0/890311433 >> [v2:172.21.15.195:6824/1278553753,v1:172.21.15.195:6825/1278553753] conn(0x55bf846517f0 msgr2=0x7eff8c088150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-06-28T16:14:34.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3302681749379600822', time=2024-06-28T17:51:24.560978+0000, chain.objs.size()=1 2024-06-28T16:14:34.064 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.S7jwq3BljmgJ771W3kocMPk1Ti_wEzD_0 2024-06-28T16:14:34.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1566416347907871939', time=2024-06-28T17:51:24.607567+0000, chain.objs.size()=1 2024-06-28T16:14:34.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.9Hq10WWdrpGJ-KXMXKZNaOIxuImSMBf_0 2024-06-28T16:14:34.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1102605593149378380', time=2024-06-28T17:51:24.609179+0000, chain.objs.size()=1 2024-06-28T16:14:34.065 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.UA5r4nWI0fZC2W7Vv48hyKWR7nyQUjP_0 2024-06-28T16:14:34.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8999444815860319587', time=2024-06-28T17:51:25.780118+0000, chain.objs.size()=1 2024-06-28T16:14:34.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.iPNN_Ze5sQ_UWbEhxCMw324wo61f8qX_0 2024-06-28T16:14:34.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17124475136748071836', time=2024-06-28T17:51:29.082435+0000, chain.objs.size()=1 2024-06-28T16:14:34.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.BxHckm2MLHHH41BxYa0312goHh8bwLJ_0 2024-06-28T16:14:34.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10446257567003849880', time=2024-06-28T17:51:29.341827+0000, chain.objs.size()=1 2024-06-28T16:14:34.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PdpOtSsGeWkOsQXSrU4AuOdnDIcts7j_0 2024-06-28T16:14:34.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1608544100522010893', time=2024-06-28T17:51:29.664184+0000, chain.objs.size()=1 2024-06-28T16:14:34.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.CbOdMuLXbtp7CQwEWvtTltWQEbe3E4I_0 2024-06-28T16:14:34.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13796334170343949862', time=2024-06-28T17:51:30.080112+0000, chain.objs.size()=1 2024-06-28T16:14:34.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.LvgjUwSrbQX57PqvUEY34tjdXwrBNhD_0 2024-06-28T16:14:34.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13197633803220680064', time=2024-06-28T17:51:30.304051+0000, chain.objs.size()=1 2024-06-28T16:14:34.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.055+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.34zsRWucPrUtAi8j2ZN08-5D4voXNAp_0 2024-06-28T16:14:34.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8837241244017193392', time=2024-06-28T17:51:32.670942+0000, chain.objs.size()=1 2024-06-28T16:14:34.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.011s8-W06W89PPwH6x7ePe8kuXdoux-_0 2024-06-28T16:14:34.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3102637969707031753', time=2024-06-28T17:51:33.957092+0000, chain.objs.size()=1 2024-06-28T16:14:34.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.E9bvOS8OxhPSyAuz-6iwLyPavbIQPSn_0 2024-06-28T16:14:34.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5134229982858916281', time=2024-06-28T17:51:34.531915+0000, chain.objs.size()=1 2024-06-28T16:14:34.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xmrh_6aeZ5Od227x9UbpjEHpPhnrlyr_0 2024-06-28T16:14:34.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13669775532582054655', time=2024-06-28T17:51:35.083389+0000, chain.objs.size()=1 2024-06-28T16:14:34.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.8E756RtBPcjCVuWD3j6MuIKW7KhRytr_0 2024-06-28T16:14:34.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17118966244030657036', time=2024-06-28T17:51:37.037186+0000, chain.objs.size()=1 2024-06-28T16:14:34.070 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xfSiw2dDP-MzRmTWZbvwdsduHqV0hrH_0 2024-06-28T16:14:34.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.059+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13362555713349888514', time=2024-06-28T17:51:37.447039+0000, chain.objs.size()=1 2024-06-28T16:14:34.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.059+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.mluWHFkJo3gcS-ERWwWJDMIirT9wK-R_0 2024-06-28T16:14:34.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5578429677579402494', time=2024-06-28T17:51:38.216751+0000, chain.objs.size()=1 2024-06-28T16:14:34.071 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YAwxs_Pw3qR95_zzXU07nP37jD0jPb5_0 2024-06-28T16:14:34.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14288941437288661812', time=2024-06-28T17:51:38.383465+0000, chain.objs.size()=1 2024-06-28T16:14:34.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.dLOn-cjhZdy75fvHEII_9U5W2AuMuJR_0 2024-06-28T16:14:34.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5444518309653547210', time=2024-06-28T17:53:23.127837+0000, chain.objs.size()=1 2024-06-28T16:14:34.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.dRKyvgnPB9pf_sFzyRHem3NZgT5I29J_0 2024-06-28T16:14:34.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13464042794426587189', time=2024-06-28T17:53:24.002441+0000, chain.objs.size()=1 2024-06-28T16:14:34.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.RI5I60IqIm5kNItGPK4MiSqUW8eKF-X_0 2024-06-28T16:14:34.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2871270632460064338', time=2024-06-28T17:53:24.127438+0000, chain.objs.size()=1 2024-06-28T16:14:34.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.YFPXGpM0Ol8vUY_3bSnSNkcopDl_1Jt_0 2024-06-28T16:14:34.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11663405572947146512', time=2024-06-28T17:53:25.817566+0000, chain.objs.size()=1 2024-06-28T16:14:34.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.9dZ3D_Olzknx27BBephiig4mISPUvcj_0 2024-06-28T16:14:34.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11671186018628846952', time=2024-06-28T17:53:26.084801+0000, chain.objs.size()=1 2024-06-28T16:14:34.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4Hoeh3u4q7wIG2WaZaAzlS2yPAti6oY_0 2024-06-28T16:14:34.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16249838663243629281', time=2024-06-28T17:53:28.157499+0000, chain.objs.size()=1 2024-06-28T16:14:34.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.063+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.oXQJDiWP8r9bMn32p-LG5raWf0jaeYB_0 2024-06-28T16:14:34.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.067+0000 7f00c1713640 0 -- 172.21.15.178:0/890311433 >> [v2:172.21.15.178:6800/2650530773,v1:172.21.15.178:6801/2650530773] conn(0x7f00b80343c0 msgr2=0x7f00b8034840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T16:14:34.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.067+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.374017674796801001', time=2024-06-28T17:53:28.163450+0000, chain.objs.size()=1 2024-06-28T16:14:34.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.067+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.hF99G8u_lcUpWuQJw7RYWrJIzYTaKdp_0 2024-06-28T16:14:34.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.067+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4159506297923151660', time=2024-06-28T17:53:31.015060+0000, chain.objs.size()=1 2024-06-28T16:14:34.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.067+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.IIaI-4l5-rtTyWufWcU8RN9fITh7fZO_0 2024-06-28T16:14:34.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.067+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3128149920752659966', time=2024-06-28T17:53:31.629020+0000, chain.objs.size()=1 2024-06-28T16:14:34.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.067+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.e8Bsf0HHWsUd2csRqiq2uGXgiddUES-_0 2024-06-28T16:14:34.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.067+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10043073125172761465', time=2024-06-28T17:53:33.734074+0000, chain.objs.size()=1 2024-06-28T16:14:34.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.067+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.kVQZ9-0CJA_GAY3Jhj3O-6pMZ3cA2bH_0 2024-06-28T16:14:34.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.067+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7532112790229666936', time=2024-06-28T17:53:38.243326+0000, chain.objs.size()=1 2024-06-28T16:14:34.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.067+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.H04NQC6zdL96yg4uME-9rgZuqL61_1q_0 2024-06-28T16:14:34.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5160130644371557018', time=2024-06-28T17:53:38.791166+0000, chain.objs.size()=1 2024-06-28T16:14:34.079 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.iIKSlo4Hyx7xElEc54QZgoyyInZtKav_0 2024-06-28T16:14:34.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3632529347091314549', time=2024-06-28T17:53:40.383897+0000, chain.objs.size()=1 2024-06-28T16:14:34.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.aDPOWW7VNW17Zsr_5_NlTXwLEG1J6j-_0 2024-06-28T16:14:34.080 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2441626491798359808', time=2024-06-28T17:53:41.830379+0000, chain.objs.size()=1 2024-06-28T16:14:34.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.OGe0uFoPJUJBoLQlZLtR9jvNRMvpY4l_0 2024-06-28T16:14:34.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1999309579445150833', time=2024-06-28T17:53:43.470779+0000, chain.objs.size()=1 2024-06-28T16:14:34.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.tY_fqoSQQAUsp3EoV5ZRiUOnS_GFNnj_0 2024-06-28T16:14:34.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15655014869448330435', time=2024-06-28T17:53:43.472253+0000, chain.objs.size()=1 2024-06-28T16:14:34.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.L4vSGKOkSoWm-84F6GaccTXG1pJiBW6_0 2024-06-28T16:14:34.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2681603224934439252', time=2024-06-28T17:53:43.790019+0000, chain.objs.size()=1 2024-06-28T16:14:34.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.dtuBfVipZ-yO2pwyiKuDggeev4xiGh8_0 2024-06-28T16:14:34.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15208134814987290078', time=2024-06-28T17:53:44.237439+0000, chain.objs.size()=1 2024-06-28T16:14:34.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.32wvaoWqoVGon6DVCoG2YCYigZFbnNr_0 2024-06-28T16:14:34.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c0f12640 0 -- 172.21.15.178:0/890311433 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x7f00b801c9c0 msgr2=0x7eff8c08d3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T16:14:34.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3604807216496830177', time=2024-06-28T17:53:45.002819+0000, chain.objs.size()=1 2024-06-28T16:14:34.082 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.071+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.VxEthMJ9U4GhUAGjL5qRE4vMf4mh1a-_0 2024-06-28T16:14:34.085 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.075+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/42645 2024-06-28T16:14:34.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=23, truncated=0, next_marker='' 2024-06-28T16:14:34.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10276593840581603189', time=2024-06-28T17:53:46.774847+0000, chain.objs.size()=1 2024-06-28T16:14:34.088 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.FDQcbMyK0JFUxrBzc-IPoQatLX61hvi_0 2024-06-28T16:14:34.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11274714942389298771', time=2024-06-28T17:53:46.900572+0000, chain.objs.size()=1 2024-06-28T16:14:34.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.HJMYPHYXrYHH225LzmO7aAyCtx-c5G-_0 2024-06-28T16:14:34.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9310540489007821732', time=2024-06-28T17:53:47.333039+0000, chain.objs.size()=1 2024-06-28T16:14:34.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gk3xl9M0CWnuc_JdeeO-QUKACsZSSk2_0 2024-06-28T16:14:34.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3489734491351012681', time=2024-06-28T17:53:48.419705+0000, chain.objs.size()=1 2024-06-28T16:14:34.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Mp37Clo7yGLDvT42bwEvQWwQaCvDafD_0 2024-06-28T16:14:34.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12132021246152006070', time=2024-06-28T17:53:48.443045+0000, chain.objs.size()=1 2024-06-28T16:14:34.089 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.b2GQmIh_qR6YFii9O5lo6dVdomLA7_F_0 2024-06-28T16:14:34.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13253299476001370820', time=2024-06-28T17:53:48.910794+0000, chain.objs.size()=1 2024-06-28T16:14:34.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.aNAl-8-C5WGNiaCN9dk63-UsDe4FoR3_0 2024-06-28T16:14:34.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.402890373856556860', time=2024-06-28T17:53:49.925350+0000, chain.objs.size()=1 2024-06-28T16:14:34.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.eVr0H3JLC9rsQmJC2hq04Nhu0A0clb4_0 2024-06-28T16:14:34.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14213062985580484957', time=2024-06-28T17:53:50.016343+0000, chain.objs.size()=1 2024-06-28T16:14:34.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.0XeSCS5W1qbqp7OfhgRwEj7exay0Rsx_0 2024-06-28T16:14:34.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10404492598083771187', time=2024-06-28T17:53:50.399303+0000, chain.objs.size()=1 2024-06-28T16:14:34.090 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.QAjtJHDU5nblrfWGGgRsGdElkbw4HMe_0 2024-06-28T16:14:34.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6946777763694044501', time=2024-06-28T17:53:50.407201+0000, chain.objs.size()=1 2024-06-28T16:14:34.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.e8N61yHxzCUsH41wxi0EGY945bnSIhA_0 2024-06-28T16:14:34.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5273177200692350919', time=2024-06-28T17:53:52.125456+0000, chain.objs.size()=1 2024-06-28T16:14:34.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BQYYcIxRj8n_-wiZ5u88Uw8y1mM0cav_0 2024-06-28T16:14:34.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17651694035718446192', time=2024-06-28T17:53:56.448310+0000, chain.objs.size()=1 2024-06-28T16:14:34.091 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.079+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.319__shadow_.uh9jio4bbc-ProG_eVJB-Wgi1VUCHd7_0 2024-06-28T16:14:34.092 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6939835176051617888', time=2024-06-28T17:56:04.887311+0000, chain.objs.size()=1 2024-06-28T16:14:34.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.451__shadow_.9pRAyprXr2V6vEZPe9ndFzSO63C4zdi_0 2024-06-28T16:14:34.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1975518991747732854', time=2024-06-28T17:56:29.630913+0000, chain.objs.size()=1 2024-06-28T16:14:34.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.474__shadow_.Gu4r_6mwg7kvRjZAcOokkqNIPRBqYiH_0 2024-06-28T16:14:34.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.493__shadow:rinwpm3WJlZ1AcGvTYYbpbEzG8SjNkf_.0OEFPT4L0L8Hzv3rUAZWjoQj5ExOwB1_1_8475e97d12ded1e3bf254b21d873a495', time=2024-06-28T17:56:55.857564+0000, chain.objs.size()=1 2024-06-28T16:14:34.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.493__shadow:rinwpm3WJlZ1AcGvTYYbpbEzG8SjNkf_.0OEFPT4L0L8Hzv3rUAZWjoQj5ExOwB1_0 2024-06-28T16:14:34.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11753889087189548512', time=2024-06-28T17:57:15.907547+0000, chain.objs.size()=1 2024-06-28T16:14:34.093 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.503__multipart_mymultipart.2~e-VcObqMjp_2HgQhUBrHtpLcp2pLKk-.1 2024-06-28T16:14:34.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6804033408574378575', time=2024-06-28T17:59:54.321952+0000, chain.objs.size()=2 2024-06-28T16:14:34.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.515__multipart_mymultipart.2~6V7OfNyKS9P_N5uy5RWzhKotLNvA4nj.1 2024-06-28T16:14:34.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.515__multipart_mymultipart.2~6V7OfNyKS9P_N5uy5RWzhKotLNvA4nj.2 2024-06-28T16:14:34.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:63C.NZOu-V2ScGWmT27Bj4CNweN3JuO_.wze7dUjVhkbvyWhLsfojeq6MpbpcSqX_1_a536bae686e8591fc2889e03632433f2', time=2024-06-28T18:04:00.515439+0000, chain.objs.size()=1 2024-06-28T16:14:34.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:63C.NZOu-V2ScGWmT27Bj4CNweN3JuO_.wze7dUjVhkbvyWhLsfojeq6MpbpcSqX_0 2024-06-28T16:14:34.094 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18413636404971458918', time=2024-06-28T18:07:02.317938+0000, chain.objs.size()=1 2024-06-28T16:14:34.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.626__shadow_.0W5NlqabiF0osAldEuJ_WWX6Z1eCwAJ_0 2024-06-28T16:14:34.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17048729205145365964', time=2024-06-28T18:08:05.136590+0000, chain.objs.size()=1 2024-06-28T16:14:34.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.651__shadow_.jeii3PvZHO_mnarpdhVEIypPVX1wVqn_0 2024-06-28T16:14:34.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9697069056911963232', time=2024-06-28T18:08:07.259340+0000, chain.objs.size()=1 2024-06-28T16:14:34.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.653__shadow_.1Am_woH9iXwK5pxEKgh_w_IW2U0BCPm_0 2024-06-28T16:14:34.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11767084487476331367', time=2024-06-28T18:08:39.928407+0000, chain.objs.size()=1 2024-06-28T16:14:34.095 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.083+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.687__multipart_file1.2~H2ur5mQXtR20Nov8BFRNC_E5GYgn6hM.1 2024-06-28T16:14:34.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.087+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.704__shadow_.IRIaciI89yIJtsgIXKk3QeDa0XQKLPY_1_ff8bae5d8768761bfe960f3148aafca2', time=2024-06-28T18:09:19.672350+0000, chain.objs.size()=1 2024-06-28T16:14:34.096 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.087+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.704__shadow_.IRIaciI89yIJtsgIXKk3QeDa0XQKLPY_0 2024-06-28T16:14:34.101 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.091+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:34.103 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-06-28T16:14:34.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:34.104 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:34.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/43992' 2024-06-28T16:14:34.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_luZfgz84BEAK8qB1tDethAHDrTMpXBt', time=2024-06-28T17:40:34.850390+0000, chain.objs.size()=2 2024-06-28T16:14:34.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.151__shadow_.tSCZYcW_KPDOzPiNfosYGqLV1H2xRtn_0 2024-06-28T16:14:34.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.151__shadow_.tSCZYcW_KPDOzPiNfosYGqLV1H2xRtn_1 2024-06-28T16:14:34.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1550955381888049602', time=2024-06-28T17:44:35.764633+0000, chain.objs.size()=1 2024-06-28T16:14:34.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.221__shadow_.GpBhyC7iBwAktLqSfG5KkS7p0q8eJ7U_0 2024-06-28T16:14:34.106 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12070129971705182229', time=2024-06-28T17:45:03.357421+0000, chain.objs.size()=1 2024-06-28T16:14:34.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.237__shadow_.8EmI63bOqXwegrA_jPOMoplFDVMFmu8_0 2024-06-28T16:14:34.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15283182037990057905', time=2024-06-28T17:45:13.069008+0000, chain.objs.size()=1 2024-06-28T16:14:34.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.243__shadow_.VFLr_bDg5k0ulNYPZG1i_bvL7kKCUw1_0 2024-06-28T16:14:34.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1553197565636156820', time=2024-06-28T17:47:08.062162+0000, chain.objs.size()=1 2024-06-28T16:14:34.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.I62QaOh1qmh0RmNbIeW4ZxcueovmE9e_0 2024-06-28T16:14:34.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.850786080066908979', time=2024-06-28T17:47:08.783662+0000, chain.objs.size()=1 2024-06-28T16:14:34.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.WPiuXemY4vd4qFXK43y5-6_cI-Ig-87_0 2024-06-28T16:14:34.107 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17560269634955020431', time=2024-06-28T17:47:09.642268+0000, chain.objs.size()=1 2024-06-28T16:14:34.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.GIK_mlVjhNZincO1wJXWAHGUKyAevJp_0 2024-06-28T16:14:34.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4121380670103484656', time=2024-06-28T17:47:10.740493+0000, chain.objs.size()=1 2024-06-28T16:14:34.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5v8Y-aS50RGtkQ9jd1_iCbS1B3gbl23_0 2024-06-28T16:14:34.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2806645366596356325', time=2024-06-28T17:47:12.588364+0000, chain.objs.size()=1 2024-06-28T16:14:34.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dAiIbzFmLlNJSbHgMeQl7I0s5lWRqGM_0 2024-06-28T16:14:34.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13756077030212037801', time=2024-06-28T17:47:12.906685+0000, chain.objs.size()=1 2024-06-28T16:14:34.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.16LmhhimKGAHOzB19aiVhFbZWALsX5Y_0 2024-06-28T16:14:34.108 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11017226534920680158', time=2024-06-28T17:47:13.663184+0000, chain.objs.size()=1 2024-06-28T16:14:34.109 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.095+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.nbAsEuWRZ9vtkyry-UZD8ZaaK7QsMf5_0 2024-06-28T16:14:34.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3640260703060159862', time=2024-06-28T17:47:13.831562+0000, chain.objs.size()=1 2024-06-28T16:14:34.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Afo6piUtcrqTBhdZYXPoVG0gyyyfeUv_0 2024-06-28T16:14:34.110 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4795332266330089535', time=2024-06-28T17:47:15.561742+0000, chain.objs.size()=1 2024-06-28T16:14:34.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oDOu9zFIlvvrGwJyNWaUZGzg30XTSfp_0 2024-06-28T16:14:34.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14908397709826503198', time=2024-06-28T17:47:17.225365+0000, chain.objs.size()=1 2024-06-28T16:14:34.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.kghM_vRacaQHjWOURxIrLhWrjXuKyJ9_0 2024-06-28T16:14:34.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.737423491136933433', time=2024-06-28T17:47:17.227439+0000, chain.objs.size()=1 2024-06-28T16:14:34.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.cArBsiaa6CUg-oBSMGP9KCqerRTx65K_0 2024-06-28T16:14:34.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4139671758250469257', time=2024-06-28T17:47:20.778784+0000, chain.objs.size()=1 2024-06-28T16:14:34.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.0TsRef5uKcVYahmUFl1JSLoQ7nlmleo_0 2024-06-28T16:14:34.111 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11117142515403365462', time=2024-06-28T17:47:21.440529+0000, chain.objs.size()=1 2024-06-28T16:14:34.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oJOS26z9ahDwUHH-gPYGxhABRQXxHe__0 2024-06-28T16:14:34.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16914505047335996259', time=2024-06-28T17:47:22.330562+0000, chain.objs.size()=1 2024-06-28T16:14:34.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.2RvyyPjnhqYMIJJnJpQ79NszCEpz7qx_0 2024-06-28T16:14:34.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3095896385959522293', time=2024-06-28T17:47:23.656206+0000, chain.objs.size()=1 2024-06-28T16:14:34.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.20tdreKGvdVycCujlZYZA8BXwueSAH4_0 2024-06-28T16:14:34.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7682673228801050837', time=2024-06-28T17:47:23.763721+0000, chain.objs.size()=1 2024-06-28T16:14:34.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.OyEnC2MTcwDvDhnxi73kFosXPrdcZFm_0 2024-06-28T16:14:34.112 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17101669141500897074', time=2024-06-28T17:47:25.127587+0000, chain.objs.size()=1 2024-06-28T16:14:34.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.zNieiNnvyy-b7WUGXimdFEkect7i0xD_0 2024-06-28T16:14:34.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17216568945615238104', time=2024-06-28T17:47:25.368186+0000, chain.objs.size()=1 2024-06-28T16:14:34.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.099+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Gl5W4myMTl8Qcg2T-XVbKsF_AMHTFAx_0 2024-06-28T16:14:34.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6802856546113560556', time=2024-06-28T17:47:25.656016+0000, chain.objs.size()=1 2024-06-28T16:14:34.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.eiZNA-FTQdKU1UIOlGjujo9PiiMlqtC_0 2024-06-28T16:14:34.115 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5705943596557857921', time=2024-06-28T17:47:25.805282+0000, chain.objs.size()=1 2024-06-28T16:14:34.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.HVImuGTLXuRBRjeBBu6uJM7GmKHxYgr_0 2024-06-28T16:14:34.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7513843890057345925', time=2024-06-28T17:47:26.155785+0000, chain.objs.size()=1 2024-06-28T16:14:34.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.xIATK-TKIYc1XDEouahra7N7b0XvFeg_0 2024-06-28T16:14:34.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10595724552032010282', time=2024-06-28T17:47:26.257667+0000, chain.objs.size()=1 2024-06-28T16:14:34.116 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.lz_wDAHW0wtEpC69txn6rw4P5wkQjkF_0 2024-06-28T16:14:34.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17107392124980972803', time=2024-06-28T17:47:26.620642+0000, chain.objs.size()=1 2024-06-28T16:14:34.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.DxMMAT3VmDRJE88iuKYAGczUxeiHplh_0 2024-06-28T16:14:34.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.771149435993249603', time=2024-06-28T17:47:27.143289+0000, chain.objs.size()=1 2024-06-28T16:14:34.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.6PolkEah8u1EYuvOUmG1cuzE-mzrtdC_0 2024-06-28T16:14:34.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5840570030594601751', time=2024-06-28T17:47:27.954854+0000, chain.objs.size()=1 2024-06-28T16:14:34.117 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.iJwncbHPkE2HxFqN5jwOS2gNDfFN-VV_0 2024-06-28T16:14:34.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5056901336979747214', time=2024-06-28T17:47:27.999917+0000, chain.objs.size()=1 2024-06-28T16:14:34.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.fs94PbnYHSfHOV5wLIkUUArYc2Jamvb_0 2024-06-28T16:14:34.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11697307181193034435', time=2024-06-28T17:47:29.014222+0000, chain.objs.size()=1 2024-06-28T16:14:34.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.3kiud-5yMON5_f8iHUovsWU4xc21W3n_0 2024-06-28T16:14:34.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18149706013691046134', time=2024-06-28T17:47:29.856258+0000, chain.objs.size()=1 2024-06-28T16:14:34.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KspTn_EoZI64m1vHX9wgagFpOA4Ek35_0 2024-06-28T16:14:34.118 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2154700807400841218', time=2024-06-28T17:47:31.299834+0000, chain.objs.size()=1 2024-06-28T16:14:34.119 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.107+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.d3ZYUBItkA-YU9OcoCjt_a02Ig1ikej_0 2024-06-28T16:14:34.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11804530019082486477', time=2024-06-28T17:47:31.680449+0000, chain.objs.size()=1 2024-06-28T16:14:34.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.BWrjVssxZlIE2t2WLHOq59MFMql2ehO_0 2024-06-28T16:14:34.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14564473542999579879', time=2024-06-28T17:47:31.682247+0000, chain.objs.size()=1 2024-06-28T16:14:34.120 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.UXJqOtgV-Inq1_WVfuhNNebVN7bCzc7_0 2024-06-28T16:14:34.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2925204316060286610', time=2024-06-28T17:47:32.073649+0000, chain.objs.size()=1 2024-06-28T16:14:34.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.zEsufQkFC1dMEvCnHRpeitPqMMPc-q-_0 2024-06-28T16:14:34.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13094850218137925834', time=2024-06-28T17:47:32.075797+0000, chain.objs.size()=1 2024-06-28T16:14:34.121 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.TPv86c0QKzVbRPHaH23aM9qEkLodGKa_0 2024-06-28T16:14:34.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.111+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15301217968723218858', time=2024-06-28T17:47:33.575008+0000, chain.objs.size()=1 2024-06-28T16:14:34.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.111+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-bORnQuNeBBFe_mKPxB6gswHA3yB2XR_0 2024-06-28T16:14:34.122 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17025614444430542509', time=2024-06-28T17:47:33.912714+0000, chain.objs.size()=1 2024-06-28T16:14:34.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5x5OzqORWvfNZu4dULxvt_LniHLQ9oZ_0 2024-06-28T16:14:34.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1101463696290936107', time=2024-06-28T17:47:35.229424+0000, chain.objs.size()=1 2024-06-28T16:14:34.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.GIBumyKZNsSpNKtdPUh4jMhpEx_6OKo_0 2024-06-28T16:14:34.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7244846306100242252', time=2024-06-28T17:47:35.434908+0000, chain.objs.size()=1 2024-06-28T16:14:34.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ycmCZaHEmrcZKScabzpLS4MGXufJBPz_0 2024-06-28T16:14:34.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17115256243722840288', time=2024-06-28T17:47:35.435899+0000, chain.objs.size()=1 2024-06-28T16:14:34.123 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.rEZsotpKNbeIIBHJ4D5POsO11IDFQYt_0 2024-06-28T16:14:34.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7153011170105141193', time=2024-06-28T17:47:35.722001+0000, chain.objs.size()=1 2024-06-28T16:14:34.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.LG0baYRuneCJJMN2mpnY-a6_tYygpPV_0 2024-06-28T16:14:34.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2534411117201299346', time=2024-06-28T17:47:39.587289+0000, chain.objs.size()=1 2024-06-28T16:14:34.124 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.p3xCRugFg0LsuUqhtEoqj9M4jaYqri5_0 2024-06-28T16:14:34.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1557757557360840457', time=2024-06-28T17:47:42.559905+0000, chain.objs.size()=1 2024-06-28T16:14:34.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.254__shadow_.aTvvPYQYfpuTl7cMXpyL02rJxYKK-4K_0 2024-06-28T16:14:34.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6525379782104812962', time=2024-06-28T17:47:55.879493+0000, chain.objs.size()=1 2024-06-28T16:14:34.125 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.263__shadow_.mH0scfys-UYSaFUi-Q_denzcK5mDBCK_0 2024-06-28T16:14:34.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10631640056137832905', time=2024-06-28T17:48:10.120654+0000, chain.objs.size()=1 2024-06-28T16:14:34.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.272__shadow_.aBy_JkZ1U6i7g6B31XVBesaQg-Kyn-m_0 2024-06-28T16:14:34.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6503309056652788570', time=2024-06-28T17:48:30.011610+0000, chain.objs.size()=1 2024-06-28T16:14:34.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.-dDfi-Os6HrPywwE6vG5nRypDjLHaIJ_0 2024-06-28T16:14:34.126 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11877727490422461465', time=2024-06-28T17:48:34.853469+0000, chain.objs.size()=1 2024-06-28T16:14:34.127 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.115+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.Qk1Sg_mSX7zXGG-z6CmRLLLid8tL2kQ_0 2024-06-28T16:14:34.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7275732724924895665', time=2024-06-28T17:48:34.922023+0000, chain.objs.size()=1 2024-06-28T16:14:34.129 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.C9x4F97QYW-L5QIIobkyL1MOdgAkP54_0 2024-06-28T16:14:34.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16453302998817958852', time=2024-06-28T17:48:44.707143+0000, chain.objs.size()=1 2024-06-28T16:14:34.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.289__shadow_.DOkz_ex82Z_Oy7XAF_NF4ZgNxSSa6vQ_0 2024-06-28T16:14:34.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2018469496882976039', time=2024-06-28T17:51:08.365009+0000, chain.objs.size()=1 2024-06-28T16:14:34.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Y87I1HdySgtxyreILW4WAjUNWpd5zef_0 2024-06-28T16:14:34.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2211619215980871742', time=2024-06-28T17:51:08.573761+0000, chain.objs.size()=1 2024-06-28T16:14:34.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.7XZ_VKwAvkzziOJTuDBKNxqtlmmNI6M_0 2024-06-28T16:14:34.130 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4779249240296497629', time=2024-06-28T17:51:09.016577+0000, chain.objs.size()=1 2024-06-28T16:14:34.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.MG3Uw0U8cPgpOwDdtr46tUkc_vr5r1K_0 2024-06-28T16:14:34.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8468154659760890305', time=2024-06-28T17:51:09.459718+0000, chain.objs.size()=1 2024-06-28T16:14:34.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.oA5FwsagysTN7DLKSOqkuD2m7fqD6I-_0 2024-06-28T16:14:34.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.119+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10265567845252575382', time=2024-06-28T17:51:11.041966+0000, chain.objs.size()=1 2024-06-28T16:14:34.131 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.119+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.JXWhV7OoZYU7mPOM44MVCPTGG5Rltv8_0 2024-06-28T16:14:34.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.123+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5733284221549349584', time=2024-06-28T17:51:11.613448+0000, chain.objs.size()=1 2024-06-28T16:14:34.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.123+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.mDXBOZiq1jCm452976uJFGTSv__Iws4_0 2024-06-28T16:14:34.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.123+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13018231022655960718', time=2024-06-28T17:51:12.101890+0000, chain.objs.size()=1 2024-06-28T16:14:34.132 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.123+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.EnlGZZIcdCY2m2dWN8CeDesTudSPYDb_0 2024-06-28T16:14:34.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.123+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4117837826935928522', time=2024-06-28T17:51:12.103834+0000, chain.objs.size()=1 2024-06-28T16:14:34.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.123+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ktriyt6MreAMpg27jr-3gw8uHIHQRDc_0 2024-06-28T16:14:34.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.123+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1523306472469010718', time=2024-06-28T17:51:13.354648+0000, chain.objs.size()=1 2024-06-28T16:14:34.133 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.123+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fgiQeLvxodh_o5iGSO9rdXFP9cCBA7X_0 2024-06-28T16:14:34.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.123+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14732712003436092782', time=2024-06-28T17:51:13.358212+0000, chain.objs.size()=1 2024-06-28T16:14:34.134 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.123+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.B161GUH7Fq8Aj4epRUe3KpCwcaHarK-_0 2024-06-28T16:14:34.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8301098602336364161', time=2024-06-28T17:51:14.119493+0000, chain.objs.size()=1 2024-06-28T16:14:34.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.I4nDKpvC2uogDdGTZJ-1yHxyd74B9X4_0 2024-06-28T16:14:34.135 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8880413074844140469', time=2024-06-28T17:51:14.162426+0000, chain.objs.size()=1 2024-06-28T16:14:34.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.elhLeBnznqKMhucySg22puYfluxlOWT_0 2024-06-28T16:14:34.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12602482487688776023', time=2024-06-28T17:51:15.780511+0000, chain.objs.size()=1 2024-06-28T16:14:34.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.kl5BkUt0zueEw_cZDo5DquB-Wz7Oloo_0 2024-06-28T16:14:34.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11505333001695677703', time=2024-06-28T17:51:16.840439+0000, chain.objs.size()=1 2024-06-28T16:14:34.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.I3LlmQ5v-6J0xX72Al_06s0eQfVIb-Y_0 2024-06-28T16:14:34.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14339042556287696461', time=2024-06-28T17:51:17.218150+0000, chain.objs.size()=1 2024-06-28T16:14:34.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.FCHCRrFOsS-PkaHNA4uCSVPG9itRozx_0 2024-06-28T16:14:34.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13235775644668767218', time=2024-06-28T17:51:18.341385+0000, chain.objs.size()=1 2024-06-28T16:14:34.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.4RCFN4TRUQ7jbJaky1-4n4hC0YUqB4k_0 2024-06-28T16:14:34.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2575697330960354419', time=2024-06-28T17:51:19.684103+0000, chain.objs.size()=1 2024-06-28T16:14:34.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yS-6xX0VjYtvREUvVT8IYxOjJ7HqPgy_0 2024-06-28T16:14:34.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10092811501280261266', time=2024-06-28T17:51:20.652706+0000, chain.objs.size()=1 2024-06-28T16:14:34.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.o9_vdWv93xVFy2hpfrMAGA9qPPPfZDl_0 2024-06-28T16:14:34.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7990859113997052164', time=2024-06-28T17:51:21.629275+0000, chain.objs.size()=1 2024-06-28T16:14:34.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.2vGz3QG5vQ6Sgwnk2r3WSEjIYx7qi1V_0 2024-06-28T16:14:34.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1399132979707896198', time=2024-06-28T17:51:21.782221+0000, chain.objs.size()=1 2024-06-28T16:14:34.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.127+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.BfOBbtv13UI96icrhqeH02FEnTVGdEs_0 2024-06-28T16:14:34.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11190063523977505783', time=2024-06-28T17:51:26.342912+0000, chain.objs.size()=1 2024-06-28T16:14:34.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.K8KrHU8wktRSeA0_-1jFPtBclAh-ey5_0 2024-06-28T16:14:34.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9482682308545472662', time=2024-06-28T17:51:26.571330+0000, chain.objs.size()=1 2024-06-28T16:14:34.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.U8bjhIHBSAQoxCVLfuRwsRxqhwWHqGX_0 2024-06-28T16:14:34.141 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14295639493349472609', time=2024-06-28T17:51:27.723487+0000, chain.objs.size()=1 2024-06-28T16:14:34.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Kz3cF9pk3d2a0b7LeFMrmpzWDFhigQY_0 2024-06-28T16:14:34.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2610670488244845241', time=2024-06-28T17:51:27.725697+0000, chain.objs.size()=1 2024-06-28T16:14:34.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.jmUfSHlAWTRpsaH-UngoJdWFOE1PFnS_0 2024-06-28T16:14:34.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2426523256404015017', time=2024-06-28T17:51:28.908668+0000, chain.objs.size()=1 2024-06-28T16:14:34.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.zyThQxrDRJiDz5FzfOU1BVDihlG04WA_0 2024-06-28T16:14:34.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11424740891720402343', time=2024-06-28T17:51:30.180782+0000, chain.objs.size()=1 2024-06-28T16:14:34.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6eXEXoebRr3pktEmVty_WISDbk-Ei83_0 2024-06-28T16:14:34.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9530600381663874807', time=2024-06-28T17:51:31.060622+0000, chain.objs.size()=1 2024-06-28T16:14:34.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.RsqR8lnTK8MqVloul8SOcWvY4NvGM_u_0 2024-06-28T16:14:34.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13643758102879765690', time=2024-06-28T17:51:31.905287+0000, chain.objs.size()=1 2024-06-28T16:14:34.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PoDrsmp6xJsCAWvsXVW182OFis-ThqJ_0 2024-06-28T16:14:34.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10849721634868957037', time=2024-06-28T17:51:33.401339+0000, chain.objs.size()=1 2024-06-28T16:14:34.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ct8tZ1GJmrRtFbAQVVh5xwz1iAapawp_0 2024-06-28T16:14:34.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3651915209494318050', time=2024-06-28T17:51:33.680678+0000, chain.objs.size()=1 2024-06-28T16:14:34.143 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PuyBbeyGR5Kwmuc-SedXwyBuEmmtDfm_0 2024-06-28T16:14:34.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3672796123057776027', time=2024-06-28T17:51:34.610343+0000, chain.objs.size()=1 2024-06-28T16:14:34.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.131+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.mMYTp8-7ZsSE161fDq0RAgQ5yu23tMa_0 2024-06-28T16:14:34.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.135+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6992682029000030519', time=2024-06-28T17:51:34.941502+0000, chain.objs.size()=1 2024-06-28T16:14:34.146 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.135+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nP7N05XdIxiw93yG0TwfbRyrAEZpw-U_0 2024-06-28T16:14:34.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14755644699062494091', time=2024-06-28T17:51:34.942858+0000, chain.objs.size()=1 2024-06-28T16:14:34.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.kop7H3kCL9VRWXeO9HiFRxtPRsTcTBY_0 2024-06-28T16:14:34.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13219278387272779336', time=2024-06-28T17:51:38.306327+0000, chain.objs.size()=1 2024-06-28T16:14:34.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.wUv-qbl2zbnGOU0mAdCd61p1uqoaQ6-_0 2024-06-28T16:14:34.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15360695989535832695', time=2024-06-28T17:53:21.049866+0000, chain.objs.size()=1 2024-06-28T16:14:34.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_._wvtVbBEaJNKs4gzqkoJmfws8CMcEii_0 2024-06-28T16:14:34.148 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4226104796735750', time=2024-06-28T17:53:21.854181+0000, chain.objs.size()=1 2024-06-28T16:14:34.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CzSG0Ixsi1NhxWPkJK38Mvv6o1_qb55_0 2024-06-28T16:14:34.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5699508581385050541', time=2024-06-28T17:53:23.591306+0000, chain.objs.size()=1 2024-06-28T16:14:34.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vOtcoan4JD0dDW9LNSDQVabGYoHnxy9_0 2024-06-28T16:14:34.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13093372840906182742', time=2024-06-28T17:53:24.015945+0000, chain.objs.size()=1 2024-06-28T16:14:34.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1ZAqXKt_pt2yITm1RZ-QW2FCJ-piBHg_0 2024-06-28T16:14:34.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5087633971149460844', time=2024-06-28T17:53:24.127184+0000, chain.objs.size()=1 2024-06-28T16:14:34.149 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wLIVdL4uqv9m3h_3ePJ2s4jGb9Z_e2h_0 2024-06-28T16:14:34.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15745054244089510722', time=2024-06-28T17:53:24.162314+0000, chain.objs.size()=1 2024-06-28T16:14:34.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.cWVkLKicIZim2irZC6gjybEwQUygr7m_0 2024-06-28T16:14:34.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11246921879854851809', time=2024-06-28T17:53:26.096557+0000, chain.objs.size()=1 2024-06-28T16:14:34.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.9KeN92dkA6Ioqrr2XOIj0jay7x0bBCC_0 2024-06-28T16:14:34.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6744054917076177188', time=2024-06-28T17:53:26.496006+0000, chain.objs.size()=1 2024-06-28T16:14:34.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.139+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.l9y6HRH8dDxRo1LgTpXje9cNOfTRqEs_0 2024-06-28T16:14:34.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14536639115311390425', time=2024-06-28T17:53:26.867141+0000, chain.objs.size()=1 2024-06-28T16:14:34.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.fjcqv57NzB0HK5eJP-3avSbNOKTWS7H_0 2024-06-28T16:14:34.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.372155725331418316', time=2024-06-28T17:53:27.094445+0000, chain.objs.size()=1 2024-06-28T16:14:34.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.cSpZaaJnm4ZBVlZ7WxQZuXeZy-d4XW4_0 2024-06-28T16:14:34.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15392568476143557342', time=2024-06-28T17:53:28.355739+0000, chain.objs.size()=1 2024-06-28T16:14:34.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.xcNSg-YyErUzE0X6SSxrH129RRVrNvB_0 2024-06-28T16:14:34.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.143+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12286825508546346618', time=2024-06-28T17:53:29.476446+0000, chain.objs.size()=1 2024-06-28T16:14:34.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.143+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1PLhLjsjYAWsRMDm-t4CqeVPBwmHnMB_0 2024-06-28T16:14:34.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.147+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1144517848388042884', time=2024-06-28T17:53:29.593530+0000, chain.objs.size()=1 2024-06-28T16:14:34.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.147+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.U_yxbjTHBpjU7jW7ls7gD4us6U0-GKU_0 2024-06-28T16:14:34.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.147+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18064362992758314078', time=2024-06-28T17:53:30.190153+0000, chain.objs.size()=1 2024-06-28T16:14:34.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.147+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.36Ly6n0yvsBUxpZ_hhGczzy2ia-ion-_0 2024-06-28T16:14:34.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.147+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14856559097603339026', time=2024-06-28T17:53:30.380962+0000, chain.objs.size()=1 2024-06-28T16:14:34.155 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.147+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Pd0toxbH5MoHobKgfYsE9pdqqLgzUUP_0 2024-06-28T16:14:34.160 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/43992 2024-06-28T16:14:34.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=31, truncated=0, next_marker='' 2024-06-28T16:14:34.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8381524936014153380', time=2024-06-28T17:53:30.844979+0000, chain.objs.size()=1 2024-06-28T16:14:34.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CnUS22fUrLTfD4alAJO6kLA3BcdOtoh_0 2024-06-28T16:14:34.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2628109269969491425', time=2024-06-28T17:53:33.818090+0000, chain.objs.size()=1 2024-06-28T16:14:34.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_._b2u5Sdh9t-kS54XmLJWLLyTuNKJ2MZ_0 2024-06-28T16:14:34.163 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.24635313443157306', time=2024-06-28T17:53:34.415131+0000, chain.objs.size()=1 2024-06-28T16:14:34.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Kx_6VZOjxz1W7Vl1T5vqKJk_wYExWGs_0 2024-06-28T16:14:34.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8899046785950932567', time=2024-06-28T17:53:35.060986+0000, chain.objs.size()=1 2024-06-28T16:14:34.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Avqie5THCRSExTPsco6q-j-ySy885Bl_0 2024-06-28T16:14:34.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12852934901982781594', time=2024-06-28T17:53:37.755668+0000, chain.objs.size()=1 2024-06-28T16:14:34.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.cF-pNmlceIjwNNQEw5Daza2PlilJTZf_0 2024-06-28T16:14:34.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10549755893924924253', time=2024-06-28T17:53:37.795593+0000, chain.objs.size()=1 2024-06-28T16:14:34.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.9Nb-JiuMoxEXzaRTdX3OKJJxupPufpf_0 2024-06-28T16:14:34.164 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12934718473126954467', time=2024-06-28T17:53:40.580215+0000, chain.objs.size()=1 2024-06-28T16:14:34.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.151+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.UZ0bp4-CGOfBwgTnpmuNOCLxG7ZCzWJ_0 2024-06-28T16:14:34.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1325218741436673888', time=2024-06-28T17:53:40.905363+0000, chain.objs.size()=1 2024-06-28T16:14:34.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.t28I3gmitaN6Vcnd3yJdxqB2p6kQbYZ_0 2024-06-28T16:14:34.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.149285134689800362', time=2024-06-28T17:53:41.958182+0000, chain.objs.size()=1 2024-06-28T16:14:34.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.dmSywErNbiW3JXc0spNmrnZpUwcX_5A_0 2024-06-28T16:14:34.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12629100553940298623', time=2024-06-28T17:53:42.304491+0000, chain.objs.size()=1 2024-06-28T16:14:34.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.V_i3ljyG09pQtAkMhTpp0bBo8KHPEI1_0 2024-06-28T16:14:34.165 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8138256213533450444', time=2024-06-28T17:53:43.477578+0000, chain.objs.size()=1 2024-06-28T16:14:34.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.s-vr4kwU4EpIlTzGwCZuzg4GTk-FEtb_0 2024-06-28T16:14:34.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.155+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8024831744377904897', time=2024-06-28T17:53:43.953139+0000, chain.objs.size()=1 2024-06-28T16:14:34.166 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.155+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.jS8EXxICKz8lC4s0YSDVkCLTP5KacY6_0 2024-06-28T16:14:34.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.159+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5277037680651927089', time=2024-06-28T17:53:44.117628+0000, chain.objs.size()=1 2024-06-28T16:14:34.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.159+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.o5mlgaHh4jALA8p0PrJPUhu_Z3Cndmv_0 2024-06-28T16:14:34.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.159+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15769852653749588825', time=2024-06-28T17:53:45.101575+0000, chain.objs.size()=1 2024-06-28T16:14:34.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.159+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.i-cFTKO8cOg-ClJjubuTuxxk-uWRK-z_0 2024-06-28T16:14:34.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.159+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16807910958924208075', time=2024-06-28T17:53:46.332451+0000, chain.objs.size()=1 2024-06-28T16:14:34.169 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.159+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.n2P5BCAAP7G5Pes4kr7wEPKykm-cfEq_0 2024-06-28T16:14:34.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.159+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16832914027524957531', time=2024-06-28T17:53:46.334166+0000, chain.objs.size()=1 2024-06-28T16:14:34.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.159+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.0YuwdZH_IpSR2NPhSWtnmG6fMK-ylbl_0 2024-06-28T16:14:34.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.159+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4945644400161578505', time=2024-06-28T17:53:50.420711+0000, chain.objs.size()=1 2024-06-28T16:14:34.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.159+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.NVp47947J4wRWEz5FBvcUy4FIN9NVlO_0 2024-06-28T16:14:34.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.159+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9209179049592626274', time=2024-06-28T17:53:51.179989+0000, chain.objs.size()=1 2024-06-28T16:14:34.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.159+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.f4dGsKnqC1cFfAvbJtLLFGLoua_0lvV_0 2024-06-28T16:14:34.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12040903751120836704', time=2024-06-28T17:53:52.206671+0000, chain.objs.size()=1 2024-06-28T16:14:34.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.tm74IjKlgg-QrYaw-nkqN7Z_ltrFilx_0 2024-06-28T16:14:34.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9136122413479876285', time=2024-06-28T17:53:59.508558+0000, chain.objs.size()=1 2024-06-28T16:14:34.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.322__shadow_.AGW9jj412zPWuDFCAMmWgqGvsmWowVv_0 2024-06-28T16:14:34.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14225037720839709952', time=2024-06-28T17:55:29.457805+0000, chain.objs.size()=1 2024-06-28T16:14:34.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.422__shadow_.4K2qtog-8toIaOlTutDXek2kUH0c_F5_0 2024-06-28T16:14:34.172 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9245184445919927831', time=2024-06-28T17:55:30.415325+0000, chain.objs.size()=1 2024-06-28T16:14:34.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.423__shadow_.4vbXb2fSVSFN3dCQusx2Tej-_euKWOh_0 2024-06-28T16:14:34.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5709107790298023836', time=2024-06-28T17:55:36.902547+0000, chain.objs.size()=1 2024-06-28T16:14:34.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.429__shadow_.DRd984OP_g7OCit_doraKZP3LpED_hF_0 2024-06-28T16:14:34.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9520033300739185353', time=2024-06-28T17:56:42.829056+0000, chain.objs.size()=1 2024-06-28T16:14:34.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.483__shadow_.eSOBTEG9xWFqF2d4MZ3P-ZlaL8vsjZa_0 2024-06-28T16:14:34.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11911184080739442398', time=2024-06-28T17:59:40.369427+0000, chain.objs.size()=2 2024-06-28T16:14:34.173 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.513__multipart_mymultipart.2~o8dKqj4ct8nY2f705EwCTaZEbCxtvVS.1 2024-06-28T16:14:34.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.513__multipart_mymultipart.2~o8dKqj4ct8nY2f705EwCTaZEbCxtvVS.2 2024-06-28T16:14:34.174 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13784068534673361487', time=2024-06-28T18:01:37.544181+0000, chain.objs.size()=1 2024-06-28T16:14:34.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.547__shadow_.V8LxmJGOYToiK51VLyvbMrC8xRbBKxy_0 2024-06-28T16:14:34.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18294338041036503973', time=2024-06-28T18:02:00.370963+0000, chain.objs.size()=1 2024-06-28T16:14:34.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.163+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:G-U94oMPBLRGS9mvhkhcIM.tGRXV1pC_.8pgONLUjsiSznMdfD2cCVhIeLavxAHp_0 2024-06-28T16:14:34.175 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15933202024459480394', time=2024-06-28T18:02:22.379603+0000, chain.objs.size()=1 2024-06-28T16:14:34.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.559__shadow:cbhjLUEUjv5EYLaqEFj4IjbwtjAiqIg_.0rn9Em5azO48I3WGQRaGP1Zc5sbfEpV_0 2024-06-28T16:14:34.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18331007334981383638', time=2024-06-28T18:03:55.894297+0000, chain.objs.size()=1 2024-06-28T16:14:34.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.568__shadow:DYiuOdFur6lDdwa1FPkVWBrRTnk0Scm_.O1Ipn53AIR7jgPoWEAJV4zPq-sYY-QP_0 2024-06-28T16:14:34.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16973554205865627831', time=2024-06-28T18:07:59.205689+0000, chain.objs.size()=1 2024-06-28T16:14:34.176 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.648__shadow_.tY8xIqUSRZopFILkO4KFXlNbDTjejd3_0 2024-06-28T16:14:34.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.167+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17160923195196249829', time=2024-06-28T18:09:17.209813+0000, chain.objs.size()=1 2024-06-28T16:14:34.177 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.167+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.702__shadow:x6m2KWCW880EMTkX06hFRPvS.NRVhed_.rObGCh4u-18zhWJWtYx2npis9TtOr85_0 2024-06-28T16:14:34.180 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.171+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:34.183 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-06-28T16:14:34.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:34.184 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:34.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45990' 2024-06-28T16:14:34.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10819645983365362856', time=2024-06-28T17:40:12.886722+0000, chain.objs.size()=3 2024-06-28T16:14:34.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.149__multipart_mymultipart-FROZEN.2~UkHAvXZ9x4ZhX5-urSpvhX3vqxZtvB0.1 2024-06-28T16:14:34.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.149__multipart_mymultipart-FROZEN.2~UkHAvXZ9x4ZhX5-urSpvhX3vqxZtvB0.2 2024-06-28T16:14:34.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.149__multipart_mymultipart-FROZEN.2~UkHAvXZ9x4ZhX5-urSpvhX3vqxZtvB0.3 2024-06-28T16:14:34.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8990579386377182280', time=2024-06-28T17:40:48.198660+0000, chain.objs.size()=1 2024-06-28T16:14:34.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.161__shadow_.lw09SUOrvxJsQZFVr0wRH4_s54oUgKS_0 2024-06-28T16:14:34.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8140807087586690500', time=2024-06-28T17:41:09.379380+0000, chain.objs.size()=1 2024-06-28T16:14:34.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.169__shadow_.i7Ub34JVq8BkF7spexRzOxQL4cKZNBg_0 2024-06-28T16:14:34.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8684741097136524129', time=2024-06-28T17:44:39.022508+0000, chain.objs.size()=1 2024-06-28T16:14:34.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.223__shadow_.BlkL6IitlSH_Em5Uf1FTXEfUoTUFp2a_0 2024-06-28T16:14:34.187 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3240318319040662236', time=2024-06-28T17:45:01.777599+0000, chain.objs.size()=1 2024-06-28T16:14:34.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.236__shadow_.hlV-0x1v-WnDt-lkFyCPqZXeHZ7NB9r_0 2024-06-28T16:14:34.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10091020520262365678', time=2024-06-28T17:45:04.979327+0000, chain.objs.size()=1 2024-06-28T16:14:34.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.238__shadow_.dQi4n_lIugP3tYI1-kW9HLG03MnkX1T_0 2024-06-28T16:14:34.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14241438175357777953', time=2024-06-28T17:47:08.054043+0000, chain.objs.size()=1 2024-06-28T16:14:34.188 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.SuX4GHxXGBZ-DqAaW-RtuWdiWOytKNb_0 2024-06-28T16:14:34.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13146161359152290081', time=2024-06-28T17:47:08.432974+0000, chain.objs.size()=1 2024-06-28T16:14:34.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.T03knbHhwYBYR7q2WdGGnopAJDTxqkW_0 2024-06-28T16:14:34.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11607909945694842461', time=2024-06-28T17:47:09.589183+0000, chain.objs.size()=1 2024-06-28T16:14:34.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.LdjHVwJ_llV8x_9xMxTafiQ8ACBhjet_0 2024-06-28T16:14:34.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16495237474243793741', time=2024-06-28T17:47:09.802106+0000, chain.objs.size()=1 2024-06-28T16:14:34.189 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.175+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.pTSd-aMJU5O6tERgB8_d3om5M2s3SbX_0 2024-06-28T16:14:34.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18111691426677302401', time=2024-06-28T17:47:09.936273+0000, chain.objs.size()=1 2024-06-28T16:14:34.190 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-R3asLeZhHNzGXMRXIFlQF3Ab9i92zm_0 2024-06-28T16:14:34.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1493826662146155429', time=2024-06-28T17:47:10.415601+0000, chain.objs.size()=1 2024-06-28T16:14:34.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.GtW6AGv4M8_Q7ya6J6sTYj4giekoBzp_0 2024-06-28T16:14:34.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7651291469669358479', time=2024-06-28T17:47:13.080130+0000, chain.objs.size()=1 2024-06-28T16:14:34.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ebeq6WcAoWBnDcsujcIHGdO7HMk2iyQ_0 2024-06-28T16:14:34.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18112445721178965898', time=2024-06-28T17:47:13.626912+0000, chain.objs.size()=1 2024-06-28T16:14:34.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.rIGQSetsv7Poi_JrXrDLuMFP9O8nqwW_0 2024-06-28T16:14:34.191 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11487390424954725919', time=2024-06-28T17:47:14.545376+0000, chain.objs.size()=1 2024-06-28T16:14:34.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.lVmTQkST1cMrN6QAmsZBajb90OgoP9N_0 2024-06-28T16:14:34.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15480839120019018768', time=2024-06-28T17:47:15.037130+0000, chain.objs.size()=1 2024-06-28T16:14:34.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.G1kFdo_mN2U1UGE5Fu0LtYecFwu8PmI_0 2024-06-28T16:14:34.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13988343304706196882', time=2024-06-28T17:47:15.144882+0000, chain.objs.size()=1 2024-06-28T16:14:34.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.RUI8AARPyPAWuJzxFq9AgT8PJGHb99g_0 2024-06-28T16:14:34.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11668868500564848494', time=2024-06-28T17:47:15.201335+0000, chain.objs.size()=1 2024-06-28T16:14:34.192 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.nBlT0Vb0KAk5Jz3tny1SKCtE8K_bNJX_0 2024-06-28T16:14:34.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11517490744796006905', time=2024-06-28T17:47:15.560056+0000, chain.objs.size()=1 2024-06-28T16:14:34.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.179+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ntdT8nn85HfEj4c0B595tyaOQvfOu4J_0 2024-06-28T16:14:34.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.183+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10729027597504510644', time=2024-06-28T17:47:16.783866+0000, chain.objs.size()=1 2024-06-28T16:14:34.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Qtos-LV8x9qBpvNSavLZbvdJEA6zO5U_0 2024-06-28T16:14:34.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.183+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5287083827359212356', time=2024-06-28T17:47:17.274865+0000, chain.objs.size()=1 2024-06-28T16:14:34.193 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.RGpGdx6JpP-COrPlrj1yavUE6FgLFz1_0 2024-06-28T16:14:34.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.183+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8526467283617604690', time=2024-06-28T17:47:18.071974+0000, chain.objs.size()=1 2024-06-28T16:14:34.194 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.183+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.g4xaBhl8vdv6KIqkKvQ0MHUP5iIclW1_0 2024-06-28T16:14:34.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6155789265392175283', time=2024-06-28T17:47:19.210856+0000, chain.objs.size()=1 2024-06-28T16:14:34.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.V6tERZNY67oMPSFxPLA1omY1BG3ssoa_0 2024-06-28T16:14:34.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15437195601455055700', time=2024-06-28T17:47:20.841293+0000, chain.objs.size()=1 2024-06-28T16:14:34.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ygVO98YbW5XRqH5DaRyrPJ6HPRAIVMV_0 2024-06-28T16:14:34.195 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16865843157370640804', time=2024-06-28T17:47:23.772883+0000, chain.objs.size()=1 2024-06-28T16:14:34.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.GfvTuOsC2Mh5A4JMB-kwxpOSaeZ01AK_0 2024-06-28T16:14:34.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11678189820771686330', time=2024-06-28T17:47:23.900096+0000, chain.objs.size()=1 2024-06-28T16:14:34.196 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.aSl1hNPPfaqn1DkgKbAGCeBUqbwahN3_0 2024-06-28T16:14:34.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12514551198277415822', time=2024-06-28T17:47:25.107159+0000, chain.objs.size()=1 2024-06-28T16:14:34.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4sRda3gjeBwSd4pSBD_UuZRjIX6m77A_0 2024-06-28T16:14:34.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13213120936533539012', time=2024-06-28T17:47:25.922223+0000, chain.objs.size()=1 2024-06-28T16:14:34.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.1tHwy-iyGQoC5sI0NUtnvpI2pZ-Dt-n_0 2024-06-28T16:14:34.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14865143524504700830', time=2024-06-28T17:47:26.151573+0000, chain.objs.size()=1 2024-06-28T16:14:34.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.A6vN2dbDjXL-ogpTr8FpZkM6mIVt6Zz_0 2024-06-28T16:14:34.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9313822292757796645', time=2024-06-28T17:47:27.806552+0000, chain.objs.size()=1 2024-06-28T16:14:34.197 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.j1LIMM34k2EY37QEIazn1S_IQMttV-b_0 2024-06-28T16:14:34.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15787414903865296597', time=2024-06-28T17:47:28.956109+0000, chain.objs.size()=1 2024-06-28T16:14:34.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.A6D3IQhAZcg2nIUaeBeS2-WyhduLt-8_0 2024-06-28T16:14:34.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13410016162711741365', time=2024-06-28T17:47:30.301152+0000, chain.objs.size()=1 2024-06-28T16:14:34.198 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.187+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VZbp1HXzDTP3OKP5FkRJk-Ui9OzwSZX_0 2024-06-28T16:14:34.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.191+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10884745013599024911', time=2024-06-28T17:47:34.298062+0000, chain.objs.size()=1 2024-06-28T16:14:34.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.191+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.PRWE7lJtmwj1MOoXEfIhRLC2npzjuAp_0 2024-06-28T16:14:34.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.191+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8586806832219502126', time=2024-06-28T17:47:36.035710+0000, chain.objs.size()=1 2024-06-28T16:14:34.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.191+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.0Z8kj4V5dS6Iks6hD85yOgsdO7TQ9SB_0 2024-06-28T16:14:34.201 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.191+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18202366332602593926', time=2024-06-28T17:47:36.037324+0000, chain.objs.size()=1 2024-06-28T16:14:34.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.191+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Tt-rLtah05SHxVbwbiZOFCnWULElogQ_0 2024-06-28T16:14:34.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.191+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10581523853095599578', time=2024-06-28T17:47:38.147906+0000, chain.objs.size()=1 2024-06-28T16:14:34.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.191+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.C0QmvTdodCjJCnpY_l-qL8uGtOLGQ9c_0 2024-06-28T16:14:34.202 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.195+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12213727170122299822', time=2024-06-28T17:47:38.357720+0000, chain.objs.size()=1 2024-06-28T16:14:34.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.195+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.JdGRTiajS-dGwyGjSxTpmo3Ut2XwCAb_0 2024-06-28T16:14:34.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.195+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6441533508857893252', time=2024-06-28T17:47:39.246439+0000, chain.objs.size()=1 2024-06-28T16:14:34.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.195+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.u3I0Q6jAgAua58stnw1iG3wOUJazzQp_0 2024-06-28T16:14:34.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.195+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8561470733543468724', time=2024-06-28T17:47:39.431546+0000, chain.objs.size()=1 2024-06-28T16:14:34.203 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.195+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KEDb0iFj7O6s3UfZ6HtRxf8xDa4K52d_0 2024-06-28T16:14:34.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.195+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1007262104991496820', time=2024-06-28T17:48:30.014625+0000, chain.objs.size()=1 2024-06-28T16:14:34.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.195+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.WQAgLoGAJEsX9o1R2cUUyLf0zmnVt73_0 2024-06-28T16:14:34.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.195+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15512904978305050606', time=2024-06-28T17:48:30.016364+0000, chain.objs.size()=1 2024-06-28T16:14:34.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.195+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.bfB7JaF3LNBeKJnp5-2TTRG4zQqcuv-_0 2024-06-28T16:14:34.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.195+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17676180342695154834', time=2024-06-28T17:48:34.632395+0000, chain.objs.size()=1 2024-06-28T16:14:34.204 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.195+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.c7h1T0G-dQ-FVTBzm2ZW9_BQ5JuKGbV_0 2024-06-28T16:14:34.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.195+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4039703483984252950', time=2024-06-28T17:48:43.038408+0000, chain.objs.size()=1 2024-06-28T16:14:34.205 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.195+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.288__shadow_.SGr0HLPhj2aZa6qXD5nKRccQUAfd0-i_0 2024-06-28T16:14:34.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6575591677767231969', time=2024-06-28T17:48:44.723083+0000, chain.objs.size()=1 2024-06-28T16:14:34.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.289__shadow_.dIUuR5TAqr_EbiQXqMZtLzQwl-jJ7pX_0 2024-06-28T16:14:34.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18343976622797952864', time=2024-06-28T17:51:06.776053+0000, chain.objs.size()=1 2024-06-28T16:14:34.207 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.5NSDOBPxcl1KnVK2FvLxjzH_WmgyPiY_0 2024-06-28T16:14:34.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6076049599279267332', time=2024-06-28T17:51:09.453564+0000, chain.objs.size()=1 2024-06-28T16:14:34.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Z8DKCgZmA4wiIlz8cr5sZUK3najMLYB_0 2024-06-28T16:14:34.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15006973845448973695', time=2024-06-28T17:51:09.457800+0000, chain.objs.size()=1 2024-06-28T16:14:34.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.XfDIVo3e6WSuL_a4r6VTF4r-gnXUUEW_0 2024-06-28T16:14:34.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7753146546423700040', time=2024-06-28T17:51:09.907062+0000, chain.objs.size()=1 2024-06-28T16:14:34.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.jyY4lnRQn8fFm6sONI2W7MU8AwJl1vG_0 2024-06-28T16:14:34.208 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6316118655607338505', time=2024-06-28T17:51:11.638861+0000, chain.objs.size()=1 2024-06-28T16:14:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.jUmtuNElIYqpL4uxMivCArsm00yNN4V_0 2024-06-28T16:14:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9559918697805164043', time=2024-06-28T17:51:12.460759+0000, chain.objs.size()=1 2024-06-28T16:14:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YBuygcfLmGx_JvbM_neAxE1zlUqHnkv_0 2024-06-28T16:14:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8385610847686299578', time=2024-06-28T17:51:15.753184+0000, chain.objs.size()=1 2024-06-28T16:14:34.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PIw4WFH4rj1HoFWo4wjC9mgZnGwpn-2_0 2024-06-28T16:14:34.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8055471690280566594', time=2024-06-28T17:51:17.817816+0000, chain.objs.size()=1 2024-06-28T16:14:34.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.oNE0cRG-rCFpo89ZbdUBRPQ9iyoBptq_0 2024-06-28T16:14:34.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1030760740002312643', time=2024-06-28T17:51:18.552228+0000, chain.objs.size()=1 2024-06-28T16:14:34.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.jYveU7OrT5u82aIw5GOsR2-SDfxcSn6_0 2024-06-28T16:14:34.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9445584128695088943', time=2024-06-28T17:51:19.910315+0000, chain.objs.size()=1 2024-06-28T16:14:34.210 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.199+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.RF8Z-SyN9r2PAuloX_1O0omJpyfvP4d_0 2024-06-28T16:14:34.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9222771277898133993', time=2024-06-28T17:51:20.643920+0000, chain.objs.size()=1 2024-06-28T16:14:34.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.iekp0YVg_ufAJm3_IVUR7hwzxD1NMh-_0 2024-06-28T16:14:34.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9032731755789013973', time=2024-06-28T17:51:21.845502+0000, chain.objs.size()=1 2024-06-28T16:14:34.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.OCPO1rNxi1-8Cxoh2qUwD-b8Ex8F4rT_0 2024-06-28T16:14:34.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8957556095079136316', time=2024-06-28T17:51:22.063451+0000, chain.objs.size()=1 2024-06-28T16:14:34.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nYtjshKixqLTOwQWj2WhPT4NEL2IjBa_0 2024-06-28T16:14:34.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1477164438060352187', time=2024-06-28T17:51:22.552158+0000, chain.objs.size()=1 2024-06-28T16:14:34.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Y8M4PG6QSSws10PlcIFv0YHryfvpp7h_0 2024-06-28T16:14:34.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.502422254555702231', time=2024-06-28T17:51:22.649123+0000, chain.objs.size()=1 2024-06-28T16:14:34.213 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.daV_39TS0iYCgdbI6OnTey95VhfB2Hl_0 2024-06-28T16:14:34.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6477971995896569030', time=2024-06-28T17:51:23.952395+0000, chain.objs.size()=1 2024-06-28T16:14:34.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.0DadTth61xYakuvqv5jNfLxIo2MAkyK_0 2024-06-28T16:14:34.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4255297892115574818', time=2024-06-28T17:51:25.976850+0000, chain.objs.size()=1 2024-06-28T16:14:34.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ATfqb0--3ohZ3Wi5JEDvWkL1gG1oKXK_0 2024-06-28T16:14:34.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5020364775742883379', time=2024-06-28T17:51:26.072980+0000, chain.objs.size()=1 2024-06-28T16:14:34.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.0TX5ciz9USkbtl3Q7ZqzWGq3ZfMXCQR_0 2024-06-28T16:14:34.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12497004581965005890', time=2024-06-28T17:51:26.338556+0000, chain.objs.size()=1 2024-06-28T16:14:34.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.JmlOXg6Z09S_28dqDLqibblgHxci0f__0 2024-06-28T16:14:34.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14537400784329609524', time=2024-06-28T17:51:26.597883+0000, chain.objs.size()=1 2024-06-28T16:14:34.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1EafmCaJ3JF4aNXoU1tee2l54knDxm2_0 2024-06-28T16:14:34.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3344790790822275981', time=2024-06-28T17:51:28.456519+0000, chain.objs.size()=1 2024-06-28T16:14:34.215 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.203+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ue8kSkO77jDrXYJlGagQJMpyYXjqwKz_0 2024-06-28T16:14:34.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.789827875239476785', time=2024-06-28T17:51:32.233835+0000, chain.objs.size()=1 2024-06-28T16:14:34.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6tmRNWJj_xE1JxyPItXCxnT1vOpR8VS_0 2024-06-28T16:14:34.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17800840081098889844', time=2024-06-28T17:51:32.752888+0000, chain.objs.size()=1 2024-06-28T16:14:34.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YjPUPHgUA7mY8sVSDkB_N5u07gDJUfA_0 2024-06-28T16:14:34.217 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.207+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1115070549074528756', time=2024-06-28T17:51:32.927339+0000, chain.objs.size()=1 2024-06-28T16:14:34.218 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.207+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.iCez4KkVsrgZ_Qv54FD7PemdG8OyoZz_0 2024-06-28T16:14:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2090563803682662256', time=2024-06-28T17:51:34.363788+0000, chain.objs.size()=1 2024-06-28T16:14:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.7y019As1KwOfkRtFCxStr6fppC65Ad0_0 2024-06-28T16:14:34.219 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10494035215067630355', time=2024-06-28T17:51:35.797730+0000, chain.objs.size()=1 2024-06-28T16:14:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.XAHIKYjSSVNj3OaiNBpLilCddZR28gl_0 2024-06-28T16:14:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17087495782392770319', time=2024-06-28T17:51:35.992178+0000, chain.objs.size()=1 2024-06-28T16:14:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.gDlds4dJcG_wriuLms89QVpnP4Vt3oT_0 2024-06-28T16:14:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.159189174849592454', time=2024-06-28T17:51:37.342396+0000, chain.objs.size()=1 2024-06-28T16:14:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.OcsZF25TYRjKts2-A2AKQqXNjZBJVJ8_0 2024-06-28T16:14:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7546940499168374568', time=2024-06-28T17:51:37.980550+0000, chain.objs.size()=1 2024-06-28T16:14:34.220 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.U48aYVA7qILFRf2larKSuwXdNSOTiwG_0 2024-06-28T16:14:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14007448939775071607', time=2024-06-28T17:53:20.983037+0000, chain.objs.size()=1 2024-06-28T16:14:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.DElRcTAsl1x3OgjTnLR6BwocrV6QTNE_0 2024-06-28T16:14:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10001283820777549376', time=2024-06-28T17:53:22.224362+0000, chain.objs.size()=1 2024-06-28T16:14:34.221 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.891DeDeUQh0dSjkR_0v2Q3WssCp78ur_0 2024-06-28T16:14:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12481395717588968637', time=2024-06-28T17:53:22.676294+0000, chain.objs.size()=1 2024-06-28T16:14:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.pObzdtaRZ06JzSrDh9GMkxHDUoMeXcK_0 2024-06-28T16:14:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7997007064194288416', time=2024-06-28T17:53:24.069590+0000, chain.objs.size()=1 2024-06-28T16:14:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.P94_zdztZrCh8EU166_qhlDOMjo2Z5j_0 2024-06-28T16:14:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17864186488967730991', time=2024-06-28T17:53:25.283799+0000, chain.objs.size()=1 2024-06-28T16:14:34.222 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.MXXHZJADgyKlhSGP7PgeK1Rc_HUX9b1_0 2024-06-28T16:14:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15121778884663324741', time=2024-06-28T17:53:25.699774+0000, chain.objs.size()=1 2024-06-28T16:14:34.223 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.211+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.WMvzSP2bwyW95SbY0akkey2FW_queif_0 2024-06-28T16:14:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10250227420580680388', time=2024-06-28T17:53:26.607382+0000, chain.objs.size()=1 2024-06-28T16:14:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.E47RM8MuonUJhCLwjgpCfmhBWpEFGa1_0 2024-06-28T16:14:34.225 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17633898380220986403', time=2024-06-28T17:53:26.756511+0000, chain.objs.size()=1 2024-06-28T16:14:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZqMtTW2d24oSNV7lM_gd1O6N3iQPQ4i_0 2024-06-28T16:14:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8700285379731438313', time=2024-06-28T17:53:30.365129+0000, chain.objs.size()=1 2024-06-28T16:14:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.aybC871o9XE5lzneSLwanPExcDVcCyk_0 2024-06-28T16:14:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1888586343734579396', time=2024-06-28T17:53:30.850487+0000, chain.objs.size()=1 2024-06-28T16:14:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.TkSzjsAv5c6P0VGkuzz-qyXfsVukz5P_0 2024-06-28T16:14:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17800971948364206309', time=2024-06-28T17:53:31.122269+0000, chain.objs.size()=1 2024-06-28T16:14:34.226 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.XEM4AKvqQZarh-PSRFrAruOUHSimFi9_0 2024-06-28T16:14:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10666354276407669187', time=2024-06-28T17:53:31.511037+0000, chain.objs.size()=1 2024-06-28T16:14:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GK0EbydGe-YnkpwattR1_dAzVL76nsI_0 2024-06-28T16:14:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1847252891408059682', time=2024-06-28T17:53:32.472915+0000, chain.objs.size()=1 2024-06-28T16:14:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.s5u_KaCQCQMLjdvlFIzb6D-hQKFlOq-_0 2024-06-28T16:14:34.227 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2192809940269528729', time=2024-06-28T17:53:32.627174+0000, chain.objs.size()=1 2024-06-28T16:14:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.YeY9kjWOnvF0E-XpZdVSHwHRUgdEJgB_0 2024-06-28T16:14:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5023533595199008250', time=2024-06-28T17:53:32.628534+0000, chain.objs.size()=1 2024-06-28T16:14:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.9gill-HFPQc97zuXgGvwlaC1FCl-aal_0 2024-06-28T16:14:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18144365880149263498', time=2024-06-28T17:53:34.405395+0000, chain.objs.size()=1 2024-06-28T16:14:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.fx5Hu3fqSEINsqBeoYRheqVRm1AoIos_0 2024-06-28T16:14:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4385739431889275968', time=2024-06-28T17:53:39.267662+0000, chain.objs.size()=1 2024-06-28T16:14:34.228 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.215+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.NE5wJhfyYOBHDbK0K_kugqfuxd5iEI1_0 2024-06-28T16:14:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17434090226445637753', time=2024-06-28T17:53:39.269871+0000, chain.objs.size()=1 2024-06-28T16:14:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.75ZZqwK2Pg_whchKOmrQaZE7omgrPrF_0 2024-06-28T16:14:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18117550596046122384', time=2024-06-28T17:53:39.751589+0000, chain.objs.size()=1 2024-06-28T16:14:34.229 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.khz92RqfRbGS_o3EsclRl3wOw5W4m43_0 2024-06-28T16:14:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6473995458570011381', time=2024-06-28T17:53:40.113929+0000, chain.objs.size()=1 2024-06-28T16:14:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Lh_rutgxlcCZViJsGVTHNzpT2822OqU_0 2024-06-28T16:14:34.230 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13651463233573078967', time=2024-06-28T17:53:40.562326+0000, chain.objs.size()=1 2024-06-28T16:14:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wwBdUj2Sv_-UnJEONCNKPFolwhyr-J3_0 2024-06-28T16:14:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12096234018690742496', time=2024-06-28T17:53:40.956640+0000, chain.objs.size()=1 2024-06-28T16:14:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.btPcUOiZwywaQAW3Llxjq99yakezzq2_0 2024-06-28T16:14:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17217407600377196800', time=2024-06-28T17:53:45.019839+0000, chain.objs.size()=1 2024-06-28T16:14:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.H0HJGQBxEoFXqXouQRZOR3KCuOOKoqE_0 2024-06-28T16:14:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.219+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15186931044058194016', time=2024-06-28T17:53:46.345098+0000, chain.objs.size()=1 2024-06-28T16:14:34.231 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.219+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZQCFYGM9S9tlZ4vXyPTq1hC73Yq0Sc1_0 2024-06-28T16:14:34.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.223+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14074260425525064852', time=2024-06-28T17:53:46.791724+0000, chain.objs.size()=1 2024-06-28T16:14:34.232 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.223+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.aoz5mLjA96UsNGI6XhQww8J5lKB3CCb_0 2024-06-28T16:14:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.223+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8730161212760365397', time=2024-06-28T17:53:47.445872+0000, chain.objs.size()=1 2024-06-28T16:14:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.223+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.58ly1mFBWaWwnjX0Z3Gpbr4oRs6XxB9_0 2024-06-28T16:14:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.223+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16684995639433563513', time=2024-06-28T17:53:47.730497+0000, chain.objs.size()=1 2024-06-28T16:14:34.233 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.223+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bO2IISGwR07ODPjMd4tNKKV0x5q69k5_0 2024-06-28T16:14:34.237 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.227+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/45990 2024-06-28T16:14:34.239 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=25, truncated=0, next_marker='' 2024-06-28T16:14:34.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10062460252453531783', time=2024-06-28T17:53:48.590375+0000, chain.objs.size()=1 2024-06-28T16:14:34.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.7KUI2dGx42nR3KxuXXC-Q6lP99ZKoZf_0 2024-06-28T16:14:34.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14381791353147873909', time=2024-06-28T17:53:48.666961+0000, chain.objs.size()=1 2024-06-28T16:14:34.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.UqJ4p4_R5XXF9KpDA90-J9jga5R_Qsy_0 2024-06-28T16:14:34.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11269585506559478130', time=2024-06-28T17:53:50.960037+0000, chain.objs.size()=1 2024-06-28T16:14:34.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ljWZ9vDs-6fhU8KpYJH6kTQLwGeAnds_0 2024-06-28T16:14:34.240 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5647043169614633835', time=2024-06-28T17:53:51.179242+0000, chain.objs.size()=1 2024-06-28T16:14:34.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZlQv-pBgDu35gkBUYT1XPh_Y_0UaaoV_0 2024-06-28T16:14:34.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14313415898652871706', time=2024-06-28T17:53:51.681530+0000, chain.objs.size()=1 2024-06-28T16:14:34.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_._IO5mWtsQYVfyXATGynQDfDJzTzN_VG_0 2024-06-28T16:14:34.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3616030805490055773', time=2024-06-28T17:53:51.844002+0000, chain.objs.size()=1 2024-06-28T16:14:34.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.OttwISFnnVlPaq8oVZ3jWiNBrI8aECG_0 2024-06-28T16:14:34.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.549417319094351840', time=2024-06-28T17:54:40.606981+0000, chain.objs.size()=1 2024-06-28T16:14:34.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.369__shadow_.3zdLk4m99EK1Y_92XNK8CD4mF1vSHkl_0 2024-06-28T16:14:34.241 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4907918587533766488', time=2024-06-28T17:55:46.391391+0000, chain.objs.size()=1 2024-06-28T16:14:34.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.437__shadow_.vEgpFj0omamkyL7VoqWPuzFjg3HEymW_0 2024-06-28T16:14:34.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5631458728761248551', time=2024-06-28T17:56:28.139550+0000, chain.objs.size()=1 2024-06-28T16:14:34.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.473__shadow_.tblD4uj474WSwq6H5tqNhPyLaboAxn4_0 2024-06-28T16:14:34.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1918850593022973749', time=2024-06-28T17:56:39.100745+0000, chain.objs.size()=1 2024-06-28T16:14:34.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.479__shadow_.HBtimoVt5SzBtvhWPWQUS8aYv-SvhZs_0 2024-06-28T16:14:34.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.2_d1611a62b185c414a589524dd14f305b', time=2024-06-28T17:57:08.979315+0000, chain.objs.size()=6 2024-06-28T16:14:34.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.1 2024-06-28T16:14:34.242 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.231+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.2 2024-06-28T16:14:34.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.3 2024-06-28T16:14:34.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.4 2024-06-28T16:14:34.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.5 2024-06-28T16:14:34.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.6 2024-06-28T16:14:34.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11923910262914713406', time=2024-06-28T17:57:28.639394+0000, chain.objs.size()=1 2024-06-28T16:14:34.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.507__shadow_.r62FlrfuAr7Lz83auH2DZH5vQA1WBsc_0 2024-06-28T16:14:34.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5737296177600895368', time=2024-06-28T17:58:10.521698+0000, chain.objs.size()=3 2024-06-28T16:14:34.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.511__multipart_mymultipart.2~lQas6hjeyZn_5CeY-yiIf7qhT4pdj_y.1 2024-06-28T16:14:34.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.511__multipart_mymultipart.2~lQas6hjeyZn_5CeY-yiIf7qhT4pdj_y.2 2024-06-28T16:14:34.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.511__multipart_mymultipart.2~lQas6hjeyZn_5CeY-yiIf7qhT4pdj_y.3 2024-06-28T16:14:34.245 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~QNBtUMmzeRqaHJCWiGvRtIJx3LM7Ap-', time=2024-06-28T17:58:50.949997+0000, chain.objs.size()=1 2024-06-28T16:14:34.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~QNBtUMmzeRqaHJCWiGvRtIJx3LM7Ap-.3 2024-06-28T16:14:34.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16664605075386074903', time=2024-06-28T18:01:15.615472+0000, chain.objs.size()=1 2024-06-28T16:14:34.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.541__shadow_.Npb4KdwEzJRhnVndWKZ58IjHphN8y9f_0 2024-06-28T16:14:34.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10048483466946870836', time=2024-06-28T18:01:27.103265+0000, chain.objs.size()=1 2024-06-28T16:14:34.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.235+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.545__shadow_.G9HUFkFeTN4rRALoCWG7zXfvZfgcqAh_0 2024-06-28T16:14:34.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15601984821826274186', time=2024-06-28T18:01:55.028014+0000, chain.objs.size()=1 2024-06-28T16:14:34.249 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.555__shadow_.VhIWtP4UmvfSkdGEDcK5-XT50WCe5u5_0 2024-06-28T16:14:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7914825711541189547', time=2024-06-28T18:03:15.240969+0000, chain.objs.size()=1 2024-06-28T16:14:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:hxLHf68MeiXOZBqu-ZmbojsbfRcwd5n_.SmvJon-cyET0xij4nY-CYqHYXXXmdGU_0 2024-06-28T16:14:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.214126061139000447', time=2024-06-28T18:03:19.747601+0000, chain.objs.size()=1 2024-06-28T16:14:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:rukptj-d0oVSG3T04S5eljAzYjheC7l_.5MAVk8Ywcn-nq_WBaHdP84IFtrNn398_0 2024-06-28T16:14:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5663990669808130856', time=2024-06-28T18:03:28.075198+0000, chain.objs.size()=1 2024-06-28T16:14:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:iwU0YcuEojteqf3U5kFFg6M6EeHP1DZ_.mJAy9QZDNxXVcNQpN9iWAGMVpqnan2J_0 2024-06-28T16:14:34.250 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.932293401047792032', time=2024-06-28T18:04:13.867306+0000, chain.objs.size()=1 2024-06-28T16:14:34.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.576__shadow:SdDVG-KaAacC9oT6pj18tVUaRK1kLnm_.S0NLP_G1gh9xOXUxlDZHFCdT7iyI1mS_0 2024-06-28T16:14:34.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.533023465992676361', time=2024-06-28T18:04:27.209974+0000, chain.objs.size()=1 2024-06-28T16:14:34.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.592__shadow_.ju0v7uArwB7Ghy3xnJuiPMsX3C94vhO_0 2024-06-28T16:14:34.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1915893319541473002', time=2024-06-28T18:07:17.224358+0000, chain.objs.size()=1 2024-06-28T16:14:34.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.633__shadow_.0hcjWf24EESmnWzRtaAQzHETfYptBNZ_0 2024-06-28T16:14:34.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8924174104632891697', time=2024-06-28T18:08:28.922489+0000, chain.objs.size()=1 2024-06-28T16:14:34.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.678__shadow:zumehEI0NdZ.DoTiOwJzlLaLZ4VUL9q_.dtujSf5hi3roYih5mTe3vcYWu4Xev5S_0 2024-06-28T16:14:34.251 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4700425025709264913', time=2024-06-28T18:08:32.189960+0000, chain.objs.size()=1 2024-06-28T16:14:34.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.239+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.681__shadow:d-rC..8bIoz68hOMvdDoDT2rfLL5AgK_.-BlK1LQIhSmdZ09VuqQUgI4O74zqcqF_0 2024-06-28T16:14:34.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.247+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:34.259 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-06-28T16:14:34.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:34.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c0f12640 0 -- 172.21.15.178:0/890311433 >> [v2:172.21.15.178:6808/4178996824,v1:172.21.15.178:6809/4178996824] conn(0x7f00b80343c0 msgr2=0x7eff8c092300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-06-28T16:14:34.261 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:34.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/40850' 2024-06-28T16:14:34.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3149742290478744996', time=2024-06-28T17:44:35.754527+0000, chain.objs.size()=1 2024-06-28T16:14:34.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.221__shadow_.1gNDFdS0nl86xH7AfMHdHkaHUmYpjqD_0 2024-06-28T16:14:34.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1719406659619471611', time=2024-06-28T17:44:54.048783+0000, chain.objs.size()=1 2024-06-28T16:14:34.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.232__shadow_.4xQv00dVaxiXp_XP16tT4uiwW3YJql7_0 2024-06-28T16:14:34.262 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16770212954279761955', time=2024-06-28T17:45:19.340527+0000, chain.objs.size()=1 2024-06-28T16:14:34.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.247__shadow_.oh7TB2yL_59VUTxxwqfjRIftbUgIZbV_0 2024-06-28T16:14:34.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1209707135918244076', time=2024-06-28T17:47:08.305224+0000, chain.objs.size()=1 2024-06-28T16:14:34.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.JI9M6QhnxEvSuArqp5ziQYGV9VhYyII_0 2024-06-28T16:14:34.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10155422899201523662', time=2024-06-28T17:47:09.330576+0000, chain.objs.size()=1 2024-06-28T16:14:34.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.coqqbu1-NOIF_T0YwAxp7x3A0pXFtS3_0 2024-06-28T16:14:34.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10054829869808486077', time=2024-06-28T17:47:11.136848+0000, chain.objs.size()=1 2024-06-28T16:14:34.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.b3TwGOrFrt4RkwyJC6mRRlusah3mD1M_0 2024-06-28T16:14:34.263 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1275313996321562918', time=2024-06-28T17:47:13.133806+0000, chain.objs.size()=1 2024-06-28T16:14:34.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.D2P4q3ak6jeJbUp5IBakBAEYoDTePGA_0 2024-06-28T16:14:34.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14958950323615811085', time=2024-06-28T17:47:14.991290+0000, chain.objs.size()=1 2024-06-28T16:14:34.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-pu3ru1M_guGiMwLqpVDm3gRzEcvL1Y_0 2024-06-28T16:14:34.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6847087616506497130', time=2024-06-28T17:47:15.365801+0000, chain.objs.size()=1 2024-06-28T16:14:34.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.cia9Unyt6RYyubUhaUHg03yawvtmRqM_0 2024-06-28T16:14:34.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15773632613179679760', time=2024-06-28T17:47:16.675711+0000, chain.objs.size()=1 2024-06-28T16:14:34.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.hOMCNLTPVKWIye40Miyc7YN03H_iJ79_0 2024-06-28T16:14:34.264 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8281917714556615410', time=2024-06-28T17:47:17.225217+0000, chain.objs.size()=1 2024-06-28T16:14:34.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.kp-d6LWC4WWJSJEhH6lUVBCYOTFCjTN_0 2024-06-28T16:14:34.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7650537729222811856', time=2024-06-28T17:47:18.764524+0000, chain.objs.size()=1 2024-06-28T16:14:34.265 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.251+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Iwkyn_byci9YLbIYaKEID4Vr8DnapI0_0 2024-06-28T16:14:34.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.255+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6343870146828016293', time=2024-06-28T17:47:21.363862+0000, chain.objs.size()=1 2024-06-28T16:14:34.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.255+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.NwFg6O4j57xZOVWjJopm9y-LGk89I1P_0 2024-06-28T16:14:34.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.255+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4078327172994016044', time=2024-06-28T17:47:21.492660+0000, chain.objs.size()=1 2024-06-28T16:14:34.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.255+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.o6SMUdtU-4_-_E20GuHhtSQuCzC9YJn_0 2024-06-28T16:14:34.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9803084681716695006', time=2024-06-28T17:47:25.931617+0000, chain.objs.size()=1 2024-06-28T16:14:34.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.T4-N5wILtmCAJPqqXpoPKy0PmUqpP5__0 2024-06-28T16:14:34.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18347685126297953871', time=2024-06-28T17:47:26.257469+0000, chain.objs.size()=1 2024-06-28T16:14:34.267 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-EOi_WjExTe-KHK3hF5gJE_3EGkrsD6_0 2024-06-28T16:14:34.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.232430792356379763', time=2024-06-28T17:47:27.143550+0000, chain.objs.size()=1 2024-06-28T16:14:34.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.K2BuSUkti9vnAhZK67GBhd7ae978R3l_0 2024-06-28T16:14:34.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16889599309898294928', time=2024-06-28T17:47:27.511780+0000, chain.objs.size()=1 2024-06-28T16:14:34.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ng9PfNymUo1HAeRqkHPrPyXZlqmlSOJ_0 2024-06-28T16:14:34.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16534362520856848280', time=2024-06-28T17:47:27.654245+0000, chain.objs.size()=1 2024-06-28T16:14:34.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.slWn2lID5DvqX7WiNKDs0x0mjvymYFO_0 2024-06-28T16:14:34.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8420691100818381510', time=2024-06-28T17:47:27.788679+0000, chain.objs.size()=1 2024-06-28T16:14:34.268 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.2oGpLX7O5pBnzVfIo39S8V9mL3mXGkb_0 2024-06-28T16:14:34.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17570122289723973139', time=2024-06-28T17:47:28.136510+0000, chain.objs.size()=1 2024-06-28T16:14:34.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5rFKjq2uLS3MyHJZHo6i2o5--0Lrtjo_0 2024-06-28T16:14:34.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7922985618405451440', time=2024-06-28T17:47:29.483739+0000, chain.objs.size()=1 2024-06-28T16:14:34.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.xQ8uFyYmvHtjmMsmvkWiJlbsCS12_Nt_0 2024-06-28T16:14:34.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1694759438286302420', time=2024-06-28T17:47:30.687609+0000, chain.objs.size()=1 2024-06-28T16:14:34.269 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.259+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.gDHBm3dAEup4jqKjqlAJO7hbJg5qngp_0 2024-06-28T16:14:34.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10727106471516645091', time=2024-06-28T17:47:31.680554+0000, chain.objs.size()=1 2024-06-28T16:14:34.271 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.tqIQvrJFnVSgUPkk38PNhURzryfHEWt_0 2024-06-28T16:14:34.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.646576922615430900', time=2024-06-28T17:47:33.062818+0000, chain.objs.size()=1 2024-06-28T16:14:34.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dl_1zWR4hLZEMrGG8MtB-AJ_EJmT5Sq_0 2024-06-28T16:14:34.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18331243764043773479', time=2024-06-28T17:47:34.090321+0000, chain.objs.size()=1 2024-06-28T16:14:34.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.qxfH1mqzLbJzDkTnbV9DD0avCLMx-AU_0 2024-06-28T16:14:34.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14226777915619392543', time=2024-06-28T17:47:34.946963+0000, chain.objs.size()=1 2024-06-28T16:14:34.272 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.AbW5DKlsDEc9En7EYlJn9yosHKMp4It_0 2024-06-28T16:14:34.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6810954433164756615', time=2024-06-28T17:47:35.721696+0000, chain.objs.size()=1 2024-06-28T16:14:34.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.E46rvbkg_wmzXEXbiwEY9t6ciG0g-5F_0 2024-06-28T16:14:34.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.470748667292433991', time=2024-06-28T17:47:38.325159+0000, chain.objs.size()=1 2024-06-28T16:14:34.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4YfTeXX5Uy8Hw1-P2SmgM-ihag_ciVQ_0 2024-06-28T16:14:34.274 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14863878988190371827', time=2024-06-28T17:47:38.728788+0000, chain.objs.size()=1 2024-06-28T16:14:34.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.gy3gUyxzSG2PRE1u0HTwpDGeUWBIxKm_0 2024-06-28T16:14:34.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1457777566332135776', time=2024-06-28T17:47:39.587031+0000, chain.objs.size()=1 2024-06-28T16:14:34.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dvTzPhOkSJvlQ1_5GDyt4YpPk3-OQiL_0 2024-06-28T16:14:34.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10306101179712765485', time=2024-06-28T17:47:49.959719+0000, chain.objs.size()=1 2024-06-28T16:14:34.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.259__shadow_.OIF5JgN-6tgQz64gBQVEPjYfKfcIudt_0 2024-06-28T16:14:34.275 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16114731869863040246', time=2024-06-28T17:47:51.424179+0000, chain.objs.size()=1 2024-06-28T16:14:34.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.260__shadow_.77QNivmlEhagOR1sCsCxDBCgUusOriE_0 2024-06-28T16:14:34.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1056514673546502155', time=2024-06-28T17:48:11.672792+0000, chain.objs.size()=1 2024-06-28T16:14:34.276 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.263+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.273__shadow_.9X1HC4jvdkLw5LedHbDpaQXcH8GyHu9_0 2024-06-28T16:14:34.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13891041530212627863', time=2024-06-28T17:48:21.410286+0000, chain.objs.size()=1 2024-06-28T16:14:34.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.279__shadow_.3dYHtNnEJCUmly1hipBCNXDP1H29weJ_0 2024-06-28T16:14:34.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9485265017145427990', time=2024-06-28T17:48:34.853440+0000, chain.objs.size()=1 2024-06-28T16:14:34.277 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.ZP1NHlMMr7Jy9I8r9TlCyCB3azjlKV5_0 2024-06-28T16:14:34.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14609416836212805227', time=2024-06-28T17:48:39.763980+0000, chain.objs.size()=1 2024-06-28T16:14:34.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.286__shadow_.g_Gg7PunQpF_1Glx-Hm9wRIiuTVlMJk_0 2024-06-28T16:14:34.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5061201698047146798', time=2024-06-28T17:48:41.343505+0000, chain.objs.size()=1 2024-06-28T16:14:34.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.287__shadow_.4L3nuJ8mzp_24Ow7xbHVICUTUP9YgUM_0 2024-06-28T16:14:34.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2786078971654260628', time=2024-06-28T17:48:46.315789+0000, chain.objs.size()=1 2024-06-28T16:14:34.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.290__shadow_.6fOlOUjs1s6xYazmJyz3INWdedZEYLM_0 2024-06-28T16:14:34.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5805523347189601115', time=2024-06-28T17:48:49.547715+0000, chain.objs.size()=1 2024-06-28T16:14:34.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.292__shadow_.k8SkE1ovv736HoKw3SFj9A_2YD6EcMl_0 2024-06-28T16:14:34.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2796999979981142171', time=2024-06-28T17:48:56.351896+0000, chain.objs.size()=1 2024-06-28T16:14:34.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.296__shadow_.0-lqByibxGNVs4L_anQg0qvD_dFgCqJ_0 2024-06-28T16:14:34.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15942309973316292250', time=2024-06-28T17:49:06.090150+0000, chain.objs.size()=1 2024-06-28T16:14:34.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.267+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.303__shadow_.qUBOGDRJI3wNRDP15e8R_7Y3a07PNso_0 2024-06-28T16:14:34.280 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11112728034014393005', time=2024-06-28T17:51:06.782332+0000, chain.objs.size()=1 2024-06-28T16:14:34.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Leisuulx1SHWauCcyy-wqrogE1K_-xI_0 2024-06-28T16:14:34.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10717287328492261448', time=2024-06-28T17:51:06.783695+0000, chain.objs.size()=1 2024-06-28T16:14:34.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.O8jazMfwFrFQSWgjlWv7psBjaCWdQkg_0 2024-06-28T16:14:34.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15447208541362836228', time=2024-06-28T17:51:11.876703+0000, chain.objs.size()=1 2024-06-28T16:14:34.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.MxtLSRZssfcHi3KikLkVtoUfjQMrKKI_0 2024-06-28T16:14:34.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15069259952886601563', time=2024-06-28T17:51:12.837928+0000, chain.objs.size()=1 2024-06-28T16:14:34.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.X5uRoQFVyYd3-EH7XXKTTdhjpPVO4kv_0 2024-06-28T16:14:34.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3858432079522726189', time=2024-06-28T17:51:12.840124+0000, chain.objs.size()=1 2024-06-28T16:14:34.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.kPyhkINNUzUKtlX4x4BEr2Zh1Alwhm3_0 2024-06-28T16:14:34.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4629871632089707440', time=2024-06-28T17:51:13.277579+0000, chain.objs.size()=1 2024-06-28T16:14:34.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.NNRWtlQKicTriN_7bHsaKHUgQUOLwa2_0 2024-06-28T16:14:34.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.271+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1648377135084444084', time=2024-06-28T17:51:13.666056+0000, chain.objs.size()=1 2024-06-28T16:14:34.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.271+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.SqoL8HMKZXirur_JpUvDo7kuSPsBNGq_0 2024-06-28T16:14:34.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13069571648851544655', time=2024-06-28T17:51:15.780508+0000, chain.objs.size()=1 2024-06-28T16:14:34.283 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.TvYa6JvbctxDIDv110IBuGo0PlrCt6D_0 2024-06-28T16:14:34.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16690940290057915161', time=2024-06-28T17:51:20.545221+0000, chain.objs.size()=1 2024-06-28T16:14:34.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.isWHSoLiQAMG-vewT39Kl0pY0IeCMCj_0 2024-06-28T16:14:34.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18071237085873775450', time=2024-06-28T17:51:21.182679+0000, chain.objs.size()=1 2024-06-28T16:14:34.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.OwwPDdYjV_WBX8SuUVqtTW0ZHw_-d4H_0 2024-06-28T16:14:34.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.275+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14713595029564826217', time=2024-06-28T17:51:21.286995+0000, chain.objs.size()=1 2024-06-28T16:14:34.285 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.275+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ufr-Sk9oLGXwRkzyOKiX9E7FdWjWoMe_0 2024-06-28T16:14:34.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2966511373458755307', time=2024-06-28T17:51:21.629111+0000, chain.objs.size()=1 2024-06-28T16:14:34.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.awUTne-RQZ-fpfgu9CXzr69ZCczNh_z_0 2024-06-28T16:14:34.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12949277922751557892', time=2024-06-28T17:51:23.967542+0000, chain.objs.size()=1 2024-06-28T16:14:34.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.BReHR2iwgrST7lQlYtTFWAGcHmE9bgY_0 2024-06-28T16:14:34.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13957191476300540393', time=2024-06-28T17:51:24.667073+0000, chain.objs.size()=1 2024-06-28T16:14:34.288 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.wZVYOmJSe7djLLoxcvzypUX5r4UMEMD_0 2024-06-28T16:14:34.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10321849296608193876', time=2024-06-28T17:51:27.723476+0000, chain.objs.size()=1 2024-06-28T16:14:34.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_._6cMo2V9BI3Ub-vKsYln8jdqarhV-9x_0 2024-06-28T16:14:34.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12593180572296048301', time=2024-06-28T17:51:28.340205+0000, chain.objs.size()=1 2024-06-28T16:14:34.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nLUAmcnu0h6zL8H-FjyNVO3-Yq7Kxoh_0 2024-06-28T16:14:34.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2153331077924322463', time=2024-06-28T17:51:29.407615+0000, chain.objs.size()=1 2024-06-28T16:14:34.289 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1lBHOAYGSW9ibPozC8SEBISkcq6tCXM_0 2024-06-28T16:14:34.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4737631802510056389', time=2024-06-28T17:51:29.577645+0000, chain.objs.size()=1 2024-06-28T16:14:34.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.0lbys3lQ1STr3ZSD6IVI1NWPeJZ3Q_c_0 2024-06-28T16:14:34.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7915721242852645948', time=2024-06-28T17:51:30.045953+0000, chain.objs.size()=1 2024-06-28T16:14:34.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.279+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-7LQpExkvDESDZwXYNxAT6CSEYaVN8p_0 2024-06-28T16:14:34.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11630387268145152414', time=2024-06-28T17:51:32.034805+0000, chain.objs.size()=1 2024-06-28T16:14:34.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Mo1BU-DFurUNGIl6DdPMJpjSD0SqMi8_0 2024-06-28T16:14:34.292 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16698664832007823105', time=2024-06-28T17:51:32.563131+0000, chain.objs.size()=1 2024-06-28T16:14:34.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.9HvTRAOSlphI87_noKksbf6sFvZcYAj_0 2024-06-28T16:14:34.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2629369884947993250', time=2024-06-28T17:51:34.610442+0000, chain.objs.size()=1 2024-06-28T16:14:34.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.4Qj_sEpSQLuIbP7pbd9pHnILtAxG1Kz_0 2024-06-28T16:14:34.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4263550049156813038', time=2024-06-28T17:51:36.628302+0000, chain.objs.size()=1 2024-06-28T16:14:34.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.slxFTAacb2bJAuqwCMqPNOJeNICZFX8_0 2024-06-28T16:14:34.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14407397489268369796', time=2024-06-28T17:51:36.854382+0000, chain.objs.size()=1 2024-06-28T16:14:34.293 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.oBnzQuf65vI1C-UQACSekKsaXG3XCrq_0 2024-06-28T16:14:34.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.964646706564475632', time=2024-06-28T17:51:38.126025+0000, chain.objs.size()=1 2024-06-28T16:14:34.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.DR6q5f7RucqLdQXe5hsU5FEDY7Cmmos_0 2024-06-28T16:14:34.294 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8843362075578725966', time=2024-06-28T17:51:38.394645+0000, chain.objs.size()=1 2024-06-28T16:14:34.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ROmUFRxQBA_dzfhStbaD5HN7uRGbUJb_0 2024-06-28T16:14:34.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5540817053257983554', time=2024-06-28T17:51:38.396523+0000, chain.objs.size()=1 2024-06-28T16:14:34.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.283+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ZTo8HLRwLy6K9nkClR3Yd2QJYgtjG6e_0 2024-06-28T16:14:34.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7178947400667801775', time=2024-06-28T17:53:21.492731+0000, chain.objs.size()=1 2024-06-28T16:14:34.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.OIZXILpUHvCy_9V1jZl8HyoeDkLUOd-_0 2024-06-28T16:14:34.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2869515875488902', time=2024-06-28T17:53:21.853471+0000, chain.objs.size()=1 2024-06-28T16:14:34.295 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.tV9pujjqWfvVsJu-F9fcMwt9q0N272y_0 2024-06-28T16:14:34.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.522906771748009591', time=2024-06-28T17:53:21.995073+0000, chain.objs.size()=1 2024-06-28T16:14:34.296 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.a0P9hzz3AKLl2v46PKJcvrDz-4gJFM3_0 2024-06-28T16:14:34.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17174414523192990149', time=2024-06-28T17:53:22.751068+0000, chain.objs.size()=1 2024-06-28T16:14:34.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-4dpwgY9thqYMSZevIC46xP0QUZ2HIJ_0 2024-06-28T16:14:34.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16592506432782957273', time=2024-06-28T17:53:27.094540+0000, chain.objs.size()=1 2024-06-28T16:14:34.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.xSmwx4rI4bx8Qx8vhb7taCdL6DgiOvi_0 2024-06-28T16:14:34.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12759109222047671757', time=2024-06-28T17:53:28.453789+0000, chain.objs.size()=1 2024-06-28T16:14:34.297 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.O3URf-RNeHIKa0llsYFzqVzYYfv7ydB_0 2024-06-28T16:14:34.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11972647141448403974', time=2024-06-28T17:53:28.455320+0000, chain.objs.size()=1 2024-06-28T16:14:34.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.onOkL9hUZDiYvH8HAfk-3Y9PAp5VBd-_0 2024-06-28T16:14:34.298 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3245186567275370399', time=2024-06-28T17:53:31.764428+0000, chain.objs.size()=1 2024-06-28T16:14:34.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.N5oECYZSCKT58yUBwpGECWoi9A8RpZd_0 2024-06-28T16:14:34.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18077886183219258753', time=2024-06-28T17:53:32.076405+0000, chain.objs.size()=1 2024-06-28T16:14:34.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.QKX5r1SYxGxCL-qqSJMt2sf4fkXHDTP_0 2024-06-28T16:14:34.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2929021420805713086', time=2024-06-28T17:53:32.180912+0000, chain.objs.size()=1 2024-06-28T16:14:34.299 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.287+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.6vnYhaQ7wWTL66E95ikxU4RX7_UwvhU_0 2024-06-28T16:14:34.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10113134872186075959', time=2024-06-28T17:53:33.817945+0000, chain.objs.size()=1 2024-06-28T16:14:34.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.EZR1tQoxwobFDM6tdxfqp465NfT4vRn_0 2024-06-28T16:14:34.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2235887126456494384', time=2024-06-28T17:53:34.776563+0000, chain.objs.size()=1 2024-06-28T16:14:34.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.8Kx5b34lHmDg-B7y_eUhuABYjMDUZvK_0 2024-06-28T16:14:34.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15761732039701098667', time=2024-06-28T17:53:34.858967+0000, chain.objs.size()=1 2024-06-28T16:14:34.300 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.pOrYk6VCJo9usC6qrCxuuEGfxqO5-hy_0 2024-06-28T16:14:34.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14433423502529969402', time=2024-06-28T17:53:34.861236+0000, chain.objs.size()=1 2024-06-28T16:14:34.301 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.9jPqpUl3heDvniJwcmGMJINIWVlf6_Z_0 2024-06-28T16:14:34.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9781185610270896616', time=2024-06-28T17:53:35.061044+0000, chain.objs.size()=1 2024-06-28T16:14:34.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.2zFSKDjxYt1ifF9-XXkOyV3CDk6EUXl_0 2024-06-28T16:14:34.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6099481290258461786', time=2024-06-28T17:53:35.260574+0000, chain.objs.size()=1 2024-06-28T16:14:34.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.as6uD4FVUwg4pbVAxDCRjbYRFPl-WsU_0 2024-06-28T16:14:34.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4484926512063171306', time=2024-06-28T17:53:38.288472+0000, chain.objs.size()=1 2024-06-28T16:14:34.302 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.2Il24Wj4feoc-ORvQ1jThfR9A0fgywv_0 2024-06-28T16:14:34.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7301077001282654091', time=2024-06-28T17:53:39.387314+0000, chain.objs.size()=1 2024-06-28T16:14:34.303 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.291+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.YBhG6SWzAyXCDR434AvNHL6YC77zyKV_0 2024-06-28T16:14:34.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10197064122702999136', time=2024-06-28T17:53:40.905360+0000, chain.objs.size()=1 2024-06-28T16:14:34.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GAl0lDTmwDqiBPGP3LHeaN6B8KbNYrm_0 2024-06-28T16:14:34.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3523963610595986728', time=2024-06-28T17:53:42.656645+0000, chain.objs.size()=1 2024-06-28T16:14:34.306 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.6kCEaEDkzZfwCRjCDs6SgAuRNB1d9tL_0 2024-06-28T16:14:34.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.988458930328957586', time=2024-06-28T17:53:44.548082+0000, chain.objs.size()=1 2024-06-28T16:14:34.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_._yAP8szVdxHY5fEZBcrijqhhqoh7WTt_0 2024-06-28T16:14:34.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10928351079755087659', time=2024-06-28T17:53:45.919833+0000, chain.objs.size()=1 2024-06-28T16:14:34.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.R1S8YuPJgODuuok-H2YodRIY2GXMS2E_0 2024-06-28T16:14:34.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2100686094313509544', time=2024-06-28T17:53:45.921840+0000, chain.objs.size()=1 2024-06-28T16:14:34.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.nsIwU9jMgCH6yGlUtGSQMbD6c0q9MGC_0 2024-06-28T16:14:34.307 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7004574113409304097', time=2024-06-28T17:53:48.737990+0000, chain.objs.size()=1 2024-06-28T16:14:34.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.U_68NXLNbcJbHw_ne4IaqsBKOG1AXss_0 2024-06-28T16:14:34.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.295+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9956124905633823304', time=2024-06-28T17:53:50.420557+0000, chain.objs.size()=1 2024-06-28T16:14:34.308 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.295+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.W0KexGrd0j5ElRupVXmkFWY3N9274Om_0 2024-06-28T16:14:34.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4852359508085577646', time=2024-06-28T17:53:50.811179+0000, chain.objs.size()=1 2024-06-28T16:14:34.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4wgof5Jc1D38yF9iZ0S8PoMMkah-2UD_0 2024-06-28T16:14:34.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12067877415529887021', time=2024-06-28T17:53:51.296735+0000, chain.objs.size()=1 2024-06-28T16:14:34.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.PNo51JmVITXUIAIy6AkG0iB9eQ57cs6_0 2024-06-28T16:14:34.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4162157552590066295', time=2024-06-28T17:54:46.558936+0000, chain.objs.size()=1 2024-06-28T16:14:34.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.374__shadow_.SXdkf55HtiBVOlAlS6EJNhLATr3E3fA_0 2024-06-28T16:14:34.309 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15642645451523333123', time=2024-06-28T17:56:49.904950+0000, chain.objs.size()=1 2024-06-28T16:14:34.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.489__shadow_.tsA53JmfkJoyXHBK6RIFTS3XaQXOHVQ_0 2024-06-28T16:14:34.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~NfT0D8bLVyyZAgfSytxCDSpO4_EjdPN', time=2024-06-28T17:59:05.960157+0000, chain.objs.size()=1 2024-06-28T16:14:34.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~NfT0D8bLVyyZAgfSytxCDSpO4_EjdPN.2 2024-06-28T16:14:34.310 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.299+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~NfT0D8bLVyyZAgfSytxCDSpO4_EjdPN', time=2024-06-28T17:59:05.969981+0000, chain.objs.size()=1 2024-06-28T16:14:34.311 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.299+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~NfT0D8bLVyyZAgfSytxCDSpO4_EjdPN.3 2024-06-28T16:14:34.313 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.303+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/40850 2024-06-28T16:14:34.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=8, truncated=0, next_marker='' 2024-06-28T16:14:34.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6671182923052376139', time=2024-06-28T18:02:04.695660+0000, chain.objs.size()=1 2024-06-28T16:14:34.315 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:y42i-L7.E96OJhb4qcELII2V2.niq.n_.3Ym1O0UgFiif3eSv-nO9H7DMW1-gSbL_0 2024-06-28T16:14:34.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17695957123195636543', time=2024-06-28T18:03:56.453372+0000, chain.objs.size()=1 2024-06-28T16:14:34.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.568__shadow:Swmi5fEKDkB2wu9F-ivSBGjk4Tkh1F9_.qLggOeoAa3kqyPXNVgKZMNmZmliMsm__0 2024-06-28T16:14:34.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8318360945423791865', time=2024-06-28T18:04:13.776024+0000, chain.objs.size()=1 2024-06-28T16:14:34.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.576__shadow:zVmRx31BA3nbxDdEb7V-6lSnNGmcIgp_.xOxgWAXKl48xTKj7lRzIR2chPpCDese_0 2024-06-28T16:14:34.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7727247169116330197', time=2024-06-28T18:04:28.727060+0000, chain.objs.size()=1 2024-06-28T16:14:34.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.593__shadow_.i03YjhDHVWYSC6xp4ZT4g91Tx2cY7gk_0 2024-06-28T16:14:34.316 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13765847285388592093', time=2024-06-28T18:06:03.106304+0000, chain.objs.size()=1 2024-06-28T16:14:34.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.607__shadow_.TaxQkf81xxkuQ42DXHSstH5dMXx70QY_0 2024-06-28T16:14:34.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16554211169607490087', time=2024-06-28T18:06:53.916279+0000, chain.objs.size()=1 2024-06-28T16:14:34.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.618__shadow_.JpYJp79xfHIw7ZYY-pt_oeUKJl8Rc16_0 2024-06-28T16:14:34.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5058636240318313613', time=2024-06-28T18:08:05.134267+0000, chain.objs.size()=1 2024-06-28T16:14:34.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.651__shadow_.27YBSpQgHxw5StUYgl3qY6yteCUAuXV_0 2024-06-28T16:14:34.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1953522468878877777', time=2024-06-28T18:08:26.858989+0000, chain.objs.size()=1 2024-06-28T16:14:34.317 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.307+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.676__shadow_.hbxuf1ECPaSNljNe0NpsLTQ3dEYwggo_0 2024-06-28T16:14:34.319 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.311+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:34.321 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.311+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-06-28T16:14:34.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.311+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:34.323 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:34.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/44825' 2024-06-28T16:14:34.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11141409663516344095', time=2024-06-28T17:41:48.130139+0000, chain.objs.size()=1 2024-06-28T16:14:34.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.190__shadow_.1W6pl2VsxcLgSls2EbWQd4k0ZtvUGbp_0 2024-06-28T16:14:34.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15162639444505074331', time=2024-06-28T17:44:42.229890+0000, chain.objs.size()=1 2024-06-28T16:14:34.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.225__shadow_.G28_0j-5LN01vDX9ta_EzRiUrzqUK2I_0 2024-06-28T16:14:34.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1315481431064569069', time=2024-06-28T17:45:01.771382+0000, chain.objs.size()=1 2024-06-28T16:14:34.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.236__shadow_.iIfhKDBy7_bdoX5ksn_N8TeR4vMFcMw_0 2024-06-28T16:14:34.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5687001070590906711', time=2024-06-28T17:45:22.316116+0000, chain.objs.size()=1 2024-06-28T16:14:34.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.249__shadow_.rhNeNcG1Eumlwvdk1AAHzwbvWO7K_GH_0 2024-06-28T16:14:34.325 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18316534487607548344', time=2024-06-28T17:47:08.063816+0000, chain.objs.size()=1 2024-06-28T16:14:34.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.c0HukvlkcwLpqeHzPGrFk9ph2O9z0ns_0 2024-06-28T16:14:34.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12618688389403304489', time=2024-06-28T17:47:08.903453+0000, chain.objs.size()=1 2024-06-28T16:14:34.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.cdz6L1jwmH6cd29dM62vQ4vMVkqTvyO_0 2024-06-28T16:14:34.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5902809354734861014', time=2024-06-28T17:47:09.905422+0000, chain.objs.size()=1 2024-06-28T16:14:34.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.gJ0wi01NLo2GWxIh2oCiL62fnzyXKPg_0 2024-06-28T16:14:34.326 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11946551310799576885', time=2024-06-28T17:47:12.876849+0000, chain.objs.size()=1 2024-06-28T16:14:34.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4Jj_S6VfKT75xRsdCyHSiNscZVPk7qi_0 2024-06-28T16:14:34.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3791871578593193219', time=2024-06-28T17:47:13.501089+0000, chain.objs.size()=1 2024-06-28T16:14:34.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VS1BmPac4FLVLNKPzg70se57bWbHqRZ_0 2024-06-28T16:14:34.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14006098853485196443', time=2024-06-28T17:47:13.972093+0000, chain.objs.size()=1 2024-06-28T16:14:34.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.h6z6uA9e9qnQxyTKQ9-tbcAFIUWKKX0_0 2024-06-28T16:14:34.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16125295123941922454', time=2024-06-28T17:47:14.991504+0000, chain.objs.size()=1 2024-06-28T16:14:34.327 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.zkhtAWxfVL0GO3isVzdY2EcmoF20wwC_0 2024-06-28T16:14:34.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9649860365470733736', time=2024-06-28T17:47:17.643403+0000, chain.objs.size()=1 2024-06-28T16:14:34.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.315+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.yOW_vCQIoStTjzIAyooxfL4OjKyqsgf_0 2024-06-28T16:14:34.328 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8321129372010439892', time=2024-06-28T17:47:18.328211+0000, chain.objs.size()=1 2024-06-28T16:14:34.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.rsMvAcTdfnDnfr70KEc2f-xspj1P69T_0 2024-06-28T16:14:34.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2313844367193294693', time=2024-06-28T17:47:19.704236+0000, chain.objs.size()=1 2024-06-28T16:14:34.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.AUkK_sSJw9RJ_hJ5XVtMlRsWPubswcr_0 2024-06-28T16:14:34.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12295497555613691275', time=2024-06-28T17:47:19.792633+0000, chain.objs.size()=1 2024-06-28T16:14:34.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.cjoW0WKv6MYA3o5yced2aiC_w695nKD_0 2024-06-28T16:14:34.329 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2873885220704244451', time=2024-06-28T17:47:20.835571+0000, chain.objs.size()=1 2024-06-28T16:14:34.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-D2T1sa-47uhb85ONLKY0N8KC_d-wbP_0 2024-06-28T16:14:34.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7330574809806805973', time=2024-06-28T17:47:20.837395+0000, chain.objs.size()=1 2024-06-28T16:14:34.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.g3I7to_wKzLQ1ojSBkROQtWZ_I7CsGS_0 2024-06-28T16:14:34.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10957160631564023053', time=2024-06-28T17:47:21.502315+0000, chain.objs.size()=1 2024-06-28T16:14:34.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.REOWxz8Ut6uEsWGRpo2OrvgTQSJ8lXW_0 2024-06-28T16:14:34.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12913606516432242773', time=2024-06-28T17:47:21.736997+0000, chain.objs.size()=1 2024-06-28T16:14:34.330 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.UzGOl9CHFT5iC1VytBDJBlnygKhniuf_0 2024-06-28T16:14:34.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10301832477097060833', time=2024-06-28T17:47:21.738610+0000, chain.objs.size()=1 2024-06-28T16:14:34.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.aW3vuzUOgS9qqJYtTEemqLgHUfVv23t_0 2024-06-28T16:14:34.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4238537656109096656', time=2024-06-28T17:47:22.817944+0000, chain.objs.size()=1 2024-06-28T16:14:34.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.xH1mxW1_jULEtgZbm8epqDdfqCw0Efj_0 2024-06-28T16:14:34.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7747137333345729400', time=2024-06-28T17:47:24.019867+0000, chain.objs.size()=1 2024-06-28T16:14:34.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oLyPsaYavUyl1TbfTk_DgFdRa7KzOaK_0 2024-06-28T16:14:34.331 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8460117630725048619', time=2024-06-28T17:47:25.895363+0000, chain.objs.size()=1 2024-06-28T16:14:34.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.319+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.wUPdrWs59EPt5igzrKOC-yMUkyYDNc1_0 2024-06-28T16:14:34.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2553687323936154955', time=2024-06-28T17:47:25.896707+0000, chain.objs.size()=1 2024-06-28T16:14:34.332 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.BPDIZTixGbqNZdTYRU9D96ZJiOIR4xQ_0 2024-06-28T16:14:34.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7788584253153717887', time=2024-06-28T17:47:26.846706+0000, chain.objs.size()=1 2024-06-28T16:14:34.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.hqEzAfzLunRN6rTcdGsm68e3rG-pCOm_0 2024-06-28T16:14:34.334 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7142702386239954479', time=2024-06-28T17:47:28.224843+0000, chain.objs.size()=1 2024-06-28T16:14:34.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.nWGjf2ezXyyaRTuokH74GwyeImxNTTh_0 2024-06-28T16:14:34.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13718757898335045818', time=2024-06-28T17:47:29.002903+0000, chain.objs.size()=1 2024-06-28T16:14:34.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.OEuCv_T2jjxGzua5pTl4Oi46d1vENmC_0 2024-06-28T16:14:34.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3979203053238401760', time=2024-06-28T17:47:29.230701+0000, chain.objs.size()=1 2024-06-28T16:14:34.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5UXTb4znhT5F_u9HAIM79JT8yqNiVbE_0 2024-06-28T16:14:34.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6623021082862536252', time=2024-06-28T17:47:29.364858+0000, chain.objs.size()=1 2024-06-28T16:14:34.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.AbYDjdXWT5AjITvCUt83175R3v8a31B_0 2024-06-28T16:14:34.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5375441079927960078', time=2024-06-28T17:47:29.427873+0000, chain.objs.size()=1 2024-06-28T16:14:34.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dhSfrHMtD0bFCYakVluDNHMXaNrsrF1_0 2024-06-28T16:14:34.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14129632126968727632', time=2024-06-28T17:47:31.363385+0000, chain.objs.size()=1 2024-06-28T16:14:34.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.qDiGdzPWp4npLa3P5qQT3ebrzIaFdL0_0 2024-06-28T16:14:34.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4336742448100701234', time=2024-06-28T17:47:32.996264+0000, chain.objs.size()=1 2024-06-28T16:14:34.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oDS8tYh1y-AuzeMFvhTxCNDvqjzIWxh_0 2024-06-28T16:14:34.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1051683312927058367', time=2024-06-28T17:47:33.478984+0000, chain.objs.size()=1 2024-06-28T16:14:34.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.vFfRw_jq90T8o14tCsq3yFtvMCAeLyi_0 2024-06-28T16:14:34.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4840792634702405029', time=2024-06-28T17:47:35.280964+0000, chain.objs.size()=1 2024-06-28T16:14:34.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.323+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Egcw4Ubf2SeLvWwa01nzLGX8hr_3ioL_0 2024-06-28T16:14:34.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.327+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1322953517156206433', time=2024-06-28T17:47:36.970573+0000, chain.objs.size()=1 2024-06-28T16:14:34.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.327+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.FRW1TyWc1hNQEEmsVUu1-hb7FiIc5ds_0 2024-06-28T16:14:34.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2407119784497296916', time=2024-06-28T17:47:36.972103+0000, chain.objs.size()=1 2024-06-28T16:14:34.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.08pMuphUnzVUEhWUKfIsijZeEsdLjmV_0 2024-06-28T16:14:34.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12532624811201025566', time=2024-06-28T17:47:36.974206+0000, chain.objs.size()=1 2024-06-28T16:14:34.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.qQ0VVEhUT-Sr7qcQ4PeOwvBvmI730b4_0 2024-06-28T16:14:34.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15290628540908798680', time=2024-06-28T17:48:16.676086+0000, chain.objs.size()=1 2024-06-28T16:14:34.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.276__shadow_.gq9ZMWB5kVrZ42M5LWDlEh4iWu12q-f_0 2024-06-28T16:14:34.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10930413659450308675', time=2024-06-28T17:48:16.677792+0000, chain.objs.size()=1 2024-06-28T16:14:34.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.276__shadow_.IuDSvL3PimwD-PyaeTB8lHklS4eGrU__0 2024-06-28T16:14:34.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17060491488184742057', time=2024-06-28T17:48:21.416363+0000, chain.objs.size()=1 2024-06-28T16:14:34.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.279__shadow_.xp02I6NaY6-qzX09bJCoFpPnVaC8DOQ_0 2024-06-28T16:14:34.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10083945973012103121', time=2024-06-28T17:48:22.999616+0000, chain.objs.size()=1 2024-06-28T16:14:34.341 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.280__shadow_.JmBIFNQrhHpkECv-UHjM5raYcexrGTi_0 2024-06-28T16:14:34.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3235256177285019997', time=2024-06-28T17:48:36.479601+0000, chain.objs.size()=1 2024-06-28T16:14:34.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.284__shadow_.bW72QDR9aG1Q5GGCPF2WqJmLcApK5c6_0 2024-06-28T16:14:34.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3839871392021354709', time=2024-06-28T17:48:38.072384+0000, chain.objs.size()=1 2024-06-28T16:14:34.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.285__shadow_.4AfQpF4QnHJmBwUVK_LqSM3A9wju8Pb_0 2024-06-28T16:14:34.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17024429311945479987', time=2024-06-28T17:51:06.761472+0000, chain.objs.size()=1 2024-06-28T16:14:34.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.HF3GCt93uVkhT1U0z4sVMPVW38zbdDi_0 2024-06-28T16:14:34.342 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1768951792898691401', time=2024-06-28T17:51:07.221018+0000, chain.objs.size()=1 2024-06-28T16:14:34.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.331+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.WPO48iK0-IxUhjUZvLo70U-fzW62dM6_0 2024-06-28T16:14:34.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.335+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9033592583493149894', time=2024-06-28T17:51:07.588071+0000, chain.objs.size()=1 2024-06-28T16:14:34.344 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.USgI3nPA6MXAuT87UuHUfMPKYdNbqos_0 2024-06-28T16:14:34.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.335+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11992596823652874878', time=2024-06-28T17:51:08.854623+0000, chain.objs.size()=1 2024-06-28T16:14:34.345 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GaC3xDQRBidgfYXWn5JWuuT2_dBpwKy_0 2024-06-28T16:14:34.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.335+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5430927180868254621', time=2024-06-28T17:51:09.219831+0000, chain.objs.size()=1 2024-06-28T16:14:34.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.WirGPtYuyZtEmOzGVKqiiX2I3WYL5WW_0 2024-06-28T16:14:34.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.335+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15307521552573301732', time=2024-06-28T17:51:11.405013+0000, chain.objs.size()=1 2024-06-28T16:14:34.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GsfccDDBd_e8sqws9nvGi0mA7FGEisL_0 2024-06-28T16:14:34.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.335+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11140414763063909672', time=2024-06-28T17:51:12.035938+0000, chain.objs.size()=1 2024-06-28T16:14:34.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.OrfHfJbEjF6f9WGA6_ifq3Z8dQXMCTu_0 2024-06-28T16:14:34.346 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.335+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11662741982422133182', time=2024-06-28T17:51:12.666757+0000, chain.objs.size()=1 2024-06-28T16:14:34.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.C0chv4EM7eeTuNnZ_o6v91FygvIJVZt_0 2024-06-28T16:14:34.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.335+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1725062108505284590', time=2024-06-28T17:51:12.985378+0000, chain.objs.size()=1 2024-06-28T16:14:34.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.335+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-9_HIVvlMXSkyeTQMZPs6CEQ4HCeFnG_0 2024-06-28T16:14:34.348 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4140552373023779120', time=2024-06-28T17:51:12.988460+0000, chain.objs.size()=1 2024-06-28T16:14:34.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_._EctV15cyMfZTM4uupW3oMeoPqvenhj_0 2024-06-28T16:14:34.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3220598913309783328', time=2024-06-28T17:51:13.833014+0000, chain.objs.size()=1 2024-06-28T16:14:34.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.768rqR9s18rxV_aPbKjXUy15PLwGo9Y_0 2024-06-28T16:14:34.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12286002854810263225', time=2024-06-28T17:51:14.255112+0000, chain.objs.size()=1 2024-06-28T16:14:34.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.kT8sWyN0xgn2Oucr7H1wymymFS4znv5_0 2024-06-28T16:14:34.349 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14346206862614393059', time=2024-06-28T17:51:16.327935+0000, chain.objs.size()=1 2024-06-28T16:14:34.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ex6hsjI_k4m5m1eWXDuk45Ihz3D8Jwj_0 2024-06-28T16:14:34.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16704258724940651292', time=2024-06-28T17:51:17.360864+0000, chain.objs.size()=1 2024-06-28T16:14:34.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.NGo_Qmgel3t7HcSb6nyX0ng2_SOLSCR_0 2024-06-28T16:14:34.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10101259662835362055', time=2024-06-28T17:51:17.709218+0000, chain.objs.size()=1 2024-06-28T16:14:34.350 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xONd8HjHff1kt869NxC2btgSxrpfFij_0 2024-06-28T16:14:34.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17838287970660542348', time=2024-06-28T17:51:20.162101+0000, chain.objs.size()=1 2024-06-28T16:14:34.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.V8T0YMwT3-Y0uummb1nLOUNSNJfWB5__0 2024-06-28T16:14:34.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18078900383258264287', time=2024-06-28T17:51:20.466335+0000, chain.objs.size()=1 2024-06-28T16:14:34.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.cduQV3392DOkAC4w2MHRoQlE5c39DhM_0 2024-06-28T16:14:34.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15079678884197539906', time=2024-06-28T17:51:20.773174+0000, chain.objs.size()=1 2024-06-28T16:14:34.351 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.7hphO5LhXghyGjZM6yQv8SEMwbA-ytL_0 2024-06-28T16:14:34.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6865563971665096106', time=2024-06-28T17:51:21.067295+0000, chain.objs.size()=1 2024-06-28T16:14:34.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Btszzz5JTdJRJqu7DtYRXc5QOG_Num4_0 2024-06-28T16:14:34.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2664074828015889378', time=2024-06-28T17:51:21.226781+0000, chain.objs.size()=1 2024-06-28T16:14:34.352 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.339+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.8ootcL1gv4r1aNpy9pmx1mjVWHK9ibK_0 2024-06-28T16:14:34.353 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17852088663277540942', time=2024-06-28T17:51:21.480932+0000, chain.objs.size()=1 2024-06-28T16:14:34.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GmfZpeyewMR-HI8IOHBjm1s8X4BfxOi_0 2024-06-28T16:14:34.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13572396332107136368', time=2024-06-28T17:51:21.682067+0000, chain.objs.size()=1 2024-06-28T16:14:34.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nl0Sfep_PuwlHPA88X9j-pwu3Py0bfW_0 2024-06-28T16:14:34.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10786091531452738713', time=2024-06-28T17:51:24.916429+0000, chain.objs.size()=1 2024-06-28T16:14:34.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.NrljsMxYeMWs9Zl_hIGrU2DYCrsbOwU_0 2024-06-28T16:14:34.354 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3612880207575257801', time=2024-06-28T17:51:25.604907+0000, chain.objs.size()=1 2024-06-28T16:14:34.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.WHs6gIJUDdZYtD-fMMu4qJzlVkqsksk_0 2024-06-28T16:14:34.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10921563790755437000', time=2024-06-28T17:51:25.790269+0000, chain.objs.size()=1 2024-06-28T16:14:34.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Kw7HkkOgoLOB_bT2JXpG2XRE7Q1TjW4_0 2024-06-28T16:14:34.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9848004406112448679', time=2024-06-28T17:51:26.294547+0000, chain.objs.size()=1 2024-06-28T16:14:34.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.wpCOkGWNBz1owhGlZH0gmk_ZR0q11Vm_0 2024-06-28T16:14:34.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2600579533940608129', time=2024-06-28T17:51:27.660215+0000, chain.objs.size()=1 2024-06-28T16:14:34.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.VwSgeK-nlMeoqmUZlNdIxrBz-pHWkI4_0 2024-06-28T16:14:34.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3227916574791329109', time=2024-06-28T17:51:27.742331+0000, chain.objs.size()=1 2024-06-28T16:14:34.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yRJ89Sfa9QuLGso_Pwb2YU4zT8s5m2C_0 2024-06-28T16:14:34.356 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17874575872819936507', time=2024-06-28T17:51:33.190331+0000, chain.objs.size()=1 2024-06-28T16:14:34.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PoO4kzwjPAI37vBN5A_ZjdHE2KK00Ou_0 2024-06-28T16:14:34.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14908109472315684732', time=2024-06-28T17:51:33.365480+0000, chain.objs.size()=1 2024-06-28T16:14:34.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.HZKFIJ5DusTH6V2GzVd8pEdxzS_BfW9_0 2024-06-28T16:14:34.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13606137517774167479', time=2024-06-28T17:51:37.567595+0000, chain.objs.size()=1 2024-06-28T16:14:34.357 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.343+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.3qBERKvKO0QmUrZvQ5aF2vH9minn8f9_0 2024-06-28T16:14:34.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13497953097901809309', time=2024-06-28T17:53:20.939861+0000, chain.objs.size()=1 2024-06-28T16:14:34.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Hp4J47DAUtzLPZYCplBofgDs6V4Flua_0 2024-06-28T16:14:34.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13402636189357051336', time=2024-06-28T17:53:21.259580+0000, chain.objs.size()=1 2024-06-28T16:14:34.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.OwrfIo57aXntnl7KdEA2UOWzmg_3S17_0 2024-06-28T16:14:34.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5726442575018232589', time=2024-06-28T17:53:23.697747+0000, chain.objs.size()=1 2024-06-28T16:14:34.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1HlQRPSl7AIq40PZGkHiGmdYY33SAFo_0 2024-06-28T16:14:34.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4046751969169017899', time=2024-06-28T17:53:27.392881+0000, chain.objs.size()=1 2024-06-28T16:14:34.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.5cfVveqBCK1LqigYkuQDJfEa6KsLbzf_0 2024-06-28T16:14:34.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18348133666127603781', time=2024-06-28T17:53:28.235115+0000, chain.objs.size()=1 2024-06-28T16:14:34.360 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.7KWQPIIv89GwOzNL0dV9nOhfM5Tg2FM_0 2024-06-28T16:14:34.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6340512035944444931', time=2024-06-28T17:53:29.700239+0000, chain.objs.size()=1 2024-06-28T16:14:34.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.HfzSl9WTfqrQ1Ijt7nxWNHM2kN9d6K7_0 2024-06-28T16:14:34.361 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13958618481761641111', time=2024-06-28T17:53:30.319741+0000, chain.objs.size()=1 2024-06-28T16:14:34.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.w90CXZv7QDNb4dViVr16q9W52HFN3Vw_0 2024-06-28T16:14:34.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3726330397343728929', time=2024-06-28T17:53:32.639199+0000, chain.objs.size()=1 2024-06-28T16:14:34.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.3BC-cYHgcxrei-Uzc2CX9r2uTPhbDeb_0 2024-06-28T16:14:34.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15939862909434950214', time=2024-06-28T17:53:34.184891+0000, chain.objs.size()=1 2024-06-28T16:14:34.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vXVm5-i78yj25k01ToEzEGRCcBXvsu9_0 2024-06-28T16:14:34.362 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6345983605103969785', time=2024-06-28T17:53:34.776697+0000, chain.objs.size()=1 2024-06-28T16:14:34.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.6vvozXWM8Dh2l8mCA5yXYpNrwWUI5kv_0 2024-06-28T16:14:34.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11979584538582218880', time=2024-06-28T17:53:35.799005+0000, chain.objs.size()=1 2024-06-28T16:14:34.363 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.351+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.3a33Cr1mBp6tjEmj4WfKLk453tvcNdP_0 2024-06-28T16:14:34.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13441756195786525878', time=2024-06-28T17:53:36.295091+0000, chain.objs.size()=1 2024-06-28T16:14:34.367 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.HoLBXpxGNbKVX2rDT05FbYXGap82p84_0 2024-06-28T16:14:34.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13088175163954467120', time=2024-06-28T17:53:36.669322+0000, chain.objs.size()=1 2024-06-28T16:14:34.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.z-LMVU3sQcqP70wX6j95SNzTSabCGcZ_0 2024-06-28T16:14:34.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7273435797095240571', time=2024-06-28T17:53:37.802712+0000, chain.objs.size()=1 2024-06-28T16:14:34.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wotVvD_UQbIB1faAUlVGTXJY-aNauSj_0 2024-06-28T16:14:34.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17008558547004179860', time=2024-06-28T17:53:39.213230+0000, chain.objs.size()=1 2024-06-28T16:14:34.368 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.MR8hYbWyy6l9nqxiUntBiFWZ72Pom6x_0 2024-06-28T16:14:34.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12147488021160553772', time=2024-06-28T17:53:39.215795+0000, chain.objs.size()=1 2024-06-28T16:14:34.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.AAvVPw_cR4tLSjk634uK1-7oBIoorkJ_0 2024-06-28T16:14:34.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2141329881164515602', time=2024-06-28T17:53:39.252743+0000, chain.objs.size()=1 2024-06-28T16:14:34.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wXNUbHXhaeaUyv1vP3RtjPk7BlHnLm-_0 2024-06-28T16:14:34.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15513795190257901732', time=2024-06-28T17:53:39.484820+0000, chain.objs.size()=1 2024-06-28T16:14:34.371 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.B_CELbg73Rj3NZr5Bp5CX5UY7qjeQHY_0 2024-06-28T16:14:34.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16663458580729145951', time=2024-06-28T17:53:40.776445+0000, chain.objs.size()=1 2024-06-28T16:14:34.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.359+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.nkBpNeYAT4QD_R6E65oCw-JsQqrzLi0_0 2024-06-28T16:14:34.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9690099153063774459', time=2024-06-28T17:53:44.518234+0000, chain.objs.size()=1 2024-06-28T16:14:34.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.etETVjObv2F5taa0DZaU-4F_3Rx7O06_0 2024-06-28T16:14:34.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5193521961412269016', time=2024-06-28T17:53:44.993594+0000, chain.objs.size()=1 2024-06-28T16:14:34.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.fO5m4DKnyaKXWnfVNqwTysSeKEPOZMq_0 2024-06-28T16:14:34.372 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7415884991162610753', time=2024-06-28T17:53:48.112248+0000, chain.objs.size()=1 2024-06-28T16:14:34.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.uc1pwTjlLptQYczKVYfN327_riREtzE_0 2024-06-28T16:14:34.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14847620285882222256', time=2024-06-28T17:53:49.660438+0000, chain.objs.size()=1 2024-06-28T16:14:34.373 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.LtQsOjLeTs_iRnJtTWXZv8BRPZsLm0u_0 2024-06-28T16:14:34.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13824628916901757125', time=2024-06-28T17:53:49.686532+0000, chain.objs.size()=1 2024-06-28T16:14:34.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.RQ4MP4Vd-vZtounuO5svbsvYdc61q9L_0 2024-06-28T16:14:34.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6322534407701099281', time=2024-06-28T17:53:50.215474+0000, chain.objs.size()=1 2024-06-28T16:14:34.374 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.tMFT8c0E4kM1v7T3QYBU7kbAN9LDK6U_0 2024-06-28T16:14:34.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.363+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10759093370506004219', time=2024-06-28T17:53:51.074086+0000, chain.objs.size()=1 2024-06-28T16:14:34.375 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.363+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.H-VNwMO35Dn5RLTXUyTelJSM2kcBtnT_0 2024-06-28T16:14:34.377 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.367+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/44825 2024-06-28T16:14:34.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=20, truncated=0, next_marker='' 2024-06-28T16:14:34.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14825647488598371923', time=2024-06-28T17:53:51.243632+0000, chain.objs.size()=1 2024-06-28T16:14:34.380 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.iIWrNuHijCPi47r4W87eNrEDs9wL7Qa_0 2024-06-28T16:14:34.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7342278972893938880', time=2024-06-28T17:53:51.616819+0000, chain.objs.size()=1 2024-06-28T16:14:34.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.RUp8WK3Qeei7v4rcL0bHN5ZopMu94l7_0 2024-06-28T16:14:34.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1821720874118198637', time=2024-06-28T17:54:38.214152+0000, chain.objs.size()=1 2024-06-28T16:14:34.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.367__shadow_.dilGxpA40qTJnpWBEcOVOc4VAcdTVlg_0 2024-06-28T16:14:34.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12799954324181160653', time=2024-06-28T17:56:04.886800+0000, chain.objs.size()=1 2024-06-28T16:14:34.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.451__shadow_.s95rn4Xa_9fISoUBX-qF3sGLd6immq1_0 2024-06-28T16:14:34.381 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2668538792110299861', time=2024-06-28T17:56:10.174037+0000, chain.objs.size()=1 2024-06-28T16:14:34.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.454__shadow_.U0DRRk30qKyyepQ9AJrBlQ3D4Bin3p__0 2024-06-28T16:14:34.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.2_ecd622e0394b1a131703acde0358a6b8', time=2024-06-28T17:57:09.480140+0000, chain.objs.size()=6 2024-06-28T16:14:34.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.1 2024-06-28T16:14:34.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.2 2024-06-28T16:14:34.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.3 2024-06-28T16:14:34.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.4 2024-06-28T16:14:34.382 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.5 2024-06-28T16:14:34.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.497__multipart_srcmultipart.2~Z8hMDJkc4tcfBI6YABDcr-X7UC66viB.6 2024-06-28T16:14:34.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17351837531679972965', time=2024-06-28T17:57:28.616804+0000, chain.objs.size()=1 2024-06-28T16:14:34.383 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.371+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.507__shadow_.syf1802nirHwrB2VOYjrB0gJtW4aQ3h_0 2024-06-28T16:14:34.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13167674078086556535', time=2024-06-28T17:59:38.319412+0000, chain.objs.size()=2 2024-06-28T16:14:34.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.513__multipart_mymultipart.2~deLQDFwKobr6Go7icLyRLNuB-DyMxya.1 2024-06-28T16:14:34.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.513__multipart_mymultipart.2~deLQDFwKobr6Go7icLyRLNuB-DyMxya.2 2024-06-28T16:14:34.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.90649582599949749', time=2024-06-28T18:00:46.175252+0000, chain.objs.size()=4 2024-06-28T16:14:34.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.525__multipart_mymultipart.2~JZ5WK0E6QAirjm-LPZBTgBGxIdPdMnV.1 2024-06-28T16:14:34.384 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.525__multipart_mymultipart.2~JZ5WK0E6QAirjm-LPZBTgBGxIdPdMnV.2 2024-06-28T16:14:34.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.525__multipart_mymultipart.i1QX89qvdLUM8a2p5BXWSAzHoprGyKd9.3 2024-06-28T16:14:34.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.525__multipart_mymultipart.2~JZ5WK0E6QAirjm-LPZBTgBGxIdPdMnV.4 2024-06-28T16:14:34.386 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14100182473994733822', time=2024-06-28T18:02:20.084444+0000, chain.objs.size()=1 2024-06-28T16:14:34.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:tbNyhOWk1quG6z-tlcKINqMvR2WqFK8_.N6VoS0swgObrU_Zdi90qjXlKgBMmkQg_0 2024-06-28T16:14:34.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6724043515708687064', time=2024-06-28T18:02:37.204415+0000, chain.objs.size()=1 2024-06-28T16:14:34.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.563__shadow:XEdIUPvv8Jnm8UL9KVfCKHCH6u6NJ84_.ZUmMW1SxqwBO3IvHzEqYTmUkiLdFG-J_0 2024-06-28T16:14:34.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='vjc3luv8y9hr6ax77x95tqws3quxk17n', time=2024-06-28T18:02:39.185001+0000, chain.objs.size()=1 2024-06-28T16:14:34.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.564__shadow_.px_UYFSpyiaCBXG7zMb6eNn1PETwRxE_0 2024-06-28T16:14:34.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9220867787183988476', time=2024-06-28T18:03:18.800744+0000, chain.objs.size()=1 2024-06-28T16:14:34.387 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:i.UNFAjb-5voLp6XWUpEOVjgrEOC-Vg_.6DiziFp2vOE3RmhQBNn6haRXD6FRfDC_0 2024-06-28T16:14:34.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7317986061218025606', time=2024-06-28T18:04:32.605513+0000, chain.objs.size()=1 2024-06-28T16:14:34.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.597__shadow_.RXGci1TBukhkg117FGfH256Yyi3PzlG_0 2024-06-28T16:14:34.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12794475636057634104', time=2024-06-28T18:06:49.628161+0000, chain.objs.size()=1 2024-06-28T16:14:34.388 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.375+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.614__shadow_.0auYUuWAC9WkF9bwMzxo8BBF4wWZBDc_0 2024-06-28T16:14:34.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.379+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2825903963827497334', time=2024-06-28T18:06:52.975207+0000, chain.objs.size()=1 2024-06-28T16:14:34.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.379+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.617__shadow_.SF9R2EcHj57thqbXa9D3H_SsGZFt-5L_0 2024-06-28T16:14:34.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.379+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4335771676963373525', time=2024-06-28T18:08:01.140705+0000, chain.objs.size()=1 2024-06-28T16:14:34.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.379+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.649__shadow_.kS7yWin7ruID-WV2u_eqekkuSLh8WGu_0 2024-06-28T16:14:34.390 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.379+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15822805105737776825', time=2024-06-28T18:08:10.209081+0000, chain.objs.size()=1 2024-06-28T16:14:34.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.379+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.656__shadow_.uNgB8QtQf5wz0HeGlIuUgvSxBgZvL_e_0 2024-06-28T16:14:34.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.379+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8856598849609236319', time=2024-06-28T18:08:53.628904+0000, chain.objs.size()=1 2024-06-28T16:14:34.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.379+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.698__shadow:PijuoaRx9aJhoXt1NZ3IXwupaw7VRAs_.8BH1OsWFH-O9GZVd3XknhLApML42HN0_0 2024-06-28T16:14:34.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.379+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11394880954489828365', time=2024-06-28T18:09:18.409910+0000, chain.objs.size()=1 2024-06-28T16:14:34.391 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.379+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.703__shadow_.ntjfTUhMHFRYNd9bEtiv7Ab-Oima2We_0 2024-06-28T16:14:34.395 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.387+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:34.398 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.387+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-06-28T16:14:34.399 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:34.400 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/45320' 2024-06-28T16:14:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9822486570915579246', time=2024-06-28T17:39:51.612777+0000, chain.objs.size()=1 2024-06-28T16:14:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.138__shadow_.fXKxqYePao9PhxkKgdsfAkX_fP8zx1q_0 2024-06-28T16:14:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15324808512458422175', time=2024-06-28T17:44:33.998085+0000, chain.objs.size()=1 2024-06-28T16:14:34.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.220__shadow_.I1x0nDOao0SuGJQV8tBmy6XyffFt7nh_0 2024-06-28T16:14:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15310095684056447778', time=2024-06-28T17:44:39.040168+0000, chain.objs.size()=1 2024-06-28T16:14:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.223__shadow_.nrw53BrW1tdUKo4dnhPUHSsN5Lhawwx_0 2024-06-28T16:14:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9334402401637785200', time=2024-06-28T17:45:11.444872+0000, chain.objs.size()=1 2024-06-28T16:14:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.242__shadow_.sL5TefQYfkh5-8uBbBYx6tsfrZ6-FRE_0 2024-06-28T16:14:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4321941673112691934', time=2024-06-28T17:47:07.850704+0000, chain.objs.size()=1 2024-06-28T16:14:34.402 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.edqUQLklT1l5Pn_I_HA37cp007FY6LZ_0 2024-06-28T16:14:34.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14530554436920000912', time=2024-06-28T17:47:08.350319+0000, chain.objs.size()=1 2024-06-28T16:14:34.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.bgyJ2jswZzQ4jI2WMBMDlg5ViwF-9I2_0 2024-06-28T16:14:34.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9787542896664824467', time=2024-06-28T17:47:12.836926+0000, chain.objs.size()=1 2024-06-28T16:14:34.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.rAYe5Srt4EKqFKGoWqiMP0_cEOo6QwJ_0 2024-06-28T16:14:34.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9728920553850260257', time=2024-06-28T17:47:16.650265+0000, chain.objs.size()=1 2024-06-28T16:14:34.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.t8__mN7BVzX3_2siqGcfaNnDxq7zEno_0 2024-06-28T16:14:34.403 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4154939960334629408', time=2024-06-28T17:47:18.329687+0000, chain.objs.size()=1 2024-06-28T16:14:34.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.vW1WtOftzMo24Km6IPRgbmh226KoCHO_0 2024-06-28T16:14:34.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2810006694435606281', time=2024-06-28T17:47:21.069021+0000, chain.objs.size()=1 2024-06-28T16:14:34.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.AzVbmkohl0m8p0TVclqHONfcg-M6LxJ_0 2024-06-28T16:14:34.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5484399454668628960', time=2024-06-28T17:47:21.256374+0000, chain.objs.size()=1 2024-06-28T16:14:34.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.BtIkRa0nGtx7WO2jrxASsRN2hbwX80S_0 2024-06-28T16:14:34.404 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12151755263475967190', time=2024-06-28T17:47:21.439591+0000, chain.objs.size()=1 2024-06-28T16:14:34.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.391+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oNLA8519R626DqF0i3RcnLpnOF0uR54_0 2024-06-28T16:14:34.405 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9172749352273304625', time=2024-06-28T17:47:22.865391+0000, chain.objs.size()=1 2024-06-28T16:14:34.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.xrQjR6rYA93WkMM0oD9mONVskxrUUUa_0 2024-06-28T16:14:34.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7378770153151229919', time=2024-06-28T17:47:23.741382+0000, chain.objs.size()=1 2024-06-28T16:14:34.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ahfWhRtxPT46rm-XLsIExbS-CT3Bshl_0 2024-06-28T16:14:34.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11198118935617225149', time=2024-06-28T17:47:25.030814+0000, chain.objs.size()=1 2024-06-28T16:14:34.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.GeBUfGnjPagCYQwSXGOKmwyDkq-zFnk_0 2024-06-28T16:14:34.406 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1239064929560360811', time=2024-06-28T17:47:28.202671+0000, chain.objs.size()=1 2024-06-28T16:14:34.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.zAS9XPxbjnmHLS76szfbtiaSQjo6Uld_0 2024-06-28T16:14:34.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3022232537651151583', time=2024-06-28T17:47:29.021879+0000, chain.objs.size()=1 2024-06-28T16:14:34.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.gvl8E9y_jnfA7sKd8fBReTQRbNc4ng7_0 2024-06-28T16:14:34.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12775944727945044375', time=2024-06-28T17:47:29.942858+0000, chain.objs.size()=1 2024-06-28T16:14:34.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.SOepqu_B51avyvFvCtWKPKjp-szYWJT_0 2024-06-28T16:14:34.407 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.353728440558187196', time=2024-06-28T17:47:30.078306+0000, chain.objs.size()=1 2024-06-28T16:14:34.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.UENqtBD0ZTzdEf-0hrwsmifHCnWqdaO_0 2024-06-28T16:14:34.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3788187275275780263', time=2024-06-28T17:47:30.191230+0000, chain.objs.size()=1 2024-06-28T16:14:34.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4xGaNtr2GDZ_RRCLy7qah29dIRGUfqW_0 2024-06-28T16:14:34.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15090569340035476618', time=2024-06-28T17:47:30.639060+0000, chain.objs.size()=1 2024-06-28T16:14:34.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.OOoVHORZu7fqG_DmgVSyHPCqAeuKwvV_0 2024-06-28T16:14:34.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2117788444322187842', time=2024-06-28T17:47:30.640372+0000, chain.objs.size()=1 2024-06-28T16:14:34.408 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.fm3-ba0j-nEN9QUyZMlF7BGQqgSecFe_0 2024-06-28T16:14:34.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6914922388369929358', time=2024-06-28T17:47:30.641816+0000, chain.objs.size()=1 2024-06-28T16:14:34.409 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.395+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.vKw8MwmSXmtMNZVNYLHFLzurtA8st-g_0 2024-06-28T16:14:34.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9829374377968933476', time=2024-06-28T17:47:33.324562+0000, chain.objs.size()=1 2024-06-28T16:14:34.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ZgeTg6JVobPncgIngT1JcEAD3pZcib7_0 2024-06-28T16:14:34.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11458475252630993820', time=2024-06-28T17:47:33.726597+0000, chain.objs.size()=1 2024-06-28T16:14:34.411 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.tKF91KBkx1faS-He70a8ROGT65bbFVP_0 2024-06-28T16:14:34.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16700153631794458898', time=2024-06-28T17:47:34.625356+0000, chain.objs.size()=1 2024-06-28T16:14:34.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4o8NBaofoPAcMK92aQ2bMWDvojJntqc_0 2024-06-28T16:14:34.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5746396925888688315', time=2024-06-28T17:47:35.723244+0000, chain.objs.size()=1 2024-06-28T16:14:34.412 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.NpRhh8sWfmZjeavCYRg5ruxRdlC4wDq_0 2024-06-28T16:14:34.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2416553827002912722', time=2024-06-28T17:47:35.956501+0000, chain.objs.size()=1 2024-06-28T16:14:34.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Zynqh1kfxQ_ctWZyBd-0EABA0_UAFD6_0 2024-06-28T16:14:34.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5453884555569861806', time=2024-06-28T17:47:36.969137+0000, chain.objs.size()=1 2024-06-28T16:14:34.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.WJOZ-mjKYN7jYJTauf6Zny4LJfZ3NEn_0 2024-06-28T16:14:34.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14956838029285119925', time=2024-06-28T17:47:37.373257+0000, chain.objs.size()=1 2024-06-28T16:14:34.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.pt32QkEY4ShxgHW_bmXW6X5_PutG1FL_0 2024-06-28T16:14:34.413 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10117962854787307178', time=2024-06-28T17:47:37.570914+0000, chain.objs.size()=1 2024-06-28T16:14:34.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.sz7mcIr_hRwpDddJx6DPJr8wIh4CDeQ_0 2024-06-28T16:14:34.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14413981294010080087', time=2024-06-28T17:47:37.955626+0000, chain.objs.size()=1 2024-06-28T16:14:34.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7q_AGRyCpuXBfe7aa4LZG16bSvgR4Z3_0 2024-06-28T16:14:34.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4781487695983818439', time=2024-06-28T17:47:39.189445+0000, chain.objs.size()=1 2024-06-28T16:14:34.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.WMZm-qu9XT_yFNP6G0XZ2uKauJH8Bmr_0 2024-06-28T16:14:34.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.527938494159735085', time=2024-06-28T17:47:39.461466+0000, chain.objs.size()=1 2024-06-28T16:14:34.414 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.403+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.1lrvPaUCI9Z7OrLhQ2XEMn5p-tuTaZz_0 2024-06-28T16:14:34.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12258300990177553814', time=2024-06-28T17:47:44.095833+0000, chain.objs.size()=1 2024-06-28T16:14:34.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.255__shadow_.vv9dz4dFL1G4oLO8HDowD8wY2VVRdk1_0 2024-06-28T16:14:34.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2055029029932185615', time=2024-06-28T17:47:52.878259+0000, chain.objs.size()=1 2024-06-28T16:14:34.416 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.261__shadow_.ae1row0tOzjtZ8lYE-6rwe5SbuDMIsb_0 2024-06-28T16:14:34.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6029583320219694884', time=2024-06-28T17:48:02.082297+0000, chain.objs.size()=1 2024-06-28T16:14:34.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.267__shadow_.-Ua-nrlBc9sMinosInu9CjHz2V0rY1L_0 2024-06-28T16:14:34.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12049554967433770455', time=2024-06-28T17:48:05.327717+0000, chain.objs.size()=1 2024-06-28T16:14:34.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.269__shadow_.lBEF8MlT1CyPFBFByK12qWPETUU8tU__0 2024-06-28T16:14:34.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12838707354995707484', time=2024-06-28T17:48:11.667721+0000, chain.objs.size()=1 2024-06-28T16:14:34.417 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.273__shadow_.iTKTO_Y3I941VQO9c-PQunHI-q34wjV_0 2024-06-28T16:14:34.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2438082482538057695', time=2024-06-28T17:48:34.650901+0000, chain.objs.size()=1 2024-06-28T16:14:34.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.Wwlw3TTbJOfOnV_epU4aFX1zodlQ4Ar_0 2024-06-28T16:14:34.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12997036683548854232', time=2024-06-28T17:48:54.331757+0000, chain.objs.size()=1 2024-06-28T16:14:34.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.295__shadow_.BUU5UaurGDZJ5UcCwe03mfBO4O9H9Sm_0 2024-06-28T16:14:34.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18347424713950521732', time=2024-06-28T17:48:58.948264+0000, chain.objs.size()=1 2024-06-28T16:14:34.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.297__shadow:7930J6TpDD37PYUV0z7kFUmeajghm9k_.hxmsTyBu5eMl3XwFhEytelj7cvmwzcd_0 2024-06-28T16:14:34.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10745739373015766408', time=2024-06-28T17:51:07.158096+0000, chain.objs.size()=1 2024-06-28T16:14:34.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.G9fBF2uqH-C0V-lNgrgiDgSgXIsl6LK_0 2024-06-28T16:14:34.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3434221505852530225', time=2024-06-28T17:51:07.219340+0000, chain.objs.size()=1 2024-06-28T16:14:34.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.F9ZG7FoO2XJjXC6SzYviPYDNoX-yGWS_0 2024-06-28T16:14:34.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3001131951971338524', time=2024-06-28T17:51:08.156028+0000, chain.objs.size()=1 2024-06-28T16:14:34.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.407+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.BMhc0yQ2EhGpEnHb7qMS5bZFBoC7g6h_0 2024-06-28T16:14:34.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7780134397220759681', time=2024-06-28T17:51:08.204666+0000, chain.objs.size()=1 2024-06-28T16:14:34.420 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Cul7JlDfAsDHnnOoZTsT96EXTfmpKEl_0 2024-06-28T16:14:34.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.664941584127952087', time=2024-06-28T17:51:09.057163+0000, chain.objs.size()=1 2024-06-28T16:14:34.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_._qO39FVQledLwI1a4FDAjHx9w3kP23v_0 2024-06-28T16:14:34.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15242240732131275515', time=2024-06-28T17:51:09.662240+0000, chain.objs.size()=1 2024-06-28T16:14:34.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.HdAsCegWTasnGJBuABbU6w8bWE1HAYZ_0 2024-06-28T16:14:34.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11192537717232070568', time=2024-06-28T17:51:12.448379+0000, chain.objs.size()=1 2024-06-28T16:14:34.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.WLJ94kLnB8XPeP8ygABMYrR9IkW8b8k_0 2024-06-28T16:14:34.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2508762411179323640', time=2024-06-28T17:51:12.882458+0000, chain.objs.size()=1 2024-06-28T16:14:34.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ckaR0eyQrZHYoc5Cfg52dckWf4bWiF__0 2024-06-28T16:14:34.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6280793795731141708', time=2024-06-28T17:51:14.306666+0000, chain.objs.size()=1 2024-06-28T16:14:34.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.EIUHc97XpFUdV8vETQlzMb4J1FUIpZa_0 2024-06-28T16:14:34.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17179574741294304092', time=2024-06-28T17:51:18.252041+0000, chain.objs.size()=1 2024-06-28T16:14:34.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.BNHgbidYtOo6PMxCUpl-3s7IDB8suoW_0 2024-06-28T16:14:34.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9090466055640789', time=2024-06-28T17:51:19.337172+0000, chain.objs.size()=1 2024-06-28T16:14:34.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.LoaeviqizrkaANUUY-TmE0yEJ4olg6c_0 2024-06-28T16:14:34.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16895626993627661821', time=2024-06-28T17:51:19.734190+0000, chain.objs.size()=1 2024-06-28T16:14:34.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.z8CgYKRQQZsMmojDVktlnG6BdD3TFhs_0 2024-06-28T16:14:34.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16277753306516239644', time=2024-06-28T17:51:20.086269+0000, chain.objs.size()=1 2024-06-28T16:14:34.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.kEgmyiQP0qBVbghJ9PnDT8Ync6CTOcK_0 2024-06-28T16:14:34.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13639064130749319836', time=2024-06-28T17:51:21.088774+0000, chain.objs.size()=1 2024-06-28T16:14:34.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.411+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nvXMtc665ETybgMfOI6Qd2_mdoFoht6_0 2024-06-28T16:14:34.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.419+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16778514336326400647', time=2024-06-28T17:51:21.736949+0000, chain.objs.size()=1 2024-06-28T16:14:34.428 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.awq2y0s0H-zAwIKKWuhFD1iyfq4x5ij_0 2024-06-28T16:14:34.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.419+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3874194133696094550', time=2024-06-28T17:51:23.646631+0000, chain.objs.size()=1 2024-06-28T16:14:34.429 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.419+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.q9D_liW9-ALGt_NW-XFD164dhgHOP_s_0 2024-06-28T16:14:34.431 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11339767305290633489', time=2024-06-28T17:51:24.120810+0000, chain.objs.size()=1 2024-06-28T16:14:34.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6SsTsNm_BR6aOU6U2FX7KUX8AKawfxl_0 2024-06-28T16:14:34.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4374557734187387043', time=2024-06-28T17:51:24.896122+0000, chain.objs.size()=1 2024-06-28T16:14:34.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Yai_x9rEBhZvtZ0HQIIJcc35dJjPdPF_0 2024-06-28T16:14:34.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4486571070057679995', time=2024-06-28T17:51:26.321486+0000, chain.objs.size()=1 2024-06-28T16:14:34.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Gr71qiwKoGZK5RF_xl6SRUGs9HtPm5d_0 2024-06-28T16:14:34.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5269073922184041310', time=2024-06-28T17:51:26.642192+0000, chain.objs.size()=1 2024-06-28T16:14:34.432 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Ska0YQXBRr8ro92uJ8bztg0UW3SIxZT_0 2024-06-28T16:14:34.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16047504969477884616', time=2024-06-28T17:51:28.611658+0000, chain.objs.size()=1 2024-06-28T16:14:34.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.gWzUnR0O4hvfQbiU8a7OGUGws1rF5CR_0 2024-06-28T16:14:34.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9674260334339229966', time=2024-06-28T17:51:29.109631+0000, chain.objs.size()=1 2024-06-28T16:14:34.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.gfGOwmGF2SlhiIz8KsEQe2-ox374Vim_0 2024-06-28T16:14:34.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8370448809434550741', time=2024-06-28T17:51:29.824607+0000, chain.objs.size()=1 2024-06-28T16:14:34.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.cLu-aY0xlmJjmniHdXsrVu-umxrvoA1_0 2024-06-28T16:14:34.433 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16331695541463265762', time=2024-06-28T17:51:30.603433+0000, chain.objs.size()=1 2024-06-28T16:14:34.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.qpo-ivNuEoOm_eWGqrauG3hqWje-wV6_0 2024-06-28T16:14:34.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2104477448381004097', time=2024-06-28T17:51:30.890954+0000, chain.objs.size()=1 2024-06-28T16:14:34.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ytocuwkZdvSc_JV0yOxss63dFCcbXwN_0 2024-06-28T16:14:34.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10119466115841993473', time=2024-06-28T17:51:32.245783+0000, chain.objs.size()=1 2024-06-28T16:14:34.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.EkcIVjyUE_kqV0oB9RtHQvqdvM3CdYy_0 2024-06-28T16:14:34.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5543716848889842950', time=2024-06-28T17:51:33.128565+0000, chain.objs.size()=1 2024-06-28T16:14:34.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.423+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_._9WofYfUt6JNXMR44P0NJub6M2aQsMh_0 2024-06-28T16:14:34.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2201245210379410159', time=2024-06-28T17:51:34.824925+0000, chain.objs.size()=1 2024-06-28T16:14:34.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.asW5nu1iOJdgcywaMrD6XcW8HL1fGeL_0 2024-06-28T16:14:34.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11378313686431572230', time=2024-06-28T17:51:35.852769+0000, chain.objs.size()=1 2024-06-28T16:14:34.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.pfWjAAuwm9iiAK2cghsJxrg1IjzBvog_0 2024-06-28T16:14:34.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7443329862268592395', time=2024-06-28T17:51:35.953291+0000, chain.objs.size()=1 2024-06-28T16:14:34.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.vqzEJU2Qmvdd24lSAMev4oWsO7Y6qdP_0 2024-06-28T16:14:34.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.300256838925454646', time=2024-06-28T17:51:35.955194+0000, chain.objs.size()=1 2024-06-28T16:14:34.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fJa17HquOi0FQ5prsBbNidPdcr_wpM2_0 2024-06-28T16:14:34.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8567807107577968455', time=2024-06-28T17:51:36.671613+0000, chain.objs.size()=1 2024-06-28T16:14:34.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.JBGVtqwatZ4VCFXPQAV_PCq6XgucvYP_0 2024-06-28T16:14:34.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12485559407432521589', time=2024-06-28T17:51:38.135090+0000, chain.objs.size()=1 2024-06-28T16:14:34.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ARHXxJDgo20ShOaJ7xCE_vzQBcpeoqf_0 2024-06-28T16:14:34.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4162696076983697811', time=2024-06-28T17:51:38.405655+0000, chain.objs.size()=1 2024-06-28T16:14:34.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.lOvCyWdZL35-vTM_k0_PD_FKNlNhG-C_0 2024-06-28T16:14:34.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16159812946705330074', time=2024-06-28T17:53:20.910698+0000, chain.objs.size()=1 2024-06-28T16:14:34.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.tQD37guaGv7qujwmUhsaJpM-AHa-lUT_0 2024-06-28T16:14:34.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16841918140439865931', time=2024-06-28T17:53:25.430661+0000, chain.objs.size()=1 2024-06-28T16:14:34.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.XeDZw97IFMQwp2gqQpFN_aYeQEpDqcZ_0 2024-06-28T16:14:34.439 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7309982127311665103', time=2024-06-28T17:53:25.805131+0000, chain.objs.size()=1 2024-06-28T16:14:34.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.427+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GrTRSwqLYZvVyH0xyS_BCCYFkKgUFI__0 2024-06-28T16:14:34.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16361628830866812861', time=2024-06-28T17:53:29.696021+0000, chain.objs.size()=1 2024-06-28T16:14:34.440 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.a_OmsSvo-oAvkf2BttOZVTL8n__VhGj_0 2024-06-28T16:14:34.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1447144344540994375', time=2024-06-28T17:53:29.907665+0000, chain.objs.size()=1 2024-06-28T16:14:34.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.X5RlHu_t0dvCaBBDZ8Zex9DDAkrTdPY_0 2024-06-28T16:14:34.441 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11493545680948503277', time=2024-06-28T17:53:30.084716+0000, chain.objs.size()=1 2024-06-28T16:14:34.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qnK7DKYbCbg2AjHCl9Ldr8dz-gG8OYQ_0 2024-06-28T16:14:34.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8009058042162096270', time=2024-06-28T17:53:34.344820+0000, chain.objs.size()=1 2024-06-28T16:14:34.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_._eScULf2Lc9GsMmJGGGBNkkqgihm9Kc_0 2024-06-28T16:14:34.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3273931026198074910', time=2024-06-28T17:53:35.490747+0000, chain.objs.size()=1 2024-06-28T16:14:34.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.aonEOaB1s_hhCZVdtDbJG5G_x5L2M0H_0 2024-06-28T16:14:34.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7263783757523623023', time=2024-06-28T17:53:35.492661+0000, chain.objs.size()=1 2024-06-28T16:14:34.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Lnkh8f05NKMY2kuukGRvksp_tOAxeYT_0 2024-06-28T16:14:34.443 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6247947321030410758', time=2024-06-28T17:53:35.595804+0000, chain.objs.size()=1 2024-06-28T16:14:34.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1sWAIad9KNsy9yfJCdZmO0xYnu7hoau_0 2024-06-28T16:14:34.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2910990122231029254', time=2024-06-28T17:53:35.979303+0000, chain.objs.size()=1 2024-06-28T16:14:34.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.kEn0JmFWAHqVuNHITfEAFK1bWSfDNYl_0 2024-06-28T16:14:34.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18003973862665957778', time=2024-06-28T17:53:36.210601+0000, chain.objs.size()=1 2024-06-28T16:14:34.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.i-ILz9_YEAhkd1Y1f1Cyr3a6tPGJ2jF_0 2024-06-28T16:14:34.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8264077936514848283', time=2024-06-28T17:53:37.809096+0000, chain.objs.size()=1 2024-06-28T16:14:34.444 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.3IBr5XtaZ4vDRiJDnK8ysVh0Dw6HGq8_0 2024-06-28T16:14:34.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17540139146097392839', time=2024-06-28T17:53:37.810964+0000, chain.objs.size()=1 2024-06-28T16:14:34.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.431+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.psikqMxeIaN8aS7TIvdRrRFRBm-eVZj_0 2024-06-28T16:14:34.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.435+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13826308032473572296', time=2024-06-28T17:53:39.645952+0000, chain.objs.size()=1 2024-06-28T16:14:34.445 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.435+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.jbdYGHxqn7giFfheqPfyLvCp5C_e5UO_0 2024-06-28T16:14:34.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11092808146231223829', time=2024-06-28T17:53:39.815197+0000, chain.objs.size()=1 2024-06-28T16:14:34.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.zlCkIFzmbxfB4safwxVluvH6iwAzeIl_0 2024-06-28T16:14:34.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9643234875597730262', time=2024-06-28T17:53:39.817586+0000, chain.objs.size()=1 2024-06-28T16:14:34.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.p31Mpdm9AzZ9gRfqgDD4xJkcZZDxCvU_0 2024-06-28T16:14:34.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13798881189633893589', time=2024-06-28T17:53:40.250915+0000, chain.objs.size()=1 2024-06-28T16:14:34.447 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.FOiTCc9V8od2hOyV1VN6Fbu0wnNjvdz_0 2024-06-28T16:14:34.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12870319815806081532', time=2024-06-28T17:53:40.368340+0000, chain.objs.size()=1 2024-06-28T16:14:34.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.a2st4n0yCrJgXZJ5J9lwoSS_ZEU02RG_0 2024-06-28T16:14:34.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12951179201540236819', time=2024-06-28T17:53:44.474507+0000, chain.objs.size()=1 2024-06-28T16:14:34.448 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.XSr1YfD7Sjma3CV9jZeYd2jpRs6HjKn_0 2024-06-28T16:14:34.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13413755888720845345', time=2024-06-28T17:53:44.587273+0000, chain.objs.size()=1 2024-06-28T16:14:34.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_._H6evN_ibgMQE0aa8nheRfAuLVeFG1b_0 2024-06-28T16:14:34.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5080336957884753852', time=2024-06-28T17:53:45.114347+0000, chain.objs.size()=1 2024-06-28T16:14:34.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.GiVW0jYbYpugH4tv09N8dKMIerRaG7g_0 2024-06-28T16:14:34.449 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9845318980418309102', time=2024-06-28T17:53:46.250523+0000, chain.objs.size()=1 2024-06-28T16:14:34.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.SgZYeuoaZ7KusIWy6WvbZVSM5eLQpXK_0 2024-06-28T16:14:34.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6220453360211573989', time=2024-06-28T17:53:47.302248+0000, chain.objs.size()=1 2024-06-28T16:14:34.450 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.439+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.r4T2Daus3ciRsYD2iorS3f-DKaFg2uE_0 2024-06-28T16:14:34.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.443+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/45320 2024-06-28T16:14:34.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=19, truncated=0, next_marker='' 2024-06-28T16:14:34.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7116495379492658774', time=2024-06-28T17:53:50.910548+0000, chain.objs.size()=1 2024-06-28T16:14:34.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.s8NPlCoMqXk04KniXB_VsGakvGPeL24_0 2024-06-28T16:14:34.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5037678950901244691', time=2024-06-28T17:53:51.304186+0000, chain.objs.size()=1 2024-06-28T16:14:34.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.N2P_Srz00QeDJrYlbWwvWNq0yPbWgoE_0 2024-06-28T16:14:34.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6015593014050181849', time=2024-06-28T17:53:52.120831+0000, chain.objs.size()=1 2024-06-28T16:14:34.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.28iI4niAKMFpzB2wT9bifuL2LDfQN-3_0 2024-06-28T16:14:34.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13916214803256780740', time=2024-06-28T17:53:58.865546+0000, chain.objs.size()=1 2024-06-28T16:14:34.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.321__shadow_.I2XvGvQ5rt8HVr3sfeM6D2xZAXLWjVc_0 2024-06-28T16:14:34.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14264693520042239794', time=2024-06-28T17:54:52.088937+0000, chain.objs.size()=1 2024-06-28T16:14:34.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.380__shadow_.82II3Ss6HCfxORU129SHWSZ1qXUCG79_0 2024-06-28T16:14:34.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.470155798252338857', time=2024-06-28T17:55:26.395059+0000, chain.objs.size()=1 2024-06-28T16:14:34.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.419__shadow_.T86D_poja5vo8ZycLA5B7-Zu67a2cAU_0 2024-06-28T16:14:34.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9511924785708247397', time=2024-06-28T17:55:39.133001+0000, chain.objs.size()=1 2024-06-28T16:14:34.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.431__shadow_.y3zZR_LR9FNjYHU-GO1eVQ61-4jIrzL_0 2024-06-28T16:14:34.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6227838289997309542', time=2024-06-28T17:56:01.591230+0000, chain.objs.size()=1 2024-06-28T16:14:34.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.449__shadow_.r_0uG2LLmw9hu0c5k1Hso12sa1E8MR5_0 2024-06-28T16:14:34.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10779923753383803106', time=2024-06-28T17:56:13.803663+0000, chain.objs.size()=1 2024-06-28T16:14:34.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.456__shadow_.-YfAFVZQZ3X7wNwgvtxPB6d5icKnzUT_0 2024-06-28T16:14:34.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16224875976241398037', time=2024-06-28T17:56:28.108798+0000, chain.objs.size()=1 2024-06-28T16:14:34.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.473__shadow_.dfPR-RJF5a-xthuM7J-ywQz9Zxyz1uf_0 2024-06-28T16:14:34.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7156434283679941218', time=2024-06-28T17:56:54.951517+0000, chain.objs.size()=1 2024-06-28T16:14:34.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.493__shadow:rinwpm3WJlZ1AcGvTYYbpbEzG8SjNkf_.0OEFPT4L0L8Hzv3rUAZWjoQj5ExOwB1_0 2024-06-28T16:14:34.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6435247639739047175', time=2024-06-28T17:59:06.033960+0000, chain.objs.size()=6 2024-06-28T16:14:34.460 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.447+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~QNBtUMmzeRqaHJCWiGvRtIJx3LM7Ap-.1 2024-06-28T16:14:34.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~QNBtUMmzeRqaHJCWiGvRtIJx3LM7Ap-.2 2024-06-28T16:14:34.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.-2mB5Rjc8nkIKNKFG51GvoIb7jpvIDeS.3 2024-06-28T16:14:34.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~QNBtUMmzeRqaHJCWiGvRtIJx3LM7Ap-.4 2024-06-28T16:14:34.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~QNBtUMmzeRqaHJCWiGvRtIJx3LM7Ap-.5 2024-06-28T16:14:34.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~QNBtUMmzeRqaHJCWiGvRtIJx3LM7Ap-.6 2024-06-28T16:14:34.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11870662991326818495', time=2024-06-28T18:00:02.232810+0000, chain.objs.size()=2 2024-06-28T16:14:34.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.515__multipart_mymultipart.2~C6P7Z-JXsn3Q9ZEQxUZyzjxLq0P3KNK.1 2024-06-28T16:14:34.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.515__multipart_mymultipart.2~C6P7Z-JXsn3Q9ZEQxUZyzjxLq0P3KNK.2 2024-06-28T16:14:34.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12475438574954481888', time=2024-06-28T18:02:36.714225+0000, chain.objs.size()=1 2024-06-28T16:14:34.462 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.563__shadow:UJ8vKHh69ih4pqKkg6mMjswxBV8qy9F_.ldQVMHbKHdtaYtb4r1rjG41AIjO-uVH_0 2024-06-28T16:14:34.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5320603290046955770', time=2024-06-28T18:02:49.530009+0000, chain.objs.size()=1 2024-06-28T16:14:34.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.565__shadow:LWAJGtvNKNbEavGsKcOFG1ouQr.UAui_.9NF6Iu3RRH3r3z5NMbkzh4reWD6rgQY_0 2024-06-28T16:14:34.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.202393428638362119', time=2024-06-28T18:03:12.117245+0000, chain.objs.size()=1 2024-06-28T16:14:34.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:DVdehbGpERqMGAF5pd4pAbI9zC0EzAS_.Qaq7L_7sZb0OIyAVR95-EXCD8wcz9Dt_0 2024-06-28T16:14:34.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12912086760426576985', time=2024-06-28T18:03:56.568536+0000, chain.objs.size()=1 2024-06-28T16:14:34.463 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.451+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.568__shadow:V-apET8J.lgnDWW8q9W5Mj2f1b9HyT9_.7oaj1hqURn9eTtTWaKx9A3pYXk-au_h_0 2024-06-28T16:14:34.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.455+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8880037268707333216', time=2024-06-28T18:05:50.747423+0000, chain.objs.size()=6 2024-06-28T16:14:34.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.603__multipart_multipart_enc.2~oZgCaTNUl6m1hwaLVMa2n-qg1Fn9-FH.1 2024-06-28T16:14:34.464 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.603__multipart_multipart_enc.2~oZgCaTNUl6m1hwaLVMa2n-qg1Fn9-FH.2 2024-06-28T16:14:34.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.603__multipart_multipart_enc.2~oZgCaTNUl6m1hwaLVMa2n-qg1Fn9-FH.3 2024-06-28T16:14:34.465 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.603__multipart_multipart_enc.2~oZgCaTNUl6m1hwaLVMa2n-qg1Fn9-FH.4 2024-06-28T16:14:34.466 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.603__multipart_multipart_enc.2~oZgCaTNUl6m1hwaLVMa2n-qg1Fn9-FH.5 2024-06-28T16:14:34.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.603__multipart_multipart_enc.2~oZgCaTNUl6m1hwaLVMa2n-qg1Fn9-FH.6 2024-06-28T16:14:34.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.455+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5634170323895908386', time=2024-06-28T18:06:01.210318+0000, chain.objs.size()=1 2024-06-28T16:14:34.467 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.455+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.605__shadow_.HvSqoMFmbUu0xEsduxK-FJLwDp_KFX__0 2024-06-28T16:14:34.472 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.463+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:34.475 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-06-28T16:14:34.476 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:34.477 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:34.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/40223' 2024-06-28T16:14:34.478 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4911767290600858406', time=2024-06-28T17:40:06.371393+0000, chain.objs.size()=2 2024-06-28T16:14:34.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.148__shadow_.obxhpSRm8rBTObMHlTBOyBwym9l7TNb_0 2024-06-28T16:14:34.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:843e24ce-085a-4043-8f55-38e05370e790.4686.148__shadow_.obxhpSRm8rBTObMHlTBOyBwym9l7TNb_1 2024-06-28T16:14:34.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.770975069693951462', time=2024-06-28T17:45:08.255264+0000, chain.objs.size()=1 2024-06-28T16:14:34.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.240__shadow_.OHVgSAaRUFULcM0rQsF5PYdAUb1CrzG_0 2024-06-28T16:14:34.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7601254775683913830', time=2024-06-28T17:45:19.329944+0000, chain.objs.size()=1 2024-06-28T16:14:34.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.247__shadow_.g89ZEXyek0iTamNrgMP9BVmPClLWWXH_0 2024-06-28T16:14:34.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.282419736669251957', time=2024-06-28T17:45:19.341495+0000, chain.objs.size()=1 2024-06-28T16:14:34.479 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.247__shadow_.tRMxkDvvWsevVufP5pTUWtCULS9qENI_0 2024-06-28T16:14:34.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16210638119104649074', time=2024-06-28T17:47:09.156913+0000, chain.objs.size()=1 2024-06-28T16:14:34.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.QNZvLypIro6kKJYorqcU_m2m74hLet3_0 2024-06-28T16:14:34.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4523742745963550380', time=2024-06-28T17:47:10.054380+0000, chain.objs.size()=1 2024-06-28T16:14:34.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.rl4uBMLtSup_eTjjOU3vNviwjSG27Ek_0 2024-06-28T16:14:34.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6038760693471151324', time=2024-06-28T17:47:10.187018+0000, chain.objs.size()=1 2024-06-28T16:14:34.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.J512qM5xWE2XtuhJp3Tozc8JaqgvznX_0 2024-06-28T16:14:34.480 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14414954326465401611', time=2024-06-28T17:47:10.487625+0000, chain.objs.size()=1 2024-06-28T16:14:34.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.76EKMh1v92zPQf3oUvoBE6Uz0QFHZ1X_0 2024-06-28T16:14:34.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2509117468166737201', time=2024-06-28T17:47:10.492728+0000, chain.objs.size()=1 2024-06-28T16:14:34.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.1LhfcUb2x292H6sHW3aJvY02MCVc89f_0 2024-06-28T16:14:34.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13869811871165570886', time=2024-06-28T17:47:10.724083+0000, chain.objs.size()=1 2024-06-28T16:14:34.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.vAdjlGWbSho3UL6dwvVsFjoc2tcnVGR_0 2024-06-28T16:14:34.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11808498825407430132', time=2024-06-28T17:47:10.905989+0000, chain.objs.size()=1 2024-06-28T16:14:34.481 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.467+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.PovqYtKKt-R8eka3Rh0AZiH_JwZTzuR_0 2024-06-28T16:14:34.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.471+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8025481618817541722', time=2024-06-28T17:47:13.308204+0000, chain.objs.size()=1 2024-06-28T16:14:34.482 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.471+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.rUqEQIv3EsXLg-EVV7gPBxi6ola1gVG_0 2024-06-28T16:14:34.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17247555356356000909', time=2024-06-28T17:47:13.769533+0000, chain.objs.size()=1 2024-06-28T16:14:34.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.qnx9apWiM5inSPQ3AITBlS7wqxto9uZ_0 2024-06-28T16:14:34.483 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.669164918649086661', time=2024-06-28T17:47:15.652854+0000, chain.objs.size()=1 2024-06-28T16:14:34.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9pAvWkHl8g03g7v8ioiWFDQzlx_G7hk_0 2024-06-28T16:14:34.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17630307089443210589', time=2024-06-28T17:47:16.637328+0000, chain.objs.size()=1 2024-06-28T16:14:34.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Tgvap3GQ1VztACKRPLNNV4nxcxdFVYh_0 2024-06-28T16:14:34.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1186313674742130734', time=2024-06-28T17:47:17.190606+0000, chain.objs.size()=1 2024-06-28T16:14:34.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Q8ai2NjH-lztTteHT4cTIcIrRxM67s5_0 2024-06-28T16:14:34.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15812146957908860647', time=2024-06-28T17:47:19.318483+0000, chain.objs.size()=1 2024-06-28T16:14:34.484 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.4Vz2CNBtMka_ZQ6PWKLUan1J_Wfj397_0 2024-06-28T16:14:34.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16354294599271771732', time=2024-06-28T17:47:20.899543+0000, chain.objs.size()=1 2024-06-28T16:14:34.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.asSZjMHNO-ylRdgi8poHmtict2RcC3X_0 2024-06-28T16:14:34.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10267119779405028922', time=2024-06-28T17:47:21.091526+0000, chain.objs.size()=1 2024-06-28T16:14:34.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.qLUDCW-i183LaVnRJ-V5suVVzfd2efC_0 2024-06-28T16:14:34.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6368042476353476811', time=2024-06-28T17:47:21.744343+0000, chain.objs.size()=1 2024-06-28T16:14:34.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.B2yWub9Es0n0xFqQlfvGD_zxOQzuk8F_0 2024-06-28T16:14:34.485 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8016518222511222202', time=2024-06-28T17:47:22.097176+0000, chain.objs.size()=1 2024-06-28T16:14:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.JrDishnWX9626qmUQzht2qyr-S3uxyx_0 2024-06-28T16:14:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5909311305349304757', time=2024-06-28T17:47:23.260345+0000, chain.objs.size()=1 2024-06-28T16:14:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.CrxUJO788QmK2poFpurdVhIlIe8Q1vC_0 2024-06-28T16:14:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1840560865528504103', time=2024-06-28T17:47:28.925962+0000, chain.objs.size()=1 2024-06-28T16:14:34.486 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.475+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.NntLiu88pDQzuO3RYa83bf3UdFSCLSI_0 2024-06-28T16:14:34.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.479+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16627196324020161518', time=2024-06-28T17:47:29.846665+0000, chain.objs.size()=1 2024-06-28T16:14:34.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.479+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.IPh7RR0O0iMsm7YpAggFpjjORZKcO1a_0 2024-06-28T16:14:34.487 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.479+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10669929529652061997', time=2024-06-28T17:47:29.857457+0000, chain.objs.size()=1 2024-06-28T16:14:34.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.479+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.RqcrYu_Y-00S4RJRLTgrBc-J2na9gHS_0 2024-06-28T16:14:34.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.479+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.112908130589093974', time=2024-06-28T17:47:30.204126+0000, chain.objs.size()=1 2024-06-28T16:14:34.488 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.479+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.TXGa6esWU-1S9zH1QzmgGlDTW5iE0Mu_0 2024-06-28T16:14:34.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.479+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10067942327185255907', time=2024-06-28T17:47:31.044353+0000, chain.objs.size()=1 2024-06-28T16:14:34.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.479+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.EJWU5mFTdYfP3cp-hFqhexDRqH1m1aP_0 2024-06-28T16:14:34.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.479+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.812040722271480717', time=2024-06-28T17:47:31.512857+0000, chain.objs.size()=1 2024-06-28T16:14:34.489 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.479+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.59iq9TlFJ1-r9ipbYl0sQtsrsSNEJLC_0 2024-06-28T16:14:34.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.479+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11062240087906226307', time=2024-06-28T17:47:31.937263+0000, chain.objs.size()=1 2024-06-28T16:14:34.490 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.479+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_._BjWi873ssW_cHLS0KPYQXNNyOU51Nc_0 2024-06-28T16:14:34.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16634096835686195696', time=2024-06-28T17:47:33.685006+0000, chain.objs.size()=1 2024-06-28T16:14:34.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.QUVsjDiDW55bEShyJf6U4MS2S7h2r1J_0 2024-06-28T16:14:34.491 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7662562408900072272', time=2024-06-28T17:47:34.146298+0000, chain.objs.size()=1 2024-06-28T16:14:34.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.j_pInakuMkmmuCPO9vf_hESpg92e4lt_0 2024-06-28T16:14:34.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7172619567242603625', time=2024-06-28T17:47:34.263841+0000, chain.objs.size()=1 2024-06-28T16:14:34.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.saoxZHdIjb8m35qvPQ6A1vh6cL2NGYZ_0 2024-06-28T16:14:34.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16719485802391136493', time=2024-06-28T17:47:38.104852+0000, chain.objs.size()=1 2024-06-28T16:14:34.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oDIxHEdM8qfllgRM3qbpy2Z0NU8f3kr_0 2024-06-28T16:14:34.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2464741566048232595', time=2024-06-28T17:47:38.729532+0000, chain.objs.size()=1 2024-06-28T16:14:34.492 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.hiqW6oaA6Dh2s1PlF9qRhRFBInUCBuB_0 2024-06-28T16:14:34.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1033099705153352018', time=2024-06-28T17:47:39.268538+0000, chain.objs.size()=1 2024-06-28T16:14:34.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9ZEp9Nvs-rFwtZofGV-yESsvP7Zmfv9_0 2024-06-28T16:14:34.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4218198067279844101', time=2024-06-28T17:47:49.962902+0000, chain.objs.size()=1 2024-06-28T16:14:34.493 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.259__shadow_.HYNTxxunQJ8ZFoADxIku6pVgmmA97Mc_0 2024-06-28T16:14:34.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8865727674317951788', time=2024-06-28T17:47:58.925670+0000, chain.objs.size()=1 2024-06-28T16:14:34.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.265__shadow_.OrxAqaYAxbeKnvO-O1yPrtMiqu3otRH_0 2024-06-28T16:14:34.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7115665295405008001', time=2024-06-28T17:48:29.562982+0000, chain.objs.size()=1 2024-06-28T16:14:34.494 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.483+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.gbr3vG9_o4vyK0bMpQySLnUPN2eP1iw_0 2024-06-28T16:14:34.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12075398627841904019', time=2024-06-28T17:48:34.570980+0000, chain.objs.size()=1 2024-06-28T16:14:34.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.XjnTb4r5AQDPyrHL9XTyCtv3q_5Ar5g_0 2024-06-28T16:14:34.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12207137255741800746', time=2024-06-28T17:48:34.572419+0000, chain.objs.size()=1 2024-06-28T16:14:34.496 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.Dt8SnmFbCwOAdfOZ6Jgh8ryJkvceznl_0 2024-06-28T16:14:34.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1110427592964844370', time=2024-06-28T17:48:34.887198+0000, chain.objs.size()=1 2024-06-28T16:14:34.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.283__shadow_.Rk_7xhlrWpIlnvAuF7umgCu6bqEfqRV_0 2024-06-28T16:14:34.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8381206171784395519', time=2024-06-28T17:49:19.397290+0000, chain.objs.size()=1 2024-06-28T16:14:34.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.311__shadow_.AYj-EoUyhP8F-P49m9uACSKFNbTyXXs_0 2024-06-28T16:14:34.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5690438153656153501', time=2024-06-28T17:51:08.393959+0000, chain.objs.size()=1 2024-06-28T16:14:34.497 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Dkjd-PNNVxNM0PXBoUlfUx6eGwVrfFT_0 2024-06-28T16:14:34.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14134775467863998254', time=2024-06-28T17:51:09.018720+0000, chain.objs.size()=1 2024-06-28T16:14:34.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.EhxK9xYGzzMDLxhGXlNhdBkTQBMVPlX_0 2024-06-28T16:14:34.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9577342378306641537', time=2024-06-28T17:51:09.837877+0000, chain.objs.size()=1 2024-06-28T16:14:34.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.V__kVY2Erbnq_N-l_mi5Lcab1E10l2h_0 2024-06-28T16:14:34.498 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3195893035149240895', time=2024-06-28T17:51:10.012345+0000, chain.objs.size()=1 2024-06-28T16:14:34.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.p-3zRYpfQ22KSe5DUzoE6Lqur3Vladh_0 2024-06-28T16:14:34.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10424348958508539717', time=2024-06-28T17:51:11.581446+0000, chain.objs.size()=1 2024-06-28T16:14:34.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.o006tq1zKqdsucMcDU0x9448FJfVglK_0 2024-06-28T16:14:34.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17229083755763421681', time=2024-06-28T17:51:11.706645+0000, chain.objs.size()=1 2024-06-28T16:14:34.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.487+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_._YiT3IAcF5lJSK4tYtkd6fpQ9E8IjtZ_0 2024-06-28T16:14:34.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7515980068888092157', time=2024-06-28T17:51:14.062289+0000, chain.objs.size()=1 2024-06-28T16:14:34.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.vN_RJUXGKZasHSzxSjZHVb6YY81NWto_0 2024-06-28T16:14:34.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10984528078836172686', time=2024-06-28T17:51:15.340106+0000, chain.objs.size()=1 2024-06-28T16:14:34.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fMCOpDz_DnCaijjG3l-g_RWwlZab2Bd_0 2024-06-28T16:14:34.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8019512263347978762', time=2024-06-28T17:51:15.446906+0000, chain.objs.size()=1 2024-06-28T16:14:34.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.vt7VPtfYmoPKyLeGHFrYooTgAJh6o-i_0 2024-06-28T16:14:34.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.491+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.819994544254770465', time=2024-06-28T17:51:15.456664+0000, chain.objs.size()=1 2024-06-28T16:14:34.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.491+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ADXLn-5ESEf5GfMXDZZFUfE-ZhO4Wbi_0 2024-06-28T16:14:34.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5456777176402707729', time=2024-06-28T17:51:17.223748+0000, chain.objs.size()=1 2024-06-28T16:14:34.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.vJ_H_Dx3SaPb7N2n0_4OZ9OD4wi_Zix_0 2024-06-28T16:14:34.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2021152605424110023', time=2024-06-28T17:51:18.404349+0000, chain.objs.size()=1 2024-06-28T16:14:34.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.KHwBHoPsmiYBKHsvA4iW9PmJTheWx89_0 2024-06-28T16:14:34.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7004736000583469855', time=2024-06-28T17:51:18.526011+0000, chain.objs.size()=1 2024-06-28T16:14:34.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.bMmjcBkfBDtrPRVd9brgqUFOFRpNNcq_0 2024-06-28T16:14:34.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11842940991729861756', time=2024-06-28T17:51:19.575506+0000, chain.objs.size()=1 2024-06-28T16:14:34.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.kFgPo_e2hNcbbDFuRa2YZ9olLdDDkDM_0 2024-06-28T16:14:34.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8159933411674929427', time=2024-06-28T17:51:20.570861+0000, chain.objs.size()=1 2024-06-28T16:14:34.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.zV74-J7e-lcpSMsKNjKAgAavRIF4b_b_0 2024-06-28T16:14:34.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7131389688862915295', time=2024-06-28T17:51:22.136662+0000, chain.objs.size()=1 2024-06-28T16:14:34.504 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Ihpvw0iqx-uznSHMxAdty8-uPbuWIAE_0 2024-06-28T16:14:34.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16502452384271181637', time=2024-06-28T17:51:23.631858+0000, chain.objs.size()=1 2024-06-28T16:14:34.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ihAC1zL3LFAkBVYu50hBMcETsCxKUPA_0 2024-06-28T16:14:34.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5396714601671752774', time=2024-06-28T17:51:24.061583+0000, chain.objs.size()=1 2024-06-28T16:14:34.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.0zvLMniUSoqRLeMqpWsqzsWIEahrmHh_0 2024-06-28T16:14:34.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15733711816716217012', time=2024-06-28T17:51:24.220576+0000, chain.objs.size()=1 2024-06-28T16:14:34.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.495+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.iejKjY6_cvBOJZw8DUXc92vq8DOCNWw_0 2024-06-28T16:14:34.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8568864992071188915', time=2024-06-28T17:51:24.445929+0000, chain.objs.size()=1 2024-06-28T16:14:34.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yspBU0kw3ddJ9WIR3wwKcWwKrZWW_Fb_0 2024-06-28T16:14:34.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7951107572781809417', time=2024-06-28T17:51:25.413299+0000, chain.objs.size()=1 2024-06-28T16:14:34.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.uVFz-gNQXw6Nm4CrYCMMwSWylqbL2D1_0 2024-06-28T16:14:34.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4172484525784106725', time=2024-06-28T17:51:26.260311+0000, chain.objs.size()=1 2024-06-28T16:14:34.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.W0zXNOPuqnZVwSoPM5mGV30RE5xR9AD_0 2024-06-28T16:14:34.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10200297416565241232', time=2024-06-28T17:51:26.355745+0000, chain.objs.size()=1 2024-06-28T16:14:34.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YAPF-k4Y79_WQRjYdMG2aHWJfVVk0Sr_0 2024-06-28T16:14:34.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16789520147298670707', time=2024-06-28T17:51:28.284401+0000, chain.objs.size()=1 2024-06-28T16:14:34.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.c6ssmYvWBvkkkin0XfoE4wOyYiscXjc_0 2024-06-28T16:14:34.510 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1412768544572545457', time=2024-06-28T17:51:29.189198+0000, chain.objs.size()=1 2024-06-28T16:14:34.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.gO2mwvA-7pnSuXtuRZOlKmsO2c_kf1m_0 2024-06-28T16:14:34.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4158157102845964491', time=2024-06-28T17:51:29.449765+0000, chain.objs.size()=1 2024-06-28T16:14:34.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Tf-ntf1I2vi7EyYZoshEuqt6QlqXKDf_0 2024-06-28T16:14:34.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13224727545117576682', time=2024-06-28T17:51:29.552110+0000, chain.objs.size()=1 2024-06-28T16:14:34.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.kAGZMWF5h7bDOtv3kW_J1qOMaz15_Iz_0 2024-06-28T16:14:34.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5942419019185159057', time=2024-06-28T17:51:30.677724+0000, chain.objs.size()=1 2024-06-28T16:14:34.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.AyYqN_TUHvQqjl85bKDeTM0Mo0DK6-y_0 2024-06-28T16:14:34.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6730871130471844157', time=2024-06-28T17:51:31.873823+0000, chain.objs.size()=1 2024-06-28T16:14:34.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.499+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.0V_LPqyaxjGQtKepRaBqe-dFpD1CUc0_0 2024-06-28T16:14:34.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.503+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1041615276747672477', time=2024-06-28T17:51:34.038199+0000, chain.objs.size()=1 2024-06-28T16:14:34.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.503+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.mtyklfZOWZZ127JH78rsORht7rEEauD_0 2024-06-28T16:14:34.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.503+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11565057214107740719', time=2024-06-28T17:51:36.442857+0000, chain.objs.size()=1 2024-06-28T16:14:34.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.503+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nI6E7-YATFVx0AL8xHxqAJtqW0KGmtV_0 2024-06-28T16:14:34.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.503+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15896723024824758771', time=2024-06-28T17:51:37.071689+0000, chain.objs.size()=1 2024-06-28T16:14:34.516 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.503+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.zdXnUdFx_-_5ifBtJ5QMvoOF0gjsVQi_0 2024-06-28T16:14:34.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.503+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6437011259453687896', time=2024-06-28T17:51:38.253413+0000, chain.objs.size()=1 2024-06-28T16:14:34.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.503+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.T6EuLBnUkq4HCfLUF0J1XYAF8mva8U4_0 2024-06-28T16:14:34.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.503+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.247287108148621808', time=2024-06-28T17:51:38.388028+0000, chain.objs.size()=1 2024-06-28T16:14:34.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.503+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.f6RxOovWBNcfUXGjuIC0SxSjPUQDGB9_0 2024-06-28T16:14:34.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1901592190598877300', time=2024-06-28T17:53:20.943259+0000, chain.objs.size()=1 2024-06-28T16:14:34.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ZkxmiT028BMPkUIzvYExYGLxyS3BKIw_0 2024-06-28T16:14:34.517 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1072165665438925909', time=2024-06-28T17:53:23.702471+0000, chain.objs.size()=1 2024-06-28T16:14:34.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4QslJoWo-dnglXjQ5Ydzjw_T5XJW221_0 2024-06-28T16:14:34.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5251518050144678948', time=2024-06-28T17:53:25.818557+0000, chain.objs.size()=1 2024-06-28T16:14:34.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.lBNuxIfhY0B-0XaGGBZH3O6AmJoUTMY_0 2024-06-28T16:14:34.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13467983511445605604', time=2024-06-28T17:53:26.929938+0000, chain.objs.size()=1 2024-06-28T16:14:34.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qRvw2P3QCAXUzwnAvUCu3kvivk07EIg_0 2024-06-28T16:14:34.518 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14540638796250346861', time=2024-06-28T17:53:27.000925+0000, chain.objs.size()=1 2024-06-28T16:14:34.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CZcGhrDySsyiN3bHPCxVoNSYHbXzhmM_0 2024-06-28T16:14:34.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.507+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7620430601805094860', time=2024-06-28T17:53:27.854035+0000, chain.objs.size()=1 2024-06-28T16:14:34.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.507+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gIHiATMuRZGWR-Z2JLMJTHF2huxbLnL_0 2024-06-28T16:14:34.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12198277968109942213', time=2024-06-28T17:53:28.231760+0000, chain.objs.size()=1 2024-06-28T16:14:34.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4rCHaBikEuB7_U2g6uHJqQk_3757pkG_0 2024-06-28T16:14:34.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1903301347554512682', time=2024-06-28T17:53:29.714678+0000, chain.objs.size()=1 2024-06-28T16:14:34.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.7Dib3Erl2IKF8y3IOUT_l9xXfnYHSfC_0 2024-06-28T16:14:34.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1112231338725449396', time=2024-06-28T17:53:29.833891+0000, chain.objs.size()=1 2024-06-28T16:14:34.520 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.tG0QYG3BnalXW9zzuoirgYQF0xRlD9Z_0 2024-06-28T16:14:34.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18213161078640324528', time=2024-06-28T17:53:29.974119+0000, chain.objs.size()=1 2024-06-28T16:14:34.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.UV__PYoy6307XNQhSSDjbEdOXfqN_na_0 2024-06-28T16:14:34.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3642137968498773343', time=2024-06-28T17:53:30.525560+0000, chain.objs.size()=1 2024-06-28T16:14:34.521 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.oC4Fl1aFBxXSbfluZo2G_2aCAAAXa3K_0 2024-06-28T16:14:34.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7197486119239539065', time=2024-06-28T17:53:32.036639+0000, chain.objs.size()=1 2024-06-28T16:14:34.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.BNtg8wZmL2Ayg_7Vf4FN91c3apKZQ0z_0 2024-06-28T16:14:34.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18105445097336813766', time=2024-06-28T17:53:33.614450+0000, chain.objs.size()=1 2024-06-28T16:14:34.522 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.-rK4lq3HfP_ya4eQIWANU98fJ18460e_0 2024-06-28T16:14:34.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14790350423820538305', time=2024-06-28T17:53:33.768353+0000, chain.objs.size()=1 2024-06-28T16:14:34.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.JzV0he538J88cEJTNZH1kG7tpmzTaEQ_0 2024-06-28T16:14:34.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4410155889266250332', time=2024-06-28T17:53:34.354596+0000, chain.objs.size()=1 2024-06-28T16:14:34.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.omZOiM9J6QPf8QDUJNg2tOsFMi7ET2o_0 2024-06-28T16:14:34.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5429298076287005720', time=2024-06-28T17:53:34.358577+0000, chain.objs.size()=1 2024-06-28T16:14:34.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.7PfpwBRDra_8Y-ODq0l5oq922_ObM0k_0 2024-06-28T16:14:34.523 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12995497627956617333', time=2024-06-28T17:53:36.677129+0000, chain.objs.size()=1 2024-06-28T16:14:34.524 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.511+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.9BuMiu6MklWghUQ_8PFitleASIFO0cM_0 2024-06-28T16:14:34.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14669548343560801029', time=2024-06-28T17:53:38.655426+0000, chain.objs.size()=1 2024-06-28T16:14:34.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.zOvj5aVGU-EomSeQG4H57j_j0UDPhcX_0 2024-06-28T16:14:34.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10026667258953551904', time=2024-06-28T17:53:39.496467+0000, chain.objs.size()=1 2024-06-28T16:14:34.525 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.viom7kMR_U_qBCaJ5b5WG77sQUywevM_0 2024-06-28T16:14:34.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1926878190236825859', time=2024-06-28T17:53:43.774342+0000, chain.objs.size()=1 2024-06-28T16:14:34.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.hlo7FCiXa-dTvQkemLcZjr_nDTU2xkk_0 2024-06-28T16:14:34.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1082077329225326151', time=2024-06-28T17:53:44.543543+0000, chain.objs.size()=1 2024-06-28T16:14:34.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.64Q9n97MIfayLht01CcqtfvSREPxDbt_0 2024-06-28T16:14:34.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17602396361063926045', time=2024-06-28T17:53:47.359788+0000, chain.objs.size()=1 2024-06-28T16:14:34.526 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.G3bUVU3bGUP_U35PnghVbR3Es1H_o44_0 2024-06-28T16:14:34.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17715724154044061632', time=2024-06-28T17:53:48.730044+0000, chain.objs.size()=1 2024-06-28T16:14:34.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.AnQCUuLnEQ2zyR3vHRUe0UGWTLPujmz_0 2024-06-28T16:14:34.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.515+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7287354466944642432', time=2024-06-28T17:53:49.006038+0000, chain.objs.size()=1 2024-06-28T16:14:34.527 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.515+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qcBCQGc9OLqLVMbjLZbfTqDdkYJRteY_0 2024-06-28T16:14:34.531 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/40223 2024-06-28T16:14:34.534 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=16, truncated=0, next_marker='' 2024-06-28T16:14:34.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16634390308803118576', time=2024-06-28T17:53:50.230262+0000, chain.objs.size()=1 2024-06-28T16:14:34.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.jgphrbug1ldlvmcOohVlU_t4gr1GZR0_0 2024-06-28T16:14:34.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18123133927571722717', time=2024-06-28T17:54:01.902303+0000, chain.objs.size()=1 2024-06-28T16:14:34.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.324__shadow_.qLcdti2pZIvE5v4vwlT62kyctQoSVhu_0 2024-06-28T16:14:34.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12046615775906100774', time=2024-06-28T17:54:30.669360+0000, chain.objs.size()=1 2024-06-28T16:14:34.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.360__shadow_.klcy4xogcKkvbt23-wAvyfHpOz5AynY_0 2024-06-28T16:14:34.535 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13795053905736319893', time=2024-06-28T17:55:56.566660+0000, chain.objs.size()=1 2024-06-28T16:14:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.446__shadow_.8GAZp6KvLhmT_o6POG2m9AvkgS4ikiX_0 2024-06-28T16:14:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.481__shadow_.Pd7f2HEnQIc26it6ObxFEv-EGpgSjmZ_1_2143ca1bf086e7ed233b628b16e824b0', time=2024-06-28T17:56:41.816710+0000, chain.objs.size()=1 2024-06-28T16:14:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.481__shadow_.Pd7f2HEnQIc26it6ObxFEv-EGpgSjmZ_0 2024-06-28T16:14:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15074734687461794139', time=2024-06-28T17:57:28.979548+0000, chain.objs.size()=1 2024-06-28T16:14:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.508__multipart_mymultipart.2~rrKnQMA21XwALCpQaxc0YmiWd6EU5YA.1 2024-06-28T16:14:34.536 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4384297585052544567', time=2024-06-28T18:01:09.039047+0000, chain.objs.size()=1 2024-06-28T16:14:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.539__shadow_.UVF6exIfiYKHLLToab7zcULtVg71tJ__0 2024-06-28T16:14:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~oYe_bBJq03g5WM4XV6Oy_7lb0Ctzvdp', time=2024-06-28T18:01:43.636681+0000, chain.objs.size()=1 2024-06-28T16:14:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.550__multipart_mymultipart.2~oYe_bBJq03g5WM4XV6Oy_7lb0Ctzvdp.1 2024-06-28T16:14:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5599960598727508576', time=2024-06-28T18:02:23.758470+0000, chain.objs.size()=1 2024-06-28T16:14:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.560__shadow_.dkzWtigeJ6FwQUeW5roqwBNvpfAg_Ic_0 2024-06-28T16:14:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17816781535312601372', time=2024-06-28T18:03:21.282105+0000, chain.objs.size()=1 2024-06-28T16:14:34.537 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:rtLw8.jvI-N3UkfZLk1V1QYRoD3E4XE_.aS2SHT_rnjT8JrrkhUqNzTlI-_aEm5H_0 2024-06-28T16:14:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15032277995760767816', time=2024-06-28T18:03:56.016405+0000, chain.objs.size()=1 2024-06-28T16:14:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.568__shadow:0LRIb1ym7GGY9xzxoF6OuJxV2VDEKj2_.YzTaf0XLMpTVKsLiunOxhWsyIu0q5wA_0 2024-06-28T16:14:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9411537830458679692', time=2024-06-28T18:07:08.394788+0000, chain.objs.size()=1 2024-06-28T16:14:34.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.523+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.629__shadow_.iK6vMyxNiIVy0qxPdMJBBv_kOjqwk47_0 2024-06-28T16:14:34.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10263029566289475365', time=2024-06-28T18:07:24.085478+0000, chain.objs.size()=1 2024-06-28T16:14:34.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.637__shadow_.b-dRs1C5WisWT5RRvj0c548kFlIifld_0 2024-06-28T16:14:34.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5666928173400444479', time=2024-06-28T18:07:31.607760+0000, chain.objs.size()=1 2024-06-28T16:14:34.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.641__shadow_.WZ5zzUi5qywBqRDu5s0e2v70QAFduTi_0 2024-06-28T16:14:34.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5151770938360132105', time=2024-06-28T18:08:10.585983+0000, chain.objs.size()=1 2024-06-28T16:14:34.539 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.657__shadow_.c-emZamw1Ir2seg89AlnBzJTh4pea6j_0 2024-06-28T16:14:34.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.527+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4371686532524059342', time=2024-06-28T18:08:42.612423+0000, chain.objs.size()=1 2024-06-28T16:14:34.540 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.527+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.689__shadow:4QzH3A7nODBbcoHV9EpFHzArS4GpNo._.bXJDe973hsvAC6_7XkIaIv68gqgIpFM_0 2024-06-28T16:14:34.541 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.531+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:34.544 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-06-28T16:14:34.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:34.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:34.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/43043' 2024-06-28T16:14:34.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16821070141615851085', time=2024-06-28T17:45:01.777597+0000, chain.objs.size()=1 2024-06-28T16:14:34.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.236__shadow_.yNMXEFziryZyEYGhhuGM9_OvdhmfFX7_0 2024-06-28T16:14:34.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1642246551723371243', time=2024-06-28T17:45:08.236907+0000, chain.objs.size()=1 2024-06-28T16:14:34.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.240__shadow_.5u8bGdzK_tDB_k8Efyd-_oxrzTwjCv0_0 2024-06-28T16:14:34.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4416526629750499422', time=2024-06-28T17:45:08.250494+0000, chain.objs.size()=1 2024-06-28T16:14:34.547 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.240__shadow_._rSr692PXnBAmCOyzrfIuZs_Ti0s2z__0 2024-06-28T16:14:34.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5601837939328637912', time=2024-06-28T17:45:11.431057+0000, chain.objs.size()=1 2024-06-28T16:14:34.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.242__shadow_.XBph5Qxr4xtWR-0EXvrP3NBktfLvLxI_0 2024-06-28T16:14:34.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5825799247597902243', time=2024-06-28T17:45:17.866034+0000, chain.objs.size()=1 2024-06-28T16:14:34.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.246__shadow_.-pGu9UU97MNuGbZchrRdbcSY3Lz3xyF_0 2024-06-28T16:14:34.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3454677446256326320', time=2024-06-28T17:45:17.894085+0000, chain.objs.size()=1 2024-06-28T16:14:34.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.246__shadow_.hAuIvfy13WJpPEAyrmkruExlViGnhMh_0 2024-06-28T16:14:34.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6520582401385036993', time=2024-06-28T17:47:08.054121+0000, chain.objs.size()=1 2024-06-28T16:14:34.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.X-68YF70c1R0lW1cinb9bBev1YxFmEX_0 2024-06-28T16:14:34.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3003161674909611834', time=2024-06-28T17:47:08.432951+0000, chain.objs.size()=1 2024-06-28T16:14:34.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.UpY_hr3HRG93tz_Gwmi_l2imtb-Bt4d_0 2024-06-28T16:14:34.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9978310724225125778', time=2024-06-28T17:47:09.199855+0000, chain.objs.size()=1 2024-06-28T16:14:34.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.1M9UzXnMYbA4LogccNPR3pMZVINws-o_0 2024-06-28T16:14:34.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8834625721610316224', time=2024-06-28T17:47:13.166349+0000, chain.objs.size()=1 2024-06-28T16:14:34.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.wFz_TZy-brWs7knbi1jeZ48DX7wNPXC_0 2024-06-28T16:14:34.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3194542373329244327', time=2024-06-28T17:47:14.028192+0000, chain.objs.size()=1 2024-06-28T16:14:34.549 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ps01GVXboSwkEd817EtciRaJ2ICyL22_0 2024-06-28T16:14:34.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9135330511950279852', time=2024-06-28T17:47:16.954297+0000, chain.objs.size()=1 2024-06-28T16:14:34.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.535+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.5ptEwXksh-we5gdiWuO-B0kUaCZIUCW_0 2024-06-28T16:14:34.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.539+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2772550856974777224', time=2024-06-28T17:47:17.152064+0000, chain.objs.size()=1 2024-06-28T16:14:34.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.539+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.a2JI6FnXbWDDKXAy9os43dXzTfvDFgG_0 2024-06-28T16:14:34.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.539+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16381124991040582895', time=2024-06-28T17:47:17.430729+0000, chain.objs.size()=1 2024-06-28T16:14:34.550 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.539+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.S6mXIEorBw_GQOvdDwjkaHSTBgthaEP_0 2024-06-28T16:14:34.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.539+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8306941195149187437', time=2024-06-28T17:47:19.488997+0000, chain.objs.size()=1 2024-06-28T16:14:34.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.539+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.kvnKwGIJyPl1vpULBdYsAaPJOvPR-Az_0 2024-06-28T16:14:34.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15677545677544922806', time=2024-06-28T17:47:23.272833+0000, chain.objs.size()=1 2024-06-28T16:14:34.551 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.QZl8TXGmVyONYgc4ktq6B1DeSQDS2J4_0 2024-06-28T16:14:34.552 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16588068232251481431', time=2024-06-28T17:47:24.071534+0000, chain.objs.size()=1 2024-06-28T16:14:34.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.-hyKevfEEJYEEorO6SuCwl1Vd-VKRso_0 2024-06-28T16:14:34.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5547070465054966183', time=2024-06-28T17:47:25.519950+0000, chain.objs.size()=1 2024-06-28T16:14:34.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.PiVUICQSbnvDK4YQ-cF0d4HCeuJMNHx_0 2024-06-28T16:14:34.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17523784145573929842', time=2024-06-28T17:47:25.922222+0000, chain.objs.size()=1 2024-06-28T16:14:34.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.W_2K5fZpsO2MXyMqRzCiOloxu2wEpy7_0 2024-06-28T16:14:34.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6167177833960202151', time=2024-06-28T17:47:26.321273+0000, chain.objs.size()=1 2024-06-28T16:14:34.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.KJY4rnih1xanD578flDbAe2_slroXK7_0 2024-06-28T16:14:34.553 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.135028993734194557', time=2024-06-28T17:47:27.925998+0000, chain.objs.size()=1 2024-06-28T16:14:34.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.51NfFrnovgOPUHQkrWNRR4xjfXr3sdi_0 2024-06-28T16:14:34.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1747800807530746885', time=2024-06-28T17:47:29.125505+0000, chain.objs.size()=1 2024-06-28T16:14:34.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.a1LrOYmkvXUySwTJlb5hShqNe_ZCvHk_0 2024-06-28T16:14:34.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5929026349506746816', time=2024-06-28T17:47:30.666115+0000, chain.objs.size()=1 2024-06-28T16:14:34.554 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.MFiUcLY__1fJ2Gf4V_mKCHKrvOHtrCD_0 2024-06-28T16:14:34.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9763143325083912042', time=2024-06-28T17:47:33.349831+0000, chain.objs.size()=1 2024-06-28T16:14:34.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.543+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.U3SF5LJuw-VUCDqYAq7IxrROX1-TkQX_0 2024-06-28T16:14:34.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4704418880479475554', time=2024-06-28T17:47:34.905356+0000, chain.objs.size()=1 2024-06-28T16:14:34.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.NtSByg2tzeaSsyMGPxbD9Zc1lcYCNWe_0 2024-06-28T16:14:34.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15817402855745374358', time=2024-06-28T17:47:35.421765+0000, chain.objs.size()=1 2024-06-28T16:14:34.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9_FX_vX3D6bQmfjIkDFpjAzu9awLmoV_0 2024-06-28T16:14:34.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11969702035704469925', time=2024-06-28T17:47:37.298771+0000, chain.objs.size()=1 2024-06-28T16:14:34.556 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.CcqW1l_p8__0NZMPFMCzuHoYrt3qr3h_0 2024-06-28T16:14:34.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16754758707788857055', time=2024-06-28T17:47:38.731606+0000, chain.objs.size()=1 2024-06-28T16:14:34.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.EzHp8eIhV5Q7XqaVAiGbcnJHgh2vSvl_0 2024-06-28T16:14:34.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10023724004355008341', time=2024-06-28T17:47:38.891437+0000, chain.objs.size()=1 2024-06-28T16:14:34.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Rhg5JyRL_Gf_U70F2PjpCw9V-rKresw_0 2024-06-28T16:14:34.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.547+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11150711639039101307', time=2024-06-28T17:47:39.523217+0000, chain.objs.size()=1 2024-06-28T16:14:34.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.547+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Mc8CFadH9Bje5T99GjWN5x196udbe_T_0 2024-06-28T16:14:34.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.551+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9439398981642506278', time=2024-06-28T17:47:51.423191+0000, chain.objs.size()=1 2024-06-28T16:14:34.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.551+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.260__shadow_.XJFp-3korL5qjoLz3fFLMcLB9LeJKkT_0 2024-06-28T16:14:34.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.551+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4290114767202082212', time=2024-06-28T17:47:52.863587+0000, chain.objs.size()=1 2024-06-28T16:14:34.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.551+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.261__shadow_.JOZChRHcxasfvOf5DDS3UMbDZDXlrBT_0 2024-06-28T16:14:34.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.551+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7340516061854464455', time=2024-06-28T17:48:05.355816+0000, chain.objs.size()=1 2024-06-28T16:14:34.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.551+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.269__shadow_.q8wcy14zf-wxnwcb-uAnCzAXYOWh4Zq_0 2024-06-28T16:14:34.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.551+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11548220486282610344', time=2024-06-28T17:48:11.692743+0000, chain.objs.size()=1 2024-06-28T16:14:34.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.551+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.273__shadow_.wvsOF87NTD_LyN9PGynF7p30Ndk5Mae_0 2024-06-28T16:14:34.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.551+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11404484576516232575', time=2024-06-28T17:48:18.242051+0000, chain.objs.size()=1 2024-06-28T16:14:34.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.551+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.277__shadow_.bCa48ppA8pCkRTF206LVlAhPwExL89f_0 2024-06-28T16:14:34.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1082009257755966957', time=2024-06-28T17:48:29.579316+0000, chain.objs.size()=1 2024-06-28T16:14:34.563 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.xJEVt-yw-zHZ_a0j7biFEwg3-13bfZD_0 2024-06-28T16:14:34.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8782231355597386117', time=2024-06-28T17:48:52.751801+0000, chain.objs.size()=1 2024-06-28T16:14:34.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.294__shadow_.OLkoyi_tkNPrfN7uYhqBWYlKpKVpSFa_0 2024-06-28T16:14:34.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16156374291669751688', time=2024-06-28T17:51:06.630872+0000, chain.objs.size()=1 2024-06-28T16:14:34.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yHoxRBnzvhDm7eIf34JspLLG0FPZWUS_0 2024-06-28T16:14:34.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5701478255915657065', time=2024-06-28T17:51:07.415673+0000, chain.objs.size()=1 2024-06-28T16:14:34.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.DnDG5kOKnKCKrHgH4YQOW6PgaZkIRRY_0 2024-06-28T16:14:34.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8668373752780457288', time=2024-06-28T17:51:11.210598+0000, chain.objs.size()=1 2024-06-28T16:14:34.564 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.FfOVhDcnuXWkr-kAe7LimgVmGCUmyik_0 2024-06-28T16:14:34.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16266005007999796681', time=2024-06-28T17:51:11.545333+0000, chain.objs.size()=1 2024-06-28T16:14:34.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.wQC9bPpoorNn6cL4w_uDWSRN0dfiXZ0_0 2024-06-28T16:14:34.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1461972951134891361', time=2024-06-28T17:51:11.637371+0000, chain.objs.size()=1 2024-06-28T16:14:34.565 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xpUnSMGTlG7rtIMM4xPmHBb5npO2uhy_0 2024-06-28T16:14:34.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10097199645723678131', time=2024-06-28T17:51:13.883874+0000, chain.objs.size()=1 2024-06-28T16:14:34.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hu5WO2fe8odNUSTbHBtcifRhZY5Uskd_0 2024-06-28T16:14:34.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4441303931628624988', time=2024-06-28T17:51:14.318946+0000, chain.objs.size()=1 2024-06-28T16:14:34.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.q6cnFEMRTI20hBpLisz7IwLzxELdwVF_0 2024-06-28T16:14:34.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2302877335784204907', time=2024-06-28T17:51:15.477990+0000, chain.objs.size()=1 2024-06-28T16:14:34.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xf_5YWkLEfVdoHUPGo1ka88YerMSpeY_0 2024-06-28T16:14:34.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13290200523900154145', time=2024-06-28T17:51:16.121024+0000, chain.objs.size()=1 2024-06-28T16:14:34.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.555+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.W1Zo7d0vyT-DnVzvIwlY0GP1PRnV2Qu_0 2024-06-28T16:14:34.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3814338011068706009', time=2024-06-28T17:51:16.297656+0000, chain.objs.size()=1 2024-06-28T16:14:34.567 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.EtbBmYiPBXInDGWlWDsGuy-NJ2rwGn__0 2024-06-28T16:14:34.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13750161403056308448', time=2024-06-28T17:51:16.824251+0000, chain.objs.size()=1 2024-06-28T16:14:34.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.sW1QR2hjuwVCS-RaETqKEZUi65iZgS2_0 2024-06-28T16:14:34.569 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2476620771897289532', time=2024-06-28T17:51:17.217190+0000, chain.objs.size()=1 2024-06-28T16:14:34.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.NOVbGtq3C6paAevbldHO7IWvQIpYClZ_0 2024-06-28T16:14:34.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14376530662090349102', time=2024-06-28T17:51:18.552635+0000, chain.objs.size()=1 2024-06-28T16:14:34.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.pgi8yChtPC9vtUevO2PWfnjHR_gAmNW_0 2024-06-28T16:14:34.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16983197256000349121', time=2024-06-28T17:51:18.567005+0000, chain.objs.size()=1 2024-06-28T16:14:34.570 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.hmAipxcMVxuRTx2wQdtn5uiopVv-PJY_0 2024-06-28T16:14:34.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11236843591508286491', time=2024-06-28T17:51:20.110195+0000, chain.objs.size()=1 2024-06-28T16:14:34.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.J0oTcr4Xak6QLQt8Z5v9D0weJAGL4vl_0 2024-06-28T16:14:34.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.559+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6923389072116420466', time=2024-06-28T17:51:20.112484+0000, chain.objs.size()=1 2024-06-28T16:14:34.571 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.559+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GSrETSoeJdtdDLIUq2GE93VNhLhM5K6_0 2024-06-28T16:14:34.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5632350718844457955', time=2024-06-28T17:51:20.538652+0000, chain.objs.size()=1 2024-06-28T16:14:34.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.F5Fjqh4N3rJvVrgia9mCiohVt5GqC5S_0 2024-06-28T16:14:34.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7975481928412402716', time=2024-06-28T17:51:21.069886+0000, chain.objs.size()=1 2024-06-28T16:14:34.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.OMOgG4Bbw2fmEKElpGTCg7OQu1t5yw__0 2024-06-28T16:14:34.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16670637122836362292', time=2024-06-28T17:51:21.195039+0000, chain.objs.size()=1 2024-06-28T16:14:34.572 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Y7g72xCDSV5KFPyUNxP5GHwon7g9wIY_0 2024-06-28T16:14:34.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9178842713009025617', time=2024-06-28T17:51:21.586810+0000, chain.objs.size()=1 2024-06-28T16:14:34.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_._y117u7J7p5rbgpeklj_VqVCvcm5CIK_0 2024-06-28T16:14:34.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6164800136491009274', time=2024-06-28T17:51:21.758793+0000, chain.objs.size()=1 2024-06-28T16:14:34.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ry4_tm4Og8LqOgUVjWzsqJcNCbAh4nP_0 2024-06-28T16:14:34.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.563+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7043637004916747457', time=2024-06-28T17:51:22.595335+0000, chain.objs.size()=1 2024-06-28T16:14:34.573 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.563+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.6Zg1E0Nl5Y_X-a21x1VUX0HKAxB3mhD_0 2024-06-28T16:14:34.574 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14214423276792512777', time=2024-06-28T17:51:24.216348+0000, chain.objs.size()=1 2024-06-28T16:14:34.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.FZv9bRtwkpHv7SnY_eEmBIzCdOlYbzB_0 2024-06-28T16:14:34.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12174835123013411683', time=2024-06-28T17:51:26.788997+0000, chain.objs.size()=1 2024-06-28T16:14:34.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.JmfCCqYFw_T6XnII8f6JZXfVyf2iTY2_0 2024-06-28T16:14:34.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9815473212395724018', time=2024-06-28T17:51:26.790947+0000, chain.objs.size()=1 2024-06-28T16:14:34.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.kty-tvvPJhoggE_cXaiPe4DMOutLWyx_0 2024-06-28T16:14:34.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9351877267812559949', time=2024-06-28T17:51:28.218879+0000, chain.objs.size()=1 2024-06-28T16:14:34.575 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-wK7bl8CrfE4M1s2fwngajG8yc8SW_P_0 2024-06-28T16:14:34.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17464716402575394814', time=2024-06-28T17:51:28.625445+0000, chain.objs.size()=1 2024-06-28T16:14:34.576 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.XNffcjWQBaMF9fLf-xxVDAkU3ktJrmn_0 2024-06-28T16:14:34.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9470303448060966703', time=2024-06-28T17:51:32.022088+0000, chain.objs.size()=1 2024-06-28T16:14:34.577 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-hSGHLwBs4y9v6IZRieMi1wYqy4Ju8e_0 2024-06-28T16:14:34.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9888463608857001222', time=2024-06-28T17:51:32.426443+0000, chain.objs.size()=1 2024-06-28T16:14:34.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Q3296d_khNhrYg8oKWza0vMcTgURLcw_0 2024-06-28T16:14:34.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7621803080082193455', time=2024-06-28T17:51:32.666378+0000, chain.objs.size()=1 2024-06-28T16:14:34.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.K-F960cBtLvWdL9DXctBjo0yQXpbHLc_0 2024-06-28T16:14:34.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1648114156473953804', time=2024-06-28T17:51:33.241697+0000, chain.objs.size()=1 2024-06-28T16:14:34.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.tfsehTW_5NIWPUeMAIG4R3K3BUqWAn4_0 2024-06-28T16:14:34.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13655179060049641510', time=2024-06-28T17:51:33.669714+0000, chain.objs.size()=1 2024-06-28T16:14:34.578 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.igKYKCUpgRvI8x_3Qxwbxt4nUOPIXBx_0 2024-06-28T16:14:34.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1785634220668756467', time=2024-06-28T17:51:33.996264+0000, chain.objs.size()=1 2024-06-28T16:14:34.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.a4OSlA1q-6iDDW_ENAXpe-tPqSHhXP4_0 2024-06-28T16:14:34.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12724948711147170154', time=2024-06-28T17:51:34.584427+0000, chain.objs.size()=1 2024-06-28T16:14:34.579 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.567+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.brOXMdXToMcRKvLywyWOCjpTSyoOJWn_0 2024-06-28T16:14:34.580 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.760328803985041733', time=2024-06-28T17:51:36.274651+0000, chain.objs.size()=1 2024-06-28T16:14:34.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.0Xcy1JSL8qlVgN00b9J53gGrjJkSyxg_0 2024-06-28T16:14:34.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9100126410319556151', time=2024-06-28T17:51:37.270231+0000, chain.objs.size()=1 2024-06-28T16:14:34.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.mHzQtRFYnDa6QoAsAzS1HY4wEORXWDk_0 2024-06-28T16:14:34.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.818391022846418153', time=2024-06-28T17:51:37.342769+0000, chain.objs.size()=1 2024-06-28T16:14:34.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.QQWHGXFJepqW2B59Vwg_D3LFWFgO4PB_0 2024-06-28T16:14:34.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13974517808458966138', time=2024-06-28T17:51:37.777604+0000, chain.objs.size()=1 2024-06-28T16:14:34.581 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.PW9mJ-G4NiB1364OCA8TF1UClewOIdZ_0 2024-06-28T16:14:34.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9359921091539834927', time=2024-06-28T17:51:37.779706+0000, chain.objs.size()=1 2024-06-28T16:14:34.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.O4fWY8MAsAQO11uJp5pjVgnESScN_xK_0 2024-06-28T16:14:34.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1097201266524037862', time=2024-06-28T17:53:20.985315+0000, chain.objs.size()=1 2024-06-28T16:14:34.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bgq0TnfMihhEWe9R8AOL96NBxYA5N6L_0 2024-06-28T16:14:34.582 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.571+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13668870277388547070', time=2024-06-28T17:53:22.296629+0000, chain.objs.size()=1 2024-06-28T16:14:34.583 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.571+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CDiDCH4kyEQMFtUAyZKc99KX19cDx-H_0 2024-06-28T16:14:34.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13042092437065819090', time=2024-06-28T17:53:22.677903+0000, chain.objs.size()=1 2024-06-28T16:14:34.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.wZiAq-QyQWhZ4h9jPK5eWB2ShkfE4vh_0 2024-06-28T16:14:34.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15905613635898224762', time=2024-06-28T17:53:23.592750+0000, chain.objs.size()=1 2024-06-28T16:14:34.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.y7qa1I-tOb6-N9Ivf3GqDWzLNVF9dNY_0 2024-06-28T16:14:34.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.567069010456426609', time=2024-06-28T17:53:25.285262+0000, chain.objs.size()=1 2024-06-28T16:14:34.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.237ZbO62SkvpbDBeT0tsraNLg9sKqcm_0 2024-06-28T16:14:34.585 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3233871941442941227', time=2024-06-28T17:53:25.431138+0000, chain.objs.size()=1 2024-06-28T16:14:34.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.EZLQpaWoUQUmoPaoZ2FRltrVFHJ9PZ2_0 2024-06-28T16:14:34.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12164268024908701799', time=2024-06-28T17:53:25.620817+0000, chain.objs.size()=1 2024-06-28T16:14:34.586 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.z7ADlUVGrbCuws78g2yFlKpSLvOVZ56_0 2024-06-28T16:14:34.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9326084604097700582', time=2024-06-28T17:53:27.194586+0000, chain.objs.size()=1 2024-06-28T16:14:34.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.yIXbdJJn8RCU_mKvlP6Gfvt1UozJRPE_0 2024-06-28T16:14:34.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14839117891350681405', time=2024-06-28T17:53:30.289651+0000, chain.objs.size()=1 2024-06-28T16:14:34.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.scsbIevFeiGDGw50jBtSqwahRW22mEI_0 2024-06-28T16:14:34.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7521014928797962678', time=2024-06-28T17:53:31.026037+0000, chain.objs.size()=1 2024-06-28T16:14:34.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.huaaVsHjJpTrBSzwC7wdGYqF3pWIjBH_0 2024-06-28T16:14:34.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4574942036564320575', time=2024-06-28T17:53:32.278564+0000, chain.objs.size()=1 2024-06-28T16:14:34.587 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.575+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.oy58vT4iZWjActlbAwMd6JAC1q4hdW5_0 2024-06-28T16:14:34.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17025513932819340297', time=2024-06-28T17:53:32.472915+0000, chain.objs.size()=1 2024-06-28T16:14:34.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.70o7wLo_yDcPYi6SQFhF2HblLfxwMeQ_0 2024-06-28T16:14:34.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3610223779579751889', time=2024-06-28T17:53:36.529961+0000, chain.objs.size()=1 2024-06-28T16:14:34.588 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.cmWBOmlyyMlUIdP3rcW6qgPUe_4stot_0 2024-06-28T16:14:34.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5657230359856671236', time=2024-06-28T17:53:40.562333+0000, chain.objs.size()=1 2024-06-28T16:14:34.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.hs4HxsZN4kp389Lj5fuGL9a8ySPKOUi_0 2024-06-28T16:14:34.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11867504767637928726', time=2024-06-28T17:53:40.834088+0000, chain.objs.size()=1 2024-06-28T16:14:34.589 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4OwBGDMps9vcY1FA_kPPt1QqMFdDwdu_0 2024-06-28T16:14:34.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17178378624344424475', time=2024-06-28T17:53:40.903961+0000, chain.objs.size()=1 2024-06-28T16:14:34.590 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.7mK0QNsL7KNN_S_5HyGDYlWQ0szi3kJ_0 2024-06-28T16:14:34.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4487355687265723046', time=2024-06-28T17:53:43.516289+0000, chain.objs.size()=1 2024-06-28T16:14:34.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CTsQMVqw7d02tTuZKi2iKoCFay18xod_0 2024-06-28T16:14:34.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.579+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8827936294206308775', time=2024-06-28T17:53:44.615205+0000, chain.objs.size()=1 2024-06-28T16:14:34.591 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.579+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.e7J9eWlaKxJC6gY9T_5YZTJE9A1kgEt_0 2024-06-28T16:14:34.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15448353953679520062', time=2024-06-28T17:53:44.825292+0000, chain.objs.size()=1 2024-06-28T16:14:34.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.AVNySEL-rph7X4AkjAbFA1EM042fefa_0 2024-06-28T16:14:34.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6201072349012093815', time=2024-06-28T17:53:45.910092+0000, chain.objs.size()=1 2024-06-28T16:14:34.592 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.R-jIjjACkxXvf8MNVfNxRhBsVjRSTKQ_0 2024-06-28T16:14:34.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13334360473373088265', time=2024-06-28T17:53:46.570834+0000, chain.objs.size()=1 2024-06-28T16:14:34.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1oEKppz57LgCTVJOR4VTifm2n361dpc_0 2024-06-28T16:14:34.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.675813017406859319', time=2024-06-28T17:53:48.119985+0000, chain.objs.size()=1 2024-06-28T16:14:34.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.HO3tQbSUjHjwgCXZ8-LlGV93_DeoHSF_0 2024-06-28T16:14:34.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4452745749617884444', time=2024-06-28T17:53:49.714018+0000, chain.objs.size()=1 2024-06-28T16:14:34.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.xibOWT5waaNfQCsO7N9EVH7DzwrC-wE_0 2024-06-28T16:14:34.595 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.583+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16279291887510876430', time=2024-06-28T17:53:50.798500+0000, chain.objs.size()=1 2024-06-28T16:14:34.596 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.583+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.fjSqDKqA_8t4mxKNS8nuLELRcKd_YPE_0 2024-06-28T16:14:34.598 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.587+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/43043 2024-06-28T16:14:34.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=11, truncated=0, next_marker='' 2024-06-28T16:14:34.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4329666028230060467', time=2024-06-28T17:54:57.583960+0000, chain.objs.size()=1 2024-06-28T16:14:34.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.384__shadow:u8aHDTE6pWRWCRHb-KdNu2K7nDThgus_.Qp9JKmff4K0ZhYYUWm9_BtRo67o6r14_0 2024-06-28T16:14:34.601 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4190208502011429474', time=2024-06-28T17:56:06.665988+0000, chain.objs.size()=1 2024-06-28T16:14:34.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.452__shadow_.Swfh0n2poScuHSzAQcufsaQtqOpPcaf_0 2024-06-28T16:14:34.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2245268052685905688', time=2024-06-28T17:56:47.982668+0000, chain.objs.size()=1 2024-06-28T16:14:34.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.488__shadow_.bR9TQ5lPZzjoZKkOpHeUZq_U-s7jNij_0 2024-06-28T16:14:34.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.489__shadow_.tsA53JmfkJoyXHBK6RIFTS3XaQXOHVQ_1_e472a9c4998a014cd02e326c9f9f5995', time=2024-06-28T17:56:49.902570+0000, chain.objs.size()=1 2024-06-28T16:14:34.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.489__shadow_.tsA53JmfkJoyXHBK6RIFTS3XaQXOHVQ_0 2024-06-28T16:14:34.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12933323259262559961', time=2024-06-28T17:56:50.302901+0000, chain.objs.size()=1 2024-06-28T16:14:34.602 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.490__shadow_.kGeXBSp5hgeaAc9Qmx89aDaRZegftIR_0 2024-06-28T16:14:34.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15030983207607557402', time=2024-06-28T17:57:28.624400+0000, chain.objs.size()=1 2024-06-28T16:14:34.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.507__shadow_.EP17jrlt-HzREhl4Or44uM0rQATqzvV_0 2024-06-28T16:14:34.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12166394804129965250', time=2024-06-28T17:57:47.478454+0000, chain.objs.size()=6 2024-06-28T16:14:34.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.509__multipart_mymultipart.2~ot7nn5CwvQG-PlLKnkSOlggw5712hEE.1 2024-06-28T16:14:34.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.509__multipart_mymultipart.2~ot7nn5CwvQG-PlLKnkSOlggw5712hEE.2 2024-06-28T16:14:34.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.509__multipart_mymultipart.2~ot7nn5CwvQG-PlLKnkSOlggw5712hEE.3 2024-06-28T16:14:34.603 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.509__multipart_mymultipart.2~ot7nn5CwvQG-PlLKnkSOlggw5712hEE.4 2024-06-28T16:14:34.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.509__multipart_mymultipart.2~ot7nn5CwvQG-PlLKnkSOlggw5712hEE.5 2024-06-28T16:14:34.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.591+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.509__multipart_mymultipart.2~ot7nn5CwvQG-PlLKnkSOlggw5712hEE.6 2024-06-28T16:14:34.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.595+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13157258767461381950', time=2024-06-28T18:02:53.152863+0000, chain.objs.size()=1 2024-06-28T16:14:34.604 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.595+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:Q5ONfVW3J.o3ek.slCQ29UQynm3R0zc_.aDgZT74SrNEGh4gsjxmHzyzNmt2k_1T_0 2024-06-28T16:14:34.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.595+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12165781980515660840', time=2024-06-28T18:04:10.317802+0000, chain.objs.size()=1 2024-06-28T16:14:34.605 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.595+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.575__shadow:wu3MqWiNR9z-sR0jHxy3hsOfvwoLrLB_.j6UxAa1hTa8ypbkyUkf6XOfZYvyLudH_0 2024-06-28T16:14:34.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.595+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3583553799652278467', time=2024-06-28T18:04:29.670705+0000, chain.objs.size()=1 2024-06-28T16:14:34.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.595+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.594__shadow_.Nmj66jw-9X1YVisp1lgMuXN87IqNyey_0 2024-06-28T16:14:34.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.595+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3426447779699747222', time=2024-06-28T18:08:38.527754+0000, chain.objs.size()=1 2024-06-28T16:14:34.606 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.595+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.686__shadow:ZeEXRabi0zM.X.O82euxpv3iFKlHMUa_.KjmVNu1BYPdNc49UrKQu_s3Vykc7do6_0 2024-06-28T16:14:34.608 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.599+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:34.611 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-06-28T16:14:34.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:34.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:34.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/48931' 2024-06-28T16:14:34.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6077672912051794492', time=2024-06-28T17:44:32.159158+0000, chain.objs.size()=1 2024-06-28T16:14:34.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.218__shadow_.J7H3cXv4wQ0v0TYTly49aVmSJVsE5JW_0 2024-06-28T16:14:34.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10759163703293357228', time=2024-06-28T17:45:14.664151+0000, chain.objs.size()=1 2024-06-28T16:14:34.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.244__shadow_.3jcn0YyMubLNnZund5OAggtnPqQCx1C_0 2024-06-28T16:14:34.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10018005285810832270', time=2024-06-28T17:45:22.307300+0000, chain.objs.size()=1 2024-06-28T16:14:34.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.249__shadow_.0wx-Z1lVYjPbL7XtONlMrERediHaVH2_0 2024-06-28T16:14:34.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.998749980135403426', time=2024-06-28T17:47:08.605246+0000, chain.objs.size()=1 2024-06-28T16:14:34.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.RyovFEl0-neLOWd0BG1jFBAJv6F9knA_0 2024-06-28T16:14:34.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6115183560166702274', time=2024-06-28T17:47:09.199089+0000, chain.objs.size()=1 2024-06-28T16:14:34.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ea1iypv844l3mqC4jsu7UUQusOTjvPj_0 2024-06-28T16:14:34.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5089039999826136658', time=2024-06-28T17:47:09.201388+0000, chain.objs.size()=1 2024-06-28T16:14:34.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.vkrn28oJ41yCD5syOfAraVT2GtJz0Ki_0 2024-06-28T16:14:34.615 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5410155201412775522', time=2024-06-28T17:47:10.451977+0000, chain.objs.size()=1 2024-06-28T16:14:34.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.PfeY1Kl7fM9keEMiI1V3m3E-xolw0gw_0 2024-06-28T16:14:34.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5770471368048500187', time=2024-06-28T17:47:10.743162+0000, chain.objs.size()=1 2024-06-28T16:14:34.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.W99qC3RXKf49VYf4qatnZhg35VAklby_0 2024-06-28T16:14:34.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4412099325636322102', time=2024-06-28T17:47:11.135171+0000, chain.objs.size()=1 2024-06-28T16:14:34.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Hem4S2bPb2tTtZWc93dXwLWvFuBSteH_0 2024-06-28T16:14:34.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2598921966388355166', time=2024-06-28T17:47:12.908136+0000, chain.objs.size()=1 2024-06-28T16:14:34.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Yfh7GwSxzkUQYs2k83A66nzHd_LcHSh_0 2024-06-28T16:14:34.616 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5125842531141621802', time=2024-06-28T17:47:14.070493+0000, chain.objs.size()=1 2024-06-28T16:14:34.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9no8exV8SM0IqYnu--Y8xMu_yhvt69W_0 2024-06-28T16:14:34.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8663335130449776305', time=2024-06-28T17:47:15.667359+0000, chain.objs.size()=1 2024-06-28T16:14:34.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.603+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Qn5NhKmaG_Hr5D3CFCgqF0mdiDiDSwy_0 2024-06-28T16:14:34.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.607+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3668936635243409143', time=2024-06-28T17:47:19.335283+0000, chain.objs.size()=1 2024-06-28T16:14:34.617 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.607+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Fh9ChttC6sQxQyBZneogsNpT-eRYyhC_0 2024-06-28T16:14:34.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1734810830694813457', time=2024-06-28T17:47:22.025245+0000, chain.objs.size()=1 2024-06-28T16:14:34.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.eYjdWXSiV6QDymG-PZbWZCK-EQognTP_0 2024-06-28T16:14:34.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2931756099422742218', time=2024-06-28T17:47:23.271813+0000, chain.objs.size()=1 2024-06-28T16:14:34.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7YsN75CfhXwEJNwC-Ilhc2HuH_ARUG__0 2024-06-28T16:14:34.620 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8660039544041894038', time=2024-06-28T17:47:23.273963+0000, chain.objs.size()=1 2024-06-28T16:14:34.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.bqYEyht-U7x9vHFObg26fgWZLz0dKvP_0 2024-06-28T16:14:34.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4641857639578793426', time=2024-06-28T17:47:23.440556+0000, chain.objs.size()=1 2024-06-28T16:14:34.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.UrcnpB7JpLPqokCaqZVbABMkp-Pt9Hr_0 2024-06-28T16:14:34.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15709922420881550448', time=2024-06-28T17:47:24.072904+0000, chain.objs.size()=1 2024-06-28T16:14:34.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.XGUW5bNUFDPyBObxounUb6N5UUUdg_a_0 2024-06-28T16:14:34.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.411568214880106905', time=2024-06-28T17:47:24.136886+0000, chain.objs.size()=1 2024-06-28T16:14:34.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.eN4pf_2p8VJHtC91mJuApxSm9cdPNRv_0 2024-06-28T16:14:34.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14131230318280955598', time=2024-06-28T17:47:26.592151+0000, chain.objs.size()=1 2024-06-28T16:14:34.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oyqVahBUwl2KWJXBP9rrv9a2hdALPnp_0 2024-06-28T16:14:34.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10220575284096545', time=2024-06-28T17:47:26.915665+0000, chain.objs.size()=1 2024-06-28T16:14:34.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.cDwXJxYf9PHsJFoFOmI0h3DVGPNd2US_0 2024-06-28T16:14:34.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5146899350762202721', time=2024-06-28T17:47:27.997213+0000, chain.objs.size()=1 2024-06-28T16:14:34.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.IeEdsTAv6BF_OBqk324MxEC7-Ad6H7E_0 2024-06-28T16:14:34.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12015761518048214683', time=2024-06-28T17:47:28.180159+0000, chain.objs.size()=1 2024-06-28T16:14:34.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.cz14cVVBRr_6w9K-ndBrHBnwzbMZHc2_0 2024-06-28T16:14:34.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13619669171534414233', time=2024-06-28T17:47:28.958277+0000, chain.objs.size()=1 2024-06-28T16:14:34.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.611+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.zpBsdAPtN3YE2tjEBtsaueyEASTcyz9_0 2024-06-28T16:14:34.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10469137824964189835', time=2024-06-28T17:47:29.375768+0000, chain.objs.size()=1 2024-06-28T16:14:34.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.8Tf_FYAa7twrZpdssQASCdbLpvW2Ycp_0 2024-06-28T16:14:34.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10215017434798715467', time=2024-06-28T17:47:29.613979+0000, chain.objs.size()=1 2024-06-28T16:14:34.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.AkiiUkuWbeptl103RmtlUse3A3azQaf_0 2024-06-28T16:14:34.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5086227457137682652', time=2024-06-28T17:47:30.182274+0000, chain.objs.size()=1 2024-06-28T16:14:34.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.HrjHfB_H6x-Xj6n_MZZ_TrGbh1zHQn8_0 2024-06-28T16:14:34.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9223505658628113628', time=2024-06-28T17:47:30.241273+0000, chain.objs.size()=1 2024-06-28T16:14:34.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.o1tF4j1n_8KHmcPCaJHn-P1OSBDljZP_0 2024-06-28T16:14:34.625 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9586149571872494390', time=2024-06-28T17:47:30.657530+0000, chain.objs.size()=1 2024-06-28T16:14:34.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.cNfadvc51fgyagfAGT601COsFF28y7W_0 2024-06-28T16:14:34.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4845053275601425925', time=2024-06-28T17:47:31.119426+0000, chain.objs.size()=1 2024-06-28T16:14:34.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.X3lyHs5hbcDDnzPJ0p8f-76_T-cPTL6_0 2024-06-28T16:14:34.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7240174343701939195', time=2024-06-28T17:47:32.069607+0000, chain.objs.size()=1 2024-06-28T16:14:34.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.CkmESDCiIGyMjC1rTWsj2naUL-jddcT_0 2024-06-28T16:14:34.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14368339208444415833', time=2024-06-28T17:47:33.577309+0000, chain.objs.size()=1 2024-06-28T16:14:34.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.s1hz8KBvj99wKJtvVi8F1LJCAYT6S-B_0 2024-06-28T16:14:34.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4926530112813378643', time=2024-06-28T17:47:33.904573+0000, chain.objs.size()=1 2024-06-28T16:14:34.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.e78cTNoqLoagEkBzmAAPKTfGoArYbS2_0 2024-06-28T16:14:34.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8526083168801544780', time=2024-06-28T17:47:33.906711+0000, chain.objs.size()=1 2024-06-28T16:14:34.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.8pcOAe2QRYygdvZ4mqddSwHqIUK2-LH_0 2024-06-28T16:14:34.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5748935799709397007', time=2024-06-28T17:47:34.403162+0000, chain.objs.size()=1 2024-06-28T16:14:34.627 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.615+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Ctu-M6UeEwRNIhrVsg4Ut2qHrsG3p4J_0 2024-06-28T16:14:34.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12460584690132752639', time=2024-06-28T17:47:35.066613+0000, chain.objs.size()=1 2024-06-28T16:14:34.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.wwlhXYgX8CREWxhlEh0KtT86ArwPHjk_0 2024-06-28T16:14:34.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1139113697857739512', time=2024-06-28T17:47:35.228056+0000, chain.objs.size()=1 2024-06-28T16:14:34.628 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.eoyDzKSU7suBC2MiyQykH4tSToj0dZq_0 2024-06-28T16:14:34.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15490139431228924905', time=2024-06-28T17:47:37.853401+0000, chain.objs.size()=1 2024-06-28T16:14:34.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.wCn6T7wfjNT2JPtHIzbz2iHSaV_yshi_0 2024-06-28T16:14:34.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10076860260906985417', time=2024-06-28T17:47:37.944053+0000, chain.objs.size()=1 2024-06-28T16:14:34.630 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.0zxHiIPbS5DPaUTB9qAaqwTqTrauqa0_0 2024-06-28T16:14:34.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16941058095863803831', time=2024-06-28T17:47:38.208214+0000, chain.objs.size()=1 2024-06-28T16:14:34.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_._bfYU_UQgYGGl5hYCfDRjf5BQN0kUQV_0 2024-06-28T16:14:34.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9503032078674356084', time=2024-06-28T17:48:02.075767+0000, chain.objs.size()=1 2024-06-28T16:14:34.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.267__shadow_.Ay0O2OIvUwK7GkNGT-WJKPZZ2c-O-h2_0 2024-06-28T16:14:34.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10240376859624669355', time=2024-06-28T17:48:18.241236+0000, chain.objs.size()=1 2024-06-28T16:14:34.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.277__shadow_.LLr8FhNsxzTU-gThlH-aYgnUQgS7vVD_0 2024-06-28T16:14:34.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17862569680974092774', time=2024-06-28T17:48:29.771921+0000, chain.objs.size()=1 2024-06-28T16:14:34.631 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.pHdLQqREykkygNifULjfWtZo0odd7-2_0 2024-06-28T16:14:34.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13198205631051353510', time=2024-06-28T17:48:29.773981+0000, chain.objs.size()=1 2024-06-28T16:14:34.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.ssOyipW31W43O-5-o-5rtfrjzs4z2Oj_0 2024-06-28T16:14:34.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8889263306466571895', time=2024-06-28T17:48:47.891583+0000, chain.objs.size()=1 2024-06-28T16:14:34.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.619+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.291__shadow_.mOZCCU3QOv181_5rcMHb9hniBRHOcB9_0 2024-06-28T16:14:34.632 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.623+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6756809699662481250', time=2024-06-28T17:48:52.752983+0000, chain.objs.size()=1 2024-06-28T16:14:34.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.623+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.294__shadow_.fXw1RB98CAefF5QHxlQU7OMlfAfJTCE_0 2024-06-28T16:14:34.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.623+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10641795834890744275', time=2024-06-28T17:48:56.362546+0000, chain.objs.size()=1 2024-06-28T16:14:34.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.623+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.296__shadow_.HD_YuWWZI6uMOkEzx-Dte38SWzs-H4t_0 2024-06-28T16:14:34.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.623+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5328284813267789218', time=2024-06-28T17:48:58.966979+0000, chain.objs.size()=1 2024-06-28T16:14:34.633 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.623+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.297__shadow:cQ2RuD4jgCV.b96bdxmG8NVV6dYdXuO_.8HbNzfLoktQqzrxalsU8o5IybiYfKGf_0 2024-06-28T16:14:34.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16793234678832443285', time=2024-06-28T17:51:07.803228+0000, chain.objs.size()=1 2024-06-28T16:14:34.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.4ZdT7JJk9cNz6TAkOAfbNqjD2SVaiyE_0 2024-06-28T16:14:34.635 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16593232846286222665', time=2024-06-28T17:51:08.263754+0000, chain.objs.size()=1 2024-06-28T16:14:34.636 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.blVZEKrtRV3hOqRhrBgUGEdIzIdzV_W_0 2024-06-28T16:14:34.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18089805001861688534', time=2024-06-28T17:51:09.963782+0000, chain.objs.size()=1 2024-06-28T16:14:34.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.WNPQ1L8GhFCW6fzd6Z1W9LtBZmktKPn_0 2024-06-28T16:14:34.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15420174961691130405', time=2024-06-28T17:51:11.701340+0000, chain.objs.size()=1 2024-06-28T16:14:34.637 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Vr4917Cbk40EPLV15txE8NMqvueMZeX_0 2024-06-28T16:14:34.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7732237582209324977', time=2024-06-28T17:51:13.387903+0000, chain.objs.size()=1 2024-06-28T16:14:34.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.YkqWJgetleG-ZIrc6nC9gRr5j9U0kFn_0 2024-06-28T16:14:34.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16411065646370700590', time=2024-06-28T17:51:13.737493+0000, chain.objs.size()=1 2024-06-28T16:14:34.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.AG4E8Hn5J8UAPWXob4jFS1wXxZTBSij_0 2024-06-28T16:14:34.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5636452103257368018', time=2024-06-28T17:51:13.906179+0000, chain.objs.size()=1 2024-06-28T16:14:34.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.q2tsSmYIJwG4nCkKb3H1Bqown8dTA8e_0 2024-06-28T16:14:34.638 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12792149255259266115', time=2024-06-28T17:51:16.835340+0000, chain.objs.size()=1 2024-06-28T16:14:34.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ZnVnhcaKiN1vB56loFCDXnkNPUDZ1lD_0 2024-06-28T16:14:34.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6711925817049304953', time=2024-06-28T17:51:16.837045+0000, chain.objs.size()=1 2024-06-28T16:14:34.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-GFywywijg9zSiSRaJcVqqUpaaotZOb_0 2024-06-28T16:14:34.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15549986628844430778', time=2024-06-28T17:51:18.065869+0000, chain.objs.size()=1 2024-06-28T16:14:34.639 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.b7hqamxMPgXV55PR9nioez7IrQsmN5X_0 2024-06-28T16:14:34.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9822136657191750620', time=2024-06-28T17:51:18.435345+0000, chain.objs.size()=1 2024-06-28T16:14:34.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.627+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.dbpayiecL79qU5PoxTeF4I8gnlVMobI_0 2024-06-28T16:14:34.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8590383255851641821', time=2024-06-28T17:51:18.522079+0000, chain.objs.size()=1 2024-06-28T16:14:34.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.o7d6TvQUrXiZCJ61M4KZtyFL1XkrLg__0 2024-06-28T16:14:34.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14189003597398848615', time=2024-06-28T17:51:24.921264+0000, chain.objs.size()=1 2024-06-28T16:14:34.640 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.CV81SecphA7XLKL-W-TIMiDzSTflak5_0 2024-06-28T16:14:34.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14191090070184109709', time=2024-06-28T17:51:26.347086+0000, chain.objs.size()=1 2024-06-28T16:14:34.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.8Vsfle30Dw6Ua5u6TXXpnJ-QZj2e50a_0 2024-06-28T16:14:34.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3242605684050800121', time=2024-06-28T17:51:26.628554+0000, chain.objs.size()=1 2024-06-28T16:14:34.641 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Fv4qQXB3tTubFpP_kmkAy_EYFKYuEm1_0 2024-06-28T16:14:34.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13000581951244421803', time=2024-06-28T17:51:26.794677+0000, chain.objs.size()=1 2024-06-28T16:14:34.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.yUzCxqKgJEI-s695JlIXhbkEs1m5gGt_0 2024-06-28T16:14:34.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.631+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4929982325019316818', time=2024-06-28T17:51:29.073422+0000, chain.objs.size()=1 2024-06-28T16:14:34.642 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.631+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.nKW20fzYpV-DInCAO6WQ5SvMAtDGI_2_0 2024-06-28T16:14:34.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17686460025781719751', time=2024-06-28T17:51:29.075707+0000, chain.objs.size()=1 2024-06-28T16:14:34.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.-oNhAfF5D7WJbbokBfHNj_rpIat-ZEq_0 2024-06-28T16:14:34.643 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9832394375643665447', time=2024-06-28T17:51:29.198567+0000, chain.objs.size()=1 2024-06-28T16:14:34.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.7-5RT8ooIRBFprTYuzNqm2bvsdydYbJ_0 2024-06-28T16:14:34.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5480443544083922376', time=2024-06-28T17:51:30.841415+0000, chain.objs.size()=1 2024-06-28T16:14:34.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.h6W1565tsUzIYNb1coQt1vPSPBWr0Xd_0 2024-06-28T16:14:34.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9978598594513366781', time=2024-06-28T17:51:32.761584+0000, chain.objs.size()=1 2024-06-28T16:14:34.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.34XWO1q1p5a6b7jg4xOS9ipqg4rnrev_0 2024-06-28T16:14:34.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.483537028195837967', time=2024-06-28T17:51:34.685407+0000, chain.objs.size()=1 2024-06-28T16:14:34.644 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.SuUkxR-x6elsg7CI9XNNEP4tuugUaBX_0 2024-06-28T16:14:34.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16383771742399064395', time=2024-06-28T17:51:36.661385+0000, chain.objs.size()=1 2024-06-28T16:14:34.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.dB-TjSMquz91iDHRSRjTQejB3Ow6ZJb_0 2024-06-28T16:14:34.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10438940030077330373', time=2024-06-28T17:51:36.847070+0000, chain.objs.size()=1 2024-06-28T16:14:34.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_._almEowzllSjTZafDYwT_0oXpppcuk3_0 2024-06-28T16:14:34.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.173946694883477428', time=2024-06-28T17:51:37.127235+0000, chain.objs.size()=1 2024-06-28T16:14:34.645 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.SvDiy__o7JHN0W4a7NtNtOlkQTkulEb_0 2024-06-28T16:14:34.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9598540559292041634', time=2024-06-28T17:51:37.619587+0000, chain.objs.size()=1 2024-06-28T16:14:34.646 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.635+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.HhCWXzSKx-U9qVGIbpjcLSDeMoYJR5t_0 2024-06-28T16:14:34.647 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18400388186911034528', time=2024-06-28T17:51:37.902586+0000, chain.objs.size()=1 2024-06-28T16:14:34.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ouBd2yslG2xNZnW7JaCITrG4QFKzomr_0 2024-06-28T16:14:34.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4184142281386433496', time=2024-06-28T17:51:38.395438+0000, chain.objs.size()=1 2024-06-28T16:14:34.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.RkyJqsJ0hLq-giRxZwmNVNGAOtPk3wn_0 2024-06-28T16:14:34.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.387691072176930761', time=2024-06-28T17:53:21.496615+0000, chain.objs.size()=1 2024-06-28T16:14:34.648 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.bvTZEU5MSHLhItbhbCADgO9GQgDwhPt_0 2024-06-28T16:14:34.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9539742591742913825', time=2024-06-28T17:53:21.765277+0000, chain.objs.size()=1 2024-06-28T16:14:34.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4ZchpeooMpC0juhmQTIFtw2_-_pbZy-_0 2024-06-28T16:14:34.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13297682353201346770', time=2024-06-28T17:53:21.996972+0000, chain.objs.size()=1 2024-06-28T16:14:34.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ynCSHTb7sdKYN5HYKVydmBmljIG1WU5_0 2024-06-28T16:14:34.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11799455065339158446', time=2024-06-28T17:53:25.292421+0000, chain.objs.size()=1 2024-06-28T16:14:34.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.YPGEFuEpV3G7IYuFrTCepAhLIfvwKVO_0 2024-06-28T16:14:34.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17234085146878732197', time=2024-06-28T17:53:25.600364+0000, chain.objs.size()=1 2024-06-28T16:14:34.649 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.a7EEx_iKW46XmaimAlOeKhJsWBwnCEp_0 2024-06-28T16:14:34.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13311808134436954116', time=2024-06-28T17:53:26.097119+0000, chain.objs.size()=1 2024-06-28T16:14:34.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.NVEAkgXOiBxaU_uUf2-XCu45niJbVLK_0 2024-06-28T16:14:34.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5471350123067658259', time=2024-06-28T17:53:26.972700+0000, chain.objs.size()=1 2024-06-28T16:14:34.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vMDkmJk_MmPzaV8R9ZnAFZnpGZrPcO7_0 2024-06-28T16:14:34.650 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13723098954536012946', time=2024-06-28T17:53:28.180358+0000, chain.objs.size()=1 2024-06-28T16:14:34.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.639+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.WpSRlrDgdpLU4L9-ArllvmRbpOyzFmk_0 2024-06-28T16:14:34.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16273026731033916344', time=2024-06-28T17:53:28.182186+0000, chain.objs.size()=1 2024-06-28T16:14:34.651 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1REyX2JcTaVvMIrMNt4zPiyRf29HQMF_0 2024-06-28T16:14:34.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13509843855155802933', time=2024-06-28T17:53:31.506805+0000, chain.objs.size()=1 2024-06-28T16:14:34.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.9Ew9HFH5r6SrWsvTpvdgbq2aKsk0Np3_0 2024-06-28T16:14:34.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1911395695524945391', time=2024-06-28T17:53:31.761051+0000, chain.objs.size()=1 2024-06-28T16:14:34.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Iy_6TUiaLLRvgXT4yvs3FCIBhamJeF3_0 2024-06-28T16:14:34.652 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16663262606280306530', time=2024-06-28T17:53:32.636215+0000, chain.objs.size()=1 2024-06-28T16:14:34.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vVIUZA2VxoXAcw3dpOO2jFsFbI_PIsr_0 2024-06-28T16:14:34.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4903409933954929075', time=2024-06-28T17:53:34.249423+0000, chain.objs.size()=1 2024-06-28T16:14:34.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.e84WmbYsrU8h35NEwq5IXDBx2xUZfzc_0 2024-06-28T16:14:34.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4600044001855931587', time=2024-06-28T17:53:34.633891+0000, chain.objs.size()=1 2024-06-28T16:14:34.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1RH2Oovq7aCeY0v4XeX7RNr06HPItKm_0 2024-06-28T16:14:34.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14051717344463383586', time=2024-06-28T17:53:35.731121+0000, chain.objs.size()=1 2024-06-28T16:14:34.653 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.NV9Halaqv_DFCrqMuKF-L9PJpfzSkI5_0 2024-06-28T16:14:34.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8564708552189165291', time=2024-06-28T17:53:36.664818+0000, chain.objs.size()=1 2024-06-28T16:14:34.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.LsPNiJqEYHLVs-bah5lx3WS6RqA9jGh_0 2024-06-28T16:14:34.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17807825945170428507', time=2024-06-28T17:53:36.803863+0000, chain.objs.size()=1 2024-06-28T16:14:34.654 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.643+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.k0N9NSSmOb0UhGlBMVFB9qvAnnnVnpl_0 2024-06-28T16:14:34.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.647+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1964401025416804416', time=2024-06-28T17:53:36.869978+0000, chain.objs.size()=1 2024-06-28T16:14:34.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.647+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qVZ5XjlqXdIYy3nrtw89nshH5Cv2Mb9_0 2024-06-28T16:14:34.655 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.647+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5782025807404033128', time=2024-06-28T17:53:36.872136+0000, chain.objs.size()=1 2024-06-28T16:14:34.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.647+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.p0F2hJMqU5kFAkshq2jQSp3ydYEx08x_0 2024-06-28T16:14:34.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.647+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7151050878004013826', time=2024-06-28T17:53:38.866841+0000, chain.objs.size()=1 2024-06-28T16:14:34.656 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.647+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.joO6FYc-fZXGCiACaCJ8aCK0BzXgb-H_0 2024-06-28T16:14:34.659 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16564560528346252589', time=2024-06-28T17:53:38.868268+0000, chain.objs.size()=1 2024-06-28T16:14:34.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.w5Q7dhVYpetbVoI9UeysASyXBeszCU9_0 2024-06-28T16:14:34.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.217708607294931645', time=2024-06-28T17:53:39.581478+0000, chain.objs.size()=1 2024-06-28T16:14:34.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_._9idesVxX3CJCM_Xp-YG5aSd5r3sXAY_0 2024-06-28T16:14:34.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1471935456994516378', time=2024-06-28T17:53:40.719304+0000, chain.objs.size()=1 2024-06-28T16:14:34.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.9hSeQAnBPcclxu5MzEgE9RUro06eI2F_0 2024-06-28T16:14:34.661 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.651+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9858341090303537575', time=2024-06-28T17:53:40.789741+0000, chain.objs.size()=1 2024-06-28T16:14:34.662 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.651+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.y6GpCwpFRues4KyjpDSEEG-twqg0EdH_0 2024-06-28T16:14:34.663 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/48931 2024-06-28T16:14:34.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=32, truncated=0, next_marker='' 2024-06-28T16:14:34.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12821305851731424927', time=2024-06-28T17:53:42.338603+0000, chain.objs.size()=1 2024-06-28T16:14:34.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.yG4M0WsUsbkUGbECDtdzZcsk0PXUhjf_0 2024-06-28T16:14:34.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17457497528262170680', time=2024-06-28T17:53:42.791073+0000, chain.objs.size()=1 2024-06-28T16:14:34.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.aoqg0NBOE4hjUdO6PpX9X6OBmVtIoWh_0 2024-06-28T16:14:34.666 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18173650454763418392', time=2024-06-28T17:53:42.793355+0000, chain.objs.size()=1 2024-06-28T16:14:34.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.U4gEXsdmomInCKRy4UnmjmkQNzll4JP_0 2024-06-28T16:14:34.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6786883749130737493', time=2024-06-28T17:53:43.337734+0000, chain.objs.size()=1 2024-06-28T16:14:34.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_._6QyUNjQojaLknTURENfNu2lXnAcZ91_0 2024-06-28T16:14:34.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7356158090079997387', time=2024-06-28T17:53:43.339257+0000, chain.objs.size()=1 2024-06-28T16:14:34.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.pfiwwPInoZ9kA8NkoUVZez4gU8ZiZxc_0 2024-06-28T16:14:34.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8284486520238654822', time=2024-06-28T17:53:45.796610+0000, chain.objs.size()=1 2024-06-28T16:14:34.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.gFR4xBsW5zq1PHOqQ6ltYOGoqC3tK3e_0 2024-06-28T16:14:34.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7665109415305013281', time=2024-06-28T17:53:46.801678+0000, chain.objs.size()=1 2024-06-28T16:14:34.667 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.K0mrH5dWFFnu1D6xCEg4ankf5GHdqcM_0 2024-06-28T16:14:34.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4101997470601042875', time=2024-06-28T17:53:46.803124+0000, chain.objs.size()=1 2024-06-28T16:14:34.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.YBY1kxnE-WcGcCOeI84UOSa3d-fV9NX_0 2024-06-28T16:14:34.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.270434422822795484', time=2024-06-28T17:53:50.960647+0000, chain.objs.size()=1 2024-06-28T16:14:34.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.7V65JYPG1mR9KVcKHoazIVHffABZAeT_0 2024-06-28T16:14:34.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1807943316523563406', time=2024-06-28T17:53:51.827442+0000, chain.objs.size()=1 2024-06-28T16:14:34.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.vBJwlWHJRwegoHd-LgkLM52ZAxzfgjx_0 2024-06-28T16:14:34.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1788565101369689844', time=2024-06-28T17:54:08.551465+0000, chain.objs.size()=1 2024-06-28T16:14:34.668 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.331__shadow_.9zeAmAc8ljLc5qwYSpmsel-_uCaNGuO_0 2024-06-28T16:14:34.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4475511470641546838', time=2024-06-28T17:55:28.533645+0000, chain.objs.size()=1 2024-06-28T16:14:34.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.655+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.421__shadow_.ddLXdy0tlRa5v1f27FZ-ya4K9w90yne_0 2024-06-28T16:14:34.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7775643187788085181', time=2024-06-28T17:55:31.018749+0000, chain.objs.size()=1 2024-06-28T16:14:34.669 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.424__shadow_.P2I86f_H-oUDIEZrsAk5Z25lbQlUwTl_0 2024-06-28T16:14:34.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10049358382253978437', time=2024-06-28T17:55:32.148056+0000, chain.objs.size()=1 2024-06-28T16:14:34.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.425__shadow_.-qfX8k0z_cDrGNzXpLWo-ETwb76cdOg_0 2024-06-28T16:14:34.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7767982173196577883', time=2024-06-28T17:55:53.239195+0000, chain.objs.size()=1 2024-06-28T16:14:34.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.444__shadow_.y-G8VL7pVpL2ZuxZW23D7SewOXihK8p_0 2024-06-28T16:14:34.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.659+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16073679780898372139', time=2024-06-28T17:58:35.999902+0000, chain.objs.size()=6 2024-06-28T16:14:34.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.gVZ6mMY99c5SpEKi-bUXxo0duv9u4yCy.1 2024-06-28T16:14:34.670 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~72uj7Oz_zf5a3iRxeLnWsnMGff6IikF.2 2024-06-28T16:14:34.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.659+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~72uj7Oz_zf5a3iRxeLnWsnMGff6IikF.3 2024-06-28T16:14:34.671 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.663+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~72uj7Oz_zf5a3iRxeLnWsnMGff6IikF.4 2024-06-28T16:14:34.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.663+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~72uj7Oz_zf5a3iRxeLnWsnMGff6IikF.5 2024-06-28T16:14:34.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.663+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~72uj7Oz_zf5a3iRxeLnWsnMGff6IikF.6 2024-06-28T16:14:34.672 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.663+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8634198993214433224', time=2024-06-28T18:01:17.698051+0000, chain.objs.size()=1 2024-06-28T16:14:34.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.663+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.542__shadow_.6TSub3_kju5ebPSTdKf1FK2gBGufd50_0 2024-06-28T16:14:34.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.663+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2402402155501860814', time=2024-06-28T18:01:21.106376+0000, chain.objs.size()=1 2024-06-28T16:14:34.673 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.663+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.543__shadow_.2KLnZF9_INcQ_G3-5GPf8evvMAUSVTk_0 2024-06-28T16:14:34.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.631811574942377935', time=2024-06-28T18:01:31.561241+0000, chain.objs.size()=1 2024-06-28T16:14:34.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.546__shadow_.4RMg3MuaGJaB4h4stVBHPOJL_NLgJVN_0 2024-06-28T16:14:34.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17421755729025506674', time=2024-06-28T18:02:27.575054+0000, chain.objs.size()=1 2024-06-28T16:14:34.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.562__shadow_.eykBEFodvO1f13ZIuoofYejutU7RjoH_0 2024-06-28T16:14:34.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16681908654777201113', time=2024-06-28T18:02:32.556122+0000, chain.objs.size()=1 2024-06-28T16:14:34.676 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.563__shadow:hWEORZW4jSIFdspki721squby2KGZcz_.RI6OIgj6rcz68orI1a-KBzshfwlQkb1_0 2024-06-28T16:14:34.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2817500695769383316', time=2024-06-28T18:02:34.546547+0000, chain.objs.size()=1 2024-06-28T16:14:34.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.563__shadow:sgH39E7sJrigFdiz.fNpyvCjxGq904J_.TnvTS3BKF1t-xfS3MtMUMaLpfutpqem_0 2024-06-28T16:14:34.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.540242741888468400', time=2024-06-28T18:03:24.138314+0000, chain.objs.size()=1 2024-06-28T16:14:34.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:gE1LzD9UJD7c9VSbtlWvmg2upcKQ6Ep_.vTxtC7vrzzcADNPY6sg7Z28Rm4KZBjc_0 2024-06-28T16:14:34.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13892985497015311524', time=2024-06-28T18:04:10.451424+0000, chain.objs.size()=1 2024-06-28T16:14:34.677 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.575__shadow:ajsOdlQ1CbPbAQy10ivTipEjTmfkrET_.N9ZdQlJTUBb5ustCdNizx2U5iEoThUC_0 2024-06-28T16:14:34.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7519699685702954245', time=2024-06-28T18:04:13.553553+0000, chain.objs.size()=1 2024-06-28T16:14:34.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.576__shadow:BT5mWO39zstafQc5XLjp9eJVm5sPnj8_.5iS-AcThRCyUFCzzQx_hvNkFpe1ybtg_0 2024-06-28T16:14:34.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1796216379194143240', time=2024-06-28T18:04:26.245505+0000, chain.objs.size()=1 2024-06-28T16:14:34.678 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.591__shadow_.m5hHWFDiHjDCIu9gz5LNRyGphjDsuaR_0 2024-06-28T16:14:34.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16009951440073931330', time=2024-06-28T18:08:05.144909+0000, chain.objs.size()=1 2024-06-28T16:14:34.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.651__shadow_.mQzRm61yMKM4sYHFauC6YJtvZibLs3V_0 2024-06-28T16:14:34.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1889684330651259178', time=2024-06-28T18:08:16.932283+0000, chain.objs.size()=1 2024-06-28T16:14:34.679 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.667+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.664__shadow_.ug7amQYB5MhxZdN_uvspqmgCFA_Gdsl_0 2024-06-28T16:14:34.680 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6152305130592003263', time=2024-06-28T18:08:16.953796+0000, chain.objs.size()=1 2024-06-28T16:14:34.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.664__shadow_.tji5aklUeH58QiXttR4bSMCFDUqnaNv_0 2024-06-28T16:14:34.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.453813714944083849', time=2024-06-28T18:08:47.747618+0000, chain.objs.size()=1 2024-06-28T16:14:34.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.693__shadow:7G7s7fX4CL3eyz.1OAREuM6yJ7.jCd3_.IDO2eF-SWts0vLlisSiQ8b48gqxLVT2_0 2024-06-28T16:14:34.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.671+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14256643792836219519', time=2024-06-28T18:08:51.469187+0000, chain.objs.size()=1 2024-06-28T16:14:34.681 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.671+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.696__multipart_file1.2~vl9Xj8Kf-nTVo_H_uoh2CaJzVPXudDk.1 2024-06-28T16:14:34.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.675+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15414581579863410032', time=2024-06-28T18:09:06.782958+0000, chain.objs.size()=1 2024-06-28T16:14:34.683 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.675+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.701__shadow:8Udu0bVjkf597tfXX9xXV9R3j50GyME_.oNIQ4SrcMtTKia6KjdW42B7x3LynC04_0 2024-06-28T16:14:34.687 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.679+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:34.689 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.679+0000 7f00c3c02980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-06-28T16:14:34.690 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-06-28T16:14:34.691 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-06-28T16:14:34.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=100, truncated=1, next_marker='0/43414' 2024-06-28T16:14:34.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='_eqEuzyV0jQ6_0t6A1ObzwJrOp5yanSm', time=2024-06-28T17:40:34.926268+0000, chain.objs.size()=2 2024-06-28T16:14:34.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.151__shadow_.iHsdO8t51qn26N_id9E_8veZB793lox_0 2024-06-28T16:14:34.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:843e24ce-085a-4043-8f55-38e05370e790.4686.151__shadow_.iHsdO8t51qn26N_id9E_8veZB793lox_1 2024-06-28T16:14:34.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.868082858324100538', time=2024-06-28T17:40:42.927178+0000, chain.objs.size()=1 2024-06-28T16:14:34.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.156__shadow_.9--j3tfG-o5vgng_IgTlp_QRHydCKuR_0 2024-06-28T16:14:34.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8843889933337607431', time=2024-06-28T17:44:46.086899+0000, chain.objs.size()=1 2024-06-28T16:14:34.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.227__shadow_.yLvE-ZMktajlO6XwVVTlJgatIaEoJ9F_0 2024-06-28T16:14:34.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9620074422515768326', time=2024-06-28T17:44:46.088426+0000, chain.objs.size()=1 2024-06-28T16:14:34.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.227__shadow_.Qe7NAvBQTPHAGEQUG9JAnVfCc2Gke5w_0 2024-06-28T16:14:34.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14188416002619866049', time=2024-06-28T17:44:54.063102+0000, chain.objs.size()=1 2024-06-28T16:14:34.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.232__shadow_.Du98JgkJLLeAP1dJOXrL2c-Oc6sFAR9_0 2024-06-28T16:14:34.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11420587505584022885', time=2024-06-28T17:44:58.705851+0000, chain.objs.size()=1 2024-06-28T16:14:34.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.234__shadow_.f3szNImy_GNKLTdqtAFK5DsviAqJdOn_0 2024-06-28T16:14:34.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2270996409499699970', time=2024-06-28T17:45:06.608381+0000, chain.objs.size()=1 2024-06-28T16:14:34.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.239__shadow_.psGTAynL1-d8aE1tW1J31l5S_nqzfUb_0 2024-06-28T16:14:34.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14645607950526765870', time=2024-06-28T17:47:08.029246+0000, chain.objs.size()=1 2024-06-28T16:14:34.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.wJWjQhALbt5BazVPrDhrRYNEVB8DFdh_0 2024-06-28T16:14:34.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10078682595431511611', time=2024-06-28T17:47:08.249168+0000, chain.objs.size()=1 2024-06-28T16:14:34.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.7K7lT3T7Fnn486oVYZFqRKWJqBQgu7v_0 2024-06-28T16:14:34.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10971132997514751726', time=2024-06-28T17:47:08.935579+0000, chain.objs.size()=1 2024-06-28T16:14:34.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.kBDhPGR5w9SS7gg9AtP852mrwjcY_6A_0 2024-06-28T16:14:34.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14742612190980072192', time=2024-06-28T17:47:15.631432+0000, chain.objs.size()=1 2024-06-28T16:14:34.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.683+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.t-BuXiqFExKYfebH2-M3ucvM82YMagE_0 2024-06-28T16:14:34.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14912104379361134149', time=2024-06-28T17:47:16.756368+0000, chain.objs.size()=1 2024-06-28T16:14:34.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.zzc2AyRbQLDQV0kcrRCAztlC9NEWTfw_0 2024-06-28T16:14:34.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16330657141241672177', time=2024-06-28T17:47:18.714309+0000, chain.objs.size()=1 2024-06-28T16:14:34.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.9jUHAjjNDTZX0nPelJFw56VpIxHF8AM_0 2024-06-28T16:14:34.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8473385341647084890', time=2024-06-28T17:47:18.856622+0000, chain.objs.size()=1 2024-06-28T16:14:34.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.jkJzfSkRlFgdZDkIvRf5p2xzMYx12c5_0 2024-06-28T16:14:34.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8967119679107094794', time=2024-06-28T17:47:19.636940+0000, chain.objs.size()=1 2024-06-28T16:14:34.698 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.HI3mC6WYIAXLt-hSLWl63K2SlxWViSD_0 2024-06-28T16:14:34.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9606412484198038968', time=2024-06-28T17:47:20.901186+0000, chain.objs.size()=1 2024-06-28T16:14:34.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.wHQ-sVskvdXWcKewc7HJrxLUCGH3vYL_0 2024-06-28T16:14:34.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.687+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4614683196762373521', time=2024-06-28T17:47:21.688942+0000, chain.objs.size()=1 2024-06-28T16:14:34.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.687+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.izCHQDaU73Jeab7l2k0t8pyWMBAICmo_0 2024-06-28T16:14:34.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7488898333666228262', time=2024-06-28T17:47:22.157288+0000, chain.objs.size()=1 2024-06-28T16:14:34.699 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.NpMYQht1KjLMA-YbhKYtdeAk5G_SHGs_0 2024-06-28T16:14:34.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7149566212155750058', time=2024-06-28T17:47:22.331663+0000, chain.objs.size()=1 2024-06-28T16:14:34.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.oPUryA3ppqZlnGLdToXhdpUK2gI-Ifi_0 2024-06-28T16:14:34.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14843660518386685278', time=2024-06-28T17:47:25.000578+0000, chain.objs.size()=1 2024-06-28T16:14:34.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.mUXziNszLRi5FiwES_ZPaCp1pzXVZL9_0 2024-06-28T16:14:34.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12748585209244567218', time=2024-06-28T17:47:26.912983+0000, chain.objs.size()=1 2024-06-28T16:14:34.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.VDNnHWI9I7gu5g5kL8dDfiYDqUmNqYc_0 2024-06-28T16:14:34.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5294750670266949096', time=2024-06-28T17:47:27.126317+0000, chain.objs.size()=1 2024-06-28T16:14:34.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.g9VMgYCQHVM1XCXHk25CXd4LKpqW49Y_0 2024-06-28T16:14:34.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11726898481149407672', time=2024-06-28T17:47:28.948028+0000, chain.objs.size()=1 2024-06-28T16:14:34.701 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.dTno3LnHYQs30dCww407Mnw3c-VWlCa_0 2024-06-28T16:14:34.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10502105560425516689', time=2024-06-28T17:47:29.660036+0000, chain.objs.size()=1 2024-06-28T16:14:34.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.CBv86lfVrz0xDXkmomMtnTNIKQB0pGH_0 2024-06-28T16:14:34.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4309177716404435850', time=2024-06-28T17:47:31.650500+0000, chain.objs.size()=1 2024-06-28T16:14:34.702 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.HdGL-bMgiSsyeYz5YoPb_JQ_QWRBIIl_0 2024-06-28T16:14:34.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13800129508024784730', time=2024-06-28T17:47:31.942898+0000, chain.objs.size()=1 2024-06-28T16:14:34.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.mtg7yBQ6IzX_uQgGLUDFpmdbAJHWtxe_0 2024-06-28T16:14:34.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18168801974270380094', time=2024-06-28T17:47:33.042694+0000, chain.objs.size()=1 2024-06-28T16:14:34.703 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.ev33WmVFj6Z6SHW6CxG0CjJWiqRlxIK_0 2024-06-28T16:14:34.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12017346918847744581', time=2024-06-28T17:47:34.042477+0000, chain.objs.size()=1 2024-06-28T16:14:34.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.691+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.zIeBadivTwSDVCzSu1OcEeYYHC_OVqr_0 2024-06-28T16:14:34.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.695+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15358813917991205400', time=2024-06-28T17:47:34.293530+0000, chain.objs.size()=1 2024-06-28T16:14:34.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.NxyJ1aqf6ILUUdGQ4o9-0cNKoE9VLJ2_0 2024-06-28T16:14:34.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.695+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5248302176806128394', time=2024-06-28T17:47:36.231681+0000, chain.objs.size()=1 2024-06-28T16:14:34.704 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.FaNj8LFATH9AeWN8cDb2H8lf8G_cHOL_0 2024-06-28T16:14:34.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.695+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2845181809105215388', time=2024-06-28T17:47:37.742040+0000, chain.objs.size()=1 2024-06-28T16:14:34.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.IbtY0bBhlVbSOZ8DwztrzPkd4PpXf6v_0 2024-06-28T16:14:34.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.695+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6978824655256058637', time=2024-06-28T17:47:39.096900+0000, chain.objs.size()=1 2024-06-28T16:14:34.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.252__shadow_.Za_wXlaDxeEE8h7zpkBLEZMwfvbKvVN_0 2024-06-28T16:14:34.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.695+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2441605154233858800', time=2024-06-28T17:47:49.962897+0000, chain.objs.size()=1 2024-06-28T16:14:34.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.259__shadow_.BDOsfFxbDEXzLsSDf_AIJBLIWME2wyq_0 2024-06-28T16:14:34.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.695+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15819850224571780720', time=2024-06-28T17:47:58.946864+0000, chain.objs.size()=1 2024-06-28T16:14:34.705 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.695+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.265__shadow_.YdDvY_dlzJFhL3iIpcjIKQin_ODNKK2_0 2024-06-28T16:14:34.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4676383964296494473', time=2024-06-28T17:48:29.434737+0000, chain.objs.size()=1 2024-06-28T16:14:34.709 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.282__shadow_.S2F20kyv4RiwM2xaVkJHYOdg00de783_0 2024-06-28T16:14:34.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4104746984151201331', time=2024-06-28T17:51:06.833309+0000, chain.objs.size()=1 2024-06-28T16:14:34.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.9pE3fQkr6uyr7L1_gRrE-a7olIAcoq0_0 2024-06-28T16:14:34.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5857150836495730404', time=2024-06-28T17:51:07.746871+0000, chain.objs.size()=1 2024-06-28T16:14:34.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.OeK0gcxzn7iQlRBwiv-12BQ6B-J85K4_0 2024-06-28T16:14:34.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14759493296902063812', time=2024-06-28T17:51:07.770884+0000, chain.objs.size()=1 2024-06-28T16:14:34.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Klj7F23Pkvce2xyJpFHooj_nPCryR_C_0 2024-06-28T16:14:34.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6530694931626325396', time=2024-06-28T17:51:08.051477+0000, chain.objs.size()=1 2024-06-28T16:14:34.710 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ZZkKTRq0FmhKbDZS0QVNYU6suDiG-TT_0 2024-06-28T16:14:34.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15062255169336071970', time=2024-06-28T17:51:08.673019+0000, chain.objs.size()=1 2024-06-28T16:14:34.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.bS7mz0jjGTGzpf6yg3d30fZEzdslE0z_0 2024-06-28T16:14:34.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6911170793125821647', time=2024-06-28T17:51:09.567264+0000, chain.objs.size()=1 2024-06-28T16:14:34.711 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.M0xskLNa2j2vfw9vdUn0y6oBswVEFVs_0 2024-06-28T16:14:34.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2250358146307444175', time=2024-06-28T17:51:10.011368+0000, chain.objs.size()=1 2024-06-28T16:14:34.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.NL-konq0XfGo9EodL5tmxNM0-URtRvZ_0 2024-06-28T16:14:34.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8431923758706020125', time=2024-06-28T17:51:11.065424+0000, chain.objs.size()=1 2024-06-28T16:14:34.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.ANBgY7-6hfq8W63FPxjyFCQgKElPns6_0 2024-06-28T16:14:34.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8707137132399687126', time=2024-06-28T17:51:11.078226+0000, chain.objs.size()=1 2024-06-28T16:14:34.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.aYDainZ8n1tBGAWlNt1feheUNfBv7t1_0 2024-06-28T16:14:34.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15198674336751081036', time=2024-06-28T17:51:12.679352+0000, chain.objs.size()=1 2024-06-28T16:14:34.712 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.699+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.sqU6fLQrWaludiauCA2ueH6QL57E7ay_0 2024-06-28T16:14:34.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8070962637554230070', time=2024-06-28T17:51:14.052612+0000, chain.objs.size()=1 2024-06-28T16:14:34.714 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.i8fK3oNGESTjNHnYLmvA9gMXhRSe9bn_0 2024-06-28T16:14:34.715 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.3797579868875169522', time=2024-06-28T17:51:15.447934+0000, chain.objs.size()=1 2024-06-28T16:14:34.717 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.mrTPwKfBfb4nnDg5yuSb3_ahsmQWHYp_0 2024-06-28T16:14:34.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.703+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2751802360436751680', time=2024-06-28T17:51:16.588533+0000, chain.objs.size()=1 2024-06-28T16:14:34.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.703+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.xoBiE-JlKxut1VkJyfdmD0SlUIBK3IR_0 2024-06-28T16:14:34.718 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10809573630710049630', time=2024-06-28T17:51:17.173041+0000, chain.objs.size()=1 2024-06-28T16:14:34.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.1MzeVuXYadgUh8zL78g4-1fjxBHft_R_0 2024-06-28T16:14:34.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2044160242373819673', time=2024-06-28T17:51:17.373435+0000, chain.objs.size()=1 2024-06-28T16:14:34.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.mVcy-e9bccOujSKbNS84r4udiHW-R2z_0 2024-06-28T16:14:34.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16909828566596740090', time=2024-06-28T17:51:17.573215+0000, chain.objs.size()=1 2024-06-28T16:14:34.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.mZ2j2FNDBeLZ4LV2LfLC7BYZFc0Vbon_0 2024-06-28T16:14:34.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14940234496386697173', time=2024-06-28T17:51:18.193067+0000, chain.objs.size()=1 2024-06-28T16:14:34.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.GSs7iZx-dBIlX5Qrtiw1o1HOwENrRJr_0 2024-06-28T16:14:34.719 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6481134630551406067', time=2024-06-28T17:51:18.486715+0000, chain.objs.size()=1 2024-06-28T16:14:34.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.7s9OXoH6_9wxlIy5tNcacVMGIkg1ceO_0 2024-06-28T16:14:34.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12387455313884658634', time=2024-06-28T17:51:20.648882+0000, chain.objs.size()=1 2024-06-28T16:14:34.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.Z7zqHGI_VEnnKTBW0a6CAaX4-yERBPk_0 2024-06-28T16:14:34.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5027447582279888174', time=2024-06-28T17:51:20.774220+0000, chain.objs.size()=1 2024-06-28T16:14:34.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fNuZ9AOH2BoHKYJ724S5akUe9wDea5j_0 2024-06-28T16:14:34.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14644628969495206342', time=2024-06-28T17:51:22.652745+0000, chain.objs.size()=1 2024-06-28T16:14:34.720 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.707+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.aoZzV03i5uT9t9apPpCLqrkvSBDepQP_0 2024-06-28T16:14:34.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9094598314653107431', time=2024-06-28T17:51:25.400339+0000, chain.objs.size()=1 2024-06-28T16:14:34.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.H_w4Mvzzcl7iOBsebkRLI8oqo-PF_Ma_0 2024-06-28T16:14:34.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2247318219828508364', time=2024-06-28T17:51:25.406253+0000, chain.objs.size()=1 2024-06-28T16:14:34.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.CP0AoWKOpNMgobXuplHXodsRsTvTqeF_0 2024-06-28T16:14:34.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12695675221930336840', time=2024-06-28T17:51:28.116296+0000, chain.objs.size()=1 2024-06-28T16:14:34.721 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.8GJ5-KsY5rhoApZsBM5ETnJx9HaVfMh_0 2024-06-28T16:14:34.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14216933874474756818', time=2024-06-28T17:51:28.284584+0000, chain.objs.size()=1 2024-06-28T16:14:34.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.gQaQjs0Wj1qzzgF5dw6197SEJ3WwF7V_0 2024-06-28T16:14:34.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11033549990810005844', time=2024-06-28T17:51:30.053550+0000, chain.objs.size()=1 2024-06-28T16:14:34.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.237hJHwCjnRK_VMZ0WfoPfmxpuHjxVM_0 2024-06-28T16:14:34.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8106679046893038257', time=2024-06-28T17:51:31.881303+0000, chain.objs.size()=1 2024-06-28T16:14:34.722 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.tJMEZlxg03LAWb-DWDX1f00rF1NsdTu_0 2024-06-28T16:14:34.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17519124202735662226', time=2024-06-28T17:51:33.265827+0000, chain.objs.size()=1 2024-06-28T16:14:34.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.TTQDCRC1bvBdTjHg8-aOx66PdvRlZio_0 2024-06-28T16:14:34.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12498409244831393596', time=2024-06-28T17:51:34.038513+0000, chain.objs.size()=1 2024-06-28T16:14:34.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.711+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.fKAr1Unt7GchLroCzdjxNFua0Sc3WBU_0 2024-06-28T16:14:34.723 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.8336547972141342251', time=2024-06-28T17:51:35.996787+0000, chain.objs.size()=1 2024-06-28T16:14:34.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.8fI1nKYCbUs_I3SXnAVsOKLKwic-HMU_0 2024-06-28T16:14:34.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15192891914753467173', time=2024-06-28T17:51:36.312946+0000, chain.objs.size()=1 2024-06-28T16:14:34.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.pi8QBxPjBkvdUox_Uc-UG6K2O3try-P_0 2024-06-28T16:14:34.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.7252450145639688878', time=2024-06-28T17:51:36.556266+0000, chain.objs.size()=1 2024-06-28T16:14:34.724 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.gyWF2icmMHx248RGfEjeEn77PqmR1Si_0 2024-06-28T16:14:34.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10672735711444191023', time=2024-06-28T17:51:36.877383+0000, chain.objs.size()=1 2024-06-28T16:14:34.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.LP_p4_L-GsWIydpawsrmF8b5S-AefHX_0 2024-06-28T16:14:34.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10865015055050919961', time=2024-06-28T17:51:37.066877+0000, chain.objs.size()=1 2024-06-28T16:14:34.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.313__shadow_.MCpGV6N52ghxSz_kJe4ejtQeI4iVL_D_0 2024-06-28T16:14:34.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.871859076808402347', time=2024-06-28T17:53:20.981586+0000, chain.objs.size()=1 2024-06-28T16:14:34.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.cdkyjmQ4V_2eqg1xvLn7BR13GMOYh5__0 2024-06-28T16:14:34.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14901898007645050552', time=2024-06-28T17:53:21.402084+0000, chain.objs.size()=1 2024-06-28T16:14:34.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.CcQbYmM5Bjufsk4PusJLtyd8i8BkplX_0 2024-06-28T16:14:34.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10075711500637095356', time=2024-06-28T17:53:21.994498+0000, chain.objs.size()=1 2024-06-28T16:14:34.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.zt9akX8utqBtRLl3ZyaQIk3DErxCfd6_0 2024-06-28T16:14:34.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17552288683598507934', time=2024-06-28T17:53:22.650842+0000, chain.objs.size()=1 2024-06-28T16:14:34.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.FOem_FuzOi_wAWVt3Ye3ICt86J3Picy_0 2024-06-28T16:14:34.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6090784014355795285', time=2024-06-28T17:53:22.663257+0000, chain.objs.size()=1 2024-06-28T16:14:34.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.715+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.0PjLswl6aZhQFaW7JWsD38we8ITxxDt_0 2024-06-28T16:14:34.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17022021034069819584', time=2024-06-28T17:53:22.667796+0000, chain.objs.size()=1 2024-06-28T16:14:34.727 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.I7TV1z8GUeoec5Ak-21AeE0BYUbZwC1_0 2024-06-28T16:14:34.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1735008853859101846', time=2024-06-28T17:53:25.300689+0000, chain.objs.size()=1 2024-06-28T16:14:34.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.SbbegAjQOxtEmYZ20Y-CSi9FT7wPWi0_0 2024-06-28T16:14:34.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10209880660959846925', time=2024-06-28T17:53:27.270933+0000, chain.objs.size()=1 2024-06-28T16:14:34.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.HbNnzD6NO9Z_MnLceG3hprgPvjRjNEO_0 2024-06-28T16:14:34.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9695261789250615286', time=2024-06-28T17:53:27.406046+0000, chain.objs.size()=1 2024-06-28T16:14:34.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.oHpQX_O3mfCgcK3-nqSC_5LBVyO828G_0 2024-06-28T16:14:34.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4675980760348924652', time=2024-06-28T17:53:27.467164+0000, chain.objs.size()=1 2024-06-28T16:14:34.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.2QS-d0Qe-1DmU9EvLHhqfHURKCJRwGU_0 2024-06-28T16:14:34.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18253382185862152508', time=2024-06-28T17:53:28.434394+0000, chain.objs.size()=1 2024-06-28T16:14:34.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Wu4mHo4J7Tujxm1WobcH19SOgAN23Ri_0 2024-06-28T16:14:34.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16943110251591825985', time=2024-06-28T17:53:29.476808+0000, chain.objs.size()=1 2024-06-28T16:14:34.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.3nBGdBvSMGevwwRGXBMJKRQi3EgY2w-_0 2024-06-28T16:14:34.730 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18105292946929701910', time=2024-06-28T17:53:29.560115+0000, chain.objs.size()=1 2024-06-28T16:14:34.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.3VeAVAwKpTeMQhEKzJ_0lDVQ2usQ_c8_0 2024-06-28T16:14:34.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9125977045809161059', time=2024-06-28T17:53:29.591651+0000, chain.objs.size()=1 2024-06-28T16:14:34.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.rFMQZlcTXUSzTzaYQr5Pe-Ta8LfVQGp_0 2024-06-28T16:14:34.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11520561715541491284', time=2024-06-28T17:53:30.827037+0000, chain.objs.size()=1 2024-06-28T16:14:34.731 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.1frMjEjyXkjF25dyhNeTcXXu-VscyWy_0 2024-06-28T16:14:34.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15299928877888710253', time=2024-06-28T17:53:31.313343+0000, chain.objs.size()=1 2024-06-28T16:14:34.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.719+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.hT_z-H9-qYwn8ZO75Kil3IjZ4owTo4B_0 2024-06-28T16:14:34.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.723+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.6028102300827200310', time=2024-06-28T17:53:32.456213+0000, chain.objs.size()=1 2024-06-28T16:14:34.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.723+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.I1Dd8nwI3flAcevBQH0myQPZH_w7x6p_0 2024-06-28T16:14:34.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.723+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15874705656169006423', time=2024-06-28T17:53:32.586454+0000, chain.objs.size()=1 2024-06-28T16:14:34.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.723+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.LJULo16ZMYR_b7bPrdt3YzVRLKrHxF1_0 2024-06-28T16:14:34.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.723+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16905098756978734572', time=2024-06-28T17:53:33.659618+0000, chain.objs.size()=1 2024-06-28T16:14:34.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.723+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.YnRRmuX6MLkpMtq1FAdGTiVUp9bA0gw_0 2024-06-28T16:14:34.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.723+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1018842816867588569', time=2024-06-28T17:53:34.707135+0000, chain.objs.size()=1 2024-06-28T16:14:34.733 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.723+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.4ZfTlVVaaKW-AJcYdY28qGgHf63I4CK_0 2024-06-28T16:14:34.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.723+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16556819652507899127', time=2024-06-28T17:53:38.734570+0000, chain.objs.size()=1 2024-06-28T16:14:34.734 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.723+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.r8GzoXuoKjzZ7XUDwEf6q4kmRffopnd_0 2024-06-28T16:14:34.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17270161876728581054', time=2024-06-28T17:53:38.859140+0000, chain.objs.size()=1 2024-06-28T16:14:34.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.ouS0EFBFKuF4HeJyc4ukErWH-dkaU-J_0 2024-06-28T16:14:34.736 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1461840076691845633', time=2024-06-28T17:53:40.661434+0000, chain.objs.size()=1 2024-06-28T16:14:34.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Ew4BZVaYBFCzcIMJkFk2uGUu01aMRbD_0 2024-06-28T16:14:34.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10722252102577347861', time=2024-06-28T17:53:42.913487+0000, chain.objs.size()=1 2024-06-28T16:14:34.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.AGx_Dso2m0GFMn8wZ_i-g2n-wNt5iw2_0 2024-06-28T16:14:34.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.781828479001573471', time=2024-06-28T17:53:43.603083+0000, chain.objs.size()=1 2024-06-28T16:14:34.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.G0-unkpidhYVe5TqWs8AzBLGEdPdamF_0 2024-06-28T16:14:34.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9186616630289365743', time=2024-06-28T17:53:43.610367+0000, chain.objs.size()=1 2024-06-28T16:14:34.737 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.Du2wWL-mzMs0bscSqHwSM3kGqFbpwV2_0 2024-06-28T16:14:34.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.1933647725850231567', time=2024-06-28T17:53:45.030784+0000, chain.objs.size()=1 2024-06-28T16:14:34.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.HAEu218WlLfyWZipDNUYJmQioeMdG5K_0 2024-06-28T16:14:34.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9628541084787271710', time=2024-06-28T17:53:45.036351+0000, chain.objs.size()=1 2024-06-28T16:14:34.738 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.eMD2NVtbslv4-4WabHHDLu4nOyV2D8l_0 2024-06-28T16:14:34.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2534578295816581296', time=2024-06-28T17:53:48.249304+0000, chain.objs.size()=1 2024-06-28T16:14:34.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.qpsUCKBxfKFk4UE2HjC0N8y9PtKymPO_0 2024-06-28T16:14:34.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.16580351906638306665', time=2024-06-28T17:53:51.175318+0000, chain.objs.size()=1 2024-06-28T16:14:34.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.H9ZttfQvKo6GHalj7fbmoMnysYgSwEG_0 2024-06-28T16:14:34.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11921484291782837542', time=2024-06-28T17:53:51.675893+0000, chain.objs.size()=1 2024-06-28T16:14:34.739 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.727+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.TNT4736vTu_vkDvfI3H87KaEFbD7Rh8_0 2024-06-28T16:14:34.742 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.731+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 0/43414 2024-06-28T16:14:34.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=23, truncated=0, next_marker='' 2024-06-28T16:14:34.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5522210596079440479', time=2024-06-28T17:53:51.802871+0000, chain.objs.size()=1 2024-06-28T16:14:34.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.314__shadow_.mrAGBxdNDO2lHOsM6JDutHLZG0xahWO_0 2024-06-28T16:14:34.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9085674831790224175', time=2024-06-28T17:54:00.537540+0000, chain.objs.size()=1 2024-06-28T16:14:34.745 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.323__shadow_.u1nNtRt4gm-Fx_uaW3WXnSDSIloeGQr_0 2024-06-28T16:14:34.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.9427000727081426082', time=2024-06-28T17:56:13.786440+0000, chain.objs.size()=1 2024-06-28T16:14:34.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.456__shadow_.gcQkK1cuGutW88S2UTRFvQly3lisVh-_0 2024-06-28T16:14:34.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.2922328432845971369', time=2024-06-28T17:56:28.063982+0000, chain.objs.size()=1 2024-06-28T16:14:34.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.473__shadow_.isZANpsXac0KTEpU6V5ltAjPrPZAEyT_0 2024-06-28T16:14:34.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.689059382174409576', time=2024-06-28T17:56:28.116358+0000, chain.objs.size()=1 2024-06-28T16:14:34.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.473__shadow_.2DtD3nqV07xQXNnWO5-QK5zzLK354Ro_0 2024-06-28T16:14:34.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.17618728537952591364', time=2024-06-28T17:56:47.604946+0000, chain.objs.size()=1 2024-06-28T16:14:34.746 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.487__shadow_.GweOq1OxYOFRv_11Y02969l2p84muei_0 2024-06-28T16:14:34.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10886977372532559654', time=2024-06-28T17:58:03.280331+0000, chain.objs.size()=3 2024-06-28T16:14:34.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.511__multipart_mymultipart.2~JXE4wGFgGPPXwbkZCRooS7FQJMgrexH.1 2024-06-28T16:14:34.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.511__multipart_mymultipart.2~JXE4wGFgGPPXwbkZCRooS7FQJMgrexH.2 2024-06-28T16:14:34.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.511__multipart_mymultipart.2~JXE4wGFgGPPXwbkZCRooS7FQJMgrexH.3 2024-06-28T16:14:34.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~zdhOaZ9lQdy2CERMBS9Zz_HWTjHY2T9', time=2024-06-28T17:58:35.940300+0000, chain.objs.size()=1 2024-06-28T16:14:34.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.512__multipart_mymultipart.2~zdhOaZ9lQdy2CERMBS9Zz_HWTjHY2T9.2 2024-06-28T16:14:34.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='2~JZ5WK0E6QAirjm-LPZBTgBGxIdPdMnV', time=2024-06-28T18:00:29.717798+0000, chain.objs.size()=1 2024-06-28T16:14:34.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.525__multipart_mymultipart.2~JZ5WK0E6QAirjm-LPZBTgBGxIdPdMnV.3 2024-06-28T16:14:34.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.18243645083797574824', time=2024-06-28T18:01:38.129990+0000, chain.objs.size()=1 2024-06-28T16:14:34.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.735+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.547__shadow_.gPwHgizLwDmCm951p4FevRdz9_DA7v3_0 2024-06-28T16:14:34.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.14912235941681386288', time=2024-06-28T18:02:12.617962+0000, chain.objs.size()=1 2024-06-28T16:14:34.748 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.558__shadow:QeZ1p78jgsVfc4njWTqXHc2fVlVvWx-_.h0kWJ7njm-hqQ_nGktWdQMFkIbyKrFp_0 2024-06-28T16:14:34.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10715269324463960159', time=2024-06-28T18:03:07.451566+0000, chain.objs.size()=1 2024-06-28T16:14:34.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:i4HJ6qkcDaJ8dCjuxjJnuoIqXo8MTvU_.hmFv7YOey0ncV2sqA7tj8npUdYe9iQP_0 2024-06-28T16:14:34.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.11090697937123455726', time=2024-06-28T18:03:29.132922+0000, chain.objs.size()=1 2024-06-28T16:14:34.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:syryDFhFAKHoo5C6ir7M9MKhQ9m2z1v_.KXzWr4PXJO9CCHUkoyb8Jq_E65dPCEF_0 2024-06-28T16:14:34.749 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.4348559707151707739', time=2024-06-28T18:03:32.259734+0000, chain.objs.size()=1 2024-06-28T16:14:34.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.566__shadow:YsOs4BAKvAI4Ebm.6ukMslaHjMuy3pR_.WKtU31oy1_IL0e2-feyzDE0zdOZeIhj_0 2024-06-28T16:14:34.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.739+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:63C.NZOu-V2ScGWmT27Bj4CNweN3JuO_.wze7dUjVhkbvyWhLsfojeq6MpbpcSqX_1_cc901741c1fbe9f2bc8ef9e2ec07414a', time=2024-06-28T18:04:01.205488+0000, chain.objs.size()=1 2024-06-28T16:14:34.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.739+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.569__shadow:63C.NZOu-V2ScGWmT27Bj4CNweN3JuO_.wze7dUjVhkbvyWhLsfojeq6MpbpcSqX_0 2024-06-28T16:14:34.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.5361036542396034261', time=2024-06-28T18:05:13.508139+0000, chain.objs.size()=6 2024-06-28T16:14:34.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.602__multipart_multipart_enc.2~5sXKh6sK6pYY51qVs6OqecXZDqZD0ay.1 2024-06-28T16:14:34.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.602__multipart_multipart_enc.2~5sXKh6sK6pYY51qVs6OqecXZDqZD0ay.2 2024-06-28T16:14:34.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.602__multipart_multipart_enc.2~5sXKh6sK6pYY51qVs6OqecXZDqZD0ay.3 2024-06-28T16:14:34.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.602__multipart_multipart_enc.2~5sXKh6sK6pYY51qVs6OqecXZDqZD0ay.4 2024-06-28T16:14:34.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.602__multipart_multipart_enc.2~5sXKh6sK6pYY51qVs6OqecXZDqZD0ay.5 2024-06-28T16:14:34.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.602__multipart_multipart_enc.2~5sXKh6sK6pYY51qVs6OqecXZDqZD0ay.6 2024-06-28T16:14:34.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15787798896273813880', time=2024-06-28T18:06:50.569846+0000, chain.objs.size()=1 2024-06-28T16:14:34.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.615__shadow_._zk-YUdtT7151QXOOpZDb9gJozTF8fv_0 2024-06-28T16:14:34.753 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.13585918141725306570', time=2024-06-28T18:07:29.336434+0000, chain.objs.size()=1 2024-06-28T16:14:34.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.640__shadow_.tYu_v3eZHqfEcaV6dYIxpqapVwmq42D_0 2024-06-28T16:14:34.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.743+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.15747623511484147529', time=2024-06-28T18:08:25.615422+0000, chain.objs.size()=1 2024-06-28T16:14:34.754 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.743+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.675__shadow:U65IfNhPZynsu7zVVgmJBxzIl.4TQjY_.FqRB_VB73NiqqeqeKZB_V6BrATqywfM_0 2024-06-28T16:14:34.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.747+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10102426543714074530', time=2024-06-28T18:08:44.300699+0000, chain.objs.size()=1 2024-06-28T16:14:34.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.747+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.690__shadow:3k29JjkcNNOIMO6BmzV-pcADhvDEDt1_.6bWECn_c79soy0s09RDazyb3f2XjYHd_0 2024-06-28T16:14:34.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.747+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.10511768620969459033', time=2024-06-28T18:08:49.988754+0000, chain.objs.size()=1 2024-06-28T16:14:34.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.747+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.695__shadow:ZwCfpyQWAfRU3oXJf21kfdFJ-lWCuBp_.OwKhC7owNVp8Ui-ZWv6pEAsawXc72Oo_0 2024-06-28T16:14:34.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.747+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.703__shadow_.ntjfTUhMHFRYNd9bEtiv7Ab-Oima2We_1_5210168a6f1a172fa0927d5dfe8d5eb0', time=2024-06-28T18:09:18.399015+0000, chain.objs.size()=1 2024-06-28T16:14:34.755 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.747+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.703__shadow_.ntjfTUhMHFRYNd9bEtiv7Ab-Oima2We_0 2024-06-28T16:14:34.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.747+0000 7f00c3c02980 20 garbage collection: RGWGC::process iterating over entry tag='843e24ce-085a-4043-8f55-38e05370e790.4686.12012369652752523067', time=2024-06-28T18:09:46.912935+0000, chain.objs.size()=1 2024-06-28T16:14:34.756 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.747+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:843e24ce-085a-4043-8f55-38e05370e790.4686.723__shadow_.1oGqq_OquyqOmRtAyQA9_hBY-uy1lj5_0 2024-06-28T16:14:34.761 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.751+0000 7f00c3c02980 5 garbage collection: RGWGC::process removing entries, marker: 2024-06-28T16:14:34.764 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.755+0000 7eff777e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-06-28T16:14:34.766 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.759+0000 7f00c3c02980 20 remove_watcher() i=0 2024-06-28T16:14:34.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.759+0000 7f00c3c02980 2 removed watcher, disabling cache 2024-06-28T16:14:34.767 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.759+0000 7f00c3c02980 20 remove_watcher() i=1 2024-06-28T16:14:34.768 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.759+0000 7f00c3c02980 20 remove_watcher() i=2 2024-06-28T16:14:34.769 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.759+0000 7f00c3c02980 20 remove_watcher() i=3 2024-06-28T16:14:34.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.759+0000 7f00c3c02980 20 remove_watcher() i=4 2024-06-28T16:14:34.771 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.763+0000 7f00c3c02980 20 remove_watcher() i=5 2024-06-28T16:14:34.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.763+0000 7f00c3c02980 20 remove_watcher() i=6 2024-06-28T16:14:34.773 INFO:teuthology.orchestra.run.smithi178.stderr:2024-06-28T16:14:34.763+0000 7f00c3c02980 20 remove_watcher() i=7 2024-06-28T16:14:34.784 DEBUG:teuthology.run_tasks:Unwinding manager openssl_keys 2024-06-28T16:14:34.796 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-06-28T16:14:34.819 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-06-28T16:14:34.820 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-28T16:14:36.004 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:36.017 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-06-28T16:14:36.024 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":2487,"stamp":"2024-06-28T16:14:35.607276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":227680302,"num_objects":1411,"num_object_clones":0,"num_object_copies":4932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1411,"num_whiteouts":0,"num_read":160453,"num_read_kb":2265609,"num_write":146008,"num_write_kb":2275003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12397,"ondisk_log_size":12397,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2106280,"kb_used_data":424952,"kb_used_omap":12,"kb_used_meta":1680371,"kb_avail":752868440,"statfs":{"total":773094113280,"available":770937282560,"internally_reserved":0,"allocated":435150848,"data_stored":419457843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":1720700467},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-457593513,"num_objects":-3393,"num_object_clones":0,"num_object_copies":-13508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3393,"num_whiteouts":0,"num_read":3820,"num_read_kb":3315,"num_write":3571,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007566"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":26,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.063084+0000","last_change":"2024-06-28T15:37:33.329429+0000","last_active":"2024-06-28T16:10:05.063084+0000","last_peered":"2024-06-28T16:10:05.063084+0000","last_clean":"2024-06-28T16:10:05.063084+0000","last_became_active":"2024-06-28T15:37:33.328626+0000","last_became_peered":"2024-06-28T15:37:33.328626+0000","last_unstale":"2024-06-28T16:10:05.063084+0000","last_undegraded":"2024-06-28T16:10:05.063084+0000","last_fullsized":"2024-06-28T16:10:05.063084+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:48:45.377436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"257'3","reported_seq":49,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.078088+0000","last_change":"2024-06-28T15:21:11.079460+0000","last_active":"2024-06-28T16:11:10.078088+0000","last_peered":"2024-06-28T16:11:10.078088+0000","last_clean":"2024-06-28T16:11:10.078088+0000","last_became_active":"2024-06-28T15:21:11.078250+0000","last_became_peered":"2024-06-28T15:21:11.078250+0000","last_unstale":"2024-06-28T16:11:10.078088+0000","last_undegraded":"2024-06-28T16:11:10.078088+0000","last_fullsized":"2024-06-28T16:11:10.078088+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:32:28.636744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"260'4","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.900204+0000","last_change":"2024-06-28T15:21:07.047340+0000","last_active":"2024-06-28T16:14:32.900204+0000","last_peered":"2024-06-28T16:14:32.900204+0000","last_clean":"2024-06-28T16:14:32.900204+0000","last_became_active":"2024-06-28T15:21:07.046517+0000","last_became_peered":"2024-06-28T15:21:07.046517+0000","last_unstale":"2024-06-28T16:14:32.900204+0000","last_undegraded":"2024-06-28T16:14:32.900204+0000","last_fullsized":"2024-06-28T16:14:32.900204+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:20:57.841643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3076,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"249'7","reported_seq":82,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.563726+0000","last_change":"2024-06-28T15:17:24.031700+0000","last_active":"2024-06-28T16:11:35.563726+0000","last_peered":"2024-06-28T16:11:35.563726+0000","last_clean":"2024-06-28T16:11:35.563726+0000","last_became_active":"2024-06-28T15:17:24.028787+0000","last_became_peered":"2024-06-28T15:17:24.028787+0000","last_unstale":"2024-06-28T16:11:35.563726+0000","last_undegraded":"2024-06-28T16:11:35.563726+0000","last_fullsized":"2024-06-28T16:11:35.563726+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:38:55.771845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":30,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":26,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.252697+0000","last_change":"2024-06-28T15:37:33.337334+0000","last_active":"2024-06-28T16:10:05.252697+0000","last_peered":"2024-06-28T16:10:05.252697+0000","last_clean":"2024-06-28T16:10:05.252697+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:10:05.252697+0000","last_undegraded":"2024-06-28T16:10:05.252697+0000","last_fullsized":"2024-06-28T16:10:05.252697+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:34:53.771735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"260'2","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.119791+0000","last_change":"2024-06-28T15:21:11.105141+0000","last_active":"2024-06-28T16:13:45.119791+0000","last_peered":"2024-06-28T16:13:45.119791+0000","last_clean":"2024-06-28T16:13:45.119791+0000","last_became_active":"2024-06-28T15:21:11.104387+0000","last_became_peered":"2024-06-28T15:21:11.104387+0000","last_unstale":"2024-06-28T16:13:45.119791+0000","last_undegraded":"2024-06-28T16:13:45.119791+0000","last_fullsized":"2024-06-28T16:13:45.119791+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:29:31.608922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"260'2","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.126798+0000","last_change":"2024-06-28T15:21:07.049830+0000","last_active":"2024-06-28T16:13:40.126798+0000","last_peered":"2024-06-28T16:13:40.126798+0000","last_clean":"2024-06-28T16:13:40.126798+0000","last_became_active":"2024-06-28T15:21:07.049141+0000","last_became_peered":"2024-06-28T15:21:07.049141+0000","last_unstale":"2024-06-28T16:13:40.126798+0000","last_undegraded":"2024-06-28T16:13:40.126798+0000","last_fullsized":"2024-06-28T16:13:40.126798+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:18:35.987689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"260'18","reported_seq":110,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:40.271668+0000","last_change":"2024-06-28T15:17:23.994609+0000","last_active":"2024-06-28T16:11:40.271668+0000","last_peered":"2024-06-28T16:11:40.271668+0000","last_clean":"2024-06-28T16:11:40.271668+0000","last_became_active":"2024-06-28T15:17:23.993905+0000","last_became_peered":"2024-06-28T15:17:23.993905+0000","last_unstale":"2024-06-28T16:11:40.271668+0000","last_undegraded":"2024-06-28T16:11:40.271668+0000","last_fullsized":"2024-06-28T16:11:40.271668+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:33:37.469328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":64,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.087475+0000","last_change":"2024-06-28T15:37:33.353498+0000","last_active":"2024-06-28T16:11:25.087475+0000","last_peered":"2024-06-28T16:11:25.087475+0000","last_clean":"2024-06-28T16:11:25.087475+0000","last_became_active":"2024-06-28T15:37:33.345642+0000","last_became_peered":"2024-06-28T15:37:33.345642+0000","last_unstale":"2024-06-28T16:11:25.087475+0000","last_undegraded":"2024-06-28T16:11:25.087475+0000","last_fullsized":"2024-06-28T16:11:25.087475+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:38:21.634963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"257'5","reported_seq":57,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.558957+0000","last_change":"2024-06-28T15:21:11.098339+0000","last_active":"2024-06-28T16:11:15.558957+0000","last_peered":"2024-06-28T16:11:15.558957+0000","last_clean":"2024-06-28T16:11:15.558957+0000","last_became_active":"2024-06-28T15:21:11.097619+0000","last_became_peered":"2024-06-28T15:21:11.097619+0000","last_unstale":"2024-06-28T16:11:15.558957+0000","last_undegraded":"2024-06-28T16:11:15.558957+0000","last_fullsized":"2024-06-28T16:11:15.558957+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:23:45.881180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5438,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"0'0","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.268802+0000","last_change":"2024-06-28T15:21:07.020860+0000","last_active":"2024-06-28T16:11:25.268802+0000","last_peered":"2024-06-28T16:11:25.268802+0000","last_clean":"2024-06-28T16:11:25.268802+0000","last_became_active":"2024-06-28T15:21:07.020090+0000","last_became_peered":"2024-06-28T15:21:07.020090+0000","last_unstale":"2024-06-28T16:11:25.268802+0000","last_undegraded":"2024-06-28T16:11:25.268802+0000","last_fullsized":"2024-06-28T16:11:25.268802+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:42:14.199161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"260'43","reported_seq":174,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:09:55.540550+0000","last_change":"2024-06-28T15:17:24.014535+0000","last_active":"2024-06-28T16:09:55.540550+0000","last_peered":"2024-06-28T16:09:55.540550+0000","last_clean":"2024-06-28T16:09:55.540550+0000","last_became_active":"2024-06-28T15:17:24.012739+0000","last_became_peered":"2024-06-28T15:17:24.012739+0000","last_unstale":"2024-06-28T16:09:55.540550+0000","last_undegraded":"2024-06-28T16:09:55.540550+0000","last_fullsized":"2024-06-28T16:09:55.540550+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:08:38.008559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":221,"num_read_kb":154,"num_write":91,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.509029+0000","last_change":"2024-06-28T15:37:33.345746+0000","last_active":"2024-06-28T16:11:30.509029+0000","last_peered":"2024-06-28T16:11:30.509029+0000","last_clean":"2024-06-28T16:11:30.509029+0000","last_became_active":"2024-06-28T15:37:33.344850+0000","last_became_peered":"2024-06-28T15:37:33.344850+0000","last_unstale":"2024-06-28T16:11:30.509029+0000","last_undegraded":"2024-06-28T16:11:30.509029+0000","last_fullsized":"2024-06-28T16:11:30.509029+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:52:03.083724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"260'3","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.320060+0000","last_change":"2024-06-28T15:21:11.107158+0000","last_active":"2024-06-28T16:13:20.320060+0000","last_peered":"2024-06-28T16:13:20.320060+0000","last_clean":"2024-06-28T16:13:20.320060+0000","last_became_active":"2024-06-28T15:21:11.102572+0000","last_became_peered":"2024-06-28T15:21:11.102572+0000","last_unstale":"2024-06-28T16:13:20.320060+0000","last_undegraded":"2024-06-28T16:13:20.320060+0000","last_fullsized":"2024-06-28T16:13:20.320060+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:06:32.919789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.085295+0000","last_change":"2024-06-28T15:21:07.032950+0000","last_active":"2024-06-28T16:11:10.085295+0000","last_peered":"2024-06-28T16:11:10.085295+0000","last_clean":"2024-06-28T16:11:10.085295+0000","last_became_active":"2024-06-28T15:21:07.030576+0000","last_became_peered":"2024-06-28T15:21:07.030576+0000","last_unstale":"2024-06-28T16:11:10.085295+0000","last_undegraded":"2024-06-28T16:11:10.085295+0000","last_fullsized":"2024-06-28T16:11:10.085295+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:28:35.851727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'23","reported_seq":116,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:09:55.485817+0000","last_change":"2024-06-28T15:17:24.019871+0000","last_active":"2024-06-28T16:09:55.485817+0000","last_peered":"2024-06-28T16:09:55.485817+0000","last_clean":"2024-06-28T16:09:55.485817+0000","last_became_active":"2024-06-28T15:17:24.019120+0000","last_became_peered":"2024-06-28T15:17:24.019120+0000","last_unstale":"2024-06-28T16:09:55.485817+0000","last_undegraded":"2024-06-28T16:09:55.485817+0000","last_fullsized":"2024-06-28T16:09:55.485817+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:25:56.355193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":82,"num_write":51,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:40.511313+0000","last_change":"2024-06-28T15:37:33.339459+0000","last_active":"2024-06-28T16:11:40.511313+0000","last_peered":"2024-06-28T16:11:40.511313+0000","last_clean":"2024-06-28T16:11:40.511313+0000","last_became_active":"2024-06-28T15:37:33.335045+0000","last_became_peered":"2024-06-28T15:37:33.335045+0000","last_unstale":"2024-06-28T16:11:40.511313+0000","last_undegraded":"2024-06-28T16:11:40.511313+0000","last_fullsized":"2024-06-28T16:11:40.511313+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:39:33.932638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"0'0","reported_seq":44,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:10.091403+0000","last_change":"2024-06-28T15:21:11.094697+0000","last_active":"2024-06-28T16:12:10.091403+0000","last_peered":"2024-06-28T16:12:10.091403+0000","last_clean":"2024-06-28T16:12:10.091403+0000","last_became_active":"2024-06-28T15:21:11.094122+0000","last_became_peered":"2024-06-28T15:21:11.094122+0000","last_unstale":"2024-06-28T16:12:10.091403+0000","last_undegraded":"2024-06-28T16:12:10.091403+0000","last_fullsized":"2024-06-28T16:12:10.091403+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:48:32.491192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.569464+0000","last_change":"2024-06-28T15:21:07.027869+0000","last_active":"2024-06-28T16:11:55.569464+0000","last_peered":"2024-06-28T16:11:55.569464+0000","last_clean":"2024-06-28T16:11:55.569464+0000","last_became_active":"2024-06-28T15:21:07.025549+0000","last_became_peered":"2024-06-28T15:21:07.025549+0000","last_unstale":"2024-06-28T16:11:55.569464+0000","last_undegraded":"2024-06-28T16:11:55.569464+0000","last_fullsized":"2024-06-28T16:11:55.569464+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:37:01.202123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"260'24","reported_seq":120,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:09:55.485705+0000","last_change":"2024-06-28T15:17:24.011769+0000","last_active":"2024-06-28T16:09:55.485705+0000","last_peered":"2024-06-28T16:09:55.485705+0000","last_clean":"2024-06-28T16:09:55.485705+0000","last_became_active":"2024-06-28T15:17:24.010441+0000","last_became_peered":"2024-06-28T15:17:24.010441+0000","last_unstale":"2024-06-28T16:09:55.485705+0000","last_undegraded":"2024-06-28T16:09:55.485705+0000","last_fullsized":"2024-06-28T16:09:55.485705+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:28:32.900550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":85,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.091008+0000","last_change":"2024-06-28T15:37:33.341710+0000","last_active":"2024-06-28T16:10:55.091008+0000","last_peered":"2024-06-28T16:10:55.091008+0000","last_clean":"2024-06-28T16:10:55.091008+0000","last_became_active":"2024-06-28T15:37:33.336623+0000","last_became_peered":"2024-06-28T15:37:33.336623+0000","last_unstale":"2024-06-28T16:10:55.091008+0000","last_undegraded":"2024-06-28T16:10:55.091008+0000","last_fullsized":"2024-06-28T16:10:55.091008+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:49:52.074185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"260'11","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.517566+0000","last_change":"2024-06-28T15:21:11.100151+0000","last_active":"2024-06-28T16:14:32.517566+0000","last_peered":"2024-06-28T16:14:32.517566+0000","last_clean":"2024-06-28T16:14:32.517566+0000","last_became_active":"2024-06-28T15:21:11.099434+0000","last_became_peered":"2024-06-28T15:21:11.099434+0000","last_unstale":"2024-06-28T16:14:32.517566+0000","last_undegraded":"2024-06-28T16:14:32.517566+0000","last_fullsized":"2024-06-28T16:14:32.517566+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:21:06.014702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":20482,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"257'5","reported_seq":65,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:10.098868+0000","last_change":"2024-06-28T15:21:07.044767+0000","last_active":"2024-06-28T16:12:10.098868+0000","last_peered":"2024-06-28T16:12:10.098868+0000","last_clean":"2024-06-28T16:12:10.098868+0000","last_became_active":"2024-06-28T15:21:07.043987+0000","last_became_peered":"2024-06-28T15:21:07.043987+0000","last_unstale":"2024-06-28T16:12:10.098868+0000","last_undegraded":"2024-06-28T16:12:10.098868+0000","last_fullsized":"2024-06-28T16:12:10.098868+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:53:56.640453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"260'19","reported_seq":100,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:09:55.067186+0000","last_change":"2024-06-28T15:17:24.007176+0000","last_active":"2024-06-28T16:09:55.067186+0000","last_peered":"2024-06-28T16:09:55.067186+0000","last_clean":"2024-06-28T16:09:55.067186+0000","last_became_active":"2024-06-28T15:17:24.006314+0000","last_became_peered":"2024-06-28T15:17:24.006314+0000","last_unstale":"2024-06-28T16:09:55.067186+0000","last_undegraded":"2024-06-28T16:09:55.067186+0000","last_fullsized":"2024-06-28T16:09:55.067186+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:20:19.639267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":54,"num_write":49,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"260'1","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.076954+0000","last_change":"2024-06-28T15:37:33.348503+0000","last_active":"2024-06-28T16:10:05.076954+0000","last_peered":"2024-06-28T16:10:05.076954+0000","last_clean":"2024-06-28T16:10:05.076954+0000","last_became_active":"2024-06-28T15:37:33.347715+0000","last_became_peered":"2024-06-28T15:37:33.347715+0000","last_unstale":"2024-06-28T16:10:05.076954+0000","last_undegraded":"2024-06-28T16:10:05.076954+0000","last_fullsized":"2024-06-28T16:10:05.076954+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:01:24.044514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":42,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.085434+0000","last_change":"2024-06-28T15:21:11.089867+0000","last_active":"2024-06-28T16:11:15.085434+0000","last_peered":"2024-06-28T16:11:15.085434+0000","last_clean":"2024-06-28T16:11:15.085434+0000","last_became_active":"2024-06-28T15:21:11.088625+0000","last_became_peered":"2024-06-28T15:21:11.088625+0000","last_unstale":"2024-06-28T16:11:15.085434+0000","last_undegraded":"2024-06-28T16:11:15.085434+0000","last_fullsized":"2024-06-28T16:11:15.085434+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:32:47.953440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"260'6","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.118688+0000","last_change":"2024-06-28T15:21:07.031673+0000","last_active":"2024-06-28T16:13:40.118688+0000","last_peered":"2024-06-28T16:13:40.118688+0000","last_clean":"2024-06-28T16:13:40.118688+0000","last_became_active":"2024-06-28T15:21:07.030888+0000","last_became_peered":"2024-06-28T15:21:07.030888+0000","last_unstale":"2024-06-28T16:13:40.118688+0000","last_undegraded":"2024-06-28T16:13:40.118688+0000","last_fullsized":"2024-06-28T16:13:40.118688+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:54.912979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":6,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"260'72","reported_seq":247,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:45.552156+0000","last_change":"2024-06-28T15:17:23.993803+0000","last_active":"2024-06-28T16:10:45.552156+0000","last_peered":"2024-06-28T16:10:45.552156+0000","last_clean":"2024-06-28T16:10:45.552156+0000","last_became_active":"2024-06-28T15:17:23.993026+0000","last_became_peered":"2024-06-28T15:17:23.993026+0000","last_unstale":"2024-06-28T16:10:45.552156+0000","last_undegraded":"2024-06-28T16:10:45.552156+0000","last_fullsized":"2024-06-28T16:10:45.552156+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:12:38.304019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":255,"num_write":131,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":26,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.077019+0000","last_change":"2024-06-28T15:37:33.351154+0000","last_active":"2024-06-28T16:10:05.077019+0000","last_peered":"2024-06-28T16:10:05.077019+0000","last_clean":"2024-06-28T16:10:05.077019+0000","last_became_active":"2024-06-28T15:37:33.350447+0000","last_became_peered":"2024-06-28T15:37:33.350447+0000","last_unstale":"2024-06-28T16:10:05.077019+0000","last_undegraded":"2024-06-28T16:10:05.077019+0000","last_fullsized":"2024-06-28T16:10:05.077019+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:41:10.113052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":43,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.504142+0000","last_change":"2024-06-28T15:21:11.101407+0000","last_active":"2024-06-28T16:11:10.504142+0000","last_peered":"2024-06-28T16:11:10.504142+0000","last_clean":"2024-06-28T16:11:10.504142+0000","last_became_active":"2024-06-28T15:21:11.100603+0000","last_became_peered":"2024-06-28T15:21:11.100603+0000","last_unstale":"2024-06-28T16:11:10.504142+0000","last_undegraded":"2024-06-28T16:11:10.504142+0000","last_fullsized":"2024-06-28T16:11:10.504142+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:33:54.057658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"260'3","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.557448+0000","last_change":"2024-06-28T15:21:07.047854+0000","last_active":"2024-06-28T16:11:10.557448+0000","last_peered":"2024-06-28T16:11:10.557448+0000","last_clean":"2024-06-28T16:11:10.557448+0000","last_became_active":"2024-06-28T15:21:07.045902+0000","last_became_peered":"2024-06-28T15:21:07.045902+0000","last_unstale":"2024-06-28T16:11:10.557448+0000","last_undegraded":"2024-06-28T16:11:10.557448+0000","last_fullsized":"2024-06-28T16:11:10.557448+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:00:14.390394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1088721,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":40,"num_write":8,"num_write_kb":1064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"260'48","reported_seq":192,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.087151+0000","last_change":"2024-06-28T15:17:24.025635+0000","last_active":"2024-06-28T16:10:35.087151+0000","last_peered":"2024-06-28T16:10:35.087151+0000","last_clean":"2024-06-28T16:10:35.087151+0000","last_became_active":"2024-06-28T15:17:24.016249+0000","last_became_peered":"2024-06-28T15:17:24.016249+0000","last_unstale":"2024-06-28T16:10:35.087151+0000","last_undegraded":"2024-06-28T16:10:35.087151+0000","last_fullsized":"2024-06-28T16:10:35.087151+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:59:25.184524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":260,"num_read_kb":188,"num_write":81,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.083231+0000","last_change":"2024-06-28T15:37:33.317386+0000","last_active":"2024-06-28T16:10:15.083231+0000","last_peered":"2024-06-28T16:10:15.083231+0000","last_clean":"2024-06-28T16:10:15.083231+0000","last_became_active":"2024-06-28T15:37:33.316616+0000","last_became_peered":"2024-06-28T15:37:33.316616+0000","last_unstale":"2024-06-28T16:10:15.083231+0000","last_undegraded":"2024-06-28T16:10:15.083231+0000","last_fullsized":"2024-06-28T16:10:15.083231+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:02:05.364651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"260'8","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:20.333490+0000","last_change":"2024-06-28T15:21:11.099574+0000","last_active":"2024-06-28T16:14:20.333490+0000","last_peered":"2024-06-28T16:14:20.333490+0000","last_clean":"2024-06-28T16:14:20.333490+0000","last_became_active":"2024-06-28T15:21:11.094947+0000","last_became_peered":"2024-06-28T15:21:11.094947+0000","last_unstale":"2024-06-28T16:14:20.333490+0000","last_undegraded":"2024-06-28T16:14:20.333490+0000","last_fullsized":"2024-06-28T16:14:20.333490+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:14:27.139498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":12406,"num_write":8,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"260'5","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:45.099524+0000","last_change":"2024-06-28T15:21:07.026138+0000","last_active":"2024-06-28T16:12:45.099524+0000","last_peered":"2024-06-28T16:12:45.099524+0000","last_clean":"2024-06-28T16:12:45.099524+0000","last_became_active":"2024-06-28T15:21:07.025396+0000","last_became_peered":"2024-06-28T15:21:07.025396+0000","last_unstale":"2024-06-28T16:12:45.099524+0000","last_undegraded":"2024-06-28T16:12:45.099524+0000","last_fullsized":"2024-06-28T16:12:45.099524+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:48:53.728948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":5,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"257'14","reported_seq":96,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.550345+0000","last_change":"2024-06-28T15:17:24.015094+0000","last_active":"2024-06-28T16:10:35.550345+0000","last_peered":"2024-06-28T16:10:35.550345+0000","last_clean":"2024-06-28T16:10:35.550345+0000","last_became_active":"2024-06-28T15:17:24.010190+0000","last_became_peered":"2024-06-28T15:17:24.010190+0000","last_unstale":"2024-06-28T16:10:35.550345+0000","last_undegraded":"2024-06-28T16:10:35.550345+0000","last_fullsized":"2024-06-28T16:10:35.550345+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:20:30.481056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":51,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"260'1","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.255026+0000","last_change":"2024-06-28T15:37:33.338222+0000","last_active":"2024-06-28T16:10:15.255026+0000","last_peered":"2024-06-28T16:10:15.255026+0000","last_clean":"2024-06-28T16:10:15.255026+0000","last_became_active":"2024-06-28T15:37:33.334097+0000","last_became_peered":"2024-06-28T15:37:33.334097+0000","last_unstale":"2024-06-28T16:10:15.255026+0000","last_undegraded":"2024-06-28T16:10:15.255026+0000","last_fullsized":"2024-06-28T16:10:15.255026+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:04:55.107050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"260'9","reported_seq":62,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.508721+0000","last_change":"2024-06-28T15:21:11.107909+0000","last_active":"2024-06-28T16:14:33.508721+0000","last_peered":"2024-06-28T16:14:33.508721+0000","last_clean":"2024-06-28T16:14:33.508721+0000","last_became_active":"2024-06-28T15:21:11.107155+0000","last_became_peered":"2024-06-28T15:21:11.107155+0000","last_unstale":"2024-06-28T16:14:33.508721+0000","last_undegraded":"2024-06-28T16:14:33.508721+0000","last_fullsized":"2024-06-28T16:14:33.508721+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:27:41.181651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21504,"num_write":14,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":49,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.102713+0000","last_change":"2024-06-28T15:21:07.043526+0000","last_active":"2024-06-28T16:11:55.102713+0000","last_peered":"2024-06-28T16:11:55.102713+0000","last_clean":"2024-06-28T16:11:55.102713+0000","last_became_active":"2024-06-28T15:21:07.042691+0000","last_became_peered":"2024-06-28T15:21:07.042691+0000","last_unstale":"2024-06-28T16:11:55.102713+0000","last_undegraded":"2024-06-28T16:11:55.102713+0000","last_fullsized":"2024-06-28T16:11:55.102713+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:56:49.487276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"257'73","reported_seq":289,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.496134+0000","last_change":"2024-06-28T15:17:24.028644+0000","last_active":"2024-06-28T16:10:35.496134+0000","last_peered":"2024-06-28T16:10:35.496134+0000","last_clean":"2024-06-28T16:10:35.496134+0000","last_became_active":"2024-06-28T15:17:24.027786+0000","last_became_peered":"2024-06-28T15:17:24.027786+0000","last_unstale":"2024-06-28T16:10:35.496134+0000","last_undegraded":"2024-06-28T16:10:35.496134+0000","last_fullsized":"2024-06-28T16:10:35.496134+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:26:10.319359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":461,"num_read_kb":332,"num_write":85,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.079074+0000","last_change":"2024-06-28T15:37:33.355373+0000","last_active":"2024-06-28T16:10:15.079074+0000","last_peered":"2024-06-28T16:10:15.079074+0000","last_clean":"2024-06-28T16:10:15.079074+0000","last_became_active":"2024-06-28T15:37:33.354730+0000","last_became_peered":"2024-06-28T15:37:33.354730+0000","last_unstale":"2024-06-28T16:10:15.079074+0000","last_undegraded":"2024-06-28T16:10:15.079074+0000","last_fullsized":"2024-06-28T16:10:15.079074+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:33:00.069224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"260'4","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:20.333564+0000","last_change":"2024-06-28T15:21:11.094517+0000","last_active":"2024-06-28T16:14:20.333564+0000","last_peered":"2024-06-28T16:14:20.333564+0000","last_clean":"2024-06-28T16:14:20.333564+0000","last_became_active":"2024-06-28T15:21:11.093751+0000","last_became_peered":"2024-06-28T15:21:11.093751+0000","last_unstale":"2024-06-28T16:14:20.333564+0000","last_undegraded":"2024-06-28T16:14:20.333564+0000","last_fullsized":"2024-06-28T16:14:20.333564+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:37:10.229225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"260'2","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:20.305025+0000","last_change":"2024-06-28T15:21:07.043427+0000","last_active":"2024-06-28T16:14:20.305025+0000","last_peered":"2024-06-28T16:14:20.305025+0000","last_clean":"2024-06-28T16:14:20.305025+0000","last_became_active":"2024-06-28T15:21:07.042846+0000","last_became_peered":"2024-06-28T15:21:07.042846+0000","last_unstale":"2024-06-28T16:14:20.305025+0000","last_undegraded":"2024-06-28T16:14:20.305025+0000","last_fullsized":"2024-06-28T16:14:20.305025+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:41:49.061527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":59,"num_write":2,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"260'31","reported_seq":144,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.258858+0000","last_change":"2024-06-28T15:17:24.017530+0000","last_active":"2024-06-28T16:10:35.258858+0000","last_peered":"2024-06-28T16:10:35.258858+0000","last_clean":"2024-06-28T16:10:35.258858+0000","last_became_active":"2024-06-28T15:17:24.016275+0000","last_became_peered":"2024-06-28T15:17:24.016275+0000","last_unstale":"2024-06-28T16:10:35.258858+0000","last_undegraded":"2024-06-28T16:10:35.258858+0000","last_fullsized":"2024-06-28T16:10:35.258858+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:46:12.119777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":116,"num_write":60,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.065428+0000","last_change":"2024-06-28T15:37:33.323939+0000","last_active":"2024-06-28T16:10:15.065428+0000","last_peered":"2024-06-28T16:10:15.065428+0000","last_clean":"2024-06-28T16:10:15.065428+0000","last_became_active":"2024-06-28T15:37:33.323174+0000","last_became_peered":"2024-06-28T15:37:33.323174+0000","last_unstale":"2024-06-28T16:10:15.065428+0000","last_undegraded":"2024-06-28T16:10:15.065428+0000","last_fullsized":"2024-06-28T16:10:15.065428+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:23:02.255965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"260'9","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.510895+0000","last_change":"2024-06-28T15:21:11.088257+0000","last_active":"2024-06-28T16:14:33.510895+0000","last_peered":"2024-06-28T16:14:33.510895+0000","last_clean":"2024-06-28T16:14:33.510895+0000","last_became_active":"2024-06-28T15:21:11.087119+0000","last_became_peered":"2024-06-28T15:21:11.087119+0000","last_unstale":"2024-06-28T16:14:33.510895+0000","last_undegraded":"2024-06-28T16:14:33.510895+0000","last_fullsized":"2024-06-28T16:14:33.510895+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:24:43.264477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":24576,"num_write":19,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.569812+0000","last_change":"2024-06-28T15:21:07.032975+0000","last_active":"2024-06-28T16:11:55.569812+0000","last_peered":"2024-06-28T16:11:55.569812+0000","last_clean":"2024-06-28T16:11:55.569812+0000","last_became_active":"2024-06-28T15:21:07.032092+0000","last_became_peered":"2024-06-28T15:21:07.032092+0000","last_unstale":"2024-06-28T16:11:55.569812+0000","last_undegraded":"2024-06-28T16:11:55.569812+0000","last_fullsized":"2024-06-28T16:11:55.569812+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:29:54.468683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"260'21","reported_seq":120,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.094899+0000","last_change":"2024-06-28T15:17:24.021516+0000","last_active":"2024-06-28T16:11:20.094899+0000","last_peered":"2024-06-28T16:11:20.094899+0000","last_clean":"2024-06-28T16:11:20.094899+0000","last_became_active":"2024-06-28T15:17:24.020899+0000","last_became_peered":"2024-06-28T15:17:24.020899+0000","last_unstale":"2024-06-28T16:11:20.094899+0000","last_undegraded":"2024-06-28T16:11:20.094899+0000","last_fullsized":"2024-06-28T16:11:20.094899+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:16:16.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":86,"num_write":39,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.508021+0000","last_change":"2024-06-28T15:37:33.339666+0000","last_active":"2024-06-28T16:11:25.508021+0000","last_peered":"2024-06-28T16:11:25.508021+0000","last_clean":"2024-06-28T16:11:25.508021+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:11:25.508021+0000","last_undegraded":"2024-06-28T16:11:25.508021+0000","last_fullsized":"2024-06-28T16:11:25.508021+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:16:22.839274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"257'3","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.083204+0000","last_change":"2024-06-28T15:21:11.098494+0000","last_active":"2024-06-28T16:10:15.083204+0000","last_peered":"2024-06-28T16:10:15.083204+0000","last_clean":"2024-06-28T16:10:15.083204+0000","last_became_active":"2024-06-28T15:21:11.097622+0000","last_became_peered":"2024-06-28T15:21:11.097622+0000","last_unstale":"2024-06-28T16:10:15.083204+0000","last_undegraded":"2024-06-28T16:10:15.083204+0000","last_fullsized":"2024-06-28T16:10:15.083204+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:35:49.979919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"0'0","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.071391+0000","last_change":"2024-06-28T15:21:07.039533+0000","last_active":"2024-06-28T16:10:10.071391+0000","last_peered":"2024-06-28T16:10:10.071391+0000","last_clean":"2024-06-28T16:10:10.071391+0000","last_became_active":"2024-06-28T15:21:07.038746+0000","last_became_peered":"2024-06-28T15:21:07.038746+0000","last_unstale":"2024-06-28T16:10:10.071391+0000","last_undegraded":"2024-06-28T16:10:10.071391+0000","last_fullsized":"2024-06-28T16:10:10.071391+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:11:32.609450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"257'9","reported_seq":81,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.091202+0000","last_change":"2024-06-28T15:17:24.024616+0000","last_active":"2024-06-28T16:11:05.091202+0000","last_peered":"2024-06-28T16:11:05.091202+0000","last_clean":"2024-06-28T16:11:05.091202+0000","last_became_active":"2024-06-28T15:17:24.020276+0000","last_became_peered":"2024-06-28T15:17:24.020276+0000","last_unstale":"2024-06-28T16:11:05.091202+0000","last_undegraded":"2024-06-28T16:11:05.091202+0000","last_fullsized":"2024-06-28T16:11:05.091202+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:06.656845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.561512+0000","last_change":"2024-06-28T15:37:33.328340+0000","last_active":"2024-06-28T16:11:25.561512+0000","last_peered":"2024-06-28T16:11:25.561512+0000","last_clean":"2024-06-28T16:11:25.561512+0000","last_became_active":"2024-06-28T15:37:33.327497+0000","last_became_peered":"2024-06-28T15:37:33.327497+0000","last_unstale":"2024-06-28T16:11:25.561512+0000","last_undegraded":"2024-06-28T16:11:25.561512+0000","last_fullsized":"2024-06-28T16:11:25.561512+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:28:42.508678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"260'4","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.136460+0000","last_change":"2024-06-28T15:21:11.092761+0000","last_active":"2024-06-28T16:14:33.136460+0000","last_peered":"2024-06-28T16:14:33.136460+0000","last_clean":"2024-06-28T16:14:33.136460+0000","last_became_active":"2024-06-28T15:21:11.092033+0000","last_became_peered":"2024-06-28T15:21:11.092033+0000","last_unstale":"2024-06-28T16:14:33.136460+0000","last_undegraded":"2024-06-28T16:14:33.136460+0000","last_fullsized":"2024-06-28T16:14:33.136460+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:19:30.265533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6152,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.489165+0000","last_change":"2024-06-28T15:21:07.047668+0000","last_active":"2024-06-28T16:10:10.489165+0000","last_peered":"2024-06-28T16:10:10.489165+0000","last_clean":"2024-06-28T16:10:10.489165+0000","last_became_active":"2024-06-28T15:21:07.045842+0000","last_became_peered":"2024-06-28T15:21:07.045842+0000","last_unstale":"2024-06-28T16:10:10.489165+0000","last_undegraded":"2024-06-28T16:10:10.489165+0000","last_fullsized":"2024-06-28T16:10:10.489165+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:42:33.570243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"260'17","reported_seq":106,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.556141+0000","last_change":"2024-06-28T15:17:24.016733+0000","last_active":"2024-06-28T16:11:05.556141+0000","last_peered":"2024-06-28T16:11:05.556141+0000","last_clean":"2024-06-28T16:11:05.556141+0000","last_became_active":"2024-06-28T15:17:24.013118+0000","last_became_peered":"2024-06-28T15:17:24.013118+0000","last_unstale":"2024-06-28T16:11:05.556141+0000","last_undegraded":"2024-06-28T16:11:05.556141+0000","last_fullsized":"2024-06-28T16:11:05.556141+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:57:44.752951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":63,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"259'1","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.546888+0000","last_change":"2024-06-28T15:37:33.342777+0000","last_active":"2024-06-28T16:10:20.546888+0000","last_peered":"2024-06-28T16:10:20.546888+0000","last_clean":"2024-06-28T16:10:20.546888+0000","last_became_active":"2024-06-28T15:37:33.339097+0000","last_became_peered":"2024-06-28T15:37:33.339097+0000","last_unstale":"2024-06-28T16:10:20.546888+0000","last_undegraded":"2024-06-28T16:10:20.546888+0000","last_fullsized":"2024-06-28T16:10:20.546888+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:34:20.645575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"257'3","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.489379+0000","last_change":"2024-06-28T15:21:11.101867+0000","last_active":"2024-06-28T16:10:10.489379+0000","last_peered":"2024-06-28T16:10:10.489379+0000","last_clean":"2024-06-28T16:10:10.489379+0000","last_became_active":"2024-06-28T15:21:11.101063+0000","last_became_peered":"2024-06-28T15:21:11.101063+0000","last_unstale":"2024-06-28T16:10:10.489379+0000","last_undegraded":"2024-06-28T16:10:10.489379+0000","last_fullsized":"2024-06-28T16:10:10.489379+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:58:49.328893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"257'3","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.070602+0000","last_change":"2024-06-28T15:21:07.042183+0000","last_active":"2024-06-28T16:10:10.070602+0000","last_peered":"2024-06-28T16:10:10.070602+0000","last_clean":"2024-06-28T16:10:10.070602+0000","last_became_active":"2024-06-28T15:21:07.040973+0000","last_became_peered":"2024-06-28T15:21:07.040973+0000","last_unstale":"2024-06-28T16:10:10.070602+0000","last_undegraded":"2024-06-28T16:10:10.070602+0000","last_fullsized":"2024-06-28T16:10:10.070602+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:38:10.417534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"260'22","reported_seq":115,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.083945+0000","last_change":"2024-06-28T15:17:24.029346+0000","last_active":"2024-06-28T16:11:05.083945+0000","last_peered":"2024-06-28T16:11:05.083945+0000","last_clean":"2024-06-28T16:11:05.083945+0000","last_became_active":"2024-06-28T15:17:24.028410+0000","last_became_peered":"2024-06-28T15:17:24.028410+0000","last_unstale":"2024-06-28T16:11:05.083945+0000","last_undegraded":"2024-06-28T16:11:05.083945+0000","last_fullsized":"2024-06-28T16:11:05.083945+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:25:05.370434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"260'1","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.507957+0000","last_change":"2024-06-28T15:37:33.341173+0000","last_active":"2024-06-28T16:11:25.507957+0000","last_peered":"2024-06-28T16:11:25.507957+0000","last_clean":"2024-06-28T16:11:25.507957+0000","last_became_active":"2024-06-28T15:37:33.340370+0000","last_became_peered":"2024-06-28T15:37:33.340370+0000","last_unstale":"2024-06-28T16:11:25.507957+0000","last_undegraded":"2024-06-28T16:11:25.507957+0000","last_fullsized":"2024-06-28T16:11:25.507957+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:07:43.988727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"257'3","reported_seq":48,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.071850+0000","last_change":"2024-06-28T15:21:11.104826+0000","last_active":"2024-06-28T16:10:15.071850+0000","last_peered":"2024-06-28T16:10:15.071850+0000","last_clean":"2024-06-28T16:10:15.071850+0000","last_became_active":"2024-06-28T15:21:11.104038+0000","last_became_peered":"2024-06-28T15:21:11.104038+0000","last_unstale":"2024-06-28T16:10:15.071850+0000","last_undegraded":"2024-06-28T16:10:15.071850+0000","last_fullsized":"2024-06-28T16:10:15.071850+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:38:01.314736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"257'8","reported_seq":71,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.064325+0000","last_change":"2024-06-28T15:21:07.010980+0000","last_active":"2024-06-28T16:10:10.064325+0000","last_peered":"2024-06-28T16:10:10.064325+0000","last_clean":"2024-06-28T16:10:10.064325+0000","last_became_active":"2024-06-28T15:21:07.008899+0000","last_became_peered":"2024-06-28T15:21:07.008899+0000","last_unstale":"2024-06-28T16:10:10.064325+0000","last_undegraded":"2024-06-28T16:10:10.064325+0000","last_fullsized":"2024-06-28T16:10:10.064325+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:39:00.266882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":7,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"260'42","reported_seq":165,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.502745+0000","last_change":"2024-06-28T15:17:24.020041+0000","last_active":"2024-06-28T16:11:05.502745+0000","last_peered":"2024-06-28T16:11:05.502745+0000","last_clean":"2024-06-28T16:11:05.502745+0000","last_became_active":"2024-06-28T15:17:24.019180+0000","last_became_peered":"2024-06-28T15:17:24.019180+0000","last_unstale":"2024-06-28T16:11:05.502745+0000","last_undegraded":"2024-06-28T16:11:05.502745+0000","last_fullsized":"2024-06-28T16:11:05.502745+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:00:51.600204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":198,"num_read_kb":139,"num_write":93,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.280511+0000","last_change":"2024-06-28T15:37:33.342459+0000","last_active":"2024-06-28T16:12:25.280511+0000","last_peered":"2024-06-28T16:12:25.280511+0000","last_clean":"2024-06-28T16:12:25.280511+0000","last_became_active":"2024-06-28T15:37:33.341716+0000","last_became_peered":"2024-06-28T15:37:33.341716+0000","last_unstale":"2024-06-28T16:12:25.280511+0000","last_undegraded":"2024-06-28T16:12:25.280511+0000","last_fullsized":"2024-06-28T16:12:25.280511+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:53:31.136353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:40.497394+0000","last_change":"2024-06-28T15:21:11.094239+0000","last_active":"2024-06-28T16:10:40.497394+0000","last_peered":"2024-06-28T16:10:40.497394+0000","last_clean":"2024-06-28T16:10:40.497394+0000","last_became_active":"2024-06-28T15:21:11.093276+0000","last_became_peered":"2024-06-28T15:21:11.093276+0000","last_unstale":"2024-06-28T16:10:40.497394+0000","last_undegraded":"2024-06-28T16:10:40.497394+0000","last_fullsized":"2024-06-28T16:10:40.497394+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:48:45.762370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"260'9","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.368805+0000","last_change":"2024-06-28T15:21:07.041391+0000","last_active":"2024-06-28T16:14:33.368805+0000","last_peered":"2024-06-28T16:14:33.368805+0000","last_clean":"2024-06-28T16:14:33.368805+0000","last_became_active":"2024-06-28T15:21:07.040691+0000","last_became_peered":"2024-06-28T15:21:07.040691+0000","last_unstale":"2024-06-28T16:14:33.368805+0000","last_undegraded":"2024-06-28T16:14:33.368805+0000","last_fullsized":"2024-06-28T16:14:33.368805+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:20:52.003993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":17408,"num_write":9,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"260'47","reported_seq":184,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.509222+0000","last_change":"2024-06-28T15:17:24.028252+0000","last_active":"2024-06-28T16:11:30.509222+0000","last_peered":"2024-06-28T16:11:30.509222+0000","last_clean":"2024-06-28T16:11:30.509222+0000","last_became_active":"2024-06-28T15:17:24.027610+0000","last_became_peered":"2024-06-28T15:17:24.027610+0000","last_unstale":"2024-06-28T16:11:30.509222+0000","last_undegraded":"2024-06-28T16:11:30.509222+0000","last_fullsized":"2024-06-28T16:11:30.509222+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:08:18.788559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":231,"num_read_kb":169,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.094821+0000","last_change":"2024-06-28T15:37:33.347375+0000","last_active":"2024-06-28T16:11:20.094821+0000","last_peered":"2024-06-28T16:11:20.094821+0000","last_clean":"2024-06-28T16:11:20.094821+0000","last_became_active":"2024-06-28T15:37:33.346414+0000","last_became_peered":"2024-06-28T15:37:33.346414+0000","last_unstale":"2024-06-28T16:11:20.094821+0000","last_undegraded":"2024-06-28T16:11:20.094821+0000","last_fullsized":"2024-06-28T16:11:20.094821+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:39:51.219545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"257'5","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.082722+0000","last_change":"2024-06-28T15:22:00.970955+0000","last_active":"2024-06-28T16:11:30.082722+0000","last_peered":"2024-06-28T16:11:30.082722+0000","last_clean":"2024-06-28T16:11:30.082722+0000","last_became_active":"2024-06-28T15:22:00.940760+0000","last_became_peered":"2024-06-28T15:22:00.940760+0000","last_unstale":"2024-06-28T16:11:30.082722+0000","last_undegraded":"2024-06-28T16:11:30.082722+0000","last_fullsized":"2024-06-28T16:11:30.082722+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:14:23.932998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"260'3","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.287511+0000","last_change":"2024-06-28T15:21:07.031060+0000","last_active":"2024-06-28T16:12:55.287511+0000","last_peered":"2024-06-28T16:12:55.287511+0000","last_clean":"2024-06-28T16:12:55.287511+0000","last_became_active":"2024-06-28T15:21:07.030163+0000","last_became_peered":"2024-06-28T15:21:07.030163+0000","last_unstale":"2024-06-28T16:12:55.287511+0000","last_undegraded":"2024-06-28T16:12:55.287511+0000","last_fullsized":"2024-06-28T16:12:55.287511+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:19:20.592250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"260'40","reported_seq":171,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.081619+0000","last_change":"2024-06-28T15:17:24.014719+0000","last_active":"2024-06-28T16:11:25.081619+0000","last_peered":"2024-06-28T16:11:25.081619+0000","last_clean":"2024-06-28T16:11:25.081619+0000","last_became_active":"2024-06-28T15:17:24.012572+0000","last_became_peered":"2024-06-28T15:17:24.012572+0000","last_unstale":"2024-06-28T16:11:25.081619+0000","last_undegraded":"2024-06-28T16:11:25.081619+0000","last_fullsized":"2024-06-28T16:11:25.081619+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:07:56.306439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":148,"num_write":94,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.099240+0000","last_change":"2024-06-28T15:37:33.353545+0000","last_active":"2024-06-28T16:12:15.099240+0000","last_peered":"2024-06-28T16:12:15.099240+0000","last_clean":"2024-06-28T16:12:15.099240+0000","last_became_active":"2024-06-28T15:37:33.352959+0000","last_became_peered":"2024-06-28T15:37:33.352959+0000","last_unstale":"2024-06-28T16:12:15.099240+0000","last_undegraded":"2024-06-28T16:12:15.099240+0000","last_fullsized":"2024-06-28T16:12:15.099240+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:42.580937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.092127+0000","last_change":"2024-06-28T15:21:11.079730+0000","last_active":"2024-06-28T16:11:00.092127+0000","last_peered":"2024-06-28T16:11:00.092127+0000","last_clean":"2024-06-28T16:11:00.092127+0000","last_became_active":"2024-06-28T15:21:11.079036+0000","last_became_peered":"2024-06-28T15:21:11.079036+0000","last_unstale":"2024-06-28T16:11:00.092127+0000","last_undegraded":"2024-06-28T16:11:00.092127+0000","last_fullsized":"2024-06-28T16:11:00.092127+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:36:54.128714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"0'0","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:40.551262+0000","last_change":"2024-06-28T15:21:07.028297+0000","last_active":"2024-06-28T16:10:40.551262+0000","last_peered":"2024-06-28T16:10:40.551262+0000","last_clean":"2024-06-28T16:10:40.551262+0000","last_became_active":"2024-06-28T15:21:07.027451+0000","last_became_peered":"2024-06-28T15:21:07.027451+0000","last_unstale":"2024-06-28T16:10:40.551262+0000","last_undegraded":"2024-06-28T16:10:40.551262+0000","last_fullsized":"2024-06-28T16:10:40.551262+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:44:47.388468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"260'16","reported_seq":97,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.087601+0000","last_change":"2024-06-28T15:17:24.027026+0000","last_active":"2024-06-28T16:11:25.087601+0000","last_peered":"2024-06-28T16:11:25.087601+0000","last_clean":"2024-06-28T16:11:25.087601+0000","last_became_active":"2024-06-28T15:17:24.026271+0000","last_became_peered":"2024-06-28T15:17:24.026271+0000","last_unstale":"2024-06-28T16:11:25.087601+0000","last_undegraded":"2024-06-28T16:11:25.087601+0000","last_fullsized":"2024-06-28T16:11:25.087601+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:23:12.426770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":48,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.107001+0000","last_change":"2024-06-28T15:37:33.326639+0000","last_active":"2024-06-28T16:12:15.107001+0000","last_peered":"2024-06-28T16:12:15.107001+0000","last_clean":"2024-06-28T16:12:15.107001+0000","last_became_active":"2024-06-28T15:37:33.325849+0000","last_became_peered":"2024-06-28T15:37:33.325849+0000","last_unstale":"2024-06-28T16:12:15.107001+0000","last_undegraded":"2024-06-28T16:12:15.107001+0000","last_fullsized":"2024-06-28T16:12:15.107001+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:14:13.968785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"257'5","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:40.497479+0000","last_change":"2024-06-28T15:21:11.108377+0000","last_active":"2024-06-28T16:10:40.497479+0000","last_peered":"2024-06-28T16:10:40.497479+0000","last_clean":"2024-06-28T16:10:40.497479+0000","last_became_active":"2024-06-28T15:21:11.107498+0000","last_became_peered":"2024-06-28T15:21:11.107498+0000","last_unstale":"2024-06-28T16:10:40.497479+0000","last_undegraded":"2024-06-28T16:10:40.497479+0000","last_fullsized":"2024-06-28T16:10:40.497479+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:33:14.411962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"257'7","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.091049+0000","last_change":"2024-06-28T15:21:07.030359+0000","last_active":"2024-06-28T16:10:55.091049+0000","last_peered":"2024-06-28T16:10:55.091049+0000","last_clean":"2024-06-28T16:10:55.091049+0000","last_became_active":"2024-06-28T15:21:07.029525+0000","last_became_peered":"2024-06-28T15:21:07.029525+0000","last_unstale":"2024-06-28T16:10:55.091049+0000","last_undegraded":"2024-06-28T16:10:55.091049+0000","last_fullsized":"2024-06-28T16:10:55.091049+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:27:20.230827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":11,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"260'34","reported_seq":151,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.268692+0000","last_change":"2024-06-28T15:17:24.008356+0000","last_active":"2024-06-28T16:11:25.268692+0000","last_peered":"2024-06-28T16:11:25.268692+0000","last_clean":"2024-06-28T16:11:25.268692+0000","last_became_active":"2024-06-28T15:17:24.007649+0000","last_became_peered":"2024-06-28T15:17:24.007649+0000","last_unstale":"2024-06-28T16:11:25.268692+0000","last_undegraded":"2024-06-28T16:11:25.268692+0000","last_fullsized":"2024-06-28T16:11:25.268692+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:41:30.857517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":172,"num_read_kb":122,"num_write":76,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.506787+0000","last_change":"2024-06-28T15:37:33.335759+0000","last_active":"2024-06-28T16:11:20.506787+0000","last_peered":"2024-06-28T16:11:20.506787+0000","last_clean":"2024-06-28T16:11:20.506787+0000","last_became_active":"2024-06-28T15:37:33.334082+0000","last_became_peered":"2024-06-28T15:37:33.334082+0000","last_unstale":"2024-06-28T16:11:20.506787+0000","last_undegraded":"2024-06-28T16:11:20.506787+0000","last_fullsized":"2024-06-28T16:11:20.506787+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:37:25.552446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"0'0","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.083900+0000","last_change":"2024-06-28T15:21:11.108205+0000","last_active":"2024-06-28T16:11:05.083900+0000","last_peered":"2024-06-28T16:11:05.083900+0000","last_clean":"2024-06-28T16:11:05.083900+0000","last_became_active":"2024-06-28T15:21:11.107686+0000","last_became_peered":"2024-06-28T15:21:11.107686+0000","last_unstale":"2024-06-28T16:11:05.083900+0000","last_undegraded":"2024-06-28T16:11:05.083900+0000","last_fullsized":"2024-06-28T16:11:05.083900+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:57:49.231179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"0'0","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.555131+0000","last_change":"2024-06-28T15:21:07.033578+0000","last_active":"2024-06-28T16:11:00.555131+0000","last_peered":"2024-06-28T16:11:00.555131+0000","last_clean":"2024-06-28T16:11:00.555131+0000","last_became_active":"2024-06-28T15:21:07.032719+0000","last_became_peered":"2024-06-28T15:21:07.032719+0000","last_unstale":"2024-06-28T16:11:00.555131+0000","last_undegraded":"2024-06-28T16:11:00.555131+0000","last_fullsized":"2024-06-28T16:11:00.555131+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:15.597833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"260'45","reported_seq":182,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.561334+0000","last_change":"2024-06-28T15:17:24.030065+0000","last_active":"2024-06-28T16:11:25.561334+0000","last_peered":"2024-06-28T16:11:25.561334+0000","last_clean":"2024-06-28T16:11:25.561334+0000","last_became_active":"2024-06-28T15:17:24.029532+0000","last_became_peered":"2024-06-28T15:17:24.029532+0000","last_unstale":"2024-06-28T16:11:25.561334+0000","last_undegraded":"2024-06-28T16:11:25.561334+0000","last_fullsized":"2024-06-28T16:11:25.561334+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:32:40.759334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":234,"num_read_kb":165,"num_write":86,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.097302+0000","last_change":"2024-06-28T15:37:33.348528+0000","last_active":"2024-06-28T16:11:30.097302+0000","last_peered":"2024-06-28T16:11:30.097302+0000","last_clean":"2024-06-28T16:11:30.097302+0000","last_became_active":"2024-06-28T15:37:33.347691+0000","last_became_peered":"2024-06-28T15:37:33.347691+0000","last_unstale":"2024-06-28T16:11:30.097302+0000","last_undegraded":"2024-06-28T16:11:30.097302+0000","last_fullsized":"2024-06-28T16:11:30.097302+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:19:46.828298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.089675+0000","last_change":"2024-06-28T15:21:11.092385+0000","last_active":"2024-06-28T16:11:00.089675+0000","last_peered":"2024-06-28T16:11:00.089675+0000","last_clean":"2024-06-28T16:11:00.089675+0000","last_became_active":"2024-06-28T15:21:11.091656+0000","last_became_peered":"2024-06-28T15:21:11.091656+0000","last_unstale":"2024-06-28T16:11:00.089675+0000","last_undegraded":"2024-06-28T16:11:00.089675+0000","last_fullsized":"2024-06-28T16:11:00.089675+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:39:05.106486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"260'5","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.120386+0000","last_change":"2024-06-28T15:21:07.030338+0000","last_active":"2024-06-28T16:13:10.120386+0000","last_peered":"2024-06-28T16:13:10.120386+0000","last_clean":"2024-06-28T16:13:10.120386+0000","last_became_active":"2024-06-28T15:21:07.029495+0000","last_became_peered":"2024-06-28T15:21:07.029495+0000","last_unstale":"2024-06-28T16:13:10.120386+0000","last_undegraded":"2024-06-28T16:13:10.120386+0000","last_fullsized":"2024-06-28T16:13:10.120386+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:18:26.892995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":318,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"260'39","reported_seq":162,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.101407+0000","last_change":"2024-06-28T15:17:24.024944+0000","last_active":"2024-06-28T16:11:50.101407+0000","last_peered":"2024-06-28T16:11:50.101407+0000","last_clean":"2024-06-28T16:11:50.101407+0000","last_became_active":"2024-06-28T15:17:24.021200+0000","last_became_peered":"2024-06-28T15:17:24.021200+0000","last_unstale":"2024-06-28T16:11:50.101407+0000","last_undegraded":"2024-06-28T16:11:50.101407+0000","last_fullsized":"2024-06-28T16:11:50.101407+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:06:00.686781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":191,"num_read_kb":133,"num_write":87,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"260'1","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.094873+0000","last_change":"2024-06-28T15:37:33.336238+0000","last_active":"2024-06-28T16:11:20.094873+0000","last_peered":"2024-06-28T16:11:20.094873+0000","last_clean":"2024-06-28T16:11:20.094873+0000","last_became_active":"2024-06-28T15:37:33.335464+0000","last_became_peered":"2024-06-28T15:37:33.335464+0000","last_unstale":"2024-06-28T16:11:20.094873+0000","last_undegraded":"2024-06-28T16:11:20.094873+0000","last_fullsized":"2024-06-28T16:11:20.094873+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:19:13.910107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"257'3","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:45.088990+0000","last_change":"2024-06-28T15:21:11.094534+0000","last_active":"2024-06-28T16:10:45.088990+0000","last_peered":"2024-06-28T16:10:45.088990+0000","last_clean":"2024-06-28T16:10:45.088990+0000","last_became_active":"2024-06-28T15:21:11.091647+0000","last_became_peered":"2024-06-28T15:21:11.091647+0000","last_unstale":"2024-06-28T16:10:45.088990+0000","last_undegraded":"2024-06-28T16:10:45.088990+0000","last_fullsized":"2024-06-28T16:10:45.088990+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:26:33.093064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"257'3","reported_seq":57,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.557369+0000","last_change":"2024-06-28T15:21:07.046952+0000","last_active":"2024-06-28T16:11:10.557369+0000","last_peered":"2024-06-28T16:11:10.557369+0000","last_clean":"2024-06-28T16:11:10.557369+0000","last_became_active":"2024-06-28T15:21:07.046218+0000","last_became_peered":"2024-06-28T15:21:07.046218+0000","last_unstale":"2024-06-28T16:11:10.557369+0000","last_undegraded":"2024-06-28T16:11:10.557369+0000","last_fullsized":"2024-06-28T16:11:10.557369+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:30:18.956581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"260'54","reported_seq":208,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.510120+0000","last_change":"2024-06-28T15:17:24.011432+0000","last_active":"2024-06-28T16:11:35.510120+0000","last_peered":"2024-06-28T16:11:35.510120+0000","last_clean":"2024-06-28T16:11:35.510120+0000","last_became_active":"2024-06-28T15:17:24.010230+0000","last_became_peered":"2024-06-28T15:17:24.010230+0000","last_unstale":"2024-06-28T16:11:35.510120+0000","last_undegraded":"2024-06-28T16:11:35.510120+0000","last_fullsized":"2024-06-28T16:11:35.510120+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:14:20.556616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":283,"num_read_kb":205,"num_write":97,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.560195+0000","last_change":"2024-06-28T15:37:33.343356+0000","last_active":"2024-06-28T16:11:20.560195+0000","last_peered":"2024-06-28T16:11:20.560195+0000","last_clean":"2024-06-28T16:11:20.560195+0000","last_became_active":"2024-06-28T15:37:33.342585+0000","last_became_peered":"2024-06-28T15:37:33.342585+0000","last_unstale":"2024-06-28T16:11:20.560195+0000","last_undegraded":"2024-06-28T16:11:20.560195+0000","last_fullsized":"2024-06-28T16:11:20.560195+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:36:10.405028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"260'3","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:25.123229+0000","last_change":"2024-06-28T15:21:11.098149+0000","last_active":"2024-06-28T16:13:25.123229+0000","last_peered":"2024-06-28T16:13:25.123229+0000","last_clean":"2024-06-28T16:13:25.123229+0000","last_became_active":"2024-06-28T15:21:11.096884+0000","last_became_peered":"2024-06-28T15:21:11.096884+0000","last_unstale":"2024-06-28T16:13:25.123229+0000","last_undegraded":"2024-06-28T16:13:25.123229+0000","last_fullsized":"2024-06-28T16:13:25.123229+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:53:18.580625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"260'11","reported_seq":79,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.109271+0000","last_change":"2024-06-28T15:21:07.037099+0000","last_active":"2024-06-28T16:12:55.109271+0000","last_peered":"2024-06-28T16:12:55.109271+0000","last_clean":"2024-06-28T16:12:55.109271+0000","last_became_active":"2024-06-28T15:21:07.032545+0000","last_became_peered":"2024-06-28T15:21:07.032545+0000","last_unstale":"2024-06-28T16:12:55.109271+0000","last_undegraded":"2024-06-28T16:12:55.109271+0000","last_fullsized":"2024-06-28T16:12:55.109271+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:28:14.596619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":35842,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"260'40","reported_seq":171,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089612+0000","last_change":"2024-06-28T15:17:24.028816+0000","last_active":"2024-06-28T16:11:35.089612+0000","last_peered":"2024-06-28T16:11:35.089612+0000","last_clean":"2024-06-28T16:11:35.089612+0000","last_became_active":"2024-06-28T15:17:24.026702+0000","last_became_peered":"2024-06-28T15:17:24.026702+0000","last_unstale":"2024-06-28T16:11:35.089612+0000","last_undegraded":"2024-06-28T16:11:35.089612+0000","last_fullsized":"2024-06-28T16:11:35.089612+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:13:09.048722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":150,"num_write":83,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.088350+0000","last_change":"2024-06-28T15:37:33.343533+0000","last_active":"2024-06-28T16:10:55.088350+0000","last_peered":"2024-06-28T16:10:55.088350+0000","last_clean":"2024-06-28T16:10:55.088350+0000","last_became_active":"2024-06-28T15:37:33.342889+0000","last_became_peered":"2024-06-28T15:37:33.342889+0000","last_unstale":"2024-06-28T16:10:55.088350+0000","last_undegraded":"2024-06-28T16:10:55.088350+0000","last_fullsized":"2024-06-28T16:10:55.088350+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:44:29.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"260'1","reported_seq":49,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.561434+0000","last_change":"2024-06-28T15:21:11.093441+0000","last_active":"2024-06-28T16:11:25.561434+0000","last_peered":"2024-06-28T16:11:25.561434+0000","last_clean":"2024-06-28T16:11:25.561434+0000","last_became_active":"2024-06-28T15:21:11.092614+0000","last_became_peered":"2024-06-28T15:21:11.092614+0000","last_unstale":"2024-06-28T16:11:25.561434+0000","last_undegraded":"2024-06-28T16:11:25.561434+0000","last_fullsized":"2024-06-28T16:11:25.561434+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:43:32.181244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"257'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.092406+0000","last_change":"2024-06-28T15:21:07.035627+0000","last_active":"2024-06-28T16:11:10.092406+0000","last_peered":"2024-06-28T16:11:10.092406+0000","last_clean":"2024-06-28T16:11:10.092406+0000","last_became_active":"2024-06-28T15:21:07.034845+0000","last_became_peered":"2024-06-28T15:21:07.034845+0000","last_unstale":"2024-06-28T16:11:10.092406+0000","last_undegraded":"2024-06-28T16:11:10.092406+0000","last_fullsized":"2024-06-28T16:11:10.092406+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:39:02.467227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"260'28","reported_seq":139,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.563793+0000","last_change":"2024-06-28T15:17:24.016960+0000","last_active":"2024-06-28T16:11:35.563793+0000","last_peered":"2024-06-28T16:11:35.563793+0000","last_clean":"2024-06-28T16:11:35.563793+0000","last_became_active":"2024-06-28T15:17:24.014034+0000","last_became_peered":"2024-06-28T15:17:24.014034+0000","last_unstale":"2024-06-28T16:11:35.563793+0000","last_undegraded":"2024-06-28T16:11:35.563793+0000","last_fullsized":"2024-06-28T16:11:35.563793+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:12:33.973492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":149,"num_read_kb":105,"num_write":55,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.074736+0000","last_change":"2024-06-28T15:37:33.340550+0000","last_active":"2024-06-28T16:10:55.074736+0000","last_peered":"2024-06-28T16:10:55.074736+0000","last_clean":"2024-06-28T16:10:55.074736+0000","last_became_active":"2024-06-28T15:37:33.339831+0000","last_became_peered":"2024-06-28T15:37:33.339831+0000","last_unstale":"2024-06-28T16:10:55.074736+0000","last_undegraded":"2024-06-28T16:10:55.074736+0000","last_fullsized":"2024-06-28T16:10:55.074736+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:10:11.303905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"260'6","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.558788+0000","last_change":"2024-06-28T15:21:11.096084+0000","last_active":"2024-06-28T16:11:15.558788+0000","last_peered":"2024-06-28T16:11:15.558788+0000","last_clean":"2024-06-28T16:11:15.558788+0000","last_became_active":"2024-06-28T15:21:11.088011+0000","last_became_peered":"2024-06-28T15:21:11.088011+0000","last_unstale":"2024-06-28T16:11:15.558788+0000","last_undegraded":"2024-06-28T16:11:15.558788+0000","last_fullsized":"2024-06-28T16:11:15.558788+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:14:39.791560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5283025,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":20522,"num_write":10,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"0'0","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.268746+0000","last_change":"2024-06-28T15:21:07.031400+0000","last_active":"2024-06-28T16:11:25.268746+0000","last_peered":"2024-06-28T16:11:25.268746+0000","last_clean":"2024-06-28T16:11:25.268746+0000","last_became_active":"2024-06-28T15:21:07.030690+0000","last_became_peered":"2024-06-28T15:21:07.030690+0000","last_unstale":"2024-06-28T16:11:25.268746+0000","last_undegraded":"2024-06-28T16:11:25.268746+0000","last_fullsized":"2024-06-28T16:11:25.268746+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:42:25.816657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"260'31","reported_seq":144,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089847+0000","last_change":"2024-06-28T15:17:24.026986+0000","last_active":"2024-06-28T16:11:35.089847+0000","last_peered":"2024-06-28T16:11:35.089847+0000","last_clean":"2024-06-28T16:11:35.089847+0000","last_became_active":"2024-06-28T15:17:24.026404+0000","last_became_peered":"2024-06-28T15:17:24.026404+0000","last_unstale":"2024-06-28T16:11:35.089847+0000","last_undegraded":"2024-06-28T16:11:35.089847+0000","last_fullsized":"2024-06-28T16:11:35.089847+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:08:40.750828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":161,"num_read_kb":111,"num_write":59,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:45.101310+0000","last_change":"2024-06-28T15:37:33.332984+0000","last_active":"2024-06-28T16:11:45.101310+0000","last_peered":"2024-06-28T16:11:45.101310+0000","last_clean":"2024-06-28T16:11:45.101310+0000","last_became_active":"2024-06-28T15:37:33.332245+0000","last_became_peered":"2024-06-28T15:37:33.332245+0000","last_unstale":"2024-06-28T16:11:45.101310+0000","last_undegraded":"2024-06-28T16:11:45.101310+0000","last_fullsized":"2024-06-28T16:11:45.101310+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:30.067085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.090010+0000","last_change":"2024-06-28T15:21:11.097568+0000","last_active":"2024-06-28T16:11:00.090010+0000","last_peered":"2024-06-28T16:11:00.090010+0000","last_clean":"2024-06-28T16:11:00.090010+0000","last_became_active":"2024-06-28T15:21:11.096926+0000","last_became_peered":"2024-06-28T15:21:11.096926+0000","last_unstale":"2024-06-28T16:11:00.090010+0000","last_undegraded":"2024-06-28T16:11:00.090010+0000","last_fullsized":"2024-06-28T16:11:00.090010+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:33:16.290705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"260'2","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.105821+0000","last_change":"2024-06-28T15:21:07.037522+0000","last_active":"2024-06-28T16:13:10.105821+0000","last_peered":"2024-06-28T16:13:10.105821+0000","last_clean":"2024-06-28T16:13:10.105821+0000","last_became_active":"2024-06-28T15:21:07.036735+0000","last_became_peered":"2024-06-28T15:21:07.036735+0000","last_unstale":"2024-06-28T16:13:10.105821+0000","last_undegraded":"2024-06-28T16:13:10.105821+0000","last_fullsized":"2024-06-28T16:13:10.105821+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:56:53.756000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"260'49","reported_seq":197,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.101490+0000","last_change":"2024-06-28T15:17:24.026580+0000","last_active":"2024-06-28T16:11:50.101490+0000","last_peered":"2024-06-28T16:11:50.101490+0000","last_clean":"2024-06-28T16:11:50.101490+0000","last_became_active":"2024-06-28T15:17:24.025855+0000","last_became_peered":"2024-06-28T15:17:24.025855+0000","last_unstale":"2024-06-28T16:11:50.101490+0000","last_undegraded":"2024-06-28T16:11:50.101490+0000","last_fullsized":"2024-06-28T16:11:50.101490+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:14:11.100104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":190,"num_write":87,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.081854+0000","last_change":"2024-06-28T15:37:33.323701+0000","last_active":"2024-06-28T16:10:55.081854+0000","last_peered":"2024-06-28T16:10:55.081854+0000","last_clean":"2024-06-28T16:10:55.081854+0000","last_became_active":"2024-06-28T15:37:33.322927+0000","last_became_peered":"2024-06-28T15:37:33.322927+0000","last_unstale":"2024-06-28T16:10:55.081854+0000","last_undegraded":"2024-06-28T16:10:55.081854+0000","last_fullsized":"2024-06-28T16:10:55.081854+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:33:37.474011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.084125+0000","last_change":"2024-06-28T15:21:11.098521+0000","last_active":"2024-06-28T16:11:05.084125+0000","last_peered":"2024-06-28T16:11:05.084125+0000","last_clean":"2024-06-28T16:11:05.084125+0000","last_became_active":"2024-06-28T15:21:11.097766+0000","last_became_peered":"2024-06-28T15:21:11.097766+0000","last_unstale":"2024-06-28T16:11:05.084125+0000","last_undegraded":"2024-06-28T16:11:05.084125+0000","last_fullsized":"2024-06-28T16:11:05.084125+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:49:28.598405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.090053+0000","last_change":"2024-06-28T15:21:07.040142+0000","last_active":"2024-06-28T16:11:00.090053+0000","last_peered":"2024-06-28T16:11:00.090053+0000","last_clean":"2024-06-28T16:11:00.090053+0000","last_became_active":"2024-06-28T15:21:07.039381+0000","last_became_peered":"2024-06-28T15:21:07.039381+0000","last_unstale":"2024-06-28T16:11:00.090053+0000","last_undegraded":"2024-06-28T16:11:00.090053+0000","last_fullsized":"2024-06-28T16:11:00.090053+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:56:40.727136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"260'26","reported_seq":137,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.099274+0000","last_change":"2024-06-28T15:17:24.027314+0000","last_active":"2024-06-28T16:11:35.099274+0000","last_peered":"2024-06-28T16:11:35.099274+0000","last_clean":"2024-06-28T16:11:35.099274+0000","last_became_active":"2024-06-28T15:17:24.026580+0000","last_became_peered":"2024-06-28T15:17:24.026580+0000","last_unstale":"2024-06-28T16:11:35.099274+0000","last_undegraded":"2024-06-28T16:11:35.099274+0000","last_fullsized":"2024-06-28T16:11:35.099274+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:45:13.623723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":143,"num_read_kb":101,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"259'1","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.278572+0000","last_change":"2024-06-28T15:37:33.349810+0000","last_active":"2024-06-28T16:12:15.278572+0000","last_peered":"2024-06-28T16:12:15.278572+0000","last_clean":"2024-06-28T16:12:15.278572+0000","last_became_active":"2024-06-28T15:37:33.349137+0000","last_became_peered":"2024-06-28T15:37:33.349137+0000","last_unstale":"2024-06-28T16:12:15.278572+0000","last_undegraded":"2024-06-28T16:12:15.278572+0000","last_fullsized":"2024-06-28T16:12:15.278572+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:47:15.998763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"260'6","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.514776+0000","last_change":"2024-06-28T15:21:11.106004+0000","last_active":"2024-06-28T16:14:32.514776+0000","last_peered":"2024-06-28T16:14:32.514776+0000","last_clean":"2024-06-28T16:14:32.514776+0000","last_became_active":"2024-06-28T15:21:11.104270+0000","last_became_peered":"2024-06-28T15:21:11.104270+0000","last_unstale":"2024-06-28T16:14:32.514776+0000","last_undegraded":"2024-06-28T16:14:32.514776+0000","last_fullsized":"2024-06-28T16:14:32.514776+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:22:50.569140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":11,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"260'6","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.311951+0000","last_change":"2024-06-28T15:21:07.049332+0000","last_active":"2024-06-28T16:14:32.311951+0000","last_peered":"2024-06-28T16:14:32.311951+0000","last_clean":"2024-06-28T16:14:32.311951+0000","last_became_active":"2024-06-28T15:21:07.048639+0000","last_became_peered":"2024-06-28T15:21:07.048639+0000","last_unstale":"2024-06-28T16:14:32.311951+0000","last_undegraded":"2024-06-28T16:14:32.311951+0000","last_fullsized":"2024-06-28T16:14:32.311951+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:02:29.473659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":12288,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"260'22","reported_seq":114,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.083766+0000","last_change":"2024-06-28T15:17:24.007520+0000","last_active":"2024-06-28T16:11:35.083766+0000","last_peered":"2024-06-28T16:11:35.083766+0000","last_clean":"2024-06-28T16:11:35.083766+0000","last_became_active":"2024-06-28T15:17:24.006784+0000","last_became_peered":"2024-06-28T15:17:24.006784+0000","last_unstale":"2024-06-28T16:11:35.083766+0000","last_undegraded":"2024-06-28T16:11:35.083766+0000","last_fullsized":"2024-06-28T16:11:35.083766+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:31:01.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":64,"num_write":51,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.574119+0000","last_change":"2024-06-28T15:37:33.334905+0000","last_active":"2024-06-28T16:12:15.574119+0000","last_peered":"2024-06-28T16:12:15.574119+0000","last_clean":"2024-06-28T16:12:15.574119+0000","last_became_active":"2024-06-28T15:37:33.333737+0000","last_became_peered":"2024-06-28T15:37:33.333737+0000","last_unstale":"2024-06-28T16:12:15.574119+0000","last_undegraded":"2024-06-28T16:12:15.574119+0000","last_fullsized":"2024-06-28T16:12:15.574119+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:02:20.700941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":48,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.081414+0000","last_change":"2024-06-28T15:21:11.092678+0000","last_active":"2024-06-28T16:11:25.081414+0000","last_peered":"2024-06-28T16:11:25.081414+0000","last_clean":"2024-06-28T16:11:25.081414+0000","last_became_active":"2024-06-28T15:21:11.092034+0000","last_became_peered":"2024-06-28T15:21:11.092034+0000","last_unstale":"2024-06-28T16:11:25.081414+0000","last_undegraded":"2024-06-28T16:11:25.081414+0000","last_fullsized":"2024-06-28T16:11:25.081414+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:08:58.423355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"260'3","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.120874+0000","last_change":"2024-06-28T15:21:07.040425+0000","last_active":"2024-06-28T16:13:15.120874+0000","last_peered":"2024-06-28T16:13:15.120874+0000","last_clean":"2024-06-28T16:13:15.120874+0000","last_became_active":"2024-06-28T15:21:07.039812+0000","last_became_peered":"2024-06-28T15:21:07.039812+0000","last_unstale":"2024-06-28T16:13:15.120874+0000","last_undegraded":"2024-06-28T16:13:15.120874+0000","last_fullsized":"2024-06-28T16:13:15.120874+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:50:26.224970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"260'77","reported_seq":307,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.090583+0000","last_change":"2024-06-28T15:17:24.013637+0000","last_active":"2024-06-28T16:11:35.090583+0000","last_peered":"2024-06-28T16:11:35.090583+0000","last_clean":"2024-06-28T16:11:35.090583+0000","last_became_active":"2024-06-28T15:17:24.012854+0000","last_became_peered":"2024-06-28T15:17:24.012854+0000","last_unstale":"2024-06-28T16:11:35.090583+0000","last_undegraded":"2024-06-28T16:11:35.090583+0000","last_fullsized":"2024-06-28T16:11:35.090583+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:10:40.982270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":483,"num_read_kb":330,"num_write":109,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.554385+0000","last_change":"2024-06-28T15:37:33.323089+0000","last_active":"2024-06-28T16:10:55.554385+0000","last_peered":"2024-06-28T16:10:55.554385+0000","last_clean":"2024-06-28T16:10:55.554385+0000","last_became_active":"2024-06-28T15:37:33.322317+0000","last_became_peered":"2024-06-28T15:37:33.322317+0000","last_unstale":"2024-06-28T16:10:55.554385+0000","last_undegraded":"2024-06-28T16:10:55.554385+0000","last_fullsized":"2024-06-28T16:10:55.554385+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:18:03.061734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.085392+0000","last_change":"2024-06-28T15:21:11.109416+0000","last_active":"2024-06-28T16:11:15.085392+0000","last_peered":"2024-06-28T16:11:15.085392+0000","last_clean":"2024-06-28T16:11:15.085392+0000","last_became_active":"2024-06-28T15:21:11.108663+0000","last_became_peered":"2024-06-28T15:21:11.108663+0000","last_unstale":"2024-06-28T16:11:15.085392+0000","last_undegraded":"2024-06-28T16:11:15.085392+0000","last_fullsized":"2024-06-28T16:11:15.085392+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:07:02.854303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"260'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.121849+0000","last_change":"2024-06-28T15:21:07.022794+0000","last_active":"2024-06-28T16:13:15.121849+0000","last_peered":"2024-06-28T16:13:15.121849+0000","last_clean":"2024-06-28T16:13:15.121849+0000","last_became_active":"2024-06-28T15:21:07.022046+0000","last_became_peered":"2024-06-28T15:21:07.022046+0000","last_unstale":"2024-06-28T16:13:15.121849+0000","last_undegraded":"2024-06-28T16:13:15.121849+0000","last_fullsized":"2024-06-28T16:13:15.121849+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:49:32.800553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"260'31","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089684+0000","last_change":"2024-06-28T15:17:24.011733+0000","last_active":"2024-06-28T16:11:35.089684+0000","last_peered":"2024-06-28T16:11:35.089684+0000","last_clean":"2024-06-28T16:11:35.089684+0000","last_became_active":"2024-06-28T15:17:24.010873+0000","last_became_peered":"2024-06-28T15:17:24.010873+0000","last_unstale":"2024-06-28T16:11:35.089684+0000","last_undegraded":"2024-06-28T16:11:35.089684+0000","last_fullsized":"2024-06-28T16:11:35.089684+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:29:45.592863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":94,"num_write":78,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.091280+0000","last_change":"2024-06-28T15:37:33.347898+0000","last_active":"2024-06-28T16:11:05.091280+0000","last_peered":"2024-06-28T16:11:05.091280+0000","last_clean":"2024-06-28T16:11:05.091280+0000","last_became_active":"2024-06-28T15:37:33.347075+0000","last_became_peered":"2024-06-28T15:37:33.347075+0000","last_unstale":"2024-06-28T16:11:05.091280+0000","last_undegraded":"2024-06-28T16:11:05.091280+0000","last_fullsized":"2024-06-28T16:11:05.091280+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:36:09.889868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":47,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.086272+0000","last_change":"2024-06-28T15:21:11.093738+0000","last_active":"2024-06-28T16:11:15.086272+0000","last_peered":"2024-06-28T16:11:15.086272+0000","last_clean":"2024-06-28T16:11:15.086272+0000","last_became_active":"2024-06-28T15:21:11.093082+0000","last_became_peered":"2024-06-28T15:21:11.093082+0000","last_unstale":"2024-06-28T16:11:15.086272+0000","last_undegraded":"2024-06-28T16:11:15.086272+0000","last_fullsized":"2024-06-28T16:11:15.086272+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:58:56.828729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"257'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.084078+0000","last_change":"2024-06-28T15:21:07.041597+0000","last_active":"2024-06-28T16:11:10.084078+0000","last_peered":"2024-06-28T16:11:10.084078+0000","last_clean":"2024-06-28T16:11:10.084078+0000","last_became_active":"2024-06-28T15:21:07.040575+0000","last_became_peered":"2024-06-28T15:21:07.040575+0000","last_unstale":"2024-06-28T16:11:10.084078+0000","last_undegraded":"2024-06-28T16:11:10.084078+0000","last_fullsized":"2024-06-28T16:11:10.084078+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:41:33.611919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"260'16","reported_seq":98,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:40.099162+0000","last_change":"2024-06-28T15:17:24.024775+0000","last_active":"2024-06-28T16:11:40.099162+0000","last_peered":"2024-06-28T16:11:40.099162+0000","last_clean":"2024-06-28T16:11:40.099162+0000","last_became_active":"2024-06-28T15:17:24.021464+0000","last_became_peered":"2024-06-28T15:17:24.021464+0000","last_unstale":"2024-06-28T16:11:40.099162+0000","last_undegraded":"2024-06-28T16:11:40.099162+0000","last_fullsized":"2024-06-28T16:11:40.099162+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:33:59.157965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":44,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"260'341","reported_seq":962,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.645969+0000","last_change":"2024-06-28T15:00:59.944074+0000","last_active":"2024-06-28T16:14:34.645969+0000","last_peered":"2024-06-28T16:14:34.645969+0000","last_clean":"2024-06-28T16:14:34.645969+0000","last_became_active":"2024-06-28T15:00:59.941681+0000","last_became_peered":"2024-06-28T15:00:59.941681+0000","last_unstale":"2024-06-28T16:14:34.645969+0000","last_undegraded":"2024-06-28T16:14:34.645969+0000","last_fullsized":"2024-06-28T16:14:34.645969+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:10:45.311266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170282,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":488,"num_read_kb":26898,"num_write":1268,"num_write_kb":25985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0,5],"acting":[2,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"260'181","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.066576+0000","last_change":"2024-06-28T15:00:17.405026+0000","last_active":"2024-06-28T16:10:20.066576+0000","last_peered":"2024-06-28T16:10:20.066576+0000","last_clean":"2024-06-28T16:10:20.066576+0000","last_became_active":"2024-06-28T15:00:17.404342+0000","last_became_peered":"2024-06-28T15:00:17.404342+0000","last_unstale":"2024-06-28T16:10:20.066576+0000","last_undegraded":"2024-06-28T16:10:20.066576+0000","last_fullsized":"2024-06-28T16:10:20.066576+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":100,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:24:42.574899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":722,"num_read_kb":569,"num_write":267,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"260'326","reported_seq":971,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.742875+0000","last_change":"2024-06-28T15:00:15.419867+0000","last_active":"2024-06-28T16:14:34.742875+0000","last_peered":"2024-06-28T16:14:34.742875+0000","last_clean":"2024-06-28T16:14:34.742875+0000","last_became_active":"2024-06-28T15:00:15.419449+0000","last_became_peered":"2024-06-28T15:00:15.419449+0000","last_unstale":"2024-06-28T16:14:34.742875+0000","last_undegraded":"2024-06-28T16:14:34.742875+0000","last_fullsized":"2024-06-28T16:14:34.742875+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:27:48.933532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5526738,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":485,"num_read_kb":15789,"num_write":1172,"num_write_kb":37304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"260'285","reported_seq":939,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080422+0000","last_change":"2024-06-28T15:00:17.409108+0000","last_active":"2024-06-28T16:10:20.080422+0000","last_peered":"2024-06-28T16:10:20.080422+0000","last_clean":"2024-06-28T16:10:20.080422+0000","last_became_active":"2024-06-28T15:00:17.408128+0000","last_became_peered":"2024-06-28T15:00:17.408128+0000","last_unstale":"2024-06-28T16:10:20.080422+0000","last_undegraded":"2024-06-28T16:10:20.080422+0000","last_fullsized":"2024-06-28T16:10:20.080422+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:38:12.539083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1157,"num_read_kb":911,"num_write":439,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"260'221","reported_seq":907,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.261930+0000","last_change":"2024-06-28T15:00:15.397113+0000","last_active":"2024-06-28T16:11:55.261930+0000","last_peered":"2024-06-28T16:11:55.261930+0000","last_clean":"2024-06-28T16:11:55.261930+0000","last_became_active":"2024-06-28T15:00:15.396469+0000","last_became_peered":"2024-06-28T15:00:15.396469+0000","last_unstale":"2024-06-28T16:11:55.261930+0000","last_undegraded":"2024-06-28T16:11:55.261930+0000","last_fullsized":"2024-06-28T16:11:55.261930+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:40:36.549494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1588322,"num_objects":66,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":442,"num_read_kb":81435,"num_write":1047,"num_write_kb":14924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"260'77","reported_seq":643,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073900+0000","last_change":"2024-06-28T15:00:17.403326+0000","last_active":"2024-06-28T16:10:20.073900+0000","last_peered":"2024-06-28T16:10:20.073900+0000","last_clean":"2024-06-28T16:10:20.073900+0000","last_became_active":"2024-06-28T15:00:17.402542+0000","last_became_peered":"2024-06-28T15:00:17.402542+0000","last_unstale":"2024-06-28T16:10:20.073900+0000","last_undegraded":"2024-06-28T16:10:20.073900+0000","last_fullsized":"2024-06-28T16:10:20.073900+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:05:41.791423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":306,"num_read_kb":248,"num_write":126,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"260'313","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.510107+0000","last_change":"2024-06-28T15:00:15.435754+0000","last_active":"2024-06-28T16:14:34.510107+0000","last_peered":"2024-06-28T16:14:34.510107+0000","last_clean":"2024-06-28T16:14:34.510107+0000","last_became_active":"2024-06-28T15:00:15.435122+0000","last_became_peered":"2024-06-28T15:00:15.435122+0000","last_unstale":"2024-06-28T16:14:34.510107+0000","last_undegraded":"2024-06-28T16:14:34.510107+0000","last_fullsized":"2024-06-28T16:14:34.510107+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:23:37.223417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1454317,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":492,"num_read_kb":48678,"num_write":1275,"num_write_kb":34259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"260'51","reported_seq":629,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.546829+0000","last_change":"2024-06-28T15:00:17.385309+0000","last_active":"2024-06-28T16:10:20.546829+0000","last_peered":"2024-06-28T16:10:20.546829+0000","last_clean":"2024-06-28T16:10:20.546829+0000","last_became_active":"2024-06-28T15:00:17.384932+0000","last_became_peered":"2024-06-28T15:00:17.384932+0000","last_unstale":"2024-06-28T16:10:20.546829+0000","last_undegraded":"2024-06-28T16:10:20.546829+0000","last_fullsized":"2024-06-28T16:10:20.546829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:12:11.868213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":197,"num_write":67,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"260'343","reported_seq":992,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.679630+0000","last_change":"2024-06-28T15:00:15.430784+0000","last_active":"2024-06-28T16:14:34.679630+0000","last_peered":"2024-06-28T16:14:34.679630+0000","last_clean":"2024-06-28T16:14:34.679630+0000","last_became_active":"2024-06-28T15:00:15.430208+0000","last_became_peered":"2024-06-28T15:00:15.430208+0000","last_unstale":"2024-06-28T16:14:34.679630+0000","last_undegraded":"2024-06-28T16:14:34.679630+0000","last_fullsized":"2024-06-28T16:14:34.679630+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:48:53.859374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437780,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":548,"num_read_kb":39482,"num_write":1215,"num_write_kb":35176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"260'185","reported_seq":837,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:59.271179+0000","last_change":"2024-06-28T15:00:17.398844+0000","last_active":"2024-06-28T16:10:59.271179+0000","last_peered":"2024-06-28T16:10:59.271179+0000","last_clean":"2024-06-28T16:10:59.271179+0000","last_became_active":"2024-06-28T15:00:17.397986+0000","last_became_peered":"2024-06-28T15:00:17.397986+0000","last_unstale":"2024-06-28T16:10:59.271179+0000","last_undegraded":"2024-06-28T16:10:59.271179+0000","last_fullsized":"2024-06-28T16:10:59.271179+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:49:08.729457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":777,"num_read_kb":632,"num_write":274,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"260'364","reported_seq":1029,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.692915+0000","last_change":"2024-06-28T15:00:15.412175+0000","last_active":"2024-06-28T16:14:34.692915+0000","last_peered":"2024-06-28T16:14:34.692915+0000","last_clean":"2024-06-28T16:14:34.692915+0000","last_became_active":"2024-06-28T15:00:15.409542+0000","last_became_peered":"2024-06-28T15:00:15.409542+0000","last_unstale":"2024-06-28T16:14:34.692915+0000","last_undegraded":"2024-06-28T16:14:34.692915+0000","last_fullsized":"2024-06-28T16:14:34.692915+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:28:00.331354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1667748,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":553,"num_read_kb":16754,"num_write":1370,"num_write_kb":32794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"260'44","reported_seq":608,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.895392+0000","last_change":"2024-06-28T15:00:17.408359+0000","last_active":"2024-06-28T16:11:10.895392+0000","last_peered":"2024-06-28T16:11:10.895392+0000","last_clean":"2024-06-28T16:11:10.895392+0000","last_became_active":"2024-06-28T15:00:17.401985+0000","last_became_peered":"2024-06-28T15:00:17.401985+0000","last_unstale":"2024-06-28T16:11:10.895392+0000","last_undegraded":"2024-06-28T16:11:10.895392+0000","last_fullsized":"2024-06-28T16:11:10.895392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:00:57.912525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":159,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"260'328","reported_seq":959,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.675966+0000","last_change":"2024-06-28T15:00:15.425314+0000","last_active":"2024-06-28T16:14:34.675966+0000","last_peered":"2024-06-28T16:14:34.675966+0000","last_clean":"2024-06-28T16:14:34.675966+0000","last_became_active":"2024-06-28T15:00:15.424430+0000","last_became_peered":"2024-06-28T15:00:15.424430+0000","last_unstale":"2024-06-28T16:14:34.675966+0000","last_undegraded":"2024-06-28T16:14:34.675966+0000","last_fullsized":"2024-06-28T16:14:34.675966+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:30:59.580102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048602,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":476,"num_read_kb":21908,"num_write":1069,"num_write_kb":36265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"260'4326","reported_seq":4996,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:30.604647+0000","last_change":"2024-06-28T15:00:17.387801+0000","last_active":"2024-06-28T16:12:30.604647+0000","last_peered":"2024-06-28T16:12:30.604647+0000","last_clean":"2024-06-28T16:12:30.604647+0000","last_became_active":"2024-06-28T15:00:17.384378+0000","last_became_peered":"2024-06-28T15:00:17.384378+0000","last_unstale":"2024-06-28T16:12:30.604647+0000","last_undegraded":"2024-06-28T16:12:30.604647+0000","last_fullsized":"2024-06-28T16:12:30.604647+0000","mapping_epoch":21,"log_start":"260'4300","ondisk_log_start":"260'4300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":3000,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:59:21.316239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15352,"num_read_kb":12408,"num_write":6557,"num_write_kb":3277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"260'411","reported_seq":1128,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.664061+0000","last_change":"2024-06-28T15:00:15.403350+0000","last_active":"2024-06-28T16:14:34.664061+0000","last_peered":"2024-06-28T16:14:34.664061+0000","last_clean":"2024-06-28T16:14:34.664061+0000","last_became_active":"2024-06-28T15:00:15.401960+0000","last_became_peered":"2024-06-28T15:00:15.401960+0000","last_unstale":"2024-06-28T16:14:34.664061+0000","last_undegraded":"2024-06-28T16:14:34.664061+0000","last_fullsized":"2024-06-28T16:14:34.664061+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":400,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:02:13.643574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324665,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":810,"num_read_kb":20615,"num_write":1466,"num_write_kb":29750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"260'70","reported_seq":685,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.247513+0000","last_change":"2024-06-28T15:00:17.381546+0000","last_active":"2024-06-28T16:14:08.247513+0000","last_peered":"2024-06-28T16:14:08.247513+0000","last_clean":"2024-06-28T16:14:08.247513+0000","last_became_active":"2024-06-28T15:00:17.381196+0000","last_became_peered":"2024-06-28T15:00:17.381196+0000","last_unstale":"2024-06-28T16:14:08.247513+0000","last_undegraded":"2024-06-28T16:14:08.247513+0000","last_fullsized":"2024-06-28T16:14:08.247513+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:47:51.139728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":309,"num_read_kb":273,"num_write":88,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"260'128","reported_seq":750,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:32.095965+0000","last_change":"2024-06-28T15:00:17.396995+0000","last_active":"2024-06-28T16:13:32.095965+0000","last_peered":"2024-06-28T16:13:32.095965+0000","last_clean":"2024-06-28T16:13:32.095965+0000","last_became_active":"2024-06-28T15:00:17.396142+0000","last_became_peered":"2024-06-28T15:00:17.396142+0000","last_unstale":"2024-06-28T16:13:32.095965+0000","last_undegraded":"2024-06-28T16:13:32.095965+0000","last_fullsized":"2024-06-28T16:13:32.095965+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:43:00.829791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":536,"num_read_kb":452,"num_write":191,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"260'416","reported_seq":1186,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.679447+0000","last_change":"2024-06-28T15:00:15.423248+0000","last_active":"2024-06-28T16:14:34.679447+0000","last_peered":"2024-06-28T16:14:34.679447+0000","last_clean":"2024-06-28T16:14:34.679447+0000","last_became_active":"2024-06-28T15:00:15.422579+0000","last_became_peered":"2024-06-28T15:00:15.422579+0000","last_unstale":"2024-06-28T16:14:34.679447+0000","last_undegraded":"2024-06-28T16:14:34.679447+0000","last_fullsized":"2024-06-28T16:14:34.679447+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":400,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:59:30.311378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":851,"num_read_kb":96606,"num_write":1464,"num_write_kb":38692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0,1],"acting":[7,5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"260'254","reported_seq":899,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080457+0000","last_change":"2024-06-28T15:00:17.410347+0000","last_active":"2024-06-28T16:10:20.080457+0000","last_peered":"2024-06-28T16:10:20.080457+0000","last_clean":"2024-06-28T16:10:20.080457+0000","last_became_active":"2024-06-28T15:00:17.409202+0000","last_became_peered":"2024-06-28T15:00:17.409202+0000","last_unstale":"2024-06-28T16:10:20.080457+0000","last_undegraded":"2024-06-28T16:10:20.080457+0000","last_fullsized":"2024-06-28T16:10:20.080457+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:33:21.410367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1059,"num_read_kb":851,"num_write":432,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"260'313","reported_seq":952,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.711448+0000","last_change":"2024-06-28T15:00:15.423492+0000","last_active":"2024-06-28T16:14:34.711448+0000","last_peered":"2024-06-28T16:14:34.711448+0000","last_clean":"2024-06-28T16:14:34.711448+0000","last_became_active":"2024-06-28T15:00:15.421980+0000","last_became_peered":"2024-06-28T15:00:15.421980+0000","last_unstale":"2024-06-28T16:14:34.711448+0000","last_undegraded":"2024-06-28T16:14:34.711448+0000","last_fullsized":"2024-06-28T16:14:34.711448+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:15:16.139016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468838,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":425,"num_read_kb":33937,"num_write":1026,"num_write_kb":43346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"260'225","reported_seq":850,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.070101+0000","last_change":"2024-06-28T15:00:17.385629+0000","last_active":"2024-06-28T16:10:35.070101+0000","last_peered":"2024-06-28T16:10:35.070101+0000","last_clean":"2024-06-28T16:10:35.070101+0000","last_became_active":"2024-06-28T15:00:17.384914+0000","last_became_peered":"2024-06-28T15:00:17.384914+0000","last_unstale":"2024-06-28T16:10:35.070101+0000","last_undegraded":"2024-06-28T16:10:35.070101+0000","last_fullsized":"2024-06-28T16:10:35.070101+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:01:45.177989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":887,"num_read_kb":706,"num_write":342,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"260'342","reported_seq":1191,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.723544+0000","last_change":"2024-06-28T15:00:15.412640+0000","last_active":"2024-06-28T16:14:34.723544+0000","last_peered":"2024-06-28T16:14:34.723544+0000","last_clean":"2024-06-28T16:14:34.723544+0000","last_became_active":"2024-06-28T15:00:15.411567+0000","last_became_peered":"2024-06-28T15:00:15.411567+0000","last_unstale":"2024-06-28T16:14:34.723544+0000","last_undegraded":"2024-06-28T16:14:34.723544+0000","last_fullsized":"2024-06-28T16:14:34.723544+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:32:11.605469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289237,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":936,"num_read_kb":14303,"num_write":1297,"num_write_kb":46873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"260'128","reported_seq":736,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:51.375718+0000","last_change":"2024-06-28T15:00:17.389926+0000","last_active":"2024-06-28T16:11:51.375718+0000","last_peered":"2024-06-28T16:11:51.375718+0000","last_clean":"2024-06-28T16:11:51.375718+0000","last_became_active":"2024-06-28T15:00:17.389613+0000","last_became_peered":"2024-06-28T15:00:17.389613+0000","last_unstale":"2024-06-28T16:11:51.375718+0000","last_undegraded":"2024-06-28T16:11:51.375718+0000","last_fullsized":"2024-06-28T16:11:51.375718+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:46:13.479655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":419,"num_write":192,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"260'320","reported_seq":1072,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.719218+0000","last_change":"2024-06-28T15:00:15.418207+0000","last_active":"2024-06-28T16:14:34.719218+0000","last_peered":"2024-06-28T16:14:34.719218+0000","last_clean":"2024-06-28T16:14:34.719218+0000","last_became_active":"2024-06-28T15:00:15.417563+0000","last_became_peered":"2024-06-28T15:00:15.417563+0000","last_unstale":"2024-06-28T16:14:34.719218+0000","last_undegraded":"2024-06-28T16:14:34.719218+0000","last_fullsized":"2024-06-28T16:14:34.719218+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:50:35.799169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":525349,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":605,"num_read_kb":12860,"num_write":1159,"num_write_kb":18563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"260'156","reported_seq":737,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.256777+0000","last_change":"2024-06-28T15:00:17.413709+0000","last_active":"2024-06-28T16:10:25.256777+0000","last_peered":"2024-06-28T16:10:25.256777+0000","last_clean":"2024-06-28T16:10:25.256777+0000","last_became_active":"2024-06-28T15:00:17.413420+0000","last_became_peered":"2024-06-28T15:00:17.413420+0000","last_unstale":"2024-06-28T16:10:25.256777+0000","last_undegraded":"2024-06-28T16:10:25.256777+0000","last_fullsized":"2024-06-28T16:10:25.256777+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:10:02.399082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":589,"num_read_kb":466,"num_write":222,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"260'357","reported_seq":1073,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.554726+0000","last_change":"2024-06-28T15:00:15.402501+0000","last_active":"2024-06-28T16:14:34.554726+0000","last_peered":"2024-06-28T16:14:34.554726+0000","last_clean":"2024-06-28T16:14:34.554726+0000","last_became_active":"2024-06-28T15:00:15.401370+0000","last_became_peered":"2024-06-28T15:00:15.401370+0000","last_unstale":"2024-06-28T16:14:34.554726+0000","last_undegraded":"2024-06-28T16:14:34.554726+0000","last_fullsized":"2024-06-28T16:14:34.554726+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:40:25.381283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324689,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":676,"num_read_kb":30262,"num_write":1536,"num_write_kb":36456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"260'253","reported_seq":902,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.053742+0000","last_change":"2024-06-28T15:00:17.410230+0000","last_active":"2024-06-28T16:13:20.053742+0000","last_peered":"2024-06-28T16:13:20.053742+0000","last_clean":"2024-06-28T16:13:20.053742+0000","last_became_active":"2024-06-28T15:00:17.408972+0000","last_became_peered":"2024-06-28T15:00:17.408972+0000","last_unstale":"2024-06-28T16:13:20.053742+0000","last_undegraded":"2024-06-28T16:13:20.053742+0000","last_fullsized":"2024-06-28T16:13:20.053742+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:48:01.967992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":990,"num_read_kb":787,"num_write":391,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"260'286","reported_seq":948,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.590888+0000","last_change":"2024-06-28T15:00:15.408546+0000","last_active":"2024-06-28T16:13:10.590888+0000","last_peered":"2024-06-28T16:13:10.590888+0000","last_clean":"2024-06-28T16:13:10.590888+0000","last_became_active":"2024-06-28T15:00:15.407552+0000","last_became_peered":"2024-06-28T15:00:15.407552+0000","last_unstale":"2024-06-28T16:13:10.590888+0000","last_undegraded":"2024-06-28T16:13:10.590888+0000","last_fullsized":"2024-06-28T16:13:10.590888+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:35:44.908220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12818085,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":453,"num_read_kb":23448,"num_write":1276,"num_write_kb":46171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"260'142","reported_seq":679,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.510437+0000","last_change":"2024-06-28T15:00:25.428551+0000","last_active":"2024-06-28T16:14:08.510437+0000","last_peered":"2024-06-28T16:14:08.510437+0000","last_clean":"2024-06-28T16:14:08.510437+0000","last_became_active":"2024-06-28T15:00:25.427734+0000","last_became_peered":"2024-06-28T15:00:25.427734+0000","last_unstale":"2024-06-28T16:14:08.510437+0000","last_undegraded":"2024-06-28T16:14:08.510437+0000","last_fullsized":"2024-06-28T16:14:08.510437+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:59:38.479139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":233,"num_read_kb":201,"num_write":347,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"260'214","reported_seq":822,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493503+0000","last_change":"2024-06-28T15:00:17.382620+0000","last_active":"2024-06-28T16:10:25.493503+0000","last_peered":"2024-06-28T16:10:25.493503+0000","last_clean":"2024-06-28T16:10:25.493503+0000","last_became_active":"2024-06-28T15:00:17.382344+0000","last_became_peered":"2024-06-28T15:00:17.382344+0000","last_unstale":"2024-06-28T16:10:25.493503+0000","last_undegraded":"2024-06-28T16:10:25.493503+0000","last_fullsized":"2024-06-28T16:10:25.493503+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":200,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:40:38.570590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":869,"num_read_kb":692,"num_write":351,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"260'292","reported_seq":944,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.632189+0000","last_change":"2024-06-28T15:00:15.423784+0000","last_active":"2024-06-28T16:14:34.632189+0000","last_peered":"2024-06-28T16:14:34.632189+0000","last_clean":"2024-06-28T16:14:34.632189+0000","last_became_active":"2024-06-28T15:00:15.420739+0000","last_became_peered":"2024-06-28T15:00:15.420739+0000","last_unstale":"2024-06-28T16:14:34.632189+0000","last_undegraded":"2024-06-28T16:14:34.632189+0000","last_fullsized":"2024-06-28T16:14:34.632189+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":200,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:38:08.814015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6022775,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":452,"num_read_kb":28797,"num_write":1102,"num_write_kb":30225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.067917+0000","last_change":"2024-06-28T15:00:19.403941+0000","last_active":"2024-06-28T16:10:25.067917+0000","last_peered":"2024-06-28T16:10:25.067917+0000","last_clean":"2024-06-28T16:10:25.067917+0000","last_became_active":"2024-06-28T15:00:19.400681+0000","last_became_peered":"2024-06-28T15:00:19.400681+0000","last_unstale":"2024-06-28T16:10:25.067917+0000","last_undegraded":"2024-06-28T16:10:25.067917+0000","last_fullsized":"2024-06-28T16:10:25.067917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:52:42.469346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"260'319","reported_seq":938,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.584308+0000","last_change":"2024-06-28T15:00:21.391364+0000","last_active":"2024-06-28T16:12:55.584308+0000","last_peered":"2024-06-28T16:12:55.584308+0000","last_clean":"2024-06-28T16:12:55.584308+0000","last_became_active":"2024-06-28T15:00:21.390609+0000","last_became_peered":"2024-06-28T15:00:21.390609+0000","last_unstale":"2024-06-28T16:12:55.584308+0000","last_undegraded":"2024-06-28T16:12:55.584308+0000","last_fullsized":"2024-06-28T16:12:55.584308+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:41:58.645414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48141,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":517,"num_read_kb":976,"num_write":400,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":493,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.082982+0000","last_change":"2024-06-28T15:00:23.408294+0000","last_active":"2024-06-28T16:10:30.082982+0000","last_peered":"2024-06-28T16:10:30.082982+0000","last_clean":"2024-06-28T16:10:30.082982+0000","last_became_active":"2024-06-28T15:00:23.407521+0000","last_became_peered":"2024-06-28T15:00:23.407521+0000","last_unstale":"2024-06-28T16:10:30.082982+0000","last_undegraded":"2024-06-28T16:10:30.082982+0000","last_fullsized":"2024-06-28T16:10:30.082982+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:06.922239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"260'2402","reported_seq":4268,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:02.134637+0000","last_change":"2024-06-28T15:00:25.440728+0000","last_active":"2024-06-28T16:14:02.134637+0000","last_peered":"2024-06-28T16:14:02.134637+0000","last_clean":"2024-06-28T16:14:02.134637+0000","last_became_active":"2024-06-28T15:00:25.440083+0000","last_became_peered":"2024-06-28T15:00:25.440083+0000","last_unstale":"2024-06-28T16:14:02.134637+0000","last_undegraded":"2024-06-28T16:14:02.134637+0000","last_fullsized":"2024-06-28T16:14:02.134637+0000","mapping_epoch":29,"log_start":"260'2300","ondisk_log_start":"260'2300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":2300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:54:57.315079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5966,"num_read_kb":5973,"num_write":4800,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"260'4179","reported_seq":4823,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.825089+0000","last_change":"2024-06-28T15:00:17.390976+0000","last_active":"2024-06-28T16:11:03.825089+0000","last_peered":"2024-06-28T16:11:03.825089+0000","last_clean":"2024-06-28T16:11:03.825089+0000","last_became_active":"2024-06-28T15:00:17.390254+0000","last_became_peered":"2024-06-28T15:00:17.390254+0000","last_unstale":"2024-06-28T16:11:03.825089+0000","last_undegraded":"2024-06-28T16:11:03.825089+0000","last_fullsized":"2024-06-28T16:11:03.825089+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":3000,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:50:01.744884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14764,"num_read_kb":11968,"num_write":6274,"num_write_kb":3129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"260'269","reported_seq":937,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.545460+0000","last_change":"2024-06-28T15:00:15.397404+0000","last_active":"2024-06-28T16:10:15.545460+0000","last_peered":"2024-06-28T16:10:15.545460+0000","last_clean":"2024-06-28T16:10:15.545460+0000","last_became_active":"2024-06-28T15:00:15.396724+0000","last_became_peered":"2024-06-28T15:00:15.396724+0000","last_unstale":"2024-06-28T16:10:15.545460+0000","last_undegraded":"2024-06-28T16:10:15.545460+0000","last_fullsized":"2024-06-28T16:10:15.545460+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:42:39.929729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2418239,"num_objects":80,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":458,"num_read_kb":7737,"num_write":1161,"num_write_kb":8076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493541+0000","last_change":"2024-06-28T15:00:19.390766+0000","last_active":"2024-06-28T16:10:25.493541+0000","last_peered":"2024-06-28T16:10:25.493541+0000","last_clean":"2024-06-28T16:10:25.493541+0000","last_became_active":"2024-06-28T15:00:19.390100+0000","last_became_peered":"2024-06-28T15:00:19.390100+0000","last_unstale":"2024-06-28T16:10:25.493541+0000","last_undegraded":"2024-06-28T16:10:25.493541+0000","last_fullsized":"2024-06-28T16:10:25.493541+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:01:20.861131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"260'123","reported_seq":662,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.316301+0000","last_change":"2024-06-28T15:00:21.374873+0000","last_active":"2024-06-28T16:11:03.316301+0000","last_peered":"2024-06-28T16:11:03.316301+0000","last_clean":"2024-06-28T16:11:03.316301+0000","last_became_active":"2024-06-28T15:00:21.374284+0000","last_became_peered":"2024-06-28T15:00:21.374284+0000","last_unstale":"2024-06-28T16:11:03.316301+0000","last_undegraded":"2024-06-28T16:11:03.316301+0000","last_fullsized":"2024-06-28T16:11:03.316301+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:17:08.459695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":127,"num_read_kb":120,"num_write":90,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":495,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.076584+0000","last_change":"2024-06-28T15:00:23.399941+0000","last_active":"2024-06-28T16:10:30.076584+0000","last_peered":"2024-06-28T16:10:30.076584+0000","last_clean":"2024-06-28T16:10:30.076584+0000","last_became_active":"2024-06-28T15:00:23.399147+0000","last_became_peered":"2024-06-28T15:00:23.399147+0000","last_unstale":"2024-06-28T16:10:30.076584+0000","last_undegraded":"2024-06-28T16:10:30.076584+0000","last_fullsized":"2024-06-28T16:10:30.076584+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:00:35.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"260'123","reported_seq":649,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.145795+0000","last_change":"2024-06-28T15:00:25.433474+0000","last_active":"2024-06-28T16:14:06.145795+0000","last_peered":"2024-06-28T16:14:06.145795+0000","last_clean":"2024-06-28T16:14:06.145795+0000","last_became_active":"2024-06-28T15:00:25.432898+0000","last_became_peered":"2024-06-28T15:00:25.432898+0000","last_unstale":"2024-06-28T16:14:06.145795+0000","last_undegraded":"2024-06-28T16:14:06.145795+0000","last_fullsized":"2024-06-28T16:14:06.145795+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:46:01.703394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":165,"num_write":302,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"260'155","reported_seq":776,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.548260+0000","last_change":"2024-06-28T15:00:17.384953+0000","last_active":"2024-06-28T16:10:25.548260+0000","last_peered":"2024-06-28T16:10:25.548260+0000","last_clean":"2024-06-28T16:10:25.548260+0000","last_became_active":"2024-06-28T15:00:17.384613+0000","last_became_peered":"2024-06-28T15:00:17.384613+0000","last_unstale":"2024-06-28T16:10:25.548260+0000","last_undegraded":"2024-06-28T16:10:25.548260+0000","last_fullsized":"2024-06-28T16:10:25.548260+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:04:42.370422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":630,"num_read_kb":508,"num_write":240,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"260'353","reported_seq":1001,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.727043+0000","last_change":"2024-06-28T15:00:15.429200+0000","last_active":"2024-06-28T16:14:34.727043+0000","last_peered":"2024-06-28T16:14:34.727043+0000","last_clean":"2024-06-28T16:14:34.727043+0000","last_became_active":"2024-06-28T15:00:15.428415+0000","last_became_peered":"2024-06-28T15:00:15.428415+0000","last_unstale":"2024-06-28T16:14:34.727043+0000","last_undegraded":"2024-06-28T16:14:34.727043+0000","last_fullsized":"2024-06-28T16:14:34.727043+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:17:28.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16055067,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":496,"num_read_kb":15197,"num_write":1261,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493690+0000","last_change":"2024-06-28T15:00:19.373751+0000","last_active":"2024-06-28T16:10:25.493690+0000","last_peered":"2024-06-28T16:10:25.493690+0000","last_clean":"2024-06-28T16:10:25.493690+0000","last_became_active":"2024-06-28T15:00:19.372840+0000","last_became_peered":"2024-06-28T15:00:19.372840+0000","last_unstale":"2024-06-28T16:10:25.493690+0000","last_undegraded":"2024-06-28T16:10:25.493690+0000","last_fullsized":"2024-06-28T16:10:25.493690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:46:30.290232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"260'123","reported_seq":686,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.516922+0000","last_change":"2024-06-28T15:00:21.377962+0000","last_active":"2024-06-28T16:11:03.516922+0000","last_peered":"2024-06-28T16:11:03.516922+0000","last_clean":"2024-06-28T16:11:03.516922+0000","last_became_active":"2024-06-28T15:00:21.377203+0000","last_became_peered":"2024-06-28T15:00:21.377203+0000","last_unstale":"2024-06-28T16:11:03.516922+0000","last_undegraded":"2024-06-28T16:11:03.516922+0000","last_fullsized":"2024-06-28T16:11:03.516922+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:14:43.436846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"260'522","reported_seq":4627,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.061536+0000","last_change":"2024-06-28T15:00:23.418328+0000","last_active":"2024-06-28T16:14:34.061536+0000","last_peered":"2024-06-28T16:14:34.061536+0000","last_clean":"2024-06-28T16:14:34.061536+0000","last_became_active":"2024-06-28T15:00:23.417490+0000","last_became_peered":"2024-06-28T15:00:23.417490+0000","last_unstale":"2024-06-28T16:14:34.061536+0000","last_undegraded":"2024-06-28T16:14:34.061536+0000","last_fullsized":"2024-06-28T16:14:34.061536+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":500,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:22:03.212204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"260'201","reported_seq":1010,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.965594+0000","last_change":"2024-06-28T15:00:25.556315+0000","last_active":"2024-06-28T16:14:06.965594+0000","last_peered":"2024-06-28T16:14:06.965594+0000","last_clean":"2024-06-28T16:14:06.965594+0000","last_became_active":"2024-06-28T15:00:25.555670+0000","last_became_peered":"2024-06-28T15:00:25.555670+0000","last_unstale":"2024-06-28T16:14:06.965594+0000","last_undegraded":"2024-06-28T16:14:06.965594+0000","last_fullsized":"2024-06-28T16:14:06.965594+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:26:34.140679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":557,"num_read_kb":555,"num_write":448,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"260'84","reported_seq":671,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493663+0000","last_change":"2024-06-28T15:00:17.390503+0000","last_active":"2024-06-28T16:10:25.493663+0000","last_peered":"2024-06-28T16:10:25.493663+0000","last_clean":"2024-06-28T16:10:25.493663+0000","last_became_active":"2024-06-28T15:00:17.389986+0000","last_became_peered":"2024-06-28T15:00:17.389986+0000","last_unstale":"2024-06-28T16:10:25.493663+0000","last_undegraded":"2024-06-28T16:10:25.493663+0000","last_fullsized":"2024-06-28T16:10:25.493663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:30:46.132610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":369,"num_read_kb":305,"num_write":130,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"260'288","reported_seq":922,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.683233+0000","last_change":"2024-06-28T15:00:15.413174+0000","last_active":"2024-06-28T16:14:34.683233+0000","last_peered":"2024-06-28T16:14:34.683233+0000","last_clean":"2024-06-28T16:14:34.683233+0000","last_became_active":"2024-06-28T15:00:15.411380+0000","last_became_peered":"2024-06-28T15:00:15.411380+0000","last_unstale":"2024-06-28T16:14:34.683233+0000","last_undegraded":"2024-06-28T16:14:34.683233+0000","last_fullsized":"2024-06-28T16:14:34.683233+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:35.364682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262217,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":24351,"num_write":1142,"num_write_kb":34218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"31'3","reported_seq":830,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:31.936121+0000","last_change":"2024-06-28T15:00:19.404519+0000","last_active":"2024-06-28T16:14:31.936121+0000","last_peered":"2024-06-28T16:14:31.936121+0000","last_clean":"2024-06-28T16:14:31.936121+0000","last_became_active":"2024-06-28T15:00:19.403623+0000","last_became_peered":"2024-06-28T15:00:19.403623+0000","last_unstale":"2024-06-28T16:14:31.936121+0000","last_undegraded":"2024-06-28T16:14:31.936121+0000","last_fullsized":"2024-06-28T16:14:31.936121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:36:48.645241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":496,"num_read_kb":494,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"260'251","reported_seq":833,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.272334+0000","last_change":"2024-06-28T15:00:21.485248+0000","last_active":"2024-06-28T16:14:33.272334+0000","last_peered":"2024-06-28T16:14:33.272334+0000","last_clean":"2024-06-28T16:14:33.272334+0000","last_became_active":"2024-06-28T15:00:21.483633+0000","last_became_peered":"2024-06-28T15:00:21.483633+0000","last_unstale":"2024-06-28T16:14:33.272334+0000","last_undegraded":"2024-06-28T16:14:33.272334+0000","last_fullsized":"2024-06-28T16:14:33.272334+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:48:04.552220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8831,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":873,"num_write":310,"num_write_kb":224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"260'537","reported_seq":2239,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.259197+0000","last_change":"2024-06-28T15:00:23.397769+0000","last_active":"2024-06-28T16:14:34.259197+0000","last_peered":"2024-06-28T16:14:34.259197+0000","last_clean":"2024-06-28T16:14:34.259197+0000","last_became_active":"2024-06-28T15:00:23.396923+0000","last_became_peered":"2024-06-28T15:00:23.396923+0000","last_unstale":"2024-06-28T16:14:34.259197+0000","last_undegraded":"2024-06-28T16:14:34.259197+0000","last_fullsized":"2024-06-28T16:14:34.259197+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:32:30.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"260'415","reported_seq":1093,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:43.529396+0000","last_change":"2024-06-28T15:00:17.378608+0000","last_active":"2024-06-28T16:12:43.529396+0000","last_peered":"2024-06-28T16:12:43.529396+0000","last_clean":"2024-06-28T16:12:43.529396+0000","last_became_active":"2024-06-28T15:00:17.376864+0000","last_became_peered":"2024-06-28T15:00:17.376864+0000","last_unstale":"2024-06-28T16:12:43.529396+0000","last_undegraded":"2024-06-28T16:12:43.529396+0000","last_fullsized":"2024-06-28T16:12:43.529396+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":400,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:13.643893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1583,"num_read_kb":1214,"num_write":634,"num_write_kb":357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"260'320","reported_seq":979,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.732986+0000","last_change":"2024-06-28T15:00:15.423161+0000","last_active":"2024-06-28T16:14:34.732986+0000","last_peered":"2024-06-28T16:14:34.732986+0000","last_clean":"2024-06-28T16:14:34.732986+0000","last_became_active":"2024-06-28T15:00:15.415163+0000","last_became_peered":"2024-06-28T15:00:15.415163+0000","last_unstale":"2024-06-28T16:14:34.732986+0000","last_undegraded":"2024-06-28T16:14:34.732986+0000","last_fullsized":"2024-06-28T16:14:34.732986+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:10:26.502858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470118,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":518,"num_read_kb":26822,"num_write":1200,"num_write_kb":41274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.101361+0000","last_change":"2024-06-28T15:00:19.390703+0000","last_active":"2024-06-28T16:11:50.101361+0000","last_peered":"2024-06-28T16:11:50.101361+0000","last_clean":"2024-06-28T16:11:50.101361+0000","last_became_active":"2024-06-28T15:00:19.389624+0000","last_became_peered":"2024-06-28T15:00:19.389624+0000","last_unstale":"2024-06-28T16:11:50.101361+0000","last_undegraded":"2024-06-28T16:11:50.101361+0000","last_fullsized":"2024-06-28T16:11:50.101361+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:38:09.613294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"260'382","reported_seq":1026,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.950175+0000","last_change":"2024-06-28T15:00:21.483120+0000","last_active":"2024-06-28T16:14:33.950175+0000","last_peered":"2024-06-28T16:14:33.950175+0000","last_clean":"2024-06-28T16:14:33.950175+0000","last_became_active":"2024-06-28T15:00:21.481315+0000","last_became_peered":"2024-06-28T15:00:21.481315+0000","last_unstale":"2024-06-28T16:14:33.950175+0000","last_undegraded":"2024-06-28T16:14:33.950175+0000","last_fullsized":"2024-06-28T16:14:33.950175+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":300,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:09:01.037761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46076,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":638,"num_read_kb":1277,"num_write":490,"num_write_kb":298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.082062+0000","last_change":"2024-06-28T15:00:23.414142+0000","last_active":"2024-06-28T16:10:10.082062+0000","last_peered":"2024-06-28T16:10:10.082062+0000","last_clean":"2024-06-28T16:10:10.082062+0000","last_became_active":"2024-06-28T15:00:23.413132+0000","last_became_peered":"2024-06-28T15:00:23.413132+0000","last_unstale":"2024-06-28T16:10:10.082062+0000","last_undegraded":"2024-06-28T16:10:10.082062+0000","last_fullsized":"2024-06-28T16:10:10.082062+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:39.827537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"260'555","reported_seq":1373,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:13.000905+0000","last_change":"2024-06-28T15:00:25.442589+0000","last_active":"2024-06-28T16:14:13.000905+0000","last_peered":"2024-06-28T16:14:13.000905+0000","last_clean":"2024-06-28T16:14:13.000905+0000","last_became_active":"2024-06-28T15:00:25.441335+0000","last_became_peered":"2024-06-28T15:00:25.441335+0000","last_unstale":"2024-06-28T16:14:13.000905+0000","last_undegraded":"2024-06-28T16:14:13.000905+0000","last_fullsized":"2024-06-28T16:14:13.000905+0000","mapping_epoch":29,"log_start":"260'500","ondisk_log_start":"260'500","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":500,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:55.109636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1277,"num_read_kb":1273,"num_write":1164,"num_write_kb":394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"260'52","reported_seq":631,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.094197+0000","last_change":"2024-06-28T15:00:17.409603+0000","last_active":"2024-06-28T16:11:50.094197+0000","last_peered":"2024-06-28T16:11:50.094197+0000","last_clean":"2024-06-28T16:11:50.094197+0000","last_became_active":"2024-06-28T15:00:17.408992+0000","last_became_peered":"2024-06-28T15:00:17.408992+0000","last_unstale":"2024-06-28T16:11:50.094197+0000","last_undegraded":"2024-06-28T16:11:50.094197+0000","last_fullsized":"2024-06-28T16:11:50.094197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:22:21.223507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":226,"num_read_kb":196,"num_write":79,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"260'326","reported_seq":945,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.655730+0000","last_change":"2024-06-28T15:00:15.402904+0000","last_active":"2024-06-28T16:14:34.655730+0000","last_peered":"2024-06-28T16:14:34.655730+0000","last_clean":"2024-06-28T16:14:34.655730+0000","last_became_active":"2024-06-28T15:00:15.402188+0000","last_became_peered":"2024-06-28T15:00:15.402188+0000","last_unstale":"2024-06-28T16:14:34.655730+0000","last_undegraded":"2024-06-28T16:14:34.655730+0000","last_fullsized":"2024-06-28T16:14:34.655730+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:19:31.626172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170347,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":7037,"num_write":1119,"num_write_kb":28395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.273401+0000","last_change":"2024-06-28T15:00:19.383547+0000","last_active":"2024-06-28T16:11:50.273401+0000","last_peered":"2024-06-28T16:11:50.273401+0000","last_clean":"2024-06-28T16:11:50.273401+0000","last_became_active":"2024-06-28T15:00:19.383002+0000","last_became_peered":"2024-06-28T15:00:19.383002+0000","last_unstale":"2024-06-28T16:11:50.273401+0000","last_undegraded":"2024-06-28T16:11:50.273401+0000","last_fullsized":"2024-06-28T16:11:50.273401+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:52.946596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"260'288","reported_seq":878,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.505294+0000","last_change":"2024-06-28T15:00:21.395366+0000","last_active":"2024-06-28T16:14:32.505294+0000","last_peered":"2024-06-28T16:14:32.505294+0000","last_clean":"2024-06-28T16:14:32.505294+0000","last_became_active":"2024-06-28T15:00:21.394580+0000","last_became_peered":"2024-06-28T15:00:21.394580+0000","last_unstale":"2024-06-28T16:14:32.505294+0000","last_undegraded":"2024-06-28T16:14:32.505294+0000","last_fullsized":"2024-06-28T16:14:32.505294+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:06:06.648827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":455,"num_read_kb":1052,"num_write":364,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:00.515835+0000","last_change":"2024-06-28T15:00:23.423231+0000","last_active":"2024-06-28T16:12:00.515835+0000","last_peered":"2024-06-28T16:12:00.515835+0000","last_clean":"2024-06-28T16:12:00.515835+0000","last_became_active":"2024-06-28T15:00:23.422567+0000","last_became_peered":"2024-06-28T15:00:23.422567+0000","last_unstale":"2024-06-28T16:12:00.515835+0000","last_undegraded":"2024-06-28T16:12:00.515835+0000","last_fullsized":"2024-06-28T16:12:00.515835+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:40.356598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"260'96","reported_seq":589,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089808+0000","last_change":"2024-06-28T15:00:25.426829+0000","last_active":"2024-06-28T16:11:35.089808+0000","last_peered":"2024-06-28T16:11:35.089808+0000","last_clean":"2024-06-28T16:11:35.089808+0000","last_became_active":"2024-06-28T15:00:25.426151+0000","last_became_peered":"2024-06-28T15:00:25.426151+0000","last_unstale":"2024-06-28T16:11:35.089808+0000","last_undegraded":"2024-06-28T16:11:35.089808+0000","last_fullsized":"2024-06-28T16:11:35.089808+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:23:26.301565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":231,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"260'173","reported_seq":777,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:10.098092+0000","last_change":"2024-06-28T15:00:17.407655+0000","last_active":"2024-06-28T16:12:10.098092+0000","last_peered":"2024-06-28T16:12:10.098092+0000","last_clean":"2024-06-28T16:12:10.098092+0000","last_became_active":"2024-06-28T15:00:17.407031+0000","last_became_peered":"2024-06-28T15:00:17.407031+0000","last_unstale":"2024-06-28T16:12:10.098092+0000","last_undegraded":"2024-06-28T16:12:10.098092+0000","last_fullsized":"2024-06-28T16:12:10.098092+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:17:37.199564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":538,"num_write":274,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"260'353","reported_seq":1038,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.682331+0000","last_change":"2024-06-28T15:00:15.428903+0000","last_active":"2024-06-28T16:14:34.682331+0000","last_peered":"2024-06-28T16:14:34.682331+0000","last_clean":"2024-06-28T16:14:34.682331+0000","last_became_active":"2024-06-28T15:00:15.427408+0000","last_became_peered":"2024-06-28T15:00:15.427408+0000","last_unstale":"2024-06-28T16:14:34.682331+0000","last_undegraded":"2024-06-28T16:14:34.682331+0000","last_fullsized":"2024-06-28T16:14:34.682331+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:52:51.484685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":574,"num_read_kb":16939,"num_write":1376,"num_write_kb":32636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.279481+0000","last_change":"2024-06-28T15:00:19.384770+0000","last_active":"2024-06-28T16:12:20.279481+0000","last_peered":"2024-06-28T16:12:20.279481+0000","last_clean":"2024-06-28T16:12:20.279481+0000","last_became_active":"2024-06-28T15:00:19.384172+0000","last_became_peered":"2024-06-28T15:00:19.384172+0000","last_unstale":"2024-06-28T16:12:20.279481+0000","last_undegraded":"2024-06-28T16:12:20.279481+0000","last_fullsized":"2024-06-28T16:12:20.279481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:13:54.648899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"260'314","reported_seq":925,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.098634+0000","last_change":"2024-06-28T15:00:21.394527+0000","last_active":"2024-06-28T16:14:34.098634+0000","last_peered":"2024-06-28T16:14:34.098634+0000","last_clean":"2024-06-28T16:14:34.098634+0000","last_became_active":"2024-06-28T15:00:21.393608+0000","last_became_peered":"2024-06-28T15:00:21.393608+0000","last_unstale":"2024-06-28T16:14:34.098634+0000","last_undegraded":"2024-06-28T16:14:34.098634+0000","last_fullsized":"2024-06-28T16:14:34.098634+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:01:39.810509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42645,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":526,"num_read_kb":1122,"num_write":410,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"260'529","reported_seq":2261,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:14.487526+0000","last_change":"2024-06-28T15:00:23.414333+0000","last_active":"2024-06-28T16:14:14.487526+0000","last_peered":"2024-06-28T16:14:14.487526+0000","last_clean":"2024-06-28T16:14:14.487526+0000","last_became_active":"2024-06-28T15:00:23.413550+0000","last_became_peered":"2024-06-28T15:00:23.413550+0000","last_unstale":"2024-06-28T16:14:14.487526+0000","last_undegraded":"2024-06-28T16:14:14.487526+0000","last_fullsized":"2024-06-28T16:14:14.487526+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":500,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:23:18.123161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"260'144","reported_seq":672,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.569778+0000","last_change":"2024-06-28T15:00:25.432580+0000","last_active":"2024-06-28T16:11:55.569778+0000","last_peered":"2024-06-28T16:11:55.569778+0000","last_clean":"2024-06-28T16:11:55.569778+0000","last_became_active":"2024-06-28T15:00:25.431835+0000","last_became_peered":"2024-06-28T15:00:25.431835+0000","last_unstale":"2024-06-28T16:11:55.569778+0000","last_undegraded":"2024-06-28T16:11:55.569778+0000","last_fullsized":"2024-06-28T16:11:55.569778+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:25:08.052594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1247,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":164,"num_write":265,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"260'4158","reported_seq":4763,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.543014+0000","last_change":"2024-06-28T15:00:17.400121+0000","last_active":"2024-06-28T16:10:05.543014+0000","last_peered":"2024-06-28T16:10:05.543014+0000","last_clean":"2024-06-28T16:10:05.543014+0000","last_became_active":"2024-06-28T15:00:17.399799+0000","last_became_peered":"2024-06-28T15:00:17.399799+0000","last_unstale":"2024-06-28T16:10:05.543014+0000","last_undegraded":"2024-06-28T16:10:05.543014+0000","last_fullsized":"2024-06-28T16:10:05.543014+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":3000,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:10:23.005648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14678,"num_read_kb":11882,"num_write":6254,"num_write_kb":3121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"260'337","reported_seq":982,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.704930+0000","last_change":"2024-06-28T15:00:15.426615+0000","last_active":"2024-06-28T16:14:34.704930+0000","last_peered":"2024-06-28T16:14:34.704930+0000","last_clean":"2024-06-28T16:14:34.704930+0000","last_became_active":"2024-06-28T15:00:15.425690+0000","last_became_peered":"2024-06-28T15:00:15.425690+0000","last_unstale":"2024-06-28T16:14:34.704930+0000","last_undegraded":"2024-06-28T16:14:34.704930+0000","last_fullsized":"2024-06-28T16:14:34.704930+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:59:45.352009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079931,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":508,"num_read_kb":13664,"num_write":1274,"num_write_kb":26574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,1],"acting":[5,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:05.276635+0000","last_change":"2024-06-28T15:00:19.385556+0000","last_active":"2024-06-28T16:12:05.276635+0000","last_peered":"2024-06-28T16:12:05.276635+0000","last_clean":"2024-06-28T16:12:05.276635+0000","last_became_active":"2024-06-28T15:00:19.385189+0000","last_became_peered":"2024-06-28T15:00:19.385189+0000","last_unstale":"2024-06-28T16:12:05.276635+0000","last_undegraded":"2024-06-28T16:12:05.276635+0000","last_fullsized":"2024-06-28T16:12:05.276635+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:12:07.124133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"260'477","reported_seq":1123,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:12.918679+0000","last_change":"2024-06-28T15:00:21.400448+0000","last_active":"2024-06-28T16:14:12.918679+0000","last_peered":"2024-06-28T16:14:12.918679+0000","last_clean":"2024-06-28T16:14:12.918679+0000","last_became_active":"2024-06-28T15:00:21.397397+0000","last_became_peered":"2024-06-28T15:00:21.397397+0000","last_unstale":"2024-06-28T16:14:12.918679+0000","last_undegraded":"2024-06-28T16:14:12.918679+0000","last_fullsized":"2024-06-28T16:14:12.918679+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":400,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:54:53.893930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109658,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":830,"num_read_kb":1847,"num_write":686,"num_write_kb":570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.274612+0000","last_change":"2024-06-28T15:00:23.422892+0000","last_active":"2024-06-28T16:11:55.274612+0000","last_peered":"2024-06-28T16:11:55.274612+0000","last_clean":"2024-06-28T16:11:55.274612+0000","last_became_active":"2024-06-28T15:00:23.416263+0000","last_became_peered":"2024-06-28T15:00:23.416263+0000","last_unstale":"2024-06-28T16:11:55.274612+0000","last_undegraded":"2024-06-28T16:11:55.274612+0000","last_fullsized":"2024-06-28T16:11:55.274612+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:12:48.735241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"260'113","reported_seq":631,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.103804+0000","last_change":"2024-06-28T15:00:25.423425+0000","last_active":"2024-06-28T16:11:55.103804+0000","last_peered":"2024-06-28T16:11:55.103804+0000","last_clean":"2024-06-28T16:11:55.103804+0000","last_became_active":"2024-06-28T15:00:25.422519+0000","last_became_peered":"2024-06-28T15:00:25.422519+0000","last_unstale":"2024-06-28T16:11:55.103804+0000","last_undegraded":"2024-06-28T16:11:55.103804+0000","last_fullsized":"2024-06-28T16:11:55.103804+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:15:27.026404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":118,"num_read_kb":116,"num_write":248,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"260'128","reported_seq":777,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101386+0000","last_change":"2024-06-28T15:00:17.409415+0000","last_active":"2024-06-28T16:12:20.101386+0000","last_peered":"2024-06-28T16:12:20.101386+0000","last_clean":"2024-06-28T16:12:20.101386+0000","last_became_active":"2024-06-28T15:00:17.404204+0000","last_became_peered":"2024-06-28T15:00:17.404204+0000","last_unstale":"2024-06-28T16:12:20.101386+0000","last_undegraded":"2024-06-28T16:12:20.101386+0000","last_fullsized":"2024-06-28T16:12:20.101386+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:25:51.372637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":562,"num_read_kb":476,"num_write":174,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"260'352","reported_seq":1059,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.701530+0000","last_change":"2024-06-28T15:00:15.402917+0000","last_active":"2024-06-28T16:14:34.701530+0000","last_peered":"2024-06-28T16:14:34.701530+0000","last_clean":"2024-06-28T16:14:34.701530+0000","last_became_active":"2024-06-28T15:00:15.402041+0000","last_became_peered":"2024-06-28T15:00:15.402041+0000","last_unstale":"2024-06-28T16:14:34.701530+0000","last_undegraded":"2024-06-28T16:14:34.701530+0000","last_fullsized":"2024-06-28T16:14:34.701530+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:50:41.105836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":603,"num_read_kb":26410,"num_write":1369,"num_write_kb":26391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.279564+0000","last_change":"2024-06-28T15:00:19.393172+0000","last_active":"2024-06-28T16:12:20.279564+0000","last_peered":"2024-06-28T16:12:20.279564+0000","last_clean":"2024-06-28T16:12:20.279564+0000","last_became_active":"2024-06-28T15:00:19.392773+0000","last_became_peered":"2024-06-28T15:00:19.392773+0000","last_unstale":"2024-06-28T16:12:20.279564+0000","last_undegraded":"2024-06-28T16:12:20.279564+0000","last_fullsized":"2024-06-28T16:12:20.279564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:03:10.710059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"260'301","reported_seq":896,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.815148+0000","last_change":"2024-06-28T15:00:21.398582+0000","last_active":"2024-06-28T16:14:32.815148+0000","last_peered":"2024-06-28T16:14:32.815148+0000","last_clean":"2024-06-28T16:14:32.815148+0000","last_became_active":"2024-06-28T15:00:21.397842+0000","last_became_peered":"2024-06-28T15:00:21.397842+0000","last_unstale":"2024-06-28T16:14:32.815148+0000","last_undegraded":"2024-06-28T16:14:32.815148+0000","last_fullsized":"2024-06-28T16:14:32.815148+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:31:25.283406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44037,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":498,"num_read_kb":1063,"num_write":434,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.548349+0000","last_change":"2024-06-28T15:00:23.406901+0000","last_active":"2024-06-28T16:10:25.548349+0000","last_peered":"2024-06-28T16:10:25.548349+0000","last_clean":"2024-06-28T16:10:25.548349+0000","last_became_active":"2024-06-28T15:00:23.406122+0000","last_became_peered":"2024-06-28T15:00:23.406122+0000","last_unstale":"2024-06-28T16:10:25.548349+0000","last_undegraded":"2024-06-28T16:10:25.548349+0000","last_fullsized":"2024-06-28T16:10:25.548349+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:58:30.933744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"260'89","reported_seq":593,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:03.580217+0000","last_change":"2024-06-28T15:00:25.439968+0000","last_active":"2024-06-28T16:14:03.580217+0000","last_peered":"2024-06-28T16:14:03.580217+0000","last_clean":"2024-06-28T16:14:03.580217+0000","last_became_active":"2024-06-28T15:00:25.439346+0000","last_became_peered":"2024-06-28T15:00:25.439346+0000","last_unstale":"2024-06-28T16:14:03.580217+0000","last_undegraded":"2024-06-28T16:14:03.580217+0000","last_fullsized":"2024-06-28T16:14:03.580217+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:31:17.344077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":207,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"260'171","reported_seq":759,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101425+0000","last_change":"2024-06-28T15:00:17.396835+0000","last_active":"2024-06-28T16:12:20.101425+0000","last_peered":"2024-06-28T16:12:20.101425+0000","last_clean":"2024-06-28T16:12:20.101425+0000","last_became_active":"2024-06-28T15:00:17.396048+0000","last_became_peered":"2024-06-28T15:00:17.396048+0000","last_unstale":"2024-06-28T16:12:20.101425+0000","last_undegraded":"2024-06-28T16:12:20.101425+0000","last_fullsized":"2024-06-28T16:12:20.101425+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:20:25.818327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":659,"num_read_kb":531,"num_write":250,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"260'376","reported_seq":1107,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.727955+0000","last_change":"2024-06-28T15:00:15.398260+0000","last_active":"2024-06-28T16:14:34.727955+0000","last_peered":"2024-06-28T16:14:34.727955+0000","last_clean":"2024-06-28T16:14:34.727955+0000","last_became_active":"2024-06-28T15:00:15.397256+0000","last_became_peered":"2024-06-28T15:00:15.397256+0000","last_unstale":"2024-06-28T16:14:34.727955+0000","last_undegraded":"2024-06-28T16:14:34.727955+0000","last_fullsized":"2024-06-28T16:14:34.727955+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":300,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:08:01.511388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2674362,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":727,"num_read_kb":7345,"num_write":1214,"num_write_kb":31713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.109069+0000","last_change":"2024-06-28T15:00:19.379868+0000","last_active":"2024-06-28T16:12:20.109069+0000","last_peered":"2024-06-28T16:12:20.109069+0000","last_clean":"2024-06-28T16:12:20.109069+0000","last_became_active":"2024-06-28T15:00:19.379281+0000","last_became_peered":"2024-06-28T15:00:19.379281+0000","last_unstale":"2024-06-28T16:12:20.109069+0000","last_undegraded":"2024-06-28T16:12:20.109069+0000","last_fullsized":"2024-06-28T16:12:20.109069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:01:53.936356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"260'350","reported_seq":968,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.682992+0000","last_change":"2024-06-28T15:00:21.396750+0000","last_active":"2024-06-28T16:14:34.682992+0000","last_peered":"2024-06-28T16:14:34.682992+0000","last_clean":"2024-06-28T16:14:34.682992+0000","last_became_active":"2024-06-28T15:00:21.395892+0000","last_became_peered":"2024-06-28T15:00:21.395892+0000","last_unstale":"2024-06-28T16:14:34.682992+0000","last_undegraded":"2024-06-28T16:14:34.682992+0000","last_fullsized":"2024-06-28T16:14:34.682992+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":300,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:49:00.898473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48931,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":579,"num_read_kb":1282,"num_write":462,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"260'501","reported_seq":2303,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.195533+0000","last_change":"2024-06-28T15:00:23.416260+0000","last_active":"2024-06-28T16:14:32.195533+0000","last_peered":"2024-06-28T16:14:32.195533+0000","last_clean":"2024-06-28T16:14:32.195533+0000","last_became_active":"2024-06-28T15:00:23.415437+0000","last_became_peered":"2024-06-28T15:00:23.415437+0000","last_unstale":"2024-06-28T16:14:32.195533+0000","last_undegraded":"2024-06-28T16:14:32.195533+0000","last_fullsized":"2024-06-28T16:14:32.195533+0000","mapping_epoch":27,"log_start":"257'400","ondisk_log_start":"257'400","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":400,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:08:34.898411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"260'99","reported_seq":608,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.122863+0000","last_change":"2024-06-28T15:00:25.425875+0000","last_active":"2024-06-28T16:13:20.122863+0000","last_peered":"2024-06-28T16:13:20.122863+0000","last_clean":"2024-06-28T16:13:20.122863+0000","last_became_active":"2024-06-28T15:00:25.425135+0000","last_became_peered":"2024-06-28T15:00:25.425135+0000","last_unstale":"2024-06-28T16:13:20.122863+0000","last_undegraded":"2024-06-28T16:13:20.122863+0000","last_fullsized":"2024-06-28T16:13:20.122863+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:13:56.075833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":245,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"260'160","reported_seq":743,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101485+0000","last_change":"2024-06-28T15:00:17.403493+0000","last_active":"2024-06-28T16:12:20.101485+0000","last_peered":"2024-06-28T16:12:20.101485+0000","last_clean":"2024-06-28T16:12:20.101485+0000","last_became_active":"2024-06-28T15:00:17.402024+0000","last_became_peered":"2024-06-28T15:00:17.402024+0000","last_unstale":"2024-06-28T16:12:20.101485+0000","last_undegraded":"2024-06-28T16:12:20.101485+0000","last_fullsized":"2024-06-28T16:12:20.101485+0000","mapping_epoch":21,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":100,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:25:59.494240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":645,"num_read_kb":505,"num_write":259,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"260'393","reported_seq":1067,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.738050+0000","last_change":"2024-06-28T15:00:15.389691+0000","last_active":"2024-06-28T16:14:34.738050+0000","last_peered":"2024-06-28T16:14:34.738050+0000","last_clean":"2024-06-28T16:14:34.738050+0000","last_became_active":"2024-06-28T15:00:15.388365+0000","last_became_peered":"2024-06-28T15:00:15.388365+0000","last_unstale":"2024-06-28T16:14:34.738050+0000","last_undegraded":"2024-06-28T16:14:34.738050+0000","last_fullsized":"2024-06-28T16:14:34.738050+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":300,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:14:31.668500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1604260,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":661,"num_read_kb":7226,"num_write":1423,"num_write_kb":30573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"24'2","reported_seq":1154,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:31.948538+0000","last_change":"2024-06-28T15:00:19.403973+0000","last_active":"2024-06-28T16:14:31.948538+0000","last_peered":"2024-06-28T16:14:31.948538+0000","last_clean":"2024-06-28T16:14:31.948538+0000","last_became_active":"2024-06-28T15:00:19.397863+0000","last_became_peered":"2024-06-28T15:00:19.397863+0000","last_unstale":"2024-06-28T16:14:31.948538+0000","last_undegraded":"2024-06-28T16:14:31.948538+0000","last_fullsized":"2024-06-28T16:14:31.948538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:35:07.475408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":648,"num_read_kb":648,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"260'113","reported_seq":657,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.163323+0000","last_change":"2024-06-28T15:00:21.484486+0000","last_active":"2024-06-28T16:11:03.163323+0000","last_peered":"2024-06-28T16:11:03.163323+0000","last_clean":"2024-06-28T16:11:03.163323+0000","last_became_active":"2024-06-28T15:00:21.483679+0000","last_became_peered":"2024-06-28T15:00:21.483679+0000","last_unstale":"2024-06-28T16:11:03.163323+0000","last_undegraded":"2024-06-28T16:11:03.163323+0000","last_fullsized":"2024-06-28T16:11:03.163323+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:16:50.618747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"260'515","reported_seq":4262,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.199674+0000","last_change":"2024-06-28T15:00:23.419813+0000","last_active":"2024-06-28T16:14:32.199674+0000","last_peered":"2024-06-28T16:14:32.199674+0000","last_clean":"2024-06-28T16:14:32.199674+0000","last_became_active":"2024-06-28T15:00:23.419182+0000","last_became_peered":"2024-06-28T15:00:23.419182+0000","last_unstale":"2024-06-28T16:14:32.199674+0000","last_undegraded":"2024-06-28T16:14:32.199674+0000","last_fullsized":"2024-06-28T16:14:32.199674+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:00:01.245043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"260'159","reported_seq":672,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.114555+0000","last_change":"2024-06-28T15:00:25.427204+0000","last_active":"2024-06-28T16:13:20.114555+0000","last_peered":"2024-06-28T16:13:20.114555+0000","last_clean":"2024-06-28T16:13:20.114555+0000","last_became_active":"2024-06-28T15:00:25.425690+0000","last_became_peered":"2024-06-28T15:00:25.425690+0000","last_unstale":"2024-06-28T16:13:20.114555+0000","last_undegraded":"2024-06-28T16:13:20.114555+0000","last_fullsized":"2024-06-28T16:13:20.114555+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:18.073869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":152,"num_write":381,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"260'74","reported_seq":619,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.075926+0000","last_change":"2024-06-28T15:00:59.800386+0000","last_active":"2024-06-28T16:11:00.075926+0000","last_peered":"2024-06-28T16:11:00.075926+0000","last_clean":"2024-06-28T16:11:00.075926+0000","last_became_active":"2024-06-28T15:00:59.799823+0000","last_became_peered":"2024-06-28T15:00:59.799823+0000","last_unstale":"2024-06-28T16:11:00.075926+0000","last_undegraded":"2024-06-28T16:11:00.075926+0000","last_fullsized":"2024-06-28T16:11:00.075926+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:43:49.439721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":247,"num_write":103,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"260'351","reported_seq":1081,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.579126+0000","last_change":"2024-06-28T15:00:15.403798+0000","last_active":"2024-06-28T16:14:34.579126+0000","last_peered":"2024-06-28T16:14:34.579126+0000","last_clean":"2024-06-28T16:14:34.579126+0000","last_became_active":"2024-06-28T15:00:15.402782+0000","last_became_peered":"2024-06-28T15:00:15.402782+0000","last_unstale":"2024-06-28T16:14:34.579126+0000","last_undegraded":"2024-06-28T16:14:34.579126+0000","last_fullsized":"2024-06-28T16:14:34.579126+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":300,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:44:32.528218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262233,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":541,"num_read_kb":3603,"num_write":1164,"num_write_kb":36556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101454+0000","last_change":"2024-06-28T15:00:19.405878+0000","last_active":"2024-06-28T16:12:20.101454+0000","last_peered":"2024-06-28T16:12:20.101454+0000","last_clean":"2024-06-28T16:12:20.101454+0000","last_became_active":"2024-06-28T15:00:19.405083+0000","last_became_peered":"2024-06-28T15:00:19.405083+0000","last_unstale":"2024-06-28T16:12:20.101454+0000","last_undegraded":"2024-06-28T16:12:20.101454+0000","last_fullsized":"2024-06-28T16:12:20.101454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:55:51.552282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"260'429","reported_seq":1063,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.718564+0000","last_change":"2024-06-28T15:00:21.393207+0000","last_active":"2024-06-28T16:14:33.718564+0000","last_peered":"2024-06-28T16:14:33.718564+0000","last_clean":"2024-06-28T16:14:33.718564+0000","last_became_active":"2024-06-28T15:00:21.392393+0000","last_became_peered":"2024-06-28T15:00:21.392393+0000","last_unstale":"2024-06-28T16:14:33.718564+0000","last_undegraded":"2024-06-28T16:14:33.718564+0000","last_fullsized":"2024-06-28T16:14:33.718564+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":400,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:47:20.793097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47467,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1282,"num_read_kb":1893,"num_write":1822,"num_write_kb":1648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":493,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.070668+0000","last_change":"2024-06-28T15:00:23.416374+0000","last_active":"2024-06-28T16:10:10.070668+0000","last_peered":"2024-06-28T16:10:10.070668+0000","last_clean":"2024-06-28T16:10:10.070668+0000","last_became_active":"2024-06-28T15:00:23.415497+0000","last_became_peered":"2024-06-28T15:00:23.415497+0000","last_unstale":"2024-06-28T16:10:10.070668+0000","last_undegraded":"2024-06-28T16:10:10.070668+0000","last_fullsized":"2024-06-28T16:10:10.070668+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:13:09.664310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"260'133","reported_seq":639,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:07.643827+0000","last_change":"2024-06-28T15:00:25.440419+0000","last_active":"2024-06-28T16:14:07.643827+0000","last_peered":"2024-06-28T16:14:07.643827+0000","last_clean":"2024-06-28T16:14:07.643827+0000","last_became_active":"2024-06-28T15:00:25.439527+0000","last_became_peered":"2024-06-28T15:00:25.439527+0000","last_unstale":"2024-06-28T16:14:07.643827+0000","last_undegraded":"2024-06-28T16:14:07.643827+0000","last_fullsized":"2024-06-28T16:14:07.643827+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:30:19.298980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":127,"num_write":301,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"260'156","reported_seq":773,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.088666+0000","last_change":"2024-06-28T15:00:17.415172+0000","last_active":"2024-06-28T16:11:30.088666+0000","last_peered":"2024-06-28T16:11:30.088666+0000","last_clean":"2024-06-28T16:11:30.088666+0000","last_became_active":"2024-06-28T15:00:17.414537+0000","last_became_peered":"2024-06-28T15:00:17.414537+0000","last_unstale":"2024-06-28T16:11:30.088666+0000","last_undegraded":"2024-06-28T16:11:30.088666+0000","last_fullsized":"2024-06-28T16:11:30.088666+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:18:02.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":522,"num_write":223,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":616,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.109225+0000","last_change":"2024-06-28T15:00:06.303487+0000","last_active":"2024-06-28T16:12:25.109225+0000","last_peered":"2024-06-28T16:12:25.109225+0000","last_clean":"2024-06-28T16:12:25.109225+0000","last_became_active":"2024-06-28T15:00:06.302725+0000","last_became_peered":"2024-06-28T15:00:06.302725+0000","last_unstale":"2024-06-28T16:12:25.109225+0000","last_undegraded":"2024-06-28T16:12:25.109225+0000","last_fullsized":"2024-06-28T16:12:25.109225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:45:32.083226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.075782+0000","last_change":"2024-06-28T15:00:06.285125+0000","last_active":"2024-06-28T16:11:00.075782+0000","last_peered":"2024-06-28T16:11:00.075782+0000","last_clean":"2024-06-28T16:11:00.075782+0000","last_became_active":"2024-06-28T15:00:06.284218+0000","last_became_peered":"2024-06-28T15:00:06.284218+0000","last_unstale":"2024-06-28T16:11:00.075782+0000","last_undegraded":"2024-06-28T16:11:00.075782+0000","last_fullsized":"2024-06-28T16:11:00.075782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:04:51.635552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"260'320","reported_seq":1115,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.735430+0000","last_change":"2024-06-28T15:00:59.940725+0000","last_active":"2024-06-28T16:14:34.735430+0000","last_peered":"2024-06-28T16:14:34.735430+0000","last_clean":"2024-06-28T16:14:34.735430+0000","last_became_active":"2024-06-28T15:00:59.939352+0000","last_became_peered":"2024-06-28T15:00:59.939352+0000","last_unstale":"2024-06-28T16:14:34.735430+0000","last_undegraded":"2024-06-28T16:14:34.735430+0000","last_fullsized":"2024-06-28T16:14:34.735430+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:16:46.674713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":804,"num_read_kb":59106,"num_write":1183,"num_write_kb":43047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6,4],"acting":[3,2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.092431+0000","last_change":"2024-06-28T15:00:19.391126+0000","last_active":"2024-06-28T16:11:10.092431+0000","last_peered":"2024-06-28T16:11:10.092431+0000","last_clean":"2024-06-28T16:11:10.092431+0000","last_became_active":"2024-06-28T15:00:19.390255+0000","last_became_peered":"2024-06-28T15:00:19.390255+0000","last_unstale":"2024-06-28T16:11:10.092431+0000","last_undegraded":"2024-06-28T16:11:10.092431+0000","last_fullsized":"2024-06-28T16:11:10.092431+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:19:02.126295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"260'730","reported_seq":1450,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.539793+0000","last_change":"2024-06-28T15:00:21.398370+0000","last_active":"2024-06-28T16:14:34.539793+0000","last_peered":"2024-06-28T16:14:34.539793+0000","last_clean":"2024-06-28T16:14:34.539793+0000","last_became_active":"2024-06-28T15:00:21.397640+0000","last_became_peered":"2024-06-28T15:00:21.397640+0000","last_unstale":"2024-06-28T16:14:34.539793+0000","last_undegraded":"2024-06-28T16:14:34.539793+0000","last_fullsized":"2024-06-28T16:14:34.539793+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":700,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:10:23.954766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134565,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1425,"num_read_kb":3272,"num_write":1360,"num_write_kb":1190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.569752+0000","last_change":"2024-06-28T15:00:23.400439+0000","last_active":"2024-06-28T16:11:55.569752+0000","last_peered":"2024-06-28T16:11:55.569752+0000","last_clean":"2024-06-28T16:11:55.569752+0000","last_became_active":"2024-06-28T15:00:23.399024+0000","last_became_peered":"2024-06-28T15:00:23.399024+0000","last_unstale":"2024-06-28T16:11:55.569752+0000","last_undegraded":"2024-06-28T16:11:55.569752+0000","last_fullsized":"2024-06-28T16:11:55.569752+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:16:52.193469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"260'186","reported_seq":876,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:05.197144+0000","last_change":"2024-06-28T15:00:25.442723+0000","last_active":"2024-06-28T16:14:05.197144+0000","last_peered":"2024-06-28T16:14:05.197144+0000","last_clean":"2024-06-28T16:14:05.197144+0000","last_became_active":"2024-06-28T15:00:25.442032+0000","last_became_peered":"2024-06-28T15:00:25.442032+0000","last_unstale":"2024-06-28T16:14:05.197144+0000","last_undegraded":"2024-06-28T16:14:05.197144+0000","last_fullsized":"2024-06-28T16:14:05.197144+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:04:44.942959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":392,"num_read_kb":388,"num_write":388,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"260'118","reported_seq":755,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:57.970334+0000","last_change":"2024-06-28T15:00:17.404538+0000","last_active":"2024-06-28T16:11:57.970334+0000","last_peered":"2024-06-28T16:11:57.970334+0000","last_clean":"2024-06-28T16:11:57.970334+0000","last_became_active":"2024-06-28T15:00:17.404200+0000","last_became_peered":"2024-06-28T15:00:17.404200+0000","last_unstale":"2024-06-28T16:11:57.970334+0000","last_undegraded":"2024-06-28T16:11:57.970334+0000","last_fullsized":"2024-06-28T16:11:57.970334+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:32:41.333506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":495,"num_read_kb":413,"num_write":159,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":526,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.109267+0000","last_change":"2024-06-28T15:00:06.286231+0000","last_active":"2024-06-28T16:12:25.109267+0000","last_peered":"2024-06-28T16:12:25.109267+0000","last_clean":"2024-06-28T16:12:25.109267+0000","last_became_active":"2024-06-28T15:00:06.285489+0000","last_became_peered":"2024-06-28T15:00:06.285489+0000","last_unstale":"2024-06-28T16:12:25.109267+0000","last_undegraded":"2024-06-28T16:12:25.109267+0000","last_fullsized":"2024-06-28T16:12:25.109267+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:05:17.528909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"260'262","reported_seq":930,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.556093+0000","last_change":"2024-06-28T15:00:15.404155+0000","last_active":"2024-06-28T16:11:05.556093+0000","last_peered":"2024-06-28T16:11:05.556093+0000","last_clean":"2024-06-28T16:11:05.556093+0000","last_became_active":"2024-06-28T15:00:15.403657+0000","last_became_peered":"2024-06-28T15:00:15.403657+0000","last_unstale":"2024-06-28T16:11:05.556093+0000","last_undegraded":"2024-06-28T16:11:05.556093+0000","last_fullsized":"2024-06-28T16:11:05.556093+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:18:38.173993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6321470,"num_objects":64,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":462,"num_read_kb":36644,"num_write":1201,"num_write_kb":16548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.087155+0000","last_change":"2024-06-28T15:00:19.407157+0000","last_active":"2024-06-28T16:11:50.087155+0000","last_peered":"2024-06-28T16:11:50.087155+0000","last_clean":"2024-06-28T16:11:50.087155+0000","last_became_active":"2024-06-28T15:00:19.406447+0000","last_became_peered":"2024-06-28T15:00:19.406447+0000","last_unstale":"2024-06-28T16:11:50.087155+0000","last_undegraded":"2024-06-28T16:11:50.087155+0000","last_fullsized":"2024-06-28T16:11:50.087155+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:12:32.253515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"260'1134","reported_seq":2908,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.789539+0000","last_change":"2024-06-28T15:00:21.399223+0000","last_active":"2024-06-28T16:14:33.789539+0000","last_peered":"2024-06-28T16:14:33.789539+0000","last_clean":"2024-06-28T16:14:33.789539+0000","last_became_active":"2024-06-28T15:00:21.398382+0000","last_became_peered":"2024-06-28T15:00:21.398382+0000","last_unstale":"2024-06-28T16:14:33.789539+0000","last_undegraded":"2024-06-28T16:14:33.789539+0000","last_fullsized":"2024-06-28T16:14:33.789539+0000","mapping_epoch":25,"log_start":"260'1100","ondisk_log_start":"260'1100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":1100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:39:26.868657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133468,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1851,"num_read_kb":3741,"num_write":1150,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":498,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.097271+0000","last_change":"2024-06-28T15:00:23.423751+0000","last_active":"2024-06-28T16:11:30.097271+0000","last_peered":"2024-06-28T16:11:30.097271+0000","last_clean":"2024-06-28T16:11:30.097271+0000","last_became_active":"2024-06-28T15:00:23.422876+0000","last_became_peered":"2024-06-28T15:00:23.422876+0000","last_unstale":"2024-06-28T16:11:30.097271+0000","last_undegraded":"2024-06-28T16:11:30.097271+0000","last_fullsized":"2024-06-28T16:11:30.097271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:18:16.152063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"260'116","reported_seq":629,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:02.832339+0000","last_change":"2024-06-28T15:00:25.554459+0000","last_active":"2024-06-28T16:14:02.832339+0000","last_peered":"2024-06-28T16:14:02.832339+0000","last_clean":"2024-06-28T16:14:02.832339+0000","last_became_active":"2024-06-28T15:00:25.553691+0000","last_became_peered":"2024-06-28T15:00:25.553691+0000","last_unstale":"2024-06-28T16:14:02.832339+0000","last_undegraded":"2024-06-28T16:14:02.832339+0000","last_fullsized":"2024-06-28T16:14:02.832339+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:37:46.194533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":862,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":120,"num_write":292,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"260'221","reported_seq":834,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:02.146046+0000","last_change":"2024-06-28T15:00:17.393736+0000","last_active":"2024-06-28T16:13:02.146046+0000","last_peered":"2024-06-28T16:13:02.146046+0000","last_clean":"2024-06-28T16:13:02.146046+0000","last_became_active":"2024-06-28T15:00:17.393361+0000","last_became_peered":"2024-06-28T15:00:17.393361+0000","last_unstale":"2024-06-28T16:13:02.146046+0000","last_undegraded":"2024-06-28T16:13:02.146046+0000","last_fullsized":"2024-06-28T16:13:02.146046+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:50.952114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":892,"num_read_kb":761,"num_write":358,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":532,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.262525+0000","last_change":"2024-06-28T15:00:06.280312+0000","last_active":"2024-06-28T16:10:55.262525+0000","last_peered":"2024-06-28T16:10:55.262525+0000","last_clean":"2024-06-28T16:10:55.262525+0000","last_became_active":"2024-06-28T15:00:06.279950+0000","last_became_peered":"2024-06-28T15:00:06.279950+0000","last_unstale":"2024-06-28T16:10:55.262525+0000","last_undegraded":"2024-06-28T16:10:55.262525+0000","last_fullsized":"2024-06-28T16:10:55.262525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:05:40.110987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"260'334","reported_seq":991,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.720045+0000","last_change":"2024-06-28T15:00:15.413730+0000","last_active":"2024-06-28T16:14:34.720045+0000","last_peered":"2024-06-28T16:14:34.720045+0000","last_clean":"2024-06-28T16:14:34.720045+0000","last_became_active":"2024-06-28T15:00:15.413348+0000","last_became_peered":"2024-06-28T15:00:15.413348+0000","last_unstale":"2024-06-28T16:14:34.720045+0000","last_undegraded":"2024-06-28T16:14:34.720045+0000","last_fullsized":"2024-06-28T16:14:34.720045+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:07:23.937007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":504,"num_read_kb":3638,"num_write":1247,"num_write_kb":36014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"24'1","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.273547+0000","last_change":"2024-06-28T15:00:19.405107+0000","last_active":"2024-06-28T16:11:50.273547+0000","last_peered":"2024-06-28T16:11:50.273547+0000","last_clean":"2024-06-28T16:11:50.273547+0000","last_became_active":"2024-06-28T15:00:19.404653+0000","last_became_peered":"2024-06-28T15:00:19.404653+0000","last_unstale":"2024-06-28T16:11:50.273547+0000","last_undegraded":"2024-06-28T16:11:50.273547+0000","last_fullsized":"2024-06-28T16:11:50.273547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:18:41.227360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"260'246","reported_seq":1691,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:11.127801+0000","last_change":"2024-06-28T15:00:21.484753+0000","last_active":"2024-06-28T16:14:11.127801+0000","last_peered":"2024-06-28T16:14:11.127801+0000","last_clean":"2024-06-28T16:14:11.127801+0000","last_became_active":"2024-06-28T15:00:21.483749+0000","last_became_peered":"2024-06-28T15:00:21.483749+0000","last_unstale":"2024-06-28T16:14:11.127801+0000","last_undegraded":"2024-06-28T16:14:11.127801+0000","last_fullsized":"2024-06-28T16:14:11.127801+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:25:11.741395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.558850+0000","last_change":"2024-06-28T15:00:23.400124+0000","last_active":"2024-06-28T16:11:15.558850+0000","last_peered":"2024-06-28T16:11:15.558850+0000","last_clean":"2024-06-28T16:11:15.558850+0000","last_became_active":"2024-06-28T15:00:23.399391+0000","last_became_peered":"2024-06-28T15:00:23.399391+0000","last_unstale":"2024-06-28T16:11:15.558850+0000","last_undegraded":"2024-06-28T16:11:15.558850+0000","last_fullsized":"2024-06-28T16:11:15.558850+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:03:52.771119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"260'239","reported_seq":844,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:07.109352+0000","last_change":"2024-06-28T15:00:25.420452+0000","last_active":"2024-06-28T16:14:07.109352+0000","last_peered":"2024-06-28T16:14:07.109352+0000","last_clean":"2024-06-28T16:14:07.109352+0000","last_became_active":"2024-06-28T15:00:25.419711+0000","last_became_peered":"2024-06-28T15:00:25.419711+0000","last_unstale":"2024-06-28T16:14:07.109352+0000","last_undegraded":"2024-06-28T16:14:07.109352+0000","last_fullsized":"2024-06-28T16:14:07.109352+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":200,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:31:14.431698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":467,"num_read_kb":491,"num_write":517,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"260'315","reported_seq":1006,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:04.988991+0000","last_change":"2024-06-28T15:00:17.398693+0000","last_active":"2024-06-28T16:12:04.988991+0000","last_peered":"2024-06-28T16:12:04.988991+0000","last_clean":"2024-06-28T16:12:04.988991+0000","last_became_active":"2024-06-28T15:00:17.393696+0000","last_became_peered":"2024-06-28T15:00:17.393696+0000","last_unstale":"2024-06-28T16:12:04.988991+0000","last_undegraded":"2024-06-28T16:12:04.988991+0000","last_fullsized":"2024-06-28T16:12:04.988991+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:48:40.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1365,"num_read_kb":1154,"num_write":468,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.074668+0000","last_change":"2024-06-28T15:00:06.284111+0000","last_active":"2024-06-28T16:10:55.074668+0000","last_peered":"2024-06-28T16:10:55.074668+0000","last_clean":"2024-06-28T16:10:55.074668+0000","last_became_active":"2024-06-28T15:00:06.283106+0000","last_became_peered":"2024-06-28T15:00:06.283106+0000","last_unstale":"2024-06-28T16:10:55.074668+0000","last_undegraded":"2024-06-28T16:10:55.074668+0000","last_fullsized":"2024-06-28T16:10:55.074668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:07.069940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"260'318","reported_seq":991,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.731218+0000","last_change":"2024-06-28T15:00:15.415068+0000","last_active":"2024-06-28T16:14:34.731218+0000","last_peered":"2024-06-28T16:14:34.731218+0000","last_clean":"2024-06-28T16:14:34.731218+0000","last_became_active":"2024-06-28T15:00:15.413880+0000","last_became_peered":"2024-06-28T15:00:15.413880+0000","last_unstale":"2024-06-28T16:14:34.731218+0000","last_undegraded":"2024-06-28T16:14:34.731218+0000","last_fullsized":"2024-06-28T16:14:34.731218+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:13:10.361144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2421978,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":465,"num_read_kb":85416,"num_write":1148,"num_write_kb":27167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.561459+0000","last_change":"2024-06-28T15:00:19.382869+0000","last_active":"2024-06-28T16:11:25.561459+0000","last_peered":"2024-06-28T16:11:25.561459+0000","last_clean":"2024-06-28T16:11:25.561459+0000","last_became_active":"2024-06-28T15:00:19.382486+0000","last_became_peered":"2024-06-28T15:00:19.382486+0000","last_unstale":"2024-06-28T16:11:25.561459+0000","last_undegraded":"2024-06-28T16:11:25.561459+0000","last_fullsized":"2024-06-28T16:11:25.561459+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:22:55.369992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"260'159","reported_seq":729,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.514897+0000","last_change":"2024-06-28T15:00:21.401345+0000","last_active":"2024-06-28T16:11:55.514897+0000","last_peered":"2024-06-28T16:11:55.514897+0000","last_clean":"2024-06-28T16:11:55.514897+0000","last_became_active":"2024-06-28T15:00:21.400535+0000","last_became_peered":"2024-06-28T15:00:21.400535+0000","last_unstale":"2024-06-28T16:11:55.514897+0000","last_undegraded":"2024-06-28T16:11:55.514897+0000","last_fullsized":"2024-06-28T16:11:55.514897+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:33:23.802275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":198,"num_read_kb":191,"num_write":142,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":495,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.094802+0000","last_change":"2024-06-28T15:00:23.417433+0000","last_active":"2024-06-28T16:11:55.094802+0000","last_peered":"2024-06-28T16:11:55.094802+0000","last_clean":"2024-06-28T16:11:55.094802+0000","last_became_active":"2024-06-28T15:00:23.416701+0000","last_became_peered":"2024-06-28T15:00:23.416701+0000","last_unstale":"2024-06-28T16:11:55.094802+0000","last_undegraded":"2024-06-28T16:11:55.094802+0000","last_fullsized":"2024-06-28T16:11:55.094802+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:59:56.613956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"260'117","reported_seq":633,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:07.191149+0000","last_change":"2024-06-28T15:00:25.439202+0000","last_active":"2024-06-28T16:13:07.191149+0000","last_peered":"2024-06-28T16:13:07.191149+0000","last_clean":"2024-06-28T16:13:07.191149+0000","last_became_active":"2024-06-28T15:00:25.438523+0000","last_became_peered":"2024-06-28T15:00:25.438523+0000","last_unstale":"2024-06-28T16:13:07.191149+0000","last_undegraded":"2024-06-28T16:13:07.191149+0000","last_fullsized":"2024-06-28T16:13:07.191149+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:31:55.560360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":129,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"260'169","reported_seq":774,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.093820+0000","last_change":"2024-06-28T15:00:17.399083+0000","last_active":"2024-06-28T16:11:10.093820+0000","last_peered":"2024-06-28T16:11:10.093820+0000","last_clean":"2024-06-28T16:11:10.093820+0000","last_became_active":"2024-06-28T15:00:17.398515+0000","last_became_peered":"2024-06-28T15:00:17.398515+0000","last_unstale":"2024-06-28T16:11:10.093820+0000","last_undegraded":"2024-06-28T16:11:10.093820+0000","last_fullsized":"2024-06-28T16:11:10.093820+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:10.485888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":644,"num_read_kb":512,"num_write":273,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"260'372","reported_seq":1043,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.710966+0000","last_change":"2024-06-28T15:00:15.417442+0000","last_active":"2024-06-28T16:14:34.710966+0000","last_peered":"2024-06-28T16:14:34.710966+0000","last_clean":"2024-06-28T16:14:34.710966+0000","last_became_active":"2024-06-28T15:00:15.414704+0000","last_became_peered":"2024-06-28T15:00:15.414704+0000","last_unstale":"2024-06-28T16:14:34.710966+0000","last_undegraded":"2024-06-28T16:14:34.710966+0000","last_fullsized":"2024-06-28T16:14:34.710966+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:54:15.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10506361,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":617,"num_read_kb":33619,"num_write":1289,"num_write_kb":23520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":529,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.092017+0000","last_change":"2024-06-28T15:00:06.302842+0000","last_active":"2024-06-28T16:11:00.092017+0000","last_peered":"2024-06-28T16:11:00.092017+0000","last_clean":"2024-06-28T16:11:00.092017+0000","last_became_active":"2024-06-28T15:00:06.302449+0000","last_became_peered":"2024-06-28T15:00:06.302449+0000","last_unstale":"2024-06-28T16:11:00.092017+0000","last_undegraded":"2024-06-28T16:11:00.092017+0000","last_fullsized":"2024-06-28T16:11:00.092017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:06:10.739681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.557574+0000","last_change":"2024-06-28T15:00:19.385492+0000","last_active":"2024-06-28T16:11:10.557574+0000","last_peered":"2024-06-28T16:11:10.557574+0000","last_clean":"2024-06-28T16:11:10.557574+0000","last_became_active":"2024-06-28T15:00:19.384933+0000","last_became_peered":"2024-06-28T15:00:19.384933+0000","last_unstale":"2024-06-28T16:11:10.557574+0000","last_undegraded":"2024-06-28T16:11:10.557574+0000","last_fullsized":"2024-06-28T16:11:10.557574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:16:09.011700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"260'577","reported_seq":1265,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:12.941428+0000","last_change":"2024-06-28T15:00:21.407998+0000","last_active":"2024-06-28T16:14:12.941428+0000","last_peered":"2024-06-28T16:14:12.941428+0000","last_clean":"2024-06-28T16:14:12.941428+0000","last_became_active":"2024-06-28T15:00:21.405762+0000","last_became_peered":"2024-06-28T15:00:21.405762+0000","last_unstale":"2024-06-28T16:14:12.941428+0000","last_undegraded":"2024-06-28T16:14:12.941428+0000","last_fullsized":"2024-06-28T16:14:12.941428+0000","mapping_epoch":25,"log_start":"260'500","ondisk_log_start":"260'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":500,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:56:43.045836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111793,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":975,"num_read_kb":1983,"num_write":826,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.086163+0000","last_change":"2024-06-28T15:00:23.405517+0000","last_active":"2024-06-28T16:11:15.086163+0000","last_peered":"2024-06-28T16:11:15.086163+0000","last_clean":"2024-06-28T16:11:15.086163+0000","last_became_active":"2024-06-28T15:00:23.404714+0000","last_became_peered":"2024-06-28T15:00:23.404714+0000","last_unstale":"2024-06-28T16:11:15.086163+0000","last_undegraded":"2024-06-28T16:11:15.086163+0000","last_fullsized":"2024-06-28T16:11:15.086163+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:54:34.158419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"260'147","reported_seq":663,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.650237+0000","last_change":"2024-06-28T15:00:25.557678+0000","last_active":"2024-06-28T16:14:06.650237+0000","last_peered":"2024-06-28T16:14:06.650237+0000","last_clean":"2024-06-28T16:14:06.650237+0000","last_became_active":"2024-06-28T15:00:25.556834+0000","last_became_peered":"2024-06-28T15:00:25.556834+0000","last_unstale":"2024-06-28T16:14:06.650237+0000","last_undegraded":"2024-06-28T16:14:06.650237+0000","last_fullsized":"2024-06-28T16:14:06.650237+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":100,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:52:25.536046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1183,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":342,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"260'139","reported_seq":768,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.557529+0000","last_change":"2024-06-28T15:00:17.372769+0000","last_active":"2024-06-28T16:11:10.557529+0000","last_peered":"2024-06-28T16:11:10.557529+0000","last_clean":"2024-06-28T16:11:10.557529+0000","last_became_active":"2024-06-28T15:00:17.372401+0000","last_became_peered":"2024-06-28T15:00:17.372401+0000","last_unstale":"2024-06-28T16:11:10.557529+0000","last_undegraded":"2024-06-28T16:11:10.557529+0000","last_fullsized":"2024-06-28T16:11:10.557529+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:34:11.796111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":577,"num_read_kb":470,"num_write":221,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"260'273","reported_seq":901,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.481261+0000","last_change":"2024-06-28T15:00:15.408605+0000","last_active":"2024-06-28T16:14:34.481261+0000","last_peered":"2024-06-28T16:14:34.481261+0000","last_clean":"2024-06-28T16:14:34.481261+0000","last_became_active":"2024-06-28T15:00:15.408140+0000","last_became_peered":"2024-06-28T15:00:15.408140+0000","last_unstale":"2024-06-28T16:14:34.481261+0000","last_undegraded":"2024-06-28T16:14:34.481261+0000","last_fullsized":"2024-06-28T16:14:34.481261+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":200,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:51:15.618780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":526399,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":387,"num_read_kb":8190,"num_write":1081,"num_write_kb":24156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.093729+0000","last_change":"2024-06-28T15:00:06.303984+0000","last_active":"2024-06-28T16:11:15.093729+0000","last_peered":"2024-06-28T16:11:15.093729+0000","last_clean":"2024-06-28T16:11:15.093729+0000","last_became_active":"2024-06-28T15:00:06.302962+0000","last_became_peered":"2024-06-28T15:00:06.302962+0000","last_unstale":"2024-06-28T16:11:15.093729+0000","last_undegraded":"2024-06-28T16:11:15.093729+0000","last_fullsized":"2024-06-28T16:11:15.093729+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:10:47.354122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.084030+0000","last_change":"2024-06-28T15:00:19.385776+0000","last_active":"2024-06-28T16:11:10.084030+0000","last_peered":"2024-06-28T16:11:10.084030+0000","last_clean":"2024-06-28T16:11:10.084030+0000","last_became_active":"2024-06-28T15:00:19.384961+0000","last_became_peered":"2024-06-28T15:00:19.384961+0000","last_unstale":"2024-06-28T16:11:10.084030+0000","last_undegraded":"2024-06-28T16:11:10.084030+0000","last_fullsized":"2024-06-28T16:11:10.084030+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:23:32.384577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"260'188","reported_seq":785,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.095201+0000","last_change":"2024-06-28T15:00:21.480780+0000","last_active":"2024-06-28T16:11:15.095201+0000","last_peered":"2024-06-28T16:11:15.095201+0000","last_clean":"2024-06-28T16:11:15.095201+0000","last_became_active":"2024-06-28T15:00:21.480107+0000","last_became_peered":"2024-06-28T15:00:21.480107+0000","last_unstale":"2024-06-28T16:11:15.095201+0000","last_undegraded":"2024-06-28T16:11:15.095201+0000","last_fullsized":"2024-06-28T16:11:15.095201+0000","mapping_epoch":25,"log_start":"257'100","ondisk_log_start":"257'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:46:14.608558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":263,"num_read_kb":253,"num_write":174,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"260'565","reported_seq":2877,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.824388+0000","last_change":"2024-06-28T15:00:23.429133+0000","last_active":"2024-06-28T16:14:32.824388+0000","last_peered":"2024-06-28T16:14:32.824388+0000","last_clean":"2024-06-28T16:14:32.824388+0000","last_became_active":"2024-06-28T15:00:23.427565+0000","last_became_peered":"2024-06-28T15:00:23.427565+0000","last_unstale":"2024-06-28T16:14:32.824388+0000","last_undegraded":"2024-06-28T16:14:32.824388+0000","last_fullsized":"2024-06-28T16:14:32.824388+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":500,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:07:13.588883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"260'124","reported_seq":638,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:30.524126+0000","last_change":"2024-06-28T15:00:25.554359+0000","last_active":"2024-06-28T16:12:30.524126+0000","last_peered":"2024-06-28T16:12:30.524126+0000","last_clean":"2024-06-28T16:12:30.524126+0000","last_became_active":"2024-06-28T15:00:25.553627+0000","last_became_peered":"2024-06-28T15:00:25.553627+0000","last_unstale":"2024-06-28T16:12:30.524126+0000","last_undegraded":"2024-06-28T16:12:30.524126+0000","last_fullsized":"2024-06-28T16:12:30.524126+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:38:46.580841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":713,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":165,"num_read_kb":148,"num_write":308,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"260'172","reported_seq":825,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.522189+0000","last_change":"2024-06-28T15:00:17.377265+0000","last_active":"2024-06-28T16:12:25.522189+0000","last_peered":"2024-06-28T16:12:25.522189+0000","last_clean":"2024-06-28T16:12:25.522189+0000","last_became_active":"2024-06-28T15:00:17.376516+0000","last_became_peered":"2024-06-28T15:00:17.376516+0000","last_unstale":"2024-06-28T16:12:25.522189+0000","last_undegraded":"2024-06-28T16:12:25.522189+0000","last_fullsized":"2024-06-28T16:12:25.522189+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:46:26.820686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":722,"num_read_kb":588,"num_write":244,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"260'336","reported_seq":980,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.728953+0000","last_change":"2024-06-28T15:00:15.403594+0000","last_active":"2024-06-28T16:14:34.728953+0000","last_peered":"2024-06-28T16:14:34.728953+0000","last_clean":"2024-06-28T16:14:34.728953+0000","last_became_active":"2024-06-28T15:00:15.403032+0000","last_became_peered":"2024-06-28T15:00:15.403032+0000","last_unstale":"2024-06-28T16:14:34.728953+0000","last_undegraded":"2024-06-28T16:14:34.728953+0000","last_fullsized":"2024-06-28T16:14:34.728953+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:18:51.571850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":968026,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":483,"num_read_kb":17032,"num_write":1147,"num_write_kb":35885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":528,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.099353+0000","last_change":"2024-06-28T15:00:06.278832+0000","last_active":"2024-06-28T16:11:35.099353+0000","last_peered":"2024-06-28T16:11:35.099353+0000","last_clean":"2024-06-28T16:11:35.099353+0000","last_became_active":"2024-06-28T15:00:06.278443+0000","last_became_peered":"2024-06-28T15:00:06.278443+0000","last_unstale":"2024-06-28T16:11:35.099353+0000","last_undegraded":"2024-06-28T16:11:35.099353+0000","last_fullsized":"2024-06-28T16:11:35.099353+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:40:27.400018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.085018+0000","last_change":"2024-06-28T15:00:19.380916+0000","last_active":"2024-06-28T16:11:10.085018+0000","last_peered":"2024-06-28T16:11:10.085018+0000","last_clean":"2024-06-28T16:11:10.085018+0000","last_became_active":"2024-06-28T15:00:19.378932+0000","last_became_peered":"2024-06-28T15:00:19.378932+0000","last_unstale":"2024-06-28T16:11:10.085018+0000","last_undegraded":"2024-06-28T16:11:10.085018+0000","last_fullsized":"2024-06-28T16:11:10.085018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:36:40.033839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"260'138","reported_seq":698,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.086320+0000","last_change":"2024-06-28T15:00:21.396435+0000","last_active":"2024-06-28T16:11:15.086320+0000","last_peered":"2024-06-28T16:11:15.086320+0000","last_clean":"2024-06-28T16:11:15.086320+0000","last_became_active":"2024-06-28T15:00:21.395462+0000","last_became_peered":"2024-06-28T15:00:21.395462+0000","last_unstale":"2024-06-28T16:11:15.086320+0000","last_undegraded":"2024-06-28T16:11:15.086320+0000","last_fullsized":"2024-06-28T16:11:15.086320+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:51:41.956055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":167,"num_read_kb":161,"num_write":112,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"260'537","reported_seq":2581,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.259280+0000","last_change":"2024-06-28T15:00:23.428281+0000","last_active":"2024-06-28T16:14:34.259280+0000","last_peered":"2024-06-28T16:14:34.259280+0000","last_clean":"2024-06-28T16:14:34.259280+0000","last_became_active":"2024-06-28T15:00:23.427568+0000","last_became_peered":"2024-06-28T15:00:23.427568+0000","last_unstale":"2024-06-28T16:14:34.259280+0000","last_undegraded":"2024-06-28T16:14:34.259280+0000","last_fullsized":"2024-06-28T16:14:34.259280+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:37:02.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"260'247","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.544258+0000","last_change":"2024-06-28T15:00:25.428796+0000","last_active":"2024-06-28T16:10:10.544258+0000","last_peered":"2024-06-28T16:10:10.544258+0000","last_clean":"2024-06-28T16:10:10.544258+0000","last_became_active":"2024-06-28T15:00:25.428067+0000","last_became_peered":"2024-06-28T15:00:25.428067+0000","last_unstale":"2024-06-28T16:10:10.544258+0000","last_undegraded":"2024-06-28T16:10:10.544258+0000","last_fullsized":"2024-06-28T16:10:10.544258+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":200,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:07:43.747052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":297,"num_write":552,"num_write_kb":170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"260'96","reported_seq":701,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:46.633284+0000","last_change":"2024-06-28T15:00:17.397233+0000","last_active":"2024-06-28T16:12:46.633284+0000","last_peered":"2024-06-28T16:12:46.633284+0000","last_clean":"2024-06-28T16:12:46.633284+0000","last_became_active":"2024-06-28T15:00:17.396413+0000","last_became_peered":"2024-06-28T15:00:17.396413+0000","last_unstale":"2024-06-28T16:12:46.633284+0000","last_undegraded":"2024-06-28T16:12:46.633284+0000","last_fullsized":"2024-06-28T16:12:46.633284+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:50:29.409467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":410,"num_read_kb":340,"num_write":139,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"260'261","reported_seq":1040,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:45.565343+0000","last_change":"2024-06-28T15:00:15.396426+0000","last_active":"2024-06-28T16:11:45.565343+0000","last_peered":"2024-06-28T16:11:45.565343+0000","last_clean":"2024-06-28T16:11:45.565343+0000","last_became_active":"2024-06-28T15:00:15.395980+0000","last_became_peered":"2024-06-28T15:00:15.395980+0000","last_unstale":"2024-06-28T16:11:45.565343+0000","last_undegraded":"2024-06-28T16:11:45.565343+0000","last_fullsized":"2024-06-28T16:11:45.565343+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:22:49.730474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2049512,"num_objects":63,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":592,"num_read_kb":125515,"num_write":1130,"num_write_kb":17513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":521,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089897+0000","last_change":"2024-06-28T15:00:06.284169+0000","last_active":"2024-06-28T16:11:35.089897+0000","last_peered":"2024-06-28T16:11:35.089897+0000","last_clean":"2024-06-28T16:11:35.089897+0000","last_became_active":"2024-06-28T15:00:06.283380+0000","last_became_peered":"2024-06-28T15:00:06.283380+0000","last_unstale":"2024-06-28T16:11:35.089897+0000","last_undegraded":"2024-06-28T16:11:35.089897+0000","last_fullsized":"2024-06-28T16:11:35.089897+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:39:59.368666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"31'5","reported_seq":842,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:31.932486+0000","last_change":"2024-06-28T15:00:19.372913+0000","last_active":"2024-06-28T16:14:31.932486+0000","last_peered":"2024-06-28T16:14:31.932486+0000","last_clean":"2024-06-28T16:14:31.932486+0000","last_became_active":"2024-06-28T15:00:19.372552+0000","last_became_peered":"2024-06-28T15:00:19.372552+0000","last_unstale":"2024-06-28T16:14:31.932486+0000","last_undegraded":"2024-06-28T16:14:31.932486+0000","last_fullsized":"2024-06-28T16:14:31.932486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:19:05.828246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":500,"num_read_kb":812,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"260'748","reported_seq":1492,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.576522+0000","last_change":"2024-06-28T15:00:21.481881+0000","last_active":"2024-06-28T16:14:33.576522+0000","last_peered":"2024-06-28T16:14:33.576522+0000","last_clean":"2024-06-28T16:14:33.576522+0000","last_became_active":"2024-06-28T15:00:21.481155+0000","last_became_peered":"2024-06-28T15:00:21.481155+0000","last_unstale":"2024-06-28T16:14:33.576522+0000","last_undegraded":"2024-06-28T16:14:33.576522+0000","last_fullsized":"2024-06-28T16:14:33.576522+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":700,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:24:28.507362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131438,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1320,"num_read_kb":3166,"num_write":1058,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.505408+0000","last_change":"2024-06-28T15:00:23.409553+0000","last_active":"2024-06-28T16:11:15.505408+0000","last_peered":"2024-06-28T16:11:15.505408+0000","last_clean":"2024-06-28T16:11:15.505408+0000","last_became_active":"2024-06-28T15:00:23.408777+0000","last_became_peered":"2024-06-28T15:00:23.408777+0000","last_unstale":"2024-06-28T16:11:15.505408+0000","last_undegraded":"2024-06-28T16:11:15.505408+0000","last_fullsized":"2024-06-28T16:11:15.505408+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:29:28.695608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"260'129","reported_seq":640,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:07.935429+0000","last_change":"2024-06-28T15:00:25.439187+0000","last_active":"2024-06-28T16:14:07.935429+0000","last_peered":"2024-06-28T16:14:07.935429+0000","last_clean":"2024-06-28T16:14:07.935429+0000","last_became_active":"2024-06-28T15:00:25.438476+0000","last_became_peered":"2024-06-28T15:00:25.438476+0000","last_unstale":"2024-06-28T16:14:07.935429+0000","last_undegraded":"2024-06-28T16:14:07.935429+0000","last_fullsized":"2024-06-28T16:14:07.935429+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:14:10.401700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":173,"num_read_kb":156,"num_write":320,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"260'151","reported_seq":718,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.829530+0000","last_change":"2024-06-28T15:00:25.440427+0000","last_active":"2024-06-28T16:14:08.829530+0000","last_peered":"2024-06-28T16:14:08.829530+0000","last_clean":"2024-06-28T16:14:08.829530+0000","last_became_active":"2024-06-28T15:00:25.439672+0000","last_became_peered":"2024-06-28T15:00:25.439672+0000","last_unstale":"2024-06-28T16:14:08.829530+0000","last_undegraded":"2024-06-28T16:14:08.829530+0000","last_fullsized":"2024-06-28T16:14:08.829530+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:21:43.580938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":207,"num_read_kb":205,"num_write":345,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"260'434","reported_seq":1071,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:58.995763+0000","last_change":"2024-06-28T15:00:17.402772+0000","last_active":"2024-06-28T16:10:58.995763+0000","last_peered":"2024-06-28T16:10:58.995763+0000","last_clean":"2024-06-28T16:10:58.995763+0000","last_became_active":"2024-06-28T15:00:17.401981+0000","last_became_peered":"2024-06-28T15:00:17.401981+0000","last_unstale":"2024-06-28T16:10:58.995763+0000","last_undegraded":"2024-06-28T16:10:58.995763+0000","last_fullsized":"2024-06-28T16:10:58.995763+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":400,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:36:44.706683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1728,"num_read_kb":1391,"num_write":746,"num_write_kb":387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"260'274","reported_seq":900,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.557629+0000","last_change":"2024-06-28T15:00:15.404138+0000","last_active":"2024-06-28T16:14:34.557629+0000","last_peered":"2024-06-28T16:14:34.557629+0000","last_clean":"2024-06-28T16:14:34.557629+0000","last_became_active":"2024-06-28T15:00:15.403450+0000","last_became_peered":"2024-06-28T15:00:15.403450+0000","last_unstale":"2024-06-28T16:14:34.557629+0000","last_undegraded":"2024-06-28T16:14:34.557629+0000","last_fullsized":"2024-06-28T16:14:34.557629+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:20:44.673833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437799,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":13452,"num_write":1073,"num_write_kb":25751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.084184+0000","last_change":"2024-06-28T15:00:19.398728+0000","last_active":"2024-06-28T16:10:20.084184+0000","last_peered":"2024-06-28T16:10:20.084184+0000","last_clean":"2024-06-28T16:10:20.084184+0000","last_became_active":"2024-06-28T15:00:19.398138+0000","last_became_peered":"2024-06-28T15:00:19.398138+0000","last_unstale":"2024-06-28T16:10:20.084184+0000","last_undegraded":"2024-06-28T16:10:20.084184+0000","last_fullsized":"2024-06-28T16:10:20.084184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:24:09.555887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"260'298","reported_seq":895,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.023577+0000","last_change":"2024-06-28T15:00:21.405835+0000","last_active":"2024-06-28T16:14:34.023577+0000","last_peered":"2024-06-28T16:14:34.023577+0000","last_clean":"2024-06-28T16:14:34.023577+0000","last_became_active":"2024-06-28T15:00:21.405016+0000","last_became_peered":"2024-06-28T15:00:21.405016+0000","last_unstale":"2024-06-28T16:14:34.023577+0000","last_undegraded":"2024-06-28T16:14:34.023577+0000","last_fullsized":"2024-06-28T16:14:34.023577+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:00:56.662170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53726,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":505,"num_read_kb":1165,"num_write":406,"num_write_kb":309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":499,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493149+0000","last_change":"2024-06-28T15:00:23.418156+0000","last_active":"2024-06-28T16:10:25.493149+0000","last_peered":"2024-06-28T16:10:25.493149+0000","last_clean":"2024-06-28T16:10:25.493149+0000","last_became_active":"2024-06-28T15:00:23.417390+0000","last_became_peered":"2024-06-28T15:00:23.417390+0000","last_unstale":"2024-06-28T16:10:25.493149+0000","last_undegraded":"2024-06-28T16:10:25.493149+0000","last_fullsized":"2024-06-28T16:10:25.493149+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:22:25.288900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"260'101","reported_seq":608,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:30.524057+0000","last_change":"2024-06-28T15:00:25.413154+0000","last_active":"2024-06-28T16:12:30.524057+0000","last_peered":"2024-06-28T16:12:30.524057+0000","last_clean":"2024-06-28T16:12:30.524057+0000","last_became_active":"2024-06-28T15:00:25.412379+0000","last_became_peered":"2024-06-28T15:00:25.412379+0000","last_unstale":"2024-06-28T16:12:30.524057+0000","last_undegraded":"2024-06-28T16:12:30.524057+0000","last_fullsized":"2024-06-28T16:12:30.524057+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:26:49.949836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":88,"num_write":238,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"260'189","reported_seq":822,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:47.151324+0000","last_change":"2024-06-28T15:00:17.406956+0000","last_active":"2024-06-28T16:11:47.151324+0000","last_peered":"2024-06-28T16:11:47.151324+0000","last_clean":"2024-06-28T16:11:47.151324+0000","last_became_active":"2024-06-28T15:00:17.406647+0000","last_became_peered":"2024-06-28T15:00:17.406647+0000","last_unstale":"2024-06-28T16:11:47.151324+0000","last_undegraded":"2024-06-28T16:11:47.151324+0000","last_fullsized":"2024-06-28T16:11:47.151324+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:18:05.135571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":748,"num_read_kb":601,"num_write":301,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"260'334","reported_seq":1001,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.613707+0000","last_change":"2024-06-28T15:00:15.413269+0000","last_active":"2024-06-28T16:14:34.613707+0000","last_peered":"2024-06-28T16:14:34.613707+0000","last_clean":"2024-06-28T16:14:34.613707+0000","last_became_active":"2024-06-28T15:00:15.402945+0000","last_became_peered":"2024-06-28T15:00:15.402945+0000","last_unstale":"2024-06-28T16:14:34.613707+0000","last_undegraded":"2024-06-28T16:14:34.613707+0000","last_fullsized":"2024-06-28T16:14:34.613707+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:41:27.586797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340417,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":534,"num_read_kb":20609,"num_write":1305,"num_write_kb":45266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,4],"acting":[7,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.492088+0000","last_change":"2024-06-28T15:00:19.383614+0000","last_active":"2024-06-28T16:10:20.492088+0000","last_peered":"2024-06-28T16:10:20.492088+0000","last_clean":"2024-06-28T16:10:20.492088+0000","last_became_active":"2024-06-28T15:00:19.382839+0000","last_became_peered":"2024-06-28T15:00:19.382839+0000","last_unstale":"2024-06-28T16:10:20.492088+0000","last_undegraded":"2024-06-28T16:10:20.492088+0000","last_fullsized":"2024-06-28T16:10:20.492088+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:09:23.668167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"260'52","reported_seq":573,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:58.477001+0000","last_change":"2024-06-28T15:00:21.407387+0000","last_active":"2024-06-28T16:10:58.477001+0000","last_peered":"2024-06-28T16:10:58.477001+0000","last_clean":"2024-06-28T16:10:58.477001+0000","last_became_active":"2024-06-28T15:00:21.404727+0000","last_became_peered":"2024-06-28T15:00:21.404727+0000","last_unstale":"2024-06-28T16:10:58.477001+0000","last_undegraded":"2024-06-28T16:10:58.477001+0000","last_fullsized":"2024-06-28T16:10:58.477001+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:02:08.588463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":48,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":494,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.081639+0000","last_change":"2024-06-28T15:00:23.419840+0000","last_active":"2024-06-28T16:10:25.081639+0000","last_peered":"2024-06-28T16:10:25.081639+0000","last_clean":"2024-06-28T16:10:25.081639+0000","last_became_active":"2024-06-28T15:00:23.419115+0000","last_became_peered":"2024-06-28T15:00:23.419115+0000","last_unstale":"2024-06-28T16:10:25.081639+0000","last_undegraded":"2024-06-28T16:10:25.081639+0000","last_fullsized":"2024-06-28T16:10:25.081639+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:05:05.710065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"260'190","reported_seq":719,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.664824+0000","last_change":"2024-06-28T15:00:25.426812+0000","last_active":"2024-06-28T16:14:06.664824+0000","last_peered":"2024-06-28T16:14:06.664824+0000","last_clean":"2024-06-28T16:14:06.664824+0000","last_became_active":"2024-06-28T15:00:25.426046+0000","last_became_peered":"2024-06-28T15:00:25.426046+0000","last_unstale":"2024-06-28T16:14:06.664824+0000","last_undegraded":"2024-06-28T16:14:06.664824+0000","last_fullsized":"2024-06-28T16:14:06.664824+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:19:11.968158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":286,"num_read_kb":283,"num_write":434,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"260'423","reported_seq":1170,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:17.460800+0000","last_change":"2024-06-28T15:00:17.390367+0000","last_active":"2024-06-28T16:11:17.460800+0000","last_peered":"2024-06-28T16:11:17.460800+0000","last_clean":"2024-06-28T16:11:17.460800+0000","last_became_active":"2024-06-28T15:00:17.389992+0000","last_became_peered":"2024-06-28T15:00:17.389992+0000","last_unstale":"2024-06-28T16:11:17.460800+0000","last_undegraded":"2024-06-28T16:11:17.460800+0000","last_fullsized":"2024-06-28T16:11:17.460800+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":400,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:13:23.718079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1786,"num_read_kb":1490,"num_write":759,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"260'328","reported_seq":971,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.635861+0000","last_change":"2024-06-28T15:00:15.427261+0000","last_active":"2024-06-28T16:14:34.635861+0000","last_peered":"2024-06-28T16:14:34.635861+0000","last_clean":"2024-06-28T16:14:34.635861+0000","last_became_active":"2024-06-28T15:00:15.419267+0000","last_became_peered":"2024-06-28T15:00:15.419267+0000","last_unstale":"2024-06-28T16:14:34.635861+0000","last_undegraded":"2024-06-28T16:14:34.635861+0000","last_fullsized":"2024-06-28T16:14:34.635861+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:49:04.986604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5413475,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":487,"num_read_kb":27459,"num_write":1288,"num_write_kb":48933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080344+0000","last_change":"2024-06-28T15:00:19.386890+0000","last_active":"2024-06-28T16:10:20.080344+0000","last_peered":"2024-06-28T16:10:20.080344+0000","last_clean":"2024-06-28T16:10:20.080344+0000","last_became_active":"2024-06-28T15:00:19.385990+0000","last_became_peered":"2024-06-28T15:00:19.385990+0000","last_unstale":"2024-06-28T16:10:20.080344+0000","last_undegraded":"2024-06-28T16:10:20.080344+0000","last_fullsized":"2024-06-28T16:10:20.080344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:39:52.204033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"260'318","reported_seq":931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.206768+0000","last_change":"2024-06-28T15:00:21.406717+0000","last_active":"2024-06-28T16:14:33.206768+0000","last_peered":"2024-06-28T16:14:33.206768+0000","last_clean":"2024-06-28T16:14:33.206768+0000","last_became_active":"2024-06-28T15:00:21.405045+0000","last_became_peered":"2024-06-28T15:00:21.405045+0000","last_unstale":"2024-06-28T16:14:33.206768+0000","last_undegraded":"2024-06-28T16:14:33.206768+0000","last_fullsized":"2024-06-28T16:14:33.206768+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:50:38.218164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48456,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":530,"num_read_kb":1162,"num_write":414,"num_write_kb":289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":492,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.074474+0000","last_change":"2024-06-28T15:00:23.415444+0000","last_active":"2024-06-28T16:10:25.074474+0000","last_peered":"2024-06-28T16:10:25.074474+0000","last_clean":"2024-06-28T16:10:25.074474+0000","last_became_active":"2024-06-28T15:00:23.411264+0000","last_became_peered":"2024-06-28T15:00:23.411264+0000","last_unstale":"2024-06-28T16:10:25.074474+0000","last_undegraded":"2024-06-28T16:10:25.074474+0000","last_fullsized":"2024-06-28T16:10:25.074474+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:58:05.328252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"260'110","reported_seq":602,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.067825+0000","last_change":"2024-06-28T15:00:25.443135+0000","last_active":"2024-06-28T16:10:25.067825+0000","last_peered":"2024-06-28T16:10:25.067825+0000","last_clean":"2024-06-28T16:10:25.067825+0000","last_became_active":"2024-06-28T15:00:25.442010+0000","last_became_peered":"2024-06-28T15:00:25.442010+0000","last_unstale":"2024-06-28T16:10:25.067825+0000","last_undegraded":"2024-06-28T16:10:25.067825+0000","last_fullsized":"2024-06-28T16:10:25.067825+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:56:35.040791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":638,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":85,"num_write":260,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"260'105","reported_seq":669,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.491831+0000","last_change":"2024-06-28T15:00:17.376172+0000","last_active":"2024-06-28T16:10:20.491831+0000","last_peered":"2024-06-28T16:10:20.491831+0000","last_clean":"2024-06-28T16:10:20.491831+0000","last_became_active":"2024-06-28T15:00:17.373396+0000","last_became_peered":"2024-06-28T15:00:17.373396+0000","last_unstale":"2024-06-28T16:10:20.491831+0000","last_undegraded":"2024-06-28T16:10:20.491831+0000","last_fullsized":"2024-06-28T16:10:20.491831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:59:28.317922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":424,"num_read_kb":340,"num_write":155,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"260'260","reported_seq":907,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:03.731453+0000","last_change":"2024-06-28T15:00:15.412826+0000","last_active":"2024-06-28T16:12:03.731453+0000","last_peered":"2024-06-28T16:12:03.731453+0000","last_clean":"2024-06-28T16:12:03.731453+0000","last_became_active":"2024-06-28T15:00:15.412333+0000","last_became_peered":"2024-06-28T15:00:15.412333+0000","last_unstale":"2024-06-28T16:12:03.731453+0000","last_undegraded":"2024-06-28T16:12:03.731453+0000","last_fullsized":"2024-06-28T16:12:03.731453+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:32:08.060194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6832535,"num_objects":55,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":411,"num_read_kb":31119,"num_write":1229,"num_write_kb":43155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073040+0000","last_change":"2024-06-28T15:00:19.387598+0000","last_active":"2024-06-28T16:10:20.073040+0000","last_peered":"2024-06-28T16:10:20.073040+0000","last_clean":"2024-06-28T16:10:20.073040+0000","last_became_active":"2024-06-28T15:00:19.386645+0000","last_became_peered":"2024-06-28T15:00:19.386645+0000","last_unstale":"2024-06-28T16:10:20.073040+0000","last_undegraded":"2024-06-28T16:10:20.073040+0000","last_fullsized":"2024-06-28T16:10:20.073040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:54:00.701086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"260'289","reported_seq":873,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.977932+0000","last_change":"2024-06-28T15:00:21.395133+0000","last_active":"2024-06-28T16:14:32.977932+0000","last_peered":"2024-06-28T16:14:32.977932+0000","last_clean":"2024-06-28T16:14:32.977932+0000","last_became_active":"2024-06-28T15:00:21.394357+0000","last_became_peered":"2024-06-28T15:00:21.394357+0000","last_unstale":"2024-06-28T16:14:32.977932+0000","last_undegraded":"2024-06-28T16:14:32.977932+0000","last_fullsized":"2024-06-28T16:14:32.977932+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:35:53.429744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53697,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":496,"num_read_kb":1194,"num_write":404,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.548098+0000","last_change":"2024-06-28T15:00:23.407033+0000","last_active":"2024-06-28T16:10:25.548098+0000","last_peered":"2024-06-28T16:10:25.548098+0000","last_clean":"2024-06-28T16:10:25.548098+0000","last_became_active":"2024-06-28T15:00:23.406296+0000","last_became_peered":"2024-06-28T15:00:23.406296+0000","last_unstale":"2024-06-28T16:10:25.548098+0000","last_undegraded":"2024-06-28T16:10:25.548098+0000","last_fullsized":"2024-06-28T16:10:25.548098+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:09:16.742634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"260'68","reported_seq":636,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.670953+0000","last_change":"2024-06-28T15:00:17.397537+0000","last_active":"2024-06-28T16:14:06.670953+0000","last_peered":"2024-06-28T16:14:06.670953+0000","last_clean":"2024-06-28T16:14:06.670953+0000","last_became_active":"2024-06-28T15:00:17.396861+0000","last_became_peered":"2024-06-28T15:00:17.396861+0000","last_unstale":"2024-06-28T16:14:06.670953+0000","last_undegraded":"2024-06-28T16:14:06.670953+0000","last_fullsized":"2024-06-28T16:14:06.670953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:27:59.755447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":279,"num_read_kb":229,"num_write":93,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"260'309","reported_seq":944,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.727038+0000","last_change":"2024-06-28T15:00:15.383453+0000","last_active":"2024-06-28T16:14:34.727038+0000","last_peered":"2024-06-28T16:14:34.727038+0000","last_clean":"2024-06-28T16:14:34.727038+0000","last_became_active":"2024-06-28T15:00:15.383008+0000","last_became_peered":"2024-06-28T15:00:15.383008+0000","last_unstale":"2024-06-28T16:14:34.727038+0000","last_undegraded":"2024-06-28T16:14:34.727038+0000","last_fullsized":"2024-06-28T16:14:34.727038+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:16:34.579751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2156401,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":459,"num_read_kb":6724,"num_write":1144,"num_write_kb":24207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.550286+0000","last_change":"2024-06-28T15:00:19.390462+0000","last_active":"2024-06-28T16:10:35.550286+0000","last_peered":"2024-06-28T16:10:35.550286+0000","last_clean":"2024-06-28T16:10:35.550286+0000","last_became_active":"2024-06-28T15:00:19.390111+0000","last_became_peered":"2024-06-28T15:00:19.390111+0000","last_unstale":"2024-06-28T16:10:35.550286+0000","last_undegraded":"2024-06-28T16:10:35.550286+0000","last_fullsized":"2024-06-28T16:10:35.550286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:44:31.427541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"260'472","reported_seq":1122,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.468574+0000","last_change":"2024-06-28T15:00:21.399070+0000","last_active":"2024-06-28T16:14:34.468574+0000","last_peered":"2024-06-28T16:14:34.468574+0000","last_clean":"2024-06-28T16:14:34.468574+0000","last_became_active":"2024-06-28T15:00:21.398262+0000","last_became_peered":"2024-06-28T15:00:21.398262+0000","last_unstale":"2024-06-28T16:14:34.468574+0000","last_undegraded":"2024-06-28T16:14:34.468574+0000","last_fullsized":"2024-06-28T16:14:34.468574+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":400,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:19:49.030651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86979,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":863,"num_read_kb":2070,"num_write":734,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":493,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.067855+0000","last_change":"2024-06-28T15:00:23.411496+0000","last_active":"2024-06-28T16:10:25.067855+0000","last_peered":"2024-06-28T16:10:25.067855+0000","last_clean":"2024-06-28T16:10:25.067855+0000","last_became_active":"2024-06-28T15:00:23.410519+0000","last_became_peered":"2024-06-28T15:00:23.410519+0000","last_unstale":"2024-06-28T16:10:25.067855+0000","last_undegraded":"2024-06-28T16:10:25.067855+0000","last_fullsized":"2024-06-28T16:10:25.067855+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:02:03.245169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"260'167","reported_seq":684,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493320+0000","last_change":"2024-06-28T15:00:25.559194+0000","last_active":"2024-06-28T16:10:25.493320+0000","last_peered":"2024-06-28T16:10:25.493320+0000","last_clean":"2024-06-28T16:10:25.493320+0000","last_became_active":"2024-06-28T15:00:25.558426+0000","last_became_peered":"2024-06-28T15:00:25.558426+0000","last_unstale":"2024-06-28T16:10:25.493320+0000","last_undegraded":"2024-06-28T16:10:25.493320+0000","last_fullsized":"2024-06-28T16:10:25.493320+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":100,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:12:12.643863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":845,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":215,"num_read_kb":219,"num_write":377,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"260'158","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.550235+0000","last_change":"2024-06-28T15:00:17.395953+0000","last_active":"2024-06-28T16:10:35.550235+0000","last_peered":"2024-06-28T16:10:35.550235+0000","last_clean":"2024-06-28T16:10:35.550235+0000","last_became_active":"2024-06-28T15:00:17.395643+0000","last_became_peered":"2024-06-28T15:00:17.395643+0000","last_unstale":"2024-06-28T16:10:35.550235+0000","last_undegraded":"2024-06-28T16:10:35.550235+0000","last_fullsized":"2024-06-28T16:10:35.550235+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:17:26.238377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":654,"num_read_kb":561,"num_write":257,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"260'290","reported_seq":930,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.634482+0000","last_change":"2024-06-28T15:00:59.940407+0000","last_active":"2024-06-28T16:14:34.634482+0000","last_peered":"2024-06-28T16:14:34.634482+0000","last_clean":"2024-06-28T16:14:34.634482+0000","last_became_active":"2024-06-28T15:00:59.939485+0000","last_became_peered":"2024-06-28T15:00:59.939485+0000","last_unstale":"2024-06-28T16:14:34.634482+0000","last_undegraded":"2024-06-28T16:14:34.634482+0000","last_fullsized":"2024-06-28T16:14:34.634482+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:30:29.418898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":430,"num_read_kb":21907,"num_write":1034,"num_write_kb":35159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,6],"acting":[1,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.084186+0000","last_change":"2024-06-28T15:00:19.385764+0000","last_active":"2024-06-28T16:11:10.084186+0000","last_peered":"2024-06-28T16:11:10.084186+0000","last_clean":"2024-06-28T16:11:10.084186+0000","last_became_active":"2024-06-28T15:00:19.385068+0000","last_became_peered":"2024-06-28T15:00:19.385068+0000","last_unstale":"2024-06-28T16:11:10.084186+0000","last_undegraded":"2024-06-28T16:11:10.084186+0000","last_fullsized":"2024-06-28T16:11:10.084186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:18:40.223976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"260'348","reported_seq":954,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.054962+0000","last_change":"2024-06-28T15:00:21.401177+0000","last_active":"2024-06-28T16:14:33.054962+0000","last_peered":"2024-06-28T16:14:33.054962+0000","last_clean":"2024-06-28T16:14:33.054962+0000","last_became_active":"2024-06-28T15:00:21.400480+0000","last_became_peered":"2024-06-28T15:00:21.400480+0000","last_unstale":"2024-06-28T16:14:33.054962+0000","last_undegraded":"2024-06-28T16:14:33.054962+0000","last_fullsized":"2024-06-28T16:14:33.054962+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":300,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:08:38.525781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47708,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":581,"num_read_kb":1249,"num_write":480,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":498,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.085271+0000","last_change":"2024-06-28T15:00:23.418653+0000","last_active":"2024-06-28T16:10:25.085271+0000","last_peered":"2024-06-28T16:10:25.085271+0000","last_clean":"2024-06-28T16:10:25.085271+0000","last_became_active":"2024-06-28T15:00:23.417580+0000","last_became_peered":"2024-06-28T15:00:23.417580+0000","last_unstale":"2024-06-28T16:10:25.085271+0000","last_undegraded":"2024-06-28T16:10:25.085271+0000","last_fullsized":"2024-06-28T16:10:25.085271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:22:55.183479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"260'170","reported_seq":677,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:07.865086+0000","last_change":"2024-06-28T15:00:25.431039+0000","last_active":"2024-06-28T16:14:07.865086+0000","last_peered":"2024-06-28T16:14:07.865086+0000","last_clean":"2024-06-28T16:14:07.865086+0000","last_became_active":"2024-06-28T15:00:25.430299+0000","last_became_peered":"2024-06-28T15:00:25.430299+0000","last_unstale":"2024-06-28T16:14:07.865086+0000","last_undegraded":"2024-06-28T16:14:07.865086+0000","last_fullsized":"2024-06-28T16:14:07.865086+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:31:33.565366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2035,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":193,"num_read_kb":191,"num_write":477,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"260'315","reported_seq":1057,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.056426+0000","last_change":"2024-06-28T15:00:17.383684+0000","last_active":"2024-06-28T16:10:30.056426+0000","last_peered":"2024-06-28T16:10:30.056426+0000","last_clean":"2024-06-28T16:10:30.056426+0000","last_became_active":"2024-06-28T15:00:17.383284+0000","last_became_peered":"2024-06-28T15:00:17.383284+0000","last_unstale":"2024-06-28T16:10:30.056426+0000","last_undegraded":"2024-06-28T16:10:30.056426+0000","last_fullsized":"2024-06-28T16:10:30.056426+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:36:05.869810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1515,"num_read_kb":1335,"num_write":548,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"260'347","reported_seq":1028,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.674376+0000","last_change":"2024-06-28T15:00:15.406314+0000","last_active":"2024-06-28T16:14:34.674376+0000","last_peered":"2024-06-28T16:14:34.674376+0000","last_clean":"2024-06-28T16:14:34.674376+0000","last_became_active":"2024-06-28T15:00:15.405499+0000","last_became_peered":"2024-06-28T15:00:15.405499+0000","last_unstale":"2024-06-28T16:14:34.674376+0000","last_undegraded":"2024-06-28T16:14:34.674376+0000","last_fullsized":"2024-06-28T16:14:34.674376+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:26:47.268928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17426850,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":585,"num_read_kb":18043,"num_write":1303,"num_write_kb":33783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.074018+0000","last_change":"2024-06-28T15:00:19.381603+0000","last_active":"2024-06-28T16:10:20.074018+0000","last_peered":"2024-06-28T16:10:20.074018+0000","last_clean":"2024-06-28T16:10:20.074018+0000","last_became_active":"2024-06-28T15:00:19.378852+0000","last_became_peered":"2024-06-28T15:00:19.378852+0000","last_unstale":"2024-06-28T16:10:20.074018+0000","last_undegraded":"2024-06-28T16:10:20.074018+0000","last_fullsized":"2024-06-28T16:10:20.074018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:45:27.874936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"260'279","reported_seq":880,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.314802+0000","last_change":"2024-06-28T15:00:21.481610+0000","last_active":"2024-06-28T16:14:34.314802+0000","last_peered":"2024-06-28T16:14:34.314802+0000","last_clean":"2024-06-28T16:14:34.314802+0000","last_became_active":"2024-06-28T15:00:21.480868+0000","last_became_peered":"2024-06-28T15:00:21.480868+0000","last_unstale":"2024-06-28T16:14:34.314802+0000","last_undegraded":"2024-06-28T16:14:34.314802+0000","last_fullsized":"2024-06-28T16:14:34.314802+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":200,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:31:53.245138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40850,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":446,"num_read_kb":956,"num_write":348,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:45.089049+0000","last_change":"2024-06-28T15:00:23.401297+0000","last_active":"2024-06-28T16:10:45.089049+0000","last_peered":"2024-06-28T16:10:45.089049+0000","last_clean":"2024-06-28T16:10:45.089049+0000","last_became_active":"2024-06-28T15:00:23.400417+0000","last_became_peered":"2024-06-28T15:00:23.400417+0000","last_unstale":"2024-06-28T16:10:45.089049+0000","last_undegraded":"2024-06-28T16:10:45.089049+0000","last_fullsized":"2024-06-28T16:10:45.089049+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:57.286666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"260'117","reported_seq":617,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:12.999377+0000","last_change":"2024-06-28T15:00:25.438977+0000","last_active":"2024-06-28T16:14:12.999377+0000","last_peered":"2024-06-28T16:14:12.999377+0000","last_clean":"2024-06-28T16:14:12.999377+0000","last_became_active":"2024-06-28T15:00:25.438176+0000","last_became_peered":"2024-06-28T15:00:25.438176+0000","last_unstale":"2024-06-28T16:14:12.999377+0000","last_undegraded":"2024-06-28T16:14:12.999377+0000","last_fullsized":"2024-06-28T16:14:12.999377+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:16:21.793451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":120,"num_read_kb":111,"num_write":277,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"260'74","reported_seq":655,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.255831+0000","last_change":"2024-06-28T15:00:17.403175+0000","last_active":"2024-06-28T16:10:20.255831+0000","last_peered":"2024-06-28T16:10:20.255831+0000","last_clean":"2024-06-28T16:10:20.255831+0000","last_became_active":"2024-06-28T15:00:17.402735+0000","last_became_peered":"2024-06-28T15:00:17.402735+0000","last_unstale":"2024-06-28T16:10:20.255831+0000","last_undegraded":"2024-06-28T16:10:20.255831+0000","last_fullsized":"2024-06-28T16:10:20.255831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:31:42.762083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":299,"num_read_kb":249,"num_write":100,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"260'261","reported_seq":983,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.545545+0000","last_change":"2024-06-28T15:00:15.415658+0000","last_active":"2024-06-28T16:10:15.545545+0000","last_peered":"2024-06-28T16:10:15.545545+0000","last_clean":"2024-06-28T16:10:15.545545+0000","last_became_active":"2024-06-28T15:00:15.415197+0000","last_became_peered":"2024-06-28T15:00:15.415197+0000","last_unstale":"2024-06-28T16:10:15.545545+0000","last_undegraded":"2024-06-28T16:10:15.545545+0000","last_fullsized":"2024-06-28T16:10:15.545545+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:16:16.807609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4058475,"num_objects":80,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":573,"num_read_kb":6210,"num_write":1088,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.084302+0000","last_change":"2024-06-28T15:00:19.379957+0000","last_active":"2024-06-28T16:10:20.084302+0000","last_peered":"2024-06-28T16:10:20.084302+0000","last_clean":"2024-06-28T16:10:20.084302+0000","last_became_active":"2024-06-28T15:00:19.378270+0000","last_became_peered":"2024-06-28T15:00:19.378270+0000","last_unstale":"2024-06-28T16:10:20.084302+0000","last_undegraded":"2024-06-28T16:10:20.084302+0000","last_fullsized":"2024-06-28T16:10:20.084302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:29:00.401820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"260'101","reported_seq":646,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.625893+0000","last_change":"2024-06-28T15:00:21.485155+0000","last_active":"2024-06-28T16:11:03.625893+0000","last_peered":"2024-06-28T16:11:03.625893+0000","last_clean":"2024-06-28T16:11:03.625893+0000","last_became_active":"2024-06-28T15:00:21.484352+0000","last_became_peered":"2024-06-28T15:00:21.484352+0000","last_unstale":"2024-06-28T16:11:03.625893+0000","last_undegraded":"2024-06-28T16:11:03.625893+0000","last_fullsized":"2024-06-28T16:11:03.625893+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:35:15.271971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":124,"num_read_kb":119,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"260'515","reported_seq":2245,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.188813+0000","last_change":"2024-06-28T15:00:23.415013+0000","last_active":"2024-06-28T16:14:32.188813+0000","last_peered":"2024-06-28T16:14:32.188813+0000","last_clean":"2024-06-28T16:14:32.188813+0000","last_became_active":"2024-06-28T15:00:23.407490+0000","last_became_peered":"2024-06-28T15:00:23.407490+0000","last_unstale":"2024-06-28T16:14:32.188813+0000","last_undegraded":"2024-06-28T16:14:32.188813+0000","last_fullsized":"2024-06-28T16:14:32.188813+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:17:37.321606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"260'102","reported_seq":606,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.032920+0000","last_change":"2024-06-28T15:00:25.446074+0000","last_active":"2024-06-28T16:14:06.032920+0000","last_peered":"2024-06-28T16:14:06.032920+0000","last_clean":"2024-06-28T16:14:06.032920+0000","last_became_active":"2024-06-28T15:00:25.445018+0000","last_became_peered":"2024-06-28T15:00:25.445018+0000","last_unstale":"2024-06-28T16:14:06.032920+0000","last_undegraded":"2024-06-28T16:14:06.032920+0000","last_fullsized":"2024-06-28T16:14:06.032920+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:03:17.977593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":117,"num_read_kb":111,"num_write":241,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"260'78","reported_seq":654,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073934+0000","last_change":"2024-06-28T15:00:17.394823+0000","last_active":"2024-06-28T16:10:20.073934+0000","last_peered":"2024-06-28T16:10:20.073934+0000","last_clean":"2024-06-28T16:10:20.073934+0000","last_became_active":"2024-06-28T15:00:17.387086+0000","last_became_peered":"2024-06-28T15:00:17.387086+0000","last_unstale":"2024-06-28T16:10:20.073934+0000","last_undegraded":"2024-06-28T16:10:20.073934+0000","last_fullsized":"2024-06-28T16:10:20.073934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:21:49.516534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":333,"num_read_kb":275,"num_write":113,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"260'328","reported_seq":960,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.721439+0000","last_change":"2024-06-28T15:00:59.943256+0000","last_active":"2024-06-28T16:14:34.721439+0000","last_peered":"2024-06-28T16:14:34.721439+0000","last_clean":"2024-06-28T16:14:34.721439+0000","last_became_active":"2024-06-28T15:00:59.941915+0000","last_became_peered":"2024-06-28T15:00:59.941915+0000","last_unstale":"2024-06-28T16:14:34.721439+0000","last_undegraded":"2024-06-28T16:14:34.721439+0000","last_fullsized":"2024-06-28T16:14:34.721439+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:05:07.672939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5506084,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":497,"num_read_kb":57892,"num_write":1218,"num_write_kb":47404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080513+0000","last_change":"2024-06-28T15:00:19.391494+0000","last_active":"2024-06-28T16:10:20.080513+0000","last_peered":"2024-06-28T16:10:20.080513+0000","last_clean":"2024-06-28T16:10:20.080513+0000","last_became_active":"2024-06-28T15:00:19.390613+0000","last_became_peered":"2024-06-28T15:00:19.390613+0000","last_unstale":"2024-06-28T16:10:20.080513+0000","last_undegraded":"2024-06-28T16:10:20.080513+0000","last_fullsized":"2024-06-28T16:10:20.080513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:27:57.018874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"260'324","reported_seq":932,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.759325+0000","last_change":"2024-06-28T15:00:21.392375+0000","last_active":"2024-06-28T16:14:34.759325+0000","last_peered":"2024-06-28T16:14:34.759325+0000","last_clean":"2024-06-28T16:14:34.759325+0000","last_became_active":"2024-06-28T15:00:21.391631+0000","last_became_peered":"2024-06-28T15:00:21.391631+0000","last_unstale":"2024-06-28T16:14:34.759325+0000","last_undegraded":"2024-06-28T16:14:34.759325+0000","last_fullsized":"2024-06-28T16:14:34.759325+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:25:50.181425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43414,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":516,"num_read_kb":1105,"num_write":400,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.549103+0000","last_change":"2024-06-28T15:00:23.405246+0000","last_active":"2024-06-28T16:10:30.549103+0000","last_peered":"2024-06-28T16:10:30.549103+0000","last_clean":"2024-06-28T16:10:30.549103+0000","last_became_active":"2024-06-28T15:00:23.404312+0000","last_became_peered":"2024-06-28T15:00:23.404312+0000","last_unstale":"2024-06-28T16:10:30.549103+0000","last_undegraded":"2024-06-28T16:10:30.549103+0000","last_fullsized":"2024-06-28T16:10:30.549103+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:40.584815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"260'131","reported_seq":642,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.235400+0000","last_change":"2024-06-28T15:00:25.425886+0000","last_active":"2024-06-28T16:14:08.235400+0000","last_peered":"2024-06-28T16:14:08.235400+0000","last_clean":"2024-06-28T16:14:08.235400+0000","last_became_active":"2024-06-28T15:00:25.424826+0000","last_became_peered":"2024-06-28T15:00:25.424826+0000","last_unstale":"2024-06-28T16:14:08.235400+0000","last_undegraded":"2024-06-28T16:14:08.235400+0000","last_fullsized":"2024-06-28T16:14:08.235400+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:56:01.501372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":159,"num_read_kb":144,"num_write":309,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"260'118","reported_seq":691,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073868+0000","last_change":"2024-06-28T15:00:17.387490+0000","last_active":"2024-06-28T16:10:20.073868+0000","last_peered":"2024-06-28T16:10:20.073868+0000","last_clean":"2024-06-28T16:10:20.073868+0000","last_became_active":"2024-06-28T15:00:17.386756+0000","last_became_peered":"2024-06-28T15:00:17.386756+0000","last_unstale":"2024-06-28T16:10:20.073868+0000","last_undegraded":"2024-06-28T16:10:20.073868+0000","last_fullsized":"2024-06-28T16:10:20.073868+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:24:23.283000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":469,"num_read_kb":378,"num_write":166,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"260'293","reported_seq":964,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.754554+0000","last_change":"2024-06-28T15:00:59.945598+0000","last_active":"2024-06-28T16:14:34.754554+0000","last_peered":"2024-06-28T16:14:34.754554+0000","last_clean":"2024-06-28T16:14:34.754554+0000","last_became_active":"2024-06-28T15:00:59.943670+0000","last_became_peered":"2024-06-28T15:00:59.943670+0000","last_unstale":"2024-06-28T16:14:34.754554+0000","last_undegraded":"2024-06-28T16:14:34.754554+0000","last_fullsized":"2024-06-28T16:14:34.754554+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:28:33.955349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306233,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":513,"num_read_kb":66858,"num_write":1091,"num_write_kb":26064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.066682+0000","last_change":"2024-06-28T15:00:19.400037+0000","last_active":"2024-06-28T16:10:20.066682+0000","last_peered":"2024-06-28T16:10:20.066682+0000","last_clean":"2024-06-28T16:10:20.066682+0000","last_became_active":"2024-06-28T15:00:19.398245+0000","last_became_peered":"2024-06-28T15:00:19.398245+0000","last_unstale":"2024-06-28T16:10:20.066682+0000","last_undegraded":"2024-06-28T16:10:20.066682+0000","last_fullsized":"2024-06-28T16:10:20.066682+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:50.601079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"260'292","reported_seq":877,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.720679+0000","last_change":"2024-06-28T15:00:21.406496+0000","last_active":"2024-06-28T16:14:06.720679+0000","last_peered":"2024-06-28T16:14:06.720679+0000","last_clean":"2024-06-28T16:14:06.720679+0000","last_became_active":"2024-06-28T15:00:21.404701+0000","last_became_peered":"2024-06-28T15:00:21.404701+0000","last_unstale":"2024-06-28T16:14:06.720679+0000","last_undegraded":"2024-06-28T16:14:06.720679+0000","last_fullsized":"2024-06-28T16:14:06.720679+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":200,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:35:29.922369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59181,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":473,"num_read_kb":1000,"num_write":388,"num_write_kb":303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":491,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:50.079705+0000","last_change":"2024-06-28T15:00:23.416260+0000","last_active":"2024-06-28T16:10:50.079705+0000","last_peered":"2024-06-28T16:10:50.079705+0000","last_clean":"2024-06-28T16:10:50.079705+0000","last_became_active":"2024-06-28T15:00:23.415531+0000","last_became_peered":"2024-06-28T15:00:23.415531+0000","last_unstale":"2024-06-28T16:10:50.079705+0000","last_undegraded":"2024-06-28T16:10:50.079705+0000","last_fullsized":"2024-06-28T16:10:50.079705+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:14:35.301127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"260'166","reported_seq":710,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:02.116164+0000","last_change":"2024-06-28T15:00:25.554319+0000","last_active":"2024-06-28T16:13:02.116164+0000","last_peered":"2024-06-28T16:13:02.116164+0000","last_clean":"2024-06-28T16:13:02.116164+0000","last_became_active":"2024-06-28T15:00:25.553724+0000","last_became_peered":"2024-06-28T15:00:25.553724+0000","last_unstale":"2024-06-28T16:13:02.116164+0000","last_undegraded":"2024-06-28T16:13:02.116164+0000","last_fullsized":"2024-06-28T16:13:02.116164+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:16:48.260247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":555,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":217,"num_write":397,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"260'143","reported_seq":806,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.494648+0000","last_change":"2024-06-28T15:00:17.383713+0000","last_active":"2024-06-28T16:10:30.494648+0000","last_peered":"2024-06-28T16:10:30.494648+0000","last_clean":"2024-06-28T16:10:30.494648+0000","last_became_active":"2024-06-28T15:00:17.383224+0000","last_became_peered":"2024-06-28T15:00:17.383224+0000","last_unstale":"2024-06-28T16:10:30.494648+0000","last_undegraded":"2024-06-28T16:10:30.494648+0000","last_fullsized":"2024-06-28T16:10:30.494648+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:23:29.150410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":649,"num_read_kb":545,"num_write":214,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"260'301","reported_seq":1066,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.691517+0000","last_change":"2024-06-28T15:00:15.424150+0000","last_active":"2024-06-28T16:14:34.691517+0000","last_peered":"2024-06-28T16:14:34.691517+0000","last_clean":"2024-06-28T16:14:34.691517+0000","last_became_active":"2024-06-28T15:00:15.423813+0000","last_became_peered":"2024-06-28T15:00:15.423813+0000","last_unstale":"2024-06-28T16:14:34.691517+0000","last_undegraded":"2024-06-28T16:14:34.691517+0000","last_fullsized":"2024-06-28T16:14:34.691517+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:57:01.721732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5724260,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":754,"num_read_kb":37474,"num_write":1107,"num_write_kb":31583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.075092+0000","last_change":"2024-06-28T15:00:19.384145+0000","last_active":"2024-06-28T16:10:25.075092+0000","last_peered":"2024-06-28T16:10:25.075092+0000","last_clean":"2024-06-28T16:10:25.075092+0000","last_became_active":"2024-06-28T15:00:19.383307+0000","last_became_peered":"2024-06-28T15:00:19.383307+0000","last_unstale":"2024-06-28T16:10:25.075092+0000","last_undegraded":"2024-06-28T16:10:25.075092+0000","last_fullsized":"2024-06-28T16:10:25.075092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:22:34.590726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"260'286","reported_seq":857,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.176534+0000","last_change":"2024-06-28T15:00:21.396994+0000","last_active":"2024-06-28T16:14:34.176534+0000","last_peered":"2024-06-28T16:14:34.176534+0000","last_clean":"2024-06-28T16:14:34.176534+0000","last_became_active":"2024-06-28T15:00:21.396257+0000","last_became_peered":"2024-06-28T15:00:21.396257+0000","last_unstale":"2024-06-28T16:14:34.176534+0000","last_undegraded":"2024-06-28T16:14:34.176534+0000","last_fullsized":"2024-06-28T16:14:34.176534+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:05:02.532173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43992,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":439,"num_read_kb":1078,"num_write":366,"num_write_kb":297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":494,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.075301+0000","last_change":"2024-06-28T15:00:23.410088+0000","last_active":"2024-06-28T16:10:25.075301+0000","last_peered":"2024-06-28T16:10:25.075301+0000","last_clean":"2024-06-28T16:10:25.075301+0000","last_became_active":"2024-06-28T15:00:23.409107+0000","last_became_peered":"2024-06-28T15:00:23.409107+0000","last_unstale":"2024-06-28T16:10:25.075301+0000","last_undegraded":"2024-06-28T16:10:25.075301+0000","last_fullsized":"2024-06-28T16:10:25.075301+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:26:16.658429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"260'144","reported_seq":876,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:02.732078+0000","last_change":"2024-06-28T15:00:25.425176+0000","last_active":"2024-06-28T16:14:02.732078+0000","last_peered":"2024-06-28T16:14:02.732078+0000","last_clean":"2024-06-28T16:14:02.732078+0000","last_became_active":"2024-06-28T15:00:25.424376+0000","last_became_peered":"2024-06-28T15:00:25.424376+0000","last_unstale":"2024-06-28T16:14:02.732078+0000","last_undegraded":"2024-06-28T16:14:02.732078+0000","last_fullsized":"2024-06-28T16:14:02.732078+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:40:22.841457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":398,"num_read_kb":429,"num_write":339,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"260'112","reported_seq":699,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.075140+0000","last_change":"2024-06-28T15:00:17.403746+0000","last_active":"2024-06-28T16:10:25.075140+0000","last_peered":"2024-06-28T16:10:25.075140+0000","last_clean":"2024-06-28T16:10:25.075140+0000","last_became_active":"2024-06-28T15:00:17.403007+0000","last_became_peered":"2024-06-28T15:00:17.403007+0000","last_unstale":"2024-06-28T16:10:25.075140+0000","last_undegraded":"2024-06-28T16:10:25.075140+0000","last_fullsized":"2024-06-28T16:10:25.075140+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:10:54.255627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":478,"num_read_kb":386,"num_write":182,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"260'311","reported_seq":978,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.556341+0000","last_change":"2024-06-28T15:00:15.415976+0000","last_active":"2024-06-28T16:11:05.556341+0000","last_peered":"2024-06-28T16:11:05.556341+0000","last_clean":"2024-06-28T16:11:05.556341+0000","last_became_active":"2024-06-28T15:00:15.414969+0000","last_became_peered":"2024-06-28T15:00:15.414969+0000","last_unstale":"2024-06-28T16:11:05.556341+0000","last_undegraded":"2024-06-28T16:11:05.556341+0000","last_fullsized":"2024-06-28T16:11:05.556341+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:03:09.425476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6850286,"num_objects":77,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":471,"num_read_kb":3182,"num_write":1253,"num_write_kb":13350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"24'1","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:45.088935+0000","last_change":"2024-06-28T15:00:19.403928+0000","last_active":"2024-06-28T16:10:45.088935+0000","last_peered":"2024-06-28T16:10:45.088935+0000","last_clean":"2024-06-28T16:10:45.088935+0000","last_became_active":"2024-06-28T15:00:19.403344+0000","last_became_peered":"2024-06-28T15:00:19.403344+0000","last_unstale":"2024-06-28T16:10:45.088935+0000","last_undegraded":"2024-06-28T16:10:45.088935+0000","last_fullsized":"2024-06-28T16:10:45.088935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:09:50.417696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"260'293","reported_seq":893,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.393958+0000","last_change":"2024-06-28T15:00:21.388828+0000","last_active":"2024-06-28T16:14:34.393958+0000","last_peered":"2024-06-28T16:14:34.393958+0000","last_clean":"2024-06-28T16:14:34.393958+0000","last_became_active":"2024-06-28T15:00:21.387539+0000","last_became_peered":"2024-06-28T15:00:21.387539+0000","last_unstale":"2024-06-28T16:14:34.393958+0000","last_undegraded":"2024-06-28T16:14:34.393958+0000","last_fullsized":"2024-06-28T16:14:34.393958+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:54:05.779834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44825,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":494,"num_read_kb":1086,"num_write":388,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":499,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.548062+0000","last_change":"2024-06-28T15:00:23.417454+0000","last_active":"2024-06-28T16:10:25.548062+0000","last_peered":"2024-06-28T16:10:25.548062+0000","last_clean":"2024-06-28T16:10:25.548062+0000","last_became_active":"2024-06-28T15:00:23.416363+0000","last_became_peered":"2024-06-28T15:00:23.416363+0000","last_unstale":"2024-06-28T16:10:25.548062+0000","last_undegraded":"2024-06-28T16:10:25.548062+0000","last_fullsized":"2024-06-28T16:10:25.548062+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:42:57.225974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"260'126","reported_seq":623,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:05.111348+0000","last_change":"2024-06-28T15:00:25.429682+0000","last_active":"2024-06-28T16:13:05.111348+0000","last_peered":"2024-06-28T16:13:05.111348+0000","last_clean":"2024-06-28T16:13:05.111348+0000","last_became_active":"2024-06-28T15:00:25.428959+0000","last_became_peered":"2024-06-28T15:00:25.428959+0000","last_unstale":"2024-06-28T16:13:05.111348+0000","last_undegraded":"2024-06-28T16:13:05.111348+0000","last_fullsized":"2024-06-28T16:13:05.111348+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:34:41.858483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":814,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":97,"num_write":299,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"260'174","reported_seq":761,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:39.929106+0000","last_change":"2024-06-28T15:00:17.407419+0000","last_active":"2024-06-28T16:13:39.929106+0000","last_peered":"2024-06-28T16:13:39.929106+0000","last_clean":"2024-06-28T16:13:39.929106+0000","last_became_active":"2024-06-28T15:00:17.406872+0000","last_became_peered":"2024-06-28T15:00:17.406872+0000","last_unstale":"2024-06-28T16:13:39.929106+0000","last_undegraded":"2024-06-28T16:13:39.929106+0000","last_fullsized":"2024-06-28T16:13:39.929106+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:52:17.662914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":660,"num_read_kb":511,"num_write":250,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"260'285","reported_seq":909,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.644073+0000","last_change":"2024-06-28T15:00:15.409170+0000","last_active":"2024-06-28T16:14:34.644073+0000","last_peered":"2024-06-28T16:14:34.644073+0000","last_clean":"2024-06-28T16:14:34.644073+0000","last_became_active":"2024-06-28T15:00:15.408337+0000","last_became_peered":"2024-06-28T15:00:15.408337+0000","last_unstale":"2024-06-28T16:14:34.644073+0000","last_undegraded":"2024-06-28T16:14:34.644073+0000","last_fullsized":"2024-06-28T16:14:34.644073+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:51:23.373884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":404,"num_read_kb":33775,"num_write":1000,"num_write_kb":10945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"260'72","reported_seq":674,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.102569+0000","last_change":"2024-06-28T15:00:17.404713+0000","last_active":"2024-06-28T16:11:50.102569+0000","last_peered":"2024-06-28T16:11:50.102569+0000","last_clean":"2024-06-28T16:11:50.102569+0000","last_became_active":"2024-06-28T15:00:17.403938+0000","last_became_peered":"2024-06-28T15:00:17.403938+0000","last_unstale":"2024-06-28T16:11:50.102569+0000","last_undegraded":"2024-06-28T16:11:50.102569+0000","last_fullsized":"2024-06-28T16:11:50.102569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:54:35.966683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":319,"num_read_kb":277,"num_write":96,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"260'322","reported_seq":995,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.576974+0000","last_change":"2024-06-28T15:00:15.420697+0000","last_active":"2024-06-28T16:14:34.576974+0000","last_peered":"2024-06-28T16:14:34.576974+0000","last_clean":"2024-06-28T16:14:34.576974+0000","last_became_active":"2024-06-28T15:00:15.420261+0000","last_became_peered":"2024-06-28T15:00:15.420261+0000","last_unstale":"2024-06-28T16:14:34.576974+0000","last_undegraded":"2024-06-28T16:14:34.576974+0000","last_fullsized":"2024-06-28T16:14:34.576974+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:57:26.418017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262189,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":529,"num_read_kb":23662,"num_write":1259,"num_write_kb":28174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"260'38","reported_seq":596,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.087084+0000","last_change":"2024-06-28T15:00:17.394654+0000","last_active":"2024-06-28T16:11:50.087084+0000","last_peered":"2024-06-28T16:11:50.087084+0000","last_clean":"2024-06-28T16:11:50.087084+0000","last_became_active":"2024-06-28T15:00:17.393889+0000","last_became_peered":"2024-06-28T15:00:17.393889+0000","last_unstale":"2024-06-28T16:11:50.087084+0000","last_undegraded":"2024-06-28T16:11:50.087084+0000","last_fullsized":"2024-06-28T16:11:50.087084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:22:39.330270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":152,"num_write":58,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"260'358","reported_seq":999,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.728265+0000","last_change":"2024-06-28T15:00:15.426603+0000","last_active":"2024-06-28T16:14:34.728265+0000","last_peered":"2024-06-28T16:14:34.728265+0000","last_clean":"2024-06-28T16:14:34.728265+0000","last_became_active":"2024-06-28T15:00:15.418773+0000","last_became_peered":"2024-06-28T15:00:15.418773+0000","last_unstale":"2024-06-28T16:14:34.728265+0000","last_undegraded":"2024-06-28T16:14:34.728265+0000","last_fullsized":"2024-06-28T16:14:34.728265+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:02:14.247776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8714448,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":553,"num_read_kb":16492,"num_write":1257,"num_write_kb":37641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"260'142","reported_seq":787,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:07.183113+0000","last_change":"2024-06-28T15:00:17.398608+0000","last_active":"2024-06-28T16:13:07.183113+0000","last_peered":"2024-06-28T16:13:07.183113+0000","last_clean":"2024-06-28T16:13:07.183113+0000","last_became_active":"2024-06-28T15:00:17.398357+0000","last_became_peered":"2024-06-28T15:00:17.398357+0000","last_unstale":"2024-06-28T16:13:07.183113+0000","last_undegraded":"2024-06-28T16:13:07.183113+0000","last_fullsized":"2024-06-28T16:13:07.183113+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:53:58.893939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":623,"num_read_kb":529,"num_write":221,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"260'269","reported_seq":913,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:09.848378+0000","last_change":"2024-06-28T15:00:15.391599+0000","last_active":"2024-06-28T16:12:09.848378+0000","last_peered":"2024-06-28T16:12:09.848378+0000","last_clean":"2024-06-28T16:12:09.848378+0000","last_became_active":"2024-06-28T15:00:15.391150+0000","last_became_peered":"2024-06-28T15:00:15.391150+0000","last_unstale":"2024-06-28T16:12:09.848378+0000","last_undegraded":"2024-06-28T16:12:09.848378+0000","last_fullsized":"2024-06-28T16:12:09.848378+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:12:30.284233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6905440,"num_objects":72,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":390,"num_read_kb":84462,"num_write":1202,"num_write_kb":41774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"260'191","reported_seq":788,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:05.276690+0000","last_change":"2024-06-28T15:00:59.807172+0000","last_active":"2024-06-28T16:12:05.276690+0000","last_peered":"2024-06-28T16:12:05.276690+0000","last_clean":"2024-06-28T16:12:05.276690+0000","last_became_active":"2024-06-28T15:00:59.806725+0000","last_became_peered":"2024-06-28T15:00:59.806725+0000","last_unstale":"2024-06-28T16:12:05.276690+0000","last_undegraded":"2024-06-28T16:12:05.276690+0000","last_fullsized":"2024-06-28T16:12:05.276690+0000","mapping_epoch":32,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:00:01.418110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":759,"num_read_kb":602,"num_write":307,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"260'334","reported_seq":998,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.618192+0000","last_change":"2024-06-28T15:00:15.383729+0000","last_active":"2024-06-28T16:14:34.618192+0000","last_peered":"2024-06-28T16:14:34.618192+0000","last_clean":"2024-06-28T16:14:34.618192+0000","last_became_active":"2024-06-28T15:00:15.383350+0000","last_became_peered":"2024-06-28T15:00:15.383350+0000","last_unstale":"2024-06-28T16:14:34.618192+0000","last_undegraded":"2024-06-28T16:14:34.618192+0000","last_fullsized":"2024-06-28T16:14:34.618192+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:22:33.747727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263219,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":502,"num_read_kb":41336,"num_write":1201,"num_write_kb":38478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0,7],"acting":[3,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"260'397","reported_seq":997,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.085126+0000","last_change":"2024-06-28T15:00:17.408600+0000","last_active":"2024-06-28T16:11:10.085126+0000","last_peered":"2024-06-28T16:11:10.085126+0000","last_clean":"2024-06-28T16:11:10.085126+0000","last_became_active":"2024-06-28T15:00:17.404844+0000","last_became_peered":"2024-06-28T15:00:17.404844+0000","last_unstale":"2024-06-28T16:11:10.085126+0000","last_undegraded":"2024-06-28T16:11:10.085126+0000","last_fullsized":"2024-06-28T16:11:10.085126+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":300,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:17:34.507155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1527,"num_read_kb":1156,"num_write":658,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"260'343","reported_seq":1027,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.657138+0000","last_change":"2024-06-28T15:00:15.401905+0000","last_active":"2024-06-28T16:14:34.657138+0000","last_peered":"2024-06-28T16:14:34.657138+0000","last_clean":"2024-06-28T16:14:34.657138+0000","last_became_active":"2024-06-28T15:00:15.401275+0000","last_became_peered":"2024-06-28T15:00:15.401275+0000","last_unstale":"2024-06-28T16:14:34.657138+0000","last_undegraded":"2024-06-28T16:14:34.657138+0000","last_fullsized":"2024-06-28T16:14:34.657138+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:00:05.589080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":559,"num_read_kb":45669,"num_write":1313,"num_write_kb":19663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"260'221","reported_seq":841,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.504084+0000","last_change":"2024-06-28T15:00:17.383279+0000","last_active":"2024-06-28T16:11:10.504084+0000","last_peered":"2024-06-28T16:11:10.504084+0000","last_clean":"2024-06-28T16:11:10.504084+0000","last_became_active":"2024-06-28T15:00:17.382739+0000","last_became_peered":"2024-06-28T15:00:17.382739+0000","last_unstale":"2024-06-28T16:11:10.504084+0000","last_undegraded":"2024-06-28T16:11:10.504084+0000","last_fullsized":"2024-06-28T16:11:10.504084+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:36:13.087157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":866,"num_read_kb":690,"num_write":335,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"260'298","reported_seq":928,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.600936+0000","last_change":"2024-06-28T15:00:15.421880+0000","last_active":"2024-06-28T16:14:34.600936+0000","last_peered":"2024-06-28T16:14:34.600936+0000","last_clean":"2024-06-28T16:14:34.600936+0000","last_became_active":"2024-06-28T15:00:15.421189+0000","last_became_peered":"2024-06-28T15:00:15.421189+0000","last_unstale":"2024-06-28T16:14:34.600936+0000","last_undegraded":"2024-06-28T16:14:34.600936+0000","last_fullsized":"2024-06-28T16:14:34.600936+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:03:21.461240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7105903,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":419,"num_read_kb":19450,"num_write":1148,"num_write_kb":38199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"260'79","reported_seq":639,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.083923+0000","last_change":"2024-06-28T15:00:17.396892+0000","last_active":"2024-06-28T16:11:10.083923+0000","last_peered":"2024-06-28T16:11:10.083923+0000","last_clean":"2024-06-28T16:11:10.083923+0000","last_became_active":"2024-06-28T15:00:17.396125+0000","last_became_peered":"2024-06-28T15:00:17.396125+0000","last_unstale":"2024-06-28T16:11:10.083923+0000","last_undegraded":"2024-06-28T16:11:10.083923+0000","last_fullsized":"2024-06-28T16:11:10.083923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:46:54.077428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":321,"num_read_kb":257,"num_write":118,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"260'270","reported_seq":927,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.735427+0000","last_change":"2024-06-28T15:00:15.408891+0000","last_active":"2024-06-28T16:14:34.735427+0000","last_peered":"2024-06-28T16:14:34.735427+0000","last_clean":"2024-06-28T16:14:34.735427+0000","last_became_active":"2024-06-28T15:00:15.408375+0000","last_became_peered":"2024-06-28T15:00:15.408375+0000","last_unstale":"2024-06-28T16:14:34.735427+0000","last_undegraded":"2024-06-28T16:14:34.735427+0000","last_fullsized":"2024-06-28T16:14:34.735427+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:29:43.369904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":484,"num_read_kb":6587,"num_write":1005,"num_write_kb":17135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"260'62","reported_seq":636,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:08.869032+0000","last_change":"2024-06-28T15:00:17.380077+0000","last_active":"2024-06-28T16:12:08.869032+0000","last_peered":"2024-06-28T16:12:08.869032+0000","last_clean":"2024-06-28T16:12:08.869032+0000","last_became_active":"2024-06-28T15:00:17.379374+0000","last_became_peered":"2024-06-28T15:00:17.379374+0000","last_unstale":"2024-06-28T16:12:08.869032+0000","last_undegraded":"2024-06-28T16:12:08.869032+0000","last_fullsized":"2024-06-28T16:12:08.869032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:19:27.611751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":216,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"260'367","reported_seq":1011,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.749490+0000","last_change":"2024-06-28T15:00:15.427903+0000","last_active":"2024-06-28T16:14:34.749490+0000","last_peered":"2024-06-28T16:14:34.749490+0000","last_clean":"2024-06-28T16:14:34.749490+0000","last_became_active":"2024-06-28T15:00:15.427171+0000","last_became_peered":"2024-06-28T15:00:15.427171+0000","last_unstale":"2024-06-28T16:14:34.749490+0000","last_undegraded":"2024-06-28T16:14:34.749490+0000","last_fullsized":"2024-06-28T16:14:34.749490+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":300,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:22:01.482231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":723921,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":536,"num_read_kb":26554,"num_write":1152,"num_write_kb":52254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"260'230","reported_seq":851,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.074074+0000","last_change":"2024-06-28T15:00:17.409608+0000","last_active":"2024-06-28T16:10:20.074074+0000","last_peered":"2024-06-28T16:10:20.074074+0000","last_clean":"2024-06-28T16:10:20.074074+0000","last_became_active":"2024-06-28T15:00:17.403760+0000","last_became_peered":"2024-06-28T15:00:17.403760+0000","last_unstale":"2024-06-28T16:10:20.074074+0000","last_undegraded":"2024-06-28T16:10:20.074074+0000","last_fullsized":"2024-06-28T16:10:20.074074+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:58:04.722934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":902,"num_read_kb":724,"num_write":338,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"260'282","reported_seq":929,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.669616+0000","last_change":"2024-06-28T15:00:15.376797+0000","last_active":"2024-06-28T16:14:34.669616+0000","last_peered":"2024-06-28T16:14:34.669616+0000","last_clean":"2024-06-28T16:14:34.669616+0000","last_became_active":"2024-06-28T15:00:15.376014+0000","last_became_peered":"2024-06-28T15:00:15.376014+0000","last_unstale":"2024-06-28T16:14:34.669616+0000","last_undegraded":"2024-06-28T16:14:34.669616+0000","last_fullsized":"2024-06-28T16:14:34.669616+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:31:24.117797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462789,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":409,"num_read_kb":32504,"num_write":910,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"260'70","reported_seq":658,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.255871+0000","last_change":"2024-06-28T15:00:17.402274+0000","last_active":"2024-06-28T16:10:20.255871+0000","last_peered":"2024-06-28T16:10:20.255871+0000","last_clean":"2024-06-28T16:10:20.255871+0000","last_became_active":"2024-06-28T15:00:17.401917+0000","last_became_peered":"2024-06-28T15:00:17.401917+0000","last_unstale":"2024-06-28T16:10:20.255871+0000","last_undegraded":"2024-06-28T16:10:20.255871+0000","last_fullsized":"2024-06-28T16:10:20.255871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:48:15.974353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":295,"num_read_kb":247,"num_write":98,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"260'337","reported_seq":973,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.694565+0000","last_change":"2024-06-28T15:00:15.434365+0000","last_active":"2024-06-28T16:14:34.694565+0000","last_peered":"2024-06-28T16:14:34.694565+0000","last_clean":"2024-06-28T16:14:34.694565+0000","last_became_active":"2024-06-28T15:00:15.431444+0000","last_became_peered":"2024-06-28T15:00:15.431444+0000","last_unstale":"2024-06-28T16:14:34.694565+0000","last_undegraded":"2024-06-28T16:14:34.694565+0000","last_fullsized":"2024-06-28T16:14:34.694565+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:01:51.800790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":461,"num_read_kb":557,"num_write":1266,"num_write_kb":33580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,3],"acting":[5,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"260'424","reported_seq":1178,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101339+0000","last_change":"2024-06-28T15:00:17.403422+0000","last_active":"2024-06-28T16:12:20.101339+0000","last_peered":"2024-06-28T16:12:20.101339+0000","last_clean":"2024-06-28T16:12:20.101339+0000","last_became_active":"2024-06-28T15:00:17.402842+0000","last_became_peered":"2024-06-28T15:00:17.402842+0000","last_unstale":"2024-06-28T16:12:20.101339+0000","last_undegraded":"2024-06-28T16:12:20.101339+0000","last_fullsized":"2024-06-28T16:12:20.101339+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":400,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:26:05.866773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1825,"num_read_kb":1465,"num_write":755,"num_write_kb":468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"260'261","reported_seq":906,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.574001+0000","last_change":"2024-06-28T15:00:15.415547+0000","last_active":"2024-06-28T16:12:15.574001+0000","last_peered":"2024-06-28T16:12:15.574001+0000","last_clean":"2024-06-28T16:12:15.574001+0000","last_became_active":"2024-06-28T15:00:15.415009+0000","last_became_peered":"2024-06-28T15:00:15.415009+0000","last_unstale":"2024-06-28T16:12:15.574001+0000","last_undegraded":"2024-06-28T16:12:15.574001+0000","last_fullsized":"2024-06-28T16:12:15.574001+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:36:33.533042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5245699,"num_objects":64,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":421,"num_read_kb":2084,"num_write":1144,"num_write_kb":29565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"260'26","reported_seq":574,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.108218+0000","last_change":"2024-06-28T15:00:17.390583+0000","last_active":"2024-06-28T16:12:20.108218+0000","last_peered":"2024-06-28T16:12:20.108218+0000","last_clean":"2024-06-28T16:12:20.108218+0000","last_became_active":"2024-06-28T15:00:17.389844+0000","last_became_peered":"2024-06-28T15:00:17.389844+0000","last_unstale":"2024-06-28T16:12:20.108218+0000","last_undegraded":"2024-06-28T16:12:20.108218+0000","last_fullsized":"2024-06-28T16:12:20.108218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:41:17.401424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":112,"num_write":34,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"260'305","reported_seq":936,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.719398+0000","last_change":"2024-06-28T15:00:15.398948+0000","last_active":"2024-06-28T16:14:34.719398+0000","last_peered":"2024-06-28T16:14:34.719398+0000","last_clean":"2024-06-28T16:14:34.719398+0000","last_became_active":"2024-06-28T15:00:15.397986+0000","last_became_peered":"2024-06-28T15:00:15.397986+0000","last_unstale":"2024-06-28T16:14:34.719398+0000","last_undegraded":"2024-06-28T16:14:34.719398+0000","last_fullsized":"2024-06-28T16:14:34.719398+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:02:19.841719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079926,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":433,"num_read_kb":10373,"num_write":1047,"num_write_kb":25421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"260'192","reported_seq":856,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.822857+0000","last_change":"2024-06-28T15:00:17.395455+0000","last_active":"2024-06-28T16:12:20.822857+0000","last_peered":"2024-06-28T16:12:20.822857+0000","last_clean":"2024-06-28T16:12:20.822857+0000","last_became_active":"2024-06-28T15:00:17.394783+0000","last_became_peered":"2024-06-28T15:00:17.394783+0000","last_unstale":"2024-06-28T16:12:20.822857+0000","last_undegraded":"2024-06-28T16:12:20.822857+0000","last_fullsized":"2024-06-28T16:12:20.822857+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:54:50.516422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":852,"num_read_kb":715,"num_write":335,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"260'302","reported_seq":913,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.681222+0000","last_change":"2024-06-28T15:00:59.939157+0000","last_active":"2024-06-28T16:14:34.681222+0000","last_peered":"2024-06-28T16:14:34.681222+0000","last_clean":"2024-06-28T16:14:34.681222+0000","last_became_active":"2024-06-28T15:00:59.938004+0000","last_became_peered":"2024-06-28T15:00:59.938004+0000","last_unstale":"2024-06-28T16:14:34.681222+0000","last_undegraded":"2024-06-28T16:14:34.681222+0000","last_fullsized":"2024-06-28T16:14:34.681222+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:35:45.871383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6542288,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":437,"num_read_kb":25077,"num_write":1246,"num_write_kb":38037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,3],"acting":[1,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"260'392","reported_seq":1062,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.520627+0000","last_change":"2024-06-28T15:00:17.374446+0000","last_active":"2024-06-28T16:12:20.520627+0000","last_peered":"2024-06-28T16:12:20.520627+0000","last_clean":"2024-06-28T16:12:20.520627+0000","last_became_active":"2024-06-28T15:00:17.374143+0000","last_became_peered":"2024-06-28T15:00:17.374143+0000","last_unstale":"2024-06-28T16:12:20.520627+0000","last_undegraded":"2024-06-28T16:12:20.520627+0000","last_fullsized":"2024-06-28T16:12:20.520627+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":300,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:31:56.644836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1520,"num_read_kb":1191,"num_write":653,"num_write_kb":382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"260'255","reported_seq":918,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.656935+0000","last_change":"2024-06-28T15:00:15.430833+0000","last_active":"2024-06-28T16:14:34.656935+0000","last_peered":"2024-06-28T16:14:34.656935+0000","last_clean":"2024-06-28T16:14:34.656935+0000","last_became_active":"2024-06-28T15:00:15.430228+0000","last_became_peered":"2024-06-28T15:00:15.430228+0000","last_unstale":"2024-06-28T16:14:34.656935+0000","last_undegraded":"2024-06-28T16:14:34.656935+0000","last_fullsized":"2024-06-28T16:14:34.656935+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:20:15.238678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170399,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":449,"num_read_kb":6008,"num_write":895,"num_write_kb":11243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"260'134","reported_seq":778,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:05.104760+0000","last_change":"2024-06-28T15:00:17.382276+0000","last_active":"2024-06-28T16:12:05.104760+0000","last_peered":"2024-06-28T16:12:05.104760+0000","last_clean":"2024-06-28T16:12:05.104760+0000","last_became_active":"2024-06-28T15:00:17.381562+0000","last_became_peered":"2024-06-28T15:00:17.381562+0000","last_unstale":"2024-06-28T16:12:05.104760+0000","last_undegraded":"2024-06-28T16:12:05.104760+0000","last_fullsized":"2024-06-28T16:12:05.104760+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:32:13.816417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":479,"num_write":190,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"260'496","reported_seq":1185,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.737064+0000","last_change":"2024-06-28T15:00:15.419485+0000","last_active":"2024-06-28T16:14:34.737064+0000","last_peered":"2024-06-28T16:14:34.737064+0000","last_clean":"2024-06-28T16:14:34.737064+0000","last_became_active":"2024-06-28T15:00:15.418850+0000","last_became_peered":"2024-06-28T15:00:15.418850+0000","last_unstale":"2024-06-28T16:14:34.737064+0000","last_undegraded":"2024-06-28T16:14:34.737064+0000","last_fullsized":"2024-06-28T16:14:34.737064+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":400,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:53:16.536329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":907,"num_read_kb":23220,"num_write":1415,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"260'338","reported_seq":1022,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.749225+0000","last_change":"2024-06-28T15:00:15.409457+0000","last_active":"2024-06-28T16:14:34.749225+0000","last_peered":"2024-06-28T16:14:34.749225+0000","last_clean":"2024-06-28T16:14:34.749225+0000","last_became_active":"2024-06-28T15:00:15.408158+0000","last_became_peered":"2024-06-28T15:00:15.408158+0000","last_unstale":"2024-06-28T16:14:34.749225+0000","last_undegraded":"2024-06-28T16:14:34.749225+0000","last_fullsized":"2024-06-28T16:14:34.749225+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:01:20.858547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6690738,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":568,"num_read_kb":38558,"num_write":1259,"num_write_kb":46809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"260'77","reported_seq":645,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:40.551311+0000","last_change":"2024-06-28T15:00:17.380351+0000","last_active":"2024-06-28T16:10:40.551311+0000","last_peered":"2024-06-28T16:10:40.551311+0000","last_clean":"2024-06-28T16:10:40.551311+0000","last_became_active":"2024-06-28T15:00:17.379771+0000","last_became_peered":"2024-06-28T15:00:17.379771+0000","last_unstale":"2024-06-28T16:10:40.551311+0000","last_undegraded":"2024-06-28T16:10:40.551311+0000","last_fullsized":"2024-06-28T16:10:40.551311+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:19:40.601549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":249,"num_write":109,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"260'387","reported_seq":1101,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.735487+0000","last_change":"2024-06-28T15:00:15.404986+0000","last_active":"2024-06-28T16:14:34.735487+0000","last_peered":"2024-06-28T16:14:34.735487+0000","last_clean":"2024-06-28T16:14:34.735487+0000","last_became_active":"2024-06-28T15:00:15.401324+0000","last_became_peered":"2024-06-28T15:00:15.401324+0000","last_unstale":"2024-06-28T16:14:34.735487+0000","last_undegraded":"2024-06-28T16:14:34.735487+0000","last_fullsized":"2024-06-28T16:14:34.735487+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":300,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:15:20.980772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079973,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":661,"num_read_kb":1946,"num_write":1429,"num_write_kb":27589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"260'176","reported_seq":777,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:14.583638+0000","last_change":"2024-06-28T15:00:17.383615+0000","last_active":"2024-06-28T16:13:14.583638+0000","last_peered":"2024-06-28T16:13:14.583638+0000","last_clean":"2024-06-28T16:13:14.583638+0000","last_became_active":"2024-06-28T15:00:17.382774+0000","last_became_peered":"2024-06-28T15:00:17.382774+0000","last_unstale":"2024-06-28T16:13:14.583638+0000","last_undegraded":"2024-06-28T16:13:14.583638+0000","last_fullsized":"2024-06-28T16:13:14.583638+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:39:45.866222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":719,"num_read_kb":570,"num_write":272,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5652,"num_read_kb":3987,"num_write":2073,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1078,"ondisk_log_size":1078,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19748,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":13201,"num_read_kb":13000,"num_write":15927,"num_write_kb":4660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":39496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1836,"ondisk_log_size":1836,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":321,"ondisk_log_size":321,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1481237,"num_objects":198,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":18084,"num_read_kb":37002,"num_write":15113,"num_write_kb":11203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":353845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1842,"ondisk_log_size":1842,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1644,"num_read_kb":1954,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87319,"num_read_kb":70822,"num_write":35298,"num_write_kb":17822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3526,"ondisk_log_size":3526,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":187758130,"num_objects":1055,"num_object_clones":0,"num_object_copies":4220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1055,"num_whiteouts":0,"num_read":34213,"num_read_kb":1786506,"num_write":77152,"num_write_kb":1982911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":273719296,"data_stored":273719296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3512,"ondisk_log_size":3512,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":19963089,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":127,"num_read_kb":165927,"num_write":152,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36741120,"data_stored":36740305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":17865937,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":135,"num_read_kb":186344,"num_write":152,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":50372608,"data_stored":50371793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575745,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":371852,"kb_used_data":51760,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":93999988,"statfs":{"total":96636764160,"available":96255987712,"internally_reserved":0,"allocated":53002240,"data_stored":51018038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":327612874},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575745,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":353092,"kb_used_data":60700,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":94018748,"statfs":{"total":96636764160,"available":96275197952,"internally_reserved":0,"allocated":62156800,"data_stored":60226794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":299170249},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575747,"num_pgs":114,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206904,"kb_used_data":40952,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94164936,"statfs":{"total":96636764160,"available":96424894464,"internally_reserved":0,"allocated":41934848,"data_stored":39931707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575747,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":146836,"kb_used_data":43468,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94225004,"statfs":{"total":96636764160,"available":96486404096,"internally_reserved":0,"allocated":44511232,"data_stored":42596897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575750,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303384,"kb_used_data":29196,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":94068456,"statfs":{"total":96636764160,"available":96326098944,"internally_reserved":0,"allocated":29896704,"data_stored":27928906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":280623559},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575748,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137428,"kb_used_data":29820,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94234412,"statfs":{"total":96636764160,"available":96496037888,"internally_reserved":0,"allocated":30535680,"data_stored":28610104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575745,"num_pgs":120,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201624,"kb_used_data":39968,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94170216,"statfs":{"total":96636764160,"available":96430301184,"internally_reserved":0,"allocated":40927232,"data_stored":38982339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575747,"num_pgs":116,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":385160,"kb_used_data":129088,"kb_used_omap":1,"kb_used_meta":256062,"kb_avail":93986680,"statfs":{"total":96636764160,"available":96242360320,"internally_reserved":0,"allocated":132186112,"data_stored":130163058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":262207941},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":96485376,"data_stored":96485376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25198592,"data_stored":25198592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21118976,"data_stored":21118976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19386368,"data_stored":19386368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29863936,"data_stored":29863936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20664320,"data_stored":20664320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28712960,"data_stored":28712960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32288768,"data_stored":32288768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":347321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":7701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":4574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":4436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":5746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16818176,"data_stored":16817361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13672448,"data_stored":13671633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-28T16:14:36.042 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-28T16:14:36.375 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:36.387 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-06-28T16:14:36.389 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":2487,"stamp":"2024-06-28T16:14:35.607276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":227680302,"num_objects":1411,"num_object_clones":0,"num_object_copies":4932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1411,"num_whiteouts":0,"num_read":160453,"num_read_kb":2265609,"num_write":146008,"num_write_kb":2275003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12397,"ondisk_log_size":12397,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2106280,"kb_used_data":424952,"kb_used_omap":12,"kb_used_meta":1680371,"kb_avail":752868440,"statfs":{"total":773094113280,"available":770937282560,"internally_reserved":0,"allocated":435150848,"data_stored":419457843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":1720700467},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-457593513,"num_objects":-3393,"num_object_clones":0,"num_object_copies":-13508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3393,"num_whiteouts":0,"num_read":3820,"num_read_kb":3315,"num_write":3571,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007566"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":26,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.063084+0000","last_change":"2024-06-28T15:37:33.329429+0000","last_active":"2024-06-28T16:10:05.063084+0000","last_peered":"2024-06-28T16:10:05.063084+0000","last_clean":"2024-06-28T16:10:05.063084+0000","last_became_active":"2024-06-28T15:37:33.328626+0000","last_became_peered":"2024-06-28T15:37:33.328626+0000","last_unstale":"2024-06-28T16:10:05.063084+0000","last_undegraded":"2024-06-28T16:10:05.063084+0000","last_fullsized":"2024-06-28T16:10:05.063084+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:48:45.377436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"257'3","reported_seq":49,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.078088+0000","last_change":"2024-06-28T15:21:11.079460+0000","last_active":"2024-06-28T16:11:10.078088+0000","last_peered":"2024-06-28T16:11:10.078088+0000","last_clean":"2024-06-28T16:11:10.078088+0000","last_became_active":"2024-06-28T15:21:11.078250+0000","last_became_peered":"2024-06-28T15:21:11.078250+0000","last_unstale":"2024-06-28T16:11:10.078088+0000","last_undegraded":"2024-06-28T16:11:10.078088+0000","last_fullsized":"2024-06-28T16:11:10.078088+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:32:28.636744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"260'4","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.900204+0000","last_change":"2024-06-28T15:21:07.047340+0000","last_active":"2024-06-28T16:14:32.900204+0000","last_peered":"2024-06-28T16:14:32.900204+0000","last_clean":"2024-06-28T16:14:32.900204+0000","last_became_active":"2024-06-28T15:21:07.046517+0000","last_became_peered":"2024-06-28T15:21:07.046517+0000","last_unstale":"2024-06-28T16:14:32.900204+0000","last_undegraded":"2024-06-28T16:14:32.900204+0000","last_fullsized":"2024-06-28T16:14:32.900204+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:20:57.841643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3076,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"249'7","reported_seq":82,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.563726+0000","last_change":"2024-06-28T15:17:24.031700+0000","last_active":"2024-06-28T16:11:35.563726+0000","last_peered":"2024-06-28T16:11:35.563726+0000","last_clean":"2024-06-28T16:11:35.563726+0000","last_became_active":"2024-06-28T15:17:24.028787+0000","last_became_peered":"2024-06-28T15:17:24.028787+0000","last_unstale":"2024-06-28T16:11:35.563726+0000","last_undegraded":"2024-06-28T16:11:35.563726+0000","last_fullsized":"2024-06-28T16:11:35.563726+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:38:55.771845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":30,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":26,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.252697+0000","last_change":"2024-06-28T15:37:33.337334+0000","last_active":"2024-06-28T16:10:05.252697+0000","last_peered":"2024-06-28T16:10:05.252697+0000","last_clean":"2024-06-28T16:10:05.252697+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:10:05.252697+0000","last_undegraded":"2024-06-28T16:10:05.252697+0000","last_fullsized":"2024-06-28T16:10:05.252697+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:34:53.771735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"260'2","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.119791+0000","last_change":"2024-06-28T15:21:11.105141+0000","last_active":"2024-06-28T16:13:45.119791+0000","last_peered":"2024-06-28T16:13:45.119791+0000","last_clean":"2024-06-28T16:13:45.119791+0000","last_became_active":"2024-06-28T15:21:11.104387+0000","last_became_peered":"2024-06-28T15:21:11.104387+0000","last_unstale":"2024-06-28T16:13:45.119791+0000","last_undegraded":"2024-06-28T16:13:45.119791+0000","last_fullsized":"2024-06-28T16:13:45.119791+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:29:31.608922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"260'2","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.126798+0000","last_change":"2024-06-28T15:21:07.049830+0000","last_active":"2024-06-28T16:13:40.126798+0000","last_peered":"2024-06-28T16:13:40.126798+0000","last_clean":"2024-06-28T16:13:40.126798+0000","last_became_active":"2024-06-28T15:21:07.049141+0000","last_became_peered":"2024-06-28T15:21:07.049141+0000","last_unstale":"2024-06-28T16:13:40.126798+0000","last_undegraded":"2024-06-28T16:13:40.126798+0000","last_fullsized":"2024-06-28T16:13:40.126798+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:18:35.987689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"260'18","reported_seq":110,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:40.271668+0000","last_change":"2024-06-28T15:17:23.994609+0000","last_active":"2024-06-28T16:11:40.271668+0000","last_peered":"2024-06-28T16:11:40.271668+0000","last_clean":"2024-06-28T16:11:40.271668+0000","last_became_active":"2024-06-28T15:17:23.993905+0000","last_became_peered":"2024-06-28T15:17:23.993905+0000","last_unstale":"2024-06-28T16:11:40.271668+0000","last_undegraded":"2024-06-28T16:11:40.271668+0000","last_fullsized":"2024-06-28T16:11:40.271668+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:33:37.469328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":64,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.087475+0000","last_change":"2024-06-28T15:37:33.353498+0000","last_active":"2024-06-28T16:11:25.087475+0000","last_peered":"2024-06-28T16:11:25.087475+0000","last_clean":"2024-06-28T16:11:25.087475+0000","last_became_active":"2024-06-28T15:37:33.345642+0000","last_became_peered":"2024-06-28T15:37:33.345642+0000","last_unstale":"2024-06-28T16:11:25.087475+0000","last_undegraded":"2024-06-28T16:11:25.087475+0000","last_fullsized":"2024-06-28T16:11:25.087475+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:38:21.634963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"257'5","reported_seq":57,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.558957+0000","last_change":"2024-06-28T15:21:11.098339+0000","last_active":"2024-06-28T16:11:15.558957+0000","last_peered":"2024-06-28T16:11:15.558957+0000","last_clean":"2024-06-28T16:11:15.558957+0000","last_became_active":"2024-06-28T15:21:11.097619+0000","last_became_peered":"2024-06-28T15:21:11.097619+0000","last_unstale":"2024-06-28T16:11:15.558957+0000","last_undegraded":"2024-06-28T16:11:15.558957+0000","last_fullsized":"2024-06-28T16:11:15.558957+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:23:45.881180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5438,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"0'0","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.268802+0000","last_change":"2024-06-28T15:21:07.020860+0000","last_active":"2024-06-28T16:11:25.268802+0000","last_peered":"2024-06-28T16:11:25.268802+0000","last_clean":"2024-06-28T16:11:25.268802+0000","last_became_active":"2024-06-28T15:21:07.020090+0000","last_became_peered":"2024-06-28T15:21:07.020090+0000","last_unstale":"2024-06-28T16:11:25.268802+0000","last_undegraded":"2024-06-28T16:11:25.268802+0000","last_fullsized":"2024-06-28T16:11:25.268802+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:42:14.199161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"260'43","reported_seq":174,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:09:55.540550+0000","last_change":"2024-06-28T15:17:24.014535+0000","last_active":"2024-06-28T16:09:55.540550+0000","last_peered":"2024-06-28T16:09:55.540550+0000","last_clean":"2024-06-28T16:09:55.540550+0000","last_became_active":"2024-06-28T15:17:24.012739+0000","last_became_peered":"2024-06-28T15:17:24.012739+0000","last_unstale":"2024-06-28T16:09:55.540550+0000","last_undegraded":"2024-06-28T16:09:55.540550+0000","last_fullsized":"2024-06-28T16:09:55.540550+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:08:38.008559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":221,"num_read_kb":154,"num_write":91,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.509029+0000","last_change":"2024-06-28T15:37:33.345746+0000","last_active":"2024-06-28T16:11:30.509029+0000","last_peered":"2024-06-28T16:11:30.509029+0000","last_clean":"2024-06-28T16:11:30.509029+0000","last_became_active":"2024-06-28T15:37:33.344850+0000","last_became_peered":"2024-06-28T15:37:33.344850+0000","last_unstale":"2024-06-28T16:11:30.509029+0000","last_undegraded":"2024-06-28T16:11:30.509029+0000","last_fullsized":"2024-06-28T16:11:30.509029+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:52:03.083724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"260'3","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.320060+0000","last_change":"2024-06-28T15:21:11.107158+0000","last_active":"2024-06-28T16:13:20.320060+0000","last_peered":"2024-06-28T16:13:20.320060+0000","last_clean":"2024-06-28T16:13:20.320060+0000","last_became_active":"2024-06-28T15:21:11.102572+0000","last_became_peered":"2024-06-28T15:21:11.102572+0000","last_unstale":"2024-06-28T16:13:20.320060+0000","last_undegraded":"2024-06-28T16:13:20.320060+0000","last_fullsized":"2024-06-28T16:13:20.320060+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:06:32.919789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.085295+0000","last_change":"2024-06-28T15:21:07.032950+0000","last_active":"2024-06-28T16:11:10.085295+0000","last_peered":"2024-06-28T16:11:10.085295+0000","last_clean":"2024-06-28T16:11:10.085295+0000","last_became_active":"2024-06-28T15:21:07.030576+0000","last_became_peered":"2024-06-28T15:21:07.030576+0000","last_unstale":"2024-06-28T16:11:10.085295+0000","last_undegraded":"2024-06-28T16:11:10.085295+0000","last_fullsized":"2024-06-28T16:11:10.085295+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:28:35.851727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'23","reported_seq":116,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:09:55.485817+0000","last_change":"2024-06-28T15:17:24.019871+0000","last_active":"2024-06-28T16:09:55.485817+0000","last_peered":"2024-06-28T16:09:55.485817+0000","last_clean":"2024-06-28T16:09:55.485817+0000","last_became_active":"2024-06-28T15:17:24.019120+0000","last_became_peered":"2024-06-28T15:17:24.019120+0000","last_unstale":"2024-06-28T16:09:55.485817+0000","last_undegraded":"2024-06-28T16:09:55.485817+0000","last_fullsized":"2024-06-28T16:09:55.485817+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:25:56.355193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":82,"num_write":51,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:40.511313+0000","last_change":"2024-06-28T15:37:33.339459+0000","last_active":"2024-06-28T16:11:40.511313+0000","last_peered":"2024-06-28T16:11:40.511313+0000","last_clean":"2024-06-28T16:11:40.511313+0000","last_became_active":"2024-06-28T15:37:33.335045+0000","last_became_peered":"2024-06-28T15:37:33.335045+0000","last_unstale":"2024-06-28T16:11:40.511313+0000","last_undegraded":"2024-06-28T16:11:40.511313+0000","last_fullsized":"2024-06-28T16:11:40.511313+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:39:33.932638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"0'0","reported_seq":44,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:10.091403+0000","last_change":"2024-06-28T15:21:11.094697+0000","last_active":"2024-06-28T16:12:10.091403+0000","last_peered":"2024-06-28T16:12:10.091403+0000","last_clean":"2024-06-28T16:12:10.091403+0000","last_became_active":"2024-06-28T15:21:11.094122+0000","last_became_peered":"2024-06-28T15:21:11.094122+0000","last_unstale":"2024-06-28T16:12:10.091403+0000","last_undegraded":"2024-06-28T16:12:10.091403+0000","last_fullsized":"2024-06-28T16:12:10.091403+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:48:32.491192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.569464+0000","last_change":"2024-06-28T15:21:07.027869+0000","last_active":"2024-06-28T16:11:55.569464+0000","last_peered":"2024-06-28T16:11:55.569464+0000","last_clean":"2024-06-28T16:11:55.569464+0000","last_became_active":"2024-06-28T15:21:07.025549+0000","last_became_peered":"2024-06-28T15:21:07.025549+0000","last_unstale":"2024-06-28T16:11:55.569464+0000","last_undegraded":"2024-06-28T16:11:55.569464+0000","last_fullsized":"2024-06-28T16:11:55.569464+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:37:01.202123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"260'24","reported_seq":120,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:09:55.485705+0000","last_change":"2024-06-28T15:17:24.011769+0000","last_active":"2024-06-28T16:09:55.485705+0000","last_peered":"2024-06-28T16:09:55.485705+0000","last_clean":"2024-06-28T16:09:55.485705+0000","last_became_active":"2024-06-28T15:17:24.010441+0000","last_became_peered":"2024-06-28T15:17:24.010441+0000","last_unstale":"2024-06-28T16:09:55.485705+0000","last_undegraded":"2024-06-28T16:09:55.485705+0000","last_fullsized":"2024-06-28T16:09:55.485705+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:28:32.900550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":85,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.091008+0000","last_change":"2024-06-28T15:37:33.341710+0000","last_active":"2024-06-28T16:10:55.091008+0000","last_peered":"2024-06-28T16:10:55.091008+0000","last_clean":"2024-06-28T16:10:55.091008+0000","last_became_active":"2024-06-28T15:37:33.336623+0000","last_became_peered":"2024-06-28T15:37:33.336623+0000","last_unstale":"2024-06-28T16:10:55.091008+0000","last_undegraded":"2024-06-28T16:10:55.091008+0000","last_fullsized":"2024-06-28T16:10:55.091008+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:49:52.074185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"260'11","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.517566+0000","last_change":"2024-06-28T15:21:11.100151+0000","last_active":"2024-06-28T16:14:32.517566+0000","last_peered":"2024-06-28T16:14:32.517566+0000","last_clean":"2024-06-28T16:14:32.517566+0000","last_became_active":"2024-06-28T15:21:11.099434+0000","last_became_peered":"2024-06-28T15:21:11.099434+0000","last_unstale":"2024-06-28T16:14:32.517566+0000","last_undegraded":"2024-06-28T16:14:32.517566+0000","last_fullsized":"2024-06-28T16:14:32.517566+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:21:06.014702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":20482,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"257'5","reported_seq":65,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:10.098868+0000","last_change":"2024-06-28T15:21:07.044767+0000","last_active":"2024-06-28T16:12:10.098868+0000","last_peered":"2024-06-28T16:12:10.098868+0000","last_clean":"2024-06-28T16:12:10.098868+0000","last_became_active":"2024-06-28T15:21:07.043987+0000","last_became_peered":"2024-06-28T15:21:07.043987+0000","last_unstale":"2024-06-28T16:12:10.098868+0000","last_undegraded":"2024-06-28T16:12:10.098868+0000","last_fullsized":"2024-06-28T16:12:10.098868+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:53:56.640453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"260'19","reported_seq":100,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:09:55.067186+0000","last_change":"2024-06-28T15:17:24.007176+0000","last_active":"2024-06-28T16:09:55.067186+0000","last_peered":"2024-06-28T16:09:55.067186+0000","last_clean":"2024-06-28T16:09:55.067186+0000","last_became_active":"2024-06-28T15:17:24.006314+0000","last_became_peered":"2024-06-28T15:17:24.006314+0000","last_unstale":"2024-06-28T16:09:55.067186+0000","last_undegraded":"2024-06-28T16:09:55.067186+0000","last_fullsized":"2024-06-28T16:09:55.067186+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:20:19.639267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":54,"num_write":49,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"260'1","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.076954+0000","last_change":"2024-06-28T15:37:33.348503+0000","last_active":"2024-06-28T16:10:05.076954+0000","last_peered":"2024-06-28T16:10:05.076954+0000","last_clean":"2024-06-28T16:10:05.076954+0000","last_became_active":"2024-06-28T15:37:33.347715+0000","last_became_peered":"2024-06-28T15:37:33.347715+0000","last_unstale":"2024-06-28T16:10:05.076954+0000","last_undegraded":"2024-06-28T16:10:05.076954+0000","last_fullsized":"2024-06-28T16:10:05.076954+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:01:24.044514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":42,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.085434+0000","last_change":"2024-06-28T15:21:11.089867+0000","last_active":"2024-06-28T16:11:15.085434+0000","last_peered":"2024-06-28T16:11:15.085434+0000","last_clean":"2024-06-28T16:11:15.085434+0000","last_became_active":"2024-06-28T15:21:11.088625+0000","last_became_peered":"2024-06-28T15:21:11.088625+0000","last_unstale":"2024-06-28T16:11:15.085434+0000","last_undegraded":"2024-06-28T16:11:15.085434+0000","last_fullsized":"2024-06-28T16:11:15.085434+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:32:47.953440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"260'6","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.118688+0000","last_change":"2024-06-28T15:21:07.031673+0000","last_active":"2024-06-28T16:13:40.118688+0000","last_peered":"2024-06-28T16:13:40.118688+0000","last_clean":"2024-06-28T16:13:40.118688+0000","last_became_active":"2024-06-28T15:21:07.030888+0000","last_became_peered":"2024-06-28T15:21:07.030888+0000","last_unstale":"2024-06-28T16:13:40.118688+0000","last_undegraded":"2024-06-28T16:13:40.118688+0000","last_fullsized":"2024-06-28T16:13:40.118688+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:54.912979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":6,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"260'72","reported_seq":247,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:45.552156+0000","last_change":"2024-06-28T15:17:23.993803+0000","last_active":"2024-06-28T16:10:45.552156+0000","last_peered":"2024-06-28T16:10:45.552156+0000","last_clean":"2024-06-28T16:10:45.552156+0000","last_became_active":"2024-06-28T15:17:23.993026+0000","last_became_peered":"2024-06-28T15:17:23.993026+0000","last_unstale":"2024-06-28T16:10:45.552156+0000","last_undegraded":"2024-06-28T16:10:45.552156+0000","last_fullsized":"2024-06-28T16:10:45.552156+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:12:38.304019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":255,"num_write":131,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":26,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.077019+0000","last_change":"2024-06-28T15:37:33.351154+0000","last_active":"2024-06-28T16:10:05.077019+0000","last_peered":"2024-06-28T16:10:05.077019+0000","last_clean":"2024-06-28T16:10:05.077019+0000","last_became_active":"2024-06-28T15:37:33.350447+0000","last_became_peered":"2024-06-28T15:37:33.350447+0000","last_unstale":"2024-06-28T16:10:05.077019+0000","last_undegraded":"2024-06-28T16:10:05.077019+0000","last_fullsized":"2024-06-28T16:10:05.077019+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:41:10.113052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":43,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.504142+0000","last_change":"2024-06-28T15:21:11.101407+0000","last_active":"2024-06-28T16:11:10.504142+0000","last_peered":"2024-06-28T16:11:10.504142+0000","last_clean":"2024-06-28T16:11:10.504142+0000","last_became_active":"2024-06-28T15:21:11.100603+0000","last_became_peered":"2024-06-28T15:21:11.100603+0000","last_unstale":"2024-06-28T16:11:10.504142+0000","last_undegraded":"2024-06-28T16:11:10.504142+0000","last_fullsized":"2024-06-28T16:11:10.504142+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:33:54.057658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"260'3","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.557448+0000","last_change":"2024-06-28T15:21:07.047854+0000","last_active":"2024-06-28T16:11:10.557448+0000","last_peered":"2024-06-28T16:11:10.557448+0000","last_clean":"2024-06-28T16:11:10.557448+0000","last_became_active":"2024-06-28T15:21:07.045902+0000","last_became_peered":"2024-06-28T15:21:07.045902+0000","last_unstale":"2024-06-28T16:11:10.557448+0000","last_undegraded":"2024-06-28T16:11:10.557448+0000","last_fullsized":"2024-06-28T16:11:10.557448+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:00:14.390394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1088721,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":40,"num_write":8,"num_write_kb":1064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"260'48","reported_seq":192,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.087151+0000","last_change":"2024-06-28T15:17:24.025635+0000","last_active":"2024-06-28T16:10:35.087151+0000","last_peered":"2024-06-28T16:10:35.087151+0000","last_clean":"2024-06-28T16:10:35.087151+0000","last_became_active":"2024-06-28T15:17:24.016249+0000","last_became_peered":"2024-06-28T15:17:24.016249+0000","last_unstale":"2024-06-28T16:10:35.087151+0000","last_undegraded":"2024-06-28T16:10:35.087151+0000","last_fullsized":"2024-06-28T16:10:35.087151+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:59:25.184524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":260,"num_read_kb":188,"num_write":81,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.083231+0000","last_change":"2024-06-28T15:37:33.317386+0000","last_active":"2024-06-28T16:10:15.083231+0000","last_peered":"2024-06-28T16:10:15.083231+0000","last_clean":"2024-06-28T16:10:15.083231+0000","last_became_active":"2024-06-28T15:37:33.316616+0000","last_became_peered":"2024-06-28T15:37:33.316616+0000","last_unstale":"2024-06-28T16:10:15.083231+0000","last_undegraded":"2024-06-28T16:10:15.083231+0000","last_fullsized":"2024-06-28T16:10:15.083231+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:02:05.364651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"260'8","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:20.333490+0000","last_change":"2024-06-28T15:21:11.099574+0000","last_active":"2024-06-28T16:14:20.333490+0000","last_peered":"2024-06-28T16:14:20.333490+0000","last_clean":"2024-06-28T16:14:20.333490+0000","last_became_active":"2024-06-28T15:21:11.094947+0000","last_became_peered":"2024-06-28T15:21:11.094947+0000","last_unstale":"2024-06-28T16:14:20.333490+0000","last_undegraded":"2024-06-28T16:14:20.333490+0000","last_fullsized":"2024-06-28T16:14:20.333490+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:14:27.139498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":12406,"num_write":8,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"260'5","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:45.099524+0000","last_change":"2024-06-28T15:21:07.026138+0000","last_active":"2024-06-28T16:12:45.099524+0000","last_peered":"2024-06-28T16:12:45.099524+0000","last_clean":"2024-06-28T16:12:45.099524+0000","last_became_active":"2024-06-28T15:21:07.025396+0000","last_became_peered":"2024-06-28T15:21:07.025396+0000","last_unstale":"2024-06-28T16:12:45.099524+0000","last_undegraded":"2024-06-28T16:12:45.099524+0000","last_fullsized":"2024-06-28T16:12:45.099524+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:48:53.728948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":5,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"257'14","reported_seq":96,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.550345+0000","last_change":"2024-06-28T15:17:24.015094+0000","last_active":"2024-06-28T16:10:35.550345+0000","last_peered":"2024-06-28T16:10:35.550345+0000","last_clean":"2024-06-28T16:10:35.550345+0000","last_became_active":"2024-06-28T15:17:24.010190+0000","last_became_peered":"2024-06-28T15:17:24.010190+0000","last_unstale":"2024-06-28T16:10:35.550345+0000","last_undegraded":"2024-06-28T16:10:35.550345+0000","last_fullsized":"2024-06-28T16:10:35.550345+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:20:30.481056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":51,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"260'1","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.255026+0000","last_change":"2024-06-28T15:37:33.338222+0000","last_active":"2024-06-28T16:10:15.255026+0000","last_peered":"2024-06-28T16:10:15.255026+0000","last_clean":"2024-06-28T16:10:15.255026+0000","last_became_active":"2024-06-28T15:37:33.334097+0000","last_became_peered":"2024-06-28T15:37:33.334097+0000","last_unstale":"2024-06-28T16:10:15.255026+0000","last_undegraded":"2024-06-28T16:10:15.255026+0000","last_fullsized":"2024-06-28T16:10:15.255026+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:04:55.107050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"260'9","reported_seq":62,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.508721+0000","last_change":"2024-06-28T15:21:11.107909+0000","last_active":"2024-06-28T16:14:33.508721+0000","last_peered":"2024-06-28T16:14:33.508721+0000","last_clean":"2024-06-28T16:14:33.508721+0000","last_became_active":"2024-06-28T15:21:11.107155+0000","last_became_peered":"2024-06-28T15:21:11.107155+0000","last_unstale":"2024-06-28T16:14:33.508721+0000","last_undegraded":"2024-06-28T16:14:33.508721+0000","last_fullsized":"2024-06-28T16:14:33.508721+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:27:41.181651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21504,"num_write":14,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":49,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.102713+0000","last_change":"2024-06-28T15:21:07.043526+0000","last_active":"2024-06-28T16:11:55.102713+0000","last_peered":"2024-06-28T16:11:55.102713+0000","last_clean":"2024-06-28T16:11:55.102713+0000","last_became_active":"2024-06-28T15:21:07.042691+0000","last_became_peered":"2024-06-28T15:21:07.042691+0000","last_unstale":"2024-06-28T16:11:55.102713+0000","last_undegraded":"2024-06-28T16:11:55.102713+0000","last_fullsized":"2024-06-28T16:11:55.102713+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:56:49.487276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"257'73","reported_seq":289,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.496134+0000","last_change":"2024-06-28T15:17:24.028644+0000","last_active":"2024-06-28T16:10:35.496134+0000","last_peered":"2024-06-28T16:10:35.496134+0000","last_clean":"2024-06-28T16:10:35.496134+0000","last_became_active":"2024-06-28T15:17:24.027786+0000","last_became_peered":"2024-06-28T15:17:24.027786+0000","last_unstale":"2024-06-28T16:10:35.496134+0000","last_undegraded":"2024-06-28T16:10:35.496134+0000","last_fullsized":"2024-06-28T16:10:35.496134+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:26:10.319359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":461,"num_read_kb":332,"num_write":85,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.079074+0000","last_change":"2024-06-28T15:37:33.355373+0000","last_active":"2024-06-28T16:10:15.079074+0000","last_peered":"2024-06-28T16:10:15.079074+0000","last_clean":"2024-06-28T16:10:15.079074+0000","last_became_active":"2024-06-28T15:37:33.354730+0000","last_became_peered":"2024-06-28T15:37:33.354730+0000","last_unstale":"2024-06-28T16:10:15.079074+0000","last_undegraded":"2024-06-28T16:10:15.079074+0000","last_fullsized":"2024-06-28T16:10:15.079074+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:33:00.069224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"260'4","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:20.333564+0000","last_change":"2024-06-28T15:21:11.094517+0000","last_active":"2024-06-28T16:14:20.333564+0000","last_peered":"2024-06-28T16:14:20.333564+0000","last_clean":"2024-06-28T16:14:20.333564+0000","last_became_active":"2024-06-28T15:21:11.093751+0000","last_became_peered":"2024-06-28T15:21:11.093751+0000","last_unstale":"2024-06-28T16:14:20.333564+0000","last_undegraded":"2024-06-28T16:14:20.333564+0000","last_fullsized":"2024-06-28T16:14:20.333564+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:37:10.229225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"260'2","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:20.305025+0000","last_change":"2024-06-28T15:21:07.043427+0000","last_active":"2024-06-28T16:14:20.305025+0000","last_peered":"2024-06-28T16:14:20.305025+0000","last_clean":"2024-06-28T16:14:20.305025+0000","last_became_active":"2024-06-28T15:21:07.042846+0000","last_became_peered":"2024-06-28T15:21:07.042846+0000","last_unstale":"2024-06-28T16:14:20.305025+0000","last_undegraded":"2024-06-28T16:14:20.305025+0000","last_fullsized":"2024-06-28T16:14:20.305025+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:41:49.061527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":59,"num_write":2,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"260'31","reported_seq":144,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.258858+0000","last_change":"2024-06-28T15:17:24.017530+0000","last_active":"2024-06-28T16:10:35.258858+0000","last_peered":"2024-06-28T16:10:35.258858+0000","last_clean":"2024-06-28T16:10:35.258858+0000","last_became_active":"2024-06-28T15:17:24.016275+0000","last_became_peered":"2024-06-28T15:17:24.016275+0000","last_unstale":"2024-06-28T16:10:35.258858+0000","last_undegraded":"2024-06-28T16:10:35.258858+0000","last_fullsized":"2024-06-28T16:10:35.258858+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:46:12.119777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":116,"num_write":60,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.065428+0000","last_change":"2024-06-28T15:37:33.323939+0000","last_active":"2024-06-28T16:10:15.065428+0000","last_peered":"2024-06-28T16:10:15.065428+0000","last_clean":"2024-06-28T16:10:15.065428+0000","last_became_active":"2024-06-28T15:37:33.323174+0000","last_became_peered":"2024-06-28T15:37:33.323174+0000","last_unstale":"2024-06-28T16:10:15.065428+0000","last_undegraded":"2024-06-28T16:10:15.065428+0000","last_fullsized":"2024-06-28T16:10:15.065428+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:23:02.255965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"260'9","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.510895+0000","last_change":"2024-06-28T15:21:11.088257+0000","last_active":"2024-06-28T16:14:33.510895+0000","last_peered":"2024-06-28T16:14:33.510895+0000","last_clean":"2024-06-28T16:14:33.510895+0000","last_became_active":"2024-06-28T15:21:11.087119+0000","last_became_peered":"2024-06-28T15:21:11.087119+0000","last_unstale":"2024-06-28T16:14:33.510895+0000","last_undegraded":"2024-06-28T16:14:33.510895+0000","last_fullsized":"2024-06-28T16:14:33.510895+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:24:43.264477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":24576,"num_write":19,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.569812+0000","last_change":"2024-06-28T15:21:07.032975+0000","last_active":"2024-06-28T16:11:55.569812+0000","last_peered":"2024-06-28T16:11:55.569812+0000","last_clean":"2024-06-28T16:11:55.569812+0000","last_became_active":"2024-06-28T15:21:07.032092+0000","last_became_peered":"2024-06-28T15:21:07.032092+0000","last_unstale":"2024-06-28T16:11:55.569812+0000","last_undegraded":"2024-06-28T16:11:55.569812+0000","last_fullsized":"2024-06-28T16:11:55.569812+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:29:54.468683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"260'21","reported_seq":120,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.094899+0000","last_change":"2024-06-28T15:17:24.021516+0000","last_active":"2024-06-28T16:11:20.094899+0000","last_peered":"2024-06-28T16:11:20.094899+0000","last_clean":"2024-06-28T16:11:20.094899+0000","last_became_active":"2024-06-28T15:17:24.020899+0000","last_became_peered":"2024-06-28T15:17:24.020899+0000","last_unstale":"2024-06-28T16:11:20.094899+0000","last_undegraded":"2024-06-28T16:11:20.094899+0000","last_fullsized":"2024-06-28T16:11:20.094899+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:16:16.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":86,"num_write":39,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.508021+0000","last_change":"2024-06-28T15:37:33.339666+0000","last_active":"2024-06-28T16:11:25.508021+0000","last_peered":"2024-06-28T16:11:25.508021+0000","last_clean":"2024-06-28T16:11:25.508021+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:11:25.508021+0000","last_undegraded":"2024-06-28T16:11:25.508021+0000","last_fullsized":"2024-06-28T16:11:25.508021+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:16:22.839274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"257'3","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.083204+0000","last_change":"2024-06-28T15:21:11.098494+0000","last_active":"2024-06-28T16:10:15.083204+0000","last_peered":"2024-06-28T16:10:15.083204+0000","last_clean":"2024-06-28T16:10:15.083204+0000","last_became_active":"2024-06-28T15:21:11.097622+0000","last_became_peered":"2024-06-28T15:21:11.097622+0000","last_unstale":"2024-06-28T16:10:15.083204+0000","last_undegraded":"2024-06-28T16:10:15.083204+0000","last_fullsized":"2024-06-28T16:10:15.083204+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:35:49.979919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"0'0","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.071391+0000","last_change":"2024-06-28T15:21:07.039533+0000","last_active":"2024-06-28T16:10:10.071391+0000","last_peered":"2024-06-28T16:10:10.071391+0000","last_clean":"2024-06-28T16:10:10.071391+0000","last_became_active":"2024-06-28T15:21:07.038746+0000","last_became_peered":"2024-06-28T15:21:07.038746+0000","last_unstale":"2024-06-28T16:10:10.071391+0000","last_undegraded":"2024-06-28T16:10:10.071391+0000","last_fullsized":"2024-06-28T16:10:10.071391+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:11:32.609450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"257'9","reported_seq":81,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.091202+0000","last_change":"2024-06-28T15:17:24.024616+0000","last_active":"2024-06-28T16:11:05.091202+0000","last_peered":"2024-06-28T16:11:05.091202+0000","last_clean":"2024-06-28T16:11:05.091202+0000","last_became_active":"2024-06-28T15:17:24.020276+0000","last_became_peered":"2024-06-28T15:17:24.020276+0000","last_unstale":"2024-06-28T16:11:05.091202+0000","last_undegraded":"2024-06-28T16:11:05.091202+0000","last_fullsized":"2024-06-28T16:11:05.091202+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:06.656845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.561512+0000","last_change":"2024-06-28T15:37:33.328340+0000","last_active":"2024-06-28T16:11:25.561512+0000","last_peered":"2024-06-28T16:11:25.561512+0000","last_clean":"2024-06-28T16:11:25.561512+0000","last_became_active":"2024-06-28T15:37:33.327497+0000","last_became_peered":"2024-06-28T15:37:33.327497+0000","last_unstale":"2024-06-28T16:11:25.561512+0000","last_undegraded":"2024-06-28T16:11:25.561512+0000","last_fullsized":"2024-06-28T16:11:25.561512+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:28:42.508678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"260'4","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.136460+0000","last_change":"2024-06-28T15:21:11.092761+0000","last_active":"2024-06-28T16:14:33.136460+0000","last_peered":"2024-06-28T16:14:33.136460+0000","last_clean":"2024-06-28T16:14:33.136460+0000","last_became_active":"2024-06-28T15:21:11.092033+0000","last_became_peered":"2024-06-28T15:21:11.092033+0000","last_unstale":"2024-06-28T16:14:33.136460+0000","last_undegraded":"2024-06-28T16:14:33.136460+0000","last_fullsized":"2024-06-28T16:14:33.136460+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:19:30.265533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6152,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.489165+0000","last_change":"2024-06-28T15:21:07.047668+0000","last_active":"2024-06-28T16:10:10.489165+0000","last_peered":"2024-06-28T16:10:10.489165+0000","last_clean":"2024-06-28T16:10:10.489165+0000","last_became_active":"2024-06-28T15:21:07.045842+0000","last_became_peered":"2024-06-28T15:21:07.045842+0000","last_unstale":"2024-06-28T16:10:10.489165+0000","last_undegraded":"2024-06-28T16:10:10.489165+0000","last_fullsized":"2024-06-28T16:10:10.489165+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:42:33.570243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"260'17","reported_seq":106,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.556141+0000","last_change":"2024-06-28T15:17:24.016733+0000","last_active":"2024-06-28T16:11:05.556141+0000","last_peered":"2024-06-28T16:11:05.556141+0000","last_clean":"2024-06-28T16:11:05.556141+0000","last_became_active":"2024-06-28T15:17:24.013118+0000","last_became_peered":"2024-06-28T15:17:24.013118+0000","last_unstale":"2024-06-28T16:11:05.556141+0000","last_undegraded":"2024-06-28T16:11:05.556141+0000","last_fullsized":"2024-06-28T16:11:05.556141+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:57:44.752951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":63,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"259'1","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.546888+0000","last_change":"2024-06-28T15:37:33.342777+0000","last_active":"2024-06-28T16:10:20.546888+0000","last_peered":"2024-06-28T16:10:20.546888+0000","last_clean":"2024-06-28T16:10:20.546888+0000","last_became_active":"2024-06-28T15:37:33.339097+0000","last_became_peered":"2024-06-28T15:37:33.339097+0000","last_unstale":"2024-06-28T16:10:20.546888+0000","last_undegraded":"2024-06-28T16:10:20.546888+0000","last_fullsized":"2024-06-28T16:10:20.546888+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:34:20.645575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"257'3","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.489379+0000","last_change":"2024-06-28T15:21:11.101867+0000","last_active":"2024-06-28T16:10:10.489379+0000","last_peered":"2024-06-28T16:10:10.489379+0000","last_clean":"2024-06-28T16:10:10.489379+0000","last_became_active":"2024-06-28T15:21:11.101063+0000","last_became_peered":"2024-06-28T15:21:11.101063+0000","last_unstale":"2024-06-28T16:10:10.489379+0000","last_undegraded":"2024-06-28T16:10:10.489379+0000","last_fullsized":"2024-06-28T16:10:10.489379+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:58:49.328893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"257'3","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.070602+0000","last_change":"2024-06-28T15:21:07.042183+0000","last_active":"2024-06-28T16:10:10.070602+0000","last_peered":"2024-06-28T16:10:10.070602+0000","last_clean":"2024-06-28T16:10:10.070602+0000","last_became_active":"2024-06-28T15:21:07.040973+0000","last_became_peered":"2024-06-28T15:21:07.040973+0000","last_unstale":"2024-06-28T16:10:10.070602+0000","last_undegraded":"2024-06-28T16:10:10.070602+0000","last_fullsized":"2024-06-28T16:10:10.070602+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:38:10.417534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"260'22","reported_seq":115,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.083945+0000","last_change":"2024-06-28T15:17:24.029346+0000","last_active":"2024-06-28T16:11:05.083945+0000","last_peered":"2024-06-28T16:11:05.083945+0000","last_clean":"2024-06-28T16:11:05.083945+0000","last_became_active":"2024-06-28T15:17:24.028410+0000","last_became_peered":"2024-06-28T15:17:24.028410+0000","last_unstale":"2024-06-28T16:11:05.083945+0000","last_undegraded":"2024-06-28T16:11:05.083945+0000","last_fullsized":"2024-06-28T16:11:05.083945+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:25:05.370434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"260'1","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.507957+0000","last_change":"2024-06-28T15:37:33.341173+0000","last_active":"2024-06-28T16:11:25.507957+0000","last_peered":"2024-06-28T16:11:25.507957+0000","last_clean":"2024-06-28T16:11:25.507957+0000","last_became_active":"2024-06-28T15:37:33.340370+0000","last_became_peered":"2024-06-28T15:37:33.340370+0000","last_unstale":"2024-06-28T16:11:25.507957+0000","last_undegraded":"2024-06-28T16:11:25.507957+0000","last_fullsized":"2024-06-28T16:11:25.507957+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:07:43.988727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"257'3","reported_seq":48,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.071850+0000","last_change":"2024-06-28T15:21:11.104826+0000","last_active":"2024-06-28T16:10:15.071850+0000","last_peered":"2024-06-28T16:10:15.071850+0000","last_clean":"2024-06-28T16:10:15.071850+0000","last_became_active":"2024-06-28T15:21:11.104038+0000","last_became_peered":"2024-06-28T15:21:11.104038+0000","last_unstale":"2024-06-28T16:10:15.071850+0000","last_undegraded":"2024-06-28T16:10:15.071850+0000","last_fullsized":"2024-06-28T16:10:15.071850+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:38:01.314736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"257'8","reported_seq":71,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.064325+0000","last_change":"2024-06-28T15:21:07.010980+0000","last_active":"2024-06-28T16:10:10.064325+0000","last_peered":"2024-06-28T16:10:10.064325+0000","last_clean":"2024-06-28T16:10:10.064325+0000","last_became_active":"2024-06-28T15:21:07.008899+0000","last_became_peered":"2024-06-28T15:21:07.008899+0000","last_unstale":"2024-06-28T16:10:10.064325+0000","last_undegraded":"2024-06-28T16:10:10.064325+0000","last_fullsized":"2024-06-28T16:10:10.064325+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:39:00.266882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":7,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"260'42","reported_seq":165,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.502745+0000","last_change":"2024-06-28T15:17:24.020041+0000","last_active":"2024-06-28T16:11:05.502745+0000","last_peered":"2024-06-28T16:11:05.502745+0000","last_clean":"2024-06-28T16:11:05.502745+0000","last_became_active":"2024-06-28T15:17:24.019180+0000","last_became_peered":"2024-06-28T15:17:24.019180+0000","last_unstale":"2024-06-28T16:11:05.502745+0000","last_undegraded":"2024-06-28T16:11:05.502745+0000","last_fullsized":"2024-06-28T16:11:05.502745+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:00:51.600204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":198,"num_read_kb":139,"num_write":93,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.280511+0000","last_change":"2024-06-28T15:37:33.342459+0000","last_active":"2024-06-28T16:12:25.280511+0000","last_peered":"2024-06-28T16:12:25.280511+0000","last_clean":"2024-06-28T16:12:25.280511+0000","last_became_active":"2024-06-28T15:37:33.341716+0000","last_became_peered":"2024-06-28T15:37:33.341716+0000","last_unstale":"2024-06-28T16:12:25.280511+0000","last_undegraded":"2024-06-28T16:12:25.280511+0000","last_fullsized":"2024-06-28T16:12:25.280511+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:53:31.136353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:40.497394+0000","last_change":"2024-06-28T15:21:11.094239+0000","last_active":"2024-06-28T16:10:40.497394+0000","last_peered":"2024-06-28T16:10:40.497394+0000","last_clean":"2024-06-28T16:10:40.497394+0000","last_became_active":"2024-06-28T15:21:11.093276+0000","last_became_peered":"2024-06-28T15:21:11.093276+0000","last_unstale":"2024-06-28T16:10:40.497394+0000","last_undegraded":"2024-06-28T16:10:40.497394+0000","last_fullsized":"2024-06-28T16:10:40.497394+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:48:45.762370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"260'9","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.368805+0000","last_change":"2024-06-28T15:21:07.041391+0000","last_active":"2024-06-28T16:14:33.368805+0000","last_peered":"2024-06-28T16:14:33.368805+0000","last_clean":"2024-06-28T16:14:33.368805+0000","last_became_active":"2024-06-28T15:21:07.040691+0000","last_became_peered":"2024-06-28T15:21:07.040691+0000","last_unstale":"2024-06-28T16:14:33.368805+0000","last_undegraded":"2024-06-28T16:14:33.368805+0000","last_fullsized":"2024-06-28T16:14:33.368805+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:20:52.003993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":17408,"num_write":9,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"260'47","reported_seq":184,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.509222+0000","last_change":"2024-06-28T15:17:24.028252+0000","last_active":"2024-06-28T16:11:30.509222+0000","last_peered":"2024-06-28T16:11:30.509222+0000","last_clean":"2024-06-28T16:11:30.509222+0000","last_became_active":"2024-06-28T15:17:24.027610+0000","last_became_peered":"2024-06-28T15:17:24.027610+0000","last_unstale":"2024-06-28T16:11:30.509222+0000","last_undegraded":"2024-06-28T16:11:30.509222+0000","last_fullsized":"2024-06-28T16:11:30.509222+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:08:18.788559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":231,"num_read_kb":169,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.094821+0000","last_change":"2024-06-28T15:37:33.347375+0000","last_active":"2024-06-28T16:11:20.094821+0000","last_peered":"2024-06-28T16:11:20.094821+0000","last_clean":"2024-06-28T16:11:20.094821+0000","last_became_active":"2024-06-28T15:37:33.346414+0000","last_became_peered":"2024-06-28T15:37:33.346414+0000","last_unstale":"2024-06-28T16:11:20.094821+0000","last_undegraded":"2024-06-28T16:11:20.094821+0000","last_fullsized":"2024-06-28T16:11:20.094821+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:39:51.219545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"257'5","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.082722+0000","last_change":"2024-06-28T15:22:00.970955+0000","last_active":"2024-06-28T16:11:30.082722+0000","last_peered":"2024-06-28T16:11:30.082722+0000","last_clean":"2024-06-28T16:11:30.082722+0000","last_became_active":"2024-06-28T15:22:00.940760+0000","last_became_peered":"2024-06-28T15:22:00.940760+0000","last_unstale":"2024-06-28T16:11:30.082722+0000","last_undegraded":"2024-06-28T16:11:30.082722+0000","last_fullsized":"2024-06-28T16:11:30.082722+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:14:23.932998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"260'3","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.287511+0000","last_change":"2024-06-28T15:21:07.031060+0000","last_active":"2024-06-28T16:12:55.287511+0000","last_peered":"2024-06-28T16:12:55.287511+0000","last_clean":"2024-06-28T16:12:55.287511+0000","last_became_active":"2024-06-28T15:21:07.030163+0000","last_became_peered":"2024-06-28T15:21:07.030163+0000","last_unstale":"2024-06-28T16:12:55.287511+0000","last_undegraded":"2024-06-28T16:12:55.287511+0000","last_fullsized":"2024-06-28T16:12:55.287511+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:19:20.592250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"260'40","reported_seq":171,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.081619+0000","last_change":"2024-06-28T15:17:24.014719+0000","last_active":"2024-06-28T16:11:25.081619+0000","last_peered":"2024-06-28T16:11:25.081619+0000","last_clean":"2024-06-28T16:11:25.081619+0000","last_became_active":"2024-06-28T15:17:24.012572+0000","last_became_peered":"2024-06-28T15:17:24.012572+0000","last_unstale":"2024-06-28T16:11:25.081619+0000","last_undegraded":"2024-06-28T16:11:25.081619+0000","last_fullsized":"2024-06-28T16:11:25.081619+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:07:56.306439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":148,"num_write":94,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.099240+0000","last_change":"2024-06-28T15:37:33.353545+0000","last_active":"2024-06-28T16:12:15.099240+0000","last_peered":"2024-06-28T16:12:15.099240+0000","last_clean":"2024-06-28T16:12:15.099240+0000","last_became_active":"2024-06-28T15:37:33.352959+0000","last_became_peered":"2024-06-28T15:37:33.352959+0000","last_unstale":"2024-06-28T16:12:15.099240+0000","last_undegraded":"2024-06-28T16:12:15.099240+0000","last_fullsized":"2024-06-28T16:12:15.099240+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:42.580937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.092127+0000","last_change":"2024-06-28T15:21:11.079730+0000","last_active":"2024-06-28T16:11:00.092127+0000","last_peered":"2024-06-28T16:11:00.092127+0000","last_clean":"2024-06-28T16:11:00.092127+0000","last_became_active":"2024-06-28T15:21:11.079036+0000","last_became_peered":"2024-06-28T15:21:11.079036+0000","last_unstale":"2024-06-28T16:11:00.092127+0000","last_undegraded":"2024-06-28T16:11:00.092127+0000","last_fullsized":"2024-06-28T16:11:00.092127+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:36:54.128714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"0'0","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:40.551262+0000","last_change":"2024-06-28T15:21:07.028297+0000","last_active":"2024-06-28T16:10:40.551262+0000","last_peered":"2024-06-28T16:10:40.551262+0000","last_clean":"2024-06-28T16:10:40.551262+0000","last_became_active":"2024-06-28T15:21:07.027451+0000","last_became_peered":"2024-06-28T15:21:07.027451+0000","last_unstale":"2024-06-28T16:10:40.551262+0000","last_undegraded":"2024-06-28T16:10:40.551262+0000","last_fullsized":"2024-06-28T16:10:40.551262+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:44:47.388468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"260'16","reported_seq":97,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.087601+0000","last_change":"2024-06-28T15:17:24.027026+0000","last_active":"2024-06-28T16:11:25.087601+0000","last_peered":"2024-06-28T16:11:25.087601+0000","last_clean":"2024-06-28T16:11:25.087601+0000","last_became_active":"2024-06-28T15:17:24.026271+0000","last_became_peered":"2024-06-28T15:17:24.026271+0000","last_unstale":"2024-06-28T16:11:25.087601+0000","last_undegraded":"2024-06-28T16:11:25.087601+0000","last_fullsized":"2024-06-28T16:11:25.087601+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:23:12.426770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":48,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.107001+0000","last_change":"2024-06-28T15:37:33.326639+0000","last_active":"2024-06-28T16:12:15.107001+0000","last_peered":"2024-06-28T16:12:15.107001+0000","last_clean":"2024-06-28T16:12:15.107001+0000","last_became_active":"2024-06-28T15:37:33.325849+0000","last_became_peered":"2024-06-28T15:37:33.325849+0000","last_unstale":"2024-06-28T16:12:15.107001+0000","last_undegraded":"2024-06-28T16:12:15.107001+0000","last_fullsized":"2024-06-28T16:12:15.107001+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:14:13.968785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"257'5","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:40.497479+0000","last_change":"2024-06-28T15:21:11.108377+0000","last_active":"2024-06-28T16:10:40.497479+0000","last_peered":"2024-06-28T16:10:40.497479+0000","last_clean":"2024-06-28T16:10:40.497479+0000","last_became_active":"2024-06-28T15:21:11.107498+0000","last_became_peered":"2024-06-28T15:21:11.107498+0000","last_unstale":"2024-06-28T16:10:40.497479+0000","last_undegraded":"2024-06-28T16:10:40.497479+0000","last_fullsized":"2024-06-28T16:10:40.497479+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:33:14.411962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"257'7","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.091049+0000","last_change":"2024-06-28T15:21:07.030359+0000","last_active":"2024-06-28T16:10:55.091049+0000","last_peered":"2024-06-28T16:10:55.091049+0000","last_clean":"2024-06-28T16:10:55.091049+0000","last_became_active":"2024-06-28T15:21:07.029525+0000","last_became_peered":"2024-06-28T15:21:07.029525+0000","last_unstale":"2024-06-28T16:10:55.091049+0000","last_undegraded":"2024-06-28T16:10:55.091049+0000","last_fullsized":"2024-06-28T16:10:55.091049+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:27:20.230827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":11,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"260'34","reported_seq":151,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.268692+0000","last_change":"2024-06-28T15:17:24.008356+0000","last_active":"2024-06-28T16:11:25.268692+0000","last_peered":"2024-06-28T16:11:25.268692+0000","last_clean":"2024-06-28T16:11:25.268692+0000","last_became_active":"2024-06-28T15:17:24.007649+0000","last_became_peered":"2024-06-28T15:17:24.007649+0000","last_unstale":"2024-06-28T16:11:25.268692+0000","last_undegraded":"2024-06-28T16:11:25.268692+0000","last_fullsized":"2024-06-28T16:11:25.268692+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:41:30.857517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":172,"num_read_kb":122,"num_write":76,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.506787+0000","last_change":"2024-06-28T15:37:33.335759+0000","last_active":"2024-06-28T16:11:20.506787+0000","last_peered":"2024-06-28T16:11:20.506787+0000","last_clean":"2024-06-28T16:11:20.506787+0000","last_became_active":"2024-06-28T15:37:33.334082+0000","last_became_peered":"2024-06-28T15:37:33.334082+0000","last_unstale":"2024-06-28T16:11:20.506787+0000","last_undegraded":"2024-06-28T16:11:20.506787+0000","last_fullsized":"2024-06-28T16:11:20.506787+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:37:25.552446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"0'0","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.083900+0000","last_change":"2024-06-28T15:21:11.108205+0000","last_active":"2024-06-28T16:11:05.083900+0000","last_peered":"2024-06-28T16:11:05.083900+0000","last_clean":"2024-06-28T16:11:05.083900+0000","last_became_active":"2024-06-28T15:21:11.107686+0000","last_became_peered":"2024-06-28T15:21:11.107686+0000","last_unstale":"2024-06-28T16:11:05.083900+0000","last_undegraded":"2024-06-28T16:11:05.083900+0000","last_fullsized":"2024-06-28T16:11:05.083900+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:57:49.231179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"0'0","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.555131+0000","last_change":"2024-06-28T15:21:07.033578+0000","last_active":"2024-06-28T16:11:00.555131+0000","last_peered":"2024-06-28T16:11:00.555131+0000","last_clean":"2024-06-28T16:11:00.555131+0000","last_became_active":"2024-06-28T15:21:07.032719+0000","last_became_peered":"2024-06-28T15:21:07.032719+0000","last_unstale":"2024-06-28T16:11:00.555131+0000","last_undegraded":"2024-06-28T16:11:00.555131+0000","last_fullsized":"2024-06-28T16:11:00.555131+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:15.597833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"260'45","reported_seq":182,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.561334+0000","last_change":"2024-06-28T15:17:24.030065+0000","last_active":"2024-06-28T16:11:25.561334+0000","last_peered":"2024-06-28T16:11:25.561334+0000","last_clean":"2024-06-28T16:11:25.561334+0000","last_became_active":"2024-06-28T15:17:24.029532+0000","last_became_peered":"2024-06-28T15:17:24.029532+0000","last_unstale":"2024-06-28T16:11:25.561334+0000","last_undegraded":"2024-06-28T16:11:25.561334+0000","last_fullsized":"2024-06-28T16:11:25.561334+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:32:40.759334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":234,"num_read_kb":165,"num_write":86,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.097302+0000","last_change":"2024-06-28T15:37:33.348528+0000","last_active":"2024-06-28T16:11:30.097302+0000","last_peered":"2024-06-28T16:11:30.097302+0000","last_clean":"2024-06-28T16:11:30.097302+0000","last_became_active":"2024-06-28T15:37:33.347691+0000","last_became_peered":"2024-06-28T15:37:33.347691+0000","last_unstale":"2024-06-28T16:11:30.097302+0000","last_undegraded":"2024-06-28T16:11:30.097302+0000","last_fullsized":"2024-06-28T16:11:30.097302+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:19:46.828298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.089675+0000","last_change":"2024-06-28T15:21:11.092385+0000","last_active":"2024-06-28T16:11:00.089675+0000","last_peered":"2024-06-28T16:11:00.089675+0000","last_clean":"2024-06-28T16:11:00.089675+0000","last_became_active":"2024-06-28T15:21:11.091656+0000","last_became_peered":"2024-06-28T15:21:11.091656+0000","last_unstale":"2024-06-28T16:11:00.089675+0000","last_undegraded":"2024-06-28T16:11:00.089675+0000","last_fullsized":"2024-06-28T16:11:00.089675+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:39:05.106486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"260'5","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.120386+0000","last_change":"2024-06-28T15:21:07.030338+0000","last_active":"2024-06-28T16:13:10.120386+0000","last_peered":"2024-06-28T16:13:10.120386+0000","last_clean":"2024-06-28T16:13:10.120386+0000","last_became_active":"2024-06-28T15:21:07.029495+0000","last_became_peered":"2024-06-28T15:21:07.029495+0000","last_unstale":"2024-06-28T16:13:10.120386+0000","last_undegraded":"2024-06-28T16:13:10.120386+0000","last_fullsized":"2024-06-28T16:13:10.120386+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:18:26.892995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":318,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"260'39","reported_seq":162,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.101407+0000","last_change":"2024-06-28T15:17:24.024944+0000","last_active":"2024-06-28T16:11:50.101407+0000","last_peered":"2024-06-28T16:11:50.101407+0000","last_clean":"2024-06-28T16:11:50.101407+0000","last_became_active":"2024-06-28T15:17:24.021200+0000","last_became_peered":"2024-06-28T15:17:24.021200+0000","last_unstale":"2024-06-28T16:11:50.101407+0000","last_undegraded":"2024-06-28T16:11:50.101407+0000","last_fullsized":"2024-06-28T16:11:50.101407+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:06:00.686781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":191,"num_read_kb":133,"num_write":87,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"260'1","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.094873+0000","last_change":"2024-06-28T15:37:33.336238+0000","last_active":"2024-06-28T16:11:20.094873+0000","last_peered":"2024-06-28T16:11:20.094873+0000","last_clean":"2024-06-28T16:11:20.094873+0000","last_became_active":"2024-06-28T15:37:33.335464+0000","last_became_peered":"2024-06-28T15:37:33.335464+0000","last_unstale":"2024-06-28T16:11:20.094873+0000","last_undegraded":"2024-06-28T16:11:20.094873+0000","last_fullsized":"2024-06-28T16:11:20.094873+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:19:13.910107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"257'3","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:45.088990+0000","last_change":"2024-06-28T15:21:11.094534+0000","last_active":"2024-06-28T16:10:45.088990+0000","last_peered":"2024-06-28T16:10:45.088990+0000","last_clean":"2024-06-28T16:10:45.088990+0000","last_became_active":"2024-06-28T15:21:11.091647+0000","last_became_peered":"2024-06-28T15:21:11.091647+0000","last_unstale":"2024-06-28T16:10:45.088990+0000","last_undegraded":"2024-06-28T16:10:45.088990+0000","last_fullsized":"2024-06-28T16:10:45.088990+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:26:33.093064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"257'3","reported_seq":57,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.557369+0000","last_change":"2024-06-28T15:21:07.046952+0000","last_active":"2024-06-28T16:11:10.557369+0000","last_peered":"2024-06-28T16:11:10.557369+0000","last_clean":"2024-06-28T16:11:10.557369+0000","last_became_active":"2024-06-28T15:21:07.046218+0000","last_became_peered":"2024-06-28T15:21:07.046218+0000","last_unstale":"2024-06-28T16:11:10.557369+0000","last_undegraded":"2024-06-28T16:11:10.557369+0000","last_fullsized":"2024-06-28T16:11:10.557369+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:30:18.956581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"260'54","reported_seq":208,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.510120+0000","last_change":"2024-06-28T15:17:24.011432+0000","last_active":"2024-06-28T16:11:35.510120+0000","last_peered":"2024-06-28T16:11:35.510120+0000","last_clean":"2024-06-28T16:11:35.510120+0000","last_became_active":"2024-06-28T15:17:24.010230+0000","last_became_peered":"2024-06-28T15:17:24.010230+0000","last_unstale":"2024-06-28T16:11:35.510120+0000","last_undegraded":"2024-06-28T16:11:35.510120+0000","last_fullsized":"2024-06-28T16:11:35.510120+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:14:20.556616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":283,"num_read_kb":205,"num_write":97,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.560195+0000","last_change":"2024-06-28T15:37:33.343356+0000","last_active":"2024-06-28T16:11:20.560195+0000","last_peered":"2024-06-28T16:11:20.560195+0000","last_clean":"2024-06-28T16:11:20.560195+0000","last_became_active":"2024-06-28T15:37:33.342585+0000","last_became_peered":"2024-06-28T15:37:33.342585+0000","last_unstale":"2024-06-28T16:11:20.560195+0000","last_undegraded":"2024-06-28T16:11:20.560195+0000","last_fullsized":"2024-06-28T16:11:20.560195+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:36:10.405028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"260'3","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:25.123229+0000","last_change":"2024-06-28T15:21:11.098149+0000","last_active":"2024-06-28T16:13:25.123229+0000","last_peered":"2024-06-28T16:13:25.123229+0000","last_clean":"2024-06-28T16:13:25.123229+0000","last_became_active":"2024-06-28T15:21:11.096884+0000","last_became_peered":"2024-06-28T15:21:11.096884+0000","last_unstale":"2024-06-28T16:13:25.123229+0000","last_undegraded":"2024-06-28T16:13:25.123229+0000","last_fullsized":"2024-06-28T16:13:25.123229+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:53:18.580625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"260'11","reported_seq":79,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.109271+0000","last_change":"2024-06-28T15:21:07.037099+0000","last_active":"2024-06-28T16:12:55.109271+0000","last_peered":"2024-06-28T16:12:55.109271+0000","last_clean":"2024-06-28T16:12:55.109271+0000","last_became_active":"2024-06-28T15:21:07.032545+0000","last_became_peered":"2024-06-28T15:21:07.032545+0000","last_unstale":"2024-06-28T16:12:55.109271+0000","last_undegraded":"2024-06-28T16:12:55.109271+0000","last_fullsized":"2024-06-28T16:12:55.109271+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:28:14.596619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":35842,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"260'40","reported_seq":171,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089612+0000","last_change":"2024-06-28T15:17:24.028816+0000","last_active":"2024-06-28T16:11:35.089612+0000","last_peered":"2024-06-28T16:11:35.089612+0000","last_clean":"2024-06-28T16:11:35.089612+0000","last_became_active":"2024-06-28T15:17:24.026702+0000","last_became_peered":"2024-06-28T15:17:24.026702+0000","last_unstale":"2024-06-28T16:11:35.089612+0000","last_undegraded":"2024-06-28T16:11:35.089612+0000","last_fullsized":"2024-06-28T16:11:35.089612+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:13:09.048722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":150,"num_write":83,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.088350+0000","last_change":"2024-06-28T15:37:33.343533+0000","last_active":"2024-06-28T16:10:55.088350+0000","last_peered":"2024-06-28T16:10:55.088350+0000","last_clean":"2024-06-28T16:10:55.088350+0000","last_became_active":"2024-06-28T15:37:33.342889+0000","last_became_peered":"2024-06-28T15:37:33.342889+0000","last_unstale":"2024-06-28T16:10:55.088350+0000","last_undegraded":"2024-06-28T16:10:55.088350+0000","last_fullsized":"2024-06-28T16:10:55.088350+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:44:29.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"260'1","reported_seq":49,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.561434+0000","last_change":"2024-06-28T15:21:11.093441+0000","last_active":"2024-06-28T16:11:25.561434+0000","last_peered":"2024-06-28T16:11:25.561434+0000","last_clean":"2024-06-28T16:11:25.561434+0000","last_became_active":"2024-06-28T15:21:11.092614+0000","last_became_peered":"2024-06-28T15:21:11.092614+0000","last_unstale":"2024-06-28T16:11:25.561434+0000","last_undegraded":"2024-06-28T16:11:25.561434+0000","last_fullsized":"2024-06-28T16:11:25.561434+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:43:32.181244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"257'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.092406+0000","last_change":"2024-06-28T15:21:07.035627+0000","last_active":"2024-06-28T16:11:10.092406+0000","last_peered":"2024-06-28T16:11:10.092406+0000","last_clean":"2024-06-28T16:11:10.092406+0000","last_became_active":"2024-06-28T15:21:07.034845+0000","last_became_peered":"2024-06-28T15:21:07.034845+0000","last_unstale":"2024-06-28T16:11:10.092406+0000","last_undegraded":"2024-06-28T16:11:10.092406+0000","last_fullsized":"2024-06-28T16:11:10.092406+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:39:02.467227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"260'28","reported_seq":139,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.563793+0000","last_change":"2024-06-28T15:17:24.016960+0000","last_active":"2024-06-28T16:11:35.563793+0000","last_peered":"2024-06-28T16:11:35.563793+0000","last_clean":"2024-06-28T16:11:35.563793+0000","last_became_active":"2024-06-28T15:17:24.014034+0000","last_became_peered":"2024-06-28T15:17:24.014034+0000","last_unstale":"2024-06-28T16:11:35.563793+0000","last_undegraded":"2024-06-28T16:11:35.563793+0000","last_fullsized":"2024-06-28T16:11:35.563793+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:12:33.973492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":149,"num_read_kb":105,"num_write":55,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.074736+0000","last_change":"2024-06-28T15:37:33.340550+0000","last_active":"2024-06-28T16:10:55.074736+0000","last_peered":"2024-06-28T16:10:55.074736+0000","last_clean":"2024-06-28T16:10:55.074736+0000","last_became_active":"2024-06-28T15:37:33.339831+0000","last_became_peered":"2024-06-28T15:37:33.339831+0000","last_unstale":"2024-06-28T16:10:55.074736+0000","last_undegraded":"2024-06-28T16:10:55.074736+0000","last_fullsized":"2024-06-28T16:10:55.074736+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:10:11.303905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"260'6","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.558788+0000","last_change":"2024-06-28T15:21:11.096084+0000","last_active":"2024-06-28T16:11:15.558788+0000","last_peered":"2024-06-28T16:11:15.558788+0000","last_clean":"2024-06-28T16:11:15.558788+0000","last_became_active":"2024-06-28T15:21:11.088011+0000","last_became_peered":"2024-06-28T15:21:11.088011+0000","last_unstale":"2024-06-28T16:11:15.558788+0000","last_undegraded":"2024-06-28T16:11:15.558788+0000","last_fullsized":"2024-06-28T16:11:15.558788+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:14:39.791560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5283025,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":20522,"num_write":10,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"0'0","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.268746+0000","last_change":"2024-06-28T15:21:07.031400+0000","last_active":"2024-06-28T16:11:25.268746+0000","last_peered":"2024-06-28T16:11:25.268746+0000","last_clean":"2024-06-28T16:11:25.268746+0000","last_became_active":"2024-06-28T15:21:07.030690+0000","last_became_peered":"2024-06-28T15:21:07.030690+0000","last_unstale":"2024-06-28T16:11:25.268746+0000","last_undegraded":"2024-06-28T16:11:25.268746+0000","last_fullsized":"2024-06-28T16:11:25.268746+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:42:25.816657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"260'31","reported_seq":144,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089847+0000","last_change":"2024-06-28T15:17:24.026986+0000","last_active":"2024-06-28T16:11:35.089847+0000","last_peered":"2024-06-28T16:11:35.089847+0000","last_clean":"2024-06-28T16:11:35.089847+0000","last_became_active":"2024-06-28T15:17:24.026404+0000","last_became_peered":"2024-06-28T15:17:24.026404+0000","last_unstale":"2024-06-28T16:11:35.089847+0000","last_undegraded":"2024-06-28T16:11:35.089847+0000","last_fullsized":"2024-06-28T16:11:35.089847+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:08:40.750828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":161,"num_read_kb":111,"num_write":59,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:45.101310+0000","last_change":"2024-06-28T15:37:33.332984+0000","last_active":"2024-06-28T16:11:45.101310+0000","last_peered":"2024-06-28T16:11:45.101310+0000","last_clean":"2024-06-28T16:11:45.101310+0000","last_became_active":"2024-06-28T15:37:33.332245+0000","last_became_peered":"2024-06-28T15:37:33.332245+0000","last_unstale":"2024-06-28T16:11:45.101310+0000","last_undegraded":"2024-06-28T16:11:45.101310+0000","last_fullsized":"2024-06-28T16:11:45.101310+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:30.067085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.090010+0000","last_change":"2024-06-28T15:21:11.097568+0000","last_active":"2024-06-28T16:11:00.090010+0000","last_peered":"2024-06-28T16:11:00.090010+0000","last_clean":"2024-06-28T16:11:00.090010+0000","last_became_active":"2024-06-28T15:21:11.096926+0000","last_became_peered":"2024-06-28T15:21:11.096926+0000","last_unstale":"2024-06-28T16:11:00.090010+0000","last_undegraded":"2024-06-28T16:11:00.090010+0000","last_fullsized":"2024-06-28T16:11:00.090010+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:33:16.290705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"260'2","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.105821+0000","last_change":"2024-06-28T15:21:07.037522+0000","last_active":"2024-06-28T16:13:10.105821+0000","last_peered":"2024-06-28T16:13:10.105821+0000","last_clean":"2024-06-28T16:13:10.105821+0000","last_became_active":"2024-06-28T15:21:07.036735+0000","last_became_peered":"2024-06-28T15:21:07.036735+0000","last_unstale":"2024-06-28T16:13:10.105821+0000","last_undegraded":"2024-06-28T16:13:10.105821+0000","last_fullsized":"2024-06-28T16:13:10.105821+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:56:53.756000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"260'49","reported_seq":197,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.101490+0000","last_change":"2024-06-28T15:17:24.026580+0000","last_active":"2024-06-28T16:11:50.101490+0000","last_peered":"2024-06-28T16:11:50.101490+0000","last_clean":"2024-06-28T16:11:50.101490+0000","last_became_active":"2024-06-28T15:17:24.025855+0000","last_became_peered":"2024-06-28T15:17:24.025855+0000","last_unstale":"2024-06-28T16:11:50.101490+0000","last_undegraded":"2024-06-28T16:11:50.101490+0000","last_fullsized":"2024-06-28T16:11:50.101490+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:14:11.100104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":190,"num_write":87,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.081854+0000","last_change":"2024-06-28T15:37:33.323701+0000","last_active":"2024-06-28T16:10:55.081854+0000","last_peered":"2024-06-28T16:10:55.081854+0000","last_clean":"2024-06-28T16:10:55.081854+0000","last_became_active":"2024-06-28T15:37:33.322927+0000","last_became_peered":"2024-06-28T15:37:33.322927+0000","last_unstale":"2024-06-28T16:10:55.081854+0000","last_undegraded":"2024-06-28T16:10:55.081854+0000","last_fullsized":"2024-06-28T16:10:55.081854+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:33:37.474011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.084125+0000","last_change":"2024-06-28T15:21:11.098521+0000","last_active":"2024-06-28T16:11:05.084125+0000","last_peered":"2024-06-28T16:11:05.084125+0000","last_clean":"2024-06-28T16:11:05.084125+0000","last_became_active":"2024-06-28T15:21:11.097766+0000","last_became_peered":"2024-06-28T15:21:11.097766+0000","last_unstale":"2024-06-28T16:11:05.084125+0000","last_undegraded":"2024-06-28T16:11:05.084125+0000","last_fullsized":"2024-06-28T16:11:05.084125+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:49:28.598405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.090053+0000","last_change":"2024-06-28T15:21:07.040142+0000","last_active":"2024-06-28T16:11:00.090053+0000","last_peered":"2024-06-28T16:11:00.090053+0000","last_clean":"2024-06-28T16:11:00.090053+0000","last_became_active":"2024-06-28T15:21:07.039381+0000","last_became_peered":"2024-06-28T15:21:07.039381+0000","last_unstale":"2024-06-28T16:11:00.090053+0000","last_undegraded":"2024-06-28T16:11:00.090053+0000","last_fullsized":"2024-06-28T16:11:00.090053+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:56:40.727136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"260'26","reported_seq":137,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.099274+0000","last_change":"2024-06-28T15:17:24.027314+0000","last_active":"2024-06-28T16:11:35.099274+0000","last_peered":"2024-06-28T16:11:35.099274+0000","last_clean":"2024-06-28T16:11:35.099274+0000","last_became_active":"2024-06-28T15:17:24.026580+0000","last_became_peered":"2024-06-28T15:17:24.026580+0000","last_unstale":"2024-06-28T16:11:35.099274+0000","last_undegraded":"2024-06-28T16:11:35.099274+0000","last_fullsized":"2024-06-28T16:11:35.099274+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:45:13.623723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":143,"num_read_kb":101,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"259'1","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.278572+0000","last_change":"2024-06-28T15:37:33.349810+0000","last_active":"2024-06-28T16:12:15.278572+0000","last_peered":"2024-06-28T16:12:15.278572+0000","last_clean":"2024-06-28T16:12:15.278572+0000","last_became_active":"2024-06-28T15:37:33.349137+0000","last_became_peered":"2024-06-28T15:37:33.349137+0000","last_unstale":"2024-06-28T16:12:15.278572+0000","last_undegraded":"2024-06-28T16:12:15.278572+0000","last_fullsized":"2024-06-28T16:12:15.278572+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:47:15.998763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"260'6","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.514776+0000","last_change":"2024-06-28T15:21:11.106004+0000","last_active":"2024-06-28T16:14:32.514776+0000","last_peered":"2024-06-28T16:14:32.514776+0000","last_clean":"2024-06-28T16:14:32.514776+0000","last_became_active":"2024-06-28T15:21:11.104270+0000","last_became_peered":"2024-06-28T15:21:11.104270+0000","last_unstale":"2024-06-28T16:14:32.514776+0000","last_undegraded":"2024-06-28T16:14:32.514776+0000","last_fullsized":"2024-06-28T16:14:32.514776+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:22:50.569140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":11,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"260'6","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.311951+0000","last_change":"2024-06-28T15:21:07.049332+0000","last_active":"2024-06-28T16:14:32.311951+0000","last_peered":"2024-06-28T16:14:32.311951+0000","last_clean":"2024-06-28T16:14:32.311951+0000","last_became_active":"2024-06-28T15:21:07.048639+0000","last_became_peered":"2024-06-28T15:21:07.048639+0000","last_unstale":"2024-06-28T16:14:32.311951+0000","last_undegraded":"2024-06-28T16:14:32.311951+0000","last_fullsized":"2024-06-28T16:14:32.311951+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:02:29.473659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":12288,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"260'22","reported_seq":114,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.083766+0000","last_change":"2024-06-28T15:17:24.007520+0000","last_active":"2024-06-28T16:11:35.083766+0000","last_peered":"2024-06-28T16:11:35.083766+0000","last_clean":"2024-06-28T16:11:35.083766+0000","last_became_active":"2024-06-28T15:17:24.006784+0000","last_became_peered":"2024-06-28T15:17:24.006784+0000","last_unstale":"2024-06-28T16:11:35.083766+0000","last_undegraded":"2024-06-28T16:11:35.083766+0000","last_fullsized":"2024-06-28T16:11:35.083766+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:31:01.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":64,"num_write":51,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.574119+0000","last_change":"2024-06-28T15:37:33.334905+0000","last_active":"2024-06-28T16:12:15.574119+0000","last_peered":"2024-06-28T16:12:15.574119+0000","last_clean":"2024-06-28T16:12:15.574119+0000","last_became_active":"2024-06-28T15:37:33.333737+0000","last_became_peered":"2024-06-28T15:37:33.333737+0000","last_unstale":"2024-06-28T16:12:15.574119+0000","last_undegraded":"2024-06-28T16:12:15.574119+0000","last_fullsized":"2024-06-28T16:12:15.574119+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:02:20.700941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":48,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.081414+0000","last_change":"2024-06-28T15:21:11.092678+0000","last_active":"2024-06-28T16:11:25.081414+0000","last_peered":"2024-06-28T16:11:25.081414+0000","last_clean":"2024-06-28T16:11:25.081414+0000","last_became_active":"2024-06-28T15:21:11.092034+0000","last_became_peered":"2024-06-28T15:21:11.092034+0000","last_unstale":"2024-06-28T16:11:25.081414+0000","last_undegraded":"2024-06-28T16:11:25.081414+0000","last_fullsized":"2024-06-28T16:11:25.081414+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:08:58.423355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"260'3","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.120874+0000","last_change":"2024-06-28T15:21:07.040425+0000","last_active":"2024-06-28T16:13:15.120874+0000","last_peered":"2024-06-28T16:13:15.120874+0000","last_clean":"2024-06-28T16:13:15.120874+0000","last_became_active":"2024-06-28T15:21:07.039812+0000","last_became_peered":"2024-06-28T15:21:07.039812+0000","last_unstale":"2024-06-28T16:13:15.120874+0000","last_undegraded":"2024-06-28T16:13:15.120874+0000","last_fullsized":"2024-06-28T16:13:15.120874+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:50:26.224970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"260'77","reported_seq":307,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.090583+0000","last_change":"2024-06-28T15:17:24.013637+0000","last_active":"2024-06-28T16:11:35.090583+0000","last_peered":"2024-06-28T16:11:35.090583+0000","last_clean":"2024-06-28T16:11:35.090583+0000","last_became_active":"2024-06-28T15:17:24.012854+0000","last_became_peered":"2024-06-28T15:17:24.012854+0000","last_unstale":"2024-06-28T16:11:35.090583+0000","last_undegraded":"2024-06-28T16:11:35.090583+0000","last_fullsized":"2024-06-28T16:11:35.090583+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:10:40.982270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":483,"num_read_kb":330,"num_write":109,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.554385+0000","last_change":"2024-06-28T15:37:33.323089+0000","last_active":"2024-06-28T16:10:55.554385+0000","last_peered":"2024-06-28T16:10:55.554385+0000","last_clean":"2024-06-28T16:10:55.554385+0000","last_became_active":"2024-06-28T15:37:33.322317+0000","last_became_peered":"2024-06-28T15:37:33.322317+0000","last_unstale":"2024-06-28T16:10:55.554385+0000","last_undegraded":"2024-06-28T16:10:55.554385+0000","last_fullsized":"2024-06-28T16:10:55.554385+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:18:03.061734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.085392+0000","last_change":"2024-06-28T15:21:11.109416+0000","last_active":"2024-06-28T16:11:15.085392+0000","last_peered":"2024-06-28T16:11:15.085392+0000","last_clean":"2024-06-28T16:11:15.085392+0000","last_became_active":"2024-06-28T15:21:11.108663+0000","last_became_peered":"2024-06-28T15:21:11.108663+0000","last_unstale":"2024-06-28T16:11:15.085392+0000","last_undegraded":"2024-06-28T16:11:15.085392+0000","last_fullsized":"2024-06-28T16:11:15.085392+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:07:02.854303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"260'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.121849+0000","last_change":"2024-06-28T15:21:07.022794+0000","last_active":"2024-06-28T16:13:15.121849+0000","last_peered":"2024-06-28T16:13:15.121849+0000","last_clean":"2024-06-28T16:13:15.121849+0000","last_became_active":"2024-06-28T15:21:07.022046+0000","last_became_peered":"2024-06-28T15:21:07.022046+0000","last_unstale":"2024-06-28T16:13:15.121849+0000","last_undegraded":"2024-06-28T16:13:15.121849+0000","last_fullsized":"2024-06-28T16:13:15.121849+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:49:32.800553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"260'31","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089684+0000","last_change":"2024-06-28T15:17:24.011733+0000","last_active":"2024-06-28T16:11:35.089684+0000","last_peered":"2024-06-28T16:11:35.089684+0000","last_clean":"2024-06-28T16:11:35.089684+0000","last_became_active":"2024-06-28T15:17:24.010873+0000","last_became_peered":"2024-06-28T15:17:24.010873+0000","last_unstale":"2024-06-28T16:11:35.089684+0000","last_undegraded":"2024-06-28T16:11:35.089684+0000","last_fullsized":"2024-06-28T16:11:35.089684+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:29:45.592863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":94,"num_write":78,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.091280+0000","last_change":"2024-06-28T15:37:33.347898+0000","last_active":"2024-06-28T16:11:05.091280+0000","last_peered":"2024-06-28T16:11:05.091280+0000","last_clean":"2024-06-28T16:11:05.091280+0000","last_became_active":"2024-06-28T15:37:33.347075+0000","last_became_peered":"2024-06-28T15:37:33.347075+0000","last_unstale":"2024-06-28T16:11:05.091280+0000","last_undegraded":"2024-06-28T16:11:05.091280+0000","last_fullsized":"2024-06-28T16:11:05.091280+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:36:09.889868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":47,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.086272+0000","last_change":"2024-06-28T15:21:11.093738+0000","last_active":"2024-06-28T16:11:15.086272+0000","last_peered":"2024-06-28T16:11:15.086272+0000","last_clean":"2024-06-28T16:11:15.086272+0000","last_became_active":"2024-06-28T15:21:11.093082+0000","last_became_peered":"2024-06-28T15:21:11.093082+0000","last_unstale":"2024-06-28T16:11:15.086272+0000","last_undegraded":"2024-06-28T16:11:15.086272+0000","last_fullsized":"2024-06-28T16:11:15.086272+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:58:56.828729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"257'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.084078+0000","last_change":"2024-06-28T15:21:07.041597+0000","last_active":"2024-06-28T16:11:10.084078+0000","last_peered":"2024-06-28T16:11:10.084078+0000","last_clean":"2024-06-28T16:11:10.084078+0000","last_became_active":"2024-06-28T15:21:07.040575+0000","last_became_peered":"2024-06-28T15:21:07.040575+0000","last_unstale":"2024-06-28T16:11:10.084078+0000","last_undegraded":"2024-06-28T16:11:10.084078+0000","last_fullsized":"2024-06-28T16:11:10.084078+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:41:33.611919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"260'16","reported_seq":98,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:40.099162+0000","last_change":"2024-06-28T15:17:24.024775+0000","last_active":"2024-06-28T16:11:40.099162+0000","last_peered":"2024-06-28T16:11:40.099162+0000","last_clean":"2024-06-28T16:11:40.099162+0000","last_became_active":"2024-06-28T15:17:24.021464+0000","last_became_peered":"2024-06-28T15:17:24.021464+0000","last_unstale":"2024-06-28T16:11:40.099162+0000","last_undegraded":"2024-06-28T16:11:40.099162+0000","last_fullsized":"2024-06-28T16:11:40.099162+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:33:59.157965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":44,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"260'341","reported_seq":962,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.645969+0000","last_change":"2024-06-28T15:00:59.944074+0000","last_active":"2024-06-28T16:14:34.645969+0000","last_peered":"2024-06-28T16:14:34.645969+0000","last_clean":"2024-06-28T16:14:34.645969+0000","last_became_active":"2024-06-28T15:00:59.941681+0000","last_became_peered":"2024-06-28T15:00:59.941681+0000","last_unstale":"2024-06-28T16:14:34.645969+0000","last_undegraded":"2024-06-28T16:14:34.645969+0000","last_fullsized":"2024-06-28T16:14:34.645969+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:10:45.311266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170282,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":488,"num_read_kb":26898,"num_write":1268,"num_write_kb":25985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0,5],"acting":[2,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"260'181","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.066576+0000","last_change":"2024-06-28T15:00:17.405026+0000","last_active":"2024-06-28T16:10:20.066576+0000","last_peered":"2024-06-28T16:10:20.066576+0000","last_clean":"2024-06-28T16:10:20.066576+0000","last_became_active":"2024-06-28T15:00:17.404342+0000","last_became_peered":"2024-06-28T15:00:17.404342+0000","last_unstale":"2024-06-28T16:10:20.066576+0000","last_undegraded":"2024-06-28T16:10:20.066576+0000","last_fullsized":"2024-06-28T16:10:20.066576+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":100,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:24:42.574899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":722,"num_read_kb":569,"num_write":267,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"260'326","reported_seq":971,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.742875+0000","last_change":"2024-06-28T15:00:15.419867+0000","last_active":"2024-06-28T16:14:34.742875+0000","last_peered":"2024-06-28T16:14:34.742875+0000","last_clean":"2024-06-28T16:14:34.742875+0000","last_became_active":"2024-06-28T15:00:15.419449+0000","last_became_peered":"2024-06-28T15:00:15.419449+0000","last_unstale":"2024-06-28T16:14:34.742875+0000","last_undegraded":"2024-06-28T16:14:34.742875+0000","last_fullsized":"2024-06-28T16:14:34.742875+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:27:48.933532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5526738,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":485,"num_read_kb":15789,"num_write":1172,"num_write_kb":37304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"260'285","reported_seq":939,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080422+0000","last_change":"2024-06-28T15:00:17.409108+0000","last_active":"2024-06-28T16:10:20.080422+0000","last_peered":"2024-06-28T16:10:20.080422+0000","last_clean":"2024-06-28T16:10:20.080422+0000","last_became_active":"2024-06-28T15:00:17.408128+0000","last_became_peered":"2024-06-28T15:00:17.408128+0000","last_unstale":"2024-06-28T16:10:20.080422+0000","last_undegraded":"2024-06-28T16:10:20.080422+0000","last_fullsized":"2024-06-28T16:10:20.080422+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:38:12.539083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1157,"num_read_kb":911,"num_write":439,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"260'221","reported_seq":907,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.261930+0000","last_change":"2024-06-28T15:00:15.397113+0000","last_active":"2024-06-28T16:11:55.261930+0000","last_peered":"2024-06-28T16:11:55.261930+0000","last_clean":"2024-06-28T16:11:55.261930+0000","last_became_active":"2024-06-28T15:00:15.396469+0000","last_became_peered":"2024-06-28T15:00:15.396469+0000","last_unstale":"2024-06-28T16:11:55.261930+0000","last_undegraded":"2024-06-28T16:11:55.261930+0000","last_fullsized":"2024-06-28T16:11:55.261930+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:40:36.549494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1588322,"num_objects":66,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":442,"num_read_kb":81435,"num_write":1047,"num_write_kb":14924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"260'77","reported_seq":643,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073900+0000","last_change":"2024-06-28T15:00:17.403326+0000","last_active":"2024-06-28T16:10:20.073900+0000","last_peered":"2024-06-28T16:10:20.073900+0000","last_clean":"2024-06-28T16:10:20.073900+0000","last_became_active":"2024-06-28T15:00:17.402542+0000","last_became_peered":"2024-06-28T15:00:17.402542+0000","last_unstale":"2024-06-28T16:10:20.073900+0000","last_undegraded":"2024-06-28T16:10:20.073900+0000","last_fullsized":"2024-06-28T16:10:20.073900+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:05:41.791423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":306,"num_read_kb":248,"num_write":126,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"260'313","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.510107+0000","last_change":"2024-06-28T15:00:15.435754+0000","last_active":"2024-06-28T16:14:34.510107+0000","last_peered":"2024-06-28T16:14:34.510107+0000","last_clean":"2024-06-28T16:14:34.510107+0000","last_became_active":"2024-06-28T15:00:15.435122+0000","last_became_peered":"2024-06-28T15:00:15.435122+0000","last_unstale":"2024-06-28T16:14:34.510107+0000","last_undegraded":"2024-06-28T16:14:34.510107+0000","last_fullsized":"2024-06-28T16:14:34.510107+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:23:37.223417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1454317,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":492,"num_read_kb":48678,"num_write":1275,"num_write_kb":34259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"260'51","reported_seq":629,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.546829+0000","last_change":"2024-06-28T15:00:17.385309+0000","last_active":"2024-06-28T16:10:20.546829+0000","last_peered":"2024-06-28T16:10:20.546829+0000","last_clean":"2024-06-28T16:10:20.546829+0000","last_became_active":"2024-06-28T15:00:17.384932+0000","last_became_peered":"2024-06-28T15:00:17.384932+0000","last_unstale":"2024-06-28T16:10:20.546829+0000","last_undegraded":"2024-06-28T16:10:20.546829+0000","last_fullsized":"2024-06-28T16:10:20.546829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:12:11.868213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":197,"num_write":67,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"260'343","reported_seq":992,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.679630+0000","last_change":"2024-06-28T15:00:15.430784+0000","last_active":"2024-06-28T16:14:34.679630+0000","last_peered":"2024-06-28T16:14:34.679630+0000","last_clean":"2024-06-28T16:14:34.679630+0000","last_became_active":"2024-06-28T15:00:15.430208+0000","last_became_peered":"2024-06-28T15:00:15.430208+0000","last_unstale":"2024-06-28T16:14:34.679630+0000","last_undegraded":"2024-06-28T16:14:34.679630+0000","last_fullsized":"2024-06-28T16:14:34.679630+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:48:53.859374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437780,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":548,"num_read_kb":39482,"num_write":1215,"num_write_kb":35176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"260'185","reported_seq":837,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:59.271179+0000","last_change":"2024-06-28T15:00:17.398844+0000","last_active":"2024-06-28T16:10:59.271179+0000","last_peered":"2024-06-28T16:10:59.271179+0000","last_clean":"2024-06-28T16:10:59.271179+0000","last_became_active":"2024-06-28T15:00:17.397986+0000","last_became_peered":"2024-06-28T15:00:17.397986+0000","last_unstale":"2024-06-28T16:10:59.271179+0000","last_undegraded":"2024-06-28T16:10:59.271179+0000","last_fullsized":"2024-06-28T16:10:59.271179+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:49:08.729457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":777,"num_read_kb":632,"num_write":274,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"260'364","reported_seq":1029,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.692915+0000","last_change":"2024-06-28T15:00:15.412175+0000","last_active":"2024-06-28T16:14:34.692915+0000","last_peered":"2024-06-28T16:14:34.692915+0000","last_clean":"2024-06-28T16:14:34.692915+0000","last_became_active":"2024-06-28T15:00:15.409542+0000","last_became_peered":"2024-06-28T15:00:15.409542+0000","last_unstale":"2024-06-28T16:14:34.692915+0000","last_undegraded":"2024-06-28T16:14:34.692915+0000","last_fullsized":"2024-06-28T16:14:34.692915+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:28:00.331354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1667748,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":553,"num_read_kb":16754,"num_write":1370,"num_write_kb":32794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"260'44","reported_seq":608,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.895392+0000","last_change":"2024-06-28T15:00:17.408359+0000","last_active":"2024-06-28T16:11:10.895392+0000","last_peered":"2024-06-28T16:11:10.895392+0000","last_clean":"2024-06-28T16:11:10.895392+0000","last_became_active":"2024-06-28T15:00:17.401985+0000","last_became_peered":"2024-06-28T15:00:17.401985+0000","last_unstale":"2024-06-28T16:11:10.895392+0000","last_undegraded":"2024-06-28T16:11:10.895392+0000","last_fullsized":"2024-06-28T16:11:10.895392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:00:57.912525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":159,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"260'328","reported_seq":959,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.675966+0000","last_change":"2024-06-28T15:00:15.425314+0000","last_active":"2024-06-28T16:14:34.675966+0000","last_peered":"2024-06-28T16:14:34.675966+0000","last_clean":"2024-06-28T16:14:34.675966+0000","last_became_active":"2024-06-28T15:00:15.424430+0000","last_became_peered":"2024-06-28T15:00:15.424430+0000","last_unstale":"2024-06-28T16:14:34.675966+0000","last_undegraded":"2024-06-28T16:14:34.675966+0000","last_fullsized":"2024-06-28T16:14:34.675966+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:30:59.580102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048602,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":476,"num_read_kb":21908,"num_write":1069,"num_write_kb":36265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"260'4326","reported_seq":4996,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:30.604647+0000","last_change":"2024-06-28T15:00:17.387801+0000","last_active":"2024-06-28T16:12:30.604647+0000","last_peered":"2024-06-28T16:12:30.604647+0000","last_clean":"2024-06-28T16:12:30.604647+0000","last_became_active":"2024-06-28T15:00:17.384378+0000","last_became_peered":"2024-06-28T15:00:17.384378+0000","last_unstale":"2024-06-28T16:12:30.604647+0000","last_undegraded":"2024-06-28T16:12:30.604647+0000","last_fullsized":"2024-06-28T16:12:30.604647+0000","mapping_epoch":21,"log_start":"260'4300","ondisk_log_start":"260'4300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":3000,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:59:21.316239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15352,"num_read_kb":12408,"num_write":6557,"num_write_kb":3277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"260'411","reported_seq":1128,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.664061+0000","last_change":"2024-06-28T15:00:15.403350+0000","last_active":"2024-06-28T16:14:34.664061+0000","last_peered":"2024-06-28T16:14:34.664061+0000","last_clean":"2024-06-28T16:14:34.664061+0000","last_became_active":"2024-06-28T15:00:15.401960+0000","last_became_peered":"2024-06-28T15:00:15.401960+0000","last_unstale":"2024-06-28T16:14:34.664061+0000","last_undegraded":"2024-06-28T16:14:34.664061+0000","last_fullsized":"2024-06-28T16:14:34.664061+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":400,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:02:13.643574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324665,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":810,"num_read_kb":20615,"num_write":1466,"num_write_kb":29750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"260'70","reported_seq":685,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.247513+0000","last_change":"2024-06-28T15:00:17.381546+0000","last_active":"2024-06-28T16:14:08.247513+0000","last_peered":"2024-06-28T16:14:08.247513+0000","last_clean":"2024-06-28T16:14:08.247513+0000","last_became_active":"2024-06-28T15:00:17.381196+0000","last_became_peered":"2024-06-28T15:00:17.381196+0000","last_unstale":"2024-06-28T16:14:08.247513+0000","last_undegraded":"2024-06-28T16:14:08.247513+0000","last_fullsized":"2024-06-28T16:14:08.247513+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:47:51.139728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":309,"num_read_kb":273,"num_write":88,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"260'128","reported_seq":750,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:32.095965+0000","last_change":"2024-06-28T15:00:17.396995+0000","last_active":"2024-06-28T16:13:32.095965+0000","last_peered":"2024-06-28T16:13:32.095965+0000","last_clean":"2024-06-28T16:13:32.095965+0000","last_became_active":"2024-06-28T15:00:17.396142+0000","last_became_peered":"2024-06-28T15:00:17.396142+0000","last_unstale":"2024-06-28T16:13:32.095965+0000","last_undegraded":"2024-06-28T16:13:32.095965+0000","last_fullsized":"2024-06-28T16:13:32.095965+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:43:00.829791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":536,"num_read_kb":452,"num_write":191,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"260'416","reported_seq":1186,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.679447+0000","last_change":"2024-06-28T15:00:15.423248+0000","last_active":"2024-06-28T16:14:34.679447+0000","last_peered":"2024-06-28T16:14:34.679447+0000","last_clean":"2024-06-28T16:14:34.679447+0000","last_became_active":"2024-06-28T15:00:15.422579+0000","last_became_peered":"2024-06-28T15:00:15.422579+0000","last_unstale":"2024-06-28T16:14:34.679447+0000","last_undegraded":"2024-06-28T16:14:34.679447+0000","last_fullsized":"2024-06-28T16:14:34.679447+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":400,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:59:30.311378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":851,"num_read_kb":96606,"num_write":1464,"num_write_kb":38692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0,1],"acting":[7,5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"260'254","reported_seq":899,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080457+0000","last_change":"2024-06-28T15:00:17.410347+0000","last_active":"2024-06-28T16:10:20.080457+0000","last_peered":"2024-06-28T16:10:20.080457+0000","last_clean":"2024-06-28T16:10:20.080457+0000","last_became_active":"2024-06-28T15:00:17.409202+0000","last_became_peered":"2024-06-28T15:00:17.409202+0000","last_unstale":"2024-06-28T16:10:20.080457+0000","last_undegraded":"2024-06-28T16:10:20.080457+0000","last_fullsized":"2024-06-28T16:10:20.080457+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:33:21.410367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1059,"num_read_kb":851,"num_write":432,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"260'313","reported_seq":952,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.711448+0000","last_change":"2024-06-28T15:00:15.423492+0000","last_active":"2024-06-28T16:14:34.711448+0000","last_peered":"2024-06-28T16:14:34.711448+0000","last_clean":"2024-06-28T16:14:34.711448+0000","last_became_active":"2024-06-28T15:00:15.421980+0000","last_became_peered":"2024-06-28T15:00:15.421980+0000","last_unstale":"2024-06-28T16:14:34.711448+0000","last_undegraded":"2024-06-28T16:14:34.711448+0000","last_fullsized":"2024-06-28T16:14:34.711448+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:15:16.139016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468838,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":425,"num_read_kb":33937,"num_write":1026,"num_write_kb":43346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"260'225","reported_seq":850,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.070101+0000","last_change":"2024-06-28T15:00:17.385629+0000","last_active":"2024-06-28T16:10:35.070101+0000","last_peered":"2024-06-28T16:10:35.070101+0000","last_clean":"2024-06-28T16:10:35.070101+0000","last_became_active":"2024-06-28T15:00:17.384914+0000","last_became_peered":"2024-06-28T15:00:17.384914+0000","last_unstale":"2024-06-28T16:10:35.070101+0000","last_undegraded":"2024-06-28T16:10:35.070101+0000","last_fullsized":"2024-06-28T16:10:35.070101+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:01:45.177989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":887,"num_read_kb":706,"num_write":342,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"260'342","reported_seq":1191,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.723544+0000","last_change":"2024-06-28T15:00:15.412640+0000","last_active":"2024-06-28T16:14:34.723544+0000","last_peered":"2024-06-28T16:14:34.723544+0000","last_clean":"2024-06-28T16:14:34.723544+0000","last_became_active":"2024-06-28T15:00:15.411567+0000","last_became_peered":"2024-06-28T15:00:15.411567+0000","last_unstale":"2024-06-28T16:14:34.723544+0000","last_undegraded":"2024-06-28T16:14:34.723544+0000","last_fullsized":"2024-06-28T16:14:34.723544+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:32:11.605469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289237,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":936,"num_read_kb":14303,"num_write":1297,"num_write_kb":46873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"260'128","reported_seq":736,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:51.375718+0000","last_change":"2024-06-28T15:00:17.389926+0000","last_active":"2024-06-28T16:11:51.375718+0000","last_peered":"2024-06-28T16:11:51.375718+0000","last_clean":"2024-06-28T16:11:51.375718+0000","last_became_active":"2024-06-28T15:00:17.389613+0000","last_became_peered":"2024-06-28T15:00:17.389613+0000","last_unstale":"2024-06-28T16:11:51.375718+0000","last_undegraded":"2024-06-28T16:11:51.375718+0000","last_fullsized":"2024-06-28T16:11:51.375718+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:46:13.479655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":419,"num_write":192,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"260'320","reported_seq":1072,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.719218+0000","last_change":"2024-06-28T15:00:15.418207+0000","last_active":"2024-06-28T16:14:34.719218+0000","last_peered":"2024-06-28T16:14:34.719218+0000","last_clean":"2024-06-28T16:14:34.719218+0000","last_became_active":"2024-06-28T15:00:15.417563+0000","last_became_peered":"2024-06-28T15:00:15.417563+0000","last_unstale":"2024-06-28T16:14:34.719218+0000","last_undegraded":"2024-06-28T16:14:34.719218+0000","last_fullsized":"2024-06-28T16:14:34.719218+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:50:35.799169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":525349,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":605,"num_read_kb":12860,"num_write":1159,"num_write_kb":18563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"260'156","reported_seq":737,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.256777+0000","last_change":"2024-06-28T15:00:17.413709+0000","last_active":"2024-06-28T16:10:25.256777+0000","last_peered":"2024-06-28T16:10:25.256777+0000","last_clean":"2024-06-28T16:10:25.256777+0000","last_became_active":"2024-06-28T15:00:17.413420+0000","last_became_peered":"2024-06-28T15:00:17.413420+0000","last_unstale":"2024-06-28T16:10:25.256777+0000","last_undegraded":"2024-06-28T16:10:25.256777+0000","last_fullsized":"2024-06-28T16:10:25.256777+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:10:02.399082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":589,"num_read_kb":466,"num_write":222,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"260'357","reported_seq":1073,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.554726+0000","last_change":"2024-06-28T15:00:15.402501+0000","last_active":"2024-06-28T16:14:34.554726+0000","last_peered":"2024-06-28T16:14:34.554726+0000","last_clean":"2024-06-28T16:14:34.554726+0000","last_became_active":"2024-06-28T15:00:15.401370+0000","last_became_peered":"2024-06-28T15:00:15.401370+0000","last_unstale":"2024-06-28T16:14:34.554726+0000","last_undegraded":"2024-06-28T16:14:34.554726+0000","last_fullsized":"2024-06-28T16:14:34.554726+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:40:25.381283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324689,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":676,"num_read_kb":30262,"num_write":1536,"num_write_kb":36456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"260'253","reported_seq":902,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.053742+0000","last_change":"2024-06-28T15:00:17.410230+0000","last_active":"2024-06-28T16:13:20.053742+0000","last_peered":"2024-06-28T16:13:20.053742+0000","last_clean":"2024-06-28T16:13:20.053742+0000","last_became_active":"2024-06-28T15:00:17.408972+0000","last_became_peered":"2024-06-28T15:00:17.408972+0000","last_unstale":"2024-06-28T16:13:20.053742+0000","last_undegraded":"2024-06-28T16:13:20.053742+0000","last_fullsized":"2024-06-28T16:13:20.053742+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:48:01.967992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":990,"num_read_kb":787,"num_write":391,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"260'286","reported_seq":948,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.590888+0000","last_change":"2024-06-28T15:00:15.408546+0000","last_active":"2024-06-28T16:13:10.590888+0000","last_peered":"2024-06-28T16:13:10.590888+0000","last_clean":"2024-06-28T16:13:10.590888+0000","last_became_active":"2024-06-28T15:00:15.407552+0000","last_became_peered":"2024-06-28T15:00:15.407552+0000","last_unstale":"2024-06-28T16:13:10.590888+0000","last_undegraded":"2024-06-28T16:13:10.590888+0000","last_fullsized":"2024-06-28T16:13:10.590888+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:35:44.908220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12818085,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":453,"num_read_kb":23448,"num_write":1276,"num_write_kb":46171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"260'142","reported_seq":679,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.510437+0000","last_change":"2024-06-28T15:00:25.428551+0000","last_active":"2024-06-28T16:14:08.510437+0000","last_peered":"2024-06-28T16:14:08.510437+0000","last_clean":"2024-06-28T16:14:08.510437+0000","last_became_active":"2024-06-28T15:00:25.427734+0000","last_became_peered":"2024-06-28T15:00:25.427734+0000","last_unstale":"2024-06-28T16:14:08.510437+0000","last_undegraded":"2024-06-28T16:14:08.510437+0000","last_fullsized":"2024-06-28T16:14:08.510437+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:59:38.479139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":233,"num_read_kb":201,"num_write":347,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"260'214","reported_seq":822,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493503+0000","last_change":"2024-06-28T15:00:17.382620+0000","last_active":"2024-06-28T16:10:25.493503+0000","last_peered":"2024-06-28T16:10:25.493503+0000","last_clean":"2024-06-28T16:10:25.493503+0000","last_became_active":"2024-06-28T15:00:17.382344+0000","last_became_peered":"2024-06-28T15:00:17.382344+0000","last_unstale":"2024-06-28T16:10:25.493503+0000","last_undegraded":"2024-06-28T16:10:25.493503+0000","last_fullsized":"2024-06-28T16:10:25.493503+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":200,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:40:38.570590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":869,"num_read_kb":692,"num_write":351,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"260'292","reported_seq":944,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.632189+0000","last_change":"2024-06-28T15:00:15.423784+0000","last_active":"2024-06-28T16:14:34.632189+0000","last_peered":"2024-06-28T16:14:34.632189+0000","last_clean":"2024-06-28T16:14:34.632189+0000","last_became_active":"2024-06-28T15:00:15.420739+0000","last_became_peered":"2024-06-28T15:00:15.420739+0000","last_unstale":"2024-06-28T16:14:34.632189+0000","last_undegraded":"2024-06-28T16:14:34.632189+0000","last_fullsized":"2024-06-28T16:14:34.632189+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":200,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:38:08.814015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6022775,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":452,"num_read_kb":28797,"num_write":1102,"num_write_kb":30225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.067917+0000","last_change":"2024-06-28T15:00:19.403941+0000","last_active":"2024-06-28T16:10:25.067917+0000","last_peered":"2024-06-28T16:10:25.067917+0000","last_clean":"2024-06-28T16:10:25.067917+0000","last_became_active":"2024-06-28T15:00:19.400681+0000","last_became_peered":"2024-06-28T15:00:19.400681+0000","last_unstale":"2024-06-28T16:10:25.067917+0000","last_undegraded":"2024-06-28T16:10:25.067917+0000","last_fullsized":"2024-06-28T16:10:25.067917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:52:42.469346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"260'319","reported_seq":938,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.584308+0000","last_change":"2024-06-28T15:00:21.391364+0000","last_active":"2024-06-28T16:12:55.584308+0000","last_peered":"2024-06-28T16:12:55.584308+0000","last_clean":"2024-06-28T16:12:55.584308+0000","last_became_active":"2024-06-28T15:00:21.390609+0000","last_became_peered":"2024-06-28T15:00:21.390609+0000","last_unstale":"2024-06-28T16:12:55.584308+0000","last_undegraded":"2024-06-28T16:12:55.584308+0000","last_fullsized":"2024-06-28T16:12:55.584308+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:41:58.645414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48141,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":517,"num_read_kb":976,"num_write":400,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":493,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.082982+0000","last_change":"2024-06-28T15:00:23.408294+0000","last_active":"2024-06-28T16:10:30.082982+0000","last_peered":"2024-06-28T16:10:30.082982+0000","last_clean":"2024-06-28T16:10:30.082982+0000","last_became_active":"2024-06-28T15:00:23.407521+0000","last_became_peered":"2024-06-28T15:00:23.407521+0000","last_unstale":"2024-06-28T16:10:30.082982+0000","last_undegraded":"2024-06-28T16:10:30.082982+0000","last_fullsized":"2024-06-28T16:10:30.082982+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:06.922239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"260'2402","reported_seq":4268,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:02.134637+0000","last_change":"2024-06-28T15:00:25.440728+0000","last_active":"2024-06-28T16:14:02.134637+0000","last_peered":"2024-06-28T16:14:02.134637+0000","last_clean":"2024-06-28T16:14:02.134637+0000","last_became_active":"2024-06-28T15:00:25.440083+0000","last_became_peered":"2024-06-28T15:00:25.440083+0000","last_unstale":"2024-06-28T16:14:02.134637+0000","last_undegraded":"2024-06-28T16:14:02.134637+0000","last_fullsized":"2024-06-28T16:14:02.134637+0000","mapping_epoch":29,"log_start":"260'2300","ondisk_log_start":"260'2300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":2300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:54:57.315079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5966,"num_read_kb":5973,"num_write":4800,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"260'4179","reported_seq":4823,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.825089+0000","last_change":"2024-06-28T15:00:17.390976+0000","last_active":"2024-06-28T16:11:03.825089+0000","last_peered":"2024-06-28T16:11:03.825089+0000","last_clean":"2024-06-28T16:11:03.825089+0000","last_became_active":"2024-06-28T15:00:17.390254+0000","last_became_peered":"2024-06-28T15:00:17.390254+0000","last_unstale":"2024-06-28T16:11:03.825089+0000","last_undegraded":"2024-06-28T16:11:03.825089+0000","last_fullsized":"2024-06-28T16:11:03.825089+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":3000,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:50:01.744884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14764,"num_read_kb":11968,"num_write":6274,"num_write_kb":3129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"260'269","reported_seq":937,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.545460+0000","last_change":"2024-06-28T15:00:15.397404+0000","last_active":"2024-06-28T16:10:15.545460+0000","last_peered":"2024-06-28T16:10:15.545460+0000","last_clean":"2024-06-28T16:10:15.545460+0000","last_became_active":"2024-06-28T15:00:15.396724+0000","last_became_peered":"2024-06-28T15:00:15.396724+0000","last_unstale":"2024-06-28T16:10:15.545460+0000","last_undegraded":"2024-06-28T16:10:15.545460+0000","last_fullsized":"2024-06-28T16:10:15.545460+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:42:39.929729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2418239,"num_objects":80,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":458,"num_read_kb":7737,"num_write":1161,"num_write_kb":8076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493541+0000","last_change":"2024-06-28T15:00:19.390766+0000","last_active":"2024-06-28T16:10:25.493541+0000","last_peered":"2024-06-28T16:10:25.493541+0000","last_clean":"2024-06-28T16:10:25.493541+0000","last_became_active":"2024-06-28T15:00:19.390100+0000","last_became_peered":"2024-06-28T15:00:19.390100+0000","last_unstale":"2024-06-28T16:10:25.493541+0000","last_undegraded":"2024-06-28T16:10:25.493541+0000","last_fullsized":"2024-06-28T16:10:25.493541+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:01:20.861131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"260'123","reported_seq":662,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.316301+0000","last_change":"2024-06-28T15:00:21.374873+0000","last_active":"2024-06-28T16:11:03.316301+0000","last_peered":"2024-06-28T16:11:03.316301+0000","last_clean":"2024-06-28T16:11:03.316301+0000","last_became_active":"2024-06-28T15:00:21.374284+0000","last_became_peered":"2024-06-28T15:00:21.374284+0000","last_unstale":"2024-06-28T16:11:03.316301+0000","last_undegraded":"2024-06-28T16:11:03.316301+0000","last_fullsized":"2024-06-28T16:11:03.316301+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:17:08.459695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":127,"num_read_kb":120,"num_write":90,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":495,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.076584+0000","last_change":"2024-06-28T15:00:23.399941+0000","last_active":"2024-06-28T16:10:30.076584+0000","last_peered":"2024-06-28T16:10:30.076584+0000","last_clean":"2024-06-28T16:10:30.076584+0000","last_became_active":"2024-06-28T15:00:23.399147+0000","last_became_peered":"2024-06-28T15:00:23.399147+0000","last_unstale":"2024-06-28T16:10:30.076584+0000","last_undegraded":"2024-06-28T16:10:30.076584+0000","last_fullsized":"2024-06-28T16:10:30.076584+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:00:35.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"260'123","reported_seq":649,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.145795+0000","last_change":"2024-06-28T15:00:25.433474+0000","last_active":"2024-06-28T16:14:06.145795+0000","last_peered":"2024-06-28T16:14:06.145795+0000","last_clean":"2024-06-28T16:14:06.145795+0000","last_became_active":"2024-06-28T15:00:25.432898+0000","last_became_peered":"2024-06-28T15:00:25.432898+0000","last_unstale":"2024-06-28T16:14:06.145795+0000","last_undegraded":"2024-06-28T16:14:06.145795+0000","last_fullsized":"2024-06-28T16:14:06.145795+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:46:01.703394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":165,"num_write":302,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"260'155","reported_seq":776,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.548260+0000","last_change":"2024-06-28T15:00:17.384953+0000","last_active":"2024-06-28T16:10:25.548260+0000","last_peered":"2024-06-28T16:10:25.548260+0000","last_clean":"2024-06-28T16:10:25.548260+0000","last_became_active":"2024-06-28T15:00:17.384613+0000","last_became_peered":"2024-06-28T15:00:17.384613+0000","last_unstale":"2024-06-28T16:10:25.548260+0000","last_undegraded":"2024-06-28T16:10:25.548260+0000","last_fullsized":"2024-06-28T16:10:25.548260+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:04:42.370422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":630,"num_read_kb":508,"num_write":240,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"260'353","reported_seq":1001,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.727043+0000","last_change":"2024-06-28T15:00:15.429200+0000","last_active":"2024-06-28T16:14:34.727043+0000","last_peered":"2024-06-28T16:14:34.727043+0000","last_clean":"2024-06-28T16:14:34.727043+0000","last_became_active":"2024-06-28T15:00:15.428415+0000","last_became_peered":"2024-06-28T15:00:15.428415+0000","last_unstale":"2024-06-28T16:14:34.727043+0000","last_undegraded":"2024-06-28T16:14:34.727043+0000","last_fullsized":"2024-06-28T16:14:34.727043+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:17:28.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16055067,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":496,"num_read_kb":15197,"num_write":1261,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493690+0000","last_change":"2024-06-28T15:00:19.373751+0000","last_active":"2024-06-28T16:10:25.493690+0000","last_peered":"2024-06-28T16:10:25.493690+0000","last_clean":"2024-06-28T16:10:25.493690+0000","last_became_active":"2024-06-28T15:00:19.372840+0000","last_became_peered":"2024-06-28T15:00:19.372840+0000","last_unstale":"2024-06-28T16:10:25.493690+0000","last_undegraded":"2024-06-28T16:10:25.493690+0000","last_fullsized":"2024-06-28T16:10:25.493690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:46:30.290232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"260'123","reported_seq":686,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.516922+0000","last_change":"2024-06-28T15:00:21.377962+0000","last_active":"2024-06-28T16:11:03.516922+0000","last_peered":"2024-06-28T16:11:03.516922+0000","last_clean":"2024-06-28T16:11:03.516922+0000","last_became_active":"2024-06-28T15:00:21.377203+0000","last_became_peered":"2024-06-28T15:00:21.377203+0000","last_unstale":"2024-06-28T16:11:03.516922+0000","last_undegraded":"2024-06-28T16:11:03.516922+0000","last_fullsized":"2024-06-28T16:11:03.516922+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:14:43.436846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"260'522","reported_seq":4627,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.061536+0000","last_change":"2024-06-28T15:00:23.418328+0000","last_active":"2024-06-28T16:14:34.061536+0000","last_peered":"2024-06-28T16:14:34.061536+0000","last_clean":"2024-06-28T16:14:34.061536+0000","last_became_active":"2024-06-28T15:00:23.417490+0000","last_became_peered":"2024-06-28T15:00:23.417490+0000","last_unstale":"2024-06-28T16:14:34.061536+0000","last_undegraded":"2024-06-28T16:14:34.061536+0000","last_fullsized":"2024-06-28T16:14:34.061536+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":500,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:22:03.212204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"260'201","reported_seq":1010,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.965594+0000","last_change":"2024-06-28T15:00:25.556315+0000","last_active":"2024-06-28T16:14:06.965594+0000","last_peered":"2024-06-28T16:14:06.965594+0000","last_clean":"2024-06-28T16:14:06.965594+0000","last_became_active":"2024-06-28T15:00:25.555670+0000","last_became_peered":"2024-06-28T15:00:25.555670+0000","last_unstale":"2024-06-28T16:14:06.965594+0000","last_undegraded":"2024-06-28T16:14:06.965594+0000","last_fullsized":"2024-06-28T16:14:06.965594+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:26:34.140679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":557,"num_read_kb":555,"num_write":448,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"260'84","reported_seq":671,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493663+0000","last_change":"2024-06-28T15:00:17.390503+0000","last_active":"2024-06-28T16:10:25.493663+0000","last_peered":"2024-06-28T16:10:25.493663+0000","last_clean":"2024-06-28T16:10:25.493663+0000","last_became_active":"2024-06-28T15:00:17.389986+0000","last_became_peered":"2024-06-28T15:00:17.389986+0000","last_unstale":"2024-06-28T16:10:25.493663+0000","last_undegraded":"2024-06-28T16:10:25.493663+0000","last_fullsized":"2024-06-28T16:10:25.493663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:30:46.132610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":369,"num_read_kb":305,"num_write":130,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"260'288","reported_seq":922,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.683233+0000","last_change":"2024-06-28T15:00:15.413174+0000","last_active":"2024-06-28T16:14:34.683233+0000","last_peered":"2024-06-28T16:14:34.683233+0000","last_clean":"2024-06-28T16:14:34.683233+0000","last_became_active":"2024-06-28T15:00:15.411380+0000","last_became_peered":"2024-06-28T15:00:15.411380+0000","last_unstale":"2024-06-28T16:14:34.683233+0000","last_undegraded":"2024-06-28T16:14:34.683233+0000","last_fullsized":"2024-06-28T16:14:34.683233+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:35.364682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262217,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":24351,"num_write":1142,"num_write_kb":34218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"31'3","reported_seq":830,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:31.936121+0000","last_change":"2024-06-28T15:00:19.404519+0000","last_active":"2024-06-28T16:14:31.936121+0000","last_peered":"2024-06-28T16:14:31.936121+0000","last_clean":"2024-06-28T16:14:31.936121+0000","last_became_active":"2024-06-28T15:00:19.403623+0000","last_became_peered":"2024-06-28T15:00:19.403623+0000","last_unstale":"2024-06-28T16:14:31.936121+0000","last_undegraded":"2024-06-28T16:14:31.936121+0000","last_fullsized":"2024-06-28T16:14:31.936121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:36:48.645241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":496,"num_read_kb":494,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"260'251","reported_seq":833,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.272334+0000","last_change":"2024-06-28T15:00:21.485248+0000","last_active":"2024-06-28T16:14:33.272334+0000","last_peered":"2024-06-28T16:14:33.272334+0000","last_clean":"2024-06-28T16:14:33.272334+0000","last_became_active":"2024-06-28T15:00:21.483633+0000","last_became_peered":"2024-06-28T15:00:21.483633+0000","last_unstale":"2024-06-28T16:14:33.272334+0000","last_undegraded":"2024-06-28T16:14:33.272334+0000","last_fullsized":"2024-06-28T16:14:33.272334+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:48:04.552220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8831,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":873,"num_write":310,"num_write_kb":224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"260'537","reported_seq":2239,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.259197+0000","last_change":"2024-06-28T15:00:23.397769+0000","last_active":"2024-06-28T16:14:34.259197+0000","last_peered":"2024-06-28T16:14:34.259197+0000","last_clean":"2024-06-28T16:14:34.259197+0000","last_became_active":"2024-06-28T15:00:23.396923+0000","last_became_peered":"2024-06-28T15:00:23.396923+0000","last_unstale":"2024-06-28T16:14:34.259197+0000","last_undegraded":"2024-06-28T16:14:34.259197+0000","last_fullsized":"2024-06-28T16:14:34.259197+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:32:30.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"260'415","reported_seq":1093,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:43.529396+0000","last_change":"2024-06-28T15:00:17.378608+0000","last_active":"2024-06-28T16:12:43.529396+0000","last_peered":"2024-06-28T16:12:43.529396+0000","last_clean":"2024-06-28T16:12:43.529396+0000","last_became_active":"2024-06-28T15:00:17.376864+0000","last_became_peered":"2024-06-28T15:00:17.376864+0000","last_unstale":"2024-06-28T16:12:43.529396+0000","last_undegraded":"2024-06-28T16:12:43.529396+0000","last_fullsized":"2024-06-28T16:12:43.529396+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":400,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:13.643893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1583,"num_read_kb":1214,"num_write":634,"num_write_kb":357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"260'320","reported_seq":979,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.732986+0000","last_change":"2024-06-28T15:00:15.423161+0000","last_active":"2024-06-28T16:14:34.732986+0000","last_peered":"2024-06-28T16:14:34.732986+0000","last_clean":"2024-06-28T16:14:34.732986+0000","last_became_active":"2024-06-28T15:00:15.415163+0000","last_became_peered":"2024-06-28T15:00:15.415163+0000","last_unstale":"2024-06-28T16:14:34.732986+0000","last_undegraded":"2024-06-28T16:14:34.732986+0000","last_fullsized":"2024-06-28T16:14:34.732986+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:10:26.502858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470118,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":518,"num_read_kb":26822,"num_write":1200,"num_write_kb":41274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.101361+0000","last_change":"2024-06-28T15:00:19.390703+0000","last_active":"2024-06-28T16:11:50.101361+0000","last_peered":"2024-06-28T16:11:50.101361+0000","last_clean":"2024-06-28T16:11:50.101361+0000","last_became_active":"2024-06-28T15:00:19.389624+0000","last_became_peered":"2024-06-28T15:00:19.389624+0000","last_unstale":"2024-06-28T16:11:50.101361+0000","last_undegraded":"2024-06-28T16:11:50.101361+0000","last_fullsized":"2024-06-28T16:11:50.101361+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:38:09.613294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"260'382","reported_seq":1026,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.950175+0000","last_change":"2024-06-28T15:00:21.483120+0000","last_active":"2024-06-28T16:14:33.950175+0000","last_peered":"2024-06-28T16:14:33.950175+0000","last_clean":"2024-06-28T16:14:33.950175+0000","last_became_active":"2024-06-28T15:00:21.481315+0000","last_became_peered":"2024-06-28T15:00:21.481315+0000","last_unstale":"2024-06-28T16:14:33.950175+0000","last_undegraded":"2024-06-28T16:14:33.950175+0000","last_fullsized":"2024-06-28T16:14:33.950175+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":300,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:09:01.037761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46076,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":638,"num_read_kb":1277,"num_write":490,"num_write_kb":298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.082062+0000","last_change":"2024-06-28T15:00:23.414142+0000","last_active":"2024-06-28T16:10:10.082062+0000","last_peered":"2024-06-28T16:10:10.082062+0000","last_clean":"2024-06-28T16:10:10.082062+0000","last_became_active":"2024-06-28T15:00:23.413132+0000","last_became_peered":"2024-06-28T15:00:23.413132+0000","last_unstale":"2024-06-28T16:10:10.082062+0000","last_undegraded":"2024-06-28T16:10:10.082062+0000","last_fullsized":"2024-06-28T16:10:10.082062+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:39.827537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"260'555","reported_seq":1373,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:13.000905+0000","last_change":"2024-06-28T15:00:25.442589+0000","last_active":"2024-06-28T16:14:13.000905+0000","last_peered":"2024-06-28T16:14:13.000905+0000","last_clean":"2024-06-28T16:14:13.000905+0000","last_became_active":"2024-06-28T15:00:25.441335+0000","last_became_peered":"2024-06-28T15:00:25.441335+0000","last_unstale":"2024-06-28T16:14:13.000905+0000","last_undegraded":"2024-06-28T16:14:13.000905+0000","last_fullsized":"2024-06-28T16:14:13.000905+0000","mapping_epoch":29,"log_start":"260'500","ondisk_log_start":"260'500","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":500,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:55.109636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1277,"num_read_kb":1273,"num_write":1164,"num_write_kb":394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"260'52","reported_seq":631,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.094197+0000","last_change":"2024-06-28T15:00:17.409603+0000","last_active":"2024-06-28T16:11:50.094197+0000","last_peered":"2024-06-28T16:11:50.094197+0000","last_clean":"2024-06-28T16:11:50.094197+0000","last_became_active":"2024-06-28T15:00:17.408992+0000","last_became_peered":"2024-06-28T15:00:17.408992+0000","last_unstale":"2024-06-28T16:11:50.094197+0000","last_undegraded":"2024-06-28T16:11:50.094197+0000","last_fullsized":"2024-06-28T16:11:50.094197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:22:21.223507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":226,"num_read_kb":196,"num_write":79,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"260'326","reported_seq":945,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.655730+0000","last_change":"2024-06-28T15:00:15.402904+0000","last_active":"2024-06-28T16:14:34.655730+0000","last_peered":"2024-06-28T16:14:34.655730+0000","last_clean":"2024-06-28T16:14:34.655730+0000","last_became_active":"2024-06-28T15:00:15.402188+0000","last_became_peered":"2024-06-28T15:00:15.402188+0000","last_unstale":"2024-06-28T16:14:34.655730+0000","last_undegraded":"2024-06-28T16:14:34.655730+0000","last_fullsized":"2024-06-28T16:14:34.655730+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:19:31.626172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170347,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":7037,"num_write":1119,"num_write_kb":28395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.273401+0000","last_change":"2024-06-28T15:00:19.383547+0000","last_active":"2024-06-28T16:11:50.273401+0000","last_peered":"2024-06-28T16:11:50.273401+0000","last_clean":"2024-06-28T16:11:50.273401+0000","last_became_active":"2024-06-28T15:00:19.383002+0000","last_became_peered":"2024-06-28T15:00:19.383002+0000","last_unstale":"2024-06-28T16:11:50.273401+0000","last_undegraded":"2024-06-28T16:11:50.273401+0000","last_fullsized":"2024-06-28T16:11:50.273401+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:52.946596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"260'288","reported_seq":878,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.505294+0000","last_change":"2024-06-28T15:00:21.395366+0000","last_active":"2024-06-28T16:14:32.505294+0000","last_peered":"2024-06-28T16:14:32.505294+0000","last_clean":"2024-06-28T16:14:32.505294+0000","last_became_active":"2024-06-28T15:00:21.394580+0000","last_became_peered":"2024-06-28T15:00:21.394580+0000","last_unstale":"2024-06-28T16:14:32.505294+0000","last_undegraded":"2024-06-28T16:14:32.505294+0000","last_fullsized":"2024-06-28T16:14:32.505294+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:06:06.648827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":455,"num_read_kb":1052,"num_write":364,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:00.515835+0000","last_change":"2024-06-28T15:00:23.423231+0000","last_active":"2024-06-28T16:12:00.515835+0000","last_peered":"2024-06-28T16:12:00.515835+0000","last_clean":"2024-06-28T16:12:00.515835+0000","last_became_active":"2024-06-28T15:00:23.422567+0000","last_became_peered":"2024-06-28T15:00:23.422567+0000","last_unstale":"2024-06-28T16:12:00.515835+0000","last_undegraded":"2024-06-28T16:12:00.515835+0000","last_fullsized":"2024-06-28T16:12:00.515835+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:40.356598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"260'96","reported_seq":589,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089808+0000","last_change":"2024-06-28T15:00:25.426829+0000","last_active":"2024-06-28T16:11:35.089808+0000","last_peered":"2024-06-28T16:11:35.089808+0000","last_clean":"2024-06-28T16:11:35.089808+0000","last_became_active":"2024-06-28T15:00:25.426151+0000","last_became_peered":"2024-06-28T15:00:25.426151+0000","last_unstale":"2024-06-28T16:11:35.089808+0000","last_undegraded":"2024-06-28T16:11:35.089808+0000","last_fullsized":"2024-06-28T16:11:35.089808+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:23:26.301565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":231,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"260'173","reported_seq":777,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:10.098092+0000","last_change":"2024-06-28T15:00:17.407655+0000","last_active":"2024-06-28T16:12:10.098092+0000","last_peered":"2024-06-28T16:12:10.098092+0000","last_clean":"2024-06-28T16:12:10.098092+0000","last_became_active":"2024-06-28T15:00:17.407031+0000","last_became_peered":"2024-06-28T15:00:17.407031+0000","last_unstale":"2024-06-28T16:12:10.098092+0000","last_undegraded":"2024-06-28T16:12:10.098092+0000","last_fullsized":"2024-06-28T16:12:10.098092+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:17:37.199564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":538,"num_write":274,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"260'353","reported_seq":1038,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.682331+0000","last_change":"2024-06-28T15:00:15.428903+0000","last_active":"2024-06-28T16:14:34.682331+0000","last_peered":"2024-06-28T16:14:34.682331+0000","last_clean":"2024-06-28T16:14:34.682331+0000","last_became_active":"2024-06-28T15:00:15.427408+0000","last_became_peered":"2024-06-28T15:00:15.427408+0000","last_unstale":"2024-06-28T16:14:34.682331+0000","last_undegraded":"2024-06-28T16:14:34.682331+0000","last_fullsized":"2024-06-28T16:14:34.682331+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:52:51.484685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":574,"num_read_kb":16939,"num_write":1376,"num_write_kb":32636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.279481+0000","last_change":"2024-06-28T15:00:19.384770+0000","last_active":"2024-06-28T16:12:20.279481+0000","last_peered":"2024-06-28T16:12:20.279481+0000","last_clean":"2024-06-28T16:12:20.279481+0000","last_became_active":"2024-06-28T15:00:19.384172+0000","last_became_peered":"2024-06-28T15:00:19.384172+0000","last_unstale":"2024-06-28T16:12:20.279481+0000","last_undegraded":"2024-06-28T16:12:20.279481+0000","last_fullsized":"2024-06-28T16:12:20.279481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:13:54.648899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"260'314","reported_seq":925,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.098634+0000","last_change":"2024-06-28T15:00:21.394527+0000","last_active":"2024-06-28T16:14:34.098634+0000","last_peered":"2024-06-28T16:14:34.098634+0000","last_clean":"2024-06-28T16:14:34.098634+0000","last_became_active":"2024-06-28T15:00:21.393608+0000","last_became_peered":"2024-06-28T15:00:21.393608+0000","last_unstale":"2024-06-28T16:14:34.098634+0000","last_undegraded":"2024-06-28T16:14:34.098634+0000","last_fullsized":"2024-06-28T16:14:34.098634+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:01:39.810509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42645,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":526,"num_read_kb":1122,"num_write":410,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"260'529","reported_seq":2261,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:14.487526+0000","last_change":"2024-06-28T15:00:23.414333+0000","last_active":"2024-06-28T16:14:14.487526+0000","last_peered":"2024-06-28T16:14:14.487526+0000","last_clean":"2024-06-28T16:14:14.487526+0000","last_became_active":"2024-06-28T15:00:23.413550+0000","last_became_peered":"2024-06-28T15:00:23.413550+0000","last_unstale":"2024-06-28T16:14:14.487526+0000","last_undegraded":"2024-06-28T16:14:14.487526+0000","last_fullsized":"2024-06-28T16:14:14.487526+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":500,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:23:18.123161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"260'144","reported_seq":672,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.569778+0000","last_change":"2024-06-28T15:00:25.432580+0000","last_active":"2024-06-28T16:11:55.569778+0000","last_peered":"2024-06-28T16:11:55.569778+0000","last_clean":"2024-06-28T16:11:55.569778+0000","last_became_active":"2024-06-28T15:00:25.431835+0000","last_became_peered":"2024-06-28T15:00:25.431835+0000","last_unstale":"2024-06-28T16:11:55.569778+0000","last_undegraded":"2024-06-28T16:11:55.569778+0000","last_fullsized":"2024-06-28T16:11:55.569778+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:25:08.052594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1247,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":164,"num_write":265,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"260'4158","reported_seq":4763,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.543014+0000","last_change":"2024-06-28T15:00:17.400121+0000","last_active":"2024-06-28T16:10:05.543014+0000","last_peered":"2024-06-28T16:10:05.543014+0000","last_clean":"2024-06-28T16:10:05.543014+0000","last_became_active":"2024-06-28T15:00:17.399799+0000","last_became_peered":"2024-06-28T15:00:17.399799+0000","last_unstale":"2024-06-28T16:10:05.543014+0000","last_undegraded":"2024-06-28T16:10:05.543014+0000","last_fullsized":"2024-06-28T16:10:05.543014+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":3000,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:10:23.005648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14678,"num_read_kb":11882,"num_write":6254,"num_write_kb":3121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"260'337","reported_seq":982,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.704930+0000","last_change":"2024-06-28T15:00:15.426615+0000","last_active":"2024-06-28T16:14:34.704930+0000","last_peered":"2024-06-28T16:14:34.704930+0000","last_clean":"2024-06-28T16:14:34.704930+0000","last_became_active":"2024-06-28T15:00:15.425690+0000","last_became_peered":"2024-06-28T15:00:15.425690+0000","last_unstale":"2024-06-28T16:14:34.704930+0000","last_undegraded":"2024-06-28T16:14:34.704930+0000","last_fullsized":"2024-06-28T16:14:34.704930+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:59:45.352009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079931,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":508,"num_read_kb":13664,"num_write":1274,"num_write_kb":26574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,1],"acting":[5,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:05.276635+0000","last_change":"2024-06-28T15:00:19.385556+0000","last_active":"2024-06-28T16:12:05.276635+0000","last_peered":"2024-06-28T16:12:05.276635+0000","last_clean":"2024-06-28T16:12:05.276635+0000","last_became_active":"2024-06-28T15:00:19.385189+0000","last_became_peered":"2024-06-28T15:00:19.385189+0000","last_unstale":"2024-06-28T16:12:05.276635+0000","last_undegraded":"2024-06-28T16:12:05.276635+0000","last_fullsized":"2024-06-28T16:12:05.276635+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:12:07.124133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"260'477","reported_seq":1123,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:12.918679+0000","last_change":"2024-06-28T15:00:21.400448+0000","last_active":"2024-06-28T16:14:12.918679+0000","last_peered":"2024-06-28T16:14:12.918679+0000","last_clean":"2024-06-28T16:14:12.918679+0000","last_became_active":"2024-06-28T15:00:21.397397+0000","last_became_peered":"2024-06-28T15:00:21.397397+0000","last_unstale":"2024-06-28T16:14:12.918679+0000","last_undegraded":"2024-06-28T16:14:12.918679+0000","last_fullsized":"2024-06-28T16:14:12.918679+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":400,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:54:53.893930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109658,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":830,"num_read_kb":1847,"num_write":686,"num_write_kb":570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.274612+0000","last_change":"2024-06-28T15:00:23.422892+0000","last_active":"2024-06-28T16:11:55.274612+0000","last_peered":"2024-06-28T16:11:55.274612+0000","last_clean":"2024-06-28T16:11:55.274612+0000","last_became_active":"2024-06-28T15:00:23.416263+0000","last_became_peered":"2024-06-28T15:00:23.416263+0000","last_unstale":"2024-06-28T16:11:55.274612+0000","last_undegraded":"2024-06-28T16:11:55.274612+0000","last_fullsized":"2024-06-28T16:11:55.274612+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:12:48.735241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"260'113","reported_seq":631,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.103804+0000","last_change":"2024-06-28T15:00:25.423425+0000","last_active":"2024-06-28T16:11:55.103804+0000","last_peered":"2024-06-28T16:11:55.103804+0000","last_clean":"2024-06-28T16:11:55.103804+0000","last_became_active":"2024-06-28T15:00:25.422519+0000","last_became_peered":"2024-06-28T15:00:25.422519+0000","last_unstale":"2024-06-28T16:11:55.103804+0000","last_undegraded":"2024-06-28T16:11:55.103804+0000","last_fullsized":"2024-06-28T16:11:55.103804+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:15:27.026404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":118,"num_read_kb":116,"num_write":248,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"260'128","reported_seq":777,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101386+0000","last_change":"2024-06-28T15:00:17.409415+0000","last_active":"2024-06-28T16:12:20.101386+0000","last_peered":"2024-06-28T16:12:20.101386+0000","last_clean":"2024-06-28T16:12:20.101386+0000","last_became_active":"2024-06-28T15:00:17.404204+0000","last_became_peered":"2024-06-28T15:00:17.404204+0000","last_unstale":"2024-06-28T16:12:20.101386+0000","last_undegraded":"2024-06-28T16:12:20.101386+0000","last_fullsized":"2024-06-28T16:12:20.101386+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:25:51.372637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":562,"num_read_kb":476,"num_write":174,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"260'352","reported_seq":1059,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.701530+0000","last_change":"2024-06-28T15:00:15.402917+0000","last_active":"2024-06-28T16:14:34.701530+0000","last_peered":"2024-06-28T16:14:34.701530+0000","last_clean":"2024-06-28T16:14:34.701530+0000","last_became_active":"2024-06-28T15:00:15.402041+0000","last_became_peered":"2024-06-28T15:00:15.402041+0000","last_unstale":"2024-06-28T16:14:34.701530+0000","last_undegraded":"2024-06-28T16:14:34.701530+0000","last_fullsized":"2024-06-28T16:14:34.701530+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:50:41.105836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":603,"num_read_kb":26410,"num_write":1369,"num_write_kb":26391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.279564+0000","last_change":"2024-06-28T15:00:19.393172+0000","last_active":"2024-06-28T16:12:20.279564+0000","last_peered":"2024-06-28T16:12:20.279564+0000","last_clean":"2024-06-28T16:12:20.279564+0000","last_became_active":"2024-06-28T15:00:19.392773+0000","last_became_peered":"2024-06-28T15:00:19.392773+0000","last_unstale":"2024-06-28T16:12:20.279564+0000","last_undegraded":"2024-06-28T16:12:20.279564+0000","last_fullsized":"2024-06-28T16:12:20.279564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:03:10.710059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"260'301","reported_seq":896,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.815148+0000","last_change":"2024-06-28T15:00:21.398582+0000","last_active":"2024-06-28T16:14:32.815148+0000","last_peered":"2024-06-28T16:14:32.815148+0000","last_clean":"2024-06-28T16:14:32.815148+0000","last_became_active":"2024-06-28T15:00:21.397842+0000","last_became_peered":"2024-06-28T15:00:21.397842+0000","last_unstale":"2024-06-28T16:14:32.815148+0000","last_undegraded":"2024-06-28T16:14:32.815148+0000","last_fullsized":"2024-06-28T16:14:32.815148+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:31:25.283406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44037,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":498,"num_read_kb":1063,"num_write":434,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.548349+0000","last_change":"2024-06-28T15:00:23.406901+0000","last_active":"2024-06-28T16:10:25.548349+0000","last_peered":"2024-06-28T16:10:25.548349+0000","last_clean":"2024-06-28T16:10:25.548349+0000","last_became_active":"2024-06-28T15:00:23.406122+0000","last_became_peered":"2024-06-28T15:00:23.406122+0000","last_unstale":"2024-06-28T16:10:25.548349+0000","last_undegraded":"2024-06-28T16:10:25.548349+0000","last_fullsized":"2024-06-28T16:10:25.548349+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:58:30.933744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"260'89","reported_seq":593,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:03.580217+0000","last_change":"2024-06-28T15:00:25.439968+0000","last_active":"2024-06-28T16:14:03.580217+0000","last_peered":"2024-06-28T16:14:03.580217+0000","last_clean":"2024-06-28T16:14:03.580217+0000","last_became_active":"2024-06-28T15:00:25.439346+0000","last_became_peered":"2024-06-28T15:00:25.439346+0000","last_unstale":"2024-06-28T16:14:03.580217+0000","last_undegraded":"2024-06-28T16:14:03.580217+0000","last_fullsized":"2024-06-28T16:14:03.580217+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:31:17.344077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":207,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"260'171","reported_seq":759,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101425+0000","last_change":"2024-06-28T15:00:17.396835+0000","last_active":"2024-06-28T16:12:20.101425+0000","last_peered":"2024-06-28T16:12:20.101425+0000","last_clean":"2024-06-28T16:12:20.101425+0000","last_became_active":"2024-06-28T15:00:17.396048+0000","last_became_peered":"2024-06-28T15:00:17.396048+0000","last_unstale":"2024-06-28T16:12:20.101425+0000","last_undegraded":"2024-06-28T16:12:20.101425+0000","last_fullsized":"2024-06-28T16:12:20.101425+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:20:25.818327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":659,"num_read_kb":531,"num_write":250,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"260'376","reported_seq":1107,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.727955+0000","last_change":"2024-06-28T15:00:15.398260+0000","last_active":"2024-06-28T16:14:34.727955+0000","last_peered":"2024-06-28T16:14:34.727955+0000","last_clean":"2024-06-28T16:14:34.727955+0000","last_became_active":"2024-06-28T15:00:15.397256+0000","last_became_peered":"2024-06-28T15:00:15.397256+0000","last_unstale":"2024-06-28T16:14:34.727955+0000","last_undegraded":"2024-06-28T16:14:34.727955+0000","last_fullsized":"2024-06-28T16:14:34.727955+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":300,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:08:01.511388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2674362,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":727,"num_read_kb":7345,"num_write":1214,"num_write_kb":31713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.109069+0000","last_change":"2024-06-28T15:00:19.379868+0000","last_active":"2024-06-28T16:12:20.109069+0000","last_peered":"2024-06-28T16:12:20.109069+0000","last_clean":"2024-06-28T16:12:20.109069+0000","last_became_active":"2024-06-28T15:00:19.379281+0000","last_became_peered":"2024-06-28T15:00:19.379281+0000","last_unstale":"2024-06-28T16:12:20.109069+0000","last_undegraded":"2024-06-28T16:12:20.109069+0000","last_fullsized":"2024-06-28T16:12:20.109069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:01:53.936356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"260'350","reported_seq":968,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.682992+0000","last_change":"2024-06-28T15:00:21.396750+0000","last_active":"2024-06-28T16:14:34.682992+0000","last_peered":"2024-06-28T16:14:34.682992+0000","last_clean":"2024-06-28T16:14:34.682992+0000","last_became_active":"2024-06-28T15:00:21.395892+0000","last_became_peered":"2024-06-28T15:00:21.395892+0000","last_unstale":"2024-06-28T16:14:34.682992+0000","last_undegraded":"2024-06-28T16:14:34.682992+0000","last_fullsized":"2024-06-28T16:14:34.682992+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":300,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:49:00.898473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48931,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":579,"num_read_kb":1282,"num_write":462,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"260'501","reported_seq":2303,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.195533+0000","last_change":"2024-06-28T15:00:23.416260+0000","last_active":"2024-06-28T16:14:32.195533+0000","last_peered":"2024-06-28T16:14:32.195533+0000","last_clean":"2024-06-28T16:14:32.195533+0000","last_became_active":"2024-06-28T15:00:23.415437+0000","last_became_peered":"2024-06-28T15:00:23.415437+0000","last_unstale":"2024-06-28T16:14:32.195533+0000","last_undegraded":"2024-06-28T16:14:32.195533+0000","last_fullsized":"2024-06-28T16:14:32.195533+0000","mapping_epoch":27,"log_start":"257'400","ondisk_log_start":"257'400","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":400,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:08:34.898411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"260'99","reported_seq":608,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.122863+0000","last_change":"2024-06-28T15:00:25.425875+0000","last_active":"2024-06-28T16:13:20.122863+0000","last_peered":"2024-06-28T16:13:20.122863+0000","last_clean":"2024-06-28T16:13:20.122863+0000","last_became_active":"2024-06-28T15:00:25.425135+0000","last_became_peered":"2024-06-28T15:00:25.425135+0000","last_unstale":"2024-06-28T16:13:20.122863+0000","last_undegraded":"2024-06-28T16:13:20.122863+0000","last_fullsized":"2024-06-28T16:13:20.122863+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:13:56.075833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":245,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"260'160","reported_seq":743,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101485+0000","last_change":"2024-06-28T15:00:17.403493+0000","last_active":"2024-06-28T16:12:20.101485+0000","last_peered":"2024-06-28T16:12:20.101485+0000","last_clean":"2024-06-28T16:12:20.101485+0000","last_became_active":"2024-06-28T15:00:17.402024+0000","last_became_peered":"2024-06-28T15:00:17.402024+0000","last_unstale":"2024-06-28T16:12:20.101485+0000","last_undegraded":"2024-06-28T16:12:20.101485+0000","last_fullsized":"2024-06-28T16:12:20.101485+0000","mapping_epoch":21,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":100,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:25:59.494240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":645,"num_read_kb":505,"num_write":259,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"260'393","reported_seq":1067,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.738050+0000","last_change":"2024-06-28T15:00:15.389691+0000","last_active":"2024-06-28T16:14:34.738050+0000","last_peered":"2024-06-28T16:14:34.738050+0000","last_clean":"2024-06-28T16:14:34.738050+0000","last_became_active":"2024-06-28T15:00:15.388365+0000","last_became_peered":"2024-06-28T15:00:15.388365+0000","last_unstale":"2024-06-28T16:14:34.738050+0000","last_undegraded":"2024-06-28T16:14:34.738050+0000","last_fullsized":"2024-06-28T16:14:34.738050+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":300,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:14:31.668500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1604260,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":661,"num_read_kb":7226,"num_write":1423,"num_write_kb":30573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"24'2","reported_seq":1154,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:31.948538+0000","last_change":"2024-06-28T15:00:19.403973+0000","last_active":"2024-06-28T16:14:31.948538+0000","last_peered":"2024-06-28T16:14:31.948538+0000","last_clean":"2024-06-28T16:14:31.948538+0000","last_became_active":"2024-06-28T15:00:19.397863+0000","last_became_peered":"2024-06-28T15:00:19.397863+0000","last_unstale":"2024-06-28T16:14:31.948538+0000","last_undegraded":"2024-06-28T16:14:31.948538+0000","last_fullsized":"2024-06-28T16:14:31.948538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:35:07.475408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":648,"num_read_kb":648,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"260'113","reported_seq":657,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.163323+0000","last_change":"2024-06-28T15:00:21.484486+0000","last_active":"2024-06-28T16:11:03.163323+0000","last_peered":"2024-06-28T16:11:03.163323+0000","last_clean":"2024-06-28T16:11:03.163323+0000","last_became_active":"2024-06-28T15:00:21.483679+0000","last_became_peered":"2024-06-28T15:00:21.483679+0000","last_unstale":"2024-06-28T16:11:03.163323+0000","last_undegraded":"2024-06-28T16:11:03.163323+0000","last_fullsized":"2024-06-28T16:11:03.163323+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:16:50.618747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"260'515","reported_seq":4262,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.199674+0000","last_change":"2024-06-28T15:00:23.419813+0000","last_active":"2024-06-28T16:14:32.199674+0000","last_peered":"2024-06-28T16:14:32.199674+0000","last_clean":"2024-06-28T16:14:32.199674+0000","last_became_active":"2024-06-28T15:00:23.419182+0000","last_became_peered":"2024-06-28T15:00:23.419182+0000","last_unstale":"2024-06-28T16:14:32.199674+0000","last_undegraded":"2024-06-28T16:14:32.199674+0000","last_fullsized":"2024-06-28T16:14:32.199674+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:00:01.245043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"260'159","reported_seq":672,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.114555+0000","last_change":"2024-06-28T15:00:25.427204+0000","last_active":"2024-06-28T16:13:20.114555+0000","last_peered":"2024-06-28T16:13:20.114555+0000","last_clean":"2024-06-28T16:13:20.114555+0000","last_became_active":"2024-06-28T15:00:25.425690+0000","last_became_peered":"2024-06-28T15:00:25.425690+0000","last_unstale":"2024-06-28T16:13:20.114555+0000","last_undegraded":"2024-06-28T16:13:20.114555+0000","last_fullsized":"2024-06-28T16:13:20.114555+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:18.073869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":152,"num_write":381,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"260'74","reported_seq":619,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.075926+0000","last_change":"2024-06-28T15:00:59.800386+0000","last_active":"2024-06-28T16:11:00.075926+0000","last_peered":"2024-06-28T16:11:00.075926+0000","last_clean":"2024-06-28T16:11:00.075926+0000","last_became_active":"2024-06-28T15:00:59.799823+0000","last_became_peered":"2024-06-28T15:00:59.799823+0000","last_unstale":"2024-06-28T16:11:00.075926+0000","last_undegraded":"2024-06-28T16:11:00.075926+0000","last_fullsized":"2024-06-28T16:11:00.075926+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:43:49.439721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":247,"num_write":103,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"260'351","reported_seq":1081,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.579126+0000","last_change":"2024-06-28T15:00:15.403798+0000","last_active":"2024-06-28T16:14:34.579126+0000","last_peered":"2024-06-28T16:14:34.579126+0000","last_clean":"2024-06-28T16:14:34.579126+0000","last_became_active":"2024-06-28T15:00:15.402782+0000","last_became_peered":"2024-06-28T15:00:15.402782+0000","last_unstale":"2024-06-28T16:14:34.579126+0000","last_undegraded":"2024-06-28T16:14:34.579126+0000","last_fullsized":"2024-06-28T16:14:34.579126+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":300,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:44:32.528218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262233,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":541,"num_read_kb":3603,"num_write":1164,"num_write_kb":36556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101454+0000","last_change":"2024-06-28T15:00:19.405878+0000","last_active":"2024-06-28T16:12:20.101454+0000","last_peered":"2024-06-28T16:12:20.101454+0000","last_clean":"2024-06-28T16:12:20.101454+0000","last_became_active":"2024-06-28T15:00:19.405083+0000","last_became_peered":"2024-06-28T15:00:19.405083+0000","last_unstale":"2024-06-28T16:12:20.101454+0000","last_undegraded":"2024-06-28T16:12:20.101454+0000","last_fullsized":"2024-06-28T16:12:20.101454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:55:51.552282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"260'429","reported_seq":1063,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.718564+0000","last_change":"2024-06-28T15:00:21.393207+0000","last_active":"2024-06-28T16:14:33.718564+0000","last_peered":"2024-06-28T16:14:33.718564+0000","last_clean":"2024-06-28T16:14:33.718564+0000","last_became_active":"2024-06-28T15:00:21.392393+0000","last_became_peered":"2024-06-28T15:00:21.392393+0000","last_unstale":"2024-06-28T16:14:33.718564+0000","last_undegraded":"2024-06-28T16:14:33.718564+0000","last_fullsized":"2024-06-28T16:14:33.718564+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":400,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:47:20.793097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47467,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1282,"num_read_kb":1893,"num_write":1822,"num_write_kb":1648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":493,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.070668+0000","last_change":"2024-06-28T15:00:23.416374+0000","last_active":"2024-06-28T16:10:10.070668+0000","last_peered":"2024-06-28T16:10:10.070668+0000","last_clean":"2024-06-28T16:10:10.070668+0000","last_became_active":"2024-06-28T15:00:23.415497+0000","last_became_peered":"2024-06-28T15:00:23.415497+0000","last_unstale":"2024-06-28T16:10:10.070668+0000","last_undegraded":"2024-06-28T16:10:10.070668+0000","last_fullsized":"2024-06-28T16:10:10.070668+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:13:09.664310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"260'133","reported_seq":639,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:07.643827+0000","last_change":"2024-06-28T15:00:25.440419+0000","last_active":"2024-06-28T16:14:07.643827+0000","last_peered":"2024-06-28T16:14:07.643827+0000","last_clean":"2024-06-28T16:14:07.643827+0000","last_became_active":"2024-06-28T15:00:25.439527+0000","last_became_peered":"2024-06-28T15:00:25.439527+0000","last_unstale":"2024-06-28T16:14:07.643827+0000","last_undegraded":"2024-06-28T16:14:07.643827+0000","last_fullsized":"2024-06-28T16:14:07.643827+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:30:19.298980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":127,"num_write":301,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"260'156","reported_seq":773,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.088666+0000","last_change":"2024-06-28T15:00:17.415172+0000","last_active":"2024-06-28T16:11:30.088666+0000","last_peered":"2024-06-28T16:11:30.088666+0000","last_clean":"2024-06-28T16:11:30.088666+0000","last_became_active":"2024-06-28T15:00:17.414537+0000","last_became_peered":"2024-06-28T15:00:17.414537+0000","last_unstale":"2024-06-28T16:11:30.088666+0000","last_undegraded":"2024-06-28T16:11:30.088666+0000","last_fullsized":"2024-06-28T16:11:30.088666+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:18:02.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":522,"num_write":223,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":616,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.109225+0000","last_change":"2024-06-28T15:00:06.303487+0000","last_active":"2024-06-28T16:12:25.109225+0000","last_peered":"2024-06-28T16:12:25.109225+0000","last_clean":"2024-06-28T16:12:25.109225+0000","last_became_active":"2024-06-28T15:00:06.302725+0000","last_became_peered":"2024-06-28T15:00:06.302725+0000","last_unstale":"2024-06-28T16:12:25.109225+0000","last_undegraded":"2024-06-28T16:12:25.109225+0000","last_fullsized":"2024-06-28T16:12:25.109225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:45:32.083226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.075782+0000","last_change":"2024-06-28T15:00:06.285125+0000","last_active":"2024-06-28T16:11:00.075782+0000","last_peered":"2024-06-28T16:11:00.075782+0000","last_clean":"2024-06-28T16:11:00.075782+0000","last_became_active":"2024-06-28T15:00:06.284218+0000","last_became_peered":"2024-06-28T15:00:06.284218+0000","last_unstale":"2024-06-28T16:11:00.075782+0000","last_undegraded":"2024-06-28T16:11:00.075782+0000","last_fullsized":"2024-06-28T16:11:00.075782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:04:51.635552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"260'320","reported_seq":1115,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.735430+0000","last_change":"2024-06-28T15:00:59.940725+0000","last_active":"2024-06-28T16:14:34.735430+0000","last_peered":"2024-06-28T16:14:34.735430+0000","last_clean":"2024-06-28T16:14:34.735430+0000","last_became_active":"2024-06-28T15:00:59.939352+0000","last_became_peered":"2024-06-28T15:00:59.939352+0000","last_unstale":"2024-06-28T16:14:34.735430+0000","last_undegraded":"2024-06-28T16:14:34.735430+0000","last_fullsized":"2024-06-28T16:14:34.735430+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:16:46.674713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":804,"num_read_kb":59106,"num_write":1183,"num_write_kb":43047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6,4],"acting":[3,2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.092431+0000","last_change":"2024-06-28T15:00:19.391126+0000","last_active":"2024-06-28T16:11:10.092431+0000","last_peered":"2024-06-28T16:11:10.092431+0000","last_clean":"2024-06-28T16:11:10.092431+0000","last_became_active":"2024-06-28T15:00:19.390255+0000","last_became_peered":"2024-06-28T15:00:19.390255+0000","last_unstale":"2024-06-28T16:11:10.092431+0000","last_undegraded":"2024-06-28T16:11:10.092431+0000","last_fullsized":"2024-06-28T16:11:10.092431+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:19:02.126295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"260'730","reported_seq":1450,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.539793+0000","last_change":"2024-06-28T15:00:21.398370+0000","last_active":"2024-06-28T16:14:34.539793+0000","last_peered":"2024-06-28T16:14:34.539793+0000","last_clean":"2024-06-28T16:14:34.539793+0000","last_became_active":"2024-06-28T15:00:21.397640+0000","last_became_peered":"2024-06-28T15:00:21.397640+0000","last_unstale":"2024-06-28T16:14:34.539793+0000","last_undegraded":"2024-06-28T16:14:34.539793+0000","last_fullsized":"2024-06-28T16:14:34.539793+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":700,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:10:23.954766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134565,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1425,"num_read_kb":3272,"num_write":1360,"num_write_kb":1190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.569752+0000","last_change":"2024-06-28T15:00:23.400439+0000","last_active":"2024-06-28T16:11:55.569752+0000","last_peered":"2024-06-28T16:11:55.569752+0000","last_clean":"2024-06-28T16:11:55.569752+0000","last_became_active":"2024-06-28T15:00:23.399024+0000","last_became_peered":"2024-06-28T15:00:23.399024+0000","last_unstale":"2024-06-28T16:11:55.569752+0000","last_undegraded":"2024-06-28T16:11:55.569752+0000","last_fullsized":"2024-06-28T16:11:55.569752+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:16:52.193469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"260'186","reported_seq":876,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:05.197144+0000","last_change":"2024-06-28T15:00:25.442723+0000","last_active":"2024-06-28T16:14:05.197144+0000","last_peered":"2024-06-28T16:14:05.197144+0000","last_clean":"2024-06-28T16:14:05.197144+0000","last_became_active":"2024-06-28T15:00:25.442032+0000","last_became_peered":"2024-06-28T15:00:25.442032+0000","last_unstale":"2024-06-28T16:14:05.197144+0000","last_undegraded":"2024-06-28T16:14:05.197144+0000","last_fullsized":"2024-06-28T16:14:05.197144+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:04:44.942959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":392,"num_read_kb":388,"num_write":388,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"260'118","reported_seq":755,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:57.970334+0000","last_change":"2024-06-28T15:00:17.404538+0000","last_active":"2024-06-28T16:11:57.970334+0000","last_peered":"2024-06-28T16:11:57.970334+0000","last_clean":"2024-06-28T16:11:57.970334+0000","last_became_active":"2024-06-28T15:00:17.404200+0000","last_became_peered":"2024-06-28T15:00:17.404200+0000","last_unstale":"2024-06-28T16:11:57.970334+0000","last_undegraded":"2024-06-28T16:11:57.970334+0000","last_fullsized":"2024-06-28T16:11:57.970334+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:32:41.333506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":495,"num_read_kb":413,"num_write":159,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":526,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.109267+0000","last_change":"2024-06-28T15:00:06.286231+0000","last_active":"2024-06-28T16:12:25.109267+0000","last_peered":"2024-06-28T16:12:25.109267+0000","last_clean":"2024-06-28T16:12:25.109267+0000","last_became_active":"2024-06-28T15:00:06.285489+0000","last_became_peered":"2024-06-28T15:00:06.285489+0000","last_unstale":"2024-06-28T16:12:25.109267+0000","last_undegraded":"2024-06-28T16:12:25.109267+0000","last_fullsized":"2024-06-28T16:12:25.109267+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:05:17.528909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"260'262","reported_seq":930,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.556093+0000","last_change":"2024-06-28T15:00:15.404155+0000","last_active":"2024-06-28T16:11:05.556093+0000","last_peered":"2024-06-28T16:11:05.556093+0000","last_clean":"2024-06-28T16:11:05.556093+0000","last_became_active":"2024-06-28T15:00:15.403657+0000","last_became_peered":"2024-06-28T15:00:15.403657+0000","last_unstale":"2024-06-28T16:11:05.556093+0000","last_undegraded":"2024-06-28T16:11:05.556093+0000","last_fullsized":"2024-06-28T16:11:05.556093+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:18:38.173993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6321470,"num_objects":64,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":462,"num_read_kb":36644,"num_write":1201,"num_write_kb":16548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.087155+0000","last_change":"2024-06-28T15:00:19.407157+0000","last_active":"2024-06-28T16:11:50.087155+0000","last_peered":"2024-06-28T16:11:50.087155+0000","last_clean":"2024-06-28T16:11:50.087155+0000","last_became_active":"2024-06-28T15:00:19.406447+0000","last_became_peered":"2024-06-28T15:00:19.406447+0000","last_unstale":"2024-06-28T16:11:50.087155+0000","last_undegraded":"2024-06-28T16:11:50.087155+0000","last_fullsized":"2024-06-28T16:11:50.087155+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:12:32.253515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"260'1134","reported_seq":2908,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.789539+0000","last_change":"2024-06-28T15:00:21.399223+0000","last_active":"2024-06-28T16:14:33.789539+0000","last_peered":"2024-06-28T16:14:33.789539+0000","last_clean":"2024-06-28T16:14:33.789539+0000","last_became_active":"2024-06-28T15:00:21.398382+0000","last_became_peered":"2024-06-28T15:00:21.398382+0000","last_unstale":"2024-06-28T16:14:33.789539+0000","last_undegraded":"2024-06-28T16:14:33.789539+0000","last_fullsized":"2024-06-28T16:14:33.789539+0000","mapping_epoch":25,"log_start":"260'1100","ondisk_log_start":"260'1100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":1100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:39:26.868657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133468,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1851,"num_read_kb":3741,"num_write":1150,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":498,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.097271+0000","last_change":"2024-06-28T15:00:23.423751+0000","last_active":"2024-06-28T16:11:30.097271+0000","last_peered":"2024-06-28T16:11:30.097271+0000","last_clean":"2024-06-28T16:11:30.097271+0000","last_became_active":"2024-06-28T15:00:23.422876+0000","last_became_peered":"2024-06-28T15:00:23.422876+0000","last_unstale":"2024-06-28T16:11:30.097271+0000","last_undegraded":"2024-06-28T16:11:30.097271+0000","last_fullsized":"2024-06-28T16:11:30.097271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:18:16.152063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"260'116","reported_seq":629,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:02.832339+0000","last_change":"2024-06-28T15:00:25.554459+0000","last_active":"2024-06-28T16:14:02.832339+0000","last_peered":"2024-06-28T16:14:02.832339+0000","last_clean":"2024-06-28T16:14:02.832339+0000","last_became_active":"2024-06-28T15:00:25.553691+0000","last_became_peered":"2024-06-28T15:00:25.553691+0000","last_unstale":"2024-06-28T16:14:02.832339+0000","last_undegraded":"2024-06-28T16:14:02.832339+0000","last_fullsized":"2024-06-28T16:14:02.832339+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:37:46.194533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":862,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":120,"num_write":292,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"260'221","reported_seq":834,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:02.146046+0000","last_change":"2024-06-28T15:00:17.393736+0000","last_active":"2024-06-28T16:13:02.146046+0000","last_peered":"2024-06-28T16:13:02.146046+0000","last_clean":"2024-06-28T16:13:02.146046+0000","last_became_active":"2024-06-28T15:00:17.393361+0000","last_became_peered":"2024-06-28T15:00:17.393361+0000","last_unstale":"2024-06-28T16:13:02.146046+0000","last_undegraded":"2024-06-28T16:13:02.146046+0000","last_fullsized":"2024-06-28T16:13:02.146046+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:50.952114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":892,"num_read_kb":761,"num_write":358,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":532,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.262525+0000","last_change":"2024-06-28T15:00:06.280312+0000","last_active":"2024-06-28T16:10:55.262525+0000","last_peered":"2024-06-28T16:10:55.262525+0000","last_clean":"2024-06-28T16:10:55.262525+0000","last_became_active":"2024-06-28T15:00:06.279950+0000","last_became_peered":"2024-06-28T15:00:06.279950+0000","last_unstale":"2024-06-28T16:10:55.262525+0000","last_undegraded":"2024-06-28T16:10:55.262525+0000","last_fullsized":"2024-06-28T16:10:55.262525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:05:40.110987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"260'334","reported_seq":991,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.720045+0000","last_change":"2024-06-28T15:00:15.413730+0000","last_active":"2024-06-28T16:14:34.720045+0000","last_peered":"2024-06-28T16:14:34.720045+0000","last_clean":"2024-06-28T16:14:34.720045+0000","last_became_active":"2024-06-28T15:00:15.413348+0000","last_became_peered":"2024-06-28T15:00:15.413348+0000","last_unstale":"2024-06-28T16:14:34.720045+0000","last_undegraded":"2024-06-28T16:14:34.720045+0000","last_fullsized":"2024-06-28T16:14:34.720045+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:07:23.937007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":504,"num_read_kb":3638,"num_write":1247,"num_write_kb":36014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"24'1","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.273547+0000","last_change":"2024-06-28T15:00:19.405107+0000","last_active":"2024-06-28T16:11:50.273547+0000","last_peered":"2024-06-28T16:11:50.273547+0000","last_clean":"2024-06-28T16:11:50.273547+0000","last_became_active":"2024-06-28T15:00:19.404653+0000","last_became_peered":"2024-06-28T15:00:19.404653+0000","last_unstale":"2024-06-28T16:11:50.273547+0000","last_undegraded":"2024-06-28T16:11:50.273547+0000","last_fullsized":"2024-06-28T16:11:50.273547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:18:41.227360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"260'246","reported_seq":1691,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:11.127801+0000","last_change":"2024-06-28T15:00:21.484753+0000","last_active":"2024-06-28T16:14:11.127801+0000","last_peered":"2024-06-28T16:14:11.127801+0000","last_clean":"2024-06-28T16:14:11.127801+0000","last_became_active":"2024-06-28T15:00:21.483749+0000","last_became_peered":"2024-06-28T15:00:21.483749+0000","last_unstale":"2024-06-28T16:14:11.127801+0000","last_undegraded":"2024-06-28T16:14:11.127801+0000","last_fullsized":"2024-06-28T16:14:11.127801+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:25:11.741395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.558850+0000","last_change":"2024-06-28T15:00:23.400124+0000","last_active":"2024-06-28T16:11:15.558850+0000","last_peered":"2024-06-28T16:11:15.558850+0000","last_clean":"2024-06-28T16:11:15.558850+0000","last_became_active":"2024-06-28T15:00:23.399391+0000","last_became_peered":"2024-06-28T15:00:23.399391+0000","last_unstale":"2024-06-28T16:11:15.558850+0000","last_undegraded":"2024-06-28T16:11:15.558850+0000","last_fullsized":"2024-06-28T16:11:15.558850+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:03:52.771119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"260'239","reported_seq":844,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:07.109352+0000","last_change":"2024-06-28T15:00:25.420452+0000","last_active":"2024-06-28T16:14:07.109352+0000","last_peered":"2024-06-28T16:14:07.109352+0000","last_clean":"2024-06-28T16:14:07.109352+0000","last_became_active":"2024-06-28T15:00:25.419711+0000","last_became_peered":"2024-06-28T15:00:25.419711+0000","last_unstale":"2024-06-28T16:14:07.109352+0000","last_undegraded":"2024-06-28T16:14:07.109352+0000","last_fullsized":"2024-06-28T16:14:07.109352+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":200,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:31:14.431698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":467,"num_read_kb":491,"num_write":517,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"260'315","reported_seq":1006,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:04.988991+0000","last_change":"2024-06-28T15:00:17.398693+0000","last_active":"2024-06-28T16:12:04.988991+0000","last_peered":"2024-06-28T16:12:04.988991+0000","last_clean":"2024-06-28T16:12:04.988991+0000","last_became_active":"2024-06-28T15:00:17.393696+0000","last_became_peered":"2024-06-28T15:00:17.393696+0000","last_unstale":"2024-06-28T16:12:04.988991+0000","last_undegraded":"2024-06-28T16:12:04.988991+0000","last_fullsized":"2024-06-28T16:12:04.988991+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:48:40.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1365,"num_read_kb":1154,"num_write":468,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.074668+0000","last_change":"2024-06-28T15:00:06.284111+0000","last_active":"2024-06-28T16:10:55.074668+0000","last_peered":"2024-06-28T16:10:55.074668+0000","last_clean":"2024-06-28T16:10:55.074668+0000","last_became_active":"2024-06-28T15:00:06.283106+0000","last_became_peered":"2024-06-28T15:00:06.283106+0000","last_unstale":"2024-06-28T16:10:55.074668+0000","last_undegraded":"2024-06-28T16:10:55.074668+0000","last_fullsized":"2024-06-28T16:10:55.074668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:07.069940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"260'318","reported_seq":991,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.731218+0000","last_change":"2024-06-28T15:00:15.415068+0000","last_active":"2024-06-28T16:14:34.731218+0000","last_peered":"2024-06-28T16:14:34.731218+0000","last_clean":"2024-06-28T16:14:34.731218+0000","last_became_active":"2024-06-28T15:00:15.413880+0000","last_became_peered":"2024-06-28T15:00:15.413880+0000","last_unstale":"2024-06-28T16:14:34.731218+0000","last_undegraded":"2024-06-28T16:14:34.731218+0000","last_fullsized":"2024-06-28T16:14:34.731218+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:13:10.361144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2421978,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":465,"num_read_kb":85416,"num_write":1148,"num_write_kb":27167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.561459+0000","last_change":"2024-06-28T15:00:19.382869+0000","last_active":"2024-06-28T16:11:25.561459+0000","last_peered":"2024-06-28T16:11:25.561459+0000","last_clean":"2024-06-28T16:11:25.561459+0000","last_became_active":"2024-06-28T15:00:19.382486+0000","last_became_peered":"2024-06-28T15:00:19.382486+0000","last_unstale":"2024-06-28T16:11:25.561459+0000","last_undegraded":"2024-06-28T16:11:25.561459+0000","last_fullsized":"2024-06-28T16:11:25.561459+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:22:55.369992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"260'159","reported_seq":729,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.514897+0000","last_change":"2024-06-28T15:00:21.401345+0000","last_active":"2024-06-28T16:11:55.514897+0000","last_peered":"2024-06-28T16:11:55.514897+0000","last_clean":"2024-06-28T16:11:55.514897+0000","last_became_active":"2024-06-28T15:00:21.400535+0000","last_became_peered":"2024-06-28T15:00:21.400535+0000","last_unstale":"2024-06-28T16:11:55.514897+0000","last_undegraded":"2024-06-28T16:11:55.514897+0000","last_fullsized":"2024-06-28T16:11:55.514897+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:33:23.802275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":198,"num_read_kb":191,"num_write":142,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":495,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.094802+0000","last_change":"2024-06-28T15:00:23.417433+0000","last_active":"2024-06-28T16:11:55.094802+0000","last_peered":"2024-06-28T16:11:55.094802+0000","last_clean":"2024-06-28T16:11:55.094802+0000","last_became_active":"2024-06-28T15:00:23.416701+0000","last_became_peered":"2024-06-28T15:00:23.416701+0000","last_unstale":"2024-06-28T16:11:55.094802+0000","last_undegraded":"2024-06-28T16:11:55.094802+0000","last_fullsized":"2024-06-28T16:11:55.094802+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:59:56.613956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"260'117","reported_seq":633,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:07.191149+0000","last_change":"2024-06-28T15:00:25.439202+0000","last_active":"2024-06-28T16:13:07.191149+0000","last_peered":"2024-06-28T16:13:07.191149+0000","last_clean":"2024-06-28T16:13:07.191149+0000","last_became_active":"2024-06-28T15:00:25.438523+0000","last_became_peered":"2024-06-28T15:00:25.438523+0000","last_unstale":"2024-06-28T16:13:07.191149+0000","last_undegraded":"2024-06-28T16:13:07.191149+0000","last_fullsized":"2024-06-28T16:13:07.191149+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:31:55.560360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":129,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"260'169","reported_seq":774,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.093820+0000","last_change":"2024-06-28T15:00:17.399083+0000","last_active":"2024-06-28T16:11:10.093820+0000","last_peered":"2024-06-28T16:11:10.093820+0000","last_clean":"2024-06-28T16:11:10.093820+0000","last_became_active":"2024-06-28T15:00:17.398515+0000","last_became_peered":"2024-06-28T15:00:17.398515+0000","last_unstale":"2024-06-28T16:11:10.093820+0000","last_undegraded":"2024-06-28T16:11:10.093820+0000","last_fullsized":"2024-06-28T16:11:10.093820+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:10.485888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":644,"num_read_kb":512,"num_write":273,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"260'372","reported_seq":1043,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.710966+0000","last_change":"2024-06-28T15:00:15.417442+0000","last_active":"2024-06-28T16:14:34.710966+0000","last_peered":"2024-06-28T16:14:34.710966+0000","last_clean":"2024-06-28T16:14:34.710966+0000","last_became_active":"2024-06-28T15:00:15.414704+0000","last_became_peered":"2024-06-28T15:00:15.414704+0000","last_unstale":"2024-06-28T16:14:34.710966+0000","last_undegraded":"2024-06-28T16:14:34.710966+0000","last_fullsized":"2024-06-28T16:14:34.710966+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:54:15.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10506361,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":617,"num_read_kb":33619,"num_write":1289,"num_write_kb":23520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":529,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.092017+0000","last_change":"2024-06-28T15:00:06.302842+0000","last_active":"2024-06-28T16:11:00.092017+0000","last_peered":"2024-06-28T16:11:00.092017+0000","last_clean":"2024-06-28T16:11:00.092017+0000","last_became_active":"2024-06-28T15:00:06.302449+0000","last_became_peered":"2024-06-28T15:00:06.302449+0000","last_unstale":"2024-06-28T16:11:00.092017+0000","last_undegraded":"2024-06-28T16:11:00.092017+0000","last_fullsized":"2024-06-28T16:11:00.092017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:06:10.739681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.557574+0000","last_change":"2024-06-28T15:00:19.385492+0000","last_active":"2024-06-28T16:11:10.557574+0000","last_peered":"2024-06-28T16:11:10.557574+0000","last_clean":"2024-06-28T16:11:10.557574+0000","last_became_active":"2024-06-28T15:00:19.384933+0000","last_became_peered":"2024-06-28T15:00:19.384933+0000","last_unstale":"2024-06-28T16:11:10.557574+0000","last_undegraded":"2024-06-28T16:11:10.557574+0000","last_fullsized":"2024-06-28T16:11:10.557574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:16:09.011700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"260'577","reported_seq":1265,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:12.941428+0000","last_change":"2024-06-28T15:00:21.407998+0000","last_active":"2024-06-28T16:14:12.941428+0000","last_peered":"2024-06-28T16:14:12.941428+0000","last_clean":"2024-06-28T16:14:12.941428+0000","last_became_active":"2024-06-28T15:00:21.405762+0000","last_became_peered":"2024-06-28T15:00:21.405762+0000","last_unstale":"2024-06-28T16:14:12.941428+0000","last_undegraded":"2024-06-28T16:14:12.941428+0000","last_fullsized":"2024-06-28T16:14:12.941428+0000","mapping_epoch":25,"log_start":"260'500","ondisk_log_start":"260'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":500,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:56:43.045836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111793,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":975,"num_read_kb":1983,"num_write":826,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.086163+0000","last_change":"2024-06-28T15:00:23.405517+0000","last_active":"2024-06-28T16:11:15.086163+0000","last_peered":"2024-06-28T16:11:15.086163+0000","last_clean":"2024-06-28T16:11:15.086163+0000","last_became_active":"2024-06-28T15:00:23.404714+0000","last_became_peered":"2024-06-28T15:00:23.404714+0000","last_unstale":"2024-06-28T16:11:15.086163+0000","last_undegraded":"2024-06-28T16:11:15.086163+0000","last_fullsized":"2024-06-28T16:11:15.086163+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:54:34.158419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"260'147","reported_seq":663,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.650237+0000","last_change":"2024-06-28T15:00:25.557678+0000","last_active":"2024-06-28T16:14:06.650237+0000","last_peered":"2024-06-28T16:14:06.650237+0000","last_clean":"2024-06-28T16:14:06.650237+0000","last_became_active":"2024-06-28T15:00:25.556834+0000","last_became_peered":"2024-06-28T15:00:25.556834+0000","last_unstale":"2024-06-28T16:14:06.650237+0000","last_undegraded":"2024-06-28T16:14:06.650237+0000","last_fullsized":"2024-06-28T16:14:06.650237+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":100,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:52:25.536046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1183,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":342,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"260'139","reported_seq":768,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.557529+0000","last_change":"2024-06-28T15:00:17.372769+0000","last_active":"2024-06-28T16:11:10.557529+0000","last_peered":"2024-06-28T16:11:10.557529+0000","last_clean":"2024-06-28T16:11:10.557529+0000","last_became_active":"2024-06-28T15:00:17.372401+0000","last_became_peered":"2024-06-28T15:00:17.372401+0000","last_unstale":"2024-06-28T16:11:10.557529+0000","last_undegraded":"2024-06-28T16:11:10.557529+0000","last_fullsized":"2024-06-28T16:11:10.557529+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:34:11.796111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":577,"num_read_kb":470,"num_write":221,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"260'273","reported_seq":901,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.481261+0000","last_change":"2024-06-28T15:00:15.408605+0000","last_active":"2024-06-28T16:14:34.481261+0000","last_peered":"2024-06-28T16:14:34.481261+0000","last_clean":"2024-06-28T16:14:34.481261+0000","last_became_active":"2024-06-28T15:00:15.408140+0000","last_became_peered":"2024-06-28T15:00:15.408140+0000","last_unstale":"2024-06-28T16:14:34.481261+0000","last_undegraded":"2024-06-28T16:14:34.481261+0000","last_fullsized":"2024-06-28T16:14:34.481261+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":200,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:51:15.618780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":526399,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":387,"num_read_kb":8190,"num_write":1081,"num_write_kb":24156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.093729+0000","last_change":"2024-06-28T15:00:06.303984+0000","last_active":"2024-06-28T16:11:15.093729+0000","last_peered":"2024-06-28T16:11:15.093729+0000","last_clean":"2024-06-28T16:11:15.093729+0000","last_became_active":"2024-06-28T15:00:06.302962+0000","last_became_peered":"2024-06-28T15:00:06.302962+0000","last_unstale":"2024-06-28T16:11:15.093729+0000","last_undegraded":"2024-06-28T16:11:15.093729+0000","last_fullsized":"2024-06-28T16:11:15.093729+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:10:47.354122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.084030+0000","last_change":"2024-06-28T15:00:19.385776+0000","last_active":"2024-06-28T16:11:10.084030+0000","last_peered":"2024-06-28T16:11:10.084030+0000","last_clean":"2024-06-28T16:11:10.084030+0000","last_became_active":"2024-06-28T15:00:19.384961+0000","last_became_peered":"2024-06-28T15:00:19.384961+0000","last_unstale":"2024-06-28T16:11:10.084030+0000","last_undegraded":"2024-06-28T16:11:10.084030+0000","last_fullsized":"2024-06-28T16:11:10.084030+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:23:32.384577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"260'188","reported_seq":785,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.095201+0000","last_change":"2024-06-28T15:00:21.480780+0000","last_active":"2024-06-28T16:11:15.095201+0000","last_peered":"2024-06-28T16:11:15.095201+0000","last_clean":"2024-06-28T16:11:15.095201+0000","last_became_active":"2024-06-28T15:00:21.480107+0000","last_became_peered":"2024-06-28T15:00:21.480107+0000","last_unstale":"2024-06-28T16:11:15.095201+0000","last_undegraded":"2024-06-28T16:11:15.095201+0000","last_fullsized":"2024-06-28T16:11:15.095201+0000","mapping_epoch":25,"log_start":"257'100","ondisk_log_start":"257'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:46:14.608558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":263,"num_read_kb":253,"num_write":174,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"260'565","reported_seq":2877,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.824388+0000","last_change":"2024-06-28T15:00:23.429133+0000","last_active":"2024-06-28T16:14:32.824388+0000","last_peered":"2024-06-28T16:14:32.824388+0000","last_clean":"2024-06-28T16:14:32.824388+0000","last_became_active":"2024-06-28T15:00:23.427565+0000","last_became_peered":"2024-06-28T15:00:23.427565+0000","last_unstale":"2024-06-28T16:14:32.824388+0000","last_undegraded":"2024-06-28T16:14:32.824388+0000","last_fullsized":"2024-06-28T16:14:32.824388+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":500,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:07:13.588883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"260'124","reported_seq":638,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:30.524126+0000","last_change":"2024-06-28T15:00:25.554359+0000","last_active":"2024-06-28T16:12:30.524126+0000","last_peered":"2024-06-28T16:12:30.524126+0000","last_clean":"2024-06-28T16:12:30.524126+0000","last_became_active":"2024-06-28T15:00:25.553627+0000","last_became_peered":"2024-06-28T15:00:25.553627+0000","last_unstale":"2024-06-28T16:12:30.524126+0000","last_undegraded":"2024-06-28T16:12:30.524126+0000","last_fullsized":"2024-06-28T16:12:30.524126+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:38:46.580841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":713,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":165,"num_read_kb":148,"num_write":308,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"260'172","reported_seq":825,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.522189+0000","last_change":"2024-06-28T15:00:17.377265+0000","last_active":"2024-06-28T16:12:25.522189+0000","last_peered":"2024-06-28T16:12:25.522189+0000","last_clean":"2024-06-28T16:12:25.522189+0000","last_became_active":"2024-06-28T15:00:17.376516+0000","last_became_peered":"2024-06-28T15:00:17.376516+0000","last_unstale":"2024-06-28T16:12:25.522189+0000","last_undegraded":"2024-06-28T16:12:25.522189+0000","last_fullsized":"2024-06-28T16:12:25.522189+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:46:26.820686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":722,"num_read_kb":588,"num_write":244,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"260'336","reported_seq":980,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.728953+0000","last_change":"2024-06-28T15:00:15.403594+0000","last_active":"2024-06-28T16:14:34.728953+0000","last_peered":"2024-06-28T16:14:34.728953+0000","last_clean":"2024-06-28T16:14:34.728953+0000","last_became_active":"2024-06-28T15:00:15.403032+0000","last_became_peered":"2024-06-28T15:00:15.403032+0000","last_unstale":"2024-06-28T16:14:34.728953+0000","last_undegraded":"2024-06-28T16:14:34.728953+0000","last_fullsized":"2024-06-28T16:14:34.728953+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:18:51.571850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":968026,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":483,"num_read_kb":17032,"num_write":1147,"num_write_kb":35885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":528,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.099353+0000","last_change":"2024-06-28T15:00:06.278832+0000","last_active":"2024-06-28T16:11:35.099353+0000","last_peered":"2024-06-28T16:11:35.099353+0000","last_clean":"2024-06-28T16:11:35.099353+0000","last_became_active":"2024-06-28T15:00:06.278443+0000","last_became_peered":"2024-06-28T15:00:06.278443+0000","last_unstale":"2024-06-28T16:11:35.099353+0000","last_undegraded":"2024-06-28T16:11:35.099353+0000","last_fullsized":"2024-06-28T16:11:35.099353+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:40:27.400018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.085018+0000","last_change":"2024-06-28T15:00:19.380916+0000","last_active":"2024-06-28T16:11:10.085018+0000","last_peered":"2024-06-28T16:11:10.085018+0000","last_clean":"2024-06-28T16:11:10.085018+0000","last_became_active":"2024-06-28T15:00:19.378932+0000","last_became_peered":"2024-06-28T15:00:19.378932+0000","last_unstale":"2024-06-28T16:11:10.085018+0000","last_undegraded":"2024-06-28T16:11:10.085018+0000","last_fullsized":"2024-06-28T16:11:10.085018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:36:40.033839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"260'138","reported_seq":698,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.086320+0000","last_change":"2024-06-28T15:00:21.396435+0000","last_active":"2024-06-28T16:11:15.086320+0000","last_peered":"2024-06-28T16:11:15.086320+0000","last_clean":"2024-06-28T16:11:15.086320+0000","last_became_active":"2024-06-28T15:00:21.395462+0000","last_became_peered":"2024-06-28T15:00:21.395462+0000","last_unstale":"2024-06-28T16:11:15.086320+0000","last_undegraded":"2024-06-28T16:11:15.086320+0000","last_fullsized":"2024-06-28T16:11:15.086320+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:51:41.956055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":167,"num_read_kb":161,"num_write":112,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"260'537","reported_seq":2581,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.259280+0000","last_change":"2024-06-28T15:00:23.428281+0000","last_active":"2024-06-28T16:14:34.259280+0000","last_peered":"2024-06-28T16:14:34.259280+0000","last_clean":"2024-06-28T16:14:34.259280+0000","last_became_active":"2024-06-28T15:00:23.427568+0000","last_became_peered":"2024-06-28T15:00:23.427568+0000","last_unstale":"2024-06-28T16:14:34.259280+0000","last_undegraded":"2024-06-28T16:14:34.259280+0000","last_fullsized":"2024-06-28T16:14:34.259280+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:37:02.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"260'247","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.544258+0000","last_change":"2024-06-28T15:00:25.428796+0000","last_active":"2024-06-28T16:10:10.544258+0000","last_peered":"2024-06-28T16:10:10.544258+0000","last_clean":"2024-06-28T16:10:10.544258+0000","last_became_active":"2024-06-28T15:00:25.428067+0000","last_became_peered":"2024-06-28T15:00:25.428067+0000","last_unstale":"2024-06-28T16:10:10.544258+0000","last_undegraded":"2024-06-28T16:10:10.544258+0000","last_fullsized":"2024-06-28T16:10:10.544258+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":200,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:07:43.747052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":297,"num_write":552,"num_write_kb":170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"260'96","reported_seq":701,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:46.633284+0000","last_change":"2024-06-28T15:00:17.397233+0000","last_active":"2024-06-28T16:12:46.633284+0000","last_peered":"2024-06-28T16:12:46.633284+0000","last_clean":"2024-06-28T16:12:46.633284+0000","last_became_active":"2024-06-28T15:00:17.396413+0000","last_became_peered":"2024-06-28T15:00:17.396413+0000","last_unstale":"2024-06-28T16:12:46.633284+0000","last_undegraded":"2024-06-28T16:12:46.633284+0000","last_fullsized":"2024-06-28T16:12:46.633284+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:50:29.409467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":410,"num_read_kb":340,"num_write":139,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"260'261","reported_seq":1040,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:45.565343+0000","last_change":"2024-06-28T15:00:15.396426+0000","last_active":"2024-06-28T16:11:45.565343+0000","last_peered":"2024-06-28T16:11:45.565343+0000","last_clean":"2024-06-28T16:11:45.565343+0000","last_became_active":"2024-06-28T15:00:15.395980+0000","last_became_peered":"2024-06-28T15:00:15.395980+0000","last_unstale":"2024-06-28T16:11:45.565343+0000","last_undegraded":"2024-06-28T16:11:45.565343+0000","last_fullsized":"2024-06-28T16:11:45.565343+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:22:49.730474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2049512,"num_objects":63,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":592,"num_read_kb":125515,"num_write":1130,"num_write_kb":17513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":521,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089897+0000","last_change":"2024-06-28T15:00:06.284169+0000","last_active":"2024-06-28T16:11:35.089897+0000","last_peered":"2024-06-28T16:11:35.089897+0000","last_clean":"2024-06-28T16:11:35.089897+0000","last_became_active":"2024-06-28T15:00:06.283380+0000","last_became_peered":"2024-06-28T15:00:06.283380+0000","last_unstale":"2024-06-28T16:11:35.089897+0000","last_undegraded":"2024-06-28T16:11:35.089897+0000","last_fullsized":"2024-06-28T16:11:35.089897+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:39:59.368666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"31'5","reported_seq":842,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:31.932486+0000","last_change":"2024-06-28T15:00:19.372913+0000","last_active":"2024-06-28T16:14:31.932486+0000","last_peered":"2024-06-28T16:14:31.932486+0000","last_clean":"2024-06-28T16:14:31.932486+0000","last_became_active":"2024-06-28T15:00:19.372552+0000","last_became_peered":"2024-06-28T15:00:19.372552+0000","last_unstale":"2024-06-28T16:14:31.932486+0000","last_undegraded":"2024-06-28T16:14:31.932486+0000","last_fullsized":"2024-06-28T16:14:31.932486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:19:05.828246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":500,"num_read_kb":812,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"260'748","reported_seq":1492,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.576522+0000","last_change":"2024-06-28T15:00:21.481881+0000","last_active":"2024-06-28T16:14:33.576522+0000","last_peered":"2024-06-28T16:14:33.576522+0000","last_clean":"2024-06-28T16:14:33.576522+0000","last_became_active":"2024-06-28T15:00:21.481155+0000","last_became_peered":"2024-06-28T15:00:21.481155+0000","last_unstale":"2024-06-28T16:14:33.576522+0000","last_undegraded":"2024-06-28T16:14:33.576522+0000","last_fullsized":"2024-06-28T16:14:33.576522+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":700,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:24:28.507362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131438,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1320,"num_read_kb":3166,"num_write":1058,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.505408+0000","last_change":"2024-06-28T15:00:23.409553+0000","last_active":"2024-06-28T16:11:15.505408+0000","last_peered":"2024-06-28T16:11:15.505408+0000","last_clean":"2024-06-28T16:11:15.505408+0000","last_became_active":"2024-06-28T15:00:23.408777+0000","last_became_peered":"2024-06-28T15:00:23.408777+0000","last_unstale":"2024-06-28T16:11:15.505408+0000","last_undegraded":"2024-06-28T16:11:15.505408+0000","last_fullsized":"2024-06-28T16:11:15.505408+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:29:28.695608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"260'129","reported_seq":640,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:07.935429+0000","last_change":"2024-06-28T15:00:25.439187+0000","last_active":"2024-06-28T16:14:07.935429+0000","last_peered":"2024-06-28T16:14:07.935429+0000","last_clean":"2024-06-28T16:14:07.935429+0000","last_became_active":"2024-06-28T15:00:25.438476+0000","last_became_peered":"2024-06-28T15:00:25.438476+0000","last_unstale":"2024-06-28T16:14:07.935429+0000","last_undegraded":"2024-06-28T16:14:07.935429+0000","last_fullsized":"2024-06-28T16:14:07.935429+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:14:10.401700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":173,"num_read_kb":156,"num_write":320,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"260'151","reported_seq":718,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.829530+0000","last_change":"2024-06-28T15:00:25.440427+0000","last_active":"2024-06-28T16:14:08.829530+0000","last_peered":"2024-06-28T16:14:08.829530+0000","last_clean":"2024-06-28T16:14:08.829530+0000","last_became_active":"2024-06-28T15:00:25.439672+0000","last_became_peered":"2024-06-28T15:00:25.439672+0000","last_unstale":"2024-06-28T16:14:08.829530+0000","last_undegraded":"2024-06-28T16:14:08.829530+0000","last_fullsized":"2024-06-28T16:14:08.829530+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:21:43.580938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":207,"num_read_kb":205,"num_write":345,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"260'434","reported_seq":1071,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:58.995763+0000","last_change":"2024-06-28T15:00:17.402772+0000","last_active":"2024-06-28T16:10:58.995763+0000","last_peered":"2024-06-28T16:10:58.995763+0000","last_clean":"2024-06-28T16:10:58.995763+0000","last_became_active":"2024-06-28T15:00:17.401981+0000","last_became_peered":"2024-06-28T15:00:17.401981+0000","last_unstale":"2024-06-28T16:10:58.995763+0000","last_undegraded":"2024-06-28T16:10:58.995763+0000","last_fullsized":"2024-06-28T16:10:58.995763+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":400,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:36:44.706683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1728,"num_read_kb":1391,"num_write":746,"num_write_kb":387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"260'274","reported_seq":900,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.557629+0000","last_change":"2024-06-28T15:00:15.404138+0000","last_active":"2024-06-28T16:14:34.557629+0000","last_peered":"2024-06-28T16:14:34.557629+0000","last_clean":"2024-06-28T16:14:34.557629+0000","last_became_active":"2024-06-28T15:00:15.403450+0000","last_became_peered":"2024-06-28T15:00:15.403450+0000","last_unstale":"2024-06-28T16:14:34.557629+0000","last_undegraded":"2024-06-28T16:14:34.557629+0000","last_fullsized":"2024-06-28T16:14:34.557629+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:20:44.673833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437799,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":13452,"num_write":1073,"num_write_kb":25751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.084184+0000","last_change":"2024-06-28T15:00:19.398728+0000","last_active":"2024-06-28T16:10:20.084184+0000","last_peered":"2024-06-28T16:10:20.084184+0000","last_clean":"2024-06-28T16:10:20.084184+0000","last_became_active":"2024-06-28T15:00:19.398138+0000","last_became_peered":"2024-06-28T15:00:19.398138+0000","last_unstale":"2024-06-28T16:10:20.084184+0000","last_undegraded":"2024-06-28T16:10:20.084184+0000","last_fullsized":"2024-06-28T16:10:20.084184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:24:09.555887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"260'298","reported_seq":895,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.023577+0000","last_change":"2024-06-28T15:00:21.405835+0000","last_active":"2024-06-28T16:14:34.023577+0000","last_peered":"2024-06-28T16:14:34.023577+0000","last_clean":"2024-06-28T16:14:34.023577+0000","last_became_active":"2024-06-28T15:00:21.405016+0000","last_became_peered":"2024-06-28T15:00:21.405016+0000","last_unstale":"2024-06-28T16:14:34.023577+0000","last_undegraded":"2024-06-28T16:14:34.023577+0000","last_fullsized":"2024-06-28T16:14:34.023577+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:00:56.662170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53726,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":505,"num_read_kb":1165,"num_write":406,"num_write_kb":309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":499,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493149+0000","last_change":"2024-06-28T15:00:23.418156+0000","last_active":"2024-06-28T16:10:25.493149+0000","last_peered":"2024-06-28T16:10:25.493149+0000","last_clean":"2024-06-28T16:10:25.493149+0000","last_became_active":"2024-06-28T15:00:23.417390+0000","last_became_peered":"2024-06-28T15:00:23.417390+0000","last_unstale":"2024-06-28T16:10:25.493149+0000","last_undegraded":"2024-06-28T16:10:25.493149+0000","last_fullsized":"2024-06-28T16:10:25.493149+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:22:25.288900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"260'101","reported_seq":608,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:30.524057+0000","last_change":"2024-06-28T15:00:25.413154+0000","last_active":"2024-06-28T16:12:30.524057+0000","last_peered":"2024-06-28T16:12:30.524057+0000","last_clean":"2024-06-28T16:12:30.524057+0000","last_became_active":"2024-06-28T15:00:25.412379+0000","last_became_peered":"2024-06-28T15:00:25.412379+0000","last_unstale":"2024-06-28T16:12:30.524057+0000","last_undegraded":"2024-06-28T16:12:30.524057+0000","last_fullsized":"2024-06-28T16:12:30.524057+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:26:49.949836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":88,"num_write":238,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"260'189","reported_seq":822,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:47.151324+0000","last_change":"2024-06-28T15:00:17.406956+0000","last_active":"2024-06-28T16:11:47.151324+0000","last_peered":"2024-06-28T16:11:47.151324+0000","last_clean":"2024-06-28T16:11:47.151324+0000","last_became_active":"2024-06-28T15:00:17.406647+0000","last_became_peered":"2024-06-28T15:00:17.406647+0000","last_unstale":"2024-06-28T16:11:47.151324+0000","last_undegraded":"2024-06-28T16:11:47.151324+0000","last_fullsized":"2024-06-28T16:11:47.151324+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:18:05.135571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":748,"num_read_kb":601,"num_write":301,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"260'334","reported_seq":1001,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.613707+0000","last_change":"2024-06-28T15:00:15.413269+0000","last_active":"2024-06-28T16:14:34.613707+0000","last_peered":"2024-06-28T16:14:34.613707+0000","last_clean":"2024-06-28T16:14:34.613707+0000","last_became_active":"2024-06-28T15:00:15.402945+0000","last_became_peered":"2024-06-28T15:00:15.402945+0000","last_unstale":"2024-06-28T16:14:34.613707+0000","last_undegraded":"2024-06-28T16:14:34.613707+0000","last_fullsized":"2024-06-28T16:14:34.613707+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:41:27.586797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340417,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":534,"num_read_kb":20609,"num_write":1305,"num_write_kb":45266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,4],"acting":[7,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.492088+0000","last_change":"2024-06-28T15:00:19.383614+0000","last_active":"2024-06-28T16:10:20.492088+0000","last_peered":"2024-06-28T16:10:20.492088+0000","last_clean":"2024-06-28T16:10:20.492088+0000","last_became_active":"2024-06-28T15:00:19.382839+0000","last_became_peered":"2024-06-28T15:00:19.382839+0000","last_unstale":"2024-06-28T16:10:20.492088+0000","last_undegraded":"2024-06-28T16:10:20.492088+0000","last_fullsized":"2024-06-28T16:10:20.492088+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:09:23.668167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"260'52","reported_seq":573,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:58.477001+0000","last_change":"2024-06-28T15:00:21.407387+0000","last_active":"2024-06-28T16:10:58.477001+0000","last_peered":"2024-06-28T16:10:58.477001+0000","last_clean":"2024-06-28T16:10:58.477001+0000","last_became_active":"2024-06-28T15:00:21.404727+0000","last_became_peered":"2024-06-28T15:00:21.404727+0000","last_unstale":"2024-06-28T16:10:58.477001+0000","last_undegraded":"2024-06-28T16:10:58.477001+0000","last_fullsized":"2024-06-28T16:10:58.477001+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:02:08.588463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":48,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":494,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.081639+0000","last_change":"2024-06-28T15:00:23.419840+0000","last_active":"2024-06-28T16:10:25.081639+0000","last_peered":"2024-06-28T16:10:25.081639+0000","last_clean":"2024-06-28T16:10:25.081639+0000","last_became_active":"2024-06-28T15:00:23.419115+0000","last_became_peered":"2024-06-28T15:00:23.419115+0000","last_unstale":"2024-06-28T16:10:25.081639+0000","last_undegraded":"2024-06-28T16:10:25.081639+0000","last_fullsized":"2024-06-28T16:10:25.081639+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:05:05.710065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"260'190","reported_seq":719,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.664824+0000","last_change":"2024-06-28T15:00:25.426812+0000","last_active":"2024-06-28T16:14:06.664824+0000","last_peered":"2024-06-28T16:14:06.664824+0000","last_clean":"2024-06-28T16:14:06.664824+0000","last_became_active":"2024-06-28T15:00:25.426046+0000","last_became_peered":"2024-06-28T15:00:25.426046+0000","last_unstale":"2024-06-28T16:14:06.664824+0000","last_undegraded":"2024-06-28T16:14:06.664824+0000","last_fullsized":"2024-06-28T16:14:06.664824+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:19:11.968158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":286,"num_read_kb":283,"num_write":434,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"260'423","reported_seq":1170,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:17.460800+0000","last_change":"2024-06-28T15:00:17.390367+0000","last_active":"2024-06-28T16:11:17.460800+0000","last_peered":"2024-06-28T16:11:17.460800+0000","last_clean":"2024-06-28T16:11:17.460800+0000","last_became_active":"2024-06-28T15:00:17.389992+0000","last_became_peered":"2024-06-28T15:00:17.389992+0000","last_unstale":"2024-06-28T16:11:17.460800+0000","last_undegraded":"2024-06-28T16:11:17.460800+0000","last_fullsized":"2024-06-28T16:11:17.460800+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":400,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:13:23.718079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1786,"num_read_kb":1490,"num_write":759,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"260'328","reported_seq":971,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.635861+0000","last_change":"2024-06-28T15:00:15.427261+0000","last_active":"2024-06-28T16:14:34.635861+0000","last_peered":"2024-06-28T16:14:34.635861+0000","last_clean":"2024-06-28T16:14:34.635861+0000","last_became_active":"2024-06-28T15:00:15.419267+0000","last_became_peered":"2024-06-28T15:00:15.419267+0000","last_unstale":"2024-06-28T16:14:34.635861+0000","last_undegraded":"2024-06-28T16:14:34.635861+0000","last_fullsized":"2024-06-28T16:14:34.635861+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:49:04.986604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5413475,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":487,"num_read_kb":27459,"num_write":1288,"num_write_kb":48933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080344+0000","last_change":"2024-06-28T15:00:19.386890+0000","last_active":"2024-06-28T16:10:20.080344+0000","last_peered":"2024-06-28T16:10:20.080344+0000","last_clean":"2024-06-28T16:10:20.080344+0000","last_became_active":"2024-06-28T15:00:19.385990+0000","last_became_peered":"2024-06-28T15:00:19.385990+0000","last_unstale":"2024-06-28T16:10:20.080344+0000","last_undegraded":"2024-06-28T16:10:20.080344+0000","last_fullsized":"2024-06-28T16:10:20.080344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:39:52.204033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"260'318","reported_seq":931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.206768+0000","last_change":"2024-06-28T15:00:21.406717+0000","last_active":"2024-06-28T16:14:33.206768+0000","last_peered":"2024-06-28T16:14:33.206768+0000","last_clean":"2024-06-28T16:14:33.206768+0000","last_became_active":"2024-06-28T15:00:21.405045+0000","last_became_peered":"2024-06-28T15:00:21.405045+0000","last_unstale":"2024-06-28T16:14:33.206768+0000","last_undegraded":"2024-06-28T16:14:33.206768+0000","last_fullsized":"2024-06-28T16:14:33.206768+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:50:38.218164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48456,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":530,"num_read_kb":1162,"num_write":414,"num_write_kb":289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":492,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.074474+0000","last_change":"2024-06-28T15:00:23.415444+0000","last_active":"2024-06-28T16:10:25.074474+0000","last_peered":"2024-06-28T16:10:25.074474+0000","last_clean":"2024-06-28T16:10:25.074474+0000","last_became_active":"2024-06-28T15:00:23.411264+0000","last_became_peered":"2024-06-28T15:00:23.411264+0000","last_unstale":"2024-06-28T16:10:25.074474+0000","last_undegraded":"2024-06-28T16:10:25.074474+0000","last_fullsized":"2024-06-28T16:10:25.074474+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:58:05.328252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"260'110","reported_seq":602,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.067825+0000","last_change":"2024-06-28T15:00:25.443135+0000","last_active":"2024-06-28T16:10:25.067825+0000","last_peered":"2024-06-28T16:10:25.067825+0000","last_clean":"2024-06-28T16:10:25.067825+0000","last_became_active":"2024-06-28T15:00:25.442010+0000","last_became_peered":"2024-06-28T15:00:25.442010+0000","last_unstale":"2024-06-28T16:10:25.067825+0000","last_undegraded":"2024-06-28T16:10:25.067825+0000","last_fullsized":"2024-06-28T16:10:25.067825+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:56:35.040791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":638,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":85,"num_write":260,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"260'105","reported_seq":669,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.491831+0000","last_change":"2024-06-28T15:00:17.376172+0000","last_active":"2024-06-28T16:10:20.491831+0000","last_peered":"2024-06-28T16:10:20.491831+0000","last_clean":"2024-06-28T16:10:20.491831+0000","last_became_active":"2024-06-28T15:00:17.373396+0000","last_became_peered":"2024-06-28T15:00:17.373396+0000","last_unstale":"2024-06-28T16:10:20.491831+0000","last_undegraded":"2024-06-28T16:10:20.491831+0000","last_fullsized":"2024-06-28T16:10:20.491831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:59:28.317922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":424,"num_read_kb":340,"num_write":155,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"260'260","reported_seq":907,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:03.731453+0000","last_change":"2024-06-28T15:00:15.412826+0000","last_active":"2024-06-28T16:12:03.731453+0000","last_peered":"2024-06-28T16:12:03.731453+0000","last_clean":"2024-06-28T16:12:03.731453+0000","last_became_active":"2024-06-28T15:00:15.412333+0000","last_became_peered":"2024-06-28T15:00:15.412333+0000","last_unstale":"2024-06-28T16:12:03.731453+0000","last_undegraded":"2024-06-28T16:12:03.731453+0000","last_fullsized":"2024-06-28T16:12:03.731453+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:32:08.060194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6832535,"num_objects":55,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":411,"num_read_kb":31119,"num_write":1229,"num_write_kb":43155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073040+0000","last_change":"2024-06-28T15:00:19.387598+0000","last_active":"2024-06-28T16:10:20.073040+0000","last_peered":"2024-06-28T16:10:20.073040+0000","last_clean":"2024-06-28T16:10:20.073040+0000","last_became_active":"2024-06-28T15:00:19.386645+0000","last_became_peered":"2024-06-28T15:00:19.386645+0000","last_unstale":"2024-06-28T16:10:20.073040+0000","last_undegraded":"2024-06-28T16:10:20.073040+0000","last_fullsized":"2024-06-28T16:10:20.073040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:54:00.701086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"260'289","reported_seq":873,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.977932+0000","last_change":"2024-06-28T15:00:21.395133+0000","last_active":"2024-06-28T16:14:32.977932+0000","last_peered":"2024-06-28T16:14:32.977932+0000","last_clean":"2024-06-28T16:14:32.977932+0000","last_became_active":"2024-06-28T15:00:21.394357+0000","last_became_peered":"2024-06-28T15:00:21.394357+0000","last_unstale":"2024-06-28T16:14:32.977932+0000","last_undegraded":"2024-06-28T16:14:32.977932+0000","last_fullsized":"2024-06-28T16:14:32.977932+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:35:53.429744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53697,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":496,"num_read_kb":1194,"num_write":404,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.548098+0000","last_change":"2024-06-28T15:00:23.407033+0000","last_active":"2024-06-28T16:10:25.548098+0000","last_peered":"2024-06-28T16:10:25.548098+0000","last_clean":"2024-06-28T16:10:25.548098+0000","last_became_active":"2024-06-28T15:00:23.406296+0000","last_became_peered":"2024-06-28T15:00:23.406296+0000","last_unstale":"2024-06-28T16:10:25.548098+0000","last_undegraded":"2024-06-28T16:10:25.548098+0000","last_fullsized":"2024-06-28T16:10:25.548098+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:09:16.742634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"260'68","reported_seq":636,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.670953+0000","last_change":"2024-06-28T15:00:17.397537+0000","last_active":"2024-06-28T16:14:06.670953+0000","last_peered":"2024-06-28T16:14:06.670953+0000","last_clean":"2024-06-28T16:14:06.670953+0000","last_became_active":"2024-06-28T15:00:17.396861+0000","last_became_peered":"2024-06-28T15:00:17.396861+0000","last_unstale":"2024-06-28T16:14:06.670953+0000","last_undegraded":"2024-06-28T16:14:06.670953+0000","last_fullsized":"2024-06-28T16:14:06.670953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:27:59.755447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":279,"num_read_kb":229,"num_write":93,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"260'309","reported_seq":944,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.727038+0000","last_change":"2024-06-28T15:00:15.383453+0000","last_active":"2024-06-28T16:14:34.727038+0000","last_peered":"2024-06-28T16:14:34.727038+0000","last_clean":"2024-06-28T16:14:34.727038+0000","last_became_active":"2024-06-28T15:00:15.383008+0000","last_became_peered":"2024-06-28T15:00:15.383008+0000","last_unstale":"2024-06-28T16:14:34.727038+0000","last_undegraded":"2024-06-28T16:14:34.727038+0000","last_fullsized":"2024-06-28T16:14:34.727038+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:16:34.579751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2156401,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":459,"num_read_kb":6724,"num_write":1144,"num_write_kb":24207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.550286+0000","last_change":"2024-06-28T15:00:19.390462+0000","last_active":"2024-06-28T16:10:35.550286+0000","last_peered":"2024-06-28T16:10:35.550286+0000","last_clean":"2024-06-28T16:10:35.550286+0000","last_became_active":"2024-06-28T15:00:19.390111+0000","last_became_peered":"2024-06-28T15:00:19.390111+0000","last_unstale":"2024-06-28T16:10:35.550286+0000","last_undegraded":"2024-06-28T16:10:35.550286+0000","last_fullsized":"2024-06-28T16:10:35.550286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:44:31.427541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"260'472","reported_seq":1122,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.468574+0000","last_change":"2024-06-28T15:00:21.399070+0000","last_active":"2024-06-28T16:14:34.468574+0000","last_peered":"2024-06-28T16:14:34.468574+0000","last_clean":"2024-06-28T16:14:34.468574+0000","last_became_active":"2024-06-28T15:00:21.398262+0000","last_became_peered":"2024-06-28T15:00:21.398262+0000","last_unstale":"2024-06-28T16:14:34.468574+0000","last_undegraded":"2024-06-28T16:14:34.468574+0000","last_fullsized":"2024-06-28T16:14:34.468574+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":400,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:19:49.030651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86979,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":863,"num_read_kb":2070,"num_write":734,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":493,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.067855+0000","last_change":"2024-06-28T15:00:23.411496+0000","last_active":"2024-06-28T16:10:25.067855+0000","last_peered":"2024-06-28T16:10:25.067855+0000","last_clean":"2024-06-28T16:10:25.067855+0000","last_became_active":"2024-06-28T15:00:23.410519+0000","last_became_peered":"2024-06-28T15:00:23.410519+0000","last_unstale":"2024-06-28T16:10:25.067855+0000","last_undegraded":"2024-06-28T16:10:25.067855+0000","last_fullsized":"2024-06-28T16:10:25.067855+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:02:03.245169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"260'167","reported_seq":684,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493320+0000","last_change":"2024-06-28T15:00:25.559194+0000","last_active":"2024-06-28T16:10:25.493320+0000","last_peered":"2024-06-28T16:10:25.493320+0000","last_clean":"2024-06-28T16:10:25.493320+0000","last_became_active":"2024-06-28T15:00:25.558426+0000","last_became_peered":"2024-06-28T15:00:25.558426+0000","last_unstale":"2024-06-28T16:10:25.493320+0000","last_undegraded":"2024-06-28T16:10:25.493320+0000","last_fullsized":"2024-06-28T16:10:25.493320+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":100,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:12:12.643863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":845,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":215,"num_read_kb":219,"num_write":377,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"260'158","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.550235+0000","last_change":"2024-06-28T15:00:17.395953+0000","last_active":"2024-06-28T16:10:35.550235+0000","last_peered":"2024-06-28T16:10:35.550235+0000","last_clean":"2024-06-28T16:10:35.550235+0000","last_became_active":"2024-06-28T15:00:17.395643+0000","last_became_peered":"2024-06-28T15:00:17.395643+0000","last_unstale":"2024-06-28T16:10:35.550235+0000","last_undegraded":"2024-06-28T16:10:35.550235+0000","last_fullsized":"2024-06-28T16:10:35.550235+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:17:26.238377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":654,"num_read_kb":561,"num_write":257,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"260'290","reported_seq":930,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.634482+0000","last_change":"2024-06-28T15:00:59.940407+0000","last_active":"2024-06-28T16:14:34.634482+0000","last_peered":"2024-06-28T16:14:34.634482+0000","last_clean":"2024-06-28T16:14:34.634482+0000","last_became_active":"2024-06-28T15:00:59.939485+0000","last_became_peered":"2024-06-28T15:00:59.939485+0000","last_unstale":"2024-06-28T16:14:34.634482+0000","last_undegraded":"2024-06-28T16:14:34.634482+0000","last_fullsized":"2024-06-28T16:14:34.634482+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:30:29.418898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":430,"num_read_kb":21907,"num_write":1034,"num_write_kb":35159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,6],"acting":[1,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.084186+0000","last_change":"2024-06-28T15:00:19.385764+0000","last_active":"2024-06-28T16:11:10.084186+0000","last_peered":"2024-06-28T16:11:10.084186+0000","last_clean":"2024-06-28T16:11:10.084186+0000","last_became_active":"2024-06-28T15:00:19.385068+0000","last_became_peered":"2024-06-28T15:00:19.385068+0000","last_unstale":"2024-06-28T16:11:10.084186+0000","last_undegraded":"2024-06-28T16:11:10.084186+0000","last_fullsized":"2024-06-28T16:11:10.084186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:18:40.223976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"260'348","reported_seq":954,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.054962+0000","last_change":"2024-06-28T15:00:21.401177+0000","last_active":"2024-06-28T16:14:33.054962+0000","last_peered":"2024-06-28T16:14:33.054962+0000","last_clean":"2024-06-28T16:14:33.054962+0000","last_became_active":"2024-06-28T15:00:21.400480+0000","last_became_peered":"2024-06-28T15:00:21.400480+0000","last_unstale":"2024-06-28T16:14:33.054962+0000","last_undegraded":"2024-06-28T16:14:33.054962+0000","last_fullsized":"2024-06-28T16:14:33.054962+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":300,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:08:38.525781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47708,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":581,"num_read_kb":1249,"num_write":480,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":498,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.085271+0000","last_change":"2024-06-28T15:00:23.418653+0000","last_active":"2024-06-28T16:10:25.085271+0000","last_peered":"2024-06-28T16:10:25.085271+0000","last_clean":"2024-06-28T16:10:25.085271+0000","last_became_active":"2024-06-28T15:00:23.417580+0000","last_became_peered":"2024-06-28T15:00:23.417580+0000","last_unstale":"2024-06-28T16:10:25.085271+0000","last_undegraded":"2024-06-28T16:10:25.085271+0000","last_fullsized":"2024-06-28T16:10:25.085271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:22:55.183479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"260'170","reported_seq":677,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:07.865086+0000","last_change":"2024-06-28T15:00:25.431039+0000","last_active":"2024-06-28T16:14:07.865086+0000","last_peered":"2024-06-28T16:14:07.865086+0000","last_clean":"2024-06-28T16:14:07.865086+0000","last_became_active":"2024-06-28T15:00:25.430299+0000","last_became_peered":"2024-06-28T15:00:25.430299+0000","last_unstale":"2024-06-28T16:14:07.865086+0000","last_undegraded":"2024-06-28T16:14:07.865086+0000","last_fullsized":"2024-06-28T16:14:07.865086+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:31:33.565366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2035,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":193,"num_read_kb":191,"num_write":477,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"260'315","reported_seq":1057,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.056426+0000","last_change":"2024-06-28T15:00:17.383684+0000","last_active":"2024-06-28T16:10:30.056426+0000","last_peered":"2024-06-28T16:10:30.056426+0000","last_clean":"2024-06-28T16:10:30.056426+0000","last_became_active":"2024-06-28T15:00:17.383284+0000","last_became_peered":"2024-06-28T15:00:17.383284+0000","last_unstale":"2024-06-28T16:10:30.056426+0000","last_undegraded":"2024-06-28T16:10:30.056426+0000","last_fullsized":"2024-06-28T16:10:30.056426+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:36:05.869810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1515,"num_read_kb":1335,"num_write":548,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"260'347","reported_seq":1028,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.674376+0000","last_change":"2024-06-28T15:00:15.406314+0000","last_active":"2024-06-28T16:14:34.674376+0000","last_peered":"2024-06-28T16:14:34.674376+0000","last_clean":"2024-06-28T16:14:34.674376+0000","last_became_active":"2024-06-28T15:00:15.405499+0000","last_became_peered":"2024-06-28T15:00:15.405499+0000","last_unstale":"2024-06-28T16:14:34.674376+0000","last_undegraded":"2024-06-28T16:14:34.674376+0000","last_fullsized":"2024-06-28T16:14:34.674376+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:26:47.268928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17426850,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":585,"num_read_kb":18043,"num_write":1303,"num_write_kb":33783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.074018+0000","last_change":"2024-06-28T15:00:19.381603+0000","last_active":"2024-06-28T16:10:20.074018+0000","last_peered":"2024-06-28T16:10:20.074018+0000","last_clean":"2024-06-28T16:10:20.074018+0000","last_became_active":"2024-06-28T15:00:19.378852+0000","last_became_peered":"2024-06-28T15:00:19.378852+0000","last_unstale":"2024-06-28T16:10:20.074018+0000","last_undegraded":"2024-06-28T16:10:20.074018+0000","last_fullsized":"2024-06-28T16:10:20.074018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:45:27.874936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"260'279","reported_seq":880,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.314802+0000","last_change":"2024-06-28T15:00:21.481610+0000","last_active":"2024-06-28T16:14:34.314802+0000","last_peered":"2024-06-28T16:14:34.314802+0000","last_clean":"2024-06-28T16:14:34.314802+0000","last_became_active":"2024-06-28T15:00:21.480868+0000","last_became_peered":"2024-06-28T15:00:21.480868+0000","last_unstale":"2024-06-28T16:14:34.314802+0000","last_undegraded":"2024-06-28T16:14:34.314802+0000","last_fullsized":"2024-06-28T16:14:34.314802+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":200,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:31:53.245138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40850,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":446,"num_read_kb":956,"num_write":348,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:45.089049+0000","last_change":"2024-06-28T15:00:23.401297+0000","last_active":"2024-06-28T16:10:45.089049+0000","last_peered":"2024-06-28T16:10:45.089049+0000","last_clean":"2024-06-28T16:10:45.089049+0000","last_became_active":"2024-06-28T15:00:23.400417+0000","last_became_peered":"2024-06-28T15:00:23.400417+0000","last_unstale":"2024-06-28T16:10:45.089049+0000","last_undegraded":"2024-06-28T16:10:45.089049+0000","last_fullsized":"2024-06-28T16:10:45.089049+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:57.286666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"260'117","reported_seq":617,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:12.999377+0000","last_change":"2024-06-28T15:00:25.438977+0000","last_active":"2024-06-28T16:14:12.999377+0000","last_peered":"2024-06-28T16:14:12.999377+0000","last_clean":"2024-06-28T16:14:12.999377+0000","last_became_active":"2024-06-28T15:00:25.438176+0000","last_became_peered":"2024-06-28T15:00:25.438176+0000","last_unstale":"2024-06-28T16:14:12.999377+0000","last_undegraded":"2024-06-28T16:14:12.999377+0000","last_fullsized":"2024-06-28T16:14:12.999377+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:16:21.793451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":120,"num_read_kb":111,"num_write":277,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"260'74","reported_seq":655,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.255831+0000","last_change":"2024-06-28T15:00:17.403175+0000","last_active":"2024-06-28T16:10:20.255831+0000","last_peered":"2024-06-28T16:10:20.255831+0000","last_clean":"2024-06-28T16:10:20.255831+0000","last_became_active":"2024-06-28T15:00:17.402735+0000","last_became_peered":"2024-06-28T15:00:17.402735+0000","last_unstale":"2024-06-28T16:10:20.255831+0000","last_undegraded":"2024-06-28T16:10:20.255831+0000","last_fullsized":"2024-06-28T16:10:20.255831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:31:42.762083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":299,"num_read_kb":249,"num_write":100,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"260'261","reported_seq":983,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.545545+0000","last_change":"2024-06-28T15:00:15.415658+0000","last_active":"2024-06-28T16:10:15.545545+0000","last_peered":"2024-06-28T16:10:15.545545+0000","last_clean":"2024-06-28T16:10:15.545545+0000","last_became_active":"2024-06-28T15:00:15.415197+0000","last_became_peered":"2024-06-28T15:00:15.415197+0000","last_unstale":"2024-06-28T16:10:15.545545+0000","last_undegraded":"2024-06-28T16:10:15.545545+0000","last_fullsized":"2024-06-28T16:10:15.545545+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:16:16.807609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4058475,"num_objects":80,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":573,"num_read_kb":6210,"num_write":1088,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.084302+0000","last_change":"2024-06-28T15:00:19.379957+0000","last_active":"2024-06-28T16:10:20.084302+0000","last_peered":"2024-06-28T16:10:20.084302+0000","last_clean":"2024-06-28T16:10:20.084302+0000","last_became_active":"2024-06-28T15:00:19.378270+0000","last_became_peered":"2024-06-28T15:00:19.378270+0000","last_unstale":"2024-06-28T16:10:20.084302+0000","last_undegraded":"2024-06-28T16:10:20.084302+0000","last_fullsized":"2024-06-28T16:10:20.084302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:29:00.401820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"260'101","reported_seq":646,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.625893+0000","last_change":"2024-06-28T15:00:21.485155+0000","last_active":"2024-06-28T16:11:03.625893+0000","last_peered":"2024-06-28T16:11:03.625893+0000","last_clean":"2024-06-28T16:11:03.625893+0000","last_became_active":"2024-06-28T15:00:21.484352+0000","last_became_peered":"2024-06-28T15:00:21.484352+0000","last_unstale":"2024-06-28T16:11:03.625893+0000","last_undegraded":"2024-06-28T16:11:03.625893+0000","last_fullsized":"2024-06-28T16:11:03.625893+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:35:15.271971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":124,"num_read_kb":119,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"260'515","reported_seq":2245,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.188813+0000","last_change":"2024-06-28T15:00:23.415013+0000","last_active":"2024-06-28T16:14:32.188813+0000","last_peered":"2024-06-28T16:14:32.188813+0000","last_clean":"2024-06-28T16:14:32.188813+0000","last_became_active":"2024-06-28T15:00:23.407490+0000","last_became_peered":"2024-06-28T15:00:23.407490+0000","last_unstale":"2024-06-28T16:14:32.188813+0000","last_undegraded":"2024-06-28T16:14:32.188813+0000","last_fullsized":"2024-06-28T16:14:32.188813+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:17:37.321606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"260'102","reported_seq":606,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.032920+0000","last_change":"2024-06-28T15:00:25.446074+0000","last_active":"2024-06-28T16:14:06.032920+0000","last_peered":"2024-06-28T16:14:06.032920+0000","last_clean":"2024-06-28T16:14:06.032920+0000","last_became_active":"2024-06-28T15:00:25.445018+0000","last_became_peered":"2024-06-28T15:00:25.445018+0000","last_unstale":"2024-06-28T16:14:06.032920+0000","last_undegraded":"2024-06-28T16:14:06.032920+0000","last_fullsized":"2024-06-28T16:14:06.032920+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:03:17.977593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":117,"num_read_kb":111,"num_write":241,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"260'78","reported_seq":654,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073934+0000","last_change":"2024-06-28T15:00:17.394823+0000","last_active":"2024-06-28T16:10:20.073934+0000","last_peered":"2024-06-28T16:10:20.073934+0000","last_clean":"2024-06-28T16:10:20.073934+0000","last_became_active":"2024-06-28T15:00:17.387086+0000","last_became_peered":"2024-06-28T15:00:17.387086+0000","last_unstale":"2024-06-28T16:10:20.073934+0000","last_undegraded":"2024-06-28T16:10:20.073934+0000","last_fullsized":"2024-06-28T16:10:20.073934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:21:49.516534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":333,"num_read_kb":275,"num_write":113,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"260'328","reported_seq":960,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.721439+0000","last_change":"2024-06-28T15:00:59.943256+0000","last_active":"2024-06-28T16:14:34.721439+0000","last_peered":"2024-06-28T16:14:34.721439+0000","last_clean":"2024-06-28T16:14:34.721439+0000","last_became_active":"2024-06-28T15:00:59.941915+0000","last_became_peered":"2024-06-28T15:00:59.941915+0000","last_unstale":"2024-06-28T16:14:34.721439+0000","last_undegraded":"2024-06-28T16:14:34.721439+0000","last_fullsized":"2024-06-28T16:14:34.721439+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:05:07.672939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5506084,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":497,"num_read_kb":57892,"num_write":1218,"num_write_kb":47404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080513+0000","last_change":"2024-06-28T15:00:19.391494+0000","last_active":"2024-06-28T16:10:20.080513+0000","last_peered":"2024-06-28T16:10:20.080513+0000","last_clean":"2024-06-28T16:10:20.080513+0000","last_became_active":"2024-06-28T15:00:19.390613+0000","last_became_peered":"2024-06-28T15:00:19.390613+0000","last_unstale":"2024-06-28T16:10:20.080513+0000","last_undegraded":"2024-06-28T16:10:20.080513+0000","last_fullsized":"2024-06-28T16:10:20.080513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:27:57.018874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"260'324","reported_seq":932,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.759325+0000","last_change":"2024-06-28T15:00:21.392375+0000","last_active":"2024-06-28T16:14:34.759325+0000","last_peered":"2024-06-28T16:14:34.759325+0000","last_clean":"2024-06-28T16:14:34.759325+0000","last_became_active":"2024-06-28T15:00:21.391631+0000","last_became_peered":"2024-06-28T15:00:21.391631+0000","last_unstale":"2024-06-28T16:14:34.759325+0000","last_undegraded":"2024-06-28T16:14:34.759325+0000","last_fullsized":"2024-06-28T16:14:34.759325+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:25:50.181425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43414,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":516,"num_read_kb":1105,"num_write":400,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.549103+0000","last_change":"2024-06-28T15:00:23.405246+0000","last_active":"2024-06-28T16:10:30.549103+0000","last_peered":"2024-06-28T16:10:30.549103+0000","last_clean":"2024-06-28T16:10:30.549103+0000","last_became_active":"2024-06-28T15:00:23.404312+0000","last_became_peered":"2024-06-28T15:00:23.404312+0000","last_unstale":"2024-06-28T16:10:30.549103+0000","last_undegraded":"2024-06-28T16:10:30.549103+0000","last_fullsized":"2024-06-28T16:10:30.549103+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:40.584815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"260'131","reported_seq":642,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.235400+0000","last_change":"2024-06-28T15:00:25.425886+0000","last_active":"2024-06-28T16:14:08.235400+0000","last_peered":"2024-06-28T16:14:08.235400+0000","last_clean":"2024-06-28T16:14:08.235400+0000","last_became_active":"2024-06-28T15:00:25.424826+0000","last_became_peered":"2024-06-28T15:00:25.424826+0000","last_unstale":"2024-06-28T16:14:08.235400+0000","last_undegraded":"2024-06-28T16:14:08.235400+0000","last_fullsized":"2024-06-28T16:14:08.235400+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:56:01.501372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":159,"num_read_kb":144,"num_write":309,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"260'118","reported_seq":691,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073868+0000","last_change":"2024-06-28T15:00:17.387490+0000","last_active":"2024-06-28T16:10:20.073868+0000","last_peered":"2024-06-28T16:10:20.073868+0000","last_clean":"2024-06-28T16:10:20.073868+0000","last_became_active":"2024-06-28T15:00:17.386756+0000","last_became_peered":"2024-06-28T15:00:17.386756+0000","last_unstale":"2024-06-28T16:10:20.073868+0000","last_undegraded":"2024-06-28T16:10:20.073868+0000","last_fullsized":"2024-06-28T16:10:20.073868+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:24:23.283000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":469,"num_read_kb":378,"num_write":166,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"260'293","reported_seq":964,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.754554+0000","last_change":"2024-06-28T15:00:59.945598+0000","last_active":"2024-06-28T16:14:34.754554+0000","last_peered":"2024-06-28T16:14:34.754554+0000","last_clean":"2024-06-28T16:14:34.754554+0000","last_became_active":"2024-06-28T15:00:59.943670+0000","last_became_peered":"2024-06-28T15:00:59.943670+0000","last_unstale":"2024-06-28T16:14:34.754554+0000","last_undegraded":"2024-06-28T16:14:34.754554+0000","last_fullsized":"2024-06-28T16:14:34.754554+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:28:33.955349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306233,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":513,"num_read_kb":66858,"num_write":1091,"num_write_kb":26064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.066682+0000","last_change":"2024-06-28T15:00:19.400037+0000","last_active":"2024-06-28T16:10:20.066682+0000","last_peered":"2024-06-28T16:10:20.066682+0000","last_clean":"2024-06-28T16:10:20.066682+0000","last_became_active":"2024-06-28T15:00:19.398245+0000","last_became_peered":"2024-06-28T15:00:19.398245+0000","last_unstale":"2024-06-28T16:10:20.066682+0000","last_undegraded":"2024-06-28T16:10:20.066682+0000","last_fullsized":"2024-06-28T16:10:20.066682+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:50.601079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"260'292","reported_seq":877,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.720679+0000","last_change":"2024-06-28T15:00:21.406496+0000","last_active":"2024-06-28T16:14:06.720679+0000","last_peered":"2024-06-28T16:14:06.720679+0000","last_clean":"2024-06-28T16:14:06.720679+0000","last_became_active":"2024-06-28T15:00:21.404701+0000","last_became_peered":"2024-06-28T15:00:21.404701+0000","last_unstale":"2024-06-28T16:14:06.720679+0000","last_undegraded":"2024-06-28T16:14:06.720679+0000","last_fullsized":"2024-06-28T16:14:06.720679+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":200,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:35:29.922369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59181,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":473,"num_read_kb":1000,"num_write":388,"num_write_kb":303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":491,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:50.079705+0000","last_change":"2024-06-28T15:00:23.416260+0000","last_active":"2024-06-28T16:10:50.079705+0000","last_peered":"2024-06-28T16:10:50.079705+0000","last_clean":"2024-06-28T16:10:50.079705+0000","last_became_active":"2024-06-28T15:00:23.415531+0000","last_became_peered":"2024-06-28T15:00:23.415531+0000","last_unstale":"2024-06-28T16:10:50.079705+0000","last_undegraded":"2024-06-28T16:10:50.079705+0000","last_fullsized":"2024-06-28T16:10:50.079705+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:14:35.301127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"260'166","reported_seq":710,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:02.116164+0000","last_change":"2024-06-28T15:00:25.554319+0000","last_active":"2024-06-28T16:13:02.116164+0000","last_peered":"2024-06-28T16:13:02.116164+0000","last_clean":"2024-06-28T16:13:02.116164+0000","last_became_active":"2024-06-28T15:00:25.553724+0000","last_became_peered":"2024-06-28T15:00:25.553724+0000","last_unstale":"2024-06-28T16:13:02.116164+0000","last_undegraded":"2024-06-28T16:13:02.116164+0000","last_fullsized":"2024-06-28T16:13:02.116164+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:16:48.260247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":555,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":217,"num_write":397,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"260'143","reported_seq":806,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.494648+0000","last_change":"2024-06-28T15:00:17.383713+0000","last_active":"2024-06-28T16:10:30.494648+0000","last_peered":"2024-06-28T16:10:30.494648+0000","last_clean":"2024-06-28T16:10:30.494648+0000","last_became_active":"2024-06-28T15:00:17.383224+0000","last_became_peered":"2024-06-28T15:00:17.383224+0000","last_unstale":"2024-06-28T16:10:30.494648+0000","last_undegraded":"2024-06-28T16:10:30.494648+0000","last_fullsized":"2024-06-28T16:10:30.494648+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:23:29.150410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":649,"num_read_kb":545,"num_write":214,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"260'301","reported_seq":1066,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.691517+0000","last_change":"2024-06-28T15:00:15.424150+0000","last_active":"2024-06-28T16:14:34.691517+0000","last_peered":"2024-06-28T16:14:34.691517+0000","last_clean":"2024-06-28T16:14:34.691517+0000","last_became_active":"2024-06-28T15:00:15.423813+0000","last_became_peered":"2024-06-28T15:00:15.423813+0000","last_unstale":"2024-06-28T16:14:34.691517+0000","last_undegraded":"2024-06-28T16:14:34.691517+0000","last_fullsized":"2024-06-28T16:14:34.691517+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:57:01.721732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5724260,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":754,"num_read_kb":37474,"num_write":1107,"num_write_kb":31583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.075092+0000","last_change":"2024-06-28T15:00:19.384145+0000","last_active":"2024-06-28T16:10:25.075092+0000","last_peered":"2024-06-28T16:10:25.075092+0000","last_clean":"2024-06-28T16:10:25.075092+0000","last_became_active":"2024-06-28T15:00:19.383307+0000","last_became_peered":"2024-06-28T15:00:19.383307+0000","last_unstale":"2024-06-28T16:10:25.075092+0000","last_undegraded":"2024-06-28T16:10:25.075092+0000","last_fullsized":"2024-06-28T16:10:25.075092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:22:34.590726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"260'286","reported_seq":857,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.176534+0000","last_change":"2024-06-28T15:00:21.396994+0000","last_active":"2024-06-28T16:14:34.176534+0000","last_peered":"2024-06-28T16:14:34.176534+0000","last_clean":"2024-06-28T16:14:34.176534+0000","last_became_active":"2024-06-28T15:00:21.396257+0000","last_became_peered":"2024-06-28T15:00:21.396257+0000","last_unstale":"2024-06-28T16:14:34.176534+0000","last_undegraded":"2024-06-28T16:14:34.176534+0000","last_fullsized":"2024-06-28T16:14:34.176534+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:05:02.532173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43992,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":439,"num_read_kb":1078,"num_write":366,"num_write_kb":297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":494,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.075301+0000","last_change":"2024-06-28T15:00:23.410088+0000","last_active":"2024-06-28T16:10:25.075301+0000","last_peered":"2024-06-28T16:10:25.075301+0000","last_clean":"2024-06-28T16:10:25.075301+0000","last_became_active":"2024-06-28T15:00:23.409107+0000","last_became_peered":"2024-06-28T15:00:23.409107+0000","last_unstale":"2024-06-28T16:10:25.075301+0000","last_undegraded":"2024-06-28T16:10:25.075301+0000","last_fullsized":"2024-06-28T16:10:25.075301+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:26:16.658429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"260'144","reported_seq":876,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:02.732078+0000","last_change":"2024-06-28T15:00:25.425176+0000","last_active":"2024-06-28T16:14:02.732078+0000","last_peered":"2024-06-28T16:14:02.732078+0000","last_clean":"2024-06-28T16:14:02.732078+0000","last_became_active":"2024-06-28T15:00:25.424376+0000","last_became_peered":"2024-06-28T15:00:25.424376+0000","last_unstale":"2024-06-28T16:14:02.732078+0000","last_undegraded":"2024-06-28T16:14:02.732078+0000","last_fullsized":"2024-06-28T16:14:02.732078+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:40:22.841457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":398,"num_read_kb":429,"num_write":339,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"260'112","reported_seq":699,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.075140+0000","last_change":"2024-06-28T15:00:17.403746+0000","last_active":"2024-06-28T16:10:25.075140+0000","last_peered":"2024-06-28T16:10:25.075140+0000","last_clean":"2024-06-28T16:10:25.075140+0000","last_became_active":"2024-06-28T15:00:17.403007+0000","last_became_peered":"2024-06-28T15:00:17.403007+0000","last_unstale":"2024-06-28T16:10:25.075140+0000","last_undegraded":"2024-06-28T16:10:25.075140+0000","last_fullsized":"2024-06-28T16:10:25.075140+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:10:54.255627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":478,"num_read_kb":386,"num_write":182,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"260'311","reported_seq":978,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.556341+0000","last_change":"2024-06-28T15:00:15.415976+0000","last_active":"2024-06-28T16:11:05.556341+0000","last_peered":"2024-06-28T16:11:05.556341+0000","last_clean":"2024-06-28T16:11:05.556341+0000","last_became_active":"2024-06-28T15:00:15.414969+0000","last_became_peered":"2024-06-28T15:00:15.414969+0000","last_unstale":"2024-06-28T16:11:05.556341+0000","last_undegraded":"2024-06-28T16:11:05.556341+0000","last_fullsized":"2024-06-28T16:11:05.556341+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:03:09.425476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6850286,"num_objects":77,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":471,"num_read_kb":3182,"num_write":1253,"num_write_kb":13350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"24'1","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:45.088935+0000","last_change":"2024-06-28T15:00:19.403928+0000","last_active":"2024-06-28T16:10:45.088935+0000","last_peered":"2024-06-28T16:10:45.088935+0000","last_clean":"2024-06-28T16:10:45.088935+0000","last_became_active":"2024-06-28T15:00:19.403344+0000","last_became_peered":"2024-06-28T15:00:19.403344+0000","last_unstale":"2024-06-28T16:10:45.088935+0000","last_undegraded":"2024-06-28T16:10:45.088935+0000","last_fullsized":"2024-06-28T16:10:45.088935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:09:50.417696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"260'293","reported_seq":893,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.393958+0000","last_change":"2024-06-28T15:00:21.388828+0000","last_active":"2024-06-28T16:14:34.393958+0000","last_peered":"2024-06-28T16:14:34.393958+0000","last_clean":"2024-06-28T16:14:34.393958+0000","last_became_active":"2024-06-28T15:00:21.387539+0000","last_became_peered":"2024-06-28T15:00:21.387539+0000","last_unstale":"2024-06-28T16:14:34.393958+0000","last_undegraded":"2024-06-28T16:14:34.393958+0000","last_fullsized":"2024-06-28T16:14:34.393958+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:54:05.779834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44825,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":494,"num_read_kb":1086,"num_write":388,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":499,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.548062+0000","last_change":"2024-06-28T15:00:23.417454+0000","last_active":"2024-06-28T16:10:25.548062+0000","last_peered":"2024-06-28T16:10:25.548062+0000","last_clean":"2024-06-28T16:10:25.548062+0000","last_became_active":"2024-06-28T15:00:23.416363+0000","last_became_peered":"2024-06-28T15:00:23.416363+0000","last_unstale":"2024-06-28T16:10:25.548062+0000","last_undegraded":"2024-06-28T16:10:25.548062+0000","last_fullsized":"2024-06-28T16:10:25.548062+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:42:57.225974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"260'126","reported_seq":623,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:05.111348+0000","last_change":"2024-06-28T15:00:25.429682+0000","last_active":"2024-06-28T16:13:05.111348+0000","last_peered":"2024-06-28T16:13:05.111348+0000","last_clean":"2024-06-28T16:13:05.111348+0000","last_became_active":"2024-06-28T15:00:25.428959+0000","last_became_peered":"2024-06-28T15:00:25.428959+0000","last_unstale":"2024-06-28T16:13:05.111348+0000","last_undegraded":"2024-06-28T16:13:05.111348+0000","last_fullsized":"2024-06-28T16:13:05.111348+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:34:41.858483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":814,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":97,"num_write":299,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"260'174","reported_seq":761,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:39.929106+0000","last_change":"2024-06-28T15:00:17.407419+0000","last_active":"2024-06-28T16:13:39.929106+0000","last_peered":"2024-06-28T16:13:39.929106+0000","last_clean":"2024-06-28T16:13:39.929106+0000","last_became_active":"2024-06-28T15:00:17.406872+0000","last_became_peered":"2024-06-28T15:00:17.406872+0000","last_unstale":"2024-06-28T16:13:39.929106+0000","last_undegraded":"2024-06-28T16:13:39.929106+0000","last_fullsized":"2024-06-28T16:13:39.929106+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:52:17.662914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":660,"num_read_kb":511,"num_write":250,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"260'285","reported_seq":909,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.644073+0000","last_change":"2024-06-28T15:00:15.409170+0000","last_active":"2024-06-28T16:14:34.644073+0000","last_peered":"2024-06-28T16:14:34.644073+0000","last_clean":"2024-06-28T16:14:34.644073+0000","last_became_active":"2024-06-28T15:00:15.408337+0000","last_became_peered":"2024-06-28T15:00:15.408337+0000","last_unstale":"2024-06-28T16:14:34.644073+0000","last_undegraded":"2024-06-28T16:14:34.644073+0000","last_fullsized":"2024-06-28T16:14:34.644073+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:51:23.373884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":404,"num_read_kb":33775,"num_write":1000,"num_write_kb":10945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"260'72","reported_seq":674,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.102569+0000","last_change":"2024-06-28T15:00:17.404713+0000","last_active":"2024-06-28T16:11:50.102569+0000","last_peered":"2024-06-28T16:11:50.102569+0000","last_clean":"2024-06-28T16:11:50.102569+0000","last_became_active":"2024-06-28T15:00:17.403938+0000","last_became_peered":"2024-06-28T15:00:17.403938+0000","last_unstale":"2024-06-28T16:11:50.102569+0000","last_undegraded":"2024-06-28T16:11:50.102569+0000","last_fullsized":"2024-06-28T16:11:50.102569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:54:35.966683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":319,"num_read_kb":277,"num_write":96,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"260'322","reported_seq":995,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.576974+0000","last_change":"2024-06-28T15:00:15.420697+0000","last_active":"2024-06-28T16:14:34.576974+0000","last_peered":"2024-06-28T16:14:34.576974+0000","last_clean":"2024-06-28T16:14:34.576974+0000","last_became_active":"2024-06-28T15:00:15.420261+0000","last_became_peered":"2024-06-28T15:00:15.420261+0000","last_unstale":"2024-06-28T16:14:34.576974+0000","last_undegraded":"2024-06-28T16:14:34.576974+0000","last_fullsized":"2024-06-28T16:14:34.576974+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:57:26.418017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262189,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":529,"num_read_kb":23662,"num_write":1259,"num_write_kb":28174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"260'38","reported_seq":596,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.087084+0000","last_change":"2024-06-28T15:00:17.394654+0000","last_active":"2024-06-28T16:11:50.087084+0000","last_peered":"2024-06-28T16:11:50.087084+0000","last_clean":"2024-06-28T16:11:50.087084+0000","last_became_active":"2024-06-28T15:00:17.393889+0000","last_became_peered":"2024-06-28T15:00:17.393889+0000","last_unstale":"2024-06-28T16:11:50.087084+0000","last_undegraded":"2024-06-28T16:11:50.087084+0000","last_fullsized":"2024-06-28T16:11:50.087084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:22:39.330270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":152,"num_write":58,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"260'358","reported_seq":999,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.728265+0000","last_change":"2024-06-28T15:00:15.426603+0000","last_active":"2024-06-28T16:14:34.728265+0000","last_peered":"2024-06-28T16:14:34.728265+0000","last_clean":"2024-06-28T16:14:34.728265+0000","last_became_active":"2024-06-28T15:00:15.418773+0000","last_became_peered":"2024-06-28T15:00:15.418773+0000","last_unstale":"2024-06-28T16:14:34.728265+0000","last_undegraded":"2024-06-28T16:14:34.728265+0000","last_fullsized":"2024-06-28T16:14:34.728265+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:02:14.247776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8714448,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":553,"num_read_kb":16492,"num_write":1257,"num_write_kb":37641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"260'142","reported_seq":787,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:07.183113+0000","last_change":"2024-06-28T15:00:17.398608+0000","last_active":"2024-06-28T16:13:07.183113+0000","last_peered":"2024-06-28T16:13:07.183113+0000","last_clean":"2024-06-28T16:13:07.183113+0000","last_became_active":"2024-06-28T15:00:17.398357+0000","last_became_peered":"2024-06-28T15:00:17.398357+0000","last_unstale":"2024-06-28T16:13:07.183113+0000","last_undegraded":"2024-06-28T16:13:07.183113+0000","last_fullsized":"2024-06-28T16:13:07.183113+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:53:58.893939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":623,"num_read_kb":529,"num_write":221,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"260'269","reported_seq":913,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:09.848378+0000","last_change":"2024-06-28T15:00:15.391599+0000","last_active":"2024-06-28T16:12:09.848378+0000","last_peered":"2024-06-28T16:12:09.848378+0000","last_clean":"2024-06-28T16:12:09.848378+0000","last_became_active":"2024-06-28T15:00:15.391150+0000","last_became_peered":"2024-06-28T15:00:15.391150+0000","last_unstale":"2024-06-28T16:12:09.848378+0000","last_undegraded":"2024-06-28T16:12:09.848378+0000","last_fullsized":"2024-06-28T16:12:09.848378+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:12:30.284233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6905440,"num_objects":72,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":390,"num_read_kb":84462,"num_write":1202,"num_write_kb":41774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"260'191","reported_seq":788,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:05.276690+0000","last_change":"2024-06-28T15:00:59.807172+0000","last_active":"2024-06-28T16:12:05.276690+0000","last_peered":"2024-06-28T16:12:05.276690+0000","last_clean":"2024-06-28T16:12:05.276690+0000","last_became_active":"2024-06-28T15:00:59.806725+0000","last_became_peered":"2024-06-28T15:00:59.806725+0000","last_unstale":"2024-06-28T16:12:05.276690+0000","last_undegraded":"2024-06-28T16:12:05.276690+0000","last_fullsized":"2024-06-28T16:12:05.276690+0000","mapping_epoch":32,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:00:01.418110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":759,"num_read_kb":602,"num_write":307,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"260'334","reported_seq":998,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.618192+0000","last_change":"2024-06-28T15:00:15.383729+0000","last_active":"2024-06-28T16:14:34.618192+0000","last_peered":"2024-06-28T16:14:34.618192+0000","last_clean":"2024-06-28T16:14:34.618192+0000","last_became_active":"2024-06-28T15:00:15.383350+0000","last_became_peered":"2024-06-28T15:00:15.383350+0000","last_unstale":"2024-06-28T16:14:34.618192+0000","last_undegraded":"2024-06-28T16:14:34.618192+0000","last_fullsized":"2024-06-28T16:14:34.618192+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:22:33.747727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263219,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":502,"num_read_kb":41336,"num_write":1201,"num_write_kb":38478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0,7],"acting":[3,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"260'397","reported_seq":997,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.085126+0000","last_change":"2024-06-28T15:00:17.408600+0000","last_active":"2024-06-28T16:11:10.085126+0000","last_peered":"2024-06-28T16:11:10.085126+0000","last_clean":"2024-06-28T16:11:10.085126+0000","last_became_active":"2024-06-28T15:00:17.404844+0000","last_became_peered":"2024-06-28T15:00:17.404844+0000","last_unstale":"2024-06-28T16:11:10.085126+0000","last_undegraded":"2024-06-28T16:11:10.085126+0000","last_fullsized":"2024-06-28T16:11:10.085126+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":300,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:17:34.507155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1527,"num_read_kb":1156,"num_write":658,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"260'343","reported_seq":1027,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.657138+0000","last_change":"2024-06-28T15:00:15.401905+0000","last_active":"2024-06-28T16:14:34.657138+0000","last_peered":"2024-06-28T16:14:34.657138+0000","last_clean":"2024-06-28T16:14:34.657138+0000","last_became_active":"2024-06-28T15:00:15.401275+0000","last_became_peered":"2024-06-28T15:00:15.401275+0000","last_unstale":"2024-06-28T16:14:34.657138+0000","last_undegraded":"2024-06-28T16:14:34.657138+0000","last_fullsized":"2024-06-28T16:14:34.657138+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:00:05.589080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":559,"num_read_kb":45669,"num_write":1313,"num_write_kb":19663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"260'221","reported_seq":841,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.504084+0000","last_change":"2024-06-28T15:00:17.383279+0000","last_active":"2024-06-28T16:11:10.504084+0000","last_peered":"2024-06-28T16:11:10.504084+0000","last_clean":"2024-06-28T16:11:10.504084+0000","last_became_active":"2024-06-28T15:00:17.382739+0000","last_became_peered":"2024-06-28T15:00:17.382739+0000","last_unstale":"2024-06-28T16:11:10.504084+0000","last_undegraded":"2024-06-28T16:11:10.504084+0000","last_fullsized":"2024-06-28T16:11:10.504084+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:36:13.087157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":866,"num_read_kb":690,"num_write":335,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"260'298","reported_seq":928,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.600936+0000","last_change":"2024-06-28T15:00:15.421880+0000","last_active":"2024-06-28T16:14:34.600936+0000","last_peered":"2024-06-28T16:14:34.600936+0000","last_clean":"2024-06-28T16:14:34.600936+0000","last_became_active":"2024-06-28T15:00:15.421189+0000","last_became_peered":"2024-06-28T15:00:15.421189+0000","last_unstale":"2024-06-28T16:14:34.600936+0000","last_undegraded":"2024-06-28T16:14:34.600936+0000","last_fullsized":"2024-06-28T16:14:34.600936+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:03:21.461240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7105903,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":419,"num_read_kb":19450,"num_write":1148,"num_write_kb":38199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"260'79","reported_seq":639,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.083923+0000","last_change":"2024-06-28T15:00:17.396892+0000","last_active":"2024-06-28T16:11:10.083923+0000","last_peered":"2024-06-28T16:11:10.083923+0000","last_clean":"2024-06-28T16:11:10.083923+0000","last_became_active":"2024-06-28T15:00:17.396125+0000","last_became_peered":"2024-06-28T15:00:17.396125+0000","last_unstale":"2024-06-28T16:11:10.083923+0000","last_undegraded":"2024-06-28T16:11:10.083923+0000","last_fullsized":"2024-06-28T16:11:10.083923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:46:54.077428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":321,"num_read_kb":257,"num_write":118,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"260'270","reported_seq":927,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.735427+0000","last_change":"2024-06-28T15:00:15.408891+0000","last_active":"2024-06-28T16:14:34.735427+0000","last_peered":"2024-06-28T16:14:34.735427+0000","last_clean":"2024-06-28T16:14:34.735427+0000","last_became_active":"2024-06-28T15:00:15.408375+0000","last_became_peered":"2024-06-28T15:00:15.408375+0000","last_unstale":"2024-06-28T16:14:34.735427+0000","last_undegraded":"2024-06-28T16:14:34.735427+0000","last_fullsized":"2024-06-28T16:14:34.735427+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:29:43.369904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":484,"num_read_kb":6587,"num_write":1005,"num_write_kb":17135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"260'62","reported_seq":636,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:08.869032+0000","last_change":"2024-06-28T15:00:17.380077+0000","last_active":"2024-06-28T16:12:08.869032+0000","last_peered":"2024-06-28T16:12:08.869032+0000","last_clean":"2024-06-28T16:12:08.869032+0000","last_became_active":"2024-06-28T15:00:17.379374+0000","last_became_peered":"2024-06-28T15:00:17.379374+0000","last_unstale":"2024-06-28T16:12:08.869032+0000","last_undegraded":"2024-06-28T16:12:08.869032+0000","last_fullsized":"2024-06-28T16:12:08.869032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:19:27.611751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":216,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"260'367","reported_seq":1011,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.749490+0000","last_change":"2024-06-28T15:00:15.427903+0000","last_active":"2024-06-28T16:14:34.749490+0000","last_peered":"2024-06-28T16:14:34.749490+0000","last_clean":"2024-06-28T16:14:34.749490+0000","last_became_active":"2024-06-28T15:00:15.427171+0000","last_became_peered":"2024-06-28T15:00:15.427171+0000","last_unstale":"2024-06-28T16:14:34.749490+0000","last_undegraded":"2024-06-28T16:14:34.749490+0000","last_fullsized":"2024-06-28T16:14:34.749490+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":300,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:22:01.482231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":723921,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":536,"num_read_kb":26554,"num_write":1152,"num_write_kb":52254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"260'230","reported_seq":851,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.074074+0000","last_change":"2024-06-28T15:00:17.409608+0000","last_active":"2024-06-28T16:10:20.074074+0000","last_peered":"2024-06-28T16:10:20.074074+0000","last_clean":"2024-06-28T16:10:20.074074+0000","last_became_active":"2024-06-28T15:00:17.403760+0000","last_became_peered":"2024-06-28T15:00:17.403760+0000","last_unstale":"2024-06-28T16:10:20.074074+0000","last_undegraded":"2024-06-28T16:10:20.074074+0000","last_fullsized":"2024-06-28T16:10:20.074074+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:58:04.722934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":902,"num_read_kb":724,"num_write":338,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"260'282","reported_seq":929,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.669616+0000","last_change":"2024-06-28T15:00:15.376797+0000","last_active":"2024-06-28T16:14:34.669616+0000","last_peered":"2024-06-28T16:14:34.669616+0000","last_clean":"2024-06-28T16:14:34.669616+0000","last_became_active":"2024-06-28T15:00:15.376014+0000","last_became_peered":"2024-06-28T15:00:15.376014+0000","last_unstale":"2024-06-28T16:14:34.669616+0000","last_undegraded":"2024-06-28T16:14:34.669616+0000","last_fullsized":"2024-06-28T16:14:34.669616+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:31:24.117797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462789,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":409,"num_read_kb":32504,"num_write":910,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"260'70","reported_seq":658,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.255871+0000","last_change":"2024-06-28T15:00:17.402274+0000","last_active":"2024-06-28T16:10:20.255871+0000","last_peered":"2024-06-28T16:10:20.255871+0000","last_clean":"2024-06-28T16:10:20.255871+0000","last_became_active":"2024-06-28T15:00:17.401917+0000","last_became_peered":"2024-06-28T15:00:17.401917+0000","last_unstale":"2024-06-28T16:10:20.255871+0000","last_undegraded":"2024-06-28T16:10:20.255871+0000","last_fullsized":"2024-06-28T16:10:20.255871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:48:15.974353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":295,"num_read_kb":247,"num_write":98,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"260'337","reported_seq":973,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.694565+0000","last_change":"2024-06-28T15:00:15.434365+0000","last_active":"2024-06-28T16:14:34.694565+0000","last_peered":"2024-06-28T16:14:34.694565+0000","last_clean":"2024-06-28T16:14:34.694565+0000","last_became_active":"2024-06-28T15:00:15.431444+0000","last_became_peered":"2024-06-28T15:00:15.431444+0000","last_unstale":"2024-06-28T16:14:34.694565+0000","last_undegraded":"2024-06-28T16:14:34.694565+0000","last_fullsized":"2024-06-28T16:14:34.694565+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:01:51.800790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":461,"num_read_kb":557,"num_write":1266,"num_write_kb":33580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,3],"acting":[5,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"260'424","reported_seq":1178,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101339+0000","last_change":"2024-06-28T15:00:17.403422+0000","last_active":"2024-06-28T16:12:20.101339+0000","last_peered":"2024-06-28T16:12:20.101339+0000","last_clean":"2024-06-28T16:12:20.101339+0000","last_became_active":"2024-06-28T15:00:17.402842+0000","last_became_peered":"2024-06-28T15:00:17.402842+0000","last_unstale":"2024-06-28T16:12:20.101339+0000","last_undegraded":"2024-06-28T16:12:20.101339+0000","last_fullsized":"2024-06-28T16:12:20.101339+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":400,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:26:05.866773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1825,"num_read_kb":1465,"num_write":755,"num_write_kb":468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"260'261","reported_seq":906,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.574001+0000","last_change":"2024-06-28T15:00:15.415547+0000","last_active":"2024-06-28T16:12:15.574001+0000","last_peered":"2024-06-28T16:12:15.574001+0000","last_clean":"2024-06-28T16:12:15.574001+0000","last_became_active":"2024-06-28T15:00:15.415009+0000","last_became_peered":"2024-06-28T15:00:15.415009+0000","last_unstale":"2024-06-28T16:12:15.574001+0000","last_undegraded":"2024-06-28T16:12:15.574001+0000","last_fullsized":"2024-06-28T16:12:15.574001+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:36:33.533042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5245699,"num_objects":64,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":421,"num_read_kb":2084,"num_write":1144,"num_write_kb":29565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"260'26","reported_seq":574,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.108218+0000","last_change":"2024-06-28T15:00:17.390583+0000","last_active":"2024-06-28T16:12:20.108218+0000","last_peered":"2024-06-28T16:12:20.108218+0000","last_clean":"2024-06-28T16:12:20.108218+0000","last_became_active":"2024-06-28T15:00:17.389844+0000","last_became_peered":"2024-06-28T15:00:17.389844+0000","last_unstale":"2024-06-28T16:12:20.108218+0000","last_undegraded":"2024-06-28T16:12:20.108218+0000","last_fullsized":"2024-06-28T16:12:20.108218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:41:17.401424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":112,"num_write":34,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"260'305","reported_seq":936,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.719398+0000","last_change":"2024-06-28T15:00:15.398948+0000","last_active":"2024-06-28T16:14:34.719398+0000","last_peered":"2024-06-28T16:14:34.719398+0000","last_clean":"2024-06-28T16:14:34.719398+0000","last_became_active":"2024-06-28T15:00:15.397986+0000","last_became_peered":"2024-06-28T15:00:15.397986+0000","last_unstale":"2024-06-28T16:14:34.719398+0000","last_undegraded":"2024-06-28T16:14:34.719398+0000","last_fullsized":"2024-06-28T16:14:34.719398+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:02:19.841719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079926,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":433,"num_read_kb":10373,"num_write":1047,"num_write_kb":25421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"260'192","reported_seq":856,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.822857+0000","last_change":"2024-06-28T15:00:17.395455+0000","last_active":"2024-06-28T16:12:20.822857+0000","last_peered":"2024-06-28T16:12:20.822857+0000","last_clean":"2024-06-28T16:12:20.822857+0000","last_became_active":"2024-06-28T15:00:17.394783+0000","last_became_peered":"2024-06-28T15:00:17.394783+0000","last_unstale":"2024-06-28T16:12:20.822857+0000","last_undegraded":"2024-06-28T16:12:20.822857+0000","last_fullsized":"2024-06-28T16:12:20.822857+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:54:50.516422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":852,"num_read_kb":715,"num_write":335,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"260'302","reported_seq":913,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.681222+0000","last_change":"2024-06-28T15:00:59.939157+0000","last_active":"2024-06-28T16:14:34.681222+0000","last_peered":"2024-06-28T16:14:34.681222+0000","last_clean":"2024-06-28T16:14:34.681222+0000","last_became_active":"2024-06-28T15:00:59.938004+0000","last_became_peered":"2024-06-28T15:00:59.938004+0000","last_unstale":"2024-06-28T16:14:34.681222+0000","last_undegraded":"2024-06-28T16:14:34.681222+0000","last_fullsized":"2024-06-28T16:14:34.681222+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:35:45.871383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6542288,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":437,"num_read_kb":25077,"num_write":1246,"num_write_kb":38037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,3],"acting":[1,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"260'392","reported_seq":1062,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.520627+0000","last_change":"2024-06-28T15:00:17.374446+0000","last_active":"2024-06-28T16:12:20.520627+0000","last_peered":"2024-06-28T16:12:20.520627+0000","last_clean":"2024-06-28T16:12:20.520627+0000","last_became_active":"2024-06-28T15:00:17.374143+0000","last_became_peered":"2024-06-28T15:00:17.374143+0000","last_unstale":"2024-06-28T16:12:20.520627+0000","last_undegraded":"2024-06-28T16:12:20.520627+0000","last_fullsized":"2024-06-28T16:12:20.520627+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":300,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:31:56.644836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1520,"num_read_kb":1191,"num_write":653,"num_write_kb":382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"260'255","reported_seq":918,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.656935+0000","last_change":"2024-06-28T15:00:15.430833+0000","last_active":"2024-06-28T16:14:34.656935+0000","last_peered":"2024-06-28T16:14:34.656935+0000","last_clean":"2024-06-28T16:14:34.656935+0000","last_became_active":"2024-06-28T15:00:15.430228+0000","last_became_peered":"2024-06-28T15:00:15.430228+0000","last_unstale":"2024-06-28T16:14:34.656935+0000","last_undegraded":"2024-06-28T16:14:34.656935+0000","last_fullsized":"2024-06-28T16:14:34.656935+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:20:15.238678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170399,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":449,"num_read_kb":6008,"num_write":895,"num_write_kb":11243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"260'134","reported_seq":778,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:05.104760+0000","last_change":"2024-06-28T15:00:17.382276+0000","last_active":"2024-06-28T16:12:05.104760+0000","last_peered":"2024-06-28T16:12:05.104760+0000","last_clean":"2024-06-28T16:12:05.104760+0000","last_became_active":"2024-06-28T15:00:17.381562+0000","last_became_peered":"2024-06-28T15:00:17.381562+0000","last_unstale":"2024-06-28T16:12:05.104760+0000","last_undegraded":"2024-06-28T16:12:05.104760+0000","last_fullsized":"2024-06-28T16:12:05.104760+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:32:13.816417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":479,"num_write":190,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"260'496","reported_seq":1185,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.737064+0000","last_change":"2024-06-28T15:00:15.419485+0000","last_active":"2024-06-28T16:14:34.737064+0000","last_peered":"2024-06-28T16:14:34.737064+0000","last_clean":"2024-06-28T16:14:34.737064+0000","last_became_active":"2024-06-28T15:00:15.418850+0000","last_became_peered":"2024-06-28T15:00:15.418850+0000","last_unstale":"2024-06-28T16:14:34.737064+0000","last_undegraded":"2024-06-28T16:14:34.737064+0000","last_fullsized":"2024-06-28T16:14:34.737064+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":400,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:53:16.536329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":907,"num_read_kb":23220,"num_write":1415,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"260'338","reported_seq":1022,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.749225+0000","last_change":"2024-06-28T15:00:15.409457+0000","last_active":"2024-06-28T16:14:34.749225+0000","last_peered":"2024-06-28T16:14:34.749225+0000","last_clean":"2024-06-28T16:14:34.749225+0000","last_became_active":"2024-06-28T15:00:15.408158+0000","last_became_peered":"2024-06-28T15:00:15.408158+0000","last_unstale":"2024-06-28T16:14:34.749225+0000","last_undegraded":"2024-06-28T16:14:34.749225+0000","last_fullsized":"2024-06-28T16:14:34.749225+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:01:20.858547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6690738,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":568,"num_read_kb":38558,"num_write":1259,"num_write_kb":46809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"260'77","reported_seq":645,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:40.551311+0000","last_change":"2024-06-28T15:00:17.380351+0000","last_active":"2024-06-28T16:10:40.551311+0000","last_peered":"2024-06-28T16:10:40.551311+0000","last_clean":"2024-06-28T16:10:40.551311+0000","last_became_active":"2024-06-28T15:00:17.379771+0000","last_became_peered":"2024-06-28T15:00:17.379771+0000","last_unstale":"2024-06-28T16:10:40.551311+0000","last_undegraded":"2024-06-28T16:10:40.551311+0000","last_fullsized":"2024-06-28T16:10:40.551311+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:19:40.601549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":249,"num_write":109,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"260'387","reported_seq":1101,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.735487+0000","last_change":"2024-06-28T15:00:15.404986+0000","last_active":"2024-06-28T16:14:34.735487+0000","last_peered":"2024-06-28T16:14:34.735487+0000","last_clean":"2024-06-28T16:14:34.735487+0000","last_became_active":"2024-06-28T15:00:15.401324+0000","last_became_peered":"2024-06-28T15:00:15.401324+0000","last_unstale":"2024-06-28T16:14:34.735487+0000","last_undegraded":"2024-06-28T16:14:34.735487+0000","last_fullsized":"2024-06-28T16:14:34.735487+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":300,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:15:20.980772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079973,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":661,"num_read_kb":1946,"num_write":1429,"num_write_kb":27589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"260'176","reported_seq":777,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:14.583638+0000","last_change":"2024-06-28T15:00:17.383615+0000","last_active":"2024-06-28T16:13:14.583638+0000","last_peered":"2024-06-28T16:13:14.583638+0000","last_clean":"2024-06-28T16:13:14.583638+0000","last_became_active":"2024-06-28T15:00:17.382774+0000","last_became_peered":"2024-06-28T15:00:17.382774+0000","last_unstale":"2024-06-28T16:13:14.583638+0000","last_undegraded":"2024-06-28T16:13:14.583638+0000","last_fullsized":"2024-06-28T16:13:14.583638+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:39:45.866222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":719,"num_read_kb":570,"num_write":272,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5652,"num_read_kb":3987,"num_write":2073,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1078,"ondisk_log_size":1078,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19748,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":13201,"num_read_kb":13000,"num_write":15927,"num_write_kb":4660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":39496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1836,"ondisk_log_size":1836,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":321,"ondisk_log_size":321,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1481237,"num_objects":198,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":18084,"num_read_kb":37002,"num_write":15113,"num_write_kb":11203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":353845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1842,"ondisk_log_size":1842,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1644,"num_read_kb":1954,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87319,"num_read_kb":70822,"num_write":35298,"num_write_kb":17822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3526,"ondisk_log_size":3526,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":187758130,"num_objects":1055,"num_object_clones":0,"num_object_copies":4220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1055,"num_whiteouts":0,"num_read":34213,"num_read_kb":1786506,"num_write":77152,"num_write_kb":1982911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":273719296,"data_stored":273719296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3512,"ondisk_log_size":3512,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":19963089,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":127,"num_read_kb":165927,"num_write":152,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36741120,"data_stored":36740305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":17865937,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":135,"num_read_kb":186344,"num_write":152,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":50372608,"data_stored":50371793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575745,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":371852,"kb_used_data":51760,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":93999988,"statfs":{"total":96636764160,"available":96255987712,"internally_reserved":0,"allocated":53002240,"data_stored":51018038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":327612874},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575745,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":353092,"kb_used_data":60700,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":94018748,"statfs":{"total":96636764160,"available":96275197952,"internally_reserved":0,"allocated":62156800,"data_stored":60226794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":299170249},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575747,"num_pgs":114,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206904,"kb_used_data":40952,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94164936,"statfs":{"total":96636764160,"available":96424894464,"internally_reserved":0,"allocated":41934848,"data_stored":39931707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575747,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":146836,"kb_used_data":43468,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94225004,"statfs":{"total":96636764160,"available":96486404096,"internally_reserved":0,"allocated":44511232,"data_stored":42596897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575750,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303384,"kb_used_data":29196,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":94068456,"statfs":{"total":96636764160,"available":96326098944,"internally_reserved":0,"allocated":29896704,"data_stored":27928906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":280623559},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575748,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137428,"kb_used_data":29820,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94234412,"statfs":{"total":96636764160,"available":96496037888,"internally_reserved":0,"allocated":30535680,"data_stored":28610104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575745,"num_pgs":120,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201624,"kb_used_data":39968,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94170216,"statfs":{"total":96636764160,"available":96430301184,"internally_reserved":0,"allocated":40927232,"data_stored":38982339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575747,"num_pgs":116,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":385160,"kb_used_data":129088,"kb_used_omap":1,"kb_used_meta":256062,"kb_avail":93986680,"statfs":{"total":96636764160,"available":96242360320,"internally_reserved":0,"allocated":132186112,"data_stored":130163058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":262207941},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":96485376,"data_stored":96485376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25198592,"data_stored":25198592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21118976,"data_stored":21118976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19386368,"data_stored":19386368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29863936,"data_stored":29863936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20664320,"data_stored":20664320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28712960,"data_stored":28712960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32288768,"data_stored":32288768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":347321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":7701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":4574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":4436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":5746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16818176,"data_stored":16817361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13672448,"data_stored":13671633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-28T16:14:36.404 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-06-28T16:14:36.404 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-28T16:14:36.765 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:36.777 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-06-28T16:14:36.779 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":2487,"stamp":"2024-06-28T16:14:35.607276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":227680302,"num_objects":1411,"num_object_clones":0,"num_object_copies":4932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1411,"num_whiteouts":0,"num_read":160453,"num_read_kb":2265609,"num_write":146008,"num_write_kb":2275003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12397,"ondisk_log_size":12397,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2106280,"kb_used_data":424952,"kb_used_omap":12,"kb_used_meta":1680371,"kb_avail":752868440,"statfs":{"total":773094113280,"available":770937282560,"internally_reserved":0,"allocated":435150848,"data_stored":419457843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":1720700467},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-457593513,"num_objects":-3393,"num_object_clones":0,"num_object_copies":-13508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3393,"num_whiteouts":0,"num_read":3820,"num_read_kb":3315,"num_write":3571,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007566"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":26,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.063084+0000","last_change":"2024-06-28T15:37:33.329429+0000","last_active":"2024-06-28T16:10:05.063084+0000","last_peered":"2024-06-28T16:10:05.063084+0000","last_clean":"2024-06-28T16:10:05.063084+0000","last_became_active":"2024-06-28T15:37:33.328626+0000","last_became_peered":"2024-06-28T15:37:33.328626+0000","last_unstale":"2024-06-28T16:10:05.063084+0000","last_undegraded":"2024-06-28T16:10:05.063084+0000","last_fullsized":"2024-06-28T16:10:05.063084+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:48:45.377436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"257'3","reported_seq":49,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.078088+0000","last_change":"2024-06-28T15:21:11.079460+0000","last_active":"2024-06-28T16:11:10.078088+0000","last_peered":"2024-06-28T16:11:10.078088+0000","last_clean":"2024-06-28T16:11:10.078088+0000","last_became_active":"2024-06-28T15:21:11.078250+0000","last_became_peered":"2024-06-28T15:21:11.078250+0000","last_unstale":"2024-06-28T16:11:10.078088+0000","last_undegraded":"2024-06-28T16:11:10.078088+0000","last_fullsized":"2024-06-28T16:11:10.078088+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:32:28.636744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"260'4","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.900204+0000","last_change":"2024-06-28T15:21:07.047340+0000","last_active":"2024-06-28T16:14:32.900204+0000","last_peered":"2024-06-28T16:14:32.900204+0000","last_clean":"2024-06-28T16:14:32.900204+0000","last_became_active":"2024-06-28T15:21:07.046517+0000","last_became_peered":"2024-06-28T15:21:07.046517+0000","last_unstale":"2024-06-28T16:14:32.900204+0000","last_undegraded":"2024-06-28T16:14:32.900204+0000","last_fullsized":"2024-06-28T16:14:32.900204+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:20:57.841643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3076,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"249'7","reported_seq":82,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.563726+0000","last_change":"2024-06-28T15:17:24.031700+0000","last_active":"2024-06-28T16:11:35.563726+0000","last_peered":"2024-06-28T16:11:35.563726+0000","last_clean":"2024-06-28T16:11:35.563726+0000","last_became_active":"2024-06-28T15:17:24.028787+0000","last_became_peered":"2024-06-28T15:17:24.028787+0000","last_unstale":"2024-06-28T16:11:35.563726+0000","last_undegraded":"2024-06-28T16:11:35.563726+0000","last_fullsized":"2024-06-28T16:11:35.563726+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:38:55.771845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":30,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":26,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.252697+0000","last_change":"2024-06-28T15:37:33.337334+0000","last_active":"2024-06-28T16:10:05.252697+0000","last_peered":"2024-06-28T16:10:05.252697+0000","last_clean":"2024-06-28T16:10:05.252697+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:10:05.252697+0000","last_undegraded":"2024-06-28T16:10:05.252697+0000","last_fullsized":"2024-06-28T16:10:05.252697+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:34:53.771735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"260'2","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.119791+0000","last_change":"2024-06-28T15:21:11.105141+0000","last_active":"2024-06-28T16:13:45.119791+0000","last_peered":"2024-06-28T16:13:45.119791+0000","last_clean":"2024-06-28T16:13:45.119791+0000","last_became_active":"2024-06-28T15:21:11.104387+0000","last_became_peered":"2024-06-28T15:21:11.104387+0000","last_unstale":"2024-06-28T16:13:45.119791+0000","last_undegraded":"2024-06-28T16:13:45.119791+0000","last_fullsized":"2024-06-28T16:13:45.119791+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:29:31.608922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"260'2","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.126798+0000","last_change":"2024-06-28T15:21:07.049830+0000","last_active":"2024-06-28T16:13:40.126798+0000","last_peered":"2024-06-28T16:13:40.126798+0000","last_clean":"2024-06-28T16:13:40.126798+0000","last_became_active":"2024-06-28T15:21:07.049141+0000","last_became_peered":"2024-06-28T15:21:07.049141+0000","last_unstale":"2024-06-28T16:13:40.126798+0000","last_undegraded":"2024-06-28T16:13:40.126798+0000","last_fullsized":"2024-06-28T16:13:40.126798+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:18:35.987689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"260'18","reported_seq":110,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:40.271668+0000","last_change":"2024-06-28T15:17:23.994609+0000","last_active":"2024-06-28T16:11:40.271668+0000","last_peered":"2024-06-28T16:11:40.271668+0000","last_clean":"2024-06-28T16:11:40.271668+0000","last_became_active":"2024-06-28T15:17:23.993905+0000","last_became_peered":"2024-06-28T15:17:23.993905+0000","last_unstale":"2024-06-28T16:11:40.271668+0000","last_undegraded":"2024-06-28T16:11:40.271668+0000","last_fullsized":"2024-06-28T16:11:40.271668+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:33:37.469328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":64,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.087475+0000","last_change":"2024-06-28T15:37:33.353498+0000","last_active":"2024-06-28T16:11:25.087475+0000","last_peered":"2024-06-28T16:11:25.087475+0000","last_clean":"2024-06-28T16:11:25.087475+0000","last_became_active":"2024-06-28T15:37:33.345642+0000","last_became_peered":"2024-06-28T15:37:33.345642+0000","last_unstale":"2024-06-28T16:11:25.087475+0000","last_undegraded":"2024-06-28T16:11:25.087475+0000","last_fullsized":"2024-06-28T16:11:25.087475+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:38:21.634963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"257'5","reported_seq":57,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.558957+0000","last_change":"2024-06-28T15:21:11.098339+0000","last_active":"2024-06-28T16:11:15.558957+0000","last_peered":"2024-06-28T16:11:15.558957+0000","last_clean":"2024-06-28T16:11:15.558957+0000","last_became_active":"2024-06-28T15:21:11.097619+0000","last_became_peered":"2024-06-28T15:21:11.097619+0000","last_unstale":"2024-06-28T16:11:15.558957+0000","last_undegraded":"2024-06-28T16:11:15.558957+0000","last_fullsized":"2024-06-28T16:11:15.558957+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:23:45.881180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5438,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"0'0","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.268802+0000","last_change":"2024-06-28T15:21:07.020860+0000","last_active":"2024-06-28T16:11:25.268802+0000","last_peered":"2024-06-28T16:11:25.268802+0000","last_clean":"2024-06-28T16:11:25.268802+0000","last_became_active":"2024-06-28T15:21:07.020090+0000","last_became_peered":"2024-06-28T15:21:07.020090+0000","last_unstale":"2024-06-28T16:11:25.268802+0000","last_undegraded":"2024-06-28T16:11:25.268802+0000","last_fullsized":"2024-06-28T16:11:25.268802+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:42:14.199161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"260'43","reported_seq":174,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:09:55.540550+0000","last_change":"2024-06-28T15:17:24.014535+0000","last_active":"2024-06-28T16:09:55.540550+0000","last_peered":"2024-06-28T16:09:55.540550+0000","last_clean":"2024-06-28T16:09:55.540550+0000","last_became_active":"2024-06-28T15:17:24.012739+0000","last_became_peered":"2024-06-28T15:17:24.012739+0000","last_unstale":"2024-06-28T16:09:55.540550+0000","last_undegraded":"2024-06-28T16:09:55.540550+0000","last_fullsized":"2024-06-28T16:09:55.540550+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:08:38.008559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":221,"num_read_kb":154,"num_write":91,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.509029+0000","last_change":"2024-06-28T15:37:33.345746+0000","last_active":"2024-06-28T16:11:30.509029+0000","last_peered":"2024-06-28T16:11:30.509029+0000","last_clean":"2024-06-28T16:11:30.509029+0000","last_became_active":"2024-06-28T15:37:33.344850+0000","last_became_peered":"2024-06-28T15:37:33.344850+0000","last_unstale":"2024-06-28T16:11:30.509029+0000","last_undegraded":"2024-06-28T16:11:30.509029+0000","last_fullsized":"2024-06-28T16:11:30.509029+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:52:03.083724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"260'3","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.320060+0000","last_change":"2024-06-28T15:21:11.107158+0000","last_active":"2024-06-28T16:13:20.320060+0000","last_peered":"2024-06-28T16:13:20.320060+0000","last_clean":"2024-06-28T16:13:20.320060+0000","last_became_active":"2024-06-28T15:21:11.102572+0000","last_became_peered":"2024-06-28T15:21:11.102572+0000","last_unstale":"2024-06-28T16:13:20.320060+0000","last_undegraded":"2024-06-28T16:13:20.320060+0000","last_fullsized":"2024-06-28T16:13:20.320060+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:06:32.919789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.085295+0000","last_change":"2024-06-28T15:21:07.032950+0000","last_active":"2024-06-28T16:11:10.085295+0000","last_peered":"2024-06-28T16:11:10.085295+0000","last_clean":"2024-06-28T16:11:10.085295+0000","last_became_active":"2024-06-28T15:21:07.030576+0000","last_became_peered":"2024-06-28T15:21:07.030576+0000","last_unstale":"2024-06-28T16:11:10.085295+0000","last_undegraded":"2024-06-28T16:11:10.085295+0000","last_fullsized":"2024-06-28T16:11:10.085295+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:28:35.851727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'23","reported_seq":116,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:09:55.485817+0000","last_change":"2024-06-28T15:17:24.019871+0000","last_active":"2024-06-28T16:09:55.485817+0000","last_peered":"2024-06-28T16:09:55.485817+0000","last_clean":"2024-06-28T16:09:55.485817+0000","last_became_active":"2024-06-28T15:17:24.019120+0000","last_became_peered":"2024-06-28T15:17:24.019120+0000","last_unstale":"2024-06-28T16:09:55.485817+0000","last_undegraded":"2024-06-28T16:09:55.485817+0000","last_fullsized":"2024-06-28T16:09:55.485817+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:25:56.355193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":82,"num_write":51,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:40.511313+0000","last_change":"2024-06-28T15:37:33.339459+0000","last_active":"2024-06-28T16:11:40.511313+0000","last_peered":"2024-06-28T16:11:40.511313+0000","last_clean":"2024-06-28T16:11:40.511313+0000","last_became_active":"2024-06-28T15:37:33.335045+0000","last_became_peered":"2024-06-28T15:37:33.335045+0000","last_unstale":"2024-06-28T16:11:40.511313+0000","last_undegraded":"2024-06-28T16:11:40.511313+0000","last_fullsized":"2024-06-28T16:11:40.511313+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:39:33.932638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"0'0","reported_seq":44,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:10.091403+0000","last_change":"2024-06-28T15:21:11.094697+0000","last_active":"2024-06-28T16:12:10.091403+0000","last_peered":"2024-06-28T16:12:10.091403+0000","last_clean":"2024-06-28T16:12:10.091403+0000","last_became_active":"2024-06-28T15:21:11.094122+0000","last_became_peered":"2024-06-28T15:21:11.094122+0000","last_unstale":"2024-06-28T16:12:10.091403+0000","last_undegraded":"2024-06-28T16:12:10.091403+0000","last_fullsized":"2024-06-28T16:12:10.091403+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:48:32.491192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.569464+0000","last_change":"2024-06-28T15:21:07.027869+0000","last_active":"2024-06-28T16:11:55.569464+0000","last_peered":"2024-06-28T16:11:55.569464+0000","last_clean":"2024-06-28T16:11:55.569464+0000","last_became_active":"2024-06-28T15:21:07.025549+0000","last_became_peered":"2024-06-28T15:21:07.025549+0000","last_unstale":"2024-06-28T16:11:55.569464+0000","last_undegraded":"2024-06-28T16:11:55.569464+0000","last_fullsized":"2024-06-28T16:11:55.569464+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:37:01.202123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"260'24","reported_seq":120,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:09:55.485705+0000","last_change":"2024-06-28T15:17:24.011769+0000","last_active":"2024-06-28T16:09:55.485705+0000","last_peered":"2024-06-28T16:09:55.485705+0000","last_clean":"2024-06-28T16:09:55.485705+0000","last_became_active":"2024-06-28T15:17:24.010441+0000","last_became_peered":"2024-06-28T15:17:24.010441+0000","last_unstale":"2024-06-28T16:09:55.485705+0000","last_undegraded":"2024-06-28T16:09:55.485705+0000","last_fullsized":"2024-06-28T16:09:55.485705+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:28:32.900550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":85,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.091008+0000","last_change":"2024-06-28T15:37:33.341710+0000","last_active":"2024-06-28T16:10:55.091008+0000","last_peered":"2024-06-28T16:10:55.091008+0000","last_clean":"2024-06-28T16:10:55.091008+0000","last_became_active":"2024-06-28T15:37:33.336623+0000","last_became_peered":"2024-06-28T15:37:33.336623+0000","last_unstale":"2024-06-28T16:10:55.091008+0000","last_undegraded":"2024-06-28T16:10:55.091008+0000","last_fullsized":"2024-06-28T16:10:55.091008+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:49:52.074185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"260'11","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.517566+0000","last_change":"2024-06-28T15:21:11.100151+0000","last_active":"2024-06-28T16:14:32.517566+0000","last_peered":"2024-06-28T16:14:32.517566+0000","last_clean":"2024-06-28T16:14:32.517566+0000","last_became_active":"2024-06-28T15:21:11.099434+0000","last_became_peered":"2024-06-28T15:21:11.099434+0000","last_unstale":"2024-06-28T16:14:32.517566+0000","last_undegraded":"2024-06-28T16:14:32.517566+0000","last_fullsized":"2024-06-28T16:14:32.517566+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:21:06.014702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":20482,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"257'5","reported_seq":65,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:10.098868+0000","last_change":"2024-06-28T15:21:07.044767+0000","last_active":"2024-06-28T16:12:10.098868+0000","last_peered":"2024-06-28T16:12:10.098868+0000","last_clean":"2024-06-28T16:12:10.098868+0000","last_became_active":"2024-06-28T15:21:07.043987+0000","last_became_peered":"2024-06-28T15:21:07.043987+0000","last_unstale":"2024-06-28T16:12:10.098868+0000","last_undegraded":"2024-06-28T16:12:10.098868+0000","last_fullsized":"2024-06-28T16:12:10.098868+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:53:56.640453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"260'19","reported_seq":100,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:09:55.067186+0000","last_change":"2024-06-28T15:17:24.007176+0000","last_active":"2024-06-28T16:09:55.067186+0000","last_peered":"2024-06-28T16:09:55.067186+0000","last_clean":"2024-06-28T16:09:55.067186+0000","last_became_active":"2024-06-28T15:17:24.006314+0000","last_became_peered":"2024-06-28T15:17:24.006314+0000","last_unstale":"2024-06-28T16:09:55.067186+0000","last_undegraded":"2024-06-28T16:09:55.067186+0000","last_fullsized":"2024-06-28T16:09:55.067186+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:20:19.639267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":54,"num_write":49,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"260'1","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.076954+0000","last_change":"2024-06-28T15:37:33.348503+0000","last_active":"2024-06-28T16:10:05.076954+0000","last_peered":"2024-06-28T16:10:05.076954+0000","last_clean":"2024-06-28T16:10:05.076954+0000","last_became_active":"2024-06-28T15:37:33.347715+0000","last_became_peered":"2024-06-28T15:37:33.347715+0000","last_unstale":"2024-06-28T16:10:05.076954+0000","last_undegraded":"2024-06-28T16:10:05.076954+0000","last_fullsized":"2024-06-28T16:10:05.076954+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:01:24.044514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":42,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.085434+0000","last_change":"2024-06-28T15:21:11.089867+0000","last_active":"2024-06-28T16:11:15.085434+0000","last_peered":"2024-06-28T16:11:15.085434+0000","last_clean":"2024-06-28T16:11:15.085434+0000","last_became_active":"2024-06-28T15:21:11.088625+0000","last_became_peered":"2024-06-28T15:21:11.088625+0000","last_unstale":"2024-06-28T16:11:15.085434+0000","last_undegraded":"2024-06-28T16:11:15.085434+0000","last_fullsized":"2024-06-28T16:11:15.085434+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:32:47.953440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"260'6","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.118688+0000","last_change":"2024-06-28T15:21:07.031673+0000","last_active":"2024-06-28T16:13:40.118688+0000","last_peered":"2024-06-28T16:13:40.118688+0000","last_clean":"2024-06-28T16:13:40.118688+0000","last_became_active":"2024-06-28T15:21:07.030888+0000","last_became_peered":"2024-06-28T15:21:07.030888+0000","last_unstale":"2024-06-28T16:13:40.118688+0000","last_undegraded":"2024-06-28T16:13:40.118688+0000","last_fullsized":"2024-06-28T16:13:40.118688+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:54.912979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":6,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"260'72","reported_seq":247,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:45.552156+0000","last_change":"2024-06-28T15:17:23.993803+0000","last_active":"2024-06-28T16:10:45.552156+0000","last_peered":"2024-06-28T16:10:45.552156+0000","last_clean":"2024-06-28T16:10:45.552156+0000","last_became_active":"2024-06-28T15:17:23.993026+0000","last_became_peered":"2024-06-28T15:17:23.993026+0000","last_unstale":"2024-06-28T16:10:45.552156+0000","last_undegraded":"2024-06-28T16:10:45.552156+0000","last_fullsized":"2024-06-28T16:10:45.552156+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:12:38.304019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":255,"num_write":131,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":26,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.077019+0000","last_change":"2024-06-28T15:37:33.351154+0000","last_active":"2024-06-28T16:10:05.077019+0000","last_peered":"2024-06-28T16:10:05.077019+0000","last_clean":"2024-06-28T16:10:05.077019+0000","last_became_active":"2024-06-28T15:37:33.350447+0000","last_became_peered":"2024-06-28T15:37:33.350447+0000","last_unstale":"2024-06-28T16:10:05.077019+0000","last_undegraded":"2024-06-28T16:10:05.077019+0000","last_fullsized":"2024-06-28T16:10:05.077019+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:41:10.113052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":43,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.504142+0000","last_change":"2024-06-28T15:21:11.101407+0000","last_active":"2024-06-28T16:11:10.504142+0000","last_peered":"2024-06-28T16:11:10.504142+0000","last_clean":"2024-06-28T16:11:10.504142+0000","last_became_active":"2024-06-28T15:21:11.100603+0000","last_became_peered":"2024-06-28T15:21:11.100603+0000","last_unstale":"2024-06-28T16:11:10.504142+0000","last_undegraded":"2024-06-28T16:11:10.504142+0000","last_fullsized":"2024-06-28T16:11:10.504142+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:33:54.057658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"260'3","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.557448+0000","last_change":"2024-06-28T15:21:07.047854+0000","last_active":"2024-06-28T16:11:10.557448+0000","last_peered":"2024-06-28T16:11:10.557448+0000","last_clean":"2024-06-28T16:11:10.557448+0000","last_became_active":"2024-06-28T15:21:07.045902+0000","last_became_peered":"2024-06-28T15:21:07.045902+0000","last_unstale":"2024-06-28T16:11:10.557448+0000","last_undegraded":"2024-06-28T16:11:10.557448+0000","last_fullsized":"2024-06-28T16:11:10.557448+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:00:14.390394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1088721,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":40,"num_write":8,"num_write_kb":1064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"260'48","reported_seq":192,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.087151+0000","last_change":"2024-06-28T15:17:24.025635+0000","last_active":"2024-06-28T16:10:35.087151+0000","last_peered":"2024-06-28T16:10:35.087151+0000","last_clean":"2024-06-28T16:10:35.087151+0000","last_became_active":"2024-06-28T15:17:24.016249+0000","last_became_peered":"2024-06-28T15:17:24.016249+0000","last_unstale":"2024-06-28T16:10:35.087151+0000","last_undegraded":"2024-06-28T16:10:35.087151+0000","last_fullsized":"2024-06-28T16:10:35.087151+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:59:25.184524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":260,"num_read_kb":188,"num_write":81,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.083231+0000","last_change":"2024-06-28T15:37:33.317386+0000","last_active":"2024-06-28T16:10:15.083231+0000","last_peered":"2024-06-28T16:10:15.083231+0000","last_clean":"2024-06-28T16:10:15.083231+0000","last_became_active":"2024-06-28T15:37:33.316616+0000","last_became_peered":"2024-06-28T15:37:33.316616+0000","last_unstale":"2024-06-28T16:10:15.083231+0000","last_undegraded":"2024-06-28T16:10:15.083231+0000","last_fullsized":"2024-06-28T16:10:15.083231+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:02:05.364651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"260'8","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:20.333490+0000","last_change":"2024-06-28T15:21:11.099574+0000","last_active":"2024-06-28T16:14:20.333490+0000","last_peered":"2024-06-28T16:14:20.333490+0000","last_clean":"2024-06-28T16:14:20.333490+0000","last_became_active":"2024-06-28T15:21:11.094947+0000","last_became_peered":"2024-06-28T15:21:11.094947+0000","last_unstale":"2024-06-28T16:14:20.333490+0000","last_undegraded":"2024-06-28T16:14:20.333490+0000","last_fullsized":"2024-06-28T16:14:20.333490+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:14:27.139498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":12406,"num_write":8,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"260'5","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:45.099524+0000","last_change":"2024-06-28T15:21:07.026138+0000","last_active":"2024-06-28T16:12:45.099524+0000","last_peered":"2024-06-28T16:12:45.099524+0000","last_clean":"2024-06-28T16:12:45.099524+0000","last_became_active":"2024-06-28T15:21:07.025396+0000","last_became_peered":"2024-06-28T15:21:07.025396+0000","last_unstale":"2024-06-28T16:12:45.099524+0000","last_undegraded":"2024-06-28T16:12:45.099524+0000","last_fullsized":"2024-06-28T16:12:45.099524+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:48:53.728948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":5,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"257'14","reported_seq":96,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.550345+0000","last_change":"2024-06-28T15:17:24.015094+0000","last_active":"2024-06-28T16:10:35.550345+0000","last_peered":"2024-06-28T16:10:35.550345+0000","last_clean":"2024-06-28T16:10:35.550345+0000","last_became_active":"2024-06-28T15:17:24.010190+0000","last_became_peered":"2024-06-28T15:17:24.010190+0000","last_unstale":"2024-06-28T16:10:35.550345+0000","last_undegraded":"2024-06-28T16:10:35.550345+0000","last_fullsized":"2024-06-28T16:10:35.550345+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:20:30.481056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":51,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"260'1","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.255026+0000","last_change":"2024-06-28T15:37:33.338222+0000","last_active":"2024-06-28T16:10:15.255026+0000","last_peered":"2024-06-28T16:10:15.255026+0000","last_clean":"2024-06-28T16:10:15.255026+0000","last_became_active":"2024-06-28T15:37:33.334097+0000","last_became_peered":"2024-06-28T15:37:33.334097+0000","last_unstale":"2024-06-28T16:10:15.255026+0000","last_undegraded":"2024-06-28T16:10:15.255026+0000","last_fullsized":"2024-06-28T16:10:15.255026+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:04:55.107050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"260'9","reported_seq":62,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.508721+0000","last_change":"2024-06-28T15:21:11.107909+0000","last_active":"2024-06-28T16:14:33.508721+0000","last_peered":"2024-06-28T16:14:33.508721+0000","last_clean":"2024-06-28T16:14:33.508721+0000","last_became_active":"2024-06-28T15:21:11.107155+0000","last_became_peered":"2024-06-28T15:21:11.107155+0000","last_unstale":"2024-06-28T16:14:33.508721+0000","last_undegraded":"2024-06-28T16:14:33.508721+0000","last_fullsized":"2024-06-28T16:14:33.508721+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:27:41.181651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21504,"num_write":14,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":49,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.102713+0000","last_change":"2024-06-28T15:21:07.043526+0000","last_active":"2024-06-28T16:11:55.102713+0000","last_peered":"2024-06-28T16:11:55.102713+0000","last_clean":"2024-06-28T16:11:55.102713+0000","last_became_active":"2024-06-28T15:21:07.042691+0000","last_became_peered":"2024-06-28T15:21:07.042691+0000","last_unstale":"2024-06-28T16:11:55.102713+0000","last_undegraded":"2024-06-28T16:11:55.102713+0000","last_fullsized":"2024-06-28T16:11:55.102713+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:56:49.487276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"257'73","reported_seq":289,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.496134+0000","last_change":"2024-06-28T15:17:24.028644+0000","last_active":"2024-06-28T16:10:35.496134+0000","last_peered":"2024-06-28T16:10:35.496134+0000","last_clean":"2024-06-28T16:10:35.496134+0000","last_became_active":"2024-06-28T15:17:24.027786+0000","last_became_peered":"2024-06-28T15:17:24.027786+0000","last_unstale":"2024-06-28T16:10:35.496134+0000","last_undegraded":"2024-06-28T16:10:35.496134+0000","last_fullsized":"2024-06-28T16:10:35.496134+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:26:10.319359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":461,"num_read_kb":332,"num_write":85,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.079074+0000","last_change":"2024-06-28T15:37:33.355373+0000","last_active":"2024-06-28T16:10:15.079074+0000","last_peered":"2024-06-28T16:10:15.079074+0000","last_clean":"2024-06-28T16:10:15.079074+0000","last_became_active":"2024-06-28T15:37:33.354730+0000","last_became_peered":"2024-06-28T15:37:33.354730+0000","last_unstale":"2024-06-28T16:10:15.079074+0000","last_undegraded":"2024-06-28T16:10:15.079074+0000","last_fullsized":"2024-06-28T16:10:15.079074+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:33:00.069224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"260'4","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:20.333564+0000","last_change":"2024-06-28T15:21:11.094517+0000","last_active":"2024-06-28T16:14:20.333564+0000","last_peered":"2024-06-28T16:14:20.333564+0000","last_clean":"2024-06-28T16:14:20.333564+0000","last_became_active":"2024-06-28T15:21:11.093751+0000","last_became_peered":"2024-06-28T15:21:11.093751+0000","last_unstale":"2024-06-28T16:14:20.333564+0000","last_undegraded":"2024-06-28T16:14:20.333564+0000","last_fullsized":"2024-06-28T16:14:20.333564+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:37:10.229225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"260'2","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:20.305025+0000","last_change":"2024-06-28T15:21:07.043427+0000","last_active":"2024-06-28T16:14:20.305025+0000","last_peered":"2024-06-28T16:14:20.305025+0000","last_clean":"2024-06-28T16:14:20.305025+0000","last_became_active":"2024-06-28T15:21:07.042846+0000","last_became_peered":"2024-06-28T15:21:07.042846+0000","last_unstale":"2024-06-28T16:14:20.305025+0000","last_undegraded":"2024-06-28T16:14:20.305025+0000","last_fullsized":"2024-06-28T16:14:20.305025+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:41:49.061527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":59,"num_write":2,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"260'31","reported_seq":144,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.258858+0000","last_change":"2024-06-28T15:17:24.017530+0000","last_active":"2024-06-28T16:10:35.258858+0000","last_peered":"2024-06-28T16:10:35.258858+0000","last_clean":"2024-06-28T16:10:35.258858+0000","last_became_active":"2024-06-28T15:17:24.016275+0000","last_became_peered":"2024-06-28T15:17:24.016275+0000","last_unstale":"2024-06-28T16:10:35.258858+0000","last_undegraded":"2024-06-28T16:10:35.258858+0000","last_fullsized":"2024-06-28T16:10:35.258858+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:46:12.119777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":116,"num_write":60,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.065428+0000","last_change":"2024-06-28T15:37:33.323939+0000","last_active":"2024-06-28T16:10:15.065428+0000","last_peered":"2024-06-28T16:10:15.065428+0000","last_clean":"2024-06-28T16:10:15.065428+0000","last_became_active":"2024-06-28T15:37:33.323174+0000","last_became_peered":"2024-06-28T15:37:33.323174+0000","last_unstale":"2024-06-28T16:10:15.065428+0000","last_undegraded":"2024-06-28T16:10:15.065428+0000","last_fullsized":"2024-06-28T16:10:15.065428+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:23:02.255965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"260'9","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.510895+0000","last_change":"2024-06-28T15:21:11.088257+0000","last_active":"2024-06-28T16:14:33.510895+0000","last_peered":"2024-06-28T16:14:33.510895+0000","last_clean":"2024-06-28T16:14:33.510895+0000","last_became_active":"2024-06-28T15:21:11.087119+0000","last_became_peered":"2024-06-28T15:21:11.087119+0000","last_unstale":"2024-06-28T16:14:33.510895+0000","last_undegraded":"2024-06-28T16:14:33.510895+0000","last_fullsized":"2024-06-28T16:14:33.510895+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:24:43.264477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":24576,"num_write":19,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.569812+0000","last_change":"2024-06-28T15:21:07.032975+0000","last_active":"2024-06-28T16:11:55.569812+0000","last_peered":"2024-06-28T16:11:55.569812+0000","last_clean":"2024-06-28T16:11:55.569812+0000","last_became_active":"2024-06-28T15:21:07.032092+0000","last_became_peered":"2024-06-28T15:21:07.032092+0000","last_unstale":"2024-06-28T16:11:55.569812+0000","last_undegraded":"2024-06-28T16:11:55.569812+0000","last_fullsized":"2024-06-28T16:11:55.569812+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:29:54.468683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"260'21","reported_seq":120,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.094899+0000","last_change":"2024-06-28T15:17:24.021516+0000","last_active":"2024-06-28T16:11:20.094899+0000","last_peered":"2024-06-28T16:11:20.094899+0000","last_clean":"2024-06-28T16:11:20.094899+0000","last_became_active":"2024-06-28T15:17:24.020899+0000","last_became_peered":"2024-06-28T15:17:24.020899+0000","last_unstale":"2024-06-28T16:11:20.094899+0000","last_undegraded":"2024-06-28T16:11:20.094899+0000","last_fullsized":"2024-06-28T16:11:20.094899+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:16:16.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":86,"num_write":39,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.508021+0000","last_change":"2024-06-28T15:37:33.339666+0000","last_active":"2024-06-28T16:11:25.508021+0000","last_peered":"2024-06-28T16:11:25.508021+0000","last_clean":"2024-06-28T16:11:25.508021+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:11:25.508021+0000","last_undegraded":"2024-06-28T16:11:25.508021+0000","last_fullsized":"2024-06-28T16:11:25.508021+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:16:22.839274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"257'3","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.083204+0000","last_change":"2024-06-28T15:21:11.098494+0000","last_active":"2024-06-28T16:10:15.083204+0000","last_peered":"2024-06-28T16:10:15.083204+0000","last_clean":"2024-06-28T16:10:15.083204+0000","last_became_active":"2024-06-28T15:21:11.097622+0000","last_became_peered":"2024-06-28T15:21:11.097622+0000","last_unstale":"2024-06-28T16:10:15.083204+0000","last_undegraded":"2024-06-28T16:10:15.083204+0000","last_fullsized":"2024-06-28T16:10:15.083204+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:35:49.979919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"0'0","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.071391+0000","last_change":"2024-06-28T15:21:07.039533+0000","last_active":"2024-06-28T16:10:10.071391+0000","last_peered":"2024-06-28T16:10:10.071391+0000","last_clean":"2024-06-28T16:10:10.071391+0000","last_became_active":"2024-06-28T15:21:07.038746+0000","last_became_peered":"2024-06-28T15:21:07.038746+0000","last_unstale":"2024-06-28T16:10:10.071391+0000","last_undegraded":"2024-06-28T16:10:10.071391+0000","last_fullsized":"2024-06-28T16:10:10.071391+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:11:32.609450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"257'9","reported_seq":81,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.091202+0000","last_change":"2024-06-28T15:17:24.024616+0000","last_active":"2024-06-28T16:11:05.091202+0000","last_peered":"2024-06-28T16:11:05.091202+0000","last_clean":"2024-06-28T16:11:05.091202+0000","last_became_active":"2024-06-28T15:17:24.020276+0000","last_became_peered":"2024-06-28T15:17:24.020276+0000","last_unstale":"2024-06-28T16:11:05.091202+0000","last_undegraded":"2024-06-28T16:11:05.091202+0000","last_fullsized":"2024-06-28T16:11:05.091202+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:06.656845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.561512+0000","last_change":"2024-06-28T15:37:33.328340+0000","last_active":"2024-06-28T16:11:25.561512+0000","last_peered":"2024-06-28T16:11:25.561512+0000","last_clean":"2024-06-28T16:11:25.561512+0000","last_became_active":"2024-06-28T15:37:33.327497+0000","last_became_peered":"2024-06-28T15:37:33.327497+0000","last_unstale":"2024-06-28T16:11:25.561512+0000","last_undegraded":"2024-06-28T16:11:25.561512+0000","last_fullsized":"2024-06-28T16:11:25.561512+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:28:42.508678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"260'4","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.136460+0000","last_change":"2024-06-28T15:21:11.092761+0000","last_active":"2024-06-28T16:14:33.136460+0000","last_peered":"2024-06-28T16:14:33.136460+0000","last_clean":"2024-06-28T16:14:33.136460+0000","last_became_active":"2024-06-28T15:21:11.092033+0000","last_became_peered":"2024-06-28T15:21:11.092033+0000","last_unstale":"2024-06-28T16:14:33.136460+0000","last_undegraded":"2024-06-28T16:14:33.136460+0000","last_fullsized":"2024-06-28T16:14:33.136460+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:19:30.265533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6152,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.489165+0000","last_change":"2024-06-28T15:21:07.047668+0000","last_active":"2024-06-28T16:10:10.489165+0000","last_peered":"2024-06-28T16:10:10.489165+0000","last_clean":"2024-06-28T16:10:10.489165+0000","last_became_active":"2024-06-28T15:21:07.045842+0000","last_became_peered":"2024-06-28T15:21:07.045842+0000","last_unstale":"2024-06-28T16:10:10.489165+0000","last_undegraded":"2024-06-28T16:10:10.489165+0000","last_fullsized":"2024-06-28T16:10:10.489165+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:42:33.570243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"260'17","reported_seq":106,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.556141+0000","last_change":"2024-06-28T15:17:24.016733+0000","last_active":"2024-06-28T16:11:05.556141+0000","last_peered":"2024-06-28T16:11:05.556141+0000","last_clean":"2024-06-28T16:11:05.556141+0000","last_became_active":"2024-06-28T15:17:24.013118+0000","last_became_peered":"2024-06-28T15:17:24.013118+0000","last_unstale":"2024-06-28T16:11:05.556141+0000","last_undegraded":"2024-06-28T16:11:05.556141+0000","last_fullsized":"2024-06-28T16:11:05.556141+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:57:44.752951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":63,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"259'1","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.546888+0000","last_change":"2024-06-28T15:37:33.342777+0000","last_active":"2024-06-28T16:10:20.546888+0000","last_peered":"2024-06-28T16:10:20.546888+0000","last_clean":"2024-06-28T16:10:20.546888+0000","last_became_active":"2024-06-28T15:37:33.339097+0000","last_became_peered":"2024-06-28T15:37:33.339097+0000","last_unstale":"2024-06-28T16:10:20.546888+0000","last_undegraded":"2024-06-28T16:10:20.546888+0000","last_fullsized":"2024-06-28T16:10:20.546888+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:34:20.645575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"257'3","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.489379+0000","last_change":"2024-06-28T15:21:11.101867+0000","last_active":"2024-06-28T16:10:10.489379+0000","last_peered":"2024-06-28T16:10:10.489379+0000","last_clean":"2024-06-28T16:10:10.489379+0000","last_became_active":"2024-06-28T15:21:11.101063+0000","last_became_peered":"2024-06-28T15:21:11.101063+0000","last_unstale":"2024-06-28T16:10:10.489379+0000","last_undegraded":"2024-06-28T16:10:10.489379+0000","last_fullsized":"2024-06-28T16:10:10.489379+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:58:49.328893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"257'3","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.070602+0000","last_change":"2024-06-28T15:21:07.042183+0000","last_active":"2024-06-28T16:10:10.070602+0000","last_peered":"2024-06-28T16:10:10.070602+0000","last_clean":"2024-06-28T16:10:10.070602+0000","last_became_active":"2024-06-28T15:21:07.040973+0000","last_became_peered":"2024-06-28T15:21:07.040973+0000","last_unstale":"2024-06-28T16:10:10.070602+0000","last_undegraded":"2024-06-28T16:10:10.070602+0000","last_fullsized":"2024-06-28T16:10:10.070602+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:38:10.417534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"260'22","reported_seq":115,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.083945+0000","last_change":"2024-06-28T15:17:24.029346+0000","last_active":"2024-06-28T16:11:05.083945+0000","last_peered":"2024-06-28T16:11:05.083945+0000","last_clean":"2024-06-28T16:11:05.083945+0000","last_became_active":"2024-06-28T15:17:24.028410+0000","last_became_peered":"2024-06-28T15:17:24.028410+0000","last_unstale":"2024-06-28T16:11:05.083945+0000","last_undegraded":"2024-06-28T16:11:05.083945+0000","last_fullsized":"2024-06-28T16:11:05.083945+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:25:05.370434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"260'1","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.507957+0000","last_change":"2024-06-28T15:37:33.341173+0000","last_active":"2024-06-28T16:11:25.507957+0000","last_peered":"2024-06-28T16:11:25.507957+0000","last_clean":"2024-06-28T16:11:25.507957+0000","last_became_active":"2024-06-28T15:37:33.340370+0000","last_became_peered":"2024-06-28T15:37:33.340370+0000","last_unstale":"2024-06-28T16:11:25.507957+0000","last_undegraded":"2024-06-28T16:11:25.507957+0000","last_fullsized":"2024-06-28T16:11:25.507957+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:07:43.988727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"257'3","reported_seq":48,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.071850+0000","last_change":"2024-06-28T15:21:11.104826+0000","last_active":"2024-06-28T16:10:15.071850+0000","last_peered":"2024-06-28T16:10:15.071850+0000","last_clean":"2024-06-28T16:10:15.071850+0000","last_became_active":"2024-06-28T15:21:11.104038+0000","last_became_peered":"2024-06-28T15:21:11.104038+0000","last_unstale":"2024-06-28T16:10:15.071850+0000","last_undegraded":"2024-06-28T16:10:15.071850+0000","last_fullsized":"2024-06-28T16:10:15.071850+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:38:01.314736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"257'8","reported_seq":71,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.064325+0000","last_change":"2024-06-28T15:21:07.010980+0000","last_active":"2024-06-28T16:10:10.064325+0000","last_peered":"2024-06-28T16:10:10.064325+0000","last_clean":"2024-06-28T16:10:10.064325+0000","last_became_active":"2024-06-28T15:21:07.008899+0000","last_became_peered":"2024-06-28T15:21:07.008899+0000","last_unstale":"2024-06-28T16:10:10.064325+0000","last_undegraded":"2024-06-28T16:10:10.064325+0000","last_fullsized":"2024-06-28T16:10:10.064325+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:39:00.266882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":7,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"260'42","reported_seq":165,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.502745+0000","last_change":"2024-06-28T15:17:24.020041+0000","last_active":"2024-06-28T16:11:05.502745+0000","last_peered":"2024-06-28T16:11:05.502745+0000","last_clean":"2024-06-28T16:11:05.502745+0000","last_became_active":"2024-06-28T15:17:24.019180+0000","last_became_peered":"2024-06-28T15:17:24.019180+0000","last_unstale":"2024-06-28T16:11:05.502745+0000","last_undegraded":"2024-06-28T16:11:05.502745+0000","last_fullsized":"2024-06-28T16:11:05.502745+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:00:51.600204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":198,"num_read_kb":139,"num_write":93,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.280511+0000","last_change":"2024-06-28T15:37:33.342459+0000","last_active":"2024-06-28T16:12:25.280511+0000","last_peered":"2024-06-28T16:12:25.280511+0000","last_clean":"2024-06-28T16:12:25.280511+0000","last_became_active":"2024-06-28T15:37:33.341716+0000","last_became_peered":"2024-06-28T15:37:33.341716+0000","last_unstale":"2024-06-28T16:12:25.280511+0000","last_undegraded":"2024-06-28T16:12:25.280511+0000","last_fullsized":"2024-06-28T16:12:25.280511+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:53:31.136353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:40.497394+0000","last_change":"2024-06-28T15:21:11.094239+0000","last_active":"2024-06-28T16:10:40.497394+0000","last_peered":"2024-06-28T16:10:40.497394+0000","last_clean":"2024-06-28T16:10:40.497394+0000","last_became_active":"2024-06-28T15:21:11.093276+0000","last_became_peered":"2024-06-28T15:21:11.093276+0000","last_unstale":"2024-06-28T16:10:40.497394+0000","last_undegraded":"2024-06-28T16:10:40.497394+0000","last_fullsized":"2024-06-28T16:10:40.497394+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:48:45.762370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"260'9","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.368805+0000","last_change":"2024-06-28T15:21:07.041391+0000","last_active":"2024-06-28T16:14:33.368805+0000","last_peered":"2024-06-28T16:14:33.368805+0000","last_clean":"2024-06-28T16:14:33.368805+0000","last_became_active":"2024-06-28T15:21:07.040691+0000","last_became_peered":"2024-06-28T15:21:07.040691+0000","last_unstale":"2024-06-28T16:14:33.368805+0000","last_undegraded":"2024-06-28T16:14:33.368805+0000","last_fullsized":"2024-06-28T16:14:33.368805+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:20:52.003993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":17408,"num_write":9,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"260'47","reported_seq":184,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.509222+0000","last_change":"2024-06-28T15:17:24.028252+0000","last_active":"2024-06-28T16:11:30.509222+0000","last_peered":"2024-06-28T16:11:30.509222+0000","last_clean":"2024-06-28T16:11:30.509222+0000","last_became_active":"2024-06-28T15:17:24.027610+0000","last_became_peered":"2024-06-28T15:17:24.027610+0000","last_unstale":"2024-06-28T16:11:30.509222+0000","last_undegraded":"2024-06-28T16:11:30.509222+0000","last_fullsized":"2024-06-28T16:11:30.509222+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:08:18.788559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":231,"num_read_kb":169,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.094821+0000","last_change":"2024-06-28T15:37:33.347375+0000","last_active":"2024-06-28T16:11:20.094821+0000","last_peered":"2024-06-28T16:11:20.094821+0000","last_clean":"2024-06-28T16:11:20.094821+0000","last_became_active":"2024-06-28T15:37:33.346414+0000","last_became_peered":"2024-06-28T15:37:33.346414+0000","last_unstale":"2024-06-28T16:11:20.094821+0000","last_undegraded":"2024-06-28T16:11:20.094821+0000","last_fullsized":"2024-06-28T16:11:20.094821+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:39:51.219545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"257'5","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.082722+0000","last_change":"2024-06-28T15:22:00.970955+0000","last_active":"2024-06-28T16:11:30.082722+0000","last_peered":"2024-06-28T16:11:30.082722+0000","last_clean":"2024-06-28T16:11:30.082722+0000","last_became_active":"2024-06-28T15:22:00.940760+0000","last_became_peered":"2024-06-28T15:22:00.940760+0000","last_unstale":"2024-06-28T16:11:30.082722+0000","last_undegraded":"2024-06-28T16:11:30.082722+0000","last_fullsized":"2024-06-28T16:11:30.082722+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:14:23.932998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"260'3","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.287511+0000","last_change":"2024-06-28T15:21:07.031060+0000","last_active":"2024-06-28T16:12:55.287511+0000","last_peered":"2024-06-28T16:12:55.287511+0000","last_clean":"2024-06-28T16:12:55.287511+0000","last_became_active":"2024-06-28T15:21:07.030163+0000","last_became_peered":"2024-06-28T15:21:07.030163+0000","last_unstale":"2024-06-28T16:12:55.287511+0000","last_undegraded":"2024-06-28T16:12:55.287511+0000","last_fullsized":"2024-06-28T16:12:55.287511+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:19:20.592250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"260'40","reported_seq":171,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.081619+0000","last_change":"2024-06-28T15:17:24.014719+0000","last_active":"2024-06-28T16:11:25.081619+0000","last_peered":"2024-06-28T16:11:25.081619+0000","last_clean":"2024-06-28T16:11:25.081619+0000","last_became_active":"2024-06-28T15:17:24.012572+0000","last_became_peered":"2024-06-28T15:17:24.012572+0000","last_unstale":"2024-06-28T16:11:25.081619+0000","last_undegraded":"2024-06-28T16:11:25.081619+0000","last_fullsized":"2024-06-28T16:11:25.081619+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:07:56.306439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":148,"num_write":94,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.099240+0000","last_change":"2024-06-28T15:37:33.353545+0000","last_active":"2024-06-28T16:12:15.099240+0000","last_peered":"2024-06-28T16:12:15.099240+0000","last_clean":"2024-06-28T16:12:15.099240+0000","last_became_active":"2024-06-28T15:37:33.352959+0000","last_became_peered":"2024-06-28T15:37:33.352959+0000","last_unstale":"2024-06-28T16:12:15.099240+0000","last_undegraded":"2024-06-28T16:12:15.099240+0000","last_fullsized":"2024-06-28T16:12:15.099240+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:42.580937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.092127+0000","last_change":"2024-06-28T15:21:11.079730+0000","last_active":"2024-06-28T16:11:00.092127+0000","last_peered":"2024-06-28T16:11:00.092127+0000","last_clean":"2024-06-28T16:11:00.092127+0000","last_became_active":"2024-06-28T15:21:11.079036+0000","last_became_peered":"2024-06-28T15:21:11.079036+0000","last_unstale":"2024-06-28T16:11:00.092127+0000","last_undegraded":"2024-06-28T16:11:00.092127+0000","last_fullsized":"2024-06-28T16:11:00.092127+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:36:54.128714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"0'0","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:40.551262+0000","last_change":"2024-06-28T15:21:07.028297+0000","last_active":"2024-06-28T16:10:40.551262+0000","last_peered":"2024-06-28T16:10:40.551262+0000","last_clean":"2024-06-28T16:10:40.551262+0000","last_became_active":"2024-06-28T15:21:07.027451+0000","last_became_peered":"2024-06-28T15:21:07.027451+0000","last_unstale":"2024-06-28T16:10:40.551262+0000","last_undegraded":"2024-06-28T16:10:40.551262+0000","last_fullsized":"2024-06-28T16:10:40.551262+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:44:47.388468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"260'16","reported_seq":97,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.087601+0000","last_change":"2024-06-28T15:17:24.027026+0000","last_active":"2024-06-28T16:11:25.087601+0000","last_peered":"2024-06-28T16:11:25.087601+0000","last_clean":"2024-06-28T16:11:25.087601+0000","last_became_active":"2024-06-28T15:17:24.026271+0000","last_became_peered":"2024-06-28T15:17:24.026271+0000","last_unstale":"2024-06-28T16:11:25.087601+0000","last_undegraded":"2024-06-28T16:11:25.087601+0000","last_fullsized":"2024-06-28T16:11:25.087601+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:23:12.426770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":48,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.107001+0000","last_change":"2024-06-28T15:37:33.326639+0000","last_active":"2024-06-28T16:12:15.107001+0000","last_peered":"2024-06-28T16:12:15.107001+0000","last_clean":"2024-06-28T16:12:15.107001+0000","last_became_active":"2024-06-28T15:37:33.325849+0000","last_became_peered":"2024-06-28T15:37:33.325849+0000","last_unstale":"2024-06-28T16:12:15.107001+0000","last_undegraded":"2024-06-28T16:12:15.107001+0000","last_fullsized":"2024-06-28T16:12:15.107001+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:14:13.968785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"257'5","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:40.497479+0000","last_change":"2024-06-28T15:21:11.108377+0000","last_active":"2024-06-28T16:10:40.497479+0000","last_peered":"2024-06-28T16:10:40.497479+0000","last_clean":"2024-06-28T16:10:40.497479+0000","last_became_active":"2024-06-28T15:21:11.107498+0000","last_became_peered":"2024-06-28T15:21:11.107498+0000","last_unstale":"2024-06-28T16:10:40.497479+0000","last_undegraded":"2024-06-28T16:10:40.497479+0000","last_fullsized":"2024-06-28T16:10:40.497479+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:33:14.411962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"257'7","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.091049+0000","last_change":"2024-06-28T15:21:07.030359+0000","last_active":"2024-06-28T16:10:55.091049+0000","last_peered":"2024-06-28T16:10:55.091049+0000","last_clean":"2024-06-28T16:10:55.091049+0000","last_became_active":"2024-06-28T15:21:07.029525+0000","last_became_peered":"2024-06-28T15:21:07.029525+0000","last_unstale":"2024-06-28T16:10:55.091049+0000","last_undegraded":"2024-06-28T16:10:55.091049+0000","last_fullsized":"2024-06-28T16:10:55.091049+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:27:20.230827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":11,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"260'34","reported_seq":151,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.268692+0000","last_change":"2024-06-28T15:17:24.008356+0000","last_active":"2024-06-28T16:11:25.268692+0000","last_peered":"2024-06-28T16:11:25.268692+0000","last_clean":"2024-06-28T16:11:25.268692+0000","last_became_active":"2024-06-28T15:17:24.007649+0000","last_became_peered":"2024-06-28T15:17:24.007649+0000","last_unstale":"2024-06-28T16:11:25.268692+0000","last_undegraded":"2024-06-28T16:11:25.268692+0000","last_fullsized":"2024-06-28T16:11:25.268692+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:41:30.857517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":172,"num_read_kb":122,"num_write":76,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.506787+0000","last_change":"2024-06-28T15:37:33.335759+0000","last_active":"2024-06-28T16:11:20.506787+0000","last_peered":"2024-06-28T16:11:20.506787+0000","last_clean":"2024-06-28T16:11:20.506787+0000","last_became_active":"2024-06-28T15:37:33.334082+0000","last_became_peered":"2024-06-28T15:37:33.334082+0000","last_unstale":"2024-06-28T16:11:20.506787+0000","last_undegraded":"2024-06-28T16:11:20.506787+0000","last_fullsized":"2024-06-28T16:11:20.506787+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:37:25.552446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"0'0","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.083900+0000","last_change":"2024-06-28T15:21:11.108205+0000","last_active":"2024-06-28T16:11:05.083900+0000","last_peered":"2024-06-28T16:11:05.083900+0000","last_clean":"2024-06-28T16:11:05.083900+0000","last_became_active":"2024-06-28T15:21:11.107686+0000","last_became_peered":"2024-06-28T15:21:11.107686+0000","last_unstale":"2024-06-28T16:11:05.083900+0000","last_undegraded":"2024-06-28T16:11:05.083900+0000","last_fullsized":"2024-06-28T16:11:05.083900+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:57:49.231179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"0'0","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.555131+0000","last_change":"2024-06-28T15:21:07.033578+0000","last_active":"2024-06-28T16:11:00.555131+0000","last_peered":"2024-06-28T16:11:00.555131+0000","last_clean":"2024-06-28T16:11:00.555131+0000","last_became_active":"2024-06-28T15:21:07.032719+0000","last_became_peered":"2024-06-28T15:21:07.032719+0000","last_unstale":"2024-06-28T16:11:00.555131+0000","last_undegraded":"2024-06-28T16:11:00.555131+0000","last_fullsized":"2024-06-28T16:11:00.555131+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:15.597833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"260'45","reported_seq":182,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.561334+0000","last_change":"2024-06-28T15:17:24.030065+0000","last_active":"2024-06-28T16:11:25.561334+0000","last_peered":"2024-06-28T16:11:25.561334+0000","last_clean":"2024-06-28T16:11:25.561334+0000","last_became_active":"2024-06-28T15:17:24.029532+0000","last_became_peered":"2024-06-28T15:17:24.029532+0000","last_unstale":"2024-06-28T16:11:25.561334+0000","last_undegraded":"2024-06-28T16:11:25.561334+0000","last_fullsized":"2024-06-28T16:11:25.561334+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:32:40.759334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":234,"num_read_kb":165,"num_write":86,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.097302+0000","last_change":"2024-06-28T15:37:33.348528+0000","last_active":"2024-06-28T16:11:30.097302+0000","last_peered":"2024-06-28T16:11:30.097302+0000","last_clean":"2024-06-28T16:11:30.097302+0000","last_became_active":"2024-06-28T15:37:33.347691+0000","last_became_peered":"2024-06-28T15:37:33.347691+0000","last_unstale":"2024-06-28T16:11:30.097302+0000","last_undegraded":"2024-06-28T16:11:30.097302+0000","last_fullsized":"2024-06-28T16:11:30.097302+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:19:46.828298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.089675+0000","last_change":"2024-06-28T15:21:11.092385+0000","last_active":"2024-06-28T16:11:00.089675+0000","last_peered":"2024-06-28T16:11:00.089675+0000","last_clean":"2024-06-28T16:11:00.089675+0000","last_became_active":"2024-06-28T15:21:11.091656+0000","last_became_peered":"2024-06-28T15:21:11.091656+0000","last_unstale":"2024-06-28T16:11:00.089675+0000","last_undegraded":"2024-06-28T16:11:00.089675+0000","last_fullsized":"2024-06-28T16:11:00.089675+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:39:05.106486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"260'5","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.120386+0000","last_change":"2024-06-28T15:21:07.030338+0000","last_active":"2024-06-28T16:13:10.120386+0000","last_peered":"2024-06-28T16:13:10.120386+0000","last_clean":"2024-06-28T16:13:10.120386+0000","last_became_active":"2024-06-28T15:21:07.029495+0000","last_became_peered":"2024-06-28T15:21:07.029495+0000","last_unstale":"2024-06-28T16:13:10.120386+0000","last_undegraded":"2024-06-28T16:13:10.120386+0000","last_fullsized":"2024-06-28T16:13:10.120386+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:18:26.892995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":318,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"260'39","reported_seq":162,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.101407+0000","last_change":"2024-06-28T15:17:24.024944+0000","last_active":"2024-06-28T16:11:50.101407+0000","last_peered":"2024-06-28T16:11:50.101407+0000","last_clean":"2024-06-28T16:11:50.101407+0000","last_became_active":"2024-06-28T15:17:24.021200+0000","last_became_peered":"2024-06-28T15:17:24.021200+0000","last_unstale":"2024-06-28T16:11:50.101407+0000","last_undegraded":"2024-06-28T16:11:50.101407+0000","last_fullsized":"2024-06-28T16:11:50.101407+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:06:00.686781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":191,"num_read_kb":133,"num_write":87,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"260'1","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.094873+0000","last_change":"2024-06-28T15:37:33.336238+0000","last_active":"2024-06-28T16:11:20.094873+0000","last_peered":"2024-06-28T16:11:20.094873+0000","last_clean":"2024-06-28T16:11:20.094873+0000","last_became_active":"2024-06-28T15:37:33.335464+0000","last_became_peered":"2024-06-28T15:37:33.335464+0000","last_unstale":"2024-06-28T16:11:20.094873+0000","last_undegraded":"2024-06-28T16:11:20.094873+0000","last_fullsized":"2024-06-28T16:11:20.094873+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:19:13.910107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"257'3","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:45.088990+0000","last_change":"2024-06-28T15:21:11.094534+0000","last_active":"2024-06-28T16:10:45.088990+0000","last_peered":"2024-06-28T16:10:45.088990+0000","last_clean":"2024-06-28T16:10:45.088990+0000","last_became_active":"2024-06-28T15:21:11.091647+0000","last_became_peered":"2024-06-28T15:21:11.091647+0000","last_unstale":"2024-06-28T16:10:45.088990+0000","last_undegraded":"2024-06-28T16:10:45.088990+0000","last_fullsized":"2024-06-28T16:10:45.088990+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:26:33.093064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"257'3","reported_seq":57,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.557369+0000","last_change":"2024-06-28T15:21:07.046952+0000","last_active":"2024-06-28T16:11:10.557369+0000","last_peered":"2024-06-28T16:11:10.557369+0000","last_clean":"2024-06-28T16:11:10.557369+0000","last_became_active":"2024-06-28T15:21:07.046218+0000","last_became_peered":"2024-06-28T15:21:07.046218+0000","last_unstale":"2024-06-28T16:11:10.557369+0000","last_undegraded":"2024-06-28T16:11:10.557369+0000","last_fullsized":"2024-06-28T16:11:10.557369+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:30:18.956581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"260'54","reported_seq":208,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.510120+0000","last_change":"2024-06-28T15:17:24.011432+0000","last_active":"2024-06-28T16:11:35.510120+0000","last_peered":"2024-06-28T16:11:35.510120+0000","last_clean":"2024-06-28T16:11:35.510120+0000","last_became_active":"2024-06-28T15:17:24.010230+0000","last_became_peered":"2024-06-28T15:17:24.010230+0000","last_unstale":"2024-06-28T16:11:35.510120+0000","last_undegraded":"2024-06-28T16:11:35.510120+0000","last_fullsized":"2024-06-28T16:11:35.510120+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:14:20.556616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":283,"num_read_kb":205,"num_write":97,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.560195+0000","last_change":"2024-06-28T15:37:33.343356+0000","last_active":"2024-06-28T16:11:20.560195+0000","last_peered":"2024-06-28T16:11:20.560195+0000","last_clean":"2024-06-28T16:11:20.560195+0000","last_became_active":"2024-06-28T15:37:33.342585+0000","last_became_peered":"2024-06-28T15:37:33.342585+0000","last_unstale":"2024-06-28T16:11:20.560195+0000","last_undegraded":"2024-06-28T16:11:20.560195+0000","last_fullsized":"2024-06-28T16:11:20.560195+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:36:10.405028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"260'3","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:25.123229+0000","last_change":"2024-06-28T15:21:11.098149+0000","last_active":"2024-06-28T16:13:25.123229+0000","last_peered":"2024-06-28T16:13:25.123229+0000","last_clean":"2024-06-28T16:13:25.123229+0000","last_became_active":"2024-06-28T15:21:11.096884+0000","last_became_peered":"2024-06-28T15:21:11.096884+0000","last_unstale":"2024-06-28T16:13:25.123229+0000","last_undegraded":"2024-06-28T16:13:25.123229+0000","last_fullsized":"2024-06-28T16:13:25.123229+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:53:18.580625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"260'11","reported_seq":79,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.109271+0000","last_change":"2024-06-28T15:21:07.037099+0000","last_active":"2024-06-28T16:12:55.109271+0000","last_peered":"2024-06-28T16:12:55.109271+0000","last_clean":"2024-06-28T16:12:55.109271+0000","last_became_active":"2024-06-28T15:21:07.032545+0000","last_became_peered":"2024-06-28T15:21:07.032545+0000","last_unstale":"2024-06-28T16:12:55.109271+0000","last_undegraded":"2024-06-28T16:12:55.109271+0000","last_fullsized":"2024-06-28T16:12:55.109271+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:28:14.596619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":35842,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"260'40","reported_seq":171,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089612+0000","last_change":"2024-06-28T15:17:24.028816+0000","last_active":"2024-06-28T16:11:35.089612+0000","last_peered":"2024-06-28T16:11:35.089612+0000","last_clean":"2024-06-28T16:11:35.089612+0000","last_became_active":"2024-06-28T15:17:24.026702+0000","last_became_peered":"2024-06-28T15:17:24.026702+0000","last_unstale":"2024-06-28T16:11:35.089612+0000","last_undegraded":"2024-06-28T16:11:35.089612+0000","last_fullsized":"2024-06-28T16:11:35.089612+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:13:09.048722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":150,"num_write":83,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.088350+0000","last_change":"2024-06-28T15:37:33.343533+0000","last_active":"2024-06-28T16:10:55.088350+0000","last_peered":"2024-06-28T16:10:55.088350+0000","last_clean":"2024-06-28T16:10:55.088350+0000","last_became_active":"2024-06-28T15:37:33.342889+0000","last_became_peered":"2024-06-28T15:37:33.342889+0000","last_unstale":"2024-06-28T16:10:55.088350+0000","last_undegraded":"2024-06-28T16:10:55.088350+0000","last_fullsized":"2024-06-28T16:10:55.088350+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:44:29.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"260'1","reported_seq":49,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.561434+0000","last_change":"2024-06-28T15:21:11.093441+0000","last_active":"2024-06-28T16:11:25.561434+0000","last_peered":"2024-06-28T16:11:25.561434+0000","last_clean":"2024-06-28T16:11:25.561434+0000","last_became_active":"2024-06-28T15:21:11.092614+0000","last_became_peered":"2024-06-28T15:21:11.092614+0000","last_unstale":"2024-06-28T16:11:25.561434+0000","last_undegraded":"2024-06-28T16:11:25.561434+0000","last_fullsized":"2024-06-28T16:11:25.561434+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:43:32.181244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3145728,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":3072,"num_write":1,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"257'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.092406+0000","last_change":"2024-06-28T15:21:07.035627+0000","last_active":"2024-06-28T16:11:10.092406+0000","last_peered":"2024-06-28T16:11:10.092406+0000","last_clean":"2024-06-28T16:11:10.092406+0000","last_became_active":"2024-06-28T15:21:07.034845+0000","last_became_peered":"2024-06-28T15:21:07.034845+0000","last_unstale":"2024-06-28T16:11:10.092406+0000","last_undegraded":"2024-06-28T16:11:10.092406+0000","last_fullsized":"2024-06-28T16:11:10.092406+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:39:02.467227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"260'28","reported_seq":139,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.563793+0000","last_change":"2024-06-28T15:17:24.016960+0000","last_active":"2024-06-28T16:11:35.563793+0000","last_peered":"2024-06-28T16:11:35.563793+0000","last_clean":"2024-06-28T16:11:35.563793+0000","last_became_active":"2024-06-28T15:17:24.014034+0000","last_became_peered":"2024-06-28T15:17:24.014034+0000","last_unstale":"2024-06-28T16:11:35.563793+0000","last_undegraded":"2024-06-28T16:11:35.563793+0000","last_fullsized":"2024-06-28T16:11:35.563793+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:12:33.973492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":149,"num_read_kb":105,"num_write":55,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.074736+0000","last_change":"2024-06-28T15:37:33.340550+0000","last_active":"2024-06-28T16:10:55.074736+0000","last_peered":"2024-06-28T16:10:55.074736+0000","last_clean":"2024-06-28T16:10:55.074736+0000","last_became_active":"2024-06-28T15:37:33.339831+0000","last_became_peered":"2024-06-28T15:37:33.339831+0000","last_unstale":"2024-06-28T16:10:55.074736+0000","last_undegraded":"2024-06-28T16:10:55.074736+0000","last_fullsized":"2024-06-28T16:10:55.074736+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:10:11.303905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"260'6","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.558788+0000","last_change":"2024-06-28T15:21:11.096084+0000","last_active":"2024-06-28T16:11:15.558788+0000","last_peered":"2024-06-28T16:11:15.558788+0000","last_clean":"2024-06-28T16:11:15.558788+0000","last_became_active":"2024-06-28T15:21:11.088011+0000","last_became_peered":"2024-06-28T15:21:11.088011+0000","last_unstale":"2024-06-28T16:11:15.558788+0000","last_undegraded":"2024-06-28T16:11:15.558788+0000","last_fullsized":"2024-06-28T16:11:15.558788+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:14:39.791560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5283025,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":20522,"num_write":10,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"0'0","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.268746+0000","last_change":"2024-06-28T15:21:07.031400+0000","last_active":"2024-06-28T16:11:25.268746+0000","last_peered":"2024-06-28T16:11:25.268746+0000","last_clean":"2024-06-28T16:11:25.268746+0000","last_became_active":"2024-06-28T15:21:07.030690+0000","last_became_peered":"2024-06-28T15:21:07.030690+0000","last_unstale":"2024-06-28T16:11:25.268746+0000","last_undegraded":"2024-06-28T16:11:25.268746+0000","last_fullsized":"2024-06-28T16:11:25.268746+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:42:25.816657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"260'31","reported_seq":144,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089847+0000","last_change":"2024-06-28T15:17:24.026986+0000","last_active":"2024-06-28T16:11:35.089847+0000","last_peered":"2024-06-28T16:11:35.089847+0000","last_clean":"2024-06-28T16:11:35.089847+0000","last_became_active":"2024-06-28T15:17:24.026404+0000","last_became_peered":"2024-06-28T15:17:24.026404+0000","last_unstale":"2024-06-28T16:11:35.089847+0000","last_undegraded":"2024-06-28T16:11:35.089847+0000","last_fullsized":"2024-06-28T16:11:35.089847+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:08:40.750828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":161,"num_read_kb":111,"num_write":59,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:45.101310+0000","last_change":"2024-06-28T15:37:33.332984+0000","last_active":"2024-06-28T16:11:45.101310+0000","last_peered":"2024-06-28T16:11:45.101310+0000","last_clean":"2024-06-28T16:11:45.101310+0000","last_became_active":"2024-06-28T15:37:33.332245+0000","last_became_peered":"2024-06-28T15:37:33.332245+0000","last_unstale":"2024-06-28T16:11:45.101310+0000","last_undegraded":"2024-06-28T16:11:45.101310+0000","last_fullsized":"2024-06-28T16:11:45.101310+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:30.067085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.090010+0000","last_change":"2024-06-28T15:21:11.097568+0000","last_active":"2024-06-28T16:11:00.090010+0000","last_peered":"2024-06-28T16:11:00.090010+0000","last_clean":"2024-06-28T16:11:00.090010+0000","last_became_active":"2024-06-28T15:21:11.096926+0000","last_became_peered":"2024-06-28T15:21:11.096926+0000","last_unstale":"2024-06-28T16:11:00.090010+0000","last_undegraded":"2024-06-28T16:11:00.090010+0000","last_fullsized":"2024-06-28T16:11:00.090010+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:33:16.290705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"260'2","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.105821+0000","last_change":"2024-06-28T15:21:07.037522+0000","last_active":"2024-06-28T16:13:10.105821+0000","last_peered":"2024-06-28T16:13:10.105821+0000","last_clean":"2024-06-28T16:13:10.105821+0000","last_became_active":"2024-06-28T15:21:07.036735+0000","last_became_peered":"2024-06-28T15:21:07.036735+0000","last_unstale":"2024-06-28T16:13:10.105821+0000","last_undegraded":"2024-06-28T16:13:10.105821+0000","last_fullsized":"2024-06-28T16:13:10.105821+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:56:53.756000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"260'49","reported_seq":197,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.101490+0000","last_change":"2024-06-28T15:17:24.026580+0000","last_active":"2024-06-28T16:11:50.101490+0000","last_peered":"2024-06-28T16:11:50.101490+0000","last_clean":"2024-06-28T16:11:50.101490+0000","last_became_active":"2024-06-28T15:17:24.025855+0000","last_became_peered":"2024-06-28T15:17:24.025855+0000","last_unstale":"2024-06-28T16:11:50.101490+0000","last_undegraded":"2024-06-28T16:11:50.101490+0000","last_fullsized":"2024-06-28T16:11:50.101490+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:14:11.100104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":190,"num_write":87,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.081854+0000","last_change":"2024-06-28T15:37:33.323701+0000","last_active":"2024-06-28T16:10:55.081854+0000","last_peered":"2024-06-28T16:10:55.081854+0000","last_clean":"2024-06-28T16:10:55.081854+0000","last_became_active":"2024-06-28T15:37:33.322927+0000","last_became_peered":"2024-06-28T15:37:33.322927+0000","last_unstale":"2024-06-28T16:10:55.081854+0000","last_undegraded":"2024-06-28T16:10:55.081854+0000","last_fullsized":"2024-06-28T16:10:55.081854+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:33:37.474011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.084125+0000","last_change":"2024-06-28T15:21:11.098521+0000","last_active":"2024-06-28T16:11:05.084125+0000","last_peered":"2024-06-28T16:11:05.084125+0000","last_clean":"2024-06-28T16:11:05.084125+0000","last_became_active":"2024-06-28T15:21:11.097766+0000","last_became_peered":"2024-06-28T15:21:11.097766+0000","last_unstale":"2024-06-28T16:11:05.084125+0000","last_undegraded":"2024-06-28T16:11:05.084125+0000","last_fullsized":"2024-06-28T16:11:05.084125+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:49:28.598405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.090053+0000","last_change":"2024-06-28T15:21:07.040142+0000","last_active":"2024-06-28T16:11:00.090053+0000","last_peered":"2024-06-28T16:11:00.090053+0000","last_clean":"2024-06-28T16:11:00.090053+0000","last_became_active":"2024-06-28T15:21:07.039381+0000","last_became_peered":"2024-06-28T15:21:07.039381+0000","last_unstale":"2024-06-28T16:11:00.090053+0000","last_undegraded":"2024-06-28T16:11:00.090053+0000","last_fullsized":"2024-06-28T16:11:00.090053+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:56:40.727136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"260'26","reported_seq":137,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.099274+0000","last_change":"2024-06-28T15:17:24.027314+0000","last_active":"2024-06-28T16:11:35.099274+0000","last_peered":"2024-06-28T16:11:35.099274+0000","last_clean":"2024-06-28T16:11:35.099274+0000","last_became_active":"2024-06-28T15:17:24.026580+0000","last_became_peered":"2024-06-28T15:17:24.026580+0000","last_unstale":"2024-06-28T16:11:35.099274+0000","last_undegraded":"2024-06-28T16:11:35.099274+0000","last_fullsized":"2024-06-28T16:11:35.099274+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:45:13.623723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":143,"num_read_kb":101,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"259'1","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.278572+0000","last_change":"2024-06-28T15:37:33.349810+0000","last_active":"2024-06-28T16:12:15.278572+0000","last_peered":"2024-06-28T16:12:15.278572+0000","last_clean":"2024-06-28T16:12:15.278572+0000","last_became_active":"2024-06-28T15:37:33.349137+0000","last_became_peered":"2024-06-28T15:37:33.349137+0000","last_unstale":"2024-06-28T16:12:15.278572+0000","last_undegraded":"2024-06-28T16:12:15.278572+0000","last_fullsized":"2024-06-28T16:12:15.278572+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:47:15.998763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"260'6","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.514776+0000","last_change":"2024-06-28T15:21:11.106004+0000","last_active":"2024-06-28T16:14:32.514776+0000","last_peered":"2024-06-28T16:14:32.514776+0000","last_clean":"2024-06-28T16:14:32.514776+0000","last_became_active":"2024-06-28T15:21:11.104270+0000","last_became_peered":"2024-06-28T15:21:11.104270+0000","last_unstale":"2024-06-28T16:14:32.514776+0000","last_undegraded":"2024-06-28T16:14:32.514776+0000","last_fullsized":"2024-06-28T16:14:32.514776+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:22:50.569140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":11,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"260'6","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.311951+0000","last_change":"2024-06-28T15:21:07.049332+0000","last_active":"2024-06-28T16:14:32.311951+0000","last_peered":"2024-06-28T16:14:32.311951+0000","last_clean":"2024-06-28T16:14:32.311951+0000","last_became_active":"2024-06-28T15:21:07.048639+0000","last_became_peered":"2024-06-28T15:21:07.048639+0000","last_unstale":"2024-06-28T16:14:32.311951+0000","last_undegraded":"2024-06-28T16:14:32.311951+0000","last_fullsized":"2024-06-28T16:14:32.311951+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:02:29.473659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":12288,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"260'22","reported_seq":114,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.083766+0000","last_change":"2024-06-28T15:17:24.007520+0000","last_active":"2024-06-28T16:11:35.083766+0000","last_peered":"2024-06-28T16:11:35.083766+0000","last_clean":"2024-06-28T16:11:35.083766+0000","last_became_active":"2024-06-28T15:17:24.006784+0000","last_became_peered":"2024-06-28T15:17:24.006784+0000","last_unstale":"2024-06-28T16:11:35.083766+0000","last_undegraded":"2024-06-28T16:11:35.083766+0000","last_fullsized":"2024-06-28T16:11:35.083766+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:31:01.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":64,"num_write":51,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.574119+0000","last_change":"2024-06-28T15:37:33.334905+0000","last_active":"2024-06-28T16:12:15.574119+0000","last_peered":"2024-06-28T16:12:15.574119+0000","last_clean":"2024-06-28T16:12:15.574119+0000","last_became_active":"2024-06-28T15:37:33.333737+0000","last_became_peered":"2024-06-28T15:37:33.333737+0000","last_unstale":"2024-06-28T16:12:15.574119+0000","last_undegraded":"2024-06-28T16:12:15.574119+0000","last_fullsized":"2024-06-28T16:12:15.574119+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:02:20.700941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":48,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.081414+0000","last_change":"2024-06-28T15:21:11.092678+0000","last_active":"2024-06-28T16:11:25.081414+0000","last_peered":"2024-06-28T16:11:25.081414+0000","last_clean":"2024-06-28T16:11:25.081414+0000","last_became_active":"2024-06-28T15:21:11.092034+0000","last_became_peered":"2024-06-28T15:21:11.092034+0000","last_unstale":"2024-06-28T16:11:25.081414+0000","last_undegraded":"2024-06-28T16:11:25.081414+0000","last_fullsized":"2024-06-28T16:11:25.081414+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:08:58.423355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"260'3","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.120874+0000","last_change":"2024-06-28T15:21:07.040425+0000","last_active":"2024-06-28T16:13:15.120874+0000","last_peered":"2024-06-28T16:13:15.120874+0000","last_clean":"2024-06-28T16:13:15.120874+0000","last_became_active":"2024-06-28T15:21:07.039812+0000","last_became_peered":"2024-06-28T15:21:07.039812+0000","last_unstale":"2024-06-28T16:13:15.120874+0000","last_undegraded":"2024-06-28T16:13:15.120874+0000","last_fullsized":"2024-06-28T16:13:15.120874+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:50:26.224970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"260'77","reported_seq":307,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.090583+0000","last_change":"2024-06-28T15:17:24.013637+0000","last_active":"2024-06-28T16:11:35.090583+0000","last_peered":"2024-06-28T16:11:35.090583+0000","last_clean":"2024-06-28T16:11:35.090583+0000","last_became_active":"2024-06-28T15:17:24.012854+0000","last_became_peered":"2024-06-28T15:17:24.012854+0000","last_unstale":"2024-06-28T16:11:35.090583+0000","last_undegraded":"2024-06-28T16:11:35.090583+0000","last_fullsized":"2024-06-28T16:11:35.090583+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:10:40.982270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":483,"num_read_kb":330,"num_write":109,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.554385+0000","last_change":"2024-06-28T15:37:33.323089+0000","last_active":"2024-06-28T16:10:55.554385+0000","last_peered":"2024-06-28T16:10:55.554385+0000","last_clean":"2024-06-28T16:10:55.554385+0000","last_became_active":"2024-06-28T15:37:33.322317+0000","last_became_peered":"2024-06-28T15:37:33.322317+0000","last_unstale":"2024-06-28T16:10:55.554385+0000","last_undegraded":"2024-06-28T16:10:55.554385+0000","last_fullsized":"2024-06-28T16:10:55.554385+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:18:03.061734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.085392+0000","last_change":"2024-06-28T15:21:11.109416+0000","last_active":"2024-06-28T16:11:15.085392+0000","last_peered":"2024-06-28T16:11:15.085392+0000","last_clean":"2024-06-28T16:11:15.085392+0000","last_became_active":"2024-06-28T15:21:11.108663+0000","last_became_peered":"2024-06-28T15:21:11.108663+0000","last_unstale":"2024-06-28T16:11:15.085392+0000","last_undegraded":"2024-06-28T16:11:15.085392+0000","last_fullsized":"2024-06-28T16:11:15.085392+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:07:02.854303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"260'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.121849+0000","last_change":"2024-06-28T15:21:07.022794+0000","last_active":"2024-06-28T16:13:15.121849+0000","last_peered":"2024-06-28T16:13:15.121849+0000","last_clean":"2024-06-28T16:13:15.121849+0000","last_became_active":"2024-06-28T15:21:07.022046+0000","last_became_peered":"2024-06-28T15:21:07.022046+0000","last_unstale":"2024-06-28T16:13:15.121849+0000","last_undegraded":"2024-06-28T16:13:15.121849+0000","last_fullsized":"2024-06-28T16:13:15.121849+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:49:32.800553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"260'31","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089684+0000","last_change":"2024-06-28T15:17:24.011733+0000","last_active":"2024-06-28T16:11:35.089684+0000","last_peered":"2024-06-28T16:11:35.089684+0000","last_clean":"2024-06-28T16:11:35.089684+0000","last_became_active":"2024-06-28T15:17:24.010873+0000","last_became_peered":"2024-06-28T15:17:24.010873+0000","last_unstale":"2024-06-28T16:11:35.089684+0000","last_undegraded":"2024-06-28T16:11:35.089684+0000","last_fullsized":"2024-06-28T16:11:35.089684+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:29:45.592863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":94,"num_write":78,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.091280+0000","last_change":"2024-06-28T15:37:33.347898+0000","last_active":"2024-06-28T16:11:05.091280+0000","last_peered":"2024-06-28T16:11:05.091280+0000","last_clean":"2024-06-28T16:11:05.091280+0000","last_became_active":"2024-06-28T15:37:33.347075+0000","last_became_peered":"2024-06-28T15:37:33.347075+0000","last_unstale":"2024-06-28T16:11:05.091280+0000","last_undegraded":"2024-06-28T16:11:05.091280+0000","last_fullsized":"2024-06-28T16:11:05.091280+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:36:09.889868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":47,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.086272+0000","last_change":"2024-06-28T15:21:11.093738+0000","last_active":"2024-06-28T16:11:15.086272+0000","last_peered":"2024-06-28T16:11:15.086272+0000","last_clean":"2024-06-28T16:11:15.086272+0000","last_became_active":"2024-06-28T15:21:11.093082+0000","last_became_peered":"2024-06-28T15:21:11.093082+0000","last_unstale":"2024-06-28T16:11:15.086272+0000","last_undegraded":"2024-06-28T16:11:15.086272+0000","last_fullsized":"2024-06-28T16:11:15.086272+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:58:56.828729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"257'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.084078+0000","last_change":"2024-06-28T15:21:07.041597+0000","last_active":"2024-06-28T16:11:10.084078+0000","last_peered":"2024-06-28T16:11:10.084078+0000","last_clean":"2024-06-28T16:11:10.084078+0000","last_became_active":"2024-06-28T15:21:07.040575+0000","last_became_peered":"2024-06-28T15:21:07.040575+0000","last_unstale":"2024-06-28T16:11:10.084078+0000","last_undegraded":"2024-06-28T16:11:10.084078+0000","last_fullsized":"2024-06-28T16:11:10.084078+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:41:33.611919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"260'16","reported_seq":98,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:40.099162+0000","last_change":"2024-06-28T15:17:24.024775+0000","last_active":"2024-06-28T16:11:40.099162+0000","last_peered":"2024-06-28T16:11:40.099162+0000","last_clean":"2024-06-28T16:11:40.099162+0000","last_became_active":"2024-06-28T15:17:24.021464+0000","last_became_peered":"2024-06-28T15:17:24.021464+0000","last_unstale":"2024-06-28T16:11:40.099162+0000","last_undegraded":"2024-06-28T16:11:40.099162+0000","last_fullsized":"2024-06-28T16:11:40.099162+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:33:59.157965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":44,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"260'341","reported_seq":962,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.645969+0000","last_change":"2024-06-28T15:00:59.944074+0000","last_active":"2024-06-28T16:14:34.645969+0000","last_peered":"2024-06-28T16:14:34.645969+0000","last_clean":"2024-06-28T16:14:34.645969+0000","last_became_active":"2024-06-28T15:00:59.941681+0000","last_became_peered":"2024-06-28T15:00:59.941681+0000","last_unstale":"2024-06-28T16:14:34.645969+0000","last_undegraded":"2024-06-28T16:14:34.645969+0000","last_fullsized":"2024-06-28T16:14:34.645969+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:10:45.311266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170282,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":488,"num_read_kb":26898,"num_write":1268,"num_write_kb":25985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0,5],"acting":[2,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"260'181","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.066576+0000","last_change":"2024-06-28T15:00:17.405026+0000","last_active":"2024-06-28T16:10:20.066576+0000","last_peered":"2024-06-28T16:10:20.066576+0000","last_clean":"2024-06-28T16:10:20.066576+0000","last_became_active":"2024-06-28T15:00:17.404342+0000","last_became_peered":"2024-06-28T15:00:17.404342+0000","last_unstale":"2024-06-28T16:10:20.066576+0000","last_undegraded":"2024-06-28T16:10:20.066576+0000","last_fullsized":"2024-06-28T16:10:20.066576+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":100,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:24:42.574899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":722,"num_read_kb":569,"num_write":267,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"260'326","reported_seq":971,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.742875+0000","last_change":"2024-06-28T15:00:15.419867+0000","last_active":"2024-06-28T16:14:34.742875+0000","last_peered":"2024-06-28T16:14:34.742875+0000","last_clean":"2024-06-28T16:14:34.742875+0000","last_became_active":"2024-06-28T15:00:15.419449+0000","last_became_peered":"2024-06-28T15:00:15.419449+0000","last_unstale":"2024-06-28T16:14:34.742875+0000","last_undegraded":"2024-06-28T16:14:34.742875+0000","last_fullsized":"2024-06-28T16:14:34.742875+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:27:48.933532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5526738,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":485,"num_read_kb":15789,"num_write":1172,"num_write_kb":37304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"260'285","reported_seq":939,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080422+0000","last_change":"2024-06-28T15:00:17.409108+0000","last_active":"2024-06-28T16:10:20.080422+0000","last_peered":"2024-06-28T16:10:20.080422+0000","last_clean":"2024-06-28T16:10:20.080422+0000","last_became_active":"2024-06-28T15:00:17.408128+0000","last_became_peered":"2024-06-28T15:00:17.408128+0000","last_unstale":"2024-06-28T16:10:20.080422+0000","last_undegraded":"2024-06-28T16:10:20.080422+0000","last_fullsized":"2024-06-28T16:10:20.080422+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:38:12.539083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1157,"num_read_kb":911,"num_write":439,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"260'221","reported_seq":907,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.261930+0000","last_change":"2024-06-28T15:00:15.397113+0000","last_active":"2024-06-28T16:11:55.261930+0000","last_peered":"2024-06-28T16:11:55.261930+0000","last_clean":"2024-06-28T16:11:55.261930+0000","last_became_active":"2024-06-28T15:00:15.396469+0000","last_became_peered":"2024-06-28T15:00:15.396469+0000","last_unstale":"2024-06-28T16:11:55.261930+0000","last_undegraded":"2024-06-28T16:11:55.261930+0000","last_fullsized":"2024-06-28T16:11:55.261930+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:40:36.549494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1588322,"num_objects":66,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":442,"num_read_kb":81435,"num_write":1047,"num_write_kb":14924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"260'77","reported_seq":643,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073900+0000","last_change":"2024-06-28T15:00:17.403326+0000","last_active":"2024-06-28T16:10:20.073900+0000","last_peered":"2024-06-28T16:10:20.073900+0000","last_clean":"2024-06-28T16:10:20.073900+0000","last_became_active":"2024-06-28T15:00:17.402542+0000","last_became_peered":"2024-06-28T15:00:17.402542+0000","last_unstale":"2024-06-28T16:10:20.073900+0000","last_undegraded":"2024-06-28T16:10:20.073900+0000","last_fullsized":"2024-06-28T16:10:20.073900+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:05:41.791423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":306,"num_read_kb":248,"num_write":126,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"260'313","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.510107+0000","last_change":"2024-06-28T15:00:15.435754+0000","last_active":"2024-06-28T16:14:34.510107+0000","last_peered":"2024-06-28T16:14:34.510107+0000","last_clean":"2024-06-28T16:14:34.510107+0000","last_became_active":"2024-06-28T15:00:15.435122+0000","last_became_peered":"2024-06-28T15:00:15.435122+0000","last_unstale":"2024-06-28T16:14:34.510107+0000","last_undegraded":"2024-06-28T16:14:34.510107+0000","last_fullsized":"2024-06-28T16:14:34.510107+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:23:37.223417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1454317,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":492,"num_read_kb":48678,"num_write":1275,"num_write_kb":34259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"260'51","reported_seq":629,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.546829+0000","last_change":"2024-06-28T15:00:17.385309+0000","last_active":"2024-06-28T16:10:20.546829+0000","last_peered":"2024-06-28T16:10:20.546829+0000","last_clean":"2024-06-28T16:10:20.546829+0000","last_became_active":"2024-06-28T15:00:17.384932+0000","last_became_peered":"2024-06-28T15:00:17.384932+0000","last_unstale":"2024-06-28T16:10:20.546829+0000","last_undegraded":"2024-06-28T16:10:20.546829+0000","last_fullsized":"2024-06-28T16:10:20.546829+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:12:11.868213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":197,"num_write":67,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"260'343","reported_seq":992,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.679630+0000","last_change":"2024-06-28T15:00:15.430784+0000","last_active":"2024-06-28T16:14:34.679630+0000","last_peered":"2024-06-28T16:14:34.679630+0000","last_clean":"2024-06-28T16:14:34.679630+0000","last_became_active":"2024-06-28T15:00:15.430208+0000","last_became_peered":"2024-06-28T15:00:15.430208+0000","last_unstale":"2024-06-28T16:14:34.679630+0000","last_undegraded":"2024-06-28T16:14:34.679630+0000","last_fullsized":"2024-06-28T16:14:34.679630+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:48:53.859374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437780,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":548,"num_read_kb":39482,"num_write":1215,"num_write_kb":35176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"260'185","reported_seq":837,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:59.271179+0000","last_change":"2024-06-28T15:00:17.398844+0000","last_active":"2024-06-28T16:10:59.271179+0000","last_peered":"2024-06-28T16:10:59.271179+0000","last_clean":"2024-06-28T16:10:59.271179+0000","last_became_active":"2024-06-28T15:00:17.397986+0000","last_became_peered":"2024-06-28T15:00:17.397986+0000","last_unstale":"2024-06-28T16:10:59.271179+0000","last_undegraded":"2024-06-28T16:10:59.271179+0000","last_fullsized":"2024-06-28T16:10:59.271179+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:49:08.729457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":777,"num_read_kb":632,"num_write":274,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"260'364","reported_seq":1029,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.692915+0000","last_change":"2024-06-28T15:00:15.412175+0000","last_active":"2024-06-28T16:14:34.692915+0000","last_peered":"2024-06-28T16:14:34.692915+0000","last_clean":"2024-06-28T16:14:34.692915+0000","last_became_active":"2024-06-28T15:00:15.409542+0000","last_became_peered":"2024-06-28T15:00:15.409542+0000","last_unstale":"2024-06-28T16:14:34.692915+0000","last_undegraded":"2024-06-28T16:14:34.692915+0000","last_fullsized":"2024-06-28T16:14:34.692915+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:28:00.331354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1667748,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":553,"num_read_kb":16754,"num_write":1370,"num_write_kb":32794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"260'44","reported_seq":608,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.895392+0000","last_change":"2024-06-28T15:00:17.408359+0000","last_active":"2024-06-28T16:11:10.895392+0000","last_peered":"2024-06-28T16:11:10.895392+0000","last_clean":"2024-06-28T16:11:10.895392+0000","last_became_active":"2024-06-28T15:00:17.401985+0000","last_became_peered":"2024-06-28T15:00:17.401985+0000","last_unstale":"2024-06-28T16:11:10.895392+0000","last_undegraded":"2024-06-28T16:11:10.895392+0000","last_fullsized":"2024-06-28T16:11:10.895392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:00:57.912525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":159,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"260'328","reported_seq":959,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.675966+0000","last_change":"2024-06-28T15:00:15.425314+0000","last_active":"2024-06-28T16:14:34.675966+0000","last_peered":"2024-06-28T16:14:34.675966+0000","last_clean":"2024-06-28T16:14:34.675966+0000","last_became_active":"2024-06-28T15:00:15.424430+0000","last_became_peered":"2024-06-28T15:00:15.424430+0000","last_unstale":"2024-06-28T16:14:34.675966+0000","last_undegraded":"2024-06-28T16:14:34.675966+0000","last_fullsized":"2024-06-28T16:14:34.675966+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:30:59.580102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048602,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":476,"num_read_kb":21908,"num_write":1069,"num_write_kb":36265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"260'4326","reported_seq":4996,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:30.604647+0000","last_change":"2024-06-28T15:00:17.387801+0000","last_active":"2024-06-28T16:12:30.604647+0000","last_peered":"2024-06-28T16:12:30.604647+0000","last_clean":"2024-06-28T16:12:30.604647+0000","last_became_active":"2024-06-28T15:00:17.384378+0000","last_became_peered":"2024-06-28T15:00:17.384378+0000","last_unstale":"2024-06-28T16:12:30.604647+0000","last_undegraded":"2024-06-28T16:12:30.604647+0000","last_fullsized":"2024-06-28T16:12:30.604647+0000","mapping_epoch":21,"log_start":"260'4300","ondisk_log_start":"260'4300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":3000,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:59:21.316239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15352,"num_read_kb":12408,"num_write":6557,"num_write_kb":3277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"260'411","reported_seq":1128,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.664061+0000","last_change":"2024-06-28T15:00:15.403350+0000","last_active":"2024-06-28T16:14:34.664061+0000","last_peered":"2024-06-28T16:14:34.664061+0000","last_clean":"2024-06-28T16:14:34.664061+0000","last_became_active":"2024-06-28T15:00:15.401960+0000","last_became_peered":"2024-06-28T15:00:15.401960+0000","last_unstale":"2024-06-28T16:14:34.664061+0000","last_undegraded":"2024-06-28T16:14:34.664061+0000","last_fullsized":"2024-06-28T16:14:34.664061+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":400,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:02:13.643574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324665,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":810,"num_read_kb":20615,"num_write":1466,"num_write_kb":29750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"260'70","reported_seq":685,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.247513+0000","last_change":"2024-06-28T15:00:17.381546+0000","last_active":"2024-06-28T16:14:08.247513+0000","last_peered":"2024-06-28T16:14:08.247513+0000","last_clean":"2024-06-28T16:14:08.247513+0000","last_became_active":"2024-06-28T15:00:17.381196+0000","last_became_peered":"2024-06-28T15:00:17.381196+0000","last_unstale":"2024-06-28T16:14:08.247513+0000","last_undegraded":"2024-06-28T16:14:08.247513+0000","last_fullsized":"2024-06-28T16:14:08.247513+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:47:51.139728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":309,"num_read_kb":273,"num_write":88,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"260'128","reported_seq":750,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:32.095965+0000","last_change":"2024-06-28T15:00:17.396995+0000","last_active":"2024-06-28T16:13:32.095965+0000","last_peered":"2024-06-28T16:13:32.095965+0000","last_clean":"2024-06-28T16:13:32.095965+0000","last_became_active":"2024-06-28T15:00:17.396142+0000","last_became_peered":"2024-06-28T15:00:17.396142+0000","last_unstale":"2024-06-28T16:13:32.095965+0000","last_undegraded":"2024-06-28T16:13:32.095965+0000","last_fullsized":"2024-06-28T16:13:32.095965+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:43:00.829791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":536,"num_read_kb":452,"num_write":191,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"260'416","reported_seq":1186,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.679447+0000","last_change":"2024-06-28T15:00:15.423248+0000","last_active":"2024-06-28T16:14:34.679447+0000","last_peered":"2024-06-28T16:14:34.679447+0000","last_clean":"2024-06-28T16:14:34.679447+0000","last_became_active":"2024-06-28T15:00:15.422579+0000","last_became_peered":"2024-06-28T15:00:15.422579+0000","last_unstale":"2024-06-28T16:14:34.679447+0000","last_undegraded":"2024-06-28T16:14:34.679447+0000","last_fullsized":"2024-06-28T16:14:34.679447+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":400,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:59:30.311378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":851,"num_read_kb":96606,"num_write":1464,"num_write_kb":38692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0,1],"acting":[7,5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"260'254","reported_seq":899,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080457+0000","last_change":"2024-06-28T15:00:17.410347+0000","last_active":"2024-06-28T16:10:20.080457+0000","last_peered":"2024-06-28T16:10:20.080457+0000","last_clean":"2024-06-28T16:10:20.080457+0000","last_became_active":"2024-06-28T15:00:17.409202+0000","last_became_peered":"2024-06-28T15:00:17.409202+0000","last_unstale":"2024-06-28T16:10:20.080457+0000","last_undegraded":"2024-06-28T16:10:20.080457+0000","last_fullsized":"2024-06-28T16:10:20.080457+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:33:21.410367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1059,"num_read_kb":851,"num_write":432,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"260'313","reported_seq":952,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.711448+0000","last_change":"2024-06-28T15:00:15.423492+0000","last_active":"2024-06-28T16:14:34.711448+0000","last_peered":"2024-06-28T16:14:34.711448+0000","last_clean":"2024-06-28T16:14:34.711448+0000","last_became_active":"2024-06-28T15:00:15.421980+0000","last_became_peered":"2024-06-28T15:00:15.421980+0000","last_unstale":"2024-06-28T16:14:34.711448+0000","last_undegraded":"2024-06-28T16:14:34.711448+0000","last_fullsized":"2024-06-28T16:14:34.711448+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:15:16.139016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468838,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":425,"num_read_kb":33937,"num_write":1026,"num_write_kb":43346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"260'225","reported_seq":850,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.070101+0000","last_change":"2024-06-28T15:00:17.385629+0000","last_active":"2024-06-28T16:10:35.070101+0000","last_peered":"2024-06-28T16:10:35.070101+0000","last_clean":"2024-06-28T16:10:35.070101+0000","last_became_active":"2024-06-28T15:00:17.384914+0000","last_became_peered":"2024-06-28T15:00:17.384914+0000","last_unstale":"2024-06-28T16:10:35.070101+0000","last_undegraded":"2024-06-28T16:10:35.070101+0000","last_fullsized":"2024-06-28T16:10:35.070101+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:01:45.177989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":887,"num_read_kb":706,"num_write":342,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"260'342","reported_seq":1191,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.723544+0000","last_change":"2024-06-28T15:00:15.412640+0000","last_active":"2024-06-28T16:14:34.723544+0000","last_peered":"2024-06-28T16:14:34.723544+0000","last_clean":"2024-06-28T16:14:34.723544+0000","last_became_active":"2024-06-28T15:00:15.411567+0000","last_became_peered":"2024-06-28T15:00:15.411567+0000","last_unstale":"2024-06-28T16:14:34.723544+0000","last_undegraded":"2024-06-28T16:14:34.723544+0000","last_fullsized":"2024-06-28T16:14:34.723544+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:32:11.605469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289237,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":936,"num_read_kb":14303,"num_write":1297,"num_write_kb":46873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"260'128","reported_seq":736,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:51.375718+0000","last_change":"2024-06-28T15:00:17.389926+0000","last_active":"2024-06-28T16:11:51.375718+0000","last_peered":"2024-06-28T16:11:51.375718+0000","last_clean":"2024-06-28T16:11:51.375718+0000","last_became_active":"2024-06-28T15:00:17.389613+0000","last_became_peered":"2024-06-28T15:00:17.389613+0000","last_unstale":"2024-06-28T16:11:51.375718+0000","last_undegraded":"2024-06-28T16:11:51.375718+0000","last_fullsized":"2024-06-28T16:11:51.375718+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:46:13.479655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":419,"num_write":192,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"260'320","reported_seq":1072,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.719218+0000","last_change":"2024-06-28T15:00:15.418207+0000","last_active":"2024-06-28T16:14:34.719218+0000","last_peered":"2024-06-28T16:14:34.719218+0000","last_clean":"2024-06-28T16:14:34.719218+0000","last_became_active":"2024-06-28T15:00:15.417563+0000","last_became_peered":"2024-06-28T15:00:15.417563+0000","last_unstale":"2024-06-28T16:14:34.719218+0000","last_undegraded":"2024-06-28T16:14:34.719218+0000","last_fullsized":"2024-06-28T16:14:34.719218+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:50:35.799169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":525349,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":605,"num_read_kb":12860,"num_write":1159,"num_write_kb":18563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"260'156","reported_seq":737,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.256777+0000","last_change":"2024-06-28T15:00:17.413709+0000","last_active":"2024-06-28T16:10:25.256777+0000","last_peered":"2024-06-28T16:10:25.256777+0000","last_clean":"2024-06-28T16:10:25.256777+0000","last_became_active":"2024-06-28T15:00:17.413420+0000","last_became_peered":"2024-06-28T15:00:17.413420+0000","last_unstale":"2024-06-28T16:10:25.256777+0000","last_undegraded":"2024-06-28T16:10:25.256777+0000","last_fullsized":"2024-06-28T16:10:25.256777+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:10:02.399082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":589,"num_read_kb":466,"num_write":222,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"260'357","reported_seq":1073,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.554726+0000","last_change":"2024-06-28T15:00:15.402501+0000","last_active":"2024-06-28T16:14:34.554726+0000","last_peered":"2024-06-28T16:14:34.554726+0000","last_clean":"2024-06-28T16:14:34.554726+0000","last_became_active":"2024-06-28T15:00:15.401370+0000","last_became_peered":"2024-06-28T15:00:15.401370+0000","last_unstale":"2024-06-28T16:14:34.554726+0000","last_undegraded":"2024-06-28T16:14:34.554726+0000","last_fullsized":"2024-06-28T16:14:34.554726+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:40:25.381283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324689,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":676,"num_read_kb":30262,"num_write":1536,"num_write_kb":36456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"260'253","reported_seq":902,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.053742+0000","last_change":"2024-06-28T15:00:17.410230+0000","last_active":"2024-06-28T16:13:20.053742+0000","last_peered":"2024-06-28T16:13:20.053742+0000","last_clean":"2024-06-28T16:13:20.053742+0000","last_became_active":"2024-06-28T15:00:17.408972+0000","last_became_peered":"2024-06-28T15:00:17.408972+0000","last_unstale":"2024-06-28T16:13:20.053742+0000","last_undegraded":"2024-06-28T16:13:20.053742+0000","last_fullsized":"2024-06-28T16:13:20.053742+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:48:01.967992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":990,"num_read_kb":787,"num_write":391,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"260'286","reported_seq":948,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.590888+0000","last_change":"2024-06-28T15:00:15.408546+0000","last_active":"2024-06-28T16:13:10.590888+0000","last_peered":"2024-06-28T16:13:10.590888+0000","last_clean":"2024-06-28T16:13:10.590888+0000","last_became_active":"2024-06-28T15:00:15.407552+0000","last_became_peered":"2024-06-28T15:00:15.407552+0000","last_unstale":"2024-06-28T16:13:10.590888+0000","last_undegraded":"2024-06-28T16:13:10.590888+0000","last_fullsized":"2024-06-28T16:13:10.590888+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:35:44.908220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12818085,"num_objects":69,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":453,"num_read_kb":23448,"num_write":1276,"num_write_kb":46171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"260'142","reported_seq":679,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.510437+0000","last_change":"2024-06-28T15:00:25.428551+0000","last_active":"2024-06-28T16:14:08.510437+0000","last_peered":"2024-06-28T16:14:08.510437+0000","last_clean":"2024-06-28T16:14:08.510437+0000","last_became_active":"2024-06-28T15:00:25.427734+0000","last_became_peered":"2024-06-28T15:00:25.427734+0000","last_unstale":"2024-06-28T16:14:08.510437+0000","last_undegraded":"2024-06-28T16:14:08.510437+0000","last_fullsized":"2024-06-28T16:14:08.510437+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:59:38.479139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":233,"num_read_kb":201,"num_write":347,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"260'214","reported_seq":822,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493503+0000","last_change":"2024-06-28T15:00:17.382620+0000","last_active":"2024-06-28T16:10:25.493503+0000","last_peered":"2024-06-28T16:10:25.493503+0000","last_clean":"2024-06-28T16:10:25.493503+0000","last_became_active":"2024-06-28T15:00:17.382344+0000","last_became_peered":"2024-06-28T15:00:17.382344+0000","last_unstale":"2024-06-28T16:10:25.493503+0000","last_undegraded":"2024-06-28T16:10:25.493503+0000","last_fullsized":"2024-06-28T16:10:25.493503+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":200,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:40:38.570590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":869,"num_read_kb":692,"num_write":351,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"260'292","reported_seq":944,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.632189+0000","last_change":"2024-06-28T15:00:15.423784+0000","last_active":"2024-06-28T16:14:34.632189+0000","last_peered":"2024-06-28T16:14:34.632189+0000","last_clean":"2024-06-28T16:14:34.632189+0000","last_became_active":"2024-06-28T15:00:15.420739+0000","last_became_peered":"2024-06-28T15:00:15.420739+0000","last_unstale":"2024-06-28T16:14:34.632189+0000","last_undegraded":"2024-06-28T16:14:34.632189+0000","last_fullsized":"2024-06-28T16:14:34.632189+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":200,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:38:08.814015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6022775,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":452,"num_read_kb":28797,"num_write":1102,"num_write_kb":30225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.067917+0000","last_change":"2024-06-28T15:00:19.403941+0000","last_active":"2024-06-28T16:10:25.067917+0000","last_peered":"2024-06-28T16:10:25.067917+0000","last_clean":"2024-06-28T16:10:25.067917+0000","last_became_active":"2024-06-28T15:00:19.400681+0000","last_became_peered":"2024-06-28T15:00:19.400681+0000","last_unstale":"2024-06-28T16:10:25.067917+0000","last_undegraded":"2024-06-28T16:10:25.067917+0000","last_fullsized":"2024-06-28T16:10:25.067917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:52:42.469346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"260'319","reported_seq":938,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.584308+0000","last_change":"2024-06-28T15:00:21.391364+0000","last_active":"2024-06-28T16:12:55.584308+0000","last_peered":"2024-06-28T16:12:55.584308+0000","last_clean":"2024-06-28T16:12:55.584308+0000","last_became_active":"2024-06-28T15:00:21.390609+0000","last_became_peered":"2024-06-28T15:00:21.390609+0000","last_unstale":"2024-06-28T16:12:55.584308+0000","last_undegraded":"2024-06-28T16:12:55.584308+0000","last_fullsized":"2024-06-28T16:12:55.584308+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:41:58.645414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48141,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":517,"num_read_kb":976,"num_write":400,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":493,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.082982+0000","last_change":"2024-06-28T15:00:23.408294+0000","last_active":"2024-06-28T16:10:30.082982+0000","last_peered":"2024-06-28T16:10:30.082982+0000","last_clean":"2024-06-28T16:10:30.082982+0000","last_became_active":"2024-06-28T15:00:23.407521+0000","last_became_peered":"2024-06-28T15:00:23.407521+0000","last_unstale":"2024-06-28T16:10:30.082982+0000","last_undegraded":"2024-06-28T16:10:30.082982+0000","last_fullsized":"2024-06-28T16:10:30.082982+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:06.922239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"260'2402","reported_seq":4268,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:02.134637+0000","last_change":"2024-06-28T15:00:25.440728+0000","last_active":"2024-06-28T16:14:02.134637+0000","last_peered":"2024-06-28T16:14:02.134637+0000","last_clean":"2024-06-28T16:14:02.134637+0000","last_became_active":"2024-06-28T15:00:25.440083+0000","last_became_peered":"2024-06-28T15:00:25.440083+0000","last_unstale":"2024-06-28T16:14:02.134637+0000","last_undegraded":"2024-06-28T16:14:02.134637+0000","last_fullsized":"2024-06-28T16:14:02.134637+0000","mapping_epoch":29,"log_start":"260'2300","ondisk_log_start":"260'2300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":2300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:54:57.315079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5966,"num_read_kb":5973,"num_write":4800,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"260'4179","reported_seq":4823,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.825089+0000","last_change":"2024-06-28T15:00:17.390976+0000","last_active":"2024-06-28T16:11:03.825089+0000","last_peered":"2024-06-28T16:11:03.825089+0000","last_clean":"2024-06-28T16:11:03.825089+0000","last_became_active":"2024-06-28T15:00:17.390254+0000","last_became_peered":"2024-06-28T15:00:17.390254+0000","last_unstale":"2024-06-28T16:11:03.825089+0000","last_undegraded":"2024-06-28T16:11:03.825089+0000","last_fullsized":"2024-06-28T16:11:03.825089+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":3000,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:50:01.744884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14764,"num_read_kb":11968,"num_write":6274,"num_write_kb":3129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"260'269","reported_seq":937,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.545460+0000","last_change":"2024-06-28T15:00:15.397404+0000","last_active":"2024-06-28T16:10:15.545460+0000","last_peered":"2024-06-28T16:10:15.545460+0000","last_clean":"2024-06-28T16:10:15.545460+0000","last_became_active":"2024-06-28T15:00:15.396724+0000","last_became_peered":"2024-06-28T15:00:15.396724+0000","last_unstale":"2024-06-28T16:10:15.545460+0000","last_undegraded":"2024-06-28T16:10:15.545460+0000","last_fullsized":"2024-06-28T16:10:15.545460+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:42:39.929729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2418239,"num_objects":80,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":458,"num_read_kb":7737,"num_write":1161,"num_write_kb":8076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493541+0000","last_change":"2024-06-28T15:00:19.390766+0000","last_active":"2024-06-28T16:10:25.493541+0000","last_peered":"2024-06-28T16:10:25.493541+0000","last_clean":"2024-06-28T16:10:25.493541+0000","last_became_active":"2024-06-28T15:00:19.390100+0000","last_became_peered":"2024-06-28T15:00:19.390100+0000","last_unstale":"2024-06-28T16:10:25.493541+0000","last_undegraded":"2024-06-28T16:10:25.493541+0000","last_fullsized":"2024-06-28T16:10:25.493541+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:01:20.861131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"260'123","reported_seq":662,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.316301+0000","last_change":"2024-06-28T15:00:21.374873+0000","last_active":"2024-06-28T16:11:03.316301+0000","last_peered":"2024-06-28T16:11:03.316301+0000","last_clean":"2024-06-28T16:11:03.316301+0000","last_became_active":"2024-06-28T15:00:21.374284+0000","last_became_peered":"2024-06-28T15:00:21.374284+0000","last_unstale":"2024-06-28T16:11:03.316301+0000","last_undegraded":"2024-06-28T16:11:03.316301+0000","last_fullsized":"2024-06-28T16:11:03.316301+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:17:08.459695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":127,"num_read_kb":120,"num_write":90,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":495,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.076584+0000","last_change":"2024-06-28T15:00:23.399941+0000","last_active":"2024-06-28T16:10:30.076584+0000","last_peered":"2024-06-28T16:10:30.076584+0000","last_clean":"2024-06-28T16:10:30.076584+0000","last_became_active":"2024-06-28T15:00:23.399147+0000","last_became_peered":"2024-06-28T15:00:23.399147+0000","last_unstale":"2024-06-28T16:10:30.076584+0000","last_undegraded":"2024-06-28T16:10:30.076584+0000","last_fullsized":"2024-06-28T16:10:30.076584+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:00:35.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"260'123","reported_seq":649,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.145795+0000","last_change":"2024-06-28T15:00:25.433474+0000","last_active":"2024-06-28T16:14:06.145795+0000","last_peered":"2024-06-28T16:14:06.145795+0000","last_clean":"2024-06-28T16:14:06.145795+0000","last_became_active":"2024-06-28T15:00:25.432898+0000","last_became_peered":"2024-06-28T15:00:25.432898+0000","last_unstale":"2024-06-28T16:14:06.145795+0000","last_undegraded":"2024-06-28T16:14:06.145795+0000","last_fullsized":"2024-06-28T16:14:06.145795+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:46:01.703394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":165,"num_write":302,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"260'155","reported_seq":776,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.548260+0000","last_change":"2024-06-28T15:00:17.384953+0000","last_active":"2024-06-28T16:10:25.548260+0000","last_peered":"2024-06-28T16:10:25.548260+0000","last_clean":"2024-06-28T16:10:25.548260+0000","last_became_active":"2024-06-28T15:00:17.384613+0000","last_became_peered":"2024-06-28T15:00:17.384613+0000","last_unstale":"2024-06-28T16:10:25.548260+0000","last_undegraded":"2024-06-28T16:10:25.548260+0000","last_fullsized":"2024-06-28T16:10:25.548260+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:04:42.370422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":630,"num_read_kb":508,"num_write":240,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"260'353","reported_seq":1001,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.727043+0000","last_change":"2024-06-28T15:00:15.429200+0000","last_active":"2024-06-28T16:14:34.727043+0000","last_peered":"2024-06-28T16:14:34.727043+0000","last_clean":"2024-06-28T16:14:34.727043+0000","last_became_active":"2024-06-28T15:00:15.428415+0000","last_became_peered":"2024-06-28T15:00:15.428415+0000","last_unstale":"2024-06-28T16:14:34.727043+0000","last_undegraded":"2024-06-28T16:14:34.727043+0000","last_fullsized":"2024-06-28T16:14:34.727043+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:17:28.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16055067,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":496,"num_read_kb":15197,"num_write":1261,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493690+0000","last_change":"2024-06-28T15:00:19.373751+0000","last_active":"2024-06-28T16:10:25.493690+0000","last_peered":"2024-06-28T16:10:25.493690+0000","last_clean":"2024-06-28T16:10:25.493690+0000","last_became_active":"2024-06-28T15:00:19.372840+0000","last_became_peered":"2024-06-28T15:00:19.372840+0000","last_unstale":"2024-06-28T16:10:25.493690+0000","last_undegraded":"2024-06-28T16:10:25.493690+0000","last_fullsized":"2024-06-28T16:10:25.493690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:46:30.290232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"260'123","reported_seq":686,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.516922+0000","last_change":"2024-06-28T15:00:21.377962+0000","last_active":"2024-06-28T16:11:03.516922+0000","last_peered":"2024-06-28T16:11:03.516922+0000","last_clean":"2024-06-28T16:11:03.516922+0000","last_became_active":"2024-06-28T15:00:21.377203+0000","last_became_peered":"2024-06-28T15:00:21.377203+0000","last_unstale":"2024-06-28T16:11:03.516922+0000","last_undegraded":"2024-06-28T16:11:03.516922+0000","last_fullsized":"2024-06-28T16:11:03.516922+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:14:43.436846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"260'522","reported_seq":4627,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.061536+0000","last_change":"2024-06-28T15:00:23.418328+0000","last_active":"2024-06-28T16:14:34.061536+0000","last_peered":"2024-06-28T16:14:34.061536+0000","last_clean":"2024-06-28T16:14:34.061536+0000","last_became_active":"2024-06-28T15:00:23.417490+0000","last_became_peered":"2024-06-28T15:00:23.417490+0000","last_unstale":"2024-06-28T16:14:34.061536+0000","last_undegraded":"2024-06-28T16:14:34.061536+0000","last_fullsized":"2024-06-28T16:14:34.061536+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":500,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:22:03.212204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"260'201","reported_seq":1010,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.965594+0000","last_change":"2024-06-28T15:00:25.556315+0000","last_active":"2024-06-28T16:14:06.965594+0000","last_peered":"2024-06-28T16:14:06.965594+0000","last_clean":"2024-06-28T16:14:06.965594+0000","last_became_active":"2024-06-28T15:00:25.555670+0000","last_became_peered":"2024-06-28T15:00:25.555670+0000","last_unstale":"2024-06-28T16:14:06.965594+0000","last_undegraded":"2024-06-28T16:14:06.965594+0000","last_fullsized":"2024-06-28T16:14:06.965594+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:26:34.140679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":557,"num_read_kb":555,"num_write":448,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"260'84","reported_seq":671,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493663+0000","last_change":"2024-06-28T15:00:17.390503+0000","last_active":"2024-06-28T16:10:25.493663+0000","last_peered":"2024-06-28T16:10:25.493663+0000","last_clean":"2024-06-28T16:10:25.493663+0000","last_became_active":"2024-06-28T15:00:17.389986+0000","last_became_peered":"2024-06-28T15:00:17.389986+0000","last_unstale":"2024-06-28T16:10:25.493663+0000","last_undegraded":"2024-06-28T16:10:25.493663+0000","last_fullsized":"2024-06-28T16:10:25.493663+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:30:46.132610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":369,"num_read_kb":305,"num_write":130,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"260'288","reported_seq":922,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.683233+0000","last_change":"2024-06-28T15:00:15.413174+0000","last_active":"2024-06-28T16:14:34.683233+0000","last_peered":"2024-06-28T16:14:34.683233+0000","last_clean":"2024-06-28T16:14:34.683233+0000","last_became_active":"2024-06-28T15:00:15.411380+0000","last_became_peered":"2024-06-28T15:00:15.411380+0000","last_unstale":"2024-06-28T16:14:34.683233+0000","last_undegraded":"2024-06-28T16:14:34.683233+0000","last_fullsized":"2024-06-28T16:14:34.683233+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:35.364682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262217,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":24351,"num_write":1142,"num_write_kb":34218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"31'3","reported_seq":830,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:31.936121+0000","last_change":"2024-06-28T15:00:19.404519+0000","last_active":"2024-06-28T16:14:31.936121+0000","last_peered":"2024-06-28T16:14:31.936121+0000","last_clean":"2024-06-28T16:14:31.936121+0000","last_became_active":"2024-06-28T15:00:19.403623+0000","last_became_peered":"2024-06-28T15:00:19.403623+0000","last_unstale":"2024-06-28T16:14:31.936121+0000","last_undegraded":"2024-06-28T16:14:31.936121+0000","last_fullsized":"2024-06-28T16:14:31.936121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:36:48.645241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":496,"num_read_kb":494,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"260'251","reported_seq":833,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.272334+0000","last_change":"2024-06-28T15:00:21.485248+0000","last_active":"2024-06-28T16:14:33.272334+0000","last_peered":"2024-06-28T16:14:33.272334+0000","last_clean":"2024-06-28T16:14:33.272334+0000","last_became_active":"2024-06-28T15:00:21.483633+0000","last_became_peered":"2024-06-28T15:00:21.483633+0000","last_unstale":"2024-06-28T16:14:33.272334+0000","last_undegraded":"2024-06-28T16:14:33.272334+0000","last_fullsized":"2024-06-28T16:14:33.272334+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:48:04.552220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8831,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":873,"num_write":310,"num_write_kb":224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"260'537","reported_seq":2239,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.259197+0000","last_change":"2024-06-28T15:00:23.397769+0000","last_active":"2024-06-28T16:14:34.259197+0000","last_peered":"2024-06-28T16:14:34.259197+0000","last_clean":"2024-06-28T16:14:34.259197+0000","last_became_active":"2024-06-28T15:00:23.396923+0000","last_became_peered":"2024-06-28T15:00:23.396923+0000","last_unstale":"2024-06-28T16:14:34.259197+0000","last_undegraded":"2024-06-28T16:14:34.259197+0000","last_fullsized":"2024-06-28T16:14:34.259197+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:32:30.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"260'415","reported_seq":1093,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:43.529396+0000","last_change":"2024-06-28T15:00:17.378608+0000","last_active":"2024-06-28T16:12:43.529396+0000","last_peered":"2024-06-28T16:12:43.529396+0000","last_clean":"2024-06-28T16:12:43.529396+0000","last_became_active":"2024-06-28T15:00:17.376864+0000","last_became_peered":"2024-06-28T15:00:17.376864+0000","last_unstale":"2024-06-28T16:12:43.529396+0000","last_undegraded":"2024-06-28T16:12:43.529396+0000","last_fullsized":"2024-06-28T16:12:43.529396+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":400,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:13.643893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1583,"num_read_kb":1214,"num_write":634,"num_write_kb":357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"260'320","reported_seq":979,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.732986+0000","last_change":"2024-06-28T15:00:15.423161+0000","last_active":"2024-06-28T16:14:34.732986+0000","last_peered":"2024-06-28T16:14:34.732986+0000","last_clean":"2024-06-28T16:14:34.732986+0000","last_became_active":"2024-06-28T15:00:15.415163+0000","last_became_peered":"2024-06-28T15:00:15.415163+0000","last_unstale":"2024-06-28T16:14:34.732986+0000","last_undegraded":"2024-06-28T16:14:34.732986+0000","last_fullsized":"2024-06-28T16:14:34.732986+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:10:26.502858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470118,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":518,"num_read_kb":26822,"num_write":1200,"num_write_kb":41274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.101361+0000","last_change":"2024-06-28T15:00:19.390703+0000","last_active":"2024-06-28T16:11:50.101361+0000","last_peered":"2024-06-28T16:11:50.101361+0000","last_clean":"2024-06-28T16:11:50.101361+0000","last_became_active":"2024-06-28T15:00:19.389624+0000","last_became_peered":"2024-06-28T15:00:19.389624+0000","last_unstale":"2024-06-28T16:11:50.101361+0000","last_undegraded":"2024-06-28T16:11:50.101361+0000","last_fullsized":"2024-06-28T16:11:50.101361+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:38:09.613294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"260'382","reported_seq":1026,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.950175+0000","last_change":"2024-06-28T15:00:21.483120+0000","last_active":"2024-06-28T16:14:33.950175+0000","last_peered":"2024-06-28T16:14:33.950175+0000","last_clean":"2024-06-28T16:14:33.950175+0000","last_became_active":"2024-06-28T15:00:21.481315+0000","last_became_peered":"2024-06-28T15:00:21.481315+0000","last_unstale":"2024-06-28T16:14:33.950175+0000","last_undegraded":"2024-06-28T16:14:33.950175+0000","last_fullsized":"2024-06-28T16:14:33.950175+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":300,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:09:01.037761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46076,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":638,"num_read_kb":1277,"num_write":490,"num_write_kb":298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.082062+0000","last_change":"2024-06-28T15:00:23.414142+0000","last_active":"2024-06-28T16:10:10.082062+0000","last_peered":"2024-06-28T16:10:10.082062+0000","last_clean":"2024-06-28T16:10:10.082062+0000","last_became_active":"2024-06-28T15:00:23.413132+0000","last_became_peered":"2024-06-28T15:00:23.413132+0000","last_unstale":"2024-06-28T16:10:10.082062+0000","last_undegraded":"2024-06-28T16:10:10.082062+0000","last_fullsized":"2024-06-28T16:10:10.082062+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:39.827537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"260'555","reported_seq":1373,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:13.000905+0000","last_change":"2024-06-28T15:00:25.442589+0000","last_active":"2024-06-28T16:14:13.000905+0000","last_peered":"2024-06-28T16:14:13.000905+0000","last_clean":"2024-06-28T16:14:13.000905+0000","last_became_active":"2024-06-28T15:00:25.441335+0000","last_became_peered":"2024-06-28T15:00:25.441335+0000","last_unstale":"2024-06-28T16:14:13.000905+0000","last_undegraded":"2024-06-28T16:14:13.000905+0000","last_fullsized":"2024-06-28T16:14:13.000905+0000","mapping_epoch":29,"log_start":"260'500","ondisk_log_start":"260'500","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":500,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:55.109636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1277,"num_read_kb":1273,"num_write":1164,"num_write_kb":394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"260'52","reported_seq":631,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.094197+0000","last_change":"2024-06-28T15:00:17.409603+0000","last_active":"2024-06-28T16:11:50.094197+0000","last_peered":"2024-06-28T16:11:50.094197+0000","last_clean":"2024-06-28T16:11:50.094197+0000","last_became_active":"2024-06-28T15:00:17.408992+0000","last_became_peered":"2024-06-28T15:00:17.408992+0000","last_unstale":"2024-06-28T16:11:50.094197+0000","last_undegraded":"2024-06-28T16:11:50.094197+0000","last_fullsized":"2024-06-28T16:11:50.094197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:22:21.223507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":226,"num_read_kb":196,"num_write":79,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"260'326","reported_seq":945,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.655730+0000","last_change":"2024-06-28T15:00:15.402904+0000","last_active":"2024-06-28T16:14:34.655730+0000","last_peered":"2024-06-28T16:14:34.655730+0000","last_clean":"2024-06-28T16:14:34.655730+0000","last_became_active":"2024-06-28T15:00:15.402188+0000","last_became_peered":"2024-06-28T15:00:15.402188+0000","last_unstale":"2024-06-28T16:14:34.655730+0000","last_undegraded":"2024-06-28T16:14:34.655730+0000","last_fullsized":"2024-06-28T16:14:34.655730+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:19:31.626172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170347,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":7037,"num_write":1119,"num_write_kb":28395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.273401+0000","last_change":"2024-06-28T15:00:19.383547+0000","last_active":"2024-06-28T16:11:50.273401+0000","last_peered":"2024-06-28T16:11:50.273401+0000","last_clean":"2024-06-28T16:11:50.273401+0000","last_became_active":"2024-06-28T15:00:19.383002+0000","last_became_peered":"2024-06-28T15:00:19.383002+0000","last_unstale":"2024-06-28T16:11:50.273401+0000","last_undegraded":"2024-06-28T16:11:50.273401+0000","last_fullsized":"2024-06-28T16:11:50.273401+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:52.946596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"260'288","reported_seq":878,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.505294+0000","last_change":"2024-06-28T15:00:21.395366+0000","last_active":"2024-06-28T16:14:32.505294+0000","last_peered":"2024-06-28T16:14:32.505294+0000","last_clean":"2024-06-28T16:14:32.505294+0000","last_became_active":"2024-06-28T15:00:21.394580+0000","last_became_peered":"2024-06-28T15:00:21.394580+0000","last_unstale":"2024-06-28T16:14:32.505294+0000","last_undegraded":"2024-06-28T16:14:32.505294+0000","last_fullsized":"2024-06-28T16:14:32.505294+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:06:06.648827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":455,"num_read_kb":1052,"num_write":364,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:00.515835+0000","last_change":"2024-06-28T15:00:23.423231+0000","last_active":"2024-06-28T16:12:00.515835+0000","last_peered":"2024-06-28T16:12:00.515835+0000","last_clean":"2024-06-28T16:12:00.515835+0000","last_became_active":"2024-06-28T15:00:23.422567+0000","last_became_peered":"2024-06-28T15:00:23.422567+0000","last_unstale":"2024-06-28T16:12:00.515835+0000","last_undegraded":"2024-06-28T16:12:00.515835+0000","last_fullsized":"2024-06-28T16:12:00.515835+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:40.356598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"260'96","reported_seq":589,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089808+0000","last_change":"2024-06-28T15:00:25.426829+0000","last_active":"2024-06-28T16:11:35.089808+0000","last_peered":"2024-06-28T16:11:35.089808+0000","last_clean":"2024-06-28T16:11:35.089808+0000","last_became_active":"2024-06-28T15:00:25.426151+0000","last_became_peered":"2024-06-28T15:00:25.426151+0000","last_unstale":"2024-06-28T16:11:35.089808+0000","last_undegraded":"2024-06-28T16:11:35.089808+0000","last_fullsized":"2024-06-28T16:11:35.089808+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:23:26.301565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":231,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"260'173","reported_seq":777,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:10.098092+0000","last_change":"2024-06-28T15:00:17.407655+0000","last_active":"2024-06-28T16:12:10.098092+0000","last_peered":"2024-06-28T16:12:10.098092+0000","last_clean":"2024-06-28T16:12:10.098092+0000","last_became_active":"2024-06-28T15:00:17.407031+0000","last_became_peered":"2024-06-28T15:00:17.407031+0000","last_unstale":"2024-06-28T16:12:10.098092+0000","last_undegraded":"2024-06-28T16:12:10.098092+0000","last_fullsized":"2024-06-28T16:12:10.098092+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:17:37.199564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":538,"num_write":274,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"260'353","reported_seq":1038,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.682331+0000","last_change":"2024-06-28T15:00:15.428903+0000","last_active":"2024-06-28T16:14:34.682331+0000","last_peered":"2024-06-28T16:14:34.682331+0000","last_clean":"2024-06-28T16:14:34.682331+0000","last_became_active":"2024-06-28T15:00:15.427408+0000","last_became_peered":"2024-06-28T15:00:15.427408+0000","last_unstale":"2024-06-28T16:14:34.682331+0000","last_undegraded":"2024-06-28T16:14:34.682331+0000","last_fullsized":"2024-06-28T16:14:34.682331+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:52:51.484685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":574,"num_read_kb":16939,"num_write":1376,"num_write_kb":32636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.279481+0000","last_change":"2024-06-28T15:00:19.384770+0000","last_active":"2024-06-28T16:12:20.279481+0000","last_peered":"2024-06-28T16:12:20.279481+0000","last_clean":"2024-06-28T16:12:20.279481+0000","last_became_active":"2024-06-28T15:00:19.384172+0000","last_became_peered":"2024-06-28T15:00:19.384172+0000","last_unstale":"2024-06-28T16:12:20.279481+0000","last_undegraded":"2024-06-28T16:12:20.279481+0000","last_fullsized":"2024-06-28T16:12:20.279481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:13:54.648899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"260'314","reported_seq":925,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.098634+0000","last_change":"2024-06-28T15:00:21.394527+0000","last_active":"2024-06-28T16:14:34.098634+0000","last_peered":"2024-06-28T16:14:34.098634+0000","last_clean":"2024-06-28T16:14:34.098634+0000","last_became_active":"2024-06-28T15:00:21.393608+0000","last_became_peered":"2024-06-28T15:00:21.393608+0000","last_unstale":"2024-06-28T16:14:34.098634+0000","last_undegraded":"2024-06-28T16:14:34.098634+0000","last_fullsized":"2024-06-28T16:14:34.098634+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:01:39.810509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42645,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":526,"num_read_kb":1122,"num_write":410,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"260'529","reported_seq":2261,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:14.487526+0000","last_change":"2024-06-28T15:00:23.414333+0000","last_active":"2024-06-28T16:14:14.487526+0000","last_peered":"2024-06-28T16:14:14.487526+0000","last_clean":"2024-06-28T16:14:14.487526+0000","last_became_active":"2024-06-28T15:00:23.413550+0000","last_became_peered":"2024-06-28T15:00:23.413550+0000","last_unstale":"2024-06-28T16:14:14.487526+0000","last_undegraded":"2024-06-28T16:14:14.487526+0000","last_fullsized":"2024-06-28T16:14:14.487526+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":500,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:23:18.123161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"260'144","reported_seq":672,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.569778+0000","last_change":"2024-06-28T15:00:25.432580+0000","last_active":"2024-06-28T16:11:55.569778+0000","last_peered":"2024-06-28T16:11:55.569778+0000","last_clean":"2024-06-28T16:11:55.569778+0000","last_became_active":"2024-06-28T15:00:25.431835+0000","last_became_peered":"2024-06-28T15:00:25.431835+0000","last_unstale":"2024-06-28T16:11:55.569778+0000","last_undegraded":"2024-06-28T16:11:55.569778+0000","last_fullsized":"2024-06-28T16:11:55.569778+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:25:08.052594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1247,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":164,"num_write":265,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"260'4158","reported_seq":4763,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.543014+0000","last_change":"2024-06-28T15:00:17.400121+0000","last_active":"2024-06-28T16:10:05.543014+0000","last_peered":"2024-06-28T16:10:05.543014+0000","last_clean":"2024-06-28T16:10:05.543014+0000","last_became_active":"2024-06-28T15:00:17.399799+0000","last_became_peered":"2024-06-28T15:00:17.399799+0000","last_unstale":"2024-06-28T16:10:05.543014+0000","last_undegraded":"2024-06-28T16:10:05.543014+0000","last_fullsized":"2024-06-28T16:10:05.543014+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":3000,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:10:23.005648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14678,"num_read_kb":11882,"num_write":6254,"num_write_kb":3121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"260'337","reported_seq":982,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.704930+0000","last_change":"2024-06-28T15:00:15.426615+0000","last_active":"2024-06-28T16:14:34.704930+0000","last_peered":"2024-06-28T16:14:34.704930+0000","last_clean":"2024-06-28T16:14:34.704930+0000","last_became_active":"2024-06-28T15:00:15.425690+0000","last_became_peered":"2024-06-28T15:00:15.425690+0000","last_unstale":"2024-06-28T16:14:34.704930+0000","last_undegraded":"2024-06-28T16:14:34.704930+0000","last_fullsized":"2024-06-28T16:14:34.704930+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:59:45.352009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079931,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":508,"num_read_kb":13664,"num_write":1274,"num_write_kb":26574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,1],"acting":[5,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:05.276635+0000","last_change":"2024-06-28T15:00:19.385556+0000","last_active":"2024-06-28T16:12:05.276635+0000","last_peered":"2024-06-28T16:12:05.276635+0000","last_clean":"2024-06-28T16:12:05.276635+0000","last_became_active":"2024-06-28T15:00:19.385189+0000","last_became_peered":"2024-06-28T15:00:19.385189+0000","last_unstale":"2024-06-28T16:12:05.276635+0000","last_undegraded":"2024-06-28T16:12:05.276635+0000","last_fullsized":"2024-06-28T16:12:05.276635+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:12:07.124133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"260'477","reported_seq":1123,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:12.918679+0000","last_change":"2024-06-28T15:00:21.400448+0000","last_active":"2024-06-28T16:14:12.918679+0000","last_peered":"2024-06-28T16:14:12.918679+0000","last_clean":"2024-06-28T16:14:12.918679+0000","last_became_active":"2024-06-28T15:00:21.397397+0000","last_became_peered":"2024-06-28T15:00:21.397397+0000","last_unstale":"2024-06-28T16:14:12.918679+0000","last_undegraded":"2024-06-28T16:14:12.918679+0000","last_fullsized":"2024-06-28T16:14:12.918679+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":400,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:54:53.893930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109658,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":830,"num_read_kb":1847,"num_write":686,"num_write_kb":570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.274612+0000","last_change":"2024-06-28T15:00:23.422892+0000","last_active":"2024-06-28T16:11:55.274612+0000","last_peered":"2024-06-28T16:11:55.274612+0000","last_clean":"2024-06-28T16:11:55.274612+0000","last_became_active":"2024-06-28T15:00:23.416263+0000","last_became_peered":"2024-06-28T15:00:23.416263+0000","last_unstale":"2024-06-28T16:11:55.274612+0000","last_undegraded":"2024-06-28T16:11:55.274612+0000","last_fullsized":"2024-06-28T16:11:55.274612+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:12:48.735241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"260'113","reported_seq":631,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.103804+0000","last_change":"2024-06-28T15:00:25.423425+0000","last_active":"2024-06-28T16:11:55.103804+0000","last_peered":"2024-06-28T16:11:55.103804+0000","last_clean":"2024-06-28T16:11:55.103804+0000","last_became_active":"2024-06-28T15:00:25.422519+0000","last_became_peered":"2024-06-28T15:00:25.422519+0000","last_unstale":"2024-06-28T16:11:55.103804+0000","last_undegraded":"2024-06-28T16:11:55.103804+0000","last_fullsized":"2024-06-28T16:11:55.103804+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:15:27.026404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":118,"num_read_kb":116,"num_write":248,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"260'128","reported_seq":777,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101386+0000","last_change":"2024-06-28T15:00:17.409415+0000","last_active":"2024-06-28T16:12:20.101386+0000","last_peered":"2024-06-28T16:12:20.101386+0000","last_clean":"2024-06-28T16:12:20.101386+0000","last_became_active":"2024-06-28T15:00:17.404204+0000","last_became_peered":"2024-06-28T15:00:17.404204+0000","last_unstale":"2024-06-28T16:12:20.101386+0000","last_undegraded":"2024-06-28T16:12:20.101386+0000","last_fullsized":"2024-06-28T16:12:20.101386+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:25:51.372637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":562,"num_read_kb":476,"num_write":174,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"260'352","reported_seq":1059,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.701530+0000","last_change":"2024-06-28T15:00:15.402917+0000","last_active":"2024-06-28T16:14:34.701530+0000","last_peered":"2024-06-28T16:14:34.701530+0000","last_clean":"2024-06-28T16:14:34.701530+0000","last_became_active":"2024-06-28T15:00:15.402041+0000","last_became_peered":"2024-06-28T15:00:15.402041+0000","last_unstale":"2024-06-28T16:14:34.701530+0000","last_undegraded":"2024-06-28T16:14:34.701530+0000","last_fullsized":"2024-06-28T16:14:34.701530+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:50:41.105836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":603,"num_read_kb":26410,"num_write":1369,"num_write_kb":26391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.279564+0000","last_change":"2024-06-28T15:00:19.393172+0000","last_active":"2024-06-28T16:12:20.279564+0000","last_peered":"2024-06-28T16:12:20.279564+0000","last_clean":"2024-06-28T16:12:20.279564+0000","last_became_active":"2024-06-28T15:00:19.392773+0000","last_became_peered":"2024-06-28T15:00:19.392773+0000","last_unstale":"2024-06-28T16:12:20.279564+0000","last_undegraded":"2024-06-28T16:12:20.279564+0000","last_fullsized":"2024-06-28T16:12:20.279564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:03:10.710059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"260'301","reported_seq":896,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.815148+0000","last_change":"2024-06-28T15:00:21.398582+0000","last_active":"2024-06-28T16:14:32.815148+0000","last_peered":"2024-06-28T16:14:32.815148+0000","last_clean":"2024-06-28T16:14:32.815148+0000","last_became_active":"2024-06-28T15:00:21.397842+0000","last_became_peered":"2024-06-28T15:00:21.397842+0000","last_unstale":"2024-06-28T16:14:32.815148+0000","last_undegraded":"2024-06-28T16:14:32.815148+0000","last_fullsized":"2024-06-28T16:14:32.815148+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:31:25.283406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44037,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":498,"num_read_kb":1063,"num_write":434,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.548349+0000","last_change":"2024-06-28T15:00:23.406901+0000","last_active":"2024-06-28T16:10:25.548349+0000","last_peered":"2024-06-28T16:10:25.548349+0000","last_clean":"2024-06-28T16:10:25.548349+0000","last_became_active":"2024-06-28T15:00:23.406122+0000","last_became_peered":"2024-06-28T15:00:23.406122+0000","last_unstale":"2024-06-28T16:10:25.548349+0000","last_undegraded":"2024-06-28T16:10:25.548349+0000","last_fullsized":"2024-06-28T16:10:25.548349+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:58:30.933744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"260'89","reported_seq":593,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:03.580217+0000","last_change":"2024-06-28T15:00:25.439968+0000","last_active":"2024-06-28T16:14:03.580217+0000","last_peered":"2024-06-28T16:14:03.580217+0000","last_clean":"2024-06-28T16:14:03.580217+0000","last_became_active":"2024-06-28T15:00:25.439346+0000","last_became_peered":"2024-06-28T15:00:25.439346+0000","last_unstale":"2024-06-28T16:14:03.580217+0000","last_undegraded":"2024-06-28T16:14:03.580217+0000","last_fullsized":"2024-06-28T16:14:03.580217+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:31:17.344077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":207,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"260'171","reported_seq":759,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101425+0000","last_change":"2024-06-28T15:00:17.396835+0000","last_active":"2024-06-28T16:12:20.101425+0000","last_peered":"2024-06-28T16:12:20.101425+0000","last_clean":"2024-06-28T16:12:20.101425+0000","last_became_active":"2024-06-28T15:00:17.396048+0000","last_became_peered":"2024-06-28T15:00:17.396048+0000","last_unstale":"2024-06-28T16:12:20.101425+0000","last_undegraded":"2024-06-28T16:12:20.101425+0000","last_fullsized":"2024-06-28T16:12:20.101425+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:20:25.818327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":659,"num_read_kb":531,"num_write":250,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"260'376","reported_seq":1107,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.727955+0000","last_change":"2024-06-28T15:00:15.398260+0000","last_active":"2024-06-28T16:14:34.727955+0000","last_peered":"2024-06-28T16:14:34.727955+0000","last_clean":"2024-06-28T16:14:34.727955+0000","last_became_active":"2024-06-28T15:00:15.397256+0000","last_became_peered":"2024-06-28T15:00:15.397256+0000","last_unstale":"2024-06-28T16:14:34.727955+0000","last_undegraded":"2024-06-28T16:14:34.727955+0000","last_fullsized":"2024-06-28T16:14:34.727955+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":300,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:08:01.511388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2674362,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":727,"num_read_kb":7345,"num_write":1214,"num_write_kb":31713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.109069+0000","last_change":"2024-06-28T15:00:19.379868+0000","last_active":"2024-06-28T16:12:20.109069+0000","last_peered":"2024-06-28T16:12:20.109069+0000","last_clean":"2024-06-28T16:12:20.109069+0000","last_became_active":"2024-06-28T15:00:19.379281+0000","last_became_peered":"2024-06-28T15:00:19.379281+0000","last_unstale":"2024-06-28T16:12:20.109069+0000","last_undegraded":"2024-06-28T16:12:20.109069+0000","last_fullsized":"2024-06-28T16:12:20.109069+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:01:53.936356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"260'350","reported_seq":968,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.682992+0000","last_change":"2024-06-28T15:00:21.396750+0000","last_active":"2024-06-28T16:14:34.682992+0000","last_peered":"2024-06-28T16:14:34.682992+0000","last_clean":"2024-06-28T16:14:34.682992+0000","last_became_active":"2024-06-28T15:00:21.395892+0000","last_became_peered":"2024-06-28T15:00:21.395892+0000","last_unstale":"2024-06-28T16:14:34.682992+0000","last_undegraded":"2024-06-28T16:14:34.682992+0000","last_fullsized":"2024-06-28T16:14:34.682992+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":300,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:49:00.898473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48931,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":579,"num_read_kb":1282,"num_write":462,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"260'501","reported_seq":2303,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.195533+0000","last_change":"2024-06-28T15:00:23.416260+0000","last_active":"2024-06-28T16:14:32.195533+0000","last_peered":"2024-06-28T16:14:32.195533+0000","last_clean":"2024-06-28T16:14:32.195533+0000","last_became_active":"2024-06-28T15:00:23.415437+0000","last_became_peered":"2024-06-28T15:00:23.415437+0000","last_unstale":"2024-06-28T16:14:32.195533+0000","last_undegraded":"2024-06-28T16:14:32.195533+0000","last_fullsized":"2024-06-28T16:14:32.195533+0000","mapping_epoch":27,"log_start":"257'400","ondisk_log_start":"257'400","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":400,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:08:34.898411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"260'99","reported_seq":608,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.122863+0000","last_change":"2024-06-28T15:00:25.425875+0000","last_active":"2024-06-28T16:13:20.122863+0000","last_peered":"2024-06-28T16:13:20.122863+0000","last_clean":"2024-06-28T16:13:20.122863+0000","last_became_active":"2024-06-28T15:00:25.425135+0000","last_became_peered":"2024-06-28T15:00:25.425135+0000","last_unstale":"2024-06-28T16:13:20.122863+0000","last_undegraded":"2024-06-28T16:13:20.122863+0000","last_fullsized":"2024-06-28T16:13:20.122863+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:13:56.075833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":245,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"260'160","reported_seq":743,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101485+0000","last_change":"2024-06-28T15:00:17.403493+0000","last_active":"2024-06-28T16:12:20.101485+0000","last_peered":"2024-06-28T16:12:20.101485+0000","last_clean":"2024-06-28T16:12:20.101485+0000","last_became_active":"2024-06-28T15:00:17.402024+0000","last_became_peered":"2024-06-28T15:00:17.402024+0000","last_unstale":"2024-06-28T16:12:20.101485+0000","last_undegraded":"2024-06-28T16:12:20.101485+0000","last_fullsized":"2024-06-28T16:12:20.101485+0000","mapping_epoch":21,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":100,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:25:59.494240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":645,"num_read_kb":505,"num_write":259,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"260'393","reported_seq":1067,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.738050+0000","last_change":"2024-06-28T15:00:15.389691+0000","last_active":"2024-06-28T16:14:34.738050+0000","last_peered":"2024-06-28T16:14:34.738050+0000","last_clean":"2024-06-28T16:14:34.738050+0000","last_became_active":"2024-06-28T15:00:15.388365+0000","last_became_peered":"2024-06-28T15:00:15.388365+0000","last_unstale":"2024-06-28T16:14:34.738050+0000","last_undegraded":"2024-06-28T16:14:34.738050+0000","last_fullsized":"2024-06-28T16:14:34.738050+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":300,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:14:31.668500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1604260,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":661,"num_read_kb":7226,"num_write":1423,"num_write_kb":30573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"24'2","reported_seq":1154,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:31.948538+0000","last_change":"2024-06-28T15:00:19.403973+0000","last_active":"2024-06-28T16:14:31.948538+0000","last_peered":"2024-06-28T16:14:31.948538+0000","last_clean":"2024-06-28T16:14:31.948538+0000","last_became_active":"2024-06-28T15:00:19.397863+0000","last_became_peered":"2024-06-28T15:00:19.397863+0000","last_unstale":"2024-06-28T16:14:31.948538+0000","last_undegraded":"2024-06-28T16:14:31.948538+0000","last_fullsized":"2024-06-28T16:14:31.948538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:35:07.475408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":648,"num_read_kb":648,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"260'113","reported_seq":657,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.163323+0000","last_change":"2024-06-28T15:00:21.484486+0000","last_active":"2024-06-28T16:11:03.163323+0000","last_peered":"2024-06-28T16:11:03.163323+0000","last_clean":"2024-06-28T16:11:03.163323+0000","last_became_active":"2024-06-28T15:00:21.483679+0000","last_became_peered":"2024-06-28T15:00:21.483679+0000","last_unstale":"2024-06-28T16:11:03.163323+0000","last_undegraded":"2024-06-28T16:11:03.163323+0000","last_fullsized":"2024-06-28T16:11:03.163323+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:16:50.618747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"260'515","reported_seq":4262,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.199674+0000","last_change":"2024-06-28T15:00:23.419813+0000","last_active":"2024-06-28T16:14:32.199674+0000","last_peered":"2024-06-28T16:14:32.199674+0000","last_clean":"2024-06-28T16:14:32.199674+0000","last_became_active":"2024-06-28T15:00:23.419182+0000","last_became_peered":"2024-06-28T15:00:23.419182+0000","last_unstale":"2024-06-28T16:14:32.199674+0000","last_undegraded":"2024-06-28T16:14:32.199674+0000","last_fullsized":"2024-06-28T16:14:32.199674+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:00:01.245043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"260'159","reported_seq":672,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.114555+0000","last_change":"2024-06-28T15:00:25.427204+0000","last_active":"2024-06-28T16:13:20.114555+0000","last_peered":"2024-06-28T16:13:20.114555+0000","last_clean":"2024-06-28T16:13:20.114555+0000","last_became_active":"2024-06-28T15:00:25.425690+0000","last_became_peered":"2024-06-28T15:00:25.425690+0000","last_unstale":"2024-06-28T16:13:20.114555+0000","last_undegraded":"2024-06-28T16:13:20.114555+0000","last_fullsized":"2024-06-28T16:13:20.114555+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:18.073869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":152,"num_write":381,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"260'74","reported_seq":619,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.075926+0000","last_change":"2024-06-28T15:00:59.800386+0000","last_active":"2024-06-28T16:11:00.075926+0000","last_peered":"2024-06-28T16:11:00.075926+0000","last_clean":"2024-06-28T16:11:00.075926+0000","last_became_active":"2024-06-28T15:00:59.799823+0000","last_became_peered":"2024-06-28T15:00:59.799823+0000","last_unstale":"2024-06-28T16:11:00.075926+0000","last_undegraded":"2024-06-28T16:11:00.075926+0000","last_fullsized":"2024-06-28T16:11:00.075926+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:43:49.439721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":247,"num_write":103,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"260'351","reported_seq":1081,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.579126+0000","last_change":"2024-06-28T15:00:15.403798+0000","last_active":"2024-06-28T16:14:34.579126+0000","last_peered":"2024-06-28T16:14:34.579126+0000","last_clean":"2024-06-28T16:14:34.579126+0000","last_became_active":"2024-06-28T15:00:15.402782+0000","last_became_peered":"2024-06-28T15:00:15.402782+0000","last_unstale":"2024-06-28T16:14:34.579126+0000","last_undegraded":"2024-06-28T16:14:34.579126+0000","last_fullsized":"2024-06-28T16:14:34.579126+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":300,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:44:32.528218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262233,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":541,"num_read_kb":3603,"num_write":1164,"num_write_kb":36556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101454+0000","last_change":"2024-06-28T15:00:19.405878+0000","last_active":"2024-06-28T16:12:20.101454+0000","last_peered":"2024-06-28T16:12:20.101454+0000","last_clean":"2024-06-28T16:12:20.101454+0000","last_became_active":"2024-06-28T15:00:19.405083+0000","last_became_peered":"2024-06-28T15:00:19.405083+0000","last_unstale":"2024-06-28T16:12:20.101454+0000","last_undegraded":"2024-06-28T16:12:20.101454+0000","last_fullsized":"2024-06-28T16:12:20.101454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:55:51.552282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"260'429","reported_seq":1063,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.718564+0000","last_change":"2024-06-28T15:00:21.393207+0000","last_active":"2024-06-28T16:14:33.718564+0000","last_peered":"2024-06-28T16:14:33.718564+0000","last_clean":"2024-06-28T16:14:33.718564+0000","last_became_active":"2024-06-28T15:00:21.392393+0000","last_became_peered":"2024-06-28T15:00:21.392393+0000","last_unstale":"2024-06-28T16:14:33.718564+0000","last_undegraded":"2024-06-28T16:14:33.718564+0000","last_fullsized":"2024-06-28T16:14:33.718564+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":400,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:47:20.793097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47467,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1282,"num_read_kb":1893,"num_write":1822,"num_write_kb":1648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":493,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.070668+0000","last_change":"2024-06-28T15:00:23.416374+0000","last_active":"2024-06-28T16:10:10.070668+0000","last_peered":"2024-06-28T16:10:10.070668+0000","last_clean":"2024-06-28T16:10:10.070668+0000","last_became_active":"2024-06-28T15:00:23.415497+0000","last_became_peered":"2024-06-28T15:00:23.415497+0000","last_unstale":"2024-06-28T16:10:10.070668+0000","last_undegraded":"2024-06-28T16:10:10.070668+0000","last_fullsized":"2024-06-28T16:10:10.070668+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:13:09.664310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"260'133","reported_seq":639,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:07.643827+0000","last_change":"2024-06-28T15:00:25.440419+0000","last_active":"2024-06-28T16:14:07.643827+0000","last_peered":"2024-06-28T16:14:07.643827+0000","last_clean":"2024-06-28T16:14:07.643827+0000","last_became_active":"2024-06-28T15:00:25.439527+0000","last_became_peered":"2024-06-28T15:00:25.439527+0000","last_unstale":"2024-06-28T16:14:07.643827+0000","last_undegraded":"2024-06-28T16:14:07.643827+0000","last_fullsized":"2024-06-28T16:14:07.643827+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:30:19.298980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":127,"num_write":301,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"260'156","reported_seq":773,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.088666+0000","last_change":"2024-06-28T15:00:17.415172+0000","last_active":"2024-06-28T16:11:30.088666+0000","last_peered":"2024-06-28T16:11:30.088666+0000","last_clean":"2024-06-28T16:11:30.088666+0000","last_became_active":"2024-06-28T15:00:17.414537+0000","last_became_peered":"2024-06-28T15:00:17.414537+0000","last_unstale":"2024-06-28T16:11:30.088666+0000","last_undegraded":"2024-06-28T16:11:30.088666+0000","last_fullsized":"2024-06-28T16:11:30.088666+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:18:02.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":522,"num_write":223,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":616,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.109225+0000","last_change":"2024-06-28T15:00:06.303487+0000","last_active":"2024-06-28T16:12:25.109225+0000","last_peered":"2024-06-28T16:12:25.109225+0000","last_clean":"2024-06-28T16:12:25.109225+0000","last_became_active":"2024-06-28T15:00:06.302725+0000","last_became_peered":"2024-06-28T15:00:06.302725+0000","last_unstale":"2024-06-28T16:12:25.109225+0000","last_undegraded":"2024-06-28T16:12:25.109225+0000","last_fullsized":"2024-06-28T16:12:25.109225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:45:32.083226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.075782+0000","last_change":"2024-06-28T15:00:06.285125+0000","last_active":"2024-06-28T16:11:00.075782+0000","last_peered":"2024-06-28T16:11:00.075782+0000","last_clean":"2024-06-28T16:11:00.075782+0000","last_became_active":"2024-06-28T15:00:06.284218+0000","last_became_peered":"2024-06-28T15:00:06.284218+0000","last_unstale":"2024-06-28T16:11:00.075782+0000","last_undegraded":"2024-06-28T16:11:00.075782+0000","last_fullsized":"2024-06-28T16:11:00.075782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:04:51.635552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"260'320","reported_seq":1115,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.735430+0000","last_change":"2024-06-28T15:00:59.940725+0000","last_active":"2024-06-28T16:14:34.735430+0000","last_peered":"2024-06-28T16:14:34.735430+0000","last_clean":"2024-06-28T16:14:34.735430+0000","last_became_active":"2024-06-28T15:00:59.939352+0000","last_became_peered":"2024-06-28T15:00:59.939352+0000","last_unstale":"2024-06-28T16:14:34.735430+0000","last_undegraded":"2024-06-28T16:14:34.735430+0000","last_fullsized":"2024-06-28T16:14:34.735430+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:16:46.674713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":804,"num_read_kb":59106,"num_write":1183,"num_write_kb":43047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6,4],"acting":[3,2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.092431+0000","last_change":"2024-06-28T15:00:19.391126+0000","last_active":"2024-06-28T16:11:10.092431+0000","last_peered":"2024-06-28T16:11:10.092431+0000","last_clean":"2024-06-28T16:11:10.092431+0000","last_became_active":"2024-06-28T15:00:19.390255+0000","last_became_peered":"2024-06-28T15:00:19.390255+0000","last_unstale":"2024-06-28T16:11:10.092431+0000","last_undegraded":"2024-06-28T16:11:10.092431+0000","last_fullsized":"2024-06-28T16:11:10.092431+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:19:02.126295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"260'730","reported_seq":1450,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.539793+0000","last_change":"2024-06-28T15:00:21.398370+0000","last_active":"2024-06-28T16:14:34.539793+0000","last_peered":"2024-06-28T16:14:34.539793+0000","last_clean":"2024-06-28T16:14:34.539793+0000","last_became_active":"2024-06-28T15:00:21.397640+0000","last_became_peered":"2024-06-28T15:00:21.397640+0000","last_unstale":"2024-06-28T16:14:34.539793+0000","last_undegraded":"2024-06-28T16:14:34.539793+0000","last_fullsized":"2024-06-28T16:14:34.539793+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":700,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:10:23.954766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134565,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1425,"num_read_kb":3272,"num_write":1360,"num_write_kb":1190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.569752+0000","last_change":"2024-06-28T15:00:23.400439+0000","last_active":"2024-06-28T16:11:55.569752+0000","last_peered":"2024-06-28T16:11:55.569752+0000","last_clean":"2024-06-28T16:11:55.569752+0000","last_became_active":"2024-06-28T15:00:23.399024+0000","last_became_peered":"2024-06-28T15:00:23.399024+0000","last_unstale":"2024-06-28T16:11:55.569752+0000","last_undegraded":"2024-06-28T16:11:55.569752+0000","last_fullsized":"2024-06-28T16:11:55.569752+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:16:52.193469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"260'186","reported_seq":876,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:05.197144+0000","last_change":"2024-06-28T15:00:25.442723+0000","last_active":"2024-06-28T16:14:05.197144+0000","last_peered":"2024-06-28T16:14:05.197144+0000","last_clean":"2024-06-28T16:14:05.197144+0000","last_became_active":"2024-06-28T15:00:25.442032+0000","last_became_peered":"2024-06-28T15:00:25.442032+0000","last_unstale":"2024-06-28T16:14:05.197144+0000","last_undegraded":"2024-06-28T16:14:05.197144+0000","last_fullsized":"2024-06-28T16:14:05.197144+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:04:44.942959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":392,"num_read_kb":388,"num_write":388,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"260'118","reported_seq":755,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:57.970334+0000","last_change":"2024-06-28T15:00:17.404538+0000","last_active":"2024-06-28T16:11:57.970334+0000","last_peered":"2024-06-28T16:11:57.970334+0000","last_clean":"2024-06-28T16:11:57.970334+0000","last_became_active":"2024-06-28T15:00:17.404200+0000","last_became_peered":"2024-06-28T15:00:17.404200+0000","last_unstale":"2024-06-28T16:11:57.970334+0000","last_undegraded":"2024-06-28T16:11:57.970334+0000","last_fullsized":"2024-06-28T16:11:57.970334+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:32:41.333506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":495,"num_read_kb":413,"num_write":159,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":526,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.109267+0000","last_change":"2024-06-28T15:00:06.286231+0000","last_active":"2024-06-28T16:12:25.109267+0000","last_peered":"2024-06-28T16:12:25.109267+0000","last_clean":"2024-06-28T16:12:25.109267+0000","last_became_active":"2024-06-28T15:00:06.285489+0000","last_became_peered":"2024-06-28T15:00:06.285489+0000","last_unstale":"2024-06-28T16:12:25.109267+0000","last_undegraded":"2024-06-28T16:12:25.109267+0000","last_fullsized":"2024-06-28T16:12:25.109267+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:05:17.528909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"260'262","reported_seq":930,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.556093+0000","last_change":"2024-06-28T15:00:15.404155+0000","last_active":"2024-06-28T16:11:05.556093+0000","last_peered":"2024-06-28T16:11:05.556093+0000","last_clean":"2024-06-28T16:11:05.556093+0000","last_became_active":"2024-06-28T15:00:15.403657+0000","last_became_peered":"2024-06-28T15:00:15.403657+0000","last_unstale":"2024-06-28T16:11:05.556093+0000","last_undegraded":"2024-06-28T16:11:05.556093+0000","last_fullsized":"2024-06-28T16:11:05.556093+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":200,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:18:38.173993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6321470,"num_objects":64,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":462,"num_read_kb":36644,"num_write":1201,"num_write_kb":16548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.087155+0000","last_change":"2024-06-28T15:00:19.407157+0000","last_active":"2024-06-28T16:11:50.087155+0000","last_peered":"2024-06-28T16:11:50.087155+0000","last_clean":"2024-06-28T16:11:50.087155+0000","last_became_active":"2024-06-28T15:00:19.406447+0000","last_became_peered":"2024-06-28T15:00:19.406447+0000","last_unstale":"2024-06-28T16:11:50.087155+0000","last_undegraded":"2024-06-28T16:11:50.087155+0000","last_fullsized":"2024-06-28T16:11:50.087155+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:12:32.253515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"260'1134","reported_seq":2908,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.789539+0000","last_change":"2024-06-28T15:00:21.399223+0000","last_active":"2024-06-28T16:14:33.789539+0000","last_peered":"2024-06-28T16:14:33.789539+0000","last_clean":"2024-06-28T16:14:33.789539+0000","last_became_active":"2024-06-28T15:00:21.398382+0000","last_became_peered":"2024-06-28T15:00:21.398382+0000","last_unstale":"2024-06-28T16:14:33.789539+0000","last_undegraded":"2024-06-28T16:14:33.789539+0000","last_fullsized":"2024-06-28T16:14:33.789539+0000","mapping_epoch":25,"log_start":"260'1100","ondisk_log_start":"260'1100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":1100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:39:26.868657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133468,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1851,"num_read_kb":3741,"num_write":1150,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":498,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.097271+0000","last_change":"2024-06-28T15:00:23.423751+0000","last_active":"2024-06-28T16:11:30.097271+0000","last_peered":"2024-06-28T16:11:30.097271+0000","last_clean":"2024-06-28T16:11:30.097271+0000","last_became_active":"2024-06-28T15:00:23.422876+0000","last_became_peered":"2024-06-28T15:00:23.422876+0000","last_unstale":"2024-06-28T16:11:30.097271+0000","last_undegraded":"2024-06-28T16:11:30.097271+0000","last_fullsized":"2024-06-28T16:11:30.097271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:18:16.152063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"260'116","reported_seq":629,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:02.832339+0000","last_change":"2024-06-28T15:00:25.554459+0000","last_active":"2024-06-28T16:14:02.832339+0000","last_peered":"2024-06-28T16:14:02.832339+0000","last_clean":"2024-06-28T16:14:02.832339+0000","last_became_active":"2024-06-28T15:00:25.553691+0000","last_became_peered":"2024-06-28T15:00:25.553691+0000","last_unstale":"2024-06-28T16:14:02.832339+0000","last_undegraded":"2024-06-28T16:14:02.832339+0000","last_fullsized":"2024-06-28T16:14:02.832339+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:37:46.194533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":862,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":120,"num_write":292,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"260'221","reported_seq":834,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:02.146046+0000","last_change":"2024-06-28T15:00:17.393736+0000","last_active":"2024-06-28T16:13:02.146046+0000","last_peered":"2024-06-28T16:13:02.146046+0000","last_clean":"2024-06-28T16:13:02.146046+0000","last_became_active":"2024-06-28T15:00:17.393361+0000","last_became_peered":"2024-06-28T15:00:17.393361+0000","last_unstale":"2024-06-28T16:13:02.146046+0000","last_undegraded":"2024-06-28T16:13:02.146046+0000","last_fullsized":"2024-06-28T16:13:02.146046+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:50.952114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":892,"num_read_kb":761,"num_write":358,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":532,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.262525+0000","last_change":"2024-06-28T15:00:06.280312+0000","last_active":"2024-06-28T16:10:55.262525+0000","last_peered":"2024-06-28T16:10:55.262525+0000","last_clean":"2024-06-28T16:10:55.262525+0000","last_became_active":"2024-06-28T15:00:06.279950+0000","last_became_peered":"2024-06-28T15:00:06.279950+0000","last_unstale":"2024-06-28T16:10:55.262525+0000","last_undegraded":"2024-06-28T16:10:55.262525+0000","last_fullsized":"2024-06-28T16:10:55.262525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:05:40.110987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"260'334","reported_seq":991,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.720045+0000","last_change":"2024-06-28T15:00:15.413730+0000","last_active":"2024-06-28T16:14:34.720045+0000","last_peered":"2024-06-28T16:14:34.720045+0000","last_clean":"2024-06-28T16:14:34.720045+0000","last_became_active":"2024-06-28T15:00:15.413348+0000","last_became_peered":"2024-06-28T15:00:15.413348+0000","last_unstale":"2024-06-28T16:14:34.720045+0000","last_undegraded":"2024-06-28T16:14:34.720045+0000","last_fullsized":"2024-06-28T16:14:34.720045+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:07:23.937007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":504,"num_read_kb":3638,"num_write":1247,"num_write_kb":36014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"24'1","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.273547+0000","last_change":"2024-06-28T15:00:19.405107+0000","last_active":"2024-06-28T16:11:50.273547+0000","last_peered":"2024-06-28T16:11:50.273547+0000","last_clean":"2024-06-28T16:11:50.273547+0000","last_became_active":"2024-06-28T15:00:19.404653+0000","last_became_peered":"2024-06-28T15:00:19.404653+0000","last_unstale":"2024-06-28T16:11:50.273547+0000","last_undegraded":"2024-06-28T16:11:50.273547+0000","last_fullsized":"2024-06-28T16:11:50.273547+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:18:41.227360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"260'246","reported_seq":1691,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:11.127801+0000","last_change":"2024-06-28T15:00:21.484753+0000","last_active":"2024-06-28T16:14:11.127801+0000","last_peered":"2024-06-28T16:14:11.127801+0000","last_clean":"2024-06-28T16:14:11.127801+0000","last_became_active":"2024-06-28T15:00:21.483749+0000","last_became_peered":"2024-06-28T15:00:21.483749+0000","last_unstale":"2024-06-28T16:14:11.127801+0000","last_undegraded":"2024-06-28T16:14:11.127801+0000","last_fullsized":"2024-06-28T16:14:11.127801+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:25:11.741395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.558850+0000","last_change":"2024-06-28T15:00:23.400124+0000","last_active":"2024-06-28T16:11:15.558850+0000","last_peered":"2024-06-28T16:11:15.558850+0000","last_clean":"2024-06-28T16:11:15.558850+0000","last_became_active":"2024-06-28T15:00:23.399391+0000","last_became_peered":"2024-06-28T15:00:23.399391+0000","last_unstale":"2024-06-28T16:11:15.558850+0000","last_undegraded":"2024-06-28T16:11:15.558850+0000","last_fullsized":"2024-06-28T16:11:15.558850+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:03:52.771119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"260'239","reported_seq":844,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:07.109352+0000","last_change":"2024-06-28T15:00:25.420452+0000","last_active":"2024-06-28T16:14:07.109352+0000","last_peered":"2024-06-28T16:14:07.109352+0000","last_clean":"2024-06-28T16:14:07.109352+0000","last_became_active":"2024-06-28T15:00:25.419711+0000","last_became_peered":"2024-06-28T15:00:25.419711+0000","last_unstale":"2024-06-28T16:14:07.109352+0000","last_undegraded":"2024-06-28T16:14:07.109352+0000","last_fullsized":"2024-06-28T16:14:07.109352+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":200,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:31:14.431698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":467,"num_read_kb":491,"num_write":517,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"260'315","reported_seq":1006,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:04.988991+0000","last_change":"2024-06-28T15:00:17.398693+0000","last_active":"2024-06-28T16:12:04.988991+0000","last_peered":"2024-06-28T16:12:04.988991+0000","last_clean":"2024-06-28T16:12:04.988991+0000","last_became_active":"2024-06-28T15:00:17.393696+0000","last_became_peered":"2024-06-28T15:00:17.393696+0000","last_unstale":"2024-06-28T16:12:04.988991+0000","last_undegraded":"2024-06-28T16:12:04.988991+0000","last_fullsized":"2024-06-28T16:12:04.988991+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:48:40.719944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1365,"num_read_kb":1154,"num_write":468,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.074668+0000","last_change":"2024-06-28T15:00:06.284111+0000","last_active":"2024-06-28T16:10:55.074668+0000","last_peered":"2024-06-28T16:10:55.074668+0000","last_clean":"2024-06-28T16:10:55.074668+0000","last_became_active":"2024-06-28T15:00:06.283106+0000","last_became_peered":"2024-06-28T15:00:06.283106+0000","last_unstale":"2024-06-28T16:10:55.074668+0000","last_undegraded":"2024-06-28T16:10:55.074668+0000","last_fullsized":"2024-06-28T16:10:55.074668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:07.069940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"260'318","reported_seq":991,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.731218+0000","last_change":"2024-06-28T15:00:15.415068+0000","last_active":"2024-06-28T16:14:34.731218+0000","last_peered":"2024-06-28T16:14:34.731218+0000","last_clean":"2024-06-28T16:14:34.731218+0000","last_became_active":"2024-06-28T15:00:15.413880+0000","last_became_peered":"2024-06-28T15:00:15.413880+0000","last_unstale":"2024-06-28T16:14:34.731218+0000","last_undegraded":"2024-06-28T16:14:34.731218+0000","last_fullsized":"2024-06-28T16:14:34.731218+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:13:10.361144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2421978,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":465,"num_read_kb":85416,"num_write":1148,"num_write_kb":27167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.561459+0000","last_change":"2024-06-28T15:00:19.382869+0000","last_active":"2024-06-28T16:11:25.561459+0000","last_peered":"2024-06-28T16:11:25.561459+0000","last_clean":"2024-06-28T16:11:25.561459+0000","last_became_active":"2024-06-28T15:00:19.382486+0000","last_became_peered":"2024-06-28T15:00:19.382486+0000","last_unstale":"2024-06-28T16:11:25.561459+0000","last_undegraded":"2024-06-28T16:11:25.561459+0000","last_fullsized":"2024-06-28T16:11:25.561459+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:22:55.369992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"260'159","reported_seq":729,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.514897+0000","last_change":"2024-06-28T15:00:21.401345+0000","last_active":"2024-06-28T16:11:55.514897+0000","last_peered":"2024-06-28T16:11:55.514897+0000","last_clean":"2024-06-28T16:11:55.514897+0000","last_became_active":"2024-06-28T15:00:21.400535+0000","last_became_peered":"2024-06-28T15:00:21.400535+0000","last_unstale":"2024-06-28T16:11:55.514897+0000","last_undegraded":"2024-06-28T16:11:55.514897+0000","last_fullsized":"2024-06-28T16:11:55.514897+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:33:23.802275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":198,"num_read_kb":191,"num_write":142,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":495,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.094802+0000","last_change":"2024-06-28T15:00:23.417433+0000","last_active":"2024-06-28T16:11:55.094802+0000","last_peered":"2024-06-28T16:11:55.094802+0000","last_clean":"2024-06-28T16:11:55.094802+0000","last_became_active":"2024-06-28T15:00:23.416701+0000","last_became_peered":"2024-06-28T15:00:23.416701+0000","last_unstale":"2024-06-28T16:11:55.094802+0000","last_undegraded":"2024-06-28T16:11:55.094802+0000","last_fullsized":"2024-06-28T16:11:55.094802+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:59:56.613956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"260'117","reported_seq":633,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:07.191149+0000","last_change":"2024-06-28T15:00:25.439202+0000","last_active":"2024-06-28T16:13:07.191149+0000","last_peered":"2024-06-28T16:13:07.191149+0000","last_clean":"2024-06-28T16:13:07.191149+0000","last_became_active":"2024-06-28T15:00:25.438523+0000","last_became_peered":"2024-06-28T15:00:25.438523+0000","last_unstale":"2024-06-28T16:13:07.191149+0000","last_undegraded":"2024-06-28T16:13:07.191149+0000","last_fullsized":"2024-06-28T16:13:07.191149+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:31:55.560360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":129,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"260'169","reported_seq":774,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.093820+0000","last_change":"2024-06-28T15:00:17.399083+0000","last_active":"2024-06-28T16:11:10.093820+0000","last_peered":"2024-06-28T16:11:10.093820+0000","last_clean":"2024-06-28T16:11:10.093820+0000","last_became_active":"2024-06-28T15:00:17.398515+0000","last_became_peered":"2024-06-28T15:00:17.398515+0000","last_unstale":"2024-06-28T16:11:10.093820+0000","last_undegraded":"2024-06-28T16:11:10.093820+0000","last_fullsized":"2024-06-28T16:11:10.093820+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:10.485888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":644,"num_read_kb":512,"num_write":273,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"260'372","reported_seq":1043,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.710966+0000","last_change":"2024-06-28T15:00:15.417442+0000","last_active":"2024-06-28T16:14:34.710966+0000","last_peered":"2024-06-28T16:14:34.710966+0000","last_clean":"2024-06-28T16:14:34.710966+0000","last_became_active":"2024-06-28T15:00:15.414704+0000","last_became_peered":"2024-06-28T15:00:15.414704+0000","last_unstale":"2024-06-28T16:14:34.710966+0000","last_undegraded":"2024-06-28T16:14:34.710966+0000","last_fullsized":"2024-06-28T16:14:34.710966+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:54:15.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10506361,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":617,"num_read_kb":33619,"num_write":1289,"num_write_kb":23520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":529,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.092017+0000","last_change":"2024-06-28T15:00:06.302842+0000","last_active":"2024-06-28T16:11:00.092017+0000","last_peered":"2024-06-28T16:11:00.092017+0000","last_clean":"2024-06-28T16:11:00.092017+0000","last_became_active":"2024-06-28T15:00:06.302449+0000","last_became_peered":"2024-06-28T15:00:06.302449+0000","last_unstale":"2024-06-28T16:11:00.092017+0000","last_undegraded":"2024-06-28T16:11:00.092017+0000","last_fullsized":"2024-06-28T16:11:00.092017+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:06:10.739681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.557574+0000","last_change":"2024-06-28T15:00:19.385492+0000","last_active":"2024-06-28T16:11:10.557574+0000","last_peered":"2024-06-28T16:11:10.557574+0000","last_clean":"2024-06-28T16:11:10.557574+0000","last_became_active":"2024-06-28T15:00:19.384933+0000","last_became_peered":"2024-06-28T15:00:19.384933+0000","last_unstale":"2024-06-28T16:11:10.557574+0000","last_undegraded":"2024-06-28T16:11:10.557574+0000","last_fullsized":"2024-06-28T16:11:10.557574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:16:09.011700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"260'577","reported_seq":1265,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:12.941428+0000","last_change":"2024-06-28T15:00:21.407998+0000","last_active":"2024-06-28T16:14:12.941428+0000","last_peered":"2024-06-28T16:14:12.941428+0000","last_clean":"2024-06-28T16:14:12.941428+0000","last_became_active":"2024-06-28T15:00:21.405762+0000","last_became_peered":"2024-06-28T15:00:21.405762+0000","last_unstale":"2024-06-28T16:14:12.941428+0000","last_undegraded":"2024-06-28T16:14:12.941428+0000","last_fullsized":"2024-06-28T16:14:12.941428+0000","mapping_epoch":25,"log_start":"260'500","ondisk_log_start":"260'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":500,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:56:43.045836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":111793,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":975,"num_read_kb":1983,"num_write":826,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.086163+0000","last_change":"2024-06-28T15:00:23.405517+0000","last_active":"2024-06-28T16:11:15.086163+0000","last_peered":"2024-06-28T16:11:15.086163+0000","last_clean":"2024-06-28T16:11:15.086163+0000","last_became_active":"2024-06-28T15:00:23.404714+0000","last_became_peered":"2024-06-28T15:00:23.404714+0000","last_unstale":"2024-06-28T16:11:15.086163+0000","last_undegraded":"2024-06-28T16:11:15.086163+0000","last_fullsized":"2024-06-28T16:11:15.086163+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:54:34.158419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"260'147","reported_seq":663,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.650237+0000","last_change":"2024-06-28T15:00:25.557678+0000","last_active":"2024-06-28T16:14:06.650237+0000","last_peered":"2024-06-28T16:14:06.650237+0000","last_clean":"2024-06-28T16:14:06.650237+0000","last_became_active":"2024-06-28T15:00:25.556834+0000","last_became_peered":"2024-06-28T15:00:25.556834+0000","last_unstale":"2024-06-28T16:14:06.650237+0000","last_undegraded":"2024-06-28T16:14:06.650237+0000","last_fullsized":"2024-06-28T16:14:06.650237+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":100,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:52:25.536046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1183,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":342,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"260'139","reported_seq":768,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.557529+0000","last_change":"2024-06-28T15:00:17.372769+0000","last_active":"2024-06-28T16:11:10.557529+0000","last_peered":"2024-06-28T16:11:10.557529+0000","last_clean":"2024-06-28T16:11:10.557529+0000","last_became_active":"2024-06-28T15:00:17.372401+0000","last_became_peered":"2024-06-28T15:00:17.372401+0000","last_unstale":"2024-06-28T16:11:10.557529+0000","last_undegraded":"2024-06-28T16:11:10.557529+0000","last_fullsized":"2024-06-28T16:11:10.557529+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:34:11.796111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":577,"num_read_kb":470,"num_write":221,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"260'273","reported_seq":901,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.481261+0000","last_change":"2024-06-28T15:00:15.408605+0000","last_active":"2024-06-28T16:14:34.481261+0000","last_peered":"2024-06-28T16:14:34.481261+0000","last_clean":"2024-06-28T16:14:34.481261+0000","last_became_active":"2024-06-28T15:00:15.408140+0000","last_became_peered":"2024-06-28T15:00:15.408140+0000","last_unstale":"2024-06-28T16:14:34.481261+0000","last_undegraded":"2024-06-28T16:14:34.481261+0000","last_fullsized":"2024-06-28T16:14:34.481261+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":200,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:51:15.618780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":526399,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":387,"num_read_kb":8190,"num_write":1081,"num_write_kb":24156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.093729+0000","last_change":"2024-06-28T15:00:06.303984+0000","last_active":"2024-06-28T16:11:15.093729+0000","last_peered":"2024-06-28T16:11:15.093729+0000","last_clean":"2024-06-28T16:11:15.093729+0000","last_became_active":"2024-06-28T15:00:06.302962+0000","last_became_peered":"2024-06-28T15:00:06.302962+0000","last_unstale":"2024-06-28T16:11:15.093729+0000","last_undegraded":"2024-06-28T16:11:15.093729+0000","last_fullsized":"2024-06-28T16:11:15.093729+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:10:47.354122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.084030+0000","last_change":"2024-06-28T15:00:19.385776+0000","last_active":"2024-06-28T16:11:10.084030+0000","last_peered":"2024-06-28T16:11:10.084030+0000","last_clean":"2024-06-28T16:11:10.084030+0000","last_became_active":"2024-06-28T15:00:19.384961+0000","last_became_peered":"2024-06-28T15:00:19.384961+0000","last_unstale":"2024-06-28T16:11:10.084030+0000","last_undegraded":"2024-06-28T16:11:10.084030+0000","last_fullsized":"2024-06-28T16:11:10.084030+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:23:32.384577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"260'188","reported_seq":785,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.095201+0000","last_change":"2024-06-28T15:00:21.480780+0000","last_active":"2024-06-28T16:11:15.095201+0000","last_peered":"2024-06-28T16:11:15.095201+0000","last_clean":"2024-06-28T16:11:15.095201+0000","last_became_active":"2024-06-28T15:00:21.480107+0000","last_became_peered":"2024-06-28T15:00:21.480107+0000","last_unstale":"2024-06-28T16:11:15.095201+0000","last_undegraded":"2024-06-28T16:11:15.095201+0000","last_fullsized":"2024-06-28T16:11:15.095201+0000","mapping_epoch":25,"log_start":"257'100","ondisk_log_start":"257'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:46:14.608558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":263,"num_read_kb":253,"num_write":174,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"260'565","reported_seq":2877,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.824388+0000","last_change":"2024-06-28T15:00:23.429133+0000","last_active":"2024-06-28T16:14:32.824388+0000","last_peered":"2024-06-28T16:14:32.824388+0000","last_clean":"2024-06-28T16:14:32.824388+0000","last_became_active":"2024-06-28T15:00:23.427565+0000","last_became_peered":"2024-06-28T15:00:23.427565+0000","last_unstale":"2024-06-28T16:14:32.824388+0000","last_undegraded":"2024-06-28T16:14:32.824388+0000","last_fullsized":"2024-06-28T16:14:32.824388+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":500,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:07:13.588883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"260'124","reported_seq":638,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:30.524126+0000","last_change":"2024-06-28T15:00:25.554359+0000","last_active":"2024-06-28T16:12:30.524126+0000","last_peered":"2024-06-28T16:12:30.524126+0000","last_clean":"2024-06-28T16:12:30.524126+0000","last_became_active":"2024-06-28T15:00:25.553627+0000","last_became_peered":"2024-06-28T15:00:25.553627+0000","last_unstale":"2024-06-28T16:12:30.524126+0000","last_undegraded":"2024-06-28T16:12:30.524126+0000","last_fullsized":"2024-06-28T16:12:30.524126+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:38:46.580841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":713,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":165,"num_read_kb":148,"num_write":308,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"260'172","reported_seq":825,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.522189+0000","last_change":"2024-06-28T15:00:17.377265+0000","last_active":"2024-06-28T16:12:25.522189+0000","last_peered":"2024-06-28T16:12:25.522189+0000","last_clean":"2024-06-28T16:12:25.522189+0000","last_became_active":"2024-06-28T15:00:17.376516+0000","last_became_peered":"2024-06-28T15:00:17.376516+0000","last_unstale":"2024-06-28T16:12:25.522189+0000","last_undegraded":"2024-06-28T16:12:25.522189+0000","last_fullsized":"2024-06-28T16:12:25.522189+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:46:26.820686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":722,"num_read_kb":588,"num_write":244,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"260'336","reported_seq":980,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.728953+0000","last_change":"2024-06-28T15:00:15.403594+0000","last_active":"2024-06-28T16:14:34.728953+0000","last_peered":"2024-06-28T16:14:34.728953+0000","last_clean":"2024-06-28T16:14:34.728953+0000","last_became_active":"2024-06-28T15:00:15.403032+0000","last_became_peered":"2024-06-28T15:00:15.403032+0000","last_unstale":"2024-06-28T16:14:34.728953+0000","last_undegraded":"2024-06-28T16:14:34.728953+0000","last_fullsized":"2024-06-28T16:14:34.728953+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:18:51.571850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":968026,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":483,"num_read_kb":17032,"num_write":1147,"num_write_kb":35885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":528,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.099353+0000","last_change":"2024-06-28T15:00:06.278832+0000","last_active":"2024-06-28T16:11:35.099353+0000","last_peered":"2024-06-28T16:11:35.099353+0000","last_clean":"2024-06-28T16:11:35.099353+0000","last_became_active":"2024-06-28T15:00:06.278443+0000","last_became_peered":"2024-06-28T15:00:06.278443+0000","last_unstale":"2024-06-28T16:11:35.099353+0000","last_undegraded":"2024-06-28T16:11:35.099353+0000","last_fullsized":"2024-06-28T16:11:35.099353+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:40:27.400018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.085018+0000","last_change":"2024-06-28T15:00:19.380916+0000","last_active":"2024-06-28T16:11:10.085018+0000","last_peered":"2024-06-28T16:11:10.085018+0000","last_clean":"2024-06-28T16:11:10.085018+0000","last_became_active":"2024-06-28T15:00:19.378932+0000","last_became_peered":"2024-06-28T15:00:19.378932+0000","last_unstale":"2024-06-28T16:11:10.085018+0000","last_undegraded":"2024-06-28T16:11:10.085018+0000","last_fullsized":"2024-06-28T16:11:10.085018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:36:40.033839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"260'138","reported_seq":698,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.086320+0000","last_change":"2024-06-28T15:00:21.396435+0000","last_active":"2024-06-28T16:11:15.086320+0000","last_peered":"2024-06-28T16:11:15.086320+0000","last_clean":"2024-06-28T16:11:15.086320+0000","last_became_active":"2024-06-28T15:00:21.395462+0000","last_became_peered":"2024-06-28T15:00:21.395462+0000","last_unstale":"2024-06-28T16:11:15.086320+0000","last_undegraded":"2024-06-28T16:11:15.086320+0000","last_fullsized":"2024-06-28T16:11:15.086320+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:51:41.956055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":167,"num_read_kb":161,"num_write":112,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"260'537","reported_seq":2581,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.259280+0000","last_change":"2024-06-28T15:00:23.428281+0000","last_active":"2024-06-28T16:14:34.259280+0000","last_peered":"2024-06-28T16:14:34.259280+0000","last_clean":"2024-06-28T16:14:34.259280+0000","last_became_active":"2024-06-28T15:00:23.427568+0000","last_became_peered":"2024-06-28T15:00:23.427568+0000","last_unstale":"2024-06-28T16:14:34.259280+0000","last_undegraded":"2024-06-28T16:14:34.259280+0000","last_fullsized":"2024-06-28T16:14:34.259280+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:37:02.302827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"260'247","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.544258+0000","last_change":"2024-06-28T15:00:25.428796+0000","last_active":"2024-06-28T16:10:10.544258+0000","last_peered":"2024-06-28T16:10:10.544258+0000","last_clean":"2024-06-28T16:10:10.544258+0000","last_became_active":"2024-06-28T15:00:25.428067+0000","last_became_peered":"2024-06-28T15:00:25.428067+0000","last_unstale":"2024-06-28T16:10:10.544258+0000","last_undegraded":"2024-06-28T16:10:10.544258+0000","last_fullsized":"2024-06-28T16:10:10.544258+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":200,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:07:43.747052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":297,"num_write":552,"num_write_kb":170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"260'96","reported_seq":701,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:46.633284+0000","last_change":"2024-06-28T15:00:17.397233+0000","last_active":"2024-06-28T16:12:46.633284+0000","last_peered":"2024-06-28T16:12:46.633284+0000","last_clean":"2024-06-28T16:12:46.633284+0000","last_became_active":"2024-06-28T15:00:17.396413+0000","last_became_peered":"2024-06-28T15:00:17.396413+0000","last_unstale":"2024-06-28T16:12:46.633284+0000","last_undegraded":"2024-06-28T16:12:46.633284+0000","last_fullsized":"2024-06-28T16:12:46.633284+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:50:29.409467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":410,"num_read_kb":340,"num_write":139,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"260'261","reported_seq":1040,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:45.565343+0000","last_change":"2024-06-28T15:00:15.396426+0000","last_active":"2024-06-28T16:11:45.565343+0000","last_peered":"2024-06-28T16:11:45.565343+0000","last_clean":"2024-06-28T16:11:45.565343+0000","last_became_active":"2024-06-28T15:00:15.395980+0000","last_became_peered":"2024-06-28T15:00:15.395980+0000","last_unstale":"2024-06-28T16:11:45.565343+0000","last_undegraded":"2024-06-28T16:11:45.565343+0000","last_fullsized":"2024-06-28T16:11:45.565343+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:22:49.730474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2049512,"num_objects":63,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":592,"num_read_kb":125515,"num_write":1130,"num_write_kb":17513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":521,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089897+0000","last_change":"2024-06-28T15:00:06.284169+0000","last_active":"2024-06-28T16:11:35.089897+0000","last_peered":"2024-06-28T16:11:35.089897+0000","last_clean":"2024-06-28T16:11:35.089897+0000","last_became_active":"2024-06-28T15:00:06.283380+0000","last_became_peered":"2024-06-28T15:00:06.283380+0000","last_unstale":"2024-06-28T16:11:35.089897+0000","last_undegraded":"2024-06-28T16:11:35.089897+0000","last_fullsized":"2024-06-28T16:11:35.089897+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:39:59.368666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"31'5","reported_seq":842,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:31.932486+0000","last_change":"2024-06-28T15:00:19.372913+0000","last_active":"2024-06-28T16:14:31.932486+0000","last_peered":"2024-06-28T16:14:31.932486+0000","last_clean":"2024-06-28T16:14:31.932486+0000","last_became_active":"2024-06-28T15:00:19.372552+0000","last_became_peered":"2024-06-28T15:00:19.372552+0000","last_unstale":"2024-06-28T16:14:31.932486+0000","last_undegraded":"2024-06-28T16:14:31.932486+0000","last_fullsized":"2024-06-28T16:14:31.932486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:19:05.828246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":500,"num_read_kb":812,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"260'748","reported_seq":1492,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.576522+0000","last_change":"2024-06-28T15:00:21.481881+0000","last_active":"2024-06-28T16:14:33.576522+0000","last_peered":"2024-06-28T16:14:33.576522+0000","last_clean":"2024-06-28T16:14:33.576522+0000","last_became_active":"2024-06-28T15:00:21.481155+0000","last_became_peered":"2024-06-28T15:00:21.481155+0000","last_unstale":"2024-06-28T16:14:33.576522+0000","last_undegraded":"2024-06-28T16:14:33.576522+0000","last_fullsized":"2024-06-28T16:14:33.576522+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":700,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:24:28.507362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131438,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1320,"num_read_kb":3166,"num_write":1058,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.505408+0000","last_change":"2024-06-28T15:00:23.409553+0000","last_active":"2024-06-28T16:11:15.505408+0000","last_peered":"2024-06-28T16:11:15.505408+0000","last_clean":"2024-06-28T16:11:15.505408+0000","last_became_active":"2024-06-28T15:00:23.408777+0000","last_became_peered":"2024-06-28T15:00:23.408777+0000","last_unstale":"2024-06-28T16:11:15.505408+0000","last_undegraded":"2024-06-28T16:11:15.505408+0000","last_fullsized":"2024-06-28T16:11:15.505408+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:29:28.695608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"260'129","reported_seq":640,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:07.935429+0000","last_change":"2024-06-28T15:00:25.439187+0000","last_active":"2024-06-28T16:14:07.935429+0000","last_peered":"2024-06-28T16:14:07.935429+0000","last_clean":"2024-06-28T16:14:07.935429+0000","last_became_active":"2024-06-28T15:00:25.438476+0000","last_became_peered":"2024-06-28T15:00:25.438476+0000","last_unstale":"2024-06-28T16:14:07.935429+0000","last_undegraded":"2024-06-28T16:14:07.935429+0000","last_fullsized":"2024-06-28T16:14:07.935429+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:14:10.401700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":173,"num_read_kb":156,"num_write":320,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"260'151","reported_seq":718,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.829530+0000","last_change":"2024-06-28T15:00:25.440427+0000","last_active":"2024-06-28T16:14:08.829530+0000","last_peered":"2024-06-28T16:14:08.829530+0000","last_clean":"2024-06-28T16:14:08.829530+0000","last_became_active":"2024-06-28T15:00:25.439672+0000","last_became_peered":"2024-06-28T15:00:25.439672+0000","last_unstale":"2024-06-28T16:14:08.829530+0000","last_undegraded":"2024-06-28T16:14:08.829530+0000","last_fullsized":"2024-06-28T16:14:08.829530+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:21:43.580938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":207,"num_read_kb":205,"num_write":345,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"260'434","reported_seq":1071,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:58.995763+0000","last_change":"2024-06-28T15:00:17.402772+0000","last_active":"2024-06-28T16:10:58.995763+0000","last_peered":"2024-06-28T16:10:58.995763+0000","last_clean":"2024-06-28T16:10:58.995763+0000","last_became_active":"2024-06-28T15:00:17.401981+0000","last_became_peered":"2024-06-28T15:00:17.401981+0000","last_unstale":"2024-06-28T16:10:58.995763+0000","last_undegraded":"2024-06-28T16:10:58.995763+0000","last_fullsized":"2024-06-28T16:10:58.995763+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":400,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:36:44.706683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1728,"num_read_kb":1391,"num_write":746,"num_write_kb":387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"260'274","reported_seq":900,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.557629+0000","last_change":"2024-06-28T15:00:15.404138+0000","last_active":"2024-06-28T16:14:34.557629+0000","last_peered":"2024-06-28T16:14:34.557629+0000","last_clean":"2024-06-28T16:14:34.557629+0000","last_became_active":"2024-06-28T15:00:15.403450+0000","last_became_peered":"2024-06-28T15:00:15.403450+0000","last_unstale":"2024-06-28T16:14:34.557629+0000","last_undegraded":"2024-06-28T16:14:34.557629+0000","last_fullsized":"2024-06-28T16:14:34.557629+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:20:44.673833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437799,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":13452,"num_write":1073,"num_write_kb":25751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.084184+0000","last_change":"2024-06-28T15:00:19.398728+0000","last_active":"2024-06-28T16:10:20.084184+0000","last_peered":"2024-06-28T16:10:20.084184+0000","last_clean":"2024-06-28T16:10:20.084184+0000","last_became_active":"2024-06-28T15:00:19.398138+0000","last_became_peered":"2024-06-28T15:00:19.398138+0000","last_unstale":"2024-06-28T16:10:20.084184+0000","last_undegraded":"2024-06-28T16:10:20.084184+0000","last_fullsized":"2024-06-28T16:10:20.084184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:24:09.555887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"260'298","reported_seq":895,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.023577+0000","last_change":"2024-06-28T15:00:21.405835+0000","last_active":"2024-06-28T16:14:34.023577+0000","last_peered":"2024-06-28T16:14:34.023577+0000","last_clean":"2024-06-28T16:14:34.023577+0000","last_became_active":"2024-06-28T15:00:21.405016+0000","last_became_peered":"2024-06-28T15:00:21.405016+0000","last_unstale":"2024-06-28T16:14:34.023577+0000","last_undegraded":"2024-06-28T16:14:34.023577+0000","last_fullsized":"2024-06-28T16:14:34.023577+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:00:56.662170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53726,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":505,"num_read_kb":1165,"num_write":406,"num_write_kb":309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":499,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493149+0000","last_change":"2024-06-28T15:00:23.418156+0000","last_active":"2024-06-28T16:10:25.493149+0000","last_peered":"2024-06-28T16:10:25.493149+0000","last_clean":"2024-06-28T16:10:25.493149+0000","last_became_active":"2024-06-28T15:00:23.417390+0000","last_became_peered":"2024-06-28T15:00:23.417390+0000","last_unstale":"2024-06-28T16:10:25.493149+0000","last_undegraded":"2024-06-28T16:10:25.493149+0000","last_fullsized":"2024-06-28T16:10:25.493149+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:22:25.288900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"260'101","reported_seq":608,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:30.524057+0000","last_change":"2024-06-28T15:00:25.413154+0000","last_active":"2024-06-28T16:12:30.524057+0000","last_peered":"2024-06-28T16:12:30.524057+0000","last_clean":"2024-06-28T16:12:30.524057+0000","last_became_active":"2024-06-28T15:00:25.412379+0000","last_became_peered":"2024-06-28T15:00:25.412379+0000","last_unstale":"2024-06-28T16:12:30.524057+0000","last_undegraded":"2024-06-28T16:12:30.524057+0000","last_fullsized":"2024-06-28T16:12:30.524057+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:26:49.949836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":88,"num_write":238,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"260'189","reported_seq":822,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:47.151324+0000","last_change":"2024-06-28T15:00:17.406956+0000","last_active":"2024-06-28T16:11:47.151324+0000","last_peered":"2024-06-28T16:11:47.151324+0000","last_clean":"2024-06-28T16:11:47.151324+0000","last_became_active":"2024-06-28T15:00:17.406647+0000","last_became_peered":"2024-06-28T15:00:17.406647+0000","last_unstale":"2024-06-28T16:11:47.151324+0000","last_undegraded":"2024-06-28T16:11:47.151324+0000","last_fullsized":"2024-06-28T16:11:47.151324+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:18:05.135571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":748,"num_read_kb":601,"num_write":301,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"260'334","reported_seq":1001,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.613707+0000","last_change":"2024-06-28T15:00:15.413269+0000","last_active":"2024-06-28T16:14:34.613707+0000","last_peered":"2024-06-28T16:14:34.613707+0000","last_clean":"2024-06-28T16:14:34.613707+0000","last_became_active":"2024-06-28T15:00:15.402945+0000","last_became_peered":"2024-06-28T15:00:15.402945+0000","last_unstale":"2024-06-28T16:14:34.613707+0000","last_undegraded":"2024-06-28T16:14:34.613707+0000","last_fullsized":"2024-06-28T16:14:34.613707+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:41:27.586797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340417,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":534,"num_read_kb":20609,"num_write":1305,"num_write_kb":45266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,4],"acting":[7,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.492088+0000","last_change":"2024-06-28T15:00:19.383614+0000","last_active":"2024-06-28T16:10:20.492088+0000","last_peered":"2024-06-28T16:10:20.492088+0000","last_clean":"2024-06-28T16:10:20.492088+0000","last_became_active":"2024-06-28T15:00:19.382839+0000","last_became_peered":"2024-06-28T15:00:19.382839+0000","last_unstale":"2024-06-28T16:10:20.492088+0000","last_undegraded":"2024-06-28T16:10:20.492088+0000","last_fullsized":"2024-06-28T16:10:20.492088+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:09:23.668167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"260'52","reported_seq":573,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:58.477001+0000","last_change":"2024-06-28T15:00:21.407387+0000","last_active":"2024-06-28T16:10:58.477001+0000","last_peered":"2024-06-28T16:10:58.477001+0000","last_clean":"2024-06-28T16:10:58.477001+0000","last_became_active":"2024-06-28T15:00:21.404727+0000","last_became_peered":"2024-06-28T15:00:21.404727+0000","last_unstale":"2024-06-28T16:10:58.477001+0000","last_undegraded":"2024-06-28T16:10:58.477001+0000","last_fullsized":"2024-06-28T16:10:58.477001+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:02:08.588463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":48,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":494,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.081639+0000","last_change":"2024-06-28T15:00:23.419840+0000","last_active":"2024-06-28T16:10:25.081639+0000","last_peered":"2024-06-28T16:10:25.081639+0000","last_clean":"2024-06-28T16:10:25.081639+0000","last_became_active":"2024-06-28T15:00:23.419115+0000","last_became_peered":"2024-06-28T15:00:23.419115+0000","last_unstale":"2024-06-28T16:10:25.081639+0000","last_undegraded":"2024-06-28T16:10:25.081639+0000","last_fullsized":"2024-06-28T16:10:25.081639+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:05:05.710065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"260'190","reported_seq":719,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.664824+0000","last_change":"2024-06-28T15:00:25.426812+0000","last_active":"2024-06-28T16:14:06.664824+0000","last_peered":"2024-06-28T16:14:06.664824+0000","last_clean":"2024-06-28T16:14:06.664824+0000","last_became_active":"2024-06-28T15:00:25.426046+0000","last_became_peered":"2024-06-28T15:00:25.426046+0000","last_unstale":"2024-06-28T16:14:06.664824+0000","last_undegraded":"2024-06-28T16:14:06.664824+0000","last_fullsized":"2024-06-28T16:14:06.664824+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:19:11.968158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":286,"num_read_kb":283,"num_write":434,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"260'423","reported_seq":1170,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:17.460800+0000","last_change":"2024-06-28T15:00:17.390367+0000","last_active":"2024-06-28T16:11:17.460800+0000","last_peered":"2024-06-28T16:11:17.460800+0000","last_clean":"2024-06-28T16:11:17.460800+0000","last_became_active":"2024-06-28T15:00:17.389992+0000","last_became_peered":"2024-06-28T15:00:17.389992+0000","last_unstale":"2024-06-28T16:11:17.460800+0000","last_undegraded":"2024-06-28T16:11:17.460800+0000","last_fullsized":"2024-06-28T16:11:17.460800+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":400,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:13:23.718079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1786,"num_read_kb":1490,"num_write":759,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"260'328","reported_seq":971,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.635861+0000","last_change":"2024-06-28T15:00:15.427261+0000","last_active":"2024-06-28T16:14:34.635861+0000","last_peered":"2024-06-28T16:14:34.635861+0000","last_clean":"2024-06-28T16:14:34.635861+0000","last_became_active":"2024-06-28T15:00:15.419267+0000","last_became_peered":"2024-06-28T15:00:15.419267+0000","last_unstale":"2024-06-28T16:14:34.635861+0000","last_undegraded":"2024-06-28T16:14:34.635861+0000","last_fullsized":"2024-06-28T16:14:34.635861+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:49:04.986604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5413475,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":487,"num_read_kb":27459,"num_write":1288,"num_write_kb":48933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080344+0000","last_change":"2024-06-28T15:00:19.386890+0000","last_active":"2024-06-28T16:10:20.080344+0000","last_peered":"2024-06-28T16:10:20.080344+0000","last_clean":"2024-06-28T16:10:20.080344+0000","last_became_active":"2024-06-28T15:00:19.385990+0000","last_became_peered":"2024-06-28T15:00:19.385990+0000","last_unstale":"2024-06-28T16:10:20.080344+0000","last_undegraded":"2024-06-28T16:10:20.080344+0000","last_fullsized":"2024-06-28T16:10:20.080344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:39:52.204033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"260'318","reported_seq":931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.206768+0000","last_change":"2024-06-28T15:00:21.406717+0000","last_active":"2024-06-28T16:14:33.206768+0000","last_peered":"2024-06-28T16:14:33.206768+0000","last_clean":"2024-06-28T16:14:33.206768+0000","last_became_active":"2024-06-28T15:00:21.405045+0000","last_became_peered":"2024-06-28T15:00:21.405045+0000","last_unstale":"2024-06-28T16:14:33.206768+0000","last_undegraded":"2024-06-28T16:14:33.206768+0000","last_fullsized":"2024-06-28T16:14:33.206768+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:50:38.218164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48456,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":530,"num_read_kb":1162,"num_write":414,"num_write_kb":289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":492,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.074474+0000","last_change":"2024-06-28T15:00:23.415444+0000","last_active":"2024-06-28T16:10:25.074474+0000","last_peered":"2024-06-28T16:10:25.074474+0000","last_clean":"2024-06-28T16:10:25.074474+0000","last_became_active":"2024-06-28T15:00:23.411264+0000","last_became_peered":"2024-06-28T15:00:23.411264+0000","last_unstale":"2024-06-28T16:10:25.074474+0000","last_undegraded":"2024-06-28T16:10:25.074474+0000","last_fullsized":"2024-06-28T16:10:25.074474+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:58:05.328252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"260'110","reported_seq":602,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.067825+0000","last_change":"2024-06-28T15:00:25.443135+0000","last_active":"2024-06-28T16:10:25.067825+0000","last_peered":"2024-06-28T16:10:25.067825+0000","last_clean":"2024-06-28T16:10:25.067825+0000","last_became_active":"2024-06-28T15:00:25.442010+0000","last_became_peered":"2024-06-28T15:00:25.442010+0000","last_unstale":"2024-06-28T16:10:25.067825+0000","last_undegraded":"2024-06-28T16:10:25.067825+0000","last_fullsized":"2024-06-28T16:10:25.067825+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:56:35.040791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":638,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":85,"num_write":260,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"260'105","reported_seq":669,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.491831+0000","last_change":"2024-06-28T15:00:17.376172+0000","last_active":"2024-06-28T16:10:20.491831+0000","last_peered":"2024-06-28T16:10:20.491831+0000","last_clean":"2024-06-28T16:10:20.491831+0000","last_became_active":"2024-06-28T15:00:17.373396+0000","last_became_peered":"2024-06-28T15:00:17.373396+0000","last_unstale":"2024-06-28T16:10:20.491831+0000","last_undegraded":"2024-06-28T16:10:20.491831+0000","last_fullsized":"2024-06-28T16:10:20.491831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:59:28.317922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":424,"num_read_kb":340,"num_write":155,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"260'260","reported_seq":907,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:03.731453+0000","last_change":"2024-06-28T15:00:15.412826+0000","last_active":"2024-06-28T16:12:03.731453+0000","last_peered":"2024-06-28T16:12:03.731453+0000","last_clean":"2024-06-28T16:12:03.731453+0000","last_became_active":"2024-06-28T15:00:15.412333+0000","last_became_peered":"2024-06-28T15:00:15.412333+0000","last_unstale":"2024-06-28T16:12:03.731453+0000","last_undegraded":"2024-06-28T16:12:03.731453+0000","last_fullsized":"2024-06-28T16:12:03.731453+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":200,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:32:08.060194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6832535,"num_objects":55,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":411,"num_read_kb":31119,"num_write":1229,"num_write_kb":43155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073040+0000","last_change":"2024-06-28T15:00:19.387598+0000","last_active":"2024-06-28T16:10:20.073040+0000","last_peered":"2024-06-28T16:10:20.073040+0000","last_clean":"2024-06-28T16:10:20.073040+0000","last_became_active":"2024-06-28T15:00:19.386645+0000","last_became_peered":"2024-06-28T15:00:19.386645+0000","last_unstale":"2024-06-28T16:10:20.073040+0000","last_undegraded":"2024-06-28T16:10:20.073040+0000","last_fullsized":"2024-06-28T16:10:20.073040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:54:00.701086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"260'289","reported_seq":873,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.977932+0000","last_change":"2024-06-28T15:00:21.395133+0000","last_active":"2024-06-28T16:14:32.977932+0000","last_peered":"2024-06-28T16:14:32.977932+0000","last_clean":"2024-06-28T16:14:32.977932+0000","last_became_active":"2024-06-28T15:00:21.394357+0000","last_became_peered":"2024-06-28T15:00:21.394357+0000","last_unstale":"2024-06-28T16:14:32.977932+0000","last_undegraded":"2024-06-28T16:14:32.977932+0000","last_fullsized":"2024-06-28T16:14:32.977932+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:35:53.429744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53697,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":496,"num_read_kb":1194,"num_write":404,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.548098+0000","last_change":"2024-06-28T15:00:23.407033+0000","last_active":"2024-06-28T16:10:25.548098+0000","last_peered":"2024-06-28T16:10:25.548098+0000","last_clean":"2024-06-28T16:10:25.548098+0000","last_became_active":"2024-06-28T15:00:23.406296+0000","last_became_peered":"2024-06-28T15:00:23.406296+0000","last_unstale":"2024-06-28T16:10:25.548098+0000","last_undegraded":"2024-06-28T16:10:25.548098+0000","last_fullsized":"2024-06-28T16:10:25.548098+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:09:16.742634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"260'68","reported_seq":636,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.670953+0000","last_change":"2024-06-28T15:00:17.397537+0000","last_active":"2024-06-28T16:14:06.670953+0000","last_peered":"2024-06-28T16:14:06.670953+0000","last_clean":"2024-06-28T16:14:06.670953+0000","last_became_active":"2024-06-28T15:00:17.396861+0000","last_became_peered":"2024-06-28T15:00:17.396861+0000","last_unstale":"2024-06-28T16:14:06.670953+0000","last_undegraded":"2024-06-28T16:14:06.670953+0000","last_fullsized":"2024-06-28T16:14:06.670953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:27:59.755447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":279,"num_read_kb":229,"num_write":93,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"260'309","reported_seq":944,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.727038+0000","last_change":"2024-06-28T15:00:15.383453+0000","last_active":"2024-06-28T16:14:34.727038+0000","last_peered":"2024-06-28T16:14:34.727038+0000","last_clean":"2024-06-28T16:14:34.727038+0000","last_became_active":"2024-06-28T15:00:15.383008+0000","last_became_peered":"2024-06-28T15:00:15.383008+0000","last_unstale":"2024-06-28T16:14:34.727038+0000","last_undegraded":"2024-06-28T16:14:34.727038+0000","last_fullsized":"2024-06-28T16:14:34.727038+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:16:34.579751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2156401,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":459,"num_read_kb":6724,"num_write":1144,"num_write_kb":24207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.550286+0000","last_change":"2024-06-28T15:00:19.390462+0000","last_active":"2024-06-28T16:10:35.550286+0000","last_peered":"2024-06-28T16:10:35.550286+0000","last_clean":"2024-06-28T16:10:35.550286+0000","last_became_active":"2024-06-28T15:00:19.390111+0000","last_became_peered":"2024-06-28T15:00:19.390111+0000","last_unstale":"2024-06-28T16:10:35.550286+0000","last_undegraded":"2024-06-28T16:10:35.550286+0000","last_fullsized":"2024-06-28T16:10:35.550286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:44:31.427541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"260'472","reported_seq":1122,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.468574+0000","last_change":"2024-06-28T15:00:21.399070+0000","last_active":"2024-06-28T16:14:34.468574+0000","last_peered":"2024-06-28T16:14:34.468574+0000","last_clean":"2024-06-28T16:14:34.468574+0000","last_became_active":"2024-06-28T15:00:21.398262+0000","last_became_peered":"2024-06-28T15:00:21.398262+0000","last_unstale":"2024-06-28T16:14:34.468574+0000","last_undegraded":"2024-06-28T16:14:34.468574+0000","last_fullsized":"2024-06-28T16:14:34.468574+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":400,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:19:49.030651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86979,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":863,"num_read_kb":2070,"num_write":734,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":493,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.067855+0000","last_change":"2024-06-28T15:00:23.411496+0000","last_active":"2024-06-28T16:10:25.067855+0000","last_peered":"2024-06-28T16:10:25.067855+0000","last_clean":"2024-06-28T16:10:25.067855+0000","last_became_active":"2024-06-28T15:00:23.410519+0000","last_became_peered":"2024-06-28T15:00:23.410519+0000","last_unstale":"2024-06-28T16:10:25.067855+0000","last_undegraded":"2024-06-28T16:10:25.067855+0000","last_fullsized":"2024-06-28T16:10:25.067855+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:02:03.245169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"260'167","reported_seq":684,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.493320+0000","last_change":"2024-06-28T15:00:25.559194+0000","last_active":"2024-06-28T16:10:25.493320+0000","last_peered":"2024-06-28T16:10:25.493320+0000","last_clean":"2024-06-28T16:10:25.493320+0000","last_became_active":"2024-06-28T15:00:25.558426+0000","last_became_peered":"2024-06-28T15:00:25.558426+0000","last_unstale":"2024-06-28T16:10:25.493320+0000","last_undegraded":"2024-06-28T16:10:25.493320+0000","last_fullsized":"2024-06-28T16:10:25.493320+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":100,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:12:12.643863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":845,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":215,"num_read_kb":219,"num_write":377,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"260'158","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.550235+0000","last_change":"2024-06-28T15:00:17.395953+0000","last_active":"2024-06-28T16:10:35.550235+0000","last_peered":"2024-06-28T16:10:35.550235+0000","last_clean":"2024-06-28T16:10:35.550235+0000","last_became_active":"2024-06-28T15:00:17.395643+0000","last_became_peered":"2024-06-28T15:00:17.395643+0000","last_unstale":"2024-06-28T16:10:35.550235+0000","last_undegraded":"2024-06-28T16:10:35.550235+0000","last_fullsized":"2024-06-28T16:10:35.550235+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:17:26.238377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":654,"num_read_kb":561,"num_write":257,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"260'290","reported_seq":930,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.634482+0000","last_change":"2024-06-28T15:00:59.940407+0000","last_active":"2024-06-28T16:14:34.634482+0000","last_peered":"2024-06-28T16:14:34.634482+0000","last_clean":"2024-06-28T16:14:34.634482+0000","last_became_active":"2024-06-28T15:00:59.939485+0000","last_became_peered":"2024-06-28T15:00:59.939485+0000","last_unstale":"2024-06-28T16:14:34.634482+0000","last_undegraded":"2024-06-28T16:14:34.634482+0000","last_fullsized":"2024-06-28T16:14:34.634482+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:30:29.418898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":430,"num_read_kb":21907,"num_write":1034,"num_write_kb":35159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,6],"acting":[1,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.084186+0000","last_change":"2024-06-28T15:00:19.385764+0000","last_active":"2024-06-28T16:11:10.084186+0000","last_peered":"2024-06-28T16:11:10.084186+0000","last_clean":"2024-06-28T16:11:10.084186+0000","last_became_active":"2024-06-28T15:00:19.385068+0000","last_became_peered":"2024-06-28T15:00:19.385068+0000","last_unstale":"2024-06-28T16:11:10.084186+0000","last_undegraded":"2024-06-28T16:11:10.084186+0000","last_fullsized":"2024-06-28T16:11:10.084186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:18:40.223976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"260'348","reported_seq":954,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.054962+0000","last_change":"2024-06-28T15:00:21.401177+0000","last_active":"2024-06-28T16:14:33.054962+0000","last_peered":"2024-06-28T16:14:33.054962+0000","last_clean":"2024-06-28T16:14:33.054962+0000","last_became_active":"2024-06-28T15:00:21.400480+0000","last_became_peered":"2024-06-28T15:00:21.400480+0000","last_unstale":"2024-06-28T16:14:33.054962+0000","last_undegraded":"2024-06-28T16:14:33.054962+0000","last_fullsized":"2024-06-28T16:14:33.054962+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":300,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:08:38.525781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47708,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":581,"num_read_kb":1249,"num_write":480,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":498,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.085271+0000","last_change":"2024-06-28T15:00:23.418653+0000","last_active":"2024-06-28T16:10:25.085271+0000","last_peered":"2024-06-28T16:10:25.085271+0000","last_clean":"2024-06-28T16:10:25.085271+0000","last_became_active":"2024-06-28T15:00:23.417580+0000","last_became_peered":"2024-06-28T15:00:23.417580+0000","last_unstale":"2024-06-28T16:10:25.085271+0000","last_undegraded":"2024-06-28T16:10:25.085271+0000","last_fullsized":"2024-06-28T16:10:25.085271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:22:55.183479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"260'170","reported_seq":677,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:07.865086+0000","last_change":"2024-06-28T15:00:25.431039+0000","last_active":"2024-06-28T16:14:07.865086+0000","last_peered":"2024-06-28T16:14:07.865086+0000","last_clean":"2024-06-28T16:14:07.865086+0000","last_became_active":"2024-06-28T15:00:25.430299+0000","last_became_peered":"2024-06-28T15:00:25.430299+0000","last_unstale":"2024-06-28T16:14:07.865086+0000","last_undegraded":"2024-06-28T16:14:07.865086+0000","last_fullsized":"2024-06-28T16:14:07.865086+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:31:33.565366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2035,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":193,"num_read_kb":191,"num_write":477,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"260'315","reported_seq":1057,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.056426+0000","last_change":"2024-06-28T15:00:17.383684+0000","last_active":"2024-06-28T16:10:30.056426+0000","last_peered":"2024-06-28T16:10:30.056426+0000","last_clean":"2024-06-28T16:10:30.056426+0000","last_became_active":"2024-06-28T15:00:17.383284+0000","last_became_peered":"2024-06-28T15:00:17.383284+0000","last_unstale":"2024-06-28T16:10:30.056426+0000","last_undegraded":"2024-06-28T16:10:30.056426+0000","last_fullsized":"2024-06-28T16:10:30.056426+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:36:05.869810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1515,"num_read_kb":1335,"num_write":548,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"260'347","reported_seq":1028,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.674376+0000","last_change":"2024-06-28T15:00:15.406314+0000","last_active":"2024-06-28T16:14:34.674376+0000","last_peered":"2024-06-28T16:14:34.674376+0000","last_clean":"2024-06-28T16:14:34.674376+0000","last_became_active":"2024-06-28T15:00:15.405499+0000","last_became_peered":"2024-06-28T15:00:15.405499+0000","last_unstale":"2024-06-28T16:14:34.674376+0000","last_undegraded":"2024-06-28T16:14:34.674376+0000","last_fullsized":"2024-06-28T16:14:34.674376+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:26:47.268928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17426850,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":585,"num_read_kb":18043,"num_write":1303,"num_write_kb":33783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.074018+0000","last_change":"2024-06-28T15:00:19.381603+0000","last_active":"2024-06-28T16:10:20.074018+0000","last_peered":"2024-06-28T16:10:20.074018+0000","last_clean":"2024-06-28T16:10:20.074018+0000","last_became_active":"2024-06-28T15:00:19.378852+0000","last_became_peered":"2024-06-28T15:00:19.378852+0000","last_unstale":"2024-06-28T16:10:20.074018+0000","last_undegraded":"2024-06-28T16:10:20.074018+0000","last_fullsized":"2024-06-28T16:10:20.074018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:45:27.874936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"260'279","reported_seq":880,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.314802+0000","last_change":"2024-06-28T15:00:21.481610+0000","last_active":"2024-06-28T16:14:34.314802+0000","last_peered":"2024-06-28T16:14:34.314802+0000","last_clean":"2024-06-28T16:14:34.314802+0000","last_became_active":"2024-06-28T15:00:21.480868+0000","last_became_peered":"2024-06-28T15:00:21.480868+0000","last_unstale":"2024-06-28T16:14:34.314802+0000","last_undegraded":"2024-06-28T16:14:34.314802+0000","last_fullsized":"2024-06-28T16:14:34.314802+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":200,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:31:53.245138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40850,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":446,"num_read_kb":956,"num_write":348,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:45.089049+0000","last_change":"2024-06-28T15:00:23.401297+0000","last_active":"2024-06-28T16:10:45.089049+0000","last_peered":"2024-06-28T16:10:45.089049+0000","last_clean":"2024-06-28T16:10:45.089049+0000","last_became_active":"2024-06-28T15:00:23.400417+0000","last_became_peered":"2024-06-28T15:00:23.400417+0000","last_unstale":"2024-06-28T16:10:45.089049+0000","last_undegraded":"2024-06-28T16:10:45.089049+0000","last_fullsized":"2024-06-28T16:10:45.089049+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:57.286666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"260'117","reported_seq":617,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:12.999377+0000","last_change":"2024-06-28T15:00:25.438977+0000","last_active":"2024-06-28T16:14:12.999377+0000","last_peered":"2024-06-28T16:14:12.999377+0000","last_clean":"2024-06-28T16:14:12.999377+0000","last_became_active":"2024-06-28T15:00:25.438176+0000","last_became_peered":"2024-06-28T15:00:25.438176+0000","last_unstale":"2024-06-28T16:14:12.999377+0000","last_undegraded":"2024-06-28T16:14:12.999377+0000","last_fullsized":"2024-06-28T16:14:12.999377+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:16:21.793451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":120,"num_read_kb":111,"num_write":277,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"260'74","reported_seq":655,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.255831+0000","last_change":"2024-06-28T15:00:17.403175+0000","last_active":"2024-06-28T16:10:20.255831+0000","last_peered":"2024-06-28T16:10:20.255831+0000","last_clean":"2024-06-28T16:10:20.255831+0000","last_became_active":"2024-06-28T15:00:17.402735+0000","last_became_peered":"2024-06-28T15:00:17.402735+0000","last_unstale":"2024-06-28T16:10:20.255831+0000","last_undegraded":"2024-06-28T16:10:20.255831+0000","last_fullsized":"2024-06-28T16:10:20.255831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:31:42.762083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":299,"num_read_kb":249,"num_write":100,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"260'261","reported_seq":983,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.545545+0000","last_change":"2024-06-28T15:00:15.415658+0000","last_active":"2024-06-28T16:10:15.545545+0000","last_peered":"2024-06-28T16:10:15.545545+0000","last_clean":"2024-06-28T16:10:15.545545+0000","last_became_active":"2024-06-28T15:00:15.415197+0000","last_became_peered":"2024-06-28T15:00:15.415197+0000","last_unstale":"2024-06-28T16:10:15.545545+0000","last_undegraded":"2024-06-28T16:10:15.545545+0000","last_fullsized":"2024-06-28T16:10:15.545545+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:16:16.807609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4058475,"num_objects":80,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":573,"num_read_kb":6210,"num_write":1088,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.084302+0000","last_change":"2024-06-28T15:00:19.379957+0000","last_active":"2024-06-28T16:10:20.084302+0000","last_peered":"2024-06-28T16:10:20.084302+0000","last_clean":"2024-06-28T16:10:20.084302+0000","last_became_active":"2024-06-28T15:00:19.378270+0000","last_became_peered":"2024-06-28T15:00:19.378270+0000","last_unstale":"2024-06-28T16:10:20.084302+0000","last_undegraded":"2024-06-28T16:10:20.084302+0000","last_fullsized":"2024-06-28T16:10:20.084302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:29:00.401820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"260'101","reported_seq":646,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.625893+0000","last_change":"2024-06-28T15:00:21.485155+0000","last_active":"2024-06-28T16:11:03.625893+0000","last_peered":"2024-06-28T16:11:03.625893+0000","last_clean":"2024-06-28T16:11:03.625893+0000","last_became_active":"2024-06-28T15:00:21.484352+0000","last_became_peered":"2024-06-28T15:00:21.484352+0000","last_unstale":"2024-06-28T16:11:03.625893+0000","last_undegraded":"2024-06-28T16:11:03.625893+0000","last_fullsized":"2024-06-28T16:11:03.625893+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:35:15.271971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":124,"num_read_kb":119,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"260'515","reported_seq":2245,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.188813+0000","last_change":"2024-06-28T15:00:23.415013+0000","last_active":"2024-06-28T16:14:32.188813+0000","last_peered":"2024-06-28T16:14:32.188813+0000","last_clean":"2024-06-28T16:14:32.188813+0000","last_became_active":"2024-06-28T15:00:23.407490+0000","last_became_peered":"2024-06-28T15:00:23.407490+0000","last_unstale":"2024-06-28T16:14:32.188813+0000","last_undegraded":"2024-06-28T16:14:32.188813+0000","last_fullsized":"2024-06-28T16:14:32.188813+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:17:37.321606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"260'102","reported_seq":606,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.032920+0000","last_change":"2024-06-28T15:00:25.446074+0000","last_active":"2024-06-28T16:14:06.032920+0000","last_peered":"2024-06-28T16:14:06.032920+0000","last_clean":"2024-06-28T16:14:06.032920+0000","last_became_active":"2024-06-28T15:00:25.445018+0000","last_became_peered":"2024-06-28T15:00:25.445018+0000","last_unstale":"2024-06-28T16:14:06.032920+0000","last_undegraded":"2024-06-28T16:14:06.032920+0000","last_fullsized":"2024-06-28T16:14:06.032920+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:03:17.977593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":117,"num_read_kb":111,"num_write":241,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"260'78","reported_seq":654,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073934+0000","last_change":"2024-06-28T15:00:17.394823+0000","last_active":"2024-06-28T16:10:20.073934+0000","last_peered":"2024-06-28T16:10:20.073934+0000","last_clean":"2024-06-28T16:10:20.073934+0000","last_became_active":"2024-06-28T15:00:17.387086+0000","last_became_peered":"2024-06-28T15:00:17.387086+0000","last_unstale":"2024-06-28T16:10:20.073934+0000","last_undegraded":"2024-06-28T16:10:20.073934+0000","last_fullsized":"2024-06-28T16:10:20.073934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:21:49.516534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":333,"num_read_kb":275,"num_write":113,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"260'328","reported_seq":960,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.721439+0000","last_change":"2024-06-28T15:00:59.943256+0000","last_active":"2024-06-28T16:14:34.721439+0000","last_peered":"2024-06-28T16:14:34.721439+0000","last_clean":"2024-06-28T16:14:34.721439+0000","last_became_active":"2024-06-28T15:00:59.941915+0000","last_became_peered":"2024-06-28T15:00:59.941915+0000","last_unstale":"2024-06-28T16:14:34.721439+0000","last_undegraded":"2024-06-28T16:14:34.721439+0000","last_fullsized":"2024-06-28T16:14:34.721439+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:05:07.672939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5506084,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":497,"num_read_kb":57892,"num_write":1218,"num_write_kb":47404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080513+0000","last_change":"2024-06-28T15:00:19.391494+0000","last_active":"2024-06-28T16:10:20.080513+0000","last_peered":"2024-06-28T16:10:20.080513+0000","last_clean":"2024-06-28T16:10:20.080513+0000","last_became_active":"2024-06-28T15:00:19.390613+0000","last_became_peered":"2024-06-28T15:00:19.390613+0000","last_unstale":"2024-06-28T16:10:20.080513+0000","last_undegraded":"2024-06-28T16:10:20.080513+0000","last_fullsized":"2024-06-28T16:10:20.080513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:27:57.018874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"260'324","reported_seq":932,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.759325+0000","last_change":"2024-06-28T15:00:21.392375+0000","last_active":"2024-06-28T16:14:34.759325+0000","last_peered":"2024-06-28T16:14:34.759325+0000","last_clean":"2024-06-28T16:14:34.759325+0000","last_became_active":"2024-06-28T15:00:21.391631+0000","last_became_peered":"2024-06-28T15:00:21.391631+0000","last_unstale":"2024-06-28T16:14:34.759325+0000","last_undegraded":"2024-06-28T16:14:34.759325+0000","last_fullsized":"2024-06-28T16:14:34.759325+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:25:50.181425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43414,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":516,"num_read_kb":1105,"num_write":400,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.549103+0000","last_change":"2024-06-28T15:00:23.405246+0000","last_active":"2024-06-28T16:10:30.549103+0000","last_peered":"2024-06-28T16:10:30.549103+0000","last_clean":"2024-06-28T16:10:30.549103+0000","last_became_active":"2024-06-28T15:00:23.404312+0000","last_became_peered":"2024-06-28T15:00:23.404312+0000","last_unstale":"2024-06-28T16:10:30.549103+0000","last_undegraded":"2024-06-28T16:10:30.549103+0000","last_fullsized":"2024-06-28T16:10:30.549103+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:40.584815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"260'131","reported_seq":642,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.235400+0000","last_change":"2024-06-28T15:00:25.425886+0000","last_active":"2024-06-28T16:14:08.235400+0000","last_peered":"2024-06-28T16:14:08.235400+0000","last_clean":"2024-06-28T16:14:08.235400+0000","last_became_active":"2024-06-28T15:00:25.424826+0000","last_became_peered":"2024-06-28T15:00:25.424826+0000","last_unstale":"2024-06-28T16:14:08.235400+0000","last_undegraded":"2024-06-28T16:14:08.235400+0000","last_fullsized":"2024-06-28T16:14:08.235400+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:56:01.501372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":159,"num_read_kb":144,"num_write":309,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"260'118","reported_seq":691,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073868+0000","last_change":"2024-06-28T15:00:17.387490+0000","last_active":"2024-06-28T16:10:20.073868+0000","last_peered":"2024-06-28T16:10:20.073868+0000","last_clean":"2024-06-28T16:10:20.073868+0000","last_became_active":"2024-06-28T15:00:17.386756+0000","last_became_peered":"2024-06-28T15:00:17.386756+0000","last_unstale":"2024-06-28T16:10:20.073868+0000","last_undegraded":"2024-06-28T16:10:20.073868+0000","last_fullsized":"2024-06-28T16:10:20.073868+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:24:23.283000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":469,"num_read_kb":378,"num_write":166,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"260'293","reported_seq":964,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.754554+0000","last_change":"2024-06-28T15:00:59.945598+0000","last_active":"2024-06-28T16:14:34.754554+0000","last_peered":"2024-06-28T16:14:34.754554+0000","last_clean":"2024-06-28T16:14:34.754554+0000","last_became_active":"2024-06-28T15:00:59.943670+0000","last_became_peered":"2024-06-28T15:00:59.943670+0000","last_unstale":"2024-06-28T16:14:34.754554+0000","last_undegraded":"2024-06-28T16:14:34.754554+0000","last_fullsized":"2024-06-28T16:14:34.754554+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:28:33.955349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306233,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":513,"num_read_kb":66858,"num_write":1091,"num_write_kb":26064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.066682+0000","last_change":"2024-06-28T15:00:19.400037+0000","last_active":"2024-06-28T16:10:20.066682+0000","last_peered":"2024-06-28T16:10:20.066682+0000","last_clean":"2024-06-28T16:10:20.066682+0000","last_became_active":"2024-06-28T15:00:19.398245+0000","last_became_peered":"2024-06-28T15:00:19.398245+0000","last_unstale":"2024-06-28T16:10:20.066682+0000","last_undegraded":"2024-06-28T16:10:20.066682+0000","last_fullsized":"2024-06-28T16:10:20.066682+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:50.601079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"260'292","reported_seq":877,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.720679+0000","last_change":"2024-06-28T15:00:21.406496+0000","last_active":"2024-06-28T16:14:06.720679+0000","last_peered":"2024-06-28T16:14:06.720679+0000","last_clean":"2024-06-28T16:14:06.720679+0000","last_became_active":"2024-06-28T15:00:21.404701+0000","last_became_peered":"2024-06-28T15:00:21.404701+0000","last_unstale":"2024-06-28T16:14:06.720679+0000","last_undegraded":"2024-06-28T16:14:06.720679+0000","last_fullsized":"2024-06-28T16:14:06.720679+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":200,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:35:29.922369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59181,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":473,"num_read_kb":1000,"num_write":388,"num_write_kb":303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":491,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:50.079705+0000","last_change":"2024-06-28T15:00:23.416260+0000","last_active":"2024-06-28T16:10:50.079705+0000","last_peered":"2024-06-28T16:10:50.079705+0000","last_clean":"2024-06-28T16:10:50.079705+0000","last_became_active":"2024-06-28T15:00:23.415531+0000","last_became_peered":"2024-06-28T15:00:23.415531+0000","last_unstale":"2024-06-28T16:10:50.079705+0000","last_undegraded":"2024-06-28T16:10:50.079705+0000","last_fullsized":"2024-06-28T16:10:50.079705+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:14:35.301127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"260'166","reported_seq":710,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:02.116164+0000","last_change":"2024-06-28T15:00:25.554319+0000","last_active":"2024-06-28T16:13:02.116164+0000","last_peered":"2024-06-28T16:13:02.116164+0000","last_clean":"2024-06-28T16:13:02.116164+0000","last_became_active":"2024-06-28T15:00:25.553724+0000","last_became_peered":"2024-06-28T15:00:25.553724+0000","last_unstale":"2024-06-28T16:13:02.116164+0000","last_undegraded":"2024-06-28T16:13:02.116164+0000","last_fullsized":"2024-06-28T16:13:02.116164+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:16:48.260247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":555,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":217,"num_write":397,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"260'143","reported_seq":806,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.494648+0000","last_change":"2024-06-28T15:00:17.383713+0000","last_active":"2024-06-28T16:10:30.494648+0000","last_peered":"2024-06-28T16:10:30.494648+0000","last_clean":"2024-06-28T16:10:30.494648+0000","last_became_active":"2024-06-28T15:00:17.383224+0000","last_became_peered":"2024-06-28T15:00:17.383224+0000","last_unstale":"2024-06-28T16:10:30.494648+0000","last_undegraded":"2024-06-28T16:10:30.494648+0000","last_fullsized":"2024-06-28T16:10:30.494648+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:23:29.150410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":649,"num_read_kb":545,"num_write":214,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"260'301","reported_seq":1066,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.691517+0000","last_change":"2024-06-28T15:00:15.424150+0000","last_active":"2024-06-28T16:14:34.691517+0000","last_peered":"2024-06-28T16:14:34.691517+0000","last_clean":"2024-06-28T16:14:34.691517+0000","last_became_active":"2024-06-28T15:00:15.423813+0000","last_became_peered":"2024-06-28T15:00:15.423813+0000","last_unstale":"2024-06-28T16:14:34.691517+0000","last_undegraded":"2024-06-28T16:14:34.691517+0000","last_fullsized":"2024-06-28T16:14:34.691517+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:57:01.721732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5724260,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":754,"num_read_kb":37474,"num_write":1107,"num_write_kb":31583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.075092+0000","last_change":"2024-06-28T15:00:19.384145+0000","last_active":"2024-06-28T16:10:25.075092+0000","last_peered":"2024-06-28T16:10:25.075092+0000","last_clean":"2024-06-28T16:10:25.075092+0000","last_became_active":"2024-06-28T15:00:19.383307+0000","last_became_peered":"2024-06-28T15:00:19.383307+0000","last_unstale":"2024-06-28T16:10:25.075092+0000","last_undegraded":"2024-06-28T16:10:25.075092+0000","last_fullsized":"2024-06-28T16:10:25.075092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:22:34.590726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"260'286","reported_seq":857,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.176534+0000","last_change":"2024-06-28T15:00:21.396994+0000","last_active":"2024-06-28T16:14:34.176534+0000","last_peered":"2024-06-28T16:14:34.176534+0000","last_clean":"2024-06-28T16:14:34.176534+0000","last_became_active":"2024-06-28T15:00:21.396257+0000","last_became_peered":"2024-06-28T15:00:21.396257+0000","last_unstale":"2024-06-28T16:14:34.176534+0000","last_undegraded":"2024-06-28T16:14:34.176534+0000","last_fullsized":"2024-06-28T16:14:34.176534+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:05:02.532173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43992,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":439,"num_read_kb":1078,"num_write":366,"num_write_kb":297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":494,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.075301+0000","last_change":"2024-06-28T15:00:23.410088+0000","last_active":"2024-06-28T16:10:25.075301+0000","last_peered":"2024-06-28T16:10:25.075301+0000","last_clean":"2024-06-28T16:10:25.075301+0000","last_became_active":"2024-06-28T15:00:23.409107+0000","last_became_peered":"2024-06-28T15:00:23.409107+0000","last_unstale":"2024-06-28T16:10:25.075301+0000","last_undegraded":"2024-06-28T16:10:25.075301+0000","last_fullsized":"2024-06-28T16:10:25.075301+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:26:16.658429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"260'144","reported_seq":876,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:02.732078+0000","last_change":"2024-06-28T15:00:25.425176+0000","last_active":"2024-06-28T16:14:02.732078+0000","last_peered":"2024-06-28T16:14:02.732078+0000","last_clean":"2024-06-28T16:14:02.732078+0000","last_became_active":"2024-06-28T15:00:25.424376+0000","last_became_peered":"2024-06-28T15:00:25.424376+0000","last_unstale":"2024-06-28T16:14:02.732078+0000","last_undegraded":"2024-06-28T16:14:02.732078+0000","last_fullsized":"2024-06-28T16:14:02.732078+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:40:22.841457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":398,"num_read_kb":429,"num_write":339,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"260'112","reported_seq":699,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.075140+0000","last_change":"2024-06-28T15:00:17.403746+0000","last_active":"2024-06-28T16:10:25.075140+0000","last_peered":"2024-06-28T16:10:25.075140+0000","last_clean":"2024-06-28T16:10:25.075140+0000","last_became_active":"2024-06-28T15:00:17.403007+0000","last_became_peered":"2024-06-28T15:00:17.403007+0000","last_unstale":"2024-06-28T16:10:25.075140+0000","last_undegraded":"2024-06-28T16:10:25.075140+0000","last_fullsized":"2024-06-28T16:10:25.075140+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:10:54.255627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":478,"num_read_kb":386,"num_write":182,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"260'311","reported_seq":978,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.556341+0000","last_change":"2024-06-28T15:00:15.415976+0000","last_active":"2024-06-28T16:11:05.556341+0000","last_peered":"2024-06-28T16:11:05.556341+0000","last_clean":"2024-06-28T16:11:05.556341+0000","last_became_active":"2024-06-28T15:00:15.414969+0000","last_became_peered":"2024-06-28T15:00:15.414969+0000","last_unstale":"2024-06-28T16:11:05.556341+0000","last_undegraded":"2024-06-28T16:11:05.556341+0000","last_fullsized":"2024-06-28T16:11:05.556341+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":300,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:03:09.425476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6850286,"num_objects":77,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":471,"num_read_kb":3182,"num_write":1253,"num_write_kb":13350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"24'1","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:45.088935+0000","last_change":"2024-06-28T15:00:19.403928+0000","last_active":"2024-06-28T16:10:45.088935+0000","last_peered":"2024-06-28T16:10:45.088935+0000","last_clean":"2024-06-28T16:10:45.088935+0000","last_became_active":"2024-06-28T15:00:19.403344+0000","last_became_peered":"2024-06-28T15:00:19.403344+0000","last_unstale":"2024-06-28T16:10:45.088935+0000","last_undegraded":"2024-06-28T16:10:45.088935+0000","last_fullsized":"2024-06-28T16:10:45.088935+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:09:50.417696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"260'293","reported_seq":893,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.393958+0000","last_change":"2024-06-28T15:00:21.388828+0000","last_active":"2024-06-28T16:14:34.393958+0000","last_peered":"2024-06-28T16:14:34.393958+0000","last_clean":"2024-06-28T16:14:34.393958+0000","last_became_active":"2024-06-28T15:00:21.387539+0000","last_became_peered":"2024-06-28T15:00:21.387539+0000","last_unstale":"2024-06-28T16:14:34.393958+0000","last_undegraded":"2024-06-28T16:14:34.393958+0000","last_fullsized":"2024-06-28T16:14:34.393958+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:54:05.779834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44825,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":494,"num_read_kb":1086,"num_write":388,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":499,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.548062+0000","last_change":"2024-06-28T15:00:23.417454+0000","last_active":"2024-06-28T16:10:25.548062+0000","last_peered":"2024-06-28T16:10:25.548062+0000","last_clean":"2024-06-28T16:10:25.548062+0000","last_became_active":"2024-06-28T15:00:23.416363+0000","last_became_peered":"2024-06-28T15:00:23.416363+0000","last_unstale":"2024-06-28T16:10:25.548062+0000","last_undegraded":"2024-06-28T16:10:25.548062+0000","last_fullsized":"2024-06-28T16:10:25.548062+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:42:57.225974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"260'126","reported_seq":623,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:05.111348+0000","last_change":"2024-06-28T15:00:25.429682+0000","last_active":"2024-06-28T16:13:05.111348+0000","last_peered":"2024-06-28T16:13:05.111348+0000","last_clean":"2024-06-28T16:13:05.111348+0000","last_became_active":"2024-06-28T15:00:25.428959+0000","last_became_peered":"2024-06-28T15:00:25.428959+0000","last_unstale":"2024-06-28T16:13:05.111348+0000","last_undegraded":"2024-06-28T16:13:05.111348+0000","last_fullsized":"2024-06-28T16:13:05.111348+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:34:41.858483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":814,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":97,"num_write":299,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"260'174","reported_seq":761,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:39.929106+0000","last_change":"2024-06-28T15:00:17.407419+0000","last_active":"2024-06-28T16:13:39.929106+0000","last_peered":"2024-06-28T16:13:39.929106+0000","last_clean":"2024-06-28T16:13:39.929106+0000","last_became_active":"2024-06-28T15:00:17.406872+0000","last_became_peered":"2024-06-28T15:00:17.406872+0000","last_unstale":"2024-06-28T16:13:39.929106+0000","last_undegraded":"2024-06-28T16:13:39.929106+0000","last_fullsized":"2024-06-28T16:13:39.929106+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:52:17.662914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":660,"num_read_kb":511,"num_write":250,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"260'285","reported_seq":909,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.644073+0000","last_change":"2024-06-28T15:00:15.409170+0000","last_active":"2024-06-28T16:14:34.644073+0000","last_peered":"2024-06-28T16:14:34.644073+0000","last_clean":"2024-06-28T16:14:34.644073+0000","last_became_active":"2024-06-28T15:00:15.408337+0000","last_became_peered":"2024-06-28T15:00:15.408337+0000","last_unstale":"2024-06-28T16:14:34.644073+0000","last_undegraded":"2024-06-28T16:14:34.644073+0000","last_fullsized":"2024-06-28T16:14:34.644073+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:51:23.373884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":404,"num_read_kb":33775,"num_write":1000,"num_write_kb":10945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"260'72","reported_seq":674,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.102569+0000","last_change":"2024-06-28T15:00:17.404713+0000","last_active":"2024-06-28T16:11:50.102569+0000","last_peered":"2024-06-28T16:11:50.102569+0000","last_clean":"2024-06-28T16:11:50.102569+0000","last_became_active":"2024-06-28T15:00:17.403938+0000","last_became_peered":"2024-06-28T15:00:17.403938+0000","last_unstale":"2024-06-28T16:11:50.102569+0000","last_undegraded":"2024-06-28T16:11:50.102569+0000","last_fullsized":"2024-06-28T16:11:50.102569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:54:35.966683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":319,"num_read_kb":277,"num_write":96,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"260'322","reported_seq":995,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.576974+0000","last_change":"2024-06-28T15:00:15.420697+0000","last_active":"2024-06-28T16:14:34.576974+0000","last_peered":"2024-06-28T16:14:34.576974+0000","last_clean":"2024-06-28T16:14:34.576974+0000","last_became_active":"2024-06-28T15:00:15.420261+0000","last_became_peered":"2024-06-28T15:00:15.420261+0000","last_unstale":"2024-06-28T16:14:34.576974+0000","last_undegraded":"2024-06-28T16:14:34.576974+0000","last_fullsized":"2024-06-28T16:14:34.576974+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:57:26.418017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262189,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":529,"num_read_kb":23662,"num_write":1259,"num_write_kb":28174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"260'38","reported_seq":596,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.087084+0000","last_change":"2024-06-28T15:00:17.394654+0000","last_active":"2024-06-28T16:11:50.087084+0000","last_peered":"2024-06-28T16:11:50.087084+0000","last_clean":"2024-06-28T16:11:50.087084+0000","last_became_active":"2024-06-28T15:00:17.393889+0000","last_became_peered":"2024-06-28T15:00:17.393889+0000","last_unstale":"2024-06-28T16:11:50.087084+0000","last_undegraded":"2024-06-28T16:11:50.087084+0000","last_fullsized":"2024-06-28T16:11:50.087084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:22:39.330270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":152,"num_write":58,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"260'358","reported_seq":999,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.728265+0000","last_change":"2024-06-28T15:00:15.426603+0000","last_active":"2024-06-28T16:14:34.728265+0000","last_peered":"2024-06-28T16:14:34.728265+0000","last_clean":"2024-06-28T16:14:34.728265+0000","last_became_active":"2024-06-28T15:00:15.418773+0000","last_became_peered":"2024-06-28T15:00:15.418773+0000","last_unstale":"2024-06-28T16:14:34.728265+0000","last_undegraded":"2024-06-28T16:14:34.728265+0000","last_fullsized":"2024-06-28T16:14:34.728265+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:02:14.247776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8714448,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":553,"num_read_kb":16492,"num_write":1257,"num_write_kb":37641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"260'142","reported_seq":787,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:07.183113+0000","last_change":"2024-06-28T15:00:17.398608+0000","last_active":"2024-06-28T16:13:07.183113+0000","last_peered":"2024-06-28T16:13:07.183113+0000","last_clean":"2024-06-28T16:13:07.183113+0000","last_became_active":"2024-06-28T15:00:17.398357+0000","last_became_peered":"2024-06-28T15:00:17.398357+0000","last_unstale":"2024-06-28T16:13:07.183113+0000","last_undegraded":"2024-06-28T16:13:07.183113+0000","last_fullsized":"2024-06-28T16:13:07.183113+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:53:58.893939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":623,"num_read_kb":529,"num_write":221,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"260'269","reported_seq":913,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:09.848378+0000","last_change":"2024-06-28T15:00:15.391599+0000","last_active":"2024-06-28T16:12:09.848378+0000","last_peered":"2024-06-28T16:12:09.848378+0000","last_clean":"2024-06-28T16:12:09.848378+0000","last_became_active":"2024-06-28T15:00:15.391150+0000","last_became_peered":"2024-06-28T15:00:15.391150+0000","last_unstale":"2024-06-28T16:12:09.848378+0000","last_undegraded":"2024-06-28T16:12:09.848378+0000","last_fullsized":"2024-06-28T16:12:09.848378+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":200,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:12:30.284233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6905440,"num_objects":72,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":390,"num_read_kb":84462,"num_write":1202,"num_write_kb":41774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"260'191","reported_seq":788,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:05.276690+0000","last_change":"2024-06-28T15:00:59.807172+0000","last_active":"2024-06-28T16:12:05.276690+0000","last_peered":"2024-06-28T16:12:05.276690+0000","last_clean":"2024-06-28T16:12:05.276690+0000","last_became_active":"2024-06-28T15:00:59.806725+0000","last_became_peered":"2024-06-28T15:00:59.806725+0000","last_unstale":"2024-06-28T16:12:05.276690+0000","last_undegraded":"2024-06-28T16:12:05.276690+0000","last_fullsized":"2024-06-28T16:12:05.276690+0000","mapping_epoch":32,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:00:01.418110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":759,"num_read_kb":602,"num_write":307,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"260'334","reported_seq":998,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.618192+0000","last_change":"2024-06-28T15:00:15.383729+0000","last_active":"2024-06-28T16:14:34.618192+0000","last_peered":"2024-06-28T16:14:34.618192+0000","last_clean":"2024-06-28T16:14:34.618192+0000","last_became_active":"2024-06-28T15:00:15.383350+0000","last_became_peered":"2024-06-28T15:00:15.383350+0000","last_unstale":"2024-06-28T16:14:34.618192+0000","last_undegraded":"2024-06-28T16:14:34.618192+0000","last_fullsized":"2024-06-28T16:14:34.618192+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:22:33.747727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263219,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":502,"num_read_kb":41336,"num_write":1201,"num_write_kb":38478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0,7],"acting":[3,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"260'397","reported_seq":997,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.085126+0000","last_change":"2024-06-28T15:00:17.408600+0000","last_active":"2024-06-28T16:11:10.085126+0000","last_peered":"2024-06-28T16:11:10.085126+0000","last_clean":"2024-06-28T16:11:10.085126+0000","last_became_active":"2024-06-28T15:00:17.404844+0000","last_became_peered":"2024-06-28T15:00:17.404844+0000","last_unstale":"2024-06-28T16:11:10.085126+0000","last_undegraded":"2024-06-28T16:11:10.085126+0000","last_fullsized":"2024-06-28T16:11:10.085126+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":300,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:17:34.507155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1527,"num_read_kb":1156,"num_write":658,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"260'343","reported_seq":1027,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.657138+0000","last_change":"2024-06-28T15:00:15.401905+0000","last_active":"2024-06-28T16:14:34.657138+0000","last_peered":"2024-06-28T16:14:34.657138+0000","last_clean":"2024-06-28T16:14:34.657138+0000","last_became_active":"2024-06-28T15:00:15.401275+0000","last_became_peered":"2024-06-28T15:00:15.401275+0000","last_unstale":"2024-06-28T16:14:34.657138+0000","last_undegraded":"2024-06-28T16:14:34.657138+0000","last_fullsized":"2024-06-28T16:14:34.657138+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:00:05.589080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":559,"num_read_kb":45669,"num_write":1313,"num_write_kb":19663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"260'221","reported_seq":841,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.504084+0000","last_change":"2024-06-28T15:00:17.383279+0000","last_active":"2024-06-28T16:11:10.504084+0000","last_peered":"2024-06-28T16:11:10.504084+0000","last_clean":"2024-06-28T16:11:10.504084+0000","last_became_active":"2024-06-28T15:00:17.382739+0000","last_became_peered":"2024-06-28T15:00:17.382739+0000","last_unstale":"2024-06-28T16:11:10.504084+0000","last_undegraded":"2024-06-28T16:11:10.504084+0000","last_fullsized":"2024-06-28T16:11:10.504084+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:36:13.087157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":866,"num_read_kb":690,"num_write":335,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"260'298","reported_seq":928,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.600936+0000","last_change":"2024-06-28T15:00:15.421880+0000","last_active":"2024-06-28T16:14:34.600936+0000","last_peered":"2024-06-28T16:14:34.600936+0000","last_clean":"2024-06-28T16:14:34.600936+0000","last_became_active":"2024-06-28T15:00:15.421189+0000","last_became_peered":"2024-06-28T15:00:15.421189+0000","last_unstale":"2024-06-28T16:14:34.600936+0000","last_undegraded":"2024-06-28T16:14:34.600936+0000","last_fullsized":"2024-06-28T16:14:34.600936+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:03:21.461240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7105903,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":419,"num_read_kb":19450,"num_write":1148,"num_write_kb":38199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"260'79","reported_seq":639,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.083923+0000","last_change":"2024-06-28T15:00:17.396892+0000","last_active":"2024-06-28T16:11:10.083923+0000","last_peered":"2024-06-28T16:11:10.083923+0000","last_clean":"2024-06-28T16:11:10.083923+0000","last_became_active":"2024-06-28T15:00:17.396125+0000","last_became_peered":"2024-06-28T15:00:17.396125+0000","last_unstale":"2024-06-28T16:11:10.083923+0000","last_undegraded":"2024-06-28T16:11:10.083923+0000","last_fullsized":"2024-06-28T16:11:10.083923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:46:54.077428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":321,"num_read_kb":257,"num_write":118,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"260'270","reported_seq":927,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.735427+0000","last_change":"2024-06-28T15:00:15.408891+0000","last_active":"2024-06-28T16:14:34.735427+0000","last_peered":"2024-06-28T16:14:34.735427+0000","last_clean":"2024-06-28T16:14:34.735427+0000","last_became_active":"2024-06-28T15:00:15.408375+0000","last_became_peered":"2024-06-28T15:00:15.408375+0000","last_unstale":"2024-06-28T16:14:34.735427+0000","last_undegraded":"2024-06-28T16:14:34.735427+0000","last_fullsized":"2024-06-28T16:14:34.735427+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:29:43.369904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":484,"num_read_kb":6587,"num_write":1005,"num_write_kb":17135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"260'62","reported_seq":636,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:08.869032+0000","last_change":"2024-06-28T15:00:17.380077+0000","last_active":"2024-06-28T16:12:08.869032+0000","last_peered":"2024-06-28T16:12:08.869032+0000","last_clean":"2024-06-28T16:12:08.869032+0000","last_became_active":"2024-06-28T15:00:17.379374+0000","last_became_peered":"2024-06-28T15:00:17.379374+0000","last_unstale":"2024-06-28T16:12:08.869032+0000","last_undegraded":"2024-06-28T16:12:08.869032+0000","last_fullsized":"2024-06-28T16:12:08.869032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:19:27.611751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":216,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"260'367","reported_seq":1011,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.749490+0000","last_change":"2024-06-28T15:00:15.427903+0000","last_active":"2024-06-28T16:14:34.749490+0000","last_peered":"2024-06-28T16:14:34.749490+0000","last_clean":"2024-06-28T16:14:34.749490+0000","last_became_active":"2024-06-28T15:00:15.427171+0000","last_became_peered":"2024-06-28T15:00:15.427171+0000","last_unstale":"2024-06-28T16:14:34.749490+0000","last_undegraded":"2024-06-28T16:14:34.749490+0000","last_fullsized":"2024-06-28T16:14:34.749490+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":300,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:22:01.482231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":723921,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":536,"num_read_kb":26554,"num_write":1152,"num_write_kb":52254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"260'230","reported_seq":851,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.074074+0000","last_change":"2024-06-28T15:00:17.409608+0000","last_active":"2024-06-28T16:10:20.074074+0000","last_peered":"2024-06-28T16:10:20.074074+0000","last_clean":"2024-06-28T16:10:20.074074+0000","last_became_active":"2024-06-28T15:00:17.403760+0000","last_became_peered":"2024-06-28T15:00:17.403760+0000","last_unstale":"2024-06-28T16:10:20.074074+0000","last_undegraded":"2024-06-28T16:10:20.074074+0000","last_fullsized":"2024-06-28T16:10:20.074074+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:58:04.722934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":902,"num_read_kb":724,"num_write":338,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"260'282","reported_seq":929,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.669616+0000","last_change":"2024-06-28T15:00:15.376797+0000","last_active":"2024-06-28T16:14:34.669616+0000","last_peered":"2024-06-28T16:14:34.669616+0000","last_clean":"2024-06-28T16:14:34.669616+0000","last_became_active":"2024-06-28T15:00:15.376014+0000","last_became_peered":"2024-06-28T15:00:15.376014+0000","last_unstale":"2024-06-28T16:14:34.669616+0000","last_undegraded":"2024-06-28T16:14:34.669616+0000","last_fullsized":"2024-06-28T16:14:34.669616+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:31:24.117797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462789,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":409,"num_read_kb":32504,"num_write":910,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"260'70","reported_seq":658,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.255871+0000","last_change":"2024-06-28T15:00:17.402274+0000","last_active":"2024-06-28T16:10:20.255871+0000","last_peered":"2024-06-28T16:10:20.255871+0000","last_clean":"2024-06-28T16:10:20.255871+0000","last_became_active":"2024-06-28T15:00:17.401917+0000","last_became_peered":"2024-06-28T15:00:17.401917+0000","last_unstale":"2024-06-28T16:10:20.255871+0000","last_undegraded":"2024-06-28T16:10:20.255871+0000","last_fullsized":"2024-06-28T16:10:20.255871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:48:15.974353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":295,"num_read_kb":247,"num_write":98,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"260'337","reported_seq":973,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.694565+0000","last_change":"2024-06-28T15:00:15.434365+0000","last_active":"2024-06-28T16:14:34.694565+0000","last_peered":"2024-06-28T16:14:34.694565+0000","last_clean":"2024-06-28T16:14:34.694565+0000","last_became_active":"2024-06-28T15:00:15.431444+0000","last_became_peered":"2024-06-28T15:00:15.431444+0000","last_unstale":"2024-06-28T16:14:34.694565+0000","last_undegraded":"2024-06-28T16:14:34.694565+0000","last_fullsized":"2024-06-28T16:14:34.694565+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:01:51.800790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":461,"num_read_kb":557,"num_write":1266,"num_write_kb":33580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,3],"acting":[5,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"260'424","reported_seq":1178,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101339+0000","last_change":"2024-06-28T15:00:17.403422+0000","last_active":"2024-06-28T16:12:20.101339+0000","last_peered":"2024-06-28T16:12:20.101339+0000","last_clean":"2024-06-28T16:12:20.101339+0000","last_became_active":"2024-06-28T15:00:17.402842+0000","last_became_peered":"2024-06-28T15:00:17.402842+0000","last_unstale":"2024-06-28T16:12:20.101339+0000","last_undegraded":"2024-06-28T16:12:20.101339+0000","last_fullsized":"2024-06-28T16:12:20.101339+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":400,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:26:05.866773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1825,"num_read_kb":1465,"num_write":755,"num_write_kb":468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"260'261","reported_seq":906,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.574001+0000","last_change":"2024-06-28T15:00:15.415547+0000","last_active":"2024-06-28T16:12:15.574001+0000","last_peered":"2024-06-28T16:12:15.574001+0000","last_clean":"2024-06-28T16:12:15.574001+0000","last_became_active":"2024-06-28T15:00:15.415009+0000","last_became_peered":"2024-06-28T15:00:15.415009+0000","last_unstale":"2024-06-28T16:12:15.574001+0000","last_undegraded":"2024-06-28T16:12:15.574001+0000","last_fullsized":"2024-06-28T16:12:15.574001+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":200,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:36:33.533042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5245699,"num_objects":64,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":421,"num_read_kb":2084,"num_write":1144,"num_write_kb":29565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"260'26","reported_seq":574,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.108218+0000","last_change":"2024-06-28T15:00:17.390583+0000","last_active":"2024-06-28T16:12:20.108218+0000","last_peered":"2024-06-28T16:12:20.108218+0000","last_clean":"2024-06-28T16:12:20.108218+0000","last_became_active":"2024-06-28T15:00:17.389844+0000","last_became_peered":"2024-06-28T15:00:17.389844+0000","last_unstale":"2024-06-28T16:12:20.108218+0000","last_undegraded":"2024-06-28T16:12:20.108218+0000","last_fullsized":"2024-06-28T16:12:20.108218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:41:17.401424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":112,"num_write":34,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"260'305","reported_seq":936,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.719398+0000","last_change":"2024-06-28T15:00:15.398948+0000","last_active":"2024-06-28T16:14:34.719398+0000","last_peered":"2024-06-28T16:14:34.719398+0000","last_clean":"2024-06-28T16:14:34.719398+0000","last_became_active":"2024-06-28T15:00:15.397986+0000","last_became_peered":"2024-06-28T15:00:15.397986+0000","last_unstale":"2024-06-28T16:14:34.719398+0000","last_undegraded":"2024-06-28T16:14:34.719398+0000","last_fullsized":"2024-06-28T16:14:34.719398+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:02:19.841719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079926,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":433,"num_read_kb":10373,"num_write":1047,"num_write_kb":25421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"260'192","reported_seq":856,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.822857+0000","last_change":"2024-06-28T15:00:17.395455+0000","last_active":"2024-06-28T16:12:20.822857+0000","last_peered":"2024-06-28T16:12:20.822857+0000","last_clean":"2024-06-28T16:12:20.822857+0000","last_became_active":"2024-06-28T15:00:17.394783+0000","last_became_peered":"2024-06-28T15:00:17.394783+0000","last_unstale":"2024-06-28T16:12:20.822857+0000","last_undegraded":"2024-06-28T16:12:20.822857+0000","last_fullsized":"2024-06-28T16:12:20.822857+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:54:50.516422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":852,"num_read_kb":715,"num_write":335,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"260'302","reported_seq":913,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.681222+0000","last_change":"2024-06-28T15:00:59.939157+0000","last_active":"2024-06-28T16:14:34.681222+0000","last_peered":"2024-06-28T16:14:34.681222+0000","last_clean":"2024-06-28T16:14:34.681222+0000","last_became_active":"2024-06-28T15:00:59.938004+0000","last_became_peered":"2024-06-28T15:00:59.938004+0000","last_unstale":"2024-06-28T16:14:34.681222+0000","last_undegraded":"2024-06-28T16:14:34.681222+0000","last_fullsized":"2024-06-28T16:14:34.681222+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:35:45.871383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6542288,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":437,"num_read_kb":25077,"num_write":1246,"num_write_kb":38037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,3],"acting":[1,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"260'392","reported_seq":1062,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.520627+0000","last_change":"2024-06-28T15:00:17.374446+0000","last_active":"2024-06-28T16:12:20.520627+0000","last_peered":"2024-06-28T16:12:20.520627+0000","last_clean":"2024-06-28T16:12:20.520627+0000","last_became_active":"2024-06-28T15:00:17.374143+0000","last_became_peered":"2024-06-28T15:00:17.374143+0000","last_unstale":"2024-06-28T16:12:20.520627+0000","last_undegraded":"2024-06-28T16:12:20.520627+0000","last_fullsized":"2024-06-28T16:12:20.520627+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":300,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:31:56.644836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1520,"num_read_kb":1191,"num_write":653,"num_write_kb":382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"260'255","reported_seq":918,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.656935+0000","last_change":"2024-06-28T15:00:15.430833+0000","last_active":"2024-06-28T16:14:34.656935+0000","last_peered":"2024-06-28T16:14:34.656935+0000","last_clean":"2024-06-28T16:14:34.656935+0000","last_became_active":"2024-06-28T15:00:15.430228+0000","last_became_peered":"2024-06-28T15:00:15.430228+0000","last_unstale":"2024-06-28T16:14:34.656935+0000","last_undegraded":"2024-06-28T16:14:34.656935+0000","last_fullsized":"2024-06-28T16:14:34.656935+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:20:15.238678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170399,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":449,"num_read_kb":6008,"num_write":895,"num_write_kb":11243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"260'134","reported_seq":778,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:05.104760+0000","last_change":"2024-06-28T15:00:17.382276+0000","last_active":"2024-06-28T16:12:05.104760+0000","last_peered":"2024-06-28T16:12:05.104760+0000","last_clean":"2024-06-28T16:12:05.104760+0000","last_became_active":"2024-06-28T15:00:17.381562+0000","last_became_peered":"2024-06-28T15:00:17.381562+0000","last_unstale":"2024-06-28T16:12:05.104760+0000","last_undegraded":"2024-06-28T16:12:05.104760+0000","last_fullsized":"2024-06-28T16:12:05.104760+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:32:13.816417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":479,"num_write":190,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"260'496","reported_seq":1185,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.737064+0000","last_change":"2024-06-28T15:00:15.419485+0000","last_active":"2024-06-28T16:14:34.737064+0000","last_peered":"2024-06-28T16:14:34.737064+0000","last_clean":"2024-06-28T16:14:34.737064+0000","last_became_active":"2024-06-28T15:00:15.418850+0000","last_became_peered":"2024-06-28T15:00:15.418850+0000","last_unstale":"2024-06-28T16:14:34.737064+0000","last_undegraded":"2024-06-28T16:14:34.737064+0000","last_fullsized":"2024-06-28T16:14:34.737064+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":400,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:53:16.536329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":907,"num_read_kb":23220,"num_write":1415,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"260'338","reported_seq":1022,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.749225+0000","last_change":"2024-06-28T15:00:15.409457+0000","last_active":"2024-06-28T16:14:34.749225+0000","last_peered":"2024-06-28T16:14:34.749225+0000","last_clean":"2024-06-28T16:14:34.749225+0000","last_became_active":"2024-06-28T15:00:15.408158+0000","last_became_peered":"2024-06-28T15:00:15.408158+0000","last_unstale":"2024-06-28T16:14:34.749225+0000","last_undegraded":"2024-06-28T16:14:34.749225+0000","last_fullsized":"2024-06-28T16:14:34.749225+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:01:20.858547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6690738,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":568,"num_read_kb":38558,"num_write":1259,"num_write_kb":46809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"260'77","reported_seq":645,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:40.551311+0000","last_change":"2024-06-28T15:00:17.380351+0000","last_active":"2024-06-28T16:10:40.551311+0000","last_peered":"2024-06-28T16:10:40.551311+0000","last_clean":"2024-06-28T16:10:40.551311+0000","last_became_active":"2024-06-28T15:00:17.379771+0000","last_became_peered":"2024-06-28T15:00:17.379771+0000","last_unstale":"2024-06-28T16:10:40.551311+0000","last_undegraded":"2024-06-28T16:10:40.551311+0000","last_fullsized":"2024-06-28T16:10:40.551311+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:19:40.601549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":249,"num_write":109,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"260'387","reported_seq":1101,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.735487+0000","last_change":"2024-06-28T15:00:15.404986+0000","last_active":"2024-06-28T16:14:34.735487+0000","last_peered":"2024-06-28T16:14:34.735487+0000","last_clean":"2024-06-28T16:14:34.735487+0000","last_became_active":"2024-06-28T15:00:15.401324+0000","last_became_peered":"2024-06-28T15:00:15.401324+0000","last_unstale":"2024-06-28T16:14:34.735487+0000","last_undegraded":"2024-06-28T16:14:34.735487+0000","last_fullsized":"2024-06-28T16:14:34.735487+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":300,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:15:20.980772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079973,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":661,"num_read_kb":1946,"num_write":1429,"num_write_kb":27589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"260'176","reported_seq":777,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:14.583638+0000","last_change":"2024-06-28T15:00:17.383615+0000","last_active":"2024-06-28T16:13:14.583638+0000","last_peered":"2024-06-28T16:13:14.583638+0000","last_clean":"2024-06-28T16:13:14.583638+0000","last_became_active":"2024-06-28T15:00:17.382774+0000","last_became_peered":"2024-06-28T15:00:17.382774+0000","last_unstale":"2024-06-28T16:13:14.583638+0000","last_undegraded":"2024-06-28T16:13:14.583638+0000","last_fullsized":"2024-06-28T16:13:14.583638+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:39:45.866222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":719,"num_read_kb":570,"num_write":272,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5652,"num_read_kb":3987,"num_write":2073,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1078,"ondisk_log_size":1078,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19748,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":13201,"num_read_kb":13000,"num_write":15927,"num_write_kb":4660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":39496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1836,"ondisk_log_size":1836,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":321,"ondisk_log_size":321,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1481237,"num_objects":198,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":18084,"num_read_kb":37002,"num_write":15113,"num_write_kb":11203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":353845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1842,"ondisk_log_size":1842,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1644,"num_read_kb":1954,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87319,"num_read_kb":70822,"num_write":35298,"num_write_kb":17822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3526,"ondisk_log_size":3526,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":187758130,"num_objects":1055,"num_object_clones":0,"num_object_copies":4220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1055,"num_whiteouts":0,"num_read":34213,"num_read_kb":1786506,"num_write":77152,"num_write_kb":1982911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":273719296,"data_stored":273719296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3512,"ondisk_log_size":3512,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":19963089,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":127,"num_read_kb":165927,"num_write":152,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36741120,"data_stored":36740305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":17865937,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":135,"num_read_kb":186344,"num_write":152,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":50372608,"data_stored":50371793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96,"ondisk_log_size":96,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575745,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":371852,"kb_used_data":51760,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":93999988,"statfs":{"total":96636764160,"available":96255987712,"internally_reserved":0,"allocated":53002240,"data_stored":51018038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":327612874},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575745,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":353092,"kb_used_data":60700,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":94018748,"statfs":{"total":96636764160,"available":96275197952,"internally_reserved":0,"allocated":62156800,"data_stored":60226794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":299170249},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575747,"num_pgs":114,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206904,"kb_used_data":40952,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94164936,"statfs":{"total":96636764160,"available":96424894464,"internally_reserved":0,"allocated":41934848,"data_stored":39931707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575747,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":146836,"kb_used_data":43468,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94225004,"statfs":{"total":96636764160,"available":96486404096,"internally_reserved":0,"allocated":44511232,"data_stored":42596897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575750,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303384,"kb_used_data":29196,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":94068456,"statfs":{"total":96636764160,"available":96326098944,"internally_reserved":0,"allocated":29896704,"data_stored":27928906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":280623559},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575748,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137428,"kb_used_data":29820,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94234412,"statfs":{"total":96636764160,"available":96496037888,"internally_reserved":0,"allocated":30535680,"data_stored":28610104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575745,"num_pgs":120,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201624,"kb_used_data":39968,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94170216,"statfs":{"total":96636764160,"available":96430301184,"internally_reserved":0,"allocated":40927232,"data_stored":38982339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575747,"num_pgs":116,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":385160,"kb_used_data":129088,"kb_used_omap":1,"kb_used_meta":256062,"kb_avail":93986680,"statfs":{"total":96636764160,"available":96242360320,"internally_reserved":0,"allocated":132186112,"data_stored":130163058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":262207941},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":96485376,"data_stored":96485376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25198592,"data_stored":25198592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21118976,"data_stored":21118976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19386368,"data_stored":19386368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29863936,"data_stored":29863936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20664320,"data_stored":20664320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28712960,"data_stored":28712960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32288768,"data_stored":32288768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":347321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":7701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":4574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":4436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":5746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16818176,"data_stored":16817361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13672448,"data_stored":13671633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-28T16:14:36.792 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-28T16:14:37.128 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:37.129 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":260,"fsid":"cc348236-88e8-4481-a265-c0b687a883fe","created":"2024-06-28T14:59:53.858969+0000","modified":"2024-06-28T15:37:34.296449+0000","last_up_change":"2024-06-28T15:00:04.244051+0000","last_in_change":"2024-06-28T14:59:56.118930+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":15,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":68,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-28T15:00:04.696005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-06-28T15:00:04.788733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.buckets.data","create_time":"2024-06-28T15:00:13.322439+0000","flags":1,"flags_names":"hashpspool","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"client.0","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":12288,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.buckets.index","create_time":"2024-06-28T15:00:15.703017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":".rgw.root","create_time":"2024-06-28T15:00:17.485887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.log","create_time":"2024-06-28T15:00:19.704279+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"default.rgw.control","create_time":"2024-06-28T15:00:21.403713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"default.rgw.meta","create_time":"2024-06-28T15:00:23.536266+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":65,"pool_name":"default.rgw.buckets.non-ec","create_time":"2024-06-28T15:17:22.056744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"249","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":66,"pool_name":"default.rgw.buckets.data.frozen","create_time":"2024-06-28T15:21:05.143976+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"252","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":67,"pool_name":"default.rgw.buckets.data.lukewarm","create_time":"2024-06-28T15:21:09.457542+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"255","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":68,"pool_name":"default.rgw.otp","create_time":"2024-06-28T15:37:31.560895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"260","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ff15d827-b738-4acc-84f1-577244382e19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":258,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6801","nonce":2650530773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6803","nonce":2650530773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6807","nonce":2650530773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2650530773},{"type":"v1","addr":"172.21.15.178:6805","nonce":2650530773}]},"public_addr":"172.21.15.178:6801/2650530773","cluster_addr":"172.21.15.178:6803/2650530773","heartbeat_back_addr":"172.21.15.178:6807/2650530773","heartbeat_front_addr":"172.21.15.178:6805/2650530773","state":["exists","up"]},{"osd":1,"uuid":"0999f2c0-81f9-4361-b3cb-e6bc5b0c25bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":258,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6812","nonce":1880801124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6816","nonce":1880801124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6824","nonce":1880801124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":1880801124},{"type":"v1","addr":"172.21.15.178:6820","nonce":1880801124}]},"public_addr":"172.21.15.178:6812/1880801124","cluster_addr":"172.21.15.178:6816/1880801124","heartbeat_back_addr":"172.21.15.178:6824/1880801124","heartbeat_front_addr":"172.21.15.178:6820/1880801124","state":["exists","up"]},{"osd":2,"uuid":"30cc4110-9499-411b-94da-b3de3c28627e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":258,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6823","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6825","nonce":1880098663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6827","nonce":1880098663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6831","nonce":1880098663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":1880098663},{"type":"v1","addr":"172.21.15.178:6829","nonce":1880098663}]},"public_addr":"172.21.15.178:6825/1880098663","cluster_addr":"172.21.15.178:6827/1880098663","heartbeat_back_addr":"172.21.15.178:6831/1880098663","heartbeat_front_addr":"172.21.15.178:6829/1880098663","state":["exists","up"]},{"osd":3,"uuid":"ec1f89b8-fcca-46c9-ba09-4cda48a142e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":258,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6809","nonce":4178996824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6811","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6813","nonce":4178996824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6819","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6821","nonce":4178996824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6815","nonce":4178996824},{"type":"v1","addr":"172.21.15.178:6817","nonce":4178996824}]},"public_addr":"172.21.15.178:6809/4178996824","cluster_addr":"172.21.15.178:6813/4178996824","heartbeat_back_addr":"172.21.15.178:6821/4178996824","heartbeat_front_addr":"172.21.15.178:6817/4178996824","state":["exists","up"]},{"osd":4,"uuid":"4fc9b653-922a-4c3c-89e7-a347e7553393","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":258,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6817","nonce":2928422746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6819","nonce":2928422746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6823","nonce":2928422746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2928422746},{"type":"v1","addr":"172.21.15.195:6821","nonce":2928422746}]},"public_addr":"172.21.15.195:6817/2928422746","cluster_addr":"172.21.15.195:6819/2928422746","heartbeat_back_addr":"172.21.15.195:6823/2928422746","heartbeat_front_addr":"172.21.15.195:6821/2928422746","state":["exists","up"]},{"osd":5,"uuid":"7a56dd0f-6d19-4c74-9108-d4cfdf51a36b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":258,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6801","nonce":2644688681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6805","nonce":2644688681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6813","nonce":2644688681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":2644688681},{"type":"v1","addr":"172.21.15.195:6809","nonce":2644688681}]},"public_addr":"172.21.15.195:6801/2644688681","cluster_addr":"172.21.15.195:6805/2644688681","heartbeat_back_addr":"172.21.15.195:6813/2644688681","heartbeat_front_addr":"172.21.15.195:6809/2644688681","state":["exists","up"]},{"osd":6,"uuid":"dcb3e382-9ad2-4ae2-9e96-332a581ec20e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":258,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6804","nonce":1595562688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6808","nonce":1595562688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6815","nonce":1595562688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1595562688},{"type":"v1","addr":"172.21.15.195:6812","nonce":1595562688}]},"public_addr":"172.21.15.195:6804/1595562688","cluster_addr":"172.21.15.195:6808/1595562688","heartbeat_back_addr":"172.21.15.195:6815/1595562688","heartbeat_front_addr":"172.21.15.195:6812/1595562688","state":["exists","up"]},{"osd":7,"uuid":"18053f3f-16ae-42af-a795-89cec10f661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":258,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6825","nonce":1278553753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6827","nonce":1278553753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6831","nonce":1278553753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":1278553753},{"type":"v1","addr":"172.21.15.195:6829","nonce":1278553753}]},"public_addr":"172.21.15.195:6825/1278553753","cluster_addr":"172.21.15.195:6827/1278553753","heartbeat_back_addr":"172.21.15.195:6831/1278553753","heartbeat_front_addr":"172.21.15.195:6829/1278553753","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.628549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.672292+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.714676+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.718739+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.580848+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.636756+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.636054+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-28T15:00:01.668751+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":2}]},{"pgid":"3.19","mappings":[{"from":4,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.23","mappings":[{"from":3,"to":2}]},{"pgid":"67.c","mappings":[{"from":3,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"client.0":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"3","m":"1","name":"testprofile","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":26,"snaps":[{"begin":1,"length":2}]},{"pool":31,"snaps":[{"begin":1,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-28T16:14:38.149 INFO:tasks.ceph:Scrubbing osd.0 2024-06-28T16:14:38.150 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-06-28T16:14:38.310 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T16:14:38.310 INFO:teuthology.orchestra.run.smithi178.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-06-28T16:14:38.310 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T16:14:38.326 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-06-28T16:14:38.614 INFO:teuthology.orchestra.run.smithi178.stderr:instructed osd(s) 0 to deep-scrub 2024-06-28T16:14:38.633 INFO:tasks.ceph:Scrubbing osd.1 2024-06-28T16:14:38.633 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-06-28T16:14:38.775 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T16:14:38.776 INFO:teuthology.orchestra.run.smithi178.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-06-28T16:14:38.776 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T16:14:38.791 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-06-28T16:14:39.079 INFO:teuthology.orchestra.run.smithi178.stderr:instructed osd(s) 1 to deep-scrub 2024-06-28T16:14:39.097 INFO:tasks.ceph:Scrubbing osd.2 2024-06-28T16:14:39.097 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-06-28T16:14:39.233 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T16:14:39.233 INFO:teuthology.orchestra.run.smithi178.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-06-28T16:14:39.234 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T16:14:39.248 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-06-28T16:14:39.526 INFO:teuthology.orchestra.run.smithi178.stderr:instructed osd(s) 2 to deep-scrub 2024-06-28T16:14:39.545 INFO:tasks.ceph:Scrubbing osd.3 2024-06-28T16:14:39.546 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-06-28T16:14:39.679 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T16:14:39.680 INFO:teuthology.orchestra.run.smithi178.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-06-28T16:14:39.680 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T16:14:39.695 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-06-28T16:14:39.977 INFO:teuthology.orchestra.run.smithi178.stderr:instructed osd(s) 3 to deep-scrub 2024-06-28T16:14:39.997 INFO:tasks.ceph:Scrubbing osd.4 2024-06-28T16:14:39.997 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-06-28T16:14:40.136 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T16:14:40.136 INFO:teuthology.orchestra.run.smithi178.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-06-28T16:14:40.136 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T16:14:40.152 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-06-28T16:14:40.425 INFO:teuthology.orchestra.run.smithi178.stderr:instructed osd(s) 4 to deep-scrub 2024-06-28T16:14:40.443 INFO:tasks.ceph:Scrubbing osd.5 2024-06-28T16:14:40.443 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-06-28T16:14:40.594 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T16:14:40.595 INFO:teuthology.orchestra.run.smithi178.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-06-28T16:14:40.595 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T16:14:40.610 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-06-28T16:14:40.880 INFO:teuthology.orchestra.run.smithi178.stderr:instructed osd(s) 5 to deep-scrub 2024-06-28T16:14:40.899 INFO:tasks.ceph:Scrubbing osd.6 2024-06-28T16:14:40.899 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-06-28T16:14:41.033 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T16:14:41.033 INFO:teuthology.orchestra.run.smithi178.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-06-28T16:14:41.033 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T16:14:41.048 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-06-28T16:14:41.335 INFO:teuthology.orchestra.run.smithi178.stderr:instructed osd(s) 6 to deep-scrub 2024-06-28T16:14:41.353 INFO:tasks.ceph:Scrubbing osd.7 2024-06-28T16:14:41.353 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-06-28T16:14:41.503 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-28T16:14:41.503 INFO:teuthology.orchestra.run.smithi178.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-06-28T16:14:41.503 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-28T16:14:41.518 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-06-28T16:14:41.805 INFO:teuthology.orchestra.run.smithi178.stderr:instructed osd(s) 7 to deep-scrub 2024-06-28T16:14:41.823 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-28T16:14:42.143 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:14:42.153 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-06-28T16:14:42.159 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":2490,"stamp":"2024-06-28T16:14:41.611360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":185196140,"num_objects":785,"num_object_clones":0,"num_object_copies":2436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":785,"num_whiteouts":0,"num_read":161161,"num_read_kb":2266365,"num_write":146672,"num_write_kb":2275015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2470,"num_omap_keys":5,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12363,"ondisk_log_size":12363,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1970096,"kb_used_data":297312,"kb_used_omap":12,"kb_used_meta":1672051,"kb_avail":753004624,"statfs":{"total":773094113280,"available":771076734976,"internally_reserved":0,"allocated":304447488,"data_stored":288792710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":1712180787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-500077675,"num_objects":-4019,"num_object_clones":0,"num_object_copies":-16004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4019,"num_whiteouts":0,"num_read":4528,"num_read_kb":4071,"num_write":4235,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2470,"num_omap_keys":5,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007936"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":26,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.063084+0000","last_change":"2024-06-28T15:37:33.329429+0000","last_active":"2024-06-28T16:10:05.063084+0000","last_peered":"2024-06-28T16:10:05.063084+0000","last_clean":"2024-06-28T16:10:05.063084+0000","last_became_active":"2024-06-28T15:37:33.328626+0000","last_became_peered":"2024-06-28T15:37:33.328626+0000","last_unstale":"2024-06-28T16:10:05.063084+0000","last_undegraded":"2024-06-28T16:10:05.063084+0000","last_fullsized":"2024-06-28T16:10:05.063084+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:48:45.377436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"257'3","reported_seq":49,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.078088+0000","last_change":"2024-06-28T15:21:11.079460+0000","last_active":"2024-06-28T16:11:10.078088+0000","last_peered":"2024-06-28T16:11:10.078088+0000","last_clean":"2024-06-28T16:11:10.078088+0000","last_became_active":"2024-06-28T15:21:11.078250+0000","last_became_peered":"2024-06-28T15:21:11.078250+0000","last_unstale":"2024-06-28T16:11:10.078088+0000","last_undegraded":"2024-06-28T16:11:10.078088+0000","last_fullsized":"2024-06-28T16:11:10.078088+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:32:28.636744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"260'4","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.364600+0000","last_change":"2024-06-28T15:21:07.047340+0000","last_active":"2024-06-28T16:14:33.364600+0000","last_peered":"2024-06-28T16:14:33.364600+0000","last_clean":"2024-06-28T16:14:33.364600+0000","last_became_active":"2024-06-28T15:21:07.046517+0000","last_became_peered":"2024-06-28T15:21:07.046517+0000","last_unstale":"2024-06-28T16:14:33.364600+0000","last_undegraded":"2024-06-28T16:14:33.364600+0000","last_fullsized":"2024-06-28T16:14:33.364600+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3076,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"249'7","reported_seq":91,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.148277+0000","last_change":"2024-06-28T16:14:39.148310+0000","last_active":"2024-06-28T16:14:39.148277+0000","last_peered":"2024-06-28T16:14:39.148277+0000","last_clean":"2024-06-28T16:14:39.148277+0000","last_became_active":"2024-06-28T15:17:24.028787+0000","last_became_peered":"2024-06-28T15:17:24.028787+0000","last_unstale":"2024-06-28T16:14:39.148277+0000","last_undegraded":"2024-06-28T16:14:39.148277+0000","last_fullsized":"2024-06-28T16:14:39.148277+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'7","last_scrub_stamp":"2024-06-28T16:14:39.148161+0000","last_deep_scrub":"249'7","last_deep_scrub_stamp":"2024-06-28T16:14:39.148161+0000","last_clean_scrub_stamp":"2024-06-28T16:14:39.148161+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:43:21.535181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":30,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:35.282884+0000","last_change":"2024-06-28T15:37:33.337334+0000","last_active":"2024-06-28T16:12:35.282884+0000","last_peered":"2024-06-28T16:12:35.282884+0000","last_clean":"2024-06-28T16:12:35.282884+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:12:35.282884+0000","last_undegraded":"2024-06-28T16:12:35.282884+0000","last_fullsized":"2024-06-28T16:12:35.282884+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"260'2","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.119791+0000","last_change":"2024-06-28T15:21:11.105141+0000","last_active":"2024-06-28T16:13:45.119791+0000","last_peered":"2024-06-28T16:13:45.119791+0000","last_clean":"2024-06-28T16:13:45.119791+0000","last_became_active":"2024-06-28T15:21:11.104387+0000","last_became_peered":"2024-06-28T15:21:11.104387+0000","last_unstale":"2024-06-28T16:13:45.119791+0000","last_undegraded":"2024-06-28T16:13:45.119791+0000","last_fullsized":"2024-06-28T16:13:45.119791+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:29:31.608922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"260'2","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.136523+0000","last_change":"2024-06-28T15:21:07.049830+0000","last_active":"2024-06-28T16:14:33.136523+0000","last_peered":"2024-06-28T16:14:33.136523+0000","last_clean":"2024-06-28T16:14:33.136523+0000","last_became_active":"2024-06-28T15:21:07.049141+0000","last_became_peered":"2024-06-28T15:21:07.049141+0000","last_unstale":"2024-06-28T16:14:33.136523+0000","last_undegraded":"2024-06-28T16:14:33.136523+0000","last_fullsized":"2024-06-28T16:14:33.136523+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"260'18","reported_seq":111,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.297119+0000","last_change":"2024-06-28T15:17:23.994609+0000","last_active":"2024-06-28T16:13:40.297119+0000","last_peered":"2024-06-28T16:13:40.297119+0000","last_clean":"2024-06-28T16:13:40.297119+0000","last_became_active":"2024-06-28T15:17:23.993905+0000","last_became_peered":"2024-06-28T15:17:23.993905+0000","last_unstale":"2024-06-28T16:13:40.297119+0000","last_undegraded":"2024-06-28T16:13:40.297119+0000","last_fullsized":"2024-06-28T16:13:40.297119+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":64,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.087475+0000","last_change":"2024-06-28T15:37:33.353498+0000","last_active":"2024-06-28T16:11:25.087475+0000","last_peered":"2024-06-28T16:11:25.087475+0000","last_clean":"2024-06-28T16:11:25.087475+0000","last_became_active":"2024-06-28T15:37:33.345642+0000","last_became_peered":"2024-06-28T15:37:33.345642+0000","last_unstale":"2024-06-28T16:11:25.087475+0000","last_undegraded":"2024-06-28T16:11:25.087475+0000","last_fullsized":"2024-06-28T16:11:25.087475+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:38:21.634963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"257'5","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.591118+0000","last_change":"2024-06-28T15:21:11.098339+0000","last_active":"2024-06-28T16:13:20.591118+0000","last_peered":"2024-06-28T16:13:20.591118+0000","last_clean":"2024-06-28T16:13:20.591118+0000","last_became_active":"2024-06-28T15:21:11.097619+0000","last_became_peered":"2024-06-28T15:21:11.097619+0000","last_unstale":"2024-06-28T16:13:20.591118+0000","last_undegraded":"2024-06-28T16:13:20.591118+0000","last_fullsized":"2024-06-28T16:13:20.591118+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5438,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"0'0","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:25.293990+0000","last_change":"2024-06-28T15:21:07.020860+0000","last_active":"2024-06-28T16:13:25.293990+0000","last_peered":"2024-06-28T16:13:25.293990+0000","last_clean":"2024-06-28T16:13:25.293990+0000","last_became_active":"2024-06-28T15:21:07.020090+0000","last_became_peered":"2024-06-28T15:21:07.020090+0000","last_unstale":"2024-06-28T16:13:25.293990+0000","last_undegraded":"2024-06-28T16:13:25.293990+0000","last_fullsized":"2024-06-28T16:13:25.293990+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"260'43","reported_seq":175,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.576417+0000","last_change":"2024-06-28T15:17:24.014535+0000","last_active":"2024-06-28T16:12:25.576417+0000","last_peered":"2024-06-28T16:12:25.576417+0000","last_clean":"2024-06-28T16:12:25.576417+0000","last_became_active":"2024-06-28T15:17:24.012739+0000","last_became_peered":"2024-06-28T15:17:24.012739+0000","last_unstale":"2024-06-28T16:12:25.576417+0000","last_undegraded":"2024-06-28T16:12:25.576417+0000","last_fullsized":"2024-06-28T16:12:25.576417+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":221,"num_read_kb":154,"num_write":91,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.328538+0000","last_change":"2024-06-28T15:37:33.345746+0000","last_active":"2024-06-28T16:14:00.328538+0000","last_peered":"2024-06-28T16:14:00.328538+0000","last_clean":"2024-06-28T16:14:00.328538+0000","last_became_active":"2024-06-28T15:37:33.344850+0000","last_became_peered":"2024-06-28T15:37:33.344850+0000","last_unstale":"2024-06-28T16:14:00.328538+0000","last_undegraded":"2024-06-28T16:14:00.328538+0000","last_fullsized":"2024-06-28T16:14:00.328538+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"260'3","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.475704+0000","last_change":"2024-06-28T15:21:11.107158+0000","last_active":"2024-06-28T16:14:34.475704+0000","last_peered":"2024-06-28T16:14:34.475704+0000","last_clean":"2024-06-28T16:14:34.475704+0000","last_became_active":"2024-06-28T15:21:11.102572+0000","last_became_peered":"2024-06-28T15:21:11.102572+0000","last_unstale":"2024-06-28T16:14:34.475704+0000","last_undegraded":"2024-06-28T16:14:34.475704+0000","last_fullsized":"2024-06-28T16:14:34.475704+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.085295+0000","last_change":"2024-06-28T15:21:07.032950+0000","last_active":"2024-06-28T16:11:10.085295+0000","last_peered":"2024-06-28T16:11:10.085295+0000","last_clean":"2024-06-28T16:11:10.085295+0000","last_became_active":"2024-06-28T15:21:07.030576+0000","last_became_peered":"2024-06-28T15:21:07.030576+0000","last_unstale":"2024-06-28T16:11:10.085295+0000","last_undegraded":"2024-06-28T16:11:10.085295+0000","last_fullsized":"2024-06-28T16:11:10.085295+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:28:35.851727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'23","reported_seq":117,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.522502+0000","last_change":"2024-06-28T15:17:24.019871+0000","last_active":"2024-06-28T16:12:25.522502+0000","last_peered":"2024-06-28T16:12:25.522502+0000","last_clean":"2024-06-28T16:12:25.522502+0000","last_became_active":"2024-06-28T15:17:24.019120+0000","last_became_peered":"2024-06-28T15:17:24.019120+0000","last_unstale":"2024-06-28T16:12:25.522502+0000","last_undegraded":"2024-06-28T16:12:25.522502+0000","last_fullsized":"2024-06-28T16:12:25.522502+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":82,"num_write":51,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":31,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.326025+0000","last_change":"2024-06-28T15:37:33.339459+0000","last_active":"2024-06-28T16:13:45.326025+0000","last_peered":"2024-06-28T16:13:45.326025+0000","last_clean":"2024-06-28T16:13:45.326025+0000","last_became_active":"2024-06-28T15:37:33.335045+0000","last_became_peered":"2024-06-28T15:37:33.335045+0000","last_unstale":"2024-06-28T16:13:45.326025+0000","last_undegraded":"2024-06-28T16:13:45.326025+0000","last_fullsized":"2024-06-28T16:13:45.326025+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"0'0","reported_seq":44,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:10.091403+0000","last_change":"2024-06-28T15:21:11.094697+0000","last_active":"2024-06-28T16:12:10.091403+0000","last_peered":"2024-06-28T16:12:10.091403+0000","last_clean":"2024-06-28T16:12:10.091403+0000","last_became_active":"2024-06-28T15:21:11.094122+0000","last_became_peered":"2024-06-28T15:21:11.094122+0000","last_unstale":"2024-06-28T16:12:10.091403+0000","last_undegraded":"2024-06-28T16:12:10.091403+0000","last_fullsized":"2024-06-28T16:12:10.091403+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:48:32.491192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:20.604372+0000","last_change":"2024-06-28T15:21:07.027869+0000","last_active":"2024-06-28T16:14:20.604372+0000","last_peered":"2024-06-28T16:14:20.604372+0000","last_clean":"2024-06-28T16:14:20.604372+0000","last_became_active":"2024-06-28T15:21:07.025549+0000","last_became_peered":"2024-06-28T15:21:07.025549+0000","last_unstale":"2024-06-28T16:14:20.604372+0000","last_undegraded":"2024-06-28T16:14:20.604372+0000","last_fullsized":"2024-06-28T16:14:20.604372+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"260'24","reported_seq":121,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.522344+0000","last_change":"2024-06-28T15:17:24.011769+0000","last_active":"2024-06-28T16:12:25.522344+0000","last_peered":"2024-06-28T16:12:25.522344+0000","last_clean":"2024-06-28T16:12:25.522344+0000","last_became_active":"2024-06-28T15:17:24.010441+0000","last_became_peered":"2024-06-28T15:17:24.010441+0000","last_unstale":"2024-06-28T16:12:25.522344+0000","last_undegraded":"2024-06-28T16:12:25.522344+0000","last_fullsized":"2024-06-28T16:12:25.522344+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":85,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.118300+0000","last_change":"2024-06-28T15:37:33.341710+0000","last_active":"2024-06-28T16:13:00.118300+0000","last_peered":"2024-06-28T16:13:00.118300+0000","last_clean":"2024-06-28T16:13:00.118300+0000","last_became_active":"2024-06-28T15:37:33.336623+0000","last_became_peered":"2024-06-28T15:37:33.336623+0000","last_unstale":"2024-06-28T16:13:00.118300+0000","last_undegraded":"2024-06-28T16:13:00.118300+0000","last_fullsized":"2024-06-28T16:13:00.118300+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"260'11","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.517566+0000","last_change":"2024-06-28T15:21:11.100151+0000","last_active":"2024-06-28T16:14:32.517566+0000","last_peered":"2024-06-28T16:14:32.517566+0000","last_clean":"2024-06-28T16:14:32.517566+0000","last_became_active":"2024-06-28T15:21:11.099434+0000","last_became_peered":"2024-06-28T15:21:11.099434+0000","last_unstale":"2024-06-28T16:14:32.517566+0000","last_undegraded":"2024-06-28T16:14:32.517566+0000","last_fullsized":"2024-06-28T16:14:32.517566+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:21:06.014702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":20482,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"257'5","reported_seq":65,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:10.098868+0000","last_change":"2024-06-28T15:21:07.044767+0000","last_active":"2024-06-28T16:12:10.098868+0000","last_peered":"2024-06-28T16:12:10.098868+0000","last_clean":"2024-06-28T16:12:10.098868+0000","last_became_active":"2024-06-28T15:21:07.043987+0000","last_became_peered":"2024-06-28T15:21:07.043987+0000","last_unstale":"2024-06-28T16:12:10.098868+0000","last_undegraded":"2024-06-28T16:12:10.098868+0000","last_fullsized":"2024-06-28T16:12:10.098868+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:53:56.640453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"260'19","reported_seq":100,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:09:55.067186+0000","last_change":"2024-06-28T15:17:24.007176+0000","last_active":"2024-06-28T16:09:55.067186+0000","last_peered":"2024-06-28T16:09:55.067186+0000","last_clean":"2024-06-28T16:09:55.067186+0000","last_became_active":"2024-06-28T15:17:24.006314+0000","last_became_peered":"2024-06-28T15:17:24.006314+0000","last_unstale":"2024-06-28T16:09:55.067186+0000","last_undegraded":"2024-06-28T16:09:55.067186+0000","last_fullsized":"2024-06-28T16:09:55.067186+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:20:19.639267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":54,"num_write":49,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"260'1","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.076954+0000","last_change":"2024-06-28T15:37:33.348503+0000","last_active":"2024-06-28T16:10:05.076954+0000","last_peered":"2024-06-28T16:10:05.076954+0000","last_clean":"2024-06-28T16:10:05.076954+0000","last_became_active":"2024-06-28T15:37:33.347715+0000","last_became_peered":"2024-06-28T15:37:33.347715+0000","last_unstale":"2024-06-28T16:10:05.076954+0000","last_undegraded":"2024-06-28T16:10:05.076954+0000","last_fullsized":"2024-06-28T16:10:05.076954+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:01:24.044514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":42,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.085434+0000","last_change":"2024-06-28T15:21:11.089867+0000","last_active":"2024-06-28T16:11:15.085434+0000","last_peered":"2024-06-28T16:11:15.085434+0000","last_clean":"2024-06-28T16:11:15.085434+0000","last_became_active":"2024-06-28T15:21:11.088625+0000","last_became_peered":"2024-06-28T15:21:11.088625+0000","last_unstale":"2024-06-28T16:11:15.085434+0000","last_undegraded":"2024-06-28T16:11:15.085434+0000","last_fullsized":"2024-06-28T16:11:15.085434+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:32:47.953440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"260'6","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.118688+0000","last_change":"2024-06-28T15:21:07.031673+0000","last_active":"2024-06-28T16:13:40.118688+0000","last_peered":"2024-06-28T16:13:40.118688+0000","last_clean":"2024-06-28T16:13:40.118688+0000","last_became_active":"2024-06-28T15:21:07.030888+0000","last_became_peered":"2024-06-28T15:21:07.030888+0000","last_unstale":"2024-06-28T16:13:40.118688+0000","last_undegraded":"2024-06-28T16:13:40.118688+0000","last_fullsized":"2024-06-28T16:13:40.118688+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:54.912979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":6,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"260'72","reported_seq":248,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.588195+0000","last_change":"2024-06-28T15:17:23.993803+0000","last_active":"2024-06-28T16:13:10.588195+0000","last_peered":"2024-06-28T16:13:10.588195+0000","last_clean":"2024-06-28T16:13:10.588195+0000","last_became_active":"2024-06-28T15:17:23.993026+0000","last_became_peered":"2024-06-28T15:17:23.993026+0000","last_unstale":"2024-06-28T16:13:10.588195+0000","last_undegraded":"2024-06-28T16:13:10.588195+0000","last_fullsized":"2024-06-28T16:13:10.588195+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":255,"num_write":131,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":26,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:05.077019+0000","last_change":"2024-06-28T15:37:33.351154+0000","last_active":"2024-06-28T16:10:05.077019+0000","last_peered":"2024-06-28T16:10:05.077019+0000","last_clean":"2024-06-28T16:10:05.077019+0000","last_became_active":"2024-06-28T15:37:33.350447+0000","last_became_peered":"2024-06-28T15:37:33.350447+0000","last_unstale":"2024-06-28T16:10:05.077019+0000","last_undegraded":"2024-06-28T16:10:05.077019+0000","last_fullsized":"2024-06-28T16:10:05.077019+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:41:10.113052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":44,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.324364+0000","last_change":"2024-06-28T15:21:11.101407+0000","last_active":"2024-06-28T16:13:40.324364+0000","last_peered":"2024-06-28T16:13:40.324364+0000","last_clean":"2024-06-28T16:13:40.324364+0000","last_became_active":"2024-06-28T15:21:11.100603+0000","last_became_peered":"2024-06-28T15:21:11.100603+0000","last_unstale":"2024-06-28T16:13:40.324364+0000","last_undegraded":"2024-06-28T16:13:40.324364+0000","last_fullsized":"2024-06-28T16:13:40.324364+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"260'5","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.183128+0000","last_change":"2024-06-28T15:21:07.047854+0000","last_active":"2024-06-28T16:14:34.183128+0000","last_peered":"2024-06-28T16:14:34.183128+0000","last_clean":"2024-06-28T16:14:34.183128+0000","last_became_active":"2024-06-28T15:21:07.045902+0000","last_became_peered":"2024-06-28T15:21:07.045902+0000","last_unstale":"2024-06-28T16:14:34.183128+0000","last_undegraded":"2024-06-28T16:14:34.183128+0000","last_fullsized":"2024-06-28T16:14:34.183128+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":40,"num_write":10,"num_write_kb":1064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"260'48","reported_seq":193,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.118076+0000","last_change":"2024-06-28T15:17:24.025635+0000","last_active":"2024-06-28T16:13:00.118076+0000","last_peered":"2024-06-28T16:13:00.118076+0000","last_clean":"2024-06-28T16:13:00.118076+0000","last_became_active":"2024-06-28T15:17:24.016249+0000","last_became_peered":"2024-06-28T15:17:24.016249+0000","last_unstale":"2024-06-28T16:13:00.118076+0000","last_undegraded":"2024-06-28T16:13:00.118076+0000","last_fullsized":"2024-06-28T16:13:00.118076+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":260,"num_read_kb":188,"num_write":81,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:35.112437+0000","last_change":"2024-06-28T15:37:33.317386+0000","last_active":"2024-06-28T16:12:35.112437+0000","last_peered":"2024-06-28T16:12:35.112437+0000","last_clean":"2024-06-28T16:12:35.112437+0000","last_became_active":"2024-06-28T15:37:33.316616+0000","last_became_peered":"2024-06-28T15:37:33.316616+0000","last_unstale":"2024-06-28T16:12:35.112437+0000","last_undegraded":"2024-06-28T16:12:35.112437+0000","last_fullsized":"2024-06-28T16:12:35.112437+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"260'8","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.507597+0000","last_change":"2024-06-28T15:21:11.099574+0000","last_active":"2024-06-28T16:14:33.507597+0000","last_peered":"2024-06-28T16:14:33.507597+0000","last_clean":"2024-06-28T16:14:33.507597+0000","last_became_active":"2024-06-28T15:21:11.094947+0000","last_became_peered":"2024-06-28T15:21:11.094947+0000","last_unstale":"2024-06-28T16:14:33.507597+0000","last_undegraded":"2024-06-28T16:14:33.507597+0000","last_fullsized":"2024-06-28T16:14:33.507597+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":12406,"num_write":8,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"260'5","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:45.099524+0000","last_change":"2024-06-28T15:21:07.026138+0000","last_active":"2024-06-28T16:12:45.099524+0000","last_peered":"2024-06-28T16:12:45.099524+0000","last_clean":"2024-06-28T16:12:45.099524+0000","last_became_active":"2024-06-28T15:21:07.025396+0000","last_became_peered":"2024-06-28T15:21:07.025396+0000","last_unstale":"2024-06-28T16:12:45.099524+0000","last_undegraded":"2024-06-28T16:12:45.099524+0000","last_fullsized":"2024-06-28T16:12:45.099524+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:48:53.728948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":5,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"257'14","reported_seq":97,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.585702+0000","last_change":"2024-06-28T15:17:24.015094+0000","last_active":"2024-06-28T16:13:00.585702+0000","last_peered":"2024-06-28T16:13:00.585702+0000","last_clean":"2024-06-28T16:13:00.585702+0000","last_became_active":"2024-06-28T15:17:24.010190+0000","last_became_peered":"2024-06-28T15:17:24.010190+0000","last_unstale":"2024-06-28T16:13:00.585702+0000","last_undegraded":"2024-06-28T16:13:00.585702+0000","last_fullsized":"2024-06-28T16:13:00.585702+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":51,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"260'1","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:35.282840+0000","last_change":"2024-06-28T15:37:33.338222+0000","last_active":"2024-06-28T16:12:35.282840+0000","last_peered":"2024-06-28T16:12:35.282840+0000","last_clean":"2024-06-28T16:12:35.282840+0000","last_became_active":"2024-06-28T15:37:33.334097+0000","last_became_peered":"2024-06-28T15:37:33.334097+0000","last_unstale":"2024-06-28T16:12:35.282840+0000","last_undegraded":"2024-06-28T16:12:35.282840+0000","last_fullsized":"2024-06-28T16:12:35.282840+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"260'9","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.104329+0000","last_change":"2024-06-28T15:21:11.107909+0000","last_active":"2024-06-28T16:14:34.104329+0000","last_peered":"2024-06-28T16:14:34.104329+0000","last_clean":"2024-06-28T16:14:34.104329+0000","last_became_active":"2024-06-28T15:21:11.107155+0000","last_became_peered":"2024-06-28T15:21:11.107155+0000","last_unstale":"2024-06-28T16:14:34.104329+0000","last_undegraded":"2024-06-28T16:14:34.104329+0000","last_fullsized":"2024-06-28T16:14:34.104329+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21504,"num_write":14,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":49,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.102713+0000","last_change":"2024-06-28T15:21:07.043526+0000","last_active":"2024-06-28T16:11:55.102713+0000","last_peered":"2024-06-28T16:11:55.102713+0000","last_clean":"2024-06-28T16:11:55.102713+0000","last_became_active":"2024-06-28T15:21:07.042691+0000","last_became_peered":"2024-06-28T15:21:07.042691+0000","last_unstale":"2024-06-28T16:11:55.102713+0000","last_undegraded":"2024-06-28T16:11:55.102713+0000","last_fullsized":"2024-06-28T16:11:55.102713+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:56:49.487276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"260'74","reported_seq":299,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.069608+0000","last_change":"2024-06-28T16:14:40.069653+0000","last_active":"2024-06-28T16:14:40.069608+0000","last_peered":"2024-06-28T16:14:40.069608+0000","last_clean":"2024-06-28T16:14:40.069608+0000","last_became_active":"2024-06-28T15:17:24.027786+0000","last_became_peered":"2024-06-28T15:17:24.027786+0000","last_unstale":"2024-06-28T16:14:40.069608+0000","last_undegraded":"2024-06-28T16:14:40.069608+0000","last_fullsized":"2024-06-28T16:14:40.069608+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'74","last_scrub_stamp":"2024-06-28T16:14:40.069506+0000","last_deep_scrub":"260'74","last_deep_scrub_stamp":"2024-06-28T16:14:40.069506+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.069506+0000","objects_scrubbed":1,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:43:30.466356+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":461,"num_read_kb":332,"num_write":85,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2470,"num_omap_keys":5,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.079074+0000","last_change":"2024-06-28T15:37:33.355373+0000","last_active":"2024-06-28T16:10:15.079074+0000","last_peered":"2024-06-28T16:10:15.079074+0000","last_clean":"2024-06-28T16:10:15.079074+0000","last_became_active":"2024-06-28T15:37:33.354730+0000","last_became_peered":"2024-06-28T15:37:33.354730+0000","last_unstale":"2024-06-28T16:10:15.079074+0000","last_undegraded":"2024-06-28T16:10:15.079074+0000","last_fullsized":"2024-06-28T16:10:15.079074+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:33:00.069224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"260'4","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.513832+0000","last_change":"2024-06-28T15:21:11.094517+0000","last_active":"2024-06-28T16:14:32.513832+0000","last_peered":"2024-06-28T16:14:32.513832+0000","last_clean":"2024-06-28T16:14:32.513832+0000","last_became_active":"2024-06-28T15:21:11.093751+0000","last_became_peered":"2024-06-28T15:21:11.093751+0000","last_unstale":"2024-06-28T16:14:32.513832+0000","last_undegraded":"2024-06-28T16:14:32.513832+0000","last_fullsized":"2024-06-28T16:14:32.513832+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"260'2","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.306442+0000","last_change":"2024-06-28T15:21:07.043427+0000","last_active":"2024-06-28T16:14:32.306442+0000","last_peered":"2024-06-28T16:14:32.306442+0000","last_clean":"2024-06-28T16:14:32.306442+0000","last_became_active":"2024-06-28T15:21:07.042846+0000","last_became_peered":"2024-06-28T15:21:07.042846+0000","last_unstale":"2024-06-28T16:14:32.306442+0000","last_undegraded":"2024-06-28T16:14:32.306442+0000","last_fullsized":"2024-06-28T16:14:32.306442+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":59,"num_write":2,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"260'31","reported_seq":145,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.288677+0000","last_change":"2024-06-28T15:17:24.017530+0000","last_active":"2024-06-28T16:13:00.288677+0000","last_peered":"2024-06-28T16:13:00.288677+0000","last_clean":"2024-06-28T16:13:00.288677+0000","last_became_active":"2024-06-28T15:17:24.016275+0000","last_became_peered":"2024-06-28T15:17:24.016275+0000","last_unstale":"2024-06-28T16:13:00.288677+0000","last_undegraded":"2024-06-28T16:13:00.288677+0000","last_fullsized":"2024-06-28T16:13:00.288677+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":116,"num_write":60,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.065428+0000","last_change":"2024-06-28T15:37:33.323939+0000","last_active":"2024-06-28T16:10:15.065428+0000","last_peered":"2024-06-28T16:10:15.065428+0000","last_clean":"2024-06-28T16:10:15.065428+0000","last_became_active":"2024-06-28T15:37:33.323174+0000","last_became_peered":"2024-06-28T15:37:33.323174+0000","last_unstale":"2024-06-28T16:10:15.065428+0000","last_undegraded":"2024-06-28T16:10:15.065428+0000","last_fullsized":"2024-06-28T16:10:15.065428+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:23:02.255965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"260'9","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.510895+0000","last_change":"2024-06-28T15:21:11.088257+0000","last_active":"2024-06-28T16:14:33.510895+0000","last_peered":"2024-06-28T16:14:33.510895+0000","last_clean":"2024-06-28T16:14:33.510895+0000","last_became_active":"2024-06-28T15:21:11.087119+0000","last_became_peered":"2024-06-28T15:21:11.087119+0000","last_unstale":"2024-06-28T16:14:33.510895+0000","last_undegraded":"2024-06-28T16:14:33.510895+0000","last_fullsized":"2024-06-28T16:14:33.510895+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:24:43.264477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":24576,"num_write":19,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:20.604613+0000","last_change":"2024-06-28T15:21:07.032975+0000","last_active":"2024-06-28T16:14:20.604613+0000","last_peered":"2024-06-28T16:14:20.604613+0000","last_clean":"2024-06-28T16:14:20.604613+0000","last_became_active":"2024-06-28T15:21:07.032092+0000","last_became_peered":"2024-06-28T15:21:07.032092+0000","last_unstale":"2024-06-28T16:14:20.604613+0000","last_undegraded":"2024-06-28T16:14:20.604613+0000","last_fullsized":"2024-06-28T16:14:20.604613+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"260'21","reported_seq":120,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.094899+0000","last_change":"2024-06-28T15:17:24.021516+0000","last_active":"2024-06-28T16:11:20.094899+0000","last_peered":"2024-06-28T16:11:20.094899+0000","last_clean":"2024-06-28T16:11:20.094899+0000","last_became_active":"2024-06-28T15:17:24.020899+0000","last_became_peered":"2024-06-28T15:17:24.020899+0000","last_unstale":"2024-06-28T16:11:20.094899+0000","last_undegraded":"2024-06-28T16:11:20.094899+0000","last_fullsized":"2024-06-28T16:11:20.094899+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:16:16.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":86,"num_write":39,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:50.326876+0000","last_change":"2024-06-28T15:37:33.339666+0000","last_active":"2024-06-28T16:13:50.326876+0000","last_peered":"2024-06-28T16:13:50.326876+0000","last_clean":"2024-06-28T16:13:50.326876+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:13:50.326876+0000","last_undegraded":"2024-06-28T16:13:50.326876+0000","last_fullsized":"2024-06-28T16:13:50.326876+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"257'3","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:35.112405+0000","last_change":"2024-06-28T15:21:11.098494+0000","last_active":"2024-06-28T16:12:35.112405+0000","last_peered":"2024-06-28T16:12:35.112405+0000","last_clean":"2024-06-28T16:12:35.112405+0000","last_became_active":"2024-06-28T15:21:11.097622+0000","last_became_peered":"2024-06-28T15:21:11.097622+0000","last_unstale":"2024-06-28T16:12:35.112405+0000","last_undegraded":"2024-06-28T16:12:35.112405+0000","last_fullsized":"2024-06-28T16:12:35.112405+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"0'0","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.071391+0000","last_change":"2024-06-28T15:21:07.039533+0000","last_active":"2024-06-28T16:10:10.071391+0000","last_peered":"2024-06-28T16:10:10.071391+0000","last_clean":"2024-06-28T16:10:10.071391+0000","last_became_active":"2024-06-28T15:21:07.038746+0000","last_became_peered":"2024-06-28T15:21:07.038746+0000","last_unstale":"2024-06-28T16:10:10.071391+0000","last_undegraded":"2024-06-28T16:10:10.071391+0000","last_fullsized":"2024-06-28T16:10:10.071391+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:11:32.609450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"257'9","reported_seq":81,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.091202+0000","last_change":"2024-06-28T15:17:24.024616+0000","last_active":"2024-06-28T16:11:05.091202+0000","last_peered":"2024-06-28T16:11:05.091202+0000","last_clean":"2024-06-28T16:11:05.091202+0000","last_became_active":"2024-06-28T15:17:24.020276+0000","last_became_peered":"2024-06-28T15:17:24.020276+0000","last_unstale":"2024-06-28T16:11:05.091202+0000","last_undegraded":"2024-06-28T16:11:05.091202+0000","last_fullsized":"2024-06-28T16:11:05.091202+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:06.656845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:50.597772+0000","last_change":"2024-06-28T15:37:33.328340+0000","last_active":"2024-06-28T16:13:50.597772+0000","last_peered":"2024-06-28T16:13:50.597772+0000","last_clean":"2024-06-28T16:13:50.597772+0000","last_became_active":"2024-06-28T15:37:33.327497+0000","last_became_peered":"2024-06-28T15:37:33.327497+0000","last_unstale":"2024-06-28T16:13:50.597772+0000","last_undegraded":"2024-06-28T16:13:50.597772+0000","last_fullsized":"2024-06-28T16:13:50.597772+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"260'4","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.507929+0000","last_change":"2024-06-28T15:21:11.092761+0000","last_active":"2024-06-28T16:14:33.507929+0000","last_peered":"2024-06-28T16:14:33.507929+0000","last_clean":"2024-06-28T16:14:33.507929+0000","last_became_active":"2024-06-28T15:21:11.092033+0000","last_became_peered":"2024-06-28T15:21:11.092033+0000","last_unstale":"2024-06-28T16:14:33.507929+0000","last_undegraded":"2024-06-28T16:14:33.507929+0000","last_fullsized":"2024-06-28T16:14:33.507929+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6152,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:30.523755+0000","last_change":"2024-06-28T15:21:07.047668+0000","last_active":"2024-06-28T16:12:30.523755+0000","last_peered":"2024-06-28T16:12:30.523755+0000","last_clean":"2024-06-28T16:12:30.523755+0000","last_became_active":"2024-06-28T15:21:07.045842+0000","last_became_peered":"2024-06-28T15:21:07.045842+0000","last_unstale":"2024-06-28T16:12:30.523755+0000","last_undegraded":"2024-06-28T16:12:30.523755+0000","last_fullsized":"2024-06-28T16:12:30.523755+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"260'17","reported_seq":107,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:25.592147+0000","last_change":"2024-06-28T15:17:24.016733+0000","last_active":"2024-06-28T16:13:25.592147+0000","last_peered":"2024-06-28T16:13:25.592147+0000","last_clean":"2024-06-28T16:13:25.592147+0000","last_became_active":"2024-06-28T15:17:24.013118+0000","last_became_peered":"2024-06-28T15:17:24.013118+0000","last_unstale":"2024-06-28T16:13:25.592147+0000","last_undegraded":"2024-06-28T16:13:25.592147+0000","last_fullsized":"2024-06-28T16:13:25.592147+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":63,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"259'1","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:40.579904+0000","last_change":"2024-06-28T15:37:33.342777+0000","last_active":"2024-06-28T16:12:40.579904+0000","last_peered":"2024-06-28T16:12:40.579904+0000","last_clean":"2024-06-28T16:12:40.579904+0000","last_became_active":"2024-06-28T15:37:33.339097+0000","last_became_peered":"2024-06-28T15:37:33.339097+0000","last_unstale":"2024-06-28T16:12:40.579904+0000","last_undegraded":"2024-06-28T16:12:40.579904+0000","last_fullsized":"2024-06-28T16:12:40.579904+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"257'3","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:30.523900+0000","last_change":"2024-06-28T15:21:11.101867+0000","last_active":"2024-06-28T16:12:30.523900+0000","last_peered":"2024-06-28T16:12:30.523900+0000","last_clean":"2024-06-28T16:12:30.523900+0000","last_became_active":"2024-06-28T15:21:11.101063+0000","last_became_peered":"2024-06-28T15:21:11.101063+0000","last_unstale":"2024-06-28T16:12:30.523900+0000","last_undegraded":"2024-06-28T16:12:30.523900+0000","last_fullsized":"2024-06-28T16:12:30.523900+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"257'3","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.070602+0000","last_change":"2024-06-28T15:21:07.042183+0000","last_active":"2024-06-28T16:10:10.070602+0000","last_peered":"2024-06-28T16:10:10.070602+0000","last_clean":"2024-06-28T16:10:10.070602+0000","last_became_active":"2024-06-28T15:21:07.040973+0000","last_became_peered":"2024-06-28T15:21:07.040973+0000","last_unstale":"2024-06-28T16:10:10.070602+0000","last_undegraded":"2024-06-28T16:10:10.070602+0000","last_fullsized":"2024-06-28T16:10:10.070602+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:38:10.417534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"260'22","reported_seq":115,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.083945+0000","last_change":"2024-06-28T15:17:24.029346+0000","last_active":"2024-06-28T16:11:05.083945+0000","last_peered":"2024-06-28T16:11:05.083945+0000","last_clean":"2024-06-28T16:11:05.083945+0000","last_became_active":"2024-06-28T15:17:24.028410+0000","last_became_peered":"2024-06-28T15:17:24.028410+0000","last_unstale":"2024-06-28T16:11:05.083945+0000","last_undegraded":"2024-06-28T16:11:05.083945+0000","last_fullsized":"2024-06-28T16:11:05.083945+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:25:05.370434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"260'1","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:50.326834+0000","last_change":"2024-06-28T15:37:33.341173+0000","last_active":"2024-06-28T16:13:50.326834+0000","last_peered":"2024-06-28T16:13:50.326834+0000","last_clean":"2024-06-28T16:13:50.326834+0000","last_became_active":"2024-06-28T15:37:33.340370+0000","last_became_peered":"2024-06-28T15:37:33.340370+0000","last_unstale":"2024-06-28T16:13:50.326834+0000","last_undegraded":"2024-06-28T16:13:50.326834+0000","last_fullsized":"2024-06-28T16:13:50.326834+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"257'3","reported_seq":48,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:15.071850+0000","last_change":"2024-06-28T15:21:11.104826+0000","last_active":"2024-06-28T16:10:15.071850+0000","last_peered":"2024-06-28T16:10:15.071850+0000","last_clean":"2024-06-28T16:10:15.071850+0000","last_became_active":"2024-06-28T15:21:11.104038+0000","last_became_peered":"2024-06-28T15:21:11.104038+0000","last_unstale":"2024-06-28T16:10:15.071850+0000","last_undegraded":"2024-06-28T16:10:15.071850+0000","last_fullsized":"2024-06-28T16:10:15.071850+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:38:01.314736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"257'8","reported_seq":71,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:10.064325+0000","last_change":"2024-06-28T15:21:07.010980+0000","last_active":"2024-06-28T16:10:10.064325+0000","last_peered":"2024-06-28T16:10:10.064325+0000","last_clean":"2024-06-28T16:10:10.064325+0000","last_became_active":"2024-06-28T15:21:07.008899+0000","last_became_peered":"2024-06-28T15:21:07.008899+0000","last_unstale":"2024-06-28T16:10:10.064325+0000","last_undegraded":"2024-06-28T16:10:10.064325+0000","last_fullsized":"2024-06-28T16:10:10.064325+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:39:00.266882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":7,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"260'42","reported_seq":166,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:25.320977+0000","last_change":"2024-06-28T15:17:24.020041+0000","last_active":"2024-06-28T16:13:25.320977+0000","last_peered":"2024-06-28T16:13:25.320977+0000","last_clean":"2024-06-28T16:13:25.320977+0000","last_became_active":"2024-06-28T15:17:24.019180+0000","last_became_peered":"2024-06-28T15:17:24.019180+0000","last_unstale":"2024-06-28T16:13:25.320977+0000","last_undegraded":"2024-06-28T16:13:25.320977+0000","last_fullsized":"2024-06-28T16:13:25.320977+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":198,"num_read_kb":139,"num_write":93,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.308181+0000","last_change":"2024-06-28T15:37:33.342459+0000","last_active":"2024-06-28T16:14:35.308181+0000","last_peered":"2024-06-28T16:14:35.308181+0000","last_clean":"2024-06-28T16:14:35.308181+0000","last_became_active":"2024-06-28T15:37:33.341716+0000","last_became_peered":"2024-06-28T15:37:33.341716+0000","last_unstale":"2024-06-28T16:14:35.308181+0000","last_undegraded":"2024-06-28T16:14:35.308181+0000","last_fullsized":"2024-06-28T16:14:35.308181+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.313538+0000","last_change":"2024-06-28T15:21:11.094239+0000","last_active":"2024-06-28T16:12:55.313538+0000","last_peered":"2024-06-28T16:12:55.313538+0000","last_clean":"2024-06-28T16:12:55.313538+0000","last_became_active":"2024-06-28T15:21:11.093276+0000","last_became_peered":"2024-06-28T15:21:11.093276+0000","last_unstale":"2024-06-28T16:12:55.313538+0000","last_undegraded":"2024-06-28T16:12:55.313538+0000","last_fullsized":"2024-06-28T16:12:55.313538+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"260'9","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.368805+0000","last_change":"2024-06-28T15:21:07.041391+0000","last_active":"2024-06-28T16:14:33.368805+0000","last_peered":"2024-06-28T16:14:33.368805+0000","last_clean":"2024-06-28T16:14:33.368805+0000","last_became_active":"2024-06-28T15:21:07.040691+0000","last_became_peered":"2024-06-28T15:21:07.040691+0000","last_unstale":"2024-06-28T16:14:33.368805+0000","last_undegraded":"2024-06-28T16:14:33.368805+0000","last_fullsized":"2024-06-28T16:14:33.368805+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:20:52.003993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":17408,"num_write":9,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"260'47","reported_seq":185,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:50.326913+0000","last_change":"2024-06-28T15:17:24.028252+0000","last_active":"2024-06-28T16:13:50.326913+0000","last_peered":"2024-06-28T16:13:50.326913+0000","last_clean":"2024-06-28T16:13:50.326913+0000","last_became_active":"2024-06-28T15:17:24.027610+0000","last_became_peered":"2024-06-28T15:17:24.027610+0000","last_unstale":"2024-06-28T16:13:50.326913+0000","last_undegraded":"2024-06-28T16:13:50.326913+0000","last_fullsized":"2024-06-28T16:13:50.326913+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":231,"num_read_kb":169,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.094821+0000","last_change":"2024-06-28T15:37:33.347375+0000","last_active":"2024-06-28T16:11:20.094821+0000","last_peered":"2024-06-28T16:11:20.094821+0000","last_clean":"2024-06-28T16:11:20.094821+0000","last_became_active":"2024-06-28T15:37:33.346414+0000","last_became_peered":"2024-06-28T15:37:33.346414+0000","last_unstale":"2024-06-28T16:11:20.094821+0000","last_undegraded":"2024-06-28T16:11:20.094821+0000","last_fullsized":"2024-06-28T16:11:20.094821+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:39:51.219545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"257'5","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.082722+0000","last_change":"2024-06-28T15:22:00.970955+0000","last_active":"2024-06-28T16:11:30.082722+0000","last_peered":"2024-06-28T16:11:30.082722+0000","last_clean":"2024-06-28T16:11:30.082722+0000","last_became_active":"2024-06-28T15:22:00.940760+0000","last_became_peered":"2024-06-28T15:22:00.940760+0000","last_unstale":"2024-06-28T16:11:30.082722+0000","last_undegraded":"2024-06-28T16:11:30.082722+0000","last_fullsized":"2024-06-28T16:11:30.082722+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:14:23.932998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"260'3","reported_seq":57,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.213929+0000","last_change":"2024-06-28T15:21:07.031060+0000","last_active":"2024-06-28T16:14:33.213929+0000","last_peered":"2024-06-28T16:14:33.213929+0000","last_clean":"2024-06-28T16:14:33.213929+0000","last_became_active":"2024-06-28T15:21:07.030163+0000","last_became_peered":"2024-06-28T15:21:07.030163+0000","last_unstale":"2024-06-28T16:14:33.213929+0000","last_undegraded":"2024-06-28T16:14:33.213929+0000","last_fullsized":"2024-06-28T16:14:33.213929+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"260'40","reported_seq":171,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.081619+0000","last_change":"2024-06-28T15:17:24.014719+0000","last_active":"2024-06-28T16:11:25.081619+0000","last_peered":"2024-06-28T16:11:25.081619+0000","last_clean":"2024-06-28T16:11:25.081619+0000","last_became_active":"2024-06-28T15:17:24.012572+0000","last_became_peered":"2024-06-28T15:17:24.012572+0000","last_unstale":"2024-06-28T16:11:25.081619+0000","last_undegraded":"2024-06-28T16:11:25.081619+0000","last_fullsized":"2024-06-28T16:11:25.081619+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:07:56.306439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":148,"num_write":94,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.099240+0000","last_change":"2024-06-28T15:37:33.353545+0000","last_active":"2024-06-28T16:12:15.099240+0000","last_peered":"2024-06-28T16:12:15.099240+0000","last_clean":"2024-06-28T16:12:15.099240+0000","last_became_active":"2024-06-28T15:37:33.352959+0000","last_became_peered":"2024-06-28T15:37:33.352959+0000","last_unstale":"2024-06-28T16:12:15.099240+0000","last_undegraded":"2024-06-28T16:12:15.099240+0000","last_fullsized":"2024-06-28T16:12:15.099240+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:42.580937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.122087+0000","last_change":"2024-06-28T15:21:11.079730+0000","last_active":"2024-06-28T16:13:15.122087+0000","last_peered":"2024-06-28T16:13:15.122087+0000","last_clean":"2024-06-28T16:13:15.122087+0000","last_became_active":"2024-06-28T15:21:11.079036+0000","last_became_peered":"2024-06-28T15:21:11.079036+0000","last_unstale":"2024-06-28T16:13:15.122087+0000","last_undegraded":"2024-06-28T16:13:15.122087+0000","last_fullsized":"2024-06-28T16:13:15.122087+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"0'0","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.584360+0000","last_change":"2024-06-28T15:21:07.028297+0000","last_active":"2024-06-28T16:12:55.584360+0000","last_peered":"2024-06-28T16:12:55.584360+0000","last_clean":"2024-06-28T16:12:55.584360+0000","last_became_active":"2024-06-28T15:21:07.027451+0000","last_became_peered":"2024-06-28T15:21:07.027451+0000","last_unstale":"2024-06-28T16:12:55.584360+0000","last_undegraded":"2024-06-28T16:12:55.584360+0000","last_fullsized":"2024-06-28T16:12:55.584360+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"260'16","reported_seq":97,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.087601+0000","last_change":"2024-06-28T15:17:24.027026+0000","last_active":"2024-06-28T16:11:25.087601+0000","last_peered":"2024-06-28T16:11:25.087601+0000","last_clean":"2024-06-28T16:11:25.087601+0000","last_became_active":"2024-06-28T15:17:24.026271+0000","last_became_peered":"2024-06-28T15:17:24.026271+0000","last_unstale":"2024-06-28T16:11:25.087601+0000","last_undegraded":"2024-06-28T16:11:25.087601+0000","last_fullsized":"2024-06-28T16:11:25.087601+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:23:12.426770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":48,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:15.107001+0000","last_change":"2024-06-28T15:37:33.326639+0000","last_active":"2024-06-28T16:12:15.107001+0000","last_peered":"2024-06-28T16:12:15.107001+0000","last_clean":"2024-06-28T16:12:15.107001+0000","last_became_active":"2024-06-28T15:37:33.325849+0000","last_became_peered":"2024-06-28T15:37:33.325849+0000","last_unstale":"2024-06-28T16:12:15.107001+0000","last_undegraded":"2024-06-28T16:12:15.107001+0000","last_fullsized":"2024-06-28T16:12:15.107001+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:14:13.968785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"257'5","reported_seq":57,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.313575+0000","last_change":"2024-06-28T15:21:11.108377+0000","last_active":"2024-06-28T16:12:55.313575+0000","last_peered":"2024-06-28T16:12:55.313575+0000","last_clean":"2024-06-28T16:12:55.313575+0000","last_became_active":"2024-06-28T15:21:11.107498+0000","last_became_peered":"2024-06-28T15:21:11.107498+0000","last_unstale":"2024-06-28T16:12:55.313575+0000","last_undegraded":"2024-06-28T16:12:55.313575+0000","last_fullsized":"2024-06-28T16:12:55.313575+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"257'7","reported_seq":67,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.120722+0000","last_change":"2024-06-28T15:21:07.030359+0000","last_active":"2024-06-28T16:13:10.120722+0000","last_peered":"2024-06-28T16:13:10.120722+0000","last_clean":"2024-06-28T16:13:10.120722+0000","last_became_active":"2024-06-28T15:21:07.029525+0000","last_became_peered":"2024-06-28T15:21:07.029525+0000","last_unstale":"2024-06-28T16:13:10.120722+0000","last_undegraded":"2024-06-28T16:13:10.120722+0000","last_fullsized":"2024-06-28T16:13:10.120722+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":11,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"260'34","reported_seq":152,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.297023+0000","last_change":"2024-06-28T15:17:24.008356+0000","last_active":"2024-06-28T16:13:40.297023+0000","last_peered":"2024-06-28T16:13:40.297023+0000","last_clean":"2024-06-28T16:13:40.297023+0000","last_became_active":"2024-06-28T15:17:24.007649+0000","last_became_peered":"2024-06-28T15:17:24.007649+0000","last_unstale":"2024-06-28T16:13:40.297023+0000","last_undegraded":"2024-06-28T16:13:40.297023+0000","last_fullsized":"2024-06-28T16:13:40.297023+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":172,"num_read_kb":122,"num_write":76,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:35.323323+0000","last_change":"2024-06-28T15:37:33.335759+0000","last_active":"2024-06-28T16:13:35.323323+0000","last_peered":"2024-06-28T16:13:35.323323+0000","last_clean":"2024-06-28T16:13:35.323323+0000","last_became_active":"2024-06-28T15:37:33.334082+0000","last_became_peered":"2024-06-28T15:37:33.334082+0000","last_unstale":"2024-06-28T16:13:35.323323+0000","last_undegraded":"2024-06-28T16:13:35.323323+0000","last_fullsized":"2024-06-28T16:13:35.323323+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"0'0","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.083900+0000","last_change":"2024-06-28T15:21:11.108205+0000","last_active":"2024-06-28T16:11:05.083900+0000","last_peered":"2024-06-28T16:11:05.083900+0000","last_clean":"2024-06-28T16:11:05.083900+0000","last_became_active":"2024-06-28T15:21:11.107686+0000","last_became_peered":"2024-06-28T15:21:11.107686+0000","last_unstale":"2024-06-28T16:11:05.083900+0000","last_undegraded":"2024-06-28T16:11:05.083900+0000","last_fullsized":"2024-06-28T16:11:05.083900+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:57:49.231179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"0'0","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.588099+0000","last_change":"2024-06-28T15:21:07.033578+0000","last_active":"2024-06-28T16:13:10.588099+0000","last_peered":"2024-06-28T16:13:10.588099+0000","last_clean":"2024-06-28T16:13:10.588099+0000","last_became_active":"2024-06-28T15:21:07.032719+0000","last_became_peered":"2024-06-28T15:21:07.032719+0000","last_unstale":"2024-06-28T16:13:10.588099+0000","last_undegraded":"2024-06-28T16:13:10.588099+0000","last_fullsized":"2024-06-28T16:13:10.588099+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"260'45","reported_seq":191,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.159340+0000","last_change":"2024-06-28T16:14:40.159396+0000","last_active":"2024-06-28T16:14:40.159340+0000","last_peered":"2024-06-28T16:14:40.159340+0000","last_clean":"2024-06-28T16:14:40.159340+0000","last_became_active":"2024-06-28T15:17:24.029532+0000","last_became_peered":"2024-06-28T15:17:24.029532+0000","last_unstale":"2024-06-28T16:14:40.159340+0000","last_undegraded":"2024-06-28T16:14:40.159340+0000","last_fullsized":"2024-06-28T16:14:40.159340+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'45","last_scrub_stamp":"2024-06-28T16:14:40.159221+0000","last_deep_scrub":"260'45","last_deep_scrub_stamp":"2024-06-28T16:14:40.159221+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.159221+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:08:46.885820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":234,"num_read_kb":165,"num_write":86,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.097302+0000","last_change":"2024-06-28T15:37:33.348528+0000","last_active":"2024-06-28T16:11:30.097302+0000","last_peered":"2024-06-28T16:11:30.097302+0000","last_clean":"2024-06-28T16:11:30.097302+0000","last_became_active":"2024-06-28T15:37:33.347691+0000","last_became_peered":"2024-06-28T15:37:33.347691+0000","last_unstale":"2024-06-28T16:11:30.097302+0000","last_undegraded":"2024-06-28T16:11:30.097302+0000","last_fullsized":"2024-06-28T16:11:30.097302+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:19:46.828298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.089675+0000","last_change":"2024-06-28T15:21:11.092385+0000","last_active":"2024-06-28T16:11:00.089675+0000","last_peered":"2024-06-28T16:11:00.089675+0000","last_clean":"2024-06-28T16:11:00.089675+0000","last_became_active":"2024-06-28T15:21:11.091656+0000","last_became_peered":"2024-06-28T15:21:11.091656+0000","last_unstale":"2024-06-28T16:11:00.089675+0000","last_undegraded":"2024-06-28T16:11:00.089675+0000","last_fullsized":"2024-06-28T16:11:00.089675+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:39:05.106486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"260'5","reported_seq":68,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.099809+0000","last_change":"2024-06-28T16:14:39.099851+0000","last_active":"2024-06-28T16:14:39.099809+0000","last_peered":"2024-06-28T16:14:39.099809+0000","last_clean":"2024-06-28T16:14:39.099809+0000","last_became_active":"2024-06-28T15:21:07.029495+0000","last_became_peered":"2024-06-28T15:21:07.029495+0000","last_unstale":"2024-06-28T16:14:39.099809+0000","last_undegraded":"2024-06-28T16:14:39.099809+0000","last_fullsized":"2024-06-28T16:14:39.099809+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'5","last_scrub_stamp":"2024-06-28T16:14:39.099694+0000","last_deep_scrub":"260'5","last_deep_scrub_stamp":"2024-06-28T16:14:39.099694+0000","last_clean_scrub_stamp":"2024-06-28T16:14:39.099694+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:24:20.828667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":318,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"260'39","reported_seq":162,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.101407+0000","last_change":"2024-06-28T15:17:24.024944+0000","last_active":"2024-06-28T16:11:50.101407+0000","last_peered":"2024-06-28T16:11:50.101407+0000","last_clean":"2024-06-28T16:11:50.101407+0000","last_became_active":"2024-06-28T15:17:24.021200+0000","last_became_peered":"2024-06-28T15:17:24.021200+0000","last_unstale":"2024-06-28T16:11:50.101407+0000","last_undegraded":"2024-06-28T16:11:50.101407+0000","last_fullsized":"2024-06-28T16:11:50.101407+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:06:00.686781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":191,"num_read_kb":133,"num_write":87,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"260'1","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:20.094873+0000","last_change":"2024-06-28T15:37:33.336238+0000","last_active":"2024-06-28T16:11:20.094873+0000","last_peered":"2024-06-28T16:11:20.094873+0000","last_clean":"2024-06-28T16:11:20.094873+0000","last_became_active":"2024-06-28T15:37:33.335464+0000","last_became_peered":"2024-06-28T15:37:33.335464+0000","last_unstale":"2024-06-28T16:11:20.094873+0000","last_undegraded":"2024-06-28T16:11:20.094873+0000","last_fullsized":"2024-06-28T16:11:20.094873+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:19:13.910107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"257'3","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.118183+0000","last_change":"2024-06-28T15:21:11.094534+0000","last_active":"2024-06-28T16:13:00.118183+0000","last_peered":"2024-06-28T16:13:00.118183+0000","last_clean":"2024-06-28T16:13:00.118183+0000","last_became_active":"2024-06-28T15:21:11.091647+0000","last_became_peered":"2024-06-28T15:21:11.091647+0000","last_unstale":"2024-06-28T16:13:00.118183+0000","last_undegraded":"2024-06-28T16:13:00.118183+0000","last_fullsized":"2024-06-28T16:13:00.118183+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"257'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.590841+0000","last_change":"2024-06-28T15:21:07.046952+0000","last_active":"2024-06-28T16:13:20.590841+0000","last_peered":"2024-06-28T16:13:20.590841+0000","last_clean":"2024-06-28T16:13:20.590841+0000","last_became_active":"2024-06-28T15:21:07.046218+0000","last_became_peered":"2024-06-28T15:21:07.046218+0000","last_unstale":"2024-06-28T16:13:20.590841+0000","last_undegraded":"2024-06-28T16:13:20.590841+0000","last_fullsized":"2024-06-28T16:13:20.590841+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"260'54","reported_seq":209,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.325858+0000","last_change":"2024-06-28T15:17:24.011432+0000","last_active":"2024-06-28T16:13:45.325858+0000","last_peered":"2024-06-28T16:13:45.325858+0000","last_clean":"2024-06-28T16:13:45.325858+0000","last_became_active":"2024-06-28T15:17:24.010230+0000","last_became_peered":"2024-06-28T15:17:24.010230+0000","last_unstale":"2024-06-28T16:13:45.325858+0000","last_undegraded":"2024-06-28T16:13:45.325858+0000","last_fullsized":"2024-06-28T16:13:45.325858+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":283,"num_read_kb":205,"num_write":97,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:35.594467+0000","last_change":"2024-06-28T15:37:33.343356+0000","last_active":"2024-06-28T16:13:35.594467+0000","last_peered":"2024-06-28T16:13:35.594467+0000","last_clean":"2024-06-28T16:13:35.594467+0000","last_became_active":"2024-06-28T15:37:33.342585+0000","last_became_peered":"2024-06-28T15:37:33.342585+0000","last_unstale":"2024-06-28T16:13:35.594467+0000","last_undegraded":"2024-06-28T16:13:35.594467+0000","last_fullsized":"2024-06-28T16:13:35.594467+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"260'3","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:25.123229+0000","last_change":"2024-06-28T15:21:11.098149+0000","last_active":"2024-06-28T16:13:25.123229+0000","last_peered":"2024-06-28T16:13:25.123229+0000","last_clean":"2024-06-28T16:13:25.123229+0000","last_became_active":"2024-06-28T15:21:11.096884+0000","last_became_peered":"2024-06-28T15:21:11.096884+0000","last_unstale":"2024-06-28T16:13:25.123229+0000","last_undegraded":"2024-06-28T16:13:25.123229+0000","last_fullsized":"2024-06-28T16:13:25.123229+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:53:18.580625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"260'11","reported_seq":79,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.109271+0000","last_change":"2024-06-28T15:21:07.037099+0000","last_active":"2024-06-28T16:12:55.109271+0000","last_peered":"2024-06-28T16:12:55.109271+0000","last_clean":"2024-06-28T16:12:55.109271+0000","last_became_active":"2024-06-28T15:21:07.032545+0000","last_became_peered":"2024-06-28T15:21:07.032545+0000","last_unstale":"2024-06-28T16:12:55.109271+0000","last_undegraded":"2024-06-28T16:12:55.109271+0000","last_fullsized":"2024-06-28T16:12:55.109271+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:28:14.596619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":35842,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"260'40","reported_seq":171,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089612+0000","last_change":"2024-06-28T15:17:24.028816+0000","last_active":"2024-06-28T16:11:35.089612+0000","last_peered":"2024-06-28T16:11:35.089612+0000","last_clean":"2024-06-28T16:11:35.089612+0000","last_became_active":"2024-06-28T15:17:24.026702+0000","last_became_peered":"2024-06-28T15:17:24.026702+0000","last_unstale":"2024-06-28T16:11:35.089612+0000","last_undegraded":"2024-06-28T16:11:35.089612+0000","last_fullsized":"2024-06-28T16:11:35.089612+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:13:09.048722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":150,"num_write":83,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.088350+0000","last_change":"2024-06-28T15:37:33.343533+0000","last_active":"2024-06-28T16:10:55.088350+0000","last_peered":"2024-06-28T16:10:55.088350+0000","last_clean":"2024-06-28T16:10:55.088350+0000","last_became_active":"2024-06-28T15:37:33.342889+0000","last_became_peered":"2024-06-28T15:37:33.342889+0000","last_unstale":"2024-06-28T16:10:55.088350+0000","last_undegraded":"2024-06-28T16:10:55.088350+0000","last_fullsized":"2024-06-28T16:10:55.088350+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:44:29.722036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"260'2","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.477042+0000","last_change":"2024-06-28T15:21:11.093441+0000","last_active":"2024-06-28T16:14:34.477042+0000","last_peered":"2024-06-28T16:14:34.477042+0000","last_clean":"2024-06-28T16:14:34.477042+0000","last_became_active":"2024-06-28T15:21:11.092614+0000","last_became_peered":"2024-06-28T15:21:11.092614+0000","last_unstale":"2024-06-28T16:14:34.477042+0000","last_undegraded":"2024-06-28T16:14:34.477042+0000","last_fullsized":"2024-06-28T16:14:34.477042+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"257'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.092406+0000","last_change":"2024-06-28T15:21:07.035627+0000","last_active":"2024-06-28T16:11:10.092406+0000","last_peered":"2024-06-28T16:11:10.092406+0000","last_clean":"2024-06-28T16:11:10.092406+0000","last_became_active":"2024-06-28T15:21:07.034845+0000","last_became_peered":"2024-06-28T15:21:07.034845+0000","last_unstale":"2024-06-28T16:11:10.092406+0000","last_undegraded":"2024-06-28T16:11:10.092406+0000","last_fullsized":"2024-06-28T16:11:10.092406+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:39:02.467227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"260'28","reported_seq":140,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.596949+0000","last_change":"2024-06-28T15:17:24.016960+0000","last_active":"2024-06-28T16:13:45.596949+0000","last_peered":"2024-06-28T16:13:45.596949+0000","last_clean":"2024-06-28T16:13:45.596949+0000","last_became_active":"2024-06-28T15:17:24.014034+0000","last_became_peered":"2024-06-28T15:17:24.014034+0000","last_unstale":"2024-06-28T16:13:45.596949+0000","last_undegraded":"2024-06-28T16:13:45.596949+0000","last_fullsized":"2024-06-28T16:13:45.596949+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":149,"num_read_kb":105,"num_write":55,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.074736+0000","last_change":"2024-06-28T15:37:33.340550+0000","last_active":"2024-06-28T16:10:55.074736+0000","last_peered":"2024-06-28T16:10:55.074736+0000","last_clean":"2024-06-28T16:10:55.074736+0000","last_became_active":"2024-06-28T15:37:33.339831+0000","last_became_peered":"2024-06-28T15:37:33.339831+0000","last_unstale":"2024-06-28T16:10:55.074736+0000","last_undegraded":"2024-06-28T16:10:55.074736+0000","last_fullsized":"2024-06-28T16:10:55.074736+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:10:11.303905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"260'7","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.436003+0000","last_change":"2024-06-28T15:21:11.096084+0000","last_active":"2024-06-28T16:14:33.436003+0000","last_peered":"2024-06-28T16:14:33.436003+0000","last_clean":"2024-06-28T16:14:33.436003+0000","last_became_active":"2024-06-28T15:21:11.088011+0000","last_became_peered":"2024-06-28T15:21:11.088011+0000","last_unstale":"2024-06-28T16:14:33.436003+0000","last_undegraded":"2024-06-28T16:14:33.436003+0000","last_fullsized":"2024-06-28T16:14:33.436003+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":20522,"num_write":11,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"0'0","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:30.295047+0000","last_change":"2024-06-28T15:21:07.031400+0000","last_active":"2024-06-28T16:13:30.295047+0000","last_peered":"2024-06-28T16:13:30.295047+0000","last_clean":"2024-06-28T16:13:30.295047+0000","last_became_active":"2024-06-28T15:21:07.030690+0000","last_became_peered":"2024-06-28T15:21:07.030690+0000","last_unstale":"2024-06-28T16:13:30.295047+0000","last_undegraded":"2024-06-28T16:13:30.295047+0000","last_fullsized":"2024-06-28T16:13:30.295047+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"260'31","reported_seq":144,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089847+0000","last_change":"2024-06-28T15:17:24.026986+0000","last_active":"2024-06-28T16:11:35.089847+0000","last_peered":"2024-06-28T16:11:35.089847+0000","last_clean":"2024-06-28T16:11:35.089847+0000","last_became_active":"2024-06-28T15:17:24.026404+0000","last_became_peered":"2024-06-28T15:17:24.026404+0000","last_unstale":"2024-06-28T16:11:35.089847+0000","last_undegraded":"2024-06-28T16:11:35.089847+0000","last_fullsized":"2024-06-28T16:11:35.089847+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:08:40.750828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":161,"num_read_kb":111,"num_write":59,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":31,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.127686+0000","last_change":"2024-06-28T15:37:33.332984+0000","last_active":"2024-06-28T16:13:45.127686+0000","last_peered":"2024-06-28T16:13:45.127686+0000","last_clean":"2024-06-28T16:13:45.127686+0000","last_became_active":"2024-06-28T15:37:33.332245+0000","last_became_peered":"2024-06-28T15:37:33.332245+0000","last_unstale":"2024-06-28T16:13:45.127686+0000","last_undegraded":"2024-06-28T16:13:45.127686+0000","last_fullsized":"2024-06-28T16:13:45.127686+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.090010+0000","last_change":"2024-06-28T15:21:11.097568+0000","last_active":"2024-06-28T16:11:00.090010+0000","last_peered":"2024-06-28T16:11:00.090010+0000","last_clean":"2024-06-28T16:11:00.090010+0000","last_became_active":"2024-06-28T15:21:11.096926+0000","last_became_peered":"2024-06-28T15:21:11.096926+0000","last_unstale":"2024-06-28T16:11:00.090010+0000","last_undegraded":"2024-06-28T16:11:00.090010+0000","last_fullsized":"2024-06-28T16:11:00.090010+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:33:16.290705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"260'2","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.105821+0000","last_change":"2024-06-28T15:21:07.037522+0000","last_active":"2024-06-28T16:13:10.105821+0000","last_peered":"2024-06-28T16:13:10.105821+0000","last_clean":"2024-06-28T16:13:10.105821+0000","last_became_active":"2024-06-28T15:21:07.036735+0000","last_became_peered":"2024-06-28T15:21:07.036735+0000","last_unstale":"2024-06-28T16:13:10.105821+0000","last_undegraded":"2024-06-28T16:13:10.105821+0000","last_fullsized":"2024-06-28T16:13:10.105821+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:56:53.756000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"260'49","reported_seq":197,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.101490+0000","last_change":"2024-06-28T15:17:24.026580+0000","last_active":"2024-06-28T16:11:50.101490+0000","last_peered":"2024-06-28T16:11:50.101490+0000","last_clean":"2024-06-28T16:11:50.101490+0000","last_became_active":"2024-06-28T15:17:24.025855+0000","last_became_peered":"2024-06-28T15:17:24.025855+0000","last_unstale":"2024-06-28T16:11:50.101490+0000","last_undegraded":"2024-06-28T16:11:50.101490+0000","last_fullsized":"2024-06-28T16:11:50.101490+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:14:11.100104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":190,"num_write":87,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.081854+0000","last_change":"2024-06-28T15:37:33.323701+0000","last_active":"2024-06-28T16:10:55.081854+0000","last_peered":"2024-06-28T16:10:55.081854+0000","last_clean":"2024-06-28T16:10:55.081854+0000","last_became_active":"2024-06-28T15:37:33.322927+0000","last_became_peered":"2024-06-28T15:37:33.322927+0000","last_unstale":"2024-06-28T16:10:55.081854+0000","last_undegraded":"2024-06-28T16:10:55.081854+0000","last_fullsized":"2024-06-28T16:10:55.081854+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:33:37.474011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.084125+0000","last_change":"2024-06-28T15:21:11.098521+0000","last_active":"2024-06-28T16:11:05.084125+0000","last_peered":"2024-06-28T16:11:05.084125+0000","last_clean":"2024-06-28T16:11:05.084125+0000","last_became_active":"2024-06-28T15:21:11.097766+0000","last_became_peered":"2024-06-28T15:21:11.097766+0000","last_unstale":"2024-06-28T16:11:05.084125+0000","last_undegraded":"2024-06-28T16:11:05.084125+0000","last_fullsized":"2024-06-28T16:11:05.084125+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:49:28.598405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.090053+0000","last_change":"2024-06-28T15:21:07.040142+0000","last_active":"2024-06-28T16:11:00.090053+0000","last_peered":"2024-06-28T16:11:00.090053+0000","last_clean":"2024-06-28T16:11:00.090053+0000","last_became_active":"2024-06-28T15:21:07.039381+0000","last_became_peered":"2024-06-28T15:21:07.039381+0000","last_unstale":"2024-06-28T16:11:00.090053+0000","last_undegraded":"2024-06-28T16:11:00.090053+0000","last_fullsized":"2024-06-28T16:11:00.090053+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:56:40.727136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"260'26","reported_seq":138,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.126731+0000","last_change":"2024-06-28T15:17:24.027314+0000","last_active":"2024-06-28T16:13:40.126731+0000","last_peered":"2024-06-28T16:13:40.126731+0000","last_clean":"2024-06-28T16:13:40.126731+0000","last_became_active":"2024-06-28T15:17:24.026580+0000","last_became_peered":"2024-06-28T15:17:24.026580+0000","last_unstale":"2024-06-28T16:13:40.126731+0000","last_undegraded":"2024-06-28T16:13:40.126731+0000","last_fullsized":"2024-06-28T16:13:40.126731+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":143,"num_read_kb":101,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"259'1","reported_seq":31,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:25.306146+0000","last_change":"2024-06-28T15:37:33.349810+0000","last_active":"2024-06-28T16:14:25.306146+0000","last_peered":"2024-06-28T16:14:25.306146+0000","last_clean":"2024-06-28T16:14:25.306146+0000","last_became_active":"2024-06-28T15:37:33.349137+0000","last_became_peered":"2024-06-28T15:37:33.349137+0000","last_unstale":"2024-06-28T16:14:25.306146+0000","last_undegraded":"2024-06-28T16:14:25.306146+0000","last_fullsized":"2024-06-28T16:14:25.306146+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"260'6","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.517228+0000","last_change":"2024-06-28T15:21:11.106004+0000","last_active":"2024-06-28T16:14:32.517228+0000","last_peered":"2024-06-28T16:14:32.517228+0000","last_clean":"2024-06-28T16:14:32.517228+0000","last_became_active":"2024-06-28T15:21:11.104270+0000","last_became_peered":"2024-06-28T15:21:11.104270+0000","last_unstale":"2024-06-28T16:14:32.517228+0000","last_undegraded":"2024-06-28T16:14:32.517228+0000","last_fullsized":"2024-06-28T16:14:32.517228+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":11,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"260'6","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.311951+0000","last_change":"2024-06-28T15:21:07.049332+0000","last_active":"2024-06-28T16:14:32.311951+0000","last_peered":"2024-06-28T16:14:32.311951+0000","last_clean":"2024-06-28T16:14:32.311951+0000","last_became_active":"2024-06-28T15:21:07.048639+0000","last_became_peered":"2024-06-28T15:21:07.048639+0000","last_unstale":"2024-06-28T16:14:32.311951+0000","last_undegraded":"2024-06-28T16:14:32.311951+0000","last_fullsized":"2024-06-28T16:14:32.311951+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:02:29.473659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":12288,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"260'22","reported_seq":114,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.083766+0000","last_change":"2024-06-28T15:17:24.007520+0000","last_active":"2024-06-28T16:11:35.083766+0000","last_peered":"2024-06-28T16:11:35.083766+0000","last_clean":"2024-06-28T16:11:35.083766+0000","last_became_active":"2024-06-28T15:17:24.006784+0000","last_became_peered":"2024-06-28T15:17:24.006784+0000","last_unstale":"2024-06-28T16:11:35.083766+0000","last_undegraded":"2024-06-28T16:11:35.083766+0000","last_fullsized":"2024-06-28T16:11:35.083766+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:31:01.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":64,"num_write":51,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:25.605834+0000","last_change":"2024-06-28T15:37:33.334905+0000","last_active":"2024-06-28T16:14:25.605834+0000","last_peered":"2024-06-28T16:14:25.605834+0000","last_clean":"2024-06-28T16:14:25.605834+0000","last_became_active":"2024-06-28T15:37:33.333737+0000","last_became_peered":"2024-06-28T15:37:33.333737+0000","last_unstale":"2024-06-28T16:14:25.605834+0000","last_undegraded":"2024-06-28T16:14:25.605834+0000","last_fullsized":"2024-06-28T16:14:25.605834+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":48,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:25.081414+0000","last_change":"2024-06-28T15:21:11.092678+0000","last_active":"2024-06-28T16:11:25.081414+0000","last_peered":"2024-06-28T16:11:25.081414+0000","last_clean":"2024-06-28T16:11:25.081414+0000","last_became_active":"2024-06-28T15:21:11.092034+0000","last_became_peered":"2024-06-28T15:21:11.092034+0000","last_unstale":"2024-06-28T16:11:25.081414+0000","last_undegraded":"2024-06-28T16:11:25.081414+0000","last_fullsized":"2024-06-28T16:11:25.081414+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:08:58.423355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"260'3","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.120874+0000","last_change":"2024-06-28T15:21:07.040425+0000","last_active":"2024-06-28T16:13:15.120874+0000","last_peered":"2024-06-28T16:13:15.120874+0000","last_clean":"2024-06-28T16:13:15.120874+0000","last_became_active":"2024-06-28T15:21:07.039812+0000","last_became_peered":"2024-06-28T15:21:07.039812+0000","last_unstale":"2024-06-28T16:13:15.120874+0000","last_undegraded":"2024-06-28T16:13:15.120874+0000","last_fullsized":"2024-06-28T16:13:15.120874+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:50:26.224970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"260'77","reported_seq":307,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.090583+0000","last_change":"2024-06-28T15:17:24.013637+0000","last_active":"2024-06-28T16:11:35.090583+0000","last_peered":"2024-06-28T16:11:35.090583+0000","last_clean":"2024-06-28T16:11:35.090583+0000","last_became_active":"2024-06-28T15:17:24.012854+0000","last_became_peered":"2024-06-28T15:17:24.012854+0000","last_unstale":"2024-06-28T16:11:35.090583+0000","last_undegraded":"2024-06-28T16:11:35.090583+0000","last_fullsized":"2024-06-28T16:11:35.090583+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:10:40.982270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":483,"num_read_kb":330,"num_write":109,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.585523+0000","last_change":"2024-06-28T15:37:33.323089+0000","last_active":"2024-06-28T16:13:00.585523+0000","last_peered":"2024-06-28T16:13:00.585523+0000","last_clean":"2024-06-28T16:13:00.585523+0000","last_became_active":"2024-06-28T15:37:33.322317+0000","last_became_peered":"2024-06-28T15:37:33.322317+0000","last_unstale":"2024-06-28T16:13:00.585523+0000","last_undegraded":"2024-06-28T16:13:00.585523+0000","last_fullsized":"2024-06-28T16:13:00.585523+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.085392+0000","last_change":"2024-06-28T15:21:11.109416+0000","last_active":"2024-06-28T16:11:15.085392+0000","last_peered":"2024-06-28T16:11:15.085392+0000","last_clean":"2024-06-28T16:11:15.085392+0000","last_became_active":"2024-06-28T15:21:11.108663+0000","last_became_peered":"2024-06-28T15:21:11.108663+0000","last_unstale":"2024-06-28T16:11:15.085392+0000","last_undegraded":"2024-06-28T16:11:15.085392+0000","last_fullsized":"2024-06-28T16:11:15.085392+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:07:02.854303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"260'3","reported_seq":67,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.135088+0000","last_change":"2024-06-28T16:14:40.135141+0000","last_active":"2024-06-28T16:14:40.135088+0000","last_peered":"2024-06-28T16:14:40.135088+0000","last_clean":"2024-06-28T16:14:40.135088+0000","last_became_active":"2024-06-28T15:21:07.022046+0000","last_became_peered":"2024-06-28T15:21:07.022046+0000","last_unstale":"2024-06-28T16:14:40.135088+0000","last_undegraded":"2024-06-28T16:14:40.135088+0000","last_fullsized":"2024-06-28T16:14:40.135088+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:14:40.134988+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:14:40.134988+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.134988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:06:49.985940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"260'31","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089684+0000","last_change":"2024-06-28T15:17:24.011733+0000","last_active":"2024-06-28T16:11:35.089684+0000","last_peered":"2024-06-28T16:11:35.089684+0000","last_clean":"2024-06-28T16:11:35.089684+0000","last_became_active":"2024-06-28T15:17:24.010873+0000","last_became_peered":"2024-06-28T15:17:24.010873+0000","last_unstale":"2024-06-28T16:11:35.089684+0000","last_undegraded":"2024-06-28T16:11:35.089684+0000","last_fullsized":"2024-06-28T16:11:35.089684+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:29:45.592863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":94,"num_write":78,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:05.091280+0000","last_change":"2024-06-28T15:37:33.347898+0000","last_active":"2024-06-28T16:11:05.091280+0000","last_peered":"2024-06-28T16:11:05.091280+0000","last_clean":"2024-06-28T16:11:05.091280+0000","last_became_active":"2024-06-28T15:37:33.347075+0000","last_became_peered":"2024-06-28T15:37:33.347075+0000","last_unstale":"2024-06-28T16:11:05.091280+0000","last_undegraded":"2024-06-28T16:11:05.091280+0000","last_fullsized":"2024-06-28T16:11:05.091280+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:36:09.889868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":47,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.086272+0000","last_change":"2024-06-28T15:21:11.093738+0000","last_active":"2024-06-28T16:11:15.086272+0000","last_peered":"2024-06-28T16:11:15.086272+0000","last_clean":"2024-06-28T16:11:15.086272+0000","last_became_active":"2024-06-28T15:21:11.093082+0000","last_became_peered":"2024-06-28T15:21:11.093082+0000","last_unstale":"2024-06-28T16:11:15.086272+0000","last_undegraded":"2024-06-28T16:11:15.086272+0000","last_fullsized":"2024-06-28T16:11:15.086272+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:58:56.828729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"257'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.084078+0000","last_change":"2024-06-28T15:21:07.041597+0000","last_active":"2024-06-28T16:11:10.084078+0000","last_peered":"2024-06-28T16:11:10.084078+0000","last_clean":"2024-06-28T16:11:10.084078+0000","last_became_active":"2024-06-28T15:21:07.040575+0000","last_became_peered":"2024-06-28T15:21:07.040575+0000","last_unstale":"2024-06-28T16:11:10.084078+0000","last_undegraded":"2024-06-28T16:11:10.084078+0000","last_fullsized":"2024-06-28T16:11:10.084078+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:41:33.611919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"260'16","reported_seq":98,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:40.099162+0000","last_change":"2024-06-28T15:17:24.024775+0000","last_active":"2024-06-28T16:11:40.099162+0000","last_peered":"2024-06-28T16:11:40.099162+0000","last_clean":"2024-06-28T16:11:40.099162+0000","last_became_active":"2024-06-28T15:17:24.021464+0000","last_became_peered":"2024-06-28T15:17:24.021464+0000","last_unstale":"2024-06-28T16:11:40.099162+0000","last_undegraded":"2024-06-28T16:11:40.099162+0000","last_fullsized":"2024-06-28T16:11:40.099162+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:17:22.969363+0000","last_clean_scrub_stamp":"2024-06-28T15:17:22.969363+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:33:59.157965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":44,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"260'341","reported_seq":963,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.690043+0000","last_change":"2024-06-28T15:00:59.944074+0000","last_active":"2024-06-28T16:14:34.690043+0000","last_peered":"2024-06-28T16:14:34.690043+0000","last_clean":"2024-06-28T16:14:34.690043+0000","last_became_active":"2024-06-28T15:00:59.941681+0000","last_became_peered":"2024-06-28T15:00:59.941681+0000","last_unstale":"2024-06-28T16:14:34.690043+0000","last_undegraded":"2024-06-28T16:14:34.690043+0000","last_fullsized":"2024-06-28T16:14:34.690043+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170282,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":488,"num_read_kb":26898,"num_write":1268,"num_write_kb":25985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0,5],"acting":[2,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"260'181","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.066576+0000","last_change":"2024-06-28T15:00:17.405026+0000","last_active":"2024-06-28T16:10:20.066576+0000","last_peered":"2024-06-28T16:10:20.066576+0000","last_clean":"2024-06-28T16:10:20.066576+0000","last_became_active":"2024-06-28T15:00:17.404342+0000","last_became_peered":"2024-06-28T15:00:17.404342+0000","last_unstale":"2024-06-28T16:10:20.066576+0000","last_undegraded":"2024-06-28T16:10:20.066576+0000","last_fullsized":"2024-06-28T16:10:20.066576+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":100,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:24:42.574899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":722,"num_read_kb":569,"num_write":267,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"260'326","reported_seq":972,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.743327+0000","last_change":"2024-06-28T15:00:15.419867+0000","last_active":"2024-06-28T16:14:34.743327+0000","last_peered":"2024-06-28T16:14:34.743327+0000","last_clean":"2024-06-28T16:14:34.743327+0000","last_became_active":"2024-06-28T15:00:15.419449+0000","last_became_peered":"2024-06-28T15:00:15.419449+0000","last_unstale":"2024-06-28T16:14:34.743327+0000","last_undegraded":"2024-06-28T16:14:34.743327+0000","last_fullsized":"2024-06-28T16:14:34.743327+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5526738,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":485,"num_read_kb":15789,"num_write":1172,"num_write_kb":37304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"260'285","reported_seq":939,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080422+0000","last_change":"2024-06-28T15:00:17.409108+0000","last_active":"2024-06-28T16:10:20.080422+0000","last_peered":"2024-06-28T16:10:20.080422+0000","last_clean":"2024-06-28T16:10:20.080422+0000","last_became_active":"2024-06-28T15:00:17.408128+0000","last_became_peered":"2024-06-28T15:00:17.408128+0000","last_unstale":"2024-06-28T16:10:20.080422+0000","last_undegraded":"2024-06-28T16:10:20.080422+0000","last_fullsized":"2024-06-28T16:10:20.080422+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:38:12.539083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1157,"num_read_kb":911,"num_write":439,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"260'283","reported_seq":970,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.735843+0000","last_change":"2024-06-28T15:00:15.397113+0000","last_active":"2024-06-28T16:14:34.735843+0000","last_peered":"2024-06-28T16:14:34.735843+0000","last_clean":"2024-06-28T16:14:34.735843+0000","last_became_active":"2024-06-28T15:00:15.396469+0000","last_became_peered":"2024-06-28T15:00:15.396469+0000","last_unstale":"2024-06-28T16:14:34.735843+0000","last_undegraded":"2024-06-28T16:14:34.735843+0000","last_fullsized":"2024-06-28T16:14:34.735843+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":200,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1096214,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":504,"num_read_kb":81437,"num_write":1108,"num_write_kb":14924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"260'77","reported_seq":643,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073900+0000","last_change":"2024-06-28T15:00:17.403326+0000","last_active":"2024-06-28T16:10:20.073900+0000","last_peered":"2024-06-28T16:10:20.073900+0000","last_clean":"2024-06-28T16:10:20.073900+0000","last_became_active":"2024-06-28T15:00:17.402542+0000","last_became_peered":"2024-06-28T15:00:17.402542+0000","last_unstale":"2024-06-28T16:10:20.073900+0000","last_undegraded":"2024-06-28T16:10:20.073900+0000","last_fullsized":"2024-06-28T16:10:20.073900+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:05:41.791423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":306,"num_read_kb":248,"num_write":126,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"260'313","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.510107+0000","last_change":"2024-06-28T15:00:15.435754+0000","last_active":"2024-06-28T16:14:34.510107+0000","last_peered":"2024-06-28T16:14:34.510107+0000","last_clean":"2024-06-28T16:14:34.510107+0000","last_became_active":"2024-06-28T15:00:15.435122+0000","last_became_peered":"2024-06-28T15:00:15.435122+0000","last_unstale":"2024-06-28T16:14:34.510107+0000","last_undegraded":"2024-06-28T16:14:34.510107+0000","last_fullsized":"2024-06-28T16:14:34.510107+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:23:37.223417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1454317,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":492,"num_read_kb":48678,"num_write":1275,"num_write_kb":34259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"260'51","reported_seq":630,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.582691+0000","last_change":"2024-06-28T15:00:17.385309+0000","last_active":"2024-06-28T16:12:50.582691+0000","last_peered":"2024-06-28T16:12:50.582691+0000","last_clean":"2024-06-28T16:12:50.582691+0000","last_became_active":"2024-06-28T15:00:17.384932+0000","last_became_peered":"2024-06-28T15:00:17.384932+0000","last_unstale":"2024-06-28T16:12:50.582691+0000","last_undegraded":"2024-06-28T16:12:50.582691+0000","last_fullsized":"2024-06-28T16:12:50.582691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":197,"num_write":67,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"260'343","reported_seq":992,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.679630+0000","last_change":"2024-06-28T15:00:15.430784+0000","last_active":"2024-06-28T16:14:34.679630+0000","last_peered":"2024-06-28T16:14:34.679630+0000","last_clean":"2024-06-28T16:14:34.679630+0000","last_became_active":"2024-06-28T15:00:15.430208+0000","last_became_peered":"2024-06-28T15:00:15.430208+0000","last_unstale":"2024-06-28T16:14:34.679630+0000","last_undegraded":"2024-06-28T16:14:34.679630+0000","last_fullsized":"2024-06-28T16:14:34.679630+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:48:53.859374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437780,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":548,"num_read_kb":39482,"num_write":1215,"num_write_kb":35176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"260'185","reported_seq":838,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:40.104808+0000","last_change":"2024-06-28T15:00:17.398844+0000","last_active":"2024-06-28T16:12:40.104808+0000","last_peered":"2024-06-28T16:12:40.104808+0000","last_clean":"2024-06-28T16:12:40.104808+0000","last_became_active":"2024-06-28T15:00:17.397986+0000","last_became_peered":"2024-06-28T15:00:17.397986+0000","last_unstale":"2024-06-28T16:12:40.104808+0000","last_undegraded":"2024-06-28T16:12:40.104808+0000","last_fullsized":"2024-06-28T16:12:40.104808+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":100,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:49:08.729457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":777,"num_read_kb":632,"num_write":274,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"260'364","reported_seq":1029,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.692915+0000","last_change":"2024-06-28T15:00:15.412175+0000","last_active":"2024-06-28T16:14:34.692915+0000","last_peered":"2024-06-28T16:14:34.692915+0000","last_clean":"2024-06-28T16:14:34.692915+0000","last_became_active":"2024-06-28T15:00:15.409542+0000","last_became_peered":"2024-06-28T15:00:15.409542+0000","last_unstale":"2024-06-28T16:14:34.692915+0000","last_undegraded":"2024-06-28T16:14:34.692915+0000","last_fullsized":"2024-06-28T16:14:34.692915+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:28:00.331354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1667748,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":553,"num_read_kb":16754,"num_write":1370,"num_write_kb":32794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"260'44","reported_seq":608,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.895392+0000","last_change":"2024-06-28T15:00:17.408359+0000","last_active":"2024-06-28T16:11:10.895392+0000","last_peered":"2024-06-28T16:11:10.895392+0000","last_clean":"2024-06-28T16:11:10.895392+0000","last_became_active":"2024-06-28T15:00:17.401985+0000","last_became_peered":"2024-06-28T15:00:17.401985+0000","last_unstale":"2024-06-28T16:11:10.895392+0000","last_undegraded":"2024-06-28T16:11:10.895392+0000","last_fullsized":"2024-06-28T16:11:10.895392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:00:57.912525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":159,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"260'328","reported_seq":959,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.675966+0000","last_change":"2024-06-28T15:00:15.425314+0000","last_active":"2024-06-28T16:14:34.675966+0000","last_peered":"2024-06-28T16:14:34.675966+0000","last_clean":"2024-06-28T16:14:34.675966+0000","last_became_active":"2024-06-28T15:00:15.424430+0000","last_became_peered":"2024-06-28T15:00:15.424430+0000","last_unstale":"2024-06-28T16:14:34.675966+0000","last_undegraded":"2024-06-28T16:14:34.675966+0000","last_fullsized":"2024-06-28T16:14:34.675966+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:30:59.580102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048602,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":476,"num_read_kb":21908,"num_write":1069,"num_write_kb":36265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"260'4326","reported_seq":4996,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:30.604647+0000","last_change":"2024-06-28T15:00:17.387801+0000","last_active":"2024-06-28T16:12:30.604647+0000","last_peered":"2024-06-28T16:12:30.604647+0000","last_clean":"2024-06-28T16:12:30.604647+0000","last_became_active":"2024-06-28T15:00:17.384378+0000","last_became_peered":"2024-06-28T15:00:17.384378+0000","last_unstale":"2024-06-28T16:12:30.604647+0000","last_undegraded":"2024-06-28T16:12:30.604647+0000","last_fullsized":"2024-06-28T16:12:30.604647+0000","mapping_epoch":21,"log_start":"260'4300","ondisk_log_start":"260'4300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":3000,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:59:21.316239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15352,"num_read_kb":12408,"num_write":6557,"num_write_kb":3277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"260'411","reported_seq":1129,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.680578+0000","last_change":"2024-06-28T15:00:15.403350+0000","last_active":"2024-06-28T16:14:34.680578+0000","last_peered":"2024-06-28T16:14:34.680578+0000","last_clean":"2024-06-28T16:14:34.680578+0000","last_became_active":"2024-06-28T15:00:15.401960+0000","last_became_peered":"2024-06-28T15:00:15.401960+0000","last_unstale":"2024-06-28T16:14:34.680578+0000","last_undegraded":"2024-06-28T16:14:34.680578+0000","last_fullsized":"2024-06-28T16:14:34.680578+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":400,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324665,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":810,"num_read_kb":20615,"num_write":1466,"num_write_kb":29750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"260'70","reported_seq":686,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.300073+0000","last_change":"2024-06-28T15:00:17.381546+0000","last_active":"2024-06-28T16:14:08.300073+0000","last_peered":"2024-06-28T16:14:08.300073+0000","last_clean":"2024-06-28T16:14:08.300073+0000","last_became_active":"2024-06-28T15:00:17.381196+0000","last_became_peered":"2024-06-28T15:00:17.381196+0000","last_unstale":"2024-06-28T16:14:08.300073+0000","last_undegraded":"2024-06-28T16:14:08.300073+0000","last_fullsized":"2024-06-28T16:14:08.300073+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":309,"num_read_kb":273,"num_write":88,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"260'128","reported_seq":750,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:32.095965+0000","last_change":"2024-06-28T15:00:17.396995+0000","last_active":"2024-06-28T16:13:32.095965+0000","last_peered":"2024-06-28T16:13:32.095965+0000","last_clean":"2024-06-28T16:13:32.095965+0000","last_became_active":"2024-06-28T15:00:17.396142+0000","last_became_peered":"2024-06-28T15:00:17.396142+0000","last_unstale":"2024-06-28T16:13:32.095965+0000","last_undegraded":"2024-06-28T16:13:32.095965+0000","last_fullsized":"2024-06-28T16:13:32.095965+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:43:00.829791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":536,"num_read_kb":452,"num_write":191,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"260'416","reported_seq":1186,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.679447+0000","last_change":"2024-06-28T15:00:15.423248+0000","last_active":"2024-06-28T16:14:34.679447+0000","last_peered":"2024-06-28T16:14:34.679447+0000","last_clean":"2024-06-28T16:14:34.679447+0000","last_became_active":"2024-06-28T15:00:15.422579+0000","last_became_peered":"2024-06-28T15:00:15.422579+0000","last_unstale":"2024-06-28T16:14:34.679447+0000","last_undegraded":"2024-06-28T16:14:34.679447+0000","last_fullsized":"2024-06-28T16:14:34.679447+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":400,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:59:30.311378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":851,"num_read_kb":96606,"num_write":1464,"num_write_kb":38692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0,1],"acting":[7,5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"260'254","reported_seq":899,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080457+0000","last_change":"2024-06-28T15:00:17.410347+0000","last_active":"2024-06-28T16:10:20.080457+0000","last_peered":"2024-06-28T16:10:20.080457+0000","last_clean":"2024-06-28T16:10:20.080457+0000","last_became_active":"2024-06-28T15:00:17.409202+0000","last_became_peered":"2024-06-28T15:00:17.409202+0000","last_unstale":"2024-06-28T16:10:20.080457+0000","last_undegraded":"2024-06-28T16:10:20.080457+0000","last_fullsized":"2024-06-28T16:10:20.080457+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:33:21.410367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1059,"num_read_kb":851,"num_write":432,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"260'313","reported_seq":952,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.711448+0000","last_change":"2024-06-28T15:00:15.423492+0000","last_active":"2024-06-28T16:14:34.711448+0000","last_peered":"2024-06-28T16:14:34.711448+0000","last_clean":"2024-06-28T16:14:34.711448+0000","last_became_active":"2024-06-28T15:00:15.421980+0000","last_became_peered":"2024-06-28T15:00:15.421980+0000","last_unstale":"2024-06-28T16:14:34.711448+0000","last_undegraded":"2024-06-28T16:14:34.711448+0000","last_fullsized":"2024-06-28T16:14:34.711448+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:15:16.139016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468838,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":425,"num_read_kb":33937,"num_write":1026,"num_write_kb":43346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"260'225","reported_seq":850,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:35.070101+0000","last_change":"2024-06-28T15:00:17.385629+0000","last_active":"2024-06-28T16:10:35.070101+0000","last_peered":"2024-06-28T16:10:35.070101+0000","last_clean":"2024-06-28T16:10:35.070101+0000","last_became_active":"2024-06-28T15:00:17.384914+0000","last_became_peered":"2024-06-28T15:00:17.384914+0000","last_unstale":"2024-06-28T16:10:35.070101+0000","last_undegraded":"2024-06-28T16:10:35.070101+0000","last_fullsized":"2024-06-28T16:10:35.070101+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:01:45.177989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":887,"num_read_kb":706,"num_write":342,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"260'342","reported_seq":1191,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.723544+0000","last_change":"2024-06-28T15:00:15.412640+0000","last_active":"2024-06-28T16:14:34.723544+0000","last_peered":"2024-06-28T16:14:34.723544+0000","last_clean":"2024-06-28T16:14:34.723544+0000","last_became_active":"2024-06-28T15:00:15.411567+0000","last_became_peered":"2024-06-28T15:00:15.411567+0000","last_unstale":"2024-06-28T16:14:34.723544+0000","last_undegraded":"2024-06-28T16:14:34.723544+0000","last_fullsized":"2024-06-28T16:14:34.723544+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:32:11.605469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289237,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":936,"num_read_kb":14303,"num_write":1297,"num_write_kb":46873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"260'128","reported_seq":737,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.286422+0000","last_change":"2024-06-28T15:00:17.389926+0000","last_active":"2024-06-28T16:12:50.286422+0000","last_peered":"2024-06-28T16:12:50.286422+0000","last_clean":"2024-06-28T16:12:50.286422+0000","last_became_active":"2024-06-28T15:00:17.389613+0000","last_became_peered":"2024-06-28T15:00:17.389613+0000","last_unstale":"2024-06-28T16:12:50.286422+0000","last_undegraded":"2024-06-28T16:12:50.286422+0000","last_fullsized":"2024-06-28T16:12:50.286422+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":419,"num_write":192,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"260'320","reported_seq":1072,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.719218+0000","last_change":"2024-06-28T15:00:15.418207+0000","last_active":"2024-06-28T16:14:34.719218+0000","last_peered":"2024-06-28T16:14:34.719218+0000","last_clean":"2024-06-28T16:14:34.719218+0000","last_became_active":"2024-06-28T15:00:15.417563+0000","last_became_peered":"2024-06-28T15:00:15.417563+0000","last_unstale":"2024-06-28T16:14:34.719218+0000","last_undegraded":"2024-06-28T16:14:34.719218+0000","last_fullsized":"2024-06-28T16:14:34.719218+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:50:35.799169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":525349,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":605,"num_read_kb":12860,"num_write":1159,"num_write_kb":18563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"260'156","reported_seq":738,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.286500+0000","last_change":"2024-06-28T15:00:17.413709+0000","last_active":"2024-06-28T16:12:50.286500+0000","last_peered":"2024-06-28T16:12:50.286500+0000","last_clean":"2024-06-28T16:12:50.286500+0000","last_became_active":"2024-06-28T15:00:17.413420+0000","last_became_peered":"2024-06-28T15:00:17.413420+0000","last_unstale":"2024-06-28T16:12:50.286500+0000","last_undegraded":"2024-06-28T16:12:50.286500+0000","last_fullsized":"2024-06-28T16:12:50.286500+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":589,"num_read_kb":466,"num_write":222,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"260'357","reported_seq":1074,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.742055+0000","last_change":"2024-06-28T15:00:15.402501+0000","last_active":"2024-06-28T16:14:34.742055+0000","last_peered":"2024-06-28T16:14:34.742055+0000","last_clean":"2024-06-28T16:14:34.742055+0000","last_became_active":"2024-06-28T15:00:15.401370+0000","last_became_peered":"2024-06-28T15:00:15.401370+0000","last_unstale":"2024-06-28T16:14:34.742055+0000","last_undegraded":"2024-06-28T16:14:34.742055+0000","last_fullsized":"2024-06-28T16:14:34.742055+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324689,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":676,"num_read_kb":30262,"num_write":1536,"num_write_kb":36456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"260'253","reported_seq":902,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.053742+0000","last_change":"2024-06-28T15:00:17.410230+0000","last_active":"2024-06-28T16:13:20.053742+0000","last_peered":"2024-06-28T16:13:20.053742+0000","last_clean":"2024-06-28T16:13:20.053742+0000","last_became_active":"2024-06-28T15:00:17.408972+0000","last_became_peered":"2024-06-28T15:00:17.408972+0000","last_unstale":"2024-06-28T16:13:20.053742+0000","last_undegraded":"2024-06-28T16:13:20.053742+0000","last_fullsized":"2024-06-28T16:13:20.053742+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:48:01.967992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":990,"num_read_kb":787,"num_write":391,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"260'346","reported_seq":1009,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.742591+0000","last_change":"2024-06-28T15:00:15.408546+0000","last_active":"2024-06-28T16:14:34.742591+0000","last_peered":"2024-06-28T16:14:34.742591+0000","last_clean":"2024-06-28T16:14:34.742591+0000","last_became_active":"2024-06-28T15:00:15.407552+0000","last_became_peered":"2024-06-28T15:00:15.407552+0000","last_unstale":"2024-06-28T16:14:34.742591+0000","last_undegraded":"2024-06-28T16:14:34.742591+0000","last_fullsized":"2024-06-28T16:14:34.742591+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":300,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278750,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":513,"num_read_kb":23448,"num_write":1336,"num_write_kb":46171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"260'142","reported_seq":679,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.510437+0000","last_change":"2024-06-28T15:00:25.428551+0000","last_active":"2024-06-28T16:14:08.510437+0000","last_peered":"2024-06-28T16:14:08.510437+0000","last_clean":"2024-06-28T16:14:08.510437+0000","last_became_active":"2024-06-28T15:00:25.427734+0000","last_became_peered":"2024-06-28T15:00:25.427734+0000","last_unstale":"2024-06-28T16:14:08.510437+0000","last_undegraded":"2024-06-28T16:14:08.510437+0000","last_fullsized":"2024-06-28T16:14:08.510437+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:59:38.479139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":233,"num_read_kb":201,"num_write":347,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"260'214","reported_seq":823,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.312228+0000","last_change":"2024-06-28T15:00:17.382620+0000","last_active":"2024-06-28T16:12:50.312228+0000","last_peered":"2024-06-28T16:12:50.312228+0000","last_clean":"2024-06-28T16:12:50.312228+0000","last_became_active":"2024-06-28T15:00:17.382344+0000","last_became_peered":"2024-06-28T15:00:17.382344+0000","last_unstale":"2024-06-28T16:12:50.312228+0000","last_undegraded":"2024-06-28T16:12:50.312228+0000","last_fullsized":"2024-06-28T16:12:50.312228+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":200,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":869,"num_read_kb":692,"num_write":351,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"260'292","reported_seq":944,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.632189+0000","last_change":"2024-06-28T15:00:15.423784+0000","last_active":"2024-06-28T16:14:34.632189+0000","last_peered":"2024-06-28T16:14:34.632189+0000","last_clean":"2024-06-28T16:14:34.632189+0000","last_became_active":"2024-06-28T15:00:15.420739+0000","last_became_peered":"2024-06-28T15:00:15.420739+0000","last_unstale":"2024-06-28T16:14:34.632189+0000","last_undegraded":"2024-06-28T16:14:34.632189+0000","last_fullsized":"2024-06-28T16:14:34.632189+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":200,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:38:08.814015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6022775,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":452,"num_read_kb":28797,"num_write":1102,"num_write_kb":30225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.067917+0000","last_change":"2024-06-28T15:00:19.403941+0000","last_active":"2024-06-28T16:10:25.067917+0000","last_peered":"2024-06-28T16:10:25.067917+0000","last_clean":"2024-06-28T16:10:25.067917+0000","last_became_active":"2024-06-28T15:00:19.400681+0000","last_became_peered":"2024-06-28T15:00:19.400681+0000","last_unstale":"2024-06-28T16:10:25.067917+0000","last_undegraded":"2024-06-28T16:10:25.067917+0000","last_fullsized":"2024-06-28T16:10:25.067917+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:52:42.469346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"260'324","reported_seq":949,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.645529+0000","last_change":"2024-06-28T15:00:21.391364+0000","last_active":"2024-06-28T16:14:33.645529+0000","last_peered":"2024-06-28T16:14:33.645529+0000","last_clean":"2024-06-28T16:14:33.645529+0000","last_became_active":"2024-06-28T15:00:21.390609+0000","last_became_peered":"2024-06-28T15:00:21.390609+0000","last_unstale":"2024-06-28T16:14:33.645529+0000","last_undegraded":"2024-06-28T16:14:33.645529+0000","last_fullsized":"2024-06-28T16:14:33.645529+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42630,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":530,"num_read_kb":1085,"num_write":406,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":493,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.082982+0000","last_change":"2024-06-28T15:00:23.408294+0000","last_active":"2024-06-28T16:10:30.082982+0000","last_peered":"2024-06-28T16:10:30.082982+0000","last_clean":"2024-06-28T16:10:30.082982+0000","last_became_active":"2024-06-28T15:00:23.407521+0000","last_became_peered":"2024-06-28T15:00:23.407521+0000","last_unstale":"2024-06-28T16:10:30.082982+0000","last_undegraded":"2024-06-28T16:10:30.082982+0000","last_fullsized":"2024-06-28T16:10:30.082982+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:06.922239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"260'2402","reported_seq":4268,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:02.134637+0000","last_change":"2024-06-28T15:00:25.440728+0000","last_active":"2024-06-28T16:14:02.134637+0000","last_peered":"2024-06-28T16:14:02.134637+0000","last_clean":"2024-06-28T16:14:02.134637+0000","last_became_active":"2024-06-28T15:00:25.440083+0000","last_became_peered":"2024-06-28T15:00:25.440083+0000","last_unstale":"2024-06-28T16:14:02.134637+0000","last_undegraded":"2024-06-28T16:14:02.134637+0000","last_fullsized":"2024-06-28T16:14:02.134637+0000","mapping_epoch":29,"log_start":"260'2300","ondisk_log_start":"260'2300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":2300,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:54:57.315079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5966,"num_read_kb":5973,"num_write":4800,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"260'4179","reported_seq":4823,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.825089+0000","last_change":"2024-06-28T15:00:17.390976+0000","last_active":"2024-06-28T16:11:03.825089+0000","last_peered":"2024-06-28T16:11:03.825089+0000","last_clean":"2024-06-28T16:11:03.825089+0000","last_became_active":"2024-06-28T15:00:17.390254+0000","last_became_peered":"2024-06-28T15:00:17.390254+0000","last_unstale":"2024-06-28T16:11:03.825089+0000","last_undegraded":"2024-06-28T16:11:03.825089+0000","last_fullsized":"2024-06-28T16:11:03.825089+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":3000,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:50:01.744884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14764,"num_read_kb":11968,"num_write":6274,"num_write_kb":3129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"260'343","reported_seq":1012,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.712958+0000","last_change":"2024-06-28T15:00:15.397404+0000","last_active":"2024-06-28T16:14:34.712958+0000","last_peered":"2024-06-28T16:14:34.712958+0000","last_clean":"2024-06-28T16:14:34.712958+0000","last_became_active":"2024-06-28T15:00:15.396724+0000","last_became_peered":"2024-06-28T15:00:15.396724+0000","last_unstale":"2024-06-28T16:14:34.712958+0000","last_undegraded":"2024-06-28T16:14:34.712958+0000","last_fullsized":"2024-06-28T16:14:34.712958+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262185,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":532,"num_read_kb":7737,"num_write":1235,"num_write_kb":8076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.312275+0000","last_change":"2024-06-28T15:00:19.390766+0000","last_active":"2024-06-28T16:12:50.312275+0000","last_peered":"2024-06-28T16:12:50.312275+0000","last_clean":"2024-06-28T16:12:50.312275+0000","last_became_active":"2024-06-28T15:00:19.390100+0000","last_became_peered":"2024-06-28T15:00:19.390100+0000","last_unstale":"2024-06-28T16:12:50.312275+0000","last_undegraded":"2024-06-28T16:12:50.312275+0000","last_fullsized":"2024-06-28T16:12:50.312275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"260'123","reported_seq":662,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.316301+0000","last_change":"2024-06-28T15:00:21.374873+0000","last_active":"2024-06-28T16:11:03.316301+0000","last_peered":"2024-06-28T16:11:03.316301+0000","last_clean":"2024-06-28T16:11:03.316301+0000","last_became_active":"2024-06-28T15:00:21.374284+0000","last_became_peered":"2024-06-28T15:00:21.374284+0000","last_unstale":"2024-06-28T16:11:03.316301+0000","last_undegraded":"2024-06-28T16:11:03.316301+0000","last_fullsized":"2024-06-28T16:11:03.316301+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:17:08.459695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":127,"num_read_kb":120,"num_write":90,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":495,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:30.076584+0000","last_change":"2024-06-28T15:00:23.399941+0000","last_active":"2024-06-28T16:10:30.076584+0000","last_peered":"2024-06-28T16:10:30.076584+0000","last_clean":"2024-06-28T16:10:30.076584+0000","last_became_active":"2024-06-28T15:00:23.399147+0000","last_became_peered":"2024-06-28T15:00:23.399147+0000","last_unstale":"2024-06-28T16:10:30.076584+0000","last_undegraded":"2024-06-28T16:10:30.076584+0000","last_fullsized":"2024-06-28T16:10:30.076584+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:00:35.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"260'123","reported_seq":650,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.676839+0000","last_change":"2024-06-28T15:00:25.433474+0000","last_active":"2024-06-28T16:14:06.676839+0000","last_peered":"2024-06-28T16:14:06.676839+0000","last_clean":"2024-06-28T16:14:06.676839+0000","last_became_active":"2024-06-28T15:00:25.432898+0000","last_became_peered":"2024-06-28T15:00:25.432898+0000","last_unstale":"2024-06-28T16:14:06.676839+0000","last_undegraded":"2024-06-28T16:14:06.676839+0000","last_fullsized":"2024-06-28T16:14:06.676839+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":165,"num_write":302,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"260'155","reported_seq":777,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.582816+0000","last_change":"2024-06-28T15:00:17.384953+0000","last_active":"2024-06-28T16:12:50.582816+0000","last_peered":"2024-06-28T16:12:50.582816+0000","last_clean":"2024-06-28T16:12:50.582816+0000","last_became_active":"2024-06-28T15:00:17.384613+0000","last_became_peered":"2024-06-28T15:00:17.384613+0000","last_unstale":"2024-06-28T16:12:50.582816+0000","last_undegraded":"2024-06-28T16:12:50.582816+0000","last_fullsized":"2024-06-28T16:12:50.582816+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":630,"num_read_kb":508,"num_write":240,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"260'353","reported_seq":1001,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.727043+0000","last_change":"2024-06-28T15:00:15.429200+0000","last_active":"2024-06-28T16:14:34.727043+0000","last_peered":"2024-06-28T16:14:34.727043+0000","last_clean":"2024-06-28T16:14:34.727043+0000","last_became_active":"2024-06-28T15:00:15.428415+0000","last_became_peered":"2024-06-28T15:00:15.428415+0000","last_unstale":"2024-06-28T16:14:34.727043+0000","last_undegraded":"2024-06-28T16:14:34.727043+0000","last_fullsized":"2024-06-28T16:14:34.727043+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:17:28.181075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16055067,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":496,"num_read_kb":15197,"num_write":1261,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.312405+0000","last_change":"2024-06-28T15:00:19.373751+0000","last_active":"2024-06-28T16:12:50.312405+0000","last_peered":"2024-06-28T16:12:50.312405+0000","last_clean":"2024-06-28T16:12:50.312405+0000","last_became_active":"2024-06-28T15:00:19.372840+0000","last_became_peered":"2024-06-28T15:00:19.372840+0000","last_unstale":"2024-06-28T16:12:50.312405+0000","last_undegraded":"2024-06-28T16:12:50.312405+0000","last_fullsized":"2024-06-28T16:12:50.312405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"260'123","reported_seq":687,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.288619+0000","last_change":"2024-06-28T15:00:21.377962+0000","last_active":"2024-06-28T16:13:00.288619+0000","last_peered":"2024-06-28T16:13:00.288619+0000","last_clean":"2024-06-28T16:13:00.288619+0000","last_became_active":"2024-06-28T15:00:21.377203+0000","last_became_peered":"2024-06-28T15:00:21.377203+0000","last_unstale":"2024-06-28T16:13:00.288619+0000","last_undegraded":"2024-06-28T16:13:00.288619+0000","last_fullsized":"2024-06-28T16:13:00.288619+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":100,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"260'522","reported_seq":4627,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.061536+0000","last_change":"2024-06-28T15:00:23.418328+0000","last_active":"2024-06-28T16:14:34.061536+0000","last_peered":"2024-06-28T16:14:34.061536+0000","last_clean":"2024-06-28T16:14:34.061536+0000","last_became_active":"2024-06-28T15:00:23.417490+0000","last_became_peered":"2024-06-28T15:00:23.417490+0000","last_unstale":"2024-06-28T16:14:34.061536+0000","last_undegraded":"2024-06-28T16:14:34.061536+0000","last_fullsized":"2024-06-28T16:14:34.061536+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":500,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:22:03.212204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"260'201","reported_seq":1010,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.965594+0000","last_change":"2024-06-28T15:00:25.556315+0000","last_active":"2024-06-28T16:14:06.965594+0000","last_peered":"2024-06-28T16:14:06.965594+0000","last_clean":"2024-06-28T16:14:06.965594+0000","last_became_active":"2024-06-28T15:00:25.555670+0000","last_became_peered":"2024-06-28T15:00:25.555670+0000","last_unstale":"2024-06-28T16:14:06.965594+0000","last_undegraded":"2024-06-28T16:14:06.965594+0000","last_fullsized":"2024-06-28T16:14:06.965594+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":100,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:26:34.140679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":557,"num_read_kb":555,"num_write":448,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"260'84","reported_seq":672,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.312378+0000","last_change":"2024-06-28T15:00:17.390503+0000","last_active":"2024-06-28T16:12:50.312378+0000","last_peered":"2024-06-28T16:12:50.312378+0000","last_clean":"2024-06-28T16:12:50.312378+0000","last_became_active":"2024-06-28T15:00:17.389986+0000","last_became_peered":"2024-06-28T15:00:17.389986+0000","last_unstale":"2024-06-28T16:12:50.312378+0000","last_undegraded":"2024-06-28T16:12:50.312378+0000","last_fullsized":"2024-06-28T16:12:50.312378+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":369,"num_read_kb":305,"num_write":130,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"260'288","reported_seq":922,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.683233+0000","last_change":"2024-06-28T15:00:15.413174+0000","last_active":"2024-06-28T16:14:34.683233+0000","last_peered":"2024-06-28T16:14:34.683233+0000","last_clean":"2024-06-28T16:14:34.683233+0000","last_became_active":"2024-06-28T15:00:15.411380+0000","last_became_peered":"2024-06-28T15:00:15.411380+0000","last_unstale":"2024-06-28T16:14:34.683233+0000","last_undegraded":"2024-06-28T16:14:34.683233+0000","last_fullsized":"2024-06-28T16:14:34.683233+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:35.364682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262217,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":24351,"num_write":1142,"num_write_kb":34218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"31'3","reported_seq":830,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:31.936121+0000","last_change":"2024-06-28T15:00:19.404519+0000","last_active":"2024-06-28T16:14:31.936121+0000","last_peered":"2024-06-28T16:14:31.936121+0000","last_clean":"2024-06-28T16:14:31.936121+0000","last_became_active":"2024-06-28T15:00:19.403623+0000","last_became_peered":"2024-06-28T15:00:19.403623+0000","last_unstale":"2024-06-28T16:14:31.936121+0000","last_undegraded":"2024-06-28T16:14:31.936121+0000","last_fullsized":"2024-06-28T16:14:31.936121+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:36:48.645241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":496,"num_read_kb":494,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"260'251","reported_seq":833,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.272334+0000","last_change":"2024-06-28T15:00:21.485248+0000","last_active":"2024-06-28T16:14:33.272334+0000","last_peered":"2024-06-28T16:14:33.272334+0000","last_clean":"2024-06-28T16:14:33.272334+0000","last_became_active":"2024-06-28T15:00:21.483633+0000","last_became_peered":"2024-06-28T15:00:21.483633+0000","last_unstale":"2024-06-28T16:14:33.272334+0000","last_undegraded":"2024-06-28T16:14:33.272334+0000","last_fullsized":"2024-06-28T16:14:33.272334+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:48:04.552220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8831,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":873,"num_write":310,"num_write_kb":224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"260'537","reported_seq":2240,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.766777+0000","last_change":"2024-06-28T15:00:23.397769+0000","last_active":"2024-06-28T16:14:34.766777+0000","last_peered":"2024-06-28T16:14:34.766777+0000","last_clean":"2024-06-28T16:14:34.766777+0000","last_became_active":"2024-06-28T15:00:23.396923+0000","last_became_peered":"2024-06-28T15:00:23.396923+0000","last_unstale":"2024-06-28T16:14:34.766777+0000","last_undegraded":"2024-06-28T16:14:34.766777+0000","last_fullsized":"2024-06-28T16:14:34.766777+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"260'415","reported_seq":1093,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:43.529396+0000","last_change":"2024-06-28T15:00:17.378608+0000","last_active":"2024-06-28T16:12:43.529396+0000","last_peered":"2024-06-28T16:12:43.529396+0000","last_clean":"2024-06-28T16:12:43.529396+0000","last_became_active":"2024-06-28T15:00:17.376864+0000","last_became_peered":"2024-06-28T15:00:17.376864+0000","last_unstale":"2024-06-28T16:12:43.529396+0000","last_undegraded":"2024-06-28T16:12:43.529396+0000","last_fullsized":"2024-06-28T16:12:43.529396+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":400,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:13.643893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1583,"num_read_kb":1214,"num_write":634,"num_write_kb":357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"260'320","reported_seq":979,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.732986+0000","last_change":"2024-06-28T15:00:15.423161+0000","last_active":"2024-06-28T16:14:34.732986+0000","last_peered":"2024-06-28T16:14:34.732986+0000","last_clean":"2024-06-28T16:14:34.732986+0000","last_became_active":"2024-06-28T15:00:15.415163+0000","last_became_peered":"2024-06-28T15:00:15.415163+0000","last_unstale":"2024-06-28T16:14:34.732986+0000","last_undegraded":"2024-06-28T16:14:34.732986+0000","last_fullsized":"2024-06-28T16:14:34.732986+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:10:26.502858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470118,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":518,"num_read_kb":26822,"num_write":1200,"num_write_kb":41274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.101361+0000","last_change":"2024-06-28T15:00:19.390703+0000","last_active":"2024-06-28T16:11:50.101361+0000","last_peered":"2024-06-28T16:11:50.101361+0000","last_clean":"2024-06-28T16:11:50.101361+0000","last_became_active":"2024-06-28T15:00:19.389624+0000","last_became_peered":"2024-06-28T15:00:19.389624+0000","last_unstale":"2024-06-28T16:11:50.101361+0000","last_undegraded":"2024-06-28T16:11:50.101361+0000","last_fullsized":"2024-06-28T16:11:50.101361+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:38:09.613294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"260'382","reported_seq":1026,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.950175+0000","last_change":"2024-06-28T15:00:21.483120+0000","last_active":"2024-06-28T16:14:33.950175+0000","last_peered":"2024-06-28T16:14:33.950175+0000","last_clean":"2024-06-28T16:14:33.950175+0000","last_became_active":"2024-06-28T15:00:21.481315+0000","last_became_peered":"2024-06-28T15:00:21.481315+0000","last_unstale":"2024-06-28T16:14:33.950175+0000","last_undegraded":"2024-06-28T16:14:33.950175+0000","last_fullsized":"2024-06-28T16:14:33.950175+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":300,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:09:01.037761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46076,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":638,"num_read_kb":1277,"num_write":490,"num_write_kb":298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.074002+0000","last_change":"2024-06-28T15:00:23.414142+0000","last_active":"2024-06-28T16:14:39.074002+0000","last_peered":"2024-06-28T16:14:39.074002+0000","last_clean":"2024-06-28T16:14:39.074002+0000","last_became_active":"2024-06-28T15:00:23.413132+0000","last_became_peered":"2024-06-28T15:00:23.413132+0000","last_unstale":"2024-06-28T16:14:39.074002+0000","last_undegraded":"2024-06-28T16:14:39.074002+0000","last_fullsized":"2024-06-28T16:14:39.074002+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"260'555","reported_seq":1373,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:13.000905+0000","last_change":"2024-06-28T15:00:25.442589+0000","last_active":"2024-06-28T16:14:13.000905+0000","last_peered":"2024-06-28T16:14:13.000905+0000","last_clean":"2024-06-28T16:14:13.000905+0000","last_became_active":"2024-06-28T15:00:25.441335+0000","last_became_peered":"2024-06-28T15:00:25.441335+0000","last_unstale":"2024-06-28T16:14:13.000905+0000","last_undegraded":"2024-06-28T16:14:13.000905+0000","last_fullsized":"2024-06-28T16:14:13.000905+0000","mapping_epoch":29,"log_start":"260'500","ondisk_log_start":"260'500","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":500,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:55.109636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1277,"num_read_kb":1273,"num_write":1164,"num_write_kb":394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"260'52","reported_seq":631,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.094197+0000","last_change":"2024-06-28T15:00:17.409603+0000","last_active":"2024-06-28T16:11:50.094197+0000","last_peered":"2024-06-28T16:11:50.094197+0000","last_clean":"2024-06-28T16:11:50.094197+0000","last_became_active":"2024-06-28T15:00:17.408992+0000","last_became_peered":"2024-06-28T15:00:17.408992+0000","last_unstale":"2024-06-28T16:11:50.094197+0000","last_undegraded":"2024-06-28T16:11:50.094197+0000","last_fullsized":"2024-06-28T16:11:50.094197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:22:21.223507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":226,"num_read_kb":196,"num_write":79,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"260'326","reported_seq":945,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.655730+0000","last_change":"2024-06-28T15:00:15.402904+0000","last_active":"2024-06-28T16:14:34.655730+0000","last_peered":"2024-06-28T16:14:34.655730+0000","last_clean":"2024-06-28T16:14:34.655730+0000","last_became_active":"2024-06-28T15:00:15.402188+0000","last_became_peered":"2024-06-28T15:00:15.402188+0000","last_unstale":"2024-06-28T16:14:34.655730+0000","last_undegraded":"2024-06-28T16:14:34.655730+0000","last_fullsized":"2024-06-28T16:14:34.655730+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:19:31.626172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170347,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":7037,"num_write":1119,"num_write_kb":28395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.300470+0000","last_change":"2024-06-28T15:00:19.383547+0000","last_active":"2024-06-28T16:14:00.300470+0000","last_peered":"2024-06-28T16:14:00.300470+0000","last_clean":"2024-06-28T16:14:00.300470+0000","last_became_active":"2024-06-28T15:00:19.383002+0000","last_became_peered":"2024-06-28T15:00:19.383002+0000","last_unstale":"2024-06-28T16:14:00.300470+0000","last_undegraded":"2024-06-28T16:14:00.300470+0000","last_fullsized":"2024-06-28T16:14:00.300470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"260'288","reported_seq":880,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.971707+0000","last_change":"2024-06-28T15:00:21.395366+0000","last_active":"2024-06-28T16:14:39.971707+0000","last_peered":"2024-06-28T16:14:39.971707+0000","last_clean":"2024-06-28T16:14:39.971707+0000","last_became_active":"2024-06-28T15:00:21.394580+0000","last_became_peered":"2024-06-28T15:00:21.394580+0000","last_unstale":"2024-06-28T16:14:39.971707+0000","last_undegraded":"2024-06-28T16:14:39.971707+0000","last_fullsized":"2024-06-28T16:14:39.971707+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":455,"num_read_kb":1052,"num_write":364,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:15.332154+0000","last_change":"2024-06-28T15:00:23.423231+0000","last_active":"2024-06-28T16:14:15.332154+0000","last_peered":"2024-06-28T16:14:15.332154+0000","last_clean":"2024-06-28T16:14:15.332154+0000","last_became_active":"2024-06-28T15:00:23.422567+0000","last_became_peered":"2024-06-28T15:00:23.422567+0000","last_unstale":"2024-06-28T16:14:15.332154+0000","last_undegraded":"2024-06-28T16:14:15.332154+0000","last_fullsized":"2024-06-28T16:14:15.332154+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"260'96","reported_seq":589,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089808+0000","last_change":"2024-06-28T15:00:25.426829+0000","last_active":"2024-06-28T16:11:35.089808+0000","last_peered":"2024-06-28T16:11:35.089808+0000","last_clean":"2024-06-28T16:11:35.089808+0000","last_became_active":"2024-06-28T15:00:25.426151+0000","last_became_peered":"2024-06-28T15:00:25.426151+0000","last_unstale":"2024-06-28T16:11:35.089808+0000","last_undegraded":"2024-06-28T16:11:35.089808+0000","last_fullsized":"2024-06-28T16:11:35.089808+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:23:26.301565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":231,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"260'173","reported_seq":777,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:10.098092+0000","last_change":"2024-06-28T15:00:17.407655+0000","last_active":"2024-06-28T16:12:10.098092+0000","last_peered":"2024-06-28T16:12:10.098092+0000","last_clean":"2024-06-28T16:12:10.098092+0000","last_became_active":"2024-06-28T15:00:17.407031+0000","last_became_peered":"2024-06-28T15:00:17.407031+0000","last_unstale":"2024-06-28T16:12:10.098092+0000","last_undegraded":"2024-06-28T16:12:10.098092+0000","last_fullsized":"2024-06-28T16:12:10.098092+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:17:37.199564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":538,"num_write":274,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"260'353","reported_seq":1038,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.682331+0000","last_change":"2024-06-28T15:00:15.428903+0000","last_active":"2024-06-28T16:14:34.682331+0000","last_peered":"2024-06-28T16:14:34.682331+0000","last_clean":"2024-06-28T16:14:34.682331+0000","last_became_active":"2024-06-28T15:00:15.427408+0000","last_became_peered":"2024-06-28T15:00:15.427408+0000","last_unstale":"2024-06-28T16:14:34.682331+0000","last_undegraded":"2024-06-28T16:14:34.682331+0000","last_fullsized":"2024-06-28T16:14:34.682331+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:52:51.484685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":574,"num_read_kb":16939,"num_write":1376,"num_write_kb":32636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.308227+0000","last_change":"2024-06-28T15:00:19.384770+0000","last_active":"2024-06-28T16:14:35.308227+0000","last_peered":"2024-06-28T16:14:35.308227+0000","last_clean":"2024-06-28T16:14:35.308227+0000","last_became_active":"2024-06-28T15:00:19.384172+0000","last_became_peered":"2024-06-28T15:00:19.384172+0000","last_unstale":"2024-06-28T16:14:35.308227+0000","last_undegraded":"2024-06-28T16:14:35.308227+0000","last_fullsized":"2024-06-28T16:14:35.308227+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"260'314","reported_seq":925,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.098634+0000","last_change":"2024-06-28T15:00:21.394527+0000","last_active":"2024-06-28T16:14:34.098634+0000","last_peered":"2024-06-28T16:14:34.098634+0000","last_clean":"2024-06-28T16:14:34.098634+0000","last_became_active":"2024-06-28T15:00:21.393608+0000","last_became_peered":"2024-06-28T15:00:21.393608+0000","last_unstale":"2024-06-28T16:14:34.098634+0000","last_undegraded":"2024-06-28T16:14:34.098634+0000","last_fullsized":"2024-06-28T16:14:34.098634+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:01:39.810509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42645,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":526,"num_read_kb":1122,"num_write":410,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"260'534","reported_seq":2267,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.767723+0000","last_change":"2024-06-28T15:00:23.414333+0000","last_active":"2024-06-28T16:14:34.767723+0000","last_peered":"2024-06-28T16:14:34.767723+0000","last_clean":"2024-06-28T16:14:34.767723+0000","last_became_active":"2024-06-28T15:00:23.413550+0000","last_became_peered":"2024-06-28T15:00:23.413550+0000","last_unstale":"2024-06-28T16:14:34.767723+0000","last_undegraded":"2024-06-28T16:14:34.767723+0000","last_fullsized":"2024-06-28T16:14:34.767723+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":500,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"260'144","reported_seq":673,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:05.601171+0000","last_change":"2024-06-28T15:00:25.432580+0000","last_active":"2024-06-28T16:14:05.601171+0000","last_peered":"2024-06-28T16:14:05.601171+0000","last_clean":"2024-06-28T16:14:05.601171+0000","last_became_active":"2024-06-28T15:00:25.431835+0000","last_became_peered":"2024-06-28T15:00:25.431835+0000","last_unstale":"2024-06-28T16:14:05.601171+0000","last_undegraded":"2024-06-28T16:14:05.601171+0000","last_fullsized":"2024-06-28T16:14:05.601171+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1247,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":164,"num_write":265,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"260'4158","reported_seq":4765,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.608300+0000","last_change":"2024-06-28T15:00:17.400121+0000","last_active":"2024-06-28T16:14:35.608300+0000","last_peered":"2024-06-28T16:14:35.608300+0000","last_clean":"2024-06-28T16:14:35.608300+0000","last_became_active":"2024-06-28T15:00:17.399799+0000","last_became_peered":"2024-06-28T15:00:17.399799+0000","last_unstale":"2024-06-28T16:14:35.608300+0000","last_undegraded":"2024-06-28T16:14:35.608300+0000","last_fullsized":"2024-06-28T16:14:35.608300+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":3000,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14678,"num_read_kb":11882,"num_write":6254,"num_write_kb":3121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"260'337","reported_seq":982,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.704930+0000","last_change":"2024-06-28T15:00:15.426615+0000","last_active":"2024-06-28T16:14:34.704930+0000","last_peered":"2024-06-28T16:14:34.704930+0000","last_clean":"2024-06-28T16:14:34.704930+0000","last_became_active":"2024-06-28T15:00:15.425690+0000","last_became_peered":"2024-06-28T15:00:15.425690+0000","last_unstale":"2024-06-28T16:14:34.704930+0000","last_undegraded":"2024-06-28T16:14:34.704930+0000","last_fullsized":"2024-06-28T16:14:34.704930+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:59:45.352009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079931,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":508,"num_read_kb":13664,"num_write":1274,"num_write_kb":26574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,1],"acting":[5,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:15.303994+0000","last_change":"2024-06-28T15:00:19.385556+0000","last_active":"2024-06-28T16:14:15.303994+0000","last_peered":"2024-06-28T16:14:15.303994+0000","last_clean":"2024-06-28T16:14:15.303994+0000","last_became_active":"2024-06-28T15:00:19.385189+0000","last_became_peered":"2024-06-28T15:00:19.385189+0000","last_unstale":"2024-06-28T16:14:15.303994+0000","last_undegraded":"2024-06-28T16:14:15.303994+0000","last_fullsized":"2024-06-28T16:14:15.303994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"260'487","reported_seq":1144,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.253332+0000","last_change":"2024-06-28T15:00:21.400448+0000","last_active":"2024-06-28T16:14:34.253332+0000","last_peered":"2024-06-28T16:14:34.253332+0000","last_clean":"2024-06-28T16:14:34.253332+0000","last_became_active":"2024-06-28T15:00:21.397397+0000","last_became_peered":"2024-06-28T15:00:21.397397+0000","last_unstale":"2024-06-28T16:14:34.253332+0000","last_undegraded":"2024-06-28T16:14:34.253332+0000","last_fullsized":"2024-06-28T16:14:34.253332+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":400,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92449,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":856,"num_read_kb":2091,"num_write":698,"num_write_kb":574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.129159+0000","last_change":"2024-06-28T16:14:40.129181+0000","last_active":"2024-06-28T16:14:40.129159+0000","last_peered":"2024-06-28T16:14:40.129159+0000","last_clean":"2024-06-28T16:14:40.129159+0000","last_became_active":"2024-06-28T15:00:23.416263+0000","last_became_peered":"2024-06-28T15:00:23.416263+0000","last_unstale":"2024-06-28T16:14:40.129159+0000","last_undegraded":"2024-06-28T16:14:40.129159+0000","last_fullsized":"2024-06-28T16:14:40.129159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:40.129070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:40.129070+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.129070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:20:56.813386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"260'113","reported_seq":632,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:05.132122+0000","last_change":"2024-06-28T15:00:25.423425+0000","last_active":"2024-06-28T16:14:05.132122+0000","last_peered":"2024-06-28T16:14:05.132122+0000","last_clean":"2024-06-28T16:14:05.132122+0000","last_became_active":"2024-06-28T15:00:25.422519+0000","last_became_peered":"2024-06-28T15:00:25.422519+0000","last_unstale":"2024-06-28T16:14:05.132122+0000","last_undegraded":"2024-06-28T16:14:05.132122+0000","last_fullsized":"2024-06-28T16:14:05.132122+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":118,"num_read_kb":116,"num_write":248,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"260'128","reported_seq":777,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101386+0000","last_change":"2024-06-28T15:00:17.409415+0000","last_active":"2024-06-28T16:12:20.101386+0000","last_peered":"2024-06-28T16:12:20.101386+0000","last_clean":"2024-06-28T16:12:20.101386+0000","last_became_active":"2024-06-28T15:00:17.404204+0000","last_became_peered":"2024-06-28T15:00:17.404204+0000","last_unstale":"2024-06-28T16:12:20.101386+0000","last_undegraded":"2024-06-28T16:12:20.101386+0000","last_fullsized":"2024-06-28T16:12:20.101386+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:25:51.372637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":562,"num_read_kb":476,"num_write":174,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"260'352","reported_seq":1060,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.731157+0000","last_change":"2024-06-28T15:00:15.402917+0000","last_active":"2024-06-28T16:14:34.731157+0000","last_peered":"2024-06-28T16:14:34.731157+0000","last_clean":"2024-06-28T16:14:34.731157+0000","last_became_active":"2024-06-28T15:00:15.402041+0000","last_became_peered":"2024-06-28T15:00:15.402041+0000","last_unstale":"2024-06-28T16:14:34.731157+0000","last_undegraded":"2024-06-28T16:14:34.731157+0000","last_fullsized":"2024-06-28T16:14:34.731157+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":603,"num_read_kb":26410,"num_write":1369,"num_write_kb":26391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.308286+0000","last_change":"2024-06-28T15:00:19.393172+0000","last_active":"2024-06-28T16:14:35.308286+0000","last_peered":"2024-06-28T16:14:35.308286+0000","last_clean":"2024-06-28T16:14:35.308286+0000","last_became_active":"2024-06-28T15:00:19.392773+0000","last_became_peered":"2024-06-28T15:00:19.392773+0000","last_unstale":"2024-06-28T16:14:35.308286+0000","last_undegraded":"2024-06-28T16:14:35.308286+0000","last_fullsized":"2024-06-28T16:14:35.308286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"260'301","reported_seq":897,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.816209+0000","last_change":"2024-06-28T15:00:21.398582+0000","last_active":"2024-06-28T16:14:32.816209+0000","last_peered":"2024-06-28T16:14:32.816209+0000","last_clean":"2024-06-28T16:14:32.816209+0000","last_became_active":"2024-06-28T15:00:21.397842+0000","last_became_peered":"2024-06-28T15:00:21.397842+0000","last_unstale":"2024-06-28T16:14:32.816209+0000","last_undegraded":"2024-06-28T16:14:32.816209+0000","last_fullsized":"2024-06-28T16:14:32.816209+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:31:25.283406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44037,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":498,"num_read_kb":1063,"num_write":434,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:40.580118+0000","last_change":"2024-06-28T15:00:23.406901+0000","last_active":"2024-06-28T16:12:40.580118+0000","last_peered":"2024-06-28T16:12:40.580118+0000","last_clean":"2024-06-28T16:12:40.580118+0000","last_became_active":"2024-06-28T15:00:23.406122+0000","last_became_peered":"2024-06-28T15:00:23.406122+0000","last_unstale":"2024-06-28T16:12:40.580118+0000","last_undegraded":"2024-06-28T16:12:40.580118+0000","last_fullsized":"2024-06-28T16:12:40.580118+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"260'89","reported_seq":594,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.336792+0000","last_change":"2024-06-28T15:00:25.439968+0000","last_active":"2024-06-28T16:14:35.336792+0000","last_peered":"2024-06-28T16:14:35.336792+0000","last_clean":"2024-06-28T16:14:35.336792+0000","last_became_active":"2024-06-28T15:00:25.439346+0000","last_became_peered":"2024-06-28T15:00:25.439346+0000","last_unstale":"2024-06-28T16:14:35.336792+0000","last_undegraded":"2024-06-28T16:14:35.336792+0000","last_fullsized":"2024-06-28T16:14:35.336792+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":207,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"260'171","reported_seq":759,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101425+0000","last_change":"2024-06-28T15:00:17.396835+0000","last_active":"2024-06-28T16:12:20.101425+0000","last_peered":"2024-06-28T16:12:20.101425+0000","last_clean":"2024-06-28T16:12:20.101425+0000","last_became_active":"2024-06-28T15:00:17.396048+0000","last_became_peered":"2024-06-28T15:00:17.396048+0000","last_unstale":"2024-06-28T16:12:20.101425+0000","last_undegraded":"2024-06-28T16:12:20.101425+0000","last_fullsized":"2024-06-28T16:12:20.101425+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:20:25.818327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":659,"num_read_kb":531,"num_write":250,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"260'376","reported_seq":1107,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.727955+0000","last_change":"2024-06-28T15:00:15.398260+0000","last_active":"2024-06-28T16:14:34.727955+0000","last_peered":"2024-06-28T16:14:34.727955+0000","last_clean":"2024-06-28T16:14:34.727955+0000","last_became_active":"2024-06-28T15:00:15.397256+0000","last_became_peered":"2024-06-28T15:00:15.397256+0000","last_unstale":"2024-06-28T16:14:34.727955+0000","last_undegraded":"2024-06-28T16:14:34.727955+0000","last_fullsized":"2024-06-28T16:14:34.727955+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":300,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:08:01.511388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2674362,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":727,"num_read_kb":7345,"num_write":1214,"num_write_kb":31713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":511,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.138103+0000","last_change":"2024-06-28T15:00:19.379868+0000","last_active":"2024-06-28T16:14:35.138103+0000","last_peered":"2024-06-28T16:14:35.138103+0000","last_clean":"2024-06-28T16:14:35.138103+0000","last_became_active":"2024-06-28T15:00:19.379281+0000","last_became_peered":"2024-06-28T15:00:19.379281+0000","last_unstale":"2024-06-28T16:14:35.138103+0000","last_undegraded":"2024-06-28T16:14:35.138103+0000","last_fullsized":"2024-06-28T16:14:35.138103+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"260'350","reported_seq":969,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.683918+0000","last_change":"2024-06-28T15:00:21.396750+0000","last_active":"2024-06-28T16:14:34.683918+0000","last_peered":"2024-06-28T16:14:34.683918+0000","last_clean":"2024-06-28T16:14:34.683918+0000","last_became_active":"2024-06-28T15:00:21.395892+0000","last_became_peered":"2024-06-28T15:00:21.395892+0000","last_unstale":"2024-06-28T16:14:34.683918+0000","last_undegraded":"2024-06-28T16:14:34.683918+0000","last_fullsized":"2024-06-28T16:14:34.683918+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":300,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48931,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":579,"num_read_kb":1282,"num_write":462,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"260'501","reported_seq":2305,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.521196+0000","last_change":"2024-06-28T15:00:23.416260+0000","last_active":"2024-06-28T16:14:39.521196+0000","last_peered":"2024-06-28T16:14:39.521196+0000","last_clean":"2024-06-28T16:14:39.521196+0000","last_became_active":"2024-06-28T15:00:23.415437+0000","last_became_peered":"2024-06-28T15:00:23.415437+0000","last_unstale":"2024-06-28T16:14:39.521196+0000","last_undegraded":"2024-06-28T16:14:39.521196+0000","last_fullsized":"2024-06-28T16:14:39.521196+0000","mapping_epoch":27,"log_start":"257'400","ondisk_log_start":"257'400","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":400,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"260'99","reported_seq":609,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:04.609220+0000","last_change":"2024-06-28T15:00:25.425875+0000","last_active":"2024-06-28T16:14:04.609220+0000","last_peered":"2024-06-28T16:14:04.609220+0000","last_clean":"2024-06-28T16:14:04.609220+0000","last_became_active":"2024-06-28T15:00:25.425135+0000","last_became_peered":"2024-06-28T15:00:25.425135+0000","last_unstale":"2024-06-28T16:14:04.609220+0000","last_undegraded":"2024-06-28T16:14:04.609220+0000","last_fullsized":"2024-06-28T16:14:04.609220+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":245,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"260'160","reported_seq":743,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101485+0000","last_change":"2024-06-28T15:00:17.403493+0000","last_active":"2024-06-28T16:12:20.101485+0000","last_peered":"2024-06-28T16:12:20.101485+0000","last_clean":"2024-06-28T16:12:20.101485+0000","last_became_active":"2024-06-28T15:00:17.402024+0000","last_became_peered":"2024-06-28T15:00:17.402024+0000","last_unstale":"2024-06-28T16:12:20.101485+0000","last_undegraded":"2024-06-28T16:12:20.101485+0000","last_fullsized":"2024-06-28T16:12:20.101485+0000","mapping_epoch":21,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":100,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:25:59.494240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":645,"num_read_kb":505,"num_write":259,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"260'393","reported_seq":1067,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.738050+0000","last_change":"2024-06-28T15:00:15.389691+0000","last_active":"2024-06-28T16:14:34.738050+0000","last_peered":"2024-06-28T16:14:34.738050+0000","last_clean":"2024-06-28T16:14:34.738050+0000","last_became_active":"2024-06-28T15:00:15.388365+0000","last_became_peered":"2024-06-28T15:00:15.388365+0000","last_unstale":"2024-06-28T16:14:34.738050+0000","last_undegraded":"2024-06-28T16:14:34.738050+0000","last_fullsized":"2024-06-28T16:14:34.738050+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":300,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:14:31.668500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1604260,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":661,"num_read_kb":7226,"num_write":1423,"num_write_kb":30573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"24'2","reported_seq":1154,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:31.948538+0000","last_change":"2024-06-28T15:00:19.403973+0000","last_active":"2024-06-28T16:14:31.948538+0000","last_peered":"2024-06-28T16:14:31.948538+0000","last_clean":"2024-06-28T16:14:31.948538+0000","last_became_active":"2024-06-28T15:00:19.397863+0000","last_became_peered":"2024-06-28T15:00:19.397863+0000","last_unstale":"2024-06-28T16:14:31.948538+0000","last_undegraded":"2024-06-28T16:14:31.948538+0000","last_fullsized":"2024-06-28T16:14:31.948538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:35:07.475408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":648,"num_read_kb":648,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"260'113","reported_seq":659,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.971629+0000","last_change":"2024-06-28T15:00:21.484486+0000","last_active":"2024-06-28T16:14:39.971629+0000","last_peered":"2024-06-28T16:14:39.971629+0000","last_clean":"2024-06-28T16:14:39.971629+0000","last_became_active":"2024-06-28T15:00:21.483679+0000","last_became_peered":"2024-06-28T15:00:21.483679+0000","last_unstale":"2024-06-28T16:14:39.971629+0000","last_undegraded":"2024-06-28T16:14:39.971629+0000","last_fullsized":"2024-06-28T16:14:39.971629+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"260'515","reported_seq":4263,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.766983+0000","last_change":"2024-06-28T15:00:23.419813+0000","last_active":"2024-06-28T16:14:34.766983+0000","last_peered":"2024-06-28T16:14:34.766983+0000","last_clean":"2024-06-28T16:14:34.766983+0000","last_became_active":"2024-06-28T15:00:23.419182+0000","last_became_peered":"2024-06-28T15:00:23.419182+0000","last_unstale":"2024-06-28T16:14:34.766983+0000","last_undegraded":"2024-06-28T16:14:34.766983+0000","last_fullsized":"2024-06-28T16:14:34.766983+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:00:01.245043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"260'159","reported_seq":672,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.114555+0000","last_change":"2024-06-28T15:00:25.427204+0000","last_active":"2024-06-28T16:13:20.114555+0000","last_peered":"2024-06-28T16:13:20.114555+0000","last_clean":"2024-06-28T16:13:20.114555+0000","last_became_active":"2024-06-28T15:00:25.425690+0000","last_became_peered":"2024-06-28T15:00:25.425690+0000","last_unstale":"2024-06-28T16:13:20.114555+0000","last_undegraded":"2024-06-28T16:13:20.114555+0000","last_fullsized":"2024-06-28T16:13:20.114555+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:26:18.073869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":152,"num_write":381,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"260'74","reported_seq":619,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.075926+0000","last_change":"2024-06-28T15:00:59.800386+0000","last_active":"2024-06-28T16:11:00.075926+0000","last_peered":"2024-06-28T16:11:00.075926+0000","last_clean":"2024-06-28T16:11:00.075926+0000","last_became_active":"2024-06-28T15:00:59.799823+0000","last_became_peered":"2024-06-28T15:00:59.799823+0000","last_unstale":"2024-06-28T16:11:00.075926+0000","last_undegraded":"2024-06-28T16:11:00.075926+0000","last_fullsized":"2024-06-28T16:11:00.075926+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:43:49.439721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":247,"num_write":103,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"260'351","reported_seq":1082,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.582297+0000","last_change":"2024-06-28T15:00:15.403798+0000","last_active":"2024-06-28T16:14:34.582297+0000","last_peered":"2024-06-28T16:14:34.582297+0000","last_clean":"2024-06-28T16:14:34.582297+0000","last_became_active":"2024-06-28T15:00:15.402782+0000","last_became_peered":"2024-06-28T15:00:15.402782+0000","last_unstale":"2024-06-28T16:14:34.582297+0000","last_undegraded":"2024-06-28T16:14:34.582297+0000","last_fullsized":"2024-06-28T16:14:34.582297+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":300,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262233,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":541,"num_read_kb":3603,"num_write":1164,"num_write_kb":36556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101454+0000","last_change":"2024-06-28T15:00:19.405878+0000","last_active":"2024-06-28T16:12:20.101454+0000","last_peered":"2024-06-28T16:12:20.101454+0000","last_clean":"2024-06-28T16:12:20.101454+0000","last_became_active":"2024-06-28T15:00:19.405083+0000","last_became_peered":"2024-06-28T15:00:19.405083+0000","last_unstale":"2024-06-28T16:12:20.101454+0000","last_undegraded":"2024-06-28T16:12:20.101454+0000","last_fullsized":"2024-06-28T16:12:20.101454+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:55:51.552282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"260'429","reported_seq":1064,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.719442+0000","last_change":"2024-06-28T15:00:21.393207+0000","last_active":"2024-06-28T16:14:33.719442+0000","last_peered":"2024-06-28T16:14:33.719442+0000","last_clean":"2024-06-28T16:14:33.719442+0000","last_became_active":"2024-06-28T15:00:21.392393+0000","last_became_peered":"2024-06-28T15:00:21.392393+0000","last_unstale":"2024-06-28T16:14:33.719442+0000","last_undegraded":"2024-06-28T16:14:33.719442+0000","last_fullsized":"2024-06-28T16:14:33.719442+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":400,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:47:20.793097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47467,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1282,"num_read_kb":1893,"num_write":1822,"num_write_kb":1648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":494,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.101411+0000","last_change":"2024-06-28T15:00:23.416374+0000","last_active":"2024-06-28T16:12:25.101411+0000","last_peered":"2024-06-28T16:12:25.101411+0000","last_clean":"2024-06-28T16:12:25.101411+0000","last_became_active":"2024-06-28T15:00:23.415497+0000","last_became_peered":"2024-06-28T15:00:23.415497+0000","last_unstale":"2024-06-28T16:12:25.101411+0000","last_undegraded":"2024-06-28T16:12:25.101411+0000","last_fullsized":"2024-06-28T16:12:25.101411+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:13:09.664310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"260'133","reported_seq":640,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.255577+0000","last_change":"2024-06-28T15:00:25.440419+0000","last_active":"2024-06-28T16:14:08.255577+0000","last_peered":"2024-06-28T16:14:08.255577+0000","last_clean":"2024-06-28T16:14:08.255577+0000","last_became_active":"2024-06-28T15:00:25.439527+0000","last_became_peered":"2024-06-28T15:00:25.439527+0000","last_unstale":"2024-06-28T16:14:08.255577+0000","last_undegraded":"2024-06-28T16:14:08.255577+0000","last_fullsized":"2024-06-28T16:14:08.255577+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":100,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":127,"num_write":301,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"260'156","reported_seq":773,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.088666+0000","last_change":"2024-06-28T15:00:17.415172+0000","last_active":"2024-06-28T16:11:30.088666+0000","last_peered":"2024-06-28T16:11:30.088666+0000","last_clean":"2024-06-28T16:11:30.088666+0000","last_became_active":"2024-06-28T15:00:17.414537+0000","last_became_peered":"2024-06-28T15:00:17.414537+0000","last_unstale":"2024-06-28T16:11:30.088666+0000","last_undegraded":"2024-06-28T16:11:30.088666+0000","last_fullsized":"2024-06-28T16:11:30.088666+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:18:02.691178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":522,"num_write":223,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":616,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.109225+0000","last_change":"2024-06-28T15:00:06.303487+0000","last_active":"2024-06-28T16:12:25.109225+0000","last_peered":"2024-06-28T16:12:25.109225+0000","last_clean":"2024-06-28T16:12:25.109225+0000","last_became_active":"2024-06-28T15:00:06.302725+0000","last_became_peered":"2024-06-28T15:00:06.302725+0000","last_unstale":"2024-06-28T16:12:25.109225+0000","last_undegraded":"2024-06-28T16:12:25.109225+0000","last_fullsized":"2024-06-28T16:12:25.109225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:45:32.083226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:00.075782+0000","last_change":"2024-06-28T15:00:06.285125+0000","last_active":"2024-06-28T16:11:00.075782+0000","last_peered":"2024-06-28T16:11:00.075782+0000","last_clean":"2024-06-28T16:11:00.075782+0000","last_became_active":"2024-06-28T15:00:06.284218+0000","last_became_peered":"2024-06-28T15:00:06.284218+0000","last_unstale":"2024-06-28T16:11:00.075782+0000","last_undegraded":"2024-06-28T16:11:00.075782+0000","last_fullsized":"2024-06-28T16:11:00.075782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:04:51.635552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"260'320","reported_seq":1116,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.750155+0000","last_change":"2024-06-28T15:00:59.940725+0000","last_active":"2024-06-28T16:14:34.750155+0000","last_peered":"2024-06-28T16:14:34.750155+0000","last_clean":"2024-06-28T16:14:34.750155+0000","last_became_active":"2024-06-28T15:00:59.939352+0000","last_became_peered":"2024-06-28T15:00:59.939352+0000","last_unstale":"2024-06-28T16:14:34.750155+0000","last_undegraded":"2024-06-28T16:14:34.750155+0000","last_fullsized":"2024-06-28T16:14:34.750155+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":804,"num_read_kb":59106,"num_write":1183,"num_write_kb":43047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6,4],"acting":[3,2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.092431+0000","last_change":"2024-06-28T15:00:19.391126+0000","last_active":"2024-06-28T16:11:10.092431+0000","last_peered":"2024-06-28T16:11:10.092431+0000","last_clean":"2024-06-28T16:11:10.092431+0000","last_became_active":"2024-06-28T15:00:19.390255+0000","last_became_peered":"2024-06-28T15:00:19.390255+0000","last_unstale":"2024-06-28T16:11:10.092431+0000","last_undegraded":"2024-06-28T16:11:10.092431+0000","last_fullsized":"2024-06-28T16:11:10.092431+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:19:02.126295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"260'730","reported_seq":1450,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.539793+0000","last_change":"2024-06-28T15:00:21.398370+0000","last_active":"2024-06-28T16:14:34.539793+0000","last_peered":"2024-06-28T16:14:34.539793+0000","last_clean":"2024-06-28T16:14:34.539793+0000","last_became_active":"2024-06-28T15:00:21.397640+0000","last_became_peered":"2024-06-28T15:00:21.397640+0000","last_unstale":"2024-06-28T16:14:34.539793+0000","last_undegraded":"2024-06-28T16:14:34.539793+0000","last_fullsized":"2024-06-28T16:14:34.539793+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":700,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:10:23.954766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134565,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1425,"num_read_kb":3272,"num_write":1360,"num_write_kb":1190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:05.601127+0000","last_change":"2024-06-28T15:00:23.400439+0000","last_active":"2024-06-28T16:14:05.601127+0000","last_peered":"2024-06-28T16:14:05.601127+0000","last_clean":"2024-06-28T16:14:05.601127+0000","last_became_active":"2024-06-28T15:00:23.399024+0000","last_became_peered":"2024-06-28T15:00:23.399024+0000","last_unstale":"2024-06-28T16:14:05.601127+0000","last_undegraded":"2024-06-28T16:14:05.601127+0000","last_fullsized":"2024-06-28T16:14:05.601127+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"260'186","reported_seq":876,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:05.197144+0000","last_change":"2024-06-28T15:00:25.442723+0000","last_active":"2024-06-28T16:14:05.197144+0000","last_peered":"2024-06-28T16:14:05.197144+0000","last_clean":"2024-06-28T16:14:05.197144+0000","last_became_active":"2024-06-28T15:00:25.442032+0000","last_became_peered":"2024-06-28T15:00:25.442032+0000","last_unstale":"2024-06-28T16:14:05.197144+0000","last_undegraded":"2024-06-28T16:14:05.197144+0000","last_fullsized":"2024-06-28T16:14:05.197144+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:04:44.942959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":392,"num_read_kb":388,"num_write":388,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"260'118","reported_seq":756,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.599760+0000","last_change":"2024-06-28T15:00:17.404538+0000","last_active":"2024-06-28T16:14:00.599760+0000","last_peered":"2024-06-28T16:14:00.599760+0000","last_clean":"2024-06-28T16:14:00.599760+0000","last_became_active":"2024-06-28T15:00:17.404200+0000","last_became_peered":"2024-06-28T15:00:17.404200+0000","last_unstale":"2024-06-28T16:14:00.599760+0000","last_undegraded":"2024-06-28T16:14:00.599760+0000","last_fullsized":"2024-06-28T16:14:00.599760+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":495,"num_read_kb":413,"num_write":159,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":526,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:25.109267+0000","last_change":"2024-06-28T15:00:06.286231+0000","last_active":"2024-06-28T16:12:25.109267+0000","last_peered":"2024-06-28T16:12:25.109267+0000","last_clean":"2024-06-28T16:12:25.109267+0000","last_became_active":"2024-06-28T15:00:06.285489+0000","last_became_peered":"2024-06-28T15:00:06.285489+0000","last_unstale":"2024-06-28T16:12:25.109267+0000","last_undegraded":"2024-06-28T16:12:25.109267+0000","last_fullsized":"2024-06-28T16:12:25.109267+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:05:17.528909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"260'324","reported_seq":993,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.669814+0000","last_change":"2024-06-28T15:00:15.404155+0000","last_active":"2024-06-28T16:14:34.669814+0000","last_peered":"2024-06-28T16:14:34.669814+0000","last_clean":"2024-06-28T16:14:34.669814+0000","last_became_active":"2024-06-28T15:00:15.403657+0000","last_became_peered":"2024-06-28T15:00:15.403657+0000","last_unstale":"2024-06-28T16:14:34.669814+0000","last_undegraded":"2024-06-28T16:14:34.669814+0000","last_fullsized":"2024-06-28T16:14:34.669814+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":524,"num_read_kb":36646,"num_write":1262,"num_write_kb":16548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.087155+0000","last_change":"2024-06-28T15:00:19.407157+0000","last_active":"2024-06-28T16:11:50.087155+0000","last_peered":"2024-06-28T16:11:50.087155+0000","last_clean":"2024-06-28T16:11:50.087155+0000","last_became_active":"2024-06-28T15:00:19.406447+0000","last_became_peered":"2024-06-28T15:00:19.406447+0000","last_unstale":"2024-06-28T16:11:50.087155+0000","last_undegraded":"2024-06-28T16:11:50.087155+0000","last_fullsized":"2024-06-28T16:11:50.087155+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:12:32.253515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"260'1134","reported_seq":2909,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.760247+0000","last_change":"2024-06-28T15:00:21.399223+0000","last_active":"2024-06-28T16:14:34.760247+0000","last_peered":"2024-06-28T16:14:34.760247+0000","last_clean":"2024-06-28T16:14:34.760247+0000","last_became_active":"2024-06-28T15:00:21.398382+0000","last_became_peered":"2024-06-28T15:00:21.398382+0000","last_unstale":"2024-06-28T16:14:34.760247+0000","last_undegraded":"2024-06-28T16:14:34.760247+0000","last_fullsized":"2024-06-28T16:14:34.760247+0000","mapping_epoch":25,"log_start":"260'1100","ondisk_log_start":"260'1100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":1100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133468,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1851,"num_read_kb":3741,"num_write":1150,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":498,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:30.097271+0000","last_change":"2024-06-28T15:00:23.423751+0000","last_active":"2024-06-28T16:11:30.097271+0000","last_peered":"2024-06-28T16:11:30.097271+0000","last_clean":"2024-06-28T16:11:30.097271+0000","last_became_active":"2024-06-28T15:00:23.422876+0000","last_became_peered":"2024-06-28T15:00:23.422876+0000","last_unstale":"2024-06-28T16:11:30.097271+0000","last_undegraded":"2024-06-28T16:11:30.097271+0000","last_fullsized":"2024-06-28T16:11:30.097271+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:18:16.152063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"260'116","reported_seq":630,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:25.136123+0000","last_change":"2024-06-28T15:00:25.554459+0000","last_active":"2024-06-28T16:14:25.136123+0000","last_peered":"2024-06-28T16:14:25.136123+0000","last_clean":"2024-06-28T16:14:25.136123+0000","last_became_active":"2024-06-28T15:00:25.553691+0000","last_became_peered":"2024-06-28T15:00:25.553691+0000","last_unstale":"2024-06-28T16:14:25.136123+0000","last_undegraded":"2024-06-28T16:14:25.136123+0000","last_fullsized":"2024-06-28T16:14:25.136123+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":100,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":862,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":120,"num_write":292,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"260'221","reported_seq":835,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.131001+0000","last_change":"2024-06-28T15:00:17.393736+0000","last_active":"2024-06-28T16:14:00.131001+0000","last_peered":"2024-06-28T16:14:00.131001+0000","last_clean":"2024-06-28T16:14:00.131001+0000","last_became_active":"2024-06-28T15:00:17.393361+0000","last_became_peered":"2024-06-28T15:00:17.393361+0000","last_unstale":"2024-06-28T16:14:00.131001+0000","last_undegraded":"2024-06-28T16:14:00.131001+0000","last_fullsized":"2024-06-28T16:14:00.131001+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":892,"num_read_kb":761,"num_write":358,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":533,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.288548+0000","last_change":"2024-06-28T15:00:06.280312+0000","last_active":"2024-06-28T16:13:00.288548+0000","last_peered":"2024-06-28T16:13:00.288548+0000","last_clean":"2024-06-28T16:13:00.288548+0000","last_became_active":"2024-06-28T15:00:06.279950+0000","last_became_peered":"2024-06-28T15:00:06.279950+0000","last_unstale":"2024-06-28T16:13:00.288548+0000","last_undegraded":"2024-06-28T16:13:00.288548+0000","last_fullsized":"2024-06-28T16:13:00.288548+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"260'334","reported_seq":992,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.138035+0000","last_change":"2024-06-28T15:00:15.413730+0000","last_active":"2024-06-28T16:14:35.138035+0000","last_peered":"2024-06-28T16:14:35.138035+0000","last_clean":"2024-06-28T16:14:35.138035+0000","last_became_active":"2024-06-28T15:00:15.413348+0000","last_became_peered":"2024-06-28T15:00:15.413348+0000","last_unstale":"2024-06-28T16:14:35.138035+0000","last_undegraded":"2024-06-28T16:14:35.138035+0000","last_fullsized":"2024-06-28T16:14:35.138035+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":504,"num_read_kb":3638,"num_write":1247,"num_write_kb":36014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"24'1","reported_seq":516,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.300622+0000","last_change":"2024-06-28T15:00:19.405107+0000","last_active":"2024-06-28T16:14:00.300622+0000","last_peered":"2024-06-28T16:14:00.300622+0000","last_clean":"2024-06-28T16:14:00.300622+0000","last_became_active":"2024-06-28T15:00:19.404653+0000","last_became_peered":"2024-06-28T15:00:19.404653+0000","last_unstale":"2024-06-28T16:14:00.300622+0000","last_undegraded":"2024-06-28T16:14:00.300622+0000","last_fullsized":"2024-06-28T16:14:00.300622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"260'246","reported_seq":1691,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:11.127801+0000","last_change":"2024-06-28T15:00:21.484753+0000","last_active":"2024-06-28T16:14:11.127801+0000","last_peered":"2024-06-28T16:14:11.127801+0000","last_clean":"2024-06-28T16:14:11.127801+0000","last_became_active":"2024-06-28T15:00:21.483749+0000","last_became_peered":"2024-06-28T15:00:21.483749+0000","last_unstale":"2024-06-28T16:14:11.127801+0000","last_undegraded":"2024-06-28T16:14:11.127801+0000","last_fullsized":"2024-06-28T16:14:11.127801+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:25:11.741395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.590983+0000","last_change":"2024-06-28T15:00:23.400124+0000","last_active":"2024-06-28T16:13:20.590983+0000","last_peered":"2024-06-28T16:13:20.590983+0000","last_clean":"2024-06-28T16:13:20.590983+0000","last_became_active":"2024-06-28T15:00:23.399391+0000","last_became_peered":"2024-06-28T15:00:23.399391+0000","last_unstale":"2024-06-28T16:13:20.590983+0000","last_undegraded":"2024-06-28T16:13:20.590983+0000","last_fullsized":"2024-06-28T16:13:20.590983+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"260'239","reported_seq":846,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.972031+0000","last_change":"2024-06-28T15:00:25.420452+0000","last_active":"2024-06-28T16:14:39.972031+0000","last_peered":"2024-06-28T16:14:39.972031+0000","last_clean":"2024-06-28T16:14:39.972031+0000","last_became_active":"2024-06-28T15:00:25.419711+0000","last_became_peered":"2024-06-28T15:00:25.419711+0000","last_unstale":"2024-06-28T16:14:39.972031+0000","last_undegraded":"2024-06-28T16:14:39.972031+0000","last_fullsized":"2024-06-28T16:14:39.972031+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":200,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":467,"num_read_kb":491,"num_write":517,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"260'315","reported_seq":1007,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.121987+0000","last_change":"2024-06-28T15:00:17.398693+0000","last_active":"2024-06-28T16:13:15.121987+0000","last_peered":"2024-06-28T16:13:15.121987+0000","last_clean":"2024-06-28T16:13:15.121987+0000","last_became_active":"2024-06-28T15:00:17.393696+0000","last_became_peered":"2024-06-28T15:00:17.393696+0000","last_unstale":"2024-06-28T16:13:15.121987+0000","last_undegraded":"2024-06-28T16:13:15.121987+0000","last_fullsized":"2024-06-28T16:13:15.121987+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1365,"num_read_kb":1154,"num_write":468,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:55.074668+0000","last_change":"2024-06-28T15:00:06.284111+0000","last_active":"2024-06-28T16:10:55.074668+0000","last_peered":"2024-06-28T16:10:55.074668+0000","last_clean":"2024-06-28T16:10:55.074668+0000","last_became_active":"2024-06-28T15:00:06.283106+0000","last_became_peered":"2024-06-28T15:00:06.283106+0000","last_unstale":"2024-06-28T16:10:55.074668+0000","last_undegraded":"2024-06-28T16:10:55.074668+0000","last_fullsized":"2024-06-28T16:10:55.074668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:07.069940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"260'318","reported_seq":991,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.731218+0000","last_change":"2024-06-28T15:00:15.415068+0000","last_active":"2024-06-28T16:14:34.731218+0000","last_peered":"2024-06-28T16:14:34.731218+0000","last_clean":"2024-06-28T16:14:34.731218+0000","last_became_active":"2024-06-28T15:00:15.413880+0000","last_became_peered":"2024-06-28T15:00:15.413880+0000","last_unstale":"2024-06-28T16:14:34.731218+0000","last_undegraded":"2024-06-28T16:14:34.731218+0000","last_fullsized":"2024-06-28T16:14:34.731218+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:13:10.361144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2421978,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":465,"num_read_kb":85416,"num_write":1148,"num_write_kb":27167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:30.593348+0000","last_change":"2024-06-28T15:00:19.382869+0000","last_active":"2024-06-28T16:13:30.593348+0000","last_peered":"2024-06-28T16:13:30.593348+0000","last_clean":"2024-06-28T16:13:30.593348+0000","last_became_active":"2024-06-28T15:00:19.382486+0000","last_became_peered":"2024-06-28T15:00:19.382486+0000","last_unstale":"2024-06-28T16:13:30.593348+0000","last_undegraded":"2024-06-28T16:13:30.593348+0000","last_fullsized":"2024-06-28T16:13:30.593348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"260'159","reported_seq":730,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:05.329931+0000","last_change":"2024-06-28T15:00:21.401345+0000","last_active":"2024-06-28T16:14:05.329931+0000","last_peered":"2024-06-28T16:14:05.329931+0000","last_clean":"2024-06-28T16:14:05.329931+0000","last_became_active":"2024-06-28T15:00:21.400535+0000","last_became_peered":"2024-06-28T15:00:21.400535+0000","last_unstale":"2024-06-28T16:14:05.329931+0000","last_undegraded":"2024-06-28T16:14:05.329931+0000","last_fullsized":"2024-06-28T16:14:05.329931+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":198,"num_read_kb":191,"num_write":142,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":495,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:55.094802+0000","last_change":"2024-06-28T15:00:23.417433+0000","last_active":"2024-06-28T16:11:55.094802+0000","last_peered":"2024-06-28T16:11:55.094802+0000","last_clean":"2024-06-28T16:11:55.094802+0000","last_became_active":"2024-06-28T15:00:23.416701+0000","last_became_peered":"2024-06-28T15:00:23.416701+0000","last_unstale":"2024-06-28T16:11:55.094802+0000","last_undegraded":"2024-06-28T16:11:55.094802+0000","last_fullsized":"2024-06-28T16:11:55.094802+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:59:56.613956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"260'117","reported_seq":633,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:07.191149+0000","last_change":"2024-06-28T15:00:25.439202+0000","last_active":"2024-06-28T16:13:07.191149+0000","last_peered":"2024-06-28T16:13:07.191149+0000","last_clean":"2024-06-28T16:13:07.191149+0000","last_became_active":"2024-06-28T15:00:25.438523+0000","last_became_peered":"2024-06-28T15:00:25.438523+0000","last_unstale":"2024-06-28T16:13:07.191149+0000","last_undegraded":"2024-06-28T16:13:07.191149+0000","last_fullsized":"2024-06-28T16:13:07.191149+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:31:55.560360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":129,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"260'169","reported_seq":775,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.121903+0000","last_change":"2024-06-28T15:00:17.399083+0000","last_active":"2024-06-28T16:13:15.121903+0000","last_peered":"2024-06-28T16:13:15.121903+0000","last_clean":"2024-06-28T16:13:15.121903+0000","last_became_active":"2024-06-28T15:00:17.398515+0000","last_became_peered":"2024-06-28T15:00:17.398515+0000","last_unstale":"2024-06-28T16:13:15.121903+0000","last_undegraded":"2024-06-28T16:13:15.121903+0000","last_fullsized":"2024-06-28T16:13:15.121903+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":644,"num_read_kb":512,"num_write":273,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"260'372","reported_seq":1043,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.710966+0000","last_change":"2024-06-28T15:00:15.417442+0000","last_active":"2024-06-28T16:14:34.710966+0000","last_peered":"2024-06-28T16:14:34.710966+0000","last_clean":"2024-06-28T16:14:34.710966+0000","last_became_active":"2024-06-28T15:00:15.414704+0000","last_became_peered":"2024-06-28T15:00:15.414704+0000","last_unstale":"2024-06-28T16:14:34.710966+0000","last_undegraded":"2024-06-28T16:14:34.710966+0000","last_fullsized":"2024-06-28T16:14:34.710966+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:54:15.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10506361,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":617,"num_read_kb":33619,"num_write":1289,"num_write_kb":23520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":530,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:05.119299+0000","last_change":"2024-06-28T15:00:06.302842+0000","last_active":"2024-06-28T16:13:05.119299+0000","last_peered":"2024-06-28T16:13:05.119299+0000","last_clean":"2024-06-28T16:13:05.119299+0000","last_became_active":"2024-06-28T15:00:06.302449+0000","last_became_peered":"2024-06-28T15:00:06.302449+0000","last_unstale":"2024-06-28T16:13:05.119299+0000","last_undegraded":"2024-06-28T16:13:05.119299+0000","last_fullsized":"2024-06-28T16:13:05.119299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.589775+0000","last_change":"2024-06-28T15:00:19.385492+0000","last_active":"2024-06-28T16:13:15.589775+0000","last_peered":"2024-06-28T16:13:15.589775+0000","last_clean":"2024-06-28T16:13:15.589775+0000","last_became_active":"2024-06-28T15:00:19.384933+0000","last_became_peered":"2024-06-28T15:00:19.384933+0000","last_unstale":"2024-06-28T16:13:15.589775+0000","last_undegraded":"2024-06-28T16:13:15.589775+0000","last_fullsized":"2024-06-28T16:13:15.589775+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"260'587","reported_seq":1287,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.609270+0000","last_change":"2024-06-28T15:00:21.407998+0000","last_active":"2024-06-28T16:14:38.609270+0000","last_peered":"2024-06-28T16:14:38.609270+0000","last_clean":"2024-06-28T16:14:38.609270+0000","last_became_active":"2024-06-28T15:00:21.405762+0000","last_became_peered":"2024-06-28T15:00:21.405762+0000","last_unstale":"2024-06-28T16:14:38.609270+0000","last_undegraded":"2024-06-28T16:14:38.609270+0000","last_fullsized":"2024-06-28T16:14:38.609270+0000","mapping_epoch":25,"log_start":"260'500","ondisk_log_start":"260'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":500,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92557,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1001,"num_read_kb":2235,"num_write":838,"num_write_kb":641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.086163+0000","last_change":"2024-06-28T15:00:23.405517+0000","last_active":"2024-06-28T16:11:15.086163+0000","last_peered":"2024-06-28T16:11:15.086163+0000","last_clean":"2024-06-28T16:11:15.086163+0000","last_became_active":"2024-06-28T15:00:23.404714+0000","last_became_peered":"2024-06-28T15:00:23.404714+0000","last_unstale":"2024-06-28T16:11:15.086163+0000","last_undegraded":"2024-06-28T16:11:15.086163+0000","last_fullsized":"2024-06-28T16:11:15.086163+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:54:34.158419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"260'147","reported_seq":665,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.609587+0000","last_change":"2024-06-28T15:00:25.557678+0000","last_active":"2024-06-28T16:14:38.609587+0000","last_peered":"2024-06-28T16:14:38.609587+0000","last_clean":"2024-06-28T16:14:38.609587+0000","last_became_active":"2024-06-28T15:00:25.556834+0000","last_became_peered":"2024-06-28T15:00:25.556834+0000","last_unstale":"2024-06-28T16:14:38.609587+0000","last_undegraded":"2024-06-28T16:14:38.609587+0000","last_fullsized":"2024-06-28T16:14:38.609587+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":100,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1183,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":342,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"260'139","reported_seq":769,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.589735+0000","last_change":"2024-06-28T15:00:17.372769+0000","last_active":"2024-06-28T16:13:15.589735+0000","last_peered":"2024-06-28T16:13:15.589735+0000","last_clean":"2024-06-28T16:13:15.589735+0000","last_became_active":"2024-06-28T15:00:17.372401+0000","last_became_peered":"2024-06-28T15:00:17.372401+0000","last_unstale":"2024-06-28T16:13:15.589735+0000","last_undegraded":"2024-06-28T16:13:15.589735+0000","last_fullsized":"2024-06-28T16:13:15.589735+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":577,"num_read_kb":470,"num_write":221,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"260'273","reported_seq":902,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.567969+0000","last_change":"2024-06-28T15:00:15.408605+0000","last_active":"2024-06-28T16:14:34.567969+0000","last_peered":"2024-06-28T16:14:34.567969+0000","last_clean":"2024-06-28T16:14:34.567969+0000","last_became_active":"2024-06-28T15:00:15.408140+0000","last_became_peered":"2024-06-28T15:00:15.408140+0000","last_unstale":"2024-06-28T16:14:34.567969+0000","last_undegraded":"2024-06-28T16:14:34.567969+0000","last_fullsized":"2024-06-28T16:14:34.567969+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":200,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":526399,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":387,"num_read_kb":8190,"num_write":1081,"num_write_kb":24156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.093729+0000","last_change":"2024-06-28T15:00:06.303984+0000","last_active":"2024-06-28T16:11:15.093729+0000","last_peered":"2024-06-28T16:11:15.093729+0000","last_clean":"2024-06-28T16:11:15.093729+0000","last_became_active":"2024-06-28T15:00:06.302962+0000","last_became_peered":"2024-06-28T15:00:06.302962+0000","last_unstale":"2024-06-28T16:11:15.093729+0000","last_undegraded":"2024-06-28T16:11:15.093729+0000","last_fullsized":"2024-06-28T16:11:15.093729+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:10:47.354122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.084030+0000","last_change":"2024-06-28T15:00:19.385776+0000","last_active":"2024-06-28T16:11:10.084030+0000","last_peered":"2024-06-28T16:11:10.084030+0000","last_clean":"2024-06-28T16:11:10.084030+0000","last_became_active":"2024-06-28T15:00:19.384961+0000","last_became_peered":"2024-06-28T15:00:19.384961+0000","last_unstale":"2024-06-28T16:11:10.084030+0000","last_undegraded":"2024-06-28T16:11:10.084030+0000","last_fullsized":"2024-06-28T16:11:10.084030+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:23:32.384577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"260'188","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.122878+0000","last_change":"2024-06-28T15:00:21.480780+0000","last_active":"2024-06-28T16:13:20.122878+0000","last_peered":"2024-06-28T16:13:20.122878+0000","last_clean":"2024-06-28T16:13:20.122878+0000","last_became_active":"2024-06-28T15:00:21.480107+0000","last_became_peered":"2024-06-28T15:00:21.480107+0000","last_unstale":"2024-06-28T16:13:20.122878+0000","last_undegraded":"2024-06-28T16:13:20.122878+0000","last_fullsized":"2024-06-28T16:13:20.122878+0000","mapping_epoch":25,"log_start":"257'100","ondisk_log_start":"257'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":100,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":263,"num_read_kb":253,"num_write":174,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"260'565","reported_seq":2878,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.764175+0000","last_change":"2024-06-28T15:00:23.429133+0000","last_active":"2024-06-28T16:14:34.764175+0000","last_peered":"2024-06-28T16:14:34.764175+0000","last_clean":"2024-06-28T16:14:34.764175+0000","last_became_active":"2024-06-28T15:00:23.427565+0000","last_became_peered":"2024-06-28T15:00:23.427565+0000","last_unstale":"2024-06-28T16:14:34.764175+0000","last_undegraded":"2024-06-28T16:14:34.764175+0000","last_fullsized":"2024-06-28T16:14:34.764175+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":500,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:07:13.588883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"260'124","reported_seq":639,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:13.005771+0000","last_change":"2024-06-28T15:00:25.554359+0000","last_active":"2024-06-28T16:14:13.005771+0000","last_peered":"2024-06-28T16:14:13.005771+0000","last_clean":"2024-06-28T16:14:13.005771+0000","last_became_active":"2024-06-28T15:00:25.553627+0000","last_became_peered":"2024-06-28T15:00:25.553627+0000","last_unstale":"2024-06-28T16:14:13.005771+0000","last_undegraded":"2024-06-28T16:14:13.005771+0000","last_fullsized":"2024-06-28T16:14:13.005771+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":100,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":713,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":165,"num_read_kb":148,"num_write":308,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"260'172","reported_seq":826,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:30.335672+0000","last_change":"2024-06-28T15:00:17.377265+0000","last_active":"2024-06-28T16:14:30.335672+0000","last_peered":"2024-06-28T16:14:30.335672+0000","last_clean":"2024-06-28T16:14:30.335672+0000","last_became_active":"2024-06-28T15:00:17.376516+0000","last_became_peered":"2024-06-28T15:00:17.376516+0000","last_unstale":"2024-06-28T16:14:30.335672+0000","last_undegraded":"2024-06-28T16:14:30.335672+0000","last_fullsized":"2024-06-28T16:14:30.335672+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":722,"num_read_kb":588,"num_write":244,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"260'336","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.742328+0000","last_change":"2024-06-28T15:00:15.403594+0000","last_active":"2024-06-28T16:14:34.742328+0000","last_peered":"2024-06-28T16:14:34.742328+0000","last_clean":"2024-06-28T16:14:34.742328+0000","last_became_active":"2024-06-28T15:00:15.403032+0000","last_became_peered":"2024-06-28T15:00:15.403032+0000","last_unstale":"2024-06-28T16:14:34.742328+0000","last_undegraded":"2024-06-28T16:14:34.742328+0000","last_fullsized":"2024-06-28T16:14:34.742328+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":968026,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":483,"num_read_kb":17032,"num_write":1147,"num_write_kb":35885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":529,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.127835+0000","last_change":"2024-06-28T15:00:06.278832+0000","last_active":"2024-06-28T16:13:45.127835+0000","last_peered":"2024-06-28T16:13:45.127835+0000","last_clean":"2024-06-28T16:13:45.127835+0000","last_became_active":"2024-06-28T15:00:06.278443+0000","last_became_peered":"2024-06-28T15:00:06.278443+0000","last_unstale":"2024-06-28T16:13:45.127835+0000","last_undegraded":"2024-06-28T16:13:45.127835+0000","last_fullsized":"2024-06-28T16:13:45.127835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.085018+0000","last_change":"2024-06-28T15:00:19.380916+0000","last_active":"2024-06-28T16:11:10.085018+0000","last_peered":"2024-06-28T16:11:10.085018+0000","last_clean":"2024-06-28T16:11:10.085018+0000","last_became_active":"2024-06-28T15:00:19.378932+0000","last_became_peered":"2024-06-28T15:00:19.378932+0000","last_unstale":"2024-06-28T16:11:10.085018+0000","last_undegraded":"2024-06-28T16:11:10.085018+0000","last_fullsized":"2024-06-28T16:11:10.085018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:36:40.033839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"260'138","reported_seq":698,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:15.086320+0000","last_change":"2024-06-28T15:00:21.396435+0000","last_active":"2024-06-28T16:11:15.086320+0000","last_peered":"2024-06-28T16:11:15.086320+0000","last_clean":"2024-06-28T16:11:15.086320+0000","last_became_active":"2024-06-28T15:00:21.395462+0000","last_became_peered":"2024-06-28T15:00:21.395462+0000","last_unstale":"2024-06-28T16:11:15.086320+0000","last_undegraded":"2024-06-28T16:11:15.086320+0000","last_fullsized":"2024-06-28T16:11:15.086320+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":100,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:51:41.956055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":167,"num_read_kb":161,"num_write":112,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"260'537","reported_seq":2582,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.765810+0000","last_change":"2024-06-28T15:00:23.428281+0000","last_active":"2024-06-28T16:14:34.765810+0000","last_peered":"2024-06-28T16:14:34.765810+0000","last_clean":"2024-06-28T16:14:34.765810+0000","last_became_active":"2024-06-28T15:00:23.427568+0000","last_became_peered":"2024-06-28T15:00:23.427568+0000","last_unstale":"2024-06-28T16:14:34.765810+0000","last_undegraded":"2024-06-28T16:14:34.765810+0000","last_fullsized":"2024-06-28T16:14:34.765810+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"260'247","reported_seq":788,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.609580+0000","last_change":"2024-06-28T15:00:25.428796+0000","last_active":"2024-06-28T16:14:38.609580+0000","last_peered":"2024-06-28T16:14:38.609580+0000","last_clean":"2024-06-28T16:14:38.609580+0000","last_became_active":"2024-06-28T15:00:25.428067+0000","last_became_peered":"2024-06-28T15:00:25.428067+0000","last_unstale":"2024-06-28T16:14:38.609580+0000","last_undegraded":"2024-06-28T16:14:38.609580+0000","last_fullsized":"2024-06-28T16:14:38.609580+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":200,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":297,"num_write":552,"num_write_kb":170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"260'96","reported_seq":701,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:46.633284+0000","last_change":"2024-06-28T15:00:17.397233+0000","last_active":"2024-06-28T16:12:46.633284+0000","last_peered":"2024-06-28T16:12:46.633284+0000","last_clean":"2024-06-28T16:12:46.633284+0000","last_became_active":"2024-06-28T15:00:17.396413+0000","last_became_peered":"2024-06-28T15:00:17.396413+0000","last_unstale":"2024-06-28T16:12:46.633284+0000","last_undegraded":"2024-06-28T16:12:46.633284+0000","last_fullsized":"2024-06-28T16:12:46.633284+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:50:29.409467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":410,"num_read_kb":340,"num_write":139,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"260'318","reported_seq":1098,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.743460+0000","last_change":"2024-06-28T15:00:15.396426+0000","last_active":"2024-06-28T16:14:34.743460+0000","last_peered":"2024-06-28T16:14:34.743460+0000","last_clean":"2024-06-28T16:14:34.743460+0000","last_became_active":"2024-06-28T15:00:15.395980+0000","last_became_peered":"2024-06-28T15:00:15.395980+0000","last_unstale":"2024-06-28T16:14:34.743460+0000","last_undegraded":"2024-06-28T16:14:34.743460+0000","last_fullsized":"2024-06-28T16:14:34.743460+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661262,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":649,"num_read_kb":125515,"num_write":1187,"num_write_kb":17513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":521,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:35.089897+0000","last_change":"2024-06-28T15:00:06.284169+0000","last_active":"2024-06-28T16:11:35.089897+0000","last_peered":"2024-06-28T16:11:35.089897+0000","last_clean":"2024-06-28T16:11:35.089897+0000","last_became_active":"2024-06-28T15:00:06.283380+0000","last_became_peered":"2024-06-28T15:00:06.283380+0000","last_unstale":"2024-06-28T16:11:35.089897+0000","last_undegraded":"2024-06-28T16:11:35.089897+0000","last_fullsized":"2024-06-28T16:11:35.089897+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:39:59.368666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"31'5","reported_seq":843,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:31.947541+0000","last_change":"2024-06-28T15:00:19.372913+0000","last_active":"2024-06-28T16:14:31.947541+0000","last_peered":"2024-06-28T16:14:31.947541+0000","last_clean":"2024-06-28T16:14:31.947541+0000","last_became_active":"2024-06-28T15:00:19.372552+0000","last_became_peered":"2024-06-28T15:00:19.372552+0000","last_unstale":"2024-06-28T16:14:31.947541+0000","last_undegraded":"2024-06-28T16:14:31.947541+0000","last_fullsized":"2024-06-28T16:14:31.947541+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":500,"num_read_kb":812,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"260'748","reported_seq":1492,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.576522+0000","last_change":"2024-06-28T15:00:21.481881+0000","last_active":"2024-06-28T16:14:33.576522+0000","last_peered":"2024-06-28T16:14:33.576522+0000","last_clean":"2024-06-28T16:14:33.576522+0000","last_became_active":"2024-06-28T15:00:21.481155+0000","last_became_peered":"2024-06-28T15:00:21.481155+0000","last_unstale":"2024-06-28T16:14:33.576522+0000","last_undegraded":"2024-06-28T16:14:33.576522+0000","last_fullsized":"2024-06-28T16:14:33.576522+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":700,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:24:28.507362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131438,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1320,"num_read_kb":3166,"num_write":1058,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.319861+0000","last_change":"2024-06-28T15:00:23.409553+0000","last_active":"2024-06-28T16:13:20.319861+0000","last_peered":"2024-06-28T16:13:20.319861+0000","last_clean":"2024-06-28T16:13:20.319861+0000","last_became_active":"2024-06-28T15:00:23.408777+0000","last_became_peered":"2024-06-28T15:00:23.408777+0000","last_unstale":"2024-06-28T16:13:20.319861+0000","last_undegraded":"2024-06-28T16:13:20.319861+0000","last_fullsized":"2024-06-28T16:13:20.319861+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"260'129","reported_seq":641,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.287170+0000","last_change":"2024-06-28T15:00:25.439187+0000","last_active":"2024-06-28T16:14:08.287170+0000","last_peered":"2024-06-28T16:14:08.287170+0000","last_clean":"2024-06-28T16:14:08.287170+0000","last_became_active":"2024-06-28T15:00:25.438476+0000","last_became_peered":"2024-06-28T15:00:25.438476+0000","last_unstale":"2024-06-28T16:14:08.287170+0000","last_undegraded":"2024-06-28T16:14:08.287170+0000","last_fullsized":"2024-06-28T16:14:08.287170+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:14:10.401700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":173,"num_read_kb":156,"num_write":320,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"260'151","reported_seq":719,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:13.001171+0000","last_change":"2024-06-28T15:00:25.440427+0000","last_active":"2024-06-28T16:14:13.001171+0000","last_peered":"2024-06-28T16:14:13.001171+0000","last_clean":"2024-06-28T16:14:13.001171+0000","last_became_active":"2024-06-28T15:00:25.439672+0000","last_became_peered":"2024-06-28T15:00:25.439672+0000","last_unstale":"2024-06-28T16:14:13.001171+0000","last_undegraded":"2024-06-28T16:14:13.001171+0000","last_fullsized":"2024-06-28T16:14:13.001171+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":100,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":207,"num_read_kb":205,"num_write":345,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"260'434","reported_seq":1071,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:58.995763+0000","last_change":"2024-06-28T15:00:17.402772+0000","last_active":"2024-06-28T16:10:58.995763+0000","last_peered":"2024-06-28T16:10:58.995763+0000","last_clean":"2024-06-28T16:10:58.995763+0000","last_became_active":"2024-06-28T15:00:17.401981+0000","last_became_peered":"2024-06-28T15:00:17.401981+0000","last_unstale":"2024-06-28T16:10:58.995763+0000","last_undegraded":"2024-06-28T16:10:58.995763+0000","last_fullsized":"2024-06-28T16:10:58.995763+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":400,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:36:44.706683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1728,"num_read_kb":1391,"num_write":746,"num_write_kb":387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"260'274","reported_seq":900,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.557629+0000","last_change":"2024-06-28T15:00:15.404138+0000","last_active":"2024-06-28T16:14:34.557629+0000","last_peered":"2024-06-28T16:14:34.557629+0000","last_clean":"2024-06-28T16:14:34.557629+0000","last_became_active":"2024-06-28T15:00:15.403450+0000","last_became_peered":"2024-06-28T15:00:15.403450+0000","last_unstale":"2024-06-28T16:14:34.557629+0000","last_undegraded":"2024-06-28T16:14:34.557629+0000","last_fullsized":"2024-06-28T16:14:34.557629+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:20:44.673833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437799,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":13452,"num_write":1073,"num_write_kb":25751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":509,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:40.113518+0000","last_change":"2024-06-28T15:00:19.398728+0000","last_active":"2024-06-28T16:12:40.113518+0000","last_peered":"2024-06-28T16:12:40.113518+0000","last_clean":"2024-06-28T16:12:40.113518+0000","last_became_active":"2024-06-28T15:00:19.398138+0000","last_became_peered":"2024-06-28T15:00:19.398138+0000","last_unstale":"2024-06-28T16:12:40.113518+0000","last_undegraded":"2024-06-28T16:12:40.113518+0000","last_fullsized":"2024-06-28T16:12:40.113518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"260'298","reported_seq":896,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.024496+0000","last_change":"2024-06-28T15:00:21.405835+0000","last_active":"2024-06-28T16:14:34.024496+0000","last_peered":"2024-06-28T16:14:34.024496+0000","last_clean":"2024-06-28T16:14:34.024496+0000","last_became_active":"2024-06-28T15:00:21.405016+0000","last_became_peered":"2024-06-28T15:00:21.405016+0000","last_unstale":"2024-06-28T16:14:34.024496+0000","last_undegraded":"2024-06-28T16:14:34.024496+0000","last_fullsized":"2024-06-28T16:14:34.024496+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53726,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":505,"num_read_kb":1165,"num_write":406,"num_write_kb":309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:45.310541+0000","last_change":"2024-06-28T15:00:23.418156+0000","last_active":"2024-06-28T16:12:45.310541+0000","last_peered":"2024-06-28T16:12:45.310541+0000","last_clean":"2024-06-28T16:12:45.310541+0000","last_became_active":"2024-06-28T15:00:23.417390+0000","last_became_peered":"2024-06-28T15:00:23.417390+0000","last_unstale":"2024-06-28T16:12:45.310541+0000","last_undegraded":"2024-06-28T16:12:45.310541+0000","last_fullsized":"2024-06-28T16:12:45.310541+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"260'101","reported_seq":609,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.336994+0000","last_change":"2024-06-28T15:00:25.413154+0000","last_active":"2024-06-28T16:14:35.336994+0000","last_peered":"2024-06-28T16:14:35.336994+0000","last_clean":"2024-06-28T16:14:35.336994+0000","last_became_active":"2024-06-28T15:00:25.412379+0000","last_became_peered":"2024-06-28T15:00:25.412379+0000","last_unstale":"2024-06-28T16:14:35.336994+0000","last_undegraded":"2024-06-28T16:14:35.336994+0000","last_fullsized":"2024-06-28T16:14:35.336994+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":88,"num_write":238,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"260'189","reported_seq":823,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:40.113567+0000","last_change":"2024-06-28T15:00:17.406956+0000","last_active":"2024-06-28T16:12:40.113567+0000","last_peered":"2024-06-28T16:12:40.113567+0000","last_clean":"2024-06-28T16:12:40.113567+0000","last_became_active":"2024-06-28T15:00:17.406647+0000","last_became_peered":"2024-06-28T15:00:17.406647+0000","last_unstale":"2024-06-28T16:12:40.113567+0000","last_undegraded":"2024-06-28T16:12:40.113567+0000","last_fullsized":"2024-06-28T16:12:40.113567+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":748,"num_read_kb":601,"num_write":301,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"260'334","reported_seq":1001,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.613707+0000","last_change":"2024-06-28T15:00:15.413269+0000","last_active":"2024-06-28T16:14:34.613707+0000","last_peered":"2024-06-28T16:14:34.613707+0000","last_clean":"2024-06-28T16:14:34.613707+0000","last_became_active":"2024-06-28T15:00:15.402945+0000","last_became_peered":"2024-06-28T15:00:15.402945+0000","last_unstale":"2024-06-28T16:14:34.613707+0000","last_undegraded":"2024-06-28T16:14:34.613707+0000","last_fullsized":"2024-06-28T16:14:34.613707+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:41:27.586797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340417,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":534,"num_read_kb":20609,"num_write":1305,"num_write_kb":45266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,4],"acting":[7,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:40.309352+0000","last_change":"2024-06-28T15:00:19.383614+0000","last_active":"2024-06-28T16:12:40.309352+0000","last_peered":"2024-06-28T16:12:40.309352+0000","last_clean":"2024-06-28T16:12:40.309352+0000","last_became_active":"2024-06-28T15:00:19.382839+0000","last_became_peered":"2024-06-28T15:00:19.382839+0000","last_unstale":"2024-06-28T16:12:40.309352+0000","last_undegraded":"2024-06-28T16:12:40.309352+0000","last_fullsized":"2024-06-28T16:12:40.309352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"260'52","reported_seq":574,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:45.581189+0000","last_change":"2024-06-28T15:00:21.407387+0000","last_active":"2024-06-28T16:12:45.581189+0000","last_peered":"2024-06-28T16:12:45.581189+0000","last_clean":"2024-06-28T16:12:45.581189+0000","last_became_active":"2024-06-28T15:00:21.404727+0000","last_became_peered":"2024-06-28T15:00:21.404727+0000","last_unstale":"2024-06-28T16:12:45.581189+0000","last_undegraded":"2024-06-28T16:12:45.581189+0000","last_fullsized":"2024-06-28T16:12:45.581189+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":48,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":494,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.081639+0000","last_change":"2024-06-28T15:00:23.419840+0000","last_active":"2024-06-28T16:10:25.081639+0000","last_peered":"2024-06-28T16:10:25.081639+0000","last_clean":"2024-06-28T16:10:25.081639+0000","last_became_active":"2024-06-28T15:00:23.419115+0000","last_became_peered":"2024-06-28T15:00:23.419115+0000","last_unstale":"2024-06-28T16:10:25.081639+0000","last_undegraded":"2024-06-28T16:10:25.081639+0000","last_fullsized":"2024-06-28T16:10:25.081639+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:05:05.710065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"260'190","reported_seq":720,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.726684+0000","last_change":"2024-06-28T15:00:25.426812+0000","last_active":"2024-06-28T16:14:06.726684+0000","last_peered":"2024-06-28T16:14:06.726684+0000","last_clean":"2024-06-28T16:14:06.726684+0000","last_became_active":"2024-06-28T15:00:25.426046+0000","last_became_peered":"2024-06-28T15:00:25.426046+0000","last_unstale":"2024-06-28T16:14:06.726684+0000","last_undegraded":"2024-06-28T16:14:06.726684+0000","last_fullsized":"2024-06-28T16:14:06.726684+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":286,"num_read_kb":283,"num_write":434,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"260'423","reported_seq":1171,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:40.309247+0000","last_change":"2024-06-28T15:00:17.390367+0000","last_active":"2024-06-28T16:12:40.309247+0000","last_peered":"2024-06-28T16:12:40.309247+0000","last_clean":"2024-06-28T16:12:40.309247+0000","last_became_active":"2024-06-28T15:00:17.389992+0000","last_became_peered":"2024-06-28T15:00:17.389992+0000","last_unstale":"2024-06-28T16:12:40.309247+0000","last_undegraded":"2024-06-28T16:12:40.309247+0000","last_fullsized":"2024-06-28T16:12:40.309247+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":400,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1786,"num_read_kb":1490,"num_write":759,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"260'328","reported_seq":971,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.635861+0000","last_change":"2024-06-28T15:00:15.427261+0000","last_active":"2024-06-28T16:14:34.635861+0000","last_peered":"2024-06-28T16:14:34.635861+0000","last_clean":"2024-06-28T16:14:34.635861+0000","last_became_active":"2024-06-28T15:00:15.419267+0000","last_became_peered":"2024-06-28T15:00:15.419267+0000","last_unstale":"2024-06-28T16:14:34.635861+0000","last_undegraded":"2024-06-28T16:14:34.635861+0000","last_fullsized":"2024-06-28T16:14:34.635861+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:49:04.986604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5413475,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":487,"num_read_kb":27459,"num_write":1288,"num_write_kb":48933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080344+0000","last_change":"2024-06-28T15:00:19.386890+0000","last_active":"2024-06-28T16:10:20.080344+0000","last_peered":"2024-06-28T16:10:20.080344+0000","last_clean":"2024-06-28T16:10:20.080344+0000","last_became_active":"2024-06-28T15:00:19.385990+0000","last_became_peered":"2024-06-28T15:00:19.385990+0000","last_unstale":"2024-06-28T16:10:20.080344+0000","last_undegraded":"2024-06-28T16:10:20.080344+0000","last_fullsized":"2024-06-28T16:10:20.080344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:39:52.204033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"260'318","reported_seq":932,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.207650+0000","last_change":"2024-06-28T15:00:21.406717+0000","last_active":"2024-06-28T16:14:33.207650+0000","last_peered":"2024-06-28T16:14:33.207650+0000","last_clean":"2024-06-28T16:14:33.207650+0000","last_became_active":"2024-06-28T15:00:21.405045+0000","last_became_peered":"2024-06-28T15:00:21.405045+0000","last_unstale":"2024-06-28T16:14:33.207650+0000","last_undegraded":"2024-06-28T16:14:33.207650+0000","last_fullsized":"2024-06-28T16:14:33.207650+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48456,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":530,"num_read_kb":1162,"num_write":414,"num_write_kb":289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":492,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.074474+0000","last_change":"2024-06-28T15:00:23.415444+0000","last_active":"2024-06-28T16:10:25.074474+0000","last_peered":"2024-06-28T16:10:25.074474+0000","last_clean":"2024-06-28T16:10:25.074474+0000","last_became_active":"2024-06-28T15:00:23.411264+0000","last_became_peered":"2024-06-28T15:00:23.411264+0000","last_unstale":"2024-06-28T16:10:25.074474+0000","last_undegraded":"2024-06-28T16:10:25.074474+0000","last_fullsized":"2024-06-28T16:10:25.074474+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:58:05.328252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"260'110","reported_seq":602,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.067825+0000","last_change":"2024-06-28T15:00:25.443135+0000","last_active":"2024-06-28T16:10:25.067825+0000","last_peered":"2024-06-28T16:10:25.067825+0000","last_clean":"2024-06-28T16:10:25.067825+0000","last_became_active":"2024-06-28T15:00:25.442010+0000","last_became_peered":"2024-06-28T15:00:25.442010+0000","last_unstale":"2024-06-28T16:10:25.067825+0000","last_undegraded":"2024-06-28T16:10:25.067825+0000","last_fullsized":"2024-06-28T16:10:25.067825+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:56:35.040791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":638,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":85,"num_write":260,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"260'105","reported_seq":670,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:40.309172+0000","last_change":"2024-06-28T15:00:17.376172+0000","last_active":"2024-06-28T16:12:40.309172+0000","last_peered":"2024-06-28T16:12:40.309172+0000","last_clean":"2024-06-28T16:12:40.309172+0000","last_became_active":"2024-06-28T15:00:17.373396+0000","last_became_peered":"2024-06-28T15:00:17.373396+0000","last_unstale":"2024-06-28T16:12:40.309172+0000","last_undegraded":"2024-06-28T16:12:40.309172+0000","last_fullsized":"2024-06-28T16:12:40.309172+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":424,"num_read_kb":340,"num_write":155,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"260'307","reported_seq":955,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.734689+0000","last_change":"2024-06-28T15:00:15.412826+0000","last_active":"2024-06-28T16:14:34.734689+0000","last_peered":"2024-06-28T16:14:34.734689+0000","last_clean":"2024-06-28T16:14:34.734689+0000","last_became_active":"2024-06-28T15:00:15.412333+0000","last_became_peered":"2024-06-28T15:00:15.412333+0000","last_unstale":"2024-06-28T16:14:34.734689+0000","last_undegraded":"2024-06-28T16:14:34.734689+0000","last_fullsized":"2024-06-28T16:14:34.734689+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1451455,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":458,"num_read_kb":31119,"num_write":1276,"num_write_kb":43155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073040+0000","last_change":"2024-06-28T15:00:19.387598+0000","last_active":"2024-06-28T16:10:20.073040+0000","last_peered":"2024-06-28T16:10:20.073040+0000","last_clean":"2024-06-28T16:10:20.073040+0000","last_became_active":"2024-06-28T15:00:19.386645+0000","last_became_peered":"2024-06-28T15:00:19.386645+0000","last_unstale":"2024-06-28T16:10:20.073040+0000","last_undegraded":"2024-06-28T16:10:20.073040+0000","last_fullsized":"2024-06-28T16:10:20.073040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:54:00.701086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"260'289","reported_seq":873,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.977932+0000","last_change":"2024-06-28T15:00:21.395133+0000","last_active":"2024-06-28T16:14:32.977932+0000","last_peered":"2024-06-28T16:14:32.977932+0000","last_clean":"2024-06-28T16:14:32.977932+0000","last_became_active":"2024-06-28T15:00:21.394357+0000","last_became_peered":"2024-06-28T15:00:21.394357+0000","last_unstale":"2024-06-28T16:14:32.977932+0000","last_undegraded":"2024-06-28T16:14:32.977932+0000","last_fullsized":"2024-06-28T16:14:32.977932+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:35:53.429744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53697,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":496,"num_read_kb":1194,"num_write":404,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:45.581309+0000","last_change":"2024-06-28T15:00:23.407033+0000","last_active":"2024-06-28T16:12:45.581309+0000","last_peered":"2024-06-28T16:12:45.581309+0000","last_clean":"2024-06-28T16:12:45.581309+0000","last_became_active":"2024-06-28T15:00:23.406296+0000","last_became_peered":"2024-06-28T15:00:23.406296+0000","last_unstale":"2024-06-28T16:12:45.581309+0000","last_undegraded":"2024-06-28T16:12:45.581309+0000","last_fullsized":"2024-06-28T16:12:45.581309+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"260'68","reported_seq":636,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.670953+0000","last_change":"2024-06-28T15:00:17.397537+0000","last_active":"2024-06-28T16:14:06.670953+0000","last_peered":"2024-06-28T16:14:06.670953+0000","last_clean":"2024-06-28T16:14:06.670953+0000","last_became_active":"2024-06-28T15:00:17.396861+0000","last_became_peered":"2024-06-28T15:00:17.396861+0000","last_unstale":"2024-06-28T16:14:06.670953+0000","last_undegraded":"2024-06-28T16:14:06.670953+0000","last_fullsized":"2024-06-28T16:14:06.670953+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:27:59.755447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":279,"num_read_kb":229,"num_write":93,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"260'309","reported_seq":945,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.754063+0000","last_change":"2024-06-28T15:00:15.383453+0000","last_active":"2024-06-28T16:14:34.754063+0000","last_peered":"2024-06-28T16:14:34.754063+0000","last_clean":"2024-06-28T16:14:34.754063+0000","last_became_active":"2024-06-28T15:00:15.383008+0000","last_became_peered":"2024-06-28T15:00:15.383008+0000","last_unstale":"2024-06-28T16:14:34.754063+0000","last_undegraded":"2024-06-28T16:14:34.754063+0000","last_fullsized":"2024-06-28T16:14:34.754063+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2156401,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":459,"num_read_kb":6724,"num_write":1144,"num_write_kb":24207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":516,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.608430+0000","last_change":"2024-06-28T15:00:19.390462+0000","last_active":"2024-06-28T16:14:35.608430+0000","last_peered":"2024-06-28T16:14:35.608430+0000","last_clean":"2024-06-28T16:14:35.608430+0000","last_became_active":"2024-06-28T15:00:19.390111+0000","last_became_peered":"2024-06-28T15:00:19.390111+0000","last_unstale":"2024-06-28T16:14:35.608430+0000","last_undegraded":"2024-06-28T16:14:35.608430+0000","last_fullsized":"2024-06-28T16:14:35.608430+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"260'472","reported_seq":1123,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.469535+0000","last_change":"2024-06-28T15:00:21.399070+0000","last_active":"2024-06-28T16:14:34.469535+0000","last_peered":"2024-06-28T16:14:34.469535+0000","last_clean":"2024-06-28T16:14:34.469535+0000","last_became_active":"2024-06-28T15:00:21.398262+0000","last_became_peered":"2024-06-28T15:00:21.398262+0000","last_unstale":"2024-06-28T16:14:34.469535+0000","last_undegraded":"2024-06-28T16:14:34.469535+0000","last_fullsized":"2024-06-28T16:14:34.469535+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":400,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86979,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":863,"num_read_kb":2070,"num_write":734,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":493,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.067855+0000","last_change":"2024-06-28T15:00:23.411496+0000","last_active":"2024-06-28T16:10:25.067855+0000","last_peered":"2024-06-28T16:10:25.067855+0000","last_clean":"2024-06-28T16:10:25.067855+0000","last_became_active":"2024-06-28T15:00:23.410519+0000","last_became_peered":"2024-06-28T15:00:23.410519+0000","last_unstale":"2024-06-28T16:10:25.067855+0000","last_undegraded":"2024-06-28T16:10:25.067855+0000","last_fullsized":"2024-06-28T16:10:25.067855+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:02:03.245169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"260'167","reported_seq":685,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:45.310807+0000","last_change":"2024-06-28T15:00:25.559194+0000","last_active":"2024-06-28T16:12:45.310807+0000","last_peered":"2024-06-28T16:12:45.310807+0000","last_clean":"2024-06-28T16:12:45.310807+0000","last_became_active":"2024-06-28T15:00:25.558426+0000","last_became_peered":"2024-06-28T15:00:25.558426+0000","last_unstale":"2024-06-28T16:12:45.310807+0000","last_undegraded":"2024-06-28T16:12:45.310807+0000","last_fullsized":"2024-06-28T16:12:45.310807+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":100,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":845,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":215,"num_read_kb":219,"num_write":377,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"260'158","reported_seq":788,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.608394+0000","last_change":"2024-06-28T15:00:17.395953+0000","last_active":"2024-06-28T16:14:35.608394+0000","last_peered":"2024-06-28T16:14:35.608394+0000","last_clean":"2024-06-28T16:14:35.608394+0000","last_became_active":"2024-06-28T15:00:17.395643+0000","last_became_peered":"2024-06-28T15:00:17.395643+0000","last_unstale":"2024-06-28T16:14:35.608394+0000","last_undegraded":"2024-06-28T16:14:35.608394+0000","last_fullsized":"2024-06-28T16:14:35.608394+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":654,"num_read_kb":561,"num_write":257,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"260'290","reported_seq":931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.646734+0000","last_change":"2024-06-28T15:00:59.940407+0000","last_active":"2024-06-28T16:14:34.646734+0000","last_peered":"2024-06-28T16:14:34.646734+0000","last_clean":"2024-06-28T16:14:34.646734+0000","last_became_active":"2024-06-28T15:00:59.939485+0000","last_became_peered":"2024-06-28T15:00:59.939485+0000","last_unstale":"2024-06-28T16:14:34.646734+0000","last_undegraded":"2024-06-28T16:14:34.646734+0000","last_fullsized":"2024-06-28T16:14:34.646734+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":430,"num_read_kb":21907,"num_write":1034,"num_write_kb":35159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,6],"acting":[1,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.084186+0000","last_change":"2024-06-28T15:00:19.385764+0000","last_active":"2024-06-28T16:11:10.084186+0000","last_peered":"2024-06-28T16:11:10.084186+0000","last_clean":"2024-06-28T16:11:10.084186+0000","last_became_active":"2024-06-28T15:00:19.385068+0000","last_became_peered":"2024-06-28T15:00:19.385068+0000","last_unstale":"2024-06-28T16:11:10.084186+0000","last_undegraded":"2024-06-28T16:11:10.084186+0000","last_fullsized":"2024-06-28T16:11:10.084186+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:18:40.223976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"260'348","reported_seq":954,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.054962+0000","last_change":"2024-06-28T15:00:21.401177+0000","last_active":"2024-06-28T16:14:33.054962+0000","last_peered":"2024-06-28T16:14:33.054962+0000","last_clean":"2024-06-28T16:14:33.054962+0000","last_became_active":"2024-06-28T15:00:21.400480+0000","last_became_peered":"2024-06-28T15:00:21.400480+0000","last_unstale":"2024-06-28T16:14:33.054962+0000","last_undegraded":"2024-06-28T16:14:33.054962+0000","last_fullsized":"2024-06-28T16:14:33.054962+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":300,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:08:38.525781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47708,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":581,"num_read_kb":1249,"num_write":480,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":499,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.117041+0000","last_change":"2024-06-28T15:00:23.418653+0000","last_active":"2024-06-28T16:12:55.117041+0000","last_peered":"2024-06-28T16:12:55.117041+0000","last_clean":"2024-06-28T16:12:55.117041+0000","last_became_active":"2024-06-28T15:00:23.417580+0000","last_became_peered":"2024-06-28T15:00:23.417580+0000","last_unstale":"2024-06-28T16:12:55.117041+0000","last_undegraded":"2024-06-28T16:12:55.117041+0000","last_fullsized":"2024-06-28T16:12:55.117041+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"260'170","reported_seq":677,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:07.865086+0000","last_change":"2024-06-28T15:00:25.431039+0000","last_active":"2024-06-28T16:14:07.865086+0000","last_peered":"2024-06-28T16:14:07.865086+0000","last_clean":"2024-06-28T16:14:07.865086+0000","last_became_active":"2024-06-28T15:00:25.430299+0000","last_became_peered":"2024-06-28T15:00:25.430299+0000","last_unstale":"2024-06-28T16:14:07.865086+0000","last_undegraded":"2024-06-28T16:14:07.865086+0000","last_fullsized":"2024-06-28T16:14:07.865086+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:31:33.565366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2035,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":193,"num_read_kb":191,"num_write":477,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"260'315","reported_seq":1058,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.312305+0000","last_change":"2024-06-28T15:00:17.383684+0000","last_active":"2024-06-28T16:12:50.312305+0000","last_peered":"2024-06-28T16:12:50.312305+0000","last_clean":"2024-06-28T16:12:50.312305+0000","last_became_active":"2024-06-28T15:00:17.383284+0000","last_became_peered":"2024-06-28T15:00:17.383284+0000","last_unstale":"2024-06-28T16:12:50.312305+0000","last_undegraded":"2024-06-28T16:12:50.312305+0000","last_fullsized":"2024-06-28T16:12:50.312305+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1515,"num_read_kb":1335,"num_write":548,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"260'347","reported_seq":1028,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.674376+0000","last_change":"2024-06-28T15:00:15.406314+0000","last_active":"2024-06-28T16:14:34.674376+0000","last_peered":"2024-06-28T16:14:34.674376+0000","last_clean":"2024-06-28T16:14:34.674376+0000","last_became_active":"2024-06-28T15:00:15.405499+0000","last_became_peered":"2024-06-28T15:00:15.405499+0000","last_unstale":"2024-06-28T16:14:34.674376+0000","last_undegraded":"2024-06-28T16:14:34.674376+0000","last_fullsized":"2024-06-28T16:14:34.674376+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:26:47.268928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17426850,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":585,"num_read_kb":18043,"num_write":1303,"num_write_kb":33783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.074018+0000","last_change":"2024-06-28T15:00:19.381603+0000","last_active":"2024-06-28T16:10:20.074018+0000","last_peered":"2024-06-28T16:10:20.074018+0000","last_clean":"2024-06-28T16:10:20.074018+0000","last_became_active":"2024-06-28T15:00:19.378852+0000","last_became_peered":"2024-06-28T15:00:19.378852+0000","last_unstale":"2024-06-28T16:10:20.074018+0000","last_undegraded":"2024-06-28T16:10:20.074018+0000","last_fullsized":"2024-06-28T16:10:20.074018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:45:27.874936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"260'279","reported_seq":881,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.315633+0000","last_change":"2024-06-28T15:00:21.481610+0000","last_active":"2024-06-28T16:14:34.315633+0000","last_peered":"2024-06-28T16:14:34.315633+0000","last_clean":"2024-06-28T16:14:34.315633+0000","last_became_active":"2024-06-28T15:00:21.480868+0000","last_became_peered":"2024-06-28T15:00:21.480868+0000","last_unstale":"2024-06-28T16:14:34.315633+0000","last_undegraded":"2024-06-28T16:14:34.315633+0000","last_fullsized":"2024-06-28T16:14:34.315633+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":200,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40850,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":446,"num_read_kb":956,"num_write":348,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:45.114822+0000","last_change":"2024-06-28T15:00:23.401297+0000","last_active":"2024-06-28T16:12:45.114822+0000","last_peered":"2024-06-28T16:12:45.114822+0000","last_clean":"2024-06-28T16:12:45.114822+0000","last_became_active":"2024-06-28T15:00:23.400417+0000","last_became_peered":"2024-06-28T15:00:23.400417+0000","last_unstale":"2024-06-28T16:12:45.114822+0000","last_undegraded":"2024-06-28T16:12:45.114822+0000","last_fullsized":"2024-06-28T16:12:45.114822+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"260'117","reported_seq":617,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:12.999377+0000","last_change":"2024-06-28T15:00:25.438977+0000","last_active":"2024-06-28T16:14:12.999377+0000","last_peered":"2024-06-28T16:14:12.999377+0000","last_clean":"2024-06-28T16:14:12.999377+0000","last_became_active":"2024-06-28T15:00:25.438176+0000","last_became_peered":"2024-06-28T15:00:25.438176+0000","last_unstale":"2024-06-28T16:14:12.999377+0000","last_undegraded":"2024-06-28T16:14:12.999377+0000","last_fullsized":"2024-06-28T16:14:12.999377+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:16:21.793451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":120,"num_read_kb":111,"num_write":277,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"260'74","reported_seq":656,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.286469+0000","last_change":"2024-06-28T15:00:17.403175+0000","last_active":"2024-06-28T16:12:50.286469+0000","last_peered":"2024-06-28T16:12:50.286469+0000","last_clean":"2024-06-28T16:12:50.286469+0000","last_became_active":"2024-06-28T15:00:17.402735+0000","last_became_peered":"2024-06-28T15:00:17.402735+0000","last_unstale":"2024-06-28T16:12:50.286469+0000","last_undegraded":"2024-06-28T16:12:50.286469+0000","last_fullsized":"2024-06-28T16:12:50.286469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":299,"num_read_kb":249,"num_write":100,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"260'336","reported_seq":1059,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.724393+0000","last_change":"2024-06-28T15:00:15.415658+0000","last_active":"2024-06-28T16:14:34.724393+0000","last_peered":"2024-06-28T16:14:34.724393+0000","last_clean":"2024-06-28T16:14:34.724393+0000","last_became_active":"2024-06-28T15:00:15.415197+0000","last_became_peered":"2024-06-28T15:00:15.415197+0000","last_unstale":"2024-06-28T16:14:34.724393+0000","last_undegraded":"2024-06-28T16:14:34.724393+0000","last_fullsized":"2024-06-28T16:14:34.724393+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3732560,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":648,"num_read_kb":6210,"num_write":1163,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.115855+0000","last_change":"2024-06-28T15:00:19.379957+0000","last_active":"2024-06-28T16:12:50.115855+0000","last_peered":"2024-06-28T16:12:50.115855+0000","last_clean":"2024-06-28T16:12:50.115855+0000","last_became_active":"2024-06-28T15:00:19.378270+0000","last_became_peered":"2024-06-28T15:00:19.378270+0000","last_unstale":"2024-06-28T16:12:50.115855+0000","last_undegraded":"2024-06-28T16:12:50.115855+0000","last_fullsized":"2024-06-28T16:12:50.115855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"260'101","reported_seq":646,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:03.625893+0000","last_change":"2024-06-28T15:00:21.485155+0000","last_active":"2024-06-28T16:11:03.625893+0000","last_peered":"2024-06-28T16:11:03.625893+0000","last_clean":"2024-06-28T16:11:03.625893+0000","last_became_active":"2024-06-28T15:00:21.484352+0000","last_became_peered":"2024-06-28T15:00:21.484352+0000","last_unstale":"2024-06-28T16:11:03.625893+0000","last_undegraded":"2024-06-28T16:11:03.625893+0000","last_fullsized":"2024-06-28T16:11:03.625893+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:35:15.271971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":124,"num_read_kb":119,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"260'515","reported_seq":2245,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.188813+0000","last_change":"2024-06-28T15:00:23.415013+0000","last_active":"2024-06-28T16:14:32.188813+0000","last_peered":"2024-06-28T16:14:32.188813+0000","last_clean":"2024-06-28T16:14:32.188813+0000","last_became_active":"2024-06-28T15:00:23.407490+0000","last_became_peered":"2024-06-28T15:00:23.407490+0000","last_unstale":"2024-06-28T16:14:32.188813+0000","last_undegraded":"2024-06-28T16:14:32.188813+0000","last_fullsized":"2024-06-28T16:14:32.188813+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:17:37.321606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"260'102","reported_seq":606,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.032920+0000","last_change":"2024-06-28T15:00:25.446074+0000","last_active":"2024-06-28T16:14:06.032920+0000","last_peered":"2024-06-28T16:14:06.032920+0000","last_clean":"2024-06-28T16:14:06.032920+0000","last_became_active":"2024-06-28T15:00:25.445018+0000","last_became_peered":"2024-06-28T15:00:25.445018+0000","last_unstale":"2024-06-28T16:14:06.032920+0000","last_undegraded":"2024-06-28T16:14:06.032920+0000","last_fullsized":"2024-06-28T16:14:06.032920+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:03:17.977593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":117,"num_read_kb":111,"num_write":241,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"260'78","reported_seq":654,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073934+0000","last_change":"2024-06-28T15:00:17.394823+0000","last_active":"2024-06-28T16:10:20.073934+0000","last_peered":"2024-06-28T16:10:20.073934+0000","last_clean":"2024-06-28T16:10:20.073934+0000","last_became_active":"2024-06-28T15:00:17.387086+0000","last_became_peered":"2024-06-28T15:00:17.387086+0000","last_unstale":"2024-06-28T16:10:20.073934+0000","last_undegraded":"2024-06-28T16:10:20.073934+0000","last_fullsized":"2024-06-28T16:10:20.073934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:21:49.516534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":333,"num_read_kb":275,"num_write":113,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"260'328","reported_seq":961,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.723293+0000","last_change":"2024-06-28T15:00:59.943256+0000","last_active":"2024-06-28T16:14:34.723293+0000","last_peered":"2024-06-28T16:14:34.723293+0000","last_clean":"2024-06-28T16:14:34.723293+0000","last_became_active":"2024-06-28T15:00:59.941915+0000","last_became_peered":"2024-06-28T15:00:59.941915+0000","last_unstale":"2024-06-28T16:14:34.723293+0000","last_undegraded":"2024-06-28T16:14:34.723293+0000","last_fullsized":"2024-06-28T16:14:34.723293+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5506084,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":497,"num_read_kb":57892,"num_write":1218,"num_write_kb":47404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.080513+0000","last_change":"2024-06-28T15:00:19.391494+0000","last_active":"2024-06-28T16:10:20.080513+0000","last_peered":"2024-06-28T16:10:20.080513+0000","last_clean":"2024-06-28T16:10:20.080513+0000","last_became_active":"2024-06-28T15:00:19.390613+0000","last_became_peered":"2024-06-28T15:00:19.390613+0000","last_unstale":"2024-06-28T16:10:20.080513+0000","last_undegraded":"2024-06-28T16:10:20.080513+0000","last_fullsized":"2024-06-28T16:10:20.080513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:27:57.018874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"260'324","reported_seq":932,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.759325+0000","last_change":"2024-06-28T15:00:21.392375+0000","last_active":"2024-06-28T16:14:34.759325+0000","last_peered":"2024-06-28T16:14:34.759325+0000","last_clean":"2024-06-28T16:14:34.759325+0000","last_became_active":"2024-06-28T15:00:21.391631+0000","last_became_peered":"2024-06-28T15:00:21.391631+0000","last_unstale":"2024-06-28T16:14:34.759325+0000","last_undegraded":"2024-06-28T16:14:34.759325+0000","last_fullsized":"2024-06-28T16:14:34.759325+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:25:50.181425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43414,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":516,"num_read_kb":1105,"num_write":400,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.583987+0000","last_change":"2024-06-28T15:00:23.405246+0000","last_active":"2024-06-28T16:12:55.583987+0000","last_peered":"2024-06-28T16:12:55.583987+0000","last_clean":"2024-06-28T16:12:55.583987+0000","last_became_active":"2024-06-28T15:00:23.404312+0000","last_became_peered":"2024-06-28T15:00:23.404312+0000","last_unstale":"2024-06-28T16:12:55.583987+0000","last_undegraded":"2024-06-28T16:12:55.583987+0000","last_fullsized":"2024-06-28T16:12:55.583987+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"260'131","reported_seq":642,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.235400+0000","last_change":"2024-06-28T15:00:25.425886+0000","last_active":"2024-06-28T16:14:08.235400+0000","last_peered":"2024-06-28T16:14:08.235400+0000","last_clean":"2024-06-28T16:14:08.235400+0000","last_became_active":"2024-06-28T15:00:25.424826+0000","last_became_peered":"2024-06-28T15:00:25.424826+0000","last_unstale":"2024-06-28T16:14:08.235400+0000","last_undegraded":"2024-06-28T16:14:08.235400+0000","last_fullsized":"2024-06-28T16:14:08.235400+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":100,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:56:01.501372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":159,"num_read_kb":144,"num_write":309,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"260'118","reported_seq":691,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.073868+0000","last_change":"2024-06-28T15:00:17.387490+0000","last_active":"2024-06-28T16:10:20.073868+0000","last_peered":"2024-06-28T16:10:20.073868+0000","last_clean":"2024-06-28T16:10:20.073868+0000","last_became_active":"2024-06-28T15:00:17.386756+0000","last_became_peered":"2024-06-28T15:00:17.386756+0000","last_unstale":"2024-06-28T16:10:20.073868+0000","last_undegraded":"2024-06-28T16:10:20.073868+0000","last_fullsized":"2024-06-28T16:10:20.073868+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:24:23.283000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":469,"num_read_kb":378,"num_write":166,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"260'293","reported_seq":965,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.754866+0000","last_change":"2024-06-28T15:00:59.945598+0000","last_active":"2024-06-28T16:14:34.754866+0000","last_peered":"2024-06-28T16:14:34.754866+0000","last_clean":"2024-06-28T16:14:34.754866+0000","last_became_active":"2024-06-28T15:00:59.943670+0000","last_became_peered":"2024-06-28T15:00:59.943670+0000","last_unstale":"2024-06-28T16:14:34.754866+0000","last_undegraded":"2024-06-28T16:14:34.754866+0000","last_fullsized":"2024-06-28T16:14:34.754866+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306233,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":513,"num_read_kb":66858,"num_write":1091,"num_write_kb":26064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.066682+0000","last_change":"2024-06-28T15:00:19.400037+0000","last_active":"2024-06-28T16:10:20.066682+0000","last_peered":"2024-06-28T16:10:20.066682+0000","last_clean":"2024-06-28T16:10:20.066682+0000","last_became_active":"2024-06-28T15:00:19.398245+0000","last_became_peered":"2024-06-28T15:00:19.398245+0000","last_unstale":"2024-06-28T16:10:20.066682+0000","last_undegraded":"2024-06-28T16:10:20.066682+0000","last_fullsized":"2024-06-28T16:10:20.066682+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:50.601079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"260'297","reported_seq":888,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.434499+0000","last_change":"2024-06-28T15:00:21.406496+0000","last_active":"2024-06-28T16:14:32.434499+0000","last_peered":"2024-06-28T16:14:32.434499+0000","last_clean":"2024-06-28T16:14:32.434499+0000","last_became_active":"2024-06-28T15:00:21.404701+0000","last_became_peered":"2024-06-28T15:00:21.404701+0000","last_unstale":"2024-06-28T16:14:32.434499+0000","last_undegraded":"2024-06-28T16:14:32.434499+0000","last_fullsized":"2024-06-28T16:14:32.434499+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44354,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":486,"num_read_kb":1147,"num_write":394,"num_write_kb":305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":491,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:50.079705+0000","last_change":"2024-06-28T15:00:23.416260+0000","last_active":"2024-06-28T16:10:50.079705+0000","last_peered":"2024-06-28T16:10:50.079705+0000","last_clean":"2024-06-28T16:10:50.079705+0000","last_became_active":"2024-06-28T15:00:23.415531+0000","last_became_peered":"2024-06-28T15:00:23.415531+0000","last_unstale":"2024-06-28T16:10:50.079705+0000","last_undegraded":"2024-06-28T16:10:50.079705+0000","last_fullsized":"2024-06-28T16:10:50.079705+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:14:35.301127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"260'166","reported_seq":711,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:02.151751+0000","last_change":"2024-06-28T15:00:25.554319+0000","last_active":"2024-06-28T16:13:02.151751+0000","last_peered":"2024-06-28T16:13:02.151751+0000","last_clean":"2024-06-28T16:13:02.151751+0000","last_became_active":"2024-06-28T15:00:25.553724+0000","last_became_peered":"2024-06-28T15:00:25.553724+0000","last_unstale":"2024-06-28T16:13:02.151751+0000","last_undegraded":"2024-06-28T16:13:02.151751+0000","last_fullsized":"2024-06-28T16:13:02.151751+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":555,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":217,"num_write":397,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"260'143","reported_seq":807,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.314710+0000","last_change":"2024-06-28T15:00:17.383713+0000","last_active":"2024-06-28T16:13:00.314710+0000","last_peered":"2024-06-28T16:13:00.314710+0000","last_clean":"2024-06-28T16:13:00.314710+0000","last_became_active":"2024-06-28T15:00:17.383224+0000","last_became_peered":"2024-06-28T15:00:17.383224+0000","last_unstale":"2024-06-28T16:13:00.314710+0000","last_undegraded":"2024-06-28T16:13:00.314710+0000","last_fullsized":"2024-06-28T16:13:00.314710+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":649,"num_read_kb":545,"num_write":214,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"260'301","reported_seq":1067,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.729020+0000","last_change":"2024-06-28T15:00:15.424150+0000","last_active":"2024-06-28T16:14:34.729020+0000","last_peered":"2024-06-28T16:14:34.729020+0000","last_clean":"2024-06-28T16:14:34.729020+0000","last_became_active":"2024-06-28T15:00:15.423813+0000","last_became_peered":"2024-06-28T15:00:15.423813+0000","last_unstale":"2024-06-28T16:14:34.729020+0000","last_undegraded":"2024-06-28T16:14:34.729020+0000","last_fullsized":"2024-06-28T16:14:34.729020+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5724260,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":754,"num_read_kb":37474,"num_write":1107,"num_write_kb":31583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.075092+0000","last_change":"2024-06-28T15:00:19.384145+0000","last_active":"2024-06-28T16:10:25.075092+0000","last_peered":"2024-06-28T16:10:25.075092+0000","last_clean":"2024-06-28T16:10:25.075092+0000","last_became_active":"2024-06-28T15:00:19.383307+0000","last_became_peered":"2024-06-28T15:00:19.383307+0000","last_unstale":"2024-06-28T16:10:25.075092+0000","last_undegraded":"2024-06-28T16:10:25.075092+0000","last_fullsized":"2024-06-28T16:10:25.075092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:22:34.590726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"260'286","reported_seq":858,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.177400+0000","last_change":"2024-06-28T15:00:21.396994+0000","last_active":"2024-06-28T16:14:34.177400+0000","last_peered":"2024-06-28T16:14:34.177400+0000","last_clean":"2024-06-28T16:14:34.177400+0000","last_became_active":"2024-06-28T15:00:21.396257+0000","last_became_peered":"2024-06-28T15:00:21.396257+0000","last_unstale":"2024-06-28T16:14:34.177400+0000","last_undegraded":"2024-06-28T16:14:34.177400+0000","last_fullsized":"2024-06-28T16:14:34.177400+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43992,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":439,"num_read_kb":1078,"num_write":366,"num_write_kb":297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":494,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.075301+0000","last_change":"2024-06-28T15:00:23.410088+0000","last_active":"2024-06-28T16:10:25.075301+0000","last_peered":"2024-06-28T16:10:25.075301+0000","last_clean":"2024-06-28T16:10:25.075301+0000","last_became_active":"2024-06-28T15:00:23.409107+0000","last_became_peered":"2024-06-28T15:00:23.409107+0000","last_unstale":"2024-06-28T16:10:25.075301+0000","last_undegraded":"2024-06-28T16:10:25.075301+0000","last_fullsized":"2024-06-28T16:10:25.075301+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:26:16.658429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"260'144","reported_seq":876,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:02.732078+0000","last_change":"2024-06-28T15:00:25.425176+0000","last_active":"2024-06-28T16:14:02.732078+0000","last_peered":"2024-06-28T16:14:02.732078+0000","last_clean":"2024-06-28T16:14:02.732078+0000","last_became_active":"2024-06-28T15:00:25.424376+0000","last_became_peered":"2024-06-28T15:00:25.424376+0000","last_unstale":"2024-06-28T16:14:02.732078+0000","last_undegraded":"2024-06-28T16:14:02.732078+0000","last_fullsized":"2024-06-28T16:14:02.732078+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":100,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:40:22.841457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":398,"num_read_kb":429,"num_write":339,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"260'112","reported_seq":699,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:25.075140+0000","last_change":"2024-06-28T15:00:17.403746+0000","last_active":"2024-06-28T16:10:25.075140+0000","last_peered":"2024-06-28T16:10:25.075140+0000","last_clean":"2024-06-28T16:10:25.075140+0000","last_became_active":"2024-06-28T15:00:17.403007+0000","last_became_peered":"2024-06-28T15:00:17.403007+0000","last_unstale":"2024-06-28T16:10:25.075140+0000","last_undegraded":"2024-06-28T16:10:25.075140+0000","last_fullsized":"2024-06-28T16:10:25.075140+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:10:54.255627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":478,"num_read_kb":386,"num_write":182,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"260'379","reported_seq":1047,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.657167+0000","last_change":"2024-06-28T15:00:15.415976+0000","last_active":"2024-06-28T16:14:34.657167+0000","last_peered":"2024-06-28T16:14:34.657167+0000","last_clean":"2024-06-28T16:14:34.657167+0000","last_became_active":"2024-06-28T15:00:15.414969+0000","last_became_peered":"2024-06-28T15:00:15.414969+0000","last_unstale":"2024-06-28T16:14:34.657167+0000","last_undegraded":"2024-06-28T16:14:34.657167+0000","last_fullsized":"2024-06-28T16:14:34.657167+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":300,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1604505,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":539,"num_read_kb":3182,"num_write":1321,"num_write_kb":13350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"24'1","reported_seq":509,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.120470+0000","last_change":"2024-06-28T15:00:19.403928+0000","last_active":"2024-06-28T16:13:10.120470+0000","last_peered":"2024-06-28T16:13:10.120470+0000","last_clean":"2024-06-28T16:13:10.120470+0000","last_became_active":"2024-06-28T15:00:19.403344+0000","last_became_peered":"2024-06-28T15:00:19.403344+0000","last_unstale":"2024-06-28T16:13:10.120470+0000","last_undegraded":"2024-06-28T16:13:10.120470+0000","last_fullsized":"2024-06-28T16:13:10.120470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"260'293","reported_seq":893,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.393958+0000","last_change":"2024-06-28T15:00:21.388828+0000","last_active":"2024-06-28T16:14:34.393958+0000","last_peered":"2024-06-28T16:14:34.393958+0000","last_clean":"2024-06-28T16:14:34.393958+0000","last_became_active":"2024-06-28T15:00:21.387539+0000","last_became_peered":"2024-06-28T15:00:21.387539+0000","last_unstale":"2024-06-28T16:14:34.393958+0000","last_undegraded":"2024-06-28T16:14:34.393958+0000","last_fullsized":"2024-06-28T16:14:34.393958+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:54:05.779834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44825,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":494,"num_read_kb":1086,"num_write":388,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.584120+0000","last_change":"2024-06-28T15:00:23.417454+0000","last_active":"2024-06-28T16:12:55.584120+0000","last_peered":"2024-06-28T16:12:55.584120+0000","last_clean":"2024-06-28T16:12:55.584120+0000","last_became_active":"2024-06-28T15:00:23.416363+0000","last_became_peered":"2024-06-28T15:00:23.416363+0000","last_unstale":"2024-06-28T16:12:55.584120+0000","last_undegraded":"2024-06-28T16:12:55.584120+0000","last_fullsized":"2024-06-28T16:12:55.584120+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"260'126","reported_seq":623,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:05.111348+0000","last_change":"2024-06-28T15:00:25.429682+0000","last_active":"2024-06-28T16:13:05.111348+0000","last_peered":"2024-06-28T16:13:05.111348+0000","last_clean":"2024-06-28T16:13:05.111348+0000","last_became_active":"2024-06-28T15:00:25.428959+0000","last_became_peered":"2024-06-28T15:00:25.428959+0000","last_unstale":"2024-06-28T16:13:05.111348+0000","last_undegraded":"2024-06-28T16:13:05.111348+0000","last_fullsized":"2024-06-28T16:13:05.111348+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":100,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:34:41.858483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":814,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":97,"num_write":299,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"260'174","reported_seq":762,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.130967+0000","last_change":"2024-06-28T15:00:17.407419+0000","last_active":"2024-06-28T16:14:00.130967+0000","last_peered":"2024-06-28T16:14:00.130967+0000","last_clean":"2024-06-28T16:14:00.130967+0000","last_became_active":"2024-06-28T15:00:17.406872+0000","last_became_peered":"2024-06-28T15:00:17.406872+0000","last_unstale":"2024-06-28T16:14:00.130967+0000","last_undegraded":"2024-06-28T16:14:00.130967+0000","last_fullsized":"2024-06-28T16:14:00.130967+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":100,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":660,"num_read_kb":511,"num_write":250,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"260'285","reported_seq":909,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.644073+0000","last_change":"2024-06-28T15:00:15.409170+0000","last_active":"2024-06-28T16:14:34.644073+0000","last_peered":"2024-06-28T16:14:34.644073+0000","last_clean":"2024-06-28T16:14:34.644073+0000","last_became_active":"2024-06-28T15:00:15.408337+0000","last_became_peered":"2024-06-28T15:00:15.408337+0000","last_unstale":"2024-06-28T16:14:34.644073+0000","last_undegraded":"2024-06-28T16:14:34.644073+0000","last_fullsized":"2024-06-28T16:14:34.644073+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T15:51:23.373884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":404,"num_read_kb":33775,"num_write":1000,"num_write_kb":10945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"260'72","reported_seq":675,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.130910+0000","last_change":"2024-06-28T15:00:17.404713+0000","last_active":"2024-06-28T16:14:00.130910+0000","last_peered":"2024-06-28T16:14:00.130910+0000","last_clean":"2024-06-28T16:14:00.130910+0000","last_became_active":"2024-06-28T15:00:17.403938+0000","last_became_peered":"2024-06-28T15:00:17.403938+0000","last_unstale":"2024-06-28T16:14:00.130910+0000","last_undegraded":"2024-06-28T16:14:00.130910+0000","last_fullsized":"2024-06-28T16:14:00.130910+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":319,"num_read_kb":277,"num_write":96,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"260'322","reported_seq":996,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.641104+0000","last_change":"2024-06-28T15:00:15.420697+0000","last_active":"2024-06-28T16:14:34.641104+0000","last_peered":"2024-06-28T16:14:34.641104+0000","last_clean":"2024-06-28T16:14:34.641104+0000","last_became_active":"2024-06-28T15:00:15.420261+0000","last_became_peered":"2024-06-28T15:00:15.420261+0000","last_unstale":"2024-06-28T16:14:34.641104+0000","last_undegraded":"2024-06-28T16:14:34.641104+0000","last_fullsized":"2024-06-28T16:14:34.641104+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262189,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":529,"num_read_kb":23662,"num_write":1259,"num_write_kb":28174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"260'38","reported_seq":596,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:50.087084+0000","last_change":"2024-06-28T15:00:17.394654+0000","last_active":"2024-06-28T16:11:50.087084+0000","last_peered":"2024-06-28T16:11:50.087084+0000","last_clean":"2024-06-28T16:11:50.087084+0000","last_became_active":"2024-06-28T15:00:17.393889+0000","last_became_peered":"2024-06-28T15:00:17.393889+0000","last_unstale":"2024-06-28T16:11:50.087084+0000","last_undegraded":"2024-06-28T16:11:50.087084+0000","last_fullsized":"2024-06-28T16:11:50.087084+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:22:39.330270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":152,"num_write":58,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"260'358","reported_seq":999,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.728265+0000","last_change":"2024-06-28T15:00:15.426603+0000","last_active":"2024-06-28T16:14:34.728265+0000","last_peered":"2024-06-28T16:14:34.728265+0000","last_clean":"2024-06-28T16:14:34.728265+0000","last_became_active":"2024-06-28T15:00:15.418773+0000","last_became_peered":"2024-06-28T15:00:15.418773+0000","last_unstale":"2024-06-28T16:14:34.728265+0000","last_undegraded":"2024-06-28T16:14:34.728265+0000","last_fullsized":"2024-06-28T16:14:34.728265+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:02:14.247776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8714448,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":553,"num_read_kb":16492,"num_write":1257,"num_write_kb":37641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"260'142","reported_seq":788,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.130808+0000","last_change":"2024-06-28T15:00:17.398608+0000","last_active":"2024-06-28T16:14:00.130808+0000","last_peered":"2024-06-28T16:14:00.130808+0000","last_clean":"2024-06-28T16:14:00.130808+0000","last_became_active":"2024-06-28T15:00:17.398357+0000","last_became_peered":"2024-06-28T15:00:17.398357+0000","last_unstale":"2024-06-28T16:14:00.130808+0000","last_undegraded":"2024-06-28T16:14:00.130808+0000","last_fullsized":"2024-06-28T16:14:00.130808+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":623,"num_read_kb":529,"num_write":221,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"260'332","reported_seq":977,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.723568+0000","last_change":"2024-06-28T15:00:15.391599+0000","last_active":"2024-06-28T16:14:34.723568+0000","last_peered":"2024-06-28T16:14:34.723568+0000","last_clean":"2024-06-28T16:14:34.723568+0000","last_became_active":"2024-06-28T15:00:15.391150+0000","last_became_peered":"2024-06-28T15:00:15.391150+0000","last_unstale":"2024-06-28T16:14:34.723568+0000","last_undegraded":"2024-06-28T16:14:34.723568+0000","last_fullsized":"2024-06-28T16:14:34.723568+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":300,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1605357,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":453,"num_read_kb":84462,"num_write":1265,"num_write_kb":41774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"260'191","reported_seq":789,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:10.303057+0000","last_change":"2024-06-28T15:00:59.807172+0000","last_active":"2024-06-28T16:14:10.303057+0000","last_peered":"2024-06-28T16:14:10.303057+0000","last_clean":"2024-06-28T16:14:10.303057+0000","last_became_active":"2024-06-28T15:00:59.806725+0000","last_became_peered":"2024-06-28T15:00:59.806725+0000","last_unstale":"2024-06-28T16:14:10.303057+0000","last_undegraded":"2024-06-28T16:14:10.303057+0000","last_fullsized":"2024-06-28T16:14:10.303057+0000","mapping_epoch":32,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":759,"num_read_kb":602,"num_write":307,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"260'334","reported_seq":999,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.696329+0000","last_change":"2024-06-28T15:00:15.383729+0000","last_active":"2024-06-28T16:14:34.696329+0000","last_peered":"2024-06-28T16:14:34.696329+0000","last_clean":"2024-06-28T16:14:34.696329+0000","last_became_active":"2024-06-28T15:00:15.383350+0000","last_became_peered":"2024-06-28T15:00:15.383350+0000","last_unstale":"2024-06-28T16:14:34.696329+0000","last_undegraded":"2024-06-28T16:14:34.696329+0000","last_fullsized":"2024-06-28T16:14:34.696329+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263219,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":502,"num_read_kb":41336,"num_write":1201,"num_write_kb":38478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0,7],"acting":[3,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"260'397","reported_seq":997,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.085126+0000","last_change":"2024-06-28T15:00:17.408600+0000","last_active":"2024-06-28T16:11:10.085126+0000","last_peered":"2024-06-28T16:11:10.085126+0000","last_clean":"2024-06-28T16:11:10.085126+0000","last_became_active":"2024-06-28T15:00:17.404844+0000","last_became_peered":"2024-06-28T15:00:17.404844+0000","last_unstale":"2024-06-28T16:11:10.085126+0000","last_undegraded":"2024-06-28T16:11:10.085126+0000","last_fullsized":"2024-06-28T16:11:10.085126+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":300,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:17:34.507155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1527,"num_read_kb":1156,"num_write":658,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"260'343","reported_seq":1028,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.751903+0000","last_change":"2024-06-28T15:00:15.401905+0000","last_active":"2024-06-28T16:14:34.751903+0000","last_peered":"2024-06-28T16:14:34.751903+0000","last_clean":"2024-06-28T16:14:34.751903+0000","last_became_active":"2024-06-28T15:00:15.401275+0000","last_became_peered":"2024-06-28T15:00:15.401275+0000","last_unstale":"2024-06-28T16:14:34.751903+0000","last_undegraded":"2024-06-28T16:14:34.751903+0000","last_fullsized":"2024-06-28T16:14:34.751903+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":559,"num_read_kb":45669,"num_write":1313,"num_write_kb":19663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"260'221","reported_seq":842,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.318677+0000","last_change":"2024-06-28T15:00:17.383279+0000","last_active":"2024-06-28T16:13:15.318677+0000","last_peered":"2024-06-28T16:13:15.318677+0000","last_clean":"2024-06-28T16:13:15.318677+0000","last_became_active":"2024-06-28T15:00:17.382739+0000","last_became_peered":"2024-06-28T15:00:17.382739+0000","last_unstale":"2024-06-28T16:13:15.318677+0000","last_undegraded":"2024-06-28T16:13:15.318677+0000","last_fullsized":"2024-06-28T16:13:15.318677+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":866,"num_read_kb":690,"num_write":335,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"260'298","reported_seq":928,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.600936+0000","last_change":"2024-06-28T15:00:15.421880+0000","last_active":"2024-06-28T16:14:34.600936+0000","last_peered":"2024-06-28T16:14:34.600936+0000","last_clean":"2024-06-28T16:14:34.600936+0000","last_became_active":"2024-06-28T15:00:15.421189+0000","last_became_peered":"2024-06-28T15:00:15.421189+0000","last_unstale":"2024-06-28T16:14:34.600936+0000","last_undegraded":"2024-06-28T16:14:34.600936+0000","last_fullsized":"2024-06-28T16:14:34.600936+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:03:21.461240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7105903,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":419,"num_read_kb":19450,"num_write":1148,"num_write_kb":38199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"260'79","reported_seq":639,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:11:10.083923+0000","last_change":"2024-06-28T15:00:17.396892+0000","last_active":"2024-06-28T16:11:10.083923+0000","last_peered":"2024-06-28T16:11:10.083923+0000","last_clean":"2024-06-28T16:11:10.083923+0000","last_became_active":"2024-06-28T15:00:17.396125+0000","last_became_peered":"2024-06-28T15:00:17.396125+0000","last_unstale":"2024-06-28T16:11:10.083923+0000","last_undegraded":"2024-06-28T16:11:10.083923+0000","last_fullsized":"2024-06-28T16:11:10.083923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:46:54.077428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":321,"num_read_kb":257,"num_write":118,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"260'270","reported_seq":928,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.754134+0000","last_change":"2024-06-28T15:00:15.408891+0000","last_active":"2024-06-28T16:14:34.754134+0000","last_peered":"2024-06-28T16:14:34.754134+0000","last_clean":"2024-06-28T16:14:34.754134+0000","last_became_active":"2024-06-28T15:00:15.408375+0000","last_became_peered":"2024-06-28T15:00:15.408375+0000","last_unstale":"2024-06-28T16:14:34.754134+0000","last_undegraded":"2024-06-28T16:14:34.754134+0000","last_fullsized":"2024-06-28T16:14:34.754134+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":484,"num_read_kb":6587,"num_write":1005,"num_write_kb":17135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"260'62","reported_seq":637,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:40.580068+0000","last_change":"2024-06-28T15:00:17.380077+0000","last_active":"2024-06-28T16:12:40.580068+0000","last_peered":"2024-06-28T16:12:40.580068+0000","last_clean":"2024-06-28T16:12:40.580068+0000","last_became_active":"2024-06-28T15:00:17.379374+0000","last_became_peered":"2024-06-28T15:00:17.379374+0000","last_unstale":"2024-06-28T16:12:40.580068+0000","last_undegraded":"2024-06-28T16:12:40.580068+0000","last_fullsized":"2024-06-28T16:12:40.580068+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":216,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"260'367","reported_seq":1011,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.749490+0000","last_change":"2024-06-28T15:00:15.427903+0000","last_active":"2024-06-28T16:14:34.749490+0000","last_peered":"2024-06-28T16:14:34.749490+0000","last_clean":"2024-06-28T16:14:34.749490+0000","last_became_active":"2024-06-28T15:00:15.427171+0000","last_became_peered":"2024-06-28T15:00:15.427171+0000","last_unstale":"2024-06-28T16:14:34.749490+0000","last_undegraded":"2024-06-28T16:14:34.749490+0000","last_fullsized":"2024-06-28T16:14:34.749490+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":300,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:22:01.482231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":723921,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":536,"num_read_kb":26554,"num_write":1152,"num_write_kb":52254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"260'230","reported_seq":851,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:10:20.074074+0000","last_change":"2024-06-28T15:00:17.409608+0000","last_active":"2024-06-28T16:10:20.074074+0000","last_peered":"2024-06-28T16:10:20.074074+0000","last_clean":"2024-06-28T16:10:20.074074+0000","last_became_active":"2024-06-28T15:00:17.403760+0000","last_became_peered":"2024-06-28T15:00:17.403760+0000","last_unstale":"2024-06-28T16:10:20.074074+0000","last_undegraded":"2024-06-28T16:10:20.074074+0000","last_fullsized":"2024-06-28T16:10:20.074074+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:58:04.722934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":902,"num_read_kb":724,"num_write":338,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"260'282","reported_seq":930,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.744928+0000","last_change":"2024-06-28T15:00:15.376797+0000","last_active":"2024-06-28T16:14:34.744928+0000","last_peered":"2024-06-28T16:14:34.744928+0000","last_clean":"2024-06-28T16:14:34.744928+0000","last_became_active":"2024-06-28T15:00:15.376014+0000","last_became_peered":"2024-06-28T15:00:15.376014+0000","last_unstale":"2024-06-28T16:14:34.744928+0000","last_undegraded":"2024-06-28T16:14:34.744928+0000","last_fullsized":"2024-06-28T16:14:34.744928+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462789,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":409,"num_read_kb":32504,"num_write":910,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"260'70","reported_seq":659,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:35.282944+0000","last_change":"2024-06-28T15:00:17.402274+0000","last_active":"2024-06-28T16:12:35.282944+0000","last_peered":"2024-06-28T16:12:35.282944+0000","last_clean":"2024-06-28T16:12:35.282944+0000","last_became_active":"2024-06-28T15:00:17.401917+0000","last_became_peered":"2024-06-28T15:00:17.401917+0000","last_unstale":"2024-06-28T16:12:35.282944+0000","last_undegraded":"2024-06-28T16:12:35.282944+0000","last_fullsized":"2024-06-28T16:12:35.282944+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":295,"num_read_kb":247,"num_write":98,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"260'337","reported_seq":973,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.694565+0000","last_change":"2024-06-28T15:00:15.434365+0000","last_active":"2024-06-28T16:14:34.694565+0000","last_peered":"2024-06-28T16:14:34.694565+0000","last_clean":"2024-06-28T16:14:34.694565+0000","last_became_active":"2024-06-28T15:00:15.431444+0000","last_became_peered":"2024-06-28T15:00:15.431444+0000","last_unstale":"2024-06-28T16:14:34.694565+0000","last_undegraded":"2024-06-28T16:14:34.694565+0000","last_fullsized":"2024-06-28T16:14:34.694565+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:01:51.800790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":461,"num_read_kb":557,"num_write":1266,"num_write_kb":33580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,3],"acting":[5,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"260'424","reported_seq":1178,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.101339+0000","last_change":"2024-06-28T15:00:17.403422+0000","last_active":"2024-06-28T16:12:20.101339+0000","last_peered":"2024-06-28T16:12:20.101339+0000","last_clean":"2024-06-28T16:12:20.101339+0000","last_became_active":"2024-06-28T15:00:17.402842+0000","last_became_peered":"2024-06-28T15:00:17.402842+0000","last_unstale":"2024-06-28T16:12:20.101339+0000","last_undegraded":"2024-06-28T16:12:20.101339+0000","last_fullsized":"2024-06-28T16:12:20.101339+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":400,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:26:05.866773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1825,"num_read_kb":1465,"num_write":755,"num_write_kb":468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"260'319","reported_seq":965,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.708660+0000","last_change":"2024-06-28T15:00:15.415547+0000","last_active":"2024-06-28T16:14:34.708660+0000","last_peered":"2024-06-28T16:14:34.708660+0000","last_clean":"2024-06-28T16:14:34.708660+0000","last_became_active":"2024-06-28T15:00:15.415009+0000","last_became_peered":"2024-06-28T15:00:15.415009+0000","last_unstale":"2024-06-28T16:14:34.708660+0000","last_undegraded":"2024-06-28T16:14:34.708660+0000","last_fullsized":"2024-06-28T16:14:34.708660+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242944,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":479,"num_read_kb":2084,"num_write":1202,"num_write_kb":29565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"260'26","reported_seq":574,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:20.108218+0000","last_change":"2024-06-28T15:00:17.390583+0000","last_active":"2024-06-28T16:12:20.108218+0000","last_peered":"2024-06-28T16:12:20.108218+0000","last_clean":"2024-06-28T16:12:20.108218+0000","last_became_active":"2024-06-28T15:00:17.389844+0000","last_became_peered":"2024-06-28T15:00:17.389844+0000","last_unstale":"2024-06-28T16:12:20.108218+0000","last_undegraded":"2024-06-28T16:12:20.108218+0000","last_fullsized":"2024-06-28T16:12:20.108218+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:41:17.401424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":112,"num_write":34,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"260'305","reported_seq":936,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.719398+0000","last_change":"2024-06-28T15:00:15.398948+0000","last_active":"2024-06-28T16:14:34.719398+0000","last_peered":"2024-06-28T16:14:34.719398+0000","last_clean":"2024-06-28T16:14:34.719398+0000","last_became_active":"2024-06-28T15:00:15.397986+0000","last_became_peered":"2024-06-28T15:00:15.397986+0000","last_unstale":"2024-06-28T16:14:34.719398+0000","last_undegraded":"2024-06-28T16:14:34.719398+0000","last_fullsized":"2024-06-28T16:14:34.719398+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:02:19.841719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079926,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":433,"num_read_kb":10373,"num_write":1047,"num_write_kb":25421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"260'192","reported_seq":858,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.608235+0000","last_change":"2024-06-28T15:00:17.395455+0000","last_active":"2024-06-28T16:14:35.608235+0000","last_peered":"2024-06-28T16:14:35.608235+0000","last_clean":"2024-06-28T16:14:35.608235+0000","last_became_active":"2024-06-28T15:00:17.394783+0000","last_became_peered":"2024-06-28T15:00:17.394783+0000","last_unstale":"2024-06-28T16:14:35.608235+0000","last_undegraded":"2024-06-28T16:14:35.608235+0000","last_fullsized":"2024-06-28T16:14:35.608235+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":852,"num_read_kb":715,"num_write":335,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"260'302","reported_seq":914,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.707348+0000","last_change":"2024-06-28T15:00:59.939157+0000","last_active":"2024-06-28T16:14:34.707348+0000","last_peered":"2024-06-28T16:14:34.707348+0000","last_clean":"2024-06-28T16:14:34.707348+0000","last_became_active":"2024-06-28T15:00:59.938004+0000","last_became_peered":"2024-06-28T15:00:59.938004+0000","last_unstale":"2024-06-28T16:14:34.707348+0000","last_undegraded":"2024-06-28T16:14:34.707348+0000","last_fullsized":"2024-06-28T16:14:34.707348+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6542288,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":437,"num_read_kb":25077,"num_write":1246,"num_write_kb":38037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,3],"acting":[1,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"260'392","reported_seq":1063,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.336853+0000","last_change":"2024-06-28T15:00:17.374446+0000","last_active":"2024-06-28T16:14:35.336853+0000","last_peered":"2024-06-28T16:14:35.336853+0000","last_clean":"2024-06-28T16:14:35.336853+0000","last_became_active":"2024-06-28T15:00:17.374143+0000","last_became_peered":"2024-06-28T15:00:17.374143+0000","last_unstale":"2024-06-28T16:14:35.336853+0000","last_undegraded":"2024-06-28T16:14:35.336853+0000","last_fullsized":"2024-06-28T16:14:35.336853+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":300,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1520,"num_read_kb":1191,"num_write":653,"num_write_kb":382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"260'255","reported_seq":919,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.743465+0000","last_change":"2024-06-28T15:00:15.430833+0000","last_active":"2024-06-28T16:14:34.743465+0000","last_peered":"2024-06-28T16:14:34.743465+0000","last_clean":"2024-06-28T16:14:34.743465+0000","last_became_active":"2024-06-28T15:00:15.430228+0000","last_became_peered":"2024-06-28T15:00:15.430228+0000","last_unstale":"2024-06-28T16:14:34.743465+0000","last_undegraded":"2024-06-28T16:14:34.743465+0000","last_fullsized":"2024-06-28T16:14:34.743465+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170399,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":449,"num_read_kb":6008,"num_write":895,"num_write_kb":11243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"260'134","reported_seq":778,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:05.104760+0000","last_change":"2024-06-28T15:00:17.382276+0000","last_active":"2024-06-28T16:12:05.104760+0000","last_peered":"2024-06-28T16:12:05.104760+0000","last_clean":"2024-06-28T16:12:05.104760+0000","last_became_active":"2024-06-28T15:00:17.381562+0000","last_became_peered":"2024-06-28T15:00:17.381562+0000","last_unstale":"2024-06-28T16:12:05.104760+0000","last_undegraded":"2024-06-28T16:12:05.104760+0000","last_fullsized":"2024-06-28T16:12:05.104760+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:32:13.816417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":479,"num_write":190,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"260'496","reported_seq":1185,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.737064+0000","last_change":"2024-06-28T15:00:15.419485+0000","last_active":"2024-06-28T16:14:34.737064+0000","last_peered":"2024-06-28T16:14:34.737064+0000","last_clean":"2024-06-28T16:14:34.737064+0000","last_became_active":"2024-06-28T15:00:15.418850+0000","last_became_peered":"2024-06-28T15:00:15.418850+0000","last_unstale":"2024-06-28T16:14:34.737064+0000","last_undegraded":"2024-06-28T16:14:34.737064+0000","last_fullsized":"2024-06-28T16:14:34.737064+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":400,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:53:16.536329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":907,"num_read_kb":23220,"num_write":1415,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"260'338","reported_seq":1022,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.749225+0000","last_change":"2024-06-28T15:00:15.409457+0000","last_active":"2024-06-28T16:14:34.749225+0000","last_peered":"2024-06-28T16:14:34.749225+0000","last_clean":"2024-06-28T16:14:34.749225+0000","last_became_active":"2024-06-28T15:00:15.408158+0000","last_became_peered":"2024-06-28T15:00:15.408158+0000","last_unstale":"2024-06-28T16:14:34.749225+0000","last_undegraded":"2024-06-28T16:14:34.749225+0000","last_fullsized":"2024-06-28T16:14:34.749225+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:01:20.858547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6690738,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":568,"num_read_kb":38558,"num_write":1259,"num_write_kb":46809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"260'77","reported_seq":646,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:05.586931+0000","last_change":"2024-06-28T15:00:17.380351+0000","last_active":"2024-06-28T16:13:05.586931+0000","last_peered":"2024-06-28T16:13:05.586931+0000","last_clean":"2024-06-28T16:13:05.586931+0000","last_became_active":"2024-06-28T15:00:17.379771+0000","last_became_peered":"2024-06-28T15:00:17.379771+0000","last_unstale":"2024-06-28T16:13:05.586931+0000","last_undegraded":"2024-06-28T16:13:05.586931+0000","last_fullsized":"2024-06-28T16:13:05.586931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":249,"num_write":109,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"260'387","reported_seq":1102,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.743355+0000","last_change":"2024-06-28T15:00:15.404986+0000","last_active":"2024-06-28T16:14:34.743355+0000","last_peered":"2024-06-28T16:14:34.743355+0000","last_clean":"2024-06-28T16:14:34.743355+0000","last_became_active":"2024-06-28T15:00:15.401324+0000","last_became_peered":"2024-06-28T15:00:15.401324+0000","last_unstale":"2024-06-28T16:14:34.743355+0000","last_undegraded":"2024-06-28T16:14:34.743355+0000","last_fullsized":"2024-06-28T16:14:34.743355+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":300,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079973,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":661,"num_read_kb":1946,"num_write":1429,"num_write_kb":27589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"260'176","reported_seq":777,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:14.583638+0000","last_change":"2024-06-28T15:00:17.383615+0000","last_active":"2024-06-28T16:13:14.583638+0000","last_peered":"2024-06-28T16:13:14.583638+0000","last_clean":"2024-06-28T16:13:14.583638+0000","last_became_active":"2024-06-28T15:00:17.382774+0000","last_became_peered":"2024-06-28T15:00:17.382774+0000","last_unstale":"2024-06-28T16:13:14.583638+0000","last_undegraded":"2024-06-28T16:13:14.583638+0000","last_fullsized":"2024-06-28T16:13:14.583638+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:39:45.866222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":719,"num_read_kb":570,"num_write":272,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5652,"num_read_kb":3987,"num_write":2073,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2470,"num_omap_keys":5,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1079,"ondisk_log_size":1079,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19748,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":13201,"num_read_kb":13000,"num_write":15927,"num_write_kb":4660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":39496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1836,"ondisk_log_size":1836,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1424454,"num_objects":198,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":18162,"num_read_kb":37754,"num_write":15149,"num_write_kb":11215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1872,"ondisk_log_size":1872,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1644,"num_read_kb":1954,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87319,"num_read_kb":70822,"num_write":35298,"num_write_kb":17822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3526,"ondisk_log_size":3526,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":149605345,"num_objects":433,"num_object_clones":0,"num_object_copies":1732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":433,"num_whiteouts":0,"num_read":34839,"num_read_kb":1786510,"num_write":77776,"num_write_kb":1982911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202506240,"data_stored":202506240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3438,"ondisk_log_size":3438,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":129,"num_read_kb":165927,"num_write":154,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":137,"num_read_kb":186344,"num_write":154,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575746,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356336,"kb_used_data":36244,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":94015504,"statfs":{"total":96636764160,"available":96271876096,"internally_reserved":0,"allocated":37113856,"data_stored":35129654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":327612874},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575746,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":335548,"kb_used_data":43380,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":94036292,"statfs":{"total":96636764160,"available":96293163008,"internally_reserved":0,"allocated":44421120,"data_stored":42495981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":299170249},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575748,"num_pgs":114,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201592,"kb_used_data":35592,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94170248,"statfs":{"total":96636764160,"available":96430333952,"internally_reserved":0,"allocated":36446208,"data_stored":34448502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575748,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":146836,"kb_used_data":43468,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94225004,"statfs":{"total":96636764160,"available":96486404096,"internally_reserved":0,"allocated":44511232,"data_stored":42596897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303384,"kb_used_data":29196,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":94068456,"statfs":{"total":96636764160,"available":96326098944,"internally_reserved":0,"allocated":29896704,"data_stored":27928906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":280623559},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575749,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137428,"kb_used_data":29820,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94234412,"statfs":{"total":96636764160,"available":96496037888,"internally_reserved":0,"allocated":30535680,"data_stored":28610104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575746,"num_pgs":120,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201576,"kb_used_data":39968,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94170264,"statfs":{"total":96636764160,"available":96430350336,"internally_reserved":0,"allocated":40927232,"data_stored":38982339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575749,"num_pgs":116,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287396,"kb_used_data":39644,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":94084444,"statfs":{"total":96636764160,"available":96342470656,"internally_reserved":0,"allocated":40595456,"data_stored":38600327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":253688261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25272320,"data_stored":25272320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25198592,"data_stored":25198592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21118976,"data_stored":21118976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19386368,"data_stored":19386368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29863936,"data_stored":29863936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20664320,"data_stored":20664320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28712960,"data_stored":28712960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32288768,"data_stored":32288768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":7701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":4574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":4436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":5746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-28T16:14:42.175 INFO:tasks.ceph:pgid 68.1a last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.176 INFO:tasks.ceph:pgid 67.1d last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.176 INFO:tasks.ceph:pgid 66.1c last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.176 INFO:tasks.ceph:pgid 68.1b last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.176 INFO:tasks.ceph:pgid 67.1c last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.176 INFO:tasks.ceph:pgid 66.1d last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.177 INFO:tasks.ceph:pgid 65.1e last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.177 INFO:tasks.ceph:pgid 68.18 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.177 INFO:tasks.ceph:pgid 67.1f last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.177 INFO:tasks.ceph:pgid 66.1e last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.177 INFO:tasks.ceph:pgid 65.1d last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.177 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.178 INFO:tasks.ceph:pgid 67.1e last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.178 INFO:tasks.ceph:pgid 66.1f last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.178 INFO:tasks.ceph:pgid 65.1c last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.178 INFO:tasks.ceph:pgid 68.1e last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.178 INFO:tasks.ceph:pgid 67.19 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.178 INFO:tasks.ceph:pgid 66.18 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.179 INFO:tasks.ceph:pgid 65.1b last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.179 INFO:tasks.ceph:pgid 68.1f last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.179 INFO:tasks.ceph:pgid 67.18 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.179 INFO:tasks.ceph:pgid 66.19 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.179 INFO:tasks.ceph:pgid 65.1a last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.179 INFO:tasks.ceph:pgid 68.1c last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.180 INFO:tasks.ceph:pgid 67.1b last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.180 INFO:tasks.ceph:pgid 66.1a last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.180 INFO:tasks.ceph:pgid 65.19 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.180 INFO:tasks.ceph:pgid 68.1d last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.180 INFO:tasks.ceph:pgid 67.1a last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.180 INFO:tasks.ceph:pgid 66.1b last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.181 INFO:tasks.ceph:pgid 65.18 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.181 INFO:tasks.ceph:pgid 68.12 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.181 INFO:tasks.ceph:pgid 67.15 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.181 INFO:tasks.ceph:pgid 66.14 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.181 INFO:tasks.ceph:pgid 65.17 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.181 INFO:tasks.ceph:pgid 68.13 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.182 INFO:tasks.ceph:pgid 67.14 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.182 INFO:tasks.ceph:pgid 66.15 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.182 INFO:tasks.ceph:pgid 68.10 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.182 INFO:tasks.ceph:pgid 67.17 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.182 INFO:tasks.ceph:pgid 66.16 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.182 INFO:tasks.ceph:pgid 65.15 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.183 INFO:tasks.ceph:pgid 68.11 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.183 INFO:tasks.ceph:pgid 67.16 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.183 INFO:tasks.ceph:pgid 66.17 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.183 INFO:tasks.ceph:pgid 65.14 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.183 INFO:tasks.ceph:pgid 68.16 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.183 INFO:tasks.ceph:pgid 67.11 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.183 INFO:tasks.ceph:pgid 66.10 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.184 INFO:tasks.ceph:pgid 65.13 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.184 INFO:tasks.ceph:pgid 68.17 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.184 INFO:tasks.ceph:pgid 67.10 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.184 INFO:tasks.ceph:pgid 66.11 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.184 INFO:tasks.ceph:pgid 65.12 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.184 INFO:tasks.ceph:pgid 68.14 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.185 INFO:tasks.ceph:pgid 67.13 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.185 INFO:tasks.ceph:pgid 66.12 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.185 INFO:tasks.ceph:pgid 65.11 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.185 INFO:tasks.ceph:pgid 68.15 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.185 INFO:tasks.ceph:pgid 67.12 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.185 INFO:tasks.ceph:pgid 66.13 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.185 INFO:tasks.ceph:pgid 65.10 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.186 INFO:tasks.ceph:pgid 68.a last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.186 INFO:tasks.ceph:pgid 67.d last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.186 INFO:tasks.ceph:pgid 66.c last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.186 INFO:tasks.ceph:pgid 65.f last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.186 INFO:tasks.ceph:pgid 68.b last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.186 INFO:tasks.ceph:pgid 67.c last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.187 INFO:tasks.ceph:pgid 66.d last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.187 INFO:tasks.ceph:pgid 65.e last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.187 INFO:tasks.ceph:pgid 68.8 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.187 INFO:tasks.ceph:pgid 67.f last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.187 INFO:tasks.ceph:pgid 66.e last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.187 INFO:tasks.ceph:pgid 65.d last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.188 INFO:tasks.ceph:pgid 68.9 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.188 INFO:tasks.ceph:pgid 67.e last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.188 INFO:tasks.ceph:pgid 66.f last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.188 INFO:tasks.ceph:pgid 65.c last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.188 INFO:tasks.ceph:pgid 68.e last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.188 INFO:tasks.ceph:pgid 67.9 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.188 INFO:tasks.ceph:pgid 66.8 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.189 INFO:tasks.ceph:pgid 68.f last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.189 INFO:tasks.ceph:pgid 67.8 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.189 INFO:tasks.ceph:pgid 65.a last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.189 INFO:tasks.ceph:pgid 68.c last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.189 INFO:tasks.ceph:pgid 67.b last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.189 INFO:tasks.ceph:pgid 66.a last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.190 INFO:tasks.ceph:pgid 65.9 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.190 INFO:tasks.ceph:pgid 68.d last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.190 INFO:tasks.ceph:pgid 67.a last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.190 INFO:tasks.ceph:pgid 66.b last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.190 INFO:tasks.ceph:pgid 65.8 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.191 INFO:tasks.ceph:pgid 68.2 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.191 INFO:tasks.ceph:pgid 67.5 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.191 INFO:tasks.ceph:pgid 66.4 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.191 INFO:tasks.ceph:pgid 65.7 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.191 INFO:tasks.ceph:pgid 68.3 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.191 INFO:tasks.ceph:pgid 67.4 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.191 INFO:tasks.ceph:pgid 66.5 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.192 INFO:tasks.ceph:pgid 65.6 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.192 INFO:tasks.ceph:pgid 68.0 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.192 INFO:tasks.ceph:pgid 67.7 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.192 INFO:tasks.ceph:pgid 66.6 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.192 INFO:tasks.ceph:pgid 65.5 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.193 INFO:tasks.ceph:pgid 68.1 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.193 INFO:tasks.ceph:pgid 67.6 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.193 INFO:tasks.ceph:pgid 66.7 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.193 INFO:tasks.ceph:pgid 65.4 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.193 INFO:tasks.ceph:pgid 68.6 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.193 INFO:tasks.ceph:pgid 67.1 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.193 INFO:tasks.ceph:pgid 66.0 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.194 INFO:tasks.ceph:pgid 65.3 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.194 INFO:tasks.ceph:pgid 68.7 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.194 INFO:tasks.ceph:pgid 67.0 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.194 INFO:tasks.ceph:pgid 66.1 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.194 INFO:tasks.ceph:pgid 65.2 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.194 INFO:tasks.ceph:pgid 68.4 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.195 INFO:tasks.ceph:pgid 67.3 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.195 INFO:tasks.ceph:pgid 65.1 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.195 INFO:tasks.ceph:pgid 68.5 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.195 INFO:tasks.ceph:pgid 67.2 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.195 INFO:tasks.ceph:pgid 66.3 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.195 INFO:tasks.ceph:pgid 65.0 last_scrub_stamp 2024-06-28T15:17:22.969363+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=17, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.196 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.196 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.196 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.196 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.196 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.196 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.196 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.197 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.197 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.197 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.197 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.197 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.197 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.197 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.198 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.198 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.198 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.198 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.198 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.198 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.199 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.199 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.199 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.199 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.199 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.199 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.200 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.200 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.200 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.200 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.200 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.200 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.200 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.201 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.201 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.201 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.201 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.201 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.201 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.202 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.202 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.202 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.202 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.202 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.202 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.203 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.203 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.203 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.203 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.203 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.203 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.204 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.204 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.204 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.204 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.204 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.204 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.204 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.205 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.205 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.205 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.205 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.205 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.205 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.205 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.206 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.206 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.206 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.206 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.206 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.206 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.207 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.207 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.207 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.207 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.207 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.207 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.208 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.208 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.208 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.208 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.208 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.208 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.208 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.209 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.209 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.209 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.209 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.209 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.209 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.209 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.210 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.210 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.210 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.210 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.210 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.210 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.211 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.211 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.211 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.211 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.211 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.211 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.211 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.212 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.212 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.212 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.212 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.212 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.212 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.213 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.213 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.213 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.213 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.213 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.213 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.213 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.214 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.214 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.214 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.214 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.214 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.214 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.215 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.215 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.215 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.215 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.215 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.215 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.215 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.216 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.216 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.216 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.216 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.216 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.216 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.216 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.217 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.217 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.217 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.217 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.217 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.217 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.217 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.218 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.218 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.218 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.218 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.218 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.218 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.219 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.219 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.219 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.219 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.219 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.219 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.219 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.220 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.220 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.220 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.220 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.220 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.220 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.221 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.221 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.221 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.221 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.221 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.221 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.222 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.222 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.222 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.222 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.222 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.222 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.222 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.223 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.223 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.223 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.223 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.223 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.224 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.224 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.224 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.224 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.224 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.224 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.224 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.225 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.225 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.225 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.225 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.225 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.225 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.226 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.226 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.226 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.226 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.226 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.226 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.226 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.227 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.227 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.227 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.227 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.227 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.227 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.227 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.228 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.228 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.228 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.228 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.228 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.228 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.229 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.229 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.229 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.229 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.229 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.229 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.229 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.230 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.230 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.230 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.230 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.230 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.230 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.231 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.231 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.231 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.231 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.231 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.231 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.231 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.232 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.232 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.232 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.232 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.232 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.232 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.232 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.233 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.233 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.233 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.233 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.233 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.233 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.233 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.234 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.234 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.234 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.234 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.234 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.234 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.235 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.235 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.235 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.235 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.235 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.235 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.235 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.236 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.236 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.236 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:14:42.236 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-06-28T16:15:02.238 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-28T16:15:02.570 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:15:02.585 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-06-28T16:15:02.592 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":2500,"stamp":"2024-06-28T16:15:01.624711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":185196140,"num_objects":785,"num_object_clones":0,"num_object_copies":2436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":785,"num_whiteouts":0,"num_read":161161,"num_read_kb":2266365,"num_write":146672,"num_write_kb":2275015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":648490,"num_omap_keys":1538,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12433,"ondisk_log_size":12433,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1973600,"kb_used_data":297312,"kb_used_omap":12,"kb_used_meta":1676211,"kb_avail":753001120,"statfs":{"total":773094113280,"available":771073146880,"internally_reserved":0,"allocated":304447488,"data_stored":288792710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":1716440627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":640826,"num_omap_keys":1521,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007834"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:35.097117+0000","last_change":"2024-06-28T15:37:33.329429+0000","last_active":"2024-06-28T16:12:35.097117+0000","last_peered":"2024-06-28T16:12:35.097117+0000","last_clean":"2024-06-28T16:12:35.097117+0000","last_became_active":"2024-06-28T15:37:33.328626+0000","last_became_peered":"2024-06-28T15:37:33.328626+0000","last_unstale":"2024-06-28T16:12:35.097117+0000","last_undegraded":"2024-06-28T16:12:35.097117+0000","last_fullsized":"2024-06-28T16:12:35.097117+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"257'3","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.112304+0000","last_change":"2024-06-28T15:21:11.079460+0000","last_active":"2024-06-28T16:13:40.112304+0000","last_peered":"2024-06-28T16:13:40.112304+0000","last_clean":"2024-06-28T16:13:40.112304+0000","last_became_active":"2024-06-28T15:21:11.078250+0000","last_became_peered":"2024-06-28T15:21:11.078250+0000","last_unstale":"2024-06-28T16:13:40.112304+0000","last_undegraded":"2024-06-28T16:13:40.112304+0000","last_fullsized":"2024-06-28T16:13:40.112304+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"260'4","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.100994+0000","last_change":"2024-06-28T16:14:41.101042+0000","last_active":"2024-06-28T16:14:41.100994+0000","last_peered":"2024-06-28T16:14:41.100994+0000","last_clean":"2024-06-28T16:14:41.100994+0000","last_became_active":"2024-06-28T15:21:07.046517+0000","last_became_peered":"2024-06-28T15:21:07.046517+0000","last_unstale":"2024-06-28T16:14:41.100994+0000","last_undegraded":"2024-06-28T16:14:41.100994+0000","last_fullsized":"2024-06-28T16:14:41.100994+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:14:41.100864+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:14:41.100864+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.100864+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:17:23.820622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3076,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"249'7","reported_seq":91,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.148277+0000","last_change":"2024-06-28T16:14:39.148310+0000","last_active":"2024-06-28T16:14:39.148277+0000","last_peered":"2024-06-28T16:14:39.148277+0000","last_clean":"2024-06-28T16:14:39.148277+0000","last_became_active":"2024-06-28T15:17:24.028787+0000","last_became_peered":"2024-06-28T15:17:24.028787+0000","last_unstale":"2024-06-28T16:14:39.148277+0000","last_undegraded":"2024-06-28T16:14:39.148277+0000","last_fullsized":"2024-06-28T16:14:39.148277+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'7","last_scrub_stamp":"2024-06-28T16:14:39.148161+0000","last_deep_scrub":"249'7","last_deep_scrub_stamp":"2024-06-28T16:14:39.148161+0000","last_clean_scrub_stamp":"2024-06-28T16:14:39.148161+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:43:21.535181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":30,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:35.282884+0000","last_change":"2024-06-28T15:37:33.337334+0000","last_active":"2024-06-28T16:12:35.282884+0000","last_peered":"2024-06-28T16:12:35.282884+0000","last_clean":"2024-06-28T16:12:35.282884+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:12:35.282884+0000","last_undegraded":"2024-06-28T16:12:35.282884+0000","last_fullsized":"2024-06-28T16:12:35.282884+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"260'2","reported_seq":47,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.880262+0000","last_change":"2024-06-28T15:21:11.105141+0000","last_active":"2024-06-28T16:14:33.880262+0000","last_peered":"2024-06-28T16:14:33.880262+0000","last_clean":"2024-06-28T16:14:33.880262+0000","last_became_active":"2024-06-28T15:21:11.104387+0000","last_became_peered":"2024-06-28T15:21:11.104387+0000","last_unstale":"2024-06-28T16:14:33.880262+0000","last_undegraded":"2024-06-28T16:14:33.880262+0000","last_fullsized":"2024-06-28T16:14:33.880262+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"260'2","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.043377+0000","last_change":"2024-06-28T16:14:45.043442+0000","last_active":"2024-06-28T16:14:45.043377+0000","last_peered":"2024-06-28T16:14:45.043377+0000","last_clean":"2024-06-28T16:14:45.043377+0000","last_became_active":"2024-06-28T15:21:07.049141+0000","last_became_peered":"2024-06-28T15:21:07.049141+0000","last_unstale":"2024-06-28T16:14:45.043377+0000","last_undegraded":"2024-06-28T16:14:45.043377+0000","last_fullsized":"2024-06-28T16:14:45.043377+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:45.043176+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:45.043176+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.043176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:35:34.904145+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"260'18","reported_seq":119,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.228494+0000","last_change":"2024-06-28T16:14:50.228554+0000","last_active":"2024-06-28T16:14:50.228494+0000","last_peered":"2024-06-28T16:14:50.228494+0000","last_clean":"2024-06-28T16:14:50.228494+0000","last_became_active":"2024-06-28T15:17:23.993905+0000","last_became_peered":"2024-06-28T15:17:23.993905+0000","last_unstale":"2024-06-28T16:14:50.228494+0000","last_undegraded":"2024-06-28T16:14:50.228494+0000","last_fullsized":"2024-06-28T16:14:50.228494+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'18","last_scrub_stamp":"2024-06-28T16:14:50.228287+0000","last_deep_scrub":"260'18","last_deep_scrub_stamp":"2024-06-28T16:14:50.228287+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.228287+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:42:17.443870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":64,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:50.120015+0000","last_change":"2024-06-28T15:37:33.353498+0000","last_active":"2024-06-28T16:13:50.120015+0000","last_peered":"2024-06-28T16:13:50.120015+0000","last_clean":"2024-06-28T16:13:50.120015+0000","last_became_active":"2024-06-28T15:37:33.345642+0000","last_became_peered":"2024-06-28T15:37:33.345642+0000","last_unstale":"2024-06-28T16:13:50.120015+0000","last_undegraded":"2024-06-28T16:13:50.120015+0000","last_fullsized":"2024-06-28T16:13:50.120015+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"257'5","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.591118+0000","last_change":"2024-06-28T15:21:11.098339+0000","last_active":"2024-06-28T16:13:20.591118+0000","last_peered":"2024-06-28T16:13:20.591118+0000","last_clean":"2024-06-28T16:13:20.591118+0000","last_became_active":"2024-06-28T15:21:11.097619+0000","last_became_peered":"2024-06-28T15:21:11.097619+0000","last_unstale":"2024-06-28T16:13:20.591118+0000","last_undegraded":"2024-06-28T16:13:20.591118+0000","last_fullsized":"2024-06-28T16:13:20.591118+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5438,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"0'0","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:25.293990+0000","last_change":"2024-06-28T15:21:07.020860+0000","last_active":"2024-06-28T16:13:25.293990+0000","last_peered":"2024-06-28T16:13:25.293990+0000","last_clean":"2024-06-28T16:13:25.293990+0000","last_became_active":"2024-06-28T15:21:07.020090+0000","last_became_peered":"2024-06-28T15:21:07.020090+0000","last_unstale":"2024-06-28T16:13:25.293990+0000","last_undegraded":"2024-06-28T16:13:25.293990+0000","last_fullsized":"2024-06-28T16:13:25.293990+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"260'43","reported_seq":184,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.114470+0000","last_change":"2024-06-28T16:14:45.114470+0000","last_active":"2024-06-28T16:14:45.114470+0000","last_peered":"2024-06-28T16:14:45.114470+0000","last_clean":"2024-06-28T16:14:45.114470+0000","last_became_active":"2024-06-28T15:17:24.012739+0000","last_became_peered":"2024-06-28T15:17:24.012739+0000","last_unstale":"2024-06-28T16:14:45.114470+0000","last_undegraded":"2024-06-28T16:14:45.114470+0000","last_fullsized":"2024-06-28T16:14:45.114470+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'43","last_scrub_stamp":"2024-06-28T16:14:45.114196+0000","last_deep_scrub":"260'43","last_deep_scrub_stamp":"2024-06-28T16:14:45.114196+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.114196+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:38:38.257912+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":221,"num_read_kb":154,"num_write":91,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.328538+0000","last_change":"2024-06-28T15:37:33.345746+0000","last_active":"2024-06-28T16:14:00.328538+0000","last_peered":"2024-06-28T16:14:00.328538+0000","last_clean":"2024-06-28T16:14:00.328538+0000","last_became_active":"2024-06-28T15:37:33.344850+0000","last_became_peered":"2024-06-28T15:37:33.344850+0000","last_unstale":"2024-06-28T16:14:00.328538+0000","last_undegraded":"2024-06-28T16:14:00.328538+0000","last_fullsized":"2024-06-28T16:14:00.328538+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"260'3","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.475704+0000","last_change":"2024-06-28T15:21:11.107158+0000","last_active":"2024-06-28T16:14:34.475704+0000","last_peered":"2024-06-28T16:14:34.475704+0000","last_clean":"2024-06-28T16:14:34.475704+0000","last_became_active":"2024-06-28T15:21:11.102572+0000","last_became_peered":"2024-06-28T15:21:11.102572+0000","last_unstale":"2024-06-28T16:14:34.475704+0000","last_undegraded":"2024-06-28T16:14:34.475704+0000","last_fullsized":"2024-06-28T16:14:34.475704+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":62,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.928105+0000","last_change":"2024-06-28T16:14:48.928159+0000","last_active":"2024-06-28T16:14:48.928105+0000","last_peered":"2024-06-28T16:14:48.928105+0000","last_clean":"2024-06-28T16:14:48.928105+0000","last_became_active":"2024-06-28T15:21:07.030576+0000","last_became_peered":"2024-06-28T15:21:07.030576+0000","last_unstale":"2024-06-28T16:14:48.928105+0000","last_undegraded":"2024-06-28T16:14:48.928105+0000","last_fullsized":"2024-06-28T16:14:48.928105+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:48.927902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:48.927902+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.927902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:15:53.577907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'23","reported_seq":126,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.110220+0000","last_change":"2024-06-28T16:14:44.110220+0000","last_active":"2024-06-28T16:14:44.110220+0000","last_peered":"2024-06-28T16:14:44.110220+0000","last_clean":"2024-06-28T16:14:44.110220+0000","last_became_active":"2024-06-28T15:17:24.019120+0000","last_became_peered":"2024-06-28T15:17:24.019120+0000","last_unstale":"2024-06-28T16:14:44.110220+0000","last_undegraded":"2024-06-28T16:14:44.110220+0000","last_fullsized":"2024-06-28T16:14:44.110220+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'23","last_scrub_stamp":"2024-06-28T16:14:44.109887+0000","last_deep_scrub":"260'23","last_deep_scrub_stamp":"2024-06-28T16:14:44.109887+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.109887+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:34:23.631650+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":82,"num_write":51,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":31,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.326025+0000","last_change":"2024-06-28T15:37:33.339459+0000","last_active":"2024-06-28T16:13:45.326025+0000","last_peered":"2024-06-28T16:13:45.326025+0000","last_clean":"2024-06-28T16:13:45.326025+0000","last_became_active":"2024-06-28T15:37:33.335045+0000","last_became_peered":"2024-06-28T15:37:33.335045+0000","last_unstale":"2024-06-28T16:13:45.326025+0000","last_undegraded":"2024-06-28T16:13:45.326025+0000","last_fullsized":"2024-06-28T16:13:45.326025+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.123964+0000","last_change":"2024-06-28T15:21:11.094697+0000","last_active":"2024-06-28T16:14:35.123964+0000","last_peered":"2024-06-28T16:14:35.123964+0000","last_clean":"2024-06-28T16:14:35.123964+0000","last_became_active":"2024-06-28T15:21:11.094122+0000","last_became_peered":"2024-06-28T15:21:11.094122+0000","last_unstale":"2024-06-28T16:14:35.123964+0000","last_undegraded":"2024-06-28T16:14:35.123964+0000","last_fullsized":"2024-06-28T16:14:35.123964+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.137409+0000","last_change":"2024-06-28T16:14:42.137431+0000","last_active":"2024-06-28T16:14:42.137409+0000","last_peered":"2024-06-28T16:14:42.137409+0000","last_clean":"2024-06-28T16:14:42.137409+0000","last_became_active":"2024-06-28T15:21:07.025549+0000","last_became_peered":"2024-06-28T15:21:07.025549+0000","last_unstale":"2024-06-28T16:14:42.137409+0000","last_undegraded":"2024-06-28T16:14:42.137409+0000","last_fullsized":"2024-06-28T16:14:42.137409+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:42.137318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:42.137318+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.137318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:05:06.478598+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"260'24","reported_seq":130,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.151839+0000","last_change":"2024-06-28T16:14:46.151839+0000","last_active":"2024-06-28T16:14:46.151839+0000","last_peered":"2024-06-28T16:14:46.151839+0000","last_clean":"2024-06-28T16:14:46.151839+0000","last_became_active":"2024-06-28T15:17:24.010441+0000","last_became_peered":"2024-06-28T15:17:24.010441+0000","last_unstale":"2024-06-28T16:14:46.151839+0000","last_undegraded":"2024-06-28T16:14:46.151839+0000","last_fullsized":"2024-06-28T16:14:46.151839+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'24","last_scrub_stamp":"2024-06-28T16:14:46.151597+0000","last_deep_scrub":"260'24","last_deep_scrub_stamp":"2024-06-28T16:14:46.151597+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.151597+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:33:56.078813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":85,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.118300+0000","last_change":"2024-06-28T15:37:33.341710+0000","last_active":"2024-06-28T16:13:00.118300+0000","last_peered":"2024-06-28T16:13:00.118300+0000","last_clean":"2024-06-28T16:13:00.118300+0000","last_became_active":"2024-06-28T15:37:33.336623+0000","last_became_peered":"2024-06-28T15:37:33.336623+0000","last_unstale":"2024-06-28T16:13:00.118300+0000","last_undegraded":"2024-06-28T16:13:00.118300+0000","last_fullsized":"2024-06-28T16:13:00.118300+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"260'11","reported_seq":65,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.517701+0000","last_change":"2024-06-28T15:21:11.100151+0000","last_active":"2024-06-28T16:14:32.517701+0000","last_peered":"2024-06-28T16:14:32.517701+0000","last_clean":"2024-06-28T16:14:32.517701+0000","last_became_active":"2024-06-28T15:21:11.099434+0000","last_became_peered":"2024-06-28T15:21:11.099434+0000","last_unstale":"2024-06-28T16:14:32.517701+0000","last_undegraded":"2024-06-28T16:14:32.517701+0000","last_fullsized":"2024-06-28T16:14:32.517701+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":20482,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"257'5","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.130458+0000","last_change":"2024-06-28T15:21:07.044767+0000","last_active":"2024-06-28T16:14:35.130458+0000","last_peered":"2024-06-28T16:14:35.130458+0000","last_clean":"2024-06-28T16:14:35.130458+0000","last_became_active":"2024-06-28T15:21:07.043987+0000","last_became_peered":"2024-06-28T15:21:07.043987+0000","last_unstale":"2024-06-28T16:14:35.130458+0000","last_undegraded":"2024-06-28T16:14:35.130458+0000","last_fullsized":"2024-06-28T16:14:35.130458+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"260'19","reported_seq":110,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.131322+0000","last_change":"2024-06-28T16:14:47.131322+0000","last_active":"2024-06-28T16:14:47.131322+0000","last_peered":"2024-06-28T16:14:47.131322+0000","last_clean":"2024-06-28T16:14:47.131322+0000","last_became_active":"2024-06-28T15:17:24.006314+0000","last_became_peered":"2024-06-28T15:17:24.006314+0000","last_unstale":"2024-06-28T16:14:47.131322+0000","last_undegraded":"2024-06-28T16:14:47.131322+0000","last_fullsized":"2024-06-28T16:14:47.131322+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'19","last_scrub_stamp":"2024-06-28T16:14:47.131067+0000","last_deep_scrub":"260'19","last_deep_scrub_stamp":"2024-06-28T16:14:47.131067+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.131067+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:32:41.228085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":54,"num_write":49,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"260'1","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:35.111677+0000","last_change":"2024-06-28T15:37:33.348503+0000","last_active":"2024-06-28T16:12:35.111677+0000","last_peered":"2024-06-28T16:12:35.111677+0000","last_clean":"2024-06-28T16:12:35.111677+0000","last_became_active":"2024-06-28T15:37:33.347715+0000","last_became_peered":"2024-06-28T15:37:33.347715+0000","last_unstale":"2024-06-28T16:12:35.111677+0000","last_undegraded":"2024-06-28T16:12:35.111677+0000","last_fullsized":"2024-06-28T16:12:35.111677+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":43,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.119036+0000","last_change":"2024-06-28T15:21:11.089867+0000","last_active":"2024-06-28T16:13:45.119036+0000","last_peered":"2024-06-28T16:13:45.119036+0000","last_clean":"2024-06-28T16:13:45.119036+0000","last_became_active":"2024-06-28T15:21:11.088625+0000","last_became_peered":"2024-06-28T15:21:11.088625+0000","last_unstale":"2024-06-28T16:13:45.119036+0000","last_undegraded":"2024-06-28T16:13:45.119036+0000","last_fullsized":"2024-06-28T16:13:45.119036+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"260'6","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.976747+0000","last_change":"2024-06-28T16:14:44.976815+0000","last_active":"2024-06-28T16:14:44.976747+0000","last_peered":"2024-06-28T16:14:44.976747+0000","last_clean":"2024-06-28T16:14:44.976747+0000","last_became_active":"2024-06-28T15:21:07.030888+0000","last_became_peered":"2024-06-28T15:21:07.030888+0000","last_unstale":"2024-06-28T16:14:44.976747+0000","last_undegraded":"2024-06-28T16:14:44.976747+0000","last_fullsized":"2024-06-28T16:14:44.976747+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:14:44.976529+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:14:44.976529+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.976529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:33:30.061725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":6,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"260'72","reported_seq":256,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.132912+0000","last_change":"2024-06-28T16:14:47.133001+0000","last_active":"2024-06-28T16:14:47.132912+0000","last_peered":"2024-06-28T16:14:47.132912+0000","last_clean":"2024-06-28T16:14:47.132912+0000","last_became_active":"2024-06-28T15:17:23.993026+0000","last_became_peered":"2024-06-28T15:17:23.993026+0000","last_unstale":"2024-06-28T16:14:47.132912+0000","last_undegraded":"2024-06-28T16:14:47.132912+0000","last_fullsized":"2024-06-28T16:14:47.132912+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'72","last_scrub_stamp":"2024-06-28T16:14:47.132725+0000","last_deep_scrub":"260'72","last_deep_scrub_stamp":"2024-06-28T16:14:47.132725+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.132725+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:03.563113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":255,"num_write":131,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":27,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:35.111741+0000","last_change":"2024-06-28T15:37:33.351154+0000","last_active":"2024-06-28T16:12:35.111741+0000","last_peered":"2024-06-28T16:12:35.111741+0000","last_clean":"2024-06-28T16:12:35.111741+0000","last_became_active":"2024-06-28T15:37:33.350447+0000","last_became_peered":"2024-06-28T15:37:33.350447+0000","last_unstale":"2024-06-28T16:12:35.111741+0000","last_undegraded":"2024-06-28T16:12:35.111741+0000","last_fullsized":"2024-06-28T16:12:35.111741+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":44,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.324364+0000","last_change":"2024-06-28T15:21:11.101407+0000","last_active":"2024-06-28T16:13:40.324364+0000","last_peered":"2024-06-28T16:13:40.324364+0000","last_clean":"2024-06-28T16:13:40.324364+0000","last_became_active":"2024-06-28T15:21:11.100603+0000","last_became_peered":"2024-06-28T15:21:11.100603+0000","last_unstale":"2024-06-28T16:13:40.324364+0000","last_undegraded":"2024-06-28T16:13:40.324364+0000","last_fullsized":"2024-06-28T16:13:40.324364+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"260'5","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.183128+0000","last_change":"2024-06-28T15:21:07.047854+0000","last_active":"2024-06-28T16:14:34.183128+0000","last_peered":"2024-06-28T16:14:34.183128+0000","last_clean":"2024-06-28T16:14:34.183128+0000","last_became_active":"2024-06-28T15:21:07.045902+0000","last_became_peered":"2024-06-28T15:21:07.045902+0000","last_unstale":"2024-06-28T16:14:34.183128+0000","last_undegraded":"2024-06-28T16:14:34.183128+0000","last_fullsized":"2024-06-28T16:14:34.183128+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":40,"num_write":10,"num_write_kb":1064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"260'48","reported_seq":201,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.112932+0000","last_change":"2024-06-28T16:14:43.112995+0000","last_active":"2024-06-28T16:14:43.112932+0000","last_peered":"2024-06-28T16:14:43.112932+0000","last_clean":"2024-06-28T16:14:43.112932+0000","last_became_active":"2024-06-28T15:17:24.016249+0000","last_became_peered":"2024-06-28T15:17:24.016249+0000","last_unstale":"2024-06-28T16:14:43.112932+0000","last_undegraded":"2024-06-28T16:14:43.112932+0000","last_fullsized":"2024-06-28T16:14:43.112932+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'48","last_scrub_stamp":"2024-06-28T16:14:43.112724+0000","last_deep_scrub":"260'48","last_deep_scrub_stamp":"2024-06-28T16:14:43.112724+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.112724+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:09:09.626825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":260,"num_read_kb":188,"num_write":81,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.074524+0000","last_change":"2024-06-28T15:37:33.317386+0000","last_active":"2024-06-28T16:14:39.074524+0000","last_peered":"2024-06-28T16:14:39.074524+0000","last_clean":"2024-06-28T16:14:39.074524+0000","last_became_active":"2024-06-28T15:37:33.316616+0000","last_became_peered":"2024-06-28T15:37:33.316616+0000","last_unstale":"2024-06-28T16:14:39.074524+0000","last_undegraded":"2024-06-28T16:14:39.074524+0000","last_fullsized":"2024-06-28T16:14:39.074524+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"260'8","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.133370+0000","last_change":"2024-06-28T16:14:57.133432+0000","last_active":"2024-06-28T16:14:57.133370+0000","last_peered":"2024-06-28T16:14:57.133370+0000","last_clean":"2024-06-28T16:14:57.133370+0000","last_became_active":"2024-06-28T15:21:11.094947+0000","last_became_peered":"2024-06-28T15:21:11.094947+0000","last_unstale":"2024-06-28T16:14:57.133370+0000","last_undegraded":"2024-06-28T16:14:57.133370+0000","last_fullsized":"2024-06-28T16:14:57.133370+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'8","last_scrub_stamp":"2024-06-28T16:14:57.133168+0000","last_deep_scrub":"260'8","last_deep_scrub_stamp":"2024-06-28T16:14:57.133168+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.133168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:12:28.239090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":12406,"num_write":8,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"260'5","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.646763+0000","last_change":"2024-06-28T16:14:52.646825+0000","last_active":"2024-06-28T16:14:52.646763+0000","last_peered":"2024-06-28T16:14:52.646763+0000","last_clean":"2024-06-28T16:14:52.646763+0000","last_became_active":"2024-06-28T15:21:07.025396+0000","last_became_peered":"2024-06-28T15:21:07.025396+0000","last_unstale":"2024-06-28T16:14:52.646763+0000","last_undegraded":"2024-06-28T16:14:52.646763+0000","last_fullsized":"2024-06-28T16:14:52.646763+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'5","last_scrub_stamp":"2024-06-28T16:14:52.646576+0000","last_deep_scrub":"260'5","last_deep_scrub_stamp":"2024-06-28T16:14:52.646576+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.646576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:58:00.023881+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":5,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"257'14","reported_seq":105,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.108821+0000","last_change":"2024-06-28T16:14:44.108939+0000","last_active":"2024-06-28T16:14:44.108821+0000","last_peered":"2024-06-28T16:14:44.108821+0000","last_clean":"2024-06-28T16:14:44.108821+0000","last_became_active":"2024-06-28T15:17:24.010190+0000","last_became_peered":"2024-06-28T15:17:24.010190+0000","last_unstale":"2024-06-28T16:14:44.108821+0000","last_undegraded":"2024-06-28T16:14:44.108821+0000","last_fullsized":"2024-06-28T16:14:44.108821+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'14","last_scrub_stamp":"2024-06-28T16:14:44.108596+0000","last_deep_scrub":"257'14","last_deep_scrub_stamp":"2024-06-28T16:14:44.108596+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.108596+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:06:35.967557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":51,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"260'1","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.521346+0000","last_change":"2024-06-28T15:37:33.338222+0000","last_active":"2024-06-28T16:14:39.521346+0000","last_peered":"2024-06-28T16:14:39.521346+0000","last_clean":"2024-06-28T16:14:39.521346+0000","last_became_active":"2024-06-28T15:37:33.334097+0000","last_became_peered":"2024-06-28T15:37:33.334097+0000","last_unstale":"2024-06-28T16:14:39.521346+0000","last_undegraded":"2024-06-28T16:14:39.521346+0000","last_fullsized":"2024-06-28T16:14:39.521346+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"260'9","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.972571+0000","last_change":"2024-06-28T15:21:11.107909+0000","last_active":"2024-06-28T16:14:39.972571+0000","last_peered":"2024-06-28T16:14:39.972571+0000","last_clean":"2024-06-28T16:14:39.972571+0000","last_became_active":"2024-06-28T15:21:11.107155+0000","last_became_peered":"2024-06-28T15:21:11.107155+0000","last_unstale":"2024-06-28T16:14:39.972571+0000","last_undegraded":"2024-06-28T16:14:39.972571+0000","last_fullsized":"2024-06-28T16:14:39.972571+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21504,"num_write":14,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.557445+0000","last_change":"2024-06-28T16:14:42.557501+0000","last_active":"2024-06-28T16:14:42.557445+0000","last_peered":"2024-06-28T16:14:42.557445+0000","last_clean":"2024-06-28T16:14:42.557445+0000","last_became_active":"2024-06-28T15:21:07.042691+0000","last_became_peered":"2024-06-28T15:21:07.042691+0000","last_unstale":"2024-06-28T16:14:42.557445+0000","last_undegraded":"2024-06-28T16:14:42.557445+0000","last_fullsized":"2024-06-28T16:14:42.557445+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:42.557250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:42.557250+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.557250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:17:11.170202+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"260'74","reported_seq":299,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.069608+0000","last_change":"2024-06-28T16:14:40.069653+0000","last_active":"2024-06-28T16:14:40.069608+0000","last_peered":"2024-06-28T16:14:40.069608+0000","last_clean":"2024-06-28T16:14:40.069608+0000","last_became_active":"2024-06-28T15:17:24.027786+0000","last_became_peered":"2024-06-28T15:17:24.027786+0000","last_unstale":"2024-06-28T16:14:40.069608+0000","last_undegraded":"2024-06-28T16:14:40.069608+0000","last_fullsized":"2024-06-28T16:14:40.069608+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'74","last_scrub_stamp":"2024-06-28T16:14:40.069506+0000","last_deep_scrub":"260'74","last_deep_scrub_stamp":"2024-06-28T16:14:40.069506+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.069506+0000","objects_scrubbed":1,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:43:30.466356+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":461,"num_read_kb":332,"num_write":85,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2470,"num_omap_keys":5,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.804650+0000","last_change":"2024-06-28T15:37:33.355373+0000","last_active":"2024-06-28T16:14:41.804650+0000","last_peered":"2024-06-28T16:14:41.804650+0000","last_clean":"2024-06-28T16:14:41.804650+0000","last_became_active":"2024-06-28T15:37:33.354730+0000","last_became_peered":"2024-06-28T15:37:33.354730+0000","last_unstale":"2024-06-28T16:14:41.804650+0000","last_undegraded":"2024-06-28T16:14:41.804650+0000","last_fullsized":"2024-06-28T16:14:41.804650+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"260'4","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.183946+0000","last_change":"2024-06-28T16:14:53.184018+0000","last_active":"2024-06-28T16:14:53.183946+0000","last_peered":"2024-06-28T16:14:53.183946+0000","last_clean":"2024-06-28T16:14:53.183946+0000","last_became_active":"2024-06-28T15:21:11.093751+0000","last_became_peered":"2024-06-28T15:21:11.093751+0000","last_unstale":"2024-06-28T16:14:53.183946+0000","last_undegraded":"2024-06-28T16:14:53.183946+0000","last_fullsized":"2024-06-28T16:14:53.183946+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:14:53.183733+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:14:53.183733+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.183733+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:12:53.454345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"260'2","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.306442+0000","last_change":"2024-06-28T15:21:07.043427+0000","last_active":"2024-06-28T16:14:32.306442+0000","last_peered":"2024-06-28T16:14:32.306442+0000","last_clean":"2024-06-28T16:14:32.306442+0000","last_became_active":"2024-06-28T15:21:07.042846+0000","last_became_peered":"2024-06-28T15:21:07.042846+0000","last_unstale":"2024-06-28T16:14:32.306442+0000","last_undegraded":"2024-06-28T16:14:32.306442+0000","last_fullsized":"2024-06-28T16:14:32.306442+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":59,"num_write":2,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"260'31","reported_seq":153,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.199746+0000","last_change":"2024-06-28T16:14:54.199811+0000","last_active":"2024-06-28T16:14:54.199746+0000","last_peered":"2024-06-28T16:14:54.199746+0000","last_clean":"2024-06-28T16:14:54.199746+0000","last_became_active":"2024-06-28T15:17:24.016275+0000","last_became_peered":"2024-06-28T15:17:24.016275+0000","last_unstale":"2024-06-28T16:14:54.199746+0000","last_undegraded":"2024-06-28T16:14:54.199746+0000","last_fullsized":"2024-06-28T16:14:54.199746+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'31","last_scrub_stamp":"2024-06-28T16:14:54.199529+0000","last_deep_scrub":"260'31","last_deep_scrub_stamp":"2024-06-28T16:14:54.199529+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.199529+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:16:15.219769+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":116,"num_write":60,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.878501+0000","last_change":"2024-06-28T15:37:33.323939+0000","last_active":"2024-06-28T16:14:40.878501+0000","last_peered":"2024-06-28T16:14:40.878501+0000","last_clean":"2024-06-28T16:14:40.878501+0000","last_became_active":"2024-06-28T15:37:33.323174+0000","last_became_peered":"2024-06-28T15:37:33.323174+0000","last_unstale":"2024-06-28T16:14:40.878501+0000","last_undegraded":"2024-06-28T16:14:40.878501+0000","last_fullsized":"2024-06-28T16:14:40.878501+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"260'9","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.511491+0000","last_change":"2024-06-28T15:21:11.088257+0000","last_active":"2024-06-28T16:14:33.511491+0000","last_peered":"2024-06-28T16:14:33.511491+0000","last_clean":"2024-06-28T16:14:33.511491+0000","last_became_active":"2024-06-28T15:21:11.087119+0000","last_became_peered":"2024-06-28T15:21:11.087119+0000","last_unstale":"2024-06-28T16:14:33.511491+0000","last_undegraded":"2024-06-28T16:14:33.511491+0000","last_fullsized":"2024-06-28T16:14:33.511491+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":24576,"num_write":19,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.108123+0000","last_change":"2024-06-28T16:14:50.108177+0000","last_active":"2024-06-28T16:14:50.108123+0000","last_peered":"2024-06-28T16:14:50.108123+0000","last_clean":"2024-06-28T16:14:50.108123+0000","last_became_active":"2024-06-28T15:21:07.032092+0000","last_became_peered":"2024-06-28T15:21:07.032092+0000","last_unstale":"2024-06-28T16:14:50.108123+0000","last_undegraded":"2024-06-28T16:14:50.108123+0000","last_fullsized":"2024-06-28T16:14:50.108123+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:50.107930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:50.107930+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.107930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:01:58.373114+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"260'21","reported_seq":129,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.642626+0000","last_change":"2024-06-28T16:14:54.642755+0000","last_active":"2024-06-28T16:14:54.642626+0000","last_peered":"2024-06-28T16:14:54.642626+0000","last_clean":"2024-06-28T16:14:54.642626+0000","last_became_active":"2024-06-28T15:17:24.020899+0000","last_became_peered":"2024-06-28T15:17:24.020899+0000","last_unstale":"2024-06-28T16:14:54.642626+0000","last_undegraded":"2024-06-28T16:14:54.642626+0000","last_fullsized":"2024-06-28T16:14:54.642626+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'21","last_scrub_stamp":"2024-06-28T16:14:54.642387+0000","last_deep_scrub":"260'21","last_deep_scrub_stamp":"2024-06-28T16:14:54.642387+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.642387+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:49:49.845916+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":86,"num_write":39,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:50.326876+0000","last_change":"2024-06-28T15:37:33.339666+0000","last_active":"2024-06-28T16:13:50.326876+0000","last_peered":"2024-06-28T16:13:50.326876+0000","last_clean":"2024-06-28T16:13:50.326876+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:13:50.326876+0000","last_undegraded":"2024-06-28T16:13:50.326876+0000","last_fullsized":"2024-06-28T16:13:50.326876+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"257'3","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.074682+0000","last_change":"2024-06-28T15:21:11.098494+0000","last_active":"2024-06-28T16:14:39.074682+0000","last_peered":"2024-06-28T16:14:39.074682+0000","last_clean":"2024-06-28T16:14:39.074682+0000","last_became_active":"2024-06-28T15:21:11.097622+0000","last_became_peered":"2024-06-28T15:21:11.097622+0000","last_unstale":"2024-06-28T16:14:39.074682+0000","last_undegraded":"2024-06-28T16:14:39.074682+0000","last_fullsized":"2024-06-28T16:14:39.074682+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"0'0","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.906019+0000","last_change":"2024-06-28T16:14:56.906073+0000","last_active":"2024-06-28T16:14:56.906019+0000","last_peered":"2024-06-28T16:14:56.906019+0000","last_clean":"2024-06-28T16:14:56.906019+0000","last_became_active":"2024-06-28T15:21:07.038746+0000","last_became_peered":"2024-06-28T15:21:07.038746+0000","last_unstale":"2024-06-28T16:14:56.906019+0000","last_undegraded":"2024-06-28T16:14:56.906019+0000","last_fullsized":"2024-06-28T16:14:56.906019+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.905794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.905794+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.905794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:56:22.215818+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"257'9","reported_seq":90,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.630121+0000","last_change":"2024-06-28T16:14:53.630260+0000","last_active":"2024-06-28T16:14:53.630121+0000","last_peered":"2024-06-28T16:14:53.630121+0000","last_clean":"2024-06-28T16:14:53.630121+0000","last_became_active":"2024-06-28T15:17:24.020276+0000","last_became_peered":"2024-06-28T15:17:24.020276+0000","last_unstale":"2024-06-28T16:14:53.630121+0000","last_undegraded":"2024-06-28T16:14:53.630121+0000","last_fullsized":"2024-06-28T16:14:53.630121+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'9","last_scrub_stamp":"2024-06-28T16:14:53.629897+0000","last_deep_scrub":"257'9","last_deep_scrub_stamp":"2024-06-28T16:14:53.629897+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.629897+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:40:42.297017+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:50.597772+0000","last_change":"2024-06-28T15:37:33.328340+0000","last_active":"2024-06-28T16:13:50.597772+0000","last_peered":"2024-06-28T16:13:50.597772+0000","last_clean":"2024-06-28T16:13:50.597772+0000","last_became_active":"2024-06-28T15:37:33.327497+0000","last_became_peered":"2024-06-28T15:37:33.327497+0000","last_unstale":"2024-06-28T16:13:50.597772+0000","last_undegraded":"2024-06-28T16:13:50.597772+0000","last_fullsized":"2024-06-28T16:13:50.597772+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"260'4","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.074535+0000","last_change":"2024-06-28T15:21:11.092761+0000","last_active":"2024-06-28T16:14:39.074535+0000","last_peered":"2024-06-28T16:14:39.074535+0000","last_clean":"2024-06-28T16:14:39.074535+0000","last_became_active":"2024-06-28T15:21:11.092033+0000","last_became_peered":"2024-06-28T15:21:11.092033+0000","last_unstale":"2024-06-28T16:14:39.074535+0000","last_undegraded":"2024-06-28T16:14:39.074535+0000","last_fullsized":"2024-06-28T16:14:39.074535+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6152,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.141194+0000","last_change":"2024-06-28T16:14:45.141194+0000","last_active":"2024-06-28T16:14:45.141194+0000","last_peered":"2024-06-28T16:14:45.141194+0000","last_clean":"2024-06-28T16:14:45.141194+0000","last_became_active":"2024-06-28T15:21:07.045842+0000","last_became_peered":"2024-06-28T15:21:07.045842+0000","last_unstale":"2024-06-28T16:14:45.141194+0000","last_undegraded":"2024-06-28T16:14:45.141194+0000","last_fullsized":"2024-06-28T16:14:45.141194+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:45.140747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:45.140747+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.140747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:01:27.177077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"260'17","reported_seq":115,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.153794+0000","last_change":"2024-06-28T16:14:43.153919+0000","last_active":"2024-06-28T16:14:43.153794+0000","last_peered":"2024-06-28T16:14:43.153794+0000","last_clean":"2024-06-28T16:14:43.153794+0000","last_became_active":"2024-06-28T15:17:24.013118+0000","last_became_peered":"2024-06-28T15:17:24.013118+0000","last_unstale":"2024-06-28T16:14:43.153794+0000","last_undegraded":"2024-06-28T16:14:43.153794+0000","last_fullsized":"2024-06-28T16:14:43.153794+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'17","last_scrub_stamp":"2024-06-28T16:14:43.153520+0000","last_deep_scrub":"260'17","last_deep_scrub_stamp":"2024-06-28T16:14:43.153520+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.153520+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:39:58.889835+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":63,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"259'1","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.610313+0000","last_change":"2024-06-28T15:37:33.342777+0000","last_active":"2024-06-28T16:14:38.610313+0000","last_peered":"2024-06-28T16:14:38.610313+0000","last_clean":"2024-06-28T16:14:38.610313+0000","last_became_active":"2024-06-28T15:37:33.339097+0000","last_became_peered":"2024-06-28T15:37:33.339097+0000","last_unstale":"2024-06-28T16:14:38.610313+0000","last_undegraded":"2024-06-28T16:14:38.610313+0000","last_fullsized":"2024-06-28T16:14:38.610313+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"257'3","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.972534+0000","last_change":"2024-06-28T15:21:11.101867+0000","last_active":"2024-06-28T16:14:39.972534+0000","last_peered":"2024-06-28T16:14:39.972534+0000","last_clean":"2024-06-28T16:14:39.972534+0000","last_became_active":"2024-06-28T15:21:11.101063+0000","last_became_peered":"2024-06-28T15:21:11.101063+0000","last_unstale":"2024-06-28T16:14:39.972534+0000","last_undegraded":"2024-06-28T16:14:39.972534+0000","last_fullsized":"2024-06-28T16:14:39.972534+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"257'3","reported_seq":57,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.332716+0000","last_change":"2024-06-28T15:21:07.042183+0000","last_active":"2024-06-28T16:14:41.332716+0000","last_peered":"2024-06-28T16:14:41.332716+0000","last_clean":"2024-06-28T16:14:41.332716+0000","last_became_active":"2024-06-28T15:21:07.040973+0000","last_became_peered":"2024-06-28T15:21:07.040973+0000","last_unstale":"2024-06-28T16:14:41.332716+0000","last_undegraded":"2024-06-28T16:14:41.332716+0000","last_fullsized":"2024-06-28T16:14:41.332716+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"260'22","reported_seq":124,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.889374+0000","last_change":"2024-06-28T16:14:40.889438+0000","last_active":"2024-06-28T16:14:40.889374+0000","last_peered":"2024-06-28T16:14:40.889374+0000","last_clean":"2024-06-28T16:14:40.889374+0000","last_became_active":"2024-06-28T15:17:24.028410+0000","last_became_peered":"2024-06-28T15:17:24.028410+0000","last_unstale":"2024-06-28T16:14:40.889374+0000","last_undegraded":"2024-06-28T16:14:40.889374+0000","last_fullsized":"2024-06-28T16:14:40.889374+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'22","last_scrub_stamp":"2024-06-28T16:14:40.889173+0000","last_deep_scrub":"260'22","last_deep_scrub_stamp":"2024-06-28T16:14:40.889173+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.889173+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:21.505212+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"260'1","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:50.326834+0000","last_change":"2024-06-28T15:37:33.341173+0000","last_active":"2024-06-28T16:13:50.326834+0000","last_peered":"2024-06-28T16:13:50.326834+0000","last_clean":"2024-06-28T16:13:50.326834+0000","last_became_active":"2024-06-28T15:37:33.340370+0000","last_became_peered":"2024-06-28T15:37:33.340370+0000","last_unstale":"2024-06-28T16:13:50.326834+0000","last_undegraded":"2024-06-28T16:13:50.326834+0000","last_fullsized":"2024-06-28T16:13:50.326834+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"257'3","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.332935+0000","last_change":"2024-06-28T15:21:11.104826+0000","last_active":"2024-06-28T16:14:41.332935+0000","last_peered":"2024-06-28T16:14:41.332935+0000","last_clean":"2024-06-28T16:14:41.332935+0000","last_became_active":"2024-06-28T15:21:11.104038+0000","last_became_peered":"2024-06-28T15:21:11.104038+0000","last_unstale":"2024-06-28T16:14:41.332935+0000","last_undegraded":"2024-06-28T16:14:41.332935+0000","last_fullsized":"2024-06-28T16:14:41.332935+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"260'9","reported_seq":82,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.657797+0000","last_change":"2024-06-28T16:14:48.657797+0000","last_active":"2024-06-28T16:14:48.657797+0000","last_peered":"2024-06-28T16:14:48.657797+0000","last_clean":"2024-06-28T16:14:48.657797+0000","last_became_active":"2024-06-28T15:21:07.008899+0000","last_became_peered":"2024-06-28T15:21:07.008899+0000","last_unstale":"2024-06-28T16:14:48.657797+0000","last_undegraded":"2024-06-28T16:14:48.657797+0000","last_fullsized":"2024-06-28T16:14:48.657797+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'9","last_scrub_stamp":"2024-06-28T16:14:48.657546+0000","last_deep_scrub":"260'9","last_deep_scrub_stamp":"2024-06-28T16:14:48.657546+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.657546+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:10:31.780674+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":7,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"260'42","reported_seq":174,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.123309+0000","last_change":"2024-06-28T16:14:43.123456+0000","last_active":"2024-06-28T16:14:43.123309+0000","last_peered":"2024-06-28T16:14:43.123309+0000","last_clean":"2024-06-28T16:14:43.123309+0000","last_became_active":"2024-06-28T15:17:24.019180+0000","last_became_peered":"2024-06-28T15:17:24.019180+0000","last_unstale":"2024-06-28T16:14:43.123309+0000","last_undegraded":"2024-06-28T16:14:43.123309+0000","last_fullsized":"2024-06-28T16:14:43.123309+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'42","last_scrub_stamp":"2024-06-28T16:14:43.123099+0000","last_deep_scrub":"260'42","last_deep_scrub_stamp":"2024-06-28T16:14:43.123099+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.123099+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:45:23.503458+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":198,"num_read_kb":139,"num_write":93,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.308181+0000","last_change":"2024-06-28T15:37:33.342459+0000","last_active":"2024-06-28T16:14:35.308181+0000","last_peered":"2024-06-28T16:14:35.308181+0000","last_clean":"2024-06-28T16:14:35.308181+0000","last_became_active":"2024-06-28T15:37:33.341716+0000","last_became_peered":"2024-06-28T15:37:33.341716+0000","last_unstale":"2024-06-28T16:14:35.308181+0000","last_undegraded":"2024-06-28T16:14:35.308181+0000","last_fullsized":"2024-06-28T16:14:35.308181+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.080188+0000","last_change":"2024-06-28T16:14:49.080244+0000","last_active":"2024-06-28T16:14:49.080188+0000","last_peered":"2024-06-28T16:14:49.080188+0000","last_clean":"2024-06-28T16:14:49.080188+0000","last_became_active":"2024-06-28T15:21:11.093276+0000","last_became_peered":"2024-06-28T15:21:11.093276+0000","last_unstale":"2024-06-28T16:14:49.080188+0000","last_undegraded":"2024-06-28T16:14:49.080188+0000","last_fullsized":"2024-06-28T16:14:49.080188+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.079996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.079996+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.079996+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:47:03.183258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"260'9","reported_seq":75,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.651861+0000","last_change":"2024-06-28T16:14:45.651923+0000","last_active":"2024-06-28T16:14:45.651861+0000","last_peered":"2024-06-28T16:14:45.651861+0000","last_clean":"2024-06-28T16:14:45.651861+0000","last_became_active":"2024-06-28T15:21:07.040691+0000","last_became_peered":"2024-06-28T15:21:07.040691+0000","last_unstale":"2024-06-28T16:14:45.651861+0000","last_undegraded":"2024-06-28T16:14:45.651861+0000","last_fullsized":"2024-06-28T16:14:45.651861+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'9","last_scrub_stamp":"2024-06-28T16:14:45.651669+0000","last_deep_scrub":"260'9","last_deep_scrub_stamp":"2024-06-28T16:14:45.651669+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.651669+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:32:08.870332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":17408,"num_write":9,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"260'47","reported_seq":193,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.096531+0000","last_change":"2024-06-28T16:14:42.096599+0000","last_active":"2024-06-28T16:14:42.096531+0000","last_peered":"2024-06-28T16:14:42.096531+0000","last_clean":"2024-06-28T16:14:42.096531+0000","last_became_active":"2024-06-28T15:17:24.027610+0000","last_became_peered":"2024-06-28T15:17:24.027610+0000","last_unstale":"2024-06-28T16:14:42.096531+0000","last_undegraded":"2024-06-28T16:14:42.096531+0000","last_fullsized":"2024-06-28T16:14:42.096531+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'47","last_scrub_stamp":"2024-06-28T16:14:42.096391+0000","last_deep_scrub":"260'47","last_deep_scrub_stamp":"2024-06-28T16:14:42.096391+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.096391+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:11:26.034311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":231,"num_read_kb":169,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:35.125426+0000","last_change":"2024-06-28T15:37:33.347375+0000","last_active":"2024-06-28T16:13:35.125426+0000","last_peered":"2024-06-28T16:13:35.125426+0000","last_clean":"2024-06-28T16:13:35.125426+0000","last_became_active":"2024-06-28T15:37:33.346414+0000","last_became_peered":"2024-06-28T15:37:33.346414+0000","last_unstale":"2024-06-28T16:13:35.125426+0000","last_undegraded":"2024-06-28T16:13:35.125426+0000","last_fullsized":"2024-06-28T16:13:35.125426+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"257'5","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.113353+0000","last_change":"2024-06-28T15:22:00.970955+0000","last_active":"2024-06-28T16:13:45.113353+0000","last_peered":"2024-06-28T16:13:45.113353+0000","last_clean":"2024-06-28T16:13:45.113353+0000","last_became_active":"2024-06-28T15:22:00.940760+0000","last_became_peered":"2024-06-28T15:22:00.940760+0000","last_unstale":"2024-06-28T16:13:45.113353+0000","last_undegraded":"2024-06-28T16:13:45.113353+0000","last_fullsized":"2024-06-28T16:13:45.113353+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"260'3","reported_seq":57,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.213929+0000","last_change":"2024-06-28T15:21:07.031060+0000","last_active":"2024-06-28T16:14:33.213929+0000","last_peered":"2024-06-28T16:14:33.213929+0000","last_clean":"2024-06-28T16:14:33.213929+0000","last_became_active":"2024-06-28T15:21:07.030163+0000","last_became_peered":"2024-06-28T15:21:07.030163+0000","last_unstale":"2024-06-28T16:14:33.213929+0000","last_undegraded":"2024-06-28T16:14:33.213929+0000","last_fullsized":"2024-06-28T16:14:33.213929+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"260'40","reported_seq":180,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.687315+0000","last_change":"2024-06-28T16:14:44.687376+0000","last_active":"2024-06-28T16:14:44.687315+0000","last_peered":"2024-06-28T16:14:44.687315+0000","last_clean":"2024-06-28T16:14:44.687315+0000","last_became_active":"2024-06-28T15:17:24.012572+0000","last_became_peered":"2024-06-28T15:17:24.012572+0000","last_unstale":"2024-06-28T16:14:44.687315+0000","last_undegraded":"2024-06-28T16:14:44.687315+0000","last_fullsized":"2024-06-28T16:14:44.687315+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'40","last_scrub_stamp":"2024-06-28T16:14:44.687125+0000","last_deep_scrub":"260'40","last_deep_scrub_stamp":"2024-06-28T16:14:44.687125+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.687125+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:52:09.797828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":148,"num_write":94,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:25.127461+0000","last_change":"2024-06-28T15:37:33.353545+0000","last_active":"2024-06-28T16:14:25.127461+0000","last_peered":"2024-06-28T16:14:25.127461+0000","last_clean":"2024-06-28T16:14:25.127461+0000","last_became_active":"2024-06-28T15:37:33.352959+0000","last_became_peered":"2024-06-28T15:37:33.352959+0000","last_unstale":"2024-06-28T16:14:25.127461+0000","last_undegraded":"2024-06-28T16:14:25.127461+0000","last_fullsized":"2024-06-28T16:14:25.127461+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.046074+0000","last_change":"2024-06-28T16:14:49.046127+0000","last_active":"2024-06-28T16:14:49.046074+0000","last_peered":"2024-06-28T16:14:49.046074+0000","last_clean":"2024-06-28T16:14:49.046074+0000","last_became_active":"2024-06-28T15:21:11.079036+0000","last_became_peered":"2024-06-28T15:21:11.079036+0000","last_unstale":"2024-06-28T16:14:49.046074+0000","last_undegraded":"2024-06-28T16:14:49.046074+0000","last_fullsized":"2024-06-28T16:14:49.046074+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.045873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.045873+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.045873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:29:37.775532+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"0'0","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.128278+0000","last_change":"2024-06-28T16:14:46.128341+0000","last_active":"2024-06-28T16:14:46.128278+0000","last_peered":"2024-06-28T16:14:46.128278+0000","last_clean":"2024-06-28T16:14:46.128278+0000","last_became_active":"2024-06-28T15:21:07.027451+0000","last_became_peered":"2024-06-28T15:21:07.027451+0000","last_unstale":"2024-06-28T16:14:46.128278+0000","last_undegraded":"2024-06-28T16:14:46.128278+0000","last_fullsized":"2024-06-28T16:14:46.128278+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:46.128071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:46.128071+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.128071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:30:28.326677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"260'16","reported_seq":106,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.081079+0000","last_change":"2024-06-28T16:14:43.081145+0000","last_active":"2024-06-28T16:14:43.081079+0000","last_peered":"2024-06-28T16:14:43.081079+0000","last_clean":"2024-06-28T16:14:43.081079+0000","last_became_active":"2024-06-28T15:17:24.026271+0000","last_became_peered":"2024-06-28T15:17:24.026271+0000","last_unstale":"2024-06-28T16:14:43.081079+0000","last_undegraded":"2024-06-28T16:14:43.081079+0000","last_fullsized":"2024-06-28T16:14:43.081079+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'16","last_scrub_stamp":"2024-06-28T16:14:43.080857+0000","last_deep_scrub":"260'16","last_deep_scrub_stamp":"2024-06-28T16:14:43.080857+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.080857+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:01:28.864357+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":48,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:25.137171+0000","last_change":"2024-06-28T15:37:33.326639+0000","last_active":"2024-06-28T16:14:25.137171+0000","last_peered":"2024-06-28T16:14:25.137171+0000","last_clean":"2024-06-28T16:14:25.137171+0000","last_became_active":"2024-06-28T15:37:33.325849+0000","last_became_peered":"2024-06-28T15:37:33.325849+0000","last_unstale":"2024-06-28T16:14:25.137171+0000","last_undegraded":"2024-06-28T16:14:25.137171+0000","last_fullsized":"2024-06-28T16:14:25.137171+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"257'5","reported_seq":57,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.313575+0000","last_change":"2024-06-28T15:21:11.108377+0000","last_active":"2024-06-28T16:12:55.313575+0000","last_peered":"2024-06-28T16:12:55.313575+0000","last_clean":"2024-06-28T16:12:55.313575+0000","last_became_active":"2024-06-28T15:21:11.107498+0000","last_became_peered":"2024-06-28T15:21:11.107498+0000","last_unstale":"2024-06-28T16:12:55.313575+0000","last_undegraded":"2024-06-28T16:12:55.313575+0000","last_fullsized":"2024-06-28T16:12:55.313575+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"260'8","reported_seq":76,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.130905+0000","last_change":"2024-06-28T16:14:41.130933+0000","last_active":"2024-06-28T16:14:41.130905+0000","last_peered":"2024-06-28T16:14:41.130905+0000","last_clean":"2024-06-28T16:14:41.130905+0000","last_became_active":"2024-06-28T15:21:07.029525+0000","last_became_peered":"2024-06-28T15:21:07.029525+0000","last_unstale":"2024-06-28T16:14:41.130905+0000","last_undegraded":"2024-06-28T16:14:41.130905+0000","last_fullsized":"2024-06-28T16:14:41.130905+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'8","last_scrub_stamp":"2024-06-28T16:14:41.130824+0000","last_deep_scrub":"260'8","last_deep_scrub_stamp":"2024-06-28T16:14:41.130824+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.130824+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:31:52.999376+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":11,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"260'34","reported_seq":160,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.199739+0000","last_change":"2024-06-28T16:14:52.199853+0000","last_active":"2024-06-28T16:14:52.199739+0000","last_peered":"2024-06-28T16:14:52.199739+0000","last_clean":"2024-06-28T16:14:52.199739+0000","last_became_active":"2024-06-28T15:17:24.007649+0000","last_became_peered":"2024-06-28T15:17:24.007649+0000","last_unstale":"2024-06-28T16:14:52.199739+0000","last_undegraded":"2024-06-28T16:14:52.199739+0000","last_fullsized":"2024-06-28T16:14:52.199739+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'34","last_scrub_stamp":"2024-06-28T16:14:52.199460+0000","last_deep_scrub":"260'34","last_deep_scrub_stamp":"2024-06-28T16:14:52.199460+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.199460+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:55:51.649887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":172,"num_read_kb":122,"num_write":76,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:35.323323+0000","last_change":"2024-06-28T15:37:33.335759+0000","last_active":"2024-06-28T16:13:35.323323+0000","last_peered":"2024-06-28T16:13:35.323323+0000","last_clean":"2024-06-28T16:13:35.323323+0000","last_became_active":"2024-06-28T15:37:33.334082+0000","last_became_peered":"2024-06-28T15:37:33.334082+0000","last_unstale":"2024-06-28T16:13:35.323323+0000","last_undegraded":"2024-06-28T16:13:35.323323+0000","last_fullsized":"2024-06-28T16:13:35.323323+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"0'0","reported_seq":47,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.113301+0000","last_change":"2024-06-28T15:21:11.108205+0000","last_active":"2024-06-28T16:13:15.113301+0000","last_peered":"2024-06-28T16:13:15.113301+0000","last_clean":"2024-06-28T16:13:15.113301+0000","last_became_active":"2024-06-28T15:21:11.107686+0000","last_became_peered":"2024-06-28T15:21:11.107686+0000","last_unstale":"2024-06-28T16:13:15.113301+0000","last_undegraded":"2024-06-28T16:13:15.113301+0000","last_fullsized":"2024-06-28T16:13:15.113301+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"0'0","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.051994+0000","last_change":"2024-06-28T16:14:54.052046+0000","last_active":"2024-06-28T16:14:54.051994+0000","last_peered":"2024-06-28T16:14:54.051994+0000","last_clean":"2024-06-28T16:14:54.051994+0000","last_became_active":"2024-06-28T15:21:07.032719+0000","last_became_peered":"2024-06-28T15:21:07.032719+0000","last_unstale":"2024-06-28T16:14:54.051994+0000","last_undegraded":"2024-06-28T16:14:54.051994+0000","last_fullsized":"2024-06-28T16:14:54.051994+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:54.051795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:54.051795+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.051795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:51:48.898614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"260'45","reported_seq":191,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.159340+0000","last_change":"2024-06-28T16:14:40.159396+0000","last_active":"2024-06-28T16:14:40.159340+0000","last_peered":"2024-06-28T16:14:40.159340+0000","last_clean":"2024-06-28T16:14:40.159340+0000","last_became_active":"2024-06-28T15:17:24.029532+0000","last_became_peered":"2024-06-28T15:17:24.029532+0000","last_unstale":"2024-06-28T16:14:40.159340+0000","last_undegraded":"2024-06-28T16:14:40.159340+0000","last_fullsized":"2024-06-28T16:14:40.159340+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'45","last_scrub_stamp":"2024-06-28T16:14:40.159221+0000","last_deep_scrub":"260'45","last_deep_scrub_stamp":"2024-06-28T16:14:40.159221+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.159221+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:08:46.885820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":234,"num_read_kb":165,"num_write":86,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.128083+0000","last_change":"2024-06-28T15:37:33.348528+0000","last_active":"2024-06-28T16:13:45.128083+0000","last_peered":"2024-06-28T16:13:45.128083+0000","last_clean":"2024-06-28T16:13:45.128083+0000","last_became_active":"2024-06-28T15:37:33.347691+0000","last_became_peered":"2024-06-28T15:37:33.347691+0000","last_unstale":"2024-06-28T16:13:45.128083+0000","last_undegraded":"2024-06-28T16:13:45.128083+0000","last_fullsized":"2024-06-28T16:13:45.128083+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"0'0","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.580792+0000","last_change":"2024-06-28T16:14:48.580849+0000","last_active":"2024-06-28T16:14:48.580792+0000","last_peered":"2024-06-28T16:14:48.580792+0000","last_clean":"2024-06-28T16:14:48.580792+0000","last_became_active":"2024-06-28T15:21:11.091656+0000","last_became_peered":"2024-06-28T15:21:11.091656+0000","last_unstale":"2024-06-28T16:14:48.580792+0000","last_undegraded":"2024-06-28T16:14:48.580792+0000","last_fullsized":"2024-06-28T16:14:48.580792+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:48.580598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:48.580598+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.580598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:07:09.489282+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"260'5","reported_seq":68,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.099809+0000","last_change":"2024-06-28T16:14:39.099851+0000","last_active":"2024-06-28T16:14:39.099809+0000","last_peered":"2024-06-28T16:14:39.099809+0000","last_clean":"2024-06-28T16:14:39.099809+0000","last_became_active":"2024-06-28T15:21:07.029495+0000","last_became_peered":"2024-06-28T15:21:07.029495+0000","last_unstale":"2024-06-28T16:14:39.099809+0000","last_undegraded":"2024-06-28T16:14:39.099809+0000","last_fullsized":"2024-06-28T16:14:39.099809+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'5","last_scrub_stamp":"2024-06-28T16:14:39.099694+0000","last_deep_scrub":"260'5","last_deep_scrub_stamp":"2024-06-28T16:14:39.099694+0000","last_clean_scrub_stamp":"2024-06-28T16:14:39.099694+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:24:20.828667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":318,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"260'39","reported_seq":171,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.649465+0000","last_change":"2024-06-28T16:14:50.649616+0000","last_active":"2024-06-28T16:14:50.649465+0000","last_peered":"2024-06-28T16:14:50.649465+0000","last_clean":"2024-06-28T16:14:50.649465+0000","last_became_active":"2024-06-28T15:17:24.021200+0000","last_became_peered":"2024-06-28T15:17:24.021200+0000","last_unstale":"2024-06-28T16:14:50.649465+0000","last_undegraded":"2024-06-28T16:14:50.649465+0000","last_fullsized":"2024-06-28T16:14:50.649465+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'39","last_scrub_stamp":"2024-06-28T16:14:50.649227+0000","last_deep_scrub":"260'39","last_deep_scrub_stamp":"2024-06-28T16:14:50.649227+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.649227+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:23:42.138261+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":191,"num_read_kb":133,"num_write":87,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"260'1","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:35.125503+0000","last_change":"2024-06-28T15:37:33.336238+0000","last_active":"2024-06-28T16:13:35.125503+0000","last_peered":"2024-06-28T16:13:35.125503+0000","last_clean":"2024-06-28T16:13:35.125503+0000","last_became_active":"2024-06-28T15:37:33.335464+0000","last_became_peered":"2024-06-28T15:37:33.335464+0000","last_unstale":"2024-06-28T16:13:35.125503+0000","last_undegraded":"2024-06-28T16:13:35.125503+0000","last_fullsized":"2024-06-28T16:13:35.125503+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"257'3","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.118183+0000","last_change":"2024-06-28T15:21:11.094534+0000","last_active":"2024-06-28T16:13:00.118183+0000","last_peered":"2024-06-28T16:13:00.118183+0000","last_clean":"2024-06-28T16:13:00.118183+0000","last_became_active":"2024-06-28T15:21:11.091647+0000","last_became_peered":"2024-06-28T15:21:11.091647+0000","last_unstale":"2024-06-28T16:13:00.118183+0000","last_undegraded":"2024-06-28T16:13:00.118183+0000","last_fullsized":"2024-06-28T16:13:00.118183+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"257'3","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.098521+0000","last_change":"2024-06-28T16:14:58.098585+0000","last_active":"2024-06-28T16:14:58.098521+0000","last_peered":"2024-06-28T16:14:58.098521+0000","last_clean":"2024-06-28T16:14:58.098521+0000","last_became_active":"2024-06-28T15:21:07.046218+0000","last_became_peered":"2024-06-28T15:21:07.046218+0000","last_unstale":"2024-06-28T16:14:58.098521+0000","last_undegraded":"2024-06-28T16:14:58.098521+0000","last_fullsized":"2024-06-28T16:14:58.098521+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:14:58.098319+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:14:58.098319+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.098319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:38:23.012737+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"260'54","reported_seq":217,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.106471+0000","last_change":"2024-06-28T16:14:47.106537+0000","last_active":"2024-06-28T16:14:47.106471+0000","last_peered":"2024-06-28T16:14:47.106471+0000","last_clean":"2024-06-28T16:14:47.106471+0000","last_became_active":"2024-06-28T15:17:24.010230+0000","last_became_peered":"2024-06-28T15:17:24.010230+0000","last_unstale":"2024-06-28T16:14:47.106471+0000","last_undegraded":"2024-06-28T16:14:47.106471+0000","last_fullsized":"2024-06-28T16:14:47.106471+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'54","last_scrub_stamp":"2024-06-28T16:14:47.106259+0000","last_deep_scrub":"260'54","last_deep_scrub_stamp":"2024-06-28T16:14:47.106259+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.106259+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:33:32.525834+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":283,"num_read_kb":205,"num_write":97,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:35.594467+0000","last_change":"2024-06-28T15:37:33.343356+0000","last_active":"2024-06-28T16:13:35.594467+0000","last_peered":"2024-06-28T16:13:35.594467+0000","last_clean":"2024-06-28T16:13:35.594467+0000","last_became_active":"2024-06-28T15:37:33.342585+0000","last_became_peered":"2024-06-28T15:37:33.342585+0000","last_unstale":"2024-06-28T16:13:35.594467+0000","last_undegraded":"2024-06-28T16:13:35.594467+0000","last_fullsized":"2024-06-28T16:13:35.594467+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"260'3","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.666961+0000","last_change":"2024-06-28T16:14:56.667083+0000","last_active":"2024-06-28T16:14:56.666961+0000","last_peered":"2024-06-28T16:14:56.666961+0000","last_clean":"2024-06-28T16:14:56.666961+0000","last_became_active":"2024-06-28T15:21:11.096884+0000","last_became_peered":"2024-06-28T15:21:11.096884+0000","last_unstale":"2024-06-28T16:14:56.666961+0000","last_undegraded":"2024-06-28T16:14:56.666961+0000","last_fullsized":"2024-06-28T16:14:56.666961+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:14:56.666741+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:14:56.666741+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.666741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:25:21.454528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"260'12","reported_seq":89,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.958983+0000","last_change":"2024-06-28T16:14:52.959056+0000","last_active":"2024-06-28T16:14:52.958983+0000","last_peered":"2024-06-28T16:14:52.958983+0000","last_clean":"2024-06-28T16:14:52.958983+0000","last_became_active":"2024-06-28T15:21:07.032545+0000","last_became_peered":"2024-06-28T15:21:07.032545+0000","last_unstale":"2024-06-28T16:14:52.958983+0000","last_undegraded":"2024-06-28T16:14:52.958983+0000","last_fullsized":"2024-06-28T16:14:52.958983+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'12","last_scrub_stamp":"2024-06-28T16:14:52.958803+0000","last_deep_scrub":"260'12","last_deep_scrub_stamp":"2024-06-28T16:14:52.958803+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.958803+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:27:34.294061+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":35842,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"260'40","reported_seq":180,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.119887+0000","last_change":"2024-06-28T16:14:42.119952+0000","last_active":"2024-06-28T16:14:42.119887+0000","last_peered":"2024-06-28T16:14:42.119887+0000","last_clean":"2024-06-28T16:14:42.119887+0000","last_became_active":"2024-06-28T15:17:24.026702+0000","last_became_peered":"2024-06-28T15:17:24.026702+0000","last_unstale":"2024-06-28T16:14:42.119887+0000","last_undegraded":"2024-06-28T16:14:42.119887+0000","last_fullsized":"2024-06-28T16:14:42.119887+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'40","last_scrub_stamp":"2024-06-28T16:14:42.119697+0000","last_deep_scrub":"260'40","last_deep_scrub_stamp":"2024-06-28T16:14:42.119697+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.119697+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:57:41.013540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":150,"num_write":83,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.117393+0000","last_change":"2024-06-28T15:37:33.343533+0000","last_active":"2024-06-28T16:13:00.117393+0000","last_peered":"2024-06-28T16:13:00.117393+0000","last_clean":"2024-06-28T16:13:00.117393+0000","last_became_active":"2024-06-28T15:37:33.342889+0000","last_became_peered":"2024-06-28T15:37:33.342889+0000","last_unstale":"2024-06-28T16:13:00.117393+0000","last_undegraded":"2024-06-28T16:13:00.117393+0000","last_fullsized":"2024-06-28T16:13:00.117393+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"260'2","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.477042+0000","last_change":"2024-06-28T15:21:11.093441+0000","last_active":"2024-06-28T16:14:34.477042+0000","last_peered":"2024-06-28T16:14:34.477042+0000","last_clean":"2024-06-28T16:14:34.477042+0000","last_became_active":"2024-06-28T15:21:11.092614+0000","last_became_peered":"2024-06-28T15:21:11.092614+0000","last_unstale":"2024-06-28T16:14:34.477042+0000","last_undegraded":"2024-06-28T16:14:34.477042+0000","last_fullsized":"2024-06-28T16:14:34.477042+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"257'3","reported_seq":67,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.625958+0000","last_change":"2024-06-28T16:14:47.626086+0000","last_active":"2024-06-28T16:14:47.625958+0000","last_peered":"2024-06-28T16:14:47.625958+0000","last_clean":"2024-06-28T16:14:47.625958+0000","last_became_active":"2024-06-28T15:21:07.034845+0000","last_became_peered":"2024-06-28T15:21:07.034845+0000","last_unstale":"2024-06-28T16:14:47.625958+0000","last_undegraded":"2024-06-28T16:14:47.625958+0000","last_fullsized":"2024-06-28T16:14:47.625958+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:14:47.625736+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:14:47.625736+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.625736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:32:04.945420+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"260'28","reported_seq":148,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.166666+0000","last_change":"2024-06-28T16:14:41.166687+0000","last_active":"2024-06-28T16:14:41.166666+0000","last_peered":"2024-06-28T16:14:41.166666+0000","last_clean":"2024-06-28T16:14:41.166666+0000","last_became_active":"2024-06-28T15:17:24.014034+0000","last_became_peered":"2024-06-28T15:17:24.014034+0000","last_unstale":"2024-06-28T16:14:41.166666+0000","last_undegraded":"2024-06-28T16:14:41.166666+0000","last_fullsized":"2024-06-28T16:14:41.166666+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'28","last_scrub_stamp":"2024-06-28T16:14:41.166589+0000","last_deep_scrub":"260'28","last_deep_scrub_stamp":"2024-06-28T16:14:41.166589+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.166589+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:11:28.889098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":149,"num_read_kb":105,"num_write":55,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.103364+0000","last_change":"2024-06-28T15:37:33.340550+0000","last_active":"2024-06-28T16:13:00.103364+0000","last_peered":"2024-06-28T16:13:00.103364+0000","last_clean":"2024-06-28T16:13:00.103364+0000","last_became_active":"2024-06-28T15:37:33.339831+0000","last_became_peered":"2024-06-28T15:37:33.339831+0000","last_unstale":"2024-06-28T16:13:00.103364+0000","last_undegraded":"2024-06-28T16:13:00.103364+0000","last_fullsized":"2024-06-28T16:13:00.103364+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"260'7","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.436003+0000","last_change":"2024-06-28T15:21:11.096084+0000","last_active":"2024-06-28T16:14:33.436003+0000","last_peered":"2024-06-28T16:14:33.436003+0000","last_clean":"2024-06-28T16:14:33.436003+0000","last_became_active":"2024-06-28T15:21:11.088011+0000","last_became_peered":"2024-06-28T15:21:11.088011+0000","last_unstale":"2024-06-28T16:14:33.436003+0000","last_undegraded":"2024-06-28T16:14:33.436003+0000","last_fullsized":"2024-06-28T16:14:33.436003+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":20522,"num_write":11,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"0'0","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:30.295047+0000","last_change":"2024-06-28T15:21:07.031400+0000","last_active":"2024-06-28T16:13:30.295047+0000","last_peered":"2024-06-28T16:13:30.295047+0000","last_clean":"2024-06-28T16:13:30.295047+0000","last_became_active":"2024-06-28T15:21:07.030690+0000","last_became_peered":"2024-06-28T15:21:07.030690+0000","last_unstale":"2024-06-28T16:13:30.295047+0000","last_undegraded":"2024-06-28T16:13:30.295047+0000","last_fullsized":"2024-06-28T16:13:30.295047+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"260'31","reported_seq":153,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.094693+0000","last_change":"2024-06-28T16:14:44.094763+0000","last_active":"2024-06-28T16:14:44.094693+0000","last_peered":"2024-06-28T16:14:44.094693+0000","last_clean":"2024-06-28T16:14:44.094693+0000","last_became_active":"2024-06-28T15:17:24.026404+0000","last_became_peered":"2024-06-28T15:17:24.026404+0000","last_unstale":"2024-06-28T16:14:44.094693+0000","last_undegraded":"2024-06-28T16:14:44.094693+0000","last_fullsized":"2024-06-28T16:14:44.094693+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'31","last_scrub_stamp":"2024-06-28T16:14:44.094496+0000","last_deep_scrub":"260'31","last_deep_scrub_stamp":"2024-06-28T16:14:44.094496+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.094496+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:11:57.383608+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":161,"num_read_kb":111,"num_write":59,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":31,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.127686+0000","last_change":"2024-06-28T15:37:33.332984+0000","last_active":"2024-06-28T16:13:45.127686+0000","last_peered":"2024-06-28T16:13:45.127686+0000","last_clean":"2024-06-28T16:13:45.127686+0000","last_became_active":"2024-06-28T15:37:33.332245+0000","last_became_peered":"2024-06-28T15:37:33.332245+0000","last_unstale":"2024-06-28T16:13:45.127686+0000","last_undegraded":"2024-06-28T16:13:45.127686+0000","last_fullsized":"2024-06-28T16:13:45.127686+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"0'0","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.610237+0000","last_change":"2024-06-28T16:14:52.610366+0000","last_active":"2024-06-28T16:14:52.610237+0000","last_peered":"2024-06-28T16:14:52.610237+0000","last_clean":"2024-06-28T16:14:52.610237+0000","last_became_active":"2024-06-28T15:21:11.096926+0000","last_became_peered":"2024-06-28T15:21:11.096926+0000","last_unstale":"2024-06-28T16:14:52.610237+0000","last_undegraded":"2024-06-28T16:14:52.610237+0000","last_fullsized":"2024-06-28T16:14:52.610237+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:52.610021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:52.610021+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.610021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:07:41.471423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"260'2","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.578876+0000","last_change":"2024-06-28T16:14:56.578940+0000","last_active":"2024-06-28T16:14:56.578876+0000","last_peered":"2024-06-28T16:14:56.578876+0000","last_clean":"2024-06-28T16:14:56.578876+0000","last_became_active":"2024-06-28T15:21:07.036735+0000","last_became_peered":"2024-06-28T15:21:07.036735+0000","last_unstale":"2024-06-28T16:14:56.578876+0000","last_undegraded":"2024-06-28T16:14:56.578876+0000","last_fullsized":"2024-06-28T16:14:56.578876+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:56.578669+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:56.578669+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.578669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:47:53.590130+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"260'49","reported_seq":206,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.629147+0000","last_change":"2024-06-28T16:14:49.629207+0000","last_active":"2024-06-28T16:14:49.629147+0000","last_peered":"2024-06-28T16:14:49.629147+0000","last_clean":"2024-06-28T16:14:49.629147+0000","last_became_active":"2024-06-28T15:17:24.025855+0000","last_became_peered":"2024-06-28T15:17:24.025855+0000","last_unstale":"2024-06-28T16:14:49.629147+0000","last_undegraded":"2024-06-28T16:14:49.629147+0000","last_fullsized":"2024-06-28T16:14:49.629147+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'49","last_scrub_stamp":"2024-06-28T16:14:49.628959+0000","last_deep_scrub":"260'49","last_deep_scrub_stamp":"2024-06-28T16:14:49.628959+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.628959+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:27:20.197035+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":190,"num_write":87,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.110242+0000","last_change":"2024-06-28T15:37:33.323701+0000","last_active":"2024-06-28T16:13:00.110242+0000","last_peered":"2024-06-28T16:13:00.110242+0000","last_clean":"2024-06-28T16:13:00.110242+0000","last_became_active":"2024-06-28T15:37:33.322927+0000","last_became_peered":"2024-06-28T15:37:33.322927+0000","last_unstale":"2024-06-28T16:13:00.110242+0000","last_undegraded":"2024-06-28T16:13:00.110242+0000","last_fullsized":"2024-06-28T16:13:00.110242+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":47,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.113538+0000","last_change":"2024-06-28T15:21:11.098521+0000","last_active":"2024-06-28T16:13:15.113538+0000","last_peered":"2024-06-28T16:13:15.113538+0000","last_clean":"2024-06-28T16:13:15.113538+0000","last_became_active":"2024-06-28T15:21:11.097766+0000","last_became_peered":"2024-06-28T15:21:11.097766+0000","last_unstale":"2024-06-28T16:13:15.113538+0000","last_undegraded":"2024-06-28T16:13:15.113538+0000","last_fullsized":"2024-06-28T16:13:15.113538+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.615895+0000","last_change":"2024-06-28T16:14:46.616012+0000","last_active":"2024-06-28T16:14:46.615895+0000","last_peered":"2024-06-28T16:14:46.615895+0000","last_clean":"2024-06-28T16:14:46.615895+0000","last_became_active":"2024-06-28T15:21:07.039381+0000","last_became_peered":"2024-06-28T15:21:07.039381+0000","last_unstale":"2024-06-28T16:14:46.615895+0000","last_undegraded":"2024-06-28T16:14:46.615895+0000","last_fullsized":"2024-06-28T16:14:46.615895+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:46.615676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:46.615676+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.615676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:02:38.667378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"260'26","reported_seq":146,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.130914+0000","last_change":"2024-06-28T16:14:42.130952+0000","last_active":"2024-06-28T16:14:42.130914+0000","last_peered":"2024-06-28T16:14:42.130914+0000","last_clean":"2024-06-28T16:14:42.130914+0000","last_became_active":"2024-06-28T15:17:24.026580+0000","last_became_peered":"2024-06-28T15:17:24.026580+0000","last_unstale":"2024-06-28T16:14:42.130914+0000","last_undegraded":"2024-06-28T16:14:42.130914+0000","last_fullsized":"2024-06-28T16:14:42.130914+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'26","last_scrub_stamp":"2024-06-28T16:14:42.130814+0000","last_deep_scrub":"260'26","last_deep_scrub_stamp":"2024-06-28T16:14:42.130814+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.130814+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:03:06.475528+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":143,"num_read_kb":101,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"259'1","reported_seq":31,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:25.306146+0000","last_change":"2024-06-28T15:37:33.349810+0000","last_active":"2024-06-28T16:14:25.306146+0000","last_peered":"2024-06-28T16:14:25.306146+0000","last_clean":"2024-06-28T16:14:25.306146+0000","last_became_active":"2024-06-28T15:37:33.349137+0000","last_became_peered":"2024-06-28T15:37:33.349137+0000","last_unstale":"2024-06-28T16:14:25.306146+0000","last_undegraded":"2024-06-28T16:14:25.306146+0000","last_fullsized":"2024-06-28T16:14:25.306146+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"260'6","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.517228+0000","last_change":"2024-06-28T15:21:11.106004+0000","last_active":"2024-06-28T16:14:32.517228+0000","last_peered":"2024-06-28T16:14:32.517228+0000","last_clean":"2024-06-28T16:14:32.517228+0000","last_became_active":"2024-06-28T15:21:11.104270+0000","last_became_peered":"2024-06-28T15:21:11.104270+0000","last_unstale":"2024-06-28T16:14:32.517228+0000","last_undegraded":"2024-06-28T16:14:32.517228+0000","last_fullsized":"2024-06-28T16:14:32.517228+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":11,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"260'6","reported_seq":72,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.653099+0000","last_change":"2024-06-28T16:14:44.653159+0000","last_active":"2024-06-28T16:14:44.653099+0000","last_peered":"2024-06-28T16:14:44.653099+0000","last_clean":"2024-06-28T16:14:44.653099+0000","last_became_active":"2024-06-28T15:21:07.048639+0000","last_became_peered":"2024-06-28T15:21:07.048639+0000","last_unstale":"2024-06-28T16:14:44.653099+0000","last_undegraded":"2024-06-28T16:14:44.653099+0000","last_fullsized":"2024-06-28T16:14:44.653099+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:14:44.652902+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:14:44.652902+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.652902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:58:20.490373+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":12288,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"260'22","reported_seq":123,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.693623+0000","last_change":"2024-06-28T16:14:41.693689+0000","last_active":"2024-06-28T16:14:41.693623+0000","last_peered":"2024-06-28T16:14:41.693623+0000","last_clean":"2024-06-28T16:14:41.693623+0000","last_became_active":"2024-06-28T15:17:24.006784+0000","last_became_peered":"2024-06-28T15:17:24.006784+0000","last_unstale":"2024-06-28T16:14:41.693623+0000","last_undegraded":"2024-06-28T16:14:41.693623+0000","last_fullsized":"2024-06-28T16:14:41.693623+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'22","last_scrub_stamp":"2024-06-28T16:14:41.693433+0000","last_deep_scrub":"260'22","last_deep_scrub_stamp":"2024-06-28T16:14:41.693433+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.693433+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:31:21.802215+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":64,"num_write":51,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:25.605834+0000","last_change":"2024-06-28T15:37:33.334905+0000","last_active":"2024-06-28T16:14:25.605834+0000","last_peered":"2024-06-28T16:14:25.605834+0000","last_clean":"2024-06-28T16:14:25.605834+0000","last_became_active":"2024-06-28T15:37:33.333737+0000","last_became_peered":"2024-06-28T15:37:33.333737+0000","last_unstale":"2024-06-28T16:14:25.605834+0000","last_undegraded":"2024-06-28T16:14:25.605834+0000","last_fullsized":"2024-06-28T16:14:25.605834+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":49,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:25.108936+0000","last_change":"2024-06-28T15:21:11.092678+0000","last_active":"2024-06-28T16:13:25.108936+0000","last_peered":"2024-06-28T16:13:25.108936+0000","last_clean":"2024-06-28T16:13:25.108936+0000","last_became_active":"2024-06-28T15:21:11.092034+0000","last_became_peered":"2024-06-28T15:21:11.092034+0000","last_unstale":"2024-06-28T16:13:25.108936+0000","last_undegraded":"2024-06-28T16:13:25.108936+0000","last_fullsized":"2024-06-28T16:13:25.108936+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"260'3","reported_seq":68,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.606891+0000","last_change":"2024-06-28T16:14:43.606954+0000","last_active":"2024-06-28T16:14:43.606891+0000","last_peered":"2024-06-28T16:14:43.606891+0000","last_clean":"2024-06-28T16:14:43.606891+0000","last_became_active":"2024-06-28T15:21:07.039812+0000","last_became_peered":"2024-06-28T15:21:07.039812+0000","last_unstale":"2024-06-28T16:14:43.606891+0000","last_undegraded":"2024-06-28T16:14:43.606891+0000","last_fullsized":"2024-06-28T16:14:43.606891+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:14:43.606698+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:14:43.606698+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.606698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:49:13.801868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"260'78","reported_seq":317,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.919916+0000","last_change":"2024-06-28T16:14:41.919989+0000","last_active":"2024-06-28T16:14:41.919916+0000","last_peered":"2024-06-28T16:14:41.919916+0000","last_clean":"2024-06-28T16:14:41.919916+0000","last_became_active":"2024-06-28T15:17:24.012854+0000","last_became_peered":"2024-06-28T15:17:24.012854+0000","last_unstale":"2024-06-28T16:14:41.919916+0000","last_undegraded":"2024-06-28T16:14:41.919916+0000","last_fullsized":"2024-06-28T16:14:41.919916+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'78","last_scrub_stamp":"2024-06-28T16:14:41.919720+0000","last_deep_scrub":"260'78","last_deep_scrub_stamp":"2024-06-28T16:14:41.919720+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.919720+0000","objects_scrubbed":1,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:17:37.822460+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":483,"num_read_kb":330,"num_write":109,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4960,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.585523+0000","last_change":"2024-06-28T15:37:33.323089+0000","last_active":"2024-06-28T16:13:00.585523+0000","last_peered":"2024-06-28T16:13:00.585523+0000","last_clean":"2024-06-28T16:13:00.585523+0000","last_became_active":"2024-06-28T15:37:33.322317+0000","last_became_peered":"2024-06-28T15:37:33.322317+0000","last_unstale":"2024-06-28T16:13:00.585523+0000","last_undegraded":"2024-06-28T16:13:00.585523+0000","last_fullsized":"2024-06-28T16:13:00.585523+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":47,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.113276+0000","last_change":"2024-06-28T15:21:11.109416+0000","last_active":"2024-06-28T16:13:20.113276+0000","last_peered":"2024-06-28T16:13:20.113276+0000","last_clean":"2024-06-28T16:13:20.113276+0000","last_became_active":"2024-06-28T15:21:11.108663+0000","last_became_peered":"2024-06-28T15:21:11.108663+0000","last_unstale":"2024-06-28T16:13:20.113276+0000","last_undegraded":"2024-06-28T16:13:20.113276+0000","last_fullsized":"2024-06-28T16:13:20.113276+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"260'3","reported_seq":67,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.135088+0000","last_change":"2024-06-28T16:14:40.135141+0000","last_active":"2024-06-28T16:14:40.135088+0000","last_peered":"2024-06-28T16:14:40.135088+0000","last_clean":"2024-06-28T16:14:40.135088+0000","last_became_active":"2024-06-28T15:21:07.022046+0000","last_became_peered":"2024-06-28T15:21:07.022046+0000","last_unstale":"2024-06-28T16:14:40.135088+0000","last_undegraded":"2024-06-28T16:14:40.135088+0000","last_fullsized":"2024-06-28T16:14:40.135088+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:14:40.134988+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:14:40.134988+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.134988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:06:49.985940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"260'31","reported_seq":145,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.121279+0000","last_change":"2024-06-28T16:14:45.121348+0000","last_active":"2024-06-28T16:14:45.121279+0000","last_peered":"2024-06-28T16:14:45.121279+0000","last_clean":"2024-06-28T16:14:45.121279+0000","last_became_active":"2024-06-28T15:17:24.010873+0000","last_became_peered":"2024-06-28T15:17:24.010873+0000","last_unstale":"2024-06-28T16:14:45.121279+0000","last_undegraded":"2024-06-28T16:14:45.121279+0000","last_fullsized":"2024-06-28T16:14:45.121279+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'31","last_scrub_stamp":"2024-06-28T16:14:45.121064+0000","last_deep_scrub":"260'31","last_deep_scrub_stamp":"2024-06-28T16:14:45.121064+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.121064+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:05:38.766333+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":94,"num_write":78,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:10.119681+0000","last_change":"2024-06-28T15:37:33.347898+0000","last_active":"2024-06-28T16:13:10.119681+0000","last_peered":"2024-06-28T16:13:10.119681+0000","last_clean":"2024-06-28T16:13:10.119681+0000","last_became_active":"2024-06-28T15:37:33.347075+0000","last_became_peered":"2024-06-28T15:37:33.347075+0000","last_unstale":"2024-06-28T16:13:10.119681+0000","last_undegraded":"2024-06-28T16:13:10.119681+0000","last_fullsized":"2024-06-28T16:13:10.119681+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":48,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.114588+0000","last_change":"2024-06-28T15:21:11.093738+0000","last_active":"2024-06-28T16:13:20.114588+0000","last_peered":"2024-06-28T16:13:20.114588+0000","last_clean":"2024-06-28T16:13:20.114588+0000","last_became_active":"2024-06-28T15:21:11.093082+0000","last_became_peered":"2024-06-28T15:21:11.093082+0000","last_unstale":"2024-06-28T16:13:20.114588+0000","last_undegraded":"2024-06-28T16:13:20.114588+0000","last_fullsized":"2024-06-28T16:13:20.114588+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"257'3","reported_seq":67,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.165436+0000","last_change":"2024-06-28T16:14:46.165567+0000","last_active":"2024-06-28T16:14:46.165436+0000","last_peered":"2024-06-28T16:14:46.165436+0000","last_clean":"2024-06-28T16:14:46.165436+0000","last_became_active":"2024-06-28T15:21:07.040575+0000","last_became_peered":"2024-06-28T15:21:07.040575+0000","last_unstale":"2024-06-28T16:14:46.165436+0000","last_undegraded":"2024-06-28T16:14:46.165436+0000","last_fullsized":"2024-06-28T16:14:46.165436+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:14:46.165127+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:14:46.165127+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.165127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:42:56.667095+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"260'16","reported_seq":107,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.625878+0000","last_change":"2024-06-28T16:14:51.625944+0000","last_active":"2024-06-28T16:14:51.625878+0000","last_peered":"2024-06-28T16:14:51.625878+0000","last_clean":"2024-06-28T16:14:51.625878+0000","last_became_active":"2024-06-28T15:17:24.021464+0000","last_became_peered":"2024-06-28T15:17:24.021464+0000","last_unstale":"2024-06-28T16:14:51.625878+0000","last_undegraded":"2024-06-28T16:14:51.625878+0000","last_fullsized":"2024-06-28T16:14:51.625878+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'16","last_scrub_stamp":"2024-06-28T16:14:51.625680+0000","last_deep_scrub":"260'16","last_deep_scrub_stamp":"2024-06-28T16:14:51.625680+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.625680+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:28:15.263944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":44,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"260'341","reported_seq":971,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.145524+0000","last_change":"2024-06-28T16:14:46.145629+0000","last_active":"2024-06-28T16:14:46.145524+0000","last_peered":"2024-06-28T16:14:46.145524+0000","last_clean":"2024-06-28T16:14:46.145524+0000","last_became_active":"2024-06-28T15:00:59.941681+0000","last_became_peered":"2024-06-28T15:00:59.941681+0000","last_unstale":"2024-06-28T16:14:46.145524+0000","last_undegraded":"2024-06-28T16:14:46.145524+0000","last_fullsized":"2024-06-28T16:14:46.145524+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'341","last_scrub_stamp":"2024-06-28T16:14:46.145323+0000","last_deep_scrub":"260'341","last_deep_scrub_stamp":"2024-06-28T16:14:46.145323+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.145323+0000","objects_scrubbed":6,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:36:27.892777+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170282,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":488,"num_read_kb":26898,"num_write":1268,"num_write_kb":25985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0,5],"acting":[2,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"260'181","reported_seq":787,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.100691+0000","last_change":"2024-06-28T15:00:17.405026+0000","last_active":"2024-06-28T16:12:50.100691+0000","last_peered":"2024-06-28T16:12:50.100691+0000","last_clean":"2024-06-28T16:12:50.100691+0000","last_became_active":"2024-06-28T15:00:17.404342+0000","last_became_peered":"2024-06-28T15:00:17.404342+0000","last_unstale":"2024-06-28T16:12:50.100691+0000","last_undegraded":"2024-06-28T16:12:50.100691+0000","last_fullsized":"2024-06-28T16:12:50.100691+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":100,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":722,"num_read_kb":569,"num_write":267,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"260'326","reported_seq":972,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.743327+0000","last_change":"2024-06-28T15:00:15.419867+0000","last_active":"2024-06-28T16:14:34.743327+0000","last_peered":"2024-06-28T16:14:34.743327+0000","last_clean":"2024-06-28T16:14:34.743327+0000","last_became_active":"2024-06-28T15:00:15.419449+0000","last_became_peered":"2024-06-28T15:00:15.419449+0000","last_unstale":"2024-06-28T16:14:34.743327+0000","last_undegraded":"2024-06-28T16:14:34.743327+0000","last_fullsized":"2024-06-28T16:14:34.743327+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5526738,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":485,"num_read_kb":15789,"num_write":1172,"num_write_kb":37304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"260'285","reported_seq":940,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.115046+0000","last_change":"2024-06-28T15:00:17.409108+0000","last_active":"2024-06-28T16:12:50.115046+0000","last_peered":"2024-06-28T16:12:50.115046+0000","last_clean":"2024-06-28T16:12:50.115046+0000","last_became_active":"2024-06-28T15:00:17.408128+0000","last_became_peered":"2024-06-28T15:00:17.408128+0000","last_unstale":"2024-06-28T16:12:50.115046+0000","last_undegraded":"2024-06-28T16:12:50.115046+0000","last_fullsized":"2024-06-28T16:12:50.115046+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1157,"num_read_kb":911,"num_write":439,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"260'283","reported_seq":970,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.735843+0000","last_change":"2024-06-28T15:00:15.397113+0000","last_active":"2024-06-28T16:14:34.735843+0000","last_peered":"2024-06-28T16:14:34.735843+0000","last_clean":"2024-06-28T16:14:34.735843+0000","last_became_active":"2024-06-28T15:00:15.396469+0000","last_became_peered":"2024-06-28T15:00:15.396469+0000","last_unstale":"2024-06-28T16:14:34.735843+0000","last_undegraded":"2024-06-28T16:14:34.735843+0000","last_fullsized":"2024-06-28T16:14:34.735843+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":200,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1096214,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":504,"num_read_kb":81437,"num_write":1108,"num_write_kb":14924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"260'77","reported_seq":644,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.107930+0000","last_change":"2024-06-28T15:00:17.403326+0000","last_active":"2024-06-28T16:12:50.107930+0000","last_peered":"2024-06-28T16:12:50.107930+0000","last_clean":"2024-06-28T16:12:50.107930+0000","last_became_active":"2024-06-28T15:00:17.402542+0000","last_became_peered":"2024-06-28T15:00:17.402542+0000","last_unstale":"2024-06-28T16:12:50.107930+0000","last_undegraded":"2024-06-28T16:12:50.107930+0000","last_fullsized":"2024-06-28T16:12:50.107930+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":306,"num_read_kb":248,"num_write":126,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"260'313","reported_seq":982,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.709656+0000","last_change":"2024-06-28T15:00:15.435754+0000","last_active":"2024-06-28T16:14:34.709656+0000","last_peered":"2024-06-28T16:14:34.709656+0000","last_clean":"2024-06-28T16:14:34.709656+0000","last_became_active":"2024-06-28T15:00:15.435122+0000","last_became_peered":"2024-06-28T15:00:15.435122+0000","last_unstale":"2024-06-28T16:14:34.709656+0000","last_undegraded":"2024-06-28T16:14:34.709656+0000","last_fullsized":"2024-06-28T16:14:34.709656+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1454317,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":492,"num_read_kb":48678,"num_write":1275,"num_write_kb":34259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"260'51","reported_seq":630,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.582691+0000","last_change":"2024-06-28T15:00:17.385309+0000","last_active":"2024-06-28T16:12:50.582691+0000","last_peered":"2024-06-28T16:12:50.582691+0000","last_clean":"2024-06-28T16:12:50.582691+0000","last_became_active":"2024-06-28T15:00:17.384932+0000","last_became_peered":"2024-06-28T15:00:17.384932+0000","last_unstale":"2024-06-28T16:12:50.582691+0000","last_undegraded":"2024-06-28T16:12:50.582691+0000","last_fullsized":"2024-06-28T16:12:50.582691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":197,"num_write":67,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"260'343","reported_seq":993,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.753114+0000","last_change":"2024-06-28T15:00:15.430784+0000","last_active":"2024-06-28T16:14:34.753114+0000","last_peered":"2024-06-28T16:14:34.753114+0000","last_clean":"2024-06-28T16:14:34.753114+0000","last_became_active":"2024-06-28T15:00:15.430208+0000","last_became_peered":"2024-06-28T15:00:15.430208+0000","last_unstale":"2024-06-28T16:14:34.753114+0000","last_undegraded":"2024-06-28T16:14:34.753114+0000","last_fullsized":"2024-06-28T16:14:34.753114+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437780,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":548,"num_read_kb":39482,"num_write":1215,"num_write_kb":35176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"260'186","reported_seq":848,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.448598+0000","last_change":"2024-06-28T16:14:59.448668+0000","last_active":"2024-06-28T16:14:59.448598+0000","last_peered":"2024-06-28T16:14:59.448598+0000","last_clean":"2024-06-28T16:14:59.448598+0000","last_became_active":"2024-06-28T15:00:17.397986+0000","last_became_peered":"2024-06-28T15:00:17.397986+0000","last_unstale":"2024-06-28T16:14:59.448598+0000","last_undegraded":"2024-06-28T16:14:59.448598+0000","last_fullsized":"2024-06-28T16:14:59.448598+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'186","last_scrub_stamp":"2024-06-28T16:14:59.448384+0000","last_deep_scrub":"260'186","last_deep_scrub_stamp":"2024-06-28T16:14:59.448384+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.448384+0000","objects_scrubbed":1,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:21:42.502350+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":777,"num_read_kb":632,"num_write":274,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":473,"num_omap_keys":2,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"260'364","reported_seq":1030,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.718372+0000","last_change":"2024-06-28T15:00:15.412175+0000","last_active":"2024-06-28T16:14:34.718372+0000","last_peered":"2024-06-28T16:14:34.718372+0000","last_clean":"2024-06-28T16:14:34.718372+0000","last_became_active":"2024-06-28T15:00:15.409542+0000","last_became_peered":"2024-06-28T15:00:15.409542+0000","last_unstale":"2024-06-28T16:14:34.718372+0000","last_undegraded":"2024-06-28T16:14:34.718372+0000","last_fullsized":"2024-06-28T16:14:34.718372+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1667748,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":553,"num_read_kb":16754,"num_write":1370,"num_write_kb":32794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"260'44","reported_seq":617,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.888425+0000","last_change":"2024-06-28T16:14:59.888516+0000","last_active":"2024-06-28T16:14:59.888425+0000","last_peered":"2024-06-28T16:14:59.888425+0000","last_clean":"2024-06-28T16:14:59.888425+0000","last_became_active":"2024-06-28T15:00:17.401985+0000","last_became_peered":"2024-06-28T15:00:17.401985+0000","last_unstale":"2024-06-28T16:14:59.888425+0000","last_undegraded":"2024-06-28T16:14:59.888425+0000","last_fullsized":"2024-06-28T16:14:59.888425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'44","last_scrub_stamp":"2024-06-28T16:14:59.888222+0000","last_deep_scrub":"260'44","last_deep_scrub_stamp":"2024-06-28T16:14:59.888222+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.888222+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:28:28.187398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":159,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"260'328","reported_seq":960,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.705772+0000","last_change":"2024-06-28T15:00:15.425314+0000","last_active":"2024-06-28T16:14:34.705772+0000","last_peered":"2024-06-28T16:14:34.705772+0000","last_clean":"2024-06-28T16:14:34.705772+0000","last_became_active":"2024-06-28T15:00:15.424430+0000","last_became_peered":"2024-06-28T15:00:15.424430+0000","last_unstale":"2024-06-28T16:14:34.705772+0000","last_undegraded":"2024-06-28T16:14:34.705772+0000","last_fullsized":"2024-06-28T16:14:34.705772+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048602,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":476,"num_read_kb":21908,"num_write":1069,"num_write_kb":36265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"260'4326","reported_seq":5005,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.551620+0000","last_change":"2024-06-28T16:14:59.551681+0000","last_active":"2024-06-28T16:14:59.551620+0000","last_peered":"2024-06-28T16:14:59.551620+0000","last_clean":"2024-06-28T16:14:59.551620+0000","last_became_active":"2024-06-28T15:00:17.384378+0000","last_became_peered":"2024-06-28T15:00:17.384378+0000","last_unstale":"2024-06-28T16:14:59.551620+0000","last_undegraded":"2024-06-28T16:14:59.551620+0000","last_fullsized":"2024-06-28T16:14:59.551620+0000","mapping_epoch":21,"log_start":"260'4300","ondisk_log_start":"260'4300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4326","last_scrub_stamp":"2024-06-28T16:14:59.551430+0000","last_deep_scrub":"260'4326","last_deep_scrub_stamp":"2024-06-28T16:14:59.551430+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.551430+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":3000,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:51:42.703162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15352,"num_read_kb":12408,"num_write":6557,"num_write_kb":3277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"260'411","reported_seq":1129,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.680578+0000","last_change":"2024-06-28T15:00:15.403350+0000","last_active":"2024-06-28T16:14:34.680578+0000","last_peered":"2024-06-28T16:14:34.680578+0000","last_clean":"2024-06-28T16:14:34.680578+0000","last_became_active":"2024-06-28T15:00:15.401960+0000","last_became_peered":"2024-06-28T15:00:15.401960+0000","last_unstale":"2024-06-28T16:14:34.680578+0000","last_undegraded":"2024-06-28T16:14:34.680578+0000","last_fullsized":"2024-06-28T16:14:34.680578+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":400,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324665,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":810,"num_read_kb":20615,"num_write":1466,"num_write_kb":29750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"260'70","reported_seq":694,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.302139+0000","last_change":"2024-06-28T16:14:59.302209+0000","last_active":"2024-06-28T16:14:59.302139+0000","last_peered":"2024-06-28T16:14:59.302139+0000","last_clean":"2024-06-28T16:14:59.302139+0000","last_became_active":"2024-06-28T15:00:17.381196+0000","last_became_peered":"2024-06-28T15:00:17.381196+0000","last_unstale":"2024-06-28T16:14:59.302139+0000","last_undegraded":"2024-06-28T16:14:59.302139+0000","last_fullsized":"2024-06-28T16:14:59.302139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'70","last_scrub_stamp":"2024-06-28T16:14:59.301939+0000","last_deep_scrub":"260'70","last_deep_scrub_stamp":"2024-06-28T16:14:59.301939+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.301939+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:19:16.430915+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":309,"num_read_kb":273,"num_write":88,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"260'128","reported_seq":752,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.422647+0000","last_change":"2024-06-28T15:00:17.396995+0000","last_active":"2024-06-28T16:14:40.422647+0000","last_peered":"2024-06-28T16:14:40.422647+0000","last_clean":"2024-06-28T16:14:40.422647+0000","last_became_active":"2024-06-28T15:00:17.396142+0000","last_became_peered":"2024-06-28T15:00:17.396142+0000","last_unstale":"2024-06-28T16:14:40.422647+0000","last_undegraded":"2024-06-28T16:14:40.422647+0000","last_fullsized":"2024-06-28T16:14:40.422647+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":536,"num_read_kb":452,"num_write":191,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"260'416","reported_seq":1187,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.745249+0000","last_change":"2024-06-28T15:00:15.423248+0000","last_active":"2024-06-28T16:14:34.745249+0000","last_peered":"2024-06-28T16:14:34.745249+0000","last_clean":"2024-06-28T16:14:34.745249+0000","last_became_active":"2024-06-28T15:00:15.422579+0000","last_became_peered":"2024-06-28T15:00:15.422579+0000","last_unstale":"2024-06-28T16:14:34.745249+0000","last_undegraded":"2024-06-28T16:14:34.745249+0000","last_fullsized":"2024-06-28T16:14:34.745249+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":400,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":851,"num_read_kb":96606,"num_write":1464,"num_write_kb":38692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0,1],"acting":[7,5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"260'254","reported_seq":901,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.803017+0000","last_change":"2024-06-28T15:00:17.410347+0000","last_active":"2024-06-28T16:14:41.803017+0000","last_peered":"2024-06-28T16:14:41.803017+0000","last_clean":"2024-06-28T16:14:41.803017+0000","last_became_active":"2024-06-28T15:00:17.409202+0000","last_became_peered":"2024-06-28T15:00:17.409202+0000","last_unstale":"2024-06-28T16:14:41.803017+0000","last_undegraded":"2024-06-28T16:14:41.803017+0000","last_fullsized":"2024-06-28T16:14:41.803017+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1059,"num_read_kb":851,"num_write":432,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"260'313","reported_seq":953,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.750703+0000","last_change":"2024-06-28T15:00:15.423492+0000","last_active":"2024-06-28T16:14:34.750703+0000","last_peered":"2024-06-28T16:14:34.750703+0000","last_clean":"2024-06-28T16:14:34.750703+0000","last_became_active":"2024-06-28T15:00:15.421980+0000","last_became_peered":"2024-06-28T15:00:15.421980+0000","last_unstale":"2024-06-28T16:14:34.750703+0000","last_undegraded":"2024-06-28T16:14:34.750703+0000","last_fullsized":"2024-06-28T16:14:34.750703+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468838,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":425,"num_read_kb":33937,"num_write":1026,"num_write_kb":43346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"260'225","reported_seq":851,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.102173+0000","last_change":"2024-06-28T15:00:17.385629+0000","last_active":"2024-06-28T16:12:55.102173+0000","last_peered":"2024-06-28T16:12:55.102173+0000","last_clean":"2024-06-28T16:12:55.102173+0000","last_became_active":"2024-06-28T15:00:17.384914+0000","last_became_peered":"2024-06-28T15:00:17.384914+0000","last_unstale":"2024-06-28T16:12:55.102173+0000","last_undegraded":"2024-06-28T16:12:55.102173+0000","last_fullsized":"2024-06-28T16:12:55.102173+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":200,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":887,"num_read_kb":706,"num_write":342,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"260'342","reported_seq":1192,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.726392+0000","last_change":"2024-06-28T15:00:15.412640+0000","last_active":"2024-06-28T16:14:34.726392+0000","last_peered":"2024-06-28T16:14:34.726392+0000","last_clean":"2024-06-28T16:14:34.726392+0000","last_became_active":"2024-06-28T15:00:15.411567+0000","last_became_peered":"2024-06-28T15:00:15.411567+0000","last_unstale":"2024-06-28T16:14:34.726392+0000","last_undegraded":"2024-06-28T16:14:34.726392+0000","last_fullsized":"2024-06-28T16:14:34.726392+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289237,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":936,"num_read_kb":14303,"num_write":1297,"num_write_kb":46873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"260'128","reported_seq":737,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.286422+0000","last_change":"2024-06-28T15:00:17.389926+0000","last_active":"2024-06-28T16:12:50.286422+0000","last_peered":"2024-06-28T16:12:50.286422+0000","last_clean":"2024-06-28T16:12:50.286422+0000","last_became_active":"2024-06-28T15:00:17.389613+0000","last_became_peered":"2024-06-28T15:00:17.389613+0000","last_unstale":"2024-06-28T16:12:50.286422+0000","last_undegraded":"2024-06-28T16:12:50.286422+0000","last_fullsized":"2024-06-28T16:12:50.286422+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":419,"num_write":192,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"260'320","reported_seq":1073,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.732457+0000","last_change":"2024-06-28T15:00:15.418207+0000","last_active":"2024-06-28T16:14:34.732457+0000","last_peered":"2024-06-28T16:14:34.732457+0000","last_clean":"2024-06-28T16:14:34.732457+0000","last_became_active":"2024-06-28T15:00:15.417563+0000","last_became_peered":"2024-06-28T15:00:15.417563+0000","last_unstale":"2024-06-28T16:14:34.732457+0000","last_undegraded":"2024-06-28T16:14:34.732457+0000","last_fullsized":"2024-06-28T16:14:34.732457+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":525349,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":605,"num_read_kb":12860,"num_write":1159,"num_write_kb":18563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"260'156","reported_seq":738,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.286500+0000","last_change":"2024-06-28T15:00:17.413709+0000","last_active":"2024-06-28T16:12:50.286500+0000","last_peered":"2024-06-28T16:12:50.286500+0000","last_clean":"2024-06-28T16:12:50.286500+0000","last_became_active":"2024-06-28T15:00:17.413420+0000","last_became_peered":"2024-06-28T15:00:17.413420+0000","last_unstale":"2024-06-28T16:12:50.286500+0000","last_undegraded":"2024-06-28T16:12:50.286500+0000","last_fullsized":"2024-06-28T16:12:50.286500+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":100,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":589,"num_read_kb":466,"num_write":222,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"260'357","reported_seq":1074,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.742055+0000","last_change":"2024-06-28T15:00:15.402501+0000","last_active":"2024-06-28T16:14:34.742055+0000","last_peered":"2024-06-28T16:14:34.742055+0000","last_clean":"2024-06-28T16:14:34.742055+0000","last_became_active":"2024-06-28T15:00:15.401370+0000","last_became_peered":"2024-06-28T15:00:15.401370+0000","last_unstale":"2024-06-28T16:14:34.742055+0000","last_undegraded":"2024-06-28T16:14:34.742055+0000","last_fullsized":"2024-06-28T16:14:34.742055+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324689,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":676,"num_read_kb":30262,"num_write":1536,"num_write_kb":36456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"260'253","reported_seq":903,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.092035+0000","last_change":"2024-06-28T15:00:17.410230+0000","last_active":"2024-06-28T16:13:20.092035+0000","last_peered":"2024-06-28T16:13:20.092035+0000","last_clean":"2024-06-28T16:13:20.092035+0000","last_became_active":"2024-06-28T15:00:17.408972+0000","last_became_peered":"2024-06-28T15:00:17.408972+0000","last_unstale":"2024-06-28T16:13:20.092035+0000","last_undegraded":"2024-06-28T16:13:20.092035+0000","last_fullsized":"2024-06-28T16:13:20.092035+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":990,"num_read_kb":787,"num_write":391,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"260'346","reported_seq":1009,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.742591+0000","last_change":"2024-06-28T15:00:15.408546+0000","last_active":"2024-06-28T16:14:34.742591+0000","last_peered":"2024-06-28T16:14:34.742591+0000","last_clean":"2024-06-28T16:14:34.742591+0000","last_became_active":"2024-06-28T15:00:15.407552+0000","last_became_peered":"2024-06-28T15:00:15.407552+0000","last_unstale":"2024-06-28T16:14:34.742591+0000","last_undegraded":"2024-06-28T16:14:34.742591+0000","last_fullsized":"2024-06-28T16:14:34.742591+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":300,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278750,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":513,"num_read_kb":23448,"num_write":1336,"num_write_kb":46171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"260'146","reported_seq":692,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.984240+0000","last_change":"2024-06-28T16:14:45.984342+0000","last_active":"2024-06-28T16:14:45.984240+0000","last_peered":"2024-06-28T16:14:45.984240+0000","last_clean":"2024-06-28T16:14:45.984240+0000","last_became_active":"2024-06-28T15:00:25.427734+0000","last_became_peered":"2024-06-28T15:00:25.427734+0000","last_unstale":"2024-06-28T16:14:45.984240+0000","last_undegraded":"2024-06-28T16:14:45.984240+0000","last_fullsized":"2024-06-28T16:14:45.984240+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'146","last_scrub_stamp":"2024-06-28T16:14:45.984045+0000","last_deep_scrub":"260'146","last_deep_scrub_stamp":"2024-06-28T16:14:45.984045+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.984045+0000","objects_scrubbed":4,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:59:27.363330+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":233,"num_read_kb":201,"num_write":347,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"260'214","reported_seq":823,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.312228+0000","last_change":"2024-06-28T15:00:17.382620+0000","last_active":"2024-06-28T16:12:50.312228+0000","last_peered":"2024-06-28T16:12:50.312228+0000","last_clean":"2024-06-28T16:12:50.312228+0000","last_became_active":"2024-06-28T15:00:17.382344+0000","last_became_peered":"2024-06-28T15:00:17.382344+0000","last_unstale":"2024-06-28T16:12:50.312228+0000","last_undegraded":"2024-06-28T16:12:50.312228+0000","last_fullsized":"2024-06-28T16:12:50.312228+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":200,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":869,"num_read_kb":692,"num_write":351,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"260'292","reported_seq":946,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.802685+0000","last_change":"2024-06-28T15:00:15.423784+0000","last_active":"2024-06-28T16:14:41.802685+0000","last_peered":"2024-06-28T16:14:41.802685+0000","last_clean":"2024-06-28T16:14:41.802685+0000","last_became_active":"2024-06-28T15:00:15.420739+0000","last_became_peered":"2024-06-28T15:00:15.420739+0000","last_unstale":"2024-06-28T16:14:41.802685+0000","last_undegraded":"2024-06-28T16:14:41.802685+0000","last_fullsized":"2024-06-28T16:14:41.802685+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":200,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6022775,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":452,"num_read_kb":28797,"num_write":1102,"num_write_kb":30225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.100963+0000","last_change":"2024-06-28T15:00:19.403941+0000","last_active":"2024-06-28T16:12:50.100963+0000","last_peered":"2024-06-28T16:12:50.100963+0000","last_clean":"2024-06-28T16:12:50.100963+0000","last_became_active":"2024-06-28T15:00:19.400681+0000","last_became_peered":"2024-06-28T15:00:19.400681+0000","last_unstale":"2024-06-28T16:12:50.100963+0000","last_undegraded":"2024-06-28T16:12:50.100963+0000","last_fullsized":"2024-06-28T16:12:50.100963+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"260'324","reported_seq":949,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.645529+0000","last_change":"2024-06-28T15:00:21.391364+0000","last_active":"2024-06-28T16:14:33.645529+0000","last_peered":"2024-06-28T16:14:33.645529+0000","last_clean":"2024-06-28T16:14:33.645529+0000","last_became_active":"2024-06-28T15:00:21.390609+0000","last_became_peered":"2024-06-28T15:00:21.390609+0000","last_unstale":"2024-06-28T16:14:33.645529+0000","last_undegraded":"2024-06-28T16:14:33.645529+0000","last_fullsized":"2024-06-28T16:14:33.645529+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42630,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":530,"num_read_kb":1085,"num_write":406,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":494,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.116485+0000","last_change":"2024-06-28T15:00:23.408294+0000","last_active":"2024-06-28T16:12:55.116485+0000","last_peered":"2024-06-28T16:12:55.116485+0000","last_clean":"2024-06-28T16:12:55.116485+0000","last_became_active":"2024-06-28T15:00:23.407521+0000","last_became_peered":"2024-06-28T15:00:23.407521+0000","last_unstale":"2024-06-28T16:12:55.116485+0000","last_undegraded":"2024-06-28T16:12:55.116485+0000","last_fullsized":"2024-06-28T16:12:55.116485+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"260'2406","reported_seq":4281,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.648926+0000","last_change":"2024-06-28T16:14:46.649004+0000","last_active":"2024-06-28T16:14:46.648926+0000","last_peered":"2024-06-28T16:14:46.648926+0000","last_clean":"2024-06-28T16:14:46.648926+0000","last_became_active":"2024-06-28T15:00:25.440083+0000","last_became_peered":"2024-06-28T15:00:25.440083+0000","last_unstale":"2024-06-28T16:14:46.648926+0000","last_undegraded":"2024-06-28T16:14:46.648926+0000","last_fullsized":"2024-06-28T16:14:46.648926+0000","mapping_epoch":29,"log_start":"260'2300","ondisk_log_start":"260'2300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2406","last_scrub_stamp":"2024-06-28T16:14:46.648700+0000","last_deep_scrub":"260'2406","last_deep_scrub_stamp":"2024-06-28T16:14:46.648700+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.648700+0000","objects_scrubbed":4,"log_size":106,"log_dups_size":2300,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:06:59.187449+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5966,"num_read_kb":5973,"num_write":4800,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"260'4180","reported_seq":4833,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.506734+0000","last_change":"2024-06-28T16:14:58.506798+0000","last_active":"2024-06-28T16:14:58.506734+0000","last_peered":"2024-06-28T16:14:58.506734+0000","last_clean":"2024-06-28T16:14:58.506734+0000","last_became_active":"2024-06-28T15:00:17.390254+0000","last_became_peered":"2024-06-28T15:00:17.390254+0000","last_unstale":"2024-06-28T16:14:58.506734+0000","last_undegraded":"2024-06-28T16:14:58.506734+0000","last_fullsized":"2024-06-28T16:14:58.506734+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4180","last_scrub_stamp":"2024-06-28T16:14:58.506546+0000","last_deep_scrub":"260'4180","last_deep_scrub_stamp":"2024-06-28T16:14:58.506546+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.506546+0000","objects_scrubbed":1,"log_size":80,"log_dups_size":3000,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:17:13.473175+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14764,"num_read_kb":11968,"num_write":6274,"num_write_kb":3129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"260'343","reported_seq":1013,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.608567+0000","last_change":"2024-06-28T15:00:15.397404+0000","last_active":"2024-06-28T16:14:38.608567+0000","last_peered":"2024-06-28T16:14:38.608567+0000","last_clean":"2024-06-28T16:14:38.608567+0000","last_became_active":"2024-06-28T15:00:15.396724+0000","last_became_peered":"2024-06-28T15:00:15.396724+0000","last_unstale":"2024-06-28T16:14:38.608567+0000","last_undegraded":"2024-06-28T16:14:38.608567+0000","last_fullsized":"2024-06-28T16:14:38.608567+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262185,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":532,"num_read_kb":7737,"num_write":1235,"num_write_kb":8076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.312275+0000","last_change":"2024-06-28T15:00:19.390766+0000","last_active":"2024-06-28T16:12:50.312275+0000","last_peered":"2024-06-28T16:12:50.312275+0000","last_clean":"2024-06-28T16:12:50.312275+0000","last_became_active":"2024-06-28T15:00:19.390100+0000","last_became_peered":"2024-06-28T15:00:19.390100+0000","last_unstale":"2024-06-28T16:12:50.312275+0000","last_undegraded":"2024-06-28T16:12:50.312275+0000","last_fullsized":"2024-06-28T16:12:50.312275+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"260'127","reported_seq":675,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.898608+0000","last_change":"2024-06-28T16:14:57.898682+0000","last_active":"2024-06-28T16:14:57.898608+0000","last_peered":"2024-06-28T16:14:57.898608+0000","last_clean":"2024-06-28T16:14:57.898608+0000","last_became_active":"2024-06-28T15:00:21.374284+0000","last_became_peered":"2024-06-28T15:00:21.374284+0000","last_unstale":"2024-06-28T16:14:57.898608+0000","last_undegraded":"2024-06-28T16:14:57.898608+0000","last_fullsized":"2024-06-28T16:14:57.898608+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'127","last_scrub_stamp":"2024-06-28T16:14:57.898396+0000","last_deep_scrub":"260'127","last_deep_scrub_stamp":"2024-06-28T16:14:57.898396+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.898396+0000","objects_scrubbed":4,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:35:13.361283+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":127,"num_read_kb":120,"num_write":90,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.938305+0000","last_change":"2024-06-28T16:14:50.938366+0000","last_active":"2024-06-28T16:14:50.938305+0000","last_peered":"2024-06-28T16:14:50.938305+0000","last_clean":"2024-06-28T16:14:50.938305+0000","last_became_active":"2024-06-28T15:00:23.399147+0000","last_became_peered":"2024-06-28T15:00:23.399147+0000","last_unstale":"2024-06-28T16:14:50.938305+0000","last_undegraded":"2024-06-28T16:14:50.938305+0000","last_fullsized":"2024-06-28T16:14:50.938305+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:50.938091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:50.938091+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.938091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:31:29.176572+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"260'127","reported_seq":662,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.070275+0000","last_change":"2024-06-28T16:14:51.070346+0000","last_active":"2024-06-28T16:14:51.070275+0000","last_peered":"2024-06-28T16:14:51.070275+0000","last_clean":"2024-06-28T16:14:51.070275+0000","last_became_active":"2024-06-28T15:00:25.432898+0000","last_became_peered":"2024-06-28T15:00:25.432898+0000","last_unstale":"2024-06-28T16:14:51.070275+0000","last_undegraded":"2024-06-28T16:14:51.070275+0000","last_fullsized":"2024-06-28T16:14:51.070275+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'127","last_scrub_stamp":"2024-06-28T16:14:51.070081+0000","last_deep_scrub":"260'127","last_deep_scrub_stamp":"2024-06-28T16:14:51.070081+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.070081+0000","objects_scrubbed":4,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:05:59.673050+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":165,"num_write":302,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"260'155","reported_seq":777,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.582816+0000","last_change":"2024-06-28T15:00:17.384953+0000","last_active":"2024-06-28T16:12:50.582816+0000","last_peered":"2024-06-28T16:12:50.582816+0000","last_clean":"2024-06-28T16:12:50.582816+0000","last_became_active":"2024-06-28T15:00:17.384613+0000","last_became_peered":"2024-06-28T15:00:17.384613+0000","last_unstale":"2024-06-28T16:12:50.582816+0000","last_undegraded":"2024-06-28T16:12:50.582816+0000","last_fullsized":"2024-06-28T16:12:50.582816+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":630,"num_read_kb":508,"num_write":240,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"260'353","reported_seq":1003,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.802499+0000","last_change":"2024-06-28T15:00:15.429200+0000","last_active":"2024-06-28T16:14:41.802499+0000","last_peered":"2024-06-28T16:14:41.802499+0000","last_clean":"2024-06-28T16:14:41.802499+0000","last_became_active":"2024-06-28T15:00:15.428415+0000","last_became_peered":"2024-06-28T15:00:15.428415+0000","last_unstale":"2024-06-28T16:14:41.802499+0000","last_undegraded":"2024-06-28T16:14:41.802499+0000","last_fullsized":"2024-06-28T16:14:41.802499+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16055067,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":496,"num_read_kb":15197,"num_write":1261,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.312405+0000","last_change":"2024-06-28T15:00:19.373751+0000","last_active":"2024-06-28T16:12:50.312405+0000","last_peered":"2024-06-28T16:12:50.312405+0000","last_clean":"2024-06-28T16:12:50.312405+0000","last_became_active":"2024-06-28T15:00:19.372840+0000","last_became_peered":"2024-06-28T15:00:19.372840+0000","last_unstale":"2024-06-28T16:12:50.312405+0000","last_undegraded":"2024-06-28T16:12:50.312405+0000","last_fullsized":"2024-06-28T16:12:50.312405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"260'128","reported_seq":700,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.103449+0000","last_change":"2024-06-28T16:14:45.103625+0000","last_active":"2024-06-28T16:14:45.103449+0000","last_peered":"2024-06-28T16:14:45.103449+0000","last_clean":"2024-06-28T16:14:45.103449+0000","last_became_active":"2024-06-28T15:00:21.377203+0000","last_became_peered":"2024-06-28T15:00:21.377203+0000","last_unstale":"2024-06-28T16:14:45.103449+0000","last_undegraded":"2024-06-28T16:14:45.103449+0000","last_fullsized":"2024-06-28T16:14:45.103449+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'128","last_scrub_stamp":"2024-06-28T16:14:45.103227+0000","last_deep_scrub":"260'128","last_deep_scrub_stamp":"2024-06-28T16:14:45.103227+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.103227+0000","objects_scrubbed":5,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:36:43.901132+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"260'522","reported_seq":4628,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.765082+0000","last_change":"2024-06-28T15:00:23.418328+0000","last_active":"2024-06-28T16:14:34.765082+0000","last_peered":"2024-06-28T16:14:34.765082+0000","last_clean":"2024-06-28T16:14:34.765082+0000","last_became_active":"2024-06-28T15:00:23.417490+0000","last_became_peered":"2024-06-28T15:00:23.417490+0000","last_unstale":"2024-06-28T16:14:34.765082+0000","last_undegraded":"2024-06-28T16:14:34.765082+0000","last_fullsized":"2024-06-28T16:14:34.765082+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":500,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"260'204","reported_seq":1022,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.919176+0000","last_change":"2024-06-28T16:14:42.919351+0000","last_active":"2024-06-28T16:14:42.919176+0000","last_peered":"2024-06-28T16:14:42.919176+0000","last_clean":"2024-06-28T16:14:42.919176+0000","last_became_active":"2024-06-28T15:00:25.555670+0000","last_became_peered":"2024-06-28T15:00:25.555670+0000","last_unstale":"2024-06-28T16:14:42.919176+0000","last_undegraded":"2024-06-28T16:14:42.919176+0000","last_fullsized":"2024-06-28T16:14:42.919176+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'204","last_scrub_stamp":"2024-06-28T16:14:42.918953+0000","last_deep_scrub":"260'204","last_deep_scrub_stamp":"2024-06-28T16:14:42.918953+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.918953+0000","objects_scrubbed":3,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:34:15.091552+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":557,"num_read_kb":555,"num_write":448,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"260'84","reported_seq":672,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.312378+0000","last_change":"2024-06-28T15:00:17.390503+0000","last_active":"2024-06-28T16:12:50.312378+0000","last_peered":"2024-06-28T16:12:50.312378+0000","last_clean":"2024-06-28T16:12:50.312378+0000","last_became_active":"2024-06-28T15:00:17.389986+0000","last_became_peered":"2024-06-28T15:00:17.389986+0000","last_unstale":"2024-06-28T16:12:50.312378+0000","last_undegraded":"2024-06-28T16:12:50.312378+0000","last_fullsized":"2024-06-28T16:12:50.312378+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":369,"num_read_kb":305,"num_write":130,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"260'288","reported_seq":924,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.331723+0000","last_change":"2024-06-28T15:00:15.413174+0000","last_active":"2024-06-28T16:14:41.331723+0000","last_peered":"2024-06-28T16:14:41.331723+0000","last_clean":"2024-06-28T16:14:41.331723+0000","last_became_active":"2024-06-28T15:00:15.411380+0000","last_became_peered":"2024-06-28T15:00:15.411380+0000","last_unstale":"2024-06-28T16:14:41.331723+0000","last_undegraded":"2024-06-28T16:14:41.331723+0000","last_fullsized":"2024-06-28T16:14:41.331723+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262217,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":24351,"num_write":1142,"num_write_kb":34218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"31'3","reported_seq":831,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.180682+0000","last_change":"2024-06-28T15:00:19.404519+0000","last_active":"2024-06-28T16:14:32.180682+0000","last_peered":"2024-06-28T16:14:32.180682+0000","last_clean":"2024-06-28T16:14:32.180682+0000","last_became_active":"2024-06-28T15:00:19.403623+0000","last_became_peered":"2024-06-28T15:00:19.403623+0000","last_unstale":"2024-06-28T16:14:32.180682+0000","last_undegraded":"2024-06-28T16:14:32.180682+0000","last_fullsized":"2024-06-28T16:14:32.180682+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":496,"num_read_kb":494,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"260'251","reported_seq":834,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.273384+0000","last_change":"2024-06-28T15:00:21.485248+0000","last_active":"2024-06-28T16:14:33.273384+0000","last_peered":"2024-06-28T16:14:33.273384+0000","last_clean":"2024-06-28T16:14:33.273384+0000","last_became_active":"2024-06-28T15:00:21.483633+0000","last_became_peered":"2024-06-28T15:00:21.483633+0000","last_unstale":"2024-06-28T16:14:33.273384+0000","last_undegraded":"2024-06-28T16:14:33.273384+0000","last_fullsized":"2024-06-28T16:14:33.273384+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8831,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":873,"num_write":310,"num_write_kb":224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"260'537","reported_seq":2240,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.766777+0000","last_change":"2024-06-28T15:00:23.397769+0000","last_active":"2024-06-28T16:14:34.766777+0000","last_peered":"2024-06-28T16:14:34.766777+0000","last_clean":"2024-06-28T16:14:34.766777+0000","last_became_active":"2024-06-28T15:00:23.396923+0000","last_became_peered":"2024-06-28T15:00:23.396923+0000","last_unstale":"2024-06-28T16:14:34.766777+0000","last_undegraded":"2024-06-28T16:14:34.766777+0000","last_fullsized":"2024-06-28T16:14:34.766777+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"260'415","reported_seq":1094,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.116205+0000","last_change":"2024-06-28T15:00:17.378608+0000","last_active":"2024-06-28T16:14:00.116205+0000","last_peered":"2024-06-28T16:14:00.116205+0000","last_clean":"2024-06-28T16:14:00.116205+0000","last_became_active":"2024-06-28T15:00:17.376864+0000","last_became_peered":"2024-06-28T15:00:17.376864+0000","last_unstale":"2024-06-28T16:14:00.116205+0000","last_undegraded":"2024-06-28T16:14:00.116205+0000","last_fullsized":"2024-06-28T16:14:00.116205+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":400,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1583,"num_read_kb":1214,"num_write":634,"num_write_kb":357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"260'320","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.802495+0000","last_change":"2024-06-28T15:00:15.423161+0000","last_active":"2024-06-28T16:14:41.802495+0000","last_peered":"2024-06-28T16:14:41.802495+0000","last_clean":"2024-06-28T16:14:41.802495+0000","last_became_active":"2024-06-28T15:00:15.415163+0000","last_became_peered":"2024-06-28T15:00:15.415163+0000","last_unstale":"2024-06-28T16:14:41.802495+0000","last_undegraded":"2024-06-28T16:14:41.802495+0000","last_fullsized":"2024-06-28T16:14:41.802495+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470118,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":518,"num_read_kb":26822,"num_write":1200,"num_write_kb":41274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.131452+0000","last_change":"2024-06-28T15:00:19.390703+0000","last_active":"2024-06-28T16:14:00.131452+0000","last_peered":"2024-06-28T16:14:00.131452+0000","last_clean":"2024-06-28T16:14:00.131452+0000","last_became_active":"2024-06-28T15:00:19.389624+0000","last_became_peered":"2024-06-28T15:00:19.389624+0000","last_unstale":"2024-06-28T16:14:00.131452+0000","last_undegraded":"2024-06-28T16:14:00.131452+0000","last_fullsized":"2024-06-28T16:14:00.131452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"260'390","reported_seq":1043,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.597603+0000","last_change":"2024-06-28T16:14:54.597760+0000","last_active":"2024-06-28T16:14:54.597603+0000","last_peered":"2024-06-28T16:14:54.597603+0000","last_clean":"2024-06-28T16:14:54.597603+0000","last_became_active":"2024-06-28T15:00:21.481315+0000","last_became_peered":"2024-06-28T15:00:21.481315+0000","last_unstale":"2024-06-28T16:14:54.597603+0000","last_undegraded":"2024-06-28T16:14:54.597603+0000","last_fullsized":"2024-06-28T16:14:54.597603+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'390","last_scrub_stamp":"2024-06-28T16:14:54.597422+0000","last_deep_scrub":"260'390","last_deep_scrub_stamp":"2024-06-28T16:14:54.597422+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.597422+0000","objects_scrubbed":9,"log_size":90,"log_dups_size":300,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:52:31.536722+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46076,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":638,"num_read_kb":1277,"num_write":490,"num_write_kb":298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.074002+0000","last_change":"2024-06-28T15:00:23.414142+0000","last_active":"2024-06-28T16:14:39.074002+0000","last_peered":"2024-06-28T16:14:39.074002+0000","last_clean":"2024-06-28T16:14:39.074002+0000","last_became_active":"2024-06-28T15:00:23.413132+0000","last_became_peered":"2024-06-28T15:00:23.413132+0000","last_unstale":"2024-06-28T16:14:39.074002+0000","last_undegraded":"2024-06-28T16:14:39.074002+0000","last_fullsized":"2024-06-28T16:14:39.074002+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"260'558","reported_seq":1385,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.684916+0000","last_change":"2024-06-28T16:14:45.685006+0000","last_active":"2024-06-28T16:14:45.684916+0000","last_peered":"2024-06-28T16:14:45.684916+0000","last_clean":"2024-06-28T16:14:45.684916+0000","last_became_active":"2024-06-28T15:00:25.441335+0000","last_became_peered":"2024-06-28T15:00:25.441335+0000","last_unstale":"2024-06-28T16:14:45.684916+0000","last_undegraded":"2024-06-28T16:14:45.684916+0000","last_fullsized":"2024-06-28T16:14:45.684916+0000","mapping_epoch":29,"log_start":"260'500","ondisk_log_start":"260'500","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'558","last_scrub_stamp":"2024-06-28T16:14:45.684713+0000","last_deep_scrub":"260'558","last_deep_scrub_stamp":"2024-06-28T16:14:45.684713+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.684713+0000","objects_scrubbed":3,"log_size":58,"log_dups_size":500,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:58:41.400605+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1277,"num_read_kb":1273,"num_write":1164,"num_write_kb":394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"260'52","reported_seq":632,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.122926+0000","last_change":"2024-06-28T15:00:17.409603+0000","last_active":"2024-06-28T16:14:00.122926+0000","last_peered":"2024-06-28T16:14:00.122926+0000","last_clean":"2024-06-28T16:14:00.122926+0000","last_became_active":"2024-06-28T15:00:17.408992+0000","last_became_peered":"2024-06-28T15:00:17.408992+0000","last_unstale":"2024-06-28T16:14:00.122926+0000","last_undegraded":"2024-06-28T16:14:00.122926+0000","last_fullsized":"2024-06-28T16:14:00.122926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":226,"num_read_kb":196,"num_write":79,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"260'326","reported_seq":946,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.675598+0000","last_change":"2024-06-28T15:00:15.402904+0000","last_active":"2024-06-28T16:14:34.675598+0000","last_peered":"2024-06-28T16:14:34.675598+0000","last_clean":"2024-06-28T16:14:34.675598+0000","last_became_active":"2024-06-28T15:00:15.402188+0000","last_became_peered":"2024-06-28T15:00:15.402188+0000","last_unstale":"2024-06-28T16:14:34.675598+0000","last_undegraded":"2024-06-28T16:14:34.675598+0000","last_fullsized":"2024-06-28T16:14:34.675598+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170347,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":7037,"num_write":1119,"num_write_kb":28395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":523,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.240452+0000","last_change":"2024-06-28T16:14:55.240505+0000","last_active":"2024-06-28T16:14:55.240452+0000","last_peered":"2024-06-28T16:14:55.240452+0000","last_clean":"2024-06-28T16:14:55.240452+0000","last_became_active":"2024-06-28T15:00:19.383002+0000","last_became_peered":"2024-06-28T15:00:19.383002+0000","last_unstale":"2024-06-28T16:14:55.240452+0000","last_undegraded":"2024-06-28T16:14:55.240452+0000","last_fullsized":"2024-06-28T16:14:55.240452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:55.240251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:55.240251+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.240251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:09:48.072213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"260'288","reported_seq":880,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.971707+0000","last_change":"2024-06-28T15:00:21.395366+0000","last_active":"2024-06-28T16:14:39.971707+0000","last_peered":"2024-06-28T16:14:39.971707+0000","last_clean":"2024-06-28T16:14:39.971707+0000","last_became_active":"2024-06-28T15:00:21.394580+0000","last_became_peered":"2024-06-28T15:00:21.394580+0000","last_unstale":"2024-06-28T16:14:39.971707+0000","last_undegraded":"2024-06-28T16:14:39.971707+0000","last_fullsized":"2024-06-28T16:14:39.971707+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":455,"num_read_kb":1052,"num_write":364,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":511,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.189470+0000","last_change":"2024-06-28T16:14:59.189522+0000","last_active":"2024-06-28T16:14:59.189470+0000","last_peered":"2024-06-28T16:14:59.189470+0000","last_clean":"2024-06-28T16:14:59.189470+0000","last_became_active":"2024-06-28T15:00:23.422567+0000","last_became_peered":"2024-06-28T15:00:23.422567+0000","last_unstale":"2024-06-28T16:14:59.189470+0000","last_undegraded":"2024-06-28T16:14:59.189470+0000","last_fullsized":"2024-06-28T16:14:59.189470+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:59.189280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:59.189280+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.189280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:25:01.077193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"260'96","reported_seq":590,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.117836+0000","last_change":"2024-06-28T15:00:25.426829+0000","last_active":"2024-06-28T16:13:40.117836+0000","last_peered":"2024-06-28T16:13:40.117836+0000","last_clean":"2024-06-28T16:13:40.117836+0000","last_became_active":"2024-06-28T15:00:25.426151+0000","last_became_peered":"2024-06-28T15:00:25.426151+0000","last_unstale":"2024-06-28T16:13:40.117836+0000","last_undegraded":"2024-06-28T16:13:40.117836+0000","last_fullsized":"2024-06-28T16:13:40.117836+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":231,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"260'173","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.436343+0000","last_change":"2024-06-28T16:14:57.436511+0000","last_active":"2024-06-28T16:14:57.436343+0000","last_peered":"2024-06-28T16:14:57.436343+0000","last_clean":"2024-06-28T16:14:57.436343+0000","last_became_active":"2024-06-28T15:00:17.407031+0000","last_became_peered":"2024-06-28T15:00:17.407031+0000","last_unstale":"2024-06-28T16:14:57.436343+0000","last_undegraded":"2024-06-28T16:14:57.436343+0000","last_fullsized":"2024-06-28T16:14:57.436343+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'173","last_scrub_stamp":"2024-06-28T16:14:57.436114+0000","last_deep_scrub":"260'173","last_deep_scrub_stamp":"2024-06-28T16:14:57.436114+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.436114+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:11:15.521784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":538,"num_write":274,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"260'353","reported_seq":1039,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.716076+0000","last_change":"2024-06-28T15:00:15.428903+0000","last_active":"2024-06-28T16:14:34.716076+0000","last_peered":"2024-06-28T16:14:34.716076+0000","last_clean":"2024-06-28T16:14:34.716076+0000","last_became_active":"2024-06-28T15:00:15.427408+0000","last_became_peered":"2024-06-28T15:00:15.427408+0000","last_unstale":"2024-06-28T16:14:34.716076+0000","last_undegraded":"2024-06-28T16:14:34.716076+0000","last_fullsized":"2024-06-28T16:14:34.716076+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":574,"num_read_kb":16939,"num_write":1376,"num_write_kb":32636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":522,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.193279+0000","last_change":"2024-06-28T16:14:53.193339+0000","last_active":"2024-06-28T16:14:53.193279+0000","last_peered":"2024-06-28T16:14:53.193279+0000","last_clean":"2024-06-28T16:14:53.193279+0000","last_became_active":"2024-06-28T15:00:19.384172+0000","last_became_peered":"2024-06-28T15:00:19.384172+0000","last_unstale":"2024-06-28T16:14:53.193279+0000","last_undegraded":"2024-06-28T16:14:53.193279+0000","last_fullsized":"2024-06-28T16:14:53.193279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.193045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.193045+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.193045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:10:37.938602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"260'314","reported_seq":926,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.099479+0000","last_change":"2024-06-28T15:00:21.394527+0000","last_active":"2024-06-28T16:14:34.099479+0000","last_peered":"2024-06-28T16:14:34.099479+0000","last_clean":"2024-06-28T16:14:34.099479+0000","last_became_active":"2024-06-28T15:00:21.393608+0000","last_became_peered":"2024-06-28T15:00:21.393608+0000","last_unstale":"2024-06-28T16:14:34.099479+0000","last_undegraded":"2024-06-28T16:14:34.099479+0000","last_fullsized":"2024-06-28T16:14:34.099479+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":300,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42645,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":526,"num_read_kb":1122,"num_write":410,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"260'534","reported_seq":2275,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.097241+0000","last_change":"2024-06-28T16:14:57.097387+0000","last_active":"2024-06-28T16:14:57.097241+0000","last_peered":"2024-06-28T16:14:57.097241+0000","last_clean":"2024-06-28T16:14:57.097241+0000","last_became_active":"2024-06-28T15:00:23.413550+0000","last_became_peered":"2024-06-28T15:00:23.413550+0000","last_unstale":"2024-06-28T16:14:57.097241+0000","last_undegraded":"2024-06-28T16:14:57.097241+0000","last_fullsized":"2024-06-28T16:14:57.097241+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'534","last_scrub_stamp":"2024-06-28T16:14:57.096961+0000","last_deep_scrub":"260'534","last_deep_scrub_stamp":"2024-06-28T16:14:57.096961+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.096961+0000","objects_scrubbed":1,"log_size":34,"log_dups_size":500,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:11:19.198302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"260'147","reported_seq":684,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.064684+0000","last_change":"2024-06-28T16:14:52.064763+0000","last_active":"2024-06-28T16:14:52.064684+0000","last_peered":"2024-06-28T16:14:52.064684+0000","last_clean":"2024-06-28T16:14:52.064684+0000","last_became_active":"2024-06-28T15:00:25.431835+0000","last_became_peered":"2024-06-28T15:00:25.431835+0000","last_unstale":"2024-06-28T16:14:52.064684+0000","last_undegraded":"2024-06-28T16:14:52.064684+0000","last_fullsized":"2024-06-28T16:14:52.064684+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'147","last_scrub_stamp":"2024-06-28T16:14:52.064509+0000","last_deep_scrub":"260'147","last_deep_scrub_stamp":"2024-06-28T16:14:52.064509+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.064509+0000","objects_scrubbed":3,"log_size":47,"log_dups_size":100,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:45:10.678734+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1247,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":164,"num_write":265,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"260'4158","reported_seq":4765,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.608300+0000","last_change":"2024-06-28T15:00:17.400121+0000","last_active":"2024-06-28T16:14:35.608300+0000","last_peered":"2024-06-28T16:14:35.608300+0000","last_clean":"2024-06-28T16:14:35.608300+0000","last_became_active":"2024-06-28T15:00:17.399799+0000","last_became_peered":"2024-06-28T15:00:17.399799+0000","last_unstale":"2024-06-28T16:14:35.608300+0000","last_undegraded":"2024-06-28T16:14:35.608300+0000","last_fullsized":"2024-06-28T16:14:35.608300+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":3000,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14678,"num_read_kb":11882,"num_write":6254,"num_write_kb":3121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"260'337","reported_seq":983,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.737490+0000","last_change":"2024-06-28T15:00:15.426615+0000","last_active":"2024-06-28T16:14:34.737490+0000","last_peered":"2024-06-28T16:14:34.737490+0000","last_clean":"2024-06-28T16:14:34.737490+0000","last_became_active":"2024-06-28T15:00:15.425690+0000","last_became_peered":"2024-06-28T15:00:15.425690+0000","last_unstale":"2024-06-28T16:14:34.737490+0000","last_undegraded":"2024-06-28T16:14:34.737490+0000","last_fullsized":"2024-06-28T16:14:34.737490+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079931,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":508,"num_read_kb":13664,"num_write":1274,"num_write_kb":26574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,1],"acting":[5,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":523,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.205322+0000","last_change":"2024-06-28T16:14:51.205380+0000","last_active":"2024-06-28T16:14:51.205322+0000","last_peered":"2024-06-28T16:14:51.205322+0000","last_clean":"2024-06-28T16:14:51.205322+0000","last_became_active":"2024-06-28T15:00:19.385189+0000","last_became_peered":"2024-06-28T15:00:19.385189+0000","last_unstale":"2024-06-28T16:14:51.205322+0000","last_undegraded":"2024-06-28T16:14:51.205322+0000","last_fullsized":"2024-06-28T16:14:51.205322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.205122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.205122+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.205122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:14:01.011784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"260'487","reported_seq":1144,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.253332+0000","last_change":"2024-06-28T15:00:21.400448+0000","last_active":"2024-06-28T16:14:34.253332+0000","last_peered":"2024-06-28T16:14:34.253332+0000","last_clean":"2024-06-28T16:14:34.253332+0000","last_became_active":"2024-06-28T15:00:21.397397+0000","last_became_peered":"2024-06-28T15:00:21.397397+0000","last_unstale":"2024-06-28T16:14:34.253332+0000","last_undegraded":"2024-06-28T16:14:34.253332+0000","last_fullsized":"2024-06-28T16:14:34.253332+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":400,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92449,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":856,"num_read_kb":2091,"num_write":698,"num_write_kb":574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.129159+0000","last_change":"2024-06-28T16:14:40.129181+0000","last_active":"2024-06-28T16:14:40.129159+0000","last_peered":"2024-06-28T16:14:40.129159+0000","last_clean":"2024-06-28T16:14:40.129159+0000","last_became_active":"2024-06-28T15:00:23.416263+0000","last_became_peered":"2024-06-28T15:00:23.416263+0000","last_unstale":"2024-06-28T16:14:40.129159+0000","last_undegraded":"2024-06-28T16:14:40.129159+0000","last_fullsized":"2024-06-28T16:14:40.129159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:40.129070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:40.129070+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.129070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:20:56.813386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"260'113","reported_seq":632,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:05.132122+0000","last_change":"2024-06-28T15:00:25.423425+0000","last_active":"2024-06-28T16:14:05.132122+0000","last_peered":"2024-06-28T16:14:05.132122+0000","last_clean":"2024-06-28T16:14:05.132122+0000","last_became_active":"2024-06-28T15:00:25.422519+0000","last_became_peered":"2024-06-28T15:00:25.422519+0000","last_unstale":"2024-06-28T16:14:05.132122+0000","last_undegraded":"2024-06-28T16:14:05.132122+0000","last_fullsized":"2024-06-28T16:14:05.132122+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":118,"num_read_kb":116,"num_write":248,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"260'128","reported_seq":778,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.130615+0000","last_change":"2024-06-28T15:00:17.409415+0000","last_active":"2024-06-28T16:14:35.130615+0000","last_peered":"2024-06-28T16:14:35.130615+0000","last_clean":"2024-06-28T16:14:35.130615+0000","last_became_active":"2024-06-28T15:00:17.404204+0000","last_became_peered":"2024-06-28T15:00:17.404204+0000","last_unstale":"2024-06-28T16:14:35.130615+0000","last_undegraded":"2024-06-28T16:14:35.130615+0000","last_fullsized":"2024-06-28T16:14:35.130615+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":562,"num_read_kb":476,"num_write":174,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"260'352","reported_seq":1060,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.731157+0000","last_change":"2024-06-28T15:00:15.402917+0000","last_active":"2024-06-28T16:14:34.731157+0000","last_peered":"2024-06-28T16:14:34.731157+0000","last_clean":"2024-06-28T16:14:34.731157+0000","last_became_active":"2024-06-28T15:00:15.402041+0000","last_became_peered":"2024-06-28T15:00:15.402041+0000","last_unstale":"2024-06-28T16:14:34.731157+0000","last_undegraded":"2024-06-28T16:14:34.731157+0000","last_fullsized":"2024-06-28T16:14:34.731157+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":603,"num_read_kb":26410,"num_write":1369,"num_write_kb":26391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":522,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.209018+0000","last_change":"2024-06-28T16:14:49.209075+0000","last_active":"2024-06-28T16:14:49.209018+0000","last_peered":"2024-06-28T16:14:49.209018+0000","last_clean":"2024-06-28T16:14:49.209018+0000","last_became_active":"2024-06-28T15:00:19.392773+0000","last_became_peered":"2024-06-28T15:00:19.392773+0000","last_unstale":"2024-06-28T16:14:49.209018+0000","last_undegraded":"2024-06-28T16:14:49.209018+0000","last_fullsized":"2024-06-28T16:14:49.209018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.208819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.208819+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.208819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:49:11.047171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"260'306","reported_seq":911,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.988613+0000","last_change":"2024-06-28T16:14:53.988691+0000","last_active":"2024-06-28T16:14:53.988613+0000","last_peered":"2024-06-28T16:14:53.988613+0000","last_clean":"2024-06-28T16:14:53.988613+0000","last_became_active":"2024-06-28T15:00:21.397842+0000","last_became_peered":"2024-06-28T15:00:21.397842+0000","last_unstale":"2024-06-28T16:14:53.988613+0000","last_undegraded":"2024-06-28T16:14:53.988613+0000","last_fullsized":"2024-06-28T16:14:53.988613+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'306","last_scrub_stamp":"2024-06-28T16:14:53.988406+0000","last_deep_scrub":"260'306","last_deep_scrub_stamp":"2024-06-28T16:14:53.988406+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.988406+0000","objects_scrubbed":6,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:16:38.325337+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44037,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":498,"num_read_kb":1063,"num_write":434,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15964,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":512,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.186008+0000","last_change":"2024-06-28T16:15:00.186120+0000","last_active":"2024-06-28T16:15:00.186008+0000","last_peered":"2024-06-28T16:15:00.186008+0000","last_clean":"2024-06-28T16:15:00.186008+0000","last_became_active":"2024-06-28T15:00:23.406122+0000","last_became_peered":"2024-06-28T15:00:23.406122+0000","last_unstale":"2024-06-28T16:15:00.186008+0000","last_undegraded":"2024-06-28T16:15:00.186008+0000","last_fullsized":"2024-06-28T16:15:00.186008+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:00.185778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:00.185778+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.185778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:16:27.337969+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"260'91","reported_seq":604,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.145305+0000","last_change":"2024-06-28T16:14:54.145374+0000","last_active":"2024-06-28T16:14:54.145305+0000","last_peered":"2024-06-28T16:14:54.145305+0000","last_clean":"2024-06-28T16:14:54.145305+0000","last_became_active":"2024-06-28T15:00:25.439346+0000","last_became_peered":"2024-06-28T15:00:25.439346+0000","last_unstale":"2024-06-28T16:14:54.145305+0000","last_undegraded":"2024-06-28T16:14:54.145305+0000","last_fullsized":"2024-06-28T16:14:54.145305+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'91","last_scrub_stamp":"2024-06-28T16:14:54.145105+0000","last_deep_scrub":"260'91","last_deep_scrub_stamp":"2024-06-28T16:14:54.145105+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.145105+0000","objects_scrubbed":2,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:54:30.494466+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":207,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"260'171","reported_seq":760,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.130652+0000","last_change":"2024-06-28T15:00:17.396835+0000","last_active":"2024-06-28T16:14:35.130652+0000","last_peered":"2024-06-28T16:14:35.130652+0000","last_clean":"2024-06-28T16:14:35.130652+0000","last_became_active":"2024-06-28T15:00:17.396048+0000","last_became_peered":"2024-06-28T15:00:17.396048+0000","last_unstale":"2024-06-28T16:14:35.130652+0000","last_undegraded":"2024-06-28T16:14:35.130652+0000","last_fullsized":"2024-06-28T16:14:35.130652+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":659,"num_read_kb":531,"num_write":250,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"260'376","reported_seq":1108,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.730081+0000","last_change":"2024-06-28T15:00:15.398260+0000","last_active":"2024-06-28T16:14:34.730081+0000","last_peered":"2024-06-28T16:14:34.730081+0000","last_clean":"2024-06-28T16:14:34.730081+0000","last_became_active":"2024-06-28T15:00:15.397256+0000","last_became_peered":"2024-06-28T15:00:15.397256+0000","last_unstale":"2024-06-28T16:14:34.730081+0000","last_undegraded":"2024-06-28T16:14:34.730081+0000","last_fullsized":"2024-06-28T16:14:34.730081+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":300,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2674362,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":727,"num_read_kb":7345,"num_write":1214,"num_write_kb":31713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":519,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.945272+0000","last_change":"2024-06-28T16:14:53.945393+0000","last_active":"2024-06-28T16:14:53.945272+0000","last_peered":"2024-06-28T16:14:53.945272+0000","last_clean":"2024-06-28T16:14:53.945272+0000","last_became_active":"2024-06-28T15:00:19.379281+0000","last_became_peered":"2024-06-28T15:00:19.379281+0000","last_unstale":"2024-06-28T16:14:53.945272+0000","last_undegraded":"2024-06-28T16:14:53.945272+0000","last_fullsized":"2024-06-28T16:14:53.945272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.945027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.945027+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.945027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:26:10.510879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"260'356","reported_seq":984,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.142072+0000","last_change":"2024-06-28T16:14:43.142072+0000","last_active":"2024-06-28T16:14:43.142072+0000","last_peered":"2024-06-28T16:14:43.142072+0000","last_clean":"2024-06-28T16:14:43.142072+0000","last_became_active":"2024-06-28T15:00:21.395892+0000","last_became_peered":"2024-06-28T15:00:21.395892+0000","last_unstale":"2024-06-28T16:14:43.142072+0000","last_undegraded":"2024-06-28T16:14:43.142072+0000","last_fullsized":"2024-06-28T16:14:43.142072+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'356","last_scrub_stamp":"2024-06-28T16:14:43.141802+0000","last_deep_scrub":"260'356","last_deep_scrub_stamp":"2024-06-28T16:14:43.141802+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.141802+0000","objects_scrubbed":7,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:06:31.916429+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48931,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":579,"num_read_kb":1282,"num_write":462,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"260'501","reported_seq":2313,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.168638+0000","last_change":"2024-06-28T16:14:41.168660+0000","last_active":"2024-06-28T16:14:41.168638+0000","last_peered":"2024-06-28T16:14:41.168638+0000","last_clean":"2024-06-28T16:14:41.168638+0000","last_became_active":"2024-06-28T15:00:23.415437+0000","last_became_peered":"2024-06-28T15:00:23.415437+0000","last_unstale":"2024-06-28T16:14:41.168638+0000","last_undegraded":"2024-06-28T16:14:41.168638+0000","last_fullsized":"2024-06-28T16:14:41.168638+0000","mapping_epoch":27,"log_start":"257'400","ondisk_log_start":"257'400","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'501","last_scrub_stamp":"2024-06-28T16:14:41.168573+0000","last_deep_scrub":"260'501","last_deep_scrub_stamp":"2024-06-28T16:14:41.168573+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.168573+0000","objects_scrubbed":1,"log_size":101,"log_dups_size":400,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:22:09.865296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"260'102","reported_seq":620,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.002911+0000","last_change":"2024-06-28T16:14:50.002982+0000","last_active":"2024-06-28T16:14:50.002911+0000","last_peered":"2024-06-28T16:14:50.002911+0000","last_clean":"2024-06-28T16:14:50.002911+0000","last_became_active":"2024-06-28T15:00:25.425135+0000","last_became_peered":"2024-06-28T15:00:25.425135+0000","last_unstale":"2024-06-28T16:14:50.002911+0000","last_undegraded":"2024-06-28T16:14:50.002911+0000","last_fullsized":"2024-06-28T16:14:50.002911+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'102","last_scrub_stamp":"2024-06-28T16:14:50.002732+0000","last_deep_scrub":"260'102","last_deep_scrub_stamp":"2024-06-28T16:14:50.002732+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.002732+0000","objects_scrubbed":3,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:56:55.357278+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":245,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"260'160","reported_seq":744,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.130711+0000","last_change":"2024-06-28T15:00:17.403493+0000","last_active":"2024-06-28T16:14:35.130711+0000","last_peered":"2024-06-28T16:14:35.130711+0000","last_clean":"2024-06-28T16:14:35.130711+0000","last_became_active":"2024-06-28T15:00:17.402024+0000","last_became_peered":"2024-06-28T15:00:17.402024+0000","last_unstale":"2024-06-28T16:14:35.130711+0000","last_undegraded":"2024-06-28T16:14:35.130711+0000","last_fullsized":"2024-06-28T16:14:35.130711+0000","mapping_epoch":21,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":100,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":645,"num_read_kb":505,"num_write":259,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"260'393","reported_seq":1068,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.756173+0000","last_change":"2024-06-28T15:00:15.389691+0000","last_active":"2024-06-28T16:14:34.756173+0000","last_peered":"2024-06-28T16:14:34.756173+0000","last_clean":"2024-06-28T16:14:34.756173+0000","last_became_active":"2024-06-28T15:00:15.388365+0000","last_became_peered":"2024-06-28T15:00:15.388365+0000","last_unstale":"2024-06-28T16:14:34.756173+0000","last_undegraded":"2024-06-28T16:14:34.756173+0000","last_fullsized":"2024-06-28T16:14:34.756173+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":300,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1604260,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":661,"num_read_kb":7226,"num_write":1423,"num_write_kb":30573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"24'2","reported_seq":1156,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.877355+0000","last_change":"2024-06-28T15:00:19.403973+0000","last_active":"2024-06-28T16:14:40.877355+0000","last_peered":"2024-06-28T16:14:40.877355+0000","last_clean":"2024-06-28T16:14:40.877355+0000","last_became_active":"2024-06-28T15:00:19.397863+0000","last_became_peered":"2024-06-28T15:00:19.397863+0000","last_unstale":"2024-06-28T16:14:40.877355+0000","last_undegraded":"2024-06-28T16:14:40.877355+0000","last_fullsized":"2024-06-28T16:14:40.877355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":648,"num_read_kb":648,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"260'113","reported_seq":659,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.971629+0000","last_change":"2024-06-28T15:00:21.484486+0000","last_active":"2024-06-28T16:14:39.971629+0000","last_peered":"2024-06-28T16:14:39.971629+0000","last_clean":"2024-06-28T16:14:39.971629+0000","last_became_active":"2024-06-28T15:00:21.483679+0000","last_became_peered":"2024-06-28T15:00:21.483679+0000","last_unstale":"2024-06-28T16:14:39.971629+0000","last_undegraded":"2024-06-28T16:14:39.971629+0000","last_fullsized":"2024-06-28T16:14:39.971629+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":100,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"260'515","reported_seq":4272,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.451884+0000","last_change":"2024-06-28T16:14:58.451949+0000","last_active":"2024-06-28T16:14:58.451884+0000","last_peered":"2024-06-28T16:14:58.451884+0000","last_clean":"2024-06-28T16:14:58.451884+0000","last_became_active":"2024-06-28T15:00:23.419182+0000","last_became_peered":"2024-06-28T15:00:23.419182+0000","last_unstale":"2024-06-28T16:14:58.451884+0000","last_undegraded":"2024-06-28T16:14:58.451884+0000","last_fullsized":"2024-06-28T16:14:58.451884+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'515","last_scrub_stamp":"2024-06-28T16:14:58.451684+0000","last_deep_scrub":"260'515","last_deep_scrub_stamp":"2024-06-28T16:14:58.451684+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.451684+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:15.191208+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"260'163","reported_seq":685,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.941396+0000","last_change":"2024-06-28T16:14:46.941473+0000","last_active":"2024-06-28T16:14:46.941396+0000","last_peered":"2024-06-28T16:14:46.941396+0000","last_clean":"2024-06-28T16:14:46.941396+0000","last_became_active":"2024-06-28T15:00:25.425690+0000","last_became_peered":"2024-06-28T15:00:25.425690+0000","last_unstale":"2024-06-28T16:14:46.941396+0000","last_undegraded":"2024-06-28T16:14:46.941396+0000","last_fullsized":"2024-06-28T16:14:46.941396+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'163","last_scrub_stamp":"2024-06-28T16:14:46.941194+0000","last_deep_scrub":"260'163","last_deep_scrub_stamp":"2024-06-28T16:14:46.941194+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.941194+0000","objects_scrubbed":4,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:28:38.329545+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":152,"num_write":381,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"260'74","reported_seq":628,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.650196+0000","last_change":"2024-06-28T16:14:42.650256+0000","last_active":"2024-06-28T16:14:42.650196+0000","last_peered":"2024-06-28T16:14:42.650196+0000","last_clean":"2024-06-28T16:14:42.650196+0000","last_became_active":"2024-06-28T15:00:59.799823+0000","last_became_peered":"2024-06-28T15:00:59.799823+0000","last_unstale":"2024-06-28T16:14:42.650196+0000","last_undegraded":"2024-06-28T16:14:42.650196+0000","last_fullsized":"2024-06-28T16:14:42.650196+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'74","last_scrub_stamp":"2024-06-28T16:14:42.650003+0000","last_deep_scrub":"260'74","last_deep_scrub_stamp":"2024-06-28T16:14:42.650003+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.650003+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:28:32.949699+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":247,"num_write":103,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"260'351","reported_seq":1082,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.582297+0000","last_change":"2024-06-28T15:00:15.403798+0000","last_active":"2024-06-28T16:14:34.582297+0000","last_peered":"2024-06-28T16:14:34.582297+0000","last_clean":"2024-06-28T16:14:34.582297+0000","last_became_active":"2024-06-28T15:00:15.402782+0000","last_became_peered":"2024-06-28T15:00:15.402782+0000","last_unstale":"2024-06-28T16:14:34.582297+0000","last_undegraded":"2024-06-28T16:14:34.582297+0000","last_fullsized":"2024-06-28T16:14:34.582297+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":300,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262233,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":541,"num_read_kb":3603,"num_write":1164,"num_write_kb":36556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.130681+0000","last_change":"2024-06-28T15:00:19.405878+0000","last_active":"2024-06-28T16:14:35.130681+0000","last_peered":"2024-06-28T16:14:35.130681+0000","last_clean":"2024-06-28T16:14:35.130681+0000","last_became_active":"2024-06-28T15:00:19.405083+0000","last_became_peered":"2024-06-28T15:00:19.405083+0000","last_unstale":"2024-06-28T16:14:35.130681+0000","last_undegraded":"2024-06-28T16:14:35.130681+0000","last_fullsized":"2024-06-28T16:14:35.130681+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"260'437","reported_seq":1081,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.565170+0000","last_change":"2024-06-28T16:14:57.565257+0000","last_active":"2024-06-28T16:14:57.565170+0000","last_peered":"2024-06-28T16:14:57.565170+0000","last_clean":"2024-06-28T16:14:57.565170+0000","last_became_active":"2024-06-28T15:00:21.392393+0000","last_became_peered":"2024-06-28T15:00:21.392393+0000","last_unstale":"2024-06-28T16:14:57.565170+0000","last_undegraded":"2024-06-28T16:14:57.565170+0000","last_fullsized":"2024-06-28T16:14:57.565170+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'437","last_scrub_stamp":"2024-06-28T16:14:57.564983+0000","last_deep_scrub":"260'437","last_deep_scrub_stamp":"2024-06-28T16:14:57.564983+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.564983+0000","objects_scrubbed":9,"log_size":37,"log_dups_size":400,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:47:46.699794+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47467,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1282,"num_read_kb":1893,"num_write":1822,"num_write_kb":1648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":479870,"num_omap_keys":1112,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.323011+0000","last_change":"2024-06-28T16:14:54.323067+0000","last_active":"2024-06-28T16:14:54.323011+0000","last_peered":"2024-06-28T16:14:54.323011+0000","last_clean":"2024-06-28T16:14:54.323011+0000","last_became_active":"2024-06-28T15:00:23.415497+0000","last_became_peered":"2024-06-28T15:00:23.415497+0000","last_unstale":"2024-06-28T16:14:54.323011+0000","last_undegraded":"2024-06-28T16:14:54.323011+0000","last_fullsized":"2024-06-28T16:14:54.323011+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:54.322811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:54.322811+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.322811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:40:56.629420+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"260'136","reported_seq":651,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.029537+0000","last_change":"2024-06-28T16:14:48.029609+0000","last_active":"2024-06-28T16:14:48.029537+0000","last_peered":"2024-06-28T16:14:48.029537+0000","last_clean":"2024-06-28T16:14:48.029537+0000","last_became_active":"2024-06-28T15:00:25.439527+0000","last_became_peered":"2024-06-28T15:00:25.439527+0000","last_unstale":"2024-06-28T16:14:48.029537+0000","last_undegraded":"2024-06-28T16:14:48.029537+0000","last_fullsized":"2024-06-28T16:14:48.029537+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'136","last_scrub_stamp":"2024-06-28T16:14:48.029357+0000","last_deep_scrub":"260'136","last_deep_scrub_stamp":"2024-06-28T16:14:48.029357+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.029357+0000","objects_scrubbed":3,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:09:52.677657+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":127,"num_write":301,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"260'157","reported_seq":783,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.366718+0000","last_change":"2024-06-28T16:14:55.366787+0000","last_active":"2024-06-28T16:14:55.366718+0000","last_peered":"2024-06-28T16:14:55.366718+0000","last_clean":"2024-06-28T16:14:55.366718+0000","last_became_active":"2024-06-28T15:00:17.414537+0000","last_became_peered":"2024-06-28T15:00:17.414537+0000","last_unstale":"2024-06-28T16:14:55.366718+0000","last_undegraded":"2024-06-28T16:14:55.366718+0000","last_fullsized":"2024-06-28T16:14:55.366718+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'157","last_scrub_stamp":"2024-06-28T16:14:55.366527+0000","last_deep_scrub":"260'157","last_deep_scrub_stamp":"2024-06-28T16:14:55.366527+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.366527+0000","objects_scrubbed":1,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:51:04.103193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":522,"num_write":223,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":617,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:25.137102+0000","last_change":"2024-06-28T15:00:06.303487+0000","last_active":"2024-06-28T16:14:25.137102+0000","last_peered":"2024-06-28T16:14:25.137102+0000","last_clean":"2024-06-28T16:14:25.137102+0000","last_became_active":"2024-06-28T15:00:06.302725+0000","last_became_peered":"2024-06-28T15:00:06.302725+0000","last_unstale":"2024-06-28T16:14:25.137102+0000","last_undegraded":"2024-06-28T16:14:25.137102+0000","last_fullsized":"2024-06-28T16:14:25.137102+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":525,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:05.104513+0000","last_change":"2024-06-28T15:00:06.285125+0000","last_active":"2024-06-28T16:13:05.104513+0000","last_peered":"2024-06-28T16:13:05.104513+0000","last_clean":"2024-06-28T16:13:05.104513+0000","last_became_active":"2024-06-28T15:00:06.284218+0000","last_became_peered":"2024-06-28T15:00:06.284218+0000","last_unstale":"2024-06-28T16:13:05.104513+0000","last_undegraded":"2024-06-28T16:13:05.104513+0000","last_fullsized":"2024-06-28T16:13:05.104513+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"260'320","reported_seq":1124,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.110063+0000","last_change":"2024-06-28T16:14:48.110204+0000","last_active":"2024-06-28T16:14:48.110063+0000","last_peered":"2024-06-28T16:14:48.110063+0000","last_clean":"2024-06-28T16:14:48.110063+0000","last_became_active":"2024-06-28T15:00:59.939352+0000","last_became_peered":"2024-06-28T15:00:59.939352+0000","last_unstale":"2024-06-28T16:14:48.110063+0000","last_undegraded":"2024-06-28T16:14:48.110063+0000","last_fullsized":"2024-06-28T16:14:48.110063+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'320","last_scrub_stamp":"2024-06-28T16:14:48.109838+0000","last_deep_scrub":"260'320","last_deep_scrub_stamp":"2024-06-28T16:14:48.109838+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.109838+0000","objects_scrubbed":10,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:13:21.727097+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":804,"num_read_kb":59106,"num_write":1183,"num_write_kb":43047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6,4],"acting":[3,2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.121011+0000","last_change":"2024-06-28T15:00:19.391126+0000","last_active":"2024-06-28T16:13:15.121011+0000","last_peered":"2024-06-28T16:13:15.121011+0000","last_clean":"2024-06-28T16:13:15.121011+0000","last_became_active":"2024-06-28T15:00:19.390255+0000","last_became_peered":"2024-06-28T15:00:19.390255+0000","last_unstale":"2024-06-28T16:13:15.121011+0000","last_undegraded":"2024-06-28T16:13:15.121011+0000","last_fullsized":"2024-06-28T16:13:15.121011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"260'734","reported_seq":1463,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.597944+0000","last_change":"2024-06-28T16:14:55.598090+0000","last_active":"2024-06-28T16:14:55.597944+0000","last_peered":"2024-06-28T16:14:55.597944+0000","last_clean":"2024-06-28T16:14:55.597944+0000","last_became_active":"2024-06-28T15:00:21.397640+0000","last_became_peered":"2024-06-28T15:00:21.397640+0000","last_unstale":"2024-06-28T16:14:55.597944+0000","last_undegraded":"2024-06-28T16:14:55.597944+0000","last_fullsized":"2024-06-28T16:14:55.597944+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'734","last_scrub_stamp":"2024-06-28T16:14:55.597730+0000","last_deep_scrub":"260'734","last_deep_scrub_stamp":"2024-06-28T16:14:55.597730+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.597730+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":700,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:40:40.351703+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134565,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1425,"num_read_kb":3272,"num_write":1360,"num_write_kb":1190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99450,"num_omap_keys":240,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:05.601127+0000","last_change":"2024-06-28T15:00:23.400439+0000","last_active":"2024-06-28T16:14:05.601127+0000","last_peered":"2024-06-28T16:14:05.601127+0000","last_clean":"2024-06-28T16:14:05.601127+0000","last_became_active":"2024-06-28T15:00:23.399024+0000","last_became_peered":"2024-06-28T15:00:23.399024+0000","last_unstale":"2024-06-28T16:14:05.601127+0000","last_undegraded":"2024-06-28T16:14:05.601127+0000","last_fullsized":"2024-06-28T16:14:05.601127+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"260'186","reported_seq":877,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.371120+0000","last_change":"2024-06-28T15:00:25.442723+0000","last_active":"2024-06-28T16:14:08.371120+0000","last_peered":"2024-06-28T16:14:08.371120+0000","last_clean":"2024-06-28T16:14:08.371120+0000","last_became_active":"2024-06-28T15:00:25.442032+0000","last_became_peered":"2024-06-28T15:00:25.442032+0000","last_unstale":"2024-06-28T16:14:08.371120+0000","last_undegraded":"2024-06-28T16:14:08.371120+0000","last_fullsized":"2024-06-28T16:14:08.371120+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":392,"num_read_kb":388,"num_write":388,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"260'118","reported_seq":756,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.599760+0000","last_change":"2024-06-28T15:00:17.404538+0000","last_active":"2024-06-28T16:14:00.599760+0000","last_peered":"2024-06-28T16:14:00.599760+0000","last_clean":"2024-06-28T16:14:00.599760+0000","last_became_active":"2024-06-28T15:00:17.404200+0000","last_became_peered":"2024-06-28T15:00:17.404200+0000","last_unstale":"2024-06-28T16:14:00.599760+0000","last_undegraded":"2024-06-28T16:14:00.599760+0000","last_fullsized":"2024-06-28T16:14:00.599760+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":495,"num_read_kb":413,"num_write":159,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":527,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:25.137140+0000","last_change":"2024-06-28T15:00:06.286231+0000","last_active":"2024-06-28T16:14:25.137140+0000","last_peered":"2024-06-28T16:14:25.137140+0000","last_clean":"2024-06-28T16:14:25.137140+0000","last_became_active":"2024-06-28T15:00:06.285489+0000","last_became_peered":"2024-06-28T15:00:06.285489+0000","last_unstale":"2024-06-28T16:14:25.137140+0000","last_undegraded":"2024-06-28T16:14:25.137140+0000","last_fullsized":"2024-06-28T16:14:25.137140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"260'324","reported_seq":993,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.669814+0000","last_change":"2024-06-28T15:00:15.404155+0000","last_active":"2024-06-28T16:14:34.669814+0000","last_peered":"2024-06-28T16:14:34.669814+0000","last_clean":"2024-06-28T16:14:34.669814+0000","last_became_active":"2024-06-28T15:00:15.403657+0000","last_became_peered":"2024-06-28T15:00:15.403657+0000","last_unstale":"2024-06-28T16:14:34.669814+0000","last_undegraded":"2024-06-28T16:14:34.669814+0000","last_fullsized":"2024-06-28T16:14:34.669814+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":524,"num_read_kb":36646,"num_write":1262,"num_write_kb":16548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.116389+0000","last_change":"2024-06-28T15:00:19.407157+0000","last_active":"2024-06-28T16:14:00.116389+0000","last_peered":"2024-06-28T16:14:00.116389+0000","last_clean":"2024-06-28T16:14:00.116389+0000","last_became_active":"2024-06-28T15:00:19.406447+0000","last_became_peered":"2024-06-28T15:00:19.406447+0000","last_unstale":"2024-06-28T16:14:00.116389+0000","last_undegraded":"2024-06-28T16:14:00.116389+0000","last_fullsized":"2024-06-28T16:14:00.116389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"260'1141","reported_seq":2924,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.980098+0000","last_change":"2024-06-28T16:14:56.980185+0000","last_active":"2024-06-28T16:14:56.980098+0000","last_peered":"2024-06-28T16:14:56.980098+0000","last_clean":"2024-06-28T16:14:56.980098+0000","last_became_active":"2024-06-28T15:00:21.398382+0000","last_became_peered":"2024-06-28T15:00:21.398382+0000","last_unstale":"2024-06-28T16:14:56.980098+0000","last_undegraded":"2024-06-28T16:14:56.980098+0000","last_fullsized":"2024-06-28T16:14:56.980098+0000","mapping_epoch":25,"log_start":"260'1100","ondisk_log_start":"260'1100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'1141","last_scrub_stamp":"2024-06-28T16:14:56.979906+0000","last_deep_scrub":"260'1141","last_deep_scrub_stamp":"2024-06-28T16:14:56.979906+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.979906+0000","objects_scrubbed":11,"log_size":41,"log_dups_size":1100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:56:04.659644+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133468,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1851,"num_read_kb":3741,"num_write":1150,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6022,"num_omap_keys":18,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.712239+0000","last_change":"2024-06-28T16:14:59.712290+0000","last_active":"2024-06-28T16:14:59.712239+0000","last_peered":"2024-06-28T16:14:59.712239+0000","last_clean":"2024-06-28T16:14:59.712239+0000","last_became_active":"2024-06-28T15:00:23.422876+0000","last_became_peered":"2024-06-28T15:00:23.422876+0000","last_unstale":"2024-06-28T16:14:59.712239+0000","last_undegraded":"2024-06-28T16:14:59.712239+0000","last_fullsized":"2024-06-28T16:14:59.712239+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:59.712038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:59.712038+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.712038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:40.142260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"260'120","reported_seq":642,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.019612+0000","last_change":"2024-06-28T16:14:47.019699+0000","last_active":"2024-06-28T16:14:47.019612+0000","last_peered":"2024-06-28T16:14:47.019612+0000","last_clean":"2024-06-28T16:14:47.019612+0000","last_became_active":"2024-06-28T15:00:25.553691+0000","last_became_peered":"2024-06-28T15:00:25.553691+0000","last_unstale":"2024-06-28T16:14:47.019612+0000","last_undegraded":"2024-06-28T16:14:47.019612+0000","last_fullsized":"2024-06-28T16:14:47.019612+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'120","last_scrub_stamp":"2024-06-28T16:14:47.019412+0000","last_deep_scrub":"260'120","last_deep_scrub_stamp":"2024-06-28T16:14:47.019412+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.019412+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:33:15.581216+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":862,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":120,"num_write":292,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"260'221","reported_seq":835,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.131001+0000","last_change":"2024-06-28T15:00:17.393736+0000","last_active":"2024-06-28T16:14:00.131001+0000","last_peered":"2024-06-28T16:14:00.131001+0000","last_clean":"2024-06-28T16:14:00.131001+0000","last_became_active":"2024-06-28T15:00:17.393361+0000","last_became_peered":"2024-06-28T15:00:17.393361+0000","last_unstale":"2024-06-28T16:14:00.131001+0000","last_undegraded":"2024-06-28T16:14:00.131001+0000","last_fullsized":"2024-06-28T16:14:00.131001+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":892,"num_read_kb":761,"num_write":358,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":541,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.216953+0000","last_change":"2024-06-28T16:14:56.217011+0000","last_active":"2024-06-28T16:14:56.216953+0000","last_peered":"2024-06-28T16:14:56.216953+0000","last_clean":"2024-06-28T16:14:56.216953+0000","last_became_active":"2024-06-28T15:00:06.279950+0000","last_became_peered":"2024-06-28T15:00:06.279950+0000","last_unstale":"2024-06-28T16:14:56.216953+0000","last_undegraded":"2024-06-28T16:14:56.216953+0000","last_fullsized":"2024-06-28T16:14:56.216953+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.216749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.216749+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.216749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:22:36.357975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"260'334","reported_seq":992,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.138035+0000","last_change":"2024-06-28T15:00:15.413730+0000","last_active":"2024-06-28T16:14:35.138035+0000","last_peered":"2024-06-28T16:14:35.138035+0000","last_clean":"2024-06-28T16:14:35.138035+0000","last_became_active":"2024-06-28T15:00:15.413348+0000","last_became_peered":"2024-06-28T15:00:15.413348+0000","last_unstale":"2024-06-28T16:14:35.138035+0000","last_undegraded":"2024-06-28T16:14:35.138035+0000","last_fullsized":"2024-06-28T16:14:35.138035+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":504,"num_read_kb":3638,"num_write":1247,"num_write_kb":36014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"260'2","reported_seq":525,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.146412+0000","last_change":"2024-06-28T16:14:47.146477+0000","last_active":"2024-06-28T16:14:47.146412+0000","last_peered":"2024-06-28T16:14:47.146412+0000","last_clean":"2024-06-28T16:14:47.146412+0000","last_became_active":"2024-06-28T15:00:19.404653+0000","last_became_peered":"2024-06-28T15:00:19.404653+0000","last_unstale":"2024-06-28T16:14:47.146412+0000","last_undegraded":"2024-06-28T16:14:47.146412+0000","last_fullsized":"2024-06-28T16:14:47.146412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:47.146212+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:47.146212+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.146212+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:46:59.182619+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"260'246","reported_seq":1692,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:13.424979+0000","last_change":"2024-06-28T15:00:21.484753+0000","last_active":"2024-06-28T16:14:13.424979+0000","last_peered":"2024-06-28T16:14:13.424979+0000","last_clean":"2024-06-28T16:14:13.424979+0000","last_became_active":"2024-06-28T15:00:21.483749+0000","last_became_peered":"2024-06-28T15:00:21.483749+0000","last_unstale":"2024-06-28T16:14:13.424979+0000","last_undegraded":"2024-06-28T16:14:13.424979+0000","last_fullsized":"2024-06-28T16:14:13.424979+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":506,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.590983+0000","last_change":"2024-06-28T15:00:23.400124+0000","last_active":"2024-06-28T16:13:20.590983+0000","last_peered":"2024-06-28T16:13:20.590983+0000","last_clean":"2024-06-28T16:13:20.590983+0000","last_became_active":"2024-06-28T15:00:23.399391+0000","last_became_peered":"2024-06-28T15:00:23.399391+0000","last_unstale":"2024-06-28T16:13:20.590983+0000","last_undegraded":"2024-06-28T16:13:20.590983+0000","last_fullsized":"2024-06-28T16:13:20.590983+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"260'242","reported_seq":857,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.112554+0000","last_change":"2024-06-28T16:14:55.112726+0000","last_active":"2024-06-28T16:14:55.112554+0000","last_peered":"2024-06-28T16:14:55.112554+0000","last_clean":"2024-06-28T16:14:55.112554+0000","last_became_active":"2024-06-28T15:00:25.419711+0000","last_became_peered":"2024-06-28T15:00:25.419711+0000","last_unstale":"2024-06-28T16:14:55.112554+0000","last_undegraded":"2024-06-28T16:14:55.112554+0000","last_fullsized":"2024-06-28T16:14:55.112554+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'242","last_scrub_stamp":"2024-06-28T16:14:55.112334+0000","last_deep_scrub":"260'242","last_deep_scrub_stamp":"2024-06-28T16:14:55.112334+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.112334+0000","objects_scrubbed":3,"log_size":42,"log_dups_size":200,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:15:19.328332+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":467,"num_read_kb":491,"num_write":517,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"260'315","reported_seq":1007,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.121987+0000","last_change":"2024-06-28T15:00:17.398693+0000","last_active":"2024-06-28T16:13:15.121987+0000","last_peered":"2024-06-28T16:13:15.121987+0000","last_clean":"2024-06-28T16:13:15.121987+0000","last_became_active":"2024-06-28T15:00:17.393696+0000","last_became_peered":"2024-06-28T15:00:17.393696+0000","last_unstale":"2024-06-28T16:13:15.121987+0000","last_undegraded":"2024-06-28T16:13:15.121987+0000","last_fullsized":"2024-06-28T16:13:15.121987+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1365,"num_read_kb":1154,"num_write":468,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":525,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.103311+0000","last_change":"2024-06-28T15:00:06.284111+0000","last_active":"2024-06-28T16:13:00.103311+0000","last_peered":"2024-06-28T16:13:00.103311+0000","last_clean":"2024-06-28T16:13:00.103311+0000","last_became_active":"2024-06-28T15:00:06.283106+0000","last_became_peered":"2024-06-28T15:00:06.283106+0000","last_unstale":"2024-06-28T16:13:00.103311+0000","last_undegraded":"2024-06-28T16:13:00.103311+0000","last_fullsized":"2024-06-28T16:13:00.103311+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"260'318","reported_seq":992,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.749649+0000","last_change":"2024-06-28T15:00:15.415068+0000","last_active":"2024-06-28T16:14:34.749649+0000","last_peered":"2024-06-28T16:14:34.749649+0000","last_clean":"2024-06-28T16:14:34.749649+0000","last_became_active":"2024-06-28T15:00:15.413880+0000","last_became_peered":"2024-06-28T15:00:15.413880+0000","last_unstale":"2024-06-28T16:14:34.749649+0000","last_undegraded":"2024-06-28T16:14:34.749649+0000","last_fullsized":"2024-06-28T16:14:34.749649+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2421978,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":465,"num_read_kb":85416,"num_write":1148,"num_write_kb":27167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:30.593348+0000","last_change":"2024-06-28T15:00:19.382869+0000","last_active":"2024-06-28T16:13:30.593348+0000","last_peered":"2024-06-28T16:13:30.593348+0000","last_clean":"2024-06-28T16:13:30.593348+0000","last_became_active":"2024-06-28T15:00:19.382486+0000","last_became_peered":"2024-06-28T15:00:19.382486+0000","last_unstale":"2024-06-28T16:13:30.593348+0000","last_undegraded":"2024-06-28T16:13:30.593348+0000","last_fullsized":"2024-06-28T16:13:30.593348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"260'159","reported_seq":730,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:05.329931+0000","last_change":"2024-06-28T15:00:21.401345+0000","last_active":"2024-06-28T16:14:05.329931+0000","last_peered":"2024-06-28T16:14:05.329931+0000","last_clean":"2024-06-28T16:14:05.329931+0000","last_became_active":"2024-06-28T15:00:21.400535+0000","last_became_peered":"2024-06-28T15:00:21.400535+0000","last_unstale":"2024-06-28T16:14:05.329931+0000","last_undegraded":"2024-06-28T16:14:05.329931+0000","last_fullsized":"2024-06-28T16:14:05.329931+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":198,"num_read_kb":191,"num_write":142,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.407107+0000","last_change":"2024-06-28T16:14:56.407161+0000","last_active":"2024-06-28T16:14:56.407107+0000","last_peered":"2024-06-28T16:14:56.407107+0000","last_clean":"2024-06-28T16:14:56.407107+0000","last_became_active":"2024-06-28T15:00:23.416701+0000","last_became_peered":"2024-06-28T15:00:23.416701+0000","last_unstale":"2024-06-28T16:14:56.407107+0000","last_undegraded":"2024-06-28T16:14:56.407107+0000","last_fullsized":"2024-06-28T16:14:56.407107+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.406907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.406907+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.406907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:09:09.502206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"260'117","reported_seq":642,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.620180+0000","last_change":"2024-06-28T16:14:47.620245+0000","last_active":"2024-06-28T16:14:47.620180+0000","last_peered":"2024-06-28T16:14:47.620180+0000","last_clean":"2024-06-28T16:14:47.620180+0000","last_became_active":"2024-06-28T15:00:25.438523+0000","last_became_peered":"2024-06-28T15:00:25.438523+0000","last_unstale":"2024-06-28T16:14:47.620180+0000","last_undegraded":"2024-06-28T16:14:47.620180+0000","last_fullsized":"2024-06-28T16:14:47.620180+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'117","last_scrub_stamp":"2024-06-28T16:14:47.619989+0000","last_deep_scrub":"260'117","last_deep_scrub_stamp":"2024-06-28T16:14:47.619989+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.619989+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:25:16.904136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":129,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"260'169","reported_seq":775,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.121903+0000","last_change":"2024-06-28T15:00:17.399083+0000","last_active":"2024-06-28T16:13:15.121903+0000","last_peered":"2024-06-28T16:13:15.121903+0000","last_clean":"2024-06-28T16:13:15.121903+0000","last_became_active":"2024-06-28T15:00:17.398515+0000","last_became_peered":"2024-06-28T15:00:17.398515+0000","last_unstale":"2024-06-28T16:13:15.121903+0000","last_undegraded":"2024-06-28T16:13:15.121903+0000","last_fullsized":"2024-06-28T16:13:15.121903+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":644,"num_read_kb":512,"num_write":273,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"260'372","reported_seq":1044,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.750566+0000","last_change":"2024-06-28T15:00:15.417442+0000","last_active":"2024-06-28T16:14:34.750566+0000","last_peered":"2024-06-28T16:14:34.750566+0000","last_clean":"2024-06-28T16:14:34.750566+0000","last_became_active":"2024-06-28T15:00:15.414704+0000","last_became_peered":"2024-06-28T15:00:15.414704+0000","last_unstale":"2024-06-28T16:14:34.750566+0000","last_undegraded":"2024-06-28T16:14:34.750566+0000","last_fullsized":"2024-06-28T16:14:34.750566+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10506361,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":617,"num_read_kb":33619,"num_write":1289,"num_write_kb":23520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":530,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:05.119299+0000","last_change":"2024-06-28T15:00:06.302842+0000","last_active":"2024-06-28T16:13:05.119299+0000","last_peered":"2024-06-28T16:13:05.119299+0000","last_clean":"2024-06-28T16:13:05.119299+0000","last_became_active":"2024-06-28T15:00:06.302449+0000","last_became_peered":"2024-06-28T15:00:06.302449+0000","last_unstale":"2024-06-28T16:13:05.119299+0000","last_undegraded":"2024-06-28T16:13:05.119299+0000","last_fullsized":"2024-06-28T16:13:05.119299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.589775+0000","last_change":"2024-06-28T15:00:19.385492+0000","last_active":"2024-06-28T16:13:15.589775+0000","last_peered":"2024-06-28T16:13:15.589775+0000","last_clean":"2024-06-28T16:13:15.589775+0000","last_became_active":"2024-06-28T15:00:19.384933+0000","last_became_peered":"2024-06-28T15:00:19.384933+0000","last_unstale":"2024-06-28T16:13:15.589775+0000","last_undegraded":"2024-06-28T16:13:15.589775+0000","last_fullsized":"2024-06-28T16:13:15.589775+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"260'587","reported_seq":1287,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.609270+0000","last_change":"2024-06-28T15:00:21.407998+0000","last_active":"2024-06-28T16:14:38.609270+0000","last_peered":"2024-06-28T16:14:38.609270+0000","last_clean":"2024-06-28T16:14:38.609270+0000","last_became_active":"2024-06-28T15:00:21.405762+0000","last_became_peered":"2024-06-28T15:00:21.405762+0000","last_unstale":"2024-06-28T16:14:38.609270+0000","last_undegraded":"2024-06-28T16:14:38.609270+0000","last_fullsized":"2024-06-28T16:14:38.609270+0000","mapping_epoch":25,"log_start":"260'500","ondisk_log_start":"260'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":500,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92557,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1001,"num_read_kb":2235,"num_write":838,"num_write_kb":641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":509,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.927443+0000","last_change":"2024-06-28T16:14:49.927496+0000","last_active":"2024-06-28T16:14:49.927443+0000","last_peered":"2024-06-28T16:14:49.927443+0000","last_clean":"2024-06-28T16:14:49.927443+0000","last_became_active":"2024-06-28T15:00:23.404714+0000","last_became_peered":"2024-06-28T15:00:23.404714+0000","last_unstale":"2024-06-28T16:14:49.927443+0000","last_undegraded":"2024-06-28T16:14:49.927443+0000","last_fullsized":"2024-06-28T16:14:49.927443+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.927238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.927238+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.927238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:46:09.338954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"260'154","reported_seq":680,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.151138+0000","last_change":"2024-06-28T16:14:48.151312+0000","last_active":"2024-06-28T16:14:48.151138+0000","last_peered":"2024-06-28T16:14:48.151138+0000","last_clean":"2024-06-28T16:14:48.151138+0000","last_became_active":"2024-06-28T15:00:25.556834+0000","last_became_peered":"2024-06-28T15:00:25.556834+0000","last_unstale":"2024-06-28T16:14:48.151138+0000","last_undegraded":"2024-06-28T16:14:48.151138+0000","last_fullsized":"2024-06-28T16:14:48.151138+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'154","last_scrub_stamp":"2024-06-28T16:14:48.150905+0000","last_deep_scrub":"260'154","last_deep_scrub_stamp":"2024-06-28T16:14:48.150905+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.150905+0000","objects_scrubbed":7,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:54:08.173965+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1183,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":342,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"260'139","reported_seq":769,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.589735+0000","last_change":"2024-06-28T15:00:17.372769+0000","last_active":"2024-06-28T16:13:15.589735+0000","last_peered":"2024-06-28T16:13:15.589735+0000","last_clean":"2024-06-28T16:13:15.589735+0000","last_became_active":"2024-06-28T15:00:17.372401+0000","last_became_peered":"2024-06-28T15:00:17.372401+0000","last_unstale":"2024-06-28T16:13:15.589735+0000","last_undegraded":"2024-06-28T16:13:15.589735+0000","last_fullsized":"2024-06-28T16:13:15.589735+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":577,"num_read_kb":470,"num_write":221,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"260'273","reported_seq":902,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.567969+0000","last_change":"2024-06-28T15:00:15.408605+0000","last_active":"2024-06-28T16:14:34.567969+0000","last_peered":"2024-06-28T16:14:34.567969+0000","last_clean":"2024-06-28T16:14:34.567969+0000","last_became_active":"2024-06-28T15:00:15.408140+0000","last_became_peered":"2024-06-28T15:00:15.408140+0000","last_unstale":"2024-06-28T16:14:34.567969+0000","last_undegraded":"2024-06-28T16:14:34.567969+0000","last_fullsized":"2024-06-28T16:14:34.567969+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":200,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":526399,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":387,"num_read_kb":8190,"num_write":1081,"num_write_kb":24156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":525,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.122148+0000","last_change":"2024-06-28T15:00:06.303984+0000","last_active":"2024-06-28T16:13:20.122148+0000","last_peered":"2024-06-28T16:13:20.122148+0000","last_clean":"2024-06-28T16:13:20.122148+0000","last_became_active":"2024-06-28T15:00:06.302962+0000","last_became_peered":"2024-06-28T15:00:06.302962+0000","last_unstale":"2024-06-28T16:13:20.122148+0000","last_undegraded":"2024-06-28T16:13:20.122148+0000","last_fullsized":"2024-06-28T16:13:20.122148+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.184389+0000","last_change":"2024-06-28T16:14:49.184441+0000","last_active":"2024-06-28T16:14:49.184389+0000","last_peered":"2024-06-28T16:14:49.184389+0000","last_clean":"2024-06-28T16:14:49.184389+0000","last_became_active":"2024-06-28T15:00:19.384961+0000","last_became_peered":"2024-06-28T15:00:19.384961+0000","last_unstale":"2024-06-28T16:14:49.184389+0000","last_undegraded":"2024-06-28T16:14:49.184389+0000","last_fullsized":"2024-06-28T16:14:49.184389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.184197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.184197+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.184197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:48:38.391991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"260'195","reported_seq":801,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.962522+0000","last_change":"2024-06-28T16:14:58.962600+0000","last_active":"2024-06-28T16:14:58.962522+0000","last_peered":"2024-06-28T16:14:58.962522+0000","last_clean":"2024-06-28T16:14:58.962522+0000","last_became_active":"2024-06-28T15:00:21.480107+0000","last_became_peered":"2024-06-28T15:00:21.480107+0000","last_unstale":"2024-06-28T16:14:58.962522+0000","last_undegraded":"2024-06-28T16:14:58.962522+0000","last_fullsized":"2024-06-28T16:14:58.962522+0000","mapping_epoch":25,"log_start":"257'100","ondisk_log_start":"257'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'195","last_scrub_stamp":"2024-06-28T16:14:58.962331+0000","last_deep_scrub":"260'195","last_deep_scrub_stamp":"2024-06-28T16:14:58.962331+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.962331+0000","objects_scrubbed":7,"log_size":95,"log_dups_size":100,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:08:23.304315+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":263,"num_read_kb":253,"num_write":174,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"260'565","reported_seq":2887,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.666227+0000","last_change":"2024-06-28T16:14:51.666291+0000","last_active":"2024-06-28T16:14:51.666227+0000","last_peered":"2024-06-28T16:14:51.666227+0000","last_clean":"2024-06-28T16:14:51.666227+0000","last_became_active":"2024-06-28T15:00:23.427565+0000","last_became_peered":"2024-06-28T15:00:23.427565+0000","last_unstale":"2024-06-28T16:14:51.666227+0000","last_undegraded":"2024-06-28T16:14:51.666227+0000","last_fullsized":"2024-06-28T16:14:51.666227+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'565","last_scrub_stamp":"2024-06-28T16:14:51.666022+0000","last_deep_scrub":"260'565","last_deep_scrub_stamp":"2024-06-28T16:14:51.666022+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.666022+0000","objects_scrubbed":1,"log_size":65,"log_dups_size":500,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:06:09.388341+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"260'127","reported_seq":651,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.139109+0000","last_change":"2024-06-28T16:14:51.139189+0000","last_active":"2024-06-28T16:14:51.139109+0000","last_peered":"2024-06-28T16:14:51.139109+0000","last_clean":"2024-06-28T16:14:51.139109+0000","last_became_active":"2024-06-28T15:00:25.553627+0000","last_became_peered":"2024-06-28T15:00:25.553627+0000","last_unstale":"2024-06-28T16:14:51.139109+0000","last_undegraded":"2024-06-28T16:14:51.139109+0000","last_fullsized":"2024-06-28T16:14:51.139109+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'127","last_scrub_stamp":"2024-06-28T16:14:51.138914+0000","last_deep_scrub":"260'127","last_deep_scrub_stamp":"2024-06-28T16:14:51.138914+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.138914+0000","objects_scrubbed":3,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:08:31.042999+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":713,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":165,"num_read_kb":148,"num_write":308,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"260'172","reported_seq":826,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:30.335672+0000","last_change":"2024-06-28T15:00:17.377265+0000","last_active":"2024-06-28T16:14:30.335672+0000","last_peered":"2024-06-28T16:14:30.335672+0000","last_clean":"2024-06-28T16:14:30.335672+0000","last_became_active":"2024-06-28T15:00:17.376516+0000","last_became_peered":"2024-06-28T15:00:17.376516+0000","last_unstale":"2024-06-28T16:14:30.335672+0000","last_undegraded":"2024-06-28T16:14:30.335672+0000","last_fullsized":"2024-06-28T16:14:30.335672+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":722,"num_read_kb":588,"num_write":244,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"260'336","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.742328+0000","last_change":"2024-06-28T15:00:15.403594+0000","last_active":"2024-06-28T16:14:34.742328+0000","last_peered":"2024-06-28T16:14:34.742328+0000","last_clean":"2024-06-28T16:14:34.742328+0000","last_became_active":"2024-06-28T15:00:15.403032+0000","last_became_peered":"2024-06-28T15:00:15.403032+0000","last_unstale":"2024-06-28T16:14:34.742328+0000","last_undegraded":"2024-06-28T16:14:34.742328+0000","last_fullsized":"2024-06-28T16:14:34.742328+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":968026,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":483,"num_read_kb":17032,"num_write":1147,"num_write_kb":35885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":529,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.127835+0000","last_change":"2024-06-28T15:00:06.278832+0000","last_active":"2024-06-28T16:13:45.127835+0000","last_peered":"2024-06-28T16:13:45.127835+0000","last_clean":"2024-06-28T16:13:45.127835+0000","last_became_active":"2024-06-28T15:00:06.278443+0000","last_became_peered":"2024-06-28T15:00:06.278443+0000","last_unstale":"2024-06-28T16:13:45.127835+0000","last_undegraded":"2024-06-28T16:13:45.127835+0000","last_fullsized":"2024-06-28T16:13:45.127835+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":509,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.113342+0000","last_change":"2024-06-28T15:00:19.380916+0000","last_active":"2024-06-28T16:13:15.113342+0000","last_peered":"2024-06-28T16:13:15.113342+0000","last_clean":"2024-06-28T16:13:15.113342+0000","last_became_active":"2024-06-28T15:00:19.378932+0000","last_became_peered":"2024-06-28T15:00:19.378932+0000","last_unstale":"2024-06-28T16:13:15.113342+0000","last_undegraded":"2024-06-28T16:13:15.113342+0000","last_fullsized":"2024-06-28T16:13:15.113342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"260'143","reported_seq":712,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.986875+0000","last_change":"2024-06-28T16:14:54.986950+0000","last_active":"2024-06-28T16:14:54.986875+0000","last_peered":"2024-06-28T16:14:54.986875+0000","last_clean":"2024-06-28T16:14:54.986875+0000","last_became_active":"2024-06-28T15:00:21.395462+0000","last_became_peered":"2024-06-28T15:00:21.395462+0000","last_unstale":"2024-06-28T16:14:54.986875+0000","last_undegraded":"2024-06-28T16:14:54.986875+0000","last_fullsized":"2024-06-28T16:14:54.986875+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'143","last_scrub_stamp":"2024-06-28T16:14:54.986685+0000","last_deep_scrub":"260'143","last_deep_scrub_stamp":"2024-06-28T16:14:54.986685+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.986685+0000","objects_scrubbed":5,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:03:28.897014+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":167,"num_read_kb":161,"num_write":112,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"260'537","reported_seq":2590,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.176626+0000","last_change":"2024-06-28T16:14:58.176689+0000","last_active":"2024-06-28T16:14:58.176626+0000","last_peered":"2024-06-28T16:14:58.176626+0000","last_clean":"2024-06-28T16:14:58.176626+0000","last_became_active":"2024-06-28T15:00:23.427568+0000","last_became_peered":"2024-06-28T15:00:23.427568+0000","last_unstale":"2024-06-28T16:14:58.176626+0000","last_undegraded":"2024-06-28T16:14:58.176626+0000","last_fullsized":"2024-06-28T16:14:58.176626+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'537","last_scrub_stamp":"2024-06-28T16:14:58.176428+0000","last_deep_scrub":"260'537","last_deep_scrub_stamp":"2024-06-28T16:14:58.176428+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.176428+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:07:27.328025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"260'251","reported_seq":800,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.105514+0000","last_change":"2024-06-28T16:14:53.105587+0000","last_active":"2024-06-28T16:14:53.105514+0000","last_peered":"2024-06-28T16:14:53.105514+0000","last_clean":"2024-06-28T16:14:53.105514+0000","last_became_active":"2024-06-28T15:00:25.428067+0000","last_became_peered":"2024-06-28T15:00:25.428067+0000","last_unstale":"2024-06-28T16:14:53.105514+0000","last_undegraded":"2024-06-28T16:14:53.105514+0000","last_fullsized":"2024-06-28T16:14:53.105514+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'251","last_scrub_stamp":"2024-06-28T16:14:53.105331+0000","last_deep_scrub":"260'251","last_deep_scrub_stamp":"2024-06-28T16:14:53.105331+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.105331+0000","objects_scrubbed":4,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:01:44.076283+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":297,"num_write":552,"num_write_kb":170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"260'96","reported_seq":702,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.113369+0000","last_change":"2024-06-28T15:00:17.397233+0000","last_active":"2024-06-28T16:13:15.113369+0000","last_peered":"2024-06-28T16:13:15.113369+0000","last_clean":"2024-06-28T16:13:15.113369+0000","last_became_active":"2024-06-28T15:00:17.396413+0000","last_became_peered":"2024-06-28T15:00:17.396413+0000","last_unstale":"2024-06-28T16:13:15.113369+0000","last_undegraded":"2024-06-28T16:13:15.113369+0000","last_fullsized":"2024-06-28T16:13:15.113369+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":410,"num_read_kb":340,"num_write":139,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"260'318","reported_seq":1098,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.743460+0000","last_change":"2024-06-28T15:00:15.396426+0000","last_active":"2024-06-28T16:14:34.743460+0000","last_peered":"2024-06-28T16:14:34.743460+0000","last_clean":"2024-06-28T16:14:34.743460+0000","last_became_active":"2024-06-28T15:00:15.395980+0000","last_became_peered":"2024-06-28T15:00:15.395980+0000","last_unstale":"2024-06-28T16:14:34.743460+0000","last_undegraded":"2024-06-28T16:14:34.743460+0000","last_fullsized":"2024-06-28T16:14:34.743460+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661262,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":649,"num_read_kb":125515,"num_write":1187,"num_write_kb":17513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":522,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.119119+0000","last_change":"2024-06-28T15:00:06.284169+0000","last_active":"2024-06-28T16:13:45.119119+0000","last_peered":"2024-06-28T16:13:45.119119+0000","last_clean":"2024-06-28T16:13:45.119119+0000","last_became_active":"2024-06-28T15:00:06.283380+0000","last_became_peered":"2024-06-28T15:00:06.283380+0000","last_unstale":"2024-06-28T16:13:45.119119+0000","last_undegraded":"2024-06-28T16:13:45.119119+0000","last_fullsized":"2024-06-28T16:13:45.119119+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"31'5","reported_seq":843,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:31.947541+0000","last_change":"2024-06-28T15:00:19.372913+0000","last_active":"2024-06-28T16:14:31.947541+0000","last_peered":"2024-06-28T16:14:31.947541+0000","last_clean":"2024-06-28T16:14:31.947541+0000","last_became_active":"2024-06-28T15:00:19.372552+0000","last_became_peered":"2024-06-28T15:00:19.372552+0000","last_unstale":"2024-06-28T16:14:31.947541+0000","last_undegraded":"2024-06-28T16:14:31.947541+0000","last_fullsized":"2024-06-28T16:14:31.947541+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":500,"num_read_kb":812,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"260'748","reported_seq":1493,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.577429+0000","last_change":"2024-06-28T15:00:21.481881+0000","last_active":"2024-06-28T16:14:33.577429+0000","last_peered":"2024-06-28T16:14:33.577429+0000","last_clean":"2024-06-28T16:14:33.577429+0000","last_became_active":"2024-06-28T15:00:21.481155+0000","last_became_peered":"2024-06-28T15:00:21.481155+0000","last_unstale":"2024-06-28T16:14:33.577429+0000","last_undegraded":"2024-06-28T16:14:33.577429+0000","last_fullsized":"2024-06-28T16:14:33.577429+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":700,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131438,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1320,"num_read_kb":3166,"num_write":1058,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.319861+0000","last_change":"2024-06-28T15:00:23.409553+0000","last_active":"2024-06-28T16:13:20.319861+0000","last_peered":"2024-06-28T16:13:20.319861+0000","last_clean":"2024-06-28T16:13:20.319861+0000","last_became_active":"2024-06-28T15:00:23.408777+0000","last_became_peered":"2024-06-28T15:00:23.408777+0000","last_unstale":"2024-06-28T16:13:20.319861+0000","last_undegraded":"2024-06-28T16:13:20.319861+0000","last_fullsized":"2024-06-28T16:13:20.319861+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"260'132","reported_seq":653,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.670572+0000","last_change":"2024-06-28T16:14:49.670641+0000","last_active":"2024-06-28T16:14:49.670572+0000","last_peered":"2024-06-28T16:14:49.670572+0000","last_clean":"2024-06-28T16:14:49.670572+0000","last_became_active":"2024-06-28T15:00:25.438476+0000","last_became_peered":"2024-06-28T15:00:25.438476+0000","last_unstale":"2024-06-28T16:14:49.670572+0000","last_undegraded":"2024-06-28T16:14:49.670572+0000","last_fullsized":"2024-06-28T16:14:49.670572+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'132","last_scrub_stamp":"2024-06-28T16:14:49.670359+0000","last_deep_scrub":"260'132","last_deep_scrub_stamp":"2024-06-28T16:14:49.670359+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.670359+0000","objects_scrubbed":3,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:09:32.566857+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":173,"num_read_kb":156,"num_write":320,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"260'153","reported_seq":729,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.120485+0000","last_change":"2024-06-28T16:14:49.120558+0000","last_active":"2024-06-28T16:14:49.120485+0000","last_peered":"2024-06-28T16:14:49.120485+0000","last_clean":"2024-06-28T16:14:49.120485+0000","last_became_active":"2024-06-28T15:00:25.439672+0000","last_became_peered":"2024-06-28T15:00:25.439672+0000","last_unstale":"2024-06-28T16:14:49.120485+0000","last_undegraded":"2024-06-28T16:14:49.120485+0000","last_fullsized":"2024-06-28T16:14:49.120485+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'153","last_scrub_stamp":"2024-06-28T16:14:49.120288+0000","last_deep_scrub":"260'153","last_deep_scrub_stamp":"2024-06-28T16:14:49.120288+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.120288+0000","objects_scrubbed":2,"log_size":53,"log_dups_size":100,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:05:56.112079+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":207,"num_read_kb":205,"num_write":345,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"260'434","reported_seq":1073,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.422274+0000","last_change":"2024-06-28T15:00:17.402772+0000","last_active":"2024-06-28T16:14:40.422274+0000","last_peered":"2024-06-28T16:14:40.422274+0000","last_clean":"2024-06-28T16:14:40.422274+0000","last_became_active":"2024-06-28T15:00:17.401981+0000","last_became_peered":"2024-06-28T15:00:17.401981+0000","last_unstale":"2024-06-28T16:14:40.422274+0000","last_undegraded":"2024-06-28T16:14:40.422274+0000","last_fullsized":"2024-06-28T16:14:40.422274+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":400,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1728,"num_read_kb":1391,"num_write":746,"num_write_kb":387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"260'274","reported_seq":901,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.698356+0000","last_change":"2024-06-28T15:00:15.404138+0000","last_active":"2024-06-28T16:14:34.698356+0000","last_peered":"2024-06-28T16:14:34.698356+0000","last_clean":"2024-06-28T16:14:34.698356+0000","last_became_active":"2024-06-28T15:00:15.403450+0000","last_became_peered":"2024-06-28T15:00:15.403450+0000","last_unstale":"2024-06-28T16:14:34.698356+0000","last_undegraded":"2024-06-28T16:14:34.698356+0000","last_fullsized":"2024-06-28T16:14:34.698356+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437799,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":13452,"num_write":1073,"num_write_kb":25751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":518,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.005157+0000","last_change":"2024-06-28T16:14:51.005157+0000","last_active":"2024-06-28T16:14:51.005157+0000","last_peered":"2024-06-28T16:14:51.005157+0000","last_clean":"2024-06-28T16:14:51.005157+0000","last_became_active":"2024-06-28T15:00:19.398138+0000","last_became_peered":"2024-06-28T15:00:19.398138+0000","last_unstale":"2024-06-28T16:14:51.005157+0000","last_undegraded":"2024-06-28T16:14:51.005157+0000","last_fullsized":"2024-06-28T16:14:51.005157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.004896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.004896+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.004896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:21:37.359914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"260'298","reported_seq":896,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.024496+0000","last_change":"2024-06-28T15:00:21.405835+0000","last_active":"2024-06-28T16:14:34.024496+0000","last_peered":"2024-06-28T16:14:34.024496+0000","last_clean":"2024-06-28T16:14:34.024496+0000","last_became_active":"2024-06-28T15:00:21.405016+0000","last_became_peered":"2024-06-28T15:00:21.405016+0000","last_unstale":"2024-06-28T16:14:34.024496+0000","last_undegraded":"2024-06-28T16:14:34.024496+0000","last_fullsized":"2024-06-28T16:14:34.024496+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53726,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":505,"num_read_kb":1165,"num_write":406,"num_write_kb":309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.204664+0000","last_change":"2024-06-28T16:15:00.204716+0000","last_active":"2024-06-28T16:15:00.204664+0000","last_peered":"2024-06-28T16:15:00.204664+0000","last_clean":"2024-06-28T16:15:00.204664+0000","last_became_active":"2024-06-28T15:00:23.417390+0000","last_became_peered":"2024-06-28T15:00:23.417390+0000","last_unstale":"2024-06-28T16:15:00.204664+0000","last_undegraded":"2024-06-28T16:15:00.204664+0000","last_fullsized":"2024-06-28T16:15:00.204664+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:00.204463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:00.204463+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.204463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:41:34.935690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"260'101","reported_seq":617,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.114583+0000","last_change":"2024-06-28T16:14:56.114642+0000","last_active":"2024-06-28T16:14:56.114583+0000","last_peered":"2024-06-28T16:14:56.114583+0000","last_clean":"2024-06-28T16:14:56.114583+0000","last_became_active":"2024-06-28T15:00:25.412379+0000","last_became_peered":"2024-06-28T15:00:25.412379+0000","last_unstale":"2024-06-28T16:14:56.114583+0000","last_undegraded":"2024-06-28T16:14:56.114583+0000","last_fullsized":"2024-06-28T16:14:56.114583+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'101","last_scrub_stamp":"2024-06-28T16:14:56.114398+0000","last_deep_scrub":"260'101","last_deep_scrub_stamp":"2024-06-28T16:14:56.114398+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.114398+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:50:04.756947+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":88,"num_write":238,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"260'189","reported_seq":832,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.954505+0000","last_change":"2024-06-28T16:14:57.954505+0000","last_active":"2024-06-28T16:14:57.954505+0000","last_peered":"2024-06-28T16:14:57.954505+0000","last_clean":"2024-06-28T16:14:57.954505+0000","last_became_active":"2024-06-28T15:00:17.406647+0000","last_became_peered":"2024-06-28T15:00:17.406647+0000","last_unstale":"2024-06-28T16:14:57.954505+0000","last_undegraded":"2024-06-28T16:14:57.954505+0000","last_fullsized":"2024-06-28T16:14:57.954505+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'189","last_scrub_stamp":"2024-06-28T16:14:57.954211+0000","last_deep_scrub":"260'189","last_deep_scrub_stamp":"2024-06-28T16:14:57.954211+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.954211+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:15:57.967237+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":748,"num_read_kb":601,"num_write":301,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"260'334","reported_seq":1002,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.629434+0000","last_change":"2024-06-28T15:00:15.413269+0000","last_active":"2024-06-28T16:14:34.629434+0000","last_peered":"2024-06-28T16:14:34.629434+0000","last_clean":"2024-06-28T16:14:34.629434+0000","last_became_active":"2024-06-28T15:00:15.402945+0000","last_became_peered":"2024-06-28T15:00:15.402945+0000","last_unstale":"2024-06-28T16:14:34.629434+0000","last_undegraded":"2024-06-28T16:14:34.629434+0000","last_fullsized":"2024-06-28T16:14:34.629434+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340417,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":534,"num_read_kb":20609,"num_write":1305,"num_write_kb":45266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,4],"acting":[7,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":509,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.971516+0000","last_change":"2024-06-28T15:00:19.383614+0000","last_active":"2024-06-28T16:14:39.971516+0000","last_peered":"2024-06-28T16:14:39.971516+0000","last_clean":"2024-06-28T16:14:39.971516+0000","last_became_active":"2024-06-28T15:00:19.382839+0000","last_became_peered":"2024-06-28T15:00:19.382839+0000","last_unstale":"2024-06-28T16:14:39.971516+0000","last_undegraded":"2024-06-28T16:14:39.971516+0000","last_fullsized":"2024-06-28T16:14:39.971516+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"260'52","reported_seq":574,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:45.581189+0000","last_change":"2024-06-28T15:00:21.407387+0000","last_active":"2024-06-28T16:12:45.581189+0000","last_peered":"2024-06-28T16:12:45.581189+0000","last_clean":"2024-06-28T16:12:45.581189+0000","last_became_active":"2024-06-28T15:00:21.404727+0000","last_became_peered":"2024-06-28T15:00:21.404727+0000","last_unstale":"2024-06-28T16:12:45.581189+0000","last_undegraded":"2024-06-28T16:12:45.581189+0000","last_fullsized":"2024-06-28T16:12:45.581189+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":48,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":495,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:45.113903+0000","last_change":"2024-06-28T15:00:23.419840+0000","last_active":"2024-06-28T16:12:45.113903+0000","last_peered":"2024-06-28T16:12:45.113903+0000","last_clean":"2024-06-28T16:12:45.113903+0000","last_became_active":"2024-06-28T15:00:23.419115+0000","last_became_peered":"2024-06-28T15:00:23.419115+0000","last_unstale":"2024-06-28T16:12:45.113903+0000","last_undegraded":"2024-06-28T16:12:45.113903+0000","last_fullsized":"2024-06-28T16:12:45.113903+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"260'191","reported_seq":729,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.032013+0000","last_change":"2024-06-28T16:14:55.032091+0000","last_active":"2024-06-28T16:14:55.032013+0000","last_peered":"2024-06-28T16:14:55.032013+0000","last_clean":"2024-06-28T16:14:55.032013+0000","last_became_active":"2024-06-28T15:00:25.426046+0000","last_became_peered":"2024-06-28T15:00:25.426046+0000","last_unstale":"2024-06-28T16:14:55.032013+0000","last_undegraded":"2024-06-28T16:14:55.032013+0000","last_fullsized":"2024-06-28T16:14:55.032013+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'191","last_scrub_stamp":"2024-06-28T16:14:55.031799+0000","last_deep_scrub":"260'191","last_deep_scrub_stamp":"2024-06-28T16:14:55.031799+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.031799+0000","objects_scrubbed":1,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:06:38.230720+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":286,"num_read_kb":283,"num_write":434,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"260'423","reported_seq":1172,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.971321+0000","last_change":"2024-06-28T15:00:17.390367+0000","last_active":"2024-06-28T16:14:39.971321+0000","last_peered":"2024-06-28T16:14:39.971321+0000","last_clean":"2024-06-28T16:14:39.971321+0000","last_became_active":"2024-06-28T15:00:17.389992+0000","last_became_peered":"2024-06-28T15:00:17.389992+0000","last_unstale":"2024-06-28T16:14:39.971321+0000","last_undegraded":"2024-06-28T16:14:39.971321+0000","last_fullsized":"2024-06-28T16:14:39.971321+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":400,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1786,"num_read_kb":1490,"num_write":759,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"260'328","reported_seq":972,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.700013+0000","last_change":"2024-06-28T15:00:15.427261+0000","last_active":"2024-06-28T16:14:34.700013+0000","last_peered":"2024-06-28T16:14:34.700013+0000","last_clean":"2024-06-28T16:14:34.700013+0000","last_became_active":"2024-06-28T15:00:15.419267+0000","last_became_peered":"2024-06-28T15:00:15.419267+0000","last_unstale":"2024-06-28T16:14:34.700013+0000","last_undegraded":"2024-06-28T16:14:34.700013+0000","last_fullsized":"2024-06-28T16:14:34.700013+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5413475,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":487,"num_read_kb":27459,"num_write":1288,"num_write_kb":48933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.802881+0000","last_change":"2024-06-28T15:00:19.386890+0000","last_active":"2024-06-28T16:14:41.802881+0000","last_peered":"2024-06-28T16:14:41.802881+0000","last_clean":"2024-06-28T16:14:41.802881+0000","last_became_active":"2024-06-28T15:00:19.385990+0000","last_became_peered":"2024-06-28T15:00:19.385990+0000","last_unstale":"2024-06-28T16:14:41.802881+0000","last_undegraded":"2024-06-28T16:14:41.802881+0000","last_fullsized":"2024-06-28T16:14:41.802881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"260'318","reported_seq":932,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.207650+0000","last_change":"2024-06-28T15:00:21.406717+0000","last_active":"2024-06-28T16:14:33.207650+0000","last_peered":"2024-06-28T16:14:33.207650+0000","last_clean":"2024-06-28T16:14:33.207650+0000","last_became_active":"2024-06-28T15:00:21.405045+0000","last_became_peered":"2024-06-28T15:00:21.405045+0000","last_unstale":"2024-06-28T16:14:33.207650+0000","last_undegraded":"2024-06-28T16:14:33.207650+0000","last_fullsized":"2024-06-28T16:14:33.207650+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48456,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":530,"num_read_kb":1162,"num_write":414,"num_write_kb":289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.144268+0000","last_change":"2024-06-28T16:14:50.144381+0000","last_active":"2024-06-28T16:14:50.144268+0000","last_peered":"2024-06-28T16:14:50.144268+0000","last_clean":"2024-06-28T16:14:50.144268+0000","last_became_active":"2024-06-28T15:00:23.411264+0000","last_became_peered":"2024-06-28T15:00:23.411264+0000","last_unstale":"2024-06-28T16:14:50.144268+0000","last_undegraded":"2024-06-28T16:14:50.144268+0000","last_fullsized":"2024-06-28T16:14:50.144268+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:50.144047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:50.144047+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.144047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:56:31.669987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"260'113","reported_seq":614,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.683239+0000","last_change":"2024-06-28T16:14:43.683376+0000","last_active":"2024-06-28T16:14:43.683239+0000","last_peered":"2024-06-28T16:14:43.683239+0000","last_clean":"2024-06-28T16:14:43.683239+0000","last_became_active":"2024-06-28T15:00:25.442010+0000","last_became_peered":"2024-06-28T15:00:25.442010+0000","last_unstale":"2024-06-28T16:14:43.683239+0000","last_undegraded":"2024-06-28T16:14:43.683239+0000","last_fullsized":"2024-06-28T16:14:43.683239+0000","mapping_epoch":29,"log_start":"260'101","ondisk_log_start":"260'101","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'113","last_scrub_stamp":"2024-06-28T16:14:43.682925+0000","last_deep_scrub":"260'113","last_deep_scrub_stamp":"2024-06-28T16:14:43.682925+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.682925+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":101,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:07:47.348656+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":638,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":85,"num_write":260,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"260'105","reported_seq":671,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.971513+0000","last_change":"2024-06-28T15:00:17.376172+0000","last_active":"2024-06-28T16:14:39.971513+0000","last_peered":"2024-06-28T16:14:39.971513+0000","last_clean":"2024-06-28T16:14:39.971513+0000","last_became_active":"2024-06-28T15:00:17.373396+0000","last_became_peered":"2024-06-28T15:00:17.373396+0000","last_unstale":"2024-06-28T16:14:39.971513+0000","last_undegraded":"2024-06-28T16:14:39.971513+0000","last_fullsized":"2024-06-28T16:14:39.971513+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":424,"num_read_kb":340,"num_write":155,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"260'307","reported_seq":955,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.734689+0000","last_change":"2024-06-28T15:00:15.412826+0000","last_active":"2024-06-28T16:14:34.734689+0000","last_peered":"2024-06-28T16:14:34.734689+0000","last_clean":"2024-06-28T16:14:34.734689+0000","last_became_active":"2024-06-28T15:00:15.412333+0000","last_became_peered":"2024-06-28T15:00:15.412333+0000","last_unstale":"2024-06-28T16:14:34.734689+0000","last_undegraded":"2024-06-28T16:14:34.734689+0000","last_fullsized":"2024-06-28T16:14:34.734689+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1451455,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":458,"num_read_kb":31119,"num_write":1276,"num_write_kb":43155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.188245+0000","last_change":"2024-06-28T16:14:51.188245+0000","last_active":"2024-06-28T16:14:51.188245+0000","last_peered":"2024-06-28T16:14:51.188245+0000","last_clean":"2024-06-28T16:14:51.188245+0000","last_became_active":"2024-06-28T15:00:19.386645+0000","last_became_peered":"2024-06-28T15:00:19.386645+0000","last_unstale":"2024-06-28T16:14:51.188245+0000","last_undegraded":"2024-06-28T16:14:51.188245+0000","last_fullsized":"2024-06-28T16:14:51.188245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.187878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.187878+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.187878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:53:35.500317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"260'289","reported_seq":874,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.978982+0000","last_change":"2024-06-28T15:00:21.395133+0000","last_active":"2024-06-28T16:14:32.978982+0000","last_peered":"2024-06-28T16:14:32.978982+0000","last_clean":"2024-06-28T16:14:32.978982+0000","last_became_active":"2024-06-28T15:00:21.394357+0000","last_became_peered":"2024-06-28T15:00:21.394357+0000","last_unstale":"2024-06-28T16:14:32.978982+0000","last_undegraded":"2024-06-28T16:14:32.978982+0000","last_fullsized":"2024-06-28T16:14:32.978982+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53697,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":496,"num_read_kb":1194,"num_write":404,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.135747+0000","last_change":"2024-06-28T16:14:59.135809+0000","last_active":"2024-06-28T16:14:59.135747+0000","last_peered":"2024-06-28T16:14:59.135747+0000","last_clean":"2024-06-28T16:14:59.135747+0000","last_became_active":"2024-06-28T15:00:23.406296+0000","last_became_peered":"2024-06-28T15:00:23.406296+0000","last_unstale":"2024-06-28T16:14:59.135747+0000","last_undegraded":"2024-06-28T16:14:59.135747+0000","last_fullsized":"2024-06-28T16:14:59.135747+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:59.135545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:59.135545+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.135545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:33:44.459217+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"260'68","reported_seq":637,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:06.711667+0000","last_change":"2024-06-28T15:00:17.397537+0000","last_active":"2024-06-28T16:14:06.711667+0000","last_peered":"2024-06-28T16:14:06.711667+0000","last_clean":"2024-06-28T16:14:06.711667+0000","last_became_active":"2024-06-28T15:00:17.396861+0000","last_became_peered":"2024-06-28T15:00:17.396861+0000","last_unstale":"2024-06-28T16:14:06.711667+0000","last_undegraded":"2024-06-28T16:14:06.711667+0000","last_fullsized":"2024-06-28T16:14:06.711667+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":279,"num_read_kb":229,"num_write":93,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"260'309","reported_seq":945,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.754063+0000","last_change":"2024-06-28T15:00:15.383453+0000","last_active":"2024-06-28T16:14:34.754063+0000","last_peered":"2024-06-28T16:14:34.754063+0000","last_clean":"2024-06-28T16:14:34.754063+0000","last_became_active":"2024-06-28T15:00:15.383008+0000","last_became_peered":"2024-06-28T15:00:15.383008+0000","last_unstale":"2024-06-28T16:14:34.754063+0000","last_undegraded":"2024-06-28T16:14:34.754063+0000","last_fullsized":"2024-06-28T16:14:34.754063+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2156401,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":459,"num_read_kb":6724,"num_write":1144,"num_write_kb":24207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":516,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.608430+0000","last_change":"2024-06-28T15:00:19.390462+0000","last_active":"2024-06-28T16:14:35.608430+0000","last_peered":"2024-06-28T16:14:35.608430+0000","last_clean":"2024-06-28T16:14:35.608430+0000","last_became_active":"2024-06-28T15:00:19.390111+0000","last_became_peered":"2024-06-28T15:00:19.390111+0000","last_unstale":"2024-06-28T16:14:35.608430+0000","last_undegraded":"2024-06-28T16:14:35.608430+0000","last_fullsized":"2024-06-28T16:14:35.608430+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"260'476","reported_seq":1135,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.999763+0000","last_change":"2024-06-28T16:14:54.999846+0000","last_active":"2024-06-28T16:14:54.999763+0000","last_peered":"2024-06-28T16:14:54.999763+0000","last_clean":"2024-06-28T16:14:54.999763+0000","last_became_active":"2024-06-28T15:00:21.398262+0000","last_became_peered":"2024-06-28T15:00:21.398262+0000","last_unstale":"2024-06-28T16:14:54.999763+0000","last_undegraded":"2024-06-28T16:14:54.999763+0000","last_fullsized":"2024-06-28T16:14:54.999763+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'476","last_scrub_stamp":"2024-06-28T16:14:54.999556+0000","last_deep_scrub":"260'476","last_deep_scrub_stamp":"2024-06-28T16:14:54.999556+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.999556+0000","objects_scrubbed":6,"log_size":76,"log_dups_size":400,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:46:08.548671+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86979,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":863,"num_read_kb":2070,"num_write":734,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24966,"num_omap_keys":48,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.607050+0000","last_change":"2024-06-28T16:14:53.607103+0000","last_active":"2024-06-28T16:14:53.607050+0000","last_peered":"2024-06-28T16:14:53.607050+0000","last_clean":"2024-06-28T16:14:53.607050+0000","last_became_active":"2024-06-28T15:00:23.410519+0000","last_became_peered":"2024-06-28T15:00:23.410519+0000","last_unstale":"2024-06-28T16:14:53.607050+0000","last_undegraded":"2024-06-28T16:14:53.607050+0000","last_fullsized":"2024-06-28T16:14:53.607050+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.606849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.606849+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.606849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:00:33.451749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"260'170","reported_seq":696,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.126852+0000","last_change":"2024-06-28T16:14:50.126982+0000","last_active":"2024-06-28T16:14:50.126852+0000","last_peered":"2024-06-28T16:14:50.126852+0000","last_clean":"2024-06-28T16:14:50.126852+0000","last_became_active":"2024-06-28T15:00:25.558426+0000","last_became_peered":"2024-06-28T15:00:25.558426+0000","last_unstale":"2024-06-28T16:14:50.126852+0000","last_undegraded":"2024-06-28T16:14:50.126852+0000","last_fullsized":"2024-06-28T16:14:50.126852+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'170","last_scrub_stamp":"2024-06-28T16:14:50.126565+0000","last_deep_scrub":"260'170","last_deep_scrub_stamp":"2024-06-28T16:14:50.126565+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.126565+0000","objects_scrubbed":3,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:28:56.261088+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":845,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":215,"num_read_kb":219,"num_write":377,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"260'158","reported_seq":788,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.608394+0000","last_change":"2024-06-28T15:00:17.395953+0000","last_active":"2024-06-28T16:14:35.608394+0000","last_peered":"2024-06-28T16:14:35.608394+0000","last_clean":"2024-06-28T16:14:35.608394+0000","last_became_active":"2024-06-28T15:00:17.395643+0000","last_became_peered":"2024-06-28T15:00:17.395643+0000","last_unstale":"2024-06-28T16:14:35.608394+0000","last_undegraded":"2024-06-28T16:14:35.608394+0000","last_fullsized":"2024-06-28T16:14:35.608394+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":100,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":654,"num_read_kb":561,"num_write":257,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"260'290","reported_seq":939,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.075407+0000","last_change":"2024-06-28T16:14:44.075499+0000","last_active":"2024-06-28T16:14:44.075407+0000","last_peered":"2024-06-28T16:14:44.075407+0000","last_clean":"2024-06-28T16:14:44.075407+0000","last_became_active":"2024-06-28T15:00:59.939485+0000","last_became_peered":"2024-06-28T15:00:59.939485+0000","last_unstale":"2024-06-28T16:14:44.075407+0000","last_undegraded":"2024-06-28T16:14:44.075407+0000","last_fullsized":"2024-06-28T16:14:44.075407+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'290","last_scrub_stamp":"2024-06-28T16:14:44.075189+0000","last_deep_scrub":"260'290","last_deep_scrub_stamp":"2024-06-28T16:14:44.075189+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.075189+0000","objects_scrubbed":3,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:23:04.302226+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":430,"num_read_kb":21907,"num_write":1034,"num_write_kb":35159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,6],"acting":[1,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.274518+0000","last_change":"2024-06-28T16:14:53.274570+0000","last_active":"2024-06-28T16:14:53.274518+0000","last_peered":"2024-06-28T16:14:53.274518+0000","last_clean":"2024-06-28T16:14:53.274518+0000","last_became_active":"2024-06-28T15:00:19.385068+0000","last_became_peered":"2024-06-28T15:00:19.385068+0000","last_unstale":"2024-06-28T16:14:53.274518+0000","last_undegraded":"2024-06-28T16:14:53.274518+0000","last_fullsized":"2024-06-28T16:14:53.274518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.274324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.274324+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.274324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:11:33.949188+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"260'348","reported_seq":955,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.056095+0000","last_change":"2024-06-28T15:00:21.401177+0000","last_active":"2024-06-28T16:14:33.056095+0000","last_peered":"2024-06-28T16:14:33.056095+0000","last_clean":"2024-06-28T16:14:33.056095+0000","last_became_active":"2024-06-28T15:00:21.400480+0000","last_became_peered":"2024-06-28T15:00:21.400480+0000","last_unstale":"2024-06-28T16:14:33.056095+0000","last_undegraded":"2024-06-28T16:14:33.056095+0000","last_fullsized":"2024-06-28T16:14:33.056095+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":300,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47708,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":581,"num_read_kb":1249,"num_write":480,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":499,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.117041+0000","last_change":"2024-06-28T15:00:23.418653+0000","last_active":"2024-06-28T16:12:55.117041+0000","last_peered":"2024-06-28T16:12:55.117041+0000","last_clean":"2024-06-28T16:12:55.117041+0000","last_became_active":"2024-06-28T15:00:23.417580+0000","last_became_peered":"2024-06-28T15:00:23.417580+0000","last_unstale":"2024-06-28T16:12:55.117041+0000","last_undegraded":"2024-06-28T16:12:55.117041+0000","last_fullsized":"2024-06-28T16:12:55.117041+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"260'170","reported_seq":678,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:08.583996+0000","last_change":"2024-06-28T15:00:25.431039+0000","last_active":"2024-06-28T16:14:08.583996+0000","last_peered":"2024-06-28T16:14:08.583996+0000","last_clean":"2024-06-28T16:14:08.583996+0000","last_became_active":"2024-06-28T15:00:25.430299+0000","last_became_peered":"2024-06-28T15:00:25.430299+0000","last_unstale":"2024-06-28T16:14:08.583996+0000","last_undegraded":"2024-06-28T16:14:08.583996+0000","last_fullsized":"2024-06-28T16:14:08.583996+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:24.376399+0000","last_clean_scrub_stamp":"2024-06-28T15:00:24.376399+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2035,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":193,"num_read_kb":191,"num_write":477,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"260'315","reported_seq":1058,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.312305+0000","last_change":"2024-06-28T15:00:17.383684+0000","last_active":"2024-06-28T16:12:50.312305+0000","last_peered":"2024-06-28T16:12:50.312305+0000","last_clean":"2024-06-28T16:12:50.312305+0000","last_became_active":"2024-06-28T15:00:17.383284+0000","last_became_peered":"2024-06-28T15:00:17.383284+0000","last_unstale":"2024-06-28T16:12:50.312305+0000","last_undegraded":"2024-06-28T16:12:50.312305+0000","last_fullsized":"2024-06-28T16:12:50.312305+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1515,"num_read_kb":1335,"num_write":548,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"260'347","reported_seq":1029,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.726870+0000","last_change":"2024-06-28T15:00:15.406314+0000","last_active":"2024-06-28T16:14:34.726870+0000","last_peered":"2024-06-28T16:14:34.726870+0000","last_clean":"2024-06-28T16:14:34.726870+0000","last_became_active":"2024-06-28T15:00:15.405499+0000","last_became_peered":"2024-06-28T15:00:15.405499+0000","last_unstale":"2024-06-28T16:14:34.726870+0000","last_undegraded":"2024-06-28T16:14:34.726870+0000","last_fullsized":"2024-06-28T16:14:34.726870+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17426850,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":585,"num_read_kb":18043,"num_write":1303,"num_write_kb":33783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.108070+0000","last_change":"2024-06-28T15:00:19.381603+0000","last_active":"2024-06-28T16:12:50.108070+0000","last_peered":"2024-06-28T16:12:50.108070+0000","last_clean":"2024-06-28T16:12:50.108070+0000","last_became_active":"2024-06-28T15:00:19.378852+0000","last_became_peered":"2024-06-28T15:00:19.378852+0000","last_unstale":"2024-06-28T16:12:50.108070+0000","last_undegraded":"2024-06-28T16:12:50.108070+0000","last_fullsized":"2024-06-28T16:12:50.108070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"260'279","reported_seq":881,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.315633+0000","last_change":"2024-06-28T15:00:21.481610+0000","last_active":"2024-06-28T16:14:34.315633+0000","last_peered":"2024-06-28T16:14:34.315633+0000","last_clean":"2024-06-28T16:14:34.315633+0000","last_became_active":"2024-06-28T15:00:21.480868+0000","last_became_peered":"2024-06-28T15:00:21.480868+0000","last_unstale":"2024-06-28T16:14:34.315633+0000","last_undegraded":"2024-06-28T16:14:34.315633+0000","last_fullsized":"2024-06-28T16:14:34.315633+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":200,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40850,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":446,"num_read_kb":956,"num_write":348,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.074074+0000","last_change":"2024-06-28T15:00:23.401297+0000","last_active":"2024-06-28T16:14:39.074074+0000","last_peered":"2024-06-28T16:14:39.074074+0000","last_clean":"2024-06-28T16:14:39.074074+0000","last_became_active":"2024-06-28T15:00:23.400417+0000","last_became_peered":"2024-06-28T15:00:23.400417+0000","last_unstale":"2024-06-28T16:14:39.074074+0000","last_undegraded":"2024-06-28T16:14:39.074074+0000","last_fullsized":"2024-06-28T16:14:39.074074+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"260'121","reported_seq":630,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.687607+0000","last_change":"2024-06-28T16:14:50.687682+0000","last_active":"2024-06-28T16:14:50.687607+0000","last_peered":"2024-06-28T16:14:50.687607+0000","last_clean":"2024-06-28T16:14:50.687607+0000","last_became_active":"2024-06-28T15:00:25.438176+0000","last_became_peered":"2024-06-28T15:00:25.438176+0000","last_unstale":"2024-06-28T16:14:50.687607+0000","last_undegraded":"2024-06-28T16:14:50.687607+0000","last_fullsized":"2024-06-28T16:14:50.687607+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'121","last_scrub_stamp":"2024-06-28T16:14:50.687377+0000","last_deep_scrub":"260'121","last_deep_scrub_stamp":"2024-06-28T16:14:50.687377+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.687377+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:06:30.493792+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":120,"num_read_kb":111,"num_write":277,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"260'74","reported_seq":656,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.286469+0000","last_change":"2024-06-28T15:00:17.403175+0000","last_active":"2024-06-28T16:12:50.286469+0000","last_peered":"2024-06-28T16:12:50.286469+0000","last_clean":"2024-06-28T16:12:50.286469+0000","last_became_active":"2024-06-28T15:00:17.402735+0000","last_became_peered":"2024-06-28T15:00:17.402735+0000","last_unstale":"2024-06-28T16:12:50.286469+0000","last_undegraded":"2024-06-28T16:12:50.286469+0000","last_fullsized":"2024-06-28T16:12:50.286469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":299,"num_read_kb":249,"num_write":100,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"260'336","reported_seq":1059,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.724393+0000","last_change":"2024-06-28T15:00:15.415658+0000","last_active":"2024-06-28T16:14:34.724393+0000","last_peered":"2024-06-28T16:14:34.724393+0000","last_clean":"2024-06-28T16:14:34.724393+0000","last_became_active":"2024-06-28T15:00:15.415197+0000","last_became_peered":"2024-06-28T15:00:15.415197+0000","last_unstale":"2024-06-28T16:14:34.724393+0000","last_undegraded":"2024-06-28T16:14:34.724393+0000","last_fullsized":"2024-06-28T16:14:34.724393+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3732560,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":648,"num_read_kb":6210,"num_write":1163,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.967001+0000","last_change":"2024-06-28T16:14:51.967053+0000","last_active":"2024-06-28T16:14:51.967001+0000","last_peered":"2024-06-28T16:14:51.967001+0000","last_clean":"2024-06-28T16:14:51.967001+0000","last_became_active":"2024-06-28T15:00:19.378270+0000","last_became_peered":"2024-06-28T15:00:19.378270+0000","last_unstale":"2024-06-28T16:14:51.967001+0000","last_undegraded":"2024-06-28T16:14:51.967001+0000","last_fullsized":"2024-06-28T16:14:51.967001+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.966801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.966801+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.966801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:16:26.209301+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"260'107","reported_seq":662,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.993632+0000","last_change":"2024-06-28T16:14:51.993706+0000","last_active":"2024-06-28T16:14:51.993632+0000","last_peered":"2024-06-28T16:14:51.993632+0000","last_clean":"2024-06-28T16:14:51.993632+0000","last_became_active":"2024-06-28T15:00:21.484352+0000","last_became_peered":"2024-06-28T15:00:21.484352+0000","last_unstale":"2024-06-28T16:14:51.993632+0000","last_undegraded":"2024-06-28T16:14:51.993632+0000","last_fullsized":"2024-06-28T16:14:51.993632+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'107","last_scrub_stamp":"2024-06-28T16:14:51.993438+0000","last_deep_scrub":"260'107","last_deep_scrub_stamp":"2024-06-28T16:14:51.993438+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.993438+0000","objects_scrubbed":6,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:28:06.528863+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":124,"num_read_kb":119,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1780,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"260'515","reported_seq":2254,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.146189+0000","last_change":"2024-06-28T16:14:48.146256+0000","last_active":"2024-06-28T16:14:48.146189+0000","last_peered":"2024-06-28T16:14:48.146189+0000","last_clean":"2024-06-28T16:14:48.146189+0000","last_became_active":"2024-06-28T15:00:23.407490+0000","last_became_peered":"2024-06-28T15:00:23.407490+0000","last_unstale":"2024-06-28T16:14:48.146189+0000","last_undegraded":"2024-06-28T16:14:48.146189+0000","last_fullsized":"2024-06-28T16:14:48.146189+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'515","last_scrub_stamp":"2024-06-28T16:14:48.145991+0000","last_deep_scrub":"260'515","last_deep_scrub_stamp":"2024-06-28T16:14:48.145991+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.145991+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:10:57.232491+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"260'103","reported_seq":616,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.637702+0000","last_change":"2024-06-28T16:14:55.637768+0000","last_active":"2024-06-28T16:14:55.637702+0000","last_peered":"2024-06-28T16:14:55.637702+0000","last_clean":"2024-06-28T16:14:55.637702+0000","last_became_active":"2024-06-28T15:00:25.445018+0000","last_became_peered":"2024-06-28T15:00:25.445018+0000","last_unstale":"2024-06-28T16:14:55.637702+0000","last_undegraded":"2024-06-28T16:14:55.637702+0000","last_fullsized":"2024-06-28T16:14:55.637702+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'103","last_scrub_stamp":"2024-06-28T16:14:55.637507+0000","last_deep_scrub":"260'103","last_deep_scrub_stamp":"2024-06-28T16:14:55.637507+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.637507+0000","objects_scrubbed":1,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:19:26.856020+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":117,"num_read_kb":111,"num_write":241,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"260'78","reported_seq":655,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.107990+0000","last_change":"2024-06-28T15:00:17.394823+0000","last_active":"2024-06-28T16:12:50.107990+0000","last_peered":"2024-06-28T16:12:50.107990+0000","last_clean":"2024-06-28T16:12:50.107990+0000","last_became_active":"2024-06-28T15:00:17.387086+0000","last_became_peered":"2024-06-28T15:00:17.387086+0000","last_unstale":"2024-06-28T16:12:50.107990+0000","last_undegraded":"2024-06-28T16:12:50.107990+0000","last_fullsized":"2024-06-28T16:12:50.107990+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":333,"num_read_kb":275,"num_write":113,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"260'328","reported_seq":969,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.118128+0000","last_change":"2024-06-28T16:14:44.118252+0000","last_active":"2024-06-28T16:14:44.118128+0000","last_peered":"2024-06-28T16:14:44.118128+0000","last_clean":"2024-06-28T16:14:44.118128+0000","last_became_active":"2024-06-28T15:00:59.941915+0000","last_became_peered":"2024-06-28T15:00:59.941915+0000","last_unstale":"2024-06-28T16:14:44.118128+0000","last_undegraded":"2024-06-28T16:14:44.118128+0000","last_fullsized":"2024-06-28T16:14:44.118128+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'328","last_scrub_stamp":"2024-06-28T16:14:44.117934+0000","last_deep_scrub":"260'328","last_deep_scrub_stamp":"2024-06-28T16:14:44.117934+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.117934+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:54:20.388393+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5506084,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":497,"num_read_kb":57892,"num_write":1218,"num_write_kb":47404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.115108+0000","last_change":"2024-06-28T15:00:19.391494+0000","last_active":"2024-06-28T16:12:50.115108+0000","last_peered":"2024-06-28T16:12:50.115108+0000","last_clean":"2024-06-28T16:12:50.115108+0000","last_became_active":"2024-06-28T15:00:19.390613+0000","last_became_peered":"2024-06-28T15:00:19.390613+0000","last_unstale":"2024-06-28T16:12:50.115108+0000","last_undegraded":"2024-06-28T16:12:50.115108+0000","last_fullsized":"2024-06-28T16:12:50.115108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"260'324","reported_seq":933,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.760414+0000","last_change":"2024-06-28T15:00:21.392375+0000","last_active":"2024-06-28T16:14:34.760414+0000","last_peered":"2024-06-28T16:14:34.760414+0000","last_clean":"2024-06-28T16:14:34.760414+0000","last_became_active":"2024-06-28T15:00:21.391631+0000","last_became_peered":"2024-06-28T15:00:21.391631+0000","last_unstale":"2024-06-28T16:14:34.760414+0000","last_undegraded":"2024-06-28T16:14:34.760414+0000","last_fullsized":"2024-06-28T16:14:34.760414+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43414,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":516,"num_read_kb":1105,"num_write":400,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:55.583987+0000","last_change":"2024-06-28T15:00:23.405246+0000","last_active":"2024-06-28T16:12:55.583987+0000","last_peered":"2024-06-28T16:12:55.583987+0000","last_clean":"2024-06-28T16:12:55.583987+0000","last_became_active":"2024-06-28T15:00:23.404312+0000","last_became_peered":"2024-06-28T15:00:23.404312+0000","last_unstale":"2024-06-28T16:12:55.583987+0000","last_undegraded":"2024-06-28T16:12:55.583987+0000","last_fullsized":"2024-06-28T16:12:55.583987+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:22.364686+0000","last_clean_scrub_stamp":"2024-06-28T15:00:22.364686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"260'136","reported_seq":656,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.687314+0000","last_change":"2024-06-28T16:14:57.687390+0000","last_active":"2024-06-28T16:14:57.687314+0000","last_peered":"2024-06-28T16:14:57.687314+0000","last_clean":"2024-06-28T16:14:57.687314+0000","last_became_active":"2024-06-28T15:00:25.424826+0000","last_became_peered":"2024-06-28T15:00:25.424826+0000","last_unstale":"2024-06-28T16:14:57.687314+0000","last_undegraded":"2024-06-28T16:14:57.687314+0000","last_fullsized":"2024-06-28T16:14:57.687314+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'136","last_scrub_stamp":"2024-06-28T16:14:57.687124+0000","last_deep_scrub":"260'136","last_deep_scrub_stamp":"2024-06-28T16:14:57.687124+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.687124+0000","objects_scrubbed":5,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:05:03.591761+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":159,"num_read_kb":144,"num_write":309,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"260'118","reported_seq":692,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.107888+0000","last_change":"2024-06-28T15:00:17.387490+0000","last_active":"2024-06-28T16:12:50.107888+0000","last_peered":"2024-06-28T16:12:50.107888+0000","last_clean":"2024-06-28T16:12:50.107888+0000","last_became_active":"2024-06-28T15:00:17.386756+0000","last_became_peered":"2024-06-28T15:00:17.386756+0000","last_unstale":"2024-06-28T16:12:50.107888+0000","last_undegraded":"2024-06-28T16:12:50.107888+0000","last_fullsized":"2024-06-28T16:12:50.107888+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":469,"num_read_kb":378,"num_write":166,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"260'293","reported_seq":973,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.180637+0000","last_change":"2024-06-28T16:14:48.180832+0000","last_active":"2024-06-28T16:14:48.180637+0000","last_peered":"2024-06-28T16:14:48.180637+0000","last_clean":"2024-06-28T16:14:48.180637+0000","last_became_active":"2024-06-28T15:00:59.943670+0000","last_became_peered":"2024-06-28T15:00:59.943670+0000","last_unstale":"2024-06-28T16:14:48.180637+0000","last_undegraded":"2024-06-28T16:14:48.180637+0000","last_fullsized":"2024-06-28T16:14:48.180637+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'293","last_scrub_stamp":"2024-06-28T16:14:48.180366+0000","last_deep_scrub":"260'293","last_deep_scrub_stamp":"2024-06-28T16:14:48.180366+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.180366+0000","objects_scrubbed":3,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:11:28.293389+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306233,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":513,"num_read_kb":66858,"num_write":1091,"num_write_kb":26064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.100827+0000","last_change":"2024-06-28T15:00:19.400037+0000","last_active":"2024-06-28T16:12:50.100827+0000","last_peered":"2024-06-28T16:12:50.100827+0000","last_clean":"2024-06-28T16:12:50.100827+0000","last_became_active":"2024-06-28T15:00:19.398245+0000","last_became_peered":"2024-06-28T15:00:19.398245+0000","last_unstale":"2024-06-28T16:12:50.100827+0000","last_undegraded":"2024-06-28T16:12:50.100827+0000","last_fullsized":"2024-06-28T16:12:50.100827+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"260'297","reported_seq":888,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:32.434499+0000","last_change":"2024-06-28T15:00:21.406496+0000","last_active":"2024-06-28T16:14:32.434499+0000","last_peered":"2024-06-28T16:14:32.434499+0000","last_clean":"2024-06-28T16:14:32.434499+0000","last_became_active":"2024-06-28T15:00:21.404701+0000","last_became_peered":"2024-06-28T15:00:21.404701+0000","last_unstale":"2024-06-28T16:14:32.434499+0000","last_undegraded":"2024-06-28T16:14:32.434499+0000","last_fullsized":"2024-06-28T16:14:32.434499+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44354,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":486,"num_read_kb":1147,"num_write":394,"num_write_kb":305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.234362+0000","last_change":"2024-06-28T16:14:52.234419+0000","last_active":"2024-06-28T16:14:52.234362+0000","last_peered":"2024-06-28T16:14:52.234362+0000","last_clean":"2024-06-28T16:14:52.234362+0000","last_became_active":"2024-06-28T15:00:23.415531+0000","last_became_peered":"2024-06-28T15:00:23.415531+0000","last_unstale":"2024-06-28T16:14:52.234362+0000","last_undegraded":"2024-06-28T16:14:52.234362+0000","last_fullsized":"2024-06-28T16:14:52.234362+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:52.234179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:52.234179+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.234179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:09:52.052832+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"260'168","reported_seq":721,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.154622+0000","last_change":"2024-06-28T16:14:52.154699+0000","last_active":"2024-06-28T16:14:52.154622+0000","last_peered":"2024-06-28T16:14:52.154622+0000","last_clean":"2024-06-28T16:14:52.154622+0000","last_became_active":"2024-06-28T15:00:25.553724+0000","last_became_peered":"2024-06-28T15:00:25.553724+0000","last_unstale":"2024-06-28T16:14:52.154622+0000","last_undegraded":"2024-06-28T16:14:52.154622+0000","last_fullsized":"2024-06-28T16:14:52.154622+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'168","last_scrub_stamp":"2024-06-28T16:14:52.154408+0000","last_deep_scrub":"260'168","last_deep_scrub_stamp":"2024-06-28T16:14:52.154408+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.154408+0000","objects_scrubbed":2,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:26:53.092282+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":555,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":217,"num_write":397,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"260'143","reported_seq":807,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:00.314710+0000","last_change":"2024-06-28T15:00:17.383713+0000","last_active":"2024-06-28T16:13:00.314710+0000","last_peered":"2024-06-28T16:13:00.314710+0000","last_clean":"2024-06-28T16:13:00.314710+0000","last_became_active":"2024-06-28T15:00:17.383224+0000","last_became_peered":"2024-06-28T15:00:17.383224+0000","last_unstale":"2024-06-28T16:13:00.314710+0000","last_undegraded":"2024-06-28T16:13:00.314710+0000","last_fullsized":"2024-06-28T16:13:00.314710+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":649,"num_read_kb":545,"num_write":214,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"260'301","reported_seq":1067,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.729020+0000","last_change":"2024-06-28T15:00:15.424150+0000","last_active":"2024-06-28T16:14:34.729020+0000","last_peered":"2024-06-28T16:14:34.729020+0000","last_clean":"2024-06-28T16:14:34.729020+0000","last_became_active":"2024-06-28T15:00:15.423813+0000","last_became_peered":"2024-06-28T15:00:15.423813+0000","last_unstale":"2024-06-28T16:14:34.729020+0000","last_undegraded":"2024-06-28T16:14:34.729020+0000","last_fullsized":"2024-06-28T16:14:34.729020+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5724260,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":754,"num_read_kb":37474,"num_write":1107,"num_write_kb":31583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.107725+0000","last_change":"2024-06-28T15:00:19.384145+0000","last_active":"2024-06-28T16:12:50.107725+0000","last_peered":"2024-06-28T16:12:50.107725+0000","last_clean":"2024-06-28T16:12:50.107725+0000","last_became_active":"2024-06-28T15:00:19.383307+0000","last_became_peered":"2024-06-28T15:00:19.383307+0000","last_unstale":"2024-06-28T16:12:50.107725+0000","last_undegraded":"2024-06-28T16:12:50.107725+0000","last_fullsized":"2024-06-28T16:12:50.107725+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"260'286","reported_seq":858,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.177400+0000","last_change":"2024-06-28T15:00:21.396994+0000","last_active":"2024-06-28T16:14:34.177400+0000","last_peered":"2024-06-28T16:14:34.177400+0000","last_clean":"2024-06-28T16:14:34.177400+0000","last_became_active":"2024-06-28T15:00:21.396257+0000","last_became_peered":"2024-06-28T15:00:21.396257+0000","last_unstale":"2024-06-28T16:14:34.177400+0000","last_undegraded":"2024-06-28T16:14:34.177400+0000","last_fullsized":"2024-06-28T16:14:34.177400+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43992,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":439,"num_read_kb":1078,"num_write":366,"num_write_kb":297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.973587+0000","last_change":"2024-06-28T16:14:47.973645+0000","last_active":"2024-06-28T16:14:47.973587+0000","last_peered":"2024-06-28T16:14:47.973587+0000","last_clean":"2024-06-28T16:14:47.973587+0000","last_became_active":"2024-06-28T15:00:23.409107+0000","last_became_peered":"2024-06-28T15:00:23.409107+0000","last_unstale":"2024-06-28T16:14:47.973587+0000","last_undegraded":"2024-06-28T16:14:47.973587+0000","last_fullsized":"2024-06-28T16:14:47.973587+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:47.973365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:47.973365+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.973365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:58:08.598920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"260'149","reported_seq":890,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.717743+0000","last_change":"2024-06-28T16:14:58.717817+0000","last_active":"2024-06-28T16:14:58.717743+0000","last_peered":"2024-06-28T16:14:58.717743+0000","last_clean":"2024-06-28T16:14:58.717743+0000","last_became_active":"2024-06-28T15:00:25.424376+0000","last_became_peered":"2024-06-28T15:00:25.424376+0000","last_unstale":"2024-06-28T16:14:58.717743+0000","last_undegraded":"2024-06-28T16:14:58.717743+0000","last_fullsized":"2024-06-28T16:14:58.717743+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'149","last_scrub_stamp":"2024-06-28T16:14:58.717559+0000","last_deep_scrub":"260'149","last_deep_scrub_stamp":"2024-06-28T16:14:58.717559+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.717559+0000","objects_scrubbed":5,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:45:24.602655+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":398,"num_read_kb":429,"num_write":339,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1200,"num_omap_keys":6,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"260'112","reported_seq":700,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:12:50.107774+0000","last_change":"2024-06-28T15:00:17.403746+0000","last_active":"2024-06-28T16:12:50.107774+0000","last_peered":"2024-06-28T16:12:50.107774+0000","last_clean":"2024-06-28T16:12:50.107774+0000","last_became_active":"2024-06-28T15:00:17.403007+0000","last_became_peered":"2024-06-28T15:00:17.403007+0000","last_unstale":"2024-06-28T16:12:50.107774+0000","last_undegraded":"2024-06-28T16:12:50.107774+0000","last_fullsized":"2024-06-28T16:12:50.107774+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":478,"num_read_kb":386,"num_write":182,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"260'379","reported_seq":1047,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.657167+0000","last_change":"2024-06-28T15:00:15.415976+0000","last_active":"2024-06-28T16:14:34.657167+0000","last_peered":"2024-06-28T16:14:34.657167+0000","last_clean":"2024-06-28T16:14:34.657167+0000","last_became_active":"2024-06-28T15:00:15.414969+0000","last_became_peered":"2024-06-28T15:00:15.414969+0000","last_unstale":"2024-06-28T16:14:34.657167+0000","last_undegraded":"2024-06-28T16:14:34.657167+0000","last_fullsized":"2024-06-28T16:14:34.657167+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":300,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1604505,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":539,"num_read_kb":3182,"num_write":1321,"num_write_kb":13350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"260'2","reported_seq":518,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.947057+0000","last_change":"2024-06-28T16:14:52.947126+0000","last_active":"2024-06-28T16:14:52.947057+0000","last_peered":"2024-06-28T16:14:52.947057+0000","last_clean":"2024-06-28T16:14:52.947057+0000","last_became_active":"2024-06-28T15:00:19.403344+0000","last_became_peered":"2024-06-28T15:00:19.403344+0000","last_unstale":"2024-06-28T16:14:52.947057+0000","last_undegraded":"2024-06-28T16:14:52.947057+0000","last_fullsized":"2024-06-28T16:14:52.947057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:52.946822+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:52.946822+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.946822+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:53:37.784932+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"260'297","reported_seq":906,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.943726+0000","last_change":"2024-06-28T16:14:55.943802+0000","last_active":"2024-06-28T16:14:55.943726+0000","last_peered":"2024-06-28T16:14:55.943726+0000","last_clean":"2024-06-28T16:14:55.943726+0000","last_became_active":"2024-06-28T15:00:21.387539+0000","last_became_peered":"2024-06-28T15:00:21.387539+0000","last_unstale":"2024-06-28T16:14:55.943726+0000","last_undegraded":"2024-06-28T16:14:55.943726+0000","last_fullsized":"2024-06-28T16:14:55.943726+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'297","last_scrub_stamp":"2024-06-28T16:14:55.943503+0000","last_deep_scrub":"260'297","last_deep_scrub_stamp":"2024-06-28T16:14:55.943503+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.943503+0000","objects_scrubbed":5,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:01:20.919128+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44825,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":494,"num_read_kb":1086,"num_write":388,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.070823+0000","last_change":"2024-06-28T16:14:56.070879+0000","last_active":"2024-06-28T16:14:56.070823+0000","last_peered":"2024-06-28T16:14:56.070823+0000","last_clean":"2024-06-28T16:14:56.070823+0000","last_became_active":"2024-06-28T15:00:23.416363+0000","last_became_peered":"2024-06-28T15:00:23.416363+0000","last_unstale":"2024-06-28T16:14:56.070823+0000","last_undegraded":"2024-06-28T16:14:56.070823+0000","last_fullsized":"2024-06-28T16:14:56.070823+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.070609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.070609+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.070609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:44:31.199646+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"260'129","reported_seq":635,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.940716+0000","last_change":"2024-06-28T16:14:43.940870+0000","last_active":"2024-06-28T16:14:43.940716+0000","last_peered":"2024-06-28T16:14:43.940716+0000","last_clean":"2024-06-28T16:14:43.940716+0000","last_became_active":"2024-06-28T15:00:25.428959+0000","last_became_peered":"2024-06-28T15:00:25.428959+0000","last_unstale":"2024-06-28T16:14:43.940716+0000","last_undegraded":"2024-06-28T16:14:43.940716+0000","last_fullsized":"2024-06-28T16:14:43.940716+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'129","last_scrub_stamp":"2024-06-28T16:14:43.940462+0000","last_deep_scrub":"260'129","last_deep_scrub_stamp":"2024-06-28T16:14:43.940462+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.940462+0000","objects_scrubbed":3,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:17:21.534335+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":814,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":97,"num_write":299,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"260'175","reported_seq":771,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.005191+0000","last_change":"2024-06-28T16:14:56.005265+0000","last_active":"2024-06-28T16:14:56.005191+0000","last_peered":"2024-06-28T16:14:56.005191+0000","last_clean":"2024-06-28T16:14:56.005191+0000","last_became_active":"2024-06-28T15:00:17.406872+0000","last_became_peered":"2024-06-28T15:00:17.406872+0000","last_unstale":"2024-06-28T16:14:56.005191+0000","last_undegraded":"2024-06-28T16:14:56.005191+0000","last_fullsized":"2024-06-28T16:14:56.005191+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'175","last_scrub_stamp":"2024-06-28T16:14:56.004971+0000","last_deep_scrub":"260'175","last_deep_scrub_stamp":"2024-06-28T16:14:56.004971+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.004971+0000","objects_scrubbed":1,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:44:14.471029+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":660,"num_read_kb":511,"num_write":250,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7564,"num_omap_keys":32,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"260'285","reported_seq":910,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.698690+0000","last_change":"2024-06-28T15:00:15.409170+0000","last_active":"2024-06-28T16:14:34.698690+0000","last_peered":"2024-06-28T16:14:34.698690+0000","last_clean":"2024-06-28T16:14:34.698690+0000","last_became_active":"2024-06-28T15:00:15.408337+0000","last_became_peered":"2024-06-28T15:00:15.408337+0000","last_unstale":"2024-06-28T16:14:34.698690+0000","last_undegraded":"2024-06-28T16:14:34.698690+0000","last_fullsized":"2024-06-28T16:14:34.698690+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":404,"num_read_kb":33775,"num_write":1000,"num_write_kb":10945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"260'72","reported_seq":683,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.969785+0000","last_change":"2024-06-28T16:14:59.969851+0000","last_active":"2024-06-28T16:14:59.969785+0000","last_peered":"2024-06-28T16:14:59.969785+0000","last_clean":"2024-06-28T16:14:59.969785+0000","last_became_active":"2024-06-28T15:00:17.403938+0000","last_became_peered":"2024-06-28T15:00:17.403938+0000","last_unstale":"2024-06-28T16:14:59.969785+0000","last_undegraded":"2024-06-28T16:14:59.969785+0000","last_fullsized":"2024-06-28T16:14:59.969785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'72","last_scrub_stamp":"2024-06-28T16:14:59.969594+0000","last_deep_scrub":"260'72","last_deep_scrub_stamp":"2024-06-28T16:14:59.969594+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.969594+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:54:37.592959+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":319,"num_read_kb":277,"num_write":96,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"260'322","reported_seq":1004,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.266941+0000","last_change":"2024-06-28T16:14:57.267065+0000","last_active":"2024-06-28T16:14:57.266941+0000","last_peered":"2024-06-28T16:14:57.266941+0000","last_clean":"2024-06-28T16:14:57.266941+0000","last_became_active":"2024-06-28T15:00:15.420261+0000","last_became_peered":"2024-06-28T15:00:15.420261+0000","last_unstale":"2024-06-28T16:14:57.266941+0000","last_undegraded":"2024-06-28T16:14:57.266941+0000","last_fullsized":"2024-06-28T16:14:57.266941+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'322","last_scrub_stamp":"2024-06-28T16:14:57.266750+0000","last_deep_scrub":"260'322","last_deep_scrub_stamp":"2024-06-28T16:14:57.266750+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.266750+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:20:49.339058+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262189,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":529,"num_read_kb":23662,"num_write":1259,"num_write_kb":28174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"260'38","reported_seq":597,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.116331+0000","last_change":"2024-06-28T15:00:17.394654+0000","last_active":"2024-06-28T16:14:00.116331+0000","last_peered":"2024-06-28T16:14:00.116331+0000","last_clean":"2024-06-28T16:14:00.116331+0000","last_became_active":"2024-06-28T15:00:17.393889+0000","last_became_peered":"2024-06-28T15:00:17.393889+0000","last_unstale":"2024-06-28T16:14:00.116331+0000","last_undegraded":"2024-06-28T16:14:00.116331+0000","last_fullsized":"2024-06-28T16:14:00.116331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":152,"num_write":58,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"260'358","reported_seq":1000,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.755208+0000","last_change":"2024-06-28T15:00:15.426603+0000","last_active":"2024-06-28T16:14:34.755208+0000","last_peered":"2024-06-28T16:14:34.755208+0000","last_clean":"2024-06-28T16:14:34.755208+0000","last_became_active":"2024-06-28T15:00:15.418773+0000","last_became_peered":"2024-06-28T15:00:15.418773+0000","last_unstale":"2024-06-28T16:14:34.755208+0000","last_undegraded":"2024-06-28T16:14:34.755208+0000","last_fullsized":"2024-06-28T16:14:34.755208+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8714448,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":553,"num_read_kb":16492,"num_write":1257,"num_write_kb":37641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"260'142","reported_seq":788,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.130808+0000","last_change":"2024-06-28T15:00:17.398608+0000","last_active":"2024-06-28T16:14:00.130808+0000","last_peered":"2024-06-28T16:14:00.130808+0000","last_clean":"2024-06-28T16:14:00.130808+0000","last_became_active":"2024-06-28T15:00:17.398357+0000","last_became_peered":"2024-06-28T15:00:17.398357+0000","last_unstale":"2024-06-28T16:14:00.130808+0000","last_undegraded":"2024-06-28T16:14:00.130808+0000","last_fullsized":"2024-06-28T16:14:00.130808+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":623,"num_read_kb":529,"num_write":221,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"260'332","reported_seq":977,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.723568+0000","last_change":"2024-06-28T15:00:15.391599+0000","last_active":"2024-06-28T16:14:34.723568+0000","last_peered":"2024-06-28T16:14:34.723568+0000","last_clean":"2024-06-28T16:14:34.723568+0000","last_became_active":"2024-06-28T15:00:15.391150+0000","last_became_peered":"2024-06-28T15:00:15.391150+0000","last_unstale":"2024-06-28T16:14:34.723568+0000","last_undegraded":"2024-06-28T16:14:34.723568+0000","last_fullsized":"2024-06-28T16:14:34.723568+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":300,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1605357,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":453,"num_read_kb":84462,"num_write":1265,"num_write_kb":41774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"260'192","reported_seq":798,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.132066+0000","last_change":"2024-06-28T16:14:42.132111+0000","last_active":"2024-06-28T16:14:42.132066+0000","last_peered":"2024-06-28T16:14:42.132066+0000","last_clean":"2024-06-28T16:14:42.132066+0000","last_became_active":"2024-06-28T15:00:59.806725+0000","last_became_peered":"2024-06-28T15:00:59.806725+0000","last_unstale":"2024-06-28T16:14:42.132066+0000","last_undegraded":"2024-06-28T16:14:42.132066+0000","last_fullsized":"2024-06-28T16:14:42.132066+0000","mapping_epoch":32,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'192","last_scrub_stamp":"2024-06-28T16:14:42.131976+0000","last_deep_scrub":"260'192","last_deep_scrub_stamp":"2024-06-28T16:14:42.131976+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.131976+0000","objects_scrubbed":1,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:22:19.905274+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":759,"num_read_kb":602,"num_write":307,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"260'334","reported_seq":999,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.696329+0000","last_change":"2024-06-28T15:00:15.383729+0000","last_active":"2024-06-28T16:14:34.696329+0000","last_peered":"2024-06-28T16:14:34.696329+0000","last_clean":"2024-06-28T16:14:34.696329+0000","last_became_active":"2024-06-28T15:00:15.383350+0000","last_became_peered":"2024-06-28T15:00:15.383350+0000","last_unstale":"2024-06-28T16:14:34.696329+0000","last_undegraded":"2024-06-28T16:14:34.696329+0000","last_fullsized":"2024-06-28T16:14:34.696329+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263219,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":502,"num_read_kb":41336,"num_write":1201,"num_write_kb":38478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0,7],"acting":[3,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"260'398","reported_seq":1007,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.938053+0000","last_change":"2024-06-28T16:14:58.938126+0000","last_active":"2024-06-28T16:14:58.938053+0000","last_peered":"2024-06-28T16:14:58.938053+0000","last_clean":"2024-06-28T16:14:58.938053+0000","last_became_active":"2024-06-28T15:00:17.404844+0000","last_became_peered":"2024-06-28T15:00:17.404844+0000","last_unstale":"2024-06-28T16:14:58.938053+0000","last_undegraded":"2024-06-28T16:14:58.938053+0000","last_fullsized":"2024-06-28T16:14:58.938053+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'398","last_scrub_stamp":"2024-06-28T16:14:58.937867+0000","last_deep_scrub":"260'398","last_deep_scrub_stamp":"2024-06-28T16:14:58.937867+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.937867+0000","objects_scrubbed":1,"log_size":98,"log_dups_size":300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:18:27.056087+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1527,"num_read_kb":1156,"num_write":658,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":268,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"260'343","reported_seq":1028,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.751903+0000","last_change":"2024-06-28T15:00:15.401905+0000","last_active":"2024-06-28T16:14:34.751903+0000","last_peered":"2024-06-28T16:14:34.751903+0000","last_clean":"2024-06-28T16:14:34.751903+0000","last_became_active":"2024-06-28T15:00:15.401275+0000","last_became_peered":"2024-06-28T15:00:15.401275+0000","last_unstale":"2024-06-28T16:14:34.751903+0000","last_undegraded":"2024-06-28T16:14:34.751903+0000","last_fullsized":"2024-06-28T16:14:34.751903+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":559,"num_read_kb":45669,"num_write":1313,"num_write_kb":19663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"260'221","reported_seq":842,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.318677+0000","last_change":"2024-06-28T15:00:17.383279+0000","last_active":"2024-06-28T16:13:15.318677+0000","last_peered":"2024-06-28T16:13:15.318677+0000","last_clean":"2024-06-28T16:13:15.318677+0000","last_became_active":"2024-06-28T15:00:17.382739+0000","last_became_peered":"2024-06-28T15:00:17.382739+0000","last_unstale":"2024-06-28T16:13:15.318677+0000","last_undegraded":"2024-06-28T16:13:15.318677+0000","last_fullsized":"2024-06-28T16:13:15.318677+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":866,"num_read_kb":690,"num_write":335,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"260'298","reported_seq":929,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.705842+0000","last_change":"2024-06-28T15:00:15.421880+0000","last_active":"2024-06-28T16:14:34.705842+0000","last_peered":"2024-06-28T16:14:34.705842+0000","last_clean":"2024-06-28T16:14:34.705842+0000","last_became_active":"2024-06-28T15:00:15.421189+0000","last_became_peered":"2024-06-28T15:00:15.421189+0000","last_unstale":"2024-06-28T16:14:34.705842+0000","last_undegraded":"2024-06-28T16:14:34.705842+0000","last_fullsized":"2024-06-28T16:14:34.705842+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7105903,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":419,"num_read_kb":19450,"num_write":1148,"num_write_kb":38199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"260'79","reported_seq":640,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.112012+0000","last_change":"2024-06-28T15:00:17.396892+0000","last_active":"2024-06-28T16:13:15.112012+0000","last_peered":"2024-06-28T16:13:15.112012+0000","last_clean":"2024-06-28T16:13:15.112012+0000","last_became_active":"2024-06-28T15:00:17.396125+0000","last_became_peered":"2024-06-28T15:00:17.396125+0000","last_unstale":"2024-06-28T16:13:15.112012+0000","last_undegraded":"2024-06-28T16:13:15.112012+0000","last_fullsized":"2024-06-28T16:13:15.112012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":321,"num_read_kb":257,"num_write":118,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"260'270","reported_seq":928,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.754134+0000","last_change":"2024-06-28T15:00:15.408891+0000","last_active":"2024-06-28T16:14:34.754134+0000","last_peered":"2024-06-28T16:14:34.754134+0000","last_clean":"2024-06-28T16:14:34.754134+0000","last_became_active":"2024-06-28T15:00:15.408375+0000","last_became_peered":"2024-06-28T15:00:15.408375+0000","last_unstale":"2024-06-28T16:14:34.754134+0000","last_undegraded":"2024-06-28T16:14:34.754134+0000","last_fullsized":"2024-06-28T16:14:34.754134+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":484,"num_read_kb":6587,"num_write":1005,"num_write_kb":17135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"260'62","reported_seq":638,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.609235+0000","last_change":"2024-06-28T15:00:17.380077+0000","last_active":"2024-06-28T16:14:38.609235+0000","last_peered":"2024-06-28T16:14:38.609235+0000","last_clean":"2024-06-28T16:14:38.609235+0000","last_became_active":"2024-06-28T15:00:17.379374+0000","last_became_peered":"2024-06-28T15:00:17.379374+0000","last_unstale":"2024-06-28T16:14:38.609235+0000","last_undegraded":"2024-06-28T16:14:38.609235+0000","last_fullsized":"2024-06-28T16:14:38.609235+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":216,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"260'367","reported_seq":1012,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.754359+0000","last_change":"2024-06-28T15:00:15.427903+0000","last_active":"2024-06-28T16:14:34.754359+0000","last_peered":"2024-06-28T16:14:34.754359+0000","last_clean":"2024-06-28T16:14:34.754359+0000","last_became_active":"2024-06-28T15:00:15.427171+0000","last_became_peered":"2024-06-28T15:00:15.427171+0000","last_unstale":"2024-06-28T16:14:34.754359+0000","last_undegraded":"2024-06-28T16:14:34.754359+0000","last_fullsized":"2024-06-28T16:14:34.754359+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":300,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":723921,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":536,"num_read_kb":26554,"num_write":1152,"num_write_kb":52254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"260'230","reported_seq":853,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.422539+0000","last_change":"2024-06-28T15:00:17.409608+0000","last_active":"2024-06-28T16:14:40.422539+0000","last_peered":"2024-06-28T16:14:40.422539+0000","last_clean":"2024-06-28T16:14:40.422539+0000","last_became_active":"2024-06-28T15:00:17.403760+0000","last_became_peered":"2024-06-28T15:00:17.403760+0000","last_unstale":"2024-06-28T16:14:40.422539+0000","last_undegraded":"2024-06-28T16:14:40.422539+0000","last_fullsized":"2024-06-28T16:14:40.422539+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":902,"num_read_kb":724,"num_write":338,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"260'282","reported_seq":930,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.744928+0000","last_change":"2024-06-28T15:00:15.376797+0000","last_active":"2024-06-28T16:14:34.744928+0000","last_peered":"2024-06-28T16:14:34.744928+0000","last_clean":"2024-06-28T16:14:34.744928+0000","last_became_active":"2024-06-28T15:00:15.376014+0000","last_became_peered":"2024-06-28T15:00:15.376014+0000","last_unstale":"2024-06-28T16:14:34.744928+0000","last_undegraded":"2024-06-28T16:14:34.744928+0000","last_fullsized":"2024-06-28T16:14:34.744928+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462789,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":409,"num_read_kb":32504,"num_write":910,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"260'70","reported_seq":660,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.521043+0000","last_change":"2024-06-28T15:00:17.402274+0000","last_active":"2024-06-28T16:14:39.521043+0000","last_peered":"2024-06-28T16:14:39.521043+0000","last_clean":"2024-06-28T16:14:39.521043+0000","last_became_active":"2024-06-28T15:00:17.401917+0000","last_became_peered":"2024-06-28T15:00:17.401917+0000","last_unstale":"2024-06-28T16:14:39.521043+0000","last_undegraded":"2024-06-28T16:14:39.521043+0000","last_fullsized":"2024-06-28T16:14:39.521043+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":295,"num_read_kb":247,"num_write":98,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"260'337","reported_seq":974,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.699992+0000","last_change":"2024-06-28T15:00:15.434365+0000","last_active":"2024-06-28T16:14:34.699992+0000","last_peered":"2024-06-28T16:14:34.699992+0000","last_clean":"2024-06-28T16:14:34.699992+0000","last_became_active":"2024-06-28T15:00:15.431444+0000","last_became_peered":"2024-06-28T15:00:15.431444+0000","last_unstale":"2024-06-28T16:14:34.699992+0000","last_undegraded":"2024-06-28T16:14:34.699992+0000","last_fullsized":"2024-06-28T16:14:34.699992+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":461,"num_read_kb":557,"num_write":1266,"num_write_kb":33580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,3],"acting":[5,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"260'424","reported_seq":1179,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.130582+0000","last_change":"2024-06-28T15:00:17.403422+0000","last_active":"2024-06-28T16:14:35.130582+0000","last_peered":"2024-06-28T16:14:35.130582+0000","last_clean":"2024-06-28T16:14:35.130582+0000","last_became_active":"2024-06-28T15:00:17.402842+0000","last_became_peered":"2024-06-28T15:00:17.402842+0000","last_unstale":"2024-06-28T16:14:35.130582+0000","last_undegraded":"2024-06-28T16:14:35.130582+0000","last_fullsized":"2024-06-28T16:14:35.130582+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":400,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1825,"num_read_kb":1465,"num_write":755,"num_write_kb":468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"260'319","reported_seq":965,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.708660+0000","last_change":"2024-06-28T15:00:15.415547+0000","last_active":"2024-06-28T16:14:34.708660+0000","last_peered":"2024-06-28T16:14:34.708660+0000","last_clean":"2024-06-28T16:14:34.708660+0000","last_became_active":"2024-06-28T15:00:15.415009+0000","last_became_peered":"2024-06-28T15:00:15.415009+0000","last_unstale":"2024-06-28T16:14:34.708660+0000","last_undegraded":"2024-06-28T16:14:34.708660+0000","last_fullsized":"2024-06-28T16:14:34.708660+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242944,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":479,"num_read_kb":2084,"num_write":1202,"num_write_kb":29565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"260'26","reported_seq":575,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.139576+0000","last_change":"2024-06-28T15:00:17.390583+0000","last_active":"2024-06-28T16:14:35.139576+0000","last_peered":"2024-06-28T16:14:35.139576+0000","last_clean":"2024-06-28T16:14:35.139576+0000","last_became_active":"2024-06-28T15:00:17.389844+0000","last_became_peered":"2024-06-28T15:00:17.389844+0000","last_unstale":"2024-06-28T16:14:35.139576+0000","last_undegraded":"2024-06-28T16:14:35.139576+0000","last_fullsized":"2024-06-28T16:14:35.139576+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":112,"num_write":34,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"260'305","reported_seq":937,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.756328+0000","last_change":"2024-06-28T15:00:15.398948+0000","last_active":"2024-06-28T16:14:34.756328+0000","last_peered":"2024-06-28T16:14:34.756328+0000","last_clean":"2024-06-28T16:14:34.756328+0000","last_became_active":"2024-06-28T15:00:15.397986+0000","last_became_peered":"2024-06-28T15:00:15.397986+0000","last_unstale":"2024-06-28T16:14:34.756328+0000","last_undegraded":"2024-06-28T16:14:34.756328+0000","last_fullsized":"2024-06-28T16:14:34.756328+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079926,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":433,"num_read_kb":10373,"num_write":1047,"num_write_kb":25421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"260'192","reported_seq":858,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.608235+0000","last_change":"2024-06-28T15:00:17.395455+0000","last_active":"2024-06-28T16:14:35.608235+0000","last_peered":"2024-06-28T16:14:35.608235+0000","last_clean":"2024-06-28T16:14:35.608235+0000","last_became_active":"2024-06-28T15:00:17.394783+0000","last_became_peered":"2024-06-28T15:00:17.394783+0000","last_unstale":"2024-06-28T16:14:35.608235+0000","last_undegraded":"2024-06-28T16:14:35.608235+0000","last_fullsized":"2024-06-28T16:14:35.608235+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":852,"num_read_kb":715,"num_write":335,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"260'302","reported_seq":922,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.050963+0000","last_change":"2024-06-28T16:14:46.051125+0000","last_active":"2024-06-28T16:14:46.050963+0000","last_peered":"2024-06-28T16:14:46.050963+0000","last_clean":"2024-06-28T16:14:46.050963+0000","last_became_active":"2024-06-28T15:00:59.938004+0000","last_became_peered":"2024-06-28T15:00:59.938004+0000","last_unstale":"2024-06-28T16:14:46.050963+0000","last_undegraded":"2024-06-28T16:14:46.050963+0000","last_fullsized":"2024-06-28T16:14:46.050963+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'302","last_scrub_stamp":"2024-06-28T16:14:46.050757+0000","last_deep_scrub":"260'302","last_deep_scrub_stamp":"2024-06-28T16:14:46.050757+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.050757+0000","objects_scrubbed":13,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:23:03.771050+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6542288,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":437,"num_read_kb":25077,"num_write":1246,"num_write_kb":38037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,3],"acting":[1,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"260'392","reported_seq":1063,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.336853+0000","last_change":"2024-06-28T15:00:17.374446+0000","last_active":"2024-06-28T16:14:35.336853+0000","last_peered":"2024-06-28T16:14:35.336853+0000","last_clean":"2024-06-28T16:14:35.336853+0000","last_became_active":"2024-06-28T15:00:17.374143+0000","last_became_peered":"2024-06-28T15:00:17.374143+0000","last_unstale":"2024-06-28T16:14:35.336853+0000","last_undegraded":"2024-06-28T16:14:35.336853+0000","last_fullsized":"2024-06-28T16:14:35.336853+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":300,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1520,"num_read_kb":1191,"num_write":653,"num_write_kb":382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"260'255","reported_seq":928,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.309302+0000","last_change":"2024-06-28T16:14:58.309458+0000","last_active":"2024-06-28T16:14:58.309302+0000","last_peered":"2024-06-28T16:14:58.309302+0000","last_clean":"2024-06-28T16:14:58.309302+0000","last_became_active":"2024-06-28T15:00:15.430228+0000","last_became_peered":"2024-06-28T15:00:15.430228+0000","last_unstale":"2024-06-28T16:14:58.309302+0000","last_undegraded":"2024-06-28T16:14:58.309302+0000","last_fullsized":"2024-06-28T16:14:58.309302+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'255","last_scrub_stamp":"2024-06-28T16:14:58.309078+0000","last_deep_scrub":"260'255","last_deep_scrub_stamp":"2024-06-28T16:14:58.309078+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.309078+0000","objects_scrubbed":6,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:49:38.745040+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170399,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":449,"num_read_kb":6008,"num_write":895,"num_write_kb":11243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"260'134","reported_seq":779,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:15.134839+0000","last_change":"2024-06-28T15:00:17.382276+0000","last_active":"2024-06-28T16:14:15.134839+0000","last_peered":"2024-06-28T16:14:15.134839+0000","last_clean":"2024-06-28T16:14:15.134839+0000","last_became_active":"2024-06-28T15:00:17.381562+0000","last_became_peered":"2024-06-28T15:00:17.381562+0000","last_unstale":"2024-06-28T16:14:15.134839+0000","last_undegraded":"2024-06-28T16:14:15.134839+0000","last_fullsized":"2024-06-28T16:14:15.134839+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":479,"num_write":190,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"260'496","reported_seq":1187,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.877065+0000","last_change":"2024-06-28T15:00:15.419485+0000","last_active":"2024-06-28T16:14:40.877065+0000","last_peered":"2024-06-28T16:14:40.877065+0000","last_clean":"2024-06-28T16:14:40.877065+0000","last_became_active":"2024-06-28T15:00:15.418850+0000","last_became_peered":"2024-06-28T15:00:15.418850+0000","last_unstale":"2024-06-28T16:14:40.877065+0000","last_undegraded":"2024-06-28T16:14:40.877065+0000","last_fullsized":"2024-06-28T16:14:40.877065+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":400,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":907,"num_read_kb":23220,"num_write":1415,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"260'338","reported_seq":1024,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.876974+0000","last_change":"2024-06-28T15:00:15.409457+0000","last_active":"2024-06-28T16:14:40.876974+0000","last_peered":"2024-06-28T16:14:40.876974+0000","last_clean":"2024-06-28T16:14:40.876974+0000","last_became_active":"2024-06-28T15:00:15.408158+0000","last_became_peered":"2024-06-28T15:00:15.408158+0000","last_unstale":"2024-06-28T16:14:40.876974+0000","last_undegraded":"2024-06-28T16:14:40.876974+0000","last_fullsized":"2024-06-28T16:14:40.876974+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6690738,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":568,"num_read_kb":38558,"num_write":1259,"num_write_kb":46809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"260'77","reported_seq":646,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:05.586931+0000","last_change":"2024-06-28T15:00:17.380351+0000","last_active":"2024-06-28T16:13:05.586931+0000","last_peered":"2024-06-28T16:13:05.586931+0000","last_clean":"2024-06-28T16:13:05.586931+0000","last_became_active":"2024-06-28T15:00:17.379771+0000","last_became_peered":"2024-06-28T15:00:17.379771+0000","last_unstale":"2024-06-28T16:13:05.586931+0000","last_undegraded":"2024-06-28T16:13:05.586931+0000","last_fullsized":"2024-06-28T16:13:05.586931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":249,"num_write":109,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"260'387","reported_seq":1103,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.073488+0000","last_change":"2024-06-28T15:00:15.404986+0000","last_active":"2024-06-28T16:14:39.073488+0000","last_peered":"2024-06-28T16:14:39.073488+0000","last_clean":"2024-06-28T16:14:39.073488+0000","last_became_active":"2024-06-28T15:00:15.401324+0000","last_became_peered":"2024-06-28T15:00:15.401324+0000","last_unstale":"2024-06-28T16:14:39.073488+0000","last_undegraded":"2024-06-28T16:14:39.073488+0000","last_fullsized":"2024-06-28T16:14:39.073488+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":300,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079973,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":661,"num_read_kb":1946,"num_write":1429,"num_write_kb":27589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"260'176","reported_seq":778,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:14.620968+0000","last_change":"2024-06-28T15:00:17.383615+0000","last_active":"2024-06-28T16:13:14.620968+0000","last_peered":"2024-06-28T16:13:14.620968+0000","last_clean":"2024-06-28T16:13:14.620968+0000","last_became_active":"2024-06-28T15:00:17.382774+0000","last_became_peered":"2024-06-28T15:00:17.382774+0000","last_unstale":"2024-06-28T16:13:14.620968+0000","last_undegraded":"2024-06-28T16:13:14.620968+0000","last_fullsized":"2024-06-28T16:13:14.620968+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":719,"num_read_kb":570,"num_write":272,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5652,"num_read_kb":3987,"num_write":2073,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7430,"num_omap_keys":15,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1080,"ondisk_log_size":1080,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19748,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":13201,"num_read_kb":13000,"num_write":15927,"num_write_kb":4660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4204,"num_omap_keys":24,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":39496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1821,"ondisk_log_size":1821,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1424454,"num_objects":198,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":18162,"num_read_kb":37754,"num_write":15149,"num_write_kb":11215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":628052,"num_omap_keys":1462,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1945,"ondisk_log_size":1945,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1644,"num_read_kb":1954,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87319,"num_read_kb":70822,"num_write":35298,"num_write_kb":17822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8804,"num_omap_keys":37,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3532,"ondisk_log_size":3532,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":149605345,"num_objects":433,"num_object_clones":0,"num_object_copies":1732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":433,"num_whiteouts":0,"num_read":34839,"num_read_kb":1786510,"num_write":77776,"num_write_kb":1982911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202506240,"data_stored":202506240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3438,"ondisk_log_size":3438,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":129,"num_read_kb":165927,"num_write":154,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":137,"num_read_kb":186344,"num_write":154,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575750,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356188,"kb_used_data":36244,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":94015652,"statfs":{"total":96636764160,"available":96272027648,"internally_reserved":0,"allocated":37113856,"data_stored":35129654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":327612874},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575750,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":339708,"kb_used_data":43380,"kb_used_omap":1,"kb_used_meta":296318,"kb_avail":94032132,"statfs":{"total":96636764160,"available":96288903168,"internally_reserved":0,"allocated":44421120,"data_stored":42495981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":303430089},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575752,"num_pgs":114,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201360,"kb_used_data":35592,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94170480,"statfs":{"total":96636764160,"available":96430571520,"internally_reserved":0,"allocated":36446208,"data_stored":34448502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575752,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":146836,"kb_used_data":43468,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94225004,"statfs":{"total":96636764160,"available":96486404096,"internally_reserved":0,"allocated":44511232,"data_stored":42596897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575755,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303252,"kb_used_data":29196,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":94068588,"statfs":{"total":96636764160,"available":96326234112,"internally_reserved":0,"allocated":29896704,"data_stored":27928906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":280623559},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575753,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137284,"kb_used_data":29820,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94234556,"statfs":{"total":96636764160,"available":96496185344,"internally_reserved":0,"allocated":30535680,"data_stored":28610104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575750,"num_pgs":120,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201576,"kb_used_data":39968,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94170264,"statfs":{"total":96636764160,"available":96430350336,"internally_reserved":0,"allocated":40927232,"data_stored":38982339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575753,"num_pgs":116,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287396,"kb_used_data":39644,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":94084444,"statfs":{"total":96636764160,"available":96342470656,"internally_reserved":0,"allocated":40595456,"data_stored":38600327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":253688261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25272320,"data_stored":25272320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25198592,"data_stored":25198592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21118976,"data_stored":21118976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19386368,"data_stored":19386368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29863936,"data_stored":29863936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20664320,"data_stored":20664320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28712960,"data_stored":28712960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32288768,"data_stored":32288768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":7701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":4574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":4436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":5746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-28T16:15:02.610 INFO:tasks.ceph:pgid 68.1a last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.610 INFO:tasks.ceph:pgid 67.1d last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.610 INFO:tasks.ceph:pgid 68.1b last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.610 INFO:tasks.ceph:pgid 67.1c last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.611 INFO:tasks.ceph:pgid 68.18 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.611 INFO:tasks.ceph:pgid 67.1f last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.611 INFO:tasks.ceph:pgid 66.1e last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.611 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.611 INFO:tasks.ceph:pgid 67.1e last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.611 INFO:tasks.ceph:pgid 68.1e last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.612 INFO:tasks.ceph:pgid 67.19 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.612 INFO:tasks.ceph:pgid 68.1f last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.612 INFO:tasks.ceph:pgid 67.18 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.612 INFO:tasks.ceph:pgid 66.19 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.612 INFO:tasks.ceph:pgid 68.1c last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.612 INFO:tasks.ceph:pgid 67.1b last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.613 INFO:tasks.ceph:pgid 68.1d last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.613 INFO:tasks.ceph:pgid 67.1a last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.613 INFO:tasks.ceph:pgid 66.1b last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.613 INFO:tasks.ceph:pgid 68.12 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.613 INFO:tasks.ceph:pgid 68.13 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.614 INFO:tasks.ceph:pgid 67.14 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.614 INFO:tasks.ceph:pgid 68.10 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.614 INFO:tasks.ceph:pgid 66.16 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.614 INFO:tasks.ceph:pgid 68.11 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.614 INFO:tasks.ceph:pgid 67.16 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.614 INFO:tasks.ceph:pgid 68.16 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.615 INFO:tasks.ceph:pgid 67.11 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.615 INFO:tasks.ceph:pgid 68.17 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.615 INFO:tasks.ceph:pgid 67.10 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.615 INFO:tasks.ceph:pgid 68.14 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.615 INFO:tasks.ceph:pgid 67.13 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.615 INFO:tasks.ceph:pgid 66.12 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.616 INFO:tasks.ceph:pgid 68.15 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.616 INFO:tasks.ceph:pgid 67.12 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.616 INFO:tasks.ceph:pgid 68.a last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.616 INFO:tasks.ceph:pgid 68.b last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.616 INFO:tasks.ceph:pgid 67.c last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.617 INFO:tasks.ceph:pgid 66.d last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.617 INFO:tasks.ceph:pgid 68.8 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.617 INFO:tasks.ceph:pgid 68.9 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.617 INFO:tasks.ceph:pgid 67.e last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.617 INFO:tasks.ceph:pgid 68.e last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.617 INFO:tasks.ceph:pgid 67.9 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.618 INFO:tasks.ceph:pgid 68.f last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.618 INFO:tasks.ceph:pgid 68.c last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.618 INFO:tasks.ceph:pgid 67.b last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.618 INFO:tasks.ceph:pgid 68.d last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.618 INFO:tasks.ceph:pgid 68.2 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.619 INFO:tasks.ceph:pgid 67.5 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.619 INFO:tasks.ceph:pgid 68.3 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.619 INFO:tasks.ceph:pgid 67.4 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.619 INFO:tasks.ceph:pgid 66.5 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.619 INFO:tasks.ceph:pgid 68.0 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.619 INFO:tasks.ceph:pgid 68.1 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.620 INFO:tasks.ceph:pgid 67.6 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.620 INFO:tasks.ceph:pgid 68.6 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.620 INFO:tasks.ceph:pgid 67.1 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.620 INFO:tasks.ceph:pgid 68.7 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.620 INFO:tasks.ceph:pgid 67.0 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.620 INFO:tasks.ceph:pgid 68.4 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.621 INFO:tasks.ceph:pgid 67.3 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.621 INFO:tasks.ceph:pgid 68.5 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.621 INFO:tasks.ceph:pgid 67.2 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.621 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.621 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.621 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.622 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.622 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.622 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.622 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.622 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.622 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.623 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.623 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.623 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.623 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.623 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.623 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.624 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.624 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.624 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.624 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.624 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.624 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.625 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.625 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.625 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.625 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.625 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.625 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.626 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.626 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.626 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.626 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.626 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.627 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.627 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.627 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.627 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.627 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.627 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.627 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.628 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.628 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.628 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.628 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.628 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.628 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.629 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.629 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.629 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.629 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.629 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.630 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.630 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.630 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.630 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.630 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.630 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.631 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.631 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.631 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.631 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.631 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.632 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.632 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.632 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.632 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.632 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.632 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.633 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.633 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.633 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.633 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.633 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.633 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.634 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.634 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.634 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.634 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.634 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.634 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.635 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.635 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.635 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.635 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.635 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.635 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.636 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.636 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.636 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.636 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.636 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.636 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.637 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.637 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.637 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.637 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.637 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.637 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.638 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.638 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.638 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.638 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.638 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.639 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.639 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.639 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.639 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.639 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.639 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.639 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.640 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.640 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.640 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.640 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.640 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.641 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.641 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.641 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.641 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.641 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.641 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-06-28T15:00:24.376399+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=24, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.642 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.642 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.642 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.642 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.642 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.642 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.643 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.643 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.643 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.643 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.643 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-06-28T15:00:22.364686+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=22, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.643 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.644 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.644 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.644 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.644 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.644 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.644 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.645 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.645 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.645 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.645 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.645 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.646 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.646 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.646 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.646 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.646 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.646 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.647 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.647 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.647 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.647 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.647 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.647 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.648 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.648 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.648 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.648 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.648 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.648 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.649 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.649 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.649 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.649 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.649 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.649 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.650 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.650 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:02.650 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-06-28T16:15:22.651 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-28T16:15:22.980 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:15:22.997 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-06-28T16:15:23.005 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":2510,"stamp":"2024-06-28T16:15:21.639398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":185196140,"num_objects":785,"num_object_clones":0,"num_object_copies":2436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":785,"num_whiteouts":0,"num_read":161161,"num_read_kb":2266365,"num_write":146672,"num_write_kb":2275015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":705334,"num_omap_keys":1744,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12539,"ondisk_log_size":12539,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1956960,"kb_used_data":297312,"kb_used_omap":12,"kb_used_meta":1659571,"kb_avail":753017760,"statfs":{"total":773094113280,"available":771090186240,"internally_reserved":0,"allocated":304447488,"data_stored":288792710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":1699401267},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":21318,"num_omap_keys":73,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007913"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.878549+0000","last_change":"2024-06-28T15:37:33.329429+0000","last_active":"2024-06-28T16:14:40.878549+0000","last_peered":"2024-06-28T16:14:40.878549+0000","last_clean":"2024-06-28T16:14:40.878549+0000","last_became_active":"2024-06-28T15:37:33.328626+0000","last_became_peered":"2024-06-28T15:37:33.328626+0000","last_unstale":"2024-06-28T16:14:40.878549+0000","last_undegraded":"2024-06-28T16:14:40.878549+0000","last_fullsized":"2024-06-28T16:14:40.878549+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"257'3","reported_seq":50,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:40.112304+0000","last_change":"2024-06-28T15:21:11.079460+0000","last_active":"2024-06-28T16:13:40.112304+0000","last_peered":"2024-06-28T16:13:40.112304+0000","last_clean":"2024-06-28T16:13:40.112304+0000","last_became_active":"2024-06-28T15:21:11.078250+0000","last_became_peered":"2024-06-28T15:21:11.078250+0000","last_unstale":"2024-06-28T16:13:40.112304+0000","last_undegraded":"2024-06-28T16:13:40.112304+0000","last_fullsized":"2024-06-28T16:13:40.112304+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"260'4","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.100994+0000","last_change":"2024-06-28T16:14:41.101042+0000","last_active":"2024-06-28T16:14:41.100994+0000","last_peered":"2024-06-28T16:14:41.100994+0000","last_clean":"2024-06-28T16:14:41.100994+0000","last_became_active":"2024-06-28T15:21:07.046517+0000","last_became_peered":"2024-06-28T15:21:07.046517+0000","last_unstale":"2024-06-28T16:14:41.100994+0000","last_undegraded":"2024-06-28T16:14:41.100994+0000","last_fullsized":"2024-06-28T16:14:41.100994+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:14:41.100864+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:14:41.100864+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.100864+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:17:23.820622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3076,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"249'7","reported_seq":91,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.148277+0000","last_change":"2024-06-28T16:14:39.148310+0000","last_active":"2024-06-28T16:14:39.148277+0000","last_peered":"2024-06-28T16:14:39.148277+0000","last_clean":"2024-06-28T16:14:39.148277+0000","last_became_active":"2024-06-28T15:17:24.028787+0000","last_became_peered":"2024-06-28T15:17:24.028787+0000","last_unstale":"2024-06-28T16:14:39.148277+0000","last_undegraded":"2024-06-28T16:14:39.148277+0000","last_fullsized":"2024-06-28T16:14:39.148277+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'7","last_scrub_stamp":"2024-06-28T16:14:39.148161+0000","last_deep_scrub":"249'7","last_deep_scrub_stamp":"2024-06-28T16:14:39.148161+0000","last_clean_scrub_stamp":"2024-06-28T16:14:39.148161+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:43:21.535181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":30,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.297034+0000","last_change":"2024-06-28T16:15:08.297105+0000","last_active":"2024-06-28T16:15:08.297034+0000","last_peered":"2024-06-28T16:15:08.297034+0000","last_clean":"2024-06-28T16:15:08.297034+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:15:08.297034+0000","last_undegraded":"2024-06-28T16:15:08.297034+0000","last_fullsized":"2024-06-28T16:15:08.297034+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:08.296839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:08.296839+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.296839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:48:21.812858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"260'2","reported_seq":47,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.880262+0000","last_change":"2024-06-28T15:21:11.105141+0000","last_active":"2024-06-28T16:14:33.880262+0000","last_peered":"2024-06-28T16:14:33.880262+0000","last_clean":"2024-06-28T16:14:33.880262+0000","last_became_active":"2024-06-28T15:21:11.104387+0000","last_became_peered":"2024-06-28T15:21:11.104387+0000","last_unstale":"2024-06-28T16:14:33.880262+0000","last_undegraded":"2024-06-28T16:14:33.880262+0000","last_fullsized":"2024-06-28T16:14:33.880262+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"260'2","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.043377+0000","last_change":"2024-06-28T16:14:45.043442+0000","last_active":"2024-06-28T16:14:45.043377+0000","last_peered":"2024-06-28T16:14:45.043377+0000","last_clean":"2024-06-28T16:14:45.043377+0000","last_became_active":"2024-06-28T15:21:07.049141+0000","last_became_peered":"2024-06-28T15:21:07.049141+0000","last_unstale":"2024-06-28T16:14:45.043377+0000","last_undegraded":"2024-06-28T16:14:45.043377+0000","last_fullsized":"2024-06-28T16:14:45.043377+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:45.043176+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:45.043176+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.043176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:35:34.904145+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"260'18","reported_seq":119,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.228494+0000","last_change":"2024-06-28T16:14:50.228554+0000","last_active":"2024-06-28T16:14:50.228494+0000","last_peered":"2024-06-28T16:14:50.228494+0000","last_clean":"2024-06-28T16:14:50.228494+0000","last_became_active":"2024-06-28T15:17:23.993905+0000","last_became_peered":"2024-06-28T15:17:23.993905+0000","last_unstale":"2024-06-28T16:14:50.228494+0000","last_undegraded":"2024-06-28T16:14:50.228494+0000","last_fullsized":"2024-06-28T16:14:50.228494+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'18","last_scrub_stamp":"2024-06-28T16:14:50.228287+0000","last_deep_scrub":"260'18","last_deep_scrub_stamp":"2024-06-28T16:14:50.228287+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.228287+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:42:17.443870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":64,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.398560+0000","last_change":"2024-06-28T16:15:18.398617+0000","last_active":"2024-06-28T16:15:18.398560+0000","last_peered":"2024-06-28T16:15:18.398560+0000","last_clean":"2024-06-28T16:15:18.398560+0000","last_became_active":"2024-06-28T15:37:33.345642+0000","last_became_peered":"2024-06-28T15:37:33.345642+0000","last_unstale":"2024-06-28T16:15:18.398560+0000","last_undegraded":"2024-06-28T16:15:18.398560+0000","last_fullsized":"2024-06-28T16:15:18.398560+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:18.398355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:18.398355+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.398355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:01:11.369378+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"257'5","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.591118+0000","last_change":"2024-06-28T15:21:11.098339+0000","last_active":"2024-06-28T16:13:20.591118+0000","last_peered":"2024-06-28T16:13:20.591118+0000","last_clean":"2024-06-28T16:13:20.591118+0000","last_became_active":"2024-06-28T15:21:11.097619+0000","last_became_peered":"2024-06-28T15:21:11.097619+0000","last_unstale":"2024-06-28T16:13:20.591118+0000","last_undegraded":"2024-06-28T16:13:20.591118+0000","last_fullsized":"2024-06-28T16:13:20.591118+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5438,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"0'0","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.347142+0000","last_change":"2024-06-28T16:15:04.347194+0000","last_active":"2024-06-28T16:15:04.347142+0000","last_peered":"2024-06-28T16:15:04.347142+0000","last_clean":"2024-06-28T16:15:04.347142+0000","last_became_active":"2024-06-28T15:21:07.020090+0000","last_became_peered":"2024-06-28T15:21:07.020090+0000","last_unstale":"2024-06-28T16:15:04.347142+0000","last_undegraded":"2024-06-28T16:15:04.347142+0000","last_fullsized":"2024-06-28T16:15:04.347142+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:04.346943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:04.346943+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.346943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:34:44.306374+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"260'43","reported_seq":184,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.114470+0000","last_change":"2024-06-28T16:14:45.114470+0000","last_active":"2024-06-28T16:14:45.114470+0000","last_peered":"2024-06-28T16:14:45.114470+0000","last_clean":"2024-06-28T16:14:45.114470+0000","last_became_active":"2024-06-28T15:17:24.012739+0000","last_became_peered":"2024-06-28T15:17:24.012739+0000","last_unstale":"2024-06-28T16:14:45.114470+0000","last_undegraded":"2024-06-28T16:14:45.114470+0000","last_fullsized":"2024-06-28T16:14:45.114470+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'43","last_scrub_stamp":"2024-06-28T16:14:45.114196+0000","last_deep_scrub":"260'43","last_deep_scrub_stamp":"2024-06-28T16:14:45.114196+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.114196+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:38:38.257912+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":221,"num_read_kb":154,"num_write":91,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.328538+0000","last_change":"2024-06-28T15:37:33.345746+0000","last_active":"2024-06-28T16:14:00.328538+0000","last_peered":"2024-06-28T16:14:00.328538+0000","last_clean":"2024-06-28T16:14:00.328538+0000","last_became_active":"2024-06-28T15:37:33.344850+0000","last_became_peered":"2024-06-28T15:37:33.344850+0000","last_unstale":"2024-06-28T16:14:00.328538+0000","last_undegraded":"2024-06-28T16:14:00.328538+0000","last_fullsized":"2024-06-28T16:14:00.328538+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"260'3","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.182157+0000","last_change":"2024-06-28T16:15:09.182157+0000","last_active":"2024-06-28T16:15:09.182157+0000","last_peered":"2024-06-28T16:15:09.182157+0000","last_clean":"2024-06-28T16:15:09.182157+0000","last_became_active":"2024-06-28T15:21:11.102572+0000","last_became_peered":"2024-06-28T15:21:11.102572+0000","last_unstale":"2024-06-28T16:15:09.182157+0000","last_undegraded":"2024-06-28T16:15:09.182157+0000","last_fullsized":"2024-06-28T16:15:09.182157+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:15:09.181904+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:15:09.181904+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.181904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:48:31.360817+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.928159+0000","last_change":"2024-06-28T16:14:48.928159+0000","last_active":"2024-06-28T16:14:48.928159+0000","last_peered":"2024-06-28T16:14:48.928159+0000","last_clean":"2024-06-28T16:14:48.928159+0000","last_became_active":"2024-06-28T15:21:07.030576+0000","last_became_peered":"2024-06-28T15:21:07.030576+0000","last_unstale":"2024-06-28T16:14:48.928159+0000","last_undegraded":"2024-06-28T16:14:48.928159+0000","last_fullsized":"2024-06-28T16:14:48.928159+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:48.927902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:48.927902+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.927902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:15:53.577907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'23","reported_seq":126,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.110220+0000","last_change":"2024-06-28T16:14:44.110220+0000","last_active":"2024-06-28T16:14:44.110220+0000","last_peered":"2024-06-28T16:14:44.110220+0000","last_clean":"2024-06-28T16:14:44.110220+0000","last_became_active":"2024-06-28T15:17:24.019120+0000","last_became_peered":"2024-06-28T15:17:24.019120+0000","last_unstale":"2024-06-28T16:14:44.110220+0000","last_undegraded":"2024-06-28T16:14:44.110220+0000","last_fullsized":"2024-06-28T16:14:44.110220+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'23","last_scrub_stamp":"2024-06-28T16:14:44.109887+0000","last_deep_scrub":"260'23","last_deep_scrub_stamp":"2024-06-28T16:14:44.109887+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.109887+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:34:23.631650+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":82,"num_write":51,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":31,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.326025+0000","last_change":"2024-06-28T15:37:33.339459+0000","last_active":"2024-06-28T16:13:45.326025+0000","last_peered":"2024-06-28T16:13:45.326025+0000","last_clean":"2024-06-28T16:13:45.326025+0000","last_became_active":"2024-06-28T15:37:33.335045+0000","last_became_peered":"2024-06-28T15:37:33.335045+0000","last_unstale":"2024-06-28T16:13:45.326025+0000","last_undegraded":"2024-06-28T16:13:45.326025+0000","last_fullsized":"2024-06-28T16:13:45.326025+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"0'0","reported_seq":45,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.123964+0000","last_change":"2024-06-28T15:21:11.094697+0000","last_active":"2024-06-28T16:14:35.123964+0000","last_peered":"2024-06-28T16:14:35.123964+0000","last_clean":"2024-06-28T16:14:35.123964+0000","last_became_active":"2024-06-28T15:21:11.094122+0000","last_became_peered":"2024-06-28T15:21:11.094122+0000","last_unstale":"2024-06-28T16:14:35.123964+0000","last_undegraded":"2024-06-28T16:14:35.123964+0000","last_fullsized":"2024-06-28T16:14:35.123964+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.137409+0000","last_change":"2024-06-28T16:14:42.137431+0000","last_active":"2024-06-28T16:14:42.137409+0000","last_peered":"2024-06-28T16:14:42.137409+0000","last_clean":"2024-06-28T16:14:42.137409+0000","last_became_active":"2024-06-28T15:21:07.025549+0000","last_became_peered":"2024-06-28T15:21:07.025549+0000","last_unstale":"2024-06-28T16:14:42.137409+0000","last_undegraded":"2024-06-28T16:14:42.137409+0000","last_fullsized":"2024-06-28T16:14:42.137409+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:42.137318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:42.137318+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.137318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:05:06.478598+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"260'24","reported_seq":130,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.151839+0000","last_change":"2024-06-28T16:14:46.151839+0000","last_active":"2024-06-28T16:14:46.151839+0000","last_peered":"2024-06-28T16:14:46.151839+0000","last_clean":"2024-06-28T16:14:46.151839+0000","last_became_active":"2024-06-28T15:17:24.010441+0000","last_became_peered":"2024-06-28T15:17:24.010441+0000","last_unstale":"2024-06-28T16:14:46.151839+0000","last_undegraded":"2024-06-28T16:14:46.151839+0000","last_fullsized":"2024-06-28T16:14:46.151839+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'24","last_scrub_stamp":"2024-06-28T16:14:46.151597+0000","last_deep_scrub":"260'24","last_deep_scrub_stamp":"2024-06-28T16:14:46.151597+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.151597+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:33:56.078813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":85,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":31,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.074506+0000","last_change":"2024-06-28T15:37:33.341710+0000","last_active":"2024-06-28T16:14:39.074506+0000","last_peered":"2024-06-28T16:14:39.074506+0000","last_clean":"2024-06-28T16:14:39.074506+0000","last_became_active":"2024-06-28T15:37:33.336623+0000","last_became_peered":"2024-06-28T15:37:33.336623+0000","last_unstale":"2024-06-28T16:14:39.074506+0000","last_undegraded":"2024-06-28T16:14:39.074506+0000","last_fullsized":"2024-06-28T16:14:39.074506+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"260'12","reported_seq":74,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.367282+0000","last_change":"2024-06-28T16:15:15.367353+0000","last_active":"2024-06-28T16:15:15.367282+0000","last_peered":"2024-06-28T16:15:15.367282+0000","last_clean":"2024-06-28T16:15:15.367282+0000","last_became_active":"2024-06-28T15:21:11.099434+0000","last_became_peered":"2024-06-28T15:21:11.099434+0000","last_unstale":"2024-06-28T16:15:15.367282+0000","last_undegraded":"2024-06-28T16:15:15.367282+0000","last_fullsized":"2024-06-28T16:15:15.367282+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'12","last_scrub_stamp":"2024-06-28T16:15:15.367078+0000","last_deep_scrub":"260'12","last_deep_scrub_stamp":"2024-06-28T16:15:15.367078+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.367078+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:12:37.736358+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":20482,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"257'5","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.130458+0000","last_change":"2024-06-28T15:21:07.044767+0000","last_active":"2024-06-28T16:14:35.130458+0000","last_peered":"2024-06-28T16:14:35.130458+0000","last_clean":"2024-06-28T16:14:35.130458+0000","last_became_active":"2024-06-28T15:21:07.043987+0000","last_became_peered":"2024-06-28T15:21:07.043987+0000","last_unstale":"2024-06-28T16:14:35.130458+0000","last_undegraded":"2024-06-28T16:14:35.130458+0000","last_fullsized":"2024-06-28T16:14:35.130458+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:05.985640+0000","last_clean_scrub_stamp":"2024-06-28T15:21:05.985640+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"260'19","reported_seq":110,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.131322+0000","last_change":"2024-06-28T16:14:47.131322+0000","last_active":"2024-06-28T16:14:47.131322+0000","last_peered":"2024-06-28T16:14:47.131322+0000","last_clean":"2024-06-28T16:14:47.131322+0000","last_became_active":"2024-06-28T15:17:24.006314+0000","last_became_peered":"2024-06-28T15:17:24.006314+0000","last_unstale":"2024-06-28T16:14:47.131322+0000","last_undegraded":"2024-06-28T16:14:47.131322+0000","last_fullsized":"2024-06-28T16:14:47.131322+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'19","last_scrub_stamp":"2024-06-28T16:14:47.131067+0000","last_deep_scrub":"260'19","last_deep_scrub_stamp":"2024-06-28T16:14:47.131067+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.131067+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:32:41.228085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":54,"num_write":49,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"260'1","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.804670+0000","last_change":"2024-06-28T15:37:33.348503+0000","last_active":"2024-06-28T16:14:41.804670+0000","last_peered":"2024-06-28T16:14:41.804670+0000","last_clean":"2024-06-28T16:14:41.804670+0000","last_became_active":"2024-06-28T15:37:33.347715+0000","last_became_peered":"2024-06-28T15:37:33.347715+0000","last_unstale":"2024-06-28T16:14:41.804670+0000","last_undegraded":"2024-06-28T16:14:41.804670+0000","last_fullsized":"2024-06-28T16:14:41.804670+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.386219+0000","last_change":"2024-06-28T16:15:14.386283+0000","last_active":"2024-06-28T16:15:14.386219+0000","last_peered":"2024-06-28T16:15:14.386219+0000","last_clean":"2024-06-28T16:15:14.386219+0000","last_became_active":"2024-06-28T15:21:11.088625+0000","last_became_peered":"2024-06-28T15:21:11.088625+0000","last_unstale":"2024-06-28T16:15:14.386219+0000","last_undegraded":"2024-06-28T16:15:14.386219+0000","last_fullsized":"2024-06-28T16:15:14.386219+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:14.386017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:14.386017+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.386017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:51:33.701375+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"260'6","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.976747+0000","last_change":"2024-06-28T16:14:44.976815+0000","last_active":"2024-06-28T16:14:44.976747+0000","last_peered":"2024-06-28T16:14:44.976747+0000","last_clean":"2024-06-28T16:14:44.976747+0000","last_became_active":"2024-06-28T15:21:07.030888+0000","last_became_peered":"2024-06-28T15:21:07.030888+0000","last_unstale":"2024-06-28T16:14:44.976747+0000","last_undegraded":"2024-06-28T16:14:44.976747+0000","last_fullsized":"2024-06-28T16:14:44.976747+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:14:44.976529+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:14:44.976529+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.976529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:33:30.061725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":6,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"260'72","reported_seq":256,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.132912+0000","last_change":"2024-06-28T16:14:47.133001+0000","last_active":"2024-06-28T16:14:47.132912+0000","last_peered":"2024-06-28T16:14:47.132912+0000","last_clean":"2024-06-28T16:14:47.132912+0000","last_became_active":"2024-06-28T15:17:23.993026+0000","last_became_peered":"2024-06-28T15:17:23.993026+0000","last_unstale":"2024-06-28T16:14:47.132912+0000","last_undegraded":"2024-06-28T16:14:47.132912+0000","last_fullsized":"2024-06-28T16:14:47.132912+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'72","last_scrub_stamp":"2024-06-28T16:14:47.132725+0000","last_deep_scrub":"260'72","last_deep_scrub_stamp":"2024-06-28T16:14:47.132725+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.132725+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:03.563113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":255,"num_write":131,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.804698+0000","last_change":"2024-06-28T15:37:33.351154+0000","last_active":"2024-06-28T16:14:41.804698+0000","last_peered":"2024-06-28T16:14:41.804698+0000","last_clean":"2024-06-28T16:14:41.804698+0000","last_became_active":"2024-06-28T15:37:33.350447+0000","last_became_peered":"2024-06-28T15:37:33.350447+0000","last_unstale":"2024-06-28T16:14:41.804698+0000","last_undegraded":"2024-06-28T16:14:41.804698+0000","last_fullsized":"2024-06-28T16:14:41.804698+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.251991+0000","last_change":"2024-06-28T16:15:01.252044+0000","last_active":"2024-06-28T16:15:01.251991+0000","last_peered":"2024-06-28T16:15:01.251991+0000","last_clean":"2024-06-28T16:15:01.251991+0000","last_became_active":"2024-06-28T15:21:11.100603+0000","last_became_peered":"2024-06-28T15:21:11.100603+0000","last_unstale":"2024-06-28T16:15:01.251991+0000","last_undegraded":"2024-06-28T16:15:01.251991+0000","last_fullsized":"2024-06-28T16:15:01.251991+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:01.251799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:01.251799+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.251799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:50:06.461152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"260'5","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.231661+0000","last_change":"2024-06-28T16:15:02.231723+0000","last_active":"2024-06-28T16:15:02.231661+0000","last_peered":"2024-06-28T16:15:02.231661+0000","last_clean":"2024-06-28T16:15:02.231661+0000","last_became_active":"2024-06-28T15:21:07.045902+0000","last_became_peered":"2024-06-28T15:21:07.045902+0000","last_unstale":"2024-06-28T16:15:02.231661+0000","last_undegraded":"2024-06-28T16:15:02.231661+0000","last_fullsized":"2024-06-28T16:15:02.231661+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'5","last_scrub_stamp":"2024-06-28T16:15:02.231467+0000","last_deep_scrub":"260'5","last_deep_scrub_stamp":"2024-06-28T16:15:02.231467+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.231467+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:59:29.633514+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":40,"num_write":10,"num_write_kb":1064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"260'48","reported_seq":201,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.112932+0000","last_change":"2024-06-28T16:14:43.112995+0000","last_active":"2024-06-28T16:14:43.112932+0000","last_peered":"2024-06-28T16:14:43.112932+0000","last_clean":"2024-06-28T16:14:43.112932+0000","last_became_active":"2024-06-28T15:17:24.016249+0000","last_became_peered":"2024-06-28T15:17:24.016249+0000","last_unstale":"2024-06-28T16:14:43.112932+0000","last_undegraded":"2024-06-28T16:14:43.112932+0000","last_fullsized":"2024-06-28T16:14:43.112932+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'48","last_scrub_stamp":"2024-06-28T16:14:43.112724+0000","last_deep_scrub":"260'48","last_deep_scrub_stamp":"2024-06-28T16:14:43.112724+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.112724+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:09:09.626825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":260,"num_read_kb":188,"num_write":81,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.074524+0000","last_change":"2024-06-28T15:37:33.317386+0000","last_active":"2024-06-28T16:14:39.074524+0000","last_peered":"2024-06-28T16:14:39.074524+0000","last_clean":"2024-06-28T16:14:39.074524+0000","last_became_active":"2024-06-28T15:37:33.316616+0000","last_became_peered":"2024-06-28T15:37:33.316616+0000","last_unstale":"2024-06-28T16:14:39.074524+0000","last_undegraded":"2024-06-28T16:14:39.074524+0000","last_fullsized":"2024-06-28T16:14:39.074524+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"260'8","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.133370+0000","last_change":"2024-06-28T16:14:57.133432+0000","last_active":"2024-06-28T16:14:57.133370+0000","last_peered":"2024-06-28T16:14:57.133370+0000","last_clean":"2024-06-28T16:14:57.133370+0000","last_became_active":"2024-06-28T15:21:11.094947+0000","last_became_peered":"2024-06-28T15:21:11.094947+0000","last_unstale":"2024-06-28T16:14:57.133370+0000","last_undegraded":"2024-06-28T16:14:57.133370+0000","last_fullsized":"2024-06-28T16:14:57.133370+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'8","last_scrub_stamp":"2024-06-28T16:14:57.133168+0000","last_deep_scrub":"260'8","last_deep_scrub_stamp":"2024-06-28T16:14:57.133168+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.133168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:12:28.239090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":12406,"num_write":8,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"260'5","reported_seq":70,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.646825+0000","last_change":"2024-06-28T16:14:52.646825+0000","last_active":"2024-06-28T16:14:52.646825+0000","last_peered":"2024-06-28T16:14:52.646825+0000","last_clean":"2024-06-28T16:14:52.646825+0000","last_became_active":"2024-06-28T15:21:07.025396+0000","last_became_peered":"2024-06-28T15:21:07.025396+0000","last_unstale":"2024-06-28T16:14:52.646825+0000","last_undegraded":"2024-06-28T16:14:52.646825+0000","last_fullsized":"2024-06-28T16:14:52.646825+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'5","last_scrub_stamp":"2024-06-28T16:14:52.646576+0000","last_deep_scrub":"260'5","last_deep_scrub_stamp":"2024-06-28T16:14:52.646576+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.646576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:58:00.023881+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":5,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"257'14","reported_seq":105,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.108821+0000","last_change":"2024-06-28T16:14:44.108939+0000","last_active":"2024-06-28T16:14:44.108821+0000","last_peered":"2024-06-28T16:14:44.108821+0000","last_clean":"2024-06-28T16:14:44.108821+0000","last_became_active":"2024-06-28T15:17:24.010190+0000","last_became_peered":"2024-06-28T15:17:24.010190+0000","last_unstale":"2024-06-28T16:14:44.108821+0000","last_undegraded":"2024-06-28T16:14:44.108821+0000","last_fullsized":"2024-06-28T16:14:44.108821+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'14","last_scrub_stamp":"2024-06-28T16:14:44.108596+0000","last_deep_scrub":"257'14","last_deep_scrub_stamp":"2024-06-28T16:14:44.108596+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.108596+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:06:35.967557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":51,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"260'2","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.268389+0000","last_change":"2024-06-28T16:15:09.268572+0000","last_active":"2024-06-28T16:15:09.268389+0000","last_peered":"2024-06-28T16:15:09.268389+0000","last_clean":"2024-06-28T16:15:09.268389+0000","last_became_active":"2024-06-28T15:37:33.334097+0000","last_became_peered":"2024-06-28T15:37:33.334097+0000","last_unstale":"2024-06-28T16:15:09.268389+0000","last_undegraded":"2024-06-28T16:15:09.268389+0000","last_fullsized":"2024-06-28T16:15:09.268389+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:09.268148+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:09.268148+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.268148+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:24:54.459931+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"260'9","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.972571+0000","last_change":"2024-06-28T15:21:11.107909+0000","last_active":"2024-06-28T16:14:39.972571+0000","last_peered":"2024-06-28T16:14:39.972571+0000","last_clean":"2024-06-28T16:14:39.972571+0000","last_became_active":"2024-06-28T15:21:11.107155+0000","last_became_peered":"2024-06-28T15:21:11.107155+0000","last_unstale":"2024-06-28T16:14:39.972571+0000","last_undegraded":"2024-06-28T16:14:39.972571+0000","last_fullsized":"2024-06-28T16:14:39.972571+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21504,"num_write":14,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.557445+0000","last_change":"2024-06-28T16:14:42.557501+0000","last_active":"2024-06-28T16:14:42.557445+0000","last_peered":"2024-06-28T16:14:42.557445+0000","last_clean":"2024-06-28T16:14:42.557445+0000","last_became_active":"2024-06-28T15:21:07.042691+0000","last_became_peered":"2024-06-28T15:21:07.042691+0000","last_unstale":"2024-06-28T16:14:42.557445+0000","last_undegraded":"2024-06-28T16:14:42.557445+0000","last_fullsized":"2024-06-28T16:14:42.557445+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:42.557250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:42.557250+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.557250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:17:11.170202+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"260'74","reported_seq":299,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.069608+0000","last_change":"2024-06-28T16:14:40.069653+0000","last_active":"2024-06-28T16:14:40.069608+0000","last_peered":"2024-06-28T16:14:40.069608+0000","last_clean":"2024-06-28T16:14:40.069608+0000","last_became_active":"2024-06-28T15:17:24.027786+0000","last_became_peered":"2024-06-28T15:17:24.027786+0000","last_unstale":"2024-06-28T16:14:40.069608+0000","last_undegraded":"2024-06-28T16:14:40.069608+0000","last_fullsized":"2024-06-28T16:14:40.069608+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'74","last_scrub_stamp":"2024-06-28T16:14:40.069506+0000","last_deep_scrub":"260'74","last_deep_scrub_stamp":"2024-06-28T16:14:40.069506+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.069506+0000","objects_scrubbed":1,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:43:30.466356+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":461,"num_read_kb":332,"num_write":85,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2470,"num_omap_keys":5,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.804650+0000","last_change":"2024-06-28T15:37:33.355373+0000","last_active":"2024-06-28T16:14:41.804650+0000","last_peered":"2024-06-28T16:14:41.804650+0000","last_clean":"2024-06-28T16:14:41.804650+0000","last_became_active":"2024-06-28T15:37:33.354730+0000","last_became_peered":"2024-06-28T15:37:33.354730+0000","last_unstale":"2024-06-28T16:14:41.804650+0000","last_undegraded":"2024-06-28T16:14:41.804650+0000","last_fullsized":"2024-06-28T16:14:41.804650+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"260'4","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.183946+0000","last_change":"2024-06-28T16:14:53.184018+0000","last_active":"2024-06-28T16:14:53.183946+0000","last_peered":"2024-06-28T16:14:53.183946+0000","last_clean":"2024-06-28T16:14:53.183946+0000","last_became_active":"2024-06-28T15:21:11.093751+0000","last_became_peered":"2024-06-28T15:21:11.093751+0000","last_unstale":"2024-06-28T16:14:53.183946+0000","last_undegraded":"2024-06-28T16:14:53.183946+0000","last_fullsized":"2024-06-28T16:14:53.183946+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:14:53.183733+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:14:53.183733+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.183733+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:12:53.454345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"260'2","reported_seq":62,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.333635+0000","last_change":"2024-06-28T16:15:07.333707+0000","last_active":"2024-06-28T16:15:07.333635+0000","last_peered":"2024-06-28T16:15:07.333635+0000","last_clean":"2024-06-28T16:15:07.333635+0000","last_became_active":"2024-06-28T15:21:07.042846+0000","last_became_peered":"2024-06-28T15:21:07.042846+0000","last_unstale":"2024-06-28T16:15:07.333635+0000","last_undegraded":"2024-06-28T16:15:07.333635+0000","last_fullsized":"2024-06-28T16:15:07.333635+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:07.333374+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:07.333374+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.333374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:40:39.550588+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":59,"num_write":2,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"260'31","reported_seq":153,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.199746+0000","last_change":"2024-06-28T16:14:54.199811+0000","last_active":"2024-06-28T16:14:54.199746+0000","last_peered":"2024-06-28T16:14:54.199746+0000","last_clean":"2024-06-28T16:14:54.199746+0000","last_became_active":"2024-06-28T15:17:24.016275+0000","last_became_peered":"2024-06-28T15:17:24.016275+0000","last_unstale":"2024-06-28T16:14:54.199746+0000","last_undegraded":"2024-06-28T16:14:54.199746+0000","last_fullsized":"2024-06-28T16:14:54.199746+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'31","last_scrub_stamp":"2024-06-28T16:14:54.199529+0000","last_deep_scrub":"260'31","last_deep_scrub_stamp":"2024-06-28T16:14:54.199529+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.199529+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:16:15.219769+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":116,"num_write":60,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.878501+0000","last_change":"2024-06-28T15:37:33.323939+0000","last_active":"2024-06-28T16:14:40.878501+0000","last_peered":"2024-06-28T16:14:40.878501+0000","last_clean":"2024-06-28T16:14:40.878501+0000","last_became_active":"2024-06-28T15:37:33.323174+0000","last_became_peered":"2024-06-28T15:37:33.323174+0000","last_unstale":"2024-06-28T16:14:40.878501+0000","last_undegraded":"2024-06-28T16:14:40.878501+0000","last_fullsized":"2024-06-28T16:14:40.878501+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"260'9","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.511491+0000","last_change":"2024-06-28T15:21:11.088257+0000","last_active":"2024-06-28T16:14:33.511491+0000","last_peered":"2024-06-28T16:14:33.511491+0000","last_clean":"2024-06-28T16:14:33.511491+0000","last_became_active":"2024-06-28T15:21:11.087119+0000","last_became_peered":"2024-06-28T15:21:11.087119+0000","last_unstale":"2024-06-28T16:14:33.511491+0000","last_undegraded":"2024-06-28T16:14:33.511491+0000","last_fullsized":"2024-06-28T16:14:33.511491+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":24576,"num_write":19,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.108123+0000","last_change":"2024-06-28T16:14:50.108177+0000","last_active":"2024-06-28T16:14:50.108123+0000","last_peered":"2024-06-28T16:14:50.108123+0000","last_clean":"2024-06-28T16:14:50.108123+0000","last_became_active":"2024-06-28T15:21:07.032092+0000","last_became_peered":"2024-06-28T15:21:07.032092+0000","last_unstale":"2024-06-28T16:14:50.108123+0000","last_undegraded":"2024-06-28T16:14:50.108123+0000","last_fullsized":"2024-06-28T16:14:50.108123+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:50.107930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:50.107930+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.107930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:01:58.373114+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"260'21","reported_seq":129,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.642626+0000","last_change":"2024-06-28T16:14:54.642755+0000","last_active":"2024-06-28T16:14:54.642626+0000","last_peered":"2024-06-28T16:14:54.642626+0000","last_clean":"2024-06-28T16:14:54.642626+0000","last_became_active":"2024-06-28T15:17:24.020899+0000","last_became_peered":"2024-06-28T15:17:24.020899+0000","last_unstale":"2024-06-28T16:14:54.642626+0000","last_undegraded":"2024-06-28T16:14:54.642626+0000","last_fullsized":"2024-06-28T16:14:54.642626+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'21","last_scrub_stamp":"2024-06-28T16:14:54.642387+0000","last_deep_scrub":"260'21","last_deep_scrub_stamp":"2024-06-28T16:14:54.642387+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.642387+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:49:49.845916+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":86,"num_write":39,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:50.326876+0000","last_change":"2024-06-28T15:37:33.339666+0000","last_active":"2024-06-28T16:13:50.326876+0000","last_peered":"2024-06-28T16:13:50.326876+0000","last_clean":"2024-06-28T16:13:50.326876+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:13:50.326876+0000","last_undegraded":"2024-06-28T16:13:50.326876+0000","last_fullsized":"2024-06-28T16:13:50.326876+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"257'3","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.256769+0000","last_change":"2024-06-28T16:15:18.256829+0000","last_active":"2024-06-28T16:15:18.256769+0000","last_peered":"2024-06-28T16:15:18.256769+0000","last_clean":"2024-06-28T16:15:18.256769+0000","last_became_active":"2024-06-28T15:21:11.097622+0000","last_became_peered":"2024-06-28T15:21:11.097622+0000","last_unstale":"2024-06-28T16:15:18.256769+0000","last_undegraded":"2024-06-28T16:15:18.256769+0000","last_fullsized":"2024-06-28T16:15:18.256769+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:18.256544+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:18.256544+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.256544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:33:41.559876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"0'0","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.906019+0000","last_change":"2024-06-28T16:14:56.906073+0000","last_active":"2024-06-28T16:14:56.906019+0000","last_peered":"2024-06-28T16:14:56.906019+0000","last_clean":"2024-06-28T16:14:56.906019+0000","last_became_active":"2024-06-28T15:21:07.038746+0000","last_became_peered":"2024-06-28T15:21:07.038746+0000","last_unstale":"2024-06-28T16:14:56.906019+0000","last_undegraded":"2024-06-28T16:14:56.906019+0000","last_fullsized":"2024-06-28T16:14:56.906019+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.905794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.905794+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.905794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:56:22.215818+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"257'9","reported_seq":90,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.630121+0000","last_change":"2024-06-28T16:14:53.630260+0000","last_active":"2024-06-28T16:14:53.630121+0000","last_peered":"2024-06-28T16:14:53.630121+0000","last_clean":"2024-06-28T16:14:53.630121+0000","last_became_active":"2024-06-28T15:17:24.020276+0000","last_became_peered":"2024-06-28T15:17:24.020276+0000","last_unstale":"2024-06-28T16:14:53.630121+0000","last_undegraded":"2024-06-28T16:14:53.630121+0000","last_fullsized":"2024-06-28T16:14:53.630121+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'9","last_scrub_stamp":"2024-06-28T16:14:53.629897+0000","last_deep_scrub":"257'9","last_deep_scrub_stamp":"2024-06-28T16:14:53.629897+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.629897+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:40:42.297017+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:50.597772+0000","last_change":"2024-06-28T15:37:33.328340+0000","last_active":"2024-06-28T16:13:50.597772+0000","last_peered":"2024-06-28T16:13:50.597772+0000","last_clean":"2024-06-28T16:13:50.597772+0000","last_became_active":"2024-06-28T15:37:33.327497+0000","last_became_peered":"2024-06-28T15:37:33.327497+0000","last_unstale":"2024-06-28T16:13:50.597772+0000","last_undegraded":"2024-06-28T16:13:50.597772+0000","last_fullsized":"2024-06-28T16:13:50.597772+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"260'4","reported_seq":62,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.271440+0000","last_change":"2024-06-28T16:15:16.271504+0000","last_active":"2024-06-28T16:15:16.271440+0000","last_peered":"2024-06-28T16:15:16.271440+0000","last_clean":"2024-06-28T16:15:16.271440+0000","last_became_active":"2024-06-28T15:21:11.092033+0000","last_became_peered":"2024-06-28T15:21:11.092033+0000","last_unstale":"2024-06-28T16:15:16.271440+0000","last_undegraded":"2024-06-28T16:15:16.271440+0000","last_fullsized":"2024-06-28T16:15:16.271440+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:15:16.271243+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:15:16.271243+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.271243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:12:12.186648+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6152,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.141194+0000","last_change":"2024-06-28T16:14:45.141194+0000","last_active":"2024-06-28T16:14:45.141194+0000","last_peered":"2024-06-28T16:14:45.141194+0000","last_clean":"2024-06-28T16:14:45.141194+0000","last_became_active":"2024-06-28T15:21:07.045842+0000","last_became_peered":"2024-06-28T15:21:07.045842+0000","last_unstale":"2024-06-28T16:14:45.141194+0000","last_undegraded":"2024-06-28T16:14:45.141194+0000","last_fullsized":"2024-06-28T16:14:45.141194+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:45.140747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:45.140747+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.140747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:01:27.177077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"260'17","reported_seq":115,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.153794+0000","last_change":"2024-06-28T16:14:43.153919+0000","last_active":"2024-06-28T16:14:43.153794+0000","last_peered":"2024-06-28T16:14:43.153794+0000","last_clean":"2024-06-28T16:14:43.153794+0000","last_became_active":"2024-06-28T15:17:24.013118+0000","last_became_peered":"2024-06-28T15:17:24.013118+0000","last_unstale":"2024-06-28T16:14:43.153794+0000","last_undegraded":"2024-06-28T16:14:43.153794+0000","last_fullsized":"2024-06-28T16:14:43.153794+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'17","last_scrub_stamp":"2024-06-28T16:14:43.153520+0000","last_deep_scrub":"260'17","last_deep_scrub_stamp":"2024-06-28T16:14:43.153520+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.153520+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:39:58.889835+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":63,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"259'1","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.610313+0000","last_change":"2024-06-28T15:37:33.342777+0000","last_active":"2024-06-28T16:14:38.610313+0000","last_peered":"2024-06-28T16:14:38.610313+0000","last_clean":"2024-06-28T16:14:38.610313+0000","last_became_active":"2024-06-28T15:37:33.339097+0000","last_became_peered":"2024-06-28T15:37:33.339097+0000","last_unstale":"2024-06-28T16:14:38.610313+0000","last_undegraded":"2024-06-28T16:14:38.610313+0000","last_fullsized":"2024-06-28T16:14:38.610313+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"257'3","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.217915+0000","last_change":"2024-06-28T16:15:05.217973+0000","last_active":"2024-06-28T16:15:05.217915+0000","last_peered":"2024-06-28T16:15:05.217915+0000","last_clean":"2024-06-28T16:15:05.217915+0000","last_became_active":"2024-06-28T15:21:11.101063+0000","last_became_peered":"2024-06-28T15:21:11.101063+0000","last_unstale":"2024-06-28T16:15:05.217915+0000","last_undegraded":"2024-06-28T16:15:05.217915+0000","last_fullsized":"2024-06-28T16:15:05.217915+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:05.217704+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:05.217704+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.217704+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:55:32.534220+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"257'3","reported_seq":65,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.407866+0000","last_change":"2024-06-28T16:15:13.407927+0000","last_active":"2024-06-28T16:15:13.407866+0000","last_peered":"2024-06-28T16:15:13.407866+0000","last_clean":"2024-06-28T16:15:13.407866+0000","last_became_active":"2024-06-28T15:21:07.040973+0000","last_became_peered":"2024-06-28T15:21:07.040973+0000","last_unstale":"2024-06-28T16:15:13.407866+0000","last_undegraded":"2024-06-28T16:15:13.407866+0000","last_fullsized":"2024-06-28T16:15:13.407866+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:13.407661+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:13.407661+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.407661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:18:56.741530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"260'22","reported_seq":124,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.889374+0000","last_change":"2024-06-28T16:14:40.889438+0000","last_active":"2024-06-28T16:14:40.889374+0000","last_peered":"2024-06-28T16:14:40.889374+0000","last_clean":"2024-06-28T16:14:40.889374+0000","last_became_active":"2024-06-28T15:17:24.028410+0000","last_became_peered":"2024-06-28T15:17:24.028410+0000","last_unstale":"2024-06-28T16:14:40.889374+0000","last_undegraded":"2024-06-28T16:14:40.889374+0000","last_fullsized":"2024-06-28T16:14:40.889374+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'22","last_scrub_stamp":"2024-06-28T16:14:40.889173+0000","last_deep_scrub":"260'22","last_deep_scrub_stamp":"2024-06-28T16:14:40.889173+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.889173+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:21.505212+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"260'1","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:50.326834+0000","last_change":"2024-06-28T15:37:33.341173+0000","last_active":"2024-06-28T16:13:50.326834+0000","last_peered":"2024-06-28T16:13:50.326834+0000","last_clean":"2024-06-28T16:13:50.326834+0000","last_became_active":"2024-06-28T15:37:33.340370+0000","last_became_peered":"2024-06-28T15:37:33.340370+0000","last_unstale":"2024-06-28T16:13:50.326834+0000","last_undegraded":"2024-06-28T16:13:50.326834+0000","last_fullsized":"2024-06-28T16:13:50.326834+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"257'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.353284+0000","last_change":"2024-06-28T16:15:16.353345+0000","last_active":"2024-06-28T16:15:16.353284+0000","last_peered":"2024-06-28T16:15:16.353284+0000","last_clean":"2024-06-28T16:15:16.353284+0000","last_became_active":"2024-06-28T15:21:11.104038+0000","last_became_peered":"2024-06-28T15:21:11.104038+0000","last_unstale":"2024-06-28T16:15:16.353284+0000","last_undegraded":"2024-06-28T16:15:16.353284+0000","last_fullsized":"2024-06-28T16:15:16.353284+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:16.353069+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:16.353069+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.353069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:49:31.899786+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"260'9","reported_seq":82,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.657797+0000","last_change":"2024-06-28T16:14:48.657797+0000","last_active":"2024-06-28T16:14:48.657797+0000","last_peered":"2024-06-28T16:14:48.657797+0000","last_clean":"2024-06-28T16:14:48.657797+0000","last_became_active":"2024-06-28T15:21:07.008899+0000","last_became_peered":"2024-06-28T15:21:07.008899+0000","last_unstale":"2024-06-28T16:14:48.657797+0000","last_undegraded":"2024-06-28T16:14:48.657797+0000","last_fullsized":"2024-06-28T16:14:48.657797+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'9","last_scrub_stamp":"2024-06-28T16:14:48.657546+0000","last_deep_scrub":"260'9","last_deep_scrub_stamp":"2024-06-28T16:14:48.657546+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.657546+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:10:31.780674+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":7,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"260'42","reported_seq":174,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.123309+0000","last_change":"2024-06-28T16:14:43.123456+0000","last_active":"2024-06-28T16:14:43.123309+0000","last_peered":"2024-06-28T16:14:43.123309+0000","last_clean":"2024-06-28T16:14:43.123309+0000","last_became_active":"2024-06-28T15:17:24.019180+0000","last_became_peered":"2024-06-28T15:17:24.019180+0000","last_unstale":"2024-06-28T16:14:43.123309+0000","last_undegraded":"2024-06-28T16:14:43.123309+0000","last_fullsized":"2024-06-28T16:14:43.123309+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'42","last_scrub_stamp":"2024-06-28T16:14:43.123099+0000","last_deep_scrub":"260'42","last_deep_scrub_stamp":"2024-06-28T16:14:43.123099+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.123099+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:45:23.503458+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":198,"num_read_kb":139,"num_write":93,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.312243+0000","last_change":"2024-06-28T16:15:10.312306+0000","last_active":"2024-06-28T16:15:10.312243+0000","last_peered":"2024-06-28T16:15:10.312243+0000","last_clean":"2024-06-28T16:15:10.312243+0000","last_became_active":"2024-06-28T15:37:33.341716+0000","last_became_peered":"2024-06-28T15:37:33.341716+0000","last_unstale":"2024-06-28T16:15:10.312243+0000","last_undegraded":"2024-06-28T16:15:10.312243+0000","last_fullsized":"2024-06-28T16:15:10.312243+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:10.312037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:10.312037+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.312037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:41:31.673706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.080244+0000","last_change":"2024-06-28T16:14:49.080244+0000","last_active":"2024-06-28T16:14:49.080244+0000","last_peered":"2024-06-28T16:14:49.080244+0000","last_clean":"2024-06-28T16:14:49.080244+0000","last_became_active":"2024-06-28T15:21:11.093276+0000","last_became_peered":"2024-06-28T15:21:11.093276+0000","last_unstale":"2024-06-28T16:14:49.080244+0000","last_undegraded":"2024-06-28T16:14:49.080244+0000","last_fullsized":"2024-06-28T16:14:49.080244+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.079996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.079996+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.079996+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:47:03.183258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"260'9","reported_seq":76,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.651923+0000","last_change":"2024-06-28T16:14:45.651923+0000","last_active":"2024-06-28T16:14:45.651923+0000","last_peered":"2024-06-28T16:14:45.651923+0000","last_clean":"2024-06-28T16:14:45.651923+0000","last_became_active":"2024-06-28T15:21:07.040691+0000","last_became_peered":"2024-06-28T15:21:07.040691+0000","last_unstale":"2024-06-28T16:14:45.651923+0000","last_undegraded":"2024-06-28T16:14:45.651923+0000","last_fullsized":"2024-06-28T16:14:45.651923+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'9","last_scrub_stamp":"2024-06-28T16:14:45.651669+0000","last_deep_scrub":"260'9","last_deep_scrub_stamp":"2024-06-28T16:14:45.651669+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.651669+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:32:08.870332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":17408,"num_write":9,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"260'47","reported_seq":193,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.096531+0000","last_change":"2024-06-28T16:14:42.096599+0000","last_active":"2024-06-28T16:14:42.096531+0000","last_peered":"2024-06-28T16:14:42.096531+0000","last_clean":"2024-06-28T16:14:42.096531+0000","last_became_active":"2024-06-28T15:17:24.027610+0000","last_became_peered":"2024-06-28T15:17:24.027610+0000","last_unstale":"2024-06-28T16:14:42.096531+0000","last_undegraded":"2024-06-28T16:14:42.096531+0000","last_fullsized":"2024-06-28T16:14:42.096531+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'47","last_scrub_stamp":"2024-06-28T16:14:42.096391+0000","last_deep_scrub":"260'47","last_deep_scrub_stamp":"2024-06-28T16:14:42.096391+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.096391+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:11:26.034311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":231,"num_read_kb":169,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:35.125426+0000","last_change":"2024-06-28T15:37:33.347375+0000","last_active":"2024-06-28T16:13:35.125426+0000","last_peered":"2024-06-28T16:13:35.125426+0000","last_clean":"2024-06-28T16:13:35.125426+0000","last_became_active":"2024-06-28T15:37:33.346414+0000","last_became_peered":"2024-06-28T15:37:33.346414+0000","last_unstale":"2024-06-28T16:13:35.125426+0000","last_undegraded":"2024-06-28T16:13:35.125426+0000","last_fullsized":"2024-06-28T16:13:35.125426+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"257'5","reported_seq":46,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.113353+0000","last_change":"2024-06-28T15:22:00.970955+0000","last_active":"2024-06-28T16:13:45.113353+0000","last_peered":"2024-06-28T16:13:45.113353+0000","last_clean":"2024-06-28T16:13:45.113353+0000","last_became_active":"2024-06-28T15:22:00.940760+0000","last_became_peered":"2024-06-28T15:22:00.940760+0000","last_unstale":"2024-06-28T16:13:45.113353+0000","last_undegraded":"2024-06-28T16:13:45.113353+0000","last_fullsized":"2024-06-28T16:13:45.113353+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"260'3","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.369210+0000","last_change":"2024-06-28T16:15:05.369210+0000","last_active":"2024-06-28T16:15:05.369210+0000","last_peered":"2024-06-28T16:15:05.369210+0000","last_clean":"2024-06-28T16:15:05.369210+0000","last_became_active":"2024-06-28T15:21:07.030163+0000","last_became_peered":"2024-06-28T15:21:07.030163+0000","last_unstale":"2024-06-28T16:15:05.369210+0000","last_undegraded":"2024-06-28T16:15:05.369210+0000","last_fullsized":"2024-06-28T16:15:05.369210+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:15:05.368894+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:15:05.368894+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.368894+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:40:02.616948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"260'40","reported_seq":180,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.687315+0000","last_change":"2024-06-28T16:14:44.687376+0000","last_active":"2024-06-28T16:14:44.687315+0000","last_peered":"2024-06-28T16:14:44.687315+0000","last_clean":"2024-06-28T16:14:44.687315+0000","last_became_active":"2024-06-28T15:17:24.012572+0000","last_became_peered":"2024-06-28T15:17:24.012572+0000","last_unstale":"2024-06-28T16:14:44.687315+0000","last_undegraded":"2024-06-28T16:14:44.687315+0000","last_fullsized":"2024-06-28T16:14:44.687315+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'40","last_scrub_stamp":"2024-06-28T16:14:44.687125+0000","last_deep_scrub":"260'40","last_deep_scrub_stamp":"2024-06-28T16:14:44.687125+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.687125+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:52:09.797828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":148,"num_write":94,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.397140+0000","last_change":"2024-06-28T16:15:19.397200+0000","last_active":"2024-06-28T16:15:19.397140+0000","last_peered":"2024-06-28T16:15:19.397140+0000","last_clean":"2024-06-28T16:15:19.397140+0000","last_became_active":"2024-06-28T15:37:33.352959+0000","last_became_peered":"2024-06-28T15:37:33.352959+0000","last_unstale":"2024-06-28T16:15:19.397140+0000","last_undegraded":"2024-06-28T16:15:19.397140+0000","last_fullsized":"2024-06-28T16:15:19.397140+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:19.396949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:19.396949+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.396949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:10:10.507079+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":54,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.046074+0000","last_change":"2024-06-28T16:14:49.046127+0000","last_active":"2024-06-28T16:14:49.046074+0000","last_peered":"2024-06-28T16:14:49.046074+0000","last_clean":"2024-06-28T16:14:49.046074+0000","last_became_active":"2024-06-28T15:21:11.079036+0000","last_became_peered":"2024-06-28T15:21:11.079036+0000","last_unstale":"2024-06-28T16:14:49.046074+0000","last_undegraded":"2024-06-28T16:14:49.046074+0000","last_fullsized":"2024-06-28T16:14:49.046074+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.045873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.045873+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.045873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:29:37.775532+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"0'0","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.128341+0000","last_change":"2024-06-28T16:14:46.128341+0000","last_active":"2024-06-28T16:14:46.128341+0000","last_peered":"2024-06-28T16:14:46.128341+0000","last_clean":"2024-06-28T16:14:46.128341+0000","last_became_active":"2024-06-28T15:21:07.027451+0000","last_became_peered":"2024-06-28T15:21:07.027451+0000","last_unstale":"2024-06-28T16:14:46.128341+0000","last_undegraded":"2024-06-28T16:14:46.128341+0000","last_fullsized":"2024-06-28T16:14:46.128341+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:46.128071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:46.128071+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.128071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:30:28.326677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"260'16","reported_seq":106,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.081079+0000","last_change":"2024-06-28T16:14:43.081145+0000","last_active":"2024-06-28T16:14:43.081079+0000","last_peered":"2024-06-28T16:14:43.081079+0000","last_clean":"2024-06-28T16:14:43.081079+0000","last_became_active":"2024-06-28T15:17:24.026271+0000","last_became_peered":"2024-06-28T15:17:24.026271+0000","last_unstale":"2024-06-28T16:14:43.081079+0000","last_undegraded":"2024-06-28T16:14:43.081079+0000","last_fullsized":"2024-06-28T16:14:43.081079+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'16","last_scrub_stamp":"2024-06-28T16:14:43.080857+0000","last_deep_scrub":"260'16","last_deep_scrub_stamp":"2024-06-28T16:14:43.080857+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.080857+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:01:28.864357+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":48,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.748084+0000","last_change":"2024-06-28T16:15:00.748137+0000","last_active":"2024-06-28T16:15:00.748084+0000","last_peered":"2024-06-28T16:15:00.748084+0000","last_clean":"2024-06-28T16:15:00.748084+0000","last_became_active":"2024-06-28T15:37:33.325849+0000","last_became_peered":"2024-06-28T15:37:33.325849+0000","last_unstale":"2024-06-28T16:15:00.748084+0000","last_undegraded":"2024-06-28T16:15:00.748084+0000","last_fullsized":"2024-06-28T16:15:00.748084+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:00.747890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:00.747890+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.747890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:59:35.430960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"257'5","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.972221+0000","last_change":"2024-06-28T15:21:11.108377+0000","last_active":"2024-06-28T16:14:39.972221+0000","last_peered":"2024-06-28T16:14:39.972221+0000","last_clean":"2024-06-28T16:14:39.972221+0000","last_became_active":"2024-06-28T15:21:11.107498+0000","last_became_peered":"2024-06-28T15:21:11.107498+0000","last_unstale":"2024-06-28T16:14:39.972221+0000","last_undegraded":"2024-06-28T16:14:39.972221+0000","last_fullsized":"2024-06-28T16:14:39.972221+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"260'8","reported_seq":76,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.130905+0000","last_change":"2024-06-28T16:14:41.130933+0000","last_active":"2024-06-28T16:14:41.130905+0000","last_peered":"2024-06-28T16:14:41.130905+0000","last_clean":"2024-06-28T16:14:41.130905+0000","last_became_active":"2024-06-28T15:21:07.029525+0000","last_became_peered":"2024-06-28T15:21:07.029525+0000","last_unstale":"2024-06-28T16:14:41.130905+0000","last_undegraded":"2024-06-28T16:14:41.130905+0000","last_fullsized":"2024-06-28T16:14:41.130905+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'8","last_scrub_stamp":"2024-06-28T16:14:41.130824+0000","last_deep_scrub":"260'8","last_deep_scrub_stamp":"2024-06-28T16:14:41.130824+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.130824+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:31:52.999376+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":11,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"260'34","reported_seq":160,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.199739+0000","last_change":"2024-06-28T16:14:52.199853+0000","last_active":"2024-06-28T16:14:52.199739+0000","last_peered":"2024-06-28T16:14:52.199739+0000","last_clean":"2024-06-28T16:14:52.199739+0000","last_became_active":"2024-06-28T15:17:24.007649+0000","last_became_peered":"2024-06-28T15:17:24.007649+0000","last_unstale":"2024-06-28T16:14:52.199739+0000","last_undegraded":"2024-06-28T16:14:52.199739+0000","last_fullsized":"2024-06-28T16:14:52.199739+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'34","last_scrub_stamp":"2024-06-28T16:14:52.199460+0000","last_deep_scrub":"260'34","last_deep_scrub_stamp":"2024-06-28T16:14:52.199460+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.199460+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:55:51.649887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":172,"num_read_kb":122,"num_write":76,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:35.323323+0000","last_change":"2024-06-28T15:37:33.335759+0000","last_active":"2024-06-28T16:13:35.323323+0000","last_peered":"2024-06-28T16:13:35.323323+0000","last_clean":"2024-06-28T16:13:35.323323+0000","last_became_active":"2024-06-28T15:37:33.334082+0000","last_became_peered":"2024-06-28T15:37:33.334082+0000","last_unstale":"2024-06-28T16:13:35.323323+0000","last_undegraded":"2024-06-28T16:13:35.323323+0000","last_fullsized":"2024-06-28T16:13:35.323323+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"0'0","reported_seq":47,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.113301+0000","last_change":"2024-06-28T15:21:11.108205+0000","last_active":"2024-06-28T16:13:15.113301+0000","last_peered":"2024-06-28T16:13:15.113301+0000","last_clean":"2024-06-28T16:13:15.113301+0000","last_became_active":"2024-06-28T15:21:11.107686+0000","last_became_peered":"2024-06-28T15:21:11.107686+0000","last_unstale":"2024-06-28T16:13:15.113301+0000","last_undegraded":"2024-06-28T16:13:15.113301+0000","last_fullsized":"2024-06-28T16:13:15.113301+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"0'0","reported_seq":62,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.052046+0000","last_change":"2024-06-28T16:14:54.052046+0000","last_active":"2024-06-28T16:14:54.052046+0000","last_peered":"2024-06-28T16:14:54.052046+0000","last_clean":"2024-06-28T16:14:54.052046+0000","last_became_active":"2024-06-28T15:21:07.032719+0000","last_became_peered":"2024-06-28T15:21:07.032719+0000","last_unstale":"2024-06-28T16:14:54.052046+0000","last_undegraded":"2024-06-28T16:14:54.052046+0000","last_fullsized":"2024-06-28T16:14:54.052046+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:54.051795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:54.051795+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.051795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:51:48.898614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"260'45","reported_seq":191,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.159340+0000","last_change":"2024-06-28T16:14:40.159396+0000","last_active":"2024-06-28T16:14:40.159340+0000","last_peered":"2024-06-28T16:14:40.159340+0000","last_clean":"2024-06-28T16:14:40.159340+0000","last_became_active":"2024-06-28T15:17:24.029532+0000","last_became_peered":"2024-06-28T15:17:24.029532+0000","last_unstale":"2024-06-28T16:14:40.159340+0000","last_undegraded":"2024-06-28T16:14:40.159340+0000","last_fullsized":"2024-06-28T16:14:40.159340+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'45","last_scrub_stamp":"2024-06-28T16:14:40.159221+0000","last_deep_scrub":"260'45","last_deep_scrub_stamp":"2024-06-28T16:14:40.159221+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.159221+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:08:46.885820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":234,"num_read_kb":165,"num_write":86,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.128083+0000","last_change":"2024-06-28T15:37:33.348528+0000","last_active":"2024-06-28T16:13:45.128083+0000","last_peered":"2024-06-28T16:13:45.128083+0000","last_clean":"2024-06-28T16:13:45.128083+0000","last_became_active":"2024-06-28T15:37:33.347691+0000","last_became_peered":"2024-06-28T15:37:33.347691+0000","last_unstale":"2024-06-28T16:13:45.128083+0000","last_undegraded":"2024-06-28T16:13:45.128083+0000","last_fullsized":"2024-06-28T16:13:45.128083+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"0'0","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.580849+0000","last_change":"2024-06-28T16:14:48.580849+0000","last_active":"2024-06-28T16:14:48.580849+0000","last_peered":"2024-06-28T16:14:48.580849+0000","last_clean":"2024-06-28T16:14:48.580849+0000","last_became_active":"2024-06-28T15:21:11.091656+0000","last_became_peered":"2024-06-28T15:21:11.091656+0000","last_unstale":"2024-06-28T16:14:48.580849+0000","last_undegraded":"2024-06-28T16:14:48.580849+0000","last_fullsized":"2024-06-28T16:14:48.580849+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:48.580598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:48.580598+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.580598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:07:09.489282+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"260'5","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.099851+0000","last_change":"2024-06-28T16:14:39.099851+0000","last_active":"2024-06-28T16:14:39.099851+0000","last_peered":"2024-06-28T16:14:39.099851+0000","last_clean":"2024-06-28T16:14:39.099851+0000","last_became_active":"2024-06-28T15:21:07.029495+0000","last_became_peered":"2024-06-28T15:21:07.029495+0000","last_unstale":"2024-06-28T16:14:39.099851+0000","last_undegraded":"2024-06-28T16:14:39.099851+0000","last_fullsized":"2024-06-28T16:14:39.099851+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'5","last_scrub_stamp":"2024-06-28T16:14:39.099694+0000","last_deep_scrub":"260'5","last_deep_scrub_stamp":"2024-06-28T16:14:39.099694+0000","last_clean_scrub_stamp":"2024-06-28T16:14:39.099694+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:24:20.828667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":318,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"260'39","reported_seq":171,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.649465+0000","last_change":"2024-06-28T16:14:50.649616+0000","last_active":"2024-06-28T16:14:50.649465+0000","last_peered":"2024-06-28T16:14:50.649465+0000","last_clean":"2024-06-28T16:14:50.649465+0000","last_became_active":"2024-06-28T15:17:24.021200+0000","last_became_peered":"2024-06-28T15:17:24.021200+0000","last_unstale":"2024-06-28T16:14:50.649465+0000","last_undegraded":"2024-06-28T16:14:50.649465+0000","last_fullsized":"2024-06-28T16:14:50.649465+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'39","last_scrub_stamp":"2024-06-28T16:14:50.649227+0000","last_deep_scrub":"260'39","last_deep_scrub_stamp":"2024-06-28T16:14:50.649227+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.649227+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:23:42.138261+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":191,"num_read_kb":133,"num_write":87,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"260'2","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.759533+0000","last_change":"2024-06-28T16:15:04.759599+0000","last_active":"2024-06-28T16:15:04.759533+0000","last_peered":"2024-06-28T16:15:04.759533+0000","last_clean":"2024-06-28T16:15:04.759533+0000","last_became_active":"2024-06-28T15:37:33.335464+0000","last_became_peered":"2024-06-28T15:37:33.335464+0000","last_unstale":"2024-06-28T16:15:04.759533+0000","last_undegraded":"2024-06-28T16:15:04.759533+0000","last_fullsized":"2024-06-28T16:15:04.759533+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:04.759350+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:04.759350+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.759350+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:56:34.598481+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"257'3","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.270056+0000","last_change":"2024-06-28T16:15:17.270119+0000","last_active":"2024-06-28T16:15:17.270056+0000","last_peered":"2024-06-28T16:15:17.270056+0000","last_clean":"2024-06-28T16:15:17.270056+0000","last_became_active":"2024-06-28T15:21:11.091647+0000","last_became_peered":"2024-06-28T15:21:11.091647+0000","last_unstale":"2024-06-28T16:15:17.270056+0000","last_undegraded":"2024-06-28T16:15:17.270056+0000","last_fullsized":"2024-06-28T16:15:17.270056+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:17.269853+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:17.269853+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.269853+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:16:54.117703+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"257'3","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.098521+0000","last_change":"2024-06-28T16:14:58.098585+0000","last_active":"2024-06-28T16:14:58.098521+0000","last_peered":"2024-06-28T16:14:58.098521+0000","last_clean":"2024-06-28T16:14:58.098521+0000","last_became_active":"2024-06-28T15:21:07.046218+0000","last_became_peered":"2024-06-28T15:21:07.046218+0000","last_unstale":"2024-06-28T16:14:58.098521+0000","last_undegraded":"2024-06-28T16:14:58.098521+0000","last_fullsized":"2024-06-28T16:14:58.098521+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:14:58.098319+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:14:58.098319+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.098319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:38:23.012737+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"260'54","reported_seq":217,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.106471+0000","last_change":"2024-06-28T16:14:47.106537+0000","last_active":"2024-06-28T16:14:47.106471+0000","last_peered":"2024-06-28T16:14:47.106471+0000","last_clean":"2024-06-28T16:14:47.106471+0000","last_became_active":"2024-06-28T15:17:24.010230+0000","last_became_peered":"2024-06-28T15:17:24.010230+0000","last_unstale":"2024-06-28T16:14:47.106471+0000","last_undegraded":"2024-06-28T16:14:47.106471+0000","last_fullsized":"2024-06-28T16:14:47.106471+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'54","last_scrub_stamp":"2024-06-28T16:14:47.106259+0000","last_deep_scrub":"260'54","last_deep_scrub_stamp":"2024-06-28T16:14:47.106259+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.106259+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:33:32.525834+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":283,"num_read_kb":205,"num_write":97,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:35.594467+0000","last_change":"2024-06-28T15:37:33.343356+0000","last_active":"2024-06-28T16:13:35.594467+0000","last_peered":"2024-06-28T16:13:35.594467+0000","last_clean":"2024-06-28T16:13:35.594467+0000","last_became_active":"2024-06-28T15:37:33.342585+0000","last_became_peered":"2024-06-28T15:37:33.342585+0000","last_unstale":"2024-06-28T16:13:35.594467+0000","last_undegraded":"2024-06-28T16:13:35.594467+0000","last_fullsized":"2024-06-28T16:13:35.594467+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"260'3","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.666961+0000","last_change":"2024-06-28T16:14:56.667083+0000","last_active":"2024-06-28T16:14:56.666961+0000","last_peered":"2024-06-28T16:14:56.666961+0000","last_clean":"2024-06-28T16:14:56.666961+0000","last_became_active":"2024-06-28T15:21:11.096884+0000","last_became_peered":"2024-06-28T15:21:11.096884+0000","last_unstale":"2024-06-28T16:14:56.666961+0000","last_undegraded":"2024-06-28T16:14:56.666961+0000","last_fullsized":"2024-06-28T16:14:56.666961+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:14:56.666741+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:14:56.666741+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.666741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:25:21.454528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"260'12","reported_seq":90,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.959056+0000","last_change":"2024-06-28T16:14:52.959056+0000","last_active":"2024-06-28T16:14:52.959056+0000","last_peered":"2024-06-28T16:14:52.959056+0000","last_clean":"2024-06-28T16:14:52.959056+0000","last_became_active":"2024-06-28T15:21:07.032545+0000","last_became_peered":"2024-06-28T15:21:07.032545+0000","last_unstale":"2024-06-28T16:14:52.959056+0000","last_undegraded":"2024-06-28T16:14:52.959056+0000","last_fullsized":"2024-06-28T16:14:52.959056+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'12","last_scrub_stamp":"2024-06-28T16:14:52.958803+0000","last_deep_scrub":"260'12","last_deep_scrub_stamp":"2024-06-28T16:14:52.958803+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.958803+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:27:34.294061+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":35842,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"260'40","reported_seq":180,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.119887+0000","last_change":"2024-06-28T16:14:42.119952+0000","last_active":"2024-06-28T16:14:42.119887+0000","last_peered":"2024-06-28T16:14:42.119887+0000","last_clean":"2024-06-28T16:14:42.119887+0000","last_became_active":"2024-06-28T15:17:24.026702+0000","last_became_peered":"2024-06-28T15:17:24.026702+0000","last_unstale":"2024-06-28T16:14:42.119887+0000","last_undegraded":"2024-06-28T16:14:42.119887+0000","last_fullsized":"2024-06-28T16:14:42.119887+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'40","last_scrub_stamp":"2024-06-28T16:14:42.119697+0000","last_deep_scrub":"260'40","last_deep_scrub_stamp":"2024-06-28T16:14:42.119697+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.119697+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:57:41.013540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":150,"num_write":83,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":31,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.804441+0000","last_change":"2024-06-28T15:37:33.343533+0000","last_active":"2024-06-28T16:14:41.804441+0000","last_peered":"2024-06-28T16:14:41.804441+0000","last_clean":"2024-06-28T16:14:41.804441+0000","last_became_active":"2024-06-28T15:37:33.342889+0000","last_became_peered":"2024-06-28T15:37:33.342889+0000","last_unstale":"2024-06-28T16:14:41.804441+0000","last_undegraded":"2024-06-28T16:14:41.804441+0000","last_fullsized":"2024-06-28T16:14:41.804441+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"260'2","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.187242+0000","last_change":"2024-06-28T16:15:06.187312+0000","last_active":"2024-06-28T16:15:06.187242+0000","last_peered":"2024-06-28T16:15:06.187242+0000","last_clean":"2024-06-28T16:15:06.187242+0000","last_became_active":"2024-06-28T15:21:11.092614+0000","last_became_peered":"2024-06-28T15:21:11.092614+0000","last_unstale":"2024-06-28T16:15:06.187242+0000","last_undegraded":"2024-06-28T16:15:06.187242+0000","last_fullsized":"2024-06-28T16:15:06.187242+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:06.187009+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:06.187009+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.187009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:50:30.362690+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"257'3","reported_seq":68,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.626086+0000","last_change":"2024-06-28T16:14:47.626086+0000","last_active":"2024-06-28T16:14:47.626086+0000","last_peered":"2024-06-28T16:14:47.626086+0000","last_clean":"2024-06-28T16:14:47.626086+0000","last_became_active":"2024-06-28T15:21:07.034845+0000","last_became_peered":"2024-06-28T15:21:07.034845+0000","last_unstale":"2024-06-28T16:14:47.626086+0000","last_undegraded":"2024-06-28T16:14:47.626086+0000","last_fullsized":"2024-06-28T16:14:47.626086+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:14:47.625736+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:14:47.625736+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.625736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:32:04.945420+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"260'28","reported_seq":148,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.166666+0000","last_change":"2024-06-28T16:14:41.166687+0000","last_active":"2024-06-28T16:14:41.166666+0000","last_peered":"2024-06-28T16:14:41.166666+0000","last_clean":"2024-06-28T16:14:41.166666+0000","last_became_active":"2024-06-28T15:17:24.014034+0000","last_became_peered":"2024-06-28T15:17:24.014034+0000","last_unstale":"2024-06-28T16:14:41.166666+0000","last_undegraded":"2024-06-28T16:14:41.166666+0000","last_fullsized":"2024-06-28T16:14:41.166666+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'28","last_scrub_stamp":"2024-06-28T16:14:41.166589+0000","last_deep_scrub":"260'28","last_deep_scrub_stamp":"2024-06-28T16:14:41.166589+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.166589+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:11:28.889098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":149,"num_read_kb":105,"num_write":55,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":31,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.878440+0000","last_change":"2024-06-28T15:37:33.340550+0000","last_active":"2024-06-28T16:14:40.878440+0000","last_peered":"2024-06-28T16:14:40.878440+0000","last_clean":"2024-06-28T16:14:40.878440+0000","last_became_active":"2024-06-28T15:37:33.339831+0000","last_became_peered":"2024-06-28T15:37:33.339831+0000","last_unstale":"2024-06-28T16:14:40.878440+0000","last_undegraded":"2024-06-28T16:14:40.878440+0000","last_fullsized":"2024-06-28T16:14:40.878440+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"260'7","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.436003+0000","last_change":"2024-06-28T15:21:11.096084+0000","last_active":"2024-06-28T16:14:33.436003+0000","last_peered":"2024-06-28T16:14:33.436003+0000","last_clean":"2024-06-28T16:14:33.436003+0000","last_became_active":"2024-06-28T15:21:11.088011+0000","last_became_peered":"2024-06-28T15:21:11.088011+0000","last_unstale":"2024-06-28T16:14:33.436003+0000","last_undegraded":"2024-06-28T16:14:33.436003+0000","last_fullsized":"2024-06-28T16:14:33.436003+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":20522,"num_write":11,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"0'0","reported_seq":62,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.347231+0000","last_change":"2024-06-28T16:15:06.347289+0000","last_active":"2024-06-28T16:15:06.347231+0000","last_peered":"2024-06-28T16:15:06.347231+0000","last_clean":"2024-06-28T16:15:06.347231+0000","last_became_active":"2024-06-28T15:21:07.030690+0000","last_became_peered":"2024-06-28T15:21:07.030690+0000","last_unstale":"2024-06-28T16:15:06.347231+0000","last_undegraded":"2024-06-28T16:15:06.347231+0000","last_fullsized":"2024-06-28T16:15:06.347231+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:06.347026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:06.347026+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.347026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:04:58.965982+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"260'31","reported_seq":153,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.094693+0000","last_change":"2024-06-28T16:14:44.094763+0000","last_active":"2024-06-28T16:14:44.094693+0000","last_peered":"2024-06-28T16:14:44.094693+0000","last_clean":"2024-06-28T16:14:44.094693+0000","last_became_active":"2024-06-28T15:17:24.026404+0000","last_became_peered":"2024-06-28T15:17:24.026404+0000","last_unstale":"2024-06-28T16:14:44.094693+0000","last_undegraded":"2024-06-28T16:14:44.094693+0000","last_fullsized":"2024-06-28T16:14:44.094693+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'31","last_scrub_stamp":"2024-06-28T16:14:44.094496+0000","last_deep_scrub":"260'31","last_deep_scrub_stamp":"2024-06-28T16:14:44.094496+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.094496+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:11:57.383608+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":161,"num_read_kb":111,"num_write":59,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":31,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:45.127686+0000","last_change":"2024-06-28T15:37:33.332984+0000","last_active":"2024-06-28T16:13:45.127686+0000","last_peered":"2024-06-28T16:13:45.127686+0000","last_clean":"2024-06-28T16:13:45.127686+0000","last_became_active":"2024-06-28T15:37:33.332245+0000","last_became_peered":"2024-06-28T15:37:33.332245+0000","last_unstale":"2024-06-28T16:13:45.127686+0000","last_undegraded":"2024-06-28T16:13:45.127686+0000","last_fullsized":"2024-06-28T16:13:45.127686+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"0'0","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.610366+0000","last_change":"2024-06-28T16:14:52.610366+0000","last_active":"2024-06-28T16:14:52.610366+0000","last_peered":"2024-06-28T16:14:52.610366+0000","last_clean":"2024-06-28T16:14:52.610366+0000","last_became_active":"2024-06-28T15:21:11.096926+0000","last_became_peered":"2024-06-28T15:21:11.096926+0000","last_unstale":"2024-06-28T16:14:52.610366+0000","last_undegraded":"2024-06-28T16:14:52.610366+0000","last_fullsized":"2024-06-28T16:14:52.610366+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:52.610021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:52.610021+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.610021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:07:41.471423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"260'2","reported_seq":65,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.578940+0000","last_change":"2024-06-28T16:14:56.578940+0000","last_active":"2024-06-28T16:14:56.578940+0000","last_peered":"2024-06-28T16:14:56.578940+0000","last_clean":"2024-06-28T16:14:56.578940+0000","last_became_active":"2024-06-28T15:21:07.036735+0000","last_became_peered":"2024-06-28T15:21:07.036735+0000","last_unstale":"2024-06-28T16:14:56.578940+0000","last_undegraded":"2024-06-28T16:14:56.578940+0000","last_fullsized":"2024-06-28T16:14:56.578940+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:56.578669+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:56.578669+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.578669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:47:53.590130+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"260'49","reported_seq":206,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.629147+0000","last_change":"2024-06-28T16:14:49.629207+0000","last_active":"2024-06-28T16:14:49.629147+0000","last_peered":"2024-06-28T16:14:49.629147+0000","last_clean":"2024-06-28T16:14:49.629147+0000","last_became_active":"2024-06-28T15:17:24.025855+0000","last_became_peered":"2024-06-28T15:17:24.025855+0000","last_unstale":"2024-06-28T16:14:49.629147+0000","last_undegraded":"2024-06-28T16:14:49.629147+0000","last_fullsized":"2024-06-28T16:14:49.629147+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'49","last_scrub_stamp":"2024-06-28T16:14:49.628959+0000","last_deep_scrub":"260'49","last_deep_scrub_stamp":"2024-06-28T16:14:49.628959+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.628959+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:27:20.197035+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":190,"num_write":87,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":31,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.423891+0000","last_change":"2024-06-28T15:37:33.323701+0000","last_active":"2024-06-28T16:14:40.423891+0000","last_peered":"2024-06-28T16:14:40.423891+0000","last_clean":"2024-06-28T16:14:40.423891+0000","last_became_active":"2024-06-28T15:37:33.322927+0000","last_became_peered":"2024-06-28T15:37:33.322927+0000","last_unstale":"2024-06-28T16:14:40.423891+0000","last_undegraded":"2024-06-28T16:14:40.423891+0000","last_fullsized":"2024-06-28T16:14:40.423891+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":47,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:15.113538+0000","last_change":"2024-06-28T15:21:11.098521+0000","last_active":"2024-06-28T16:13:15.113538+0000","last_peered":"2024-06-28T16:13:15.113538+0000","last_clean":"2024-06-28T16:13:15.113538+0000","last_became_active":"2024-06-28T15:21:11.097766+0000","last_became_peered":"2024-06-28T15:21:11.097766+0000","last_unstale":"2024-06-28T16:13:15.113538+0000","last_undegraded":"2024-06-28T16:13:15.113538+0000","last_fullsized":"2024-06-28T16:13:15.113538+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.616012+0000","last_change":"2024-06-28T16:14:46.616012+0000","last_active":"2024-06-28T16:14:46.616012+0000","last_peered":"2024-06-28T16:14:46.616012+0000","last_clean":"2024-06-28T16:14:46.616012+0000","last_became_active":"2024-06-28T15:21:07.039381+0000","last_became_peered":"2024-06-28T15:21:07.039381+0000","last_unstale":"2024-06-28T16:14:46.616012+0000","last_undegraded":"2024-06-28T16:14:46.616012+0000","last_fullsized":"2024-06-28T16:14:46.616012+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:46.615676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:46.615676+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.615676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:02:38.667378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"260'26","reported_seq":146,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.130914+0000","last_change":"2024-06-28T16:14:42.130952+0000","last_active":"2024-06-28T16:14:42.130914+0000","last_peered":"2024-06-28T16:14:42.130914+0000","last_clean":"2024-06-28T16:14:42.130914+0000","last_became_active":"2024-06-28T15:17:24.026580+0000","last_became_peered":"2024-06-28T15:17:24.026580+0000","last_unstale":"2024-06-28T16:14:42.130914+0000","last_undegraded":"2024-06-28T16:14:42.130914+0000","last_fullsized":"2024-06-28T16:14:42.130914+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'26","last_scrub_stamp":"2024-06-28T16:14:42.130814+0000","last_deep_scrub":"260'26","last_deep_scrub_stamp":"2024-06-28T16:14:42.130814+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.130814+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:03:06.475528+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":143,"num_read_kb":101,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"260'2","reported_seq":40,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.336046+0000","last_change":"2024-06-28T16:15:11.336114+0000","last_active":"2024-06-28T16:15:11.336046+0000","last_peered":"2024-06-28T16:15:11.336046+0000","last_clean":"2024-06-28T16:15:11.336046+0000","last_became_active":"2024-06-28T15:37:33.349137+0000","last_became_peered":"2024-06-28T15:37:33.349137+0000","last_unstale":"2024-06-28T16:15:11.336046+0000","last_undegraded":"2024-06-28T16:15:11.336046+0000","last_fullsized":"2024-06-28T16:15:11.336046+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:11.335857+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:11.335857+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.335857+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:55:35.045287+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"260'6","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.298630+0000","last_change":"2024-06-28T16:15:19.298763+0000","last_active":"2024-06-28T16:15:19.298630+0000","last_peered":"2024-06-28T16:15:19.298630+0000","last_clean":"2024-06-28T16:15:19.298630+0000","last_became_active":"2024-06-28T15:21:11.104270+0000","last_became_peered":"2024-06-28T15:21:11.104270+0000","last_unstale":"2024-06-28T16:15:19.298630+0000","last_undegraded":"2024-06-28T16:15:19.298630+0000","last_fullsized":"2024-06-28T16:15:19.298630+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:15:19.298407+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:15:19.298407+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.298407+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:47:09.226133+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":11,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"260'6","reported_seq":72,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.653099+0000","last_change":"2024-06-28T16:14:44.653159+0000","last_active":"2024-06-28T16:14:44.653099+0000","last_peered":"2024-06-28T16:14:44.653099+0000","last_clean":"2024-06-28T16:14:44.653099+0000","last_became_active":"2024-06-28T15:21:07.048639+0000","last_became_peered":"2024-06-28T15:21:07.048639+0000","last_unstale":"2024-06-28T16:14:44.653099+0000","last_undegraded":"2024-06-28T16:14:44.653099+0000","last_fullsized":"2024-06-28T16:14:44.653099+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:14:44.652902+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:14:44.652902+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.652902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:58:20.490373+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":12288,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"260'22","reported_seq":123,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.693623+0000","last_change":"2024-06-28T16:14:41.693689+0000","last_active":"2024-06-28T16:14:41.693623+0000","last_peered":"2024-06-28T16:14:41.693623+0000","last_clean":"2024-06-28T16:14:41.693623+0000","last_became_active":"2024-06-28T15:17:24.006784+0000","last_became_peered":"2024-06-28T15:17:24.006784+0000","last_unstale":"2024-06-28T16:14:41.693623+0000","last_undegraded":"2024-06-28T16:14:41.693623+0000","last_fullsized":"2024-06-28T16:14:41.693623+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'22","last_scrub_stamp":"2024-06-28T16:14:41.693433+0000","last_deep_scrub":"260'22","last_deep_scrub_stamp":"2024-06-28T16:14:41.693433+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.693433+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:31:21.802215+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":64,"num_write":51,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":30,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:25.605834+0000","last_change":"2024-06-28T15:37:33.334905+0000","last_active":"2024-06-28T16:14:25.605834+0000","last_peered":"2024-06-28T16:14:25.605834+0000","last_clean":"2024-06-28T16:14:25.605834+0000","last_became_active":"2024-06-28T15:37:33.333737+0000","last_became_peered":"2024-06-28T15:37:33.333737+0000","last_unstale":"2024-06-28T16:14:25.605834+0000","last_undegraded":"2024-06-28T16:14:25.605834+0000","last_fullsized":"2024-06-28T16:14:25.605834+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":49,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:25.108936+0000","last_change":"2024-06-28T15:21:11.092678+0000","last_active":"2024-06-28T16:13:25.108936+0000","last_peered":"2024-06-28T16:13:25.108936+0000","last_clean":"2024-06-28T16:13:25.108936+0000","last_became_active":"2024-06-28T15:21:11.092034+0000","last_became_peered":"2024-06-28T15:21:11.092034+0000","last_unstale":"2024-06-28T16:13:25.108936+0000","last_undegraded":"2024-06-28T16:13:25.108936+0000","last_fullsized":"2024-06-28T16:13:25.108936+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"260'3","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.606954+0000","last_change":"2024-06-28T16:14:43.606954+0000","last_active":"2024-06-28T16:14:43.606954+0000","last_peered":"2024-06-28T16:14:43.606954+0000","last_clean":"2024-06-28T16:14:43.606954+0000","last_became_active":"2024-06-28T15:21:07.039812+0000","last_became_peered":"2024-06-28T15:21:07.039812+0000","last_unstale":"2024-06-28T16:14:43.606954+0000","last_undegraded":"2024-06-28T16:14:43.606954+0000","last_fullsized":"2024-06-28T16:14:43.606954+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:14:43.606698+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:14:43.606698+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.606698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:49:13.801868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"260'78","reported_seq":317,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.919916+0000","last_change":"2024-06-28T16:14:41.919989+0000","last_active":"2024-06-28T16:14:41.919916+0000","last_peered":"2024-06-28T16:14:41.919916+0000","last_clean":"2024-06-28T16:14:41.919916+0000","last_became_active":"2024-06-28T15:17:24.012854+0000","last_became_peered":"2024-06-28T15:17:24.012854+0000","last_unstale":"2024-06-28T16:14:41.919916+0000","last_undegraded":"2024-06-28T16:14:41.919916+0000","last_fullsized":"2024-06-28T16:14:41.919916+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'78","last_scrub_stamp":"2024-06-28T16:14:41.919720+0000","last_deep_scrub":"260'78","last_deep_scrub_stamp":"2024-06-28T16:14:41.919720+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.919720+0000","objects_scrubbed":1,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:17:37.822460+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":483,"num_read_kb":330,"num_write":109,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4960,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":31,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.609927+0000","last_change":"2024-06-28T15:37:33.323089+0000","last_active":"2024-06-28T16:14:38.609927+0000","last_peered":"2024-06-28T16:14:38.609927+0000","last_clean":"2024-06-28T16:14:38.609927+0000","last_became_active":"2024-06-28T15:37:33.322317+0000","last_became_peered":"2024-06-28T15:37:33.322317+0000","last_unstale":"2024-06-28T16:14:38.609927+0000","last_undegraded":"2024-06-28T16:14:38.609927+0000","last_fullsized":"2024-06-28T16:14:38.609927+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.394565+0000","last_change":"2024-06-28T16:15:17.394628+0000","last_active":"2024-06-28T16:15:17.394565+0000","last_peered":"2024-06-28T16:15:17.394565+0000","last_clean":"2024-06-28T16:15:17.394565+0000","last_became_active":"2024-06-28T15:21:11.108663+0000","last_became_peered":"2024-06-28T15:21:11.108663+0000","last_unstale":"2024-06-28T16:15:17.394565+0000","last_undegraded":"2024-06-28T16:15:17.394565+0000","last_fullsized":"2024-06-28T16:15:17.394565+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:17.394367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:17.394367+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.394367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:27:51.882594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"260'3","reported_seq":68,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.135141+0000","last_change":"2024-06-28T16:14:40.135141+0000","last_active":"2024-06-28T16:14:40.135141+0000","last_peered":"2024-06-28T16:14:40.135141+0000","last_clean":"2024-06-28T16:14:40.135141+0000","last_became_active":"2024-06-28T15:21:07.022046+0000","last_became_peered":"2024-06-28T15:21:07.022046+0000","last_unstale":"2024-06-28T16:14:40.135141+0000","last_undegraded":"2024-06-28T16:14:40.135141+0000","last_fullsized":"2024-06-28T16:14:40.135141+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:14:40.134988+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:14:40.134988+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.134988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:06:49.985940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"260'31","reported_seq":145,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.121279+0000","last_change":"2024-06-28T16:14:45.121348+0000","last_active":"2024-06-28T16:14:45.121279+0000","last_peered":"2024-06-28T16:14:45.121279+0000","last_clean":"2024-06-28T16:14:45.121279+0000","last_became_active":"2024-06-28T15:17:24.010873+0000","last_became_peered":"2024-06-28T15:17:24.010873+0000","last_unstale":"2024-06-28T16:14:45.121279+0000","last_undegraded":"2024-06-28T16:14:45.121279+0000","last_fullsized":"2024-06-28T16:14:45.121279+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'31","last_scrub_stamp":"2024-06-28T16:14:45.121064+0000","last_deep_scrub":"260'31","last_deep_scrub_stamp":"2024-06-28T16:14:45.121064+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.121064+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:05:38.766333+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":94,"num_write":78,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":31,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.804295+0000","last_change":"2024-06-28T15:37:33.347898+0000","last_active":"2024-06-28T16:14:41.804295+0000","last_peered":"2024-06-28T16:14:41.804295+0000","last_clean":"2024-06-28T16:14:41.804295+0000","last_became_active":"2024-06-28T15:37:33.347075+0000","last_became_peered":"2024-06-28T15:37:33.347075+0000","last_unstale":"2024-06-28T16:14:41.804295+0000","last_undegraded":"2024-06-28T16:14:41.804295+0000","last_fullsized":"2024-06-28T16:14:41.804295+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":48,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.114588+0000","last_change":"2024-06-28T15:21:11.093738+0000","last_active":"2024-06-28T16:13:20.114588+0000","last_peered":"2024-06-28T16:13:20.114588+0000","last_clean":"2024-06-28T16:13:20.114588+0000","last_became_active":"2024-06-28T15:21:11.093082+0000","last_became_peered":"2024-06-28T15:21:11.093082+0000","last_unstale":"2024-06-28T16:13:20.114588+0000","last_undegraded":"2024-06-28T16:13:20.114588+0000","last_fullsized":"2024-06-28T16:13:20.114588+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:21:10.040344+0000","last_clean_scrub_stamp":"2024-06-28T15:21:10.040344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"257'3","reported_seq":68,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.165567+0000","last_change":"2024-06-28T16:14:46.165567+0000","last_active":"2024-06-28T16:14:46.165567+0000","last_peered":"2024-06-28T16:14:46.165567+0000","last_clean":"2024-06-28T16:14:46.165567+0000","last_became_active":"2024-06-28T15:21:07.040575+0000","last_became_peered":"2024-06-28T15:21:07.040575+0000","last_unstale":"2024-06-28T16:14:46.165567+0000","last_undegraded":"2024-06-28T16:14:46.165567+0000","last_fullsized":"2024-06-28T16:14:46.165567+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:14:46.165127+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:14:46.165127+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.165127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:42:56.667095+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"260'16","reported_seq":107,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.625878+0000","last_change":"2024-06-28T16:14:51.625944+0000","last_active":"2024-06-28T16:14:51.625878+0000","last_peered":"2024-06-28T16:14:51.625878+0000","last_clean":"2024-06-28T16:14:51.625878+0000","last_became_active":"2024-06-28T15:17:24.021464+0000","last_became_peered":"2024-06-28T15:17:24.021464+0000","last_unstale":"2024-06-28T16:14:51.625878+0000","last_undegraded":"2024-06-28T16:14:51.625878+0000","last_fullsized":"2024-06-28T16:14:51.625878+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'16","last_scrub_stamp":"2024-06-28T16:14:51.625680+0000","last_deep_scrub":"260'16","last_deep_scrub_stamp":"2024-06-28T16:14:51.625680+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.625680+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:28:15.263944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":44,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"260'341","reported_seq":971,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.145524+0000","last_change":"2024-06-28T16:14:46.145629+0000","last_active":"2024-06-28T16:14:46.145524+0000","last_peered":"2024-06-28T16:14:46.145524+0000","last_clean":"2024-06-28T16:14:46.145524+0000","last_became_active":"2024-06-28T15:00:59.941681+0000","last_became_peered":"2024-06-28T15:00:59.941681+0000","last_unstale":"2024-06-28T16:14:46.145524+0000","last_undegraded":"2024-06-28T16:14:46.145524+0000","last_fullsized":"2024-06-28T16:14:46.145524+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'341","last_scrub_stamp":"2024-06-28T16:14:46.145323+0000","last_deep_scrub":"260'341","last_deep_scrub_stamp":"2024-06-28T16:14:46.145323+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.145323+0000","objects_scrubbed":6,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:36:27.892777+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170282,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":488,"num_read_kb":26898,"num_write":1268,"num_write_kb":25985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0,5],"acting":[2,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"260'182","reported_seq":797,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.635725+0000","last_change":"2024-06-28T16:15:02.635725+0000","last_active":"2024-06-28T16:15:02.635725+0000","last_peered":"2024-06-28T16:15:02.635725+0000","last_clean":"2024-06-28T16:15:02.635725+0000","last_became_active":"2024-06-28T15:00:17.404342+0000","last_became_peered":"2024-06-28T15:00:17.404342+0000","last_unstale":"2024-06-28T16:15:02.635725+0000","last_undegraded":"2024-06-28T16:15:02.635725+0000","last_fullsized":"2024-06-28T16:15:02.635725+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'182","last_scrub_stamp":"2024-06-28T16:15:02.635444+0000","last_deep_scrub":"260'182","last_deep_scrub_stamp":"2024-06-28T16:15:02.635444+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.635444+0000","objects_scrubbed":1,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:22:41.865680+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":722,"num_read_kb":569,"num_write":267,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":272,"num_omap_keys":1,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"260'326","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.235841+0000","last_change":"2024-06-28T16:15:13.235841+0000","last_active":"2024-06-28T16:15:13.235841+0000","last_peered":"2024-06-28T16:15:13.235841+0000","last_clean":"2024-06-28T16:15:13.235841+0000","last_became_active":"2024-06-28T15:00:15.419449+0000","last_became_peered":"2024-06-28T15:00:15.419449+0000","last_unstale":"2024-06-28T16:15:13.235841+0000","last_undegraded":"2024-06-28T16:15:13.235841+0000","last_fullsized":"2024-06-28T16:15:13.235841+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'326","last_scrub_stamp":"2024-06-28T16:15:13.235513+0000","last_deep_scrub":"260'326","last_deep_scrub_stamp":"2024-06-28T16:15:13.235513+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.235513+0000","objects_scrubbed":10,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:42:49.114452+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5526738,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":485,"num_read_kb":15789,"num_write":1172,"num_write_kb":37304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"260'287","reported_seq":951,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.751954+0000","last_change":"2024-06-28T16:15:09.751954+0000","last_active":"2024-06-28T16:15:09.751954+0000","last_peered":"2024-06-28T16:15:09.751954+0000","last_clean":"2024-06-28T16:15:09.751954+0000","last_became_active":"2024-06-28T15:00:17.408128+0000","last_became_peered":"2024-06-28T15:00:17.408128+0000","last_unstale":"2024-06-28T16:15:09.751954+0000","last_undegraded":"2024-06-28T16:15:09.751954+0000","last_fullsized":"2024-06-28T16:15:09.751954+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'287","last_scrub_stamp":"2024-06-28T16:15:09.751693+0000","last_deep_scrub":"260'287","last_deep_scrub_stamp":"2024-06-28T16:15:09.751693+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.751693+0000","objects_scrubbed":2,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:50:27.173640+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1157,"num_read_kb":911,"num_write":439,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3314,"num_omap_keys":19,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"260'283","reported_seq":971,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.608868+0000","last_change":"2024-06-28T15:00:15.397113+0000","last_active":"2024-06-28T16:14:38.608868+0000","last_peered":"2024-06-28T16:14:38.608868+0000","last_clean":"2024-06-28T16:14:38.608868+0000","last_became_active":"2024-06-28T15:00:15.396469+0000","last_became_peered":"2024-06-28T15:00:15.396469+0000","last_unstale":"2024-06-28T16:14:38.608868+0000","last_undegraded":"2024-06-28T16:14:38.608868+0000","last_fullsized":"2024-06-28T16:14:38.608868+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":200,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1096214,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":504,"num_read_kb":81437,"num_write":1108,"num_write_kb":14924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"260'78","reported_seq":654,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.007487+0000","last_change":"2024-06-28T16:15:08.007487+0000","last_active":"2024-06-28T16:15:08.007487+0000","last_peered":"2024-06-28T16:15:08.007487+0000","last_clean":"2024-06-28T16:15:08.007487+0000","last_became_active":"2024-06-28T15:00:17.402542+0000","last_became_peered":"2024-06-28T15:00:17.402542+0000","last_unstale":"2024-06-28T16:15:08.007487+0000","last_undegraded":"2024-06-28T16:15:08.007487+0000","last_fullsized":"2024-06-28T16:15:08.007487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'78","last_scrub_stamp":"2024-06-28T16:15:08.007123+0000","last_deep_scrub":"260'78","last_deep_scrub_stamp":"2024-06-28T16:15:08.007123+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.007123+0000","objects_scrubbed":1,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:43:06.313520+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":306,"num_read_kb":248,"num_write":126,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":396,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"260'313","reported_seq":990,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.683417+0000","last_change":"2024-06-28T16:15:07.683559+0000","last_active":"2024-06-28T16:15:07.683417+0000","last_peered":"2024-06-28T16:15:07.683417+0000","last_clean":"2024-06-28T16:15:07.683417+0000","last_became_active":"2024-06-28T15:00:15.435122+0000","last_became_peered":"2024-06-28T15:00:15.435122+0000","last_unstale":"2024-06-28T16:15:07.683417+0000","last_undegraded":"2024-06-28T16:15:07.683417+0000","last_fullsized":"2024-06-28T16:15:07.683417+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'313","last_scrub_stamp":"2024-06-28T16:15:07.683192+0000","last_deep_scrub":"260'313","last_deep_scrub_stamp":"2024-06-28T16:15:07.683192+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.683192+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:10:01.212501+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1454317,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":492,"num_read_kb":48678,"num_write":1275,"num_write_kb":34259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"260'52","reported_seq":640,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.278889+0000","last_change":"2024-06-28T16:15:11.278889+0000","last_active":"2024-06-28T16:15:11.278889+0000","last_peered":"2024-06-28T16:15:11.278889+0000","last_clean":"2024-06-28T16:15:11.278889+0000","last_became_active":"2024-06-28T15:00:17.384932+0000","last_became_peered":"2024-06-28T15:00:17.384932+0000","last_unstale":"2024-06-28T16:15:11.278889+0000","last_undegraded":"2024-06-28T16:15:11.278889+0000","last_fullsized":"2024-06-28T16:15:11.278889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'52","last_scrub_stamp":"2024-06-28T16:15:11.278605+0000","last_deep_scrub":"260'52","last_deep_scrub_stamp":"2024-06-28T16:15:11.278605+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.278605+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:12:36.542381+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":197,"num_write":67,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"260'343","reported_seq":1002,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.835980+0000","last_change":"2024-06-28T16:15:15.836133+0000","last_active":"2024-06-28T16:15:15.835980+0000","last_peered":"2024-06-28T16:15:15.835980+0000","last_clean":"2024-06-28T16:15:15.835980+0000","last_became_active":"2024-06-28T15:00:15.430208+0000","last_became_peered":"2024-06-28T15:00:15.430208+0000","last_unstale":"2024-06-28T16:15:15.835980+0000","last_undegraded":"2024-06-28T16:15:15.835980+0000","last_fullsized":"2024-06-28T16:15:15.835980+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'343","last_scrub_stamp":"2024-06-28T16:15:15.835782+0000","last_deep_scrub":"260'343","last_deep_scrub_stamp":"2024-06-28T16:15:15.835782+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.835782+0000","objects_scrubbed":8,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:41:52.084536+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437780,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":548,"num_read_kb":39482,"num_write":1215,"num_write_kb":35176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"260'186","reported_seq":848,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.448598+0000","last_change":"2024-06-28T16:14:59.448668+0000","last_active":"2024-06-28T16:14:59.448598+0000","last_peered":"2024-06-28T16:14:59.448598+0000","last_clean":"2024-06-28T16:14:59.448598+0000","last_became_active":"2024-06-28T15:00:17.397986+0000","last_became_peered":"2024-06-28T15:00:17.397986+0000","last_unstale":"2024-06-28T16:14:59.448598+0000","last_undegraded":"2024-06-28T16:14:59.448598+0000","last_fullsized":"2024-06-28T16:14:59.448598+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'186","last_scrub_stamp":"2024-06-28T16:14:59.448384+0000","last_deep_scrub":"260'186","last_deep_scrub_stamp":"2024-06-28T16:14:59.448384+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.448384+0000","objects_scrubbed":1,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:21:42.502350+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":777,"num_read_kb":632,"num_write":274,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":473,"num_omap_keys":2,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"260'364","reported_seq":1039,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.804713+0000","last_change":"2024-06-28T16:15:17.804834+0000","last_active":"2024-06-28T16:15:17.804713+0000","last_peered":"2024-06-28T16:15:17.804713+0000","last_clean":"2024-06-28T16:15:17.804713+0000","last_became_active":"2024-06-28T15:00:15.409542+0000","last_became_peered":"2024-06-28T15:00:15.409542+0000","last_unstale":"2024-06-28T16:15:17.804713+0000","last_undegraded":"2024-06-28T16:15:17.804713+0000","last_fullsized":"2024-06-28T16:15:17.804713+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'364","last_scrub_stamp":"2024-06-28T16:15:17.804471+0000","last_deep_scrub":"260'364","last_deep_scrub_stamp":"2024-06-28T16:15:17.804471+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.804471+0000","objects_scrubbed":7,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:31:04.306481+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1667748,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":553,"num_read_kb":16754,"num_write":1370,"num_write_kb":32794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"260'44","reported_seq":618,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.888516+0000","last_change":"2024-06-28T16:14:59.888516+0000","last_active":"2024-06-28T16:14:59.888516+0000","last_peered":"2024-06-28T16:14:59.888516+0000","last_clean":"2024-06-28T16:14:59.888516+0000","last_became_active":"2024-06-28T15:00:17.401985+0000","last_became_peered":"2024-06-28T15:00:17.401985+0000","last_unstale":"2024-06-28T16:14:59.888516+0000","last_undegraded":"2024-06-28T16:14:59.888516+0000","last_fullsized":"2024-06-28T16:14:59.888516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'44","last_scrub_stamp":"2024-06-28T16:14:59.888222+0000","last_deep_scrub":"260'44","last_deep_scrub_stamp":"2024-06-28T16:14:59.888222+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.888222+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:28:28.187398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":159,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"260'328","reported_seq":961,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.802839+0000","last_change":"2024-06-28T15:00:15.425314+0000","last_active":"2024-06-28T16:14:41.802839+0000","last_peered":"2024-06-28T16:14:41.802839+0000","last_clean":"2024-06-28T16:14:41.802839+0000","last_became_active":"2024-06-28T15:00:15.424430+0000","last_became_peered":"2024-06-28T15:00:15.424430+0000","last_unstale":"2024-06-28T16:14:41.802839+0000","last_undegraded":"2024-06-28T16:14:41.802839+0000","last_fullsized":"2024-06-28T16:14:41.802839+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048602,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":476,"num_read_kb":21908,"num_write":1069,"num_write_kb":36265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"260'4326","reported_seq":5006,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.551681+0000","last_change":"2024-06-28T16:14:59.551681+0000","last_active":"2024-06-28T16:14:59.551681+0000","last_peered":"2024-06-28T16:14:59.551681+0000","last_clean":"2024-06-28T16:14:59.551681+0000","last_became_active":"2024-06-28T15:00:17.384378+0000","last_became_peered":"2024-06-28T15:00:17.384378+0000","last_unstale":"2024-06-28T16:14:59.551681+0000","last_undegraded":"2024-06-28T16:14:59.551681+0000","last_fullsized":"2024-06-28T16:14:59.551681+0000","mapping_epoch":21,"log_start":"260'4300","ondisk_log_start":"260'4300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4326","last_scrub_stamp":"2024-06-28T16:14:59.551430+0000","last_deep_scrub":"260'4326","last_deep_scrub_stamp":"2024-06-28T16:14:59.551430+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.551430+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":3000,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:51:42.703162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15352,"num_read_kb":12408,"num_write":6557,"num_write_kb":3277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"260'411","reported_seq":1130,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.971310+0000","last_change":"2024-06-28T15:00:15.403350+0000","last_active":"2024-06-28T16:14:39.971310+0000","last_peered":"2024-06-28T16:14:39.971310+0000","last_clean":"2024-06-28T16:14:39.971310+0000","last_became_active":"2024-06-28T15:00:15.401960+0000","last_became_peered":"2024-06-28T15:00:15.401960+0000","last_unstale":"2024-06-28T16:14:39.971310+0000","last_undegraded":"2024-06-28T16:14:39.971310+0000","last_fullsized":"2024-06-28T16:14:39.971310+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":400,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324665,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":810,"num_read_kb":20615,"num_write":1466,"num_write_kb":29750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"260'70","reported_seq":695,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.302209+0000","last_change":"2024-06-28T16:14:59.302209+0000","last_active":"2024-06-28T16:14:59.302209+0000","last_peered":"2024-06-28T16:14:59.302209+0000","last_clean":"2024-06-28T16:14:59.302209+0000","last_became_active":"2024-06-28T15:00:17.381196+0000","last_became_peered":"2024-06-28T15:00:17.381196+0000","last_unstale":"2024-06-28T16:14:59.302209+0000","last_undegraded":"2024-06-28T16:14:59.302209+0000","last_fullsized":"2024-06-28T16:14:59.302209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'70","last_scrub_stamp":"2024-06-28T16:14:59.301939+0000","last_deep_scrub":"260'70","last_deep_scrub_stamp":"2024-06-28T16:14:59.301939+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.301939+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:19:16.430915+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":309,"num_read_kb":273,"num_write":88,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"260'130","reported_seq":762,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.971071+0000","last_change":"2024-06-28T16:15:13.971140+0000","last_active":"2024-06-28T16:15:13.971071+0000","last_peered":"2024-06-28T16:15:13.971071+0000","last_clean":"2024-06-28T16:15:13.971071+0000","last_became_active":"2024-06-28T15:00:17.396142+0000","last_became_peered":"2024-06-28T15:00:17.396142+0000","last_unstale":"2024-06-28T16:15:13.971071+0000","last_undegraded":"2024-06-28T16:15:13.971071+0000","last_fullsized":"2024-06-28T16:15:13.971071+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'130","last_scrub_stamp":"2024-06-28T16:15:13.970878+0000","last_deep_scrub":"260'130","last_deep_scrub_stamp":"2024-06-28T16:15:13.970878+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.970878+0000","objects_scrubbed":2,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:21:34.067289+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":536,"num_read_kb":452,"num_write":191,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":546,"num_omap_keys":3,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"260'416","reported_seq":1187,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.745249+0000","last_change":"2024-06-28T15:00:15.423248+0000","last_active":"2024-06-28T16:14:34.745249+0000","last_peered":"2024-06-28T16:14:34.745249+0000","last_clean":"2024-06-28T16:14:34.745249+0000","last_became_active":"2024-06-28T15:00:15.422579+0000","last_became_peered":"2024-06-28T15:00:15.422579+0000","last_unstale":"2024-06-28T16:14:34.745249+0000","last_undegraded":"2024-06-28T16:14:34.745249+0000","last_fullsized":"2024-06-28T16:14:34.745249+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":400,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":851,"num_read_kb":96606,"num_write":1464,"num_write_kb":38692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0,1],"acting":[7,5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"260'254","reported_seq":909,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.709122+0000","last_change":"2024-06-28T16:15:07.709182+0000","last_active":"2024-06-28T16:15:07.709122+0000","last_peered":"2024-06-28T16:15:07.709122+0000","last_clean":"2024-06-28T16:15:07.709122+0000","last_became_active":"2024-06-28T15:00:17.409202+0000","last_became_peered":"2024-06-28T15:00:17.409202+0000","last_unstale":"2024-06-28T16:15:07.709122+0000","last_undegraded":"2024-06-28T16:15:07.709122+0000","last_fullsized":"2024-06-28T16:15:07.709122+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'254","last_scrub_stamp":"2024-06-28T16:15:07.708938+0000","last_deep_scrub":"260'254","last_deep_scrub_stamp":"2024-06-28T16:15:07.708938+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.708938+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:59:43.588127+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1059,"num_read_kb":851,"num_write":432,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"260'313","reported_seq":962,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.430774+0000","last_change":"2024-06-28T16:15:11.430934+0000","last_active":"2024-06-28T16:15:11.430774+0000","last_peered":"2024-06-28T16:15:11.430774+0000","last_clean":"2024-06-28T16:15:11.430774+0000","last_became_active":"2024-06-28T15:00:15.421980+0000","last_became_peered":"2024-06-28T15:00:15.421980+0000","last_unstale":"2024-06-28T16:15:11.430774+0000","last_undegraded":"2024-06-28T16:15:11.430774+0000","last_fullsized":"2024-06-28T16:15:11.430774+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'313","last_scrub_stamp":"2024-06-28T16:15:11.430569+0000","last_deep_scrub":"260'313","last_deep_scrub_stamp":"2024-06-28T16:15:11.430569+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.430569+0000","objects_scrubbed":7,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:44:39.434883+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468838,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":425,"num_read_kb":33937,"num_write":1026,"num_write_kb":43346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"260'226","reported_seq":861,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.587895+0000","last_change":"2024-06-28T16:15:01.587895+0000","last_active":"2024-06-28T16:15:01.587895+0000","last_peered":"2024-06-28T16:15:01.587895+0000","last_clean":"2024-06-28T16:15:01.587895+0000","last_became_active":"2024-06-28T15:00:17.384914+0000","last_became_peered":"2024-06-28T15:00:17.384914+0000","last_unstale":"2024-06-28T16:15:01.587895+0000","last_undegraded":"2024-06-28T16:15:01.587895+0000","last_fullsized":"2024-06-28T16:15:01.587895+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'226","last_scrub_stamp":"2024-06-28T16:15:01.587618+0000","last_deep_scrub":"260'226","last_deep_scrub_stamp":"2024-06-28T16:15:01.587618+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.587618+0000","objects_scrubbed":1,"log_size":26,"log_dups_size":200,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:02:48.171099+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":887,"num_read_kb":706,"num_write":342,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":720,"num_omap_keys":3,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"260'342","reported_seq":1201,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.377861+0000","last_change":"2024-06-28T16:15:08.377861+0000","last_active":"2024-06-28T16:15:08.377861+0000","last_peered":"2024-06-28T16:15:08.377861+0000","last_clean":"2024-06-28T16:15:08.377861+0000","last_became_active":"2024-06-28T15:00:15.411567+0000","last_became_peered":"2024-06-28T15:00:15.411567+0000","last_unstale":"2024-06-28T16:15:08.377861+0000","last_undegraded":"2024-06-28T16:15:08.377861+0000","last_fullsized":"2024-06-28T16:15:08.377861+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'342","last_scrub_stamp":"2024-06-28T16:15:08.377521+0000","last_deep_scrub":"260'342","last_deep_scrub_stamp":"2024-06-28T16:15:08.377521+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.377521+0000","objects_scrubbed":4,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:33:53.677785+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289237,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":936,"num_read_kb":14303,"num_write":1297,"num_write_kb":46873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"260'129","reported_seq":747,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.355962+0000","last_change":"2024-06-28T16:15:00.355962+0000","last_active":"2024-06-28T16:15:00.355962+0000","last_peered":"2024-06-28T16:15:00.355962+0000","last_clean":"2024-06-28T16:15:00.355962+0000","last_became_active":"2024-06-28T15:00:17.389613+0000","last_became_peered":"2024-06-28T15:00:17.389613+0000","last_unstale":"2024-06-28T16:15:00.355962+0000","last_undegraded":"2024-06-28T16:15:00.355962+0000","last_fullsized":"2024-06-28T16:15:00.355962+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'129","last_scrub_stamp":"2024-06-28T16:15:00.355683+0000","last_deep_scrub":"260'129","last_deep_scrub_stamp":"2024-06-28T16:15:00.355683+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.355683+0000","objects_scrubbed":1,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:21:03.329874+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":419,"num_write":192,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"260'320","reported_seq":1082,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.926608+0000","last_change":"2024-06-28T16:15:18.926827+0000","last_active":"2024-06-28T16:15:18.926608+0000","last_peered":"2024-06-28T16:15:18.926608+0000","last_clean":"2024-06-28T16:15:18.926608+0000","last_became_active":"2024-06-28T15:00:15.417563+0000","last_became_peered":"2024-06-28T15:00:15.417563+0000","last_unstale":"2024-06-28T16:15:18.926608+0000","last_undegraded":"2024-06-28T16:15:18.926608+0000","last_fullsized":"2024-06-28T16:15:18.926608+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'320","last_scrub_stamp":"2024-06-28T16:15:18.926356+0000","last_deep_scrub":"260'320","last_deep_scrub_stamp":"2024-06-28T16:15:18.926356+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.926356+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:35:12.865205+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":525349,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":605,"num_read_kb":12860,"num_write":1159,"num_write_kb":18563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"260'157","reported_seq":748,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.360593+0000","last_change":"2024-06-28T16:15:03.360593+0000","last_active":"2024-06-28T16:15:03.360593+0000","last_peered":"2024-06-28T16:15:03.360593+0000","last_clean":"2024-06-28T16:15:03.360593+0000","last_became_active":"2024-06-28T15:00:17.413420+0000","last_became_peered":"2024-06-28T15:00:17.413420+0000","last_unstale":"2024-06-28T16:15:03.360593+0000","last_undegraded":"2024-06-28T16:15:03.360593+0000","last_fullsized":"2024-06-28T16:15:03.360593+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'157","last_scrub_stamp":"2024-06-28T16:15:03.360224+0000","last_deep_scrub":"260'157","last_deep_scrub_stamp":"2024-06-28T16:15:03.360224+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.360224+0000","objects_scrubbed":1,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:39:07.258685+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":589,"num_read_kb":466,"num_write":222,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"260'357","reported_seq":1075,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.971220+0000","last_change":"2024-06-28T15:00:15.402501+0000","last_active":"2024-06-28T16:14:39.971220+0000","last_peered":"2024-06-28T16:14:39.971220+0000","last_clean":"2024-06-28T16:14:39.971220+0000","last_became_active":"2024-06-28T15:00:15.401370+0000","last_became_peered":"2024-06-28T15:00:15.401370+0000","last_unstale":"2024-06-28T16:14:39.971220+0000","last_undegraded":"2024-06-28T16:14:39.971220+0000","last_fullsized":"2024-06-28T16:14:39.971220+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324689,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":676,"num_read_kb":30262,"num_write":1536,"num_write_kb":36456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"260'253","reported_seq":912,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.991624+0000","last_change":"2024-06-28T16:15:06.991624+0000","last_active":"2024-06-28T16:15:06.991624+0000","last_peered":"2024-06-28T16:15:06.991624+0000","last_clean":"2024-06-28T16:15:06.991624+0000","last_became_active":"2024-06-28T15:00:17.408972+0000","last_became_peered":"2024-06-28T15:00:17.408972+0000","last_unstale":"2024-06-28T16:15:06.991624+0000","last_undegraded":"2024-06-28T16:15:06.991624+0000","last_fullsized":"2024-06-28T16:15:06.991624+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'253","last_scrub_stamp":"2024-06-28T16:15:06.991213+0000","last_deep_scrub":"260'253","last_deep_scrub_stamp":"2024-06-28T16:15:06.991213+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.991213+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:27:33.581900+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":990,"num_read_kb":787,"num_write":391,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"260'346","reported_seq":1010,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.608773+0000","last_change":"2024-06-28T15:00:15.408546+0000","last_active":"2024-06-28T16:14:38.608773+0000","last_peered":"2024-06-28T16:14:38.608773+0000","last_clean":"2024-06-28T16:14:38.608773+0000","last_became_active":"2024-06-28T15:00:15.407552+0000","last_became_peered":"2024-06-28T15:00:15.407552+0000","last_unstale":"2024-06-28T16:14:38.608773+0000","last_undegraded":"2024-06-28T16:14:38.608773+0000","last_fullsized":"2024-06-28T16:14:38.608773+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":300,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278750,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":513,"num_read_kb":23448,"num_write":1336,"num_write_kb":46171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"260'146","reported_seq":693,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.984342+0000","last_change":"2024-06-28T16:14:45.984342+0000","last_active":"2024-06-28T16:14:45.984342+0000","last_peered":"2024-06-28T16:14:45.984342+0000","last_clean":"2024-06-28T16:14:45.984342+0000","last_became_active":"2024-06-28T15:00:25.427734+0000","last_became_peered":"2024-06-28T15:00:25.427734+0000","last_unstale":"2024-06-28T16:14:45.984342+0000","last_undegraded":"2024-06-28T16:14:45.984342+0000","last_fullsized":"2024-06-28T16:14:45.984342+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'146","last_scrub_stamp":"2024-06-28T16:14:45.984045+0000","last_deep_scrub":"260'146","last_deep_scrub_stamp":"2024-06-28T16:14:45.984045+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.984045+0000","objects_scrubbed":4,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:59:27.363330+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":233,"num_read_kb":201,"num_write":347,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"260'214","reported_seq":832,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.190384+0000","last_change":"2024-06-28T16:15:16.190448+0000","last_active":"2024-06-28T16:15:16.190384+0000","last_peered":"2024-06-28T16:15:16.190384+0000","last_clean":"2024-06-28T16:15:16.190384+0000","last_became_active":"2024-06-28T15:00:17.382344+0000","last_became_peered":"2024-06-28T15:00:17.382344+0000","last_unstale":"2024-06-28T16:15:16.190384+0000","last_undegraded":"2024-06-28T16:15:16.190384+0000","last_fullsized":"2024-06-28T16:15:16.190384+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'214","last_scrub_stamp":"2024-06-28T16:15:16.190165+0000","last_deep_scrub":"260'214","last_deep_scrub_stamp":"2024-06-28T16:15:16.190165+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.190165+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":200,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:54:00.610916+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":869,"num_read_kb":692,"num_write":351,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"260'292","reported_seq":946,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.802685+0000","last_change":"2024-06-28T15:00:15.423784+0000","last_active":"2024-06-28T16:14:41.802685+0000","last_peered":"2024-06-28T16:14:41.802685+0000","last_clean":"2024-06-28T16:14:41.802685+0000","last_became_active":"2024-06-28T15:00:15.420739+0000","last_became_peered":"2024-06-28T15:00:15.420739+0000","last_unstale":"2024-06-28T16:14:41.802685+0000","last_undegraded":"2024-06-28T16:14:41.802685+0000","last_fullsized":"2024-06-28T16:14:41.802685+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":200,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6022775,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":452,"num_read_kb":28797,"num_write":1102,"num_write_kb":30225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":512,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.665224+0000","last_change":"2024-06-28T16:15:06.665224+0000","last_active":"2024-06-28T16:15:06.665224+0000","last_peered":"2024-06-28T16:15:06.665224+0000","last_clean":"2024-06-28T16:15:06.665224+0000","last_became_active":"2024-06-28T15:00:19.400681+0000","last_became_peered":"2024-06-28T15:00:19.400681+0000","last_unstale":"2024-06-28T16:15:06.665224+0000","last_undegraded":"2024-06-28T16:15:06.665224+0000","last_fullsized":"2024-06-28T16:15:06.665224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:06.664972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:06.664972+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.664972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:48:36.340497+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"260'328","reported_seq":962,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:20.216253+0000","last_change":"2024-06-28T16:15:20.216253+0000","last_active":"2024-06-28T16:15:20.216253+0000","last_peered":"2024-06-28T16:15:20.216253+0000","last_clean":"2024-06-28T16:15:20.216253+0000","last_became_active":"2024-06-28T15:00:21.390609+0000","last_became_peered":"2024-06-28T15:00:21.390609+0000","last_unstale":"2024-06-28T16:15:20.216253+0000","last_undegraded":"2024-06-28T16:15:20.216253+0000","last_fullsized":"2024-06-28T16:15:20.216253+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'328","last_scrub_stamp":"2024-06-28T16:15:20.215969+0000","last_deep_scrub":"260'328","last_deep_scrub_stamp":"2024-06-28T16:15:20.215969+0000","last_clean_scrub_stamp":"2024-06-28T16:15:20.215969+0000","objects_scrubbed":4,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:17:23.688350+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42630,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":530,"num_read_kb":1085,"num_write":406,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.711989+0000","last_change":"2024-06-28T16:15:03.711989+0000","last_active":"2024-06-28T16:15:03.711989+0000","last_peered":"2024-06-28T16:15:03.711989+0000","last_clean":"2024-06-28T16:15:03.711989+0000","last_became_active":"2024-06-28T15:00:23.407521+0000","last_became_peered":"2024-06-28T15:00:23.407521+0000","last_unstale":"2024-06-28T16:15:03.711989+0000","last_undegraded":"2024-06-28T16:15:03.711989+0000","last_fullsized":"2024-06-28T16:15:03.711989+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:03.711749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:03.711749+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.711749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:10:02.315582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"260'2406","reported_seq":4281,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.648926+0000","last_change":"2024-06-28T16:14:46.649004+0000","last_active":"2024-06-28T16:14:46.648926+0000","last_peered":"2024-06-28T16:14:46.648926+0000","last_clean":"2024-06-28T16:14:46.648926+0000","last_became_active":"2024-06-28T15:00:25.440083+0000","last_became_peered":"2024-06-28T15:00:25.440083+0000","last_unstale":"2024-06-28T16:14:46.648926+0000","last_undegraded":"2024-06-28T16:14:46.648926+0000","last_fullsized":"2024-06-28T16:14:46.648926+0000","mapping_epoch":29,"log_start":"260'2300","ondisk_log_start":"260'2300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2406","last_scrub_stamp":"2024-06-28T16:14:46.648700+0000","last_deep_scrub":"260'2406","last_deep_scrub_stamp":"2024-06-28T16:14:46.648700+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.648700+0000","objects_scrubbed":4,"log_size":106,"log_dups_size":2300,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:06:59.187449+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5966,"num_read_kb":5973,"num_write":4800,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"260'4180","reported_seq":4834,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.506798+0000","last_change":"2024-06-28T16:14:58.506798+0000","last_active":"2024-06-28T16:14:58.506798+0000","last_peered":"2024-06-28T16:14:58.506798+0000","last_clean":"2024-06-28T16:14:58.506798+0000","last_became_active":"2024-06-28T15:00:17.390254+0000","last_became_peered":"2024-06-28T15:00:17.390254+0000","last_unstale":"2024-06-28T16:14:58.506798+0000","last_undegraded":"2024-06-28T16:14:58.506798+0000","last_fullsized":"2024-06-28T16:14:58.506798+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4180","last_scrub_stamp":"2024-06-28T16:14:58.506546+0000","last_deep_scrub":"260'4180","last_deep_scrub_stamp":"2024-06-28T16:14:58.506546+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.506546+0000","objects_scrubbed":1,"log_size":80,"log_dups_size":3000,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:17:13.473175+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14764,"num_read_kb":11968,"num_write":6274,"num_write_kb":3129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"260'343","reported_seq":1013,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.608567+0000","last_change":"2024-06-28T15:00:15.397404+0000","last_active":"2024-06-28T16:14:38.608567+0000","last_peered":"2024-06-28T16:14:38.608567+0000","last_clean":"2024-06-28T16:14:38.608567+0000","last_became_active":"2024-06-28T15:00:15.396724+0000","last_became_peered":"2024-06-28T15:00:15.396724+0000","last_unstale":"2024-06-28T16:14:38.608567+0000","last_undegraded":"2024-06-28T16:14:38.608567+0000","last_fullsized":"2024-06-28T16:14:38.608567+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262185,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":532,"num_read_kb":7737,"num_write":1235,"num_write_kb":8076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":509,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.971582+0000","last_change":"2024-06-28T15:00:19.390766+0000","last_active":"2024-06-28T16:14:39.971582+0000","last_peered":"2024-06-28T16:14:39.971582+0000","last_clean":"2024-06-28T16:14:39.971582+0000","last_became_active":"2024-06-28T15:00:19.390100+0000","last_became_peered":"2024-06-28T15:00:19.390100+0000","last_unstale":"2024-06-28T16:14:39.971582+0000","last_undegraded":"2024-06-28T16:14:39.971582+0000","last_fullsized":"2024-06-28T16:14:39.971582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"260'127","reported_seq":676,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.898682+0000","last_change":"2024-06-28T16:14:57.898682+0000","last_active":"2024-06-28T16:14:57.898682+0000","last_peered":"2024-06-28T16:14:57.898682+0000","last_clean":"2024-06-28T16:14:57.898682+0000","last_became_active":"2024-06-28T15:00:21.374284+0000","last_became_peered":"2024-06-28T15:00:21.374284+0000","last_unstale":"2024-06-28T16:14:57.898682+0000","last_undegraded":"2024-06-28T16:14:57.898682+0000","last_fullsized":"2024-06-28T16:14:57.898682+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'127","last_scrub_stamp":"2024-06-28T16:14:57.898396+0000","last_deep_scrub":"260'127","last_deep_scrub_stamp":"2024-06-28T16:14:57.898396+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.898396+0000","objects_scrubbed":4,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:35:13.361283+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":127,"num_read_kb":120,"num_write":90,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.938366+0000","last_change":"2024-06-28T16:14:50.938366+0000","last_active":"2024-06-28T16:14:50.938366+0000","last_peered":"2024-06-28T16:14:50.938366+0000","last_clean":"2024-06-28T16:14:50.938366+0000","last_became_active":"2024-06-28T15:00:23.399147+0000","last_became_peered":"2024-06-28T15:00:23.399147+0000","last_unstale":"2024-06-28T16:14:50.938366+0000","last_undegraded":"2024-06-28T16:14:50.938366+0000","last_fullsized":"2024-06-28T16:14:50.938366+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:50.938091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:50.938091+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.938091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:31:29.176572+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"260'127","reported_seq":662,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.070275+0000","last_change":"2024-06-28T16:14:51.070346+0000","last_active":"2024-06-28T16:14:51.070275+0000","last_peered":"2024-06-28T16:14:51.070275+0000","last_clean":"2024-06-28T16:14:51.070275+0000","last_became_active":"2024-06-28T15:00:25.432898+0000","last_became_peered":"2024-06-28T15:00:25.432898+0000","last_unstale":"2024-06-28T16:14:51.070275+0000","last_undegraded":"2024-06-28T16:14:51.070275+0000","last_fullsized":"2024-06-28T16:14:51.070275+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'127","last_scrub_stamp":"2024-06-28T16:14:51.070081+0000","last_deep_scrub":"260'127","last_deep_scrub_stamp":"2024-06-28T16:14:51.070081+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.070081+0000","objects_scrubbed":4,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:05:59.673050+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":165,"num_write":302,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"260'155","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.296338+0000","last_change":"2024-06-28T16:15:13.296338+0000","last_active":"2024-06-28T16:15:13.296338+0000","last_peered":"2024-06-28T16:15:13.296338+0000","last_clean":"2024-06-28T16:15:13.296338+0000","last_became_active":"2024-06-28T15:00:17.384613+0000","last_became_peered":"2024-06-28T15:00:17.384613+0000","last_unstale":"2024-06-28T16:15:13.296338+0000","last_undegraded":"2024-06-28T16:15:13.296338+0000","last_fullsized":"2024-06-28T16:15:13.296338+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'155","last_scrub_stamp":"2024-06-28T16:15:13.296050+0000","last_deep_scrub":"260'155","last_deep_scrub_stamp":"2024-06-28T16:15:13.296050+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.296050+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:05:33.691767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":630,"num_read_kb":508,"num_write":240,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"260'353","reported_seq":1011,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.887613+0000","last_change":"2024-06-28T16:15:17.887833+0000","last_active":"2024-06-28T16:15:17.887613+0000","last_peered":"2024-06-28T16:15:17.887613+0000","last_clean":"2024-06-28T16:15:17.887613+0000","last_became_active":"2024-06-28T15:00:15.428415+0000","last_became_peered":"2024-06-28T15:00:15.428415+0000","last_unstale":"2024-06-28T16:15:17.887613+0000","last_undegraded":"2024-06-28T16:15:17.887613+0000","last_fullsized":"2024-06-28T16:15:17.887613+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'353","last_scrub_stamp":"2024-06-28T16:15:17.887368+0000","last_deep_scrub":"260'353","last_deep_scrub_stamp":"2024-06-28T16:15:17.887368+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.887368+0000","objects_scrubbed":10,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:26:36.084273+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16055067,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":496,"num_read_kb":15197,"num_write":1261,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":516,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.228639+0000","last_change":"2024-06-28T16:15:19.228690+0000","last_active":"2024-06-28T16:15:19.228639+0000","last_peered":"2024-06-28T16:15:19.228639+0000","last_clean":"2024-06-28T16:15:19.228639+0000","last_became_active":"2024-06-28T15:00:19.372840+0000","last_became_peered":"2024-06-28T15:00:19.372840+0000","last_unstale":"2024-06-28T16:15:19.228639+0000","last_undegraded":"2024-06-28T16:15:19.228639+0000","last_fullsized":"2024-06-28T16:15:19.228639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:19.228437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:19.228437+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.228437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:28:55.169806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"260'128","reported_seq":701,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.103625+0000","last_change":"2024-06-28T16:14:45.103625+0000","last_active":"2024-06-28T16:14:45.103625+0000","last_peered":"2024-06-28T16:14:45.103625+0000","last_clean":"2024-06-28T16:14:45.103625+0000","last_became_active":"2024-06-28T15:00:21.377203+0000","last_became_peered":"2024-06-28T15:00:21.377203+0000","last_unstale":"2024-06-28T16:14:45.103625+0000","last_undegraded":"2024-06-28T16:14:45.103625+0000","last_fullsized":"2024-06-28T16:14:45.103625+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'128","last_scrub_stamp":"2024-06-28T16:14:45.103227+0000","last_deep_scrub":"260'128","last_deep_scrub_stamp":"2024-06-28T16:14:45.103227+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.103227+0000","objects_scrubbed":5,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:36:43.901132+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"260'522","reported_seq":4637,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.759185+0000","last_change":"2024-06-28T16:15:02.759185+0000","last_active":"2024-06-28T16:15:02.759185+0000","last_peered":"2024-06-28T16:15:02.759185+0000","last_clean":"2024-06-28T16:15:02.759185+0000","last_became_active":"2024-06-28T15:00:23.417490+0000","last_became_peered":"2024-06-28T15:00:23.417490+0000","last_unstale":"2024-06-28T16:15:02.759185+0000","last_undegraded":"2024-06-28T16:15:02.759185+0000","last_fullsized":"2024-06-28T16:15:02.759185+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'522","last_scrub_stamp":"2024-06-28T16:15:02.758942+0000","last_deep_scrub":"260'522","last_deep_scrub_stamp":"2024-06-28T16:15:02.758942+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.758942+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":500,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:33:07.422217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"260'204","reported_seq":1022,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.919176+0000","last_change":"2024-06-28T16:14:42.919351+0000","last_active":"2024-06-28T16:14:42.919176+0000","last_peered":"2024-06-28T16:14:42.919176+0000","last_clean":"2024-06-28T16:14:42.919176+0000","last_became_active":"2024-06-28T15:00:25.555670+0000","last_became_peered":"2024-06-28T15:00:25.555670+0000","last_unstale":"2024-06-28T16:14:42.919176+0000","last_undegraded":"2024-06-28T16:14:42.919176+0000","last_fullsized":"2024-06-28T16:14:42.919176+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'204","last_scrub_stamp":"2024-06-28T16:14:42.918953+0000","last_deep_scrub":"260'204","last_deep_scrub_stamp":"2024-06-28T16:14:42.918953+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.918953+0000","objects_scrubbed":3,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:34:15.091552+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":557,"num_read_kb":555,"num_write":448,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"260'84","reported_seq":681,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.159682+0000","last_change":"2024-06-28T16:15:15.159682+0000","last_active":"2024-06-28T16:15:15.159682+0000","last_peered":"2024-06-28T16:15:15.159682+0000","last_clean":"2024-06-28T16:15:15.159682+0000","last_became_active":"2024-06-28T15:00:17.389986+0000","last_became_peered":"2024-06-28T15:00:17.389986+0000","last_unstale":"2024-06-28T16:15:15.159682+0000","last_undegraded":"2024-06-28T16:15:15.159682+0000","last_fullsized":"2024-06-28T16:15:15.159682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'84","last_scrub_stamp":"2024-06-28T16:15:15.159426+0000","last_deep_scrub":"260'84","last_deep_scrub_stamp":"2024-06-28T16:15:15.159426+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.159426+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:34:57.998551+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":369,"num_read_kb":305,"num_write":130,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"260'288","reported_seq":932,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.384197+0000","last_change":"2024-06-28T16:15:09.384387+0000","last_active":"2024-06-28T16:15:09.384197+0000","last_peered":"2024-06-28T16:15:09.384197+0000","last_clean":"2024-06-28T16:15:09.384197+0000","last_became_active":"2024-06-28T15:00:15.411380+0000","last_became_peered":"2024-06-28T15:00:15.411380+0000","last_unstale":"2024-06-28T16:15:09.384197+0000","last_undegraded":"2024-06-28T16:15:09.384197+0000","last_fullsized":"2024-06-28T16:15:09.384197+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'288","last_scrub_stamp":"2024-06-28T16:15:09.383997+0000","last_deep_scrub":"260'288","last_deep_scrub_stamp":"2024-06-28T16:15:09.383997+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.383997+0000","objects_scrubbed":7,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:51:32.793598+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262217,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":24351,"num_write":1142,"num_write_kb":34218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"260'4","reported_seq":841,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.683401+0000","last_change":"2024-06-28T16:15:10.683401+0000","last_active":"2024-06-28T16:15:10.683401+0000","last_peered":"2024-06-28T16:15:10.683401+0000","last_clean":"2024-06-28T16:15:10.683401+0000","last_became_active":"2024-06-28T15:00:19.403623+0000","last_became_peered":"2024-06-28T15:00:19.403623+0000","last_unstale":"2024-06-28T16:15:10.683401+0000","last_undegraded":"2024-06-28T16:15:10.683401+0000","last_fullsized":"2024-06-28T16:15:10.683401+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:15:10.683144+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:15:10.683144+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.683144+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:43:58.420576+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":496,"num_read_kb":494,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"260'255","reported_seq":847,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.728156+0000","last_change":"2024-06-28T16:15:05.728156+0000","last_active":"2024-06-28T16:15:05.728156+0000","last_peered":"2024-06-28T16:15:05.728156+0000","last_clean":"2024-06-28T16:15:05.728156+0000","last_became_active":"2024-06-28T15:00:21.483633+0000","last_became_peered":"2024-06-28T15:00:21.483633+0000","last_unstale":"2024-06-28T16:15:05.728156+0000","last_undegraded":"2024-06-28T16:15:05.728156+0000","last_fullsized":"2024-06-28T16:15:05.728156+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'255","last_scrub_stamp":"2024-06-28T16:15:05.727852+0000","last_deep_scrub":"260'255","last_deep_scrub_stamp":"2024-06-28T16:15:05.727852+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.727852+0000","objects_scrubbed":4,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:54.739144+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8831,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":873,"num_write":310,"num_write_kb":224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"260'537","reported_seq":2249,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.225909+0000","last_change":"2024-06-28T16:15:03.225909+0000","last_active":"2024-06-28T16:15:03.225909+0000","last_peered":"2024-06-28T16:15:03.225909+0000","last_clean":"2024-06-28T16:15:03.225909+0000","last_became_active":"2024-06-28T15:00:23.396923+0000","last_became_peered":"2024-06-28T15:00:23.396923+0000","last_unstale":"2024-06-28T16:15:03.225909+0000","last_undegraded":"2024-06-28T16:15:03.225909+0000","last_fullsized":"2024-06-28T16:15:03.225909+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'537","last_scrub_stamp":"2024-06-28T16:15:03.225642+0000","last_deep_scrub":"260'537","last_deep_scrub_stamp":"2024-06-28T16:15:03.225642+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.225642+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:40:58.110463+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"260'415","reported_seq":1102,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.654488+0000","last_change":"2024-06-28T16:15:05.654568+0000","last_active":"2024-06-28T16:15:05.654488+0000","last_peered":"2024-06-28T16:15:05.654488+0000","last_clean":"2024-06-28T16:15:05.654488+0000","last_became_active":"2024-06-28T15:00:17.376864+0000","last_became_peered":"2024-06-28T15:00:17.376864+0000","last_unstale":"2024-06-28T16:15:05.654488+0000","last_undegraded":"2024-06-28T16:15:05.654488+0000","last_fullsized":"2024-06-28T16:15:05.654488+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'415","last_scrub_stamp":"2024-06-28T16:15:05.654289+0000","last_deep_scrub":"260'415","last_deep_scrub_stamp":"2024-06-28T16:15:05.654289+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.654289+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":400,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:26:21.223890+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1583,"num_read_kb":1214,"num_write":634,"num_write_kb":357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"260'320","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.802495+0000","last_change":"2024-06-28T15:00:15.423161+0000","last_active":"2024-06-28T16:14:41.802495+0000","last_peered":"2024-06-28T16:14:41.802495+0000","last_clean":"2024-06-28T16:14:41.802495+0000","last_became_active":"2024-06-28T15:00:15.415163+0000","last_became_peered":"2024-06-28T15:00:15.415163+0000","last_unstale":"2024-06-28T16:14:41.802495+0000","last_undegraded":"2024-06-28T16:14:41.802495+0000","last_fullsized":"2024-06-28T16:14:41.802495+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470118,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":518,"num_read_kb":26822,"num_write":1200,"num_write_kb":41274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.707792+0000","last_change":"2024-06-28T16:15:08.707848+0000","last_active":"2024-06-28T16:15:08.707792+0000","last_peered":"2024-06-28T16:15:08.707792+0000","last_clean":"2024-06-28T16:15:08.707792+0000","last_became_active":"2024-06-28T15:00:19.389624+0000","last_became_peered":"2024-06-28T15:00:19.389624+0000","last_unstale":"2024-06-28T16:15:08.707792+0000","last_undegraded":"2024-06-28T16:15:08.707792+0000","last_fullsized":"2024-06-28T16:15:08.707792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:08.707598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:08.707598+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.707598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:30.086517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"260'390","reported_seq":1043,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.597603+0000","last_change":"2024-06-28T16:14:54.597760+0000","last_active":"2024-06-28T16:14:54.597603+0000","last_peered":"2024-06-28T16:14:54.597603+0000","last_clean":"2024-06-28T16:14:54.597603+0000","last_became_active":"2024-06-28T15:00:21.481315+0000","last_became_peered":"2024-06-28T15:00:21.481315+0000","last_unstale":"2024-06-28T16:14:54.597603+0000","last_undegraded":"2024-06-28T16:14:54.597603+0000","last_fullsized":"2024-06-28T16:14:54.597603+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'390","last_scrub_stamp":"2024-06-28T16:14:54.597422+0000","last_deep_scrub":"260'390","last_deep_scrub_stamp":"2024-06-28T16:14:54.597422+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.597422+0000","objects_scrubbed":9,"log_size":90,"log_dups_size":300,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:52:31.536722+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46076,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":638,"num_read_kb":1277,"num_write":490,"num_write_kb":298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":511,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.079204+0000","last_change":"2024-06-28T16:15:03.079255+0000","last_active":"2024-06-28T16:15:03.079204+0000","last_peered":"2024-06-28T16:15:03.079204+0000","last_clean":"2024-06-28T16:15:03.079204+0000","last_became_active":"2024-06-28T15:00:23.413132+0000","last_became_peered":"2024-06-28T15:00:23.413132+0000","last_unstale":"2024-06-28T16:15:03.079204+0000","last_undegraded":"2024-06-28T16:15:03.079204+0000","last_fullsized":"2024-06-28T16:15:03.079204+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:03.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:03.079017+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:55:30.146201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"260'558","reported_seq":1385,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.684916+0000","last_change":"2024-06-28T16:14:45.685006+0000","last_active":"2024-06-28T16:14:45.684916+0000","last_peered":"2024-06-28T16:14:45.684916+0000","last_clean":"2024-06-28T16:14:45.684916+0000","last_became_active":"2024-06-28T15:00:25.441335+0000","last_became_peered":"2024-06-28T15:00:25.441335+0000","last_unstale":"2024-06-28T16:14:45.684916+0000","last_undegraded":"2024-06-28T16:14:45.684916+0000","last_fullsized":"2024-06-28T16:14:45.684916+0000","mapping_epoch":29,"log_start":"260'500","ondisk_log_start":"260'500","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'558","last_scrub_stamp":"2024-06-28T16:14:45.684713+0000","last_deep_scrub":"260'558","last_deep_scrub_stamp":"2024-06-28T16:14:45.684713+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.684713+0000","objects_scrubbed":3,"log_size":58,"log_dups_size":500,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:58:41.400605+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1277,"num_read_kb":1273,"num_write":1164,"num_write_kb":394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"260'53","reported_seq":641,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.944152+0000","last_change":"2024-06-28T16:15:02.944224+0000","last_active":"2024-06-28T16:15:02.944152+0000","last_peered":"2024-06-28T16:15:02.944152+0000","last_clean":"2024-06-28T16:15:02.944152+0000","last_became_active":"2024-06-28T15:00:17.408992+0000","last_became_peered":"2024-06-28T15:00:17.408992+0000","last_unstale":"2024-06-28T16:15:02.944152+0000","last_undegraded":"2024-06-28T16:15:02.944152+0000","last_fullsized":"2024-06-28T16:15:02.944152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'53","last_scrub_stamp":"2024-06-28T16:15:02.943961+0000","last_deep_scrub":"260'53","last_deep_scrub_stamp":"2024-06-28T16:15:02.943961+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.943961+0000","objects_scrubbed":1,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:58:16.183710+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":226,"num_read_kb":196,"num_write":79,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"260'326","reported_seq":946,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.675598+0000","last_change":"2024-06-28T15:00:15.402904+0000","last_active":"2024-06-28T16:14:34.675598+0000","last_peered":"2024-06-28T16:14:34.675598+0000","last_clean":"2024-06-28T16:14:34.675598+0000","last_became_active":"2024-06-28T15:00:15.402188+0000","last_became_peered":"2024-06-28T15:00:15.402188+0000","last_unstale":"2024-06-28T16:14:34.675598+0000","last_undegraded":"2024-06-28T16:14:34.675598+0000","last_fullsized":"2024-06-28T16:14:34.675598+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170347,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":7037,"num_write":1119,"num_write_kb":28395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":523,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.240452+0000","last_change":"2024-06-28T16:14:55.240505+0000","last_active":"2024-06-28T16:14:55.240452+0000","last_peered":"2024-06-28T16:14:55.240452+0000","last_clean":"2024-06-28T16:14:55.240452+0000","last_became_active":"2024-06-28T15:00:19.383002+0000","last_became_peered":"2024-06-28T15:00:19.383002+0000","last_unstale":"2024-06-28T16:14:55.240452+0000","last_undegraded":"2024-06-28T16:14:55.240452+0000","last_fullsized":"2024-06-28T16:14:55.240452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:55.240251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:55.240251+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.240251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:09:48.072213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"260'288","reported_seq":880,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.971707+0000","last_change":"2024-06-28T15:00:21.395366+0000","last_active":"2024-06-28T16:14:39.971707+0000","last_peered":"2024-06-28T16:14:39.971707+0000","last_clean":"2024-06-28T16:14:39.971707+0000","last_became_active":"2024-06-28T15:00:21.394580+0000","last_became_peered":"2024-06-28T15:00:21.394580+0000","last_unstale":"2024-06-28T16:14:39.971707+0000","last_undegraded":"2024-06-28T16:14:39.971707+0000","last_fullsized":"2024-06-28T16:14:39.971707+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":455,"num_read_kb":1052,"num_write":364,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":511,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.189470+0000","last_change":"2024-06-28T16:14:59.189522+0000","last_active":"2024-06-28T16:14:59.189470+0000","last_peered":"2024-06-28T16:14:59.189470+0000","last_clean":"2024-06-28T16:14:59.189470+0000","last_became_active":"2024-06-28T15:00:23.422567+0000","last_became_peered":"2024-06-28T15:00:23.422567+0000","last_unstale":"2024-06-28T16:14:59.189470+0000","last_undegraded":"2024-06-28T16:14:59.189470+0000","last_fullsized":"2024-06-28T16:14:59.189470+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:59.189280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:59.189280+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.189280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:25:01.077193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"260'96","reported_seq":598,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.496092+0000","last_change":"2024-06-28T16:15:00.496156+0000","last_active":"2024-06-28T16:15:00.496092+0000","last_peered":"2024-06-28T16:15:00.496092+0000","last_clean":"2024-06-28T16:15:00.496092+0000","last_became_active":"2024-06-28T15:00:25.426151+0000","last_became_peered":"2024-06-28T15:00:25.426151+0000","last_unstale":"2024-06-28T16:15:00.496092+0000","last_undegraded":"2024-06-28T16:15:00.496092+0000","last_fullsized":"2024-06-28T16:15:00.496092+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'96","last_scrub_stamp":"2024-06-28T16:15:00.495898+0000","last_deep_scrub":"260'96","last_deep_scrub_stamp":"2024-06-28T16:15:00.495898+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.495898+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:31:27.205808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":231,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"260'173","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.436343+0000","last_change":"2024-06-28T16:14:57.436511+0000","last_active":"2024-06-28T16:14:57.436343+0000","last_peered":"2024-06-28T16:14:57.436343+0000","last_clean":"2024-06-28T16:14:57.436343+0000","last_became_active":"2024-06-28T15:00:17.407031+0000","last_became_peered":"2024-06-28T15:00:17.407031+0000","last_unstale":"2024-06-28T16:14:57.436343+0000","last_undegraded":"2024-06-28T16:14:57.436343+0000","last_fullsized":"2024-06-28T16:14:57.436343+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'173","last_scrub_stamp":"2024-06-28T16:14:57.436114+0000","last_deep_scrub":"260'173","last_deep_scrub_stamp":"2024-06-28T16:14:57.436114+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.436114+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:11:15.521784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":538,"num_write":274,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"260'353","reported_seq":1047,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.817239+0000","last_change":"2024-06-28T16:15:19.817365+0000","last_active":"2024-06-28T16:15:19.817239+0000","last_peered":"2024-06-28T16:15:19.817239+0000","last_clean":"2024-06-28T16:15:19.817239+0000","last_became_active":"2024-06-28T15:00:15.427408+0000","last_became_peered":"2024-06-28T15:00:15.427408+0000","last_unstale":"2024-06-28T16:15:19.817239+0000","last_undegraded":"2024-06-28T16:15:19.817239+0000","last_fullsized":"2024-06-28T16:15:19.817239+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'353","last_scrub_stamp":"2024-06-28T16:15:19.817035+0000","last_deep_scrub":"260'353","last_deep_scrub_stamp":"2024-06-28T16:15:19.817035+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.817035+0000","objects_scrubbed":6,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:12:02.971410+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":574,"num_read_kb":16939,"num_write":1376,"num_write_kb":32636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":522,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.193279+0000","last_change":"2024-06-28T16:14:53.193339+0000","last_active":"2024-06-28T16:14:53.193279+0000","last_peered":"2024-06-28T16:14:53.193279+0000","last_clean":"2024-06-28T16:14:53.193279+0000","last_became_active":"2024-06-28T15:00:19.384172+0000","last_became_peered":"2024-06-28T15:00:19.384172+0000","last_unstale":"2024-06-28T16:14:53.193279+0000","last_undegraded":"2024-06-28T16:14:53.193279+0000","last_fullsized":"2024-06-28T16:14:53.193279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.193045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.193045+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.193045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:10:37.938602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"260'321","reported_seq":941,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.794562+0000","last_change":"2024-06-28T16:15:14.794653+0000","last_active":"2024-06-28T16:15:14.794562+0000","last_peered":"2024-06-28T16:15:14.794562+0000","last_clean":"2024-06-28T16:15:14.794562+0000","last_became_active":"2024-06-28T15:00:21.393608+0000","last_became_peered":"2024-06-28T15:00:21.393608+0000","last_unstale":"2024-06-28T16:15:14.794562+0000","last_undegraded":"2024-06-28T16:15:14.794562+0000","last_fullsized":"2024-06-28T16:15:14.794562+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'321","last_scrub_stamp":"2024-06-28T16:15:14.794352+0000","last_deep_scrub":"260'321","last_deep_scrub_stamp":"2024-06-28T16:15:14.794352+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.794352+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:23:46.118111+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42645,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":526,"num_read_kb":1122,"num_write":410,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"260'534","reported_seq":2275,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.097241+0000","last_change":"2024-06-28T16:14:57.097387+0000","last_active":"2024-06-28T16:14:57.097241+0000","last_peered":"2024-06-28T16:14:57.097241+0000","last_clean":"2024-06-28T16:14:57.097241+0000","last_became_active":"2024-06-28T15:00:23.413550+0000","last_became_peered":"2024-06-28T15:00:23.413550+0000","last_unstale":"2024-06-28T16:14:57.097241+0000","last_undegraded":"2024-06-28T16:14:57.097241+0000","last_fullsized":"2024-06-28T16:14:57.097241+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'534","last_scrub_stamp":"2024-06-28T16:14:57.096961+0000","last_deep_scrub":"260'534","last_deep_scrub_stamp":"2024-06-28T16:14:57.096961+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.096961+0000","objects_scrubbed":1,"log_size":34,"log_dups_size":500,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:11:19.198302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"260'147","reported_seq":684,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.064684+0000","last_change":"2024-06-28T16:14:52.064763+0000","last_active":"2024-06-28T16:14:52.064684+0000","last_peered":"2024-06-28T16:14:52.064684+0000","last_clean":"2024-06-28T16:14:52.064684+0000","last_became_active":"2024-06-28T15:00:25.431835+0000","last_became_peered":"2024-06-28T15:00:25.431835+0000","last_unstale":"2024-06-28T16:14:52.064684+0000","last_undegraded":"2024-06-28T16:14:52.064684+0000","last_fullsized":"2024-06-28T16:14:52.064684+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'147","last_scrub_stamp":"2024-06-28T16:14:52.064509+0000","last_deep_scrub":"260'147","last_deep_scrub_stamp":"2024-06-28T16:14:52.064509+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.064509+0000","objects_scrubbed":3,"log_size":47,"log_dups_size":100,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:45:10.678734+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1247,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":164,"num_write":265,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"260'4158","reported_seq":4773,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.323709+0000","last_change":"2024-06-28T16:15:10.323773+0000","last_active":"2024-06-28T16:15:10.323709+0000","last_peered":"2024-06-28T16:15:10.323709+0000","last_clean":"2024-06-28T16:15:10.323709+0000","last_became_active":"2024-06-28T15:00:17.399799+0000","last_became_peered":"2024-06-28T15:00:17.399799+0000","last_unstale":"2024-06-28T16:15:10.323709+0000","last_undegraded":"2024-06-28T16:15:10.323709+0000","last_fullsized":"2024-06-28T16:15:10.323709+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4158","last_scrub_stamp":"2024-06-28T16:15:10.323504+0000","last_deep_scrub":"260'4158","last_deep_scrub_stamp":"2024-06-28T16:15:10.323504+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.323504+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":3000,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:49:38.330274+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14678,"num_read_kb":11882,"num_write":6254,"num_write_kb":3121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"260'337","reported_seq":983,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.737490+0000","last_change":"2024-06-28T15:00:15.426615+0000","last_active":"2024-06-28T16:14:34.737490+0000","last_peered":"2024-06-28T16:14:34.737490+0000","last_clean":"2024-06-28T16:14:34.737490+0000","last_became_active":"2024-06-28T15:00:15.425690+0000","last_became_peered":"2024-06-28T15:00:15.425690+0000","last_unstale":"2024-06-28T16:14:34.737490+0000","last_undegraded":"2024-06-28T16:14:34.737490+0000","last_fullsized":"2024-06-28T16:14:34.737490+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079931,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":508,"num_read_kb":13664,"num_write":1274,"num_write_kb":26574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,1],"acting":[5,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":523,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.205322+0000","last_change":"2024-06-28T16:14:51.205380+0000","last_active":"2024-06-28T16:14:51.205322+0000","last_peered":"2024-06-28T16:14:51.205322+0000","last_clean":"2024-06-28T16:14:51.205322+0000","last_became_active":"2024-06-28T15:00:19.385189+0000","last_became_peered":"2024-06-28T15:00:19.385189+0000","last_unstale":"2024-06-28T16:14:51.205322+0000","last_undegraded":"2024-06-28T16:14:51.205322+0000","last_fullsized":"2024-06-28T16:14:51.205322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.205122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.205122+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.205122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:14:01.011784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"260'487","reported_seq":1144,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.253332+0000","last_change":"2024-06-28T15:00:21.400448+0000","last_active":"2024-06-28T16:14:34.253332+0000","last_peered":"2024-06-28T16:14:34.253332+0000","last_clean":"2024-06-28T16:14:34.253332+0000","last_became_active":"2024-06-28T15:00:21.397397+0000","last_became_peered":"2024-06-28T15:00:21.397397+0000","last_unstale":"2024-06-28T16:14:34.253332+0000","last_undegraded":"2024-06-28T16:14:34.253332+0000","last_fullsized":"2024-06-28T16:14:34.253332+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":400,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92449,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":856,"num_read_kb":2091,"num_write":698,"num_write_kb":574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.129159+0000","last_change":"2024-06-28T16:14:40.129181+0000","last_active":"2024-06-28T16:14:40.129159+0000","last_peered":"2024-06-28T16:14:40.129159+0000","last_clean":"2024-06-28T16:14:40.129159+0000","last_became_active":"2024-06-28T15:00:23.416263+0000","last_became_peered":"2024-06-28T15:00:23.416263+0000","last_unstale":"2024-06-28T16:14:40.129159+0000","last_undegraded":"2024-06-28T16:14:40.129159+0000","last_fullsized":"2024-06-28T16:14:40.129159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:40.129070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:40.129070+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.129070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:20:56.813386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"260'117","reported_seq":644,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.256714+0000","last_change":"2024-06-28T16:15:15.256884+0000","last_active":"2024-06-28T16:15:15.256714+0000","last_peered":"2024-06-28T16:15:15.256714+0000","last_clean":"2024-06-28T16:15:15.256714+0000","last_became_active":"2024-06-28T15:00:25.422519+0000","last_became_peered":"2024-06-28T15:00:25.422519+0000","last_unstale":"2024-06-28T16:15:15.256714+0000","last_undegraded":"2024-06-28T16:15:15.256714+0000","last_fullsized":"2024-06-28T16:15:15.256714+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'117","last_scrub_stamp":"2024-06-28T16:15:15.256473+0000","last_deep_scrub":"260'117","last_deep_scrub_stamp":"2024-06-28T16:15:15.256473+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.256473+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:55:59.918202+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":118,"num_read_kb":116,"num_write":248,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"260'128","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.924442+0000","last_change":"2024-06-28T16:15:00.924534+0000","last_active":"2024-06-28T16:15:00.924442+0000","last_peered":"2024-06-28T16:15:00.924442+0000","last_clean":"2024-06-28T16:15:00.924442+0000","last_became_active":"2024-06-28T15:00:17.404204+0000","last_became_peered":"2024-06-28T15:00:17.404204+0000","last_unstale":"2024-06-28T16:15:00.924442+0000","last_undegraded":"2024-06-28T16:15:00.924442+0000","last_fullsized":"2024-06-28T16:15:00.924442+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'128","last_scrub_stamp":"2024-06-28T16:15:00.924255+0000","last_deep_scrub":"260'128","last_deep_scrub_stamp":"2024-06-28T16:15:00.924255+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.924255+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:33:46.108267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":562,"num_read_kb":476,"num_write":174,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"260'352","reported_seq":1060,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.731157+0000","last_change":"2024-06-28T15:00:15.402917+0000","last_active":"2024-06-28T16:14:34.731157+0000","last_peered":"2024-06-28T16:14:34.731157+0000","last_clean":"2024-06-28T16:14:34.731157+0000","last_became_active":"2024-06-28T15:00:15.402041+0000","last_became_peered":"2024-06-28T15:00:15.402041+0000","last_unstale":"2024-06-28T16:14:34.731157+0000","last_undegraded":"2024-06-28T16:14:34.731157+0000","last_fullsized":"2024-06-28T16:14:34.731157+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":603,"num_read_kb":26410,"num_write":1369,"num_write_kb":26391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":522,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.209018+0000","last_change":"2024-06-28T16:14:49.209075+0000","last_active":"2024-06-28T16:14:49.209018+0000","last_peered":"2024-06-28T16:14:49.209018+0000","last_clean":"2024-06-28T16:14:49.209018+0000","last_became_active":"2024-06-28T15:00:19.392773+0000","last_became_peered":"2024-06-28T15:00:19.392773+0000","last_unstale":"2024-06-28T16:14:49.209018+0000","last_undegraded":"2024-06-28T16:14:49.209018+0000","last_fullsized":"2024-06-28T16:14:49.209018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.208819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.208819+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.208819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:49:11.047171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"260'306","reported_seq":911,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.988613+0000","last_change":"2024-06-28T16:14:53.988691+0000","last_active":"2024-06-28T16:14:53.988613+0000","last_peered":"2024-06-28T16:14:53.988613+0000","last_clean":"2024-06-28T16:14:53.988613+0000","last_became_active":"2024-06-28T15:00:21.397842+0000","last_became_peered":"2024-06-28T15:00:21.397842+0000","last_unstale":"2024-06-28T16:14:53.988613+0000","last_undegraded":"2024-06-28T16:14:53.988613+0000","last_fullsized":"2024-06-28T16:14:53.988613+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'306","last_scrub_stamp":"2024-06-28T16:14:53.988406+0000","last_deep_scrub":"260'306","last_deep_scrub_stamp":"2024-06-28T16:14:53.988406+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.988406+0000","objects_scrubbed":6,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:16:38.325337+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44037,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":498,"num_read_kb":1063,"num_write":434,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15964,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":512,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.186008+0000","last_change":"2024-06-28T16:15:00.186120+0000","last_active":"2024-06-28T16:15:00.186008+0000","last_peered":"2024-06-28T16:15:00.186008+0000","last_clean":"2024-06-28T16:15:00.186008+0000","last_became_active":"2024-06-28T15:00:23.406122+0000","last_became_peered":"2024-06-28T15:00:23.406122+0000","last_unstale":"2024-06-28T16:15:00.186008+0000","last_undegraded":"2024-06-28T16:15:00.186008+0000","last_fullsized":"2024-06-28T16:15:00.186008+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:00.185778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:00.185778+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.185778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:16:27.337969+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"260'91","reported_seq":604,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.145305+0000","last_change":"2024-06-28T16:14:54.145374+0000","last_active":"2024-06-28T16:14:54.145305+0000","last_peered":"2024-06-28T16:14:54.145305+0000","last_clean":"2024-06-28T16:14:54.145305+0000","last_became_active":"2024-06-28T15:00:25.439346+0000","last_became_peered":"2024-06-28T15:00:25.439346+0000","last_unstale":"2024-06-28T16:14:54.145305+0000","last_undegraded":"2024-06-28T16:14:54.145305+0000","last_fullsized":"2024-06-28T16:14:54.145305+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'91","last_scrub_stamp":"2024-06-28T16:14:54.145105+0000","last_deep_scrub":"260'91","last_deep_scrub_stamp":"2024-06-28T16:14:54.145105+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.145105+0000","objects_scrubbed":2,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:54:30.494466+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":207,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"260'171","reported_seq":760,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.130652+0000","last_change":"2024-06-28T15:00:17.396835+0000","last_active":"2024-06-28T16:14:35.130652+0000","last_peered":"2024-06-28T16:14:35.130652+0000","last_clean":"2024-06-28T16:14:35.130652+0000","last_became_active":"2024-06-28T15:00:17.396048+0000","last_became_peered":"2024-06-28T15:00:17.396048+0000","last_unstale":"2024-06-28T16:14:35.130652+0000","last_undegraded":"2024-06-28T16:14:35.130652+0000","last_fullsized":"2024-06-28T16:14:35.130652+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":100,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":659,"num_read_kb":531,"num_write":250,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"260'376","reported_seq":1116,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.457498+0000","last_change":"2024-06-28T16:15:05.457664+0000","last_active":"2024-06-28T16:15:05.457498+0000","last_peered":"2024-06-28T16:15:05.457498+0000","last_clean":"2024-06-28T16:15:05.457498+0000","last_became_active":"2024-06-28T15:00:15.397256+0000","last_became_peered":"2024-06-28T15:00:15.397256+0000","last_unstale":"2024-06-28T16:15:05.457498+0000","last_undegraded":"2024-06-28T16:15:05.457498+0000","last_fullsized":"2024-06-28T16:15:05.457498+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'376","last_scrub_stamp":"2024-06-28T16:15:05.457272+0000","last_deep_scrub":"260'376","last_deep_scrub_stamp":"2024-06-28T16:15:05.457272+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.457272+0000","objects_scrubbed":7,"log_size":76,"log_dups_size":300,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:17:53.364489+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2674362,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":727,"num_read_kb":7345,"num_write":1214,"num_write_kb":31713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":519,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.945272+0000","last_change":"2024-06-28T16:14:53.945393+0000","last_active":"2024-06-28T16:14:53.945272+0000","last_peered":"2024-06-28T16:14:53.945272+0000","last_clean":"2024-06-28T16:14:53.945272+0000","last_became_active":"2024-06-28T15:00:19.379281+0000","last_became_peered":"2024-06-28T15:00:19.379281+0000","last_unstale":"2024-06-28T16:14:53.945272+0000","last_undegraded":"2024-06-28T16:14:53.945272+0000","last_fullsized":"2024-06-28T16:14:53.945272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.945027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.945027+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.945027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:26:10.510879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"260'356","reported_seq":984,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.142072+0000","last_change":"2024-06-28T16:14:43.142072+0000","last_active":"2024-06-28T16:14:43.142072+0000","last_peered":"2024-06-28T16:14:43.142072+0000","last_clean":"2024-06-28T16:14:43.142072+0000","last_became_active":"2024-06-28T15:00:21.395892+0000","last_became_peered":"2024-06-28T15:00:21.395892+0000","last_unstale":"2024-06-28T16:14:43.142072+0000","last_undegraded":"2024-06-28T16:14:43.142072+0000","last_fullsized":"2024-06-28T16:14:43.142072+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'356","last_scrub_stamp":"2024-06-28T16:14:43.141802+0000","last_deep_scrub":"260'356","last_deep_scrub_stamp":"2024-06-28T16:14:43.141802+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.141802+0000","objects_scrubbed":7,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:06:31.916429+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48931,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":579,"num_read_kb":1282,"num_write":462,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"260'501","reported_seq":2313,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.168638+0000","last_change":"2024-06-28T16:14:41.168660+0000","last_active":"2024-06-28T16:14:41.168638+0000","last_peered":"2024-06-28T16:14:41.168638+0000","last_clean":"2024-06-28T16:14:41.168638+0000","last_became_active":"2024-06-28T15:00:23.415437+0000","last_became_peered":"2024-06-28T15:00:23.415437+0000","last_unstale":"2024-06-28T16:14:41.168638+0000","last_undegraded":"2024-06-28T16:14:41.168638+0000","last_fullsized":"2024-06-28T16:14:41.168638+0000","mapping_epoch":27,"log_start":"257'400","ondisk_log_start":"257'400","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'501","last_scrub_stamp":"2024-06-28T16:14:41.168573+0000","last_deep_scrub":"260'501","last_deep_scrub_stamp":"2024-06-28T16:14:41.168573+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.168573+0000","objects_scrubbed":1,"log_size":101,"log_dups_size":400,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:22:09.865296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"260'102","reported_seq":620,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.002911+0000","last_change":"2024-06-28T16:14:50.002982+0000","last_active":"2024-06-28T16:14:50.002911+0000","last_peered":"2024-06-28T16:14:50.002911+0000","last_clean":"2024-06-28T16:14:50.002911+0000","last_became_active":"2024-06-28T15:00:25.425135+0000","last_became_peered":"2024-06-28T15:00:25.425135+0000","last_unstale":"2024-06-28T16:14:50.002911+0000","last_undegraded":"2024-06-28T16:14:50.002911+0000","last_fullsized":"2024-06-28T16:14:50.002911+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'102","last_scrub_stamp":"2024-06-28T16:14:50.002732+0000","last_deep_scrub":"260'102","last_deep_scrub_stamp":"2024-06-28T16:14:50.002732+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.002732+0000","objects_scrubbed":3,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:56:55.357278+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":245,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"260'160","reported_seq":752,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.949730+0000","last_change":"2024-06-28T16:15:03.949795+0000","last_active":"2024-06-28T16:15:03.949730+0000","last_peered":"2024-06-28T16:15:03.949730+0000","last_clean":"2024-06-28T16:15:03.949730+0000","last_became_active":"2024-06-28T15:00:17.402024+0000","last_became_peered":"2024-06-28T15:00:17.402024+0000","last_unstale":"2024-06-28T16:15:03.949730+0000","last_undegraded":"2024-06-28T16:15:03.949730+0000","last_fullsized":"2024-06-28T16:15:03.949730+0000","mapping_epoch":21,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'160","last_scrub_stamp":"2024-06-28T16:15:03.949540+0000","last_deep_scrub":"260'160","last_deep_scrub_stamp":"2024-06-28T16:15:03.949540+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.949540+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":100,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:07:39.846023+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":645,"num_read_kb":505,"num_write":259,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"260'393","reported_seq":1076,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.969253+0000","last_change":"2024-06-28T16:15:14.969434+0000","last_active":"2024-06-28T16:15:14.969253+0000","last_peered":"2024-06-28T16:15:14.969253+0000","last_clean":"2024-06-28T16:15:14.969253+0000","last_became_active":"2024-06-28T15:00:15.388365+0000","last_became_peered":"2024-06-28T15:00:15.388365+0000","last_unstale":"2024-06-28T16:15:14.969253+0000","last_undegraded":"2024-06-28T16:15:14.969253+0000","last_fullsized":"2024-06-28T16:15:14.969253+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'393","last_scrub_stamp":"2024-06-28T16:15:14.969046+0000","last_deep_scrub":"260'393","last_deep_scrub_stamp":"2024-06-28T16:15:14.969046+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.969046+0000","objects_scrubbed":8,"log_size":93,"log_dups_size":300,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:43:46.202161+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1604260,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":661,"num_read_kb":7226,"num_write":1423,"num_write_kb":30573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"260'4","reported_seq":1166,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.680759+0000","last_change":"2024-06-28T16:15:08.680851+0000","last_active":"2024-06-28T16:15:08.680759+0000","last_peered":"2024-06-28T16:15:08.680759+0000","last_clean":"2024-06-28T16:15:08.680759+0000","last_became_active":"2024-06-28T15:00:19.397863+0000","last_became_peered":"2024-06-28T15:00:19.397863+0000","last_unstale":"2024-06-28T16:15:08.680759+0000","last_undegraded":"2024-06-28T16:15:08.680759+0000","last_fullsized":"2024-06-28T16:15:08.680759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:15:08.680583+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:15:08.680583+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.680583+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:28:22.747616+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":648,"num_read_kb":648,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"260'118","reported_seq":672,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.248332+0000","last_change":"2024-06-28T16:15:04.248438+0000","last_active":"2024-06-28T16:15:04.248332+0000","last_peered":"2024-06-28T16:15:04.248332+0000","last_clean":"2024-06-28T16:15:04.248332+0000","last_became_active":"2024-06-28T15:00:21.483679+0000","last_became_peered":"2024-06-28T15:00:21.483679+0000","last_unstale":"2024-06-28T16:15:04.248332+0000","last_undegraded":"2024-06-28T16:15:04.248332+0000","last_fullsized":"2024-06-28T16:15:04.248332+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'118","last_scrub_stamp":"2024-06-28T16:15:04.248130+0000","last_deep_scrub":"260'118","last_deep_scrub_stamp":"2024-06-28T16:15:04.248130+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.248130+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:12:41.540178+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"260'515","reported_seq":4272,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.451884+0000","last_change":"2024-06-28T16:14:58.451949+0000","last_active":"2024-06-28T16:14:58.451884+0000","last_peered":"2024-06-28T16:14:58.451884+0000","last_clean":"2024-06-28T16:14:58.451884+0000","last_became_active":"2024-06-28T15:00:23.419182+0000","last_became_peered":"2024-06-28T15:00:23.419182+0000","last_unstale":"2024-06-28T16:14:58.451884+0000","last_undegraded":"2024-06-28T16:14:58.451884+0000","last_fullsized":"2024-06-28T16:14:58.451884+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'515","last_scrub_stamp":"2024-06-28T16:14:58.451684+0000","last_deep_scrub":"260'515","last_deep_scrub_stamp":"2024-06-28T16:14:58.451684+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.451684+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:15.191208+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"260'163","reported_seq":685,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.941396+0000","last_change":"2024-06-28T16:14:46.941473+0000","last_active":"2024-06-28T16:14:46.941396+0000","last_peered":"2024-06-28T16:14:46.941396+0000","last_clean":"2024-06-28T16:14:46.941396+0000","last_became_active":"2024-06-28T15:00:25.425690+0000","last_became_peered":"2024-06-28T15:00:25.425690+0000","last_unstale":"2024-06-28T16:14:46.941396+0000","last_undegraded":"2024-06-28T16:14:46.941396+0000","last_fullsized":"2024-06-28T16:14:46.941396+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'163","last_scrub_stamp":"2024-06-28T16:14:46.941194+0000","last_deep_scrub":"260'163","last_deep_scrub_stamp":"2024-06-28T16:14:46.941194+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.941194+0000","objects_scrubbed":4,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:28:38.329545+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":152,"num_write":381,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"260'74","reported_seq":628,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.650196+0000","last_change":"2024-06-28T16:14:42.650256+0000","last_active":"2024-06-28T16:14:42.650196+0000","last_peered":"2024-06-28T16:14:42.650196+0000","last_clean":"2024-06-28T16:14:42.650196+0000","last_became_active":"2024-06-28T15:00:59.799823+0000","last_became_peered":"2024-06-28T15:00:59.799823+0000","last_unstale":"2024-06-28T16:14:42.650196+0000","last_undegraded":"2024-06-28T16:14:42.650196+0000","last_fullsized":"2024-06-28T16:14:42.650196+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'74","last_scrub_stamp":"2024-06-28T16:14:42.650003+0000","last_deep_scrub":"260'74","last_deep_scrub_stamp":"2024-06-28T16:14:42.650003+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.650003+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:28:32.949699+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":247,"num_write":103,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"260'351","reported_seq":1082,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.582297+0000","last_change":"2024-06-28T15:00:15.403798+0000","last_active":"2024-06-28T16:14:34.582297+0000","last_peered":"2024-06-28T16:14:34.582297+0000","last_clean":"2024-06-28T16:14:34.582297+0000","last_became_active":"2024-06-28T15:00:15.402782+0000","last_became_peered":"2024-06-28T15:00:15.402782+0000","last_unstale":"2024-06-28T16:14:34.582297+0000","last_undegraded":"2024-06-28T16:14:34.582297+0000","last_fullsized":"2024-06-28T16:14:34.582297+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":300,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262233,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":541,"num_read_kb":3603,"num_write":1164,"num_write_kb":36556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.130681+0000","last_change":"2024-06-28T15:00:19.405878+0000","last_active":"2024-06-28T16:14:35.130681+0000","last_peered":"2024-06-28T16:14:35.130681+0000","last_clean":"2024-06-28T16:14:35.130681+0000","last_became_active":"2024-06-28T15:00:19.405083+0000","last_became_peered":"2024-06-28T15:00:19.405083+0000","last_unstale":"2024-06-28T16:14:35.130681+0000","last_undegraded":"2024-06-28T16:14:35.130681+0000","last_fullsized":"2024-06-28T16:14:35.130681+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"260'437","reported_seq":1081,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.565170+0000","last_change":"2024-06-28T16:14:57.565257+0000","last_active":"2024-06-28T16:14:57.565170+0000","last_peered":"2024-06-28T16:14:57.565170+0000","last_clean":"2024-06-28T16:14:57.565170+0000","last_became_active":"2024-06-28T15:00:21.392393+0000","last_became_peered":"2024-06-28T15:00:21.392393+0000","last_unstale":"2024-06-28T16:14:57.565170+0000","last_undegraded":"2024-06-28T16:14:57.565170+0000","last_fullsized":"2024-06-28T16:14:57.565170+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'437","last_scrub_stamp":"2024-06-28T16:14:57.564983+0000","last_deep_scrub":"260'437","last_deep_scrub_stamp":"2024-06-28T16:14:57.564983+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.564983+0000","objects_scrubbed":9,"log_size":37,"log_dups_size":400,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:47:46.699794+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47467,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1282,"num_read_kb":1893,"num_write":1822,"num_write_kb":1648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":479870,"num_omap_keys":1112,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.323011+0000","last_change":"2024-06-28T16:14:54.323067+0000","last_active":"2024-06-28T16:14:54.323011+0000","last_peered":"2024-06-28T16:14:54.323011+0000","last_clean":"2024-06-28T16:14:54.323011+0000","last_became_active":"2024-06-28T15:00:23.415497+0000","last_became_peered":"2024-06-28T15:00:23.415497+0000","last_unstale":"2024-06-28T16:14:54.323011+0000","last_undegraded":"2024-06-28T16:14:54.323011+0000","last_fullsized":"2024-06-28T16:14:54.323011+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:54.322811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:54.322811+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.322811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:40:56.629420+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"260'136","reported_seq":651,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.029537+0000","last_change":"2024-06-28T16:14:48.029609+0000","last_active":"2024-06-28T16:14:48.029537+0000","last_peered":"2024-06-28T16:14:48.029537+0000","last_clean":"2024-06-28T16:14:48.029537+0000","last_became_active":"2024-06-28T15:00:25.439527+0000","last_became_peered":"2024-06-28T15:00:25.439527+0000","last_unstale":"2024-06-28T16:14:48.029537+0000","last_undegraded":"2024-06-28T16:14:48.029537+0000","last_fullsized":"2024-06-28T16:14:48.029537+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'136","last_scrub_stamp":"2024-06-28T16:14:48.029357+0000","last_deep_scrub":"260'136","last_deep_scrub_stamp":"2024-06-28T16:14:48.029357+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.029357+0000","objects_scrubbed":3,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:09:52.677657+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":127,"num_write":301,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"260'157","reported_seq":783,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.366718+0000","last_change":"2024-06-28T16:14:55.366787+0000","last_active":"2024-06-28T16:14:55.366718+0000","last_peered":"2024-06-28T16:14:55.366718+0000","last_clean":"2024-06-28T16:14:55.366718+0000","last_became_active":"2024-06-28T15:00:17.414537+0000","last_became_peered":"2024-06-28T15:00:17.414537+0000","last_unstale":"2024-06-28T16:14:55.366718+0000","last_undegraded":"2024-06-28T16:14:55.366718+0000","last_fullsized":"2024-06-28T16:14:55.366718+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'157","last_scrub_stamp":"2024-06-28T16:14:55.366527+0000","last_deep_scrub":"260'157","last_deep_scrub_stamp":"2024-06-28T16:14:55.366527+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.366527+0000","objects_scrubbed":1,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:51:04.103193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":522,"num_write":223,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":617,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:25.137102+0000","last_change":"2024-06-28T15:00:06.303487+0000","last_active":"2024-06-28T16:14:25.137102+0000","last_peered":"2024-06-28T16:14:25.137102+0000","last_clean":"2024-06-28T16:14:25.137102+0000","last_became_active":"2024-06-28T15:00:06.302725+0000","last_became_peered":"2024-06-28T15:00:06.302725+0000","last_unstale":"2024-06-28T16:14:25.137102+0000","last_undegraded":"2024-06-28T16:14:25.137102+0000","last_fullsized":"2024-06-28T16:14:25.137102+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":534,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.800619+0000","last_change":"2024-06-28T16:15:15.800750+0000","last_active":"2024-06-28T16:15:15.800619+0000","last_peered":"2024-06-28T16:15:15.800619+0000","last_clean":"2024-06-28T16:15:15.800619+0000","last_became_active":"2024-06-28T15:00:06.284218+0000","last_became_peered":"2024-06-28T15:00:06.284218+0000","last_unstale":"2024-06-28T16:15:15.800619+0000","last_undegraded":"2024-06-28T16:15:15.800619+0000","last_fullsized":"2024-06-28T16:15:15.800619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:15.800331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:15.800331+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.800331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:32:57.118235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"260'320","reported_seq":1125,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.110204+0000","last_change":"2024-06-28T16:14:48.110204+0000","last_active":"2024-06-28T16:14:48.110204+0000","last_peered":"2024-06-28T16:14:48.110204+0000","last_clean":"2024-06-28T16:14:48.110204+0000","last_became_active":"2024-06-28T15:00:59.939352+0000","last_became_peered":"2024-06-28T15:00:59.939352+0000","last_unstale":"2024-06-28T16:14:48.110204+0000","last_undegraded":"2024-06-28T16:14:48.110204+0000","last_fullsized":"2024-06-28T16:14:48.110204+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'320","last_scrub_stamp":"2024-06-28T16:14:48.109838+0000","last_deep_scrub":"260'320","last_deep_scrub_stamp":"2024-06-28T16:14:48.109838+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.109838+0000","objects_scrubbed":10,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:13:21.727097+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":804,"num_read_kb":59106,"num_write":1183,"num_write_kb":43047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6,4],"acting":[3,2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":516,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.722700+0000","last_change":"2024-06-28T16:15:10.722700+0000","last_active":"2024-06-28T16:15:10.722700+0000","last_peered":"2024-06-28T16:15:10.722700+0000","last_clean":"2024-06-28T16:15:10.722700+0000","last_became_active":"2024-06-28T15:00:19.390255+0000","last_became_peered":"2024-06-28T15:00:19.390255+0000","last_unstale":"2024-06-28T16:15:10.722700+0000","last_undegraded":"2024-06-28T16:15:10.722700+0000","last_fullsized":"2024-06-28T16:15:10.722700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:10.722439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:10.722439+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.722439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:01:41.201808+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"260'734","reported_seq":1463,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.597944+0000","last_change":"2024-06-28T16:14:55.598090+0000","last_active":"2024-06-28T16:14:55.597944+0000","last_peered":"2024-06-28T16:14:55.597944+0000","last_clean":"2024-06-28T16:14:55.597944+0000","last_became_active":"2024-06-28T15:00:21.397640+0000","last_became_peered":"2024-06-28T15:00:21.397640+0000","last_unstale":"2024-06-28T16:14:55.597944+0000","last_undegraded":"2024-06-28T16:14:55.597944+0000","last_fullsized":"2024-06-28T16:14:55.597944+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'734","last_scrub_stamp":"2024-06-28T16:14:55.597730+0000","last_deep_scrub":"260'734","last_deep_scrub_stamp":"2024-06-28T16:14:55.597730+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.597730+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":700,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:40:40.351703+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134565,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1425,"num_read_kb":3272,"num_write":1360,"num_write_kb":1190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99450,"num_omap_keys":240,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.214467+0000","last_change":"2024-06-28T16:15:03.214529+0000","last_active":"2024-06-28T16:15:03.214467+0000","last_peered":"2024-06-28T16:15:03.214467+0000","last_clean":"2024-06-28T16:15:03.214467+0000","last_became_active":"2024-06-28T15:00:23.399024+0000","last_became_peered":"2024-06-28T15:00:23.399024+0000","last_unstale":"2024-06-28T16:15:03.214467+0000","last_undegraded":"2024-06-28T16:15:03.214467+0000","last_fullsized":"2024-06-28T16:15:03.214467+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:03.214271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:03.214271+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.214271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:17:32.250338+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"260'190","reported_seq":889,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.404009+0000","last_change":"2024-06-28T16:15:12.404087+0000","last_active":"2024-06-28T16:15:12.404009+0000","last_peered":"2024-06-28T16:15:12.404009+0000","last_clean":"2024-06-28T16:15:12.404009+0000","last_became_active":"2024-06-28T15:00:25.442032+0000","last_became_peered":"2024-06-28T15:00:25.442032+0000","last_unstale":"2024-06-28T16:15:12.404009+0000","last_undegraded":"2024-06-28T16:15:12.404009+0000","last_fullsized":"2024-06-28T16:15:12.404009+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'190","last_scrub_stamp":"2024-06-28T16:15:12.403829+0000","last_deep_scrub":"260'190","last_deep_scrub_stamp":"2024-06-28T16:15:12.403829+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.403829+0000","objects_scrubbed":4,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:09:00.851833+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":392,"num_read_kb":388,"num_write":388,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":66,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"260'118","reported_seq":764,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.318413+0000","last_change":"2024-06-28T16:15:12.318476+0000","last_active":"2024-06-28T16:15:12.318413+0000","last_peered":"2024-06-28T16:15:12.318413+0000","last_clean":"2024-06-28T16:15:12.318413+0000","last_became_active":"2024-06-28T15:00:17.404200+0000","last_became_peered":"2024-06-28T15:00:17.404200+0000","last_unstale":"2024-06-28T16:15:12.318413+0000","last_undegraded":"2024-06-28T16:15:12.318413+0000","last_fullsized":"2024-06-28T16:15:12.318413+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'118","last_scrub_stamp":"2024-06-28T16:15:12.318222+0000","last_deep_scrub":"260'118","last_deep_scrub_stamp":"2024-06-28T16:15:12.318222+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.318222+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:09:16.409918+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":495,"num_read_kb":413,"num_write":159,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":527,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:25.137140+0000","last_change":"2024-06-28T15:00:06.286231+0000","last_active":"2024-06-28T16:14:25.137140+0000","last_peered":"2024-06-28T16:14:25.137140+0000","last_clean":"2024-06-28T16:14:25.137140+0000","last_became_active":"2024-06-28T15:00:06.285489+0000","last_became_peered":"2024-06-28T15:00:06.285489+0000","last_unstale":"2024-06-28T16:14:25.137140+0000","last_undegraded":"2024-06-28T16:14:25.137140+0000","last_fullsized":"2024-06-28T16:14:25.137140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"260'324","reported_seq":994,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.608481+0000","last_change":"2024-06-28T15:00:15.404155+0000","last_active":"2024-06-28T16:14:38.608481+0000","last_peered":"2024-06-28T16:14:38.608481+0000","last_clean":"2024-06-28T16:14:38.608481+0000","last_became_active":"2024-06-28T15:00:15.403657+0000","last_became_peered":"2024-06-28T15:00:15.403657+0000","last_unstale":"2024-06-28T16:14:38.608481+0000","last_undegraded":"2024-06-28T16:14:38.608481+0000","last_fullsized":"2024-06-28T16:14:38.608481+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":524,"num_read_kb":36646,"num_write":1262,"num_write_kb":16548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.674081+0000","last_change":"2024-06-28T16:15:12.674142+0000","last_active":"2024-06-28T16:15:12.674081+0000","last_peered":"2024-06-28T16:15:12.674081+0000","last_clean":"2024-06-28T16:15:12.674081+0000","last_became_active":"2024-06-28T15:00:19.406447+0000","last_became_peered":"2024-06-28T15:00:19.406447+0000","last_unstale":"2024-06-28T16:15:12.674081+0000","last_undegraded":"2024-06-28T16:15:12.674081+0000","last_fullsized":"2024-06-28T16:15:12.674081+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:12.673872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:12.673872+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.673872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:57:02.704896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"260'1141","reported_seq":2924,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.980098+0000","last_change":"2024-06-28T16:14:56.980185+0000","last_active":"2024-06-28T16:14:56.980098+0000","last_peered":"2024-06-28T16:14:56.980098+0000","last_clean":"2024-06-28T16:14:56.980098+0000","last_became_active":"2024-06-28T15:00:21.398382+0000","last_became_peered":"2024-06-28T15:00:21.398382+0000","last_unstale":"2024-06-28T16:14:56.980098+0000","last_undegraded":"2024-06-28T16:14:56.980098+0000","last_fullsized":"2024-06-28T16:14:56.980098+0000","mapping_epoch":25,"log_start":"260'1100","ondisk_log_start":"260'1100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'1141","last_scrub_stamp":"2024-06-28T16:14:56.979906+0000","last_deep_scrub":"260'1141","last_deep_scrub_stamp":"2024-06-28T16:14:56.979906+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.979906+0000","objects_scrubbed":11,"log_size":41,"log_dups_size":1100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:56:04.659644+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133468,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1851,"num_read_kb":3741,"num_write":1150,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6022,"num_omap_keys":18,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.712239+0000","last_change":"2024-06-28T16:14:59.712290+0000","last_active":"2024-06-28T16:14:59.712239+0000","last_peered":"2024-06-28T16:14:59.712239+0000","last_clean":"2024-06-28T16:14:59.712239+0000","last_became_active":"2024-06-28T15:00:23.422876+0000","last_became_peered":"2024-06-28T15:00:23.422876+0000","last_unstale":"2024-06-28T16:14:59.712239+0000","last_undegraded":"2024-06-28T16:14:59.712239+0000","last_fullsized":"2024-06-28T16:14:59.712239+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:59.712038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:59.712038+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.712038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:40.142260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"260'120","reported_seq":642,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.019612+0000","last_change":"2024-06-28T16:14:47.019699+0000","last_active":"2024-06-28T16:14:47.019612+0000","last_peered":"2024-06-28T16:14:47.019612+0000","last_clean":"2024-06-28T16:14:47.019612+0000","last_became_active":"2024-06-28T15:00:25.553691+0000","last_became_peered":"2024-06-28T15:00:25.553691+0000","last_unstale":"2024-06-28T16:14:47.019612+0000","last_undegraded":"2024-06-28T16:14:47.019612+0000","last_fullsized":"2024-06-28T16:14:47.019612+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'120","last_scrub_stamp":"2024-06-28T16:14:47.019412+0000","last_deep_scrub":"260'120","last_deep_scrub_stamp":"2024-06-28T16:14:47.019412+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.019412+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:33:15.581216+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":862,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":120,"num_write":292,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"260'221","reported_seq":843,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.262910+0000","last_change":"2024-06-28T16:15:14.262978+0000","last_active":"2024-06-28T16:15:14.262910+0000","last_peered":"2024-06-28T16:15:14.262910+0000","last_clean":"2024-06-28T16:15:14.262910+0000","last_became_active":"2024-06-28T15:00:17.393361+0000","last_became_peered":"2024-06-28T15:00:17.393361+0000","last_unstale":"2024-06-28T16:15:14.262910+0000","last_undegraded":"2024-06-28T16:15:14.262910+0000","last_fullsized":"2024-06-28T16:15:14.262910+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'221","last_scrub_stamp":"2024-06-28T16:15:14.262723+0000","last_deep_scrub":"260'221","last_deep_scrub_stamp":"2024-06-28T16:15:14.262723+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.262723+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:12:41.234327+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":892,"num_read_kb":761,"num_write":358,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":542,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.217011+0000","last_change":"2024-06-28T16:14:56.217011+0000","last_active":"2024-06-28T16:14:56.217011+0000","last_peered":"2024-06-28T16:14:56.217011+0000","last_clean":"2024-06-28T16:14:56.217011+0000","last_became_active":"2024-06-28T15:00:06.279950+0000","last_became_peered":"2024-06-28T15:00:06.279950+0000","last_unstale":"2024-06-28T16:14:56.217011+0000","last_undegraded":"2024-06-28T16:14:56.217011+0000","last_fullsized":"2024-06-28T16:14:56.217011+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.216749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.216749+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.216749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:22:36.357975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"260'334","reported_seq":1000,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.220573+0000","last_change":"2024-06-28T16:15:12.220658+0000","last_active":"2024-06-28T16:15:12.220573+0000","last_peered":"2024-06-28T16:15:12.220573+0000","last_clean":"2024-06-28T16:15:12.220573+0000","last_became_active":"2024-06-28T15:00:15.413348+0000","last_became_peered":"2024-06-28T15:00:15.413348+0000","last_unstale":"2024-06-28T16:15:12.220573+0000","last_undegraded":"2024-06-28T16:15:12.220573+0000","last_fullsized":"2024-06-28T16:15:12.220573+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'334","last_scrub_stamp":"2024-06-28T16:15:12.220392+0000","last_deep_scrub":"260'334","last_deep_scrub_stamp":"2024-06-28T16:15:12.220392+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.220392+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:47:12.335485+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":504,"num_read_kb":3638,"num_write":1247,"num_write_kb":36014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"260'2","reported_seq":525,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.146412+0000","last_change":"2024-06-28T16:14:47.146477+0000","last_active":"2024-06-28T16:14:47.146412+0000","last_peered":"2024-06-28T16:14:47.146412+0000","last_clean":"2024-06-28T16:14:47.146412+0000","last_became_active":"2024-06-28T15:00:19.404653+0000","last_became_peered":"2024-06-28T15:00:19.404653+0000","last_unstale":"2024-06-28T16:14:47.146412+0000","last_undegraded":"2024-06-28T16:14:47.146412+0000","last_fullsized":"2024-06-28T16:14:47.146412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:47.146212+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:47.146212+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.146212+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:46:59.182619+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"260'246","reported_seq":1692,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:13.424979+0000","last_change":"2024-06-28T15:00:21.484753+0000","last_active":"2024-06-28T16:14:13.424979+0000","last_peered":"2024-06-28T16:14:13.424979+0000","last_clean":"2024-06-28T16:14:13.424979+0000","last_became_active":"2024-06-28T15:00:21.483749+0000","last_became_peered":"2024-06-28T15:00:21.483749+0000","last_unstale":"2024-06-28T16:14:13.424979+0000","last_undegraded":"2024-06-28T16:14:13.424979+0000","last_fullsized":"2024-06-28T16:14:13.424979+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":200,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.250781+0000","last_change":"2024-06-28T16:15:04.250838+0000","last_active":"2024-06-28T16:15:04.250781+0000","last_peered":"2024-06-28T16:15:04.250781+0000","last_clean":"2024-06-28T16:15:04.250781+0000","last_became_active":"2024-06-28T15:00:23.399391+0000","last_became_peered":"2024-06-28T15:00:23.399391+0000","last_unstale":"2024-06-28T16:15:04.250781+0000","last_undegraded":"2024-06-28T16:15:04.250781+0000","last_fullsized":"2024-06-28T16:15:04.250781+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:04.250594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:04.250594+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.250594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:04:20.072805+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"260'242","reported_seq":857,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.112554+0000","last_change":"2024-06-28T16:14:55.112726+0000","last_active":"2024-06-28T16:14:55.112554+0000","last_peered":"2024-06-28T16:14:55.112554+0000","last_clean":"2024-06-28T16:14:55.112554+0000","last_became_active":"2024-06-28T15:00:25.419711+0000","last_became_peered":"2024-06-28T15:00:25.419711+0000","last_unstale":"2024-06-28T16:14:55.112554+0000","last_undegraded":"2024-06-28T16:14:55.112554+0000","last_fullsized":"2024-06-28T16:14:55.112554+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'242","last_scrub_stamp":"2024-06-28T16:14:55.112334+0000","last_deep_scrub":"260'242","last_deep_scrub_stamp":"2024-06-28T16:14:55.112334+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.112334+0000","objects_scrubbed":3,"log_size":42,"log_dups_size":200,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:15:19.328332+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":467,"num_read_kb":491,"num_write":517,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"260'316","reported_seq":1017,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.125767+0000","last_change":"2024-06-28T16:15:04.125767+0000","last_active":"2024-06-28T16:15:04.125767+0000","last_peered":"2024-06-28T16:15:04.125767+0000","last_clean":"2024-06-28T16:15:04.125767+0000","last_became_active":"2024-06-28T15:00:17.393696+0000","last_became_peered":"2024-06-28T15:00:17.393696+0000","last_unstale":"2024-06-28T16:15:04.125767+0000","last_undegraded":"2024-06-28T16:15:04.125767+0000","last_fullsized":"2024-06-28T16:15:04.125767+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'316","last_scrub_stamp":"2024-06-28T16:15:04.125411+0000","last_deep_scrub":"260'316","last_deep_scrub_stamp":"2024-06-28T16:15:04.125411+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.125411+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":300,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:39:24.006191+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1365,"num_read_kb":1154,"num_write":468,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25681,"num_omap_keys":101,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":534,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.751576+0000","last_change":"2024-06-28T16:15:14.751632+0000","last_active":"2024-06-28T16:15:14.751576+0000","last_peered":"2024-06-28T16:15:14.751576+0000","last_clean":"2024-06-28T16:15:14.751576+0000","last_became_active":"2024-06-28T15:00:06.283106+0000","last_became_peered":"2024-06-28T15:00:06.283106+0000","last_unstale":"2024-06-28T16:15:14.751576+0000","last_undegraded":"2024-06-28T16:15:14.751576+0000","last_fullsized":"2024-06-28T16:15:14.751576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:14.751373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:14.751373+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.751373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:17:53.348364+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"260'318","reported_seq":1001,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.927891+0000","last_change":"2024-06-28T16:15:17.928148+0000","last_active":"2024-06-28T16:15:17.927891+0000","last_peered":"2024-06-28T16:15:17.927891+0000","last_clean":"2024-06-28T16:15:17.927891+0000","last_became_active":"2024-06-28T15:00:15.413880+0000","last_became_peered":"2024-06-28T15:00:15.413880+0000","last_unstale":"2024-06-28T16:15:17.927891+0000","last_undegraded":"2024-06-28T16:15:17.927891+0000","last_fullsized":"2024-06-28T16:15:17.927891+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'318","last_scrub_stamp":"2024-06-28T16:15:17.927669+0000","last_deep_scrub":"260'318","last_deep_scrub_stamp":"2024-06-28T16:15:17.927669+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.927669+0000","objects_scrubbed":7,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:19:11.398970+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2421978,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":465,"num_read_kb":85416,"num_write":1148,"num_write_kb":27167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":522,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.295458+0000","last_change":"2024-06-28T16:15:14.295510+0000","last_active":"2024-06-28T16:15:14.295458+0000","last_peered":"2024-06-28T16:15:14.295458+0000","last_clean":"2024-06-28T16:15:14.295458+0000","last_became_active":"2024-06-28T15:00:19.382486+0000","last_became_peered":"2024-06-28T15:00:19.382486+0000","last_unstale":"2024-06-28T16:15:14.295458+0000","last_undegraded":"2024-06-28T16:15:14.295458+0000","last_fullsized":"2024-06-28T16:15:14.295458+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:14.295262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:14.295262+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.295262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:15:46.049931+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"260'166","reported_seq":745,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.148880+0000","last_change":"2024-06-28T16:15:10.148966+0000","last_active":"2024-06-28T16:15:10.148880+0000","last_peered":"2024-06-28T16:15:10.148880+0000","last_clean":"2024-06-28T16:15:10.148880+0000","last_became_active":"2024-06-28T15:00:21.400535+0000","last_became_peered":"2024-06-28T15:00:21.400535+0000","last_unstale":"2024-06-28T16:15:10.148880+0000","last_undegraded":"2024-06-28T16:15:10.148880+0000","last_fullsized":"2024-06-28T16:15:10.148880+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'166","last_scrub_stamp":"2024-06-28T16:15:10.148680+0000","last_deep_scrub":"260'166","last_deep_scrub_stamp":"2024-06-28T16:15:10.148680+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.148680+0000","objects_scrubbed":7,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:40:13.912505+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":198,"num_read_kb":191,"num_write":142,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1510,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.407107+0000","last_change":"2024-06-28T16:14:56.407161+0000","last_active":"2024-06-28T16:14:56.407107+0000","last_peered":"2024-06-28T16:14:56.407107+0000","last_clean":"2024-06-28T16:14:56.407107+0000","last_became_active":"2024-06-28T15:00:23.416701+0000","last_became_peered":"2024-06-28T15:00:23.416701+0000","last_unstale":"2024-06-28T16:14:56.407107+0000","last_undegraded":"2024-06-28T16:14:56.407107+0000","last_fullsized":"2024-06-28T16:14:56.407107+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.406907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.406907+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.406907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:09:09.502206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"260'117","reported_seq":642,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.620180+0000","last_change":"2024-06-28T16:14:47.620245+0000","last_active":"2024-06-28T16:14:47.620180+0000","last_peered":"2024-06-28T16:14:47.620180+0000","last_clean":"2024-06-28T16:14:47.620180+0000","last_became_active":"2024-06-28T15:00:25.438523+0000","last_became_peered":"2024-06-28T15:00:25.438523+0000","last_unstale":"2024-06-28T16:14:47.620180+0000","last_undegraded":"2024-06-28T16:14:47.620180+0000","last_fullsized":"2024-06-28T16:14:47.620180+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'117","last_scrub_stamp":"2024-06-28T16:14:47.619989+0000","last_deep_scrub":"260'117","last_deep_scrub_stamp":"2024-06-28T16:14:47.619989+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.619989+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:25:16.904136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":129,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"260'169","reported_seq":784,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.041377+0000","last_change":"2024-06-28T16:15:02.041377+0000","last_active":"2024-06-28T16:15:02.041377+0000","last_peered":"2024-06-28T16:15:02.041377+0000","last_clean":"2024-06-28T16:15:02.041377+0000","last_became_active":"2024-06-28T15:00:17.398515+0000","last_became_peered":"2024-06-28T15:00:17.398515+0000","last_unstale":"2024-06-28T16:15:02.041377+0000","last_undegraded":"2024-06-28T16:15:02.041377+0000","last_fullsized":"2024-06-28T16:15:02.041377+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'169","last_scrub_stamp":"2024-06-28T16:15:02.041106+0000","last_deep_scrub":"260'169","last_deep_scrub_stamp":"2024-06-28T16:15:02.041106+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.041106+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:44:07.334141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":644,"num_read_kb":512,"num_write":273,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"260'372","reported_seq":1052,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.858716+0000","last_change":"2024-06-28T16:15:18.858841+0000","last_active":"2024-06-28T16:15:18.858716+0000","last_peered":"2024-06-28T16:15:18.858716+0000","last_clean":"2024-06-28T16:15:18.858716+0000","last_became_active":"2024-06-28T15:00:15.414704+0000","last_became_peered":"2024-06-28T15:00:15.414704+0000","last_unstale":"2024-06-28T16:15:18.858716+0000","last_undegraded":"2024-06-28T16:15:18.858716+0000","last_fullsized":"2024-06-28T16:15:18.858716+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'372","last_scrub_stamp":"2024-06-28T16:15:18.858489+0000","last_deep_scrub":"260'372","last_deep_scrub_stamp":"2024-06-28T16:15:18.858489+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.858489+0000","objects_scrubbed":4,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:00:26.901636+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10506361,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":617,"num_read_kb":33619,"num_write":1289,"num_write_kb":23520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":539,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.163070+0000","last_change":"2024-06-28T16:15:08.163070+0000","last_active":"2024-06-28T16:15:08.163070+0000","last_peered":"2024-06-28T16:15:08.163070+0000","last_clean":"2024-06-28T16:15:08.163070+0000","last_became_active":"2024-06-28T15:00:06.302449+0000","last_became_peered":"2024-06-28T15:00:06.302449+0000","last_unstale":"2024-06-28T16:15:08.163070+0000","last_undegraded":"2024-06-28T16:15:08.163070+0000","last_fullsized":"2024-06-28T16:15:08.163070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:08.162767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:08.162767+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.162767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:45:36.489995+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":523,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.260730+0000","last_change":"2024-06-28T16:15:16.260730+0000","last_active":"2024-06-28T16:15:16.260730+0000","last_peered":"2024-06-28T16:15:16.260730+0000","last_clean":"2024-06-28T16:15:16.260730+0000","last_became_active":"2024-06-28T15:00:19.384933+0000","last_became_peered":"2024-06-28T15:00:19.384933+0000","last_unstale":"2024-06-28T16:15:16.260730+0000","last_undegraded":"2024-06-28T16:15:16.260730+0000","last_fullsized":"2024-06-28T16:15:16.260730+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:16.260473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:16.260473+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.260473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:45:58.084956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"260'597","reported_seq":1305,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.242218+0000","last_change":"2024-06-28T16:15:05.242315+0000","last_active":"2024-06-28T16:15:05.242218+0000","last_peered":"2024-06-28T16:15:05.242218+0000","last_clean":"2024-06-28T16:15:05.242218+0000","last_became_active":"2024-06-28T15:00:21.405762+0000","last_became_peered":"2024-06-28T15:00:21.405762+0000","last_unstale":"2024-06-28T16:15:05.242218+0000","last_undegraded":"2024-06-28T16:15:05.242218+0000","last_fullsized":"2024-06-28T16:15:05.242218+0000","mapping_epoch":25,"log_start":"260'500","ondisk_log_start":"260'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'597","last_scrub_stamp":"2024-06-28T16:15:05.241998+0000","last_deep_scrub":"260'597","last_deep_scrub_stamp":"2024-06-28T16:15:05.241998+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.241998+0000","objects_scrubbed":10,"log_size":97,"log_dups_size":500,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:43:06.700587+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92557,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1001,"num_read_kb":2235,"num_write":838,"num_write_kb":641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8196,"num_omap_keys":24,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":509,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.927443+0000","last_change":"2024-06-28T16:14:49.927496+0000","last_active":"2024-06-28T16:14:49.927443+0000","last_peered":"2024-06-28T16:14:49.927443+0000","last_clean":"2024-06-28T16:14:49.927443+0000","last_became_active":"2024-06-28T15:00:23.404714+0000","last_became_peered":"2024-06-28T15:00:23.404714+0000","last_unstale":"2024-06-28T16:14:49.927443+0000","last_undegraded":"2024-06-28T16:14:49.927443+0000","last_fullsized":"2024-06-28T16:14:49.927443+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.927238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.927238+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.927238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:46:09.338954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"260'154","reported_seq":680,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.151138+0000","last_change":"2024-06-28T16:14:48.151312+0000","last_active":"2024-06-28T16:14:48.151138+0000","last_peered":"2024-06-28T16:14:48.151138+0000","last_clean":"2024-06-28T16:14:48.151138+0000","last_became_active":"2024-06-28T15:00:25.556834+0000","last_became_peered":"2024-06-28T15:00:25.556834+0000","last_unstale":"2024-06-28T16:14:48.151138+0000","last_undegraded":"2024-06-28T16:14:48.151138+0000","last_fullsized":"2024-06-28T16:14:48.151138+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'154","last_scrub_stamp":"2024-06-28T16:14:48.150905+0000","last_deep_scrub":"260'154","last_deep_scrub_stamp":"2024-06-28T16:14:48.150905+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.150905+0000","objects_scrubbed":7,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:54:08.173965+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1183,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":342,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"260'139","reported_seq":778,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.225074+0000","last_change":"2024-06-28T16:15:19.225074+0000","last_active":"2024-06-28T16:15:19.225074+0000","last_peered":"2024-06-28T16:15:19.225074+0000","last_clean":"2024-06-28T16:15:19.225074+0000","last_became_active":"2024-06-28T15:00:17.372401+0000","last_became_peered":"2024-06-28T15:00:17.372401+0000","last_unstale":"2024-06-28T16:15:19.225074+0000","last_undegraded":"2024-06-28T16:15:19.225074+0000","last_fullsized":"2024-06-28T16:15:19.225074+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'139","last_scrub_stamp":"2024-06-28T16:15:19.224695+0000","last_deep_scrub":"260'139","last_deep_scrub_stamp":"2024-06-28T16:15:19.224695+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.224695+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:34:36.503538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":577,"num_read_kb":470,"num_write":221,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"260'273","reported_seq":911,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.187159+0000","last_change":"2024-06-28T16:15:11.187159+0000","last_active":"2024-06-28T16:15:11.187159+0000","last_peered":"2024-06-28T16:15:11.187159+0000","last_clean":"2024-06-28T16:15:11.187159+0000","last_became_active":"2024-06-28T15:00:15.408140+0000","last_became_peered":"2024-06-28T15:00:15.408140+0000","last_unstale":"2024-06-28T16:15:11.187159+0000","last_undegraded":"2024-06-28T16:15:11.187159+0000","last_fullsized":"2024-06-28T16:15:11.187159+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'273","last_scrub_stamp":"2024-06-28T16:15:11.186833+0000","last_deep_scrub":"260'273","last_deep_scrub_stamp":"2024-06-28T16:15:11.186833+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.186833+0000","objects_scrubbed":10,"log_size":73,"log_dups_size":200,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:13:01.399899+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":526399,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":387,"num_read_kb":8190,"num_write":1081,"num_write_kb":24156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":525,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:20.122148+0000","last_change":"2024-06-28T15:00:06.303984+0000","last_active":"2024-06-28T16:13:20.122148+0000","last_peered":"2024-06-28T16:13:20.122148+0000","last_clean":"2024-06-28T16:13:20.122148+0000","last_became_active":"2024-06-28T15:00:06.302962+0000","last_became_peered":"2024-06-28T15:00:06.302962+0000","last_unstale":"2024-06-28T16:13:20.122148+0000","last_undegraded":"2024-06-28T16:13:20.122148+0000","last_fullsized":"2024-06-28T16:13:20.122148+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:05.258724+0000","last_clean_scrub_stamp":"2024-06-28T15:00:05.258724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.184441+0000","last_change":"2024-06-28T16:14:49.184441+0000","last_active":"2024-06-28T16:14:49.184441+0000","last_peered":"2024-06-28T16:14:49.184441+0000","last_clean":"2024-06-28T16:14:49.184441+0000","last_became_active":"2024-06-28T15:00:19.384961+0000","last_became_peered":"2024-06-28T15:00:19.384961+0000","last_unstale":"2024-06-28T16:14:49.184441+0000","last_undegraded":"2024-06-28T16:14:49.184441+0000","last_fullsized":"2024-06-28T16:14:49.184441+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.184197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.184197+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.184197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:48:38.391991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"260'195","reported_seq":801,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.962522+0000","last_change":"2024-06-28T16:14:58.962600+0000","last_active":"2024-06-28T16:14:58.962522+0000","last_peered":"2024-06-28T16:14:58.962522+0000","last_clean":"2024-06-28T16:14:58.962522+0000","last_became_active":"2024-06-28T15:00:21.480107+0000","last_became_peered":"2024-06-28T15:00:21.480107+0000","last_unstale":"2024-06-28T16:14:58.962522+0000","last_undegraded":"2024-06-28T16:14:58.962522+0000","last_fullsized":"2024-06-28T16:14:58.962522+0000","mapping_epoch":25,"log_start":"257'100","ondisk_log_start":"257'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'195","last_scrub_stamp":"2024-06-28T16:14:58.962331+0000","last_deep_scrub":"260'195","last_deep_scrub_stamp":"2024-06-28T16:14:58.962331+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.962331+0000","objects_scrubbed":7,"log_size":95,"log_dups_size":100,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:08:23.304315+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":263,"num_read_kb":253,"num_write":174,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"260'565","reported_seq":2887,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.666227+0000","last_change":"2024-06-28T16:14:51.666291+0000","last_active":"2024-06-28T16:14:51.666227+0000","last_peered":"2024-06-28T16:14:51.666227+0000","last_clean":"2024-06-28T16:14:51.666227+0000","last_became_active":"2024-06-28T15:00:23.427565+0000","last_became_peered":"2024-06-28T15:00:23.427565+0000","last_unstale":"2024-06-28T16:14:51.666227+0000","last_undegraded":"2024-06-28T16:14:51.666227+0000","last_fullsized":"2024-06-28T16:14:51.666227+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'565","last_scrub_stamp":"2024-06-28T16:14:51.666022+0000","last_deep_scrub":"260'565","last_deep_scrub_stamp":"2024-06-28T16:14:51.666022+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.666022+0000","objects_scrubbed":1,"log_size":65,"log_dups_size":500,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:06:09.388341+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"260'127","reported_seq":651,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.139109+0000","last_change":"2024-06-28T16:14:51.139189+0000","last_active":"2024-06-28T16:14:51.139109+0000","last_peered":"2024-06-28T16:14:51.139109+0000","last_clean":"2024-06-28T16:14:51.139109+0000","last_became_active":"2024-06-28T15:00:25.553627+0000","last_became_peered":"2024-06-28T15:00:25.553627+0000","last_unstale":"2024-06-28T16:14:51.139109+0000","last_undegraded":"2024-06-28T16:14:51.139109+0000","last_fullsized":"2024-06-28T16:14:51.139109+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'127","last_scrub_stamp":"2024-06-28T16:14:51.138914+0000","last_deep_scrub":"260'127","last_deep_scrub_stamp":"2024-06-28T16:14:51.138914+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.138914+0000","objects_scrubbed":3,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:08:31.042999+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":713,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":165,"num_read_kb":148,"num_write":308,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"260'172","reported_seq":834,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.189062+0000","last_change":"2024-06-28T16:15:18.189235+0000","last_active":"2024-06-28T16:15:18.189062+0000","last_peered":"2024-06-28T16:15:18.189062+0000","last_clean":"2024-06-28T16:15:18.189062+0000","last_became_active":"2024-06-28T15:00:17.376516+0000","last_became_peered":"2024-06-28T15:00:17.376516+0000","last_unstale":"2024-06-28T16:15:18.189062+0000","last_undegraded":"2024-06-28T16:15:18.189062+0000","last_fullsized":"2024-06-28T16:15:18.189062+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'172","last_scrub_stamp":"2024-06-28T16:15:18.188830+0000","last_deep_scrub":"260'172","last_deep_scrub_stamp":"2024-06-28T16:15:18.188830+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.188830+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:08:43.423312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":722,"num_read_kb":588,"num_write":244,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"260'336","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.742328+0000","last_change":"2024-06-28T15:00:15.403594+0000","last_active":"2024-06-28T16:14:34.742328+0000","last_peered":"2024-06-28T16:14:34.742328+0000","last_clean":"2024-06-28T16:14:34.742328+0000","last_became_active":"2024-06-28T15:00:15.403032+0000","last_became_peered":"2024-06-28T15:00:15.403032+0000","last_unstale":"2024-06-28T16:14:34.742328+0000","last_undegraded":"2024-06-28T16:14:34.742328+0000","last_fullsized":"2024-06-28T16:14:34.742328+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":968026,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":483,"num_read_kb":17032,"num_write":1147,"num_write_kb":35885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":537,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.172483+0000","last_change":"2024-06-28T16:15:07.172536+0000","last_active":"2024-06-28T16:15:07.172483+0000","last_peered":"2024-06-28T16:15:07.172483+0000","last_clean":"2024-06-28T16:15:07.172483+0000","last_became_active":"2024-06-28T15:00:06.278443+0000","last_became_peered":"2024-06-28T15:00:06.278443+0000","last_unstale":"2024-06-28T16:15:07.172483+0000","last_undegraded":"2024-06-28T16:15:07.172483+0000","last_fullsized":"2024-06-28T16:15:07.172483+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:07.172293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:07.172293+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.172293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:30:00.544437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":518,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.999237+0000","last_change":"2024-06-28T16:15:08.999237+0000","last_active":"2024-06-28T16:15:08.999237+0000","last_peered":"2024-06-28T16:15:08.999237+0000","last_clean":"2024-06-28T16:15:08.999237+0000","last_became_active":"2024-06-28T15:00:19.378932+0000","last_became_peered":"2024-06-28T15:00:19.378932+0000","last_unstale":"2024-06-28T16:15:08.999237+0000","last_undegraded":"2024-06-28T16:15:08.999237+0000","last_fullsized":"2024-06-28T16:15:08.999237+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:08.998981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:08.998981+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.998981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:59:56.713595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"260'143","reported_seq":712,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.986875+0000","last_change":"2024-06-28T16:14:54.986950+0000","last_active":"2024-06-28T16:14:54.986875+0000","last_peered":"2024-06-28T16:14:54.986875+0000","last_clean":"2024-06-28T16:14:54.986875+0000","last_became_active":"2024-06-28T15:00:21.395462+0000","last_became_peered":"2024-06-28T15:00:21.395462+0000","last_unstale":"2024-06-28T16:14:54.986875+0000","last_undegraded":"2024-06-28T16:14:54.986875+0000","last_fullsized":"2024-06-28T16:14:54.986875+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'143","last_scrub_stamp":"2024-06-28T16:14:54.986685+0000","last_deep_scrub":"260'143","last_deep_scrub_stamp":"2024-06-28T16:14:54.986685+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.986685+0000","objects_scrubbed":5,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:03:28.897014+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":167,"num_read_kb":161,"num_write":112,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"260'537","reported_seq":2590,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.176626+0000","last_change":"2024-06-28T16:14:58.176689+0000","last_active":"2024-06-28T16:14:58.176626+0000","last_peered":"2024-06-28T16:14:58.176626+0000","last_clean":"2024-06-28T16:14:58.176626+0000","last_became_active":"2024-06-28T15:00:23.427568+0000","last_became_peered":"2024-06-28T15:00:23.427568+0000","last_unstale":"2024-06-28T16:14:58.176626+0000","last_undegraded":"2024-06-28T16:14:58.176626+0000","last_fullsized":"2024-06-28T16:14:58.176626+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'537","last_scrub_stamp":"2024-06-28T16:14:58.176428+0000","last_deep_scrub":"260'537","last_deep_scrub_stamp":"2024-06-28T16:14:58.176428+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.176428+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:07:27.328025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"260'251","reported_seq":800,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.105514+0000","last_change":"2024-06-28T16:14:53.105587+0000","last_active":"2024-06-28T16:14:53.105514+0000","last_peered":"2024-06-28T16:14:53.105514+0000","last_clean":"2024-06-28T16:14:53.105514+0000","last_became_active":"2024-06-28T15:00:25.428067+0000","last_became_peered":"2024-06-28T15:00:25.428067+0000","last_unstale":"2024-06-28T16:14:53.105514+0000","last_undegraded":"2024-06-28T16:14:53.105514+0000","last_fullsized":"2024-06-28T16:14:53.105514+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'251","last_scrub_stamp":"2024-06-28T16:14:53.105331+0000","last_deep_scrub":"260'251","last_deep_scrub_stamp":"2024-06-28T16:14:53.105331+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.105331+0000","objects_scrubbed":4,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:01:44.076283+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":297,"num_write":552,"num_write_kb":170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"260'96","reported_seq":711,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.050892+0000","last_change":"2024-06-28T16:15:12.050892+0000","last_active":"2024-06-28T16:15:12.050892+0000","last_peered":"2024-06-28T16:15:12.050892+0000","last_clean":"2024-06-28T16:15:12.050892+0000","last_became_active":"2024-06-28T15:00:17.396413+0000","last_became_peered":"2024-06-28T15:00:17.396413+0000","last_unstale":"2024-06-28T16:15:12.050892+0000","last_undegraded":"2024-06-28T16:15:12.050892+0000","last_fullsized":"2024-06-28T16:15:12.050892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'96","last_scrub_stamp":"2024-06-28T16:15:12.050611+0000","last_deep_scrub":"260'96","last_deep_scrub_stamp":"2024-06-28T16:15:12.050611+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.050611+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:35:33.992945+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":410,"num_read_kb":340,"num_write":139,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"260'318","reported_seq":1098,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.743460+0000","last_change":"2024-06-28T15:00:15.396426+0000","last_active":"2024-06-28T16:14:34.743460+0000","last_peered":"2024-06-28T16:14:34.743460+0000","last_clean":"2024-06-28T16:14:34.743460+0000","last_became_active":"2024-06-28T15:00:15.395980+0000","last_became_peered":"2024-06-28T15:00:15.395980+0000","last_unstale":"2024-06-28T16:14:34.743460+0000","last_undegraded":"2024-06-28T16:14:34.743460+0000","last_fullsized":"2024-06-28T16:14:34.743460+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661262,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":649,"num_read_kb":125515,"num_write":1187,"num_write_kb":17513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":530,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.460147+0000","last_change":"2024-06-28T16:15:04.460199+0000","last_active":"2024-06-28T16:15:04.460147+0000","last_peered":"2024-06-28T16:15:04.460147+0000","last_clean":"2024-06-28T16:15:04.460147+0000","last_became_active":"2024-06-28T15:00:06.283380+0000","last_became_peered":"2024-06-28T15:00:06.283380+0000","last_unstale":"2024-06-28T16:15:04.460147+0000","last_undegraded":"2024-06-28T16:15:04.460147+0000","last_fullsized":"2024-06-28T16:15:04.460147+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:04.459951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:04.459951+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.459951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:06:10.142541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"260'6","reported_seq":852,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.154538+0000","last_change":"2024-06-28T16:15:17.154602+0000","last_active":"2024-06-28T16:15:17.154538+0000","last_peered":"2024-06-28T16:15:17.154538+0000","last_clean":"2024-06-28T16:15:17.154538+0000","last_became_active":"2024-06-28T15:00:19.372552+0000","last_became_peered":"2024-06-28T15:00:19.372552+0000","last_unstale":"2024-06-28T16:15:17.154538+0000","last_undegraded":"2024-06-28T16:15:17.154538+0000","last_fullsized":"2024-06-28T16:15:17.154538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:15:17.154344+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:15:17.154344+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.154344+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:46:02.206707+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":500,"num_read_kb":812,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"260'748","reported_seq":1493,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:33.577429+0000","last_change":"2024-06-28T15:00:21.481881+0000","last_active":"2024-06-28T16:14:33.577429+0000","last_peered":"2024-06-28T16:14:33.577429+0000","last_clean":"2024-06-28T16:14:33.577429+0000","last_became_active":"2024-06-28T15:00:21.481155+0000","last_became_peered":"2024-06-28T15:00:21.481155+0000","last_unstale":"2024-06-28T16:14:33.577429+0000","last_undegraded":"2024-06-28T16:14:33.577429+0000","last_fullsized":"2024-06-28T16:14:33.577429+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":700,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131438,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1320,"num_read_kb":3166,"num_write":1058,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":512,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.234931+0000","last_change":"2024-06-28T16:15:02.235034+0000","last_active":"2024-06-28T16:15:02.234931+0000","last_peered":"2024-06-28T16:15:02.234931+0000","last_clean":"2024-06-28T16:15:02.234931+0000","last_became_active":"2024-06-28T15:00:23.408777+0000","last_became_peered":"2024-06-28T15:00:23.408777+0000","last_unstale":"2024-06-28T16:15:02.234931+0000","last_undegraded":"2024-06-28T16:15:02.234931+0000","last_fullsized":"2024-06-28T16:15:02.234931+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:02.234718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:02.234718+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.234718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:27:42.722800+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"260'132","reported_seq":653,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.670572+0000","last_change":"2024-06-28T16:14:49.670641+0000","last_active":"2024-06-28T16:14:49.670572+0000","last_peered":"2024-06-28T16:14:49.670572+0000","last_clean":"2024-06-28T16:14:49.670572+0000","last_became_active":"2024-06-28T15:00:25.438476+0000","last_became_peered":"2024-06-28T15:00:25.438476+0000","last_unstale":"2024-06-28T16:14:49.670572+0000","last_undegraded":"2024-06-28T16:14:49.670572+0000","last_fullsized":"2024-06-28T16:14:49.670572+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'132","last_scrub_stamp":"2024-06-28T16:14:49.670359+0000","last_deep_scrub":"260'132","last_deep_scrub_stamp":"2024-06-28T16:14:49.670359+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.670359+0000","objects_scrubbed":3,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:09:32.566857+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":173,"num_read_kb":156,"num_write":320,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"260'153","reported_seq":729,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.120485+0000","last_change":"2024-06-28T16:14:49.120558+0000","last_active":"2024-06-28T16:14:49.120485+0000","last_peered":"2024-06-28T16:14:49.120485+0000","last_clean":"2024-06-28T16:14:49.120485+0000","last_became_active":"2024-06-28T15:00:25.439672+0000","last_became_peered":"2024-06-28T15:00:25.439672+0000","last_unstale":"2024-06-28T16:14:49.120485+0000","last_undegraded":"2024-06-28T16:14:49.120485+0000","last_fullsized":"2024-06-28T16:14:49.120485+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'153","last_scrub_stamp":"2024-06-28T16:14:49.120288+0000","last_deep_scrub":"260'153","last_deep_scrub_stamp":"2024-06-28T16:14:49.120288+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.120288+0000","objects_scrubbed":2,"log_size":53,"log_dups_size":100,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:05:56.112079+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":207,"num_read_kb":205,"num_write":345,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"260'435","reported_seq":1082,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.041132+0000","last_change":"2024-06-28T16:15:10.041196+0000","last_active":"2024-06-28T16:15:10.041132+0000","last_peered":"2024-06-28T16:15:10.041132+0000","last_clean":"2024-06-28T16:15:10.041132+0000","last_became_active":"2024-06-28T15:00:17.401981+0000","last_became_peered":"2024-06-28T15:00:17.401981+0000","last_unstale":"2024-06-28T16:15:10.041132+0000","last_undegraded":"2024-06-28T16:15:10.041132+0000","last_fullsized":"2024-06-28T16:15:10.041132+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'435","last_scrub_stamp":"2024-06-28T16:15:10.040927+0000","last_deep_scrub":"260'435","last_deep_scrub_stamp":"2024-06-28T16:15:10.040927+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.040927+0000","objects_scrubbed":1,"log_size":35,"log_dups_size":400,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:52:25.537900+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1728,"num_read_kb":1391,"num_write":746,"num_write_kb":387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":230,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"260'274","reported_seq":910,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.992519+0000","last_change":"2024-06-28T16:15:15.992709+0000","last_active":"2024-06-28T16:15:15.992519+0000","last_peered":"2024-06-28T16:15:15.992519+0000","last_clean":"2024-06-28T16:15:15.992519+0000","last_became_active":"2024-06-28T15:00:15.403450+0000","last_became_peered":"2024-06-28T15:00:15.403450+0000","last_unstale":"2024-06-28T16:15:15.992519+0000","last_undegraded":"2024-06-28T16:15:15.992519+0000","last_fullsized":"2024-06-28T16:15:15.992519+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'274","last_scrub_stamp":"2024-06-28T16:15:15.992200+0000","last_deep_scrub":"260'274","last_deep_scrub_stamp":"2024-06-28T16:15:15.992200+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.992200+0000","objects_scrubbed":4,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:27:40.312409+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437799,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":13452,"num_write":1073,"num_write_kb":25751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":518,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.005157+0000","last_change":"2024-06-28T16:14:51.005157+0000","last_active":"2024-06-28T16:14:51.005157+0000","last_peered":"2024-06-28T16:14:51.005157+0000","last_clean":"2024-06-28T16:14:51.005157+0000","last_became_active":"2024-06-28T15:00:19.398138+0000","last_became_peered":"2024-06-28T15:00:19.398138+0000","last_unstale":"2024-06-28T16:14:51.005157+0000","last_undegraded":"2024-06-28T16:14:51.005157+0000","last_fullsized":"2024-06-28T16:14:51.005157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.004896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.004896+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.004896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:21:37.359914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"260'304","reported_seq":911,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.170902+0000","last_change":"2024-06-28T16:15:08.170995+0000","last_active":"2024-06-28T16:15:08.170902+0000","last_peered":"2024-06-28T16:15:08.170902+0000","last_clean":"2024-06-28T16:15:08.170902+0000","last_became_active":"2024-06-28T15:00:21.405016+0000","last_became_peered":"2024-06-28T15:00:21.405016+0000","last_unstale":"2024-06-28T16:15:08.170902+0000","last_undegraded":"2024-06-28T16:15:08.170902+0000","last_fullsized":"2024-06-28T16:15:08.170902+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'304","last_scrub_stamp":"2024-06-28T16:15:08.170703+0000","last_deep_scrub":"260'304","last_deep_scrub_stamp":"2024-06-28T16:15:08.170703+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.170703+0000","objects_scrubbed":6,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:57:06.513866+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53726,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":505,"num_read_kb":1165,"num_write":406,"num_write_kb":309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":509,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.204716+0000","last_change":"2024-06-28T16:15:00.204716+0000","last_active":"2024-06-28T16:15:00.204716+0000","last_peered":"2024-06-28T16:15:00.204716+0000","last_clean":"2024-06-28T16:15:00.204716+0000","last_became_active":"2024-06-28T15:00:23.417390+0000","last_became_peered":"2024-06-28T15:00:23.417390+0000","last_unstale":"2024-06-28T16:15:00.204716+0000","last_undegraded":"2024-06-28T16:15:00.204716+0000","last_fullsized":"2024-06-28T16:15:00.204716+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:00.204463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:00.204463+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.204463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:41:34.935690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"260'101","reported_seq":617,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.114583+0000","last_change":"2024-06-28T16:14:56.114642+0000","last_active":"2024-06-28T16:14:56.114583+0000","last_peered":"2024-06-28T16:14:56.114583+0000","last_clean":"2024-06-28T16:14:56.114583+0000","last_became_active":"2024-06-28T15:00:25.412379+0000","last_became_peered":"2024-06-28T15:00:25.412379+0000","last_unstale":"2024-06-28T16:14:56.114583+0000","last_undegraded":"2024-06-28T16:14:56.114583+0000","last_fullsized":"2024-06-28T16:14:56.114583+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'101","last_scrub_stamp":"2024-06-28T16:14:56.114398+0000","last_deep_scrub":"260'101","last_deep_scrub_stamp":"2024-06-28T16:14:56.114398+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.114398+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:50:04.756947+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":88,"num_write":238,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"260'189","reported_seq":832,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.954505+0000","last_change":"2024-06-28T16:14:57.954505+0000","last_active":"2024-06-28T16:14:57.954505+0000","last_peered":"2024-06-28T16:14:57.954505+0000","last_clean":"2024-06-28T16:14:57.954505+0000","last_became_active":"2024-06-28T15:00:17.406647+0000","last_became_peered":"2024-06-28T15:00:17.406647+0000","last_unstale":"2024-06-28T16:14:57.954505+0000","last_undegraded":"2024-06-28T16:14:57.954505+0000","last_fullsized":"2024-06-28T16:14:57.954505+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'189","last_scrub_stamp":"2024-06-28T16:14:57.954211+0000","last_deep_scrub":"260'189","last_deep_scrub_stamp":"2024-06-28T16:14:57.954211+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.954211+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:15:57.967237+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":748,"num_read_kb":601,"num_write":301,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"260'334","reported_seq":1003,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.802740+0000","last_change":"2024-06-28T15:00:15.413269+0000","last_active":"2024-06-28T16:14:41.802740+0000","last_peered":"2024-06-28T16:14:41.802740+0000","last_clean":"2024-06-28T16:14:41.802740+0000","last_became_active":"2024-06-28T15:00:15.402945+0000","last_became_peered":"2024-06-28T15:00:15.402945+0000","last_unstale":"2024-06-28T16:14:41.802740+0000","last_undegraded":"2024-06-28T16:14:41.802740+0000","last_fullsized":"2024-06-28T16:14:41.802740+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340417,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":534,"num_read_kb":20609,"num_write":1305,"num_write_kb":45266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,4],"acting":[7,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":509,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.971516+0000","last_change":"2024-06-28T15:00:19.383614+0000","last_active":"2024-06-28T16:14:39.971516+0000","last_peered":"2024-06-28T16:14:39.971516+0000","last_clean":"2024-06-28T16:14:39.971516+0000","last_became_active":"2024-06-28T15:00:19.382839+0000","last_became_peered":"2024-06-28T15:00:19.382839+0000","last_unstale":"2024-06-28T16:14:39.971516+0000","last_undegraded":"2024-06-28T16:14:39.971516+0000","last_fullsized":"2024-06-28T16:14:39.971516+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:18.353169+0000","last_clean_scrub_stamp":"2024-06-28T15:00:18.353169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"260'55","reported_seq":586,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.212479+0000","last_change":"2024-06-28T16:15:07.212638+0000","last_active":"2024-06-28T16:15:07.212479+0000","last_peered":"2024-06-28T16:15:07.212479+0000","last_clean":"2024-06-28T16:15:07.212479+0000","last_became_active":"2024-06-28T15:00:21.404727+0000","last_became_peered":"2024-06-28T15:00:21.404727+0000","last_unstale":"2024-06-28T16:15:07.212479+0000","last_undegraded":"2024-06-28T16:15:07.212479+0000","last_fullsized":"2024-06-28T16:15:07.212479+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'55","last_scrub_stamp":"2024-06-28T16:15:07.212261+0000","last_deep_scrub":"260'55","last_deep_scrub_stamp":"2024-06-28T16:15:07.212261+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.212261+0000","objects_scrubbed":3,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:08:38.234431+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":48,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.776013+0000","last_change":"2024-06-28T16:15:01.776013+0000","last_active":"2024-06-28T16:15:01.776013+0000","last_peered":"2024-06-28T16:15:01.776013+0000","last_clean":"2024-06-28T16:15:01.776013+0000","last_became_active":"2024-06-28T15:00:23.419115+0000","last_became_peered":"2024-06-28T15:00:23.419115+0000","last_unstale":"2024-06-28T16:15:01.776013+0000","last_undegraded":"2024-06-28T16:15:01.776013+0000","last_fullsized":"2024-06-28T16:15:01.776013+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:01.775641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:01.775641+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.775641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:56:43.293391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"260'191","reported_seq":729,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.032013+0000","last_change":"2024-06-28T16:14:55.032091+0000","last_active":"2024-06-28T16:14:55.032013+0000","last_peered":"2024-06-28T16:14:55.032013+0000","last_clean":"2024-06-28T16:14:55.032013+0000","last_became_active":"2024-06-28T15:00:25.426046+0000","last_became_peered":"2024-06-28T15:00:25.426046+0000","last_unstale":"2024-06-28T16:14:55.032013+0000","last_undegraded":"2024-06-28T16:14:55.032013+0000","last_fullsized":"2024-06-28T16:14:55.032013+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'191","last_scrub_stamp":"2024-06-28T16:14:55.031799+0000","last_deep_scrub":"260'191","last_deep_scrub_stamp":"2024-06-28T16:14:55.031799+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.031799+0000","objects_scrubbed":1,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:06:38.230720+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":286,"num_read_kb":283,"num_write":434,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"260'423","reported_seq":1180,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.153858+0000","last_change":"2024-06-28T16:15:13.154003+0000","last_active":"2024-06-28T16:15:13.153858+0000","last_peered":"2024-06-28T16:15:13.153858+0000","last_clean":"2024-06-28T16:15:13.153858+0000","last_became_active":"2024-06-28T15:00:17.389992+0000","last_became_peered":"2024-06-28T15:00:17.389992+0000","last_unstale":"2024-06-28T16:15:13.153858+0000","last_undegraded":"2024-06-28T16:15:13.153858+0000","last_fullsized":"2024-06-28T16:15:13.153858+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'423","last_scrub_stamp":"2024-06-28T16:15:13.153624+0000","last_deep_scrub":"260'423","last_deep_scrub_stamp":"2024-06-28T16:15:13.153624+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.153624+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":400,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:59:03.430418+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1786,"num_read_kb":1490,"num_write":759,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"260'328","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.730674+0000","last_change":"2024-06-28T16:15:13.730884+0000","last_active":"2024-06-28T16:15:13.730674+0000","last_peered":"2024-06-28T16:15:13.730674+0000","last_clean":"2024-06-28T16:15:13.730674+0000","last_became_active":"2024-06-28T15:00:15.419267+0000","last_became_peered":"2024-06-28T15:00:15.419267+0000","last_unstale":"2024-06-28T16:15:13.730674+0000","last_undegraded":"2024-06-28T16:15:13.730674+0000","last_fullsized":"2024-06-28T16:15:13.730674+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'328","last_scrub_stamp":"2024-06-28T16:15:13.730424+0000","last_deep_scrub":"260'328","last_deep_scrub_stamp":"2024-06-28T16:15:13.730424+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.730424+0000","objects_scrubbed":5,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:57:21.223073+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5413475,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":487,"num_read_kb":27459,"num_write":1288,"num_write_kb":48933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":512,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.747129+0000","last_change":"2024-06-28T16:15:06.747183+0000","last_active":"2024-06-28T16:15:06.747129+0000","last_peered":"2024-06-28T16:15:06.747129+0000","last_clean":"2024-06-28T16:15:06.747129+0000","last_became_active":"2024-06-28T15:00:19.385990+0000","last_became_peered":"2024-06-28T15:00:19.385990+0000","last_unstale":"2024-06-28T16:15:06.747129+0000","last_undegraded":"2024-06-28T16:15:06.747129+0000","last_fullsized":"2024-06-28T16:15:06.747129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:06.746938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:06.746938+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.746938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:07:08.302586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"260'323","reported_seq":946,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.152026+0000","last_change":"2024-06-28T16:15:07.152103+0000","last_active":"2024-06-28T16:15:07.152026+0000","last_peered":"2024-06-28T16:15:07.152026+0000","last_clean":"2024-06-28T16:15:07.152026+0000","last_became_active":"2024-06-28T15:00:21.405045+0000","last_became_peered":"2024-06-28T15:00:21.405045+0000","last_unstale":"2024-06-28T16:15:07.152026+0000","last_undegraded":"2024-06-28T16:15:07.152026+0000","last_fullsized":"2024-06-28T16:15:07.152026+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'323","last_scrub_stamp":"2024-06-28T16:15:07.151832+0000","last_deep_scrub":"260'323","last_deep_scrub_stamp":"2024-06-28T16:15:07.151832+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.151832+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:20:40.489212+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48456,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":530,"num_read_kb":1162,"num_write":414,"num_write_kb":289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.144381+0000","last_change":"2024-06-28T16:14:50.144381+0000","last_active":"2024-06-28T16:14:50.144381+0000","last_peered":"2024-06-28T16:14:50.144381+0000","last_clean":"2024-06-28T16:14:50.144381+0000","last_became_active":"2024-06-28T15:00:23.411264+0000","last_became_peered":"2024-06-28T15:00:23.411264+0000","last_unstale":"2024-06-28T16:14:50.144381+0000","last_undegraded":"2024-06-28T16:14:50.144381+0000","last_fullsized":"2024-06-28T16:14:50.144381+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:50.144047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:50.144047+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.144047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:56:31.669987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"260'113","reported_seq":614,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.683239+0000","last_change":"2024-06-28T16:14:43.683376+0000","last_active":"2024-06-28T16:14:43.683239+0000","last_peered":"2024-06-28T16:14:43.683239+0000","last_clean":"2024-06-28T16:14:43.683239+0000","last_became_active":"2024-06-28T15:00:25.442010+0000","last_became_peered":"2024-06-28T15:00:25.442010+0000","last_unstale":"2024-06-28T16:14:43.683239+0000","last_undegraded":"2024-06-28T16:14:43.683239+0000","last_fullsized":"2024-06-28T16:14:43.683239+0000","mapping_epoch":29,"log_start":"260'101","ondisk_log_start":"260'101","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'113","last_scrub_stamp":"2024-06-28T16:14:43.682925+0000","last_deep_scrub":"260'113","last_deep_scrub_stamp":"2024-06-28T16:14:43.682925+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.682925+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":101,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:07:47.348656+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":638,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":85,"num_write":260,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"260'105","reported_seq":679,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:20.277565+0000","last_change":"2024-06-28T16:15:20.277626+0000","last_active":"2024-06-28T16:15:20.277565+0000","last_peered":"2024-06-28T16:15:20.277565+0000","last_clean":"2024-06-28T16:15:20.277565+0000","last_became_active":"2024-06-28T15:00:17.373396+0000","last_became_peered":"2024-06-28T15:00:17.373396+0000","last_unstale":"2024-06-28T16:15:20.277565+0000","last_undegraded":"2024-06-28T16:15:20.277565+0000","last_fullsized":"2024-06-28T16:15:20.277565+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'105","last_scrub_stamp":"2024-06-28T16:15:20.277371+0000","last_deep_scrub":"260'105","last_deep_scrub_stamp":"2024-06-28T16:15:20.277371+0000","last_clean_scrub_stamp":"2024-06-28T16:15:20.277371+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:05:20.707326+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":424,"num_read_kb":340,"num_write":155,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"260'307","reported_seq":956,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.608467+0000","last_change":"2024-06-28T15:00:15.412826+0000","last_active":"2024-06-28T16:14:38.608467+0000","last_peered":"2024-06-28T16:14:38.608467+0000","last_clean":"2024-06-28T16:14:38.608467+0000","last_became_active":"2024-06-28T15:00:15.412333+0000","last_became_peered":"2024-06-28T15:00:15.412333+0000","last_unstale":"2024-06-28T16:14:38.608467+0000","last_undegraded":"2024-06-28T16:14:38.608467+0000","last_fullsized":"2024-06-28T16:14:38.608467+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1451455,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":458,"num_read_kb":31119,"num_write":1276,"num_write_kb":43155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.188245+0000","last_change":"2024-06-28T16:14:51.188245+0000","last_active":"2024-06-28T16:14:51.188245+0000","last_peered":"2024-06-28T16:14:51.188245+0000","last_clean":"2024-06-28T16:14:51.188245+0000","last_became_active":"2024-06-28T15:00:19.386645+0000","last_became_peered":"2024-06-28T15:00:19.386645+0000","last_unstale":"2024-06-28T16:14:51.188245+0000","last_undegraded":"2024-06-28T16:14:51.188245+0000","last_fullsized":"2024-06-28T16:14:51.188245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.187878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.187878+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.187878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:53:35.500317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"260'293","reported_seq":887,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.836441+0000","last_change":"2024-06-28T16:15:16.836538+0000","last_active":"2024-06-28T16:15:16.836441+0000","last_peered":"2024-06-28T16:15:16.836441+0000","last_clean":"2024-06-28T16:15:16.836441+0000","last_became_active":"2024-06-28T15:00:21.394357+0000","last_became_peered":"2024-06-28T15:00:21.394357+0000","last_unstale":"2024-06-28T16:15:16.836441+0000","last_undegraded":"2024-06-28T16:15:16.836441+0000","last_fullsized":"2024-06-28T16:15:16.836441+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'293","last_scrub_stamp":"2024-06-28T16:15:16.836265+0000","last_deep_scrub":"260'293","last_deep_scrub_stamp":"2024-06-28T16:15:16.836265+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.836265+0000","objects_scrubbed":4,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:39:45.845728+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53697,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":496,"num_read_kb":1194,"num_write":404,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":511,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.135809+0000","last_change":"2024-06-28T16:14:59.135809+0000","last_active":"2024-06-28T16:14:59.135809+0000","last_peered":"2024-06-28T16:14:59.135809+0000","last_clean":"2024-06-28T16:14:59.135809+0000","last_became_active":"2024-06-28T15:00:23.406296+0000","last_became_peered":"2024-06-28T15:00:23.406296+0000","last_unstale":"2024-06-28T16:14:59.135809+0000","last_undegraded":"2024-06-28T16:14:59.135809+0000","last_fullsized":"2024-06-28T16:14:59.135809+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:59.135545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:59.135545+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.135545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:33:44.459217+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"260'68","reported_seq":646,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.449222+0000","last_change":"2024-06-28T16:15:01.449222+0000","last_active":"2024-06-28T16:15:01.449222+0000","last_peered":"2024-06-28T16:15:01.449222+0000","last_clean":"2024-06-28T16:15:01.449222+0000","last_became_active":"2024-06-28T15:00:17.396861+0000","last_became_peered":"2024-06-28T15:00:17.396861+0000","last_unstale":"2024-06-28T16:15:01.449222+0000","last_undegraded":"2024-06-28T16:15:01.449222+0000","last_fullsized":"2024-06-28T16:15:01.449222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'68","last_scrub_stamp":"2024-06-28T16:15:01.448976+0000","last_deep_scrub":"260'68","last_deep_scrub_stamp":"2024-06-28T16:15:01.448976+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.448976+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:55:04.325605+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":279,"num_read_kb":229,"num_write":93,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"260'309","reported_seq":946,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.971144+0000","last_change":"2024-06-28T15:00:15.383453+0000","last_active":"2024-06-28T16:14:39.971144+0000","last_peered":"2024-06-28T16:14:39.971144+0000","last_clean":"2024-06-28T16:14:39.971144+0000","last_became_active":"2024-06-28T15:00:15.383008+0000","last_became_peered":"2024-06-28T15:00:15.383008+0000","last_unstale":"2024-06-28T16:14:39.971144+0000","last_undegraded":"2024-06-28T16:14:39.971144+0000","last_fullsized":"2024-06-28T16:14:39.971144+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2156401,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":459,"num_read_kb":6724,"num_write":1144,"num_write_kb":24207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.216032+0000","last_change":"2024-06-28T16:15:18.216084+0000","last_active":"2024-06-28T16:15:18.216032+0000","last_peered":"2024-06-28T16:15:18.216032+0000","last_clean":"2024-06-28T16:15:18.216032+0000","last_became_active":"2024-06-28T15:00:19.390111+0000","last_became_peered":"2024-06-28T15:00:19.390111+0000","last_unstale":"2024-06-28T16:15:18.216032+0000","last_undegraded":"2024-06-28T16:15:18.216032+0000","last_fullsized":"2024-06-28T16:15:18.216032+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:18.215836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:18.215836+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.215836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:01:06.524192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"260'476","reported_seq":1135,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.999763+0000","last_change":"2024-06-28T16:14:54.999846+0000","last_active":"2024-06-28T16:14:54.999763+0000","last_peered":"2024-06-28T16:14:54.999763+0000","last_clean":"2024-06-28T16:14:54.999763+0000","last_became_active":"2024-06-28T15:00:21.398262+0000","last_became_peered":"2024-06-28T15:00:21.398262+0000","last_unstale":"2024-06-28T16:14:54.999763+0000","last_undegraded":"2024-06-28T16:14:54.999763+0000","last_fullsized":"2024-06-28T16:14:54.999763+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'476","last_scrub_stamp":"2024-06-28T16:14:54.999556+0000","last_deep_scrub":"260'476","last_deep_scrub_stamp":"2024-06-28T16:14:54.999556+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.999556+0000","objects_scrubbed":6,"log_size":76,"log_dups_size":400,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:46:08.548671+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86979,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":863,"num_read_kb":2070,"num_write":734,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24966,"num_omap_keys":48,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.607050+0000","last_change":"2024-06-28T16:14:53.607103+0000","last_active":"2024-06-28T16:14:53.607050+0000","last_peered":"2024-06-28T16:14:53.607050+0000","last_clean":"2024-06-28T16:14:53.607050+0000","last_became_active":"2024-06-28T15:00:23.410519+0000","last_became_peered":"2024-06-28T15:00:23.410519+0000","last_unstale":"2024-06-28T16:14:53.607050+0000","last_undegraded":"2024-06-28T16:14:53.607050+0000","last_fullsized":"2024-06-28T16:14:53.607050+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.606849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.606849+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.606849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:00:33.451749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"260'170","reported_seq":697,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.126982+0000","last_change":"2024-06-28T16:14:50.126982+0000","last_active":"2024-06-28T16:14:50.126982+0000","last_peered":"2024-06-28T16:14:50.126982+0000","last_clean":"2024-06-28T16:14:50.126982+0000","last_became_active":"2024-06-28T15:00:25.558426+0000","last_became_peered":"2024-06-28T15:00:25.558426+0000","last_unstale":"2024-06-28T16:14:50.126982+0000","last_undegraded":"2024-06-28T16:14:50.126982+0000","last_fullsized":"2024-06-28T16:14:50.126982+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'170","last_scrub_stamp":"2024-06-28T16:14:50.126565+0000","last_deep_scrub":"260'170","last_deep_scrub_stamp":"2024-06-28T16:14:50.126565+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.126565+0000","objects_scrubbed":3,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:28:56.261088+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":845,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":215,"num_read_kb":219,"num_write":377,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"260'159","reported_seq":797,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.243927+0000","last_change":"2024-06-28T16:15:08.243996+0000","last_active":"2024-06-28T16:15:08.243927+0000","last_peered":"2024-06-28T16:15:08.243927+0000","last_clean":"2024-06-28T16:15:08.243927+0000","last_became_active":"2024-06-28T15:00:17.395643+0000","last_became_peered":"2024-06-28T15:00:17.395643+0000","last_unstale":"2024-06-28T16:15:08.243927+0000","last_undegraded":"2024-06-28T16:15:08.243927+0000","last_fullsized":"2024-06-28T16:15:08.243927+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'159","last_scrub_stamp":"2024-06-28T16:15:08.243717+0000","last_deep_scrub":"260'159","last_deep_scrub_stamp":"2024-06-28T16:15:08.243717+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.243717+0000","objects_scrubbed":1,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:49:47.952363+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":654,"num_read_kb":561,"num_write":257,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2876,"num_omap_keys":12,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"260'290","reported_seq":939,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.075407+0000","last_change":"2024-06-28T16:14:44.075499+0000","last_active":"2024-06-28T16:14:44.075407+0000","last_peered":"2024-06-28T16:14:44.075407+0000","last_clean":"2024-06-28T16:14:44.075407+0000","last_became_active":"2024-06-28T15:00:59.939485+0000","last_became_peered":"2024-06-28T15:00:59.939485+0000","last_unstale":"2024-06-28T16:14:44.075407+0000","last_undegraded":"2024-06-28T16:14:44.075407+0000","last_fullsized":"2024-06-28T16:14:44.075407+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'290","last_scrub_stamp":"2024-06-28T16:14:44.075189+0000","last_deep_scrub":"260'290","last_deep_scrub_stamp":"2024-06-28T16:14:44.075189+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.075189+0000","objects_scrubbed":3,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:23:04.302226+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":430,"num_read_kb":21907,"num_write":1034,"num_write_kb":35159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,6],"acting":[1,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.274570+0000","last_change":"2024-06-28T16:14:53.274570+0000","last_active":"2024-06-28T16:14:53.274570+0000","last_peered":"2024-06-28T16:14:53.274570+0000","last_clean":"2024-06-28T16:14:53.274570+0000","last_became_active":"2024-06-28T15:00:19.385068+0000","last_became_peered":"2024-06-28T15:00:19.385068+0000","last_unstale":"2024-06-28T16:14:53.274570+0000","last_undegraded":"2024-06-28T16:14:53.274570+0000","last_fullsized":"2024-06-28T16:14:53.274570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.274324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.274324+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.274324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:11:33.949188+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"260'356","reported_seq":971,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.835843+0000","last_change":"2024-06-28T16:15:18.835932+0000","last_active":"2024-06-28T16:15:18.835843+0000","last_peered":"2024-06-28T16:15:18.835843+0000","last_clean":"2024-06-28T16:15:18.835843+0000","last_became_active":"2024-06-28T15:00:21.400480+0000","last_became_peered":"2024-06-28T15:00:21.400480+0000","last_unstale":"2024-06-28T16:15:18.835843+0000","last_undegraded":"2024-06-28T16:15:18.835843+0000","last_fullsized":"2024-06-28T16:15:18.835843+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'356","last_scrub_stamp":"2024-06-28T16:15:18.835637+0000","last_deep_scrub":"260'356","last_deep_scrub_stamp":"2024-06-28T16:15:18.835637+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.835637+0000","objects_scrubbed":8,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:15:40.539768+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47708,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":581,"num_read_kb":1249,"num_write":480,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11460,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":507,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.168287+0000","last_change":"2024-06-28T16:15:05.168340+0000","last_active":"2024-06-28T16:15:05.168287+0000","last_peered":"2024-06-28T16:15:05.168287+0000","last_clean":"2024-06-28T16:15:05.168287+0000","last_became_active":"2024-06-28T15:00:23.417580+0000","last_became_peered":"2024-06-28T15:00:23.417580+0000","last_unstale":"2024-06-28T16:15:05.168287+0000","last_undegraded":"2024-06-28T16:15:05.168287+0000","last_fullsized":"2024-06-28T16:15:05.168287+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:05.168092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:05.168092+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.168092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:54:08.480636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"260'176","reported_seq":693,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.494904+0000","last_change":"2024-06-28T16:15:02.494904+0000","last_active":"2024-06-28T16:15:02.494904+0000","last_peered":"2024-06-28T16:15:02.494904+0000","last_clean":"2024-06-28T16:15:02.494904+0000","last_became_active":"2024-06-28T15:00:25.430299+0000","last_became_peered":"2024-06-28T15:00:25.430299+0000","last_unstale":"2024-06-28T16:15:02.494904+0000","last_undegraded":"2024-06-28T16:15:02.494904+0000","last_fullsized":"2024-06-28T16:15:02.494904+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'176","last_scrub_stamp":"2024-06-28T16:15:02.494614+0000","last_deep_scrub":"260'176","last_deep_scrub_stamp":"2024-06-28T16:15:02.494614+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.494614+0000","objects_scrubbed":6,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:35:42.752972+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2035,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":193,"num_read_kb":191,"num_write":477,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"260'315","reported_seq":1067,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.121432+0000","last_change":"2024-06-28T16:15:12.121432+0000","last_active":"2024-06-28T16:15:12.121432+0000","last_peered":"2024-06-28T16:15:12.121432+0000","last_clean":"2024-06-28T16:15:12.121432+0000","last_became_active":"2024-06-28T15:00:17.383284+0000","last_became_peered":"2024-06-28T15:00:17.383284+0000","last_unstale":"2024-06-28T16:15:12.121432+0000","last_undegraded":"2024-06-28T16:15:12.121432+0000","last_fullsized":"2024-06-28T16:15:12.121432+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'315","last_scrub_stamp":"2024-06-28T16:15:12.121164+0000","last_deep_scrub":"260'315","last_deep_scrub_stamp":"2024-06-28T16:15:12.121164+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.121164+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:19:12.129939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1515,"num_read_kb":1335,"num_write":548,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"260'347","reported_seq":1038,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.960069+0000","last_change":"2024-06-28T16:15:16.960532+0000","last_active":"2024-06-28T16:15:16.960069+0000","last_peered":"2024-06-28T16:15:16.960069+0000","last_clean":"2024-06-28T16:15:16.960069+0000","last_became_active":"2024-06-28T15:00:15.405499+0000","last_became_peered":"2024-06-28T15:00:15.405499+0000","last_unstale":"2024-06-28T16:15:16.960069+0000","last_undegraded":"2024-06-28T16:15:16.960069+0000","last_fullsized":"2024-06-28T16:15:16.960069+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'347","last_scrub_stamp":"2024-06-28T16:15:16.959776+0000","last_deep_scrub":"260'347","last_deep_scrub_stamp":"2024-06-28T16:15:16.959776+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.959776+0000","objects_scrubbed":13,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:25:44.951698+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17426850,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":585,"num_read_kb":18043,"num_write":1303,"num_write_kb":33783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":512,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.087242+0000","last_change":"2024-06-28T16:15:11.087242+0000","last_active":"2024-06-28T16:15:11.087242+0000","last_peered":"2024-06-28T16:15:11.087242+0000","last_clean":"2024-06-28T16:15:11.087242+0000","last_became_active":"2024-06-28T15:00:19.378852+0000","last_became_peered":"2024-06-28T15:00:19.378852+0000","last_unstale":"2024-06-28T16:15:11.087242+0000","last_undegraded":"2024-06-28T16:15:11.087242+0000","last_fullsized":"2024-06-28T16:15:11.087242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:11.086992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:11.086992+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.086992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:18:25.029287+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"260'285","reported_seq":895,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.193388+0000","last_change":"2024-06-28T16:15:06.193472+0000","last_active":"2024-06-28T16:15:06.193388+0000","last_peered":"2024-06-28T16:15:06.193388+0000","last_clean":"2024-06-28T16:15:06.193388+0000","last_became_active":"2024-06-28T15:00:21.480868+0000","last_became_peered":"2024-06-28T15:00:21.480868+0000","last_unstale":"2024-06-28T16:15:06.193388+0000","last_undegraded":"2024-06-28T16:15:06.193388+0000","last_fullsized":"2024-06-28T16:15:06.193388+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'285","last_scrub_stamp":"2024-06-28T16:15:06.193211+0000","last_deep_scrub":"260'285","last_deep_scrub_stamp":"2024-06-28T16:15:06.193211+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.193211+0000","objects_scrubbed":6,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:07:51.229985+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40850,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":446,"num_read_kb":956,"num_write":348,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.011327+0000","last_change":"2024-06-28T16:15:01.011381+0000","last_active":"2024-06-28T16:15:01.011327+0000","last_peered":"2024-06-28T16:15:01.011327+0000","last_clean":"2024-06-28T16:15:01.011327+0000","last_became_active":"2024-06-28T15:00:23.400417+0000","last_became_peered":"2024-06-28T15:00:23.400417+0000","last_unstale":"2024-06-28T16:15:01.011327+0000","last_undegraded":"2024-06-28T16:15:01.011327+0000","last_fullsized":"2024-06-28T16:15:01.011327+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:01.011100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:01.011100+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.011100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:29:13.769857+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"260'121","reported_seq":631,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.687682+0000","last_change":"2024-06-28T16:14:50.687682+0000","last_active":"2024-06-28T16:14:50.687682+0000","last_peered":"2024-06-28T16:14:50.687682+0000","last_clean":"2024-06-28T16:14:50.687682+0000","last_became_active":"2024-06-28T15:00:25.438176+0000","last_became_peered":"2024-06-28T15:00:25.438176+0000","last_unstale":"2024-06-28T16:14:50.687682+0000","last_undegraded":"2024-06-28T16:14:50.687682+0000","last_fullsized":"2024-06-28T16:14:50.687682+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'121","last_scrub_stamp":"2024-06-28T16:14:50.687377+0000","last_deep_scrub":"260'121","last_deep_scrub_stamp":"2024-06-28T16:14:50.687377+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.687377+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:06:30.493792+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":120,"num_read_kb":111,"num_write":277,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"260'74","reported_seq":665,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.334685+0000","last_change":"2024-06-28T16:15:02.334685+0000","last_active":"2024-06-28T16:15:02.334685+0000","last_peered":"2024-06-28T16:15:02.334685+0000","last_clean":"2024-06-28T16:15:02.334685+0000","last_became_active":"2024-06-28T15:00:17.402735+0000","last_became_peered":"2024-06-28T15:00:17.402735+0000","last_unstale":"2024-06-28T16:15:02.334685+0000","last_undegraded":"2024-06-28T16:15:02.334685+0000","last_fullsized":"2024-06-28T16:15:02.334685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'74","last_scrub_stamp":"2024-06-28T16:15:02.334423+0000","last_deep_scrub":"260'74","last_deep_scrub_stamp":"2024-06-28T16:15:02.334423+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.334423+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:29:13.249710+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":299,"num_read_kb":249,"num_write":100,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"260'336","reported_seq":1060,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.608585+0000","last_change":"2024-06-28T15:00:15.415658+0000","last_active":"2024-06-28T16:14:38.608585+0000","last_peered":"2024-06-28T16:14:38.608585+0000","last_clean":"2024-06-28T16:14:38.608585+0000","last_became_active":"2024-06-28T15:00:15.415197+0000","last_became_peered":"2024-06-28T15:00:15.415197+0000","last_unstale":"2024-06-28T16:14:38.608585+0000","last_undegraded":"2024-06-28T16:14:38.608585+0000","last_fullsized":"2024-06-28T16:14:38.608585+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3732560,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":648,"num_read_kb":6210,"num_write":1163,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":516,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.967053+0000","last_change":"2024-06-28T16:14:51.967053+0000","last_active":"2024-06-28T16:14:51.967053+0000","last_peered":"2024-06-28T16:14:51.967053+0000","last_clean":"2024-06-28T16:14:51.967053+0000","last_became_active":"2024-06-28T15:00:19.378270+0000","last_became_peered":"2024-06-28T15:00:19.378270+0000","last_unstale":"2024-06-28T16:14:51.967053+0000","last_undegraded":"2024-06-28T16:14:51.967053+0000","last_fullsized":"2024-06-28T16:14:51.967053+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.966801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.966801+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.966801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:16:26.209301+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"260'107","reported_seq":662,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.993632+0000","last_change":"2024-06-28T16:14:51.993706+0000","last_active":"2024-06-28T16:14:51.993632+0000","last_peered":"2024-06-28T16:14:51.993632+0000","last_clean":"2024-06-28T16:14:51.993632+0000","last_became_active":"2024-06-28T15:00:21.484352+0000","last_became_peered":"2024-06-28T15:00:21.484352+0000","last_unstale":"2024-06-28T16:14:51.993632+0000","last_undegraded":"2024-06-28T16:14:51.993632+0000","last_fullsized":"2024-06-28T16:14:51.993632+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'107","last_scrub_stamp":"2024-06-28T16:14:51.993438+0000","last_deep_scrub":"260'107","last_deep_scrub_stamp":"2024-06-28T16:14:51.993438+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.993438+0000","objects_scrubbed":6,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:28:06.528863+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":124,"num_read_kb":119,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1780,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"260'515","reported_seq":2254,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.146189+0000","last_change":"2024-06-28T16:14:48.146256+0000","last_active":"2024-06-28T16:14:48.146189+0000","last_peered":"2024-06-28T16:14:48.146189+0000","last_clean":"2024-06-28T16:14:48.146189+0000","last_became_active":"2024-06-28T15:00:23.407490+0000","last_became_peered":"2024-06-28T15:00:23.407490+0000","last_unstale":"2024-06-28T16:14:48.146189+0000","last_undegraded":"2024-06-28T16:14:48.146189+0000","last_fullsized":"2024-06-28T16:14:48.146189+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'515","last_scrub_stamp":"2024-06-28T16:14:48.145991+0000","last_deep_scrub":"260'515","last_deep_scrub_stamp":"2024-06-28T16:14:48.145991+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.145991+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:10:57.232491+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"260'103","reported_seq":617,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.637768+0000","last_change":"2024-06-28T16:14:55.637768+0000","last_active":"2024-06-28T16:14:55.637768+0000","last_peered":"2024-06-28T16:14:55.637768+0000","last_clean":"2024-06-28T16:14:55.637768+0000","last_became_active":"2024-06-28T15:00:25.445018+0000","last_became_peered":"2024-06-28T15:00:25.445018+0000","last_unstale":"2024-06-28T16:14:55.637768+0000","last_undegraded":"2024-06-28T16:14:55.637768+0000","last_fullsized":"2024-06-28T16:14:55.637768+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'103","last_scrub_stamp":"2024-06-28T16:14:55.637507+0000","last_deep_scrub":"260'103","last_deep_scrub_stamp":"2024-06-28T16:14:55.637507+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.637507+0000","objects_scrubbed":1,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:19:26.856020+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":117,"num_read_kb":111,"num_write":241,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"260'78","reported_seq":656,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.422444+0000","last_change":"2024-06-28T15:00:17.394823+0000","last_active":"2024-06-28T16:14:40.422444+0000","last_peered":"2024-06-28T16:14:40.422444+0000","last_clean":"2024-06-28T16:14:40.422444+0000","last_became_active":"2024-06-28T15:00:17.387086+0000","last_became_peered":"2024-06-28T15:00:17.387086+0000","last_unstale":"2024-06-28T16:14:40.422444+0000","last_undegraded":"2024-06-28T16:14:40.422444+0000","last_fullsized":"2024-06-28T16:14:40.422444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":333,"num_read_kb":275,"num_write":113,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"260'328","reported_seq":969,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.118128+0000","last_change":"2024-06-28T16:14:44.118252+0000","last_active":"2024-06-28T16:14:44.118128+0000","last_peered":"2024-06-28T16:14:44.118128+0000","last_clean":"2024-06-28T16:14:44.118128+0000","last_became_active":"2024-06-28T15:00:59.941915+0000","last_became_peered":"2024-06-28T15:00:59.941915+0000","last_unstale":"2024-06-28T16:14:44.118128+0000","last_undegraded":"2024-06-28T16:14:44.118128+0000","last_fullsized":"2024-06-28T16:14:44.118128+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'328","last_scrub_stamp":"2024-06-28T16:14:44.117934+0000","last_deep_scrub":"260'328","last_deep_scrub_stamp":"2024-06-28T16:14:44.117934+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.117934+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:54:20.388393+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5506084,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":497,"num_read_kb":57892,"num_write":1218,"num_write_kb":47404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.747922+0000","last_change":"2024-06-28T16:15:12.747922+0000","last_active":"2024-06-28T16:15:12.747922+0000","last_peered":"2024-06-28T16:15:12.747922+0000","last_clean":"2024-06-28T16:15:12.747922+0000","last_became_active":"2024-06-28T15:00:19.390613+0000","last_became_peered":"2024-06-28T15:00:19.390613+0000","last_unstale":"2024-06-28T16:15:12.747922+0000","last_undegraded":"2024-06-28T16:15:12.747922+0000","last_fullsized":"2024-06-28T16:15:12.747922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:12.747667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:12.747667+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.747667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:15:33.069119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"260'324","reported_seq":933,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.760414+0000","last_change":"2024-06-28T15:00:21.392375+0000","last_active":"2024-06-28T16:14:34.760414+0000","last_peered":"2024-06-28T16:14:34.760414+0000","last_clean":"2024-06-28T16:14:34.760414+0000","last_became_active":"2024-06-28T15:00:21.391631+0000","last_became_peered":"2024-06-28T15:00:21.391631+0000","last_unstale":"2024-06-28T16:14:34.760414+0000","last_undegraded":"2024-06-28T16:14:34.760414+0000","last_fullsized":"2024-06-28T16:14:34.760414+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43414,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":516,"num_read_kb":1105,"num_write":400,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.214355+0000","last_change":"2024-06-28T16:15:01.214355+0000","last_active":"2024-06-28T16:15:01.214355+0000","last_peered":"2024-06-28T16:15:01.214355+0000","last_clean":"2024-06-28T16:15:01.214355+0000","last_became_active":"2024-06-28T15:00:23.404312+0000","last_became_peered":"2024-06-28T15:00:23.404312+0000","last_unstale":"2024-06-28T16:15:01.214355+0000","last_undegraded":"2024-06-28T16:15:01.214355+0000","last_fullsized":"2024-06-28T16:15:01.214355+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:01.214103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:01.214103+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.214103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:21:36.177910+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"260'136","reported_seq":657,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.687390+0000","last_change":"2024-06-28T16:14:57.687390+0000","last_active":"2024-06-28T16:14:57.687390+0000","last_peered":"2024-06-28T16:14:57.687390+0000","last_clean":"2024-06-28T16:14:57.687390+0000","last_became_active":"2024-06-28T15:00:25.424826+0000","last_became_peered":"2024-06-28T15:00:25.424826+0000","last_unstale":"2024-06-28T16:14:57.687390+0000","last_undegraded":"2024-06-28T16:14:57.687390+0000","last_fullsized":"2024-06-28T16:14:57.687390+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'136","last_scrub_stamp":"2024-06-28T16:14:57.687124+0000","last_deep_scrub":"260'136","last_deep_scrub_stamp":"2024-06-28T16:14:57.687124+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.687124+0000","objects_scrubbed":5,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:05:03.591761+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":159,"num_read_kb":144,"num_write":309,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"260'118","reported_seq":701,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.933025+0000","last_change":"2024-06-28T16:15:19.933025+0000","last_active":"2024-06-28T16:15:19.933025+0000","last_peered":"2024-06-28T16:15:19.933025+0000","last_clean":"2024-06-28T16:15:19.933025+0000","last_became_active":"2024-06-28T15:00:17.386756+0000","last_became_peered":"2024-06-28T15:00:17.386756+0000","last_unstale":"2024-06-28T16:15:19.933025+0000","last_undegraded":"2024-06-28T16:15:19.933025+0000","last_fullsized":"2024-06-28T16:15:19.933025+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'118","last_scrub_stamp":"2024-06-28T16:15:19.932659+0000","last_deep_scrub":"260'118","last_deep_scrub_stamp":"2024-06-28T16:15:19.932659+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.932659+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:41:38.014558+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":469,"num_read_kb":378,"num_write":166,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"260'293","reported_seq":973,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.180637+0000","last_change":"2024-06-28T16:14:48.180832+0000","last_active":"2024-06-28T16:14:48.180637+0000","last_peered":"2024-06-28T16:14:48.180637+0000","last_clean":"2024-06-28T16:14:48.180637+0000","last_became_active":"2024-06-28T15:00:59.943670+0000","last_became_peered":"2024-06-28T15:00:59.943670+0000","last_unstale":"2024-06-28T16:14:48.180637+0000","last_undegraded":"2024-06-28T16:14:48.180637+0000","last_fullsized":"2024-06-28T16:14:48.180637+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'293","last_scrub_stamp":"2024-06-28T16:14:48.180366+0000","last_deep_scrub":"260'293","last_deep_scrub_stamp":"2024-06-28T16:14:48.180366+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.180366+0000","objects_scrubbed":3,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:11:28.293389+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306233,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":513,"num_read_kb":66858,"num_write":1091,"num_write_kb":26064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":511,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.644404+0000","last_change":"2024-06-28T16:15:04.644404+0000","last_active":"2024-06-28T16:15:04.644404+0000","last_peered":"2024-06-28T16:15:04.644404+0000","last_clean":"2024-06-28T16:15:04.644404+0000","last_became_active":"2024-06-28T15:00:19.398245+0000","last_became_peered":"2024-06-28T15:00:19.398245+0000","last_unstale":"2024-06-28T16:15:04.644404+0000","last_undegraded":"2024-06-28T16:15:04.644404+0000","last_fullsized":"2024-06-28T16:15:04.644404+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:04.644148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:04.644148+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.644148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:23:05.800918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"260'297","reported_seq":889,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.609444+0000","last_change":"2024-06-28T15:00:21.406496+0000","last_active":"2024-06-28T16:14:38.609444+0000","last_peered":"2024-06-28T16:14:38.609444+0000","last_clean":"2024-06-28T16:14:38.609444+0000","last_became_active":"2024-06-28T15:00:21.404701+0000","last_became_peered":"2024-06-28T15:00:21.404701+0000","last_unstale":"2024-06-28T16:14:38.609444+0000","last_undegraded":"2024-06-28T16:14:38.609444+0000","last_fullsized":"2024-06-28T16:14:38.609444+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44354,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":486,"num_read_kb":1147,"num_write":394,"num_write_kb":305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.234362+0000","last_change":"2024-06-28T16:14:52.234419+0000","last_active":"2024-06-28T16:14:52.234362+0000","last_peered":"2024-06-28T16:14:52.234362+0000","last_clean":"2024-06-28T16:14:52.234362+0000","last_became_active":"2024-06-28T15:00:23.415531+0000","last_became_peered":"2024-06-28T15:00:23.415531+0000","last_unstale":"2024-06-28T16:14:52.234362+0000","last_undegraded":"2024-06-28T16:14:52.234362+0000","last_fullsized":"2024-06-28T16:14:52.234362+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:52.234179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:52.234179+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.234179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:09:52.052832+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"260'168","reported_seq":722,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.154699+0000","last_change":"2024-06-28T16:14:52.154699+0000","last_active":"2024-06-28T16:14:52.154699+0000","last_peered":"2024-06-28T16:14:52.154699+0000","last_clean":"2024-06-28T16:14:52.154699+0000","last_became_active":"2024-06-28T15:00:25.553724+0000","last_became_peered":"2024-06-28T15:00:25.553724+0000","last_unstale":"2024-06-28T16:14:52.154699+0000","last_undegraded":"2024-06-28T16:14:52.154699+0000","last_fullsized":"2024-06-28T16:14:52.154699+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'168","last_scrub_stamp":"2024-06-28T16:14:52.154408+0000","last_deep_scrub":"260'168","last_deep_scrub_stamp":"2024-06-28T16:14:52.154408+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.154408+0000","objects_scrubbed":2,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:26:53.092282+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":555,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":217,"num_write":397,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"260'143","reported_seq":815,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.153520+0000","last_change":"2024-06-28T16:15:11.153578+0000","last_active":"2024-06-28T16:15:11.153520+0000","last_peered":"2024-06-28T16:15:11.153520+0000","last_clean":"2024-06-28T16:15:11.153520+0000","last_became_active":"2024-06-28T15:00:17.383224+0000","last_became_peered":"2024-06-28T15:00:17.383224+0000","last_unstale":"2024-06-28T16:15:11.153520+0000","last_undegraded":"2024-06-28T16:15:11.153520+0000","last_fullsized":"2024-06-28T16:15:11.153520+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'143","last_scrub_stamp":"2024-06-28T16:15:11.153330+0000","last_deep_scrub":"260'143","last_deep_scrub_stamp":"2024-06-28T16:15:11.153330+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.153330+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:59:15.862953+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":649,"num_read_kb":545,"num_write":214,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"260'301","reported_seq":1067,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.729020+0000","last_change":"2024-06-28T15:00:15.424150+0000","last_active":"2024-06-28T16:14:34.729020+0000","last_peered":"2024-06-28T16:14:34.729020+0000","last_clean":"2024-06-28T16:14:34.729020+0000","last_became_active":"2024-06-28T15:00:15.423813+0000","last_became_peered":"2024-06-28T15:00:15.423813+0000","last_unstale":"2024-06-28T16:14:34.729020+0000","last_undegraded":"2024-06-28T16:14:34.729020+0000","last_fullsized":"2024-06-28T16:14:34.729020+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5724260,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":754,"num_read_kb":37474,"num_write":1107,"num_write_kb":31583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.014004+0000","last_change":"2024-06-28T16:15:13.014004+0000","last_active":"2024-06-28T16:15:13.014004+0000","last_peered":"2024-06-28T16:15:13.014004+0000","last_clean":"2024-06-28T16:15:13.014004+0000","last_became_active":"2024-06-28T15:00:19.383307+0000","last_became_peered":"2024-06-28T15:00:19.383307+0000","last_unstale":"2024-06-28T16:15:13.014004+0000","last_undegraded":"2024-06-28T16:15:13.014004+0000","last_fullsized":"2024-06-28T16:15:13.014004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:13.013754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:13.013754+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.013754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:17:26.792620+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"260'286","reported_seq":858,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.177400+0000","last_change":"2024-06-28T15:00:21.396994+0000","last_active":"2024-06-28T16:14:34.177400+0000","last_peered":"2024-06-28T16:14:34.177400+0000","last_clean":"2024-06-28T16:14:34.177400+0000","last_became_active":"2024-06-28T15:00:21.396257+0000","last_became_peered":"2024-06-28T15:00:21.396257+0000","last_unstale":"2024-06-28T16:14:34.177400+0000","last_undegraded":"2024-06-28T16:14:34.177400+0000","last_fullsized":"2024-06-28T16:14:34.177400+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:20.358927+0000","last_clean_scrub_stamp":"2024-06-28T15:00:20.358927+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":200,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43992,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":439,"num_read_kb":1078,"num_write":366,"num_write_kb":297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.973587+0000","last_change":"2024-06-28T16:14:47.973645+0000","last_active":"2024-06-28T16:14:47.973587+0000","last_peered":"2024-06-28T16:14:47.973587+0000","last_clean":"2024-06-28T16:14:47.973587+0000","last_became_active":"2024-06-28T15:00:23.409107+0000","last_became_peered":"2024-06-28T15:00:23.409107+0000","last_unstale":"2024-06-28T16:14:47.973587+0000","last_undegraded":"2024-06-28T16:14:47.973587+0000","last_fullsized":"2024-06-28T16:14:47.973587+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:47.973365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:47.973365+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.973365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:58:08.598920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"260'149","reported_seq":891,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.717817+0000","last_change":"2024-06-28T16:14:58.717817+0000","last_active":"2024-06-28T16:14:58.717817+0000","last_peered":"2024-06-28T16:14:58.717817+0000","last_clean":"2024-06-28T16:14:58.717817+0000","last_became_active":"2024-06-28T15:00:25.424376+0000","last_became_peered":"2024-06-28T15:00:25.424376+0000","last_unstale":"2024-06-28T16:14:58.717817+0000","last_undegraded":"2024-06-28T16:14:58.717817+0000","last_fullsized":"2024-06-28T16:14:58.717817+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'149","last_scrub_stamp":"2024-06-28T16:14:58.717559+0000","last_deep_scrub":"260'149","last_deep_scrub_stamp":"2024-06-28T16:14:58.717559+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.717559+0000","objects_scrubbed":5,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:45:24.602655+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":398,"num_read_kb":429,"num_write":339,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1200,"num_omap_keys":6,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"260'112","reported_seq":709,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.936863+0000","last_change":"2024-06-28T16:15:01.936863+0000","last_active":"2024-06-28T16:15:01.936863+0000","last_peered":"2024-06-28T16:15:01.936863+0000","last_clean":"2024-06-28T16:15:01.936863+0000","last_became_active":"2024-06-28T15:00:17.403007+0000","last_became_peered":"2024-06-28T15:00:17.403007+0000","last_unstale":"2024-06-28T16:15:01.936863+0000","last_undegraded":"2024-06-28T16:15:01.936863+0000","last_fullsized":"2024-06-28T16:15:01.936863+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'112","last_scrub_stamp":"2024-06-28T16:15:01.936598+0000","last_deep_scrub":"260'112","last_deep_scrub_stamp":"2024-06-28T16:15:01.936598+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.936598+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:50:31.559519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":478,"num_read_kb":386,"num_write":182,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"260'379","reported_seq":1048,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:38.608581+0000","last_change":"2024-06-28T15:00:15.415976+0000","last_active":"2024-06-28T16:14:38.608581+0000","last_peered":"2024-06-28T16:14:38.608581+0000","last_clean":"2024-06-28T16:14:38.608581+0000","last_became_active":"2024-06-28T15:00:15.414969+0000","last_became_peered":"2024-06-28T15:00:15.414969+0000","last_unstale":"2024-06-28T16:14:38.608581+0000","last_undegraded":"2024-06-28T16:14:38.608581+0000","last_fullsized":"2024-06-28T16:14:38.608581+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":300,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1604505,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":539,"num_read_kb":3182,"num_write":1321,"num_write_kb":13350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"260'2","reported_seq":518,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.947057+0000","last_change":"2024-06-28T16:14:52.947126+0000","last_active":"2024-06-28T16:14:52.947057+0000","last_peered":"2024-06-28T16:14:52.947057+0000","last_clean":"2024-06-28T16:14:52.947057+0000","last_became_active":"2024-06-28T15:00:19.403344+0000","last_became_peered":"2024-06-28T15:00:19.403344+0000","last_unstale":"2024-06-28T16:14:52.947057+0000","last_undegraded":"2024-06-28T16:14:52.947057+0000","last_fullsized":"2024-06-28T16:14:52.947057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:52.946822+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:52.946822+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.946822+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:53:37.784932+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"260'297","reported_seq":906,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.943726+0000","last_change":"2024-06-28T16:14:55.943802+0000","last_active":"2024-06-28T16:14:55.943726+0000","last_peered":"2024-06-28T16:14:55.943726+0000","last_clean":"2024-06-28T16:14:55.943726+0000","last_became_active":"2024-06-28T15:00:21.387539+0000","last_became_peered":"2024-06-28T15:00:21.387539+0000","last_unstale":"2024-06-28T16:14:55.943726+0000","last_undegraded":"2024-06-28T16:14:55.943726+0000","last_fullsized":"2024-06-28T16:14:55.943726+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'297","last_scrub_stamp":"2024-06-28T16:14:55.943503+0000","last_deep_scrub":"260'297","last_deep_scrub_stamp":"2024-06-28T16:14:55.943503+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.943503+0000","objects_scrubbed":5,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:01:20.919128+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44825,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":494,"num_read_kb":1086,"num_write":388,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.070823+0000","last_change":"2024-06-28T16:14:56.070879+0000","last_active":"2024-06-28T16:14:56.070823+0000","last_peered":"2024-06-28T16:14:56.070823+0000","last_clean":"2024-06-28T16:14:56.070823+0000","last_became_active":"2024-06-28T15:00:23.416363+0000","last_became_peered":"2024-06-28T15:00:23.416363+0000","last_unstale":"2024-06-28T16:14:56.070823+0000","last_undegraded":"2024-06-28T16:14:56.070823+0000","last_fullsized":"2024-06-28T16:14:56.070823+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.070609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.070609+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.070609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:44:31.199646+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"260'129","reported_seq":635,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.940716+0000","last_change":"2024-06-28T16:14:43.940870+0000","last_active":"2024-06-28T16:14:43.940716+0000","last_peered":"2024-06-28T16:14:43.940716+0000","last_clean":"2024-06-28T16:14:43.940716+0000","last_became_active":"2024-06-28T15:00:25.428959+0000","last_became_peered":"2024-06-28T15:00:25.428959+0000","last_unstale":"2024-06-28T16:14:43.940716+0000","last_undegraded":"2024-06-28T16:14:43.940716+0000","last_fullsized":"2024-06-28T16:14:43.940716+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'129","last_scrub_stamp":"2024-06-28T16:14:43.940462+0000","last_deep_scrub":"260'129","last_deep_scrub_stamp":"2024-06-28T16:14:43.940462+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.940462+0000","objects_scrubbed":3,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:17:21.534335+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":814,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":97,"num_write":299,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"260'175","reported_seq":771,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.005191+0000","last_change":"2024-06-28T16:14:56.005265+0000","last_active":"2024-06-28T16:14:56.005191+0000","last_peered":"2024-06-28T16:14:56.005191+0000","last_clean":"2024-06-28T16:14:56.005191+0000","last_became_active":"2024-06-28T15:00:17.406872+0000","last_became_peered":"2024-06-28T15:00:17.406872+0000","last_unstale":"2024-06-28T16:14:56.005191+0000","last_undegraded":"2024-06-28T16:14:56.005191+0000","last_fullsized":"2024-06-28T16:14:56.005191+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'175","last_scrub_stamp":"2024-06-28T16:14:56.004971+0000","last_deep_scrub":"260'175","last_deep_scrub_stamp":"2024-06-28T16:14:56.004971+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.004971+0000","objects_scrubbed":1,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:44:14.471029+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":660,"num_read_kb":511,"num_write":250,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7564,"num_omap_keys":32,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"260'285","reported_seq":919,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.394078+0000","last_change":"2024-06-28T16:15:07.394078+0000","last_active":"2024-06-28T16:15:07.394078+0000","last_peered":"2024-06-28T16:15:07.394078+0000","last_clean":"2024-06-28T16:15:07.394078+0000","last_became_active":"2024-06-28T15:00:15.408337+0000","last_became_peered":"2024-06-28T15:00:15.408337+0000","last_unstale":"2024-06-28T16:15:07.394078+0000","last_undegraded":"2024-06-28T16:15:07.394078+0000","last_fullsized":"2024-06-28T16:15:07.394078+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'285","last_scrub_stamp":"2024-06-28T16:15:07.393622+0000","last_deep_scrub":"260'285","last_deep_scrub_stamp":"2024-06-28T16:15:07.393622+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.393622+0000","objects_scrubbed":5,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:14:07.466937+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":404,"num_read_kb":33775,"num_write":1000,"num_write_kb":10945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"260'72","reported_seq":683,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.969785+0000","last_change":"2024-06-28T16:14:59.969851+0000","last_active":"2024-06-28T16:14:59.969785+0000","last_peered":"2024-06-28T16:14:59.969785+0000","last_clean":"2024-06-28T16:14:59.969785+0000","last_became_active":"2024-06-28T15:00:17.403938+0000","last_became_peered":"2024-06-28T15:00:17.403938+0000","last_unstale":"2024-06-28T16:14:59.969785+0000","last_undegraded":"2024-06-28T16:14:59.969785+0000","last_fullsized":"2024-06-28T16:14:59.969785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'72","last_scrub_stamp":"2024-06-28T16:14:59.969594+0000","last_deep_scrub":"260'72","last_deep_scrub_stamp":"2024-06-28T16:14:59.969594+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.969594+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:54:37.592959+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":319,"num_read_kb":277,"num_write":96,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"260'322","reported_seq":1005,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.267065+0000","last_change":"2024-06-28T16:14:57.267065+0000","last_active":"2024-06-28T16:14:57.267065+0000","last_peered":"2024-06-28T16:14:57.267065+0000","last_clean":"2024-06-28T16:14:57.267065+0000","last_became_active":"2024-06-28T15:00:15.420261+0000","last_became_peered":"2024-06-28T15:00:15.420261+0000","last_unstale":"2024-06-28T16:14:57.267065+0000","last_undegraded":"2024-06-28T16:14:57.267065+0000","last_fullsized":"2024-06-28T16:14:57.267065+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'322","last_scrub_stamp":"2024-06-28T16:14:57.266750+0000","last_deep_scrub":"260'322","last_deep_scrub_stamp":"2024-06-28T16:14:57.266750+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.266750+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:20:49.339058+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262189,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":529,"num_read_kb":23662,"num_write":1259,"num_write_kb":28174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"260'38","reported_seq":605,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.589352+0000","last_change":"2024-06-28T16:15:00.589416+0000","last_active":"2024-06-28T16:15:00.589352+0000","last_peered":"2024-06-28T16:15:00.589352+0000","last_clean":"2024-06-28T16:15:00.589352+0000","last_became_active":"2024-06-28T15:00:17.393889+0000","last_became_peered":"2024-06-28T15:00:17.393889+0000","last_unstale":"2024-06-28T16:15:00.589352+0000","last_undegraded":"2024-06-28T16:15:00.589352+0000","last_fullsized":"2024-06-28T16:15:00.589352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'38","last_scrub_stamp":"2024-06-28T16:15:00.589144+0000","last_deep_scrub":"260'38","last_deep_scrub_stamp":"2024-06-28T16:15:00.589144+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.589144+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:04:36.901823+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":152,"num_write":58,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"260'358","reported_seq":1001,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.877308+0000","last_change":"2024-06-28T15:00:15.426603+0000","last_active":"2024-06-28T16:14:40.877308+0000","last_peered":"2024-06-28T16:14:40.877308+0000","last_clean":"2024-06-28T16:14:40.877308+0000","last_became_active":"2024-06-28T15:00:15.418773+0000","last_became_peered":"2024-06-28T15:00:15.418773+0000","last_unstale":"2024-06-28T16:14:40.877308+0000","last_undegraded":"2024-06-28T16:14:40.877308+0000","last_fullsized":"2024-06-28T16:14:40.877308+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8714448,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":553,"num_read_kb":16492,"num_write":1257,"num_write_kb":37641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"260'142","reported_seq":796,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.147070+0000","last_change":"2024-06-28T16:15:06.147193+0000","last_active":"2024-06-28T16:15:06.147070+0000","last_peered":"2024-06-28T16:15:06.147070+0000","last_clean":"2024-06-28T16:15:06.147070+0000","last_became_active":"2024-06-28T15:00:17.398357+0000","last_became_peered":"2024-06-28T15:00:17.398357+0000","last_unstale":"2024-06-28T16:15:06.147070+0000","last_undegraded":"2024-06-28T16:15:06.147070+0000","last_fullsized":"2024-06-28T16:15:06.147070+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'142","last_scrub_stamp":"2024-06-28T16:15:06.146782+0000","last_deep_scrub":"260'142","last_deep_scrub_stamp":"2024-06-28T16:15:06.146782+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.146782+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:02:03.196875+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":623,"num_read_kb":529,"num_write":221,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"260'332","reported_seq":977,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.723568+0000","last_change":"2024-06-28T15:00:15.391599+0000","last_active":"2024-06-28T16:14:34.723568+0000","last_peered":"2024-06-28T16:14:34.723568+0000","last_clean":"2024-06-28T16:14:34.723568+0000","last_became_active":"2024-06-28T15:00:15.391150+0000","last_became_peered":"2024-06-28T15:00:15.391150+0000","last_unstale":"2024-06-28T16:14:34.723568+0000","last_undegraded":"2024-06-28T16:14:34.723568+0000","last_fullsized":"2024-06-28T16:14:34.723568+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":300,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1605357,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":453,"num_read_kb":84462,"num_write":1265,"num_write_kb":41774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"260'192","reported_seq":798,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.132066+0000","last_change":"2024-06-28T16:14:42.132111+0000","last_active":"2024-06-28T16:14:42.132066+0000","last_peered":"2024-06-28T16:14:42.132066+0000","last_clean":"2024-06-28T16:14:42.132066+0000","last_became_active":"2024-06-28T15:00:59.806725+0000","last_became_peered":"2024-06-28T15:00:59.806725+0000","last_unstale":"2024-06-28T16:14:42.132066+0000","last_undegraded":"2024-06-28T16:14:42.132066+0000","last_fullsized":"2024-06-28T16:14:42.132066+0000","mapping_epoch":32,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'192","last_scrub_stamp":"2024-06-28T16:14:42.131976+0000","last_deep_scrub":"260'192","last_deep_scrub_stamp":"2024-06-28T16:14:42.131976+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.131976+0000","objects_scrubbed":1,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:22:19.905274+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":759,"num_read_kb":602,"num_write":307,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"260'334","reported_seq":999,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.696329+0000","last_change":"2024-06-28T15:00:15.383729+0000","last_active":"2024-06-28T16:14:34.696329+0000","last_peered":"2024-06-28T16:14:34.696329+0000","last_clean":"2024-06-28T16:14:34.696329+0000","last_became_active":"2024-06-28T15:00:15.383350+0000","last_became_peered":"2024-06-28T15:00:15.383350+0000","last_unstale":"2024-06-28T16:14:34.696329+0000","last_undegraded":"2024-06-28T16:14:34.696329+0000","last_fullsized":"2024-06-28T16:14:34.696329+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263219,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":502,"num_read_kb":41336,"num_write":1201,"num_write_kb":38478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0,7],"acting":[3,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"260'398","reported_seq":1008,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.938126+0000","last_change":"2024-06-28T16:14:58.938126+0000","last_active":"2024-06-28T16:14:58.938126+0000","last_peered":"2024-06-28T16:14:58.938126+0000","last_clean":"2024-06-28T16:14:58.938126+0000","last_became_active":"2024-06-28T15:00:17.404844+0000","last_became_peered":"2024-06-28T15:00:17.404844+0000","last_unstale":"2024-06-28T16:14:58.938126+0000","last_undegraded":"2024-06-28T16:14:58.938126+0000","last_fullsized":"2024-06-28T16:14:58.938126+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'398","last_scrub_stamp":"2024-06-28T16:14:58.937867+0000","last_deep_scrub":"260'398","last_deep_scrub_stamp":"2024-06-28T16:14:58.937867+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.937867+0000","objects_scrubbed":1,"log_size":98,"log_dups_size":300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:18:27.056087+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1527,"num_read_kb":1156,"num_write":658,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":268,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"260'343","reported_seq":1036,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.134211+0000","last_change":"2024-06-28T16:15:09.134294+0000","last_active":"2024-06-28T16:15:09.134211+0000","last_peered":"2024-06-28T16:15:09.134211+0000","last_clean":"2024-06-28T16:15:09.134211+0000","last_became_active":"2024-06-28T15:00:15.401275+0000","last_became_peered":"2024-06-28T15:00:15.401275+0000","last_unstale":"2024-06-28T16:15:09.134211+0000","last_undegraded":"2024-06-28T16:15:09.134211+0000","last_fullsized":"2024-06-28T16:15:09.134211+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'343","last_scrub_stamp":"2024-06-28T16:15:09.133999+0000","last_deep_scrub":"260'343","last_deep_scrub_stamp":"2024-06-28T16:15:09.133999+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.133999+0000","objects_scrubbed":3,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:37:29.220333+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":559,"num_read_kb":45669,"num_write":1313,"num_write_kb":19663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"260'221","reported_seq":851,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.135956+0000","last_change":"2024-06-28T16:15:14.135956+0000","last_active":"2024-06-28T16:15:14.135956+0000","last_peered":"2024-06-28T16:15:14.135956+0000","last_clean":"2024-06-28T16:15:14.135956+0000","last_became_active":"2024-06-28T15:00:17.382739+0000","last_became_peered":"2024-06-28T15:00:17.382739+0000","last_unstale":"2024-06-28T16:15:14.135956+0000","last_undegraded":"2024-06-28T16:15:14.135956+0000","last_fullsized":"2024-06-28T16:15:14.135956+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'221","last_scrub_stamp":"2024-06-28T16:15:14.135703+0000","last_deep_scrub":"260'221","last_deep_scrub_stamp":"2024-06-28T16:15:14.135703+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.135703+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:03:26.946056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":866,"num_read_kb":690,"num_write":335,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"260'298","reported_seq":937,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.436658+0000","last_change":"2024-06-28T16:15:10.436794+0000","last_active":"2024-06-28T16:15:10.436658+0000","last_peered":"2024-06-28T16:15:10.436658+0000","last_clean":"2024-06-28T16:15:10.436658+0000","last_became_active":"2024-06-28T15:00:15.421189+0000","last_became_peered":"2024-06-28T15:00:15.421189+0000","last_unstale":"2024-06-28T16:15:10.436658+0000","last_undegraded":"2024-06-28T16:15:10.436658+0000","last_fullsized":"2024-06-28T16:15:10.436658+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'298","last_scrub_stamp":"2024-06-28T16:15:10.436402+0000","last_deep_scrub":"260'298","last_deep_scrub_stamp":"2024-06-28T16:15:10.436402+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.436402+0000","objects_scrubbed":7,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:19:16.974922+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7105903,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":419,"num_read_kb":19450,"num_write":1148,"num_write_kb":38199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"260'80","reported_seq":650,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.448105+0000","last_change":"2024-06-28T16:15:03.448105+0000","last_active":"2024-06-28T16:15:03.448105+0000","last_peered":"2024-06-28T16:15:03.448105+0000","last_clean":"2024-06-28T16:15:03.448105+0000","last_became_active":"2024-06-28T15:00:17.396125+0000","last_became_peered":"2024-06-28T15:00:17.396125+0000","last_unstale":"2024-06-28T16:15:03.448105+0000","last_undegraded":"2024-06-28T16:15:03.448105+0000","last_fullsized":"2024-06-28T16:15:03.448105+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'80","last_scrub_stamp":"2024-06-28T16:15:03.447853+0000","last_deep_scrub":"260'80","last_deep_scrub_stamp":"2024-06-28T16:15:03.447853+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.447853+0000","objects_scrubbed":1,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:57:53.065744+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":321,"num_read_kb":257,"num_write":118,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"260'270","reported_seq":928,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.754134+0000","last_change":"2024-06-28T15:00:15.408891+0000","last_active":"2024-06-28T16:14:34.754134+0000","last_peered":"2024-06-28T16:14:34.754134+0000","last_clean":"2024-06-28T16:14:34.754134+0000","last_became_active":"2024-06-28T15:00:15.408375+0000","last_became_peered":"2024-06-28T15:00:15.408375+0000","last_unstale":"2024-06-28T16:14:34.754134+0000","last_undegraded":"2024-06-28T16:14:34.754134+0000","last_fullsized":"2024-06-28T16:14:34.754134+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":484,"num_read_kb":6587,"num_write":1005,"num_write_kb":17135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"260'62","reported_seq":646,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.240437+0000","last_change":"2024-06-28T16:15:17.240573+0000","last_active":"2024-06-28T16:15:17.240437+0000","last_peered":"2024-06-28T16:15:17.240437+0000","last_clean":"2024-06-28T16:15:17.240437+0000","last_became_active":"2024-06-28T15:00:17.379374+0000","last_became_peered":"2024-06-28T15:00:17.379374+0000","last_unstale":"2024-06-28T16:15:17.240437+0000","last_undegraded":"2024-06-28T16:15:17.240437+0000","last_fullsized":"2024-06-28T16:15:17.240437+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'62","last_scrub_stamp":"2024-06-28T16:15:17.240048+0000","last_deep_scrub":"260'62","last_deep_scrub_stamp":"2024-06-28T16:15:17.240048+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.240048+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:17:53.458911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":216,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"260'367","reported_seq":1020,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.676929+0000","last_change":"2024-06-28T16:15:11.677022+0000","last_active":"2024-06-28T16:15:11.676929+0000","last_peered":"2024-06-28T16:15:11.676929+0000","last_clean":"2024-06-28T16:15:11.676929+0000","last_became_active":"2024-06-28T15:00:15.427171+0000","last_became_peered":"2024-06-28T15:00:15.427171+0000","last_unstale":"2024-06-28T16:15:11.676929+0000","last_undegraded":"2024-06-28T16:15:11.676929+0000","last_fullsized":"2024-06-28T16:15:11.676929+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'367","last_scrub_stamp":"2024-06-28T16:15:11.676722+0000","last_deep_scrub":"260'367","last_deep_scrub_stamp":"2024-06-28T16:15:11.676722+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.676722+0000","objects_scrubbed":4,"log_size":67,"log_dups_size":300,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:35.302039+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":723921,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":536,"num_read_kb":26554,"num_write":1152,"num_write_kb":52254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"260'230","reported_seq":861,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.989876+0000","last_change":"2024-06-28T16:15:04.989943+0000","last_active":"2024-06-28T16:15:04.989876+0000","last_peered":"2024-06-28T16:15:04.989876+0000","last_clean":"2024-06-28T16:15:04.989876+0000","last_became_active":"2024-06-28T15:00:17.403760+0000","last_became_peered":"2024-06-28T15:00:17.403760+0000","last_unstale":"2024-06-28T16:15:04.989876+0000","last_undegraded":"2024-06-28T16:15:04.989876+0000","last_fullsized":"2024-06-28T16:15:04.989876+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'230","last_scrub_stamp":"2024-06-28T16:15:04.989667+0000","last_deep_scrub":"260'230","last_deep_scrub_stamp":"2024-06-28T16:15:04.989667+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.989667+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:58.817243+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":902,"num_read_kb":724,"num_write":338,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"260'282","reported_seq":930,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.744928+0000","last_change":"2024-06-28T15:00:15.376797+0000","last_active":"2024-06-28T16:14:34.744928+0000","last_peered":"2024-06-28T16:14:34.744928+0000","last_clean":"2024-06-28T16:14:34.744928+0000","last_became_active":"2024-06-28T15:00:15.376014+0000","last_became_peered":"2024-06-28T15:00:15.376014+0000","last_unstale":"2024-06-28T16:14:34.744928+0000","last_undegraded":"2024-06-28T16:14:34.744928+0000","last_fullsized":"2024-06-28T16:14:34.744928+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462789,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":409,"num_read_kb":32504,"num_write":910,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"260'70","reported_seq":668,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.378898+0000","last_change":"2024-06-28T16:15:01.378961+0000","last_active":"2024-06-28T16:15:01.378898+0000","last_peered":"2024-06-28T16:15:01.378898+0000","last_clean":"2024-06-28T16:15:01.378898+0000","last_became_active":"2024-06-28T15:00:17.401917+0000","last_became_peered":"2024-06-28T15:00:17.401917+0000","last_unstale":"2024-06-28T16:15:01.378898+0000","last_undegraded":"2024-06-28T16:15:01.378898+0000","last_fullsized":"2024-06-28T16:15:01.378898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'70","last_scrub_stamp":"2024-06-28T16:15:01.378703+0000","last_deep_scrub":"260'70","last_deep_scrub_stamp":"2024-06-28T16:15:01.378703+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.378703+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:54:33.487185+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":295,"num_read_kb":247,"num_write":98,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"260'337","reported_seq":982,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.634962+0000","last_change":"2024-06-28T16:15:09.635188+0000","last_active":"2024-06-28T16:15:09.634962+0000","last_peered":"2024-06-28T16:15:09.634962+0000","last_clean":"2024-06-28T16:15:09.634962+0000","last_became_active":"2024-06-28T15:00:15.431444+0000","last_became_peered":"2024-06-28T15:00:15.431444+0000","last_unstale":"2024-06-28T16:15:09.634962+0000","last_undegraded":"2024-06-28T16:15:09.634962+0000","last_fullsized":"2024-06-28T16:15:09.634962+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'337","last_scrub_stamp":"2024-06-28T16:15:09.634716+0000","last_deep_scrub":"260'337","last_deep_scrub_stamp":"2024-06-28T16:15:09.634716+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.634716+0000","objects_scrubbed":6,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:58:38.266578+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":461,"num_read_kb":557,"num_write":1266,"num_write_kb":33580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,3],"acting":[5,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"260'424","reported_seq":1187,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.016891+0000","last_change":"2024-06-28T16:15:06.016973+0000","last_active":"2024-06-28T16:15:06.016891+0000","last_peered":"2024-06-28T16:15:06.016891+0000","last_clean":"2024-06-28T16:15:06.016891+0000","last_became_active":"2024-06-28T15:00:17.402842+0000","last_became_peered":"2024-06-28T15:00:17.402842+0000","last_unstale":"2024-06-28T16:15:06.016891+0000","last_undegraded":"2024-06-28T16:15:06.016891+0000","last_fullsized":"2024-06-28T16:15:06.016891+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'424","last_scrub_stamp":"2024-06-28T16:15:06.016665+0000","last_deep_scrub":"260'424","last_deep_scrub_stamp":"2024-06-28T16:15:06.016665+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.016665+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":400,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:27:00.891642+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1825,"num_read_kb":1465,"num_write":755,"num_write_kb":468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"260'319","reported_seq":965,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:34.708660+0000","last_change":"2024-06-28T15:00:15.415547+0000","last_active":"2024-06-28T16:14:34.708660+0000","last_peered":"2024-06-28T16:14:34.708660+0000","last_clean":"2024-06-28T16:14:34.708660+0000","last_became_active":"2024-06-28T15:00:15.415009+0000","last_became_peered":"2024-06-28T15:00:15.415009+0000","last_unstale":"2024-06-28T16:14:34.708660+0000","last_undegraded":"2024-06-28T16:14:34.708660+0000","last_fullsized":"2024-06-28T16:14:34.708660+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:14.314850+0000","last_clean_scrub_stamp":"2024-06-28T15:00:14.314850+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242944,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":479,"num_read_kb":2084,"num_write":1202,"num_write_kb":29565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"260'26","reported_seq":583,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.742665+0000","last_change":"2024-06-28T16:15:11.742725+0000","last_active":"2024-06-28T16:15:11.742665+0000","last_peered":"2024-06-28T16:15:11.742665+0000","last_clean":"2024-06-28T16:15:11.742665+0000","last_became_active":"2024-06-28T15:00:17.389844+0000","last_became_peered":"2024-06-28T15:00:17.389844+0000","last_unstale":"2024-06-28T16:15:11.742665+0000","last_undegraded":"2024-06-28T16:15:11.742665+0000","last_fullsized":"2024-06-28T16:15:11.742665+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'26","last_scrub_stamp":"2024-06-28T16:15:11.742459+0000","last_deep_scrub":"260'26","last_deep_scrub_stamp":"2024-06-28T16:15:11.742459+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.742459+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:47:41.122636+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":112,"num_write":34,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"260'305","reported_seq":945,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.433717+0000","last_change":"2024-06-28T16:15:06.433887+0000","last_active":"2024-06-28T16:15:06.433717+0000","last_peered":"2024-06-28T16:15:06.433717+0000","last_clean":"2024-06-28T16:15:06.433717+0000","last_became_active":"2024-06-28T15:00:15.397986+0000","last_became_peered":"2024-06-28T15:00:15.397986+0000","last_unstale":"2024-06-28T16:15:06.433717+0000","last_undegraded":"2024-06-28T16:15:06.433717+0000","last_fullsized":"2024-06-28T16:15:06.433717+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'305","last_scrub_stamp":"2024-06-28T16:15:06.433513+0000","last_deep_scrub":"260'305","last_deep_scrub_stamp":"2024-06-28T16:15:06.433513+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.433513+0000","objects_scrubbed":7,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:04:10.170914+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079926,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":433,"num_read_kb":10373,"num_write":1047,"num_write_kb":25421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"260'192","reported_seq":866,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.290301+0000","last_change":"2024-06-28T16:15:09.290366+0000","last_active":"2024-06-28T16:15:09.290301+0000","last_peered":"2024-06-28T16:15:09.290301+0000","last_clean":"2024-06-28T16:15:09.290301+0000","last_became_active":"2024-06-28T15:00:17.394783+0000","last_became_peered":"2024-06-28T15:00:17.394783+0000","last_unstale":"2024-06-28T16:15:09.290301+0000","last_undegraded":"2024-06-28T16:15:09.290301+0000","last_fullsized":"2024-06-28T16:15:09.290301+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'192","last_scrub_stamp":"2024-06-28T16:15:09.290105+0000","last_deep_scrub":"260'192","last_deep_scrub_stamp":"2024-06-28T16:15:09.290105+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.290105+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:44:41.337242+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":852,"num_read_kb":715,"num_write":335,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"260'302","reported_seq":922,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.050963+0000","last_change":"2024-06-28T16:14:46.051125+0000","last_active":"2024-06-28T16:14:46.050963+0000","last_peered":"2024-06-28T16:14:46.050963+0000","last_clean":"2024-06-28T16:14:46.050963+0000","last_became_active":"2024-06-28T15:00:59.938004+0000","last_became_peered":"2024-06-28T15:00:59.938004+0000","last_unstale":"2024-06-28T16:14:46.050963+0000","last_undegraded":"2024-06-28T16:14:46.050963+0000","last_fullsized":"2024-06-28T16:14:46.050963+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'302","last_scrub_stamp":"2024-06-28T16:14:46.050757+0000","last_deep_scrub":"260'302","last_deep_scrub_stamp":"2024-06-28T16:14:46.050757+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.050757+0000","objects_scrubbed":13,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:23:03.771050+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6542288,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":437,"num_read_kb":25077,"num_write":1246,"num_write_kb":38037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,3],"acting":[1,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"260'392","reported_seq":1063,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:35.336853+0000","last_change":"2024-06-28T15:00:17.374446+0000","last_active":"2024-06-28T16:14:35.336853+0000","last_peered":"2024-06-28T16:14:35.336853+0000","last_clean":"2024-06-28T16:14:35.336853+0000","last_became_active":"2024-06-28T15:00:17.374143+0000","last_became_peered":"2024-06-28T15:00:17.374143+0000","last_unstale":"2024-06-28T16:14:35.336853+0000","last_undegraded":"2024-06-28T16:14:35.336853+0000","last_fullsized":"2024-06-28T16:14:35.336853+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:00:16.335946+0000","last_clean_scrub_stamp":"2024-06-28T15:00:16.335946+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":300,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1520,"num_read_kb":1191,"num_write":653,"num_write_kb":382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"260'255","reported_seq":928,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.309302+0000","last_change":"2024-06-28T16:14:58.309458+0000","last_active":"2024-06-28T16:14:58.309302+0000","last_peered":"2024-06-28T16:14:58.309302+0000","last_clean":"2024-06-28T16:14:58.309302+0000","last_became_active":"2024-06-28T15:00:15.430228+0000","last_became_peered":"2024-06-28T15:00:15.430228+0000","last_unstale":"2024-06-28T16:14:58.309302+0000","last_undegraded":"2024-06-28T16:14:58.309302+0000","last_fullsized":"2024-06-28T16:14:58.309302+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'255","last_scrub_stamp":"2024-06-28T16:14:58.309078+0000","last_deep_scrub":"260'255","last_deep_scrub_stamp":"2024-06-28T16:14:58.309078+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.309078+0000","objects_scrubbed":6,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:49:38.745040+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170399,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":449,"num_read_kb":6008,"num_write":895,"num_write_kb":11243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"260'134","reported_seq":787,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.792855+0000","last_change":"2024-06-28T16:15:13.793011+0000","last_active":"2024-06-28T16:15:13.792855+0000","last_peered":"2024-06-28T16:15:13.792855+0000","last_clean":"2024-06-28T16:15:13.792855+0000","last_became_active":"2024-06-28T15:00:17.381562+0000","last_became_peered":"2024-06-28T15:00:17.381562+0000","last_unstale":"2024-06-28T16:15:13.792855+0000","last_undegraded":"2024-06-28T16:15:13.792855+0000","last_fullsized":"2024-06-28T16:15:13.792855+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'134","last_scrub_stamp":"2024-06-28T16:15:13.792604+0000","last_deep_scrub":"260'134","last_deep_scrub_stamp":"2024-06-28T16:15:13.792604+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.792604+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:32:09.953292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":479,"num_write":190,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"260'496","reported_seq":1195,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.851543+0000","last_change":"2024-06-28T16:15:19.851688+0000","last_active":"2024-06-28T16:15:19.851543+0000","last_peered":"2024-06-28T16:15:19.851543+0000","last_clean":"2024-06-28T16:15:19.851543+0000","last_became_active":"2024-06-28T15:00:15.418850+0000","last_became_peered":"2024-06-28T15:00:15.418850+0000","last_unstale":"2024-06-28T16:15:19.851543+0000","last_undegraded":"2024-06-28T16:15:19.851543+0000","last_fullsized":"2024-06-28T16:15:19.851543+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'496","last_scrub_stamp":"2024-06-28T16:15:19.851317+0000","last_deep_scrub":"260'496","last_deep_scrub_stamp":"2024-06-28T16:15:19.851317+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.851317+0000","objects_scrubbed":1,"log_size":96,"log_dups_size":400,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:36:52.931988+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":907,"num_read_kb":23220,"num_write":1415,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"260'338","reported_seq":1032,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.827560+0000","last_change":"2024-06-28T16:15:16.827810+0000","last_active":"2024-06-28T16:15:16.827560+0000","last_peered":"2024-06-28T16:15:16.827560+0000","last_clean":"2024-06-28T16:15:16.827560+0000","last_became_active":"2024-06-28T15:00:15.408158+0000","last_became_peered":"2024-06-28T15:00:15.408158+0000","last_unstale":"2024-06-28T16:15:16.827560+0000","last_undegraded":"2024-06-28T16:15:16.827560+0000","last_fullsized":"2024-06-28T16:15:16.827560+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'338","last_scrub_stamp":"2024-06-28T16:15:16.827294+0000","last_deep_scrub":"260'338","last_deep_scrub_stamp":"2024-06-28T16:15:16.827294+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.827294+0000","objects_scrubbed":9,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:01:49.282108+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6690738,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":568,"num_read_kb":38558,"num_write":1259,"num_write_kb":46809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"260'77","reported_seq":654,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.305915+0000","last_change":"2024-06-28T16:15:15.306055+0000","last_active":"2024-06-28T16:15:15.305915+0000","last_peered":"2024-06-28T16:15:15.305915+0000","last_clean":"2024-06-28T16:15:15.305915+0000","last_became_active":"2024-06-28T15:00:17.379771+0000","last_became_peered":"2024-06-28T15:00:17.379771+0000","last_unstale":"2024-06-28T16:15:15.305915+0000","last_undegraded":"2024-06-28T16:15:15.305915+0000","last_fullsized":"2024-06-28T16:15:15.305915+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'77","last_scrub_stamp":"2024-06-28T16:15:15.305668+0000","last_deep_scrub":"260'77","last_deep_scrub_stamp":"2024-06-28T16:15:15.305668+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.305668+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:33:25.492967+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":249,"num_write":109,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"260'387","reported_seq":1111,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.144751+0000","last_change":"2024-06-28T16:15:10.144995+0000","last_active":"2024-06-28T16:15:10.144751+0000","last_peered":"2024-06-28T16:15:10.144751+0000","last_clean":"2024-06-28T16:15:10.144751+0000","last_became_active":"2024-06-28T15:00:15.401324+0000","last_became_peered":"2024-06-28T15:00:15.401324+0000","last_unstale":"2024-06-28T16:15:10.144751+0000","last_undegraded":"2024-06-28T16:15:10.144751+0000","last_fullsized":"2024-06-28T16:15:10.144751+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'387","last_scrub_stamp":"2024-06-28T16:15:10.144446+0000","last_deep_scrub":"260'387","last_deep_scrub_stamp":"2024-06-28T16:15:10.144446+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.144446+0000","objects_scrubbed":9,"log_size":87,"log_dups_size":300,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:10:22.329753+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079973,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":661,"num_read_kb":1946,"num_write":1429,"num_write_kb":27589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"260'176","reported_seq":787,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.646038+0000","last_change":"2024-06-28T16:15:03.646038+0000","last_active":"2024-06-28T16:15:03.646038+0000","last_peered":"2024-06-28T16:15:03.646038+0000","last_clean":"2024-06-28T16:15:03.646038+0000","last_became_active":"2024-06-28T15:00:17.382774+0000","last_became_peered":"2024-06-28T15:00:17.382774+0000","last_unstale":"2024-06-28T16:15:03.646038+0000","last_undegraded":"2024-06-28T16:15:03.646038+0000","last_fullsized":"2024-06-28T16:15:03.646038+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'176","last_scrub_stamp":"2024-06-28T16:15:03.645757+0000","last_deep_scrub":"260'176","last_deep_scrub_stamp":"2024-06-28T16:15:03.645757+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.645757+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:35:27.591692+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":719,"num_read_kb":570,"num_write":272,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5652,"num_read_kb":3987,"num_write":2073,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7430,"num_omap_keys":15,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1080,"ondisk_log_size":1080,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19748,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":13201,"num_read_kb":13000,"num_write":15927,"num_write_kb":4660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4270,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":39496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1835,"ondisk_log_size":1835,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1424454,"num_objects":198,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":18162,"num_read_kb":37754,"num_write":15149,"num_write_kb":11215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":649512,"num_omap_keys":1516,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2014,"ondisk_log_size":2014,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1644,"num_read_kb":1954,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87319,"num_read_kb":70822,"num_write":35298,"num_write_kb":17822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":43558,"num_omap_keys":182,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3547,"ondisk_log_size":3547,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":564,"num_omap_keys":6,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":149605345,"num_objects":433,"num_object_clones":0,"num_object_copies":1732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":433,"num_whiteouts":0,"num_read":34839,"num_read_kb":1786510,"num_write":77776,"num_write_kb":1982911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202506240,"data_stored":202506240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3438,"ondisk_log_size":3438,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":129,"num_read_kb":165927,"num_write":154,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":137,"num_read_kb":186344,"num_write":154,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575754,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356188,"kb_used_data":36244,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":94015652,"statfs":{"total":96636764160,"available":96272027648,"internally_reserved":0,"allocated":37113856,"data_stored":35129654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":327612874},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575754,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323068,"kb_used_data":43380,"kb_used_omap":1,"kb_used_meta":279678,"kb_avail":94048772,"statfs":{"total":96636764160,"available":96305942528,"internally_reserved":0,"allocated":44421120,"data_stored":42495981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":286390729},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575756,"num_pgs":114,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201360,"kb_used_data":35592,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94170480,"statfs":{"total":96636764160,"available":96430571520,"internally_reserved":0,"allocated":36446208,"data_stored":34448502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575756,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":146836,"kb_used_data":43468,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94225004,"statfs":{"total":96636764160,"available":96486404096,"internally_reserved":0,"allocated":44511232,"data_stored":42596897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575759,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303252,"kb_used_data":29196,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":94068588,"statfs":{"total":96636764160,"available":96326234112,"internally_reserved":0,"allocated":29896704,"data_stored":27928906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":280623559},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575757,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137284,"kb_used_data":29820,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94234556,"statfs":{"total":96636764160,"available":96496185344,"internally_reserved":0,"allocated":30535680,"data_stored":28610104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575754,"num_pgs":120,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201576,"kb_used_data":39968,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94170264,"statfs":{"total":96636764160,"available":96430350336,"internally_reserved":0,"allocated":40927232,"data_stored":38982339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575757,"num_pgs":116,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287396,"kb_used_data":39644,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":94084444,"statfs":{"total":96636764160,"available":96342470656,"internally_reserved":0,"allocated":40595456,"data_stored":38600327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":253688261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25272320,"data_stored":25272320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25198592,"data_stored":25198592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21118976,"data_stored":21118976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19386368,"data_stored":19386368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29863936,"data_stored":29863936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20664320,"data_stored":20664320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28712960,"data_stored":28712960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32288768,"data_stored":32288768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":7701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":4574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":4436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":5746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-28T16:15:23.022 INFO:tasks.ceph:pgid 68.1a last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.023 INFO:tasks.ceph:pgid 67.1d last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.023 INFO:tasks.ceph:pgid 67.1c last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.023 INFO:tasks.ceph:pgid 67.1f last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.023 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.023 INFO:tasks.ceph:pgid 68.1e last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.024 INFO:tasks.ceph:pgid 67.19 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.024 INFO:tasks.ceph:pgid 68.1f last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.024 INFO:tasks.ceph:pgid 66.19 last_scrub_stamp 2024-06-28T15:21:05.985640+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.024 INFO:tasks.ceph:pgid 68.1c last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.024 INFO:tasks.ceph:pgid 68.1d last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.025 INFO:tasks.ceph:pgid 68.12 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.025 INFO:tasks.ceph:pgid 67.14 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.025 INFO:tasks.ceph:pgid 68.10 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.025 INFO:tasks.ceph:pgid 68.11 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.025 INFO:tasks.ceph:pgid 67.16 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.026 INFO:tasks.ceph:pgid 68.16 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.026 INFO:tasks.ceph:pgid 68.17 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.027 INFO:tasks.ceph:pgid 68.14 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.027 INFO:tasks.ceph:pgid 68.15 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.028 INFO:tasks.ceph:pgid 68.b last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.028 INFO:tasks.ceph:pgid 67.c last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.028 INFO:tasks.ceph:pgid 67.e last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.029 INFO:tasks.ceph:pgid 68.e last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.029 INFO:tasks.ceph:pgid 67.9 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.029 INFO:tasks.ceph:pgid 68.f last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.030 INFO:tasks.ceph:pgid 68.d last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.030 INFO:tasks.ceph:pgid 68.2 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.030 INFO:tasks.ceph:pgid 68.3 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.031 INFO:tasks.ceph:pgid 67.4 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.031 INFO:tasks.ceph:pgid 68.0 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.031 INFO:tasks.ceph:pgid 68.1 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.032 INFO:tasks.ceph:pgid 67.6 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.032 INFO:tasks.ceph:pgid 68.7 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.032 INFO:tasks.ceph:pgid 67.0 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.033 INFO:tasks.ceph:pgid 68.4 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.033 INFO:tasks.ceph:pgid 68.5 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.033 INFO:tasks.ceph:pgid 67.2 last_scrub_stamp 2024-06-28T15:21:10.040344+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=21, tm_sec=10, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.034 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.034 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.035 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.035 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.035 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.036 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.036 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.036 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.037 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.037 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.038 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.039 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.040 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.040 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.041 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.041 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.041 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.042 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.042 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.042 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.042 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.043 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.043 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-06-28T15:00:05.258724+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=5, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.044 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.044 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.044 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.044 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.045 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-06-28T15:00:18.353169+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=18, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.045 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.045 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.046 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.046 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.046 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.046 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.047 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.047 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-06-28T15:00:20.358927+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=20, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.047 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.047 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.047 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.048 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.048 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.048 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.048 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-06-28T15:00:14.314850+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=14, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.049 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-06-28T15:00:16.335946+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=0, tm_sec=16, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:23.049 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-06-28T16:15:43.050 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-28T16:15:43.389 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:15:43.402 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-06-28T16:15:43.408 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":2520,"stamp":"2024-06-28T16:15:41.653712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":185196140,"num_objects":785,"num_object_clones":0,"num_object_copies":2436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":785,"num_whiteouts":0,"num_read":161161,"num_read_kb":2266365,"num_write":146672,"num_write_kb":2275015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":711534,"num_omap_keys":1762,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12590,"ondisk_log_size":12590,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1956960,"kb_used_data":297312,"kb_used_omap":12,"kb_used_meta":1659571,"kb_avail":753017760,"statfs":{"total":773094113280,"available":771090186240,"internally_reserved":0,"allocated":304447488,"data_stored":288792710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":1699401267},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6200,"num_omap_keys":18,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007879"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:28.856570+0000","last_change":"2024-06-28T16:15:28.856633+0000","last_active":"2024-06-28T16:15:28.856570+0000","last_peered":"2024-06-28T16:15:28.856570+0000","last_clean":"2024-06-28T16:15:28.856570+0000","last_became_active":"2024-06-28T15:37:33.328626+0000","last_became_peered":"2024-06-28T15:37:33.328626+0000","last_unstale":"2024-06-28T16:15:28.856570+0000","last_undegraded":"2024-06-28T16:15:28.856570+0000","last_fullsized":"2024-06-28T16:15:28.856570+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:28.856334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:28.856334+0000","last_clean_scrub_stamp":"2024-06-28T16:15:28.856334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:58:02.210395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"257'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:23.876306+0000","last_change":"2024-06-28T16:15:23.876367+0000","last_active":"2024-06-28T16:15:23.876306+0000","last_peered":"2024-06-28T16:15:23.876306+0000","last_clean":"2024-06-28T16:15:23.876306+0000","last_became_active":"2024-06-28T15:21:11.078250+0000","last_became_peered":"2024-06-28T15:21:11.078250+0000","last_unstale":"2024-06-28T16:15:23.876306+0000","last_undegraded":"2024-06-28T16:15:23.876306+0000","last_fullsized":"2024-06-28T16:15:23.876306+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:23.876096+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:23.876096+0000","last_clean_scrub_stamp":"2024-06-28T16:15:23.876096+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:51:03.261049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"260'4","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.100994+0000","last_change":"2024-06-28T16:14:41.101042+0000","last_active":"2024-06-28T16:14:41.100994+0000","last_peered":"2024-06-28T16:14:41.100994+0000","last_clean":"2024-06-28T16:14:41.100994+0000","last_became_active":"2024-06-28T15:21:07.046517+0000","last_became_peered":"2024-06-28T15:21:07.046517+0000","last_unstale":"2024-06-28T16:14:41.100994+0000","last_undegraded":"2024-06-28T16:14:41.100994+0000","last_fullsized":"2024-06-28T16:14:41.100994+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:14:41.100864+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:14:41.100864+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.100864+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:17:23.820622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3076,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"249'7","reported_seq":91,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.148277+0000","last_change":"2024-06-28T16:14:39.148310+0000","last_active":"2024-06-28T16:14:39.148277+0000","last_peered":"2024-06-28T16:14:39.148277+0000","last_clean":"2024-06-28T16:14:39.148277+0000","last_became_active":"2024-06-28T15:17:24.028787+0000","last_became_peered":"2024-06-28T15:17:24.028787+0000","last_unstale":"2024-06-28T16:14:39.148277+0000","last_undegraded":"2024-06-28T16:14:39.148277+0000","last_fullsized":"2024-06-28T16:14:39.148277+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'7","last_scrub_stamp":"2024-06-28T16:14:39.148161+0000","last_deep_scrub":"249'7","last_deep_scrub_stamp":"2024-06-28T16:14:39.148161+0000","last_clean_scrub_stamp":"2024-06-28T16:14:39.148161+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:43:21.535181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":30,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.297034+0000","last_change":"2024-06-28T16:15:08.297105+0000","last_active":"2024-06-28T16:15:08.297034+0000","last_peered":"2024-06-28T16:15:08.297034+0000","last_clean":"2024-06-28T16:15:08.297034+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:15:08.297034+0000","last_undegraded":"2024-06-28T16:15:08.297034+0000","last_fullsized":"2024-06-28T16:15:08.297034+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:08.296839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:08.296839+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.296839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:48:21.812858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"260'2","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:27.994963+0000","last_change":"2024-06-28T16:15:27.995033+0000","last_active":"2024-06-28T16:15:27.994963+0000","last_peered":"2024-06-28T16:15:27.994963+0000","last_clean":"2024-06-28T16:15:27.994963+0000","last_became_active":"2024-06-28T15:21:11.104387+0000","last_became_peered":"2024-06-28T15:21:11.104387+0000","last_unstale":"2024-06-28T16:15:27.994963+0000","last_undegraded":"2024-06-28T16:15:27.994963+0000","last_fullsized":"2024-06-28T16:15:27.994963+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:27.994749+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:27.994749+0000","last_clean_scrub_stamp":"2024-06-28T16:15:27.994749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:32:47.148847+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"260'2","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.043377+0000","last_change":"2024-06-28T16:14:45.043442+0000","last_active":"2024-06-28T16:14:45.043377+0000","last_peered":"2024-06-28T16:14:45.043377+0000","last_clean":"2024-06-28T16:14:45.043377+0000","last_became_active":"2024-06-28T15:21:07.049141+0000","last_became_peered":"2024-06-28T15:21:07.049141+0000","last_unstale":"2024-06-28T16:14:45.043377+0000","last_undegraded":"2024-06-28T16:14:45.043377+0000","last_fullsized":"2024-06-28T16:14:45.043377+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:45.043176+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:45.043176+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.043176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:35:34.904145+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"260'18","reported_seq":120,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.228554+0000","last_change":"2024-06-28T16:14:50.228554+0000","last_active":"2024-06-28T16:14:50.228554+0000","last_peered":"2024-06-28T16:14:50.228554+0000","last_clean":"2024-06-28T16:14:50.228554+0000","last_became_active":"2024-06-28T15:17:23.993905+0000","last_became_peered":"2024-06-28T15:17:23.993905+0000","last_unstale":"2024-06-28T16:14:50.228554+0000","last_undegraded":"2024-06-28T16:14:50.228554+0000","last_fullsized":"2024-06-28T16:14:50.228554+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'18","last_scrub_stamp":"2024-06-28T16:14:50.228287+0000","last_deep_scrub":"260'18","last_deep_scrub_stamp":"2024-06-28T16:14:50.228287+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.228287+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:42:17.443870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":64,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.398560+0000","last_change":"2024-06-28T16:15:18.398617+0000","last_active":"2024-06-28T16:15:18.398560+0000","last_peered":"2024-06-28T16:15:18.398560+0000","last_clean":"2024-06-28T16:15:18.398560+0000","last_became_active":"2024-06-28T15:37:33.345642+0000","last_became_peered":"2024-06-28T15:37:33.345642+0000","last_unstale":"2024-06-28T16:15:18.398560+0000","last_undegraded":"2024-06-28T16:15:18.398560+0000","last_fullsized":"2024-06-28T16:15:18.398560+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:18.398355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:18.398355+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.398355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:01:11.369378+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"257'5","reported_seq":67,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:34.424204+0000","last_change":"2024-06-28T16:15:34.424276+0000","last_active":"2024-06-28T16:15:34.424204+0000","last_peered":"2024-06-28T16:15:34.424204+0000","last_clean":"2024-06-28T16:15:34.424204+0000","last_became_active":"2024-06-28T15:21:11.097619+0000","last_became_peered":"2024-06-28T15:21:11.097619+0000","last_unstale":"2024-06-28T16:15:34.424204+0000","last_undegraded":"2024-06-28T16:15:34.424204+0000","last_fullsized":"2024-06-28T16:15:34.424204+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'5","last_scrub_stamp":"2024-06-28T16:15:34.423959+0000","last_deep_scrub":"257'5","last_deep_scrub_stamp":"2024-06-28T16:15:34.423959+0000","last_clean_scrub_stamp":"2024-06-28T16:15:34.423959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:36:18.836568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5438,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"0'0","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.347194+0000","last_change":"2024-06-28T16:15:04.347194+0000","last_active":"2024-06-28T16:15:04.347194+0000","last_peered":"2024-06-28T16:15:04.347194+0000","last_clean":"2024-06-28T16:15:04.347194+0000","last_became_active":"2024-06-28T15:21:07.020090+0000","last_became_peered":"2024-06-28T15:21:07.020090+0000","last_unstale":"2024-06-28T16:15:04.347194+0000","last_undegraded":"2024-06-28T16:15:04.347194+0000","last_fullsized":"2024-06-28T16:15:04.347194+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:04.346943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:04.346943+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.346943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:34:44.306374+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"260'43","reported_seq":184,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.114470+0000","last_change":"2024-06-28T16:14:45.114470+0000","last_active":"2024-06-28T16:14:45.114470+0000","last_peered":"2024-06-28T16:14:45.114470+0000","last_clean":"2024-06-28T16:14:45.114470+0000","last_became_active":"2024-06-28T15:17:24.012739+0000","last_became_peered":"2024-06-28T15:17:24.012739+0000","last_unstale":"2024-06-28T16:14:45.114470+0000","last_undegraded":"2024-06-28T16:14:45.114470+0000","last_fullsized":"2024-06-28T16:14:45.114470+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'43","last_scrub_stamp":"2024-06-28T16:14:45.114196+0000","last_deep_scrub":"260'43","last_deep_scrub_stamp":"2024-06-28T16:14:45.114196+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.114196+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:38:38.257912+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":221,"num_read_kb":154,"num_write":91,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":28,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:00.328538+0000","last_change":"2024-06-28T15:37:33.345746+0000","last_active":"2024-06-28T16:14:00.328538+0000","last_peered":"2024-06-28T16:14:00.328538+0000","last_clean":"2024-06-28T16:14:00.328538+0000","last_became_active":"2024-06-28T15:37:33.344850+0000","last_became_peered":"2024-06-28T15:37:33.344850+0000","last_unstale":"2024-06-28T16:14:00.328538+0000","last_undegraded":"2024-06-28T16:14:00.328538+0000","last_fullsized":"2024-06-28T16:14:00.328538+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"260'3","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.182157+0000","last_change":"2024-06-28T16:15:09.182157+0000","last_active":"2024-06-28T16:15:09.182157+0000","last_peered":"2024-06-28T16:15:09.182157+0000","last_clean":"2024-06-28T16:15:09.182157+0000","last_became_active":"2024-06-28T15:21:11.102572+0000","last_became_peered":"2024-06-28T15:21:11.102572+0000","last_unstale":"2024-06-28T16:15:09.182157+0000","last_undegraded":"2024-06-28T16:15:09.182157+0000","last_fullsized":"2024-06-28T16:15:09.182157+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:15:09.181904+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:15:09.181904+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.181904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:48:31.360817+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.928159+0000","last_change":"2024-06-28T16:14:48.928159+0000","last_active":"2024-06-28T16:14:48.928159+0000","last_peered":"2024-06-28T16:14:48.928159+0000","last_clean":"2024-06-28T16:14:48.928159+0000","last_became_active":"2024-06-28T15:21:07.030576+0000","last_became_peered":"2024-06-28T15:21:07.030576+0000","last_unstale":"2024-06-28T16:14:48.928159+0000","last_undegraded":"2024-06-28T16:14:48.928159+0000","last_fullsized":"2024-06-28T16:14:48.928159+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:48.927902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:48.927902+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.927902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:15:53.577907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'23","reported_seq":126,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.110220+0000","last_change":"2024-06-28T16:14:44.110220+0000","last_active":"2024-06-28T16:14:44.110220+0000","last_peered":"2024-06-28T16:14:44.110220+0000","last_clean":"2024-06-28T16:14:44.110220+0000","last_became_active":"2024-06-28T15:17:24.019120+0000","last_became_peered":"2024-06-28T15:17:24.019120+0000","last_unstale":"2024-06-28T16:14:44.110220+0000","last_undegraded":"2024-06-28T16:14:44.110220+0000","last_fullsized":"2024-06-28T16:14:44.110220+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'23","last_scrub_stamp":"2024-06-28T16:14:44.109887+0000","last_deep_scrub":"260'23","last_deep_scrub_stamp":"2024-06-28T16:14:44.109887+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.109887+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:34:23.631650+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":82,"num_write":51,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:39.219885+0000","last_change":"2024-06-28T16:15:39.219941+0000","last_active":"2024-06-28T16:15:39.219885+0000","last_peered":"2024-06-28T16:15:39.219885+0000","last_clean":"2024-06-28T16:15:39.219885+0000","last_became_active":"2024-06-28T15:37:33.335045+0000","last_became_peered":"2024-06-28T15:37:33.335045+0000","last_unstale":"2024-06-28T16:15:39.219885+0000","last_undegraded":"2024-06-28T16:15:39.219885+0000","last_fullsized":"2024-06-28T16:15:39.219885+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:39.219687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:39.219687+0000","last_clean_scrub_stamp":"2024-06-28T16:15:39.219687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:12:49.197514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"0'0","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:25.916735+0000","last_change":"2024-06-28T16:15:25.916839+0000","last_active":"2024-06-28T16:15:25.916735+0000","last_peered":"2024-06-28T16:15:25.916735+0000","last_clean":"2024-06-28T16:15:25.916735+0000","last_became_active":"2024-06-28T15:21:11.094122+0000","last_became_peered":"2024-06-28T15:21:11.094122+0000","last_unstale":"2024-06-28T16:15:25.916735+0000","last_undegraded":"2024-06-28T16:15:25.916735+0000","last_fullsized":"2024-06-28T16:15:25.916735+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:25.916390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:25.916390+0000","last_clean_scrub_stamp":"2024-06-28T16:15:25.916390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:08:57.050034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.137409+0000","last_change":"2024-06-28T16:14:42.137431+0000","last_active":"2024-06-28T16:14:42.137409+0000","last_peered":"2024-06-28T16:14:42.137409+0000","last_clean":"2024-06-28T16:14:42.137409+0000","last_became_active":"2024-06-28T15:21:07.025549+0000","last_became_peered":"2024-06-28T15:21:07.025549+0000","last_unstale":"2024-06-28T16:14:42.137409+0000","last_undegraded":"2024-06-28T16:14:42.137409+0000","last_fullsized":"2024-06-28T16:14:42.137409+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:42.137318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:42.137318+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.137318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:05:06.478598+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"260'24","reported_seq":130,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.151839+0000","last_change":"2024-06-28T16:14:46.151839+0000","last_active":"2024-06-28T16:14:46.151839+0000","last_peered":"2024-06-28T16:14:46.151839+0000","last_clean":"2024-06-28T16:14:46.151839+0000","last_became_active":"2024-06-28T15:17:24.010441+0000","last_became_peered":"2024-06-28T15:17:24.010441+0000","last_unstale":"2024-06-28T16:14:46.151839+0000","last_undegraded":"2024-06-28T16:14:46.151839+0000","last_fullsized":"2024-06-28T16:14:46.151839+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'24","last_scrub_stamp":"2024-06-28T16:14:46.151597+0000","last_deep_scrub":"260'24","last_deep_scrub_stamp":"2024-06-28T16:14:46.151597+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.151597+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:33:56.078813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":85,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:22.314706+0000","last_change":"2024-06-28T16:15:22.314844+0000","last_active":"2024-06-28T16:15:22.314706+0000","last_peered":"2024-06-28T16:15:22.314706+0000","last_clean":"2024-06-28T16:15:22.314706+0000","last_became_active":"2024-06-28T15:37:33.336623+0000","last_became_peered":"2024-06-28T15:37:33.336623+0000","last_unstale":"2024-06-28T16:15:22.314706+0000","last_undegraded":"2024-06-28T16:15:22.314706+0000","last_fullsized":"2024-06-28T16:15:22.314706+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:22.314470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:22.314470+0000","last_clean_scrub_stamp":"2024-06-28T16:15:22.314470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:42:33.660532+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"260'12","reported_seq":74,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.367282+0000","last_change":"2024-06-28T16:15:15.367353+0000","last_active":"2024-06-28T16:15:15.367282+0000","last_peered":"2024-06-28T16:15:15.367282+0000","last_clean":"2024-06-28T16:15:15.367282+0000","last_became_active":"2024-06-28T15:21:11.099434+0000","last_became_peered":"2024-06-28T15:21:11.099434+0000","last_unstale":"2024-06-28T16:15:15.367282+0000","last_undegraded":"2024-06-28T16:15:15.367282+0000","last_fullsized":"2024-06-28T16:15:15.367282+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'12","last_scrub_stamp":"2024-06-28T16:15:15.367078+0000","last_deep_scrub":"260'12","last_deep_scrub_stamp":"2024-06-28T16:15:15.367078+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.367078+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:12:37.736358+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":20482,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"260'6","reported_seq":75,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:23.962817+0000","last_change":"2024-06-28T16:15:23.962974+0000","last_active":"2024-06-28T16:15:23.962817+0000","last_peered":"2024-06-28T16:15:23.962817+0000","last_clean":"2024-06-28T16:15:23.962817+0000","last_became_active":"2024-06-28T15:21:07.043987+0000","last_became_peered":"2024-06-28T15:21:07.043987+0000","last_unstale":"2024-06-28T16:15:23.962817+0000","last_undegraded":"2024-06-28T16:15:23.962817+0000","last_fullsized":"2024-06-28T16:15:23.962817+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:15:23.962501+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:15:23.962501+0000","last_clean_scrub_stamp":"2024-06-28T16:15:23.962501+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:48:09.464644+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"260'19","reported_seq":110,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.131322+0000","last_change":"2024-06-28T16:14:47.131322+0000","last_active":"2024-06-28T16:14:47.131322+0000","last_peered":"2024-06-28T16:14:47.131322+0000","last_clean":"2024-06-28T16:14:47.131322+0000","last_became_active":"2024-06-28T15:17:24.006314+0000","last_became_peered":"2024-06-28T15:17:24.006314+0000","last_unstale":"2024-06-28T16:14:47.131322+0000","last_undegraded":"2024-06-28T16:14:47.131322+0000","last_fullsized":"2024-06-28T16:14:47.131322+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'19","last_scrub_stamp":"2024-06-28T16:14:47.131067+0000","last_deep_scrub":"260'19","last_deep_scrub_stamp":"2024-06-28T16:14:47.131067+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.131067+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:32:41.228085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":54,"num_write":49,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"260'2","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:35.132584+0000","last_change":"2024-06-28T16:15:35.132653+0000","last_active":"2024-06-28T16:15:35.132584+0000","last_peered":"2024-06-28T16:15:35.132584+0000","last_clean":"2024-06-28T16:15:35.132584+0000","last_became_active":"2024-06-28T15:37:33.347715+0000","last_became_peered":"2024-06-28T15:37:33.347715+0000","last_unstale":"2024-06-28T16:15:35.132584+0000","last_undegraded":"2024-06-28T16:15:35.132584+0000","last_fullsized":"2024-06-28T16:15:35.132584+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:35.132387+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:35.132387+0000","last_clean_scrub_stamp":"2024-06-28T16:15:35.132387+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:04:57.900521+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.386219+0000","last_change":"2024-06-28T16:15:14.386283+0000","last_active":"2024-06-28T16:15:14.386219+0000","last_peered":"2024-06-28T16:15:14.386219+0000","last_clean":"2024-06-28T16:15:14.386219+0000","last_became_active":"2024-06-28T15:21:11.088625+0000","last_became_peered":"2024-06-28T15:21:11.088625+0000","last_unstale":"2024-06-28T16:15:14.386219+0000","last_undegraded":"2024-06-28T16:15:14.386219+0000","last_fullsized":"2024-06-28T16:15:14.386219+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:14.386017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:14.386017+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.386017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:51:33.701375+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"260'6","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.976747+0000","last_change":"2024-06-28T16:14:44.976815+0000","last_active":"2024-06-28T16:14:44.976747+0000","last_peered":"2024-06-28T16:14:44.976747+0000","last_clean":"2024-06-28T16:14:44.976747+0000","last_became_active":"2024-06-28T15:21:07.030888+0000","last_became_peered":"2024-06-28T15:21:07.030888+0000","last_unstale":"2024-06-28T16:14:44.976747+0000","last_undegraded":"2024-06-28T16:14:44.976747+0000","last_fullsized":"2024-06-28T16:14:44.976747+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:14:44.976529+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:14:44.976529+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.976529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:33:30.061725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":6,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"260'72","reported_seq":257,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.133001+0000","last_change":"2024-06-28T16:14:47.133001+0000","last_active":"2024-06-28T16:14:47.133001+0000","last_peered":"2024-06-28T16:14:47.133001+0000","last_clean":"2024-06-28T16:14:47.133001+0000","last_became_active":"2024-06-28T15:17:23.993026+0000","last_became_peered":"2024-06-28T15:17:23.993026+0000","last_unstale":"2024-06-28T16:14:47.133001+0000","last_undegraded":"2024-06-28T16:14:47.133001+0000","last_fullsized":"2024-06-28T16:14:47.133001+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'72","last_scrub_stamp":"2024-06-28T16:14:47.132725+0000","last_deep_scrub":"260'72","last_deep_scrub_stamp":"2024-06-28T16:14:47.132725+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.132725+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:03.563113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":255,"num_write":131,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:37.123059+0000","last_change":"2024-06-28T16:15:37.123114+0000","last_active":"2024-06-28T16:15:37.123059+0000","last_peered":"2024-06-28T16:15:37.123059+0000","last_clean":"2024-06-28T16:15:37.123059+0000","last_became_active":"2024-06-28T15:37:33.350447+0000","last_became_peered":"2024-06-28T15:37:33.350447+0000","last_unstale":"2024-06-28T16:15:37.123059+0000","last_undegraded":"2024-06-28T16:15:37.123059+0000","last_fullsized":"2024-06-28T16:15:37.123059+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:37.122862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:37.122862+0000","last_clean_scrub_stamp":"2024-06-28T16:15:37.122862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:27:41.073800+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.251991+0000","last_change":"2024-06-28T16:15:01.252044+0000","last_active":"2024-06-28T16:15:01.251991+0000","last_peered":"2024-06-28T16:15:01.251991+0000","last_clean":"2024-06-28T16:15:01.251991+0000","last_became_active":"2024-06-28T15:21:11.100603+0000","last_became_peered":"2024-06-28T15:21:11.100603+0000","last_unstale":"2024-06-28T16:15:01.251991+0000","last_undegraded":"2024-06-28T16:15:01.251991+0000","last_fullsized":"2024-06-28T16:15:01.251991+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:01.251799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:01.251799+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.251799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:50:06.461152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"260'5","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.231661+0000","last_change":"2024-06-28T16:15:02.231723+0000","last_active":"2024-06-28T16:15:02.231661+0000","last_peered":"2024-06-28T16:15:02.231661+0000","last_clean":"2024-06-28T16:15:02.231661+0000","last_became_active":"2024-06-28T15:21:07.045902+0000","last_became_peered":"2024-06-28T15:21:07.045902+0000","last_unstale":"2024-06-28T16:15:02.231661+0000","last_undegraded":"2024-06-28T16:15:02.231661+0000","last_fullsized":"2024-06-28T16:15:02.231661+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'5","last_scrub_stamp":"2024-06-28T16:15:02.231467+0000","last_deep_scrub":"260'5","last_deep_scrub_stamp":"2024-06-28T16:15:02.231467+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.231467+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:59:29.633514+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":40,"num_write":10,"num_write_kb":1064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"260'48","reported_seq":202,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.112995+0000","last_change":"2024-06-28T16:14:43.112995+0000","last_active":"2024-06-28T16:14:43.112995+0000","last_peered":"2024-06-28T16:14:43.112995+0000","last_clean":"2024-06-28T16:14:43.112995+0000","last_became_active":"2024-06-28T15:17:24.016249+0000","last_became_peered":"2024-06-28T15:17:24.016249+0000","last_unstale":"2024-06-28T16:14:43.112995+0000","last_undegraded":"2024-06-28T16:14:43.112995+0000","last_fullsized":"2024-06-28T16:14:43.112995+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'48","last_scrub_stamp":"2024-06-28T16:14:43.112724+0000","last_deep_scrub":"260'48","last_deep_scrub_stamp":"2024-06-28T16:14:43.112724+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.112724+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:09:09.626825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":260,"num_read_kb":188,"num_write":81,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":37,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:20.290402+0000","last_change":"2024-06-28T16:15:20.290459+0000","last_active":"2024-06-28T16:15:20.290402+0000","last_peered":"2024-06-28T16:15:20.290402+0000","last_clean":"2024-06-28T16:15:20.290402+0000","last_became_active":"2024-06-28T15:37:33.316616+0000","last_became_peered":"2024-06-28T15:37:33.316616+0000","last_unstale":"2024-06-28T16:15:20.290402+0000","last_undegraded":"2024-06-28T16:15:20.290402+0000","last_fullsized":"2024-06-28T16:15:20.290402+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:20.290205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:20.290205+0000","last_clean_scrub_stamp":"2024-06-28T16:15:20.290205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:47:29.011484+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"260'8","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.133370+0000","last_change":"2024-06-28T16:14:57.133432+0000","last_active":"2024-06-28T16:14:57.133370+0000","last_peered":"2024-06-28T16:14:57.133370+0000","last_clean":"2024-06-28T16:14:57.133370+0000","last_became_active":"2024-06-28T15:21:11.094947+0000","last_became_peered":"2024-06-28T15:21:11.094947+0000","last_unstale":"2024-06-28T16:14:57.133370+0000","last_undegraded":"2024-06-28T16:14:57.133370+0000","last_fullsized":"2024-06-28T16:14:57.133370+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'8","last_scrub_stamp":"2024-06-28T16:14:57.133168+0000","last_deep_scrub":"260'8","last_deep_scrub_stamp":"2024-06-28T16:14:57.133168+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.133168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:12:28.239090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":12406,"num_write":8,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"260'5","reported_seq":70,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.646825+0000","last_change":"2024-06-28T16:14:52.646825+0000","last_active":"2024-06-28T16:14:52.646825+0000","last_peered":"2024-06-28T16:14:52.646825+0000","last_clean":"2024-06-28T16:14:52.646825+0000","last_became_active":"2024-06-28T15:21:07.025396+0000","last_became_peered":"2024-06-28T15:21:07.025396+0000","last_unstale":"2024-06-28T16:14:52.646825+0000","last_undegraded":"2024-06-28T16:14:52.646825+0000","last_fullsized":"2024-06-28T16:14:52.646825+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'5","last_scrub_stamp":"2024-06-28T16:14:52.646576+0000","last_deep_scrub":"260'5","last_deep_scrub_stamp":"2024-06-28T16:14:52.646576+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.646576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:58:00.023881+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":5,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"257'14","reported_seq":106,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.108939+0000","last_change":"2024-06-28T16:14:44.108939+0000","last_active":"2024-06-28T16:14:44.108939+0000","last_peered":"2024-06-28T16:14:44.108939+0000","last_clean":"2024-06-28T16:14:44.108939+0000","last_became_active":"2024-06-28T15:17:24.010190+0000","last_became_peered":"2024-06-28T15:17:24.010190+0000","last_unstale":"2024-06-28T16:14:44.108939+0000","last_undegraded":"2024-06-28T16:14:44.108939+0000","last_fullsized":"2024-06-28T16:14:44.108939+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'14","last_scrub_stamp":"2024-06-28T16:14:44.108596+0000","last_deep_scrub":"257'14","last_deep_scrub_stamp":"2024-06-28T16:14:44.108596+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.108596+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:06:35.967557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":51,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"260'2","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.268389+0000","last_change":"2024-06-28T16:15:09.268572+0000","last_active":"2024-06-28T16:15:09.268389+0000","last_peered":"2024-06-28T16:15:09.268389+0000","last_clean":"2024-06-28T16:15:09.268389+0000","last_became_active":"2024-06-28T15:37:33.334097+0000","last_became_peered":"2024-06-28T15:37:33.334097+0000","last_unstale":"2024-06-28T16:15:09.268389+0000","last_undegraded":"2024-06-28T16:15:09.268389+0000","last_fullsized":"2024-06-28T16:15:09.268389+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:09.268148+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:09.268148+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.268148+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:24:54.459931+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"260'9","reported_seq":72,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:36.208417+0000","last_change":"2024-06-28T16:15:36.208476+0000","last_active":"2024-06-28T16:15:36.208417+0000","last_peered":"2024-06-28T16:15:36.208417+0000","last_clean":"2024-06-28T16:15:36.208417+0000","last_became_active":"2024-06-28T15:21:11.107155+0000","last_became_peered":"2024-06-28T15:21:11.107155+0000","last_unstale":"2024-06-28T16:15:36.208417+0000","last_undegraded":"2024-06-28T16:15:36.208417+0000","last_fullsized":"2024-06-28T16:15:36.208417+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'9","last_scrub_stamp":"2024-06-28T16:15:36.208219+0000","last_deep_scrub":"260'9","last_deep_scrub_stamp":"2024-06-28T16:15:36.208219+0000","last_clean_scrub_stamp":"2024-06-28T16:15:36.208219+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:50:45.904748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21504,"num_write":14,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.557445+0000","last_change":"2024-06-28T16:14:42.557501+0000","last_active":"2024-06-28T16:14:42.557445+0000","last_peered":"2024-06-28T16:14:42.557445+0000","last_clean":"2024-06-28T16:14:42.557445+0000","last_became_active":"2024-06-28T15:21:07.042691+0000","last_became_peered":"2024-06-28T15:21:07.042691+0000","last_unstale":"2024-06-28T16:14:42.557445+0000","last_undegraded":"2024-06-28T16:14:42.557445+0000","last_fullsized":"2024-06-28T16:14:42.557445+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:42.557250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:42.557250+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.557250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:17:11.170202+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"260'74","reported_seq":300,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.069653+0000","last_change":"2024-06-28T16:14:40.069653+0000","last_active":"2024-06-28T16:14:40.069653+0000","last_peered":"2024-06-28T16:14:40.069653+0000","last_clean":"2024-06-28T16:14:40.069653+0000","last_became_active":"2024-06-28T15:17:24.027786+0000","last_became_peered":"2024-06-28T15:17:24.027786+0000","last_unstale":"2024-06-28T16:14:40.069653+0000","last_undegraded":"2024-06-28T16:14:40.069653+0000","last_fullsized":"2024-06-28T16:14:40.069653+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'74","last_scrub_stamp":"2024-06-28T16:14:40.069506+0000","last_deep_scrub":"260'74","last_deep_scrub_stamp":"2024-06-28T16:14:40.069506+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.069506+0000","objects_scrubbed":1,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:43:30.466356+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":461,"num_read_kb":332,"num_write":85,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2470,"num_omap_keys":5,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":37,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:38.089080+0000","last_change":"2024-06-28T16:15:38.089136+0000","last_active":"2024-06-28T16:15:38.089080+0000","last_peered":"2024-06-28T16:15:38.089080+0000","last_clean":"2024-06-28T16:15:38.089080+0000","last_became_active":"2024-06-28T15:37:33.354730+0000","last_became_peered":"2024-06-28T15:37:33.354730+0000","last_unstale":"2024-06-28T16:15:38.089080+0000","last_undegraded":"2024-06-28T16:15:38.089080+0000","last_fullsized":"2024-06-28T16:15:38.089080+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:38.088868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:38.088868+0000","last_clean_scrub_stamp":"2024-06-28T16:15:38.088868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:24:22.573921+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"260'4","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.183946+0000","last_change":"2024-06-28T16:14:53.184018+0000","last_active":"2024-06-28T16:14:53.183946+0000","last_peered":"2024-06-28T16:14:53.183946+0000","last_clean":"2024-06-28T16:14:53.183946+0000","last_became_active":"2024-06-28T15:21:11.093751+0000","last_became_peered":"2024-06-28T15:21:11.093751+0000","last_unstale":"2024-06-28T16:14:53.183946+0000","last_undegraded":"2024-06-28T16:14:53.183946+0000","last_fullsized":"2024-06-28T16:14:53.183946+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:14:53.183733+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:14:53.183733+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.183733+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:12:53.454345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"260'2","reported_seq":62,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.333635+0000","last_change":"2024-06-28T16:15:07.333707+0000","last_active":"2024-06-28T16:15:07.333635+0000","last_peered":"2024-06-28T16:15:07.333635+0000","last_clean":"2024-06-28T16:15:07.333635+0000","last_became_active":"2024-06-28T15:21:07.042846+0000","last_became_peered":"2024-06-28T15:21:07.042846+0000","last_unstale":"2024-06-28T16:15:07.333635+0000","last_undegraded":"2024-06-28T16:15:07.333635+0000","last_fullsized":"2024-06-28T16:15:07.333635+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:07.333374+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:07.333374+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.333374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:40:39.550588+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":59,"num_write":2,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"260'31","reported_seq":154,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.199811+0000","last_change":"2024-06-28T16:14:54.199811+0000","last_active":"2024-06-28T16:14:54.199811+0000","last_peered":"2024-06-28T16:14:54.199811+0000","last_clean":"2024-06-28T16:14:54.199811+0000","last_became_active":"2024-06-28T15:17:24.016275+0000","last_became_peered":"2024-06-28T15:17:24.016275+0000","last_unstale":"2024-06-28T16:14:54.199811+0000","last_undegraded":"2024-06-28T16:14:54.199811+0000","last_fullsized":"2024-06-28T16:14:54.199811+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'31","last_scrub_stamp":"2024-06-28T16:14:54.199529+0000","last_deep_scrub":"260'31","last_deep_scrub_stamp":"2024-06-28T16:14:54.199529+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.199529+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:16:15.219769+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":116,"num_write":60,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":37,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:27.863859+0000","last_change":"2024-06-28T16:15:27.863916+0000","last_active":"2024-06-28T16:15:27.863859+0000","last_peered":"2024-06-28T16:15:27.863859+0000","last_clean":"2024-06-28T16:15:27.863859+0000","last_became_active":"2024-06-28T15:37:33.323174+0000","last_became_peered":"2024-06-28T15:37:33.323174+0000","last_unstale":"2024-06-28T16:15:27.863859+0000","last_undegraded":"2024-06-28T16:15:27.863859+0000","last_fullsized":"2024-06-28T16:15:27.863859+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:27.863662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:27.863662+0000","last_clean_scrub_stamp":"2024-06-28T16:15:27.863662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:58:08.717137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"260'9","reported_seq":72,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:24.964479+0000","last_change":"2024-06-28T16:15:24.964620+0000","last_active":"2024-06-28T16:15:24.964479+0000","last_peered":"2024-06-28T16:15:24.964479+0000","last_clean":"2024-06-28T16:15:24.964479+0000","last_became_active":"2024-06-28T15:21:11.087119+0000","last_became_peered":"2024-06-28T15:21:11.087119+0000","last_unstale":"2024-06-28T16:15:24.964479+0000","last_undegraded":"2024-06-28T16:15:24.964479+0000","last_fullsized":"2024-06-28T16:15:24.964479+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'9","last_scrub_stamp":"2024-06-28T16:15:24.964220+0000","last_deep_scrub":"260'9","last_deep_scrub_stamp":"2024-06-28T16:15:24.964220+0000","last_clean_scrub_stamp":"2024-06-28T16:15:24.964220+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:50:36.338944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":24576,"num_write":19,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.108123+0000","last_change":"2024-06-28T16:14:50.108177+0000","last_active":"2024-06-28T16:14:50.108123+0000","last_peered":"2024-06-28T16:14:50.108123+0000","last_clean":"2024-06-28T16:14:50.108123+0000","last_became_active":"2024-06-28T15:21:07.032092+0000","last_became_peered":"2024-06-28T15:21:07.032092+0000","last_unstale":"2024-06-28T16:14:50.108123+0000","last_undegraded":"2024-06-28T16:14:50.108123+0000","last_fullsized":"2024-06-28T16:14:50.108123+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:50.107930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:50.107930+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.107930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:01:58.373114+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"260'21","reported_seq":129,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.642626+0000","last_change":"2024-06-28T16:14:54.642755+0000","last_active":"2024-06-28T16:14:54.642626+0000","last_peered":"2024-06-28T16:14:54.642626+0000","last_clean":"2024-06-28T16:14:54.642626+0000","last_became_active":"2024-06-28T15:17:24.020899+0000","last_became_peered":"2024-06-28T15:17:24.020899+0000","last_unstale":"2024-06-28T16:14:54.642626+0000","last_undegraded":"2024-06-28T16:14:54.642626+0000","last_fullsized":"2024-06-28T16:14:54.642626+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'21","last_scrub_stamp":"2024-06-28T16:14:54.642387+0000","last_deep_scrub":"260'21","last_deep_scrub_stamp":"2024-06-28T16:14:54.642387+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.642387+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:49:49.845916+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":86,"num_write":39,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:40.215684+0000","last_change":"2024-06-28T16:15:40.215745+0000","last_active":"2024-06-28T16:15:40.215684+0000","last_peered":"2024-06-28T16:15:40.215684+0000","last_clean":"2024-06-28T16:15:40.215684+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:15:40.215684+0000","last_undegraded":"2024-06-28T16:15:40.215684+0000","last_fullsized":"2024-06-28T16:15:40.215684+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:40.215483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:40.215483+0000","last_clean_scrub_stamp":"2024-06-28T16:15:40.215483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:01:56.072945+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"257'3","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.256769+0000","last_change":"2024-06-28T16:15:18.256829+0000","last_active":"2024-06-28T16:15:18.256769+0000","last_peered":"2024-06-28T16:15:18.256769+0000","last_clean":"2024-06-28T16:15:18.256769+0000","last_became_active":"2024-06-28T15:21:11.097622+0000","last_became_peered":"2024-06-28T15:21:11.097622+0000","last_unstale":"2024-06-28T16:15:18.256769+0000","last_undegraded":"2024-06-28T16:15:18.256769+0000","last_fullsized":"2024-06-28T16:15:18.256769+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:18.256544+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:18.256544+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.256544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:33:41.559876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"0'0","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.906019+0000","last_change":"2024-06-28T16:14:56.906073+0000","last_active":"2024-06-28T16:14:56.906019+0000","last_peered":"2024-06-28T16:14:56.906019+0000","last_clean":"2024-06-28T16:14:56.906019+0000","last_became_active":"2024-06-28T15:21:07.038746+0000","last_became_peered":"2024-06-28T15:21:07.038746+0000","last_unstale":"2024-06-28T16:14:56.906019+0000","last_undegraded":"2024-06-28T16:14:56.906019+0000","last_fullsized":"2024-06-28T16:14:56.906019+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.905794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.905794+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.905794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:56:22.215818+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"257'9","reported_seq":90,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.630121+0000","last_change":"2024-06-28T16:14:53.630260+0000","last_active":"2024-06-28T16:14:53.630121+0000","last_peered":"2024-06-28T16:14:53.630121+0000","last_clean":"2024-06-28T16:14:53.630121+0000","last_became_active":"2024-06-28T15:17:24.020276+0000","last_became_peered":"2024-06-28T15:17:24.020276+0000","last_unstale":"2024-06-28T16:14:53.630121+0000","last_undegraded":"2024-06-28T16:14:53.630121+0000","last_fullsized":"2024-06-28T16:14:53.630121+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'9","last_scrub_stamp":"2024-06-28T16:14:53.629897+0000","last_deep_scrub":"257'9","last_deep_scrub_stamp":"2024-06-28T16:14:53.629897+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.629897+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:40:42.297017+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:36.416611+0000","last_change":"2024-06-28T16:15:36.416731+0000","last_active":"2024-06-28T16:15:36.416611+0000","last_peered":"2024-06-28T16:15:36.416611+0000","last_clean":"2024-06-28T16:15:36.416611+0000","last_became_active":"2024-06-28T15:37:33.327497+0000","last_became_peered":"2024-06-28T15:37:33.327497+0000","last_unstale":"2024-06-28T16:15:36.416611+0000","last_undegraded":"2024-06-28T16:15:36.416611+0000","last_fullsized":"2024-06-28T16:15:36.416611+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:36.416382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:36.416382+0000","last_clean_scrub_stamp":"2024-06-28T16:15:36.416382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:40:33.651339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"260'4","reported_seq":62,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.271440+0000","last_change":"2024-06-28T16:15:16.271504+0000","last_active":"2024-06-28T16:15:16.271440+0000","last_peered":"2024-06-28T16:15:16.271440+0000","last_clean":"2024-06-28T16:15:16.271440+0000","last_became_active":"2024-06-28T15:21:11.092033+0000","last_became_peered":"2024-06-28T15:21:11.092033+0000","last_unstale":"2024-06-28T16:15:16.271440+0000","last_undegraded":"2024-06-28T16:15:16.271440+0000","last_fullsized":"2024-06-28T16:15:16.271440+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:15:16.271243+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:15:16.271243+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.271243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:12:12.186648+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6152,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.141194+0000","last_change":"2024-06-28T16:14:45.141194+0000","last_active":"2024-06-28T16:14:45.141194+0000","last_peered":"2024-06-28T16:14:45.141194+0000","last_clean":"2024-06-28T16:14:45.141194+0000","last_became_active":"2024-06-28T15:21:07.045842+0000","last_became_peered":"2024-06-28T15:21:07.045842+0000","last_unstale":"2024-06-28T16:14:45.141194+0000","last_undegraded":"2024-06-28T16:14:45.141194+0000","last_fullsized":"2024-06-28T16:14:45.141194+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:45.140747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:45.140747+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.140747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:01:27.177077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"260'17","reported_seq":115,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.153794+0000","last_change":"2024-06-28T16:14:43.153919+0000","last_active":"2024-06-28T16:14:43.153794+0000","last_peered":"2024-06-28T16:14:43.153794+0000","last_clean":"2024-06-28T16:14:43.153794+0000","last_became_active":"2024-06-28T15:17:24.013118+0000","last_became_peered":"2024-06-28T15:17:24.013118+0000","last_unstale":"2024-06-28T16:14:43.153794+0000","last_undegraded":"2024-06-28T16:14:43.153794+0000","last_fullsized":"2024-06-28T16:14:43.153794+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'17","last_scrub_stamp":"2024-06-28T16:14:43.153520+0000","last_deep_scrub":"260'17","last_deep_scrub_stamp":"2024-06-28T16:14:43.153520+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.153520+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:39:58.889835+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":63,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"260'2","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:38.496400+0000","last_change":"2024-06-28T16:15:38.496536+0000","last_active":"2024-06-28T16:15:38.496400+0000","last_peered":"2024-06-28T16:15:38.496400+0000","last_clean":"2024-06-28T16:15:38.496400+0000","last_became_active":"2024-06-28T15:37:33.339097+0000","last_became_peered":"2024-06-28T15:37:33.339097+0000","last_unstale":"2024-06-28T16:15:38.496400+0000","last_undegraded":"2024-06-28T16:15:38.496400+0000","last_fullsized":"2024-06-28T16:15:38.496400+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:38.496169+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:38.496169+0000","last_clean_scrub_stamp":"2024-06-28T16:15:38.496169+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:29:36.667770+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"257'3","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.217915+0000","last_change":"2024-06-28T16:15:05.217973+0000","last_active":"2024-06-28T16:15:05.217915+0000","last_peered":"2024-06-28T16:15:05.217915+0000","last_clean":"2024-06-28T16:15:05.217915+0000","last_became_active":"2024-06-28T15:21:11.101063+0000","last_became_peered":"2024-06-28T15:21:11.101063+0000","last_unstale":"2024-06-28T16:15:05.217915+0000","last_undegraded":"2024-06-28T16:15:05.217915+0000","last_fullsized":"2024-06-28T16:15:05.217915+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:05.217704+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:05.217704+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.217704+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:55:32.534220+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"257'3","reported_seq":65,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.407866+0000","last_change":"2024-06-28T16:15:13.407927+0000","last_active":"2024-06-28T16:15:13.407866+0000","last_peered":"2024-06-28T16:15:13.407866+0000","last_clean":"2024-06-28T16:15:13.407866+0000","last_became_active":"2024-06-28T15:21:07.040973+0000","last_became_peered":"2024-06-28T15:21:07.040973+0000","last_unstale":"2024-06-28T16:15:13.407866+0000","last_undegraded":"2024-06-28T16:15:13.407866+0000","last_fullsized":"2024-06-28T16:15:13.407866+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:13.407661+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:13.407661+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.407661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:18:56.741530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"260'22","reported_seq":124,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.889374+0000","last_change":"2024-06-28T16:14:40.889438+0000","last_active":"2024-06-28T16:14:40.889374+0000","last_peered":"2024-06-28T16:14:40.889374+0000","last_clean":"2024-06-28T16:14:40.889374+0000","last_became_active":"2024-06-28T15:17:24.028410+0000","last_became_peered":"2024-06-28T15:17:24.028410+0000","last_unstale":"2024-06-28T16:14:40.889374+0000","last_undegraded":"2024-06-28T16:14:40.889374+0000","last_fullsized":"2024-06-28T16:14:40.889374+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'22","last_scrub_stamp":"2024-06-28T16:14:40.889173+0000","last_deep_scrub":"260'22","last_deep_scrub_stamp":"2024-06-28T16:14:40.889173+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.889173+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:21.505212+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"260'1","reported_seq":29,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:13:50.326834+0000","last_change":"2024-06-28T15:37:33.341173+0000","last_active":"2024-06-28T16:13:50.326834+0000","last_peered":"2024-06-28T16:13:50.326834+0000","last_clean":"2024-06-28T16:13:50.326834+0000","last_became_active":"2024-06-28T15:37:33.340370+0000","last_became_peered":"2024-06-28T15:37:33.340370+0000","last_unstale":"2024-06-28T16:13:50.326834+0000","last_undegraded":"2024-06-28T16:13:50.326834+0000","last_fullsized":"2024-06-28T16:13:50.326834+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T15:37:32.285758+0000","last_clean_scrub_stamp":"2024-06-28T15:37:32.285758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"257'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.353284+0000","last_change":"2024-06-28T16:15:16.353345+0000","last_active":"2024-06-28T16:15:16.353284+0000","last_peered":"2024-06-28T16:15:16.353284+0000","last_clean":"2024-06-28T16:15:16.353284+0000","last_became_active":"2024-06-28T15:21:11.104038+0000","last_became_peered":"2024-06-28T15:21:11.104038+0000","last_unstale":"2024-06-28T16:15:16.353284+0000","last_undegraded":"2024-06-28T16:15:16.353284+0000","last_fullsized":"2024-06-28T16:15:16.353284+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:16.353069+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:16.353069+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.353069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:49:31.899786+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"260'9","reported_seq":82,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.657797+0000","last_change":"2024-06-28T16:14:48.657797+0000","last_active":"2024-06-28T16:14:48.657797+0000","last_peered":"2024-06-28T16:14:48.657797+0000","last_clean":"2024-06-28T16:14:48.657797+0000","last_became_active":"2024-06-28T15:21:07.008899+0000","last_became_peered":"2024-06-28T15:21:07.008899+0000","last_unstale":"2024-06-28T16:14:48.657797+0000","last_undegraded":"2024-06-28T16:14:48.657797+0000","last_fullsized":"2024-06-28T16:14:48.657797+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'9","last_scrub_stamp":"2024-06-28T16:14:48.657546+0000","last_deep_scrub":"260'9","last_deep_scrub_stamp":"2024-06-28T16:14:48.657546+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.657546+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:10:31.780674+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":7,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"260'42","reported_seq":174,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.123309+0000","last_change":"2024-06-28T16:14:43.123456+0000","last_active":"2024-06-28T16:14:43.123309+0000","last_peered":"2024-06-28T16:14:43.123309+0000","last_clean":"2024-06-28T16:14:43.123309+0000","last_became_active":"2024-06-28T15:17:24.019180+0000","last_became_peered":"2024-06-28T15:17:24.019180+0000","last_unstale":"2024-06-28T16:14:43.123309+0000","last_undegraded":"2024-06-28T16:14:43.123309+0000","last_fullsized":"2024-06-28T16:14:43.123309+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'42","last_scrub_stamp":"2024-06-28T16:14:43.123099+0000","last_deep_scrub":"260'42","last_deep_scrub_stamp":"2024-06-28T16:14:43.123099+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.123099+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:45:23.503458+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":198,"num_read_kb":139,"num_write":93,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.312243+0000","last_change":"2024-06-28T16:15:10.312306+0000","last_active":"2024-06-28T16:15:10.312243+0000","last_peered":"2024-06-28T16:15:10.312243+0000","last_clean":"2024-06-28T16:15:10.312243+0000","last_became_active":"2024-06-28T15:37:33.341716+0000","last_became_peered":"2024-06-28T15:37:33.341716+0000","last_unstale":"2024-06-28T16:15:10.312243+0000","last_undegraded":"2024-06-28T16:15:10.312243+0000","last_fullsized":"2024-06-28T16:15:10.312243+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:10.312037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:10.312037+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.312037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:41:31.673706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.080244+0000","last_change":"2024-06-28T16:14:49.080244+0000","last_active":"2024-06-28T16:14:49.080244+0000","last_peered":"2024-06-28T16:14:49.080244+0000","last_clean":"2024-06-28T16:14:49.080244+0000","last_became_active":"2024-06-28T15:21:11.093276+0000","last_became_peered":"2024-06-28T15:21:11.093276+0000","last_unstale":"2024-06-28T16:14:49.080244+0000","last_undegraded":"2024-06-28T16:14:49.080244+0000","last_fullsized":"2024-06-28T16:14:49.080244+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.079996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.079996+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.079996+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:47:03.183258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"260'9","reported_seq":76,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.651923+0000","last_change":"2024-06-28T16:14:45.651923+0000","last_active":"2024-06-28T16:14:45.651923+0000","last_peered":"2024-06-28T16:14:45.651923+0000","last_clean":"2024-06-28T16:14:45.651923+0000","last_became_active":"2024-06-28T15:21:07.040691+0000","last_became_peered":"2024-06-28T15:21:07.040691+0000","last_unstale":"2024-06-28T16:14:45.651923+0000","last_undegraded":"2024-06-28T16:14:45.651923+0000","last_fullsized":"2024-06-28T16:14:45.651923+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'9","last_scrub_stamp":"2024-06-28T16:14:45.651669+0000","last_deep_scrub":"260'9","last_deep_scrub_stamp":"2024-06-28T16:14:45.651669+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.651669+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:32:08.870332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":17408,"num_write":9,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"260'47","reported_seq":193,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.096531+0000","last_change":"2024-06-28T16:14:42.096599+0000","last_active":"2024-06-28T16:14:42.096531+0000","last_peered":"2024-06-28T16:14:42.096531+0000","last_clean":"2024-06-28T16:14:42.096531+0000","last_became_active":"2024-06-28T15:17:24.027610+0000","last_became_peered":"2024-06-28T15:17:24.027610+0000","last_unstale":"2024-06-28T16:14:42.096531+0000","last_undegraded":"2024-06-28T16:14:42.096531+0000","last_fullsized":"2024-06-28T16:14:42.096531+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'47","last_scrub_stamp":"2024-06-28T16:14:42.096391+0000","last_deep_scrub":"260'47","last_deep_scrub_stamp":"2024-06-28T16:14:42.096391+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.096391+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:11:26.034311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":231,"num_read_kb":169,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":37,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:33.093457+0000","last_change":"2024-06-28T16:15:33.093516+0000","last_active":"2024-06-28T16:15:33.093457+0000","last_peered":"2024-06-28T16:15:33.093457+0000","last_clean":"2024-06-28T16:15:33.093457+0000","last_became_active":"2024-06-28T15:37:33.346414+0000","last_became_peered":"2024-06-28T15:37:33.346414+0000","last_unstale":"2024-06-28T16:15:33.093457+0000","last_undegraded":"2024-06-28T16:15:33.093457+0000","last_fullsized":"2024-06-28T16:15:33.093457+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:33.093268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:33.093268+0000","last_clean_scrub_stamp":"2024-06-28T16:15:33.093268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:46:17.365812+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"260'6","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:26.891752+0000","last_change":"2024-06-28T16:15:26.891821+0000","last_active":"2024-06-28T16:15:26.891752+0000","last_peered":"2024-06-28T16:15:26.891752+0000","last_clean":"2024-06-28T16:15:26.891752+0000","last_became_active":"2024-06-28T15:22:00.940760+0000","last_became_peered":"2024-06-28T15:22:00.940760+0000","last_unstale":"2024-06-28T16:15:26.891752+0000","last_undegraded":"2024-06-28T16:15:26.891752+0000","last_fullsized":"2024-06-28T16:15:26.891752+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:15:26.891559+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:15:26.891559+0000","last_clean_scrub_stamp":"2024-06-28T16:15:26.891559+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:12:22.129920+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"260'3","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.369210+0000","last_change":"2024-06-28T16:15:05.369210+0000","last_active":"2024-06-28T16:15:05.369210+0000","last_peered":"2024-06-28T16:15:05.369210+0000","last_clean":"2024-06-28T16:15:05.369210+0000","last_became_active":"2024-06-28T15:21:07.030163+0000","last_became_peered":"2024-06-28T15:21:07.030163+0000","last_unstale":"2024-06-28T16:15:05.369210+0000","last_undegraded":"2024-06-28T16:15:05.369210+0000","last_fullsized":"2024-06-28T16:15:05.369210+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:15:05.368894+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:15:05.368894+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.368894+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:40:02.616948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"260'40","reported_seq":180,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.687315+0000","last_change":"2024-06-28T16:14:44.687376+0000","last_active":"2024-06-28T16:14:44.687315+0000","last_peered":"2024-06-28T16:14:44.687315+0000","last_clean":"2024-06-28T16:14:44.687315+0000","last_became_active":"2024-06-28T15:17:24.012572+0000","last_became_peered":"2024-06-28T15:17:24.012572+0000","last_unstale":"2024-06-28T16:14:44.687315+0000","last_undegraded":"2024-06-28T16:14:44.687315+0000","last_fullsized":"2024-06-28T16:14:44.687315+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'40","last_scrub_stamp":"2024-06-28T16:14:44.687125+0000","last_deep_scrub":"260'40","last_deep_scrub_stamp":"2024-06-28T16:14:44.687125+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.687125+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:52:09.797828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":148,"num_write":94,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.397140+0000","last_change":"2024-06-28T16:15:19.397200+0000","last_active":"2024-06-28T16:15:19.397140+0000","last_peered":"2024-06-28T16:15:19.397140+0000","last_clean":"2024-06-28T16:15:19.397140+0000","last_became_active":"2024-06-28T15:37:33.352959+0000","last_became_peered":"2024-06-28T15:37:33.352959+0000","last_unstale":"2024-06-28T16:15:19.397140+0000","last_undegraded":"2024-06-28T16:15:19.397140+0000","last_fullsized":"2024-06-28T16:15:19.397140+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:19.396949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:19.396949+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.396949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:10:10.507079+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.046127+0000","last_change":"2024-06-28T16:14:49.046127+0000","last_active":"2024-06-28T16:14:49.046127+0000","last_peered":"2024-06-28T16:14:49.046127+0000","last_clean":"2024-06-28T16:14:49.046127+0000","last_became_active":"2024-06-28T15:21:11.079036+0000","last_became_peered":"2024-06-28T15:21:11.079036+0000","last_unstale":"2024-06-28T16:14:49.046127+0000","last_undegraded":"2024-06-28T16:14:49.046127+0000","last_fullsized":"2024-06-28T16:14:49.046127+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.045873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.045873+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.045873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:29:37.775532+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"0'0","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.128341+0000","last_change":"2024-06-28T16:14:46.128341+0000","last_active":"2024-06-28T16:14:46.128341+0000","last_peered":"2024-06-28T16:14:46.128341+0000","last_clean":"2024-06-28T16:14:46.128341+0000","last_became_active":"2024-06-28T15:21:07.027451+0000","last_became_peered":"2024-06-28T15:21:07.027451+0000","last_unstale":"2024-06-28T16:14:46.128341+0000","last_undegraded":"2024-06-28T16:14:46.128341+0000","last_fullsized":"2024-06-28T16:14:46.128341+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:46.128071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:46.128071+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.128071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:30:28.326677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"260'16","reported_seq":106,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.081079+0000","last_change":"2024-06-28T16:14:43.081145+0000","last_active":"2024-06-28T16:14:43.081079+0000","last_peered":"2024-06-28T16:14:43.081079+0000","last_clean":"2024-06-28T16:14:43.081079+0000","last_became_active":"2024-06-28T15:17:24.026271+0000","last_became_peered":"2024-06-28T15:17:24.026271+0000","last_unstale":"2024-06-28T16:14:43.081079+0000","last_undegraded":"2024-06-28T16:14:43.081079+0000","last_fullsized":"2024-06-28T16:14:43.081079+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'16","last_scrub_stamp":"2024-06-28T16:14:43.080857+0000","last_deep_scrub":"260'16","last_deep_scrub_stamp":"2024-06-28T16:14:43.080857+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.080857+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:01:28.864357+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":48,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.748084+0000","last_change":"2024-06-28T16:15:00.748137+0000","last_active":"2024-06-28T16:15:00.748084+0000","last_peered":"2024-06-28T16:15:00.748084+0000","last_clean":"2024-06-28T16:15:00.748084+0000","last_became_active":"2024-06-28T15:37:33.325849+0000","last_became_peered":"2024-06-28T15:37:33.325849+0000","last_unstale":"2024-06-28T16:15:00.748084+0000","last_undegraded":"2024-06-28T16:15:00.748084+0000","last_fullsized":"2024-06-28T16:15:00.748084+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:00.747890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:00.747890+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.747890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:59:35.430960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"260'6","reported_seq":67,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:37.231881+0000","last_change":"2024-06-28T16:15:37.232010+0000","last_active":"2024-06-28T16:15:37.231881+0000","last_peered":"2024-06-28T16:15:37.231881+0000","last_clean":"2024-06-28T16:15:37.231881+0000","last_became_active":"2024-06-28T15:21:11.107498+0000","last_became_peered":"2024-06-28T15:21:11.107498+0000","last_unstale":"2024-06-28T16:15:37.231881+0000","last_undegraded":"2024-06-28T16:15:37.231881+0000","last_fullsized":"2024-06-28T16:15:37.231881+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:15:37.231595+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:15:37.231595+0000","last_clean_scrub_stamp":"2024-06-28T16:15:37.231595+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:33:36.860544+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"260'8","reported_seq":77,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.130933+0000","last_change":"2024-06-28T16:14:41.130933+0000","last_active":"2024-06-28T16:14:41.130933+0000","last_peered":"2024-06-28T16:14:41.130933+0000","last_clean":"2024-06-28T16:14:41.130933+0000","last_became_active":"2024-06-28T15:21:07.029525+0000","last_became_peered":"2024-06-28T15:21:07.029525+0000","last_unstale":"2024-06-28T16:14:41.130933+0000","last_undegraded":"2024-06-28T16:14:41.130933+0000","last_fullsized":"2024-06-28T16:14:41.130933+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'8","last_scrub_stamp":"2024-06-28T16:14:41.130824+0000","last_deep_scrub":"260'8","last_deep_scrub_stamp":"2024-06-28T16:14:41.130824+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.130824+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:31:52.999376+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":11,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"260'34","reported_seq":160,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.199739+0000","last_change":"2024-06-28T16:14:52.199853+0000","last_active":"2024-06-28T16:14:52.199739+0000","last_peered":"2024-06-28T16:14:52.199739+0000","last_clean":"2024-06-28T16:14:52.199739+0000","last_became_active":"2024-06-28T15:17:24.007649+0000","last_became_peered":"2024-06-28T15:17:24.007649+0000","last_unstale":"2024-06-28T16:14:52.199739+0000","last_undegraded":"2024-06-28T16:14:52.199739+0000","last_fullsized":"2024-06-28T16:14:52.199739+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'34","last_scrub_stamp":"2024-06-28T16:14:52.199460+0000","last_deep_scrub":"260'34","last_deep_scrub_stamp":"2024-06-28T16:14:52.199460+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.199460+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:55:51.649887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":172,"num_read_kb":122,"num_write":76,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":37,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:38.213042+0000","last_change":"2024-06-28T16:15:38.213118+0000","last_active":"2024-06-28T16:15:38.213042+0000","last_peered":"2024-06-28T16:15:38.213042+0000","last_clean":"2024-06-28T16:15:38.213042+0000","last_became_active":"2024-06-28T15:37:33.334082+0000","last_became_peered":"2024-06-28T15:37:33.334082+0000","last_unstale":"2024-06-28T16:15:38.213042+0000","last_undegraded":"2024-06-28T16:15:38.213042+0000","last_fullsized":"2024-06-28T16:15:38.213042+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:38.212835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:38.212835+0000","last_clean_scrub_stamp":"2024-06-28T16:15:38.212835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:34:21.976990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"0'0","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:28.967650+0000","last_change":"2024-06-28T16:15:28.967711+0000","last_active":"2024-06-28T16:15:28.967650+0000","last_peered":"2024-06-28T16:15:28.967650+0000","last_clean":"2024-06-28T16:15:28.967650+0000","last_became_active":"2024-06-28T15:21:11.107686+0000","last_became_peered":"2024-06-28T15:21:11.107686+0000","last_unstale":"2024-06-28T16:15:28.967650+0000","last_undegraded":"2024-06-28T16:15:28.967650+0000","last_fullsized":"2024-06-28T16:15:28.967650+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:28.967453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:28.967453+0000","last_clean_scrub_stamp":"2024-06-28T16:15:28.967453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:12:31.455209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"0'0","reported_seq":62,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.052046+0000","last_change":"2024-06-28T16:14:54.052046+0000","last_active":"2024-06-28T16:14:54.052046+0000","last_peered":"2024-06-28T16:14:54.052046+0000","last_clean":"2024-06-28T16:14:54.052046+0000","last_became_active":"2024-06-28T15:21:07.032719+0000","last_became_peered":"2024-06-28T15:21:07.032719+0000","last_unstale":"2024-06-28T16:14:54.052046+0000","last_undegraded":"2024-06-28T16:14:54.052046+0000","last_fullsized":"2024-06-28T16:14:54.052046+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:54.051795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:54.051795+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.051795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:51:48.898614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"260'45","reported_seq":191,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.159340+0000","last_change":"2024-06-28T16:14:40.159396+0000","last_active":"2024-06-28T16:14:40.159340+0000","last_peered":"2024-06-28T16:14:40.159340+0000","last_clean":"2024-06-28T16:14:40.159340+0000","last_became_active":"2024-06-28T15:17:24.029532+0000","last_became_peered":"2024-06-28T15:17:24.029532+0000","last_unstale":"2024-06-28T16:14:40.159340+0000","last_undegraded":"2024-06-28T16:14:40.159340+0000","last_fullsized":"2024-06-28T16:14:40.159340+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'45","last_scrub_stamp":"2024-06-28T16:14:40.159221+0000","last_deep_scrub":"260'45","last_deep_scrub_stamp":"2024-06-28T16:14:40.159221+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.159221+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:08:46.885820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":234,"num_read_kb":165,"num_write":86,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":37,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:36.125307+0000","last_change":"2024-06-28T16:15:36.125373+0000","last_active":"2024-06-28T16:15:36.125307+0000","last_peered":"2024-06-28T16:15:36.125307+0000","last_clean":"2024-06-28T16:15:36.125307+0000","last_became_active":"2024-06-28T15:37:33.347691+0000","last_became_peered":"2024-06-28T15:37:33.347691+0000","last_unstale":"2024-06-28T16:15:36.125307+0000","last_undegraded":"2024-06-28T16:15:36.125307+0000","last_fullsized":"2024-06-28T16:15:36.125307+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:36.125085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:36.125085+0000","last_clean_scrub_stamp":"2024-06-28T16:15:36.125085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:53:25.842476+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"0'0","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.580849+0000","last_change":"2024-06-28T16:14:48.580849+0000","last_active":"2024-06-28T16:14:48.580849+0000","last_peered":"2024-06-28T16:14:48.580849+0000","last_clean":"2024-06-28T16:14:48.580849+0000","last_became_active":"2024-06-28T15:21:11.091656+0000","last_became_peered":"2024-06-28T15:21:11.091656+0000","last_unstale":"2024-06-28T16:14:48.580849+0000","last_undegraded":"2024-06-28T16:14:48.580849+0000","last_fullsized":"2024-06-28T16:14:48.580849+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:48.580598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:48.580598+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.580598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:07:09.489282+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"260'5","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.099851+0000","last_change":"2024-06-28T16:14:39.099851+0000","last_active":"2024-06-28T16:14:39.099851+0000","last_peered":"2024-06-28T16:14:39.099851+0000","last_clean":"2024-06-28T16:14:39.099851+0000","last_became_active":"2024-06-28T15:21:07.029495+0000","last_became_peered":"2024-06-28T15:21:07.029495+0000","last_unstale":"2024-06-28T16:14:39.099851+0000","last_undegraded":"2024-06-28T16:14:39.099851+0000","last_fullsized":"2024-06-28T16:14:39.099851+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'5","last_scrub_stamp":"2024-06-28T16:14:39.099694+0000","last_deep_scrub":"260'5","last_deep_scrub_stamp":"2024-06-28T16:14:39.099694+0000","last_clean_scrub_stamp":"2024-06-28T16:14:39.099694+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:24:20.828667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":318,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"260'39","reported_seq":171,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.649465+0000","last_change":"2024-06-28T16:14:50.649616+0000","last_active":"2024-06-28T16:14:50.649465+0000","last_peered":"2024-06-28T16:14:50.649465+0000","last_clean":"2024-06-28T16:14:50.649465+0000","last_became_active":"2024-06-28T15:17:24.021200+0000","last_became_peered":"2024-06-28T15:17:24.021200+0000","last_unstale":"2024-06-28T16:14:50.649465+0000","last_undegraded":"2024-06-28T16:14:50.649465+0000","last_fullsized":"2024-06-28T16:14:50.649465+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'39","last_scrub_stamp":"2024-06-28T16:14:50.649227+0000","last_deep_scrub":"260'39","last_deep_scrub_stamp":"2024-06-28T16:14:50.649227+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.649227+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:23:42.138261+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":191,"num_read_kb":133,"num_write":87,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"260'2","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.759533+0000","last_change":"2024-06-28T16:15:04.759599+0000","last_active":"2024-06-28T16:15:04.759533+0000","last_peered":"2024-06-28T16:15:04.759533+0000","last_clean":"2024-06-28T16:15:04.759533+0000","last_became_active":"2024-06-28T15:37:33.335464+0000","last_became_peered":"2024-06-28T15:37:33.335464+0000","last_unstale":"2024-06-28T16:15:04.759533+0000","last_undegraded":"2024-06-28T16:15:04.759533+0000","last_fullsized":"2024-06-28T16:15:04.759533+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:04.759350+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:04.759350+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.759350+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:56:34.598481+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"257'3","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.270056+0000","last_change":"2024-06-28T16:15:17.270119+0000","last_active":"2024-06-28T16:15:17.270056+0000","last_peered":"2024-06-28T16:15:17.270056+0000","last_clean":"2024-06-28T16:15:17.270056+0000","last_became_active":"2024-06-28T15:21:11.091647+0000","last_became_peered":"2024-06-28T15:21:11.091647+0000","last_unstale":"2024-06-28T16:15:17.270056+0000","last_undegraded":"2024-06-28T16:15:17.270056+0000","last_fullsized":"2024-06-28T16:15:17.270056+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:17.269853+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:17.269853+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.269853+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:16:54.117703+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"257'3","reported_seq":67,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.098585+0000","last_change":"2024-06-28T16:14:58.098585+0000","last_active":"2024-06-28T16:14:58.098585+0000","last_peered":"2024-06-28T16:14:58.098585+0000","last_clean":"2024-06-28T16:14:58.098585+0000","last_became_active":"2024-06-28T15:21:07.046218+0000","last_became_peered":"2024-06-28T15:21:07.046218+0000","last_unstale":"2024-06-28T16:14:58.098585+0000","last_undegraded":"2024-06-28T16:14:58.098585+0000","last_fullsized":"2024-06-28T16:14:58.098585+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:14:58.098319+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:14:58.098319+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.098319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:38:23.012737+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"260'54","reported_seq":217,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.106471+0000","last_change":"2024-06-28T16:14:47.106537+0000","last_active":"2024-06-28T16:14:47.106471+0000","last_peered":"2024-06-28T16:14:47.106471+0000","last_clean":"2024-06-28T16:14:47.106471+0000","last_became_active":"2024-06-28T15:17:24.010230+0000","last_became_peered":"2024-06-28T15:17:24.010230+0000","last_unstale":"2024-06-28T16:14:47.106471+0000","last_undegraded":"2024-06-28T16:14:47.106471+0000","last_fullsized":"2024-06-28T16:14:47.106471+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'54","last_scrub_stamp":"2024-06-28T16:14:47.106259+0000","last_deep_scrub":"260'54","last_deep_scrub_stamp":"2024-06-28T16:14:47.106259+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.106259+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:33:32.525834+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":283,"num_read_kb":205,"num_write":97,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":37,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:39.504976+0000","last_change":"2024-06-28T16:15:39.505030+0000","last_active":"2024-06-28T16:15:39.504976+0000","last_peered":"2024-06-28T16:15:39.504976+0000","last_clean":"2024-06-28T16:15:39.504976+0000","last_became_active":"2024-06-28T15:37:33.342585+0000","last_became_peered":"2024-06-28T15:37:33.342585+0000","last_unstale":"2024-06-28T16:15:39.504976+0000","last_undegraded":"2024-06-28T16:15:39.504976+0000","last_fullsized":"2024-06-28T16:15:39.504976+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:39.504775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:39.504775+0000","last_clean_scrub_stamp":"2024-06-28T16:15:39.504775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:04:43.776599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"260'3","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.667083+0000","last_change":"2024-06-28T16:14:56.667083+0000","last_active":"2024-06-28T16:14:56.667083+0000","last_peered":"2024-06-28T16:14:56.667083+0000","last_clean":"2024-06-28T16:14:56.667083+0000","last_became_active":"2024-06-28T15:21:11.096884+0000","last_became_peered":"2024-06-28T15:21:11.096884+0000","last_unstale":"2024-06-28T16:14:56.667083+0000","last_undegraded":"2024-06-28T16:14:56.667083+0000","last_fullsized":"2024-06-28T16:14:56.667083+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:14:56.666741+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:14:56.666741+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.666741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:25:21.454528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"260'12","reported_seq":90,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.959056+0000","last_change":"2024-06-28T16:14:52.959056+0000","last_active":"2024-06-28T16:14:52.959056+0000","last_peered":"2024-06-28T16:14:52.959056+0000","last_clean":"2024-06-28T16:14:52.959056+0000","last_became_active":"2024-06-28T15:21:07.032545+0000","last_became_peered":"2024-06-28T15:21:07.032545+0000","last_unstale":"2024-06-28T16:14:52.959056+0000","last_undegraded":"2024-06-28T16:14:52.959056+0000","last_fullsized":"2024-06-28T16:14:52.959056+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'12","last_scrub_stamp":"2024-06-28T16:14:52.958803+0000","last_deep_scrub":"260'12","last_deep_scrub_stamp":"2024-06-28T16:14:52.958803+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.958803+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:27:34.294061+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":35842,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"260'40","reported_seq":180,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.119887+0000","last_change":"2024-06-28T16:14:42.119952+0000","last_active":"2024-06-28T16:14:42.119887+0000","last_peered":"2024-06-28T16:14:42.119887+0000","last_clean":"2024-06-28T16:14:42.119887+0000","last_became_active":"2024-06-28T15:17:24.026702+0000","last_became_peered":"2024-06-28T15:17:24.026702+0000","last_unstale":"2024-06-28T16:14:42.119887+0000","last_undegraded":"2024-06-28T16:14:42.119887+0000","last_fullsized":"2024-06-28T16:14:42.119887+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'40","last_scrub_stamp":"2024-06-28T16:14:42.119697+0000","last_deep_scrub":"260'40","last_deep_scrub_stamp":"2024-06-28T16:14:42.119697+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.119697+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:57:41.013540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":150,"num_write":83,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:32.063707+0000","last_change":"2024-06-28T16:15:32.063762+0000","last_active":"2024-06-28T16:15:32.063707+0000","last_peered":"2024-06-28T16:15:32.063707+0000","last_clean":"2024-06-28T16:15:32.063707+0000","last_became_active":"2024-06-28T15:37:33.342889+0000","last_became_peered":"2024-06-28T15:37:33.342889+0000","last_unstale":"2024-06-28T16:15:32.063707+0000","last_undegraded":"2024-06-28T16:15:32.063707+0000","last_fullsized":"2024-06-28T16:15:32.063707+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:32.063502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:32.063502+0000","last_clean_scrub_stamp":"2024-06-28T16:15:32.063502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:55:56.613996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"260'2","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.187312+0000","last_change":"2024-06-28T16:15:06.187312+0000","last_active":"2024-06-28T16:15:06.187312+0000","last_peered":"2024-06-28T16:15:06.187312+0000","last_clean":"2024-06-28T16:15:06.187312+0000","last_became_active":"2024-06-28T15:21:11.092614+0000","last_became_peered":"2024-06-28T15:21:11.092614+0000","last_unstale":"2024-06-28T16:15:06.187312+0000","last_undegraded":"2024-06-28T16:15:06.187312+0000","last_fullsized":"2024-06-28T16:15:06.187312+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:06.187009+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:06.187009+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.187009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:50:30.362690+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"257'3","reported_seq":68,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.626086+0000","last_change":"2024-06-28T16:14:47.626086+0000","last_active":"2024-06-28T16:14:47.626086+0000","last_peered":"2024-06-28T16:14:47.626086+0000","last_clean":"2024-06-28T16:14:47.626086+0000","last_became_active":"2024-06-28T15:21:07.034845+0000","last_became_peered":"2024-06-28T15:21:07.034845+0000","last_unstale":"2024-06-28T16:14:47.626086+0000","last_undegraded":"2024-06-28T16:14:47.626086+0000","last_fullsized":"2024-06-28T16:14:47.626086+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:14:47.625736+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:14:47.625736+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.625736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:32:04.945420+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"260'28","reported_seq":148,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.166666+0000","last_change":"2024-06-28T16:14:41.166687+0000","last_active":"2024-06-28T16:14:41.166666+0000","last_peered":"2024-06-28T16:14:41.166666+0000","last_clean":"2024-06-28T16:14:41.166666+0000","last_became_active":"2024-06-28T15:17:24.014034+0000","last_became_peered":"2024-06-28T15:17:24.014034+0000","last_unstale":"2024-06-28T16:14:41.166666+0000","last_undegraded":"2024-06-28T16:14:41.166666+0000","last_fullsized":"2024-06-28T16:14:41.166666+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'28","last_scrub_stamp":"2024-06-28T16:14:41.166589+0000","last_deep_scrub":"260'28","last_deep_scrub_stamp":"2024-06-28T16:14:41.166589+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.166589+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:11:28.889098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":149,"num_read_kb":105,"num_write":55,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:29.853051+0000","last_change":"2024-06-28T16:15:29.853176+0000","last_active":"2024-06-28T16:15:29.853051+0000","last_peered":"2024-06-28T16:15:29.853051+0000","last_clean":"2024-06-28T16:15:29.853051+0000","last_became_active":"2024-06-28T15:37:33.339831+0000","last_became_peered":"2024-06-28T15:37:33.339831+0000","last_unstale":"2024-06-28T16:15:29.853051+0000","last_undegraded":"2024-06-28T16:15:29.853051+0000","last_fullsized":"2024-06-28T16:15:29.853051+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:29.852800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:29.852800+0000","last_clean_scrub_stamp":"2024-06-28T16:15:29.852800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:45:20.965050+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"260'8","reported_seq":76,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:33.390399+0000","last_change":"2024-06-28T16:15:33.390470+0000","last_active":"2024-06-28T16:15:33.390399+0000","last_peered":"2024-06-28T16:15:33.390399+0000","last_clean":"2024-06-28T16:15:33.390399+0000","last_became_active":"2024-06-28T15:21:11.088011+0000","last_became_peered":"2024-06-28T15:21:11.088011+0000","last_unstale":"2024-06-28T16:15:33.390399+0000","last_undegraded":"2024-06-28T16:15:33.390399+0000","last_fullsized":"2024-06-28T16:15:33.390399+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'8","last_scrub_stamp":"2024-06-28T16:15:33.390135+0000","last_deep_scrub":"260'8","last_deep_scrub_stamp":"2024-06-28T16:15:33.390135+0000","last_clean_scrub_stamp":"2024-06-28T16:15:33.390135+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:14:57.591361+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":20522,"num_write":11,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"0'0","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.347289+0000","last_change":"2024-06-28T16:15:06.347289+0000","last_active":"2024-06-28T16:15:06.347289+0000","last_peered":"2024-06-28T16:15:06.347289+0000","last_clean":"2024-06-28T16:15:06.347289+0000","last_became_active":"2024-06-28T15:21:07.030690+0000","last_became_peered":"2024-06-28T15:21:07.030690+0000","last_unstale":"2024-06-28T16:15:06.347289+0000","last_undegraded":"2024-06-28T16:15:06.347289+0000","last_fullsized":"2024-06-28T16:15:06.347289+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:06.347026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:06.347026+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.347026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:04:58.965982+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"260'31","reported_seq":153,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.094693+0000","last_change":"2024-06-28T16:14:44.094763+0000","last_active":"2024-06-28T16:14:44.094693+0000","last_peered":"2024-06-28T16:14:44.094693+0000","last_clean":"2024-06-28T16:14:44.094693+0000","last_became_active":"2024-06-28T15:17:24.026404+0000","last_became_peered":"2024-06-28T15:17:24.026404+0000","last_unstale":"2024-06-28T16:14:44.094693+0000","last_undegraded":"2024-06-28T16:14:44.094693+0000","last_fullsized":"2024-06-28T16:14:44.094693+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'31","last_scrub_stamp":"2024-06-28T16:14:44.094496+0000","last_deep_scrub":"260'31","last_deep_scrub_stamp":"2024-06-28T16:14:44.094496+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.094496+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:11:57.383608+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":161,"num_read_kb":111,"num_write":59,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:21.288145+0000","last_change":"2024-06-28T16:15:21.288271+0000","last_active":"2024-06-28T16:15:21.288145+0000","last_peered":"2024-06-28T16:15:21.288145+0000","last_clean":"2024-06-28T16:15:21.288145+0000","last_became_active":"2024-06-28T15:37:33.332245+0000","last_became_peered":"2024-06-28T15:37:33.332245+0000","last_unstale":"2024-06-28T16:15:21.288145+0000","last_undegraded":"2024-06-28T16:15:21.288145+0000","last_fullsized":"2024-06-28T16:15:21.288145+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:21.287899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:21.287899+0000","last_clean_scrub_stamp":"2024-06-28T16:15:21.287899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:32:23.487275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"0'0","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.610366+0000","last_change":"2024-06-28T16:14:52.610366+0000","last_active":"2024-06-28T16:14:52.610366+0000","last_peered":"2024-06-28T16:14:52.610366+0000","last_clean":"2024-06-28T16:14:52.610366+0000","last_became_active":"2024-06-28T15:21:11.096926+0000","last_became_peered":"2024-06-28T15:21:11.096926+0000","last_unstale":"2024-06-28T16:14:52.610366+0000","last_undegraded":"2024-06-28T16:14:52.610366+0000","last_fullsized":"2024-06-28T16:14:52.610366+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:52.610021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:52.610021+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.610021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:07:41.471423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"260'2","reported_seq":65,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.578940+0000","last_change":"2024-06-28T16:14:56.578940+0000","last_active":"2024-06-28T16:14:56.578940+0000","last_peered":"2024-06-28T16:14:56.578940+0000","last_clean":"2024-06-28T16:14:56.578940+0000","last_became_active":"2024-06-28T15:21:07.036735+0000","last_became_peered":"2024-06-28T15:21:07.036735+0000","last_unstale":"2024-06-28T16:14:56.578940+0000","last_undegraded":"2024-06-28T16:14:56.578940+0000","last_fullsized":"2024-06-28T16:14:56.578940+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:56.578669+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:56.578669+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.578669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:47:53.590130+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"260'49","reported_seq":206,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.629147+0000","last_change":"2024-06-28T16:14:49.629207+0000","last_active":"2024-06-28T16:14:49.629147+0000","last_peered":"2024-06-28T16:14:49.629147+0000","last_clean":"2024-06-28T16:14:49.629147+0000","last_became_active":"2024-06-28T15:17:24.025855+0000","last_became_peered":"2024-06-28T15:17:24.025855+0000","last_unstale":"2024-06-28T16:14:49.629147+0000","last_undegraded":"2024-06-28T16:14:49.629147+0000","last_fullsized":"2024-06-28T16:14:49.629147+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'49","last_scrub_stamp":"2024-06-28T16:14:49.628959+0000","last_deep_scrub":"260'49","last_deep_scrub_stamp":"2024-06-28T16:14:49.628959+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.628959+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:27:20.197035+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":190,"num_write":87,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:29.934179+0000","last_change":"2024-06-28T16:15:29.934234+0000","last_active":"2024-06-28T16:15:29.934179+0000","last_peered":"2024-06-28T16:15:29.934179+0000","last_clean":"2024-06-28T16:15:29.934179+0000","last_became_active":"2024-06-28T15:37:33.322927+0000","last_became_peered":"2024-06-28T15:37:33.322927+0000","last_unstale":"2024-06-28T16:15:29.934179+0000","last_undegraded":"2024-06-28T16:15:29.934179+0000","last_fullsized":"2024-06-28T16:15:29.934179+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:29.933985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:29.933985+0000","last_clean_scrub_stamp":"2024-06-28T16:15:29.933985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:58:05.348138+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:26.965303+0000","last_change":"2024-06-28T16:15:26.965362+0000","last_active":"2024-06-28T16:15:26.965303+0000","last_peered":"2024-06-28T16:15:26.965303+0000","last_clean":"2024-06-28T16:15:26.965303+0000","last_became_active":"2024-06-28T15:21:11.097766+0000","last_became_peered":"2024-06-28T15:21:11.097766+0000","last_unstale":"2024-06-28T16:15:26.965303+0000","last_undegraded":"2024-06-28T16:15:26.965303+0000","last_fullsized":"2024-06-28T16:15:26.965303+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:26.965105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:26.965105+0000","last_clean_scrub_stamp":"2024-06-28T16:15:26.965105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:36:31.252819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.616012+0000","last_change":"2024-06-28T16:14:46.616012+0000","last_active":"2024-06-28T16:14:46.616012+0000","last_peered":"2024-06-28T16:14:46.616012+0000","last_clean":"2024-06-28T16:14:46.616012+0000","last_became_active":"2024-06-28T15:21:07.039381+0000","last_became_peered":"2024-06-28T15:21:07.039381+0000","last_unstale":"2024-06-28T16:14:46.616012+0000","last_undegraded":"2024-06-28T16:14:46.616012+0000","last_fullsized":"2024-06-28T16:14:46.616012+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:46.615676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:46.615676+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.615676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:02:38.667378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"260'26","reported_seq":146,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.130914+0000","last_change":"2024-06-28T16:14:42.130952+0000","last_active":"2024-06-28T16:14:42.130914+0000","last_peered":"2024-06-28T16:14:42.130914+0000","last_clean":"2024-06-28T16:14:42.130914+0000","last_became_active":"2024-06-28T15:17:24.026580+0000","last_became_peered":"2024-06-28T15:17:24.026580+0000","last_unstale":"2024-06-28T16:14:42.130914+0000","last_undegraded":"2024-06-28T16:14:42.130914+0000","last_fullsized":"2024-06-28T16:14:42.130914+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'26","last_scrub_stamp":"2024-06-28T16:14:42.130814+0000","last_deep_scrub":"260'26","last_deep_scrub_stamp":"2024-06-28T16:14:42.130814+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.130814+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:03:06.475528+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":143,"num_read_kb":101,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"260'2","reported_seq":40,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.336046+0000","last_change":"2024-06-28T16:15:11.336114+0000","last_active":"2024-06-28T16:15:11.336046+0000","last_peered":"2024-06-28T16:15:11.336046+0000","last_clean":"2024-06-28T16:15:11.336046+0000","last_became_active":"2024-06-28T15:37:33.349137+0000","last_became_peered":"2024-06-28T15:37:33.349137+0000","last_unstale":"2024-06-28T16:15:11.336046+0000","last_undegraded":"2024-06-28T16:15:11.336046+0000","last_fullsized":"2024-06-28T16:15:11.336046+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:11.335857+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:11.335857+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.335857+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:55:35.045287+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"260'6","reported_seq":65,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.298763+0000","last_change":"2024-06-28T16:15:19.298763+0000","last_active":"2024-06-28T16:15:19.298763+0000","last_peered":"2024-06-28T16:15:19.298763+0000","last_clean":"2024-06-28T16:15:19.298763+0000","last_became_active":"2024-06-28T15:21:11.104270+0000","last_became_peered":"2024-06-28T15:21:11.104270+0000","last_unstale":"2024-06-28T16:15:19.298763+0000","last_undegraded":"2024-06-28T16:15:19.298763+0000","last_fullsized":"2024-06-28T16:15:19.298763+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:15:19.298407+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:15:19.298407+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.298407+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:47:09.226133+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":11,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"260'6","reported_seq":73,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.653159+0000","last_change":"2024-06-28T16:14:44.653159+0000","last_active":"2024-06-28T16:14:44.653159+0000","last_peered":"2024-06-28T16:14:44.653159+0000","last_clean":"2024-06-28T16:14:44.653159+0000","last_became_active":"2024-06-28T15:21:07.048639+0000","last_became_peered":"2024-06-28T15:21:07.048639+0000","last_unstale":"2024-06-28T16:14:44.653159+0000","last_undegraded":"2024-06-28T16:14:44.653159+0000","last_fullsized":"2024-06-28T16:14:44.653159+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:14:44.652902+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:14:44.652902+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.652902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:58:20.490373+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":12288,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"260'22","reported_seq":123,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.693623+0000","last_change":"2024-06-28T16:14:41.693689+0000","last_active":"2024-06-28T16:14:41.693623+0000","last_peered":"2024-06-28T16:14:41.693623+0000","last_clean":"2024-06-28T16:14:41.693623+0000","last_became_active":"2024-06-28T15:17:24.006784+0000","last_became_peered":"2024-06-28T15:17:24.006784+0000","last_unstale":"2024-06-28T16:14:41.693623+0000","last_undegraded":"2024-06-28T16:14:41.693623+0000","last_fullsized":"2024-06-28T16:14:41.693623+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'22","last_scrub_stamp":"2024-06-28T16:14:41.693433+0000","last_deep_scrub":"260'22","last_deep_scrub_stamp":"2024-06-28T16:14:41.693433+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.693433+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:31:21.802215+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":64,"num_write":51,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:37.456493+0000","last_change":"2024-06-28T16:15:37.456550+0000","last_active":"2024-06-28T16:15:37.456493+0000","last_peered":"2024-06-28T16:15:37.456493+0000","last_clean":"2024-06-28T16:15:37.456493+0000","last_became_active":"2024-06-28T15:37:33.333737+0000","last_became_peered":"2024-06-28T15:37:33.333737+0000","last_unstale":"2024-06-28T16:15:37.456493+0000","last_undegraded":"2024-06-28T16:15:37.456493+0000","last_fullsized":"2024-06-28T16:15:37.456493+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:37.456285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:37.456285+0000","last_clean_scrub_stamp":"2024-06-28T16:15:37.456285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:58:21.487688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:24.889265+0000","last_change":"2024-06-28T16:15:24.889265+0000","last_active":"2024-06-28T16:15:24.889265+0000","last_peered":"2024-06-28T16:15:24.889265+0000","last_clean":"2024-06-28T16:15:24.889265+0000","last_became_active":"2024-06-28T15:21:11.092034+0000","last_became_peered":"2024-06-28T15:21:11.092034+0000","last_unstale":"2024-06-28T16:15:24.889265+0000","last_undegraded":"2024-06-28T16:15:24.889265+0000","last_fullsized":"2024-06-28T16:15:24.889265+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:24.888926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:24.888926+0000","last_clean_scrub_stamp":"2024-06-28T16:15:24.888926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:34:36.805202+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"260'3","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.606954+0000","last_change":"2024-06-28T16:14:43.606954+0000","last_active":"2024-06-28T16:14:43.606954+0000","last_peered":"2024-06-28T16:14:43.606954+0000","last_clean":"2024-06-28T16:14:43.606954+0000","last_became_active":"2024-06-28T15:21:07.039812+0000","last_became_peered":"2024-06-28T15:21:07.039812+0000","last_unstale":"2024-06-28T16:14:43.606954+0000","last_undegraded":"2024-06-28T16:14:43.606954+0000","last_fullsized":"2024-06-28T16:14:43.606954+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:14:43.606698+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:14:43.606698+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.606698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:49:13.801868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"260'78","reported_seq":317,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.919916+0000","last_change":"2024-06-28T16:14:41.919989+0000","last_active":"2024-06-28T16:14:41.919916+0000","last_peered":"2024-06-28T16:14:41.919916+0000","last_clean":"2024-06-28T16:14:41.919916+0000","last_became_active":"2024-06-28T15:17:24.012854+0000","last_became_peered":"2024-06-28T15:17:24.012854+0000","last_unstale":"2024-06-28T16:14:41.919916+0000","last_undegraded":"2024-06-28T16:14:41.919916+0000","last_fullsized":"2024-06-28T16:14:41.919916+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'78","last_scrub_stamp":"2024-06-28T16:14:41.919720+0000","last_deep_scrub":"260'78","last_deep_scrub_stamp":"2024-06-28T16:14:41.919720+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.919720+0000","objects_scrubbed":1,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:17:37.822460+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":483,"num_read_kb":330,"num_write":109,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4960,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:35.421126+0000","last_change":"2024-06-28T16:15:35.421182+0000","last_active":"2024-06-28T16:15:35.421126+0000","last_peered":"2024-06-28T16:15:35.421126+0000","last_clean":"2024-06-28T16:15:35.421126+0000","last_became_active":"2024-06-28T15:37:33.322317+0000","last_became_peered":"2024-06-28T15:37:33.322317+0000","last_unstale":"2024-06-28T16:15:35.421126+0000","last_undegraded":"2024-06-28T16:15:35.421126+0000","last_fullsized":"2024-06-28T16:15:35.421126+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:35.420912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:35.420912+0000","last_clean_scrub_stamp":"2024-06-28T16:15:35.420912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:26:11.682180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.394628+0000","last_change":"2024-06-28T16:15:17.394628+0000","last_active":"2024-06-28T16:15:17.394628+0000","last_peered":"2024-06-28T16:15:17.394628+0000","last_clean":"2024-06-28T16:15:17.394628+0000","last_became_active":"2024-06-28T15:21:11.108663+0000","last_became_peered":"2024-06-28T15:21:11.108663+0000","last_unstale":"2024-06-28T16:15:17.394628+0000","last_undegraded":"2024-06-28T16:15:17.394628+0000","last_fullsized":"2024-06-28T16:15:17.394628+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:17.394367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:17.394367+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.394367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:27:51.882594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"260'3","reported_seq":68,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.135141+0000","last_change":"2024-06-28T16:14:40.135141+0000","last_active":"2024-06-28T16:14:40.135141+0000","last_peered":"2024-06-28T16:14:40.135141+0000","last_clean":"2024-06-28T16:14:40.135141+0000","last_became_active":"2024-06-28T15:21:07.022046+0000","last_became_peered":"2024-06-28T15:21:07.022046+0000","last_unstale":"2024-06-28T16:14:40.135141+0000","last_undegraded":"2024-06-28T16:14:40.135141+0000","last_fullsized":"2024-06-28T16:14:40.135141+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:14:40.134988+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:14:40.134988+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.134988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:06:49.985940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"260'31","reported_seq":145,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.121279+0000","last_change":"2024-06-28T16:14:45.121348+0000","last_active":"2024-06-28T16:14:45.121279+0000","last_peered":"2024-06-28T16:14:45.121279+0000","last_clean":"2024-06-28T16:14:45.121279+0000","last_became_active":"2024-06-28T15:17:24.010873+0000","last_became_peered":"2024-06-28T15:17:24.010873+0000","last_unstale":"2024-06-28T16:14:45.121279+0000","last_undegraded":"2024-06-28T16:14:45.121279+0000","last_fullsized":"2024-06-28T16:14:45.121279+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'31","last_scrub_stamp":"2024-06-28T16:14:45.121064+0000","last_deep_scrub":"260'31","last_deep_scrub_stamp":"2024-06-28T16:14:45.121064+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.121064+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:05:38.766333+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":94,"num_write":78,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:34.114903+0000","last_change":"2024-06-28T16:15:34.114956+0000","last_active":"2024-06-28T16:15:34.114903+0000","last_peered":"2024-06-28T16:15:34.114903+0000","last_clean":"2024-06-28T16:15:34.114903+0000","last_became_active":"2024-06-28T15:37:33.347075+0000","last_became_peered":"2024-06-28T15:37:33.347075+0000","last_unstale":"2024-06-28T16:15:34.114903+0000","last_undegraded":"2024-06-28T16:15:34.114903+0000","last_fullsized":"2024-06-28T16:15:34.114903+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:34.114706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:34.114706+0000","last_clean_scrub_stamp":"2024-06-28T16:15:34.114706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:14:56.120842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":57,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:25.970748+0000","last_change":"2024-06-28T16:15:25.970806+0000","last_active":"2024-06-28T16:15:25.970748+0000","last_peered":"2024-06-28T16:15:25.970748+0000","last_clean":"2024-06-28T16:15:25.970748+0000","last_became_active":"2024-06-28T15:21:11.093082+0000","last_became_peered":"2024-06-28T15:21:11.093082+0000","last_unstale":"2024-06-28T16:15:25.970748+0000","last_undegraded":"2024-06-28T16:15:25.970748+0000","last_fullsized":"2024-06-28T16:15:25.970748+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:25.970539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:25.970539+0000","last_clean_scrub_stamp":"2024-06-28T16:15:25.970539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:10:13.261154+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"257'3","reported_seq":68,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.165567+0000","last_change":"2024-06-28T16:14:46.165567+0000","last_active":"2024-06-28T16:14:46.165567+0000","last_peered":"2024-06-28T16:14:46.165567+0000","last_clean":"2024-06-28T16:14:46.165567+0000","last_became_active":"2024-06-28T15:21:07.040575+0000","last_became_peered":"2024-06-28T15:21:07.040575+0000","last_unstale":"2024-06-28T16:14:46.165567+0000","last_undegraded":"2024-06-28T16:14:46.165567+0000","last_fullsized":"2024-06-28T16:14:46.165567+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:14:46.165127+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:14:46.165127+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.165127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:42:56.667095+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"260'16","reported_seq":108,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.625944+0000","last_change":"2024-06-28T16:14:51.625944+0000","last_active":"2024-06-28T16:14:51.625944+0000","last_peered":"2024-06-28T16:14:51.625944+0000","last_clean":"2024-06-28T16:14:51.625944+0000","last_became_active":"2024-06-28T15:17:24.021464+0000","last_became_peered":"2024-06-28T15:17:24.021464+0000","last_unstale":"2024-06-28T16:14:51.625944+0000","last_undegraded":"2024-06-28T16:14:51.625944+0000","last_fullsized":"2024-06-28T16:14:51.625944+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'16","last_scrub_stamp":"2024-06-28T16:14:51.625680+0000","last_deep_scrub":"260'16","last_deep_scrub_stamp":"2024-06-28T16:14:51.625680+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.625680+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:28:15.263944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":44,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"260'341","reported_seq":971,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.145524+0000","last_change":"2024-06-28T16:14:46.145629+0000","last_active":"2024-06-28T16:14:46.145524+0000","last_peered":"2024-06-28T16:14:46.145524+0000","last_clean":"2024-06-28T16:14:46.145524+0000","last_became_active":"2024-06-28T15:00:59.941681+0000","last_became_peered":"2024-06-28T15:00:59.941681+0000","last_unstale":"2024-06-28T16:14:46.145524+0000","last_undegraded":"2024-06-28T16:14:46.145524+0000","last_fullsized":"2024-06-28T16:14:46.145524+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'341","last_scrub_stamp":"2024-06-28T16:14:46.145323+0000","last_deep_scrub":"260'341","last_deep_scrub_stamp":"2024-06-28T16:14:46.145323+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.145323+0000","objects_scrubbed":6,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:36:27.892777+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170282,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":488,"num_read_kb":26898,"num_write":1268,"num_write_kb":25985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0,5],"acting":[2,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"260'182","reported_seq":797,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.635725+0000","last_change":"2024-06-28T16:15:02.635725+0000","last_active":"2024-06-28T16:15:02.635725+0000","last_peered":"2024-06-28T16:15:02.635725+0000","last_clean":"2024-06-28T16:15:02.635725+0000","last_became_active":"2024-06-28T15:00:17.404342+0000","last_became_peered":"2024-06-28T15:00:17.404342+0000","last_unstale":"2024-06-28T16:15:02.635725+0000","last_undegraded":"2024-06-28T16:15:02.635725+0000","last_fullsized":"2024-06-28T16:15:02.635725+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'182","last_scrub_stamp":"2024-06-28T16:15:02.635444+0000","last_deep_scrub":"260'182","last_deep_scrub_stamp":"2024-06-28T16:15:02.635444+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.635444+0000","objects_scrubbed":1,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:22:41.865680+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":722,"num_read_kb":569,"num_write":267,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":272,"num_omap_keys":1,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"260'326","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.235841+0000","last_change":"2024-06-28T16:15:13.235841+0000","last_active":"2024-06-28T16:15:13.235841+0000","last_peered":"2024-06-28T16:15:13.235841+0000","last_clean":"2024-06-28T16:15:13.235841+0000","last_became_active":"2024-06-28T15:00:15.419449+0000","last_became_peered":"2024-06-28T15:00:15.419449+0000","last_unstale":"2024-06-28T16:15:13.235841+0000","last_undegraded":"2024-06-28T16:15:13.235841+0000","last_fullsized":"2024-06-28T16:15:13.235841+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'326","last_scrub_stamp":"2024-06-28T16:15:13.235513+0000","last_deep_scrub":"260'326","last_deep_scrub_stamp":"2024-06-28T16:15:13.235513+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.235513+0000","objects_scrubbed":10,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:42:49.114452+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5526738,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":485,"num_read_kb":15789,"num_write":1172,"num_write_kb":37304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"260'287","reported_seq":951,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.751954+0000","last_change":"2024-06-28T16:15:09.751954+0000","last_active":"2024-06-28T16:15:09.751954+0000","last_peered":"2024-06-28T16:15:09.751954+0000","last_clean":"2024-06-28T16:15:09.751954+0000","last_became_active":"2024-06-28T15:00:17.408128+0000","last_became_peered":"2024-06-28T15:00:17.408128+0000","last_unstale":"2024-06-28T16:15:09.751954+0000","last_undegraded":"2024-06-28T16:15:09.751954+0000","last_fullsized":"2024-06-28T16:15:09.751954+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'287","last_scrub_stamp":"2024-06-28T16:15:09.751693+0000","last_deep_scrub":"260'287","last_deep_scrub_stamp":"2024-06-28T16:15:09.751693+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.751693+0000","objects_scrubbed":2,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:50:27.173640+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1157,"num_read_kb":911,"num_write":439,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3314,"num_omap_keys":19,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"260'283","reported_seq":979,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:26.213414+0000","last_change":"2024-06-28T16:15:26.213641+0000","last_active":"2024-06-28T16:15:26.213414+0000","last_peered":"2024-06-28T16:15:26.213414+0000","last_clean":"2024-06-28T16:15:26.213414+0000","last_became_active":"2024-06-28T15:00:15.396469+0000","last_became_peered":"2024-06-28T15:00:15.396469+0000","last_unstale":"2024-06-28T16:15:26.213414+0000","last_undegraded":"2024-06-28T16:15:26.213414+0000","last_fullsized":"2024-06-28T16:15:26.213414+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'283","last_scrub_stamp":"2024-06-28T16:15:26.213188+0000","last_deep_scrub":"260'283","last_deep_scrub_stamp":"2024-06-28T16:15:26.213188+0000","last_clean_scrub_stamp":"2024-06-28T16:15:26.213188+0000","objects_scrubbed":6,"log_size":83,"log_dups_size":200,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:30:45.912047+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1096214,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":504,"num_read_kb":81437,"num_write":1108,"num_write_kb":14924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"260'78","reported_seq":654,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.007487+0000","last_change":"2024-06-28T16:15:08.007487+0000","last_active":"2024-06-28T16:15:08.007487+0000","last_peered":"2024-06-28T16:15:08.007487+0000","last_clean":"2024-06-28T16:15:08.007487+0000","last_became_active":"2024-06-28T15:00:17.402542+0000","last_became_peered":"2024-06-28T15:00:17.402542+0000","last_unstale":"2024-06-28T16:15:08.007487+0000","last_undegraded":"2024-06-28T16:15:08.007487+0000","last_fullsized":"2024-06-28T16:15:08.007487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'78","last_scrub_stamp":"2024-06-28T16:15:08.007123+0000","last_deep_scrub":"260'78","last_deep_scrub_stamp":"2024-06-28T16:15:08.007123+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.007123+0000","objects_scrubbed":1,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:43:06.313520+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":306,"num_read_kb":248,"num_write":126,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":396,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"260'313","reported_seq":990,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.683417+0000","last_change":"2024-06-28T16:15:07.683559+0000","last_active":"2024-06-28T16:15:07.683417+0000","last_peered":"2024-06-28T16:15:07.683417+0000","last_clean":"2024-06-28T16:15:07.683417+0000","last_became_active":"2024-06-28T15:00:15.435122+0000","last_became_peered":"2024-06-28T15:00:15.435122+0000","last_unstale":"2024-06-28T16:15:07.683417+0000","last_undegraded":"2024-06-28T16:15:07.683417+0000","last_fullsized":"2024-06-28T16:15:07.683417+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'313","last_scrub_stamp":"2024-06-28T16:15:07.683192+0000","last_deep_scrub":"260'313","last_deep_scrub_stamp":"2024-06-28T16:15:07.683192+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.683192+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:10:01.212501+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1454317,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":492,"num_read_kb":48678,"num_write":1275,"num_write_kb":34259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"260'52","reported_seq":640,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.278889+0000","last_change":"2024-06-28T16:15:11.278889+0000","last_active":"2024-06-28T16:15:11.278889+0000","last_peered":"2024-06-28T16:15:11.278889+0000","last_clean":"2024-06-28T16:15:11.278889+0000","last_became_active":"2024-06-28T15:00:17.384932+0000","last_became_peered":"2024-06-28T15:00:17.384932+0000","last_unstale":"2024-06-28T16:15:11.278889+0000","last_undegraded":"2024-06-28T16:15:11.278889+0000","last_fullsized":"2024-06-28T16:15:11.278889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'52","last_scrub_stamp":"2024-06-28T16:15:11.278605+0000","last_deep_scrub":"260'52","last_deep_scrub_stamp":"2024-06-28T16:15:11.278605+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.278605+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:12:36.542381+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":197,"num_write":67,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"260'343","reported_seq":1002,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.835980+0000","last_change":"2024-06-28T16:15:15.836133+0000","last_active":"2024-06-28T16:15:15.835980+0000","last_peered":"2024-06-28T16:15:15.835980+0000","last_clean":"2024-06-28T16:15:15.835980+0000","last_became_active":"2024-06-28T15:00:15.430208+0000","last_became_peered":"2024-06-28T15:00:15.430208+0000","last_unstale":"2024-06-28T16:15:15.835980+0000","last_undegraded":"2024-06-28T16:15:15.835980+0000","last_fullsized":"2024-06-28T16:15:15.835980+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'343","last_scrub_stamp":"2024-06-28T16:15:15.835782+0000","last_deep_scrub":"260'343","last_deep_scrub_stamp":"2024-06-28T16:15:15.835782+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.835782+0000","objects_scrubbed":8,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:41:52.084536+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437780,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":548,"num_read_kb":39482,"num_write":1215,"num_write_kb":35176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"260'186","reported_seq":848,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.448598+0000","last_change":"2024-06-28T16:14:59.448668+0000","last_active":"2024-06-28T16:14:59.448598+0000","last_peered":"2024-06-28T16:14:59.448598+0000","last_clean":"2024-06-28T16:14:59.448598+0000","last_became_active":"2024-06-28T15:00:17.397986+0000","last_became_peered":"2024-06-28T15:00:17.397986+0000","last_unstale":"2024-06-28T16:14:59.448598+0000","last_undegraded":"2024-06-28T16:14:59.448598+0000","last_fullsized":"2024-06-28T16:14:59.448598+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'186","last_scrub_stamp":"2024-06-28T16:14:59.448384+0000","last_deep_scrub":"260'186","last_deep_scrub_stamp":"2024-06-28T16:14:59.448384+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.448384+0000","objects_scrubbed":1,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:21:42.502350+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":777,"num_read_kb":632,"num_write":274,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":473,"num_omap_keys":2,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"260'364","reported_seq":1039,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.804713+0000","last_change":"2024-06-28T16:15:17.804834+0000","last_active":"2024-06-28T16:15:17.804713+0000","last_peered":"2024-06-28T16:15:17.804713+0000","last_clean":"2024-06-28T16:15:17.804713+0000","last_became_active":"2024-06-28T15:00:15.409542+0000","last_became_peered":"2024-06-28T15:00:15.409542+0000","last_unstale":"2024-06-28T16:15:17.804713+0000","last_undegraded":"2024-06-28T16:15:17.804713+0000","last_fullsized":"2024-06-28T16:15:17.804713+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'364","last_scrub_stamp":"2024-06-28T16:15:17.804471+0000","last_deep_scrub":"260'364","last_deep_scrub_stamp":"2024-06-28T16:15:17.804471+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.804471+0000","objects_scrubbed":7,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:31:04.306481+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1667748,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":553,"num_read_kb":16754,"num_write":1370,"num_write_kb":32794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"260'44","reported_seq":618,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.888516+0000","last_change":"2024-06-28T16:14:59.888516+0000","last_active":"2024-06-28T16:14:59.888516+0000","last_peered":"2024-06-28T16:14:59.888516+0000","last_clean":"2024-06-28T16:14:59.888516+0000","last_became_active":"2024-06-28T15:00:17.401985+0000","last_became_peered":"2024-06-28T15:00:17.401985+0000","last_unstale":"2024-06-28T16:14:59.888516+0000","last_undegraded":"2024-06-28T16:14:59.888516+0000","last_fullsized":"2024-06-28T16:14:59.888516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'44","last_scrub_stamp":"2024-06-28T16:14:59.888222+0000","last_deep_scrub":"260'44","last_deep_scrub_stamp":"2024-06-28T16:14:59.888222+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.888222+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:28:28.187398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":159,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"260'328","reported_seq":969,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:21.854134+0000","last_change":"2024-06-28T16:15:21.854270+0000","last_active":"2024-06-28T16:15:21.854134+0000","last_peered":"2024-06-28T16:15:21.854134+0000","last_clean":"2024-06-28T16:15:21.854134+0000","last_became_active":"2024-06-28T15:00:15.424430+0000","last_became_peered":"2024-06-28T15:00:15.424430+0000","last_unstale":"2024-06-28T16:15:21.854134+0000","last_undegraded":"2024-06-28T16:15:21.854134+0000","last_fullsized":"2024-06-28T16:15:21.854134+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'328","last_scrub_stamp":"2024-06-28T16:15:21.853905+0000","last_deep_scrub":"260'328","last_deep_scrub_stamp":"2024-06-28T16:15:21.853905+0000","last_clean_scrub_stamp":"2024-06-28T16:15:21.853905+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:54:26.716654+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048602,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":476,"num_read_kb":21908,"num_write":1069,"num_write_kb":36265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"260'4326","reported_seq":5006,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.551681+0000","last_change":"2024-06-28T16:14:59.551681+0000","last_active":"2024-06-28T16:14:59.551681+0000","last_peered":"2024-06-28T16:14:59.551681+0000","last_clean":"2024-06-28T16:14:59.551681+0000","last_became_active":"2024-06-28T15:00:17.384378+0000","last_became_peered":"2024-06-28T15:00:17.384378+0000","last_unstale":"2024-06-28T16:14:59.551681+0000","last_undegraded":"2024-06-28T16:14:59.551681+0000","last_fullsized":"2024-06-28T16:14:59.551681+0000","mapping_epoch":21,"log_start":"260'4300","ondisk_log_start":"260'4300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4326","last_scrub_stamp":"2024-06-28T16:14:59.551430+0000","last_deep_scrub":"260'4326","last_deep_scrub_stamp":"2024-06-28T16:14:59.551430+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.551430+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":3000,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:51:42.703162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15352,"num_read_kb":12408,"num_write":6557,"num_write_kb":3277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"260'411","reported_seq":1138,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:32.273569+0000","last_change":"2024-06-28T16:15:32.273661+0000","last_active":"2024-06-28T16:15:32.273569+0000","last_peered":"2024-06-28T16:15:32.273569+0000","last_clean":"2024-06-28T16:15:32.273569+0000","last_became_active":"2024-06-28T15:00:15.401960+0000","last_became_peered":"2024-06-28T15:00:15.401960+0000","last_unstale":"2024-06-28T16:15:32.273569+0000","last_undegraded":"2024-06-28T16:15:32.273569+0000","last_fullsized":"2024-06-28T16:15:32.273569+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'411","last_scrub_stamp":"2024-06-28T16:15:32.273339+0000","last_deep_scrub":"260'411","last_deep_scrub_stamp":"2024-06-28T16:15:32.273339+0000","last_clean_scrub_stamp":"2024-06-28T16:15:32.273339+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":400,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:22:04.556909+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324665,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":810,"num_read_kb":20615,"num_write":1466,"num_write_kb":29750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"260'70","reported_seq":695,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.302209+0000","last_change":"2024-06-28T16:14:59.302209+0000","last_active":"2024-06-28T16:14:59.302209+0000","last_peered":"2024-06-28T16:14:59.302209+0000","last_clean":"2024-06-28T16:14:59.302209+0000","last_became_active":"2024-06-28T15:00:17.381196+0000","last_became_peered":"2024-06-28T15:00:17.381196+0000","last_unstale":"2024-06-28T16:14:59.302209+0000","last_undegraded":"2024-06-28T16:14:59.302209+0000","last_fullsized":"2024-06-28T16:14:59.302209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'70","last_scrub_stamp":"2024-06-28T16:14:59.301939+0000","last_deep_scrub":"260'70","last_deep_scrub_stamp":"2024-06-28T16:14:59.301939+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.301939+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:19:16.430915+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":309,"num_read_kb":273,"num_write":88,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"260'130","reported_seq":762,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.971071+0000","last_change":"2024-06-28T16:15:13.971140+0000","last_active":"2024-06-28T16:15:13.971071+0000","last_peered":"2024-06-28T16:15:13.971071+0000","last_clean":"2024-06-28T16:15:13.971071+0000","last_became_active":"2024-06-28T15:00:17.396142+0000","last_became_peered":"2024-06-28T15:00:17.396142+0000","last_unstale":"2024-06-28T16:15:13.971071+0000","last_undegraded":"2024-06-28T16:15:13.971071+0000","last_fullsized":"2024-06-28T16:15:13.971071+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'130","last_scrub_stamp":"2024-06-28T16:15:13.970878+0000","last_deep_scrub":"260'130","last_deep_scrub_stamp":"2024-06-28T16:15:13.970878+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.970878+0000","objects_scrubbed":2,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:21:34.067289+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":536,"num_read_kb":452,"num_write":191,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":546,"num_omap_keys":3,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"260'416","reported_seq":1196,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:29.038983+0000","last_change":"2024-06-28T16:15:29.039073+0000","last_active":"2024-06-28T16:15:29.038983+0000","last_peered":"2024-06-28T16:15:29.038983+0000","last_clean":"2024-06-28T16:15:29.038983+0000","last_became_active":"2024-06-28T15:00:15.422579+0000","last_became_peered":"2024-06-28T15:00:15.422579+0000","last_unstale":"2024-06-28T16:15:29.038983+0000","last_undegraded":"2024-06-28T16:15:29.038983+0000","last_fullsized":"2024-06-28T16:15:29.038983+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'416","last_scrub_stamp":"2024-06-28T16:15:29.038792+0000","last_deep_scrub":"260'416","last_deep_scrub_stamp":"2024-06-28T16:15:29.038792+0000","last_clean_scrub_stamp":"2024-06-28T16:15:29.038792+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":400,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:03:21.671624+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":851,"num_read_kb":96606,"num_write":1464,"num_write_kb":38692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0,1],"acting":[7,5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"260'254","reported_seq":909,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.709122+0000","last_change":"2024-06-28T16:15:07.709182+0000","last_active":"2024-06-28T16:15:07.709122+0000","last_peered":"2024-06-28T16:15:07.709122+0000","last_clean":"2024-06-28T16:15:07.709122+0000","last_became_active":"2024-06-28T15:00:17.409202+0000","last_became_peered":"2024-06-28T15:00:17.409202+0000","last_unstale":"2024-06-28T16:15:07.709122+0000","last_undegraded":"2024-06-28T16:15:07.709122+0000","last_fullsized":"2024-06-28T16:15:07.709122+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'254","last_scrub_stamp":"2024-06-28T16:15:07.708938+0000","last_deep_scrub":"260'254","last_deep_scrub_stamp":"2024-06-28T16:15:07.708938+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.708938+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:59:43.588127+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1059,"num_read_kb":851,"num_write":432,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"260'313","reported_seq":962,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.430774+0000","last_change":"2024-06-28T16:15:11.430934+0000","last_active":"2024-06-28T16:15:11.430774+0000","last_peered":"2024-06-28T16:15:11.430774+0000","last_clean":"2024-06-28T16:15:11.430774+0000","last_became_active":"2024-06-28T15:00:15.421980+0000","last_became_peered":"2024-06-28T15:00:15.421980+0000","last_unstale":"2024-06-28T16:15:11.430774+0000","last_undegraded":"2024-06-28T16:15:11.430774+0000","last_fullsized":"2024-06-28T16:15:11.430774+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'313","last_scrub_stamp":"2024-06-28T16:15:11.430569+0000","last_deep_scrub":"260'313","last_deep_scrub_stamp":"2024-06-28T16:15:11.430569+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.430569+0000","objects_scrubbed":7,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:44:39.434883+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468838,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":425,"num_read_kb":33937,"num_write":1026,"num_write_kb":43346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"260'226","reported_seq":861,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.587895+0000","last_change":"2024-06-28T16:15:01.587895+0000","last_active":"2024-06-28T16:15:01.587895+0000","last_peered":"2024-06-28T16:15:01.587895+0000","last_clean":"2024-06-28T16:15:01.587895+0000","last_became_active":"2024-06-28T15:00:17.384914+0000","last_became_peered":"2024-06-28T15:00:17.384914+0000","last_unstale":"2024-06-28T16:15:01.587895+0000","last_undegraded":"2024-06-28T16:15:01.587895+0000","last_fullsized":"2024-06-28T16:15:01.587895+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'226","last_scrub_stamp":"2024-06-28T16:15:01.587618+0000","last_deep_scrub":"260'226","last_deep_scrub_stamp":"2024-06-28T16:15:01.587618+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.587618+0000","objects_scrubbed":1,"log_size":26,"log_dups_size":200,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:02:48.171099+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":887,"num_read_kb":706,"num_write":342,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":720,"num_omap_keys":3,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"260'342","reported_seq":1201,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.377861+0000","last_change":"2024-06-28T16:15:08.377861+0000","last_active":"2024-06-28T16:15:08.377861+0000","last_peered":"2024-06-28T16:15:08.377861+0000","last_clean":"2024-06-28T16:15:08.377861+0000","last_became_active":"2024-06-28T15:00:15.411567+0000","last_became_peered":"2024-06-28T15:00:15.411567+0000","last_unstale":"2024-06-28T16:15:08.377861+0000","last_undegraded":"2024-06-28T16:15:08.377861+0000","last_fullsized":"2024-06-28T16:15:08.377861+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'342","last_scrub_stamp":"2024-06-28T16:15:08.377521+0000","last_deep_scrub":"260'342","last_deep_scrub_stamp":"2024-06-28T16:15:08.377521+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.377521+0000","objects_scrubbed":4,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:33:53.677785+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289237,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":936,"num_read_kb":14303,"num_write":1297,"num_write_kb":46873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"260'129","reported_seq":747,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.355962+0000","last_change":"2024-06-28T16:15:00.355962+0000","last_active":"2024-06-28T16:15:00.355962+0000","last_peered":"2024-06-28T16:15:00.355962+0000","last_clean":"2024-06-28T16:15:00.355962+0000","last_became_active":"2024-06-28T15:00:17.389613+0000","last_became_peered":"2024-06-28T15:00:17.389613+0000","last_unstale":"2024-06-28T16:15:00.355962+0000","last_undegraded":"2024-06-28T16:15:00.355962+0000","last_fullsized":"2024-06-28T16:15:00.355962+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'129","last_scrub_stamp":"2024-06-28T16:15:00.355683+0000","last_deep_scrub":"260'129","last_deep_scrub_stamp":"2024-06-28T16:15:00.355683+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.355683+0000","objects_scrubbed":1,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:21:03.329874+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":419,"num_write":192,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"260'320","reported_seq":1082,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.926608+0000","last_change":"2024-06-28T16:15:18.926827+0000","last_active":"2024-06-28T16:15:18.926608+0000","last_peered":"2024-06-28T16:15:18.926608+0000","last_clean":"2024-06-28T16:15:18.926608+0000","last_became_active":"2024-06-28T15:00:15.417563+0000","last_became_peered":"2024-06-28T15:00:15.417563+0000","last_unstale":"2024-06-28T16:15:18.926608+0000","last_undegraded":"2024-06-28T16:15:18.926608+0000","last_fullsized":"2024-06-28T16:15:18.926608+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'320","last_scrub_stamp":"2024-06-28T16:15:18.926356+0000","last_deep_scrub":"260'320","last_deep_scrub_stamp":"2024-06-28T16:15:18.926356+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.926356+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:35:12.865205+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":525349,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":605,"num_read_kb":12860,"num_write":1159,"num_write_kb":18563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"260'157","reported_seq":748,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.360593+0000","last_change":"2024-06-28T16:15:03.360593+0000","last_active":"2024-06-28T16:15:03.360593+0000","last_peered":"2024-06-28T16:15:03.360593+0000","last_clean":"2024-06-28T16:15:03.360593+0000","last_became_active":"2024-06-28T15:00:17.413420+0000","last_became_peered":"2024-06-28T15:00:17.413420+0000","last_unstale":"2024-06-28T16:15:03.360593+0000","last_undegraded":"2024-06-28T16:15:03.360593+0000","last_fullsized":"2024-06-28T16:15:03.360593+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'157","last_scrub_stamp":"2024-06-28T16:15:03.360224+0000","last_deep_scrub":"260'157","last_deep_scrub_stamp":"2024-06-28T16:15:03.360224+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.360224+0000","objects_scrubbed":1,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:39:07.258685+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":589,"num_read_kb":466,"num_write":222,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"260'357","reported_seq":1083,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:30.329671+0000","last_change":"2024-06-28T16:15:30.329853+0000","last_active":"2024-06-28T16:15:30.329671+0000","last_peered":"2024-06-28T16:15:30.329671+0000","last_clean":"2024-06-28T16:15:30.329671+0000","last_became_active":"2024-06-28T15:00:15.401370+0000","last_became_peered":"2024-06-28T15:00:15.401370+0000","last_unstale":"2024-06-28T16:15:30.329671+0000","last_undegraded":"2024-06-28T16:15:30.329671+0000","last_fullsized":"2024-06-28T16:15:30.329671+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'357","last_scrub_stamp":"2024-06-28T16:15:30.329450+0000","last_deep_scrub":"260'357","last_deep_scrub_stamp":"2024-06-28T16:15:30.329450+0000","last_clean_scrub_stamp":"2024-06-28T16:15:30.329450+0000","objects_scrubbed":7,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:12:52.635418+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324689,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":676,"num_read_kb":30262,"num_write":1536,"num_write_kb":36456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"260'253","reported_seq":912,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.991624+0000","last_change":"2024-06-28T16:15:06.991624+0000","last_active":"2024-06-28T16:15:06.991624+0000","last_peered":"2024-06-28T16:15:06.991624+0000","last_clean":"2024-06-28T16:15:06.991624+0000","last_became_active":"2024-06-28T15:00:17.408972+0000","last_became_peered":"2024-06-28T15:00:17.408972+0000","last_unstale":"2024-06-28T16:15:06.991624+0000","last_undegraded":"2024-06-28T16:15:06.991624+0000","last_fullsized":"2024-06-28T16:15:06.991624+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'253","last_scrub_stamp":"2024-06-28T16:15:06.991213+0000","last_deep_scrub":"260'253","last_deep_scrub_stamp":"2024-06-28T16:15:06.991213+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.991213+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:27:33.581900+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":990,"num_read_kb":787,"num_write":391,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"260'346","reported_seq":1018,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:29.276908+0000","last_change":"2024-06-28T16:15:29.277022+0000","last_active":"2024-06-28T16:15:29.276908+0000","last_peered":"2024-06-28T16:15:29.276908+0000","last_clean":"2024-06-28T16:15:29.276908+0000","last_became_active":"2024-06-28T15:00:15.407552+0000","last_became_peered":"2024-06-28T15:00:15.407552+0000","last_unstale":"2024-06-28T16:15:29.276908+0000","last_undegraded":"2024-06-28T16:15:29.276908+0000","last_fullsized":"2024-06-28T16:15:29.276908+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'346","last_scrub_stamp":"2024-06-28T16:15:29.276709+0000","last_deep_scrub":"260'346","last_deep_scrub_stamp":"2024-06-28T16:15:29.276709+0000","last_clean_scrub_stamp":"2024-06-28T16:15:29.276709+0000","objects_scrubbed":9,"log_size":46,"log_dups_size":300,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:13:17.059573+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278750,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":513,"num_read_kb":23448,"num_write":1336,"num_write_kb":46171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"260'146","reported_seq":693,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.984342+0000","last_change":"2024-06-28T16:14:45.984342+0000","last_active":"2024-06-28T16:14:45.984342+0000","last_peered":"2024-06-28T16:14:45.984342+0000","last_clean":"2024-06-28T16:14:45.984342+0000","last_became_active":"2024-06-28T15:00:25.427734+0000","last_became_peered":"2024-06-28T15:00:25.427734+0000","last_unstale":"2024-06-28T16:14:45.984342+0000","last_undegraded":"2024-06-28T16:14:45.984342+0000","last_fullsized":"2024-06-28T16:14:45.984342+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'146","last_scrub_stamp":"2024-06-28T16:14:45.984045+0000","last_deep_scrub":"260'146","last_deep_scrub_stamp":"2024-06-28T16:14:45.984045+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.984045+0000","objects_scrubbed":4,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:59:27.363330+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":233,"num_read_kb":201,"num_write":347,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"260'214","reported_seq":832,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.190384+0000","last_change":"2024-06-28T16:15:16.190448+0000","last_active":"2024-06-28T16:15:16.190384+0000","last_peered":"2024-06-28T16:15:16.190384+0000","last_clean":"2024-06-28T16:15:16.190384+0000","last_became_active":"2024-06-28T15:00:17.382344+0000","last_became_peered":"2024-06-28T15:00:17.382344+0000","last_unstale":"2024-06-28T16:15:16.190384+0000","last_undegraded":"2024-06-28T16:15:16.190384+0000","last_fullsized":"2024-06-28T16:15:16.190384+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'214","last_scrub_stamp":"2024-06-28T16:15:16.190165+0000","last_deep_scrub":"260'214","last_deep_scrub_stamp":"2024-06-28T16:15:16.190165+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.190165+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":200,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:54:00.610916+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":869,"num_read_kb":692,"num_write":351,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"260'292","reported_seq":954,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:30.071405+0000","last_change":"2024-06-28T16:15:30.071591+0000","last_active":"2024-06-28T16:15:30.071405+0000","last_peered":"2024-06-28T16:15:30.071405+0000","last_clean":"2024-06-28T16:15:30.071405+0000","last_became_active":"2024-06-28T15:00:15.420739+0000","last_became_peered":"2024-06-28T15:00:15.420739+0000","last_unstale":"2024-06-28T16:15:30.071405+0000","last_undegraded":"2024-06-28T16:15:30.071405+0000","last_fullsized":"2024-06-28T16:15:30.071405+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'292","last_scrub_stamp":"2024-06-28T16:15:30.071192+0000","last_deep_scrub":"260'292","last_deep_scrub_stamp":"2024-06-28T16:15:30.071192+0000","last_clean_scrub_stamp":"2024-06-28T16:15:30.071192+0000","objects_scrubbed":9,"log_size":92,"log_dups_size":200,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:50:59.150622+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6022775,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":452,"num_read_kb":28797,"num_write":1102,"num_write_kb":30225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":512,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.665224+0000","last_change":"2024-06-28T16:15:06.665224+0000","last_active":"2024-06-28T16:15:06.665224+0000","last_peered":"2024-06-28T16:15:06.665224+0000","last_clean":"2024-06-28T16:15:06.665224+0000","last_became_active":"2024-06-28T15:00:19.400681+0000","last_became_peered":"2024-06-28T15:00:19.400681+0000","last_unstale":"2024-06-28T16:15:06.665224+0000","last_undegraded":"2024-06-28T16:15:06.665224+0000","last_fullsized":"2024-06-28T16:15:06.665224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:06.664972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:06.664972+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.664972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:48:36.340497+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"260'328","reported_seq":962,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:20.216253+0000","last_change":"2024-06-28T16:15:20.216253+0000","last_active":"2024-06-28T16:15:20.216253+0000","last_peered":"2024-06-28T16:15:20.216253+0000","last_clean":"2024-06-28T16:15:20.216253+0000","last_became_active":"2024-06-28T15:00:21.390609+0000","last_became_peered":"2024-06-28T15:00:21.390609+0000","last_unstale":"2024-06-28T16:15:20.216253+0000","last_undegraded":"2024-06-28T16:15:20.216253+0000","last_fullsized":"2024-06-28T16:15:20.216253+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'328","last_scrub_stamp":"2024-06-28T16:15:20.215969+0000","last_deep_scrub":"260'328","last_deep_scrub_stamp":"2024-06-28T16:15:20.215969+0000","last_clean_scrub_stamp":"2024-06-28T16:15:20.215969+0000","objects_scrubbed":4,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:17:23.688350+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42630,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":530,"num_read_kb":1085,"num_write":406,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.711989+0000","last_change":"2024-06-28T16:15:03.711989+0000","last_active":"2024-06-28T16:15:03.711989+0000","last_peered":"2024-06-28T16:15:03.711989+0000","last_clean":"2024-06-28T16:15:03.711989+0000","last_became_active":"2024-06-28T15:00:23.407521+0000","last_became_peered":"2024-06-28T15:00:23.407521+0000","last_unstale":"2024-06-28T16:15:03.711989+0000","last_undegraded":"2024-06-28T16:15:03.711989+0000","last_fullsized":"2024-06-28T16:15:03.711989+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:03.711749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:03.711749+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.711749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:10:02.315582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"260'2406","reported_seq":4282,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.649004+0000","last_change":"2024-06-28T16:14:46.649004+0000","last_active":"2024-06-28T16:14:46.649004+0000","last_peered":"2024-06-28T16:14:46.649004+0000","last_clean":"2024-06-28T16:14:46.649004+0000","last_became_active":"2024-06-28T15:00:25.440083+0000","last_became_peered":"2024-06-28T15:00:25.440083+0000","last_unstale":"2024-06-28T16:14:46.649004+0000","last_undegraded":"2024-06-28T16:14:46.649004+0000","last_fullsized":"2024-06-28T16:14:46.649004+0000","mapping_epoch":29,"log_start":"260'2300","ondisk_log_start":"260'2300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2406","last_scrub_stamp":"2024-06-28T16:14:46.648700+0000","last_deep_scrub":"260'2406","last_deep_scrub_stamp":"2024-06-28T16:14:46.648700+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.648700+0000","objects_scrubbed":4,"log_size":106,"log_dups_size":2300,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:06:59.187449+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5966,"num_read_kb":5973,"num_write":4800,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"260'4180","reported_seq":4834,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.506798+0000","last_change":"2024-06-28T16:14:58.506798+0000","last_active":"2024-06-28T16:14:58.506798+0000","last_peered":"2024-06-28T16:14:58.506798+0000","last_clean":"2024-06-28T16:14:58.506798+0000","last_became_active":"2024-06-28T15:00:17.390254+0000","last_became_peered":"2024-06-28T15:00:17.390254+0000","last_unstale":"2024-06-28T16:14:58.506798+0000","last_undegraded":"2024-06-28T16:14:58.506798+0000","last_fullsized":"2024-06-28T16:14:58.506798+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4180","last_scrub_stamp":"2024-06-28T16:14:58.506546+0000","last_deep_scrub":"260'4180","last_deep_scrub_stamp":"2024-06-28T16:14:58.506546+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.506546+0000","objects_scrubbed":1,"log_size":80,"log_dups_size":3000,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:17:13.473175+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14764,"num_read_kb":11968,"num_write":6274,"num_write_kb":3129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"260'343","reported_seq":1021,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:27.248179+0000","last_change":"2024-06-28T16:15:27.248334+0000","last_active":"2024-06-28T16:15:27.248179+0000","last_peered":"2024-06-28T16:15:27.248179+0000","last_clean":"2024-06-28T16:15:27.248179+0000","last_became_active":"2024-06-28T15:00:15.396724+0000","last_became_peered":"2024-06-28T15:00:15.396724+0000","last_unstale":"2024-06-28T16:15:27.248179+0000","last_undegraded":"2024-06-28T16:15:27.248179+0000","last_fullsized":"2024-06-28T16:15:27.248179+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'343","last_scrub_stamp":"2024-06-28T16:15:27.247957+0000","last_deep_scrub":"260'343","last_deep_scrub_stamp":"2024-06-28T16:15:27.247957+0000","last_clean_scrub_stamp":"2024-06-28T16:15:27.247957+0000","objects_scrubbed":6,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:47:50.870403+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262185,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":532,"num_read_kb":7737,"num_write":1235,"num_write_kb":8076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":517,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:23.373405+0000","last_change":"2024-06-28T16:15:23.373459+0000","last_active":"2024-06-28T16:15:23.373405+0000","last_peered":"2024-06-28T16:15:23.373405+0000","last_clean":"2024-06-28T16:15:23.373405+0000","last_became_active":"2024-06-28T15:00:19.390100+0000","last_became_peered":"2024-06-28T15:00:19.390100+0000","last_unstale":"2024-06-28T16:15:23.373405+0000","last_undegraded":"2024-06-28T16:15:23.373405+0000","last_fullsized":"2024-06-28T16:15:23.373405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:23.373200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:23.373200+0000","last_clean_scrub_stamp":"2024-06-28T16:15:23.373200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:31:49.559018+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"260'127","reported_seq":676,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.898682+0000","last_change":"2024-06-28T16:14:57.898682+0000","last_active":"2024-06-28T16:14:57.898682+0000","last_peered":"2024-06-28T16:14:57.898682+0000","last_clean":"2024-06-28T16:14:57.898682+0000","last_became_active":"2024-06-28T15:00:21.374284+0000","last_became_peered":"2024-06-28T15:00:21.374284+0000","last_unstale":"2024-06-28T16:14:57.898682+0000","last_undegraded":"2024-06-28T16:14:57.898682+0000","last_fullsized":"2024-06-28T16:14:57.898682+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'127","last_scrub_stamp":"2024-06-28T16:14:57.898396+0000","last_deep_scrub":"260'127","last_deep_scrub_stamp":"2024-06-28T16:14:57.898396+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.898396+0000","objects_scrubbed":4,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:35:13.361283+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":127,"num_read_kb":120,"num_write":90,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.938366+0000","last_change":"2024-06-28T16:14:50.938366+0000","last_active":"2024-06-28T16:14:50.938366+0000","last_peered":"2024-06-28T16:14:50.938366+0000","last_clean":"2024-06-28T16:14:50.938366+0000","last_became_active":"2024-06-28T15:00:23.399147+0000","last_became_peered":"2024-06-28T15:00:23.399147+0000","last_unstale":"2024-06-28T16:14:50.938366+0000","last_undegraded":"2024-06-28T16:14:50.938366+0000","last_fullsized":"2024-06-28T16:14:50.938366+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:50.938091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:50.938091+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.938091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:31:29.176572+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"260'127","reported_seq":663,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.070346+0000","last_change":"2024-06-28T16:14:51.070346+0000","last_active":"2024-06-28T16:14:51.070346+0000","last_peered":"2024-06-28T16:14:51.070346+0000","last_clean":"2024-06-28T16:14:51.070346+0000","last_became_active":"2024-06-28T15:00:25.432898+0000","last_became_peered":"2024-06-28T15:00:25.432898+0000","last_unstale":"2024-06-28T16:14:51.070346+0000","last_undegraded":"2024-06-28T16:14:51.070346+0000","last_fullsized":"2024-06-28T16:14:51.070346+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'127","last_scrub_stamp":"2024-06-28T16:14:51.070081+0000","last_deep_scrub":"260'127","last_deep_scrub_stamp":"2024-06-28T16:14:51.070081+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.070081+0000","objects_scrubbed":4,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:05:59.673050+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":165,"num_write":302,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"260'155","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.296338+0000","last_change":"2024-06-28T16:15:13.296338+0000","last_active":"2024-06-28T16:15:13.296338+0000","last_peered":"2024-06-28T16:15:13.296338+0000","last_clean":"2024-06-28T16:15:13.296338+0000","last_became_active":"2024-06-28T15:00:17.384613+0000","last_became_peered":"2024-06-28T15:00:17.384613+0000","last_unstale":"2024-06-28T16:15:13.296338+0000","last_undegraded":"2024-06-28T16:15:13.296338+0000","last_fullsized":"2024-06-28T16:15:13.296338+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'155","last_scrub_stamp":"2024-06-28T16:15:13.296050+0000","last_deep_scrub":"260'155","last_deep_scrub_stamp":"2024-06-28T16:15:13.296050+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.296050+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:05:33.691767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":630,"num_read_kb":508,"num_write":240,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"260'353","reported_seq":1011,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.887613+0000","last_change":"2024-06-28T16:15:17.887833+0000","last_active":"2024-06-28T16:15:17.887613+0000","last_peered":"2024-06-28T16:15:17.887613+0000","last_clean":"2024-06-28T16:15:17.887613+0000","last_became_active":"2024-06-28T15:00:15.428415+0000","last_became_peered":"2024-06-28T15:00:15.428415+0000","last_unstale":"2024-06-28T16:15:17.887613+0000","last_undegraded":"2024-06-28T16:15:17.887613+0000","last_fullsized":"2024-06-28T16:15:17.887613+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'353","last_scrub_stamp":"2024-06-28T16:15:17.887368+0000","last_deep_scrub":"260'353","last_deep_scrub_stamp":"2024-06-28T16:15:17.887368+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.887368+0000","objects_scrubbed":10,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:26:36.084273+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16055067,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":496,"num_read_kb":15197,"num_write":1261,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":516,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.228639+0000","last_change":"2024-06-28T16:15:19.228690+0000","last_active":"2024-06-28T16:15:19.228639+0000","last_peered":"2024-06-28T16:15:19.228639+0000","last_clean":"2024-06-28T16:15:19.228639+0000","last_became_active":"2024-06-28T15:00:19.372840+0000","last_became_peered":"2024-06-28T15:00:19.372840+0000","last_unstale":"2024-06-28T16:15:19.228639+0000","last_undegraded":"2024-06-28T16:15:19.228639+0000","last_fullsized":"2024-06-28T16:15:19.228639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:19.228437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:19.228437+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.228437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:28:55.169806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"260'128","reported_seq":701,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.103625+0000","last_change":"2024-06-28T16:14:45.103625+0000","last_active":"2024-06-28T16:14:45.103625+0000","last_peered":"2024-06-28T16:14:45.103625+0000","last_clean":"2024-06-28T16:14:45.103625+0000","last_became_active":"2024-06-28T15:00:21.377203+0000","last_became_peered":"2024-06-28T15:00:21.377203+0000","last_unstale":"2024-06-28T16:14:45.103625+0000","last_undegraded":"2024-06-28T16:14:45.103625+0000","last_fullsized":"2024-06-28T16:14:45.103625+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'128","last_scrub_stamp":"2024-06-28T16:14:45.103227+0000","last_deep_scrub":"260'128","last_deep_scrub_stamp":"2024-06-28T16:14:45.103227+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.103227+0000","objects_scrubbed":5,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:36:43.901132+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"260'522","reported_seq":4637,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.759185+0000","last_change":"2024-06-28T16:15:02.759185+0000","last_active":"2024-06-28T16:15:02.759185+0000","last_peered":"2024-06-28T16:15:02.759185+0000","last_clean":"2024-06-28T16:15:02.759185+0000","last_became_active":"2024-06-28T15:00:23.417490+0000","last_became_peered":"2024-06-28T15:00:23.417490+0000","last_unstale":"2024-06-28T16:15:02.759185+0000","last_undegraded":"2024-06-28T16:15:02.759185+0000","last_fullsized":"2024-06-28T16:15:02.759185+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'522","last_scrub_stamp":"2024-06-28T16:15:02.758942+0000","last_deep_scrub":"260'522","last_deep_scrub_stamp":"2024-06-28T16:15:02.758942+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.758942+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":500,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:33:07.422217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"260'204","reported_seq":1023,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.919351+0000","last_change":"2024-06-28T16:14:42.919351+0000","last_active":"2024-06-28T16:14:42.919351+0000","last_peered":"2024-06-28T16:14:42.919351+0000","last_clean":"2024-06-28T16:14:42.919351+0000","last_became_active":"2024-06-28T15:00:25.555670+0000","last_became_peered":"2024-06-28T15:00:25.555670+0000","last_unstale":"2024-06-28T16:14:42.919351+0000","last_undegraded":"2024-06-28T16:14:42.919351+0000","last_fullsized":"2024-06-28T16:14:42.919351+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'204","last_scrub_stamp":"2024-06-28T16:14:42.918953+0000","last_deep_scrub":"260'204","last_deep_scrub_stamp":"2024-06-28T16:14:42.918953+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.918953+0000","objects_scrubbed":3,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:34:15.091552+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":557,"num_read_kb":555,"num_write":448,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"260'84","reported_seq":681,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.159682+0000","last_change":"2024-06-28T16:15:15.159682+0000","last_active":"2024-06-28T16:15:15.159682+0000","last_peered":"2024-06-28T16:15:15.159682+0000","last_clean":"2024-06-28T16:15:15.159682+0000","last_became_active":"2024-06-28T15:00:17.389986+0000","last_became_peered":"2024-06-28T15:00:17.389986+0000","last_unstale":"2024-06-28T16:15:15.159682+0000","last_undegraded":"2024-06-28T16:15:15.159682+0000","last_fullsized":"2024-06-28T16:15:15.159682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'84","last_scrub_stamp":"2024-06-28T16:15:15.159426+0000","last_deep_scrub":"260'84","last_deep_scrub_stamp":"2024-06-28T16:15:15.159426+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.159426+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:34:57.998551+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":369,"num_read_kb":305,"num_write":130,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"260'288","reported_seq":932,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.384197+0000","last_change":"2024-06-28T16:15:09.384387+0000","last_active":"2024-06-28T16:15:09.384197+0000","last_peered":"2024-06-28T16:15:09.384197+0000","last_clean":"2024-06-28T16:15:09.384197+0000","last_became_active":"2024-06-28T15:00:15.411380+0000","last_became_peered":"2024-06-28T15:00:15.411380+0000","last_unstale":"2024-06-28T16:15:09.384197+0000","last_undegraded":"2024-06-28T16:15:09.384197+0000","last_fullsized":"2024-06-28T16:15:09.384197+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'288","last_scrub_stamp":"2024-06-28T16:15:09.383997+0000","last_deep_scrub":"260'288","last_deep_scrub_stamp":"2024-06-28T16:15:09.383997+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.383997+0000","objects_scrubbed":7,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:51:32.793598+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262217,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":24351,"num_write":1142,"num_write_kb":34218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"260'4","reported_seq":841,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.683401+0000","last_change":"2024-06-28T16:15:10.683401+0000","last_active":"2024-06-28T16:15:10.683401+0000","last_peered":"2024-06-28T16:15:10.683401+0000","last_clean":"2024-06-28T16:15:10.683401+0000","last_became_active":"2024-06-28T15:00:19.403623+0000","last_became_peered":"2024-06-28T15:00:19.403623+0000","last_unstale":"2024-06-28T16:15:10.683401+0000","last_undegraded":"2024-06-28T16:15:10.683401+0000","last_fullsized":"2024-06-28T16:15:10.683401+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:15:10.683144+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:15:10.683144+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.683144+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:43:58.420576+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":496,"num_read_kb":494,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"260'255","reported_seq":847,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.728156+0000","last_change":"2024-06-28T16:15:05.728156+0000","last_active":"2024-06-28T16:15:05.728156+0000","last_peered":"2024-06-28T16:15:05.728156+0000","last_clean":"2024-06-28T16:15:05.728156+0000","last_became_active":"2024-06-28T15:00:21.483633+0000","last_became_peered":"2024-06-28T15:00:21.483633+0000","last_unstale":"2024-06-28T16:15:05.728156+0000","last_undegraded":"2024-06-28T16:15:05.728156+0000","last_fullsized":"2024-06-28T16:15:05.728156+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'255","last_scrub_stamp":"2024-06-28T16:15:05.727852+0000","last_deep_scrub":"260'255","last_deep_scrub_stamp":"2024-06-28T16:15:05.727852+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.727852+0000","objects_scrubbed":4,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:54.739144+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8831,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":873,"num_write":310,"num_write_kb":224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"260'537","reported_seq":2249,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.225909+0000","last_change":"2024-06-28T16:15:03.225909+0000","last_active":"2024-06-28T16:15:03.225909+0000","last_peered":"2024-06-28T16:15:03.225909+0000","last_clean":"2024-06-28T16:15:03.225909+0000","last_became_active":"2024-06-28T15:00:23.396923+0000","last_became_peered":"2024-06-28T15:00:23.396923+0000","last_unstale":"2024-06-28T16:15:03.225909+0000","last_undegraded":"2024-06-28T16:15:03.225909+0000","last_fullsized":"2024-06-28T16:15:03.225909+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'537","last_scrub_stamp":"2024-06-28T16:15:03.225642+0000","last_deep_scrub":"260'537","last_deep_scrub_stamp":"2024-06-28T16:15:03.225642+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.225642+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:40:58.110463+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"260'415","reported_seq":1102,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.654488+0000","last_change":"2024-06-28T16:15:05.654568+0000","last_active":"2024-06-28T16:15:05.654488+0000","last_peered":"2024-06-28T16:15:05.654488+0000","last_clean":"2024-06-28T16:15:05.654488+0000","last_became_active":"2024-06-28T15:00:17.376864+0000","last_became_peered":"2024-06-28T15:00:17.376864+0000","last_unstale":"2024-06-28T16:15:05.654488+0000","last_undegraded":"2024-06-28T16:15:05.654488+0000","last_fullsized":"2024-06-28T16:15:05.654488+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'415","last_scrub_stamp":"2024-06-28T16:15:05.654289+0000","last_deep_scrub":"260'415","last_deep_scrub_stamp":"2024-06-28T16:15:05.654289+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.654289+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":400,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:26:21.223890+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1583,"num_read_kb":1214,"num_write":634,"num_write_kb":357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"260'320","reported_seq":989,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:28.034442+0000","last_change":"2024-06-28T16:15:28.034567+0000","last_active":"2024-06-28T16:15:28.034442+0000","last_peered":"2024-06-28T16:15:28.034442+0000","last_clean":"2024-06-28T16:15:28.034442+0000","last_became_active":"2024-06-28T15:00:15.415163+0000","last_became_peered":"2024-06-28T15:00:15.415163+0000","last_unstale":"2024-06-28T16:15:28.034442+0000","last_undegraded":"2024-06-28T16:15:28.034442+0000","last_fullsized":"2024-06-28T16:15:28.034442+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'320","last_scrub_stamp":"2024-06-28T16:15:28.034253+0000","last_deep_scrub":"260'320","last_deep_scrub_stamp":"2024-06-28T16:15:28.034253+0000","last_clean_scrub_stamp":"2024-06-28T16:15:28.034253+0000","objects_scrubbed":9,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:03:07.842410+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470118,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":518,"num_read_kb":26822,"num_write":1200,"num_write_kb":41274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.707792+0000","last_change":"2024-06-28T16:15:08.707848+0000","last_active":"2024-06-28T16:15:08.707792+0000","last_peered":"2024-06-28T16:15:08.707792+0000","last_clean":"2024-06-28T16:15:08.707792+0000","last_became_active":"2024-06-28T15:00:19.389624+0000","last_became_peered":"2024-06-28T15:00:19.389624+0000","last_unstale":"2024-06-28T16:15:08.707792+0000","last_undegraded":"2024-06-28T16:15:08.707792+0000","last_fullsized":"2024-06-28T16:15:08.707792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:08.707598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:08.707598+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.707598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:30.086517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"260'390","reported_seq":1043,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.597603+0000","last_change":"2024-06-28T16:14:54.597760+0000","last_active":"2024-06-28T16:14:54.597603+0000","last_peered":"2024-06-28T16:14:54.597603+0000","last_clean":"2024-06-28T16:14:54.597603+0000","last_became_active":"2024-06-28T15:00:21.481315+0000","last_became_peered":"2024-06-28T15:00:21.481315+0000","last_unstale":"2024-06-28T16:14:54.597603+0000","last_undegraded":"2024-06-28T16:14:54.597603+0000","last_fullsized":"2024-06-28T16:14:54.597603+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'390","last_scrub_stamp":"2024-06-28T16:14:54.597422+0000","last_deep_scrub":"260'390","last_deep_scrub_stamp":"2024-06-28T16:14:54.597422+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.597422+0000","objects_scrubbed":9,"log_size":90,"log_dups_size":300,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:52:31.536722+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46076,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":638,"num_read_kb":1277,"num_write":490,"num_write_kb":298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":511,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.079204+0000","last_change":"2024-06-28T16:15:03.079255+0000","last_active":"2024-06-28T16:15:03.079204+0000","last_peered":"2024-06-28T16:15:03.079204+0000","last_clean":"2024-06-28T16:15:03.079204+0000","last_became_active":"2024-06-28T15:00:23.413132+0000","last_became_peered":"2024-06-28T15:00:23.413132+0000","last_unstale":"2024-06-28T16:15:03.079204+0000","last_undegraded":"2024-06-28T16:15:03.079204+0000","last_fullsized":"2024-06-28T16:15:03.079204+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:03.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:03.079017+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:55:30.146201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"260'558","reported_seq":1386,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.685006+0000","last_change":"2024-06-28T16:14:45.685006+0000","last_active":"2024-06-28T16:14:45.685006+0000","last_peered":"2024-06-28T16:14:45.685006+0000","last_clean":"2024-06-28T16:14:45.685006+0000","last_became_active":"2024-06-28T15:00:25.441335+0000","last_became_peered":"2024-06-28T15:00:25.441335+0000","last_unstale":"2024-06-28T16:14:45.685006+0000","last_undegraded":"2024-06-28T16:14:45.685006+0000","last_fullsized":"2024-06-28T16:14:45.685006+0000","mapping_epoch":29,"log_start":"260'500","ondisk_log_start":"260'500","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'558","last_scrub_stamp":"2024-06-28T16:14:45.684713+0000","last_deep_scrub":"260'558","last_deep_scrub_stamp":"2024-06-28T16:14:45.684713+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.684713+0000","objects_scrubbed":3,"log_size":58,"log_dups_size":500,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:58:41.400605+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1277,"num_read_kb":1273,"num_write":1164,"num_write_kb":394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"260'53","reported_seq":641,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.944152+0000","last_change":"2024-06-28T16:15:02.944224+0000","last_active":"2024-06-28T16:15:02.944152+0000","last_peered":"2024-06-28T16:15:02.944152+0000","last_clean":"2024-06-28T16:15:02.944152+0000","last_became_active":"2024-06-28T15:00:17.408992+0000","last_became_peered":"2024-06-28T15:00:17.408992+0000","last_unstale":"2024-06-28T16:15:02.944152+0000","last_undegraded":"2024-06-28T16:15:02.944152+0000","last_fullsized":"2024-06-28T16:15:02.944152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'53","last_scrub_stamp":"2024-06-28T16:15:02.943961+0000","last_deep_scrub":"260'53","last_deep_scrub_stamp":"2024-06-28T16:15:02.943961+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.943961+0000","objects_scrubbed":1,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:58:16.183710+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":226,"num_read_kb":196,"num_write":79,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"260'326","reported_seq":954,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:25.988017+0000","last_change":"2024-06-28T16:15:25.988106+0000","last_active":"2024-06-28T16:15:25.988017+0000","last_peered":"2024-06-28T16:15:25.988017+0000","last_clean":"2024-06-28T16:15:25.988017+0000","last_became_active":"2024-06-28T15:00:15.402188+0000","last_became_peered":"2024-06-28T15:00:15.402188+0000","last_unstale":"2024-06-28T16:15:25.988017+0000","last_undegraded":"2024-06-28T16:15:25.988017+0000","last_fullsized":"2024-06-28T16:15:25.988017+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'326","last_scrub_stamp":"2024-06-28T16:15:25.987822+0000","last_deep_scrub":"260'326","last_deep_scrub_stamp":"2024-06-28T16:15:25.987822+0000","last_clean_scrub_stamp":"2024-06-28T16:15:25.987822+0000","objects_scrubbed":4,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:30:56.699781+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170347,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":7037,"num_write":1119,"num_write_kb":28395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":523,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.240452+0000","last_change":"2024-06-28T16:14:55.240505+0000","last_active":"2024-06-28T16:14:55.240452+0000","last_peered":"2024-06-28T16:14:55.240452+0000","last_clean":"2024-06-28T16:14:55.240452+0000","last_became_active":"2024-06-28T15:00:19.383002+0000","last_became_peered":"2024-06-28T15:00:19.383002+0000","last_unstale":"2024-06-28T16:14:55.240452+0000","last_undegraded":"2024-06-28T16:14:55.240452+0000","last_fullsized":"2024-06-28T16:14:55.240452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:55.240251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:55.240251+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.240251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:09:48.072213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"260'293","reported_seq":893,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:25.370055+0000","last_change":"2024-06-28T16:15:25.370136+0000","last_active":"2024-06-28T16:15:25.370055+0000","last_peered":"2024-06-28T16:15:25.370055+0000","last_clean":"2024-06-28T16:15:25.370055+0000","last_became_active":"2024-06-28T15:00:21.394580+0000","last_became_peered":"2024-06-28T15:00:21.394580+0000","last_unstale":"2024-06-28T16:15:25.370055+0000","last_undegraded":"2024-06-28T16:15:25.370055+0000","last_fullsized":"2024-06-28T16:15:25.370055+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'293","last_scrub_stamp":"2024-06-28T16:15:25.369862+0000","last_deep_scrub":"260'293","last_deep_scrub_stamp":"2024-06-28T16:15:25.369862+0000","last_clean_scrub_stamp":"2024-06-28T16:15:25.369862+0000","objects_scrubbed":5,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:45:33.914241+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":455,"num_read_kb":1052,"num_write":364,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":511,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.189470+0000","last_change":"2024-06-28T16:14:59.189522+0000","last_active":"2024-06-28T16:14:59.189470+0000","last_peered":"2024-06-28T16:14:59.189470+0000","last_clean":"2024-06-28T16:14:59.189470+0000","last_became_active":"2024-06-28T15:00:23.422567+0000","last_became_peered":"2024-06-28T15:00:23.422567+0000","last_unstale":"2024-06-28T16:14:59.189470+0000","last_undegraded":"2024-06-28T16:14:59.189470+0000","last_fullsized":"2024-06-28T16:14:59.189470+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:59.189280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:59.189280+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.189280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:25:01.077193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"260'96","reported_seq":598,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.496092+0000","last_change":"2024-06-28T16:15:00.496156+0000","last_active":"2024-06-28T16:15:00.496092+0000","last_peered":"2024-06-28T16:15:00.496092+0000","last_clean":"2024-06-28T16:15:00.496092+0000","last_became_active":"2024-06-28T15:00:25.426151+0000","last_became_peered":"2024-06-28T15:00:25.426151+0000","last_unstale":"2024-06-28T16:15:00.496092+0000","last_undegraded":"2024-06-28T16:15:00.496092+0000","last_fullsized":"2024-06-28T16:15:00.496092+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'96","last_scrub_stamp":"2024-06-28T16:15:00.495898+0000","last_deep_scrub":"260'96","last_deep_scrub_stamp":"2024-06-28T16:15:00.495898+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.495898+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:31:27.205808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":231,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"260'173","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.436343+0000","last_change":"2024-06-28T16:14:57.436511+0000","last_active":"2024-06-28T16:14:57.436343+0000","last_peered":"2024-06-28T16:14:57.436343+0000","last_clean":"2024-06-28T16:14:57.436343+0000","last_became_active":"2024-06-28T15:00:17.407031+0000","last_became_peered":"2024-06-28T15:00:17.407031+0000","last_unstale":"2024-06-28T16:14:57.436343+0000","last_undegraded":"2024-06-28T16:14:57.436343+0000","last_fullsized":"2024-06-28T16:14:57.436343+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'173","last_scrub_stamp":"2024-06-28T16:14:57.436114+0000","last_deep_scrub":"260'173","last_deep_scrub_stamp":"2024-06-28T16:14:57.436114+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.436114+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:11:15.521784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":538,"num_write":274,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"260'353","reported_seq":1047,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.817239+0000","last_change":"2024-06-28T16:15:19.817365+0000","last_active":"2024-06-28T16:15:19.817239+0000","last_peered":"2024-06-28T16:15:19.817239+0000","last_clean":"2024-06-28T16:15:19.817239+0000","last_became_active":"2024-06-28T15:00:15.427408+0000","last_became_peered":"2024-06-28T15:00:15.427408+0000","last_unstale":"2024-06-28T16:15:19.817239+0000","last_undegraded":"2024-06-28T16:15:19.817239+0000","last_fullsized":"2024-06-28T16:15:19.817239+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'353","last_scrub_stamp":"2024-06-28T16:15:19.817035+0000","last_deep_scrub":"260'353","last_deep_scrub_stamp":"2024-06-28T16:15:19.817035+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.817035+0000","objects_scrubbed":6,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:12:02.971410+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":574,"num_read_kb":16939,"num_write":1376,"num_write_kb":32636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":522,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.193279+0000","last_change":"2024-06-28T16:14:53.193339+0000","last_active":"2024-06-28T16:14:53.193279+0000","last_peered":"2024-06-28T16:14:53.193279+0000","last_clean":"2024-06-28T16:14:53.193279+0000","last_became_active":"2024-06-28T15:00:19.384172+0000","last_became_peered":"2024-06-28T15:00:19.384172+0000","last_unstale":"2024-06-28T16:14:53.193279+0000","last_undegraded":"2024-06-28T16:14:53.193279+0000","last_fullsized":"2024-06-28T16:14:53.193279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.193045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.193045+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.193045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:10:37.938602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"260'321","reported_seq":941,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.794562+0000","last_change":"2024-06-28T16:15:14.794653+0000","last_active":"2024-06-28T16:15:14.794562+0000","last_peered":"2024-06-28T16:15:14.794562+0000","last_clean":"2024-06-28T16:15:14.794562+0000","last_became_active":"2024-06-28T15:00:21.393608+0000","last_became_peered":"2024-06-28T15:00:21.393608+0000","last_unstale":"2024-06-28T16:15:14.794562+0000","last_undegraded":"2024-06-28T16:15:14.794562+0000","last_fullsized":"2024-06-28T16:15:14.794562+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'321","last_scrub_stamp":"2024-06-28T16:15:14.794352+0000","last_deep_scrub":"260'321","last_deep_scrub_stamp":"2024-06-28T16:15:14.794352+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.794352+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:23:46.118111+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42645,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":526,"num_read_kb":1122,"num_write":410,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"260'534","reported_seq":2275,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.097241+0000","last_change":"2024-06-28T16:14:57.097387+0000","last_active":"2024-06-28T16:14:57.097241+0000","last_peered":"2024-06-28T16:14:57.097241+0000","last_clean":"2024-06-28T16:14:57.097241+0000","last_became_active":"2024-06-28T15:00:23.413550+0000","last_became_peered":"2024-06-28T15:00:23.413550+0000","last_unstale":"2024-06-28T16:14:57.097241+0000","last_undegraded":"2024-06-28T16:14:57.097241+0000","last_fullsized":"2024-06-28T16:14:57.097241+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'534","last_scrub_stamp":"2024-06-28T16:14:57.096961+0000","last_deep_scrub":"260'534","last_deep_scrub_stamp":"2024-06-28T16:14:57.096961+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.096961+0000","objects_scrubbed":1,"log_size":34,"log_dups_size":500,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:11:19.198302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"260'147","reported_seq":684,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.064684+0000","last_change":"2024-06-28T16:14:52.064763+0000","last_active":"2024-06-28T16:14:52.064684+0000","last_peered":"2024-06-28T16:14:52.064684+0000","last_clean":"2024-06-28T16:14:52.064684+0000","last_became_active":"2024-06-28T15:00:25.431835+0000","last_became_peered":"2024-06-28T15:00:25.431835+0000","last_unstale":"2024-06-28T16:14:52.064684+0000","last_undegraded":"2024-06-28T16:14:52.064684+0000","last_fullsized":"2024-06-28T16:14:52.064684+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'147","last_scrub_stamp":"2024-06-28T16:14:52.064509+0000","last_deep_scrub":"260'147","last_deep_scrub_stamp":"2024-06-28T16:14:52.064509+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.064509+0000","objects_scrubbed":3,"log_size":47,"log_dups_size":100,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:45:10.678734+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1247,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":164,"num_write":265,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"260'4158","reported_seq":4773,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.323709+0000","last_change":"2024-06-28T16:15:10.323773+0000","last_active":"2024-06-28T16:15:10.323709+0000","last_peered":"2024-06-28T16:15:10.323709+0000","last_clean":"2024-06-28T16:15:10.323709+0000","last_became_active":"2024-06-28T15:00:17.399799+0000","last_became_peered":"2024-06-28T15:00:17.399799+0000","last_unstale":"2024-06-28T16:15:10.323709+0000","last_undegraded":"2024-06-28T16:15:10.323709+0000","last_fullsized":"2024-06-28T16:15:10.323709+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4158","last_scrub_stamp":"2024-06-28T16:15:10.323504+0000","last_deep_scrub":"260'4158","last_deep_scrub_stamp":"2024-06-28T16:15:10.323504+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.323504+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":3000,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:49:38.330274+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14678,"num_read_kb":11882,"num_write":6254,"num_write_kb":3121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"260'337","reported_seq":991,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:21.906283+0000","last_change":"2024-06-28T16:15:21.906562+0000","last_active":"2024-06-28T16:15:21.906283+0000","last_peered":"2024-06-28T16:15:21.906283+0000","last_clean":"2024-06-28T16:15:21.906283+0000","last_became_active":"2024-06-28T15:00:15.425690+0000","last_became_peered":"2024-06-28T15:00:15.425690+0000","last_unstale":"2024-06-28T16:15:21.906283+0000","last_undegraded":"2024-06-28T16:15:21.906283+0000","last_fullsized":"2024-06-28T16:15:21.906283+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'337","last_scrub_stamp":"2024-06-28T16:15:21.906079+0000","last_deep_scrub":"260'337","last_deep_scrub_stamp":"2024-06-28T16:15:21.906079+0000","last_clean_scrub_stamp":"2024-06-28T16:15:21.906079+0000","objects_scrubbed":8,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:19.541141+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079931,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":508,"num_read_kb":13664,"num_write":1274,"num_write_kb":26574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,1],"acting":[5,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":523,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.205322+0000","last_change":"2024-06-28T16:14:51.205380+0000","last_active":"2024-06-28T16:14:51.205322+0000","last_peered":"2024-06-28T16:14:51.205322+0000","last_clean":"2024-06-28T16:14:51.205322+0000","last_became_active":"2024-06-28T15:00:19.385189+0000","last_became_peered":"2024-06-28T15:00:19.385189+0000","last_unstale":"2024-06-28T16:14:51.205322+0000","last_undegraded":"2024-06-28T16:14:51.205322+0000","last_fullsized":"2024-06-28T16:14:51.205322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.205122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.205122+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.205122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:14:01.011784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"260'493","reported_seq":1158,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:22.135776+0000","last_change":"2024-06-28T16:15:22.135863+0000","last_active":"2024-06-28T16:15:22.135776+0000","last_peered":"2024-06-28T16:15:22.135776+0000","last_clean":"2024-06-28T16:15:22.135776+0000","last_became_active":"2024-06-28T15:00:21.397397+0000","last_became_peered":"2024-06-28T15:00:21.397397+0000","last_unstale":"2024-06-28T16:15:22.135776+0000","last_undegraded":"2024-06-28T16:15:22.135776+0000","last_fullsized":"2024-06-28T16:15:22.135776+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'493","last_scrub_stamp":"2024-06-28T16:15:22.135585+0000","last_deep_scrub":"260'493","last_deep_scrub_stamp":"2024-06-28T16:15:22.135585+0000","last_clean_scrub_stamp":"2024-06-28T16:15:22.135585+0000","objects_scrubbed":6,"log_size":93,"log_dups_size":400,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:55:12.323828+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92449,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":856,"num_read_kb":2091,"num_write":698,"num_write_kb":574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.129159+0000","last_change":"2024-06-28T16:14:40.129181+0000","last_active":"2024-06-28T16:14:40.129159+0000","last_peered":"2024-06-28T16:14:40.129159+0000","last_clean":"2024-06-28T16:14:40.129159+0000","last_became_active":"2024-06-28T15:00:23.416263+0000","last_became_peered":"2024-06-28T15:00:23.416263+0000","last_unstale":"2024-06-28T16:14:40.129159+0000","last_undegraded":"2024-06-28T16:14:40.129159+0000","last_fullsized":"2024-06-28T16:14:40.129159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:40.129070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:40.129070+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.129070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:20:56.813386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"260'117","reported_seq":644,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.256714+0000","last_change":"2024-06-28T16:15:15.256884+0000","last_active":"2024-06-28T16:15:15.256714+0000","last_peered":"2024-06-28T16:15:15.256714+0000","last_clean":"2024-06-28T16:15:15.256714+0000","last_became_active":"2024-06-28T15:00:25.422519+0000","last_became_peered":"2024-06-28T15:00:25.422519+0000","last_unstale":"2024-06-28T16:15:15.256714+0000","last_undegraded":"2024-06-28T16:15:15.256714+0000","last_fullsized":"2024-06-28T16:15:15.256714+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'117","last_scrub_stamp":"2024-06-28T16:15:15.256473+0000","last_deep_scrub":"260'117","last_deep_scrub_stamp":"2024-06-28T16:15:15.256473+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.256473+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:55:59.918202+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":118,"num_read_kb":116,"num_write":248,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"260'128","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.924442+0000","last_change":"2024-06-28T16:15:00.924534+0000","last_active":"2024-06-28T16:15:00.924442+0000","last_peered":"2024-06-28T16:15:00.924442+0000","last_clean":"2024-06-28T16:15:00.924442+0000","last_became_active":"2024-06-28T15:00:17.404204+0000","last_became_peered":"2024-06-28T15:00:17.404204+0000","last_unstale":"2024-06-28T16:15:00.924442+0000","last_undegraded":"2024-06-28T16:15:00.924442+0000","last_fullsized":"2024-06-28T16:15:00.924442+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'128","last_scrub_stamp":"2024-06-28T16:15:00.924255+0000","last_deep_scrub":"260'128","last_deep_scrub_stamp":"2024-06-28T16:15:00.924255+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.924255+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:33:46.108267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":562,"num_read_kb":476,"num_write":174,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"260'352","reported_seq":1068,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:31.283604+0000","last_change":"2024-06-28T16:15:31.283702+0000","last_active":"2024-06-28T16:15:31.283604+0000","last_peered":"2024-06-28T16:15:31.283604+0000","last_clean":"2024-06-28T16:15:31.283604+0000","last_became_active":"2024-06-28T15:00:15.402041+0000","last_became_peered":"2024-06-28T15:00:15.402041+0000","last_unstale":"2024-06-28T16:15:31.283604+0000","last_undegraded":"2024-06-28T16:15:31.283604+0000","last_fullsized":"2024-06-28T16:15:31.283604+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'352","last_scrub_stamp":"2024-06-28T16:15:31.283400+0000","last_deep_scrub":"260'352","last_deep_scrub_stamp":"2024-06-28T16:15:31.283400+0000","last_clean_scrub_stamp":"2024-06-28T16:15:31.283400+0000","objects_scrubbed":7,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:28:30.802247+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":603,"num_read_kb":26410,"num_write":1369,"num_write_kb":26391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":522,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.209018+0000","last_change":"2024-06-28T16:14:49.209075+0000","last_active":"2024-06-28T16:14:49.209018+0000","last_peered":"2024-06-28T16:14:49.209018+0000","last_clean":"2024-06-28T16:14:49.209018+0000","last_became_active":"2024-06-28T15:00:19.392773+0000","last_became_peered":"2024-06-28T15:00:19.392773+0000","last_unstale":"2024-06-28T16:14:49.209018+0000","last_undegraded":"2024-06-28T16:14:49.209018+0000","last_fullsized":"2024-06-28T16:14:49.209018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.208819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.208819+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.208819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:49:11.047171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"260'306","reported_seq":911,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.988613+0000","last_change":"2024-06-28T16:14:53.988691+0000","last_active":"2024-06-28T16:14:53.988613+0000","last_peered":"2024-06-28T16:14:53.988613+0000","last_clean":"2024-06-28T16:14:53.988613+0000","last_became_active":"2024-06-28T15:00:21.397842+0000","last_became_peered":"2024-06-28T15:00:21.397842+0000","last_unstale":"2024-06-28T16:14:53.988613+0000","last_undegraded":"2024-06-28T16:14:53.988613+0000","last_fullsized":"2024-06-28T16:14:53.988613+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'306","last_scrub_stamp":"2024-06-28T16:14:53.988406+0000","last_deep_scrub":"260'306","last_deep_scrub_stamp":"2024-06-28T16:14:53.988406+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.988406+0000","objects_scrubbed":6,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:16:38.325337+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44037,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":498,"num_read_kb":1063,"num_write":434,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15964,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":512,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.186008+0000","last_change":"2024-06-28T16:15:00.186120+0000","last_active":"2024-06-28T16:15:00.186008+0000","last_peered":"2024-06-28T16:15:00.186008+0000","last_clean":"2024-06-28T16:15:00.186008+0000","last_became_active":"2024-06-28T15:00:23.406122+0000","last_became_peered":"2024-06-28T15:00:23.406122+0000","last_unstale":"2024-06-28T16:15:00.186008+0000","last_undegraded":"2024-06-28T16:15:00.186008+0000","last_fullsized":"2024-06-28T16:15:00.186008+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:00.185778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:00.185778+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.185778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:16:27.337969+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"260'91","reported_seq":604,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.145305+0000","last_change":"2024-06-28T16:14:54.145374+0000","last_active":"2024-06-28T16:14:54.145305+0000","last_peered":"2024-06-28T16:14:54.145305+0000","last_clean":"2024-06-28T16:14:54.145305+0000","last_became_active":"2024-06-28T15:00:25.439346+0000","last_became_peered":"2024-06-28T15:00:25.439346+0000","last_unstale":"2024-06-28T16:14:54.145305+0000","last_undegraded":"2024-06-28T16:14:54.145305+0000","last_fullsized":"2024-06-28T16:14:54.145305+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'91","last_scrub_stamp":"2024-06-28T16:14:54.145105+0000","last_deep_scrub":"260'91","last_deep_scrub_stamp":"2024-06-28T16:14:54.145105+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.145105+0000","objects_scrubbed":2,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:54:30.494466+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":207,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"260'172","reported_seq":769,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:21.974854+0000","last_change":"2024-06-28T16:15:21.974919+0000","last_active":"2024-06-28T16:15:21.974854+0000","last_peered":"2024-06-28T16:15:21.974854+0000","last_clean":"2024-06-28T16:15:21.974854+0000","last_became_active":"2024-06-28T15:00:17.396048+0000","last_became_peered":"2024-06-28T15:00:17.396048+0000","last_unstale":"2024-06-28T16:15:21.974854+0000","last_undegraded":"2024-06-28T16:15:21.974854+0000","last_fullsized":"2024-06-28T16:15:21.974854+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'172","last_scrub_stamp":"2024-06-28T16:15:21.974637+0000","last_deep_scrub":"260'172","last_deep_scrub_stamp":"2024-06-28T16:15:21.974637+0000","last_clean_scrub_stamp":"2024-06-28T16:15:21.974637+0000","objects_scrubbed":1,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:11:35.817304+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":659,"num_read_kb":531,"num_write":250,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"260'376","reported_seq":1116,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.457498+0000","last_change":"2024-06-28T16:15:05.457664+0000","last_active":"2024-06-28T16:15:05.457498+0000","last_peered":"2024-06-28T16:15:05.457498+0000","last_clean":"2024-06-28T16:15:05.457498+0000","last_became_active":"2024-06-28T15:00:15.397256+0000","last_became_peered":"2024-06-28T15:00:15.397256+0000","last_unstale":"2024-06-28T16:15:05.457498+0000","last_undegraded":"2024-06-28T16:15:05.457498+0000","last_fullsized":"2024-06-28T16:15:05.457498+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'376","last_scrub_stamp":"2024-06-28T16:15:05.457272+0000","last_deep_scrub":"260'376","last_deep_scrub_stamp":"2024-06-28T16:15:05.457272+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.457272+0000","objects_scrubbed":7,"log_size":76,"log_dups_size":300,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:17:53.364489+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2674362,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":727,"num_read_kb":7345,"num_write":1214,"num_write_kb":31713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":519,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.945272+0000","last_change":"2024-06-28T16:14:53.945393+0000","last_active":"2024-06-28T16:14:53.945272+0000","last_peered":"2024-06-28T16:14:53.945272+0000","last_clean":"2024-06-28T16:14:53.945272+0000","last_became_active":"2024-06-28T15:00:19.379281+0000","last_became_peered":"2024-06-28T15:00:19.379281+0000","last_unstale":"2024-06-28T16:14:53.945272+0000","last_undegraded":"2024-06-28T16:14:53.945272+0000","last_fullsized":"2024-06-28T16:14:53.945272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.945027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.945027+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.945027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:26:10.510879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"260'356","reported_seq":984,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.142072+0000","last_change":"2024-06-28T16:14:43.142072+0000","last_active":"2024-06-28T16:14:43.142072+0000","last_peered":"2024-06-28T16:14:43.142072+0000","last_clean":"2024-06-28T16:14:43.142072+0000","last_became_active":"2024-06-28T15:00:21.395892+0000","last_became_peered":"2024-06-28T15:00:21.395892+0000","last_unstale":"2024-06-28T16:14:43.142072+0000","last_undegraded":"2024-06-28T16:14:43.142072+0000","last_fullsized":"2024-06-28T16:14:43.142072+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'356","last_scrub_stamp":"2024-06-28T16:14:43.141802+0000","last_deep_scrub":"260'356","last_deep_scrub_stamp":"2024-06-28T16:14:43.141802+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.141802+0000","objects_scrubbed":7,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:06:31.916429+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48931,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":579,"num_read_kb":1282,"num_write":462,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"260'501","reported_seq":2313,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.168638+0000","last_change":"2024-06-28T16:14:41.168660+0000","last_active":"2024-06-28T16:14:41.168638+0000","last_peered":"2024-06-28T16:14:41.168638+0000","last_clean":"2024-06-28T16:14:41.168638+0000","last_became_active":"2024-06-28T15:00:23.415437+0000","last_became_peered":"2024-06-28T15:00:23.415437+0000","last_unstale":"2024-06-28T16:14:41.168638+0000","last_undegraded":"2024-06-28T16:14:41.168638+0000","last_fullsized":"2024-06-28T16:14:41.168638+0000","mapping_epoch":27,"log_start":"257'400","ondisk_log_start":"257'400","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'501","last_scrub_stamp":"2024-06-28T16:14:41.168573+0000","last_deep_scrub":"260'501","last_deep_scrub_stamp":"2024-06-28T16:14:41.168573+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.168573+0000","objects_scrubbed":1,"log_size":101,"log_dups_size":400,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:22:09.865296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"260'102","reported_seq":621,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.002982+0000","last_change":"2024-06-28T16:14:50.002982+0000","last_active":"2024-06-28T16:14:50.002982+0000","last_peered":"2024-06-28T16:14:50.002982+0000","last_clean":"2024-06-28T16:14:50.002982+0000","last_became_active":"2024-06-28T15:00:25.425135+0000","last_became_peered":"2024-06-28T15:00:25.425135+0000","last_unstale":"2024-06-28T16:14:50.002982+0000","last_undegraded":"2024-06-28T16:14:50.002982+0000","last_fullsized":"2024-06-28T16:14:50.002982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'102","last_scrub_stamp":"2024-06-28T16:14:50.002732+0000","last_deep_scrub":"260'102","last_deep_scrub_stamp":"2024-06-28T16:14:50.002732+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.002732+0000","objects_scrubbed":3,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:56:55.357278+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":245,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"260'160","reported_seq":752,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.949730+0000","last_change":"2024-06-28T16:15:03.949795+0000","last_active":"2024-06-28T16:15:03.949730+0000","last_peered":"2024-06-28T16:15:03.949730+0000","last_clean":"2024-06-28T16:15:03.949730+0000","last_became_active":"2024-06-28T15:00:17.402024+0000","last_became_peered":"2024-06-28T15:00:17.402024+0000","last_unstale":"2024-06-28T16:15:03.949730+0000","last_undegraded":"2024-06-28T16:15:03.949730+0000","last_fullsized":"2024-06-28T16:15:03.949730+0000","mapping_epoch":21,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'160","last_scrub_stamp":"2024-06-28T16:15:03.949540+0000","last_deep_scrub":"260'160","last_deep_scrub_stamp":"2024-06-28T16:15:03.949540+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.949540+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":100,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:07:39.846023+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":645,"num_read_kb":505,"num_write":259,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"260'393","reported_seq":1076,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.969253+0000","last_change":"2024-06-28T16:15:14.969434+0000","last_active":"2024-06-28T16:15:14.969253+0000","last_peered":"2024-06-28T16:15:14.969253+0000","last_clean":"2024-06-28T16:15:14.969253+0000","last_became_active":"2024-06-28T15:00:15.388365+0000","last_became_peered":"2024-06-28T15:00:15.388365+0000","last_unstale":"2024-06-28T16:15:14.969253+0000","last_undegraded":"2024-06-28T16:15:14.969253+0000","last_fullsized":"2024-06-28T16:15:14.969253+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'393","last_scrub_stamp":"2024-06-28T16:15:14.969046+0000","last_deep_scrub":"260'393","last_deep_scrub_stamp":"2024-06-28T16:15:14.969046+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.969046+0000","objects_scrubbed":8,"log_size":93,"log_dups_size":300,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:43:46.202161+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1604260,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":661,"num_read_kb":7226,"num_write":1423,"num_write_kb":30573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"260'4","reported_seq":1166,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.680759+0000","last_change":"2024-06-28T16:15:08.680851+0000","last_active":"2024-06-28T16:15:08.680759+0000","last_peered":"2024-06-28T16:15:08.680759+0000","last_clean":"2024-06-28T16:15:08.680759+0000","last_became_active":"2024-06-28T15:00:19.397863+0000","last_became_peered":"2024-06-28T15:00:19.397863+0000","last_unstale":"2024-06-28T16:15:08.680759+0000","last_undegraded":"2024-06-28T16:15:08.680759+0000","last_fullsized":"2024-06-28T16:15:08.680759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:15:08.680583+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:15:08.680583+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.680583+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:28:22.747616+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":648,"num_read_kb":648,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"260'118","reported_seq":672,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.248332+0000","last_change":"2024-06-28T16:15:04.248438+0000","last_active":"2024-06-28T16:15:04.248332+0000","last_peered":"2024-06-28T16:15:04.248332+0000","last_clean":"2024-06-28T16:15:04.248332+0000","last_became_active":"2024-06-28T15:00:21.483679+0000","last_became_peered":"2024-06-28T15:00:21.483679+0000","last_unstale":"2024-06-28T16:15:04.248332+0000","last_undegraded":"2024-06-28T16:15:04.248332+0000","last_fullsized":"2024-06-28T16:15:04.248332+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'118","last_scrub_stamp":"2024-06-28T16:15:04.248130+0000","last_deep_scrub":"260'118","last_deep_scrub_stamp":"2024-06-28T16:15:04.248130+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.248130+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:12:41.540178+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"260'515","reported_seq":4272,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.451884+0000","last_change":"2024-06-28T16:14:58.451949+0000","last_active":"2024-06-28T16:14:58.451884+0000","last_peered":"2024-06-28T16:14:58.451884+0000","last_clean":"2024-06-28T16:14:58.451884+0000","last_became_active":"2024-06-28T15:00:23.419182+0000","last_became_peered":"2024-06-28T15:00:23.419182+0000","last_unstale":"2024-06-28T16:14:58.451884+0000","last_undegraded":"2024-06-28T16:14:58.451884+0000","last_fullsized":"2024-06-28T16:14:58.451884+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'515","last_scrub_stamp":"2024-06-28T16:14:58.451684+0000","last_deep_scrub":"260'515","last_deep_scrub_stamp":"2024-06-28T16:14:58.451684+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.451684+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:15.191208+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"260'163","reported_seq":686,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.941473+0000","last_change":"2024-06-28T16:14:46.941473+0000","last_active":"2024-06-28T16:14:46.941473+0000","last_peered":"2024-06-28T16:14:46.941473+0000","last_clean":"2024-06-28T16:14:46.941473+0000","last_became_active":"2024-06-28T15:00:25.425690+0000","last_became_peered":"2024-06-28T15:00:25.425690+0000","last_unstale":"2024-06-28T16:14:46.941473+0000","last_undegraded":"2024-06-28T16:14:46.941473+0000","last_fullsized":"2024-06-28T16:14:46.941473+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'163","last_scrub_stamp":"2024-06-28T16:14:46.941194+0000","last_deep_scrub":"260'163","last_deep_scrub_stamp":"2024-06-28T16:14:46.941194+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.941194+0000","objects_scrubbed":4,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:28:38.329545+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":152,"num_write":381,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"260'74","reported_seq":629,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.650256+0000","last_change":"2024-06-28T16:14:42.650256+0000","last_active":"2024-06-28T16:14:42.650256+0000","last_peered":"2024-06-28T16:14:42.650256+0000","last_clean":"2024-06-28T16:14:42.650256+0000","last_became_active":"2024-06-28T15:00:59.799823+0000","last_became_peered":"2024-06-28T15:00:59.799823+0000","last_unstale":"2024-06-28T16:14:42.650256+0000","last_undegraded":"2024-06-28T16:14:42.650256+0000","last_fullsized":"2024-06-28T16:14:42.650256+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'74","last_scrub_stamp":"2024-06-28T16:14:42.650003+0000","last_deep_scrub":"260'74","last_deep_scrub_stamp":"2024-06-28T16:14:42.650003+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.650003+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:28:32.949699+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":247,"num_write":103,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"260'351","reported_seq":1090,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:34.236597+0000","last_change":"2024-06-28T16:15:34.236827+0000","last_active":"2024-06-28T16:15:34.236597+0000","last_peered":"2024-06-28T16:15:34.236597+0000","last_clean":"2024-06-28T16:15:34.236597+0000","last_became_active":"2024-06-28T15:00:15.402782+0000","last_became_peered":"2024-06-28T15:00:15.402782+0000","last_unstale":"2024-06-28T16:15:34.236597+0000","last_undegraded":"2024-06-28T16:15:34.236597+0000","last_fullsized":"2024-06-28T16:15:34.236597+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'351","last_scrub_stamp":"2024-06-28T16:15:34.236400+0000","last_deep_scrub":"260'351","last_deep_scrub_stamp":"2024-06-28T16:15:34.236400+0000","last_clean_scrub_stamp":"2024-06-28T16:15:34.236400+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":300,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:18:57.699390+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262233,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":541,"num_read_kb":3603,"num_write":1164,"num_write_kb":36556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:22.954316+0000","last_change":"2024-06-28T16:15:22.954419+0000","last_active":"2024-06-28T16:15:22.954316+0000","last_peered":"2024-06-28T16:15:22.954316+0000","last_clean":"2024-06-28T16:15:22.954316+0000","last_became_active":"2024-06-28T15:00:19.405083+0000","last_became_peered":"2024-06-28T15:00:19.405083+0000","last_unstale":"2024-06-28T16:15:22.954316+0000","last_undegraded":"2024-06-28T16:15:22.954316+0000","last_fullsized":"2024-06-28T16:15:22.954316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:22.954093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:22.954093+0000","last_clean_scrub_stamp":"2024-06-28T16:15:22.954093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:26:57.368299+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"260'437","reported_seq":1081,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.565170+0000","last_change":"2024-06-28T16:14:57.565257+0000","last_active":"2024-06-28T16:14:57.565170+0000","last_peered":"2024-06-28T16:14:57.565170+0000","last_clean":"2024-06-28T16:14:57.565170+0000","last_became_active":"2024-06-28T15:00:21.392393+0000","last_became_peered":"2024-06-28T15:00:21.392393+0000","last_unstale":"2024-06-28T16:14:57.565170+0000","last_undegraded":"2024-06-28T16:14:57.565170+0000","last_fullsized":"2024-06-28T16:14:57.565170+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'437","last_scrub_stamp":"2024-06-28T16:14:57.564983+0000","last_deep_scrub":"260'437","last_deep_scrub_stamp":"2024-06-28T16:14:57.564983+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.564983+0000","objects_scrubbed":9,"log_size":37,"log_dups_size":400,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:47:46.699794+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47467,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1282,"num_read_kb":1893,"num_write":1822,"num_write_kb":1648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":479870,"num_omap_keys":1112,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.323011+0000","last_change":"2024-06-28T16:14:54.323067+0000","last_active":"2024-06-28T16:14:54.323011+0000","last_peered":"2024-06-28T16:14:54.323011+0000","last_clean":"2024-06-28T16:14:54.323011+0000","last_became_active":"2024-06-28T15:00:23.415497+0000","last_became_peered":"2024-06-28T15:00:23.415497+0000","last_unstale":"2024-06-28T16:14:54.323011+0000","last_undegraded":"2024-06-28T16:14:54.323011+0000","last_fullsized":"2024-06-28T16:14:54.323011+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:54.322811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:54.322811+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.322811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:40:56.629420+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"260'136","reported_seq":652,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.029609+0000","last_change":"2024-06-28T16:14:48.029609+0000","last_active":"2024-06-28T16:14:48.029609+0000","last_peered":"2024-06-28T16:14:48.029609+0000","last_clean":"2024-06-28T16:14:48.029609+0000","last_became_active":"2024-06-28T15:00:25.439527+0000","last_became_peered":"2024-06-28T15:00:25.439527+0000","last_unstale":"2024-06-28T16:14:48.029609+0000","last_undegraded":"2024-06-28T16:14:48.029609+0000","last_fullsized":"2024-06-28T16:14:48.029609+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'136","last_scrub_stamp":"2024-06-28T16:14:48.029357+0000","last_deep_scrub":"260'136","last_deep_scrub_stamp":"2024-06-28T16:14:48.029357+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.029357+0000","objects_scrubbed":3,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:09:52.677657+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":127,"num_write":301,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"260'157","reported_seq":784,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.366787+0000","last_change":"2024-06-28T16:14:55.366787+0000","last_active":"2024-06-28T16:14:55.366787+0000","last_peered":"2024-06-28T16:14:55.366787+0000","last_clean":"2024-06-28T16:14:55.366787+0000","last_became_active":"2024-06-28T15:00:17.414537+0000","last_became_peered":"2024-06-28T15:00:17.414537+0000","last_unstale":"2024-06-28T16:14:55.366787+0000","last_undegraded":"2024-06-28T16:14:55.366787+0000","last_fullsized":"2024-06-28T16:14:55.366787+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'157","last_scrub_stamp":"2024-06-28T16:14:55.366527+0000","last_deep_scrub":"260'157","last_deep_scrub_stamp":"2024-06-28T16:14:55.366527+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.366527+0000","objects_scrubbed":1,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:51:04.103193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":522,"num_write":223,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"260'74","reported_seq":627,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:23.942146+0000","last_change":"2024-06-28T16:15:23.942219+0000","last_active":"2024-06-28T16:15:23.942146+0000","last_peered":"2024-06-28T16:15:23.942146+0000","last_clean":"2024-06-28T16:15:23.942146+0000","last_became_active":"2024-06-28T15:00:06.302725+0000","last_became_peered":"2024-06-28T15:00:06.302725+0000","last_unstale":"2024-06-28T16:15:23.942146+0000","last_undegraded":"2024-06-28T16:15:23.942146+0000","last_fullsized":"2024-06-28T16:15:23.942146+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'74","last_scrub_stamp":"2024-06-28T16:15:23.941955+0000","last_deep_scrub":"260'74","last_deep_scrub_stamp":"2024-06-28T16:15:23.941955+0000","last_clean_scrub_stamp":"2024-06-28T16:15:23.941955+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:57:32.698515+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":534,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.800619+0000","last_change":"2024-06-28T16:15:15.800750+0000","last_active":"2024-06-28T16:15:15.800619+0000","last_peered":"2024-06-28T16:15:15.800619+0000","last_clean":"2024-06-28T16:15:15.800619+0000","last_became_active":"2024-06-28T15:00:06.284218+0000","last_became_peered":"2024-06-28T15:00:06.284218+0000","last_unstale":"2024-06-28T16:15:15.800619+0000","last_undegraded":"2024-06-28T16:15:15.800619+0000","last_fullsized":"2024-06-28T16:15:15.800619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:15.800331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:15.800331+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.800331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:32:57.118235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"260'320","reported_seq":1125,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.110204+0000","last_change":"2024-06-28T16:14:48.110204+0000","last_active":"2024-06-28T16:14:48.110204+0000","last_peered":"2024-06-28T16:14:48.110204+0000","last_clean":"2024-06-28T16:14:48.110204+0000","last_became_active":"2024-06-28T15:00:59.939352+0000","last_became_peered":"2024-06-28T15:00:59.939352+0000","last_unstale":"2024-06-28T16:14:48.110204+0000","last_undegraded":"2024-06-28T16:14:48.110204+0000","last_fullsized":"2024-06-28T16:14:48.110204+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'320","last_scrub_stamp":"2024-06-28T16:14:48.109838+0000","last_deep_scrub":"260'320","last_deep_scrub_stamp":"2024-06-28T16:14:48.109838+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.109838+0000","objects_scrubbed":10,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:13:21.727097+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":804,"num_read_kb":59106,"num_write":1183,"num_write_kb":43047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6,4],"acting":[3,2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":516,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.722700+0000","last_change":"2024-06-28T16:15:10.722700+0000","last_active":"2024-06-28T16:15:10.722700+0000","last_peered":"2024-06-28T16:15:10.722700+0000","last_clean":"2024-06-28T16:15:10.722700+0000","last_became_active":"2024-06-28T15:00:19.390255+0000","last_became_peered":"2024-06-28T15:00:19.390255+0000","last_unstale":"2024-06-28T16:15:10.722700+0000","last_undegraded":"2024-06-28T16:15:10.722700+0000","last_fullsized":"2024-06-28T16:15:10.722700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:10.722439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:10.722439+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.722439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:01:41.201808+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"260'734","reported_seq":1463,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.597944+0000","last_change":"2024-06-28T16:14:55.598090+0000","last_active":"2024-06-28T16:14:55.597944+0000","last_peered":"2024-06-28T16:14:55.597944+0000","last_clean":"2024-06-28T16:14:55.597944+0000","last_became_active":"2024-06-28T15:00:21.397640+0000","last_became_peered":"2024-06-28T15:00:21.397640+0000","last_unstale":"2024-06-28T16:14:55.597944+0000","last_undegraded":"2024-06-28T16:14:55.597944+0000","last_fullsized":"2024-06-28T16:14:55.597944+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'734","last_scrub_stamp":"2024-06-28T16:14:55.597730+0000","last_deep_scrub":"260'734","last_deep_scrub_stamp":"2024-06-28T16:14:55.597730+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.597730+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":700,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:40:40.351703+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134565,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1425,"num_read_kb":3272,"num_write":1360,"num_write_kb":1190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99450,"num_omap_keys":240,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.214467+0000","last_change":"2024-06-28T16:15:03.214529+0000","last_active":"2024-06-28T16:15:03.214467+0000","last_peered":"2024-06-28T16:15:03.214467+0000","last_clean":"2024-06-28T16:15:03.214467+0000","last_became_active":"2024-06-28T15:00:23.399024+0000","last_became_peered":"2024-06-28T15:00:23.399024+0000","last_unstale":"2024-06-28T16:15:03.214467+0000","last_undegraded":"2024-06-28T16:15:03.214467+0000","last_fullsized":"2024-06-28T16:15:03.214467+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:03.214271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:03.214271+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.214271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:17:32.250338+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"260'190","reported_seq":889,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.404009+0000","last_change":"2024-06-28T16:15:12.404087+0000","last_active":"2024-06-28T16:15:12.404009+0000","last_peered":"2024-06-28T16:15:12.404009+0000","last_clean":"2024-06-28T16:15:12.404009+0000","last_became_active":"2024-06-28T15:00:25.442032+0000","last_became_peered":"2024-06-28T15:00:25.442032+0000","last_unstale":"2024-06-28T16:15:12.404009+0000","last_undegraded":"2024-06-28T16:15:12.404009+0000","last_fullsized":"2024-06-28T16:15:12.404009+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'190","last_scrub_stamp":"2024-06-28T16:15:12.403829+0000","last_deep_scrub":"260'190","last_deep_scrub_stamp":"2024-06-28T16:15:12.403829+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.403829+0000","objects_scrubbed":4,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:09:00.851833+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":392,"num_read_kb":388,"num_write":388,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":66,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"260'118","reported_seq":764,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.318413+0000","last_change":"2024-06-28T16:15:12.318476+0000","last_active":"2024-06-28T16:15:12.318413+0000","last_peered":"2024-06-28T16:15:12.318413+0000","last_clean":"2024-06-28T16:15:12.318413+0000","last_became_active":"2024-06-28T15:00:17.404200+0000","last_became_peered":"2024-06-28T15:00:17.404200+0000","last_unstale":"2024-06-28T16:15:12.318413+0000","last_undegraded":"2024-06-28T16:15:12.318413+0000","last_fullsized":"2024-06-28T16:15:12.318413+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'118","last_scrub_stamp":"2024-06-28T16:15:12.318222+0000","last_deep_scrub":"260'118","last_deep_scrub_stamp":"2024-06-28T16:15:12.318222+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.318222+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:09:16.409918+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":495,"num_read_kb":413,"num_write":159,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":535,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:22.892745+0000","last_change":"2024-06-28T16:15:22.892798+0000","last_active":"2024-06-28T16:15:22.892745+0000","last_peered":"2024-06-28T16:15:22.892745+0000","last_clean":"2024-06-28T16:15:22.892745+0000","last_became_active":"2024-06-28T15:00:06.285489+0000","last_became_peered":"2024-06-28T15:00:06.285489+0000","last_unstale":"2024-06-28T16:15:22.892745+0000","last_undegraded":"2024-06-28T16:15:22.892745+0000","last_fullsized":"2024-06-28T16:15:22.892745+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:22.892521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:22.892521+0000","last_clean_scrub_stamp":"2024-06-28T16:15:22.892521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:00:17.084929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"260'324","reported_seq":1002,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:28.289381+0000","last_change":"2024-06-28T16:15:28.289485+0000","last_active":"2024-06-28T16:15:28.289381+0000","last_peered":"2024-06-28T16:15:28.289381+0000","last_clean":"2024-06-28T16:15:28.289381+0000","last_became_active":"2024-06-28T15:00:15.403657+0000","last_became_peered":"2024-06-28T15:00:15.403657+0000","last_unstale":"2024-06-28T16:15:28.289381+0000","last_undegraded":"2024-06-28T16:15:28.289381+0000","last_fullsized":"2024-06-28T16:15:28.289381+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'324","last_scrub_stamp":"2024-06-28T16:15:28.289176+0000","last_deep_scrub":"260'324","last_deep_scrub_stamp":"2024-06-28T16:15:28.289176+0000","last_clean_scrub_stamp":"2024-06-28T16:15:28.289176+0000","objects_scrubbed":4,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:25:17.991177+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":524,"num_read_kb":36646,"num_write":1262,"num_write_kb":16548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.674081+0000","last_change":"2024-06-28T16:15:12.674142+0000","last_active":"2024-06-28T16:15:12.674081+0000","last_peered":"2024-06-28T16:15:12.674081+0000","last_clean":"2024-06-28T16:15:12.674081+0000","last_became_active":"2024-06-28T15:00:19.406447+0000","last_became_peered":"2024-06-28T15:00:19.406447+0000","last_unstale":"2024-06-28T16:15:12.674081+0000","last_undegraded":"2024-06-28T16:15:12.674081+0000","last_fullsized":"2024-06-28T16:15:12.674081+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:12.673872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:12.673872+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.673872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:57:02.704896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"260'1141","reported_seq":2925,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.980185+0000","last_change":"2024-06-28T16:14:56.980185+0000","last_active":"2024-06-28T16:14:56.980185+0000","last_peered":"2024-06-28T16:14:56.980185+0000","last_clean":"2024-06-28T16:14:56.980185+0000","last_became_active":"2024-06-28T15:00:21.398382+0000","last_became_peered":"2024-06-28T15:00:21.398382+0000","last_unstale":"2024-06-28T16:14:56.980185+0000","last_undegraded":"2024-06-28T16:14:56.980185+0000","last_fullsized":"2024-06-28T16:14:56.980185+0000","mapping_epoch":25,"log_start":"260'1100","ondisk_log_start":"260'1100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'1141","last_scrub_stamp":"2024-06-28T16:14:56.979906+0000","last_deep_scrub":"260'1141","last_deep_scrub_stamp":"2024-06-28T16:14:56.979906+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.979906+0000","objects_scrubbed":11,"log_size":41,"log_dups_size":1100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:56:04.659644+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133468,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1851,"num_read_kb":3741,"num_write":1150,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6022,"num_omap_keys":18,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.712290+0000","last_change":"2024-06-28T16:14:59.712290+0000","last_active":"2024-06-28T16:14:59.712290+0000","last_peered":"2024-06-28T16:14:59.712290+0000","last_clean":"2024-06-28T16:14:59.712290+0000","last_became_active":"2024-06-28T15:00:23.422876+0000","last_became_peered":"2024-06-28T15:00:23.422876+0000","last_unstale":"2024-06-28T16:14:59.712290+0000","last_undegraded":"2024-06-28T16:14:59.712290+0000","last_fullsized":"2024-06-28T16:14:59.712290+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:59.712038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:59.712038+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.712038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:40.142260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"260'120","reported_seq":642,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.019612+0000","last_change":"2024-06-28T16:14:47.019699+0000","last_active":"2024-06-28T16:14:47.019612+0000","last_peered":"2024-06-28T16:14:47.019612+0000","last_clean":"2024-06-28T16:14:47.019612+0000","last_became_active":"2024-06-28T15:00:25.553691+0000","last_became_peered":"2024-06-28T15:00:25.553691+0000","last_unstale":"2024-06-28T16:14:47.019612+0000","last_undegraded":"2024-06-28T16:14:47.019612+0000","last_fullsized":"2024-06-28T16:14:47.019612+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'120","last_scrub_stamp":"2024-06-28T16:14:47.019412+0000","last_deep_scrub":"260'120","last_deep_scrub_stamp":"2024-06-28T16:14:47.019412+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.019412+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:33:15.581216+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":862,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":120,"num_write":292,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"260'221","reported_seq":843,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.262910+0000","last_change":"2024-06-28T16:15:14.262978+0000","last_active":"2024-06-28T16:15:14.262910+0000","last_peered":"2024-06-28T16:15:14.262910+0000","last_clean":"2024-06-28T16:15:14.262910+0000","last_became_active":"2024-06-28T15:00:17.393361+0000","last_became_peered":"2024-06-28T15:00:17.393361+0000","last_unstale":"2024-06-28T16:15:14.262910+0000","last_undegraded":"2024-06-28T16:15:14.262910+0000","last_fullsized":"2024-06-28T16:15:14.262910+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'221","last_scrub_stamp":"2024-06-28T16:15:14.262723+0000","last_deep_scrub":"260'221","last_deep_scrub_stamp":"2024-06-28T16:15:14.262723+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.262723+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:12:41.234327+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":892,"num_read_kb":761,"num_write":358,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":542,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.217011+0000","last_change":"2024-06-28T16:14:56.217011+0000","last_active":"2024-06-28T16:14:56.217011+0000","last_peered":"2024-06-28T16:14:56.217011+0000","last_clean":"2024-06-28T16:14:56.217011+0000","last_became_active":"2024-06-28T15:00:06.279950+0000","last_became_peered":"2024-06-28T15:00:06.279950+0000","last_unstale":"2024-06-28T16:14:56.217011+0000","last_undegraded":"2024-06-28T16:14:56.217011+0000","last_fullsized":"2024-06-28T16:14:56.217011+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.216749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.216749+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.216749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:22:36.357975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"260'334","reported_seq":1000,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.220573+0000","last_change":"2024-06-28T16:15:12.220658+0000","last_active":"2024-06-28T16:15:12.220573+0000","last_peered":"2024-06-28T16:15:12.220573+0000","last_clean":"2024-06-28T16:15:12.220573+0000","last_became_active":"2024-06-28T15:00:15.413348+0000","last_became_peered":"2024-06-28T15:00:15.413348+0000","last_unstale":"2024-06-28T16:15:12.220573+0000","last_undegraded":"2024-06-28T16:15:12.220573+0000","last_fullsized":"2024-06-28T16:15:12.220573+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'334","last_scrub_stamp":"2024-06-28T16:15:12.220392+0000","last_deep_scrub":"260'334","last_deep_scrub_stamp":"2024-06-28T16:15:12.220392+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.220392+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:47:12.335485+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":504,"num_read_kb":3638,"num_write":1247,"num_write_kb":36014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"260'2","reported_seq":525,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.146412+0000","last_change":"2024-06-28T16:14:47.146477+0000","last_active":"2024-06-28T16:14:47.146412+0000","last_peered":"2024-06-28T16:14:47.146412+0000","last_clean":"2024-06-28T16:14:47.146412+0000","last_became_active":"2024-06-28T15:00:19.404653+0000","last_became_peered":"2024-06-28T15:00:19.404653+0000","last_unstale":"2024-06-28T16:14:47.146412+0000","last_undegraded":"2024-06-28T16:14:47.146412+0000","last_fullsized":"2024-06-28T16:14:47.146412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:47.146212+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:47.146212+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.146212+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:46:59.182619+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"260'255","reported_seq":1710,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:31.068348+0000","last_change":"2024-06-28T16:15:31.068348+0000","last_active":"2024-06-28T16:15:31.068348+0000","last_peered":"2024-06-28T16:15:31.068348+0000","last_clean":"2024-06-28T16:15:31.068348+0000","last_became_active":"2024-06-28T15:00:21.483749+0000","last_became_peered":"2024-06-28T15:00:21.483749+0000","last_unstale":"2024-06-28T16:15:31.068348+0000","last_undegraded":"2024-06-28T16:15:31.068348+0000","last_fullsized":"2024-06-28T16:15:31.068348+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'255","last_scrub_stamp":"2024-06-28T16:15:31.068054+0000","last_deep_scrub":"260'255","last_deep_scrub_stamp":"2024-06-28T16:15:31.068054+0000","last_clean_scrub_stamp":"2024-06-28T16:15:31.068054+0000","objects_scrubbed":9,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:02:29.373520+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5824,"num_omap_keys":14,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.250838+0000","last_change":"2024-06-28T16:15:04.250838+0000","last_active":"2024-06-28T16:15:04.250838+0000","last_peered":"2024-06-28T16:15:04.250838+0000","last_clean":"2024-06-28T16:15:04.250838+0000","last_became_active":"2024-06-28T15:00:23.399391+0000","last_became_peered":"2024-06-28T15:00:23.399391+0000","last_unstale":"2024-06-28T16:15:04.250838+0000","last_undegraded":"2024-06-28T16:15:04.250838+0000","last_fullsized":"2024-06-28T16:15:04.250838+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:04.250594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:04.250594+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.250594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:04:20.072805+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"260'242","reported_seq":857,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.112554+0000","last_change":"2024-06-28T16:14:55.112726+0000","last_active":"2024-06-28T16:14:55.112554+0000","last_peered":"2024-06-28T16:14:55.112554+0000","last_clean":"2024-06-28T16:14:55.112554+0000","last_became_active":"2024-06-28T15:00:25.419711+0000","last_became_peered":"2024-06-28T15:00:25.419711+0000","last_unstale":"2024-06-28T16:14:55.112554+0000","last_undegraded":"2024-06-28T16:14:55.112554+0000","last_fullsized":"2024-06-28T16:14:55.112554+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'242","last_scrub_stamp":"2024-06-28T16:14:55.112334+0000","last_deep_scrub":"260'242","last_deep_scrub_stamp":"2024-06-28T16:14:55.112334+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.112334+0000","objects_scrubbed":3,"log_size":42,"log_dups_size":200,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:15:19.328332+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":467,"num_read_kb":491,"num_write":517,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"260'316","reported_seq":1017,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.125767+0000","last_change":"2024-06-28T16:15:04.125767+0000","last_active":"2024-06-28T16:15:04.125767+0000","last_peered":"2024-06-28T16:15:04.125767+0000","last_clean":"2024-06-28T16:15:04.125767+0000","last_became_active":"2024-06-28T15:00:17.393696+0000","last_became_peered":"2024-06-28T15:00:17.393696+0000","last_unstale":"2024-06-28T16:15:04.125767+0000","last_undegraded":"2024-06-28T16:15:04.125767+0000","last_fullsized":"2024-06-28T16:15:04.125767+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'316","last_scrub_stamp":"2024-06-28T16:15:04.125411+0000","last_deep_scrub":"260'316","last_deep_scrub_stamp":"2024-06-28T16:15:04.125411+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.125411+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":300,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:39:24.006191+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1365,"num_read_kb":1154,"num_write":468,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25681,"num_omap_keys":101,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":534,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.751576+0000","last_change":"2024-06-28T16:15:14.751632+0000","last_active":"2024-06-28T16:15:14.751576+0000","last_peered":"2024-06-28T16:15:14.751576+0000","last_clean":"2024-06-28T16:15:14.751576+0000","last_became_active":"2024-06-28T15:00:06.283106+0000","last_became_peered":"2024-06-28T15:00:06.283106+0000","last_unstale":"2024-06-28T16:15:14.751576+0000","last_undegraded":"2024-06-28T16:15:14.751576+0000","last_fullsized":"2024-06-28T16:15:14.751576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:14.751373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:14.751373+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.751373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:17:53.348364+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"260'318","reported_seq":1001,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.927891+0000","last_change":"2024-06-28T16:15:17.928148+0000","last_active":"2024-06-28T16:15:17.927891+0000","last_peered":"2024-06-28T16:15:17.927891+0000","last_clean":"2024-06-28T16:15:17.927891+0000","last_became_active":"2024-06-28T15:00:15.413880+0000","last_became_peered":"2024-06-28T15:00:15.413880+0000","last_unstale":"2024-06-28T16:15:17.927891+0000","last_undegraded":"2024-06-28T16:15:17.927891+0000","last_fullsized":"2024-06-28T16:15:17.927891+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'318","last_scrub_stamp":"2024-06-28T16:15:17.927669+0000","last_deep_scrub":"260'318","last_deep_scrub_stamp":"2024-06-28T16:15:17.927669+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.927669+0000","objects_scrubbed":7,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:19:11.398970+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2421978,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":465,"num_read_kb":85416,"num_write":1148,"num_write_kb":27167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":523,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.295510+0000","last_change":"2024-06-28T16:15:14.295510+0000","last_active":"2024-06-28T16:15:14.295510+0000","last_peered":"2024-06-28T16:15:14.295510+0000","last_clean":"2024-06-28T16:15:14.295510+0000","last_became_active":"2024-06-28T15:00:19.382486+0000","last_became_peered":"2024-06-28T15:00:19.382486+0000","last_unstale":"2024-06-28T16:15:14.295510+0000","last_undegraded":"2024-06-28T16:15:14.295510+0000","last_fullsized":"2024-06-28T16:15:14.295510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:14.295262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:14.295262+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.295262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:15:46.049931+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"260'166","reported_seq":745,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.148880+0000","last_change":"2024-06-28T16:15:10.148966+0000","last_active":"2024-06-28T16:15:10.148880+0000","last_peered":"2024-06-28T16:15:10.148880+0000","last_clean":"2024-06-28T16:15:10.148880+0000","last_became_active":"2024-06-28T15:00:21.400535+0000","last_became_peered":"2024-06-28T15:00:21.400535+0000","last_unstale":"2024-06-28T16:15:10.148880+0000","last_undegraded":"2024-06-28T16:15:10.148880+0000","last_fullsized":"2024-06-28T16:15:10.148880+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'166","last_scrub_stamp":"2024-06-28T16:15:10.148680+0000","last_deep_scrub":"260'166","last_deep_scrub_stamp":"2024-06-28T16:15:10.148680+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.148680+0000","objects_scrubbed":7,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:40:13.912505+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":198,"num_read_kb":191,"num_write":142,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1510,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.407107+0000","last_change":"2024-06-28T16:14:56.407161+0000","last_active":"2024-06-28T16:14:56.407107+0000","last_peered":"2024-06-28T16:14:56.407107+0000","last_clean":"2024-06-28T16:14:56.407107+0000","last_became_active":"2024-06-28T15:00:23.416701+0000","last_became_peered":"2024-06-28T15:00:23.416701+0000","last_unstale":"2024-06-28T16:14:56.407107+0000","last_undegraded":"2024-06-28T16:14:56.407107+0000","last_fullsized":"2024-06-28T16:14:56.407107+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.406907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.406907+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.406907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:09:09.502206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"260'117","reported_seq":642,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.620180+0000","last_change":"2024-06-28T16:14:47.620245+0000","last_active":"2024-06-28T16:14:47.620180+0000","last_peered":"2024-06-28T16:14:47.620180+0000","last_clean":"2024-06-28T16:14:47.620180+0000","last_became_active":"2024-06-28T15:00:25.438523+0000","last_became_peered":"2024-06-28T15:00:25.438523+0000","last_unstale":"2024-06-28T16:14:47.620180+0000","last_undegraded":"2024-06-28T16:14:47.620180+0000","last_fullsized":"2024-06-28T16:14:47.620180+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'117","last_scrub_stamp":"2024-06-28T16:14:47.619989+0000","last_deep_scrub":"260'117","last_deep_scrub_stamp":"2024-06-28T16:14:47.619989+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.619989+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:25:16.904136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":129,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"260'169","reported_seq":784,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.041377+0000","last_change":"2024-06-28T16:15:02.041377+0000","last_active":"2024-06-28T16:15:02.041377+0000","last_peered":"2024-06-28T16:15:02.041377+0000","last_clean":"2024-06-28T16:15:02.041377+0000","last_became_active":"2024-06-28T15:00:17.398515+0000","last_became_peered":"2024-06-28T15:00:17.398515+0000","last_unstale":"2024-06-28T16:15:02.041377+0000","last_undegraded":"2024-06-28T16:15:02.041377+0000","last_fullsized":"2024-06-28T16:15:02.041377+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'169","last_scrub_stamp":"2024-06-28T16:15:02.041106+0000","last_deep_scrub":"260'169","last_deep_scrub_stamp":"2024-06-28T16:15:02.041106+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.041106+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:44:07.334141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":644,"num_read_kb":512,"num_write":273,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"260'372","reported_seq":1053,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.858841+0000","last_change":"2024-06-28T16:15:18.858841+0000","last_active":"2024-06-28T16:15:18.858841+0000","last_peered":"2024-06-28T16:15:18.858841+0000","last_clean":"2024-06-28T16:15:18.858841+0000","last_became_active":"2024-06-28T15:00:15.414704+0000","last_became_peered":"2024-06-28T15:00:15.414704+0000","last_unstale":"2024-06-28T16:15:18.858841+0000","last_undegraded":"2024-06-28T16:15:18.858841+0000","last_fullsized":"2024-06-28T16:15:18.858841+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'372","last_scrub_stamp":"2024-06-28T16:15:18.858489+0000","last_deep_scrub":"260'372","last_deep_scrub_stamp":"2024-06-28T16:15:18.858489+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.858489+0000","objects_scrubbed":4,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:00:26.901636+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10506361,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":617,"num_read_kb":33619,"num_write":1289,"num_write_kb":23520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":539,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.163070+0000","last_change":"2024-06-28T16:15:08.163070+0000","last_active":"2024-06-28T16:15:08.163070+0000","last_peered":"2024-06-28T16:15:08.163070+0000","last_clean":"2024-06-28T16:15:08.163070+0000","last_became_active":"2024-06-28T15:00:06.302449+0000","last_became_peered":"2024-06-28T15:00:06.302449+0000","last_unstale":"2024-06-28T16:15:08.163070+0000","last_undegraded":"2024-06-28T16:15:08.163070+0000","last_fullsized":"2024-06-28T16:15:08.163070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:08.162767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:08.162767+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.162767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:45:36.489995+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":523,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.260730+0000","last_change":"2024-06-28T16:15:16.260730+0000","last_active":"2024-06-28T16:15:16.260730+0000","last_peered":"2024-06-28T16:15:16.260730+0000","last_clean":"2024-06-28T16:15:16.260730+0000","last_became_active":"2024-06-28T15:00:19.384933+0000","last_became_peered":"2024-06-28T15:00:19.384933+0000","last_unstale":"2024-06-28T16:15:16.260730+0000","last_undegraded":"2024-06-28T16:15:16.260730+0000","last_fullsized":"2024-06-28T16:15:16.260730+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:16.260473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:16.260473+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.260473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:45:58.084956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"260'597","reported_seq":1305,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.242218+0000","last_change":"2024-06-28T16:15:05.242315+0000","last_active":"2024-06-28T16:15:05.242218+0000","last_peered":"2024-06-28T16:15:05.242218+0000","last_clean":"2024-06-28T16:15:05.242218+0000","last_became_active":"2024-06-28T15:00:21.405762+0000","last_became_peered":"2024-06-28T15:00:21.405762+0000","last_unstale":"2024-06-28T16:15:05.242218+0000","last_undegraded":"2024-06-28T16:15:05.242218+0000","last_fullsized":"2024-06-28T16:15:05.242218+0000","mapping_epoch":25,"log_start":"260'500","ondisk_log_start":"260'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'597","last_scrub_stamp":"2024-06-28T16:15:05.241998+0000","last_deep_scrub":"260'597","last_deep_scrub_stamp":"2024-06-28T16:15:05.241998+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.241998+0000","objects_scrubbed":10,"log_size":97,"log_dups_size":500,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:43:06.700587+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92557,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1001,"num_read_kb":2235,"num_write":838,"num_write_kb":641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8196,"num_omap_keys":24,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.927496+0000","last_change":"2024-06-28T16:14:49.927496+0000","last_active":"2024-06-28T16:14:49.927496+0000","last_peered":"2024-06-28T16:14:49.927496+0000","last_clean":"2024-06-28T16:14:49.927496+0000","last_became_active":"2024-06-28T15:00:23.404714+0000","last_became_peered":"2024-06-28T15:00:23.404714+0000","last_unstale":"2024-06-28T16:14:49.927496+0000","last_undegraded":"2024-06-28T16:14:49.927496+0000","last_fullsized":"2024-06-28T16:14:49.927496+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.927238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.927238+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.927238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:46:09.338954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"260'154","reported_seq":680,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.151138+0000","last_change":"2024-06-28T16:14:48.151312+0000","last_active":"2024-06-28T16:14:48.151138+0000","last_peered":"2024-06-28T16:14:48.151138+0000","last_clean":"2024-06-28T16:14:48.151138+0000","last_became_active":"2024-06-28T15:00:25.556834+0000","last_became_peered":"2024-06-28T15:00:25.556834+0000","last_unstale":"2024-06-28T16:14:48.151138+0000","last_undegraded":"2024-06-28T16:14:48.151138+0000","last_fullsized":"2024-06-28T16:14:48.151138+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'154","last_scrub_stamp":"2024-06-28T16:14:48.150905+0000","last_deep_scrub":"260'154","last_deep_scrub_stamp":"2024-06-28T16:14:48.150905+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.150905+0000","objects_scrubbed":7,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:54:08.173965+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1183,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":342,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"260'139","reported_seq":778,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.225074+0000","last_change":"2024-06-28T16:15:19.225074+0000","last_active":"2024-06-28T16:15:19.225074+0000","last_peered":"2024-06-28T16:15:19.225074+0000","last_clean":"2024-06-28T16:15:19.225074+0000","last_became_active":"2024-06-28T15:00:17.372401+0000","last_became_peered":"2024-06-28T15:00:17.372401+0000","last_unstale":"2024-06-28T16:15:19.225074+0000","last_undegraded":"2024-06-28T16:15:19.225074+0000","last_fullsized":"2024-06-28T16:15:19.225074+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'139","last_scrub_stamp":"2024-06-28T16:15:19.224695+0000","last_deep_scrub":"260'139","last_deep_scrub_stamp":"2024-06-28T16:15:19.224695+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.224695+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:34:36.503538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":577,"num_read_kb":470,"num_write":221,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"260'273","reported_seq":911,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.187159+0000","last_change":"2024-06-28T16:15:11.187159+0000","last_active":"2024-06-28T16:15:11.187159+0000","last_peered":"2024-06-28T16:15:11.187159+0000","last_clean":"2024-06-28T16:15:11.187159+0000","last_became_active":"2024-06-28T15:00:15.408140+0000","last_became_peered":"2024-06-28T15:00:15.408140+0000","last_unstale":"2024-06-28T16:15:11.187159+0000","last_undegraded":"2024-06-28T16:15:11.187159+0000","last_fullsized":"2024-06-28T16:15:11.187159+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'273","last_scrub_stamp":"2024-06-28T16:15:11.186833+0000","last_deep_scrub":"260'273","last_deep_scrub_stamp":"2024-06-28T16:15:11.186833+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.186833+0000","objects_scrubbed":10,"log_size":73,"log_dups_size":200,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:13:01.399899+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":526399,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":387,"num_read_kb":8190,"num_write":1081,"num_write_kb":24156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":534,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:24.956397+0000","last_change":"2024-06-28T16:15:24.956397+0000","last_active":"2024-06-28T16:15:24.956397+0000","last_peered":"2024-06-28T16:15:24.956397+0000","last_clean":"2024-06-28T16:15:24.956397+0000","last_became_active":"2024-06-28T15:00:06.302962+0000","last_became_peered":"2024-06-28T15:00:06.302962+0000","last_unstale":"2024-06-28T16:15:24.956397+0000","last_undegraded":"2024-06-28T16:15:24.956397+0000","last_fullsized":"2024-06-28T16:15:24.956397+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:24.956071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:24.956071+0000","last_clean_scrub_stamp":"2024-06-28T16:15:24.956071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:16:18.714248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.184441+0000","last_change":"2024-06-28T16:14:49.184441+0000","last_active":"2024-06-28T16:14:49.184441+0000","last_peered":"2024-06-28T16:14:49.184441+0000","last_clean":"2024-06-28T16:14:49.184441+0000","last_became_active":"2024-06-28T15:00:19.384961+0000","last_became_peered":"2024-06-28T15:00:19.384961+0000","last_unstale":"2024-06-28T16:14:49.184441+0000","last_undegraded":"2024-06-28T16:14:49.184441+0000","last_fullsized":"2024-06-28T16:14:49.184441+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.184197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.184197+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.184197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:48:38.391991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"260'195","reported_seq":802,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.962600+0000","last_change":"2024-06-28T16:14:58.962600+0000","last_active":"2024-06-28T16:14:58.962600+0000","last_peered":"2024-06-28T16:14:58.962600+0000","last_clean":"2024-06-28T16:14:58.962600+0000","last_became_active":"2024-06-28T15:00:21.480107+0000","last_became_peered":"2024-06-28T15:00:21.480107+0000","last_unstale":"2024-06-28T16:14:58.962600+0000","last_undegraded":"2024-06-28T16:14:58.962600+0000","last_fullsized":"2024-06-28T16:14:58.962600+0000","mapping_epoch":25,"log_start":"257'100","ondisk_log_start":"257'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'195","last_scrub_stamp":"2024-06-28T16:14:58.962331+0000","last_deep_scrub":"260'195","last_deep_scrub_stamp":"2024-06-28T16:14:58.962331+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.962331+0000","objects_scrubbed":7,"log_size":95,"log_dups_size":100,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:08:23.304315+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":263,"num_read_kb":253,"num_write":174,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"260'565","reported_seq":2887,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.666227+0000","last_change":"2024-06-28T16:14:51.666291+0000","last_active":"2024-06-28T16:14:51.666227+0000","last_peered":"2024-06-28T16:14:51.666227+0000","last_clean":"2024-06-28T16:14:51.666227+0000","last_became_active":"2024-06-28T15:00:23.427565+0000","last_became_peered":"2024-06-28T15:00:23.427565+0000","last_unstale":"2024-06-28T16:14:51.666227+0000","last_undegraded":"2024-06-28T16:14:51.666227+0000","last_fullsized":"2024-06-28T16:14:51.666227+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'565","last_scrub_stamp":"2024-06-28T16:14:51.666022+0000","last_deep_scrub":"260'565","last_deep_scrub_stamp":"2024-06-28T16:14:51.666022+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.666022+0000","objects_scrubbed":1,"log_size":65,"log_dups_size":500,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:06:09.388341+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"260'127","reported_seq":651,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.139109+0000","last_change":"2024-06-28T16:14:51.139189+0000","last_active":"2024-06-28T16:14:51.139109+0000","last_peered":"2024-06-28T16:14:51.139109+0000","last_clean":"2024-06-28T16:14:51.139109+0000","last_became_active":"2024-06-28T15:00:25.553627+0000","last_became_peered":"2024-06-28T15:00:25.553627+0000","last_unstale":"2024-06-28T16:14:51.139109+0000","last_undegraded":"2024-06-28T16:14:51.139109+0000","last_fullsized":"2024-06-28T16:14:51.139109+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'127","last_scrub_stamp":"2024-06-28T16:14:51.138914+0000","last_deep_scrub":"260'127","last_deep_scrub_stamp":"2024-06-28T16:14:51.138914+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.138914+0000","objects_scrubbed":3,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:08:31.042999+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":713,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":165,"num_read_kb":148,"num_write":308,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"260'172","reported_seq":834,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.189062+0000","last_change":"2024-06-28T16:15:18.189235+0000","last_active":"2024-06-28T16:15:18.189062+0000","last_peered":"2024-06-28T16:15:18.189062+0000","last_clean":"2024-06-28T16:15:18.189062+0000","last_became_active":"2024-06-28T15:00:17.376516+0000","last_became_peered":"2024-06-28T15:00:17.376516+0000","last_unstale":"2024-06-28T16:15:18.189062+0000","last_undegraded":"2024-06-28T16:15:18.189062+0000","last_fullsized":"2024-06-28T16:15:18.189062+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'172","last_scrub_stamp":"2024-06-28T16:15:18.188830+0000","last_deep_scrub":"260'172","last_deep_scrub_stamp":"2024-06-28T16:15:18.188830+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.188830+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:08:43.423312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":722,"num_read_kb":588,"num_write":244,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"260'336","reported_seq":989,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:33.259899+0000","last_change":"2024-06-28T16:15:33.259994+0000","last_active":"2024-06-28T16:15:33.259899+0000","last_peered":"2024-06-28T16:15:33.259899+0000","last_clean":"2024-06-28T16:15:33.259899+0000","last_became_active":"2024-06-28T15:00:15.403032+0000","last_became_peered":"2024-06-28T15:00:15.403032+0000","last_unstale":"2024-06-28T16:15:33.259899+0000","last_undegraded":"2024-06-28T16:15:33.259899+0000","last_fullsized":"2024-06-28T16:15:33.259899+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'336","last_scrub_stamp":"2024-06-28T16:15:33.259679+0000","last_deep_scrub":"260'336","last_deep_scrub_stamp":"2024-06-28T16:15:33.259679+0000","last_clean_scrub_stamp":"2024-06-28T16:15:33.259679+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:44:24.072948+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":968026,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":483,"num_read_kb":17032,"num_write":1147,"num_write_kb":35885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":537,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.172483+0000","last_change":"2024-06-28T16:15:07.172536+0000","last_active":"2024-06-28T16:15:07.172483+0000","last_peered":"2024-06-28T16:15:07.172483+0000","last_clean":"2024-06-28T16:15:07.172483+0000","last_became_active":"2024-06-28T15:00:06.278443+0000","last_became_peered":"2024-06-28T15:00:06.278443+0000","last_unstale":"2024-06-28T16:15:07.172483+0000","last_undegraded":"2024-06-28T16:15:07.172483+0000","last_fullsized":"2024-06-28T16:15:07.172483+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:07.172293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:07.172293+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.172293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:30:00.544437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":518,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.999237+0000","last_change":"2024-06-28T16:15:08.999237+0000","last_active":"2024-06-28T16:15:08.999237+0000","last_peered":"2024-06-28T16:15:08.999237+0000","last_clean":"2024-06-28T16:15:08.999237+0000","last_became_active":"2024-06-28T15:00:19.378932+0000","last_became_peered":"2024-06-28T15:00:19.378932+0000","last_unstale":"2024-06-28T16:15:08.999237+0000","last_undegraded":"2024-06-28T16:15:08.999237+0000","last_fullsized":"2024-06-28T16:15:08.999237+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:08.998981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:08.998981+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.998981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:59:56.713595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"260'143","reported_seq":713,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.986950+0000","last_change":"2024-06-28T16:14:54.986950+0000","last_active":"2024-06-28T16:14:54.986950+0000","last_peered":"2024-06-28T16:14:54.986950+0000","last_clean":"2024-06-28T16:14:54.986950+0000","last_became_active":"2024-06-28T15:00:21.395462+0000","last_became_peered":"2024-06-28T15:00:21.395462+0000","last_unstale":"2024-06-28T16:14:54.986950+0000","last_undegraded":"2024-06-28T16:14:54.986950+0000","last_fullsized":"2024-06-28T16:14:54.986950+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'143","last_scrub_stamp":"2024-06-28T16:14:54.986685+0000","last_deep_scrub":"260'143","last_deep_scrub_stamp":"2024-06-28T16:14:54.986685+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.986685+0000","objects_scrubbed":5,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:03:28.897014+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":167,"num_read_kb":161,"num_write":112,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"260'537","reported_seq":2591,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.176689+0000","last_change":"2024-06-28T16:14:58.176689+0000","last_active":"2024-06-28T16:14:58.176689+0000","last_peered":"2024-06-28T16:14:58.176689+0000","last_clean":"2024-06-28T16:14:58.176689+0000","last_became_active":"2024-06-28T15:00:23.427568+0000","last_became_peered":"2024-06-28T15:00:23.427568+0000","last_unstale":"2024-06-28T16:14:58.176689+0000","last_undegraded":"2024-06-28T16:14:58.176689+0000","last_fullsized":"2024-06-28T16:14:58.176689+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'537","last_scrub_stamp":"2024-06-28T16:14:58.176428+0000","last_deep_scrub":"260'537","last_deep_scrub_stamp":"2024-06-28T16:14:58.176428+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.176428+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:07:27.328025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"260'251","reported_seq":800,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.105514+0000","last_change":"2024-06-28T16:14:53.105587+0000","last_active":"2024-06-28T16:14:53.105514+0000","last_peered":"2024-06-28T16:14:53.105514+0000","last_clean":"2024-06-28T16:14:53.105514+0000","last_became_active":"2024-06-28T15:00:25.428067+0000","last_became_peered":"2024-06-28T15:00:25.428067+0000","last_unstale":"2024-06-28T16:14:53.105514+0000","last_undegraded":"2024-06-28T16:14:53.105514+0000","last_fullsized":"2024-06-28T16:14:53.105514+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'251","last_scrub_stamp":"2024-06-28T16:14:53.105331+0000","last_deep_scrub":"260'251","last_deep_scrub_stamp":"2024-06-28T16:14:53.105331+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.105331+0000","objects_scrubbed":4,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:01:44.076283+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":297,"num_write":552,"num_write_kb":170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"260'96","reported_seq":711,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.050892+0000","last_change":"2024-06-28T16:15:12.050892+0000","last_active":"2024-06-28T16:15:12.050892+0000","last_peered":"2024-06-28T16:15:12.050892+0000","last_clean":"2024-06-28T16:15:12.050892+0000","last_became_active":"2024-06-28T15:00:17.396413+0000","last_became_peered":"2024-06-28T15:00:17.396413+0000","last_unstale":"2024-06-28T16:15:12.050892+0000","last_undegraded":"2024-06-28T16:15:12.050892+0000","last_fullsized":"2024-06-28T16:15:12.050892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'96","last_scrub_stamp":"2024-06-28T16:15:12.050611+0000","last_deep_scrub":"260'96","last_deep_scrub_stamp":"2024-06-28T16:15:12.050611+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.050611+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:35:33.992945+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":410,"num_read_kb":340,"num_write":139,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"260'318","reported_seq":1106,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:25.169011+0000","last_change":"2024-06-28T16:15:25.169146+0000","last_active":"2024-06-28T16:15:25.169011+0000","last_peered":"2024-06-28T16:15:25.169011+0000","last_clean":"2024-06-28T16:15:25.169011+0000","last_became_active":"2024-06-28T15:00:15.395980+0000","last_became_peered":"2024-06-28T15:00:15.395980+0000","last_unstale":"2024-06-28T16:15:25.169011+0000","last_undegraded":"2024-06-28T16:15:25.169011+0000","last_fullsized":"2024-06-28T16:15:25.169011+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'318","last_scrub_stamp":"2024-06-28T16:15:25.168809+0000","last_deep_scrub":"260'318","last_deep_scrub_stamp":"2024-06-28T16:15:25.168809+0000","last_clean_scrub_stamp":"2024-06-28T16:15:25.168809+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:57:31.434952+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661262,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":649,"num_read_kb":125515,"num_write":1187,"num_write_kb":17513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":530,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.460147+0000","last_change":"2024-06-28T16:15:04.460199+0000","last_active":"2024-06-28T16:15:04.460147+0000","last_peered":"2024-06-28T16:15:04.460147+0000","last_clean":"2024-06-28T16:15:04.460147+0000","last_became_active":"2024-06-28T15:00:06.283380+0000","last_became_peered":"2024-06-28T15:00:06.283380+0000","last_unstale":"2024-06-28T16:15:04.460147+0000","last_undegraded":"2024-06-28T16:15:04.460147+0000","last_fullsized":"2024-06-28T16:15:04.460147+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:04.459951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:04.459951+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.459951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:06:10.142541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"260'6","reported_seq":852,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.154538+0000","last_change":"2024-06-28T16:15:17.154602+0000","last_active":"2024-06-28T16:15:17.154538+0000","last_peered":"2024-06-28T16:15:17.154538+0000","last_clean":"2024-06-28T16:15:17.154538+0000","last_became_active":"2024-06-28T15:00:19.372552+0000","last_became_peered":"2024-06-28T15:00:19.372552+0000","last_unstale":"2024-06-28T16:15:17.154538+0000","last_undegraded":"2024-06-28T16:15:17.154538+0000","last_fullsized":"2024-06-28T16:15:17.154538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:15:17.154344+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:15:17.154344+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.154344+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:46:02.206707+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":500,"num_read_kb":812,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"260'756","reported_seq":1510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:20.845590+0000","last_change":"2024-06-28T16:15:20.845590+0000","last_active":"2024-06-28T16:15:20.845590+0000","last_peered":"2024-06-28T16:15:20.845590+0000","last_clean":"2024-06-28T16:15:20.845590+0000","last_became_active":"2024-06-28T15:00:21.481155+0000","last_became_peered":"2024-06-28T15:00:21.481155+0000","last_unstale":"2024-06-28T16:15:20.845590+0000","last_undegraded":"2024-06-28T16:15:20.845590+0000","last_fullsized":"2024-06-28T16:15:20.845590+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'756","last_scrub_stamp":"2024-06-28T16:15:20.845293+0000","last_deep_scrub":"260'756","last_deep_scrub_stamp":"2024-06-28T16:15:20.845293+0000","last_clean_scrub_stamp":"2024-06-28T16:15:20.845293+0000","objects_scrubbed":8,"log_size":56,"log_dups_size":700,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:26:10.542424+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131438,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1320,"num_read_kb":3166,"num_write":1058,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.235034+0000","last_change":"2024-06-28T16:15:02.235034+0000","last_active":"2024-06-28T16:15:02.235034+0000","last_peered":"2024-06-28T16:15:02.235034+0000","last_clean":"2024-06-28T16:15:02.235034+0000","last_became_active":"2024-06-28T15:00:23.408777+0000","last_became_peered":"2024-06-28T15:00:23.408777+0000","last_unstale":"2024-06-28T16:15:02.235034+0000","last_undegraded":"2024-06-28T16:15:02.235034+0000","last_fullsized":"2024-06-28T16:15:02.235034+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:02.234718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:02.234718+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.234718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:27:42.722800+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"260'132","reported_seq":653,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.670572+0000","last_change":"2024-06-28T16:14:49.670641+0000","last_active":"2024-06-28T16:14:49.670572+0000","last_peered":"2024-06-28T16:14:49.670572+0000","last_clean":"2024-06-28T16:14:49.670572+0000","last_became_active":"2024-06-28T15:00:25.438476+0000","last_became_peered":"2024-06-28T15:00:25.438476+0000","last_unstale":"2024-06-28T16:14:49.670572+0000","last_undegraded":"2024-06-28T16:14:49.670572+0000","last_fullsized":"2024-06-28T16:14:49.670572+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'132","last_scrub_stamp":"2024-06-28T16:14:49.670359+0000","last_deep_scrub":"260'132","last_deep_scrub_stamp":"2024-06-28T16:14:49.670359+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.670359+0000","objects_scrubbed":3,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:09:32.566857+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":173,"num_read_kb":156,"num_write":320,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"260'153","reported_seq":730,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.120558+0000","last_change":"2024-06-28T16:14:49.120558+0000","last_active":"2024-06-28T16:14:49.120558+0000","last_peered":"2024-06-28T16:14:49.120558+0000","last_clean":"2024-06-28T16:14:49.120558+0000","last_became_active":"2024-06-28T15:00:25.439672+0000","last_became_peered":"2024-06-28T15:00:25.439672+0000","last_unstale":"2024-06-28T16:14:49.120558+0000","last_undegraded":"2024-06-28T16:14:49.120558+0000","last_fullsized":"2024-06-28T16:14:49.120558+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'153","last_scrub_stamp":"2024-06-28T16:14:49.120288+0000","last_deep_scrub":"260'153","last_deep_scrub_stamp":"2024-06-28T16:14:49.120288+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.120288+0000","objects_scrubbed":2,"log_size":53,"log_dups_size":100,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:05:56.112079+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":207,"num_read_kb":205,"num_write":345,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"260'435","reported_seq":1082,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.041132+0000","last_change":"2024-06-28T16:15:10.041196+0000","last_active":"2024-06-28T16:15:10.041132+0000","last_peered":"2024-06-28T16:15:10.041132+0000","last_clean":"2024-06-28T16:15:10.041132+0000","last_became_active":"2024-06-28T15:00:17.401981+0000","last_became_peered":"2024-06-28T15:00:17.401981+0000","last_unstale":"2024-06-28T16:15:10.041132+0000","last_undegraded":"2024-06-28T16:15:10.041132+0000","last_fullsized":"2024-06-28T16:15:10.041132+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'435","last_scrub_stamp":"2024-06-28T16:15:10.040927+0000","last_deep_scrub":"260'435","last_deep_scrub_stamp":"2024-06-28T16:15:10.040927+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.040927+0000","objects_scrubbed":1,"log_size":35,"log_dups_size":400,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:52:25.537900+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1728,"num_read_kb":1391,"num_write":746,"num_write_kb":387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":230,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"260'274","reported_seq":910,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.992519+0000","last_change":"2024-06-28T16:15:15.992709+0000","last_active":"2024-06-28T16:15:15.992519+0000","last_peered":"2024-06-28T16:15:15.992519+0000","last_clean":"2024-06-28T16:15:15.992519+0000","last_became_active":"2024-06-28T15:00:15.403450+0000","last_became_peered":"2024-06-28T15:00:15.403450+0000","last_unstale":"2024-06-28T16:15:15.992519+0000","last_undegraded":"2024-06-28T16:15:15.992519+0000","last_fullsized":"2024-06-28T16:15:15.992519+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'274","last_scrub_stamp":"2024-06-28T16:15:15.992200+0000","last_deep_scrub":"260'274","last_deep_scrub_stamp":"2024-06-28T16:15:15.992200+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.992200+0000","objects_scrubbed":4,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:27:40.312409+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437799,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":13452,"num_write":1073,"num_write_kb":25751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":518,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.005157+0000","last_change":"2024-06-28T16:14:51.005157+0000","last_active":"2024-06-28T16:14:51.005157+0000","last_peered":"2024-06-28T16:14:51.005157+0000","last_clean":"2024-06-28T16:14:51.005157+0000","last_became_active":"2024-06-28T15:00:19.398138+0000","last_became_peered":"2024-06-28T15:00:19.398138+0000","last_unstale":"2024-06-28T16:14:51.005157+0000","last_undegraded":"2024-06-28T16:14:51.005157+0000","last_fullsized":"2024-06-28T16:14:51.005157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.004896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.004896+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.004896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:21:37.359914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"260'304","reported_seq":911,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.170902+0000","last_change":"2024-06-28T16:15:08.170995+0000","last_active":"2024-06-28T16:15:08.170902+0000","last_peered":"2024-06-28T16:15:08.170902+0000","last_clean":"2024-06-28T16:15:08.170902+0000","last_became_active":"2024-06-28T15:00:21.405016+0000","last_became_peered":"2024-06-28T15:00:21.405016+0000","last_unstale":"2024-06-28T16:15:08.170902+0000","last_undegraded":"2024-06-28T16:15:08.170902+0000","last_fullsized":"2024-06-28T16:15:08.170902+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'304","last_scrub_stamp":"2024-06-28T16:15:08.170703+0000","last_deep_scrub":"260'304","last_deep_scrub_stamp":"2024-06-28T16:15:08.170703+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.170703+0000","objects_scrubbed":6,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:57:06.513866+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53726,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":505,"num_read_kb":1165,"num_write":406,"num_write_kb":309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":509,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.204716+0000","last_change":"2024-06-28T16:15:00.204716+0000","last_active":"2024-06-28T16:15:00.204716+0000","last_peered":"2024-06-28T16:15:00.204716+0000","last_clean":"2024-06-28T16:15:00.204716+0000","last_became_active":"2024-06-28T15:00:23.417390+0000","last_became_peered":"2024-06-28T15:00:23.417390+0000","last_unstale":"2024-06-28T16:15:00.204716+0000","last_undegraded":"2024-06-28T16:15:00.204716+0000","last_fullsized":"2024-06-28T16:15:00.204716+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:00.204463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:00.204463+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.204463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:41:34.935690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"260'101","reported_seq":617,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.114583+0000","last_change":"2024-06-28T16:14:56.114642+0000","last_active":"2024-06-28T16:14:56.114583+0000","last_peered":"2024-06-28T16:14:56.114583+0000","last_clean":"2024-06-28T16:14:56.114583+0000","last_became_active":"2024-06-28T15:00:25.412379+0000","last_became_peered":"2024-06-28T15:00:25.412379+0000","last_unstale":"2024-06-28T16:14:56.114583+0000","last_undegraded":"2024-06-28T16:14:56.114583+0000","last_fullsized":"2024-06-28T16:14:56.114583+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'101","last_scrub_stamp":"2024-06-28T16:14:56.114398+0000","last_deep_scrub":"260'101","last_deep_scrub_stamp":"2024-06-28T16:14:56.114398+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.114398+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:50:04.756947+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":88,"num_write":238,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"260'189","reported_seq":832,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.954505+0000","last_change":"2024-06-28T16:14:57.954505+0000","last_active":"2024-06-28T16:14:57.954505+0000","last_peered":"2024-06-28T16:14:57.954505+0000","last_clean":"2024-06-28T16:14:57.954505+0000","last_became_active":"2024-06-28T15:00:17.406647+0000","last_became_peered":"2024-06-28T15:00:17.406647+0000","last_unstale":"2024-06-28T16:14:57.954505+0000","last_undegraded":"2024-06-28T16:14:57.954505+0000","last_fullsized":"2024-06-28T16:14:57.954505+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'189","last_scrub_stamp":"2024-06-28T16:14:57.954211+0000","last_deep_scrub":"260'189","last_deep_scrub_stamp":"2024-06-28T16:14:57.954211+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.954211+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:15:57.967237+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":748,"num_read_kb":601,"num_write":301,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"260'334","reported_seq":1011,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:27.040044+0000","last_change":"2024-06-28T16:15:27.040348+0000","last_active":"2024-06-28T16:15:27.040044+0000","last_peered":"2024-06-28T16:15:27.040044+0000","last_clean":"2024-06-28T16:15:27.040044+0000","last_became_active":"2024-06-28T15:00:15.402945+0000","last_became_peered":"2024-06-28T15:00:15.402945+0000","last_unstale":"2024-06-28T16:15:27.040044+0000","last_undegraded":"2024-06-28T16:15:27.040044+0000","last_fullsized":"2024-06-28T16:15:27.040044+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'334","last_scrub_stamp":"2024-06-28T16:15:27.039840+0000","last_deep_scrub":"260'334","last_deep_scrub_stamp":"2024-06-28T16:15:27.039840+0000","last_clean_scrub_stamp":"2024-06-28T16:15:27.039840+0000","objects_scrubbed":9,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:02:38.767341+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340417,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":534,"num_read_kb":20609,"num_write":1305,"num_write_kb":45266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,4],"acting":[7,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":517,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:21.320178+0000","last_change":"2024-06-28T16:15:21.320284+0000","last_active":"2024-06-28T16:15:21.320178+0000","last_peered":"2024-06-28T16:15:21.320178+0000","last_clean":"2024-06-28T16:15:21.320178+0000","last_became_active":"2024-06-28T15:00:19.382839+0000","last_became_peered":"2024-06-28T15:00:19.382839+0000","last_unstale":"2024-06-28T16:15:21.320178+0000","last_undegraded":"2024-06-28T16:15:21.320178+0000","last_fullsized":"2024-06-28T16:15:21.320178+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:21.319924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:21.319924+0000","last_clean_scrub_stamp":"2024-06-28T16:15:21.319924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:57:39.016603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"260'55","reported_seq":586,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.212479+0000","last_change":"2024-06-28T16:15:07.212638+0000","last_active":"2024-06-28T16:15:07.212479+0000","last_peered":"2024-06-28T16:15:07.212479+0000","last_clean":"2024-06-28T16:15:07.212479+0000","last_became_active":"2024-06-28T15:00:21.404727+0000","last_became_peered":"2024-06-28T15:00:21.404727+0000","last_unstale":"2024-06-28T16:15:07.212479+0000","last_undegraded":"2024-06-28T16:15:07.212479+0000","last_fullsized":"2024-06-28T16:15:07.212479+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'55","last_scrub_stamp":"2024-06-28T16:15:07.212261+0000","last_deep_scrub":"260'55","last_deep_scrub_stamp":"2024-06-28T16:15:07.212261+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.212261+0000","objects_scrubbed":3,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:08:38.234431+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":48,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.776013+0000","last_change":"2024-06-28T16:15:01.776013+0000","last_active":"2024-06-28T16:15:01.776013+0000","last_peered":"2024-06-28T16:15:01.776013+0000","last_clean":"2024-06-28T16:15:01.776013+0000","last_became_active":"2024-06-28T15:00:23.419115+0000","last_became_peered":"2024-06-28T15:00:23.419115+0000","last_unstale":"2024-06-28T16:15:01.776013+0000","last_undegraded":"2024-06-28T16:15:01.776013+0000","last_fullsized":"2024-06-28T16:15:01.776013+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:01.775641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:01.775641+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.775641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:56:43.293391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"260'191","reported_seq":730,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.032091+0000","last_change":"2024-06-28T16:14:55.032091+0000","last_active":"2024-06-28T16:14:55.032091+0000","last_peered":"2024-06-28T16:14:55.032091+0000","last_clean":"2024-06-28T16:14:55.032091+0000","last_became_active":"2024-06-28T15:00:25.426046+0000","last_became_peered":"2024-06-28T15:00:25.426046+0000","last_unstale":"2024-06-28T16:14:55.032091+0000","last_undegraded":"2024-06-28T16:14:55.032091+0000","last_fullsized":"2024-06-28T16:14:55.032091+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'191","last_scrub_stamp":"2024-06-28T16:14:55.031799+0000","last_deep_scrub":"260'191","last_deep_scrub_stamp":"2024-06-28T16:14:55.031799+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.031799+0000","objects_scrubbed":1,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:06:38.230720+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":286,"num_read_kb":283,"num_write":434,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"260'423","reported_seq":1180,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.153858+0000","last_change":"2024-06-28T16:15:13.154003+0000","last_active":"2024-06-28T16:15:13.153858+0000","last_peered":"2024-06-28T16:15:13.153858+0000","last_clean":"2024-06-28T16:15:13.153858+0000","last_became_active":"2024-06-28T15:00:17.389992+0000","last_became_peered":"2024-06-28T15:00:17.389992+0000","last_unstale":"2024-06-28T16:15:13.153858+0000","last_undegraded":"2024-06-28T16:15:13.153858+0000","last_fullsized":"2024-06-28T16:15:13.153858+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'423","last_scrub_stamp":"2024-06-28T16:15:13.153624+0000","last_deep_scrub":"260'423","last_deep_scrub_stamp":"2024-06-28T16:15:13.153624+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.153624+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":400,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:59:03.430418+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1786,"num_read_kb":1490,"num_write":759,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"260'328","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.730674+0000","last_change":"2024-06-28T16:15:13.730884+0000","last_active":"2024-06-28T16:15:13.730674+0000","last_peered":"2024-06-28T16:15:13.730674+0000","last_clean":"2024-06-28T16:15:13.730674+0000","last_became_active":"2024-06-28T15:00:15.419267+0000","last_became_peered":"2024-06-28T15:00:15.419267+0000","last_unstale":"2024-06-28T16:15:13.730674+0000","last_undegraded":"2024-06-28T16:15:13.730674+0000","last_fullsized":"2024-06-28T16:15:13.730674+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'328","last_scrub_stamp":"2024-06-28T16:15:13.730424+0000","last_deep_scrub":"260'328","last_deep_scrub_stamp":"2024-06-28T16:15:13.730424+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.730424+0000","objects_scrubbed":5,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:57:21.223073+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5413475,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":487,"num_read_kb":27459,"num_write":1288,"num_write_kb":48933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":512,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.747129+0000","last_change":"2024-06-28T16:15:06.747183+0000","last_active":"2024-06-28T16:15:06.747129+0000","last_peered":"2024-06-28T16:15:06.747129+0000","last_clean":"2024-06-28T16:15:06.747129+0000","last_became_active":"2024-06-28T15:00:19.385990+0000","last_became_peered":"2024-06-28T15:00:19.385990+0000","last_unstale":"2024-06-28T16:15:06.747129+0000","last_undegraded":"2024-06-28T16:15:06.747129+0000","last_fullsized":"2024-06-28T16:15:06.747129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:06.746938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:06.746938+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.746938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:07:08.302586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"260'323","reported_seq":946,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.152026+0000","last_change":"2024-06-28T16:15:07.152103+0000","last_active":"2024-06-28T16:15:07.152026+0000","last_peered":"2024-06-28T16:15:07.152026+0000","last_clean":"2024-06-28T16:15:07.152026+0000","last_became_active":"2024-06-28T15:00:21.405045+0000","last_became_peered":"2024-06-28T15:00:21.405045+0000","last_unstale":"2024-06-28T16:15:07.152026+0000","last_undegraded":"2024-06-28T16:15:07.152026+0000","last_fullsized":"2024-06-28T16:15:07.152026+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'323","last_scrub_stamp":"2024-06-28T16:15:07.151832+0000","last_deep_scrub":"260'323","last_deep_scrub_stamp":"2024-06-28T16:15:07.151832+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.151832+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:20:40.489212+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48456,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":530,"num_read_kb":1162,"num_write":414,"num_write_kb":289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.144381+0000","last_change":"2024-06-28T16:14:50.144381+0000","last_active":"2024-06-28T16:14:50.144381+0000","last_peered":"2024-06-28T16:14:50.144381+0000","last_clean":"2024-06-28T16:14:50.144381+0000","last_became_active":"2024-06-28T15:00:23.411264+0000","last_became_peered":"2024-06-28T15:00:23.411264+0000","last_unstale":"2024-06-28T16:14:50.144381+0000","last_undegraded":"2024-06-28T16:14:50.144381+0000","last_fullsized":"2024-06-28T16:14:50.144381+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:50.144047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:50.144047+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.144047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:56:31.669987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"260'113","reported_seq":615,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.683376+0000","last_change":"2024-06-28T16:14:43.683376+0000","last_active":"2024-06-28T16:14:43.683376+0000","last_peered":"2024-06-28T16:14:43.683376+0000","last_clean":"2024-06-28T16:14:43.683376+0000","last_became_active":"2024-06-28T15:00:25.442010+0000","last_became_peered":"2024-06-28T15:00:25.442010+0000","last_unstale":"2024-06-28T16:14:43.683376+0000","last_undegraded":"2024-06-28T16:14:43.683376+0000","last_fullsized":"2024-06-28T16:14:43.683376+0000","mapping_epoch":29,"log_start":"260'101","ondisk_log_start":"260'101","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'113","last_scrub_stamp":"2024-06-28T16:14:43.682925+0000","last_deep_scrub":"260'113","last_deep_scrub_stamp":"2024-06-28T16:14:43.682925+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.682925+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":101,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:07:47.348656+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":638,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":85,"num_write":260,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"260'105","reported_seq":679,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:20.277565+0000","last_change":"2024-06-28T16:15:20.277626+0000","last_active":"2024-06-28T16:15:20.277565+0000","last_peered":"2024-06-28T16:15:20.277565+0000","last_clean":"2024-06-28T16:15:20.277565+0000","last_became_active":"2024-06-28T15:00:17.373396+0000","last_became_peered":"2024-06-28T15:00:17.373396+0000","last_unstale":"2024-06-28T16:15:20.277565+0000","last_undegraded":"2024-06-28T16:15:20.277565+0000","last_fullsized":"2024-06-28T16:15:20.277565+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'105","last_scrub_stamp":"2024-06-28T16:15:20.277371+0000","last_deep_scrub":"260'105","last_deep_scrub_stamp":"2024-06-28T16:15:20.277371+0000","last_clean_scrub_stamp":"2024-06-28T16:15:20.277371+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:05:20.707326+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":424,"num_read_kb":340,"num_write":155,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"260'307","reported_seq":964,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:30.286189+0000","last_change":"2024-06-28T16:15:30.286346+0000","last_active":"2024-06-28T16:15:30.286189+0000","last_peered":"2024-06-28T16:15:30.286189+0000","last_clean":"2024-06-28T16:15:30.286189+0000","last_became_active":"2024-06-28T15:00:15.412333+0000","last_became_peered":"2024-06-28T15:00:15.412333+0000","last_unstale":"2024-06-28T16:15:30.286189+0000","last_undegraded":"2024-06-28T16:15:30.286189+0000","last_fullsized":"2024-06-28T16:15:30.286189+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'307","last_scrub_stamp":"2024-06-28T16:15:30.285983+0000","last_deep_scrub":"260'307","last_deep_scrub_stamp":"2024-06-28T16:15:30.285983+0000","last_clean_scrub_stamp":"2024-06-28T16:15:30.285983+0000","objects_scrubbed":8,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:20:34.060427+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1451455,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":458,"num_read_kb":31119,"num_write":1276,"num_write_kb":43155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.188245+0000","last_change":"2024-06-28T16:14:51.188245+0000","last_active":"2024-06-28T16:14:51.188245+0000","last_peered":"2024-06-28T16:14:51.188245+0000","last_clean":"2024-06-28T16:14:51.188245+0000","last_became_active":"2024-06-28T15:00:19.386645+0000","last_became_peered":"2024-06-28T15:00:19.386645+0000","last_unstale":"2024-06-28T16:14:51.188245+0000","last_undegraded":"2024-06-28T16:14:51.188245+0000","last_fullsized":"2024-06-28T16:14:51.188245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.187878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.187878+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.187878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:53:35.500317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"260'293","reported_seq":887,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.836441+0000","last_change":"2024-06-28T16:15:16.836538+0000","last_active":"2024-06-28T16:15:16.836441+0000","last_peered":"2024-06-28T16:15:16.836441+0000","last_clean":"2024-06-28T16:15:16.836441+0000","last_became_active":"2024-06-28T15:00:21.394357+0000","last_became_peered":"2024-06-28T15:00:21.394357+0000","last_unstale":"2024-06-28T16:15:16.836441+0000","last_undegraded":"2024-06-28T16:15:16.836441+0000","last_fullsized":"2024-06-28T16:15:16.836441+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'293","last_scrub_stamp":"2024-06-28T16:15:16.836265+0000","last_deep_scrub":"260'293","last_deep_scrub_stamp":"2024-06-28T16:15:16.836265+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.836265+0000","objects_scrubbed":4,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:39:45.845728+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53697,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":496,"num_read_kb":1194,"num_write":404,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":511,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.135809+0000","last_change":"2024-06-28T16:14:59.135809+0000","last_active":"2024-06-28T16:14:59.135809+0000","last_peered":"2024-06-28T16:14:59.135809+0000","last_clean":"2024-06-28T16:14:59.135809+0000","last_became_active":"2024-06-28T15:00:23.406296+0000","last_became_peered":"2024-06-28T15:00:23.406296+0000","last_unstale":"2024-06-28T16:14:59.135809+0000","last_undegraded":"2024-06-28T16:14:59.135809+0000","last_fullsized":"2024-06-28T16:14:59.135809+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:59.135545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:59.135545+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.135545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:33:44.459217+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"260'68","reported_seq":646,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.449222+0000","last_change":"2024-06-28T16:15:01.449222+0000","last_active":"2024-06-28T16:15:01.449222+0000","last_peered":"2024-06-28T16:15:01.449222+0000","last_clean":"2024-06-28T16:15:01.449222+0000","last_became_active":"2024-06-28T15:00:17.396861+0000","last_became_peered":"2024-06-28T15:00:17.396861+0000","last_unstale":"2024-06-28T16:15:01.449222+0000","last_undegraded":"2024-06-28T16:15:01.449222+0000","last_fullsized":"2024-06-28T16:15:01.449222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'68","last_scrub_stamp":"2024-06-28T16:15:01.448976+0000","last_deep_scrub":"260'68","last_deep_scrub_stamp":"2024-06-28T16:15:01.448976+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.448976+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:55:04.325605+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":279,"num_read_kb":229,"num_write":93,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"260'309","reported_seq":954,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:28.303700+0000","last_change":"2024-06-28T16:15:28.303812+0000","last_active":"2024-06-28T16:15:28.303700+0000","last_peered":"2024-06-28T16:15:28.303700+0000","last_clean":"2024-06-28T16:15:28.303700+0000","last_became_active":"2024-06-28T15:00:15.383008+0000","last_became_peered":"2024-06-28T15:00:15.383008+0000","last_unstale":"2024-06-28T16:15:28.303700+0000","last_undegraded":"2024-06-28T16:15:28.303700+0000","last_fullsized":"2024-06-28T16:15:28.303700+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'309","last_scrub_stamp":"2024-06-28T16:15:28.303480+0000","last_deep_scrub":"260'309","last_deep_scrub_stamp":"2024-06-28T16:15:28.303480+0000","last_clean_scrub_stamp":"2024-06-28T16:15:28.303480+0000","objects_scrubbed":6,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:25:28.517303+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2156401,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":459,"num_read_kb":6724,"num_write":1144,"num_write_kb":24207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.216032+0000","last_change":"2024-06-28T16:15:18.216084+0000","last_active":"2024-06-28T16:15:18.216032+0000","last_peered":"2024-06-28T16:15:18.216032+0000","last_clean":"2024-06-28T16:15:18.216032+0000","last_became_active":"2024-06-28T15:00:19.390111+0000","last_became_peered":"2024-06-28T15:00:19.390111+0000","last_unstale":"2024-06-28T16:15:18.216032+0000","last_undegraded":"2024-06-28T16:15:18.216032+0000","last_fullsized":"2024-06-28T16:15:18.216032+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:18.215836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:18.215836+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.215836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:01:06.524192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"260'476","reported_seq":1136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.999846+0000","last_change":"2024-06-28T16:14:54.999846+0000","last_active":"2024-06-28T16:14:54.999846+0000","last_peered":"2024-06-28T16:14:54.999846+0000","last_clean":"2024-06-28T16:14:54.999846+0000","last_became_active":"2024-06-28T15:00:21.398262+0000","last_became_peered":"2024-06-28T15:00:21.398262+0000","last_unstale":"2024-06-28T16:14:54.999846+0000","last_undegraded":"2024-06-28T16:14:54.999846+0000","last_fullsized":"2024-06-28T16:14:54.999846+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'476","last_scrub_stamp":"2024-06-28T16:14:54.999556+0000","last_deep_scrub":"260'476","last_deep_scrub_stamp":"2024-06-28T16:14:54.999556+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.999556+0000","objects_scrubbed":6,"log_size":76,"log_dups_size":400,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:46:08.548671+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86979,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":863,"num_read_kb":2070,"num_write":734,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24966,"num_omap_keys":48,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.607103+0000","last_change":"2024-06-28T16:14:53.607103+0000","last_active":"2024-06-28T16:14:53.607103+0000","last_peered":"2024-06-28T16:14:53.607103+0000","last_clean":"2024-06-28T16:14:53.607103+0000","last_became_active":"2024-06-28T15:00:23.410519+0000","last_became_peered":"2024-06-28T15:00:23.410519+0000","last_unstale":"2024-06-28T16:14:53.607103+0000","last_undegraded":"2024-06-28T16:14:53.607103+0000","last_fullsized":"2024-06-28T16:14:53.607103+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.606849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.606849+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.606849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:00:33.451749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"260'170","reported_seq":697,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.126982+0000","last_change":"2024-06-28T16:14:50.126982+0000","last_active":"2024-06-28T16:14:50.126982+0000","last_peered":"2024-06-28T16:14:50.126982+0000","last_clean":"2024-06-28T16:14:50.126982+0000","last_became_active":"2024-06-28T15:00:25.558426+0000","last_became_peered":"2024-06-28T15:00:25.558426+0000","last_unstale":"2024-06-28T16:14:50.126982+0000","last_undegraded":"2024-06-28T16:14:50.126982+0000","last_fullsized":"2024-06-28T16:14:50.126982+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'170","last_scrub_stamp":"2024-06-28T16:14:50.126565+0000","last_deep_scrub":"260'170","last_deep_scrub_stamp":"2024-06-28T16:14:50.126565+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.126565+0000","objects_scrubbed":3,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:28:56.261088+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":845,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":215,"num_read_kb":219,"num_write":377,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"260'159","reported_seq":797,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.243927+0000","last_change":"2024-06-28T16:15:08.243996+0000","last_active":"2024-06-28T16:15:08.243927+0000","last_peered":"2024-06-28T16:15:08.243927+0000","last_clean":"2024-06-28T16:15:08.243927+0000","last_became_active":"2024-06-28T15:00:17.395643+0000","last_became_peered":"2024-06-28T15:00:17.395643+0000","last_unstale":"2024-06-28T16:15:08.243927+0000","last_undegraded":"2024-06-28T16:15:08.243927+0000","last_fullsized":"2024-06-28T16:15:08.243927+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'159","last_scrub_stamp":"2024-06-28T16:15:08.243717+0000","last_deep_scrub":"260'159","last_deep_scrub_stamp":"2024-06-28T16:15:08.243717+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.243717+0000","objects_scrubbed":1,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:49:47.952363+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":654,"num_read_kb":561,"num_write":257,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2876,"num_omap_keys":12,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"260'290","reported_seq":940,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.075499+0000","last_change":"2024-06-28T16:14:44.075499+0000","last_active":"2024-06-28T16:14:44.075499+0000","last_peered":"2024-06-28T16:14:44.075499+0000","last_clean":"2024-06-28T16:14:44.075499+0000","last_became_active":"2024-06-28T15:00:59.939485+0000","last_became_peered":"2024-06-28T15:00:59.939485+0000","last_unstale":"2024-06-28T16:14:44.075499+0000","last_undegraded":"2024-06-28T16:14:44.075499+0000","last_fullsized":"2024-06-28T16:14:44.075499+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'290","last_scrub_stamp":"2024-06-28T16:14:44.075189+0000","last_deep_scrub":"260'290","last_deep_scrub_stamp":"2024-06-28T16:14:44.075189+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.075189+0000","objects_scrubbed":3,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:23:04.302226+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":430,"num_read_kb":21907,"num_write":1034,"num_write_kb":35159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,6],"acting":[1,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.274570+0000","last_change":"2024-06-28T16:14:53.274570+0000","last_active":"2024-06-28T16:14:53.274570+0000","last_peered":"2024-06-28T16:14:53.274570+0000","last_clean":"2024-06-28T16:14:53.274570+0000","last_became_active":"2024-06-28T15:00:19.385068+0000","last_became_peered":"2024-06-28T15:00:19.385068+0000","last_unstale":"2024-06-28T16:14:53.274570+0000","last_undegraded":"2024-06-28T16:14:53.274570+0000","last_fullsized":"2024-06-28T16:14:53.274570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.274324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.274324+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.274324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:11:33.949188+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"260'356","reported_seq":972,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.835932+0000","last_change":"2024-06-28T16:15:18.835932+0000","last_active":"2024-06-28T16:15:18.835932+0000","last_peered":"2024-06-28T16:15:18.835932+0000","last_clean":"2024-06-28T16:15:18.835932+0000","last_became_active":"2024-06-28T15:00:21.400480+0000","last_became_peered":"2024-06-28T15:00:21.400480+0000","last_unstale":"2024-06-28T16:15:18.835932+0000","last_undegraded":"2024-06-28T16:15:18.835932+0000","last_fullsized":"2024-06-28T16:15:18.835932+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'356","last_scrub_stamp":"2024-06-28T16:15:18.835637+0000","last_deep_scrub":"260'356","last_deep_scrub_stamp":"2024-06-28T16:15:18.835637+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.835637+0000","objects_scrubbed":8,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:15:40.539768+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47708,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":581,"num_read_kb":1249,"num_write":480,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11460,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.168340+0000","last_change":"2024-06-28T16:15:05.168340+0000","last_active":"2024-06-28T16:15:05.168340+0000","last_peered":"2024-06-28T16:15:05.168340+0000","last_clean":"2024-06-28T16:15:05.168340+0000","last_became_active":"2024-06-28T15:00:23.417580+0000","last_became_peered":"2024-06-28T15:00:23.417580+0000","last_unstale":"2024-06-28T16:15:05.168340+0000","last_undegraded":"2024-06-28T16:15:05.168340+0000","last_fullsized":"2024-06-28T16:15:05.168340+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:05.168092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:05.168092+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.168092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:54:08.480636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"260'176","reported_seq":693,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.494904+0000","last_change":"2024-06-28T16:15:02.494904+0000","last_active":"2024-06-28T16:15:02.494904+0000","last_peered":"2024-06-28T16:15:02.494904+0000","last_clean":"2024-06-28T16:15:02.494904+0000","last_became_active":"2024-06-28T15:00:25.430299+0000","last_became_peered":"2024-06-28T15:00:25.430299+0000","last_unstale":"2024-06-28T16:15:02.494904+0000","last_undegraded":"2024-06-28T16:15:02.494904+0000","last_fullsized":"2024-06-28T16:15:02.494904+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'176","last_scrub_stamp":"2024-06-28T16:15:02.494614+0000","last_deep_scrub":"260'176","last_deep_scrub_stamp":"2024-06-28T16:15:02.494614+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.494614+0000","objects_scrubbed":6,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:35:42.752972+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2035,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":193,"num_read_kb":191,"num_write":477,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"260'315","reported_seq":1067,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.121432+0000","last_change":"2024-06-28T16:15:12.121432+0000","last_active":"2024-06-28T16:15:12.121432+0000","last_peered":"2024-06-28T16:15:12.121432+0000","last_clean":"2024-06-28T16:15:12.121432+0000","last_became_active":"2024-06-28T15:00:17.383284+0000","last_became_peered":"2024-06-28T15:00:17.383284+0000","last_unstale":"2024-06-28T16:15:12.121432+0000","last_undegraded":"2024-06-28T16:15:12.121432+0000","last_fullsized":"2024-06-28T16:15:12.121432+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'315","last_scrub_stamp":"2024-06-28T16:15:12.121164+0000","last_deep_scrub":"260'315","last_deep_scrub_stamp":"2024-06-28T16:15:12.121164+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.121164+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:19:12.129939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1515,"num_read_kb":1335,"num_write":548,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"260'347","reported_seq":1038,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.960069+0000","last_change":"2024-06-28T16:15:16.960532+0000","last_active":"2024-06-28T16:15:16.960069+0000","last_peered":"2024-06-28T16:15:16.960069+0000","last_clean":"2024-06-28T16:15:16.960069+0000","last_became_active":"2024-06-28T15:00:15.405499+0000","last_became_peered":"2024-06-28T15:00:15.405499+0000","last_unstale":"2024-06-28T16:15:16.960069+0000","last_undegraded":"2024-06-28T16:15:16.960069+0000","last_fullsized":"2024-06-28T16:15:16.960069+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'347","last_scrub_stamp":"2024-06-28T16:15:16.959776+0000","last_deep_scrub":"260'347","last_deep_scrub_stamp":"2024-06-28T16:15:16.959776+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.959776+0000","objects_scrubbed":13,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:25:44.951698+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17426850,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":585,"num_read_kb":18043,"num_write":1303,"num_write_kb":33783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":512,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.087242+0000","last_change":"2024-06-28T16:15:11.087242+0000","last_active":"2024-06-28T16:15:11.087242+0000","last_peered":"2024-06-28T16:15:11.087242+0000","last_clean":"2024-06-28T16:15:11.087242+0000","last_became_active":"2024-06-28T15:00:19.378852+0000","last_became_peered":"2024-06-28T15:00:19.378852+0000","last_unstale":"2024-06-28T16:15:11.087242+0000","last_undegraded":"2024-06-28T16:15:11.087242+0000","last_fullsized":"2024-06-28T16:15:11.087242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:11.086992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:11.086992+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.086992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:18:25.029287+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"260'285","reported_seq":896,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.193472+0000","last_change":"2024-06-28T16:15:06.193472+0000","last_active":"2024-06-28T16:15:06.193472+0000","last_peered":"2024-06-28T16:15:06.193472+0000","last_clean":"2024-06-28T16:15:06.193472+0000","last_became_active":"2024-06-28T15:00:21.480868+0000","last_became_peered":"2024-06-28T15:00:21.480868+0000","last_unstale":"2024-06-28T16:15:06.193472+0000","last_undegraded":"2024-06-28T16:15:06.193472+0000","last_fullsized":"2024-06-28T16:15:06.193472+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'285","last_scrub_stamp":"2024-06-28T16:15:06.193211+0000","last_deep_scrub":"260'285","last_deep_scrub_stamp":"2024-06-28T16:15:06.193211+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.193211+0000","objects_scrubbed":6,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:07:51.229985+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40850,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":446,"num_read_kb":956,"num_write":348,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.011327+0000","last_change":"2024-06-28T16:15:01.011381+0000","last_active":"2024-06-28T16:15:01.011327+0000","last_peered":"2024-06-28T16:15:01.011327+0000","last_clean":"2024-06-28T16:15:01.011327+0000","last_became_active":"2024-06-28T15:00:23.400417+0000","last_became_peered":"2024-06-28T15:00:23.400417+0000","last_unstale":"2024-06-28T16:15:01.011327+0000","last_undegraded":"2024-06-28T16:15:01.011327+0000","last_fullsized":"2024-06-28T16:15:01.011327+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:01.011100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:01.011100+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.011100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:29:13.769857+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"260'121","reported_seq":631,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.687682+0000","last_change":"2024-06-28T16:14:50.687682+0000","last_active":"2024-06-28T16:14:50.687682+0000","last_peered":"2024-06-28T16:14:50.687682+0000","last_clean":"2024-06-28T16:14:50.687682+0000","last_became_active":"2024-06-28T15:00:25.438176+0000","last_became_peered":"2024-06-28T15:00:25.438176+0000","last_unstale":"2024-06-28T16:14:50.687682+0000","last_undegraded":"2024-06-28T16:14:50.687682+0000","last_fullsized":"2024-06-28T16:14:50.687682+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'121","last_scrub_stamp":"2024-06-28T16:14:50.687377+0000","last_deep_scrub":"260'121","last_deep_scrub_stamp":"2024-06-28T16:14:50.687377+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.687377+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:06:30.493792+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":120,"num_read_kb":111,"num_write":277,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"260'74","reported_seq":665,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.334685+0000","last_change":"2024-06-28T16:15:02.334685+0000","last_active":"2024-06-28T16:15:02.334685+0000","last_peered":"2024-06-28T16:15:02.334685+0000","last_clean":"2024-06-28T16:15:02.334685+0000","last_became_active":"2024-06-28T15:00:17.402735+0000","last_became_peered":"2024-06-28T15:00:17.402735+0000","last_unstale":"2024-06-28T16:15:02.334685+0000","last_undegraded":"2024-06-28T16:15:02.334685+0000","last_fullsized":"2024-06-28T16:15:02.334685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'74","last_scrub_stamp":"2024-06-28T16:15:02.334423+0000","last_deep_scrub":"260'74","last_deep_scrub_stamp":"2024-06-28T16:15:02.334423+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.334423+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:29:13.249710+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":299,"num_read_kb":249,"num_write":100,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"260'336","reported_seq":1068,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:23.105933+0000","last_change":"2024-06-28T16:15:23.106192+0000","last_active":"2024-06-28T16:15:23.105933+0000","last_peered":"2024-06-28T16:15:23.105933+0000","last_clean":"2024-06-28T16:15:23.105933+0000","last_became_active":"2024-06-28T15:00:15.415197+0000","last_became_peered":"2024-06-28T15:00:15.415197+0000","last_unstale":"2024-06-28T16:15:23.105933+0000","last_undegraded":"2024-06-28T16:15:23.105933+0000","last_fullsized":"2024-06-28T16:15:23.105933+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'336","last_scrub_stamp":"2024-06-28T16:15:23.105650+0000","last_deep_scrub":"260'336","last_deep_scrub_stamp":"2024-06-28T16:15:23.105650+0000","last_clean_scrub_stamp":"2024-06-28T16:15:23.105650+0000","objects_scrubbed":5,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:23:46.778415+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3732560,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":648,"num_read_kb":6210,"num_write":1163,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":516,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.967053+0000","last_change":"2024-06-28T16:14:51.967053+0000","last_active":"2024-06-28T16:14:51.967053+0000","last_peered":"2024-06-28T16:14:51.967053+0000","last_clean":"2024-06-28T16:14:51.967053+0000","last_became_active":"2024-06-28T15:00:19.378270+0000","last_became_peered":"2024-06-28T15:00:19.378270+0000","last_unstale":"2024-06-28T16:14:51.967053+0000","last_undegraded":"2024-06-28T16:14:51.967053+0000","last_fullsized":"2024-06-28T16:14:51.967053+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.966801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.966801+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.966801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:16:26.209301+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"260'107","reported_seq":662,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.993632+0000","last_change":"2024-06-28T16:14:51.993706+0000","last_active":"2024-06-28T16:14:51.993632+0000","last_peered":"2024-06-28T16:14:51.993632+0000","last_clean":"2024-06-28T16:14:51.993632+0000","last_became_active":"2024-06-28T15:00:21.484352+0000","last_became_peered":"2024-06-28T15:00:21.484352+0000","last_unstale":"2024-06-28T16:14:51.993632+0000","last_undegraded":"2024-06-28T16:14:51.993632+0000","last_fullsized":"2024-06-28T16:14:51.993632+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'107","last_scrub_stamp":"2024-06-28T16:14:51.993438+0000","last_deep_scrub":"260'107","last_deep_scrub_stamp":"2024-06-28T16:14:51.993438+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.993438+0000","objects_scrubbed":6,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:28:06.528863+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":124,"num_read_kb":119,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1780,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"260'515","reported_seq":2255,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.146256+0000","last_change":"2024-06-28T16:14:48.146256+0000","last_active":"2024-06-28T16:14:48.146256+0000","last_peered":"2024-06-28T16:14:48.146256+0000","last_clean":"2024-06-28T16:14:48.146256+0000","last_became_active":"2024-06-28T15:00:23.407490+0000","last_became_peered":"2024-06-28T15:00:23.407490+0000","last_unstale":"2024-06-28T16:14:48.146256+0000","last_undegraded":"2024-06-28T16:14:48.146256+0000","last_fullsized":"2024-06-28T16:14:48.146256+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'515","last_scrub_stamp":"2024-06-28T16:14:48.145991+0000","last_deep_scrub":"260'515","last_deep_scrub_stamp":"2024-06-28T16:14:48.145991+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.145991+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:10:57.232491+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"260'103","reported_seq":617,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.637768+0000","last_change":"2024-06-28T16:14:55.637768+0000","last_active":"2024-06-28T16:14:55.637768+0000","last_peered":"2024-06-28T16:14:55.637768+0000","last_clean":"2024-06-28T16:14:55.637768+0000","last_became_active":"2024-06-28T15:00:25.445018+0000","last_became_peered":"2024-06-28T15:00:25.445018+0000","last_unstale":"2024-06-28T16:14:55.637768+0000","last_undegraded":"2024-06-28T16:14:55.637768+0000","last_fullsized":"2024-06-28T16:14:55.637768+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'103","last_scrub_stamp":"2024-06-28T16:14:55.637507+0000","last_deep_scrub":"260'103","last_deep_scrub_stamp":"2024-06-28T16:14:55.637507+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.637507+0000","objects_scrubbed":1,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:19:26.856020+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":117,"num_read_kb":111,"num_write":241,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"260'78","reported_seq":664,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:20.971945+0000","last_change":"2024-06-28T16:15:20.972078+0000","last_active":"2024-06-28T16:15:20.971945+0000","last_peered":"2024-06-28T16:15:20.971945+0000","last_clean":"2024-06-28T16:15:20.971945+0000","last_became_active":"2024-06-28T15:00:17.387086+0000","last_became_peered":"2024-06-28T15:00:17.387086+0000","last_unstale":"2024-06-28T16:15:20.971945+0000","last_undegraded":"2024-06-28T16:15:20.971945+0000","last_fullsized":"2024-06-28T16:15:20.971945+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'78","last_scrub_stamp":"2024-06-28T16:15:20.971625+0000","last_deep_scrub":"260'78","last_deep_scrub_stamp":"2024-06-28T16:15:20.971625+0000","last_clean_scrub_stamp":"2024-06-28T16:15:20.971625+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:39:40.485714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":333,"num_read_kb":275,"num_write":113,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"260'328","reported_seq":969,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.118128+0000","last_change":"2024-06-28T16:14:44.118252+0000","last_active":"2024-06-28T16:14:44.118128+0000","last_peered":"2024-06-28T16:14:44.118128+0000","last_clean":"2024-06-28T16:14:44.118128+0000","last_became_active":"2024-06-28T15:00:59.941915+0000","last_became_peered":"2024-06-28T15:00:59.941915+0000","last_unstale":"2024-06-28T16:14:44.118128+0000","last_undegraded":"2024-06-28T16:14:44.118128+0000","last_fullsized":"2024-06-28T16:14:44.118128+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'328","last_scrub_stamp":"2024-06-28T16:14:44.117934+0000","last_deep_scrub":"260'328","last_deep_scrub_stamp":"2024-06-28T16:14:44.117934+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.117934+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:54:20.388393+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5506084,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":497,"num_read_kb":57892,"num_write":1218,"num_write_kb":47404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.747922+0000","last_change":"2024-06-28T16:15:12.747922+0000","last_active":"2024-06-28T16:15:12.747922+0000","last_peered":"2024-06-28T16:15:12.747922+0000","last_clean":"2024-06-28T16:15:12.747922+0000","last_became_active":"2024-06-28T15:00:19.390613+0000","last_became_peered":"2024-06-28T15:00:19.390613+0000","last_unstale":"2024-06-28T16:15:12.747922+0000","last_undegraded":"2024-06-28T16:15:12.747922+0000","last_fullsized":"2024-06-28T16:15:12.747922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:12.747667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:12.747667+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.747667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:15:33.069119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"260'331","reported_seq":949,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:22.930866+0000","last_change":"2024-06-28T16:15:22.930866+0000","last_active":"2024-06-28T16:15:22.930866+0000","last_peered":"2024-06-28T16:15:22.930866+0000","last_clean":"2024-06-28T16:15:22.930866+0000","last_became_active":"2024-06-28T15:00:21.391631+0000","last_became_peered":"2024-06-28T15:00:21.391631+0000","last_unstale":"2024-06-28T16:15:22.930866+0000","last_undegraded":"2024-06-28T16:15:22.930866+0000","last_fullsized":"2024-06-28T16:15:22.930866+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'331","last_scrub_stamp":"2024-06-28T16:15:22.930580+0000","last_deep_scrub":"260'331","last_deep_scrub_stamp":"2024-06-28T16:15:22.930580+0000","last_clean_scrub_stamp":"2024-06-28T16:15:22.930580+0000","objects_scrubbed":7,"log_size":31,"log_dups_size":300,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:45:49.759442+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43414,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":516,"num_read_kb":1105,"num_write":400,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.214355+0000","last_change":"2024-06-28T16:15:01.214355+0000","last_active":"2024-06-28T16:15:01.214355+0000","last_peered":"2024-06-28T16:15:01.214355+0000","last_clean":"2024-06-28T16:15:01.214355+0000","last_became_active":"2024-06-28T15:00:23.404312+0000","last_became_peered":"2024-06-28T15:00:23.404312+0000","last_unstale":"2024-06-28T16:15:01.214355+0000","last_undegraded":"2024-06-28T16:15:01.214355+0000","last_fullsized":"2024-06-28T16:15:01.214355+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:01.214103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:01.214103+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.214103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:21:36.177910+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"260'136","reported_seq":657,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.687390+0000","last_change":"2024-06-28T16:14:57.687390+0000","last_active":"2024-06-28T16:14:57.687390+0000","last_peered":"2024-06-28T16:14:57.687390+0000","last_clean":"2024-06-28T16:14:57.687390+0000","last_became_active":"2024-06-28T15:00:25.424826+0000","last_became_peered":"2024-06-28T15:00:25.424826+0000","last_unstale":"2024-06-28T16:14:57.687390+0000","last_undegraded":"2024-06-28T16:14:57.687390+0000","last_fullsized":"2024-06-28T16:14:57.687390+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'136","last_scrub_stamp":"2024-06-28T16:14:57.687124+0000","last_deep_scrub":"260'136","last_deep_scrub_stamp":"2024-06-28T16:14:57.687124+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.687124+0000","objects_scrubbed":5,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:05:03.591761+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":159,"num_read_kb":144,"num_write":309,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"260'118","reported_seq":701,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.933025+0000","last_change":"2024-06-28T16:15:19.933025+0000","last_active":"2024-06-28T16:15:19.933025+0000","last_peered":"2024-06-28T16:15:19.933025+0000","last_clean":"2024-06-28T16:15:19.933025+0000","last_became_active":"2024-06-28T15:00:17.386756+0000","last_became_peered":"2024-06-28T15:00:17.386756+0000","last_unstale":"2024-06-28T16:15:19.933025+0000","last_undegraded":"2024-06-28T16:15:19.933025+0000","last_fullsized":"2024-06-28T16:15:19.933025+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'118","last_scrub_stamp":"2024-06-28T16:15:19.932659+0000","last_deep_scrub":"260'118","last_deep_scrub_stamp":"2024-06-28T16:15:19.932659+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.932659+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:41:38.014558+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":469,"num_read_kb":378,"num_write":166,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"260'293","reported_seq":973,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.180637+0000","last_change":"2024-06-28T16:14:48.180832+0000","last_active":"2024-06-28T16:14:48.180637+0000","last_peered":"2024-06-28T16:14:48.180637+0000","last_clean":"2024-06-28T16:14:48.180637+0000","last_became_active":"2024-06-28T15:00:59.943670+0000","last_became_peered":"2024-06-28T15:00:59.943670+0000","last_unstale":"2024-06-28T16:14:48.180637+0000","last_undegraded":"2024-06-28T16:14:48.180637+0000","last_fullsized":"2024-06-28T16:14:48.180637+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'293","last_scrub_stamp":"2024-06-28T16:14:48.180366+0000","last_deep_scrub":"260'293","last_deep_scrub_stamp":"2024-06-28T16:14:48.180366+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.180366+0000","objects_scrubbed":3,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:11:28.293389+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306233,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":513,"num_read_kb":66858,"num_write":1091,"num_write_kb":26064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":511,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.644404+0000","last_change":"2024-06-28T16:15:04.644404+0000","last_active":"2024-06-28T16:15:04.644404+0000","last_peered":"2024-06-28T16:15:04.644404+0000","last_clean":"2024-06-28T16:15:04.644404+0000","last_became_active":"2024-06-28T15:00:19.398245+0000","last_became_peered":"2024-06-28T15:00:19.398245+0000","last_unstale":"2024-06-28T16:15:04.644404+0000","last_undegraded":"2024-06-28T16:15:04.644404+0000","last_fullsized":"2024-06-28T16:15:04.644404+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:04.644148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:04.644148+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.644148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:23:05.800918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"260'301","reported_seq":901,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:32.343362+0000","last_change":"2024-06-28T16:15:32.343498+0000","last_active":"2024-06-28T16:15:32.343362+0000","last_peered":"2024-06-28T16:15:32.343362+0000","last_clean":"2024-06-28T16:15:32.343362+0000","last_became_active":"2024-06-28T15:00:21.404701+0000","last_became_peered":"2024-06-28T15:00:21.404701+0000","last_unstale":"2024-06-28T16:15:32.343362+0000","last_undegraded":"2024-06-28T16:15:32.343362+0000","last_fullsized":"2024-06-28T16:15:32.343362+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'301","last_scrub_stamp":"2024-06-28T16:15:32.343097+0000","last_deep_scrub":"260'301","last_deep_scrub_stamp":"2024-06-28T16:15:32.343097+0000","last_clean_scrub_stamp":"2024-06-28T16:15:32.343097+0000","objects_scrubbed":4,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:19:20.550850+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44354,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":486,"num_read_kb":1147,"num_write":394,"num_write_kb":305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.234419+0000","last_change":"2024-06-28T16:14:52.234419+0000","last_active":"2024-06-28T16:14:52.234419+0000","last_peered":"2024-06-28T16:14:52.234419+0000","last_clean":"2024-06-28T16:14:52.234419+0000","last_became_active":"2024-06-28T15:00:23.415531+0000","last_became_peered":"2024-06-28T15:00:23.415531+0000","last_unstale":"2024-06-28T16:14:52.234419+0000","last_undegraded":"2024-06-28T16:14:52.234419+0000","last_fullsized":"2024-06-28T16:14:52.234419+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:52.234179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:52.234179+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.234179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:09:52.052832+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"260'168","reported_seq":722,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.154699+0000","last_change":"2024-06-28T16:14:52.154699+0000","last_active":"2024-06-28T16:14:52.154699+0000","last_peered":"2024-06-28T16:14:52.154699+0000","last_clean":"2024-06-28T16:14:52.154699+0000","last_became_active":"2024-06-28T15:00:25.553724+0000","last_became_peered":"2024-06-28T15:00:25.553724+0000","last_unstale":"2024-06-28T16:14:52.154699+0000","last_undegraded":"2024-06-28T16:14:52.154699+0000","last_fullsized":"2024-06-28T16:14:52.154699+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'168","last_scrub_stamp":"2024-06-28T16:14:52.154408+0000","last_deep_scrub":"260'168","last_deep_scrub_stamp":"2024-06-28T16:14:52.154408+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.154408+0000","objects_scrubbed":2,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:26:53.092282+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":555,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":217,"num_write":397,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"260'143","reported_seq":816,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.153578+0000","last_change":"2024-06-28T16:15:11.153578+0000","last_active":"2024-06-28T16:15:11.153578+0000","last_peered":"2024-06-28T16:15:11.153578+0000","last_clean":"2024-06-28T16:15:11.153578+0000","last_became_active":"2024-06-28T15:00:17.383224+0000","last_became_peered":"2024-06-28T15:00:17.383224+0000","last_unstale":"2024-06-28T16:15:11.153578+0000","last_undegraded":"2024-06-28T16:15:11.153578+0000","last_fullsized":"2024-06-28T16:15:11.153578+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'143","last_scrub_stamp":"2024-06-28T16:15:11.153330+0000","last_deep_scrub":"260'143","last_deep_scrub_stamp":"2024-06-28T16:15:11.153330+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.153330+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:59:15.862953+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":649,"num_read_kb":545,"num_write":214,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"260'301","reported_seq":1075,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:24.365485+0000","last_change":"2024-06-28T16:15:24.365603+0000","last_active":"2024-06-28T16:15:24.365485+0000","last_peered":"2024-06-28T16:15:24.365485+0000","last_clean":"2024-06-28T16:15:24.365485+0000","last_became_active":"2024-06-28T15:00:15.423813+0000","last_became_peered":"2024-06-28T15:00:15.423813+0000","last_unstale":"2024-06-28T16:15:24.365485+0000","last_undegraded":"2024-06-28T16:15:24.365485+0000","last_fullsized":"2024-06-28T16:15:24.365485+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'301","last_scrub_stamp":"2024-06-28T16:15:24.365252+0000","last_deep_scrub":"260'301","last_deep_scrub_stamp":"2024-06-28T16:15:24.365252+0000","last_clean_scrub_stamp":"2024-06-28T16:15:24.365252+0000","objects_scrubbed":6,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:53:47.692150+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5724260,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":754,"num_read_kb":37474,"num_write":1107,"num_write_kb":31583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.014004+0000","last_change":"2024-06-28T16:15:13.014004+0000","last_active":"2024-06-28T16:15:13.014004+0000","last_peered":"2024-06-28T16:15:13.014004+0000","last_clean":"2024-06-28T16:15:13.014004+0000","last_became_active":"2024-06-28T15:00:19.383307+0000","last_became_peered":"2024-06-28T15:00:19.383307+0000","last_unstale":"2024-06-28T16:15:13.014004+0000","last_undegraded":"2024-06-28T16:15:13.014004+0000","last_fullsized":"2024-06-28T16:15:13.014004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:13.013754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:13.013754+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.013754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:17:26.792620+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"260'289","reported_seq":870,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:35.247053+0000","last_change":"2024-06-28T16:15:35.247132+0000","last_active":"2024-06-28T16:15:35.247053+0000","last_peered":"2024-06-28T16:15:35.247053+0000","last_clean":"2024-06-28T16:15:35.247053+0000","last_became_active":"2024-06-28T15:00:21.396257+0000","last_became_peered":"2024-06-28T15:00:21.396257+0000","last_unstale":"2024-06-28T16:15:35.247053+0000","last_undegraded":"2024-06-28T16:15:35.247053+0000","last_fullsized":"2024-06-28T16:15:35.247053+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'289","last_scrub_stamp":"2024-06-28T16:15:35.246851+0000","last_deep_scrub":"260'289","last_deep_scrub_stamp":"2024-06-28T16:15:35.246851+0000","last_clean_scrub_stamp":"2024-06-28T16:15:35.246851+0000","objects_scrubbed":3,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:29:48.575801+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43992,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":439,"num_read_kb":1078,"num_write":366,"num_write_kb":297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.973645+0000","last_change":"2024-06-28T16:14:47.973645+0000","last_active":"2024-06-28T16:14:47.973645+0000","last_peered":"2024-06-28T16:14:47.973645+0000","last_clean":"2024-06-28T16:14:47.973645+0000","last_became_active":"2024-06-28T15:00:23.409107+0000","last_became_peered":"2024-06-28T15:00:23.409107+0000","last_unstale":"2024-06-28T16:14:47.973645+0000","last_undegraded":"2024-06-28T16:14:47.973645+0000","last_fullsized":"2024-06-28T16:14:47.973645+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:47.973365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:47.973365+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.973365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:58:08.598920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"260'149","reported_seq":891,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.717817+0000","last_change":"2024-06-28T16:14:58.717817+0000","last_active":"2024-06-28T16:14:58.717817+0000","last_peered":"2024-06-28T16:14:58.717817+0000","last_clean":"2024-06-28T16:14:58.717817+0000","last_became_active":"2024-06-28T15:00:25.424376+0000","last_became_peered":"2024-06-28T15:00:25.424376+0000","last_unstale":"2024-06-28T16:14:58.717817+0000","last_undegraded":"2024-06-28T16:14:58.717817+0000","last_fullsized":"2024-06-28T16:14:58.717817+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'149","last_scrub_stamp":"2024-06-28T16:14:58.717559+0000","last_deep_scrub":"260'149","last_deep_scrub_stamp":"2024-06-28T16:14:58.717559+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.717559+0000","objects_scrubbed":5,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:45:24.602655+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":398,"num_read_kb":429,"num_write":339,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1200,"num_omap_keys":6,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"260'112","reported_seq":709,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.936863+0000","last_change":"2024-06-28T16:15:01.936863+0000","last_active":"2024-06-28T16:15:01.936863+0000","last_peered":"2024-06-28T16:15:01.936863+0000","last_clean":"2024-06-28T16:15:01.936863+0000","last_became_active":"2024-06-28T15:00:17.403007+0000","last_became_peered":"2024-06-28T15:00:17.403007+0000","last_unstale":"2024-06-28T16:15:01.936863+0000","last_undegraded":"2024-06-28T16:15:01.936863+0000","last_fullsized":"2024-06-28T16:15:01.936863+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'112","last_scrub_stamp":"2024-06-28T16:15:01.936598+0000","last_deep_scrub":"260'112","last_deep_scrub_stamp":"2024-06-28T16:15:01.936598+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.936598+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:50:31.559519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":478,"num_read_kb":386,"num_write":182,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"260'379","reported_seq":1056,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:21.180633+0000","last_change":"2024-06-28T16:15:21.180834+0000","last_active":"2024-06-28T16:15:21.180633+0000","last_peered":"2024-06-28T16:15:21.180633+0000","last_clean":"2024-06-28T16:15:21.180633+0000","last_became_active":"2024-06-28T15:00:15.414969+0000","last_became_peered":"2024-06-28T15:00:15.414969+0000","last_unstale":"2024-06-28T16:15:21.180633+0000","last_undegraded":"2024-06-28T16:15:21.180633+0000","last_fullsized":"2024-06-28T16:15:21.180633+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'379","last_scrub_stamp":"2024-06-28T16:15:21.180331+0000","last_deep_scrub":"260'379","last_deep_scrub_stamp":"2024-06-28T16:15:21.180331+0000","last_clean_scrub_stamp":"2024-06-28T16:15:21.180331+0000","objects_scrubbed":9,"log_size":79,"log_dups_size":300,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:59:23.438744+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1604505,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":539,"num_read_kb":3182,"num_write":1321,"num_write_kb":13350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"260'2","reported_seq":519,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.947126+0000","last_change":"2024-06-28T16:14:52.947126+0000","last_active":"2024-06-28T16:14:52.947126+0000","last_peered":"2024-06-28T16:14:52.947126+0000","last_clean":"2024-06-28T16:14:52.947126+0000","last_became_active":"2024-06-28T15:00:19.403344+0000","last_became_peered":"2024-06-28T15:00:19.403344+0000","last_unstale":"2024-06-28T16:14:52.947126+0000","last_undegraded":"2024-06-28T16:14:52.947126+0000","last_fullsized":"2024-06-28T16:14:52.947126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:52.946822+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:52.946822+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.946822+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:53:37.784932+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"260'297","reported_seq":907,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.943802+0000","last_change":"2024-06-28T16:14:55.943802+0000","last_active":"2024-06-28T16:14:55.943802+0000","last_peered":"2024-06-28T16:14:55.943802+0000","last_clean":"2024-06-28T16:14:55.943802+0000","last_became_active":"2024-06-28T15:00:21.387539+0000","last_became_peered":"2024-06-28T15:00:21.387539+0000","last_unstale":"2024-06-28T16:14:55.943802+0000","last_undegraded":"2024-06-28T16:14:55.943802+0000","last_fullsized":"2024-06-28T16:14:55.943802+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'297","last_scrub_stamp":"2024-06-28T16:14:55.943503+0000","last_deep_scrub":"260'297","last_deep_scrub_stamp":"2024-06-28T16:14:55.943503+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.943503+0000","objects_scrubbed":5,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:01:20.919128+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44825,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":494,"num_read_kb":1086,"num_write":388,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":509,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.070879+0000","last_change":"2024-06-28T16:14:56.070879+0000","last_active":"2024-06-28T16:14:56.070879+0000","last_peered":"2024-06-28T16:14:56.070879+0000","last_clean":"2024-06-28T16:14:56.070879+0000","last_became_active":"2024-06-28T15:00:23.416363+0000","last_became_peered":"2024-06-28T15:00:23.416363+0000","last_unstale":"2024-06-28T16:14:56.070879+0000","last_undegraded":"2024-06-28T16:14:56.070879+0000","last_fullsized":"2024-06-28T16:14:56.070879+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.070609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.070609+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.070609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:44:31.199646+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"260'129","reported_seq":636,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.940870+0000","last_change":"2024-06-28T16:14:43.940870+0000","last_active":"2024-06-28T16:14:43.940870+0000","last_peered":"2024-06-28T16:14:43.940870+0000","last_clean":"2024-06-28T16:14:43.940870+0000","last_became_active":"2024-06-28T15:00:25.428959+0000","last_became_peered":"2024-06-28T15:00:25.428959+0000","last_unstale":"2024-06-28T16:14:43.940870+0000","last_undegraded":"2024-06-28T16:14:43.940870+0000","last_fullsized":"2024-06-28T16:14:43.940870+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'129","last_scrub_stamp":"2024-06-28T16:14:43.940462+0000","last_deep_scrub":"260'129","last_deep_scrub_stamp":"2024-06-28T16:14:43.940462+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.940462+0000","objects_scrubbed":3,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:17:21.534335+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":814,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":97,"num_write":299,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"260'175","reported_seq":771,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.005191+0000","last_change":"2024-06-28T16:14:56.005265+0000","last_active":"2024-06-28T16:14:56.005191+0000","last_peered":"2024-06-28T16:14:56.005191+0000","last_clean":"2024-06-28T16:14:56.005191+0000","last_became_active":"2024-06-28T15:00:17.406872+0000","last_became_peered":"2024-06-28T15:00:17.406872+0000","last_unstale":"2024-06-28T16:14:56.005191+0000","last_undegraded":"2024-06-28T16:14:56.005191+0000","last_fullsized":"2024-06-28T16:14:56.005191+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'175","last_scrub_stamp":"2024-06-28T16:14:56.004971+0000","last_deep_scrub":"260'175","last_deep_scrub_stamp":"2024-06-28T16:14:56.004971+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.004971+0000","objects_scrubbed":1,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:44:14.471029+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":660,"num_read_kb":511,"num_write":250,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7564,"num_omap_keys":32,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"260'285","reported_seq":919,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.394078+0000","last_change":"2024-06-28T16:15:07.394078+0000","last_active":"2024-06-28T16:15:07.394078+0000","last_peered":"2024-06-28T16:15:07.394078+0000","last_clean":"2024-06-28T16:15:07.394078+0000","last_became_active":"2024-06-28T15:00:15.408337+0000","last_became_peered":"2024-06-28T15:00:15.408337+0000","last_unstale":"2024-06-28T16:15:07.394078+0000","last_undegraded":"2024-06-28T16:15:07.394078+0000","last_fullsized":"2024-06-28T16:15:07.394078+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'285","last_scrub_stamp":"2024-06-28T16:15:07.393622+0000","last_deep_scrub":"260'285","last_deep_scrub_stamp":"2024-06-28T16:15:07.393622+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.393622+0000","objects_scrubbed":5,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:14:07.466937+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":404,"num_read_kb":33775,"num_write":1000,"num_write_kb":10945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"260'72","reported_seq":683,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.969785+0000","last_change":"2024-06-28T16:14:59.969851+0000","last_active":"2024-06-28T16:14:59.969785+0000","last_peered":"2024-06-28T16:14:59.969785+0000","last_clean":"2024-06-28T16:14:59.969785+0000","last_became_active":"2024-06-28T15:00:17.403938+0000","last_became_peered":"2024-06-28T15:00:17.403938+0000","last_unstale":"2024-06-28T16:14:59.969785+0000","last_undegraded":"2024-06-28T16:14:59.969785+0000","last_fullsized":"2024-06-28T16:14:59.969785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'72","last_scrub_stamp":"2024-06-28T16:14:59.969594+0000","last_deep_scrub":"260'72","last_deep_scrub_stamp":"2024-06-28T16:14:59.969594+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.969594+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:54:37.592959+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":319,"num_read_kb":277,"num_write":96,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"260'322","reported_seq":1005,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.267065+0000","last_change":"2024-06-28T16:14:57.267065+0000","last_active":"2024-06-28T16:14:57.267065+0000","last_peered":"2024-06-28T16:14:57.267065+0000","last_clean":"2024-06-28T16:14:57.267065+0000","last_became_active":"2024-06-28T15:00:15.420261+0000","last_became_peered":"2024-06-28T15:00:15.420261+0000","last_unstale":"2024-06-28T16:14:57.267065+0000","last_undegraded":"2024-06-28T16:14:57.267065+0000","last_fullsized":"2024-06-28T16:14:57.267065+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'322","last_scrub_stamp":"2024-06-28T16:14:57.266750+0000","last_deep_scrub":"260'322","last_deep_scrub_stamp":"2024-06-28T16:14:57.266750+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.266750+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:20:49.339058+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262189,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":529,"num_read_kb":23662,"num_write":1259,"num_write_kb":28174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"260'38","reported_seq":605,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.589352+0000","last_change":"2024-06-28T16:15:00.589416+0000","last_active":"2024-06-28T16:15:00.589352+0000","last_peered":"2024-06-28T16:15:00.589352+0000","last_clean":"2024-06-28T16:15:00.589352+0000","last_became_active":"2024-06-28T15:00:17.393889+0000","last_became_peered":"2024-06-28T15:00:17.393889+0000","last_unstale":"2024-06-28T16:15:00.589352+0000","last_undegraded":"2024-06-28T16:15:00.589352+0000","last_fullsized":"2024-06-28T16:15:00.589352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'38","last_scrub_stamp":"2024-06-28T16:15:00.589144+0000","last_deep_scrub":"260'38","last_deep_scrub_stamp":"2024-06-28T16:15:00.589144+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.589144+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:04:36.901823+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":152,"num_write":58,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"260'358","reported_seq":1009,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:20.874114+0000","last_change":"2024-06-28T16:15:20.874504+0000","last_active":"2024-06-28T16:15:20.874114+0000","last_peered":"2024-06-28T16:15:20.874114+0000","last_clean":"2024-06-28T16:15:20.874114+0000","last_became_active":"2024-06-28T15:00:15.418773+0000","last_became_peered":"2024-06-28T15:00:15.418773+0000","last_unstale":"2024-06-28T16:15:20.874114+0000","last_undegraded":"2024-06-28T16:15:20.874114+0000","last_fullsized":"2024-06-28T16:15:20.874114+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'358","last_scrub_stamp":"2024-06-28T16:15:20.873913+0000","last_deep_scrub":"260'358","last_deep_scrub_stamp":"2024-06-28T16:15:20.873913+0000","last_clean_scrub_stamp":"2024-06-28T16:15:20.873913+0000","objects_scrubbed":9,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:50:36.816524+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8714448,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":553,"num_read_kb":16492,"num_write":1257,"num_write_kb":37641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"260'142","reported_seq":796,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.147070+0000","last_change":"2024-06-28T16:15:06.147193+0000","last_active":"2024-06-28T16:15:06.147070+0000","last_peered":"2024-06-28T16:15:06.147070+0000","last_clean":"2024-06-28T16:15:06.147070+0000","last_became_active":"2024-06-28T15:00:17.398357+0000","last_became_peered":"2024-06-28T15:00:17.398357+0000","last_unstale":"2024-06-28T16:15:06.147070+0000","last_undegraded":"2024-06-28T16:15:06.147070+0000","last_fullsized":"2024-06-28T16:15:06.147070+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'142","last_scrub_stamp":"2024-06-28T16:15:06.146782+0000","last_deep_scrub":"260'142","last_deep_scrub_stamp":"2024-06-28T16:15:06.146782+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.146782+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:02:03.196875+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":623,"num_read_kb":529,"num_write":221,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"260'332","reported_seq":986,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:24.148018+0000","last_change":"2024-06-28T16:15:24.148018+0000","last_active":"2024-06-28T16:15:24.148018+0000","last_peered":"2024-06-28T16:15:24.148018+0000","last_clean":"2024-06-28T16:15:24.148018+0000","last_became_active":"2024-06-28T15:00:15.391150+0000","last_became_peered":"2024-06-28T15:00:15.391150+0000","last_unstale":"2024-06-28T16:15:24.148018+0000","last_undegraded":"2024-06-28T16:15:24.148018+0000","last_fullsized":"2024-06-28T16:15:24.148018+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'332","last_scrub_stamp":"2024-06-28T16:15:24.147482+0000","last_deep_scrub":"260'332","last_deep_scrub_stamp":"2024-06-28T16:15:24.147482+0000","last_clean_scrub_stamp":"2024-06-28T16:15:24.147482+0000","objects_scrubbed":9,"log_size":32,"log_dups_size":300,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:55:18.493042+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1605357,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":453,"num_read_kb":84462,"num_write":1265,"num_write_kb":41774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"260'192","reported_seq":798,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.132066+0000","last_change":"2024-06-28T16:14:42.132111+0000","last_active":"2024-06-28T16:14:42.132066+0000","last_peered":"2024-06-28T16:14:42.132066+0000","last_clean":"2024-06-28T16:14:42.132066+0000","last_became_active":"2024-06-28T15:00:59.806725+0000","last_became_peered":"2024-06-28T15:00:59.806725+0000","last_unstale":"2024-06-28T16:14:42.132066+0000","last_undegraded":"2024-06-28T16:14:42.132066+0000","last_fullsized":"2024-06-28T16:14:42.132066+0000","mapping_epoch":32,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'192","last_scrub_stamp":"2024-06-28T16:14:42.131976+0000","last_deep_scrub":"260'192","last_deep_scrub_stamp":"2024-06-28T16:14:42.131976+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.131976+0000","objects_scrubbed":1,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:22:19.905274+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":759,"num_read_kb":602,"num_write":307,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"260'334","reported_seq":1007,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:29.347623+0000","last_change":"2024-06-28T16:15:29.347949+0000","last_active":"2024-06-28T16:15:29.347623+0000","last_peered":"2024-06-28T16:15:29.347623+0000","last_clean":"2024-06-28T16:15:29.347623+0000","last_became_active":"2024-06-28T15:00:15.383350+0000","last_became_peered":"2024-06-28T15:00:15.383350+0000","last_unstale":"2024-06-28T16:15:29.347623+0000","last_undegraded":"2024-06-28T16:15:29.347623+0000","last_fullsized":"2024-06-28T16:15:29.347623+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'334","last_scrub_stamp":"2024-06-28T16:15:29.347354+0000","last_deep_scrub":"260'334","last_deep_scrub_stamp":"2024-06-28T16:15:29.347354+0000","last_clean_scrub_stamp":"2024-06-28T16:15:29.347354+0000","objects_scrubbed":9,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:10:43.722259+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263219,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":502,"num_read_kb":41336,"num_write":1201,"num_write_kb":38478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0,7],"acting":[3,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"260'398","reported_seq":1008,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.938126+0000","last_change":"2024-06-28T16:14:58.938126+0000","last_active":"2024-06-28T16:14:58.938126+0000","last_peered":"2024-06-28T16:14:58.938126+0000","last_clean":"2024-06-28T16:14:58.938126+0000","last_became_active":"2024-06-28T15:00:17.404844+0000","last_became_peered":"2024-06-28T15:00:17.404844+0000","last_unstale":"2024-06-28T16:14:58.938126+0000","last_undegraded":"2024-06-28T16:14:58.938126+0000","last_fullsized":"2024-06-28T16:14:58.938126+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'398","last_scrub_stamp":"2024-06-28T16:14:58.937867+0000","last_deep_scrub":"260'398","last_deep_scrub_stamp":"2024-06-28T16:14:58.937867+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.937867+0000","objects_scrubbed":1,"log_size":98,"log_dups_size":300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:18:27.056087+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1527,"num_read_kb":1156,"num_write":658,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":268,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"260'343","reported_seq":1036,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.134211+0000","last_change":"2024-06-28T16:15:09.134294+0000","last_active":"2024-06-28T16:15:09.134211+0000","last_peered":"2024-06-28T16:15:09.134211+0000","last_clean":"2024-06-28T16:15:09.134211+0000","last_became_active":"2024-06-28T15:00:15.401275+0000","last_became_peered":"2024-06-28T15:00:15.401275+0000","last_unstale":"2024-06-28T16:15:09.134211+0000","last_undegraded":"2024-06-28T16:15:09.134211+0000","last_fullsized":"2024-06-28T16:15:09.134211+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'343","last_scrub_stamp":"2024-06-28T16:15:09.133999+0000","last_deep_scrub":"260'343","last_deep_scrub_stamp":"2024-06-28T16:15:09.133999+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.133999+0000","objects_scrubbed":3,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:37:29.220333+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":559,"num_read_kb":45669,"num_write":1313,"num_write_kb":19663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"260'221","reported_seq":851,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.135956+0000","last_change":"2024-06-28T16:15:14.135956+0000","last_active":"2024-06-28T16:15:14.135956+0000","last_peered":"2024-06-28T16:15:14.135956+0000","last_clean":"2024-06-28T16:15:14.135956+0000","last_became_active":"2024-06-28T15:00:17.382739+0000","last_became_peered":"2024-06-28T15:00:17.382739+0000","last_unstale":"2024-06-28T16:15:14.135956+0000","last_undegraded":"2024-06-28T16:15:14.135956+0000","last_fullsized":"2024-06-28T16:15:14.135956+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'221","last_scrub_stamp":"2024-06-28T16:15:14.135703+0000","last_deep_scrub":"260'221","last_deep_scrub_stamp":"2024-06-28T16:15:14.135703+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.135703+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:03:26.946056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":866,"num_read_kb":690,"num_write":335,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"260'298","reported_seq":937,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.436658+0000","last_change":"2024-06-28T16:15:10.436794+0000","last_active":"2024-06-28T16:15:10.436658+0000","last_peered":"2024-06-28T16:15:10.436658+0000","last_clean":"2024-06-28T16:15:10.436658+0000","last_became_active":"2024-06-28T15:00:15.421189+0000","last_became_peered":"2024-06-28T15:00:15.421189+0000","last_unstale":"2024-06-28T16:15:10.436658+0000","last_undegraded":"2024-06-28T16:15:10.436658+0000","last_fullsized":"2024-06-28T16:15:10.436658+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'298","last_scrub_stamp":"2024-06-28T16:15:10.436402+0000","last_deep_scrub":"260'298","last_deep_scrub_stamp":"2024-06-28T16:15:10.436402+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.436402+0000","objects_scrubbed":7,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:19:16.974922+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7105903,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":419,"num_read_kb":19450,"num_write":1148,"num_write_kb":38199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"260'80","reported_seq":650,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.448105+0000","last_change":"2024-06-28T16:15:03.448105+0000","last_active":"2024-06-28T16:15:03.448105+0000","last_peered":"2024-06-28T16:15:03.448105+0000","last_clean":"2024-06-28T16:15:03.448105+0000","last_became_active":"2024-06-28T15:00:17.396125+0000","last_became_peered":"2024-06-28T15:00:17.396125+0000","last_unstale":"2024-06-28T16:15:03.448105+0000","last_undegraded":"2024-06-28T16:15:03.448105+0000","last_fullsized":"2024-06-28T16:15:03.448105+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'80","last_scrub_stamp":"2024-06-28T16:15:03.447853+0000","last_deep_scrub":"260'80","last_deep_scrub_stamp":"2024-06-28T16:15:03.447853+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.447853+0000","objects_scrubbed":1,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:57:53.065744+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":321,"num_read_kb":257,"num_write":118,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"260'270","reported_seq":936,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:26.330845+0000","last_change":"2024-06-28T16:15:26.331055+0000","last_active":"2024-06-28T16:15:26.330845+0000","last_peered":"2024-06-28T16:15:26.330845+0000","last_clean":"2024-06-28T16:15:26.330845+0000","last_became_active":"2024-06-28T15:00:15.408375+0000","last_became_peered":"2024-06-28T15:00:15.408375+0000","last_unstale":"2024-06-28T16:15:26.330845+0000","last_undegraded":"2024-06-28T16:15:26.330845+0000","last_fullsized":"2024-06-28T16:15:26.330845+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'270","last_scrub_stamp":"2024-06-28T16:15:26.330596+0000","last_deep_scrub":"260'270","last_deep_scrub_stamp":"2024-06-28T16:15:26.330596+0000","last_clean_scrub_stamp":"2024-06-28T16:15:26.330596+0000","objects_scrubbed":5,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:26:19.674184+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":484,"num_read_kb":6587,"num_write":1005,"num_write_kb":17135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"260'62","reported_seq":646,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.240437+0000","last_change":"2024-06-28T16:15:17.240573+0000","last_active":"2024-06-28T16:15:17.240437+0000","last_peered":"2024-06-28T16:15:17.240437+0000","last_clean":"2024-06-28T16:15:17.240437+0000","last_became_active":"2024-06-28T15:00:17.379374+0000","last_became_peered":"2024-06-28T15:00:17.379374+0000","last_unstale":"2024-06-28T16:15:17.240437+0000","last_undegraded":"2024-06-28T16:15:17.240437+0000","last_fullsized":"2024-06-28T16:15:17.240437+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'62","last_scrub_stamp":"2024-06-28T16:15:17.240048+0000","last_deep_scrub":"260'62","last_deep_scrub_stamp":"2024-06-28T16:15:17.240048+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.240048+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:17:53.458911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":216,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"260'367","reported_seq":1020,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.676929+0000","last_change":"2024-06-28T16:15:11.677022+0000","last_active":"2024-06-28T16:15:11.676929+0000","last_peered":"2024-06-28T16:15:11.676929+0000","last_clean":"2024-06-28T16:15:11.676929+0000","last_became_active":"2024-06-28T15:00:15.427171+0000","last_became_peered":"2024-06-28T15:00:15.427171+0000","last_unstale":"2024-06-28T16:15:11.676929+0000","last_undegraded":"2024-06-28T16:15:11.676929+0000","last_fullsized":"2024-06-28T16:15:11.676929+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'367","last_scrub_stamp":"2024-06-28T16:15:11.676722+0000","last_deep_scrub":"260'367","last_deep_scrub_stamp":"2024-06-28T16:15:11.676722+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.676722+0000","objects_scrubbed":4,"log_size":67,"log_dups_size":300,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:35.302039+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":723921,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":536,"num_read_kb":26554,"num_write":1152,"num_write_kb":52254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"260'230","reported_seq":861,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.989876+0000","last_change":"2024-06-28T16:15:04.989943+0000","last_active":"2024-06-28T16:15:04.989876+0000","last_peered":"2024-06-28T16:15:04.989876+0000","last_clean":"2024-06-28T16:15:04.989876+0000","last_became_active":"2024-06-28T15:00:17.403760+0000","last_became_peered":"2024-06-28T15:00:17.403760+0000","last_unstale":"2024-06-28T16:15:04.989876+0000","last_undegraded":"2024-06-28T16:15:04.989876+0000","last_fullsized":"2024-06-28T16:15:04.989876+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'230","last_scrub_stamp":"2024-06-28T16:15:04.989667+0000","last_deep_scrub":"260'230","last_deep_scrub_stamp":"2024-06-28T16:15:04.989667+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.989667+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:58.817243+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":902,"num_read_kb":724,"num_write":338,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"260'282","reported_seq":938,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:27.318871+0000","last_change":"2024-06-28T16:15:27.319004+0000","last_active":"2024-06-28T16:15:27.318871+0000","last_peered":"2024-06-28T16:15:27.318871+0000","last_clean":"2024-06-28T16:15:27.318871+0000","last_became_active":"2024-06-28T15:00:15.376014+0000","last_became_peered":"2024-06-28T15:00:15.376014+0000","last_unstale":"2024-06-28T16:15:27.318871+0000","last_undegraded":"2024-06-28T16:15:27.318871+0000","last_fullsized":"2024-06-28T16:15:27.318871+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'282","last_scrub_stamp":"2024-06-28T16:15:27.318626+0000","last_deep_scrub":"260'282","last_deep_scrub_stamp":"2024-06-28T16:15:27.318626+0000","last_clean_scrub_stamp":"2024-06-28T16:15:27.318626+0000","objects_scrubbed":7,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:54:14.907511+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462789,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":409,"num_read_kb":32504,"num_write":910,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"260'70","reported_seq":668,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.378898+0000","last_change":"2024-06-28T16:15:01.378961+0000","last_active":"2024-06-28T16:15:01.378898+0000","last_peered":"2024-06-28T16:15:01.378898+0000","last_clean":"2024-06-28T16:15:01.378898+0000","last_became_active":"2024-06-28T15:00:17.401917+0000","last_became_peered":"2024-06-28T15:00:17.401917+0000","last_unstale":"2024-06-28T16:15:01.378898+0000","last_undegraded":"2024-06-28T16:15:01.378898+0000","last_fullsized":"2024-06-28T16:15:01.378898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'70","last_scrub_stamp":"2024-06-28T16:15:01.378703+0000","last_deep_scrub":"260'70","last_deep_scrub_stamp":"2024-06-28T16:15:01.378703+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.378703+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:54:33.487185+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":295,"num_read_kb":247,"num_write":98,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"260'337","reported_seq":982,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.634962+0000","last_change":"2024-06-28T16:15:09.635188+0000","last_active":"2024-06-28T16:15:09.634962+0000","last_peered":"2024-06-28T16:15:09.634962+0000","last_clean":"2024-06-28T16:15:09.634962+0000","last_became_active":"2024-06-28T15:00:15.431444+0000","last_became_peered":"2024-06-28T15:00:15.431444+0000","last_unstale":"2024-06-28T16:15:09.634962+0000","last_undegraded":"2024-06-28T16:15:09.634962+0000","last_fullsized":"2024-06-28T16:15:09.634962+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'337","last_scrub_stamp":"2024-06-28T16:15:09.634716+0000","last_deep_scrub":"260'337","last_deep_scrub_stamp":"2024-06-28T16:15:09.634716+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.634716+0000","objects_scrubbed":6,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:58:38.266578+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":461,"num_read_kb":557,"num_write":1266,"num_write_kb":33580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,3],"acting":[5,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"260'424","reported_seq":1187,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.016891+0000","last_change":"2024-06-28T16:15:06.016973+0000","last_active":"2024-06-28T16:15:06.016891+0000","last_peered":"2024-06-28T16:15:06.016891+0000","last_clean":"2024-06-28T16:15:06.016891+0000","last_became_active":"2024-06-28T15:00:17.402842+0000","last_became_peered":"2024-06-28T15:00:17.402842+0000","last_unstale":"2024-06-28T16:15:06.016891+0000","last_undegraded":"2024-06-28T16:15:06.016891+0000","last_fullsized":"2024-06-28T16:15:06.016891+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'424","last_scrub_stamp":"2024-06-28T16:15:06.016665+0000","last_deep_scrub":"260'424","last_deep_scrub_stamp":"2024-06-28T16:15:06.016665+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.016665+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":400,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:27:00.891642+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1825,"num_read_kb":1465,"num_write":755,"num_write_kb":468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"260'319","reported_seq":973,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:31.310383+0000","last_change":"2024-06-28T16:15:31.310482+0000","last_active":"2024-06-28T16:15:31.310383+0000","last_peered":"2024-06-28T16:15:31.310383+0000","last_clean":"2024-06-28T16:15:31.310383+0000","last_became_active":"2024-06-28T15:00:15.415009+0000","last_became_peered":"2024-06-28T15:00:15.415009+0000","last_unstale":"2024-06-28T16:15:31.310383+0000","last_undegraded":"2024-06-28T16:15:31.310383+0000","last_fullsized":"2024-06-28T16:15:31.310383+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'319","last_scrub_stamp":"2024-06-28T16:15:31.310182+0000","last_deep_scrub":"260'319","last_deep_scrub_stamp":"2024-06-28T16:15:31.310182+0000","last_clean_scrub_stamp":"2024-06-28T16:15:31.310182+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:45:14.627771+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242944,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":479,"num_read_kb":2084,"num_write":1202,"num_write_kb":29565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"260'26","reported_seq":583,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.742665+0000","last_change":"2024-06-28T16:15:11.742725+0000","last_active":"2024-06-28T16:15:11.742665+0000","last_peered":"2024-06-28T16:15:11.742665+0000","last_clean":"2024-06-28T16:15:11.742665+0000","last_became_active":"2024-06-28T15:00:17.389844+0000","last_became_peered":"2024-06-28T15:00:17.389844+0000","last_unstale":"2024-06-28T16:15:11.742665+0000","last_undegraded":"2024-06-28T16:15:11.742665+0000","last_fullsized":"2024-06-28T16:15:11.742665+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'26","last_scrub_stamp":"2024-06-28T16:15:11.742459+0000","last_deep_scrub":"260'26","last_deep_scrub_stamp":"2024-06-28T16:15:11.742459+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.742459+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:47:41.122636+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":112,"num_write":34,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"260'305","reported_seq":945,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.433717+0000","last_change":"2024-06-28T16:15:06.433887+0000","last_active":"2024-06-28T16:15:06.433717+0000","last_peered":"2024-06-28T16:15:06.433717+0000","last_clean":"2024-06-28T16:15:06.433717+0000","last_became_active":"2024-06-28T15:00:15.397986+0000","last_became_peered":"2024-06-28T15:00:15.397986+0000","last_unstale":"2024-06-28T16:15:06.433717+0000","last_undegraded":"2024-06-28T16:15:06.433717+0000","last_fullsized":"2024-06-28T16:15:06.433717+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'305","last_scrub_stamp":"2024-06-28T16:15:06.433513+0000","last_deep_scrub":"260'305","last_deep_scrub_stamp":"2024-06-28T16:15:06.433513+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.433513+0000","objects_scrubbed":7,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:04:10.170914+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079926,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":433,"num_read_kb":10373,"num_write":1047,"num_write_kb":25421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"260'192","reported_seq":866,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.290301+0000","last_change":"2024-06-28T16:15:09.290366+0000","last_active":"2024-06-28T16:15:09.290301+0000","last_peered":"2024-06-28T16:15:09.290301+0000","last_clean":"2024-06-28T16:15:09.290301+0000","last_became_active":"2024-06-28T15:00:17.394783+0000","last_became_peered":"2024-06-28T15:00:17.394783+0000","last_unstale":"2024-06-28T16:15:09.290301+0000","last_undegraded":"2024-06-28T16:15:09.290301+0000","last_fullsized":"2024-06-28T16:15:09.290301+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'192","last_scrub_stamp":"2024-06-28T16:15:09.290105+0000","last_deep_scrub":"260'192","last_deep_scrub_stamp":"2024-06-28T16:15:09.290105+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.290105+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:44:41.337242+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":852,"num_read_kb":715,"num_write":335,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"260'302","reported_seq":923,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.051125+0000","last_change":"2024-06-28T16:14:46.051125+0000","last_active":"2024-06-28T16:14:46.051125+0000","last_peered":"2024-06-28T16:14:46.051125+0000","last_clean":"2024-06-28T16:14:46.051125+0000","last_became_active":"2024-06-28T15:00:59.938004+0000","last_became_peered":"2024-06-28T15:00:59.938004+0000","last_unstale":"2024-06-28T16:14:46.051125+0000","last_undegraded":"2024-06-28T16:14:46.051125+0000","last_fullsized":"2024-06-28T16:14:46.051125+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'302","last_scrub_stamp":"2024-06-28T16:14:46.050757+0000","last_deep_scrub":"260'302","last_deep_scrub_stamp":"2024-06-28T16:14:46.050757+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.050757+0000","objects_scrubbed":13,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:23:03.771050+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6542288,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":437,"num_read_kb":25077,"num_write":1246,"num_write_kb":38037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,3],"acting":[1,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"260'392","reported_seq":1071,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:22.358463+0000","last_change":"2024-06-28T16:15:22.358526+0000","last_active":"2024-06-28T16:15:22.358463+0000","last_peered":"2024-06-28T16:15:22.358463+0000","last_clean":"2024-06-28T16:15:22.358463+0000","last_became_active":"2024-06-28T15:00:17.374143+0000","last_became_peered":"2024-06-28T15:00:17.374143+0000","last_unstale":"2024-06-28T16:15:22.358463+0000","last_undegraded":"2024-06-28T16:15:22.358463+0000","last_fullsized":"2024-06-28T16:15:22.358463+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'392","last_scrub_stamp":"2024-06-28T16:15:22.358270+0000","last_deep_scrub":"260'392","last_deep_scrub_stamp":"2024-06-28T16:15:22.358270+0000","last_clean_scrub_stamp":"2024-06-28T16:15:22.358270+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":300,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:17:15.649882+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1520,"num_read_kb":1191,"num_write":653,"num_write_kb":382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"260'255","reported_seq":928,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.309302+0000","last_change":"2024-06-28T16:14:58.309458+0000","last_active":"2024-06-28T16:14:58.309302+0000","last_peered":"2024-06-28T16:14:58.309302+0000","last_clean":"2024-06-28T16:14:58.309302+0000","last_became_active":"2024-06-28T15:00:15.430228+0000","last_became_peered":"2024-06-28T15:00:15.430228+0000","last_unstale":"2024-06-28T16:14:58.309302+0000","last_undegraded":"2024-06-28T16:14:58.309302+0000","last_fullsized":"2024-06-28T16:14:58.309302+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'255","last_scrub_stamp":"2024-06-28T16:14:58.309078+0000","last_deep_scrub":"260'255","last_deep_scrub_stamp":"2024-06-28T16:14:58.309078+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.309078+0000","objects_scrubbed":6,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:49:38.745040+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170399,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":449,"num_read_kb":6008,"num_write":895,"num_write_kb":11243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"260'134","reported_seq":787,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.792855+0000","last_change":"2024-06-28T16:15:13.793011+0000","last_active":"2024-06-28T16:15:13.792855+0000","last_peered":"2024-06-28T16:15:13.792855+0000","last_clean":"2024-06-28T16:15:13.792855+0000","last_became_active":"2024-06-28T15:00:17.381562+0000","last_became_peered":"2024-06-28T15:00:17.381562+0000","last_unstale":"2024-06-28T16:15:13.792855+0000","last_undegraded":"2024-06-28T16:15:13.792855+0000","last_fullsized":"2024-06-28T16:15:13.792855+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'134","last_scrub_stamp":"2024-06-28T16:15:13.792604+0000","last_deep_scrub":"260'134","last_deep_scrub_stamp":"2024-06-28T16:15:13.792604+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.792604+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:32:09.953292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":479,"num_write":190,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"260'496","reported_seq":1195,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.851543+0000","last_change":"2024-06-28T16:15:19.851688+0000","last_active":"2024-06-28T16:15:19.851543+0000","last_peered":"2024-06-28T16:15:19.851543+0000","last_clean":"2024-06-28T16:15:19.851543+0000","last_became_active":"2024-06-28T15:00:15.418850+0000","last_became_peered":"2024-06-28T15:00:15.418850+0000","last_unstale":"2024-06-28T16:15:19.851543+0000","last_undegraded":"2024-06-28T16:15:19.851543+0000","last_fullsized":"2024-06-28T16:15:19.851543+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'496","last_scrub_stamp":"2024-06-28T16:15:19.851317+0000","last_deep_scrub":"260'496","last_deep_scrub_stamp":"2024-06-28T16:15:19.851317+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.851317+0000","objects_scrubbed":1,"log_size":96,"log_dups_size":400,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:36:52.931988+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":907,"num_read_kb":23220,"num_write":1415,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"260'338","reported_seq":1032,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.827560+0000","last_change":"2024-06-28T16:15:16.827810+0000","last_active":"2024-06-28T16:15:16.827560+0000","last_peered":"2024-06-28T16:15:16.827560+0000","last_clean":"2024-06-28T16:15:16.827560+0000","last_became_active":"2024-06-28T15:00:15.408158+0000","last_became_peered":"2024-06-28T15:00:15.408158+0000","last_unstale":"2024-06-28T16:15:16.827560+0000","last_undegraded":"2024-06-28T16:15:16.827560+0000","last_fullsized":"2024-06-28T16:15:16.827560+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'338","last_scrub_stamp":"2024-06-28T16:15:16.827294+0000","last_deep_scrub":"260'338","last_deep_scrub_stamp":"2024-06-28T16:15:16.827294+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.827294+0000","objects_scrubbed":9,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:01:49.282108+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6690738,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":568,"num_read_kb":38558,"num_write":1259,"num_write_kb":46809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"260'77","reported_seq":655,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.306055+0000","last_change":"2024-06-28T16:15:15.306055+0000","last_active":"2024-06-28T16:15:15.306055+0000","last_peered":"2024-06-28T16:15:15.306055+0000","last_clean":"2024-06-28T16:15:15.306055+0000","last_became_active":"2024-06-28T15:00:17.379771+0000","last_became_peered":"2024-06-28T15:00:17.379771+0000","last_unstale":"2024-06-28T16:15:15.306055+0000","last_undegraded":"2024-06-28T16:15:15.306055+0000","last_fullsized":"2024-06-28T16:15:15.306055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'77","last_scrub_stamp":"2024-06-28T16:15:15.305668+0000","last_deep_scrub":"260'77","last_deep_scrub_stamp":"2024-06-28T16:15:15.305668+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.305668+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:33:25.492967+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":249,"num_write":109,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"260'387","reported_seq":1111,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.144751+0000","last_change":"2024-06-28T16:15:10.144995+0000","last_active":"2024-06-28T16:15:10.144751+0000","last_peered":"2024-06-28T16:15:10.144751+0000","last_clean":"2024-06-28T16:15:10.144751+0000","last_became_active":"2024-06-28T15:00:15.401324+0000","last_became_peered":"2024-06-28T15:00:15.401324+0000","last_unstale":"2024-06-28T16:15:10.144751+0000","last_undegraded":"2024-06-28T16:15:10.144751+0000","last_fullsized":"2024-06-28T16:15:10.144751+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'387","last_scrub_stamp":"2024-06-28T16:15:10.144446+0000","last_deep_scrub":"260'387","last_deep_scrub_stamp":"2024-06-28T16:15:10.144446+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.144446+0000","objects_scrubbed":9,"log_size":87,"log_dups_size":300,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:10:22.329753+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079973,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":661,"num_read_kb":1946,"num_write":1429,"num_write_kb":27589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"260'176","reported_seq":787,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.646038+0000","last_change":"2024-06-28T16:15:03.646038+0000","last_active":"2024-06-28T16:15:03.646038+0000","last_peered":"2024-06-28T16:15:03.646038+0000","last_clean":"2024-06-28T16:15:03.646038+0000","last_became_active":"2024-06-28T15:00:17.382774+0000","last_became_peered":"2024-06-28T15:00:17.382774+0000","last_unstale":"2024-06-28T16:15:03.646038+0000","last_undegraded":"2024-06-28T16:15:03.646038+0000","last_fullsized":"2024-06-28T16:15:03.646038+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'176","last_scrub_stamp":"2024-06-28T16:15:03.645757+0000","last_deep_scrub":"260'176","last_deep_scrub_stamp":"2024-06-28T16:15:03.645757+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.645757+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:35:27.591692+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":719,"num_read_kb":570,"num_write":272,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5652,"num_read_kb":3987,"num_write":2073,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7430,"num_omap_keys":15,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1080,"ondisk_log_size":1080,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19748,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":13201,"num_read_kb":13000,"num_write":15927,"num_write_kb":4660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4270,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":39496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1835,"ondisk_log_size":1835,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1424454,"num_objects":198,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":18162,"num_read_kb":37754,"num_write":15149,"num_write_kb":11215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":655336,"num_omap_keys":1530,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2056,"ondisk_log_size":2056,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1644,"num_read_kb":1954,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87319,"num_read_kb":70822,"num_write":35298,"num_write_kb":17822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":43558,"num_omap_keys":182,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3548,"ondisk_log_size":3548,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":940,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":149605345,"num_objects":433,"num_object_clones":0,"num_object_copies":1732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":433,"num_whiteouts":0,"num_read":34839,"num_read_kb":1786510,"num_write":77776,"num_write_kb":1982911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202506240,"data_stored":202506240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3438,"ondisk_log_size":3438,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":129,"num_read_kb":165927,"num_write":154,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":137,"num_read_kb":186344,"num_write":154,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575758,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356188,"kb_used_data":36244,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":94015652,"statfs":{"total":96636764160,"available":96272027648,"internally_reserved":0,"allocated":37113856,"data_stored":35129654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":327612874},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575758,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323068,"kb_used_data":43380,"kb_used_omap":1,"kb_used_meta":279678,"kb_avail":94048772,"statfs":{"total":96636764160,"available":96305942528,"internally_reserved":0,"allocated":44421120,"data_stored":42495981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":286390729},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575760,"num_pgs":114,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201360,"kb_used_data":35592,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94170480,"statfs":{"total":96636764160,"available":96430571520,"internally_reserved":0,"allocated":36446208,"data_stored":34448502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575760,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":146836,"kb_used_data":43468,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94225004,"statfs":{"total":96636764160,"available":96486404096,"internally_reserved":0,"allocated":44511232,"data_stored":42596897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575763,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303252,"kb_used_data":29196,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":94068588,"statfs":{"total":96636764160,"available":96326234112,"internally_reserved":0,"allocated":29896704,"data_stored":27928906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":280623559},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575761,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137284,"kb_used_data":29820,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94234556,"statfs":{"total":96636764160,"available":96496185344,"internally_reserved":0,"allocated":30535680,"data_stored":28610104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575758,"num_pgs":120,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201576,"kb_used_data":39968,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94170264,"statfs":{"total":96636764160,"available":96430350336,"internally_reserved":0,"allocated":40927232,"data_stored":38982339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575761,"num_pgs":116,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287396,"kb_used_data":39644,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":94084444,"statfs":{"total":96636764160,"available":96342470656,"internally_reserved":0,"allocated":40595456,"data_stored":38600327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":253688261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25272320,"data_stored":25272320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25198592,"data_stored":25198592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21118976,"data_stored":21118976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19386368,"data_stored":19386368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29863936,"data_stored":29863936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20664320,"data_stored":20664320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28712960,"data_stored":28712960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32288768,"data_stored":32288768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":7701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":4574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":4436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":5746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-28T16:15:43.431 INFO:tasks.ceph:pgid 68.19 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:43.432 INFO:tasks.ceph:pgid 68.15 last_scrub_stamp 2024-06-28T15:37:32.285758+0000 time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=15, tm_min=37, tm_sec=32, tm_wday=4, tm_yday=180, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=6, tm_mday=28, tm_hour=16, tm_min=14, tm_sec=37, tm_wday=4, tm_yday=180, tm_isdst=0) 2024-06-28T16:15:43.436 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-06-28T16:16:03.437 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-28T16:16:03.807 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-28T16:16:03.820 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2024-06-28T16:16:03.824 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":2531,"stamp":"2024-06-28T16:16:03.671213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":185196140,"num_objects":785,"num_object_clones":0,"num_object_copies":2436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":785,"num_whiteouts":0,"num_read":161161,"num_read_kb":2266365,"num_write":146672,"num_write_kb":2275015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":68,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":711722,"num_omap_keys":1764,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12591,"ondisk_log_size":12591,"up":914,"acting":914,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":914,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1956960,"kb_used_data":297312,"kb_used_omap":12,"kb_used_meta":1659571,"kb_avail":753017760,"statfs":{"total":773094113280,"available":771090186240,"internally_reserved":0,"allocated":304447488,"data_stored":288792710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":1699401267},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009952"},"pg_stats":[{"pgid":"68.1a","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:28.856570+0000","last_change":"2024-06-28T16:15:28.856633+0000","last_active":"2024-06-28T16:15:28.856570+0000","last_peered":"2024-06-28T16:15:28.856570+0000","last_clean":"2024-06-28T16:15:28.856570+0000","last_became_active":"2024-06-28T15:37:33.328626+0000","last_became_peered":"2024-06-28T15:37:33.328626+0000","last_unstale":"2024-06-28T16:15:28.856570+0000","last_undegraded":"2024-06-28T16:15:28.856570+0000","last_fullsized":"2024-06-28T16:15:28.856570+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:28.856334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:28.856334+0000","last_clean_scrub_stamp":"2024-06-28T16:15:28.856334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:58:02.210395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.1d","version":"257'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:23.876306+0000","last_change":"2024-06-28T16:15:23.876367+0000","last_active":"2024-06-28T16:15:23.876306+0000","last_peered":"2024-06-28T16:15:23.876306+0000","last_clean":"2024-06-28T16:15:23.876306+0000","last_became_active":"2024-06-28T15:21:11.078250+0000","last_became_peered":"2024-06-28T15:21:11.078250+0000","last_unstale":"2024-06-28T16:15:23.876306+0000","last_undegraded":"2024-06-28T16:15:23.876306+0000","last_fullsized":"2024-06-28T16:15:23.876306+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:23.876096+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:23.876096+0000","last_clean_scrub_stamp":"2024-06-28T16:15:23.876096+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:51:03.261049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1c","version":"260'4","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.100994+0000","last_change":"2024-06-28T16:14:41.101042+0000","last_active":"2024-06-28T16:14:41.100994+0000","last_peered":"2024-06-28T16:14:41.100994+0000","last_clean":"2024-06-28T16:14:41.100994+0000","last_became_active":"2024-06-28T15:21:07.046517+0000","last_became_peered":"2024-06-28T15:21:07.046517+0000","last_unstale":"2024-06-28T16:14:41.100994+0000","last_undegraded":"2024-06-28T16:14:41.100994+0000","last_fullsized":"2024-06-28T16:14:41.100994+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:14:41.100864+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:14:41.100864+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.100864+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:17:23.820622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3076,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.1f","version":"249'7","reported_seq":92,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.148310+0000","last_change":"2024-06-28T16:14:39.148310+0000","last_active":"2024-06-28T16:14:39.148310+0000","last_peered":"2024-06-28T16:14:39.148310+0000","last_clean":"2024-06-28T16:14:39.148310+0000","last_became_active":"2024-06-28T15:17:24.028787+0000","last_became_peered":"2024-06-28T15:17:24.028787+0000","last_unstale":"2024-06-28T16:14:39.148310+0000","last_undegraded":"2024-06-28T16:14:39.148310+0000","last_fullsized":"2024-06-28T16:14:39.148310+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"249'7","last_scrub_stamp":"2024-06-28T16:14:39.148161+0000","last_deep_scrub":"249'7","last_deep_scrub_stamp":"2024-06-28T16:14:39.148161+0000","last_clean_scrub_stamp":"2024-06-28T16:14:39.148161+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:43:21.535181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":30,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1b","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.297034+0000","last_change":"2024-06-28T16:15:08.297105+0000","last_active":"2024-06-28T16:15:08.297034+0000","last_peered":"2024-06-28T16:15:08.297034+0000","last_clean":"2024-06-28T16:15:08.297034+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:15:08.297034+0000","last_undegraded":"2024-06-28T16:15:08.297034+0000","last_fullsized":"2024-06-28T16:15:08.297034+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:08.296839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:08.296839+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.296839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:48:21.812858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1c","version":"260'2","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:27.994963+0000","last_change":"2024-06-28T16:15:27.995033+0000","last_active":"2024-06-28T16:15:27.994963+0000","last_peered":"2024-06-28T16:15:27.994963+0000","last_clean":"2024-06-28T16:15:27.994963+0000","last_became_active":"2024-06-28T15:21:11.104387+0000","last_became_peered":"2024-06-28T15:21:11.104387+0000","last_unstale":"2024-06-28T16:15:27.994963+0000","last_undegraded":"2024-06-28T16:15:27.994963+0000","last_fullsized":"2024-06-28T16:15:27.994963+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:27.994749+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:27.994749+0000","last_clean_scrub_stamp":"2024-06-28T16:15:27.994749+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:32:47.148847+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.1d","version":"260'2","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.043377+0000","last_change":"2024-06-28T16:14:45.043442+0000","last_active":"2024-06-28T16:14:45.043377+0000","last_peered":"2024-06-28T16:14:45.043377+0000","last_clean":"2024-06-28T16:14:45.043377+0000","last_became_active":"2024-06-28T15:21:07.049141+0000","last_became_peered":"2024-06-28T15:21:07.049141+0000","last_unstale":"2024-06-28T16:14:45.043377+0000","last_undegraded":"2024-06-28T16:14:45.043377+0000","last_fullsized":"2024-06-28T16:14:45.043377+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:45.043176+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:45.043176+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.043176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:35:34.904145+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1e","version":"260'18","reported_seq":120,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.228554+0000","last_change":"2024-06-28T16:14:50.228554+0000","last_active":"2024-06-28T16:14:50.228554+0000","last_peered":"2024-06-28T16:14:50.228554+0000","last_clean":"2024-06-28T16:14:50.228554+0000","last_became_active":"2024-06-28T15:17:23.993905+0000","last_became_peered":"2024-06-28T15:17:23.993905+0000","last_unstale":"2024-06-28T16:14:50.228554+0000","last_undegraded":"2024-06-28T16:14:50.228554+0000","last_fullsized":"2024-06-28T16:14:50.228554+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'18","last_scrub_stamp":"2024-06-28T16:14:50.228287+0000","last_deep_scrub":"260'18","last_deep_scrub_stamp":"2024-06-28T16:14:50.228287+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.228287+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:42:17.443870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":64,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.18","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.398560+0000","last_change":"2024-06-28T16:15:18.398617+0000","last_active":"2024-06-28T16:15:18.398560+0000","last_peered":"2024-06-28T16:15:18.398560+0000","last_clean":"2024-06-28T16:15:18.398560+0000","last_became_active":"2024-06-28T15:37:33.345642+0000","last_became_peered":"2024-06-28T15:37:33.345642+0000","last_unstale":"2024-06-28T16:15:18.398560+0000","last_undegraded":"2024-06-28T16:15:18.398560+0000","last_fullsized":"2024-06-28T16:15:18.398560+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:18.398355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:18.398355+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.398355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:01:11.369378+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.1f","version":"257'5","reported_seq":67,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:34.424204+0000","last_change":"2024-06-28T16:15:34.424276+0000","last_active":"2024-06-28T16:15:34.424204+0000","last_peered":"2024-06-28T16:15:34.424204+0000","last_clean":"2024-06-28T16:15:34.424204+0000","last_became_active":"2024-06-28T15:21:11.097619+0000","last_became_peered":"2024-06-28T15:21:11.097619+0000","last_unstale":"2024-06-28T16:15:34.424204+0000","last_undegraded":"2024-06-28T16:15:34.424204+0000","last_fullsized":"2024-06-28T16:15:34.424204+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'5","last_scrub_stamp":"2024-06-28T16:15:34.423959+0000","last_deep_scrub":"257'5","last_deep_scrub_stamp":"2024-06-28T16:15:34.423959+0000","last_clean_scrub_stamp":"2024-06-28T16:15:34.423959+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:36:18.836568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5438,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.1e","version":"0'0","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.347194+0000","last_change":"2024-06-28T16:15:04.347194+0000","last_active":"2024-06-28T16:15:04.347194+0000","last_peered":"2024-06-28T16:15:04.347194+0000","last_clean":"2024-06-28T16:15:04.347194+0000","last_became_active":"2024-06-28T15:21:07.020090+0000","last_became_peered":"2024-06-28T15:21:07.020090+0000","last_unstale":"2024-06-28T16:15:04.347194+0000","last_undegraded":"2024-06-28T16:15:04.347194+0000","last_fullsized":"2024-06-28T16:15:04.347194+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:04.346943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:04.346943+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.346943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:34:44.306374+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.1d","version":"260'43","reported_seq":184,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.114470+0000","last_change":"2024-06-28T16:14:45.114470+0000","last_active":"2024-06-28T16:14:45.114470+0000","last_peered":"2024-06-28T16:14:45.114470+0000","last_clean":"2024-06-28T16:14:45.114470+0000","last_became_active":"2024-06-28T15:17:24.012739+0000","last_became_peered":"2024-06-28T15:17:24.012739+0000","last_unstale":"2024-06-28T16:14:45.114470+0000","last_undegraded":"2024-06-28T16:14:45.114470+0000","last_fullsized":"2024-06-28T16:14:45.114470+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'43","last_scrub_stamp":"2024-06-28T16:14:45.114196+0000","last_deep_scrub":"260'43","last_deep_scrub_stamp":"2024-06-28T16:14:45.114196+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.114196+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:38:38.257912+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":221,"num_read_kb":154,"num_write":91,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.19","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:42.252622+0000","last_change":"2024-06-28T16:15:42.252744+0000","last_active":"2024-06-28T16:15:42.252622+0000","last_peered":"2024-06-28T16:15:42.252622+0000","last_clean":"2024-06-28T16:15:42.252622+0000","last_became_active":"2024-06-28T15:37:33.344850+0000","last_became_peered":"2024-06-28T15:37:33.344850+0000","last_unstale":"2024-06-28T16:15:42.252622+0000","last_undegraded":"2024-06-28T16:15:42.252622+0000","last_fullsized":"2024-06-28T16:15:42.252622+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:42.252388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:42.252388+0000","last_clean_scrub_stamp":"2024-06-28T16:15:42.252388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:17:49.113664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.1e","version":"260'3","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.182157+0000","last_change":"2024-06-28T16:15:09.182157+0000","last_active":"2024-06-28T16:15:09.182157+0000","last_peered":"2024-06-28T16:15:09.182157+0000","last_clean":"2024-06-28T16:15:09.182157+0000","last_became_active":"2024-06-28T15:21:11.102572+0000","last_became_peered":"2024-06-28T15:21:11.102572+0000","last_unstale":"2024-06-28T16:15:09.182157+0000","last_undegraded":"2024-06-28T16:15:09.182157+0000","last_fullsized":"2024-06-28T16:15:09.182157+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:15:09.181904+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:15:09.181904+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.181904+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:48:31.360817+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1f","version":"0'0","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.928159+0000","last_change":"2024-06-28T16:14:48.928159+0000","last_active":"2024-06-28T16:14:48.928159+0000","last_peered":"2024-06-28T16:14:48.928159+0000","last_clean":"2024-06-28T16:14:48.928159+0000","last_became_active":"2024-06-28T15:21:07.030576+0000","last_became_peered":"2024-06-28T15:21:07.030576+0000","last_unstale":"2024-06-28T16:14:48.928159+0000","last_undegraded":"2024-06-28T16:14:48.928159+0000","last_fullsized":"2024-06-28T16:14:48.928159+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:48.927902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:48.927902+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.927902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:15:53.577907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1c","version":"260'23","reported_seq":126,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.110220+0000","last_change":"2024-06-28T16:14:44.110220+0000","last_active":"2024-06-28T16:14:44.110220+0000","last_peered":"2024-06-28T16:14:44.110220+0000","last_clean":"2024-06-28T16:14:44.110220+0000","last_became_active":"2024-06-28T15:17:24.019120+0000","last_became_peered":"2024-06-28T15:17:24.019120+0000","last_unstale":"2024-06-28T16:14:44.110220+0000","last_undegraded":"2024-06-28T16:14:44.110220+0000","last_fullsized":"2024-06-28T16:14:44.110220+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'23","last_scrub_stamp":"2024-06-28T16:14:44.109887+0000","last_deep_scrub":"260'23","last_deep_scrub_stamp":"2024-06-28T16:14:44.109887+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.109887+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:34:23.631650+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":82,"num_write":51,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1e","version":"0'0","reported_seq":40,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:39.219941+0000","last_change":"2024-06-28T16:15:39.219941+0000","last_active":"2024-06-28T16:15:39.219941+0000","last_peered":"2024-06-28T16:15:39.219941+0000","last_clean":"2024-06-28T16:15:39.219941+0000","last_became_active":"2024-06-28T15:37:33.335045+0000","last_became_peered":"2024-06-28T15:37:33.335045+0000","last_unstale":"2024-06-28T16:15:39.219941+0000","last_undegraded":"2024-06-28T16:15:39.219941+0000","last_fullsized":"2024-06-28T16:15:39.219941+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:39.219687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:39.219687+0000","last_clean_scrub_stamp":"2024-06-28T16:15:39.219687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:12:49.197514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.19","version":"0'0","reported_seq":53,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:25.916735+0000","last_change":"2024-06-28T16:15:25.916839+0000","last_active":"2024-06-28T16:15:25.916735+0000","last_peered":"2024-06-28T16:15:25.916735+0000","last_clean":"2024-06-28T16:15:25.916735+0000","last_became_active":"2024-06-28T15:21:11.094122+0000","last_became_peered":"2024-06-28T15:21:11.094122+0000","last_unstale":"2024-06-28T16:15:25.916735+0000","last_undegraded":"2024-06-28T16:15:25.916735+0000","last_fullsized":"2024-06-28T16:15:25.916735+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:25.916390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:25.916390+0000","last_clean_scrub_stamp":"2024-06-28T16:15:25.916390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:08:57.050034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.18","version":"0'0","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.137409+0000","last_change":"2024-06-28T16:14:42.137431+0000","last_active":"2024-06-28T16:14:42.137409+0000","last_peered":"2024-06-28T16:14:42.137409+0000","last_clean":"2024-06-28T16:14:42.137409+0000","last_became_active":"2024-06-28T15:21:07.025549+0000","last_became_peered":"2024-06-28T15:21:07.025549+0000","last_unstale":"2024-06-28T16:14:42.137409+0000","last_undegraded":"2024-06-28T16:14:42.137409+0000","last_fullsized":"2024-06-28T16:14:42.137409+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:42.137318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:42.137318+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.137318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:05:06.478598+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.1b","version":"260'24","reported_seq":130,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.151839+0000","last_change":"2024-06-28T16:14:46.151839+0000","last_active":"2024-06-28T16:14:46.151839+0000","last_peered":"2024-06-28T16:14:46.151839+0000","last_clean":"2024-06-28T16:14:46.151839+0000","last_became_active":"2024-06-28T15:17:24.010441+0000","last_became_peered":"2024-06-28T15:17:24.010441+0000","last_unstale":"2024-06-28T16:14:46.151839+0000","last_undegraded":"2024-06-28T16:14:46.151839+0000","last_fullsized":"2024-06-28T16:14:46.151839+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'24","last_scrub_stamp":"2024-06-28T16:14:46.151597+0000","last_deep_scrub":"260'24","last_deep_scrub_stamp":"2024-06-28T16:14:46.151597+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.151597+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:33:56.078813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":117,"num_read_kb":85,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.1f","version":"0'0","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:22.314706+0000","last_change":"2024-06-28T16:15:22.314844+0000","last_active":"2024-06-28T16:15:22.314706+0000","last_peered":"2024-06-28T16:15:22.314706+0000","last_clean":"2024-06-28T16:15:22.314706+0000","last_became_active":"2024-06-28T15:37:33.336623+0000","last_became_peered":"2024-06-28T15:37:33.336623+0000","last_unstale":"2024-06-28T16:15:22.314706+0000","last_undegraded":"2024-06-28T16:15:22.314706+0000","last_fullsized":"2024-06-28T16:15:22.314706+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:22.314470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:22.314470+0000","last_clean_scrub_stamp":"2024-06-28T16:15:22.314470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:42:33.660532+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.18","version":"260'12","reported_seq":74,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.367282+0000","last_change":"2024-06-28T16:15:15.367353+0000","last_active":"2024-06-28T16:15:15.367282+0000","last_peered":"2024-06-28T16:15:15.367282+0000","last_clean":"2024-06-28T16:15:15.367282+0000","last_became_active":"2024-06-28T15:21:11.099434+0000","last_became_peered":"2024-06-28T15:21:11.099434+0000","last_unstale":"2024-06-28T16:15:15.367282+0000","last_undegraded":"2024-06-28T16:15:15.367282+0000","last_fullsized":"2024-06-28T16:15:15.367282+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'12","last_scrub_stamp":"2024-06-28T16:15:15.367078+0000","last_deep_scrub":"260'12","last_deep_scrub_stamp":"2024-06-28T16:15:15.367078+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.367078+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:12:37.736358+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":20482,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.19","version":"260'6","reported_seq":75,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:23.962817+0000","last_change":"2024-06-28T16:15:23.962974+0000","last_active":"2024-06-28T16:15:23.962817+0000","last_peered":"2024-06-28T16:15:23.962817+0000","last_clean":"2024-06-28T16:15:23.962817+0000","last_became_active":"2024-06-28T15:21:07.043987+0000","last_became_peered":"2024-06-28T15:21:07.043987+0000","last_unstale":"2024-06-28T16:15:23.962817+0000","last_undegraded":"2024-06-28T16:15:23.962817+0000","last_fullsized":"2024-06-28T16:15:23.962817+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:15:23.962501+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:15:23.962501+0000","last_clean_scrub_stamp":"2024-06-28T16:15:23.962501+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:48:09.464644+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":20482,"num_write":9,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.1a","version":"260'19","reported_seq":110,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.131322+0000","last_change":"2024-06-28T16:14:47.131322+0000","last_active":"2024-06-28T16:14:47.131322+0000","last_peered":"2024-06-28T16:14:47.131322+0000","last_clean":"2024-06-28T16:14:47.131322+0000","last_became_active":"2024-06-28T15:17:24.006314+0000","last_became_peered":"2024-06-28T15:17:24.006314+0000","last_unstale":"2024-06-28T16:14:47.131322+0000","last_undegraded":"2024-06-28T16:14:47.131322+0000","last_fullsized":"2024-06-28T16:14:47.131322+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'19","last_scrub_stamp":"2024-06-28T16:14:47.131067+0000","last_deep_scrub":"260'19","last_deep_scrub_stamp":"2024-06-28T16:14:47.131067+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.131067+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:32:41.228085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":54,"num_write":49,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.1c","version":"260'2","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:35.132584+0000","last_change":"2024-06-28T16:15:35.132653+0000","last_active":"2024-06-28T16:15:35.132584+0000","last_peered":"2024-06-28T16:15:35.132584+0000","last_clean":"2024-06-28T16:15:35.132584+0000","last_became_active":"2024-06-28T15:37:33.347715+0000","last_became_peered":"2024-06-28T15:37:33.347715+0000","last_unstale":"2024-06-28T16:15:35.132584+0000","last_undegraded":"2024-06-28T16:15:35.132584+0000","last_fullsized":"2024-06-28T16:15:35.132584+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:35.132387+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:35.132387+0000","last_clean_scrub_stamp":"2024-06-28T16:15:35.132387+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:04:57.900521+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1b","version":"0'0","reported_seq":51,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.386219+0000","last_change":"2024-06-28T16:15:14.386283+0000","last_active":"2024-06-28T16:15:14.386219+0000","last_peered":"2024-06-28T16:15:14.386219+0000","last_clean":"2024-06-28T16:15:14.386219+0000","last_became_active":"2024-06-28T15:21:11.088625+0000","last_became_peered":"2024-06-28T15:21:11.088625+0000","last_unstale":"2024-06-28T16:15:14.386219+0000","last_undegraded":"2024-06-28T16:15:14.386219+0000","last_fullsized":"2024-06-28T16:15:14.386219+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:14.386017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:14.386017+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.386017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:51:33.701375+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.1a","version":"260'6","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.976747+0000","last_change":"2024-06-28T16:14:44.976815+0000","last_active":"2024-06-28T16:14:44.976747+0000","last_peered":"2024-06-28T16:14:44.976747+0000","last_clean":"2024-06-28T16:14:44.976747+0000","last_became_active":"2024-06-28T15:21:07.030888+0000","last_became_peered":"2024-06-28T15:21:07.030888+0000","last_unstale":"2024-06-28T16:14:44.976747+0000","last_undegraded":"2024-06-28T16:14:44.976747+0000","last_fullsized":"2024-06-28T16:14:44.976747+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:14:44.976529+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:14:44.976529+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.976529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:33:30.061725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":6,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.19","version":"260'72","reported_seq":257,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.133001+0000","last_change":"2024-06-28T16:14:47.133001+0000","last_active":"2024-06-28T16:14:47.133001+0000","last_peered":"2024-06-28T16:14:47.133001+0000","last_clean":"2024-06-28T16:14:47.133001+0000","last_became_active":"2024-06-28T15:17:23.993026+0000","last_became_peered":"2024-06-28T15:17:23.993026+0000","last_unstale":"2024-06-28T16:14:47.133001+0000","last_undegraded":"2024-06-28T16:14:47.133001+0000","last_fullsized":"2024-06-28T16:14:47.133001+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'72","last_scrub_stamp":"2024-06-28T16:14:47.132725+0000","last_deep_scrub":"260'72","last_deep_scrub_stamp":"2024-06-28T16:14:47.132725+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.132725+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:48:03.563113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":358,"num_read_kb":255,"num_write":131,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.1d","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:37.123059+0000","last_change":"2024-06-28T16:15:37.123114+0000","last_active":"2024-06-28T16:15:37.123059+0000","last_peered":"2024-06-28T16:15:37.123059+0000","last_clean":"2024-06-28T16:15:37.123059+0000","last_became_active":"2024-06-28T15:37:33.350447+0000","last_became_peered":"2024-06-28T15:37:33.350447+0000","last_unstale":"2024-06-28T16:15:37.123059+0000","last_undegraded":"2024-06-28T16:15:37.123059+0000","last_fullsized":"2024-06-28T16:15:37.123059+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:37.122862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:37.122862+0000","last_clean_scrub_stamp":"2024-06-28T16:15:37.122862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:27:41.073800+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.1a","version":"0'0","reported_seq":52,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.251991+0000","last_change":"2024-06-28T16:15:01.252044+0000","last_active":"2024-06-28T16:15:01.251991+0000","last_peered":"2024-06-28T16:15:01.251991+0000","last_clean":"2024-06-28T16:15:01.251991+0000","last_became_active":"2024-06-28T15:21:11.100603+0000","last_became_peered":"2024-06-28T15:21:11.100603+0000","last_unstale":"2024-06-28T16:15:01.251991+0000","last_undegraded":"2024-06-28T16:15:01.251991+0000","last_fullsized":"2024-06-28T16:15:01.251991+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:01.251799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:01.251799+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.251799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:50:06.461152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.1b","version":"260'5","reported_seq":64,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.231661+0000","last_change":"2024-06-28T16:15:02.231723+0000","last_active":"2024-06-28T16:15:02.231661+0000","last_peered":"2024-06-28T16:15:02.231661+0000","last_clean":"2024-06-28T16:15:02.231661+0000","last_became_active":"2024-06-28T15:21:07.045902+0000","last_became_peered":"2024-06-28T15:21:07.045902+0000","last_unstale":"2024-06-28T16:15:02.231661+0000","last_undegraded":"2024-06-28T16:15:02.231661+0000","last_fullsized":"2024-06-28T16:15:02.231661+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'5","last_scrub_stamp":"2024-06-28T16:15:02.231467+0000","last_deep_scrub":"260'5","last_deep_scrub_stamp":"2024-06-28T16:15:02.231467+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.231467+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:59:29.633514+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":40,"num_write":10,"num_write_kb":1064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.18","version":"260'48","reported_seq":202,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.112995+0000","last_change":"2024-06-28T16:14:43.112995+0000","last_active":"2024-06-28T16:14:43.112995+0000","last_peered":"2024-06-28T16:14:43.112995+0000","last_clean":"2024-06-28T16:14:43.112995+0000","last_became_active":"2024-06-28T15:17:24.016249+0000","last_became_peered":"2024-06-28T15:17:24.016249+0000","last_unstale":"2024-06-28T16:14:43.112995+0000","last_undegraded":"2024-06-28T16:14:43.112995+0000","last_fullsized":"2024-06-28T16:14:43.112995+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'48","last_scrub_stamp":"2024-06-28T16:14:43.112724+0000","last_deep_scrub":"260'48","last_deep_scrub_stamp":"2024-06-28T16:14:43.112724+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.112724+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:09:09.626825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":260,"num_read_kb":188,"num_write":81,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.12","version":"0'0","reported_seq":37,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:20.290402+0000","last_change":"2024-06-28T16:15:20.290459+0000","last_active":"2024-06-28T16:15:20.290402+0000","last_peered":"2024-06-28T16:15:20.290402+0000","last_clean":"2024-06-28T16:15:20.290402+0000","last_became_active":"2024-06-28T15:37:33.316616+0000","last_became_peered":"2024-06-28T15:37:33.316616+0000","last_unstale":"2024-06-28T16:15:20.290402+0000","last_undegraded":"2024-06-28T16:15:20.290402+0000","last_fullsized":"2024-06-28T16:15:20.290402+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:20.290205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:20.290205+0000","last_clean_scrub_stamp":"2024-06-28T16:15:20.290205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:47:29.011484+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.15","version":"260'8","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.133370+0000","last_change":"2024-06-28T16:14:57.133432+0000","last_active":"2024-06-28T16:14:57.133370+0000","last_peered":"2024-06-28T16:14:57.133370+0000","last_clean":"2024-06-28T16:14:57.133370+0000","last_became_active":"2024-06-28T15:21:11.094947+0000","last_became_peered":"2024-06-28T15:21:11.094947+0000","last_unstale":"2024-06-28T16:14:57.133370+0000","last_undegraded":"2024-06-28T16:14:57.133370+0000","last_fullsized":"2024-06-28T16:14:57.133370+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'8","last_scrub_stamp":"2024-06-28T16:14:57.133168+0000","last_deep_scrub":"260'8","last_deep_scrub_stamp":"2024-06-28T16:14:57.133168+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.133168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:12:28.239090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":12406,"num_write":8,"num_write_kb":12347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.14","version":"260'5","reported_seq":70,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.646825+0000","last_change":"2024-06-28T16:14:52.646825+0000","last_active":"2024-06-28T16:14:52.646825+0000","last_peered":"2024-06-28T16:14:52.646825+0000","last_clean":"2024-06-28T16:14:52.646825+0000","last_became_active":"2024-06-28T15:21:07.025396+0000","last_became_peered":"2024-06-28T15:21:07.025396+0000","last_unstale":"2024-06-28T16:14:52.646825+0000","last_undegraded":"2024-06-28T16:14:52.646825+0000","last_fullsized":"2024-06-28T16:14:52.646825+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'5","last_scrub_stamp":"2024-06-28T16:14:52.646576+0000","last_deep_scrub":"260'5","last_deep_scrub_stamp":"2024-06-28T16:14:52.646576+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.646576+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:58:00.023881+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":12288,"num_write":5,"num_write_kb":9216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.17","version":"257'14","reported_seq":106,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.108939+0000","last_change":"2024-06-28T16:14:44.108939+0000","last_active":"2024-06-28T16:14:44.108939+0000","last_peered":"2024-06-28T16:14:44.108939+0000","last_clean":"2024-06-28T16:14:44.108939+0000","last_became_active":"2024-06-28T15:17:24.010190+0000","last_became_peered":"2024-06-28T15:17:24.010190+0000","last_unstale":"2024-06-28T16:14:44.108939+0000","last_undegraded":"2024-06-28T16:14:44.108939+0000","last_fullsized":"2024-06-28T16:14:44.108939+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'14","last_scrub_stamp":"2024-06-28T16:14:44.108596+0000","last_deep_scrub":"257'14","last_deep_scrub_stamp":"2024-06-28T16:14:44.108596+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.108596+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:06:35.967557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":51,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.13","version":"260'2","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.268389+0000","last_change":"2024-06-28T16:15:09.268572+0000","last_active":"2024-06-28T16:15:09.268389+0000","last_peered":"2024-06-28T16:15:09.268389+0000","last_clean":"2024-06-28T16:15:09.268389+0000","last_became_active":"2024-06-28T15:37:33.334097+0000","last_became_peered":"2024-06-28T15:37:33.334097+0000","last_unstale":"2024-06-28T16:15:09.268389+0000","last_undegraded":"2024-06-28T16:15:09.268389+0000","last_fullsized":"2024-06-28T16:15:09.268389+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:09.268148+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:09.268148+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.268148+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:24:54.459931+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.14","version":"260'9","reported_seq":72,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:36.208417+0000","last_change":"2024-06-28T16:15:36.208476+0000","last_active":"2024-06-28T16:15:36.208417+0000","last_peered":"2024-06-28T16:15:36.208417+0000","last_clean":"2024-06-28T16:15:36.208417+0000","last_became_active":"2024-06-28T15:21:11.107155+0000","last_became_peered":"2024-06-28T15:21:11.107155+0000","last_unstale":"2024-06-28T16:15:36.208417+0000","last_undegraded":"2024-06-28T16:15:36.208417+0000","last_fullsized":"2024-06-28T16:15:36.208417+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'9","last_scrub_stamp":"2024-06-28T16:15:36.208219+0000","last_deep_scrub":"260'9","last_deep_scrub_stamp":"2024-06-28T16:15:36.208219+0000","last_clean_scrub_stamp":"2024-06-28T16:15:36.208219+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:50:45.904748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":21504,"num_write":14,"num_write_kb":16384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.15","version":"0'0","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.557445+0000","last_change":"2024-06-28T16:14:42.557501+0000","last_active":"2024-06-28T16:14:42.557445+0000","last_peered":"2024-06-28T16:14:42.557445+0000","last_clean":"2024-06-28T16:14:42.557445+0000","last_became_active":"2024-06-28T15:21:07.042691+0000","last_became_peered":"2024-06-28T15:21:07.042691+0000","last_unstale":"2024-06-28T16:14:42.557445+0000","last_undegraded":"2024-06-28T16:14:42.557445+0000","last_fullsized":"2024-06-28T16:14:42.557445+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:42.557250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:42.557250+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.557250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:17:11.170202+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.16","version":"260'74","reported_seq":300,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.069653+0000","last_change":"2024-06-28T16:14:40.069653+0000","last_active":"2024-06-28T16:14:40.069653+0000","last_peered":"2024-06-28T16:14:40.069653+0000","last_clean":"2024-06-28T16:14:40.069653+0000","last_became_active":"2024-06-28T15:17:24.027786+0000","last_became_peered":"2024-06-28T15:17:24.027786+0000","last_unstale":"2024-06-28T16:14:40.069653+0000","last_undegraded":"2024-06-28T16:14:40.069653+0000","last_fullsized":"2024-06-28T16:14:40.069653+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'74","last_scrub_stamp":"2024-06-28T16:14:40.069506+0000","last_deep_scrub":"260'74","last_deep_scrub_stamp":"2024-06-28T16:14:40.069506+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.069506+0000","objects_scrubbed":1,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:43:30.466356+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":461,"num_read_kb":332,"num_write":85,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2470,"num_omap_keys":5,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.10","version":"0'0","reported_seq":37,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:38.089080+0000","last_change":"2024-06-28T16:15:38.089136+0000","last_active":"2024-06-28T16:15:38.089080+0000","last_peered":"2024-06-28T16:15:38.089080+0000","last_clean":"2024-06-28T16:15:38.089080+0000","last_became_active":"2024-06-28T15:37:33.354730+0000","last_became_peered":"2024-06-28T15:37:33.354730+0000","last_unstale":"2024-06-28T16:15:38.089080+0000","last_undegraded":"2024-06-28T16:15:38.089080+0000","last_fullsized":"2024-06-28T16:15:38.089080+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:38.088868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:38.088868+0000","last_clean_scrub_stamp":"2024-06-28T16:15:38.088868+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:24:22.573921+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.17","version":"260'4","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.183946+0000","last_change":"2024-06-28T16:14:53.184018+0000","last_active":"2024-06-28T16:14:53.183946+0000","last_peered":"2024-06-28T16:14:53.183946+0000","last_clean":"2024-06-28T16:14:53.183946+0000","last_became_active":"2024-06-28T15:21:11.093751+0000","last_became_peered":"2024-06-28T15:21:11.093751+0000","last_unstale":"2024-06-28T16:14:53.183946+0000","last_undegraded":"2024-06-28T16:14:53.183946+0000","last_fullsized":"2024-06-28T16:14:53.183946+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:14:53.183733+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:14:53.183733+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.183733+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:12:53.454345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":7168,"num_write":4,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.16","version":"260'2","reported_seq":62,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.333635+0000","last_change":"2024-06-28T16:15:07.333707+0000","last_active":"2024-06-28T16:15:07.333635+0000","last_peered":"2024-06-28T16:15:07.333635+0000","last_clean":"2024-06-28T16:15:07.333635+0000","last_became_active":"2024-06-28T15:21:07.042846+0000","last_became_peered":"2024-06-28T15:21:07.042846+0000","last_unstale":"2024-06-28T16:15:07.333635+0000","last_undegraded":"2024-06-28T16:15:07.333635+0000","last_fullsized":"2024-06-28T16:15:07.333635+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:07.333374+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:07.333374+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.333374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:40:39.550588+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":59,"num_write":2,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.15","version":"260'31","reported_seq":154,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.199811+0000","last_change":"2024-06-28T16:14:54.199811+0000","last_active":"2024-06-28T16:14:54.199811+0000","last_peered":"2024-06-28T16:14:54.199811+0000","last_clean":"2024-06-28T16:14:54.199811+0000","last_became_active":"2024-06-28T15:17:24.016275+0000","last_became_peered":"2024-06-28T15:17:24.016275+0000","last_unstale":"2024-06-28T16:14:54.199811+0000","last_undegraded":"2024-06-28T16:14:54.199811+0000","last_fullsized":"2024-06-28T16:14:54.199811+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'31","last_scrub_stamp":"2024-06-28T16:14:54.199529+0000","last_deep_scrub":"260'31","last_deep_scrub_stamp":"2024-06-28T16:14:54.199529+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.199529+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:16:15.219769+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":166,"num_read_kb":116,"num_write":60,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.11","version":"0'0","reported_seq":37,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:27.863859+0000","last_change":"2024-06-28T16:15:27.863916+0000","last_active":"2024-06-28T16:15:27.863859+0000","last_peered":"2024-06-28T16:15:27.863859+0000","last_clean":"2024-06-28T16:15:27.863859+0000","last_became_active":"2024-06-28T15:37:33.323174+0000","last_became_peered":"2024-06-28T15:37:33.323174+0000","last_unstale":"2024-06-28T16:15:27.863859+0000","last_undegraded":"2024-06-28T16:15:27.863859+0000","last_fullsized":"2024-06-28T16:15:27.863859+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:27.863662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:27.863662+0000","last_clean_scrub_stamp":"2024-06-28T16:15:27.863662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:58:08.717137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.16","version":"260'9","reported_seq":72,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:24.964479+0000","last_change":"2024-06-28T16:15:24.964620+0000","last_active":"2024-06-28T16:15:24.964479+0000","last_peered":"2024-06-28T16:15:24.964479+0000","last_clean":"2024-06-28T16:15:24.964479+0000","last_became_active":"2024-06-28T15:21:11.087119+0000","last_became_peered":"2024-06-28T15:21:11.087119+0000","last_unstale":"2024-06-28T16:15:24.964479+0000","last_undegraded":"2024-06-28T16:15:24.964479+0000","last_fullsized":"2024-06-28T16:15:24.964479+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'9","last_scrub_stamp":"2024-06-28T16:15:24.964220+0000","last_deep_scrub":"260'9","last_deep_scrub_stamp":"2024-06-28T16:15:24.964220+0000","last_clean_scrub_stamp":"2024-06-28T16:15:24.964220+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:50:36.338944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":24576,"num_write":19,"num_write_kb":12288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.17","version":"0'0","reported_seq":59,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.108123+0000","last_change":"2024-06-28T16:14:50.108177+0000","last_active":"2024-06-28T16:14:50.108123+0000","last_peered":"2024-06-28T16:14:50.108123+0000","last_clean":"2024-06-28T16:14:50.108123+0000","last_became_active":"2024-06-28T15:21:07.032092+0000","last_became_peered":"2024-06-28T15:21:07.032092+0000","last_unstale":"2024-06-28T16:14:50.108123+0000","last_undegraded":"2024-06-28T16:14:50.108123+0000","last_fullsized":"2024-06-28T16:14:50.108123+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:50.107930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:50.107930+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.107930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:01:58.373114+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.14","version":"260'21","reported_seq":130,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.642755+0000","last_change":"2024-06-28T16:14:54.642755+0000","last_active":"2024-06-28T16:14:54.642755+0000","last_peered":"2024-06-28T16:14:54.642755+0000","last_clean":"2024-06-28T16:14:54.642755+0000","last_became_active":"2024-06-28T15:17:24.020899+0000","last_became_peered":"2024-06-28T15:17:24.020899+0000","last_unstale":"2024-06-28T16:14:54.642755+0000","last_undegraded":"2024-06-28T16:14:54.642755+0000","last_fullsized":"2024-06-28T16:14:54.642755+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'21","last_scrub_stamp":"2024-06-28T16:14:54.642387+0000","last_deep_scrub":"260'21","last_deep_scrub_stamp":"2024-06-28T16:14:54.642387+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.642387+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:49:49.845916+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":86,"num_write":39,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.16","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:40.215684+0000","last_change":"2024-06-28T16:15:40.215745+0000","last_active":"2024-06-28T16:15:40.215684+0000","last_peered":"2024-06-28T16:15:40.215684+0000","last_clean":"2024-06-28T16:15:40.215684+0000","last_became_active":"2024-06-28T15:37:33.336587+0000","last_became_peered":"2024-06-28T15:37:33.336587+0000","last_unstale":"2024-06-28T16:15:40.215684+0000","last_undegraded":"2024-06-28T16:15:40.215684+0000","last_fullsized":"2024-06-28T16:15:40.215684+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:40.215483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:40.215483+0000","last_clean_scrub_stamp":"2024-06-28T16:15:40.215483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:01:56.072945+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.11","version":"257'3","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.256769+0000","last_change":"2024-06-28T16:15:18.256829+0000","last_active":"2024-06-28T16:15:18.256769+0000","last_peered":"2024-06-28T16:15:18.256769+0000","last_clean":"2024-06-28T16:15:18.256769+0000","last_became_active":"2024-06-28T15:21:11.097622+0000","last_became_peered":"2024-06-28T15:21:11.097622+0000","last_unstale":"2024-06-28T16:15:18.256769+0000","last_undegraded":"2024-06-28T16:15:18.256769+0000","last_fullsized":"2024-06-28T16:15:18.256769+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:18.256544+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:18.256544+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.256544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:33:41.559876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.10","version":"0'0","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.906019+0000","last_change":"2024-06-28T16:14:56.906073+0000","last_active":"2024-06-28T16:14:56.906019+0000","last_peered":"2024-06-28T16:14:56.906019+0000","last_clean":"2024-06-28T16:14:56.906019+0000","last_became_active":"2024-06-28T15:21:07.038746+0000","last_became_peered":"2024-06-28T15:21:07.038746+0000","last_unstale":"2024-06-28T16:14:56.906019+0000","last_undegraded":"2024-06-28T16:14:56.906019+0000","last_fullsized":"2024-06-28T16:14:56.906019+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.905794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.905794+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.905794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:56:22.215818+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.13","version":"257'9","reported_seq":91,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.630260+0000","last_change":"2024-06-28T16:14:53.630260+0000","last_active":"2024-06-28T16:14:53.630260+0000","last_peered":"2024-06-28T16:14:53.630260+0000","last_clean":"2024-06-28T16:14:53.630260+0000","last_became_active":"2024-06-28T15:17:24.020276+0000","last_became_peered":"2024-06-28T15:17:24.020276+0000","last_unstale":"2024-06-28T16:14:53.630260+0000","last_undegraded":"2024-06-28T16:14:53.630260+0000","last_fullsized":"2024-06-28T16:14:53.630260+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'9","last_scrub_stamp":"2024-06-28T16:14:53.629897+0000","last_deep_scrub":"257'9","last_deep_scrub_stamp":"2024-06-28T16:14:53.629897+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.629897+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:40:42.297017+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":33,"num_write":17,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.17","version":"0'0","reported_seq":36,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:36.416611+0000","last_change":"2024-06-28T16:15:36.416731+0000","last_active":"2024-06-28T16:15:36.416611+0000","last_peered":"2024-06-28T16:15:36.416611+0000","last_clean":"2024-06-28T16:15:36.416611+0000","last_became_active":"2024-06-28T15:37:33.327497+0000","last_became_peered":"2024-06-28T15:37:33.327497+0000","last_unstale":"2024-06-28T16:15:36.416611+0000","last_undegraded":"2024-06-28T16:15:36.416611+0000","last_fullsized":"2024-06-28T16:15:36.416611+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:36.416382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:36.416382+0000","last_clean_scrub_stamp":"2024-06-28T16:15:36.416382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:40:33.651339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.10","version":"260'4","reported_seq":62,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.271440+0000","last_change":"2024-06-28T16:15:16.271504+0000","last_active":"2024-06-28T16:15:16.271440+0000","last_peered":"2024-06-28T16:15:16.271440+0000","last_clean":"2024-06-28T16:15:16.271440+0000","last_became_active":"2024-06-28T15:21:11.092033+0000","last_became_peered":"2024-06-28T15:21:11.092033+0000","last_unstale":"2024-06-28T16:15:16.271440+0000","last_undegraded":"2024-06-28T16:15:16.271440+0000","last_fullsized":"2024-06-28T16:15:16.271440+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:15:16.271243+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:15:16.271243+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.271243+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:12:12.186648+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6152,"num_write":4,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.11","version":"0'0","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.141194+0000","last_change":"2024-06-28T16:14:45.141194+0000","last_active":"2024-06-28T16:14:45.141194+0000","last_peered":"2024-06-28T16:14:45.141194+0000","last_clean":"2024-06-28T16:14:45.141194+0000","last_became_active":"2024-06-28T15:21:07.045842+0000","last_became_peered":"2024-06-28T15:21:07.045842+0000","last_unstale":"2024-06-28T16:14:45.141194+0000","last_undegraded":"2024-06-28T16:14:45.141194+0000","last_fullsized":"2024-06-28T16:14:45.141194+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:45.140747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:45.140747+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.140747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:01:27.177077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"65.12","version":"260'17","reported_seq":116,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.153919+0000","last_change":"2024-06-28T16:14:43.153919+0000","last_active":"2024-06-28T16:14:43.153919+0000","last_peered":"2024-06-28T16:14:43.153919+0000","last_clean":"2024-06-28T16:14:43.153919+0000","last_became_active":"2024-06-28T15:17:24.013118+0000","last_became_peered":"2024-06-28T15:17:24.013118+0000","last_unstale":"2024-06-28T16:14:43.153919+0000","last_undegraded":"2024-06-28T16:14:43.153919+0000","last_fullsized":"2024-06-28T16:14:43.153919+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'17","last_scrub_stamp":"2024-06-28T16:14:43.153520+0000","last_deep_scrub":"260'17","last_deep_scrub_stamp":"2024-06-28T16:14:43.153520+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.153520+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:39:58.889835+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":63,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.14","version":"260'2","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:38.496400+0000","last_change":"2024-06-28T16:15:38.496536+0000","last_active":"2024-06-28T16:15:38.496400+0000","last_peered":"2024-06-28T16:15:38.496400+0000","last_clean":"2024-06-28T16:15:38.496400+0000","last_became_active":"2024-06-28T15:37:33.339097+0000","last_became_peered":"2024-06-28T15:37:33.339097+0000","last_unstale":"2024-06-28T16:15:38.496400+0000","last_undegraded":"2024-06-28T16:15:38.496400+0000","last_fullsized":"2024-06-28T16:15:38.496400+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:38.496169+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:38.496169+0000","last_clean_scrub_stamp":"2024-06-28T16:15:38.496169+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:29:36.667770+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.13","version":"257'3","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.217915+0000","last_change":"2024-06-28T16:15:05.217973+0000","last_active":"2024-06-28T16:15:05.217915+0000","last_peered":"2024-06-28T16:15:05.217915+0000","last_clean":"2024-06-28T16:15:05.217915+0000","last_became_active":"2024-06-28T15:21:11.101063+0000","last_became_peered":"2024-06-28T15:21:11.101063+0000","last_unstale":"2024-06-28T16:15:05.217915+0000","last_undegraded":"2024-06-28T16:15:05.217915+0000","last_fullsized":"2024-06-28T16:15:05.217915+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:05.217704+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:05.217704+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.217704+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:55:32.534220+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.12","version":"257'3","reported_seq":65,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.407866+0000","last_change":"2024-06-28T16:15:13.407927+0000","last_active":"2024-06-28T16:15:13.407866+0000","last_peered":"2024-06-28T16:15:13.407866+0000","last_clean":"2024-06-28T16:15:13.407866+0000","last_became_active":"2024-06-28T15:21:07.040973+0000","last_became_peered":"2024-06-28T15:21:07.040973+0000","last_unstale":"2024-06-28T16:15:13.407866+0000","last_undegraded":"2024-06-28T16:15:13.407866+0000","last_fullsized":"2024-06-28T16:15:13.407866+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:13.407661+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:13.407661+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.407661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:18:56.741530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.11","version":"260'22","reported_seq":125,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.889438+0000","last_change":"2024-06-28T16:14:40.889438+0000","last_active":"2024-06-28T16:14:40.889438+0000","last_peered":"2024-06-28T16:14:40.889438+0000","last_clean":"2024-06-28T16:14:40.889438+0000","last_became_active":"2024-06-28T15:17:24.028410+0000","last_became_peered":"2024-06-28T15:17:24.028410+0000","last_unstale":"2024-06-28T16:14:40.889438+0000","last_undegraded":"2024-06-28T16:14:40.889438+0000","last_fullsized":"2024-06-28T16:14:40.889438+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'22","last_scrub_stamp":"2024-06-28T16:14:40.889173+0000","last_deep_scrub":"260'22","last_deep_scrub_stamp":"2024-06-28T16:14:40.889173+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.889173+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:21.505212+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":76,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.15","version":"260'2","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:41.228703+0000","last_change":"2024-06-28T16:15:41.228850+0000","last_active":"2024-06-28T16:15:41.228703+0000","last_peered":"2024-06-28T16:15:41.228703+0000","last_clean":"2024-06-28T16:15:41.228703+0000","last_became_active":"2024-06-28T15:37:33.340370+0000","last_became_peered":"2024-06-28T15:37:33.340370+0000","last_unstale":"2024-06-28T16:15:41.228703+0000","last_undegraded":"2024-06-28T16:15:41.228703+0000","last_fullsized":"2024-06-28T16:15:41.228703+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:41.228459+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:41.228459+0000","last_clean_scrub_stamp":"2024-06-28T16:15:41.228459+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:32:55.510009+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.12","version":"257'3","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.353284+0000","last_change":"2024-06-28T16:15:16.353345+0000","last_active":"2024-06-28T16:15:16.353284+0000","last_peered":"2024-06-28T16:15:16.353284+0000","last_clean":"2024-06-28T16:15:16.353284+0000","last_became_active":"2024-06-28T15:21:11.104038+0000","last_became_peered":"2024-06-28T15:21:11.104038+0000","last_unstale":"2024-06-28T16:15:16.353284+0000","last_undegraded":"2024-06-28T16:15:16.353284+0000","last_fullsized":"2024-06-28T16:15:16.353284+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:16.353069+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:16.353069+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.353069+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:49:31.899786+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.13","version":"260'9","reported_seq":82,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.657797+0000","last_change":"2024-06-28T16:14:48.657797+0000","last_active":"2024-06-28T16:14:48.657797+0000","last_peered":"2024-06-28T16:14:48.657797+0000","last_clean":"2024-06-28T16:14:48.657797+0000","last_became_active":"2024-06-28T15:21:07.008899+0000","last_became_peered":"2024-06-28T15:21:07.008899+0000","last_unstale":"2024-06-28T16:14:48.657797+0000","last_undegraded":"2024-06-28T16:14:48.657797+0000","last_fullsized":"2024-06-28T16:14:48.657797+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'9","last_scrub_stamp":"2024-06-28T16:14:48.657546+0000","last_deep_scrub":"260'9","last_deep_scrub_stamp":"2024-06-28T16:14:48.657546+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.657546+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:10:31.780674+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":25602,"num_write":7,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.10","version":"260'42","reported_seq":175,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.123456+0000","last_change":"2024-06-28T16:14:43.123456+0000","last_active":"2024-06-28T16:14:43.123456+0000","last_peered":"2024-06-28T16:14:43.123456+0000","last_clean":"2024-06-28T16:14:43.123456+0000","last_became_active":"2024-06-28T15:17:24.019180+0000","last_became_peered":"2024-06-28T15:17:24.019180+0000","last_unstale":"2024-06-28T16:14:43.123456+0000","last_undegraded":"2024-06-28T16:14:43.123456+0000","last_fullsized":"2024-06-28T16:14:43.123456+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'42","last_scrub_stamp":"2024-06-28T16:14:43.123099+0000","last_deep_scrub":"260'42","last_deep_scrub_stamp":"2024-06-28T16:14:43.123099+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.123099+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:45:23.503458+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":198,"num_read_kb":139,"num_write":93,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.a","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.312243+0000","last_change":"2024-06-28T16:15:10.312306+0000","last_active":"2024-06-28T16:15:10.312243+0000","last_peered":"2024-06-28T16:15:10.312243+0000","last_clean":"2024-06-28T16:15:10.312243+0000","last_became_active":"2024-06-28T15:37:33.341716+0000","last_became_peered":"2024-06-28T15:37:33.341716+0000","last_unstale":"2024-06-28T16:15:10.312243+0000","last_undegraded":"2024-06-28T16:15:10.312243+0000","last_fullsized":"2024-06-28T16:15:10.312243+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:10.312037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:10.312037+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.312037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:41:31.673706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.d","version":"0'0","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.080244+0000","last_change":"2024-06-28T16:14:49.080244+0000","last_active":"2024-06-28T16:14:49.080244+0000","last_peered":"2024-06-28T16:14:49.080244+0000","last_clean":"2024-06-28T16:14:49.080244+0000","last_became_active":"2024-06-28T15:21:11.093276+0000","last_became_peered":"2024-06-28T15:21:11.093276+0000","last_unstale":"2024-06-28T16:14:49.080244+0000","last_undegraded":"2024-06-28T16:14:49.080244+0000","last_fullsized":"2024-06-28T16:14:49.080244+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.079996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.079996+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.079996+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:47:03.183258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.c","version":"260'9","reported_seq":76,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.651923+0000","last_change":"2024-06-28T16:14:45.651923+0000","last_active":"2024-06-28T16:14:45.651923+0000","last_peered":"2024-06-28T16:14:45.651923+0000","last_clean":"2024-06-28T16:14:45.651923+0000","last_became_active":"2024-06-28T15:21:07.040691+0000","last_became_peered":"2024-06-28T15:21:07.040691+0000","last_unstale":"2024-06-28T16:14:45.651923+0000","last_undegraded":"2024-06-28T16:14:45.651923+0000","last_fullsized":"2024-06-28T16:14:45.651923+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'9","last_scrub_stamp":"2024-06-28T16:14:45.651669+0000","last_deep_scrub":"260'9","last_deep_scrub_stamp":"2024-06-28T16:14:45.651669+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.651669+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:32:08.870332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":17408,"num_write":9,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.f","version":"260'47","reported_seq":193,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.096531+0000","last_change":"2024-06-28T16:14:42.096599+0000","last_active":"2024-06-28T16:14:42.096531+0000","last_peered":"2024-06-28T16:14:42.096531+0000","last_clean":"2024-06-28T16:14:42.096531+0000","last_became_active":"2024-06-28T15:17:24.027610+0000","last_became_peered":"2024-06-28T15:17:24.027610+0000","last_unstale":"2024-06-28T16:14:42.096531+0000","last_undegraded":"2024-06-28T16:14:42.096531+0000","last_fullsized":"2024-06-28T16:14:42.096531+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'47","last_scrub_stamp":"2024-06-28T16:14:42.096391+0000","last_deep_scrub":"260'47","last_deep_scrub_stamp":"2024-06-28T16:14:42.096391+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.096391+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:11:26.034311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":231,"num_read_kb":169,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.b","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:33.093516+0000","last_change":"2024-06-28T16:15:33.093516+0000","last_active":"2024-06-28T16:15:33.093516+0000","last_peered":"2024-06-28T16:15:33.093516+0000","last_clean":"2024-06-28T16:15:33.093516+0000","last_became_active":"2024-06-28T15:37:33.346414+0000","last_became_peered":"2024-06-28T15:37:33.346414+0000","last_unstale":"2024-06-28T16:15:33.093516+0000","last_undegraded":"2024-06-28T16:15:33.093516+0000","last_fullsized":"2024-06-28T16:15:33.093516+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:33.093268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:33.093268+0000","last_clean_scrub_stamp":"2024-06-28T16:15:33.093268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:46:17.365812+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.c","version":"260'6","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:26.891821+0000","last_change":"2024-06-28T16:15:26.891821+0000","last_active":"2024-06-28T16:15:26.891821+0000","last_peered":"2024-06-28T16:15:26.891821+0000","last_clean":"2024-06-28T16:15:26.891821+0000","last_became_active":"2024-06-28T15:22:00.940760+0000","last_became_peered":"2024-06-28T15:22:00.940760+0000","last_unstale":"2024-06-28T16:15:26.891821+0000","last_undegraded":"2024-06-28T16:15:26.891821+0000","last_fullsized":"2024-06-28T16:15:26.891821+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:15:26.891559+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:15:26.891559+0000","last_clean_scrub_stamp":"2024-06-28T16:15:26.891559+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:12:22.129920+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":4,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.d","version":"260'3","reported_seq":66,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.369210+0000","last_change":"2024-06-28T16:15:05.369210+0000","last_active":"2024-06-28T16:15:05.369210+0000","last_peered":"2024-06-28T16:15:05.369210+0000","last_clean":"2024-06-28T16:15:05.369210+0000","last_became_active":"2024-06-28T15:21:07.030163+0000","last_became_peered":"2024-06-28T15:21:07.030163+0000","last_unstale":"2024-06-28T16:15:05.369210+0000","last_undegraded":"2024-06-28T16:15:05.369210+0000","last_fullsized":"2024-06-28T16:15:05.369210+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:15:05.368894+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:15:05.368894+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.368894+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:40:02.616948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2048,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.e","version":"260'40","reported_seq":181,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.687376+0000","last_change":"2024-06-28T16:14:44.687376+0000","last_active":"2024-06-28T16:14:44.687376+0000","last_peered":"2024-06-28T16:14:44.687376+0000","last_clean":"2024-06-28T16:14:44.687376+0000","last_became_active":"2024-06-28T15:17:24.012572+0000","last_became_peered":"2024-06-28T15:17:24.012572+0000","last_unstale":"2024-06-28T16:14:44.687376+0000","last_undegraded":"2024-06-28T16:14:44.687376+0000","last_fullsized":"2024-06-28T16:14:44.687376+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'40","last_scrub_stamp":"2024-06-28T16:14:44.687125+0000","last_deep_scrub":"260'40","last_deep_scrub_stamp":"2024-06-28T16:14:44.687125+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.687125+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:52:09.797828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":148,"num_write":94,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.8","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.397140+0000","last_change":"2024-06-28T16:15:19.397200+0000","last_active":"2024-06-28T16:15:19.397140+0000","last_peered":"2024-06-28T16:15:19.397140+0000","last_clean":"2024-06-28T16:15:19.397140+0000","last_became_active":"2024-06-28T15:37:33.352959+0000","last_became_peered":"2024-06-28T15:37:33.352959+0000","last_unstale":"2024-06-28T16:15:19.397140+0000","last_undegraded":"2024-06-28T16:15:19.397140+0000","last_fullsized":"2024-06-28T16:15:19.397140+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:19.396949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:19.396949+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.396949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:10:10.507079+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"67.f","version":"0'0","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.046127+0000","last_change":"2024-06-28T16:14:49.046127+0000","last_active":"2024-06-28T16:14:49.046127+0000","last_peered":"2024-06-28T16:14:49.046127+0000","last_clean":"2024-06-28T16:14:49.046127+0000","last_became_active":"2024-06-28T15:21:11.079036+0000","last_became_peered":"2024-06-28T15:21:11.079036+0000","last_unstale":"2024-06-28T16:14:49.046127+0000","last_undegraded":"2024-06-28T16:14:49.046127+0000","last_fullsized":"2024-06-28T16:14:49.046127+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.045873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.045873+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.045873+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:29:37.775532+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.e","version":"0'0","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.128341+0000","last_change":"2024-06-28T16:14:46.128341+0000","last_active":"2024-06-28T16:14:46.128341+0000","last_peered":"2024-06-28T16:14:46.128341+0000","last_clean":"2024-06-28T16:14:46.128341+0000","last_became_active":"2024-06-28T15:21:07.027451+0000","last_became_peered":"2024-06-28T15:21:07.027451+0000","last_unstale":"2024-06-28T16:14:46.128341+0000","last_undegraded":"2024-06-28T16:14:46.128341+0000","last_fullsized":"2024-06-28T16:14:46.128341+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:46.128071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:46.128071+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.128071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:30:28.326677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.d","version":"260'16","reported_seq":107,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.081145+0000","last_change":"2024-06-28T16:14:43.081145+0000","last_active":"2024-06-28T16:14:43.081145+0000","last_peered":"2024-06-28T16:14:43.081145+0000","last_clean":"2024-06-28T16:14:43.081145+0000","last_became_active":"2024-06-28T15:17:24.026271+0000","last_became_peered":"2024-06-28T15:17:24.026271+0000","last_unstale":"2024-06-28T16:14:43.081145+0000","last_undegraded":"2024-06-28T16:14:43.081145+0000","last_fullsized":"2024-06-28T16:14:43.081145+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'16","last_scrub_stamp":"2024-06-28T16:14:43.080857+0000","last_deep_scrub":"260'16","last_deep_scrub_stamp":"2024-06-28T16:14:43.080857+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.080857+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:01:28.864357+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":48,"num_write":36,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.9","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.748084+0000","last_change":"2024-06-28T16:15:00.748137+0000","last_active":"2024-06-28T16:15:00.748084+0000","last_peered":"2024-06-28T16:15:00.748084+0000","last_clean":"2024-06-28T16:15:00.748084+0000","last_became_active":"2024-06-28T15:37:33.325849+0000","last_became_peered":"2024-06-28T15:37:33.325849+0000","last_unstale":"2024-06-28T16:15:00.748084+0000","last_undegraded":"2024-06-28T16:15:00.748084+0000","last_fullsized":"2024-06-28T16:15:00.748084+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:00.747890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:00.747890+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.747890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:59:35.430960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.e","version":"260'6","reported_seq":67,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:37.231881+0000","last_change":"2024-06-28T16:15:37.232010+0000","last_active":"2024-06-28T16:15:37.231881+0000","last_peered":"2024-06-28T16:15:37.231881+0000","last_clean":"2024-06-28T16:15:37.231881+0000","last_became_active":"2024-06-28T15:21:11.107498+0000","last_became_peered":"2024-06-28T15:21:11.107498+0000","last_unstale":"2024-06-28T16:15:37.231881+0000","last_undegraded":"2024-06-28T16:15:37.231881+0000","last_fullsized":"2024-06-28T16:15:37.231881+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:15:37.231595+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:15:37.231595+0000","last_clean_scrub_stamp":"2024-06-28T16:15:37.231595+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:33:36.860544+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":4098,"num_write":9,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"66.f","version":"260'8","reported_seq":77,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.130933+0000","last_change":"2024-06-28T16:14:41.130933+0000","last_active":"2024-06-28T16:14:41.130933+0000","last_peered":"2024-06-28T16:14:41.130933+0000","last_clean":"2024-06-28T16:14:41.130933+0000","last_became_active":"2024-06-28T15:21:07.029525+0000","last_became_peered":"2024-06-28T15:21:07.029525+0000","last_unstale":"2024-06-28T16:14:41.130933+0000","last_undegraded":"2024-06-28T16:14:41.130933+0000","last_fullsized":"2024-06-28T16:14:41.130933+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'8","last_scrub_stamp":"2024-06-28T16:14:41.130824+0000","last_deep_scrub":"260'8","last_deep_scrub_stamp":"2024-06-28T16:14:41.130824+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.130824+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:31:52.999376+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8194,"num_write":11,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.c","version":"260'34","reported_seq":161,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.199853+0000","last_change":"2024-06-28T16:14:52.199853+0000","last_active":"2024-06-28T16:14:52.199853+0000","last_peered":"2024-06-28T16:14:52.199853+0000","last_clean":"2024-06-28T16:14:52.199853+0000","last_became_active":"2024-06-28T15:17:24.007649+0000","last_became_peered":"2024-06-28T15:17:24.007649+0000","last_unstale":"2024-06-28T16:14:52.199853+0000","last_undegraded":"2024-06-28T16:14:52.199853+0000","last_fullsized":"2024-06-28T16:14:52.199853+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'34","last_scrub_stamp":"2024-06-28T16:14:52.199460+0000","last_deep_scrub":"260'34","last_deep_scrub_stamp":"2024-06-28T16:14:52.199460+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.199460+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:55:51.649887+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":172,"num_read_kb":122,"num_write":76,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"68.e","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:38.213118+0000","last_change":"2024-06-28T16:15:38.213118+0000","last_active":"2024-06-28T16:15:38.213118+0000","last_peered":"2024-06-28T16:15:38.213118+0000","last_clean":"2024-06-28T16:15:38.213118+0000","last_became_active":"2024-06-28T15:37:33.334082+0000","last_became_peered":"2024-06-28T15:37:33.334082+0000","last_unstale":"2024-06-28T16:15:38.213118+0000","last_undegraded":"2024-06-28T16:15:38.213118+0000","last_fullsized":"2024-06-28T16:15:38.213118+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:38.212835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:38.212835+0000","last_clean_scrub_stamp":"2024-06-28T16:15:38.212835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:34:21.976990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"67.9","version":"0'0","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:28.967650+0000","last_change":"2024-06-28T16:15:28.967711+0000","last_active":"2024-06-28T16:15:28.967650+0000","last_peered":"2024-06-28T16:15:28.967650+0000","last_clean":"2024-06-28T16:15:28.967650+0000","last_became_active":"2024-06-28T15:21:11.107686+0000","last_became_peered":"2024-06-28T15:21:11.107686+0000","last_unstale":"2024-06-28T16:15:28.967650+0000","last_undegraded":"2024-06-28T16:15:28.967650+0000","last_fullsized":"2024-06-28T16:15:28.967650+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:28.967453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:28.967453+0000","last_clean_scrub_stamp":"2024-06-28T16:15:28.967453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:12:31.455209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.8","version":"0'0","reported_seq":62,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.052046+0000","last_change":"2024-06-28T16:14:54.052046+0000","last_active":"2024-06-28T16:14:54.052046+0000","last_peered":"2024-06-28T16:14:54.052046+0000","last_clean":"2024-06-28T16:14:54.052046+0000","last_became_active":"2024-06-28T15:21:07.032719+0000","last_became_peered":"2024-06-28T15:21:07.032719+0000","last_unstale":"2024-06-28T16:14:54.052046+0000","last_undegraded":"2024-06-28T16:14:54.052046+0000","last_fullsized":"2024-06-28T16:14:54.052046+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:54.051795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:54.051795+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.051795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:51:48.898614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.b","version":"260'45","reported_seq":192,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.159396+0000","last_change":"2024-06-28T16:14:40.159396+0000","last_active":"2024-06-28T16:14:40.159396+0000","last_peered":"2024-06-28T16:14:40.159396+0000","last_clean":"2024-06-28T16:14:40.159396+0000","last_became_active":"2024-06-28T15:17:24.029532+0000","last_became_peered":"2024-06-28T15:17:24.029532+0000","last_unstale":"2024-06-28T16:14:40.159396+0000","last_undegraded":"2024-06-28T16:14:40.159396+0000","last_fullsized":"2024-06-28T16:14:40.159396+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'45","last_scrub_stamp":"2024-06-28T16:14:40.159221+0000","last_deep_scrub":"260'45","last_deep_scrub_stamp":"2024-06-28T16:14:40.159221+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.159221+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:08:46.885820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":234,"num_read_kb":165,"num_write":86,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.f","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:36.125373+0000","last_change":"2024-06-28T16:15:36.125373+0000","last_active":"2024-06-28T16:15:36.125373+0000","last_peered":"2024-06-28T16:15:36.125373+0000","last_clean":"2024-06-28T16:15:36.125373+0000","last_became_active":"2024-06-28T15:37:33.347691+0000","last_became_peered":"2024-06-28T15:37:33.347691+0000","last_unstale":"2024-06-28T16:15:36.125373+0000","last_undegraded":"2024-06-28T16:15:36.125373+0000","last_fullsized":"2024-06-28T16:15:36.125373+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:36.125085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:36.125085+0000","last_clean_scrub_stamp":"2024-06-28T16:15:36.125085+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:53:25.842476+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.8","version":"0'0","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.580849+0000","last_change":"2024-06-28T16:14:48.580849+0000","last_active":"2024-06-28T16:14:48.580849+0000","last_peered":"2024-06-28T16:14:48.580849+0000","last_clean":"2024-06-28T16:14:48.580849+0000","last_became_active":"2024-06-28T15:21:11.091656+0000","last_became_peered":"2024-06-28T15:21:11.091656+0000","last_unstale":"2024-06-28T16:14:48.580849+0000","last_undegraded":"2024-06-28T16:14:48.580849+0000","last_fullsized":"2024-06-28T16:14:48.580849+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:48.580598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:48.580598+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.580598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:07:09.489282+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.9","version":"260'5","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:39.099851+0000","last_change":"2024-06-28T16:14:39.099851+0000","last_active":"2024-06-28T16:14:39.099851+0000","last_peered":"2024-06-28T16:14:39.099851+0000","last_clean":"2024-06-28T16:14:39.099851+0000","last_became_active":"2024-06-28T15:21:07.029495+0000","last_became_peered":"2024-06-28T15:21:07.029495+0000","last_unstale":"2024-06-28T16:14:39.099851+0000","last_undegraded":"2024-06-28T16:14:39.099851+0000","last_fullsized":"2024-06-28T16:14:39.099851+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'5","last_scrub_stamp":"2024-06-28T16:14:39.099694+0000","last_deep_scrub":"260'5","last_deep_scrub_stamp":"2024-06-28T16:14:39.099694+0000","last_clean_scrub_stamp":"2024-06-28T16:14:39.099694+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:24:20.828667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":318,"num_write":10,"num_write_kb":5438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.a","version":"260'39","reported_seq":171,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.649465+0000","last_change":"2024-06-28T16:14:50.649616+0000","last_active":"2024-06-28T16:14:50.649465+0000","last_peered":"2024-06-28T16:14:50.649465+0000","last_clean":"2024-06-28T16:14:50.649465+0000","last_became_active":"2024-06-28T15:17:24.021200+0000","last_became_peered":"2024-06-28T15:17:24.021200+0000","last_unstale":"2024-06-28T16:14:50.649465+0000","last_undegraded":"2024-06-28T16:14:50.649465+0000","last_fullsized":"2024-06-28T16:14:50.649465+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'39","last_scrub_stamp":"2024-06-28T16:14:50.649227+0000","last_deep_scrub":"260'39","last_deep_scrub_stamp":"2024-06-28T16:14:50.649227+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.649227+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:23:42.138261+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":191,"num_read_kb":133,"num_write":87,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.c","version":"260'2","reported_seq":40,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.759599+0000","last_change":"2024-06-28T16:15:04.759599+0000","last_active":"2024-06-28T16:15:04.759599+0000","last_peered":"2024-06-28T16:15:04.759599+0000","last_clean":"2024-06-28T16:15:04.759599+0000","last_became_active":"2024-06-28T15:37:33.335464+0000","last_became_peered":"2024-06-28T15:37:33.335464+0000","last_unstale":"2024-06-28T16:15:04.759599+0000","last_undegraded":"2024-06-28T16:15:04.759599+0000","last_fullsized":"2024-06-28T16:15:04.759599+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:04.759350+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:04.759350+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.759350+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:56:34.598481+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.b","version":"257'3","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.270056+0000","last_change":"2024-06-28T16:15:17.270119+0000","last_active":"2024-06-28T16:15:17.270056+0000","last_peered":"2024-06-28T16:15:17.270056+0000","last_clean":"2024-06-28T16:15:17.270056+0000","last_became_active":"2024-06-28T15:21:11.091647+0000","last_became_peered":"2024-06-28T15:21:11.091647+0000","last_unstale":"2024-06-28T16:15:17.270056+0000","last_undegraded":"2024-06-28T16:15:17.270056+0000","last_fullsized":"2024-06-28T16:15:17.270056+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:15:17.269853+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:15:17.269853+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.269853+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:16:54.117703+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":3,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.a","version":"257'3","reported_seq":67,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.098585+0000","last_change":"2024-06-28T16:14:58.098585+0000","last_active":"2024-06-28T16:14:58.098585+0000","last_peered":"2024-06-28T16:14:58.098585+0000","last_clean":"2024-06-28T16:14:58.098585+0000","last_became_active":"2024-06-28T15:21:07.046218+0000","last_became_peered":"2024-06-28T15:21:07.046218+0000","last_unstale":"2024-06-28T16:14:58.098585+0000","last_undegraded":"2024-06-28T16:14:58.098585+0000","last_fullsized":"2024-06-28T16:14:58.098585+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:14:58.098319+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:14:58.098319+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.098319+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:38:23.012737+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":1024,"num_write":8,"num_write_kb":1024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"65.9","version":"260'54","reported_seq":218,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.106537+0000","last_change":"2024-06-28T16:14:47.106537+0000","last_active":"2024-06-28T16:14:47.106537+0000","last_peered":"2024-06-28T16:14:47.106537+0000","last_clean":"2024-06-28T16:14:47.106537+0000","last_became_active":"2024-06-28T15:17:24.010230+0000","last_became_peered":"2024-06-28T15:17:24.010230+0000","last_unstale":"2024-06-28T16:14:47.106537+0000","last_undegraded":"2024-06-28T16:14:47.106537+0000","last_fullsized":"2024-06-28T16:14:47.106537+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'54","last_scrub_stamp":"2024-06-28T16:14:47.106259+0000","last_deep_scrub":"260'54","last_deep_scrub_stamp":"2024-06-28T16:14:47.106259+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.106259+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:33:32.525834+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":283,"num_read_kb":205,"num_write":97,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"68.d","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:39.505030+0000","last_change":"2024-06-28T16:15:39.505030+0000","last_active":"2024-06-28T16:15:39.505030+0000","last_peered":"2024-06-28T16:15:39.505030+0000","last_clean":"2024-06-28T16:15:39.505030+0000","last_became_active":"2024-06-28T15:37:33.342585+0000","last_became_peered":"2024-06-28T15:37:33.342585+0000","last_unstale":"2024-06-28T16:15:39.505030+0000","last_undegraded":"2024-06-28T16:15:39.505030+0000","last_fullsized":"2024-06-28T16:15:39.505030+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:39.504775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:39.504775+0000","last_clean_scrub_stamp":"2024-06-28T16:15:39.504775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:04:43.776599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.a","version":"260'3","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.667083+0000","last_change":"2024-06-28T16:14:56.667083+0000","last_active":"2024-06-28T16:14:56.667083+0000","last_peered":"2024-06-28T16:14:56.667083+0000","last_clean":"2024-06-28T16:14:56.667083+0000","last_became_active":"2024-06-28T15:21:11.096884+0000","last_became_peered":"2024-06-28T15:21:11.096884+0000","last_unstale":"2024-06-28T16:14:56.667083+0000","last_undegraded":"2024-06-28T16:14:56.667083+0000","last_fullsized":"2024-06-28T16:14:56.667083+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:14:56.666741+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:14:56.666741+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.666741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:25:21.454528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.b","version":"260'12","reported_seq":90,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.959056+0000","last_change":"2024-06-28T16:14:52.959056+0000","last_active":"2024-06-28T16:14:52.959056+0000","last_peered":"2024-06-28T16:14:52.959056+0000","last_clean":"2024-06-28T16:14:52.959056+0000","last_became_active":"2024-06-28T15:21:07.032545+0000","last_became_peered":"2024-06-28T15:21:07.032545+0000","last_unstale":"2024-06-28T16:14:52.959056+0000","last_undegraded":"2024-06-28T16:14:52.959056+0000","last_fullsized":"2024-06-28T16:14:52.959056+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'12","last_scrub_stamp":"2024-06-28T16:14:52.958803+0000","last_deep_scrub":"260'12","last_deep_scrub_stamp":"2024-06-28T16:14:52.958803+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.958803+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:27:34.294061+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":35842,"num_write":25,"num_write_kb":15360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"65.8","version":"260'40","reported_seq":181,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.119952+0000","last_change":"2024-06-28T16:14:42.119952+0000","last_active":"2024-06-28T16:14:42.119952+0000","last_peered":"2024-06-28T16:14:42.119952+0000","last_clean":"2024-06-28T16:14:42.119952+0000","last_became_active":"2024-06-28T15:17:24.026702+0000","last_became_peered":"2024-06-28T15:17:24.026702+0000","last_unstale":"2024-06-28T16:14:42.119952+0000","last_undegraded":"2024-06-28T16:14:42.119952+0000","last_fullsized":"2024-06-28T16:14:42.119952+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'40","last_scrub_stamp":"2024-06-28T16:14:42.119697+0000","last_deep_scrub":"260'40","last_deep_scrub_stamp":"2024-06-28T16:14:42.119697+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.119697+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:57:41.013540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":211,"num_read_kb":150,"num_write":83,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.2","version":"0'0","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:32.063707+0000","last_change":"2024-06-28T16:15:32.063762+0000","last_active":"2024-06-28T16:15:32.063707+0000","last_peered":"2024-06-28T16:15:32.063707+0000","last_clean":"2024-06-28T16:15:32.063707+0000","last_became_active":"2024-06-28T15:37:33.342889+0000","last_became_peered":"2024-06-28T15:37:33.342889+0000","last_unstale":"2024-06-28T16:15:32.063707+0000","last_undegraded":"2024-06-28T16:15:32.063707+0000","last_fullsized":"2024-06-28T16:15:32.063707+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:32.063502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:32.063502+0000","last_clean_scrub_stamp":"2024-06-28T16:15:32.063502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:55:56.613996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.5","version":"260'2","reported_seq":60,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.187312+0000","last_change":"2024-06-28T16:15:06.187312+0000","last_active":"2024-06-28T16:15:06.187312+0000","last_peered":"2024-06-28T16:15:06.187312+0000","last_clean":"2024-06-28T16:15:06.187312+0000","last_became_active":"2024-06-28T15:21:11.092614+0000","last_became_peered":"2024-06-28T15:21:11.092614+0000","last_unstale":"2024-06-28T16:15:06.187312+0000","last_undegraded":"2024-06-28T16:15:06.187312+0000","last_fullsized":"2024-06-28T16:15:06.187312+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:06.187009+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:06.187009+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.187009+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:50:30.362690+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3072,"num_write":2,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.4","version":"257'3","reported_seq":68,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.626086+0000","last_change":"2024-06-28T16:14:47.626086+0000","last_active":"2024-06-28T16:14:47.626086+0000","last_peered":"2024-06-28T16:14:47.626086+0000","last_clean":"2024-06-28T16:14:47.626086+0000","last_became_active":"2024-06-28T15:21:07.034845+0000","last_became_peered":"2024-06-28T15:21:07.034845+0000","last_unstale":"2024-06-28T16:14:47.626086+0000","last_undegraded":"2024-06-28T16:14:47.626086+0000","last_fullsized":"2024-06-28T16:14:47.626086+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:14:47.625736+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:14:47.625736+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.625736+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:32:04.945420+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.7","version":"260'28","reported_seq":149,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.166687+0000","last_change":"2024-06-28T16:14:41.166687+0000","last_active":"2024-06-28T16:14:41.166687+0000","last_peered":"2024-06-28T16:14:41.166687+0000","last_clean":"2024-06-28T16:14:41.166687+0000","last_became_active":"2024-06-28T15:17:24.014034+0000","last_became_peered":"2024-06-28T15:17:24.014034+0000","last_unstale":"2024-06-28T16:14:41.166687+0000","last_undegraded":"2024-06-28T16:14:41.166687+0000","last_fullsized":"2024-06-28T16:14:41.166687+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'28","last_scrub_stamp":"2024-06-28T16:14:41.166589+0000","last_deep_scrub":"260'28","last_deep_scrub_stamp":"2024-06-28T16:14:41.166589+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.166589+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:11:28.889098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":149,"num_read_kb":105,"num_write":55,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"68.3","version":"0'0","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:29.853051+0000","last_change":"2024-06-28T16:15:29.853176+0000","last_active":"2024-06-28T16:15:29.853051+0000","last_peered":"2024-06-28T16:15:29.853051+0000","last_clean":"2024-06-28T16:15:29.853051+0000","last_became_active":"2024-06-28T15:37:33.339831+0000","last_became_peered":"2024-06-28T15:37:33.339831+0000","last_unstale":"2024-06-28T16:15:29.853051+0000","last_undegraded":"2024-06-28T16:15:29.853051+0000","last_fullsized":"2024-06-28T16:15:29.853051+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:29.852800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:29.852800+0000","last_clean_scrub_stamp":"2024-06-28T16:15:29.852800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:45:20.965050+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"67.4","version":"260'8","reported_seq":76,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:33.390399+0000","last_change":"2024-06-28T16:15:33.390470+0000","last_active":"2024-06-28T16:15:33.390399+0000","last_peered":"2024-06-28T16:15:33.390399+0000","last_clean":"2024-06-28T16:15:33.390399+0000","last_became_active":"2024-06-28T15:21:11.088011+0000","last_became_peered":"2024-06-28T15:21:11.088011+0000","last_unstale":"2024-06-28T16:15:33.390399+0000","last_undegraded":"2024-06-28T16:15:33.390399+0000","last_fullsized":"2024-06-28T16:15:33.390399+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'8","last_scrub_stamp":"2024-06-28T16:15:33.390135+0000","last_deep_scrub":"260'8","last_deep_scrub_stamp":"2024-06-28T16:15:33.390135+0000","last_clean_scrub_stamp":"2024-06-28T16:15:33.390135+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:14:57.591361+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242880,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":20522,"num_write":11,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"66.5","version":"0'0","reported_seq":63,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.347289+0000","last_change":"2024-06-28T16:15:06.347289+0000","last_active":"2024-06-28T16:15:06.347289+0000","last_peered":"2024-06-28T16:15:06.347289+0000","last_clean":"2024-06-28T16:15:06.347289+0000","last_became_active":"2024-06-28T15:21:07.030690+0000","last_became_peered":"2024-06-28T15:21:07.030690+0000","last_unstale":"2024-06-28T16:15:06.347289+0000","last_undegraded":"2024-06-28T16:15:06.347289+0000","last_fullsized":"2024-06-28T16:15:06.347289+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:06.347026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:06.347026+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.347026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:04:58.965982+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"65.6","version":"260'31","reported_seq":154,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.094763+0000","last_change":"2024-06-28T16:14:44.094763+0000","last_active":"2024-06-28T16:14:44.094763+0000","last_peered":"2024-06-28T16:14:44.094763+0000","last_clean":"2024-06-28T16:14:44.094763+0000","last_became_active":"2024-06-28T15:17:24.026404+0000","last_became_peered":"2024-06-28T15:17:24.026404+0000","last_unstale":"2024-06-28T16:14:44.094763+0000","last_undegraded":"2024-06-28T16:14:44.094763+0000","last_fullsized":"2024-06-28T16:14:44.094763+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'31","last_scrub_stamp":"2024-06-28T16:14:44.094496+0000","last_deep_scrub":"260'31","last_deep_scrub_stamp":"2024-06-28T16:14:44.094496+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.094496+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:11:57.383608+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":161,"num_read_kb":111,"num_write":59,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.0","version":"0'0","reported_seq":40,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:21.288271+0000","last_change":"2024-06-28T16:15:21.288271+0000","last_active":"2024-06-28T16:15:21.288271+0000","last_peered":"2024-06-28T16:15:21.288271+0000","last_clean":"2024-06-28T16:15:21.288271+0000","last_became_active":"2024-06-28T15:37:33.332245+0000","last_became_peered":"2024-06-28T15:37:33.332245+0000","last_unstale":"2024-06-28T16:15:21.288271+0000","last_undegraded":"2024-06-28T16:15:21.288271+0000","last_fullsized":"2024-06-28T16:15:21.288271+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:21.287899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:21.287899+0000","last_clean_scrub_stamp":"2024-06-28T16:15:21.287899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:32:23.487275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"67.7","version":"0'0","reported_seq":55,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.610366+0000","last_change":"2024-06-28T16:14:52.610366+0000","last_active":"2024-06-28T16:14:52.610366+0000","last_peered":"2024-06-28T16:14:52.610366+0000","last_clean":"2024-06-28T16:14:52.610366+0000","last_became_active":"2024-06-28T15:21:11.096926+0000","last_became_peered":"2024-06-28T15:21:11.096926+0000","last_unstale":"2024-06-28T16:14:52.610366+0000","last_undegraded":"2024-06-28T16:14:52.610366+0000","last_fullsized":"2024-06-28T16:14:52.610366+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:52.610021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:52.610021+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.610021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:07:41.471423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"66.6","version":"260'2","reported_seq":65,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.578940+0000","last_change":"2024-06-28T16:14:56.578940+0000","last_active":"2024-06-28T16:14:56.578940+0000","last_peered":"2024-06-28T16:14:56.578940+0000","last_clean":"2024-06-28T16:14:56.578940+0000","last_became_active":"2024-06-28T15:21:07.036735+0000","last_became_peered":"2024-06-28T15:21:07.036735+0000","last_unstale":"2024-06-28T16:14:56.578940+0000","last_undegraded":"2024-06-28T16:14:56.578940+0000","last_fullsized":"2024-06-28T16:14:56.578940+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:56.578669+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:56.578669+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.578669+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:47:53.590130+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":571,"num_write":2,"num_write_kb":571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"65.5","version":"260'49","reported_seq":207,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.629207+0000","last_change":"2024-06-28T16:14:49.629207+0000","last_active":"2024-06-28T16:14:49.629207+0000","last_peered":"2024-06-28T16:14:49.629207+0000","last_clean":"2024-06-28T16:14:49.629207+0000","last_became_active":"2024-06-28T15:17:24.025855+0000","last_became_peered":"2024-06-28T15:17:24.025855+0000","last_unstale":"2024-06-28T16:14:49.629207+0000","last_undegraded":"2024-06-28T16:14:49.629207+0000","last_fullsized":"2024-06-28T16:14:49.629207+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'49","last_scrub_stamp":"2024-06-28T16:14:49.628959+0000","last_deep_scrub":"260'49","last_deep_scrub_stamp":"2024-06-28T16:14:49.628959+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.628959+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:27:20.197035+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":264,"num_read_kb":190,"num_write":87,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"68.1","version":"0'0","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:29.934179+0000","last_change":"2024-06-28T16:15:29.934234+0000","last_active":"2024-06-28T16:15:29.934179+0000","last_peered":"2024-06-28T16:15:29.934179+0000","last_clean":"2024-06-28T16:15:29.934179+0000","last_became_active":"2024-06-28T15:37:33.322927+0000","last_became_peered":"2024-06-28T15:37:33.322927+0000","last_unstale":"2024-06-28T16:15:29.934179+0000","last_undegraded":"2024-06-28T16:15:29.934179+0000","last_fullsized":"2024-06-28T16:15:29.934179+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:29.933985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:29.933985+0000","last_clean_scrub_stamp":"2024-06-28T16:15:29.933985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:58:05.348138+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"67.6","version":"0'0","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:26.965303+0000","last_change":"2024-06-28T16:15:26.965362+0000","last_active":"2024-06-28T16:15:26.965303+0000","last_peered":"2024-06-28T16:15:26.965303+0000","last_clean":"2024-06-28T16:15:26.965303+0000","last_became_active":"2024-06-28T15:21:11.097766+0000","last_became_peered":"2024-06-28T15:21:11.097766+0000","last_unstale":"2024-06-28T16:15:26.965303+0000","last_undegraded":"2024-06-28T16:15:26.965303+0000","last_fullsized":"2024-06-28T16:15:26.965303+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:26.965105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:26.965105+0000","last_clean_scrub_stamp":"2024-06-28T16:15:26.965105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:36:31.252819+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.7","version":"0'0","reported_seq":61,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.616012+0000","last_change":"2024-06-28T16:14:46.616012+0000","last_active":"2024-06-28T16:14:46.616012+0000","last_peered":"2024-06-28T16:14:46.616012+0000","last_clean":"2024-06-28T16:14:46.616012+0000","last_became_active":"2024-06-28T15:21:07.039381+0000","last_became_peered":"2024-06-28T15:21:07.039381+0000","last_unstale":"2024-06-28T16:14:46.616012+0000","last_undegraded":"2024-06-28T16:14:46.616012+0000","last_fullsized":"2024-06-28T16:14:46.616012+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:46.615676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:46.615676+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.615676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:02:38.667378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.4","version":"260'26","reported_seq":147,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.130952+0000","last_change":"2024-06-28T16:14:42.130952+0000","last_active":"2024-06-28T16:14:42.130952+0000","last_peered":"2024-06-28T16:14:42.130952+0000","last_clean":"2024-06-28T16:14:42.130952+0000","last_became_active":"2024-06-28T15:17:24.026580+0000","last_became_peered":"2024-06-28T15:17:24.026580+0000","last_unstale":"2024-06-28T16:14:42.130952+0000","last_undegraded":"2024-06-28T16:14:42.130952+0000","last_fullsized":"2024-06-28T16:14:42.130952+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'26","last_scrub_stamp":"2024-06-28T16:14:42.130814+0000","last_deep_scrub":"260'26","last_deep_scrub_stamp":"2024-06-28T16:14:42.130814+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.130814+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:03:06.475528+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":143,"num_read_kb":101,"num_write":52,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"68.6","version":"260'2","reported_seq":40,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.336046+0000","last_change":"2024-06-28T16:15:11.336114+0000","last_active":"2024-06-28T16:15:11.336046+0000","last_peered":"2024-06-28T16:15:11.336046+0000","last_clean":"2024-06-28T16:15:11.336046+0000","last_became_active":"2024-06-28T15:37:33.349137+0000","last_became_peered":"2024-06-28T15:37:33.349137+0000","last_unstale":"2024-06-28T16:15:11.336046+0000","last_undegraded":"2024-06-28T16:15:11.336046+0000","last_fullsized":"2024-06-28T16:15:11.336046+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:15:11.335857+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:15:11.335857+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.335857+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:55:35.045287+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":188,"num_omap_keys":2,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"67.1","version":"260'6","reported_seq":65,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.298763+0000","last_change":"2024-06-28T16:15:19.298763+0000","last_active":"2024-06-28T16:15:19.298763+0000","last_peered":"2024-06-28T16:15:19.298763+0000","last_clean":"2024-06-28T16:15:19.298763+0000","last_became_active":"2024-06-28T15:21:11.104270+0000","last_became_peered":"2024-06-28T15:21:11.104270+0000","last_unstale":"2024-06-28T16:15:19.298763+0000","last_undegraded":"2024-06-28T16:15:19.298763+0000","last_fullsized":"2024-06-28T16:15:19.298763+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:15:19.298407+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:15:19.298407+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.298407+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:47:09.226133+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6144,"num_write":11,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"66.0","version":"260'6","reported_seq":73,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.653159+0000","last_change":"2024-06-28T16:14:44.653159+0000","last_active":"2024-06-28T16:14:44.653159+0000","last_peered":"2024-06-28T16:14:44.653159+0000","last_clean":"2024-06-28T16:14:44.653159+0000","last_became_active":"2024-06-28T15:21:07.048639+0000","last_became_peered":"2024-06-28T15:21:07.048639+0000","last_unstale":"2024-06-28T16:14:44.653159+0000","last_undegraded":"2024-06-28T16:14:44.653159+0000","last_fullsized":"2024-06-28T16:14:44.653159+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:14:44.652902+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:14:44.652902+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.652902+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:58:20.490373+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":12288,"num_write":11,"num_write_kb":11264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.3","version":"260'22","reported_seq":124,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.693689+0000","last_change":"2024-06-28T16:14:41.693689+0000","last_active":"2024-06-28T16:14:41.693689+0000","last_peered":"2024-06-28T16:14:41.693689+0000","last_clean":"2024-06-28T16:14:41.693689+0000","last_became_active":"2024-06-28T15:17:24.006784+0000","last_became_peered":"2024-06-28T15:17:24.006784+0000","last_unstale":"2024-06-28T16:14:41.693689+0000","last_undegraded":"2024-06-28T16:14:41.693689+0000","last_fullsized":"2024-06-28T16:14:41.693689+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'22","last_scrub_stamp":"2024-06-28T16:14:41.693433+0000","last_deep_scrub":"260'22","last_deep_scrub_stamp":"2024-06-28T16:14:41.693433+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.693433+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:31:21.802215+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":64,"num_write":51,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"68.7","version":"0'0","reported_seq":38,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:37.456493+0000","last_change":"2024-06-28T16:15:37.456550+0000","last_active":"2024-06-28T16:15:37.456493+0000","last_peered":"2024-06-28T16:15:37.456493+0000","last_clean":"2024-06-28T16:15:37.456493+0000","last_became_active":"2024-06-28T15:37:33.333737+0000","last_became_peered":"2024-06-28T15:37:33.333737+0000","last_unstale":"2024-06-28T16:15:37.456493+0000","last_undegraded":"2024-06-28T16:15:37.456493+0000","last_fullsized":"2024-06-28T16:15:37.456493+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:37.456285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:37.456285+0000","last_clean_scrub_stamp":"2024-06-28T16:15:37.456285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:58:21.487688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.0","version":"0'0","reported_seq":58,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:24.889265+0000","last_change":"2024-06-28T16:15:24.889265+0000","last_active":"2024-06-28T16:15:24.889265+0000","last_peered":"2024-06-28T16:15:24.889265+0000","last_clean":"2024-06-28T16:15:24.889265+0000","last_became_active":"2024-06-28T15:21:11.092034+0000","last_became_peered":"2024-06-28T15:21:11.092034+0000","last_unstale":"2024-06-28T16:15:24.889265+0000","last_undegraded":"2024-06-28T16:15:24.889265+0000","last_fullsized":"2024-06-28T16:15:24.889265+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:24.888926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:24.888926+0000","last_clean_scrub_stamp":"2024-06-28T16:15:24.888926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:34:36.805202+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"66.1","version":"260'3","reported_seq":69,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.606954+0000","last_change":"2024-06-28T16:14:43.606954+0000","last_active":"2024-06-28T16:14:43.606954+0000","last_peered":"2024-06-28T16:14:43.606954+0000","last_clean":"2024-06-28T16:14:43.606954+0000","last_became_active":"2024-06-28T15:21:07.039812+0000","last_became_peered":"2024-06-28T15:21:07.039812+0000","last_unstale":"2024-06-28T16:14:43.606954+0000","last_undegraded":"2024-06-28T16:14:43.606954+0000","last_fullsized":"2024-06-28T16:14:43.606954+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:14:43.606698+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:14:43.606698+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.606698+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:49:13.801868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10240,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"65.2","version":"260'78","reported_seq":318,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.919989+0000","last_change":"2024-06-28T16:14:41.919989+0000","last_active":"2024-06-28T16:14:41.919989+0000","last_peered":"2024-06-28T16:14:41.919989+0000","last_clean":"2024-06-28T16:14:41.919989+0000","last_became_active":"2024-06-28T15:17:24.012854+0000","last_became_peered":"2024-06-28T15:17:24.012854+0000","last_unstale":"2024-06-28T16:14:41.919989+0000","last_undegraded":"2024-06-28T16:14:41.919989+0000","last_fullsized":"2024-06-28T16:14:41.919989+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'78","last_scrub_stamp":"2024-06-28T16:14:41.919720+0000","last_deep_scrub":"260'78","last_deep_scrub_stamp":"2024-06-28T16:14:41.919720+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.919720+0000","objects_scrubbed":1,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:17:37.822460+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":483,"num_read_kb":330,"num_write":109,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4960,"num_omap_keys":10,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"68.4","version":"0'0","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:35.421126+0000","last_change":"2024-06-28T16:15:35.421182+0000","last_active":"2024-06-28T16:15:35.421126+0000","last_peered":"2024-06-28T16:15:35.421126+0000","last_clean":"2024-06-28T16:15:35.421126+0000","last_became_active":"2024-06-28T15:37:33.322317+0000","last_became_peered":"2024-06-28T15:37:33.322317+0000","last_unstale":"2024-06-28T16:15:35.421126+0000","last_undegraded":"2024-06-28T16:15:35.421126+0000","last_fullsized":"2024-06-28T16:15:35.421126+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:35.420912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:35.420912+0000","last_clean_scrub_stamp":"2024-06-28T16:15:35.420912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:26:11.682180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"67.3","version":"0'0","reported_seq":56,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.394628+0000","last_change":"2024-06-28T16:15:17.394628+0000","last_active":"2024-06-28T16:15:17.394628+0000","last_peered":"2024-06-28T16:15:17.394628+0000","last_clean":"2024-06-28T16:15:17.394628+0000","last_became_active":"2024-06-28T15:21:11.108663+0000","last_became_peered":"2024-06-28T15:21:11.108663+0000","last_unstale":"2024-06-28T16:15:17.394628+0000","last_undegraded":"2024-06-28T16:15:17.394628+0000","last_fullsized":"2024-06-28T16:15:17.394628+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:17.394367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:17.394367+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.394367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:27:51.882594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"66.2","version":"260'3","reported_seq":68,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.135141+0000","last_change":"2024-06-28T16:14:40.135141+0000","last_active":"2024-06-28T16:14:40.135141+0000","last_peered":"2024-06-28T16:14:40.135141+0000","last_clean":"2024-06-28T16:14:40.135141+0000","last_became_active":"2024-06-28T15:21:07.022046+0000","last_became_peered":"2024-06-28T15:21:07.022046+0000","last_unstale":"2024-06-28T16:14:40.135141+0000","last_undegraded":"2024-06-28T16:14:40.135141+0000","last_fullsized":"2024-06-28T16:14:40.135141+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-06-28T16:14:40.134988+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-06-28T16:14:40.134988+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.134988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:06:49.985940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":5120,"num_write":3,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"65.1","version":"260'31","reported_seq":146,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.121348+0000","last_change":"2024-06-28T16:14:45.121348+0000","last_active":"2024-06-28T16:14:45.121348+0000","last_peered":"2024-06-28T16:14:45.121348+0000","last_clean":"2024-06-28T16:14:45.121348+0000","last_became_active":"2024-06-28T15:17:24.010873+0000","last_became_peered":"2024-06-28T15:17:24.010873+0000","last_unstale":"2024-06-28T16:14:45.121348+0000","last_undegraded":"2024-06-28T16:14:45.121348+0000","last_fullsized":"2024-06-28T16:14:45.121348+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'31","last_scrub_stamp":"2024-06-28T16:14:45.121064+0000","last_deep_scrub":"260'31","last_deep_scrub_stamp":"2024-06-28T16:14:45.121064+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.121064+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:05:38.766333+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":139,"num_read_kb":94,"num_write":78,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"68.5","version":"0'0","reported_seq":39,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:34.114903+0000","last_change":"2024-06-28T16:15:34.114956+0000","last_active":"2024-06-28T16:15:34.114903+0000","last_peered":"2024-06-28T16:15:34.114903+0000","last_clean":"2024-06-28T16:15:34.114903+0000","last_became_active":"2024-06-28T15:37:33.347075+0000","last_became_peered":"2024-06-28T15:37:33.347075+0000","last_unstale":"2024-06-28T16:15:34.114903+0000","last_undegraded":"2024-06-28T16:15:34.114903+0000","last_fullsized":"2024-06-28T16:15:34.114903+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":258,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:34.114706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:34.114706+0000","last_clean_scrub_stamp":"2024-06-28T16:15:34.114706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:14:56.120842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"67.2","version":"0'0","reported_seq":57,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:25.970748+0000","last_change":"2024-06-28T16:15:25.970806+0000","last_active":"2024-06-28T16:15:25.970748+0000","last_peered":"2024-06-28T16:15:25.970748+0000","last_clean":"2024-06-28T16:15:25.970748+0000","last_became_active":"2024-06-28T15:21:11.093082+0000","last_became_peered":"2024-06-28T15:21:11.093082+0000","last_unstale":"2024-06-28T16:15:25.970748+0000","last_undegraded":"2024-06-28T16:15:25.970748+0000","last_fullsized":"2024-06-28T16:15:25.970748+0000","mapping_epoch":253,"log_start":"0'0","ondisk_log_start":"0'0","created":253,"last_epoch_clean":254,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:25.970539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:25.970539+0000","last_clean_scrub_stamp":"2024-06-28T16:15:25.970539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:10:13.261154+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"66.3","version":"257'3","reported_seq":68,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.165567+0000","last_change":"2024-06-28T16:14:46.165567+0000","last_active":"2024-06-28T16:14:46.165567+0000","last_peered":"2024-06-28T16:14:46.165567+0000","last_clean":"2024-06-28T16:14:46.165567+0000","last_became_active":"2024-06-28T15:21:07.040575+0000","last_became_peered":"2024-06-28T15:21:07.040575+0000","last_unstale":"2024-06-28T16:14:46.165567+0000","last_undegraded":"2024-06-28T16:14:46.165567+0000","last_fullsized":"2024-06-28T16:14:46.165567+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":250,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"257'3","last_scrub_stamp":"2024-06-28T16:14:46.165127+0000","last_deep_scrub":"257'3","last_deep_scrub_stamp":"2024-06-28T16:14:46.165127+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.165127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:42:56.667095+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":5120,"num_write":8,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"65.0","version":"260'16","reported_seq":108,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.625944+0000","last_change":"2024-06-28T16:14:51.625944+0000","last_active":"2024-06-28T16:14:51.625944+0000","last_peered":"2024-06-28T16:14:51.625944+0000","last_clean":"2024-06-28T16:14:51.625944+0000","last_became_active":"2024-06-28T15:17:24.021464+0000","last_became_peered":"2024-06-28T15:17:24.021464+0000","last_unstale":"2024-06-28T16:14:51.625944+0000","last_undegraded":"2024-06-28T16:14:51.625944+0000","last_fullsized":"2024-06-28T16:14:51.625944+0000","mapping_epoch":247,"log_start":"0'0","ondisk_log_start":"0'0","created":247,"last_epoch_clean":248,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'16","last_scrub_stamp":"2024-06-28T16:14:51.625680+0000","last_deep_scrub":"260'16","last_deep_scrub_stamp":"2024-06-28T16:14:51.625680+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.625680+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:28:15.263944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":44,"num_write":37,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3f","version":"260'341","reported_seq":972,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.145629+0000","last_change":"2024-06-28T16:14:46.145629+0000","last_active":"2024-06-28T16:14:46.145629+0000","last_peered":"2024-06-28T16:14:46.145629+0000","last_clean":"2024-06-28T16:14:46.145629+0000","last_became_active":"2024-06-28T15:00:59.941681+0000","last_became_peered":"2024-06-28T15:00:59.941681+0000","last_unstale":"2024-06-28T16:14:46.145629+0000","last_undegraded":"2024-06-28T16:14:46.145629+0000","last_fullsized":"2024-06-28T16:14:46.145629+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'341","last_scrub_stamp":"2024-06-28T16:14:46.145323+0000","last_deep_scrub":"260'341","last_deep_scrub_stamp":"2024-06-28T16:14:46.145323+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.145323+0000","objects_scrubbed":6,"log_size":41,"log_dups_size":300,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:36:27.892777+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170282,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":488,"num_read_kb":26898,"num_write":1268,"num_write_kb":25985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0,5],"acting":[2,6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"260'182","reported_seq":797,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.635725+0000","last_change":"2024-06-28T16:15:02.635725+0000","last_active":"2024-06-28T16:15:02.635725+0000","last_peered":"2024-06-28T16:15:02.635725+0000","last_clean":"2024-06-28T16:15:02.635725+0000","last_became_active":"2024-06-28T15:00:17.404342+0000","last_became_peered":"2024-06-28T15:00:17.404342+0000","last_unstale":"2024-06-28T16:15:02.635725+0000","last_undegraded":"2024-06-28T16:15:02.635725+0000","last_fullsized":"2024-06-28T16:15:02.635725+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'182","last_scrub_stamp":"2024-06-28T16:15:02.635444+0000","last_deep_scrub":"260'182","last_deep_scrub_stamp":"2024-06-28T16:15:02.635444+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.635444+0000","objects_scrubbed":1,"log_size":82,"log_dups_size":100,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:22:41.865680+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":722,"num_read_kb":569,"num_write":267,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":272,"num_omap_keys":1,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"260'326","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.235841+0000","last_change":"2024-06-28T16:15:13.235841+0000","last_active":"2024-06-28T16:15:13.235841+0000","last_peered":"2024-06-28T16:15:13.235841+0000","last_clean":"2024-06-28T16:15:13.235841+0000","last_became_active":"2024-06-28T15:00:15.419449+0000","last_became_peered":"2024-06-28T15:00:15.419449+0000","last_unstale":"2024-06-28T16:15:13.235841+0000","last_undegraded":"2024-06-28T16:15:13.235841+0000","last_fullsized":"2024-06-28T16:15:13.235841+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'326","last_scrub_stamp":"2024-06-28T16:15:13.235513+0000","last_deep_scrub":"260'326","last_deep_scrub_stamp":"2024-06-28T16:15:13.235513+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.235513+0000","objects_scrubbed":10,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:42:49.114452+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5526738,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":485,"num_read_kb":15789,"num_write":1172,"num_write_kb":37304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,6,4],"acting":[1,0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.39","version":"260'287","reported_seq":951,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.751954+0000","last_change":"2024-06-28T16:15:09.751954+0000","last_active":"2024-06-28T16:15:09.751954+0000","last_peered":"2024-06-28T16:15:09.751954+0000","last_clean":"2024-06-28T16:15:09.751954+0000","last_became_active":"2024-06-28T15:00:17.408128+0000","last_became_peered":"2024-06-28T15:00:17.408128+0000","last_unstale":"2024-06-28T16:15:09.751954+0000","last_undegraded":"2024-06-28T16:15:09.751954+0000","last_fullsized":"2024-06-28T16:15:09.751954+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'287","last_scrub_stamp":"2024-06-28T16:15:09.751693+0000","last_deep_scrub":"260'287","last_deep_scrub_stamp":"2024-06-28T16:15:09.751693+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.751693+0000","objects_scrubbed":2,"log_size":87,"log_dups_size":200,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:50:27.173640+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1157,"num_read_kb":911,"num_write":439,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3314,"num_omap_keys":19,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3d","version":"260'283","reported_seq":979,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:26.213414+0000","last_change":"2024-06-28T16:15:26.213641+0000","last_active":"2024-06-28T16:15:26.213414+0000","last_peered":"2024-06-28T16:15:26.213414+0000","last_clean":"2024-06-28T16:15:26.213414+0000","last_became_active":"2024-06-28T15:00:15.396469+0000","last_became_peered":"2024-06-28T15:00:15.396469+0000","last_unstale":"2024-06-28T16:15:26.213414+0000","last_undegraded":"2024-06-28T16:15:26.213414+0000","last_fullsized":"2024-06-28T16:15:26.213414+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'283","last_scrub_stamp":"2024-06-28T16:15:26.213188+0000","last_deep_scrub":"260'283","last_deep_scrub_stamp":"2024-06-28T16:15:26.213188+0000","last_clean_scrub_stamp":"2024-06-28T16:15:26.213188+0000","objects_scrubbed":6,"log_size":83,"log_dups_size":200,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:30:45.912047+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1096214,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":504,"num_read_kb":81437,"num_write":1108,"num_write_kb":14924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,7],"acting":[0,2,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"260'78","reported_seq":654,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.007487+0000","last_change":"2024-06-28T16:15:08.007487+0000","last_active":"2024-06-28T16:15:08.007487+0000","last_peered":"2024-06-28T16:15:08.007487+0000","last_clean":"2024-06-28T16:15:08.007487+0000","last_became_active":"2024-06-28T15:00:17.402542+0000","last_became_peered":"2024-06-28T15:00:17.402542+0000","last_unstale":"2024-06-28T16:15:08.007487+0000","last_undegraded":"2024-06-28T16:15:08.007487+0000","last_fullsized":"2024-06-28T16:15:08.007487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'78","last_scrub_stamp":"2024-06-28T16:15:08.007123+0000","last_deep_scrub":"260'78","last_deep_scrub_stamp":"2024-06-28T16:15:08.007123+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.007123+0000","objects_scrubbed":1,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:43:06.313520+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":306,"num_read_kb":248,"num_write":126,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":396,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3c","version":"260'313","reported_seq":990,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.683417+0000","last_change":"2024-06-28T16:15:07.683559+0000","last_active":"2024-06-28T16:15:07.683417+0000","last_peered":"2024-06-28T16:15:07.683417+0000","last_clean":"2024-06-28T16:15:07.683417+0000","last_became_active":"2024-06-28T15:00:15.435122+0000","last_became_peered":"2024-06-28T15:00:15.435122+0000","last_unstale":"2024-06-28T16:15:07.683417+0000","last_undegraded":"2024-06-28T16:15:07.683417+0000","last_fullsized":"2024-06-28T16:15:07.683417+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'313","last_scrub_stamp":"2024-06-28T16:15:07.683192+0000","last_deep_scrub":"260'313","last_deep_scrub_stamp":"2024-06-28T16:15:07.683192+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.683192+0000","objects_scrubbed":6,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:10:01.212501+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1454317,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":492,"num_read_kb":48678,"num_write":1275,"num_write_kb":34259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7,2],"acting":[5,3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3b","version":"260'52","reported_seq":640,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.278889+0000","last_change":"2024-06-28T16:15:11.278889+0000","last_active":"2024-06-28T16:15:11.278889+0000","last_peered":"2024-06-28T16:15:11.278889+0000","last_clean":"2024-06-28T16:15:11.278889+0000","last_became_active":"2024-06-28T15:00:17.384932+0000","last_became_peered":"2024-06-28T15:00:17.384932+0000","last_unstale":"2024-06-28T16:15:11.278889+0000","last_undegraded":"2024-06-28T16:15:11.278889+0000","last_fullsized":"2024-06-28T16:15:11.278889+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'52","last_scrub_stamp":"2024-06-28T16:15:11.278605+0000","last_deep_scrub":"260'52","last_deep_scrub_stamp":"2024-06-28T16:15:11.278605+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.278605+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:12:36.542381+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":227,"num_read_kb":197,"num_write":67,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3b","version":"260'343","reported_seq":1002,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.835980+0000","last_change":"2024-06-28T16:15:15.836133+0000","last_active":"2024-06-28T16:15:15.835980+0000","last_peered":"2024-06-28T16:15:15.835980+0000","last_clean":"2024-06-28T16:15:15.835980+0000","last_became_active":"2024-06-28T15:00:15.430208+0000","last_became_peered":"2024-06-28T15:00:15.430208+0000","last_unstale":"2024-06-28T16:15:15.835980+0000","last_undegraded":"2024-06-28T16:15:15.835980+0000","last_fullsized":"2024-06-28T16:15:15.835980+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'343","last_scrub_stamp":"2024-06-28T16:15:15.835782+0000","last_deep_scrub":"260'343","last_deep_scrub_stamp":"2024-06-28T16:15:15.835782+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.835782+0000","objects_scrubbed":8,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:41:52.084536+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437780,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":548,"num_read_kb":39482,"num_write":1215,"num_write_kb":35176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3,6],"acting":[7,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3c","version":"260'186","reported_seq":848,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.448598+0000","last_change":"2024-06-28T16:14:59.448668+0000","last_active":"2024-06-28T16:14:59.448598+0000","last_peered":"2024-06-28T16:14:59.448598+0000","last_clean":"2024-06-28T16:14:59.448598+0000","last_became_active":"2024-06-28T15:00:17.397986+0000","last_became_peered":"2024-06-28T15:00:17.397986+0000","last_unstale":"2024-06-28T16:14:59.448598+0000","last_undegraded":"2024-06-28T16:14:59.448598+0000","last_fullsized":"2024-06-28T16:14:59.448598+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'186","last_scrub_stamp":"2024-06-28T16:14:59.448384+0000","last_deep_scrub":"260'186","last_deep_scrub_stamp":"2024-06-28T16:14:59.448384+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.448384+0000","objects_scrubbed":1,"log_size":86,"log_dups_size":100,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:21:42.502350+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":777,"num_read_kb":632,"num_write":274,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":473,"num_omap_keys":2,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3a","version":"260'364","reported_seq":1039,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.804713+0000","last_change":"2024-06-28T16:15:17.804834+0000","last_active":"2024-06-28T16:15:17.804713+0000","last_peered":"2024-06-28T16:15:17.804713+0000","last_clean":"2024-06-28T16:15:17.804713+0000","last_became_active":"2024-06-28T15:00:15.409542+0000","last_became_peered":"2024-06-28T15:00:15.409542+0000","last_unstale":"2024-06-28T16:15:17.804713+0000","last_undegraded":"2024-06-28T16:15:17.804713+0000","last_fullsized":"2024-06-28T16:15:17.804713+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'364","last_scrub_stamp":"2024-06-28T16:15:17.804471+0000","last_deep_scrub":"260'364","last_deep_scrub_stamp":"2024-06-28T16:15:17.804471+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.804471+0000","objects_scrubbed":7,"log_size":64,"log_dups_size":300,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:31:04.306481+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1667748,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":553,"num_read_kb":16754,"num_write":1370,"num_write_kb":32794,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4,1],"acting":[5,0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"260'44","reported_seq":618,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.888516+0000","last_change":"2024-06-28T16:14:59.888516+0000","last_active":"2024-06-28T16:14:59.888516+0000","last_peered":"2024-06-28T16:14:59.888516+0000","last_clean":"2024-06-28T16:14:59.888516+0000","last_became_active":"2024-06-28T15:00:17.401985+0000","last_became_peered":"2024-06-28T15:00:17.401985+0000","last_unstale":"2024-06-28T16:14:59.888516+0000","last_undegraded":"2024-06-28T16:14:59.888516+0000","last_fullsized":"2024-06-28T16:14:59.888516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'44","last_scrub_stamp":"2024-06-28T16:14:59.888222+0000","last_deep_scrub":"260'44","last_deep_scrub_stamp":"2024-06-28T16:14:59.888222+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.888222+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:28:28.187398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":187,"num_read_kb":159,"num_write":58,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.39","version":"260'328","reported_seq":969,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:21.854134+0000","last_change":"2024-06-28T16:15:21.854270+0000","last_active":"2024-06-28T16:15:21.854134+0000","last_peered":"2024-06-28T16:15:21.854134+0000","last_clean":"2024-06-28T16:15:21.854134+0000","last_became_active":"2024-06-28T15:00:15.424430+0000","last_became_peered":"2024-06-28T15:00:15.424430+0000","last_unstale":"2024-06-28T16:15:21.854134+0000","last_undegraded":"2024-06-28T16:15:21.854134+0000","last_fullsized":"2024-06-28T16:15:21.854134+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'328","last_scrub_stamp":"2024-06-28T16:15:21.853905+0000","last_deep_scrub":"260'328","last_deep_scrub_stamp":"2024-06-28T16:15:21.853905+0000","last_clean_scrub_stamp":"2024-06-28T16:15:21.853905+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:54:26.716654+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048602,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":476,"num_read_kb":21908,"num_write":1069,"num_write_kb":36265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4,6],"acting":[7,0,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3e","version":"260'4326","reported_seq":5006,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.551681+0000","last_change":"2024-06-28T16:14:59.551681+0000","last_active":"2024-06-28T16:14:59.551681+0000","last_peered":"2024-06-28T16:14:59.551681+0000","last_clean":"2024-06-28T16:14:59.551681+0000","last_became_active":"2024-06-28T15:00:17.384378+0000","last_became_peered":"2024-06-28T15:00:17.384378+0000","last_unstale":"2024-06-28T16:14:59.551681+0000","last_undegraded":"2024-06-28T16:14:59.551681+0000","last_fullsized":"2024-06-28T16:14:59.551681+0000","mapping_epoch":21,"log_start":"260'4300","ondisk_log_start":"260'4300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4326","last_scrub_stamp":"2024-06-28T16:14:59.551430+0000","last_deep_scrub":"260'4326","last_deep_scrub_stamp":"2024-06-28T16:14:59.551430+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.551430+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":3000,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:51:42.703162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15352,"num_read_kb":12408,"num_write":6557,"num_write_kb":3277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.38","version":"260'411","reported_seq":1138,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:32.273569+0000","last_change":"2024-06-28T16:15:32.273661+0000","last_active":"2024-06-28T16:15:32.273569+0000","last_peered":"2024-06-28T16:15:32.273569+0000","last_clean":"2024-06-28T16:15:32.273569+0000","last_became_active":"2024-06-28T15:00:15.401960+0000","last_became_peered":"2024-06-28T15:00:15.401960+0000","last_unstale":"2024-06-28T16:15:32.273569+0000","last_undegraded":"2024-06-28T16:15:32.273569+0000","last_fullsized":"2024-06-28T16:15:32.273569+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'411","last_scrub_stamp":"2024-06-28T16:15:32.273339+0000","last_deep_scrub":"260'411","last_deep_scrub_stamp":"2024-06-28T16:15:32.273339+0000","last_clean_scrub_stamp":"2024-06-28T16:15:32.273339+0000","objects_scrubbed":3,"log_size":11,"log_dups_size":400,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:22:04.556909+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324665,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":810,"num_read_kb":20615,"num_write":1466,"num_write_kb":29750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,0,2],"acting":[3,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3f","version":"260'70","reported_seq":695,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.302209+0000","last_change":"2024-06-28T16:14:59.302209+0000","last_active":"2024-06-28T16:14:59.302209+0000","last_peered":"2024-06-28T16:14:59.302209+0000","last_clean":"2024-06-28T16:14:59.302209+0000","last_became_active":"2024-06-28T15:00:17.381196+0000","last_became_peered":"2024-06-28T15:00:17.381196+0000","last_unstale":"2024-06-28T16:14:59.302209+0000","last_undegraded":"2024-06-28T16:14:59.302209+0000","last_fullsized":"2024-06-28T16:14:59.302209+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'70","last_scrub_stamp":"2024-06-28T16:14:59.301939+0000","last_deep_scrub":"260'70","last_deep_scrub_stamp":"2024-06-28T16:14:59.301939+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.301939+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:19:16.430915+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":309,"num_read_kb":273,"num_write":88,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"260'130","reported_seq":762,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.971071+0000","last_change":"2024-06-28T16:15:13.971140+0000","last_active":"2024-06-28T16:15:13.971071+0000","last_peered":"2024-06-28T16:15:13.971071+0000","last_clean":"2024-06-28T16:15:13.971071+0000","last_became_active":"2024-06-28T15:00:17.396142+0000","last_became_peered":"2024-06-28T15:00:17.396142+0000","last_unstale":"2024-06-28T16:15:13.971071+0000","last_undegraded":"2024-06-28T16:15:13.971071+0000","last_fullsized":"2024-06-28T16:15:13.971071+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'130","last_scrub_stamp":"2024-06-28T16:15:13.970878+0000","last_deep_scrub":"260'130","last_deep_scrub_stamp":"2024-06-28T16:15:13.970878+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.970878+0000","objects_scrubbed":2,"log_size":30,"log_dups_size":100,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:21:34.067289+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":536,"num_read_kb":452,"num_write":191,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":546,"num_omap_keys":3,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"260'416","reported_seq":1196,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:29.038983+0000","last_change":"2024-06-28T16:15:29.039073+0000","last_active":"2024-06-28T16:15:29.038983+0000","last_peered":"2024-06-28T16:15:29.038983+0000","last_clean":"2024-06-28T16:15:29.038983+0000","last_became_active":"2024-06-28T15:00:15.422579+0000","last_became_peered":"2024-06-28T15:00:15.422579+0000","last_unstale":"2024-06-28T16:15:29.038983+0000","last_undegraded":"2024-06-28T16:15:29.038983+0000","last_fullsized":"2024-06-28T16:15:29.038983+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'416","last_scrub_stamp":"2024-06-28T16:15:29.038792+0000","last_deep_scrub":"260'416","last_deep_scrub_stamp":"2024-06-28T16:15:29.038792+0000","last_clean_scrub_stamp":"2024-06-28T16:15:29.038792+0000","objects_scrubbed":6,"log_size":16,"log_dups_size":400,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:03:21.671624+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":851,"num_read_kb":96606,"num_write":1464,"num_write_kb":38692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,0,1],"acting":[7,5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.31","version":"260'254","reported_seq":909,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.709122+0000","last_change":"2024-06-28T16:15:07.709182+0000","last_active":"2024-06-28T16:15:07.709122+0000","last_peered":"2024-06-28T16:15:07.709122+0000","last_clean":"2024-06-28T16:15:07.709122+0000","last_became_active":"2024-06-28T15:00:17.409202+0000","last_became_peered":"2024-06-28T15:00:17.409202+0000","last_unstale":"2024-06-28T16:15:07.709122+0000","last_undegraded":"2024-06-28T16:15:07.709122+0000","last_fullsized":"2024-06-28T16:15:07.709122+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'254","last_scrub_stamp":"2024-06-28T16:15:07.708938+0000","last_deep_scrub":"260'254","last_deep_scrub_stamp":"2024-06-28T16:15:07.708938+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.708938+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":200,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:59:43.588127+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1059,"num_read_kb":851,"num_write":432,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"260'313","reported_seq":962,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.430774+0000","last_change":"2024-06-28T16:15:11.430934+0000","last_active":"2024-06-28T16:15:11.430774+0000","last_peered":"2024-06-28T16:15:11.430774+0000","last_clean":"2024-06-28T16:15:11.430774+0000","last_became_active":"2024-06-28T15:00:15.421980+0000","last_became_peered":"2024-06-28T15:00:15.421980+0000","last_unstale":"2024-06-28T16:15:11.430774+0000","last_undegraded":"2024-06-28T16:15:11.430774+0000","last_fullsized":"2024-06-28T16:15:11.430774+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'313","last_scrub_stamp":"2024-06-28T16:15:11.430569+0000","last_deep_scrub":"260'313","last_deep_scrub_stamp":"2024-06-28T16:15:11.430569+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.430569+0000","objects_scrubbed":7,"log_size":13,"log_dups_size":300,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:44:39.434883+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468838,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":425,"num_read_kb":33937,"num_write":1026,"num_write_kb":43346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2,1],"acting":[6,5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"260'226","reported_seq":861,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.587895+0000","last_change":"2024-06-28T16:15:01.587895+0000","last_active":"2024-06-28T16:15:01.587895+0000","last_peered":"2024-06-28T16:15:01.587895+0000","last_clean":"2024-06-28T16:15:01.587895+0000","last_became_active":"2024-06-28T15:00:17.384914+0000","last_became_peered":"2024-06-28T15:00:17.384914+0000","last_unstale":"2024-06-28T16:15:01.587895+0000","last_undegraded":"2024-06-28T16:15:01.587895+0000","last_fullsized":"2024-06-28T16:15:01.587895+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'226","last_scrub_stamp":"2024-06-28T16:15:01.587618+0000","last_deep_scrub":"260'226","last_deep_scrub_stamp":"2024-06-28T16:15:01.587618+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.587618+0000","objects_scrubbed":1,"log_size":26,"log_dups_size":200,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:02:48.171099+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":887,"num_read_kb":706,"num_write":342,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":720,"num_omap_keys":3,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"260'342","reported_seq":1201,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.377861+0000","last_change":"2024-06-28T16:15:08.377861+0000","last_active":"2024-06-28T16:15:08.377861+0000","last_peered":"2024-06-28T16:15:08.377861+0000","last_clean":"2024-06-28T16:15:08.377861+0000","last_became_active":"2024-06-28T15:00:15.411567+0000","last_became_peered":"2024-06-28T15:00:15.411567+0000","last_unstale":"2024-06-28T16:15:08.377861+0000","last_undegraded":"2024-06-28T16:15:08.377861+0000","last_fullsized":"2024-06-28T16:15:08.377861+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'342","last_scrub_stamp":"2024-06-28T16:15:08.377521+0000","last_deep_scrub":"260'342","last_deep_scrub_stamp":"2024-06-28T16:15:08.377521+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.377521+0000","objects_scrubbed":4,"log_size":42,"log_dups_size":300,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:33:53.677785+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":289237,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":936,"num_read_kb":14303,"num_write":1297,"num_write_kb":46873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0,3],"acting":[6,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.33","version":"260'129","reported_seq":747,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.355962+0000","last_change":"2024-06-28T16:15:00.355962+0000","last_active":"2024-06-28T16:15:00.355962+0000","last_peered":"2024-06-28T16:15:00.355962+0000","last_clean":"2024-06-28T16:15:00.355962+0000","last_became_active":"2024-06-28T15:00:17.389613+0000","last_became_peered":"2024-06-28T15:00:17.389613+0000","last_unstale":"2024-06-28T16:15:00.355962+0000","last_undegraded":"2024-06-28T16:15:00.355962+0000","last_fullsized":"2024-06-28T16:15:00.355962+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'129","last_scrub_stamp":"2024-06-28T16:15:00.355683+0000","last_deep_scrub":"260'129","last_deep_scrub_stamp":"2024-06-28T16:15:00.355683+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.355683+0000","objects_scrubbed":1,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:21:03.329874+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":512,"num_read_kb":419,"num_write":192,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":219,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"260'320","reported_seq":1082,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.926608+0000","last_change":"2024-06-28T16:15:18.926827+0000","last_active":"2024-06-28T16:15:18.926608+0000","last_peered":"2024-06-28T16:15:18.926608+0000","last_clean":"2024-06-28T16:15:18.926608+0000","last_became_active":"2024-06-28T15:00:15.417563+0000","last_became_peered":"2024-06-28T15:00:15.417563+0000","last_unstale":"2024-06-28T16:15:18.926608+0000","last_undegraded":"2024-06-28T16:15:18.926608+0000","last_fullsized":"2024-06-28T16:15:18.926608+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'320","last_scrub_stamp":"2024-06-28T16:15:18.926356+0000","last_deep_scrub":"260'320","last_deep_scrub_stamp":"2024-06-28T16:15:18.926356+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.926356+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:35:12.865205+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":525349,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":605,"num_read_kb":12860,"num_write":1159,"num_write_kb":18563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,0],"acting":[4,3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"260'157","reported_seq":748,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.360593+0000","last_change":"2024-06-28T16:15:03.360593+0000","last_active":"2024-06-28T16:15:03.360593+0000","last_peered":"2024-06-28T16:15:03.360593+0000","last_clean":"2024-06-28T16:15:03.360593+0000","last_became_active":"2024-06-28T15:00:17.413420+0000","last_became_peered":"2024-06-28T15:00:17.413420+0000","last_unstale":"2024-06-28T16:15:03.360593+0000","last_undegraded":"2024-06-28T16:15:03.360593+0000","last_fullsized":"2024-06-28T16:15:03.360593+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'157","last_scrub_stamp":"2024-06-28T16:15:03.360224+0000","last_deep_scrub":"260'157","last_deep_scrub_stamp":"2024-06-28T16:15:03.360224+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.360224+0000","objects_scrubbed":1,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:39:07.258685+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":589,"num_read_kb":466,"num_write":222,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"260'357","reported_seq":1083,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:30.329671+0000","last_change":"2024-06-28T16:15:30.329853+0000","last_active":"2024-06-28T16:15:30.329671+0000","last_peered":"2024-06-28T16:15:30.329671+0000","last_clean":"2024-06-28T16:15:30.329671+0000","last_became_active":"2024-06-28T15:00:15.401370+0000","last_became_peered":"2024-06-28T15:00:15.401370+0000","last_unstale":"2024-06-28T16:15:30.329671+0000","last_undegraded":"2024-06-28T16:15:30.329671+0000","last_fullsized":"2024-06-28T16:15:30.329671+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'357","last_scrub_stamp":"2024-06-28T16:15:30.329450+0000","last_deep_scrub":"260'357","last_deep_scrub_stamp":"2024-06-28T16:15:30.329450+0000","last_clean_scrub_stamp":"2024-06-28T16:15:30.329450+0000","objects_scrubbed":7,"log_size":57,"log_dups_size":300,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:12:52.635418+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324689,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":676,"num_read_kb":30262,"num_write":1536,"num_write_kb":36456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,1,5],"acting":[3,6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.35","version":"260'253","reported_seq":912,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.991624+0000","last_change":"2024-06-28T16:15:06.991624+0000","last_active":"2024-06-28T16:15:06.991624+0000","last_peered":"2024-06-28T16:15:06.991624+0000","last_clean":"2024-06-28T16:15:06.991624+0000","last_became_active":"2024-06-28T15:00:17.408972+0000","last_became_peered":"2024-06-28T15:00:17.408972+0000","last_unstale":"2024-06-28T16:15:06.991624+0000","last_undegraded":"2024-06-28T16:15:06.991624+0000","last_fullsized":"2024-06-28T16:15:06.991624+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'253","last_scrub_stamp":"2024-06-28T16:15:06.991213+0000","last_deep_scrub":"260'253","last_deep_scrub_stamp":"2024-06-28T16:15:06.991213+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.991213+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":200,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:27:33.581900+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":990,"num_read_kb":787,"num_write":391,"num_write_kb":195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"260'346","reported_seq":1018,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:29.276908+0000","last_change":"2024-06-28T16:15:29.277022+0000","last_active":"2024-06-28T16:15:29.276908+0000","last_peered":"2024-06-28T16:15:29.276908+0000","last_clean":"2024-06-28T16:15:29.276908+0000","last_became_active":"2024-06-28T15:00:15.407552+0000","last_became_peered":"2024-06-28T15:00:15.407552+0000","last_unstale":"2024-06-28T16:15:29.276908+0000","last_undegraded":"2024-06-28T16:15:29.276908+0000","last_fullsized":"2024-06-28T16:15:29.276908+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'346","last_scrub_stamp":"2024-06-28T16:15:29.276709+0000","last_deep_scrub":"260'346","last_deep_scrub_stamp":"2024-06-28T16:15:29.276709+0000","last_clean_scrub_stamp":"2024-06-28T16:15:29.276709+0000","objects_scrubbed":9,"log_size":46,"log_dups_size":300,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:13:17.059573+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1278750,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":513,"num_read_kb":23448,"num_write":1336,"num_write_kb":46171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1,5],"acting":[0,4,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"260'146","reported_seq":693,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.984342+0000","last_change":"2024-06-28T16:14:45.984342+0000","last_active":"2024-06-28T16:14:45.984342+0000","last_peered":"2024-06-28T16:14:45.984342+0000","last_clean":"2024-06-28T16:14:45.984342+0000","last_became_active":"2024-06-28T15:00:25.427734+0000","last_became_peered":"2024-06-28T15:00:25.427734+0000","last_unstale":"2024-06-28T16:14:45.984342+0000","last_undegraded":"2024-06-28T16:14:45.984342+0000","last_fullsized":"2024-06-28T16:14:45.984342+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'146","last_scrub_stamp":"2024-06-28T16:14:45.984045+0000","last_deep_scrub":"260'146","last_deep_scrub_stamp":"2024-06-28T16:14:45.984045+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.984045+0000","objects_scrubbed":4,"log_size":46,"log_dups_size":100,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:59:27.363330+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":233,"num_read_kb":201,"num_write":347,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"260'214","reported_seq":832,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.190384+0000","last_change":"2024-06-28T16:15:16.190448+0000","last_active":"2024-06-28T16:15:16.190384+0000","last_peered":"2024-06-28T16:15:16.190384+0000","last_clean":"2024-06-28T16:15:16.190384+0000","last_became_active":"2024-06-28T15:00:17.382344+0000","last_became_peered":"2024-06-28T15:00:17.382344+0000","last_unstale":"2024-06-28T16:15:16.190384+0000","last_undegraded":"2024-06-28T16:15:16.190384+0000","last_fullsized":"2024-06-28T16:15:16.190384+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'214","last_scrub_stamp":"2024-06-28T16:15:16.190165+0000","last_deep_scrub":"260'214","last_deep_scrub_stamp":"2024-06-28T16:15:16.190165+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.190165+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":200,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:54:00.610916+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":869,"num_read_kb":692,"num_write":351,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"260'292","reported_seq":954,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:30.071405+0000","last_change":"2024-06-28T16:15:30.071591+0000","last_active":"2024-06-28T16:15:30.071405+0000","last_peered":"2024-06-28T16:15:30.071405+0000","last_clean":"2024-06-28T16:15:30.071405+0000","last_became_active":"2024-06-28T15:00:15.420739+0000","last_became_peered":"2024-06-28T15:00:15.420739+0000","last_unstale":"2024-06-28T16:15:30.071405+0000","last_undegraded":"2024-06-28T16:15:30.071405+0000","last_fullsized":"2024-06-28T16:15:30.071405+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'292","last_scrub_stamp":"2024-06-28T16:15:30.071192+0000","last_deep_scrub":"260'292","last_deep_scrub_stamp":"2024-06-28T16:15:30.071192+0000","last_clean_scrub_stamp":"2024-06-28T16:15:30.071192+0000","objects_scrubbed":9,"log_size":92,"log_dups_size":200,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:50:59.150622+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6022775,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":452,"num_read_kb":28797,"num_write":1102,"num_write_kb":30225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,2],"acting":[7,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":512,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.665224+0000","last_change":"2024-06-28T16:15:06.665224+0000","last_active":"2024-06-28T16:15:06.665224+0000","last_peered":"2024-06-28T16:15:06.665224+0000","last_clean":"2024-06-28T16:15:06.665224+0000","last_became_active":"2024-06-28T15:00:19.400681+0000","last_became_peered":"2024-06-28T15:00:19.400681+0000","last_unstale":"2024-06-28T16:15:06.665224+0000","last_undegraded":"2024-06-28T16:15:06.665224+0000","last_fullsized":"2024-06-28T16:15:06.665224+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:06.664972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:06.664972+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.664972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:48:36.340497+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"260'328","reported_seq":962,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:20.216253+0000","last_change":"2024-06-28T16:15:20.216253+0000","last_active":"2024-06-28T16:15:20.216253+0000","last_peered":"2024-06-28T16:15:20.216253+0000","last_clean":"2024-06-28T16:15:20.216253+0000","last_became_active":"2024-06-28T15:00:21.390609+0000","last_became_peered":"2024-06-28T15:00:21.390609+0000","last_unstale":"2024-06-28T16:15:20.216253+0000","last_undegraded":"2024-06-28T16:15:20.216253+0000","last_fullsized":"2024-06-28T16:15:20.216253+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'328","last_scrub_stamp":"2024-06-28T16:15:20.215969+0000","last_deep_scrub":"260'328","last_deep_scrub_stamp":"2024-06-28T16:15:20.215969+0000","last_clean_scrub_stamp":"2024-06-28T16:15:20.215969+0000","objects_scrubbed":4,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:17:23.688350+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42630,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":530,"num_read_kb":1085,"num_write":406,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.711989+0000","last_change":"2024-06-28T16:15:03.711989+0000","last_active":"2024-06-28T16:15:03.711989+0000","last_peered":"2024-06-28T16:15:03.711989+0000","last_clean":"2024-06-28T16:15:03.711989+0000","last_became_active":"2024-06-28T15:00:23.407521+0000","last_became_peered":"2024-06-28T15:00:23.407521+0000","last_unstale":"2024-06-28T16:15:03.711989+0000","last_undegraded":"2024-06-28T16:15:03.711989+0000","last_fullsized":"2024-06-28T16:15:03.711989+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:03.711749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:03.711749+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.711749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:10:02.315582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"260'2406","reported_seq":4282,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.649004+0000","last_change":"2024-06-28T16:14:46.649004+0000","last_active":"2024-06-28T16:14:46.649004+0000","last_peered":"2024-06-28T16:14:46.649004+0000","last_clean":"2024-06-28T16:14:46.649004+0000","last_became_active":"2024-06-28T15:00:25.440083+0000","last_became_peered":"2024-06-28T15:00:25.440083+0000","last_unstale":"2024-06-28T16:14:46.649004+0000","last_undegraded":"2024-06-28T16:14:46.649004+0000","last_fullsized":"2024-06-28T16:14:46.649004+0000","mapping_epoch":29,"log_start":"260'2300","ondisk_log_start":"260'2300","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2406","last_scrub_stamp":"2024-06-28T16:14:46.648700+0000","last_deep_scrub":"260'2406","last_deep_scrub_stamp":"2024-06-28T16:14:46.648700+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.648700+0000","objects_scrubbed":4,"log_size":106,"log_dups_size":2300,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:06:59.187449+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":5966,"num_read_kb":5973,"num_write":4800,"num_write_kb":1767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":547,"num_omap_keys":3,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"260'4180","reported_seq":4834,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.506798+0000","last_change":"2024-06-28T16:14:58.506798+0000","last_active":"2024-06-28T16:14:58.506798+0000","last_peered":"2024-06-28T16:14:58.506798+0000","last_clean":"2024-06-28T16:14:58.506798+0000","last_became_active":"2024-06-28T15:00:17.390254+0000","last_became_peered":"2024-06-28T15:00:17.390254+0000","last_unstale":"2024-06-28T16:14:58.506798+0000","last_undegraded":"2024-06-28T16:14:58.506798+0000","last_fullsized":"2024-06-28T16:14:58.506798+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4180","last_scrub_stamp":"2024-06-28T16:14:58.506546+0000","last_deep_scrub":"260'4180","last_deep_scrub_stamp":"2024-06-28T16:14:58.506546+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.506546+0000","objects_scrubbed":1,"log_size":80,"log_dups_size":3000,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:17:13.473175+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":14764,"num_read_kb":11968,"num_write":6274,"num_write_kb":3129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":249,"num_omap_keys":1,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"260'343","reported_seq":1021,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:27.248179+0000","last_change":"2024-06-28T16:15:27.248334+0000","last_active":"2024-06-28T16:15:27.248179+0000","last_peered":"2024-06-28T16:15:27.248179+0000","last_clean":"2024-06-28T16:15:27.248179+0000","last_became_active":"2024-06-28T15:00:15.396724+0000","last_became_peered":"2024-06-28T15:00:15.396724+0000","last_unstale":"2024-06-28T16:15:27.248179+0000","last_undegraded":"2024-06-28T16:15:27.248179+0000","last_fullsized":"2024-06-28T16:15:27.248179+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'343","last_scrub_stamp":"2024-06-28T16:15:27.247957+0000","last_deep_scrub":"260'343","last_deep_scrub_stamp":"2024-06-28T16:15:27.247957+0000","last_clean_scrub_stamp":"2024-06-28T16:15:27.247957+0000","objects_scrubbed":6,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:47:50.870403+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262185,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":532,"num_read_kb":7737,"num_write":1235,"num_write_kb":8076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":517,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:23.373405+0000","last_change":"2024-06-28T16:15:23.373459+0000","last_active":"2024-06-28T16:15:23.373405+0000","last_peered":"2024-06-28T16:15:23.373405+0000","last_clean":"2024-06-28T16:15:23.373405+0000","last_became_active":"2024-06-28T15:00:19.390100+0000","last_became_peered":"2024-06-28T15:00:19.390100+0000","last_unstale":"2024-06-28T16:15:23.373405+0000","last_undegraded":"2024-06-28T16:15:23.373405+0000","last_fullsized":"2024-06-28T16:15:23.373405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:23.373200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:23.373200+0000","last_clean_scrub_stamp":"2024-06-28T16:15:23.373200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:31:49.559018+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"260'127","reported_seq":676,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.898682+0000","last_change":"2024-06-28T16:14:57.898682+0000","last_active":"2024-06-28T16:14:57.898682+0000","last_peered":"2024-06-28T16:14:57.898682+0000","last_clean":"2024-06-28T16:14:57.898682+0000","last_became_active":"2024-06-28T15:00:21.374284+0000","last_became_peered":"2024-06-28T15:00:21.374284+0000","last_unstale":"2024-06-28T16:14:57.898682+0000","last_undegraded":"2024-06-28T16:14:57.898682+0000","last_fullsized":"2024-06-28T16:14:57.898682+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'127","last_scrub_stamp":"2024-06-28T16:14:57.898396+0000","last_deep_scrub":"260'127","last_deep_scrub_stamp":"2024-06-28T16:14:57.898396+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.898396+0000","objects_scrubbed":4,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:35:13.361283+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":127,"num_read_kb":120,"num_write":90,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":505,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.938366+0000","last_change":"2024-06-28T16:14:50.938366+0000","last_active":"2024-06-28T16:14:50.938366+0000","last_peered":"2024-06-28T16:14:50.938366+0000","last_clean":"2024-06-28T16:14:50.938366+0000","last_became_active":"2024-06-28T15:00:23.399147+0000","last_became_peered":"2024-06-28T15:00:23.399147+0000","last_unstale":"2024-06-28T16:14:50.938366+0000","last_undegraded":"2024-06-28T16:14:50.938366+0000","last_fullsized":"2024-06-28T16:14:50.938366+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:50.938091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:50.938091+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.938091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:31:29.176572+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"260'127","reported_seq":663,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.070346+0000","last_change":"2024-06-28T16:14:51.070346+0000","last_active":"2024-06-28T16:14:51.070346+0000","last_peered":"2024-06-28T16:14:51.070346+0000","last_clean":"2024-06-28T16:14:51.070346+0000","last_became_active":"2024-06-28T15:00:25.432898+0000","last_became_peered":"2024-06-28T15:00:25.432898+0000","last_unstale":"2024-06-28T16:14:51.070346+0000","last_undegraded":"2024-06-28T16:14:51.070346+0000","last_fullsized":"2024-06-28T16:14:51.070346+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'127","last_scrub_stamp":"2024-06-28T16:14:51.070081+0000","last_deep_scrub":"260'127","last_deep_scrub_stamp":"2024-06-28T16:14:51.070081+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.070081+0000","objects_scrubbed":4,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:05:59.673050+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":183,"num_read_kb":165,"num_write":302,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"260'155","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.296338+0000","last_change":"2024-06-28T16:15:13.296338+0000","last_active":"2024-06-28T16:15:13.296338+0000","last_peered":"2024-06-28T16:15:13.296338+0000","last_clean":"2024-06-28T16:15:13.296338+0000","last_became_active":"2024-06-28T15:00:17.384613+0000","last_became_peered":"2024-06-28T15:00:17.384613+0000","last_unstale":"2024-06-28T16:15:13.296338+0000","last_undegraded":"2024-06-28T16:15:13.296338+0000","last_fullsized":"2024-06-28T16:15:13.296338+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'155","last_scrub_stamp":"2024-06-28T16:15:13.296050+0000","last_deep_scrub":"260'155","last_deep_scrub_stamp":"2024-06-28T16:15:13.296050+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.296050+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":100,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:05:33.691767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":630,"num_read_kb":508,"num_write":240,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"260'353","reported_seq":1011,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.887613+0000","last_change":"2024-06-28T16:15:17.887833+0000","last_active":"2024-06-28T16:15:17.887613+0000","last_peered":"2024-06-28T16:15:17.887613+0000","last_clean":"2024-06-28T16:15:17.887613+0000","last_became_active":"2024-06-28T15:00:15.428415+0000","last_became_peered":"2024-06-28T15:00:15.428415+0000","last_unstale":"2024-06-28T16:15:17.887613+0000","last_undegraded":"2024-06-28T16:15:17.887613+0000","last_fullsized":"2024-06-28T16:15:17.887613+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'353","last_scrub_stamp":"2024-06-28T16:15:17.887368+0000","last_deep_scrub":"260'353","last_deep_scrub_stamp":"2024-06-28T16:15:17.887368+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.887368+0000","objects_scrubbed":10,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:26:36.084273+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16055067,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":496,"num_read_kb":15197,"num_write":1261,"num_write_kb":16935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":516,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.228639+0000","last_change":"2024-06-28T16:15:19.228690+0000","last_active":"2024-06-28T16:15:19.228639+0000","last_peered":"2024-06-28T16:15:19.228639+0000","last_clean":"2024-06-28T16:15:19.228639+0000","last_became_active":"2024-06-28T15:00:19.372840+0000","last_became_peered":"2024-06-28T15:00:19.372840+0000","last_unstale":"2024-06-28T16:15:19.228639+0000","last_undegraded":"2024-06-28T16:15:19.228639+0000","last_fullsized":"2024-06-28T16:15:19.228639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:19.228437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:19.228437+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.228437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:28:55.169806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"260'128","reported_seq":701,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.103625+0000","last_change":"2024-06-28T16:14:45.103625+0000","last_active":"2024-06-28T16:14:45.103625+0000","last_peered":"2024-06-28T16:14:45.103625+0000","last_clean":"2024-06-28T16:14:45.103625+0000","last_became_active":"2024-06-28T15:00:21.377203+0000","last_became_peered":"2024-06-28T15:00:21.377203+0000","last_unstale":"2024-06-28T16:14:45.103625+0000","last_undegraded":"2024-06-28T16:14:45.103625+0000","last_fullsized":"2024-06-28T16:14:45.103625+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'128","last_scrub_stamp":"2024-06-28T16:14:45.103227+0000","last_deep_scrub":"260'128","last_deep_scrub_stamp":"2024-06-28T16:14:45.103227+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.103227+0000","objects_scrubbed":5,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:36:43.901132+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":164,"num_read_kb":159,"num_write":108,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"260'522","reported_seq":4637,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.759185+0000","last_change":"2024-06-28T16:15:02.759185+0000","last_active":"2024-06-28T16:15:02.759185+0000","last_peered":"2024-06-28T16:15:02.759185+0000","last_clean":"2024-06-28T16:15:02.759185+0000","last_became_active":"2024-06-28T15:00:23.417490+0000","last_became_peered":"2024-06-28T15:00:23.417490+0000","last_unstale":"2024-06-28T16:15:02.759185+0000","last_undegraded":"2024-06-28T16:15:02.759185+0000","last_fullsized":"2024-06-28T16:15:02.759185+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'522","last_scrub_stamp":"2024-06-28T16:15:02.758942+0000","last_deep_scrub":"260'522","last_deep_scrub_stamp":"2024-06-28T16:15:02.758942+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.758942+0000","objects_scrubbed":1,"log_size":22,"log_dups_size":500,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:33:07.422217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"260'204","reported_seq":1023,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.919351+0000","last_change":"2024-06-28T16:14:42.919351+0000","last_active":"2024-06-28T16:14:42.919351+0000","last_peered":"2024-06-28T16:14:42.919351+0000","last_clean":"2024-06-28T16:14:42.919351+0000","last_became_active":"2024-06-28T15:00:25.555670+0000","last_became_peered":"2024-06-28T15:00:25.555670+0000","last_unstale":"2024-06-28T16:14:42.919351+0000","last_undegraded":"2024-06-28T16:14:42.919351+0000","last_fullsized":"2024-06-28T16:14:42.919351+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'204","last_scrub_stamp":"2024-06-28T16:14:42.918953+0000","last_deep_scrub":"260'204","last_deep_scrub_stamp":"2024-06-28T16:14:42.918953+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.918953+0000","objects_scrubbed":3,"log_size":104,"log_dups_size":100,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:34:15.091552+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":557,"num_read_kb":555,"num_write":448,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":234,"num_omap_keys":2,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"260'84","reported_seq":681,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.159682+0000","last_change":"2024-06-28T16:15:15.159682+0000","last_active":"2024-06-28T16:15:15.159682+0000","last_peered":"2024-06-28T16:15:15.159682+0000","last_clean":"2024-06-28T16:15:15.159682+0000","last_became_active":"2024-06-28T15:00:17.389986+0000","last_became_peered":"2024-06-28T15:00:17.389986+0000","last_unstale":"2024-06-28T16:15:15.159682+0000","last_undegraded":"2024-06-28T16:15:15.159682+0000","last_fullsized":"2024-06-28T16:15:15.159682+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'84","last_scrub_stamp":"2024-06-28T16:15:15.159426+0000","last_deep_scrub":"260'84","last_deep_scrub_stamp":"2024-06-28T16:15:15.159426+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.159426+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:34:57.998551+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":369,"num_read_kb":305,"num_write":130,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"260'288","reported_seq":932,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.384197+0000","last_change":"2024-06-28T16:15:09.384387+0000","last_active":"2024-06-28T16:15:09.384197+0000","last_peered":"2024-06-28T16:15:09.384197+0000","last_clean":"2024-06-28T16:15:09.384197+0000","last_became_active":"2024-06-28T15:00:15.411380+0000","last_became_peered":"2024-06-28T15:00:15.411380+0000","last_unstale":"2024-06-28T16:15:09.384197+0000","last_undegraded":"2024-06-28T16:15:09.384197+0000","last_fullsized":"2024-06-28T16:15:09.384197+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'288","last_scrub_stamp":"2024-06-28T16:15:09.383997+0000","last_deep_scrub":"260'288","last_deep_scrub_stamp":"2024-06-28T16:15:09.383997+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.383997+0000","objects_scrubbed":7,"log_size":88,"log_dups_size":200,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:51:32.793598+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262217,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":459,"num_read_kb":24351,"num_write":1142,"num_write_kb":34218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,3],"acting":[6,0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.16","version":"260'4","reported_seq":841,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.683401+0000","last_change":"2024-06-28T16:15:10.683401+0000","last_active":"2024-06-28T16:15:10.683401+0000","last_peered":"2024-06-28T16:15:10.683401+0000","last_clean":"2024-06-28T16:15:10.683401+0000","last_became_active":"2024-06-28T15:00:19.403623+0000","last_became_peered":"2024-06-28T15:00:19.403623+0000","last_unstale":"2024-06-28T16:15:10.683401+0000","last_undegraded":"2024-06-28T16:15:10.683401+0000","last_fullsized":"2024-06-28T16:15:10.683401+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:15:10.683144+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:15:10.683144+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.683144+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:43:58.420576+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":496,"num_read_kb":494,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"260'255","reported_seq":847,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.728156+0000","last_change":"2024-06-28T16:15:05.728156+0000","last_active":"2024-06-28T16:15:05.728156+0000","last_peered":"2024-06-28T16:15:05.728156+0000","last_clean":"2024-06-28T16:15:05.728156+0000","last_became_active":"2024-06-28T15:00:21.483633+0000","last_became_peered":"2024-06-28T15:00:21.483633+0000","last_unstale":"2024-06-28T16:15:05.728156+0000","last_undegraded":"2024-06-28T16:15:05.728156+0000","last_fullsized":"2024-06-28T16:15:05.728156+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'255","last_scrub_stamp":"2024-06-28T16:15:05.727852+0000","last_deep_scrub":"260'255","last_deep_scrub_stamp":"2024-06-28T16:15:05.727852+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.727852+0000","objects_scrubbed":4,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:36:54.739144+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8831,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":400,"num_read_kb":873,"num_write":310,"num_write_kb":224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.14","version":"260'537","reported_seq":2249,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.225909+0000","last_change":"2024-06-28T16:15:03.225909+0000","last_active":"2024-06-28T16:15:03.225909+0000","last_peered":"2024-06-28T16:15:03.225909+0000","last_clean":"2024-06-28T16:15:03.225909+0000","last_became_active":"2024-06-28T15:00:23.396923+0000","last_became_peered":"2024-06-28T15:00:23.396923+0000","last_unstale":"2024-06-28T16:15:03.225909+0000","last_undegraded":"2024-06-28T16:15:03.225909+0000","last_fullsized":"2024-06-28T16:15:03.225909+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'537","last_scrub_stamp":"2024-06-28T16:15:03.225642+0000","last_deep_scrub":"260'537","last_deep_scrub_stamp":"2024-06-28T16:15:03.225642+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.225642+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:40:58.110463+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"260'415","reported_seq":1102,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.654488+0000","last_change":"2024-06-28T16:15:05.654568+0000","last_active":"2024-06-28T16:15:05.654488+0000","last_peered":"2024-06-28T16:15:05.654488+0000","last_clean":"2024-06-28T16:15:05.654488+0000","last_became_active":"2024-06-28T15:00:17.376864+0000","last_became_peered":"2024-06-28T15:00:17.376864+0000","last_unstale":"2024-06-28T16:15:05.654488+0000","last_undegraded":"2024-06-28T16:15:05.654488+0000","last_fullsized":"2024-06-28T16:15:05.654488+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'415","last_scrub_stamp":"2024-06-28T16:15:05.654289+0000","last_deep_scrub":"260'415","last_deep_scrub_stamp":"2024-06-28T16:15:05.654289+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.654289+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":400,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:26:21.223890+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1583,"num_read_kb":1214,"num_write":634,"num_write_kb":357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"260'320","reported_seq":989,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:28.034442+0000","last_change":"2024-06-28T16:15:28.034567+0000","last_active":"2024-06-28T16:15:28.034442+0000","last_peered":"2024-06-28T16:15:28.034442+0000","last_clean":"2024-06-28T16:15:28.034442+0000","last_became_active":"2024-06-28T15:00:15.415163+0000","last_became_peered":"2024-06-28T15:00:15.415163+0000","last_unstale":"2024-06-28T16:15:28.034442+0000","last_undegraded":"2024-06-28T16:15:28.034442+0000","last_fullsized":"2024-06-28T16:15:28.034442+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'320","last_scrub_stamp":"2024-06-28T16:15:28.034253+0000","last_deep_scrub":"260'320","last_deep_scrub_stamp":"2024-06-28T16:15:28.034253+0000","last_clean_scrub_stamp":"2024-06-28T16:15:28.034253+0000","objects_scrubbed":9,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:03:07.842410+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470118,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":518,"num_read_kb":26822,"num_write":1200,"num_write_kb":41274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0,1],"acting":[7,4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.707792+0000","last_change":"2024-06-28T16:15:08.707848+0000","last_active":"2024-06-28T16:15:08.707792+0000","last_peered":"2024-06-28T16:15:08.707792+0000","last_clean":"2024-06-28T16:15:08.707792+0000","last_became_active":"2024-06-28T15:00:19.389624+0000","last_became_peered":"2024-06-28T15:00:19.389624+0000","last_unstale":"2024-06-28T16:15:08.707792+0000","last_undegraded":"2024-06-28T16:15:08.707792+0000","last_fullsized":"2024-06-28T16:15:08.707792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:08.707598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:08.707598+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.707598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:30.086517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"260'390","reported_seq":1043,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.597603+0000","last_change":"2024-06-28T16:14:54.597760+0000","last_active":"2024-06-28T16:14:54.597603+0000","last_peered":"2024-06-28T16:14:54.597603+0000","last_clean":"2024-06-28T16:14:54.597603+0000","last_became_active":"2024-06-28T15:00:21.481315+0000","last_became_peered":"2024-06-28T15:00:21.481315+0000","last_unstale":"2024-06-28T16:14:54.597603+0000","last_undegraded":"2024-06-28T16:14:54.597603+0000","last_fullsized":"2024-06-28T16:14:54.597603+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'390","last_scrub_stamp":"2024-06-28T16:14:54.597422+0000","last_deep_scrub":"260'390","last_deep_scrub_stamp":"2024-06-28T16:14:54.597422+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.597422+0000","objects_scrubbed":9,"log_size":90,"log_dups_size":300,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:52:31.536722+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46076,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":638,"num_read_kb":1277,"num_write":490,"num_write_kb":298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":511,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.079204+0000","last_change":"2024-06-28T16:15:03.079255+0000","last_active":"2024-06-28T16:15:03.079204+0000","last_peered":"2024-06-28T16:15:03.079204+0000","last_clean":"2024-06-28T16:15:03.079204+0000","last_became_active":"2024-06-28T15:00:23.413132+0000","last_became_peered":"2024-06-28T15:00:23.413132+0000","last_unstale":"2024-06-28T16:15:03.079204+0000","last_undegraded":"2024-06-28T16:15:03.079204+0000","last_fullsized":"2024-06-28T16:15:03.079204+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:03.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:03.079017+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:55:30.146201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.4","version":"260'558","reported_seq":1386,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:45.685006+0000","last_change":"2024-06-28T16:14:45.685006+0000","last_active":"2024-06-28T16:14:45.685006+0000","last_peered":"2024-06-28T16:14:45.685006+0000","last_clean":"2024-06-28T16:14:45.685006+0000","last_became_active":"2024-06-28T15:00:25.441335+0000","last_became_peered":"2024-06-28T15:00:25.441335+0000","last_unstale":"2024-06-28T16:14:45.685006+0000","last_undegraded":"2024-06-28T16:14:45.685006+0000","last_fullsized":"2024-06-28T16:14:45.685006+0000","mapping_epoch":29,"log_start":"260'500","ondisk_log_start":"260'500","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'558","last_scrub_stamp":"2024-06-28T16:14:45.684713+0000","last_deep_scrub":"260'558","last_deep_scrub_stamp":"2024-06-28T16:14:45.684713+0000","last_clean_scrub_stamp":"2024-06-28T16:14:45.684713+0000","objects_scrubbed":3,"log_size":58,"log_dups_size":500,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:58:41.400605+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1277,"num_read_kb":1273,"num_write":1164,"num_write_kb":394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":381,"num_omap_keys":2,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"260'53","reported_seq":641,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.944152+0000","last_change":"2024-06-28T16:15:02.944224+0000","last_active":"2024-06-28T16:15:02.944152+0000","last_peered":"2024-06-28T16:15:02.944152+0000","last_clean":"2024-06-28T16:15:02.944152+0000","last_became_active":"2024-06-28T15:00:17.408992+0000","last_became_peered":"2024-06-28T15:00:17.408992+0000","last_unstale":"2024-06-28T16:15:02.944152+0000","last_undegraded":"2024-06-28T16:15:02.944152+0000","last_fullsized":"2024-06-28T16:15:02.944152+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'53","last_scrub_stamp":"2024-06-28T16:15:02.943961+0000","last_deep_scrub":"260'53","last_deep_scrub_stamp":"2024-06-28T16:15:02.943961+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.943961+0000","objects_scrubbed":1,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:58:16.183710+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":226,"num_read_kb":196,"num_write":79,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"260'326","reported_seq":954,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:25.988017+0000","last_change":"2024-06-28T16:15:25.988106+0000","last_active":"2024-06-28T16:15:25.988017+0000","last_peered":"2024-06-28T16:15:25.988017+0000","last_clean":"2024-06-28T16:15:25.988017+0000","last_became_active":"2024-06-28T15:00:15.402188+0000","last_became_peered":"2024-06-28T15:00:15.402188+0000","last_unstale":"2024-06-28T16:15:25.988017+0000","last_undegraded":"2024-06-28T16:15:25.988017+0000","last_fullsized":"2024-06-28T16:15:25.988017+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'326","last_scrub_stamp":"2024-06-28T16:15:25.987822+0000","last_deep_scrub":"260'326","last_deep_scrub_stamp":"2024-06-28T16:15:25.987822+0000","last_clean_scrub_stamp":"2024-06-28T16:15:25.987822+0000","objects_scrubbed":4,"log_size":26,"log_dups_size":300,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:30:56.699781+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170347,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":7037,"num_write":1119,"num_write_kb":28395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,2],"acting":[7,4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":523,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.240452+0000","last_change":"2024-06-28T16:14:55.240505+0000","last_active":"2024-06-28T16:14:55.240452+0000","last_peered":"2024-06-28T16:14:55.240452+0000","last_clean":"2024-06-28T16:14:55.240452+0000","last_became_active":"2024-06-28T15:00:19.383002+0000","last_became_peered":"2024-06-28T15:00:19.383002+0000","last_unstale":"2024-06-28T16:14:55.240452+0000","last_undegraded":"2024-06-28T16:14:55.240452+0000","last_fullsized":"2024-06-28T16:14:55.240452+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:55.240251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:55.240251+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.240251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:09:48.072213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"260'293","reported_seq":893,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:25.370055+0000","last_change":"2024-06-28T16:15:25.370136+0000","last_active":"2024-06-28T16:15:25.370055+0000","last_peered":"2024-06-28T16:15:25.370055+0000","last_clean":"2024-06-28T16:15:25.370055+0000","last_became_active":"2024-06-28T15:00:21.394580+0000","last_became_peered":"2024-06-28T15:00:21.394580+0000","last_unstale":"2024-06-28T16:15:25.370055+0000","last_undegraded":"2024-06-28T16:15:25.370055+0000","last_fullsized":"2024-06-28T16:15:25.370055+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'293","last_scrub_stamp":"2024-06-28T16:15:25.369862+0000","last_deep_scrub":"260'293","last_deep_scrub_stamp":"2024-06-28T16:15:25.369862+0000","last_clean_scrub_stamp":"2024-06-28T16:15:25.369862+0000","objects_scrubbed":5,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:45:33.914241+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":455,"num_read_kb":1052,"num_write":364,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":511,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.189470+0000","last_change":"2024-06-28T16:14:59.189522+0000","last_active":"2024-06-28T16:14:59.189470+0000","last_peered":"2024-06-28T16:14:59.189470+0000","last_clean":"2024-06-28T16:14:59.189470+0000","last_became_active":"2024-06-28T15:00:23.422567+0000","last_became_peered":"2024-06-28T15:00:23.422567+0000","last_unstale":"2024-06-28T16:14:59.189470+0000","last_undegraded":"2024-06-28T16:14:59.189470+0000","last_fullsized":"2024-06-28T16:14:59.189470+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:59.189280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:59.189280+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.189280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:25:01.077193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"260'96","reported_seq":599,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.496156+0000","last_change":"2024-06-28T16:15:00.496156+0000","last_active":"2024-06-28T16:15:00.496156+0000","last_peered":"2024-06-28T16:15:00.496156+0000","last_clean":"2024-06-28T16:15:00.496156+0000","last_became_active":"2024-06-28T15:00:25.426151+0000","last_became_peered":"2024-06-28T15:00:25.426151+0000","last_unstale":"2024-06-28T16:15:00.496156+0000","last_undegraded":"2024-06-28T16:15:00.496156+0000","last_fullsized":"2024-06-28T16:15:00.496156+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'96","last_scrub_stamp":"2024-06-28T16:15:00.495898+0000","last_deep_scrub":"260'96","last_deep_scrub_stamp":"2024-06-28T16:15:00.495898+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.495898+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:31:27.205808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":231,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"260'173","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.436343+0000","last_change":"2024-06-28T16:14:57.436511+0000","last_active":"2024-06-28T16:14:57.436343+0000","last_peered":"2024-06-28T16:14:57.436343+0000","last_clean":"2024-06-28T16:14:57.436343+0000","last_became_active":"2024-06-28T15:00:17.407031+0000","last_became_peered":"2024-06-28T15:00:17.407031+0000","last_unstale":"2024-06-28T16:14:57.436343+0000","last_undegraded":"2024-06-28T16:14:57.436343+0000","last_fullsized":"2024-06-28T16:14:57.436343+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'173","last_scrub_stamp":"2024-06-28T16:14:57.436114+0000","last_deep_scrub":"260'173","last_deep_scrub_stamp":"2024-06-28T16:14:57.436114+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.436114+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":100,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:11:15.521784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":676,"num_read_kb":538,"num_write":274,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"260'353","reported_seq":1047,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.817239+0000","last_change":"2024-06-28T16:15:19.817365+0000","last_active":"2024-06-28T16:15:19.817239+0000","last_peered":"2024-06-28T16:15:19.817239+0000","last_clean":"2024-06-28T16:15:19.817239+0000","last_became_active":"2024-06-28T15:00:15.427408+0000","last_became_peered":"2024-06-28T15:00:15.427408+0000","last_unstale":"2024-06-28T16:15:19.817239+0000","last_undegraded":"2024-06-28T16:15:19.817239+0000","last_fullsized":"2024-06-28T16:15:19.817239+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'353","last_scrub_stamp":"2024-06-28T16:15:19.817035+0000","last_deep_scrub":"260'353","last_deep_scrub_stamp":"2024-06-28T16:15:19.817035+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.817035+0000","objects_scrubbed":6,"log_size":53,"log_dups_size":300,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:12:02.971410+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":574,"num_read_kb":16939,"num_write":1376,"num_write_kb":32636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4,6],"acting":[7,5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":522,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.193279+0000","last_change":"2024-06-28T16:14:53.193339+0000","last_active":"2024-06-28T16:14:53.193279+0000","last_peered":"2024-06-28T16:14:53.193279+0000","last_clean":"2024-06-28T16:14:53.193279+0000","last_became_active":"2024-06-28T15:00:19.384172+0000","last_became_peered":"2024-06-28T15:00:19.384172+0000","last_unstale":"2024-06-28T16:14:53.193279+0000","last_undegraded":"2024-06-28T16:14:53.193279+0000","last_fullsized":"2024-06-28T16:14:53.193279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.193045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.193045+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.193045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:10:37.938602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"260'321","reported_seq":941,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.794562+0000","last_change":"2024-06-28T16:15:14.794653+0000","last_active":"2024-06-28T16:15:14.794562+0000","last_peered":"2024-06-28T16:15:14.794562+0000","last_clean":"2024-06-28T16:15:14.794562+0000","last_became_active":"2024-06-28T15:00:21.393608+0000","last_became_peered":"2024-06-28T15:00:21.393608+0000","last_unstale":"2024-06-28T16:15:14.794562+0000","last_undegraded":"2024-06-28T16:15:14.794562+0000","last_fullsized":"2024-06-28T16:15:14.794562+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'321","last_scrub_stamp":"2024-06-28T16:15:14.794352+0000","last_deep_scrub":"260'321","last_deep_scrub_stamp":"2024-06-28T16:15:14.794352+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.794352+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":300,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:23:46.118111+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42645,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":526,"num_read_kb":1122,"num_write":410,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.9","version":"260'534","reported_seq":2275,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.097241+0000","last_change":"2024-06-28T16:14:57.097387+0000","last_active":"2024-06-28T16:14:57.097241+0000","last_peered":"2024-06-28T16:14:57.097241+0000","last_clean":"2024-06-28T16:14:57.097241+0000","last_became_active":"2024-06-28T15:00:23.413550+0000","last_became_peered":"2024-06-28T15:00:23.413550+0000","last_unstale":"2024-06-28T16:14:57.097241+0000","last_undegraded":"2024-06-28T16:14:57.097241+0000","last_fullsized":"2024-06-28T16:14:57.097241+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'534","last_scrub_stamp":"2024-06-28T16:14:57.096961+0000","last_deep_scrub":"260'534","last_deep_scrub_stamp":"2024-06-28T16:14:57.096961+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.096961+0000","objects_scrubbed":1,"log_size":34,"log_dups_size":500,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:11:19.198302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"260'147","reported_seq":684,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.064684+0000","last_change":"2024-06-28T16:14:52.064763+0000","last_active":"2024-06-28T16:14:52.064684+0000","last_peered":"2024-06-28T16:14:52.064684+0000","last_clean":"2024-06-28T16:14:52.064684+0000","last_became_active":"2024-06-28T15:00:25.431835+0000","last_became_peered":"2024-06-28T15:00:25.431835+0000","last_unstale":"2024-06-28T16:14:52.064684+0000","last_undegraded":"2024-06-28T16:14:52.064684+0000","last_fullsized":"2024-06-28T16:14:52.064684+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'147","last_scrub_stamp":"2024-06-28T16:14:52.064509+0000","last_deep_scrub":"260'147","last_deep_scrub_stamp":"2024-06-28T16:14:52.064509+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.064509+0000","objects_scrubbed":3,"log_size":47,"log_dups_size":100,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:45:10.678734+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1247,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":164,"num_write":265,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"260'4158","reported_seq":4773,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.323709+0000","last_change":"2024-06-28T16:15:10.323773+0000","last_active":"2024-06-28T16:15:10.323709+0000","last_peered":"2024-06-28T16:15:10.323709+0000","last_clean":"2024-06-28T16:15:10.323709+0000","last_became_active":"2024-06-28T15:00:17.399799+0000","last_became_peered":"2024-06-28T15:00:17.399799+0000","last_unstale":"2024-06-28T16:15:10.323709+0000","last_undegraded":"2024-06-28T16:15:10.323709+0000","last_fullsized":"2024-06-28T16:15:10.323709+0000","mapping_epoch":21,"log_start":"260'4100","ondisk_log_start":"260'4100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4158","last_scrub_stamp":"2024-06-28T16:15:10.323504+0000","last_deep_scrub":"260'4158","last_deep_scrub_stamp":"2024-06-28T16:15:10.323504+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.323504+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":3000,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:49:38.330274+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14678,"num_read_kb":11882,"num_write":6254,"num_write_kb":3121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"260'337","reported_seq":991,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:21.906283+0000","last_change":"2024-06-28T16:15:21.906562+0000","last_active":"2024-06-28T16:15:21.906283+0000","last_peered":"2024-06-28T16:15:21.906283+0000","last_clean":"2024-06-28T16:15:21.906283+0000","last_became_active":"2024-06-28T15:00:15.425690+0000","last_became_peered":"2024-06-28T15:00:15.425690+0000","last_unstale":"2024-06-28T16:15:21.906283+0000","last_undegraded":"2024-06-28T16:15:21.906283+0000","last_fullsized":"2024-06-28T16:15:21.906283+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'337","last_scrub_stamp":"2024-06-28T16:15:21.906079+0000","last_deep_scrub":"260'337","last_deep_scrub_stamp":"2024-06-28T16:15:21.906079+0000","last_clean_scrub_stamp":"2024-06-28T16:15:21.906079+0000","objects_scrubbed":8,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:19.541141+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079931,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":508,"num_read_kb":13664,"num_write":1274,"num_write_kb":26574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6,1],"acting":[5,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":523,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.205322+0000","last_change":"2024-06-28T16:14:51.205380+0000","last_active":"2024-06-28T16:14:51.205322+0000","last_peered":"2024-06-28T16:14:51.205322+0000","last_clean":"2024-06-28T16:14:51.205322+0000","last_became_active":"2024-06-28T15:00:19.385189+0000","last_became_peered":"2024-06-28T15:00:19.385189+0000","last_unstale":"2024-06-28T16:14:51.205322+0000","last_undegraded":"2024-06-28T16:14:51.205322+0000","last_fullsized":"2024-06-28T16:14:51.205322+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.205122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.205122+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.205122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:14:01.011784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"260'493","reported_seq":1158,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:22.135776+0000","last_change":"2024-06-28T16:15:22.135863+0000","last_active":"2024-06-28T16:15:22.135776+0000","last_peered":"2024-06-28T16:15:22.135776+0000","last_clean":"2024-06-28T16:15:22.135776+0000","last_became_active":"2024-06-28T15:00:21.397397+0000","last_became_peered":"2024-06-28T15:00:21.397397+0000","last_unstale":"2024-06-28T16:15:22.135776+0000","last_undegraded":"2024-06-28T16:15:22.135776+0000","last_fullsized":"2024-06-28T16:15:22.135776+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'493","last_scrub_stamp":"2024-06-28T16:15:22.135585+0000","last_deep_scrub":"260'493","last_deep_scrub_stamp":"2024-06-28T16:15:22.135585+0000","last_clean_scrub_stamp":"2024-06-28T16:15:22.135585+0000","objects_scrubbed":6,"log_size":93,"log_dups_size":400,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:55:12.323828+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92449,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":856,"num_read_kb":2091,"num_write":698,"num_write_kb":574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:40.129159+0000","last_change":"2024-06-28T16:14:40.129181+0000","last_active":"2024-06-28T16:14:40.129159+0000","last_peered":"2024-06-28T16:14:40.129159+0000","last_clean":"2024-06-28T16:14:40.129159+0000","last_became_active":"2024-06-28T15:00:23.416263+0000","last_became_peered":"2024-06-28T15:00:23.416263+0000","last_unstale":"2024-06-28T16:14:40.129159+0000","last_undegraded":"2024-06-28T16:14:40.129159+0000","last_fullsized":"2024-06-28T16:14:40.129159+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:40.129070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:40.129070+0000","last_clean_scrub_stamp":"2024-06-28T16:14:40.129070+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:20:56.813386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"260'117","reported_seq":644,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.256714+0000","last_change":"2024-06-28T16:15:15.256884+0000","last_active":"2024-06-28T16:15:15.256714+0000","last_peered":"2024-06-28T16:15:15.256714+0000","last_clean":"2024-06-28T16:15:15.256714+0000","last_became_active":"2024-06-28T15:00:25.422519+0000","last_became_peered":"2024-06-28T15:00:25.422519+0000","last_unstale":"2024-06-28T16:15:15.256714+0000","last_undegraded":"2024-06-28T16:15:15.256714+0000","last_fullsized":"2024-06-28T16:15:15.256714+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'117","last_scrub_stamp":"2024-06-28T16:15:15.256473+0000","last_deep_scrub":"260'117","last_deep_scrub_stamp":"2024-06-28T16:15:15.256473+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.256473+0000","objects_scrubbed":4,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:55:59.918202+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":118,"num_read_kb":116,"num_write":248,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"260'128","reported_seq":786,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.924442+0000","last_change":"2024-06-28T16:15:00.924534+0000","last_active":"2024-06-28T16:15:00.924442+0000","last_peered":"2024-06-28T16:15:00.924442+0000","last_clean":"2024-06-28T16:15:00.924442+0000","last_became_active":"2024-06-28T15:00:17.404204+0000","last_became_peered":"2024-06-28T15:00:17.404204+0000","last_unstale":"2024-06-28T16:15:00.924442+0000","last_undegraded":"2024-06-28T16:15:00.924442+0000","last_fullsized":"2024-06-28T16:15:00.924442+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'128","last_scrub_stamp":"2024-06-28T16:15:00.924255+0000","last_deep_scrub":"260'128","last_deep_scrub_stamp":"2024-06-28T16:15:00.924255+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.924255+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":100,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:33:46.108267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":562,"num_read_kb":476,"num_write":174,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"260'352","reported_seq":1068,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:31.283604+0000","last_change":"2024-06-28T16:15:31.283702+0000","last_active":"2024-06-28T16:15:31.283604+0000","last_peered":"2024-06-28T16:15:31.283604+0000","last_clean":"2024-06-28T16:15:31.283604+0000","last_became_active":"2024-06-28T15:00:15.402041+0000","last_became_peered":"2024-06-28T15:00:15.402041+0000","last_unstale":"2024-06-28T16:15:31.283604+0000","last_undegraded":"2024-06-28T16:15:31.283604+0000","last_fullsized":"2024-06-28T16:15:31.283604+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'352","last_scrub_stamp":"2024-06-28T16:15:31.283400+0000","last_deep_scrub":"260'352","last_deep_scrub_stamp":"2024-06-28T16:15:31.283400+0000","last_clean_scrub_stamp":"2024-06-28T16:15:31.283400+0000","objects_scrubbed":7,"log_size":52,"log_dups_size":300,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:28:30.802247+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":603,"num_read_kb":26410,"num_write":1369,"num_write_kb":26391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4,7],"acting":[3,0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":522,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.209018+0000","last_change":"2024-06-28T16:14:49.209075+0000","last_active":"2024-06-28T16:14:49.209018+0000","last_peered":"2024-06-28T16:14:49.209018+0000","last_clean":"2024-06-28T16:14:49.209018+0000","last_became_active":"2024-06-28T15:00:19.392773+0000","last_became_peered":"2024-06-28T15:00:19.392773+0000","last_unstale":"2024-06-28T16:14:49.209018+0000","last_undegraded":"2024-06-28T16:14:49.209018+0000","last_fullsized":"2024-06-28T16:14:49.209018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.208819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.208819+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.208819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:49:11.047171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"260'306","reported_seq":911,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.988613+0000","last_change":"2024-06-28T16:14:53.988691+0000","last_active":"2024-06-28T16:14:53.988613+0000","last_peered":"2024-06-28T16:14:53.988613+0000","last_clean":"2024-06-28T16:14:53.988613+0000","last_became_active":"2024-06-28T15:00:21.397842+0000","last_became_peered":"2024-06-28T15:00:21.397842+0000","last_unstale":"2024-06-28T16:14:53.988613+0000","last_undegraded":"2024-06-28T16:14:53.988613+0000","last_fullsized":"2024-06-28T16:14:53.988613+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'306","last_scrub_stamp":"2024-06-28T16:14:53.988406+0000","last_deep_scrub":"260'306","last_deep_scrub_stamp":"2024-06-28T16:14:53.988406+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.988406+0000","objects_scrubbed":6,"log_size":106,"log_dups_size":200,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:16:38.325337+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44037,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":498,"num_read_kb":1063,"num_write":434,"num_write_kb":334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":15964,"num_omap_keys":38,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":512,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.186008+0000","last_change":"2024-06-28T16:15:00.186120+0000","last_active":"2024-06-28T16:15:00.186008+0000","last_peered":"2024-06-28T16:15:00.186008+0000","last_clean":"2024-06-28T16:15:00.186008+0000","last_became_active":"2024-06-28T15:00:23.406122+0000","last_became_peered":"2024-06-28T15:00:23.406122+0000","last_unstale":"2024-06-28T16:15:00.186008+0000","last_undegraded":"2024-06-28T16:15:00.186008+0000","last_fullsized":"2024-06-28T16:15:00.186008+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:00.185778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:00.185778+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.185778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:16:27.337969+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.0","version":"260'91","reported_seq":604,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.145305+0000","last_change":"2024-06-28T16:14:54.145374+0000","last_active":"2024-06-28T16:14:54.145305+0000","last_peered":"2024-06-28T16:14:54.145305+0000","last_clean":"2024-06-28T16:14:54.145305+0000","last_became_active":"2024-06-28T15:00:25.439346+0000","last_became_peered":"2024-06-28T15:00:25.439346+0000","last_unstale":"2024-06-28T16:14:54.145305+0000","last_undegraded":"2024-06-28T16:14:54.145305+0000","last_fullsized":"2024-06-28T16:14:54.145305+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'91","last_scrub_stamp":"2024-06-28T16:14:54.145105+0000","last_deep_scrub":"260'91","last_deep_scrub_stamp":"2024-06-28T16:14:54.145105+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.145105+0000","objects_scrubbed":2,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:54:30.494466+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":207,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"260'172","reported_seq":769,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:21.974854+0000","last_change":"2024-06-28T16:15:21.974919+0000","last_active":"2024-06-28T16:15:21.974854+0000","last_peered":"2024-06-28T16:15:21.974854+0000","last_clean":"2024-06-28T16:15:21.974854+0000","last_became_active":"2024-06-28T15:00:17.396048+0000","last_became_peered":"2024-06-28T15:00:17.396048+0000","last_unstale":"2024-06-28T16:15:21.974854+0000","last_undegraded":"2024-06-28T16:15:21.974854+0000","last_fullsized":"2024-06-28T16:15:21.974854+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'172","last_scrub_stamp":"2024-06-28T16:15:21.974637+0000","last_deep_scrub":"260'172","last_deep_scrub_stamp":"2024-06-28T16:15:21.974637+0000","last_clean_scrub_stamp":"2024-06-28T16:15:21.974637+0000","objects_scrubbed":1,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:11:35.817304+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":659,"num_read_kb":531,"num_write":250,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"260'376","reported_seq":1116,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.457498+0000","last_change":"2024-06-28T16:15:05.457664+0000","last_active":"2024-06-28T16:15:05.457498+0000","last_peered":"2024-06-28T16:15:05.457498+0000","last_clean":"2024-06-28T16:15:05.457498+0000","last_became_active":"2024-06-28T15:00:15.397256+0000","last_became_peered":"2024-06-28T15:00:15.397256+0000","last_unstale":"2024-06-28T16:15:05.457498+0000","last_undegraded":"2024-06-28T16:15:05.457498+0000","last_fullsized":"2024-06-28T16:15:05.457498+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'376","last_scrub_stamp":"2024-06-28T16:15:05.457272+0000","last_deep_scrub":"260'376","last_deep_scrub_stamp":"2024-06-28T16:15:05.457272+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.457272+0000","objects_scrubbed":7,"log_size":76,"log_dups_size":300,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:17:53.364489+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2674362,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":727,"num_read_kb":7345,"num_write":1214,"num_write_kb":31713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1,7],"acting":[6,4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":519,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.945272+0000","last_change":"2024-06-28T16:14:53.945393+0000","last_active":"2024-06-28T16:14:53.945272+0000","last_peered":"2024-06-28T16:14:53.945272+0000","last_clean":"2024-06-28T16:14:53.945272+0000","last_became_active":"2024-06-28T15:00:19.379281+0000","last_became_peered":"2024-06-28T15:00:19.379281+0000","last_unstale":"2024-06-28T16:14:53.945272+0000","last_undegraded":"2024-06-28T16:14:53.945272+0000","last_fullsized":"2024-06-28T16:14:53.945272+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.945027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.945027+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.945027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:26:10.510879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"260'356","reported_seq":984,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.142072+0000","last_change":"2024-06-28T16:14:43.142072+0000","last_active":"2024-06-28T16:14:43.142072+0000","last_peered":"2024-06-28T16:14:43.142072+0000","last_clean":"2024-06-28T16:14:43.142072+0000","last_became_active":"2024-06-28T15:00:21.395892+0000","last_became_peered":"2024-06-28T15:00:21.395892+0000","last_unstale":"2024-06-28T16:14:43.142072+0000","last_undegraded":"2024-06-28T16:14:43.142072+0000","last_fullsized":"2024-06-28T16:14:43.142072+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'356","last_scrub_stamp":"2024-06-28T16:14:43.141802+0000","last_deep_scrub":"260'356","last_deep_scrub_stamp":"2024-06-28T16:14:43.141802+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.141802+0000","objects_scrubbed":7,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:06:31.916429+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48931,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":579,"num_read_kb":1282,"num_write":462,"num_write_kb":330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"260'501","reported_seq":2313,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:41.168638+0000","last_change":"2024-06-28T16:14:41.168660+0000","last_active":"2024-06-28T16:14:41.168638+0000","last_peered":"2024-06-28T16:14:41.168638+0000","last_clean":"2024-06-28T16:14:41.168638+0000","last_became_active":"2024-06-28T15:00:23.415437+0000","last_became_peered":"2024-06-28T15:00:23.415437+0000","last_unstale":"2024-06-28T16:14:41.168638+0000","last_undegraded":"2024-06-28T16:14:41.168638+0000","last_fullsized":"2024-06-28T16:14:41.168638+0000","mapping_epoch":27,"log_start":"257'400","ondisk_log_start":"257'400","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'501","last_scrub_stamp":"2024-06-28T16:14:41.168573+0000","last_deep_scrub":"260'501","last_deep_scrub_stamp":"2024-06-28T16:14:41.168573+0000","last_clean_scrub_stamp":"2024-06-28T16:14:41.168573+0000","objects_scrubbed":1,"log_size":101,"log_dups_size":400,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:22:09.865296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"260'102","reported_seq":621,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.002982+0000","last_change":"2024-06-28T16:14:50.002982+0000","last_active":"2024-06-28T16:14:50.002982+0000","last_peered":"2024-06-28T16:14:50.002982+0000","last_clean":"2024-06-28T16:14:50.002982+0000","last_became_active":"2024-06-28T15:00:25.425135+0000","last_became_peered":"2024-06-28T15:00:25.425135+0000","last_unstale":"2024-06-28T16:14:50.002982+0000","last_undegraded":"2024-06-28T16:14:50.002982+0000","last_fullsized":"2024-06-28T16:14:50.002982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'102","last_scrub_stamp":"2024-06-28T16:14:50.002732+0000","last_deep_scrub":"260'102","last_deep_scrub_stamp":"2024-06-28T16:14:50.002732+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.002732+0000","objects_scrubbed":3,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:56:55.357278+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1256,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":97,"num_write":245,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"260'160","reported_seq":752,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.949730+0000","last_change":"2024-06-28T16:15:03.949795+0000","last_active":"2024-06-28T16:15:03.949730+0000","last_peered":"2024-06-28T16:15:03.949730+0000","last_clean":"2024-06-28T16:15:03.949730+0000","last_became_active":"2024-06-28T15:00:17.402024+0000","last_became_peered":"2024-06-28T15:00:17.402024+0000","last_unstale":"2024-06-28T16:15:03.949730+0000","last_undegraded":"2024-06-28T16:15:03.949730+0000","last_fullsized":"2024-06-28T16:15:03.949730+0000","mapping_epoch":21,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'160","last_scrub_stamp":"2024-06-28T16:15:03.949540+0000","last_deep_scrub":"260'160","last_deep_scrub_stamp":"2024-06-28T16:15:03.949540+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.949540+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":100,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:07:39.846023+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":645,"num_read_kb":505,"num_write":259,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"260'393","reported_seq":1076,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.969253+0000","last_change":"2024-06-28T16:15:14.969434+0000","last_active":"2024-06-28T16:15:14.969253+0000","last_peered":"2024-06-28T16:15:14.969253+0000","last_clean":"2024-06-28T16:15:14.969253+0000","last_became_active":"2024-06-28T15:00:15.388365+0000","last_became_peered":"2024-06-28T15:00:15.388365+0000","last_unstale":"2024-06-28T16:15:14.969253+0000","last_undegraded":"2024-06-28T16:15:14.969253+0000","last_fullsized":"2024-06-28T16:15:14.969253+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'393","last_scrub_stamp":"2024-06-28T16:15:14.969046+0000","last_deep_scrub":"260'393","last_deep_scrub_stamp":"2024-06-28T16:15:14.969046+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.969046+0000","objects_scrubbed":8,"log_size":93,"log_dups_size":300,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:43:46.202161+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1604260,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":661,"num_read_kb":7226,"num_write":1423,"num_write_kb":30573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7,1],"acting":[4,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"260'4","reported_seq":1166,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.680759+0000","last_change":"2024-06-28T16:15:08.680851+0000","last_active":"2024-06-28T16:15:08.680759+0000","last_peered":"2024-06-28T16:15:08.680759+0000","last_clean":"2024-06-28T16:15:08.680759+0000","last_became_active":"2024-06-28T15:00:19.397863+0000","last_became_peered":"2024-06-28T15:00:19.397863+0000","last_unstale":"2024-06-28T16:15:08.680759+0000","last_undegraded":"2024-06-28T16:15:08.680759+0000","last_fullsized":"2024-06-28T16:15:08.680759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'4","last_scrub_stamp":"2024-06-28T16:15:08.680583+0000","last_deep_scrub":"260'4","last_deep_scrub_stamp":"2024-06-28T16:15:08.680583+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.680583+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:28:22.747616+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":648,"num_read_kb":648,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"260'118","reported_seq":672,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.248332+0000","last_change":"2024-06-28T16:15:04.248438+0000","last_active":"2024-06-28T16:15:04.248332+0000","last_peered":"2024-06-28T16:15:04.248332+0000","last_clean":"2024-06-28T16:15:04.248332+0000","last_became_active":"2024-06-28T15:00:21.483679+0000","last_became_peered":"2024-06-28T15:00:21.483679+0000","last_unstale":"2024-06-28T16:15:04.248332+0000","last_undegraded":"2024-06-28T16:15:04.248332+0000","last_fullsized":"2024-06-28T16:15:04.248332+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'118","last_scrub_stamp":"2024-06-28T16:15:04.248130+0000","last_deep_scrub":"260'118","last_deep_scrub_stamp":"2024-06-28T16:15:04.248130+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.248130+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:12:41.540178+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":115,"num_write":80,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"260'515","reported_seq":4272,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.451884+0000","last_change":"2024-06-28T16:14:58.451949+0000","last_active":"2024-06-28T16:14:58.451884+0000","last_peered":"2024-06-28T16:14:58.451884+0000","last_clean":"2024-06-28T16:14:58.451884+0000","last_became_active":"2024-06-28T15:00:23.419182+0000","last_became_peered":"2024-06-28T15:00:23.419182+0000","last_unstale":"2024-06-28T16:14:58.451884+0000","last_undegraded":"2024-06-28T16:14:58.451884+0000","last_fullsized":"2024-06-28T16:14:58.451884+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'515","last_scrub_stamp":"2024-06-28T16:14:58.451684+0000","last_deep_scrub":"260'515","last_deep_scrub_stamp":"2024-06-28T16:14:58.451684+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.451684+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:15.191208+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2","version":"260'163","reported_seq":686,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.941473+0000","last_change":"2024-06-28T16:14:46.941473+0000","last_active":"2024-06-28T16:14:46.941473+0000","last_peered":"2024-06-28T16:14:46.941473+0000","last_clean":"2024-06-28T16:14:46.941473+0000","last_became_active":"2024-06-28T15:00:25.425690+0000","last_became_peered":"2024-06-28T15:00:25.425690+0000","last_unstale":"2024-06-28T16:14:46.941473+0000","last_undegraded":"2024-06-28T16:14:46.941473+0000","last_fullsized":"2024-06-28T16:14:46.941473+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'163","last_scrub_stamp":"2024-06-28T16:14:46.941194+0000","last_deep_scrub":"260'163","last_deep_scrub_stamp":"2024-06-28T16:14:46.941194+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.941194+0000","objects_scrubbed":4,"log_size":63,"log_dups_size":100,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:28:38.329545+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":170,"num_read_kb":152,"num_write":381,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"260'74","reported_seq":629,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.650256+0000","last_change":"2024-06-28T16:14:42.650256+0000","last_active":"2024-06-28T16:14:42.650256+0000","last_peered":"2024-06-28T16:14:42.650256+0000","last_clean":"2024-06-28T16:14:42.650256+0000","last_became_active":"2024-06-28T15:00:59.799823+0000","last_became_peered":"2024-06-28T15:00:59.799823+0000","last_unstale":"2024-06-28T16:14:42.650256+0000","last_undegraded":"2024-06-28T16:14:42.650256+0000","last_fullsized":"2024-06-28T16:14:42.650256+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'74","last_scrub_stamp":"2024-06-28T16:14:42.650003+0000","last_deep_scrub":"260'74","last_deep_scrub_stamp":"2024-06-28T16:14:42.650003+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.650003+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:28:32.949699+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":304,"num_read_kb":247,"num_write":103,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"260'351","reported_seq":1090,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:34.236597+0000","last_change":"2024-06-28T16:15:34.236827+0000","last_active":"2024-06-28T16:15:34.236597+0000","last_peered":"2024-06-28T16:15:34.236597+0000","last_clean":"2024-06-28T16:15:34.236597+0000","last_became_active":"2024-06-28T15:00:15.402782+0000","last_became_peered":"2024-06-28T15:00:15.402782+0000","last_unstale":"2024-06-28T16:15:34.236597+0000","last_undegraded":"2024-06-28T16:15:34.236597+0000","last_fullsized":"2024-06-28T16:15:34.236597+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'351","last_scrub_stamp":"2024-06-28T16:15:34.236400+0000","last_deep_scrub":"260'351","last_deep_scrub_stamp":"2024-06-28T16:15:34.236400+0000","last_clean_scrub_stamp":"2024-06-28T16:15:34.236400+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":300,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:18:57.699390+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262233,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":541,"num_read_kb":3603,"num_write":1164,"num_write_kb":36556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,4],"acting":[3,1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:22.954316+0000","last_change":"2024-06-28T16:15:22.954419+0000","last_active":"2024-06-28T16:15:22.954316+0000","last_peered":"2024-06-28T16:15:22.954316+0000","last_clean":"2024-06-28T16:15:22.954316+0000","last_became_active":"2024-06-28T15:00:19.405083+0000","last_became_peered":"2024-06-28T15:00:19.405083+0000","last_unstale":"2024-06-28T16:15:22.954316+0000","last_undegraded":"2024-06-28T16:15:22.954316+0000","last_fullsized":"2024-06-28T16:15:22.954316+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:22.954093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:22.954093+0000","last_clean_scrub_stamp":"2024-06-28T16:15:22.954093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:26:57.368299+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"260'437","reported_seq":1081,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.565170+0000","last_change":"2024-06-28T16:14:57.565257+0000","last_active":"2024-06-28T16:14:57.565170+0000","last_peered":"2024-06-28T16:14:57.565170+0000","last_clean":"2024-06-28T16:14:57.565170+0000","last_became_active":"2024-06-28T15:00:21.392393+0000","last_became_peered":"2024-06-28T15:00:21.392393+0000","last_unstale":"2024-06-28T16:14:57.565170+0000","last_undegraded":"2024-06-28T16:14:57.565170+0000","last_fullsized":"2024-06-28T16:14:57.565170+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'437","last_scrub_stamp":"2024-06-28T16:14:57.564983+0000","last_deep_scrub":"260'437","last_deep_scrub_stamp":"2024-06-28T16:14:57.564983+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.564983+0000","objects_scrubbed":9,"log_size":37,"log_dups_size":400,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:47:46.699794+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47467,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1282,"num_read_kb":1893,"num_write":1822,"num_write_kb":1648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":479870,"num_omap_keys":1112,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.323011+0000","last_change":"2024-06-28T16:14:54.323067+0000","last_active":"2024-06-28T16:14:54.323011+0000","last_peered":"2024-06-28T16:14:54.323011+0000","last_clean":"2024-06-28T16:14:54.323011+0000","last_became_active":"2024-06-28T15:00:23.415497+0000","last_became_peered":"2024-06-28T15:00:23.415497+0000","last_unstale":"2024-06-28T16:14:54.323011+0000","last_undegraded":"2024-06-28T16:14:54.323011+0000","last_fullsized":"2024-06-28T16:14:54.323011+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:54.322811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:54.322811+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.322811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:40:56.629420+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"260'136","reported_seq":652,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.029609+0000","last_change":"2024-06-28T16:14:48.029609+0000","last_active":"2024-06-28T16:14:48.029609+0000","last_peered":"2024-06-28T16:14:48.029609+0000","last_clean":"2024-06-28T16:14:48.029609+0000","last_became_active":"2024-06-28T15:00:25.439527+0000","last_became_peered":"2024-06-28T15:00:25.439527+0000","last_unstale":"2024-06-28T16:14:48.029609+0000","last_undegraded":"2024-06-28T16:14:48.029609+0000","last_fullsized":"2024-06-28T16:14:48.029609+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'136","last_scrub_stamp":"2024-06-28T16:14:48.029357+0000","last_deep_scrub":"260'136","last_deep_scrub_stamp":"2024-06-28T16:14:48.029357+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.029357+0000","objects_scrubbed":3,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:09:52.677657+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":127,"num_write":301,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"260'157","reported_seq":784,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.366787+0000","last_change":"2024-06-28T16:14:55.366787+0000","last_active":"2024-06-28T16:14:55.366787+0000","last_peered":"2024-06-28T16:14:55.366787+0000","last_clean":"2024-06-28T16:14:55.366787+0000","last_became_active":"2024-06-28T15:00:17.414537+0000","last_became_peered":"2024-06-28T15:00:17.414537+0000","last_unstale":"2024-06-28T16:14:55.366787+0000","last_undegraded":"2024-06-28T16:14:55.366787+0000","last_fullsized":"2024-06-28T16:14:55.366787+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'157","last_scrub_stamp":"2024-06-28T16:14:55.366527+0000","last_deep_scrub":"260'157","last_deep_scrub_stamp":"2024-06-28T16:14:55.366527+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.366527+0000","objects_scrubbed":1,"log_size":57,"log_dups_size":100,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:51:04.103193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":646,"num_read_kb":522,"num_write":223,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"260'74","reported_seq":627,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:23.942146+0000","last_change":"2024-06-28T16:15:23.942219+0000","last_active":"2024-06-28T16:15:23.942146+0000","last_peered":"2024-06-28T16:15:23.942146+0000","last_clean":"2024-06-28T16:15:23.942146+0000","last_became_active":"2024-06-28T15:00:06.302725+0000","last_became_peered":"2024-06-28T15:00:06.302725+0000","last_unstale":"2024-06-28T16:15:23.942146+0000","last_undegraded":"2024-06-28T16:15:23.942146+0000","last_fullsized":"2024-06-28T16:15:23.942146+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'74","last_scrub_stamp":"2024-06-28T16:15:23.941955+0000","last_deep_scrub":"260'74","last_deep_scrub_stamp":"2024-06-28T16:15:23.941955+0000","last_clean_scrub_stamp":"2024-06-28T16:15:23.941955+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:57:32.698515+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":534,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.800619+0000","last_change":"2024-06-28T16:15:15.800750+0000","last_active":"2024-06-28T16:15:15.800619+0000","last_peered":"2024-06-28T16:15:15.800619+0000","last_clean":"2024-06-28T16:15:15.800619+0000","last_became_active":"2024-06-28T15:00:06.284218+0000","last_became_peered":"2024-06-28T15:00:06.284218+0000","last_unstale":"2024-06-28T16:15:15.800619+0000","last_undegraded":"2024-06-28T16:15:15.800619+0000","last_fullsized":"2024-06-28T16:15:15.800619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:15.800331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:15.800331+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.800331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:32:57.118235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"260'320","reported_seq":1125,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.110204+0000","last_change":"2024-06-28T16:14:48.110204+0000","last_active":"2024-06-28T16:14:48.110204+0000","last_peered":"2024-06-28T16:14:48.110204+0000","last_clean":"2024-06-28T16:14:48.110204+0000","last_became_active":"2024-06-28T15:00:59.939352+0000","last_became_peered":"2024-06-28T15:00:59.939352+0000","last_unstale":"2024-06-28T16:14:48.110204+0000","last_undegraded":"2024-06-28T16:14:48.110204+0000","last_fullsized":"2024-06-28T16:14:48.110204+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'320","last_scrub_stamp":"2024-06-28T16:14:48.109838+0000","last_deep_scrub":"260'320","last_deep_scrub_stamp":"2024-06-28T16:14:48.109838+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.109838+0000","objects_scrubbed":10,"log_size":20,"log_dups_size":300,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:13:21.727097+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":804,"num_read_kb":59106,"num_write":1183,"num_write_kb":43047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6,4],"acting":[3,2,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":516,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.722700+0000","last_change":"2024-06-28T16:15:10.722700+0000","last_active":"2024-06-28T16:15:10.722700+0000","last_peered":"2024-06-28T16:15:10.722700+0000","last_clean":"2024-06-28T16:15:10.722700+0000","last_became_active":"2024-06-28T15:00:19.390255+0000","last_became_peered":"2024-06-28T15:00:19.390255+0000","last_unstale":"2024-06-28T16:15:10.722700+0000","last_undegraded":"2024-06-28T16:15:10.722700+0000","last_fullsized":"2024-06-28T16:15:10.722700+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:10.722439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:10.722439+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.722439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:01:41.201808+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"260'734","reported_seq":1463,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.597944+0000","last_change":"2024-06-28T16:14:55.598090+0000","last_active":"2024-06-28T16:14:55.597944+0000","last_peered":"2024-06-28T16:14:55.597944+0000","last_clean":"2024-06-28T16:14:55.597944+0000","last_became_active":"2024-06-28T15:00:21.397640+0000","last_became_peered":"2024-06-28T15:00:21.397640+0000","last_unstale":"2024-06-28T16:14:55.597944+0000","last_undegraded":"2024-06-28T16:14:55.597944+0000","last_fullsized":"2024-06-28T16:14:55.597944+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'734","last_scrub_stamp":"2024-06-28T16:14:55.597730+0000","last_deep_scrub":"260'734","last_deep_scrub_stamp":"2024-06-28T16:14:55.597730+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.597730+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":700,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:40:40.351703+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134565,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1425,"num_read_kb":3272,"num_write":1360,"num_write_kb":1190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99450,"num_omap_keys":240,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.214467+0000","last_change":"2024-06-28T16:15:03.214529+0000","last_active":"2024-06-28T16:15:03.214467+0000","last_peered":"2024-06-28T16:15:03.214467+0000","last_clean":"2024-06-28T16:15:03.214467+0000","last_became_active":"2024-06-28T15:00:23.399024+0000","last_became_peered":"2024-06-28T15:00:23.399024+0000","last_unstale":"2024-06-28T16:15:03.214467+0000","last_undegraded":"2024-06-28T16:15:03.214467+0000","last_fullsized":"2024-06-28T16:15:03.214467+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:03.214271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:03.214271+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.214271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:17:32.250338+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"260'190","reported_seq":889,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.404009+0000","last_change":"2024-06-28T16:15:12.404087+0000","last_active":"2024-06-28T16:15:12.404009+0000","last_peered":"2024-06-28T16:15:12.404009+0000","last_clean":"2024-06-28T16:15:12.404009+0000","last_became_active":"2024-06-28T15:00:25.442032+0000","last_became_peered":"2024-06-28T15:00:25.442032+0000","last_unstale":"2024-06-28T16:15:12.404009+0000","last_undegraded":"2024-06-28T16:15:12.404009+0000","last_fullsized":"2024-06-28T16:15:12.404009+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'190","last_scrub_stamp":"2024-06-28T16:15:12.403829+0000","last_deep_scrub":"260'190","last_deep_scrub_stamp":"2024-06-28T16:15:12.403829+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.403829+0000","objects_scrubbed":4,"log_size":90,"log_dups_size":100,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:09:00.851833+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1020,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":392,"num_read_kb":388,"num_write":388,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":66,"num_omap_keys":1,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"260'118","reported_seq":764,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.318413+0000","last_change":"2024-06-28T16:15:12.318476+0000","last_active":"2024-06-28T16:15:12.318413+0000","last_peered":"2024-06-28T16:15:12.318413+0000","last_clean":"2024-06-28T16:15:12.318413+0000","last_became_active":"2024-06-28T15:00:17.404200+0000","last_became_peered":"2024-06-28T15:00:17.404200+0000","last_unstale":"2024-06-28T16:15:12.318413+0000","last_undegraded":"2024-06-28T16:15:12.318413+0000","last_fullsized":"2024-06-28T16:15:12.318413+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'118","last_scrub_stamp":"2024-06-28T16:15:12.318222+0000","last_deep_scrub":"260'118","last_deep_scrub_stamp":"2024-06-28T16:15:12.318222+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.318222+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:09:16.409918+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":495,"num_read_kb":413,"num_write":159,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":535,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:22.892745+0000","last_change":"2024-06-28T16:15:22.892798+0000","last_active":"2024-06-28T16:15:22.892745+0000","last_peered":"2024-06-28T16:15:22.892745+0000","last_clean":"2024-06-28T16:15:22.892745+0000","last_became_active":"2024-06-28T15:00:06.285489+0000","last_became_peered":"2024-06-28T15:00:06.285489+0000","last_unstale":"2024-06-28T16:15:22.892745+0000","last_undegraded":"2024-06-28T16:15:22.892745+0000","last_fullsized":"2024-06-28T16:15:22.892745+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:22.892521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:22.892521+0000","last_clean_scrub_stamp":"2024-06-28T16:15:22.892521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:00:17.084929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"260'324","reported_seq":1002,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:28.289381+0000","last_change":"2024-06-28T16:15:28.289485+0000","last_active":"2024-06-28T16:15:28.289381+0000","last_peered":"2024-06-28T16:15:28.289381+0000","last_clean":"2024-06-28T16:15:28.289381+0000","last_became_active":"2024-06-28T15:00:15.403657+0000","last_became_peered":"2024-06-28T15:00:15.403657+0000","last_unstale":"2024-06-28T16:15:28.289381+0000","last_undegraded":"2024-06-28T16:15:28.289381+0000","last_fullsized":"2024-06-28T16:15:28.289381+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'324","last_scrub_stamp":"2024-06-28T16:15:28.289176+0000","last_deep_scrub":"260'324","last_deep_scrub_stamp":"2024-06-28T16:15:28.289176+0000","last_clean_scrub_stamp":"2024-06-28T16:15:28.289176+0000","objects_scrubbed":4,"log_size":24,"log_dups_size":300,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:25:17.991177+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":524,"num_read_kb":36646,"num_write":1262,"num_write_kb":16548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3,5],"acting":[0,4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.674081+0000","last_change":"2024-06-28T16:15:12.674142+0000","last_active":"2024-06-28T16:15:12.674081+0000","last_peered":"2024-06-28T16:15:12.674081+0000","last_clean":"2024-06-28T16:15:12.674081+0000","last_became_active":"2024-06-28T15:00:19.406447+0000","last_became_peered":"2024-06-28T15:00:19.406447+0000","last_unstale":"2024-06-28T16:15:12.674081+0000","last_undegraded":"2024-06-28T16:15:12.674081+0000","last_fullsized":"2024-06-28T16:15:12.674081+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:12.673872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:12.673872+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.673872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:57:02.704896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"260'1141","reported_seq":2925,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.980185+0000","last_change":"2024-06-28T16:14:56.980185+0000","last_active":"2024-06-28T16:14:56.980185+0000","last_peered":"2024-06-28T16:14:56.980185+0000","last_clean":"2024-06-28T16:14:56.980185+0000","last_became_active":"2024-06-28T15:00:21.398382+0000","last_became_peered":"2024-06-28T15:00:21.398382+0000","last_unstale":"2024-06-28T16:14:56.980185+0000","last_undegraded":"2024-06-28T16:14:56.980185+0000","last_fullsized":"2024-06-28T16:14:56.980185+0000","mapping_epoch":25,"log_start":"260'1100","ondisk_log_start":"260'1100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'1141","last_scrub_stamp":"2024-06-28T16:14:56.979906+0000","last_deep_scrub":"260'1141","last_deep_scrub_stamp":"2024-06-28T16:14:56.979906+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.979906+0000","objects_scrubbed":11,"log_size":41,"log_dups_size":1100,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:56:04.659644+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133468,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1851,"num_read_kb":3741,"num_write":1150,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":6022,"num_omap_keys":18,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.712290+0000","last_change":"2024-06-28T16:14:59.712290+0000","last_active":"2024-06-28T16:14:59.712290+0000","last_peered":"2024-06-28T16:14:59.712290+0000","last_clean":"2024-06-28T16:14:59.712290+0000","last_became_active":"2024-06-28T15:00:23.422876+0000","last_became_peered":"2024-06-28T15:00:23.422876+0000","last_unstale":"2024-06-28T16:14:59.712290+0000","last_undegraded":"2024-06-28T16:14:59.712290+0000","last_fullsized":"2024-06-28T16:14:59.712290+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:59.712038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:59.712038+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.712038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:14:40.142260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"260'120","reported_seq":642,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.019612+0000","last_change":"2024-06-28T16:14:47.019699+0000","last_active":"2024-06-28T16:14:47.019612+0000","last_peered":"2024-06-28T16:14:47.019612+0000","last_clean":"2024-06-28T16:14:47.019612+0000","last_became_active":"2024-06-28T15:00:25.553691+0000","last_became_peered":"2024-06-28T15:00:25.553691+0000","last_unstale":"2024-06-28T16:14:47.019612+0000","last_undegraded":"2024-06-28T16:14:47.019612+0000","last_fullsized":"2024-06-28T16:14:47.019612+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'120","last_scrub_stamp":"2024-06-28T16:14:47.019412+0000","last_deep_scrub":"260'120","last_deep_scrub_stamp":"2024-06-28T16:14:47.019412+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.019412+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":100,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:33:15.581216+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":862,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":126,"num_read_kb":120,"num_write":292,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"260'221","reported_seq":843,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.262910+0000","last_change":"2024-06-28T16:15:14.262978+0000","last_active":"2024-06-28T16:15:14.262910+0000","last_peered":"2024-06-28T16:15:14.262910+0000","last_clean":"2024-06-28T16:15:14.262910+0000","last_became_active":"2024-06-28T15:00:17.393361+0000","last_became_peered":"2024-06-28T15:00:17.393361+0000","last_unstale":"2024-06-28T16:15:14.262910+0000","last_undegraded":"2024-06-28T16:15:14.262910+0000","last_fullsized":"2024-06-28T16:15:14.262910+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'221","last_scrub_stamp":"2024-06-28T16:15:14.262723+0000","last_deep_scrub":"260'221","last_deep_scrub_stamp":"2024-06-28T16:15:14.262723+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.262723+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:12:41.234327+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":892,"num_read_kb":761,"num_write":358,"num_write_kb":174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":542,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.217011+0000","last_change":"2024-06-28T16:14:56.217011+0000","last_active":"2024-06-28T16:14:56.217011+0000","last_peered":"2024-06-28T16:14:56.217011+0000","last_clean":"2024-06-28T16:14:56.217011+0000","last_became_active":"2024-06-28T15:00:06.279950+0000","last_became_peered":"2024-06-28T15:00:06.279950+0000","last_unstale":"2024-06-28T16:14:56.217011+0000","last_undegraded":"2024-06-28T16:14:56.217011+0000","last_fullsized":"2024-06-28T16:14:56.217011+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.216749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.216749+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.216749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:22:36.357975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"260'334","reported_seq":1000,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.220573+0000","last_change":"2024-06-28T16:15:12.220658+0000","last_active":"2024-06-28T16:15:12.220573+0000","last_peered":"2024-06-28T16:15:12.220573+0000","last_clean":"2024-06-28T16:15:12.220573+0000","last_became_active":"2024-06-28T15:00:15.413348+0000","last_became_peered":"2024-06-28T15:00:15.413348+0000","last_unstale":"2024-06-28T16:15:12.220573+0000","last_undegraded":"2024-06-28T16:15:12.220573+0000","last_fullsized":"2024-06-28T16:15:12.220573+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'334","last_scrub_stamp":"2024-06-28T16:15:12.220392+0000","last_deep_scrub":"260'334","last_deep_scrub_stamp":"2024-06-28T16:15:12.220392+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.220392+0000","objects_scrubbed":4,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:47:12.335485+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":504,"num_read_kb":3638,"num_write":1247,"num_write_kb":36014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,5],"acting":[1,2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"260'2","reported_seq":525,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.146412+0000","last_change":"2024-06-28T16:14:47.146477+0000","last_active":"2024-06-28T16:14:47.146412+0000","last_peered":"2024-06-28T16:14:47.146412+0000","last_clean":"2024-06-28T16:14:47.146412+0000","last_became_active":"2024-06-28T15:00:19.404653+0000","last_became_peered":"2024-06-28T15:00:19.404653+0000","last_unstale":"2024-06-28T16:14:47.146412+0000","last_undegraded":"2024-06-28T16:14:47.146412+0000","last_fullsized":"2024-06-28T16:14:47.146412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:47.146212+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:47.146212+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.146212+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:46:59.182619+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"260'255","reported_seq":1710,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:31.068348+0000","last_change":"2024-06-28T16:15:31.068348+0000","last_active":"2024-06-28T16:15:31.068348+0000","last_peered":"2024-06-28T16:15:31.068348+0000","last_clean":"2024-06-28T16:15:31.068348+0000","last_became_active":"2024-06-28T15:00:21.483749+0000","last_became_peered":"2024-06-28T15:00:21.483749+0000","last_unstale":"2024-06-28T16:15:31.068348+0000","last_undegraded":"2024-06-28T16:15:31.068348+0000","last_fullsized":"2024-06-28T16:15:31.068348+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'255","last_scrub_stamp":"2024-06-28T16:15:31.068054+0000","last_deep_scrub":"260'255","last_deep_scrub_stamp":"2024-06-28T16:15:31.068054+0000","last_clean_scrub_stamp":"2024-06-28T16:15:31.068054+0000","objects_scrubbed":9,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:02:29.373520+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":232,"num_read_kb":224,"num_write":183,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5824,"num_omap_keys":14,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.250838+0000","last_change":"2024-06-28T16:15:04.250838+0000","last_active":"2024-06-28T16:15:04.250838+0000","last_peered":"2024-06-28T16:15:04.250838+0000","last_clean":"2024-06-28T16:15:04.250838+0000","last_became_active":"2024-06-28T15:00:23.399391+0000","last_became_peered":"2024-06-28T15:00:23.399391+0000","last_unstale":"2024-06-28T16:15:04.250838+0000","last_undegraded":"2024-06-28T16:15:04.250838+0000","last_fullsized":"2024-06-28T16:15:04.250838+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:04.250594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:04.250594+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.250594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:04:20.072805+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"260'242","reported_seq":857,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.112554+0000","last_change":"2024-06-28T16:14:55.112726+0000","last_active":"2024-06-28T16:14:55.112554+0000","last_peered":"2024-06-28T16:14:55.112554+0000","last_clean":"2024-06-28T16:14:55.112554+0000","last_became_active":"2024-06-28T15:00:25.419711+0000","last_became_peered":"2024-06-28T15:00:25.419711+0000","last_unstale":"2024-06-28T16:14:55.112554+0000","last_undegraded":"2024-06-28T16:14:55.112554+0000","last_fullsized":"2024-06-28T16:14:55.112554+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'242","last_scrub_stamp":"2024-06-28T16:14:55.112334+0000","last_deep_scrub":"260'242","last_deep_scrub_stamp":"2024-06-28T16:14:55.112334+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.112334+0000","objects_scrubbed":3,"log_size":42,"log_dups_size":200,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:15:19.328332+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":416,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":467,"num_read_kb":491,"num_write":517,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1193,"num_omap_keys":7,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"260'316","reported_seq":1017,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.125767+0000","last_change":"2024-06-28T16:15:04.125767+0000","last_active":"2024-06-28T16:15:04.125767+0000","last_peered":"2024-06-28T16:15:04.125767+0000","last_clean":"2024-06-28T16:15:04.125767+0000","last_became_active":"2024-06-28T15:00:17.393696+0000","last_became_peered":"2024-06-28T15:00:17.393696+0000","last_unstale":"2024-06-28T16:15:04.125767+0000","last_undegraded":"2024-06-28T16:15:04.125767+0000","last_fullsized":"2024-06-28T16:15:04.125767+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'316","last_scrub_stamp":"2024-06-28T16:15:04.125411+0000","last_deep_scrub":"260'316","last_deep_scrub_stamp":"2024-06-28T16:15:04.125411+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.125411+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":300,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:39:24.006191+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1365,"num_read_kb":1154,"num_write":468,"num_write_kb":275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25681,"num_omap_keys":101,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":534,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.751576+0000","last_change":"2024-06-28T16:15:14.751632+0000","last_active":"2024-06-28T16:15:14.751576+0000","last_peered":"2024-06-28T16:15:14.751576+0000","last_clean":"2024-06-28T16:15:14.751576+0000","last_became_active":"2024-06-28T15:00:06.283106+0000","last_became_peered":"2024-06-28T15:00:06.283106+0000","last_unstale":"2024-06-28T16:15:14.751576+0000","last_undegraded":"2024-06-28T16:15:14.751576+0000","last_fullsized":"2024-06-28T16:15:14.751576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:14.751373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:14.751373+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.751373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:17:53.348364+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"260'318","reported_seq":1001,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.927891+0000","last_change":"2024-06-28T16:15:17.928148+0000","last_active":"2024-06-28T16:15:17.927891+0000","last_peered":"2024-06-28T16:15:17.927891+0000","last_clean":"2024-06-28T16:15:17.927891+0000","last_became_active":"2024-06-28T15:00:15.413880+0000","last_became_peered":"2024-06-28T15:00:15.413880+0000","last_unstale":"2024-06-28T16:15:17.927891+0000","last_undegraded":"2024-06-28T16:15:17.927891+0000","last_fullsized":"2024-06-28T16:15:17.927891+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'318","last_scrub_stamp":"2024-06-28T16:15:17.927669+0000","last_deep_scrub":"260'318","last_deep_scrub_stamp":"2024-06-28T16:15:17.927669+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.927669+0000","objects_scrubbed":7,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:19:11.398970+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2421978,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":465,"num_read_kb":85416,"num_write":1148,"num_write_kb":27167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,6],"acting":[4,5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":523,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.295510+0000","last_change":"2024-06-28T16:15:14.295510+0000","last_active":"2024-06-28T16:15:14.295510+0000","last_peered":"2024-06-28T16:15:14.295510+0000","last_clean":"2024-06-28T16:15:14.295510+0000","last_became_active":"2024-06-28T15:00:19.382486+0000","last_became_peered":"2024-06-28T15:00:19.382486+0000","last_unstale":"2024-06-28T16:15:14.295510+0000","last_undegraded":"2024-06-28T16:15:14.295510+0000","last_fullsized":"2024-06-28T16:15:14.295510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:14.295262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:14.295262+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.295262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:15:46.049931+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"260'166","reported_seq":745,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.148880+0000","last_change":"2024-06-28T16:15:10.148966+0000","last_active":"2024-06-28T16:15:10.148880+0000","last_peered":"2024-06-28T16:15:10.148880+0000","last_clean":"2024-06-28T16:15:10.148880+0000","last_became_active":"2024-06-28T15:00:21.400535+0000","last_became_peered":"2024-06-28T15:00:21.400535+0000","last_unstale":"2024-06-28T16:15:10.148880+0000","last_undegraded":"2024-06-28T16:15:10.148880+0000","last_fullsized":"2024-06-28T16:15:10.148880+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'166","last_scrub_stamp":"2024-06-28T16:15:10.148680+0000","last_deep_scrub":"260'166","last_deep_scrub_stamp":"2024-06-28T16:15:10.148680+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.148680+0000","objects_scrubbed":7,"log_size":66,"log_dups_size":100,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:40:13.912505+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":198,"num_read_kb":191,"num_write":142,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1510,"num_omap_keys":4,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.407107+0000","last_change":"2024-06-28T16:14:56.407161+0000","last_active":"2024-06-28T16:14:56.407107+0000","last_peered":"2024-06-28T16:14:56.407107+0000","last_clean":"2024-06-28T16:14:56.407107+0000","last_became_active":"2024-06-28T15:00:23.416701+0000","last_became_peered":"2024-06-28T15:00:23.416701+0000","last_unstale":"2024-06-28T16:14:56.407107+0000","last_undegraded":"2024-06-28T16:14:56.407107+0000","last_fullsized":"2024-06-28T16:14:56.407107+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.406907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.406907+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.406907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:09:09.502206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.8","version":"260'117","reported_seq":642,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.620180+0000","last_change":"2024-06-28T16:14:47.620245+0000","last_active":"2024-06-28T16:14:47.620180+0000","last_peered":"2024-06-28T16:14:47.620180+0000","last_clean":"2024-06-28T16:14:47.620180+0000","last_became_active":"2024-06-28T15:00:25.438523+0000","last_became_peered":"2024-06-28T15:00:25.438523+0000","last_unstale":"2024-06-28T16:14:47.620180+0000","last_undegraded":"2024-06-28T16:14:47.620180+0000","last_fullsized":"2024-06-28T16:14:47.620180+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'117","last_scrub_stamp":"2024-06-28T16:14:47.619989+0000","last_deep_scrub":"260'117","last_deep_scrub_stamp":"2024-06-28T16:14:47.619989+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.619989+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:25:16.904136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":129,"num_write":276,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"260'169","reported_seq":784,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.041377+0000","last_change":"2024-06-28T16:15:02.041377+0000","last_active":"2024-06-28T16:15:02.041377+0000","last_peered":"2024-06-28T16:15:02.041377+0000","last_clean":"2024-06-28T16:15:02.041377+0000","last_became_active":"2024-06-28T15:00:17.398515+0000","last_became_peered":"2024-06-28T15:00:17.398515+0000","last_unstale":"2024-06-28T16:15:02.041377+0000","last_undegraded":"2024-06-28T16:15:02.041377+0000","last_fullsized":"2024-06-28T16:15:02.041377+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'169","last_scrub_stamp":"2024-06-28T16:15:02.041106+0000","last_deep_scrub":"260'169","last_deep_scrub_stamp":"2024-06-28T16:15:02.041106+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.041106+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":100,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:44:07.334141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":644,"num_read_kb":512,"num_write":273,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"260'372","reported_seq":1053,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.858841+0000","last_change":"2024-06-28T16:15:18.858841+0000","last_active":"2024-06-28T16:15:18.858841+0000","last_peered":"2024-06-28T16:15:18.858841+0000","last_clean":"2024-06-28T16:15:18.858841+0000","last_became_active":"2024-06-28T15:00:15.414704+0000","last_became_peered":"2024-06-28T15:00:15.414704+0000","last_unstale":"2024-06-28T16:15:18.858841+0000","last_undegraded":"2024-06-28T16:15:18.858841+0000","last_fullsized":"2024-06-28T16:15:18.858841+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'372","last_scrub_stamp":"2024-06-28T16:15:18.858489+0000","last_deep_scrub":"260'372","last_deep_scrub_stamp":"2024-06-28T16:15:18.858489+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.858489+0000","objects_scrubbed":4,"log_size":72,"log_dups_size":300,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:00:26.901636+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10506361,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":617,"num_read_kb":33619,"num_write":1289,"num_write_kb":23520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":539,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.163070+0000","last_change":"2024-06-28T16:15:08.163070+0000","last_active":"2024-06-28T16:15:08.163070+0000","last_peered":"2024-06-28T16:15:08.163070+0000","last_clean":"2024-06-28T16:15:08.163070+0000","last_became_active":"2024-06-28T15:00:06.302449+0000","last_became_peered":"2024-06-28T15:00:06.302449+0000","last_unstale":"2024-06-28T16:15:08.163070+0000","last_undegraded":"2024-06-28T16:15:08.163070+0000","last_fullsized":"2024-06-28T16:15:08.163070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:08.162767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:08.162767+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.162767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:45:36.489995+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":523,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.260730+0000","last_change":"2024-06-28T16:15:16.260730+0000","last_active":"2024-06-28T16:15:16.260730+0000","last_peered":"2024-06-28T16:15:16.260730+0000","last_clean":"2024-06-28T16:15:16.260730+0000","last_became_active":"2024-06-28T15:00:19.384933+0000","last_became_peered":"2024-06-28T15:00:19.384933+0000","last_unstale":"2024-06-28T16:15:16.260730+0000","last_undegraded":"2024-06-28T16:15:16.260730+0000","last_fullsized":"2024-06-28T16:15:16.260730+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:16.260473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:16.260473+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.260473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:45:58.084956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"260'597","reported_seq":1305,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.242218+0000","last_change":"2024-06-28T16:15:05.242315+0000","last_active":"2024-06-28T16:15:05.242218+0000","last_peered":"2024-06-28T16:15:05.242218+0000","last_clean":"2024-06-28T16:15:05.242218+0000","last_became_active":"2024-06-28T15:00:21.405762+0000","last_became_peered":"2024-06-28T15:00:21.405762+0000","last_unstale":"2024-06-28T16:15:05.242218+0000","last_undegraded":"2024-06-28T16:15:05.242218+0000","last_fullsized":"2024-06-28T16:15:05.242218+0000","mapping_epoch":25,"log_start":"260'500","ondisk_log_start":"260'500","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'597","last_scrub_stamp":"2024-06-28T16:15:05.241998+0000","last_deep_scrub":"260'597","last_deep_scrub_stamp":"2024-06-28T16:15:05.241998+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.241998+0000","objects_scrubbed":10,"log_size":97,"log_dups_size":500,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:43:06.700587+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92557,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1001,"num_read_kb":2235,"num_write":838,"num_write_kb":641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":8196,"num_omap_keys":24,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.927496+0000","last_change":"2024-06-28T16:14:49.927496+0000","last_active":"2024-06-28T16:14:49.927496+0000","last_peered":"2024-06-28T16:14:49.927496+0000","last_clean":"2024-06-28T16:14:49.927496+0000","last_became_active":"2024-06-28T15:00:23.404714+0000","last_became_peered":"2024-06-28T15:00:23.404714+0000","last_unstale":"2024-06-28T16:14:49.927496+0000","last_undegraded":"2024-06-28T16:14:49.927496+0000","last_fullsized":"2024-06-28T16:14:49.927496+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.927238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.927238+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.927238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:46:09.338954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"260'154","reported_seq":680,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.151138+0000","last_change":"2024-06-28T16:14:48.151312+0000","last_active":"2024-06-28T16:14:48.151138+0000","last_peered":"2024-06-28T16:14:48.151138+0000","last_clean":"2024-06-28T16:14:48.151138+0000","last_became_active":"2024-06-28T15:00:25.556834+0000","last_became_peered":"2024-06-28T15:00:25.556834+0000","last_unstale":"2024-06-28T16:14:48.151138+0000","last_undegraded":"2024-06-28T16:14:48.151138+0000","last_fullsized":"2024-06-28T16:14:48.151138+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'154","last_scrub_stamp":"2024-06-28T16:14:48.150905+0000","last_deep_scrub":"260'154","last_deep_scrub_stamp":"2024-06-28T16:14:48.150905+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.150905+0000","objects_scrubbed":7,"log_size":54,"log_dups_size":100,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:54:08.173965+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1183,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":133,"num_read_kb":127,"num_write":342,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":73,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"260'139","reported_seq":778,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.225074+0000","last_change":"2024-06-28T16:15:19.225074+0000","last_active":"2024-06-28T16:15:19.225074+0000","last_peered":"2024-06-28T16:15:19.225074+0000","last_clean":"2024-06-28T16:15:19.225074+0000","last_became_active":"2024-06-28T15:00:17.372401+0000","last_became_peered":"2024-06-28T15:00:17.372401+0000","last_unstale":"2024-06-28T16:15:19.225074+0000","last_undegraded":"2024-06-28T16:15:19.225074+0000","last_fullsized":"2024-06-28T16:15:19.225074+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'139","last_scrub_stamp":"2024-06-28T16:15:19.224695+0000","last_deep_scrub":"260'139","last_deep_scrub_stamp":"2024-06-28T16:15:19.224695+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.224695+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":100,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:34:36.503538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":577,"num_read_kb":470,"num_write":221,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"260'273","reported_seq":911,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.187159+0000","last_change":"2024-06-28T16:15:11.187159+0000","last_active":"2024-06-28T16:15:11.187159+0000","last_peered":"2024-06-28T16:15:11.187159+0000","last_clean":"2024-06-28T16:15:11.187159+0000","last_became_active":"2024-06-28T15:00:15.408140+0000","last_became_peered":"2024-06-28T15:00:15.408140+0000","last_unstale":"2024-06-28T16:15:11.187159+0000","last_undegraded":"2024-06-28T16:15:11.187159+0000","last_fullsized":"2024-06-28T16:15:11.187159+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'273","last_scrub_stamp":"2024-06-28T16:15:11.186833+0000","last_deep_scrub":"260'273","last_deep_scrub_stamp":"2024-06-28T16:15:11.186833+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.186833+0000","objects_scrubbed":10,"log_size":73,"log_dups_size":200,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:13:01.399899+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":526399,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":387,"num_read_kb":8190,"num_write":1081,"num_write_kb":24156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,5],"acting":[1,2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":534,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:24.956397+0000","last_change":"2024-06-28T16:15:24.956397+0000","last_active":"2024-06-28T16:15:24.956397+0000","last_peered":"2024-06-28T16:15:24.956397+0000","last_clean":"2024-06-28T16:15:24.956397+0000","last_became_active":"2024-06-28T15:00:06.302962+0000","last_became_peered":"2024-06-28T15:00:06.302962+0000","last_unstale":"2024-06-28T16:15:24.956397+0000","last_undegraded":"2024-06-28T16:15:24.956397+0000","last_fullsized":"2024-06-28T16:15:24.956397+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:24.956071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:24.956071+0000","last_clean_scrub_stamp":"2024-06-28T16:15:24.956071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:16:18.714248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.184441+0000","last_change":"2024-06-28T16:14:49.184441+0000","last_active":"2024-06-28T16:14:49.184441+0000","last_peered":"2024-06-28T16:14:49.184441+0000","last_clean":"2024-06-28T16:14:49.184441+0000","last_became_active":"2024-06-28T15:00:19.384961+0000","last_became_peered":"2024-06-28T15:00:19.384961+0000","last_unstale":"2024-06-28T16:14:49.184441+0000","last_undegraded":"2024-06-28T16:14:49.184441+0000","last_fullsized":"2024-06-28T16:14:49.184441+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:49.184197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:49.184197+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.184197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:48:38.391991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"260'195","reported_seq":802,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.962600+0000","last_change":"2024-06-28T16:14:58.962600+0000","last_active":"2024-06-28T16:14:58.962600+0000","last_peered":"2024-06-28T16:14:58.962600+0000","last_clean":"2024-06-28T16:14:58.962600+0000","last_became_active":"2024-06-28T15:00:21.480107+0000","last_became_peered":"2024-06-28T15:00:21.480107+0000","last_unstale":"2024-06-28T16:14:58.962600+0000","last_undegraded":"2024-06-28T16:14:58.962600+0000","last_fullsized":"2024-06-28T16:14:58.962600+0000","mapping_epoch":25,"log_start":"257'100","ondisk_log_start":"257'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'195","last_scrub_stamp":"2024-06-28T16:14:58.962331+0000","last_deep_scrub":"260'195","last_deep_scrub_stamp":"2024-06-28T16:14:58.962331+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.962331+0000","objects_scrubbed":7,"log_size":95,"log_dups_size":100,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:08:23.304315+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":263,"num_read_kb":253,"num_write":174,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"260'565","reported_seq":2887,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.666227+0000","last_change":"2024-06-28T16:14:51.666291+0000","last_active":"2024-06-28T16:14:51.666227+0000","last_peered":"2024-06-28T16:14:51.666227+0000","last_clean":"2024-06-28T16:14:51.666227+0000","last_became_active":"2024-06-28T15:00:23.427565+0000","last_became_peered":"2024-06-28T15:00:23.427565+0000","last_unstale":"2024-06-28T16:14:51.666227+0000","last_undegraded":"2024-06-28T16:14:51.666227+0000","last_fullsized":"2024-06-28T16:14:51.666227+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'565","last_scrub_stamp":"2024-06-28T16:14:51.666022+0000","last_deep_scrub":"260'565","last_deep_scrub_stamp":"2024-06-28T16:14:51.666022+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.666022+0000","objects_scrubbed":1,"log_size":65,"log_dups_size":500,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:06:09.388341+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"260'127","reported_seq":651,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.139109+0000","last_change":"2024-06-28T16:14:51.139189+0000","last_active":"2024-06-28T16:14:51.139109+0000","last_peered":"2024-06-28T16:14:51.139109+0000","last_clean":"2024-06-28T16:14:51.139109+0000","last_became_active":"2024-06-28T15:00:25.553627+0000","last_became_peered":"2024-06-28T15:00:25.553627+0000","last_unstale":"2024-06-28T16:14:51.139109+0000","last_undegraded":"2024-06-28T16:14:51.139109+0000","last_fullsized":"2024-06-28T16:14:51.139109+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'127","last_scrub_stamp":"2024-06-28T16:14:51.138914+0000","last_deep_scrub":"260'127","last_deep_scrub_stamp":"2024-06-28T16:14:51.138914+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.138914+0000","objects_scrubbed":3,"log_size":27,"log_dups_size":100,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:08:31.042999+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":713,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":165,"num_read_kb":148,"num_write":308,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"260'172","reported_seq":834,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.189062+0000","last_change":"2024-06-28T16:15:18.189235+0000","last_active":"2024-06-28T16:15:18.189062+0000","last_peered":"2024-06-28T16:15:18.189062+0000","last_clean":"2024-06-28T16:15:18.189062+0000","last_became_active":"2024-06-28T15:00:17.376516+0000","last_became_peered":"2024-06-28T15:00:17.376516+0000","last_unstale":"2024-06-28T16:15:18.189062+0000","last_undegraded":"2024-06-28T16:15:18.189062+0000","last_fullsized":"2024-06-28T16:15:18.189062+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'172","last_scrub_stamp":"2024-06-28T16:15:18.188830+0000","last_deep_scrub":"260'172","last_deep_scrub_stamp":"2024-06-28T16:15:18.188830+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.188830+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":100,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:08:43.423312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":722,"num_read_kb":588,"num_write":244,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"260'336","reported_seq":989,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:33.259899+0000","last_change":"2024-06-28T16:15:33.259994+0000","last_active":"2024-06-28T16:15:33.259899+0000","last_peered":"2024-06-28T16:15:33.259899+0000","last_clean":"2024-06-28T16:15:33.259899+0000","last_became_active":"2024-06-28T15:00:15.403032+0000","last_became_peered":"2024-06-28T15:00:15.403032+0000","last_unstale":"2024-06-28T16:15:33.259899+0000","last_undegraded":"2024-06-28T16:15:33.259899+0000","last_fullsized":"2024-06-28T16:15:33.259899+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'336","last_scrub_stamp":"2024-06-28T16:15:33.259679+0000","last_deep_scrub":"260'336","last_deep_scrub_stamp":"2024-06-28T16:15:33.259679+0000","last_clean_scrub_stamp":"2024-06-28T16:15:33.259679+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:44:24.072948+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":968026,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":483,"num_read_kb":17032,"num_write":1147,"num_write_kb":35885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,0],"acting":[3,5,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":538,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.172536+0000","last_change":"2024-06-28T16:15:07.172536+0000","last_active":"2024-06-28T16:15:07.172536+0000","last_peered":"2024-06-28T16:15:07.172536+0000","last_clean":"2024-06-28T16:15:07.172536+0000","last_became_active":"2024-06-28T15:00:06.278443+0000","last_became_peered":"2024-06-28T15:00:06.278443+0000","last_unstale":"2024-06-28T16:15:07.172536+0000","last_undegraded":"2024-06-28T16:15:07.172536+0000","last_fullsized":"2024-06-28T16:15:07.172536+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:07.172293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:07.172293+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.172293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:30:00.544437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":518,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.999237+0000","last_change":"2024-06-28T16:15:08.999237+0000","last_active":"2024-06-28T16:15:08.999237+0000","last_peered":"2024-06-28T16:15:08.999237+0000","last_clean":"2024-06-28T16:15:08.999237+0000","last_became_active":"2024-06-28T15:00:19.378932+0000","last_became_peered":"2024-06-28T15:00:19.378932+0000","last_unstale":"2024-06-28T16:15:08.999237+0000","last_undegraded":"2024-06-28T16:15:08.999237+0000","last_fullsized":"2024-06-28T16:15:08.999237+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:08.998981+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:08.998981+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.998981+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:59:56.713595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"260'143","reported_seq":713,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.986950+0000","last_change":"2024-06-28T16:14:54.986950+0000","last_active":"2024-06-28T16:14:54.986950+0000","last_peered":"2024-06-28T16:14:54.986950+0000","last_clean":"2024-06-28T16:14:54.986950+0000","last_became_active":"2024-06-28T15:00:21.395462+0000","last_became_peered":"2024-06-28T15:00:21.395462+0000","last_unstale":"2024-06-28T16:14:54.986950+0000","last_undegraded":"2024-06-28T16:14:54.986950+0000","last_fullsized":"2024-06-28T16:14:54.986950+0000","mapping_epoch":25,"log_start":"260'100","ondisk_log_start":"260'100","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'143","last_scrub_stamp":"2024-06-28T16:14:54.986685+0000","last_deep_scrub":"260'143","last_deep_scrub_stamp":"2024-06-28T16:14:54.986685+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.986685+0000","objects_scrubbed":5,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:03:28.897014+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":167,"num_read_kb":161,"num_write":112,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"260'537","reported_seq":2591,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.176689+0000","last_change":"2024-06-28T16:14:58.176689+0000","last_active":"2024-06-28T16:14:58.176689+0000","last_peered":"2024-06-28T16:14:58.176689+0000","last_clean":"2024-06-28T16:14:58.176689+0000","last_became_active":"2024-06-28T15:00:23.427568+0000","last_became_peered":"2024-06-28T15:00:23.427568+0000","last_unstale":"2024-06-28T16:14:58.176689+0000","last_undegraded":"2024-06-28T16:14:58.176689+0000","last_fullsized":"2024-06-28T16:14:58.176689+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'537","last_scrub_stamp":"2024-06-28T16:14:58.176428+0000","last_deep_scrub":"260'537","last_deep_scrub_stamp":"2024-06-28T16:14:58.176428+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.176428+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":500,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:07:27.328025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"260'251","reported_seq":800,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.105514+0000","last_change":"2024-06-28T16:14:53.105587+0000","last_active":"2024-06-28T16:14:53.105514+0000","last_peered":"2024-06-28T16:14:53.105514+0000","last_clean":"2024-06-28T16:14:53.105514+0000","last_became_active":"2024-06-28T15:00:25.428067+0000","last_became_peered":"2024-06-28T15:00:25.428067+0000","last_unstale":"2024-06-28T16:14:53.105514+0000","last_undegraded":"2024-06-28T16:14:53.105514+0000","last_fullsized":"2024-06-28T16:14:53.105514+0000","mapping_epoch":29,"log_start":"260'200","ondisk_log_start":"260'200","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'251","last_scrub_stamp":"2024-06-28T16:14:53.105331+0000","last_deep_scrub":"260'251","last_deep_scrub_stamp":"2024-06-28T16:14:53.105331+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.105331+0000","objects_scrubbed":4,"log_size":51,"log_dups_size":200,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:01:44.076283+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":324,"num_read_kb":297,"num_write":552,"num_write_kb":170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":178,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"260'96","reported_seq":711,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.050892+0000","last_change":"2024-06-28T16:15:12.050892+0000","last_active":"2024-06-28T16:15:12.050892+0000","last_peered":"2024-06-28T16:15:12.050892+0000","last_clean":"2024-06-28T16:15:12.050892+0000","last_became_active":"2024-06-28T15:00:17.396413+0000","last_became_peered":"2024-06-28T15:00:17.396413+0000","last_unstale":"2024-06-28T16:15:12.050892+0000","last_undegraded":"2024-06-28T16:15:12.050892+0000","last_fullsized":"2024-06-28T16:15:12.050892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'96","last_scrub_stamp":"2024-06-28T16:15:12.050611+0000","last_deep_scrub":"260'96","last_deep_scrub_stamp":"2024-06-28T16:15:12.050611+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.050611+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:35:33.992945+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":410,"num_read_kb":340,"num_write":139,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"260'318","reported_seq":1106,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:25.169011+0000","last_change":"2024-06-28T16:15:25.169146+0000","last_active":"2024-06-28T16:15:25.169011+0000","last_peered":"2024-06-28T16:15:25.169011+0000","last_clean":"2024-06-28T16:15:25.169011+0000","last_became_active":"2024-06-28T15:00:15.395980+0000","last_became_peered":"2024-06-28T15:00:15.395980+0000","last_unstale":"2024-06-28T16:15:25.169011+0000","last_undegraded":"2024-06-28T16:15:25.169011+0000","last_fullsized":"2024-06-28T16:15:25.169011+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'318","last_scrub_stamp":"2024-06-28T16:15:25.168809+0000","last_deep_scrub":"260'318","last_deep_scrub_stamp":"2024-06-28T16:15:25.168809+0000","last_clean_scrub_stamp":"2024-06-28T16:15:25.168809+0000","objects_scrubbed":6,"log_size":18,"log_dups_size":300,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:57:31.434952+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":661262,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":649,"num_read_kb":125515,"num_write":1187,"num_write_kb":17513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,7],"acting":[0,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":531,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.460199+0000","last_change":"2024-06-28T16:15:04.460199+0000","last_active":"2024-06-28T16:15:04.460199+0000","last_peered":"2024-06-28T16:15:04.460199+0000","last_clean":"2024-06-28T16:15:04.460199+0000","last_became_active":"2024-06-28T15:00:06.283380+0000","last_became_peered":"2024-06-28T15:00:06.283380+0000","last_unstale":"2024-06-28T16:15:04.460199+0000","last_undegraded":"2024-06-28T16:15:04.460199+0000","last_fullsized":"2024-06-28T16:15:04.460199+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:04.459951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:04.459951+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.459951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:06:10.142541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"260'6","reported_seq":852,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.154538+0000","last_change":"2024-06-28T16:15:17.154602+0000","last_active":"2024-06-28T16:15:17.154538+0000","last_peered":"2024-06-28T16:15:17.154538+0000","last_clean":"2024-06-28T16:15:17.154538+0000","last_became_active":"2024-06-28T15:00:19.372552+0000","last_became_peered":"2024-06-28T15:00:19.372552+0000","last_unstale":"2024-06-28T16:15:17.154538+0000","last_undegraded":"2024-06-28T16:15:17.154538+0000","last_fullsized":"2024-06-28T16:15:17.154538+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'6","last_scrub_stamp":"2024-06-28T16:15:17.154344+0000","last_deep_scrub":"260'6","last_deep_scrub_stamp":"2024-06-28T16:15:17.154344+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.154344+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:46:02.206707+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1139,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":500,"num_read_kb":812,"num_write":10,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"260'756","reported_seq":1510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:20.845590+0000","last_change":"2024-06-28T16:15:20.845590+0000","last_active":"2024-06-28T16:15:20.845590+0000","last_peered":"2024-06-28T16:15:20.845590+0000","last_clean":"2024-06-28T16:15:20.845590+0000","last_became_active":"2024-06-28T15:00:21.481155+0000","last_became_peered":"2024-06-28T15:00:21.481155+0000","last_unstale":"2024-06-28T16:15:20.845590+0000","last_undegraded":"2024-06-28T16:15:20.845590+0000","last_fullsized":"2024-06-28T16:15:20.845590+0000","mapping_epoch":25,"log_start":"260'700","ondisk_log_start":"260'700","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'756","last_scrub_stamp":"2024-06-28T16:15:20.845293+0000","last_deep_scrub":"260'756","last_deep_scrub_stamp":"2024-06-28T16:15:20.845293+0000","last_clean_scrub_stamp":"2024-06-28T16:15:20.845293+0000","objects_scrubbed":8,"log_size":56,"log_dups_size":700,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:26:10.542424+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131438,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1320,"num_read_kb":3166,"num_write":1058,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.235034+0000","last_change":"2024-06-28T16:15:02.235034+0000","last_active":"2024-06-28T16:15:02.235034+0000","last_peered":"2024-06-28T16:15:02.235034+0000","last_clean":"2024-06-28T16:15:02.235034+0000","last_became_active":"2024-06-28T15:00:23.408777+0000","last_became_peered":"2024-06-28T15:00:23.408777+0000","last_unstale":"2024-06-28T16:15:02.235034+0000","last_undegraded":"2024-06-28T16:15:02.235034+0000","last_fullsized":"2024-06-28T16:15:02.235034+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:02.234718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:02.234718+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.234718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:27:42.722800+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"260'132","reported_seq":653,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.670572+0000","last_change":"2024-06-28T16:14:49.670641+0000","last_active":"2024-06-28T16:14:49.670572+0000","last_peered":"2024-06-28T16:14:49.670572+0000","last_clean":"2024-06-28T16:14:49.670572+0000","last_became_active":"2024-06-28T15:00:25.438476+0000","last_became_peered":"2024-06-28T15:00:25.438476+0000","last_unstale":"2024-06-28T16:14:49.670572+0000","last_undegraded":"2024-06-28T16:14:49.670572+0000","last_fullsized":"2024-06-28T16:14:49.670572+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'132","last_scrub_stamp":"2024-06-28T16:14:49.670359+0000","last_deep_scrub":"260'132","last_deep_scrub_stamp":"2024-06-28T16:14:49.670359+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.670359+0000","objects_scrubbed":3,"log_size":32,"log_dups_size":100,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:09:32.566857+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":620,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":173,"num_read_kb":156,"num_write":320,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1f","version":"260'153","reported_seq":730,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:49.120558+0000","last_change":"2024-06-28T16:14:49.120558+0000","last_active":"2024-06-28T16:14:49.120558+0000","last_peered":"2024-06-28T16:14:49.120558+0000","last_clean":"2024-06-28T16:14:49.120558+0000","last_became_active":"2024-06-28T15:00:25.439672+0000","last_became_peered":"2024-06-28T15:00:25.439672+0000","last_unstale":"2024-06-28T16:14:49.120558+0000","last_undegraded":"2024-06-28T16:14:49.120558+0000","last_fullsized":"2024-06-28T16:14:49.120558+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'153","last_scrub_stamp":"2024-06-28T16:14:49.120288+0000","last_deep_scrub":"260'153","last_deep_scrub_stamp":"2024-06-28T16:14:49.120288+0000","last_clean_scrub_stamp":"2024-06-28T16:14:49.120288+0000","objects_scrubbed":2,"log_size":53,"log_dups_size":100,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:05:56.112079+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":419,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":207,"num_read_kb":205,"num_write":345,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"260'435","reported_seq":1082,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.041132+0000","last_change":"2024-06-28T16:15:10.041196+0000","last_active":"2024-06-28T16:15:10.041132+0000","last_peered":"2024-06-28T16:15:10.041132+0000","last_clean":"2024-06-28T16:15:10.041132+0000","last_became_active":"2024-06-28T15:00:17.401981+0000","last_became_peered":"2024-06-28T15:00:17.401981+0000","last_unstale":"2024-06-28T16:15:10.041132+0000","last_undegraded":"2024-06-28T16:15:10.041132+0000","last_fullsized":"2024-06-28T16:15:10.041132+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'435","last_scrub_stamp":"2024-06-28T16:15:10.040927+0000","last_deep_scrub":"260'435","last_deep_scrub_stamp":"2024-06-28T16:15:10.040927+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.040927+0000","objects_scrubbed":1,"log_size":35,"log_dups_size":400,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:52:25.537900+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1728,"num_read_kb":1391,"num_write":746,"num_write_kb":387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":230,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"260'274","reported_seq":910,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.992519+0000","last_change":"2024-06-28T16:15:15.992709+0000","last_active":"2024-06-28T16:15:15.992519+0000","last_peered":"2024-06-28T16:15:15.992519+0000","last_clean":"2024-06-28T16:15:15.992519+0000","last_became_active":"2024-06-28T15:00:15.403450+0000","last_became_peered":"2024-06-28T15:00:15.403450+0000","last_unstale":"2024-06-28T16:15:15.992519+0000","last_undegraded":"2024-06-28T16:15:15.992519+0000","last_fullsized":"2024-06-28T16:15:15.992519+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'274","last_scrub_stamp":"2024-06-28T16:15:15.992200+0000","last_deep_scrub":"260'274","last_deep_scrub_stamp":"2024-06-28T16:15:15.992200+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.992200+0000","objects_scrubbed":4,"log_size":74,"log_dups_size":200,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:27:40.312409+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437799,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":413,"num_read_kb":13452,"num_write":1073,"num_write_kb":25751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,2,6],"acting":[4,7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":518,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.005157+0000","last_change":"2024-06-28T16:14:51.005157+0000","last_active":"2024-06-28T16:14:51.005157+0000","last_peered":"2024-06-28T16:14:51.005157+0000","last_clean":"2024-06-28T16:14:51.005157+0000","last_became_active":"2024-06-28T15:00:19.398138+0000","last_became_peered":"2024-06-28T15:00:19.398138+0000","last_unstale":"2024-06-28T16:14:51.005157+0000","last_undegraded":"2024-06-28T16:14:51.005157+0000","last_fullsized":"2024-06-28T16:14:51.005157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.004896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.004896+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.004896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:21:37.359914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"260'304","reported_seq":911,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.170902+0000","last_change":"2024-06-28T16:15:08.170995+0000","last_active":"2024-06-28T16:15:08.170902+0000","last_peered":"2024-06-28T16:15:08.170902+0000","last_clean":"2024-06-28T16:15:08.170902+0000","last_became_active":"2024-06-28T15:00:21.405016+0000","last_became_peered":"2024-06-28T15:00:21.405016+0000","last_unstale":"2024-06-28T16:15:08.170902+0000","last_undegraded":"2024-06-28T16:15:08.170902+0000","last_fullsized":"2024-06-28T16:15:08.170902+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'304","last_scrub_stamp":"2024-06-28T16:15:08.170703+0000","last_deep_scrub":"260'304","last_deep_scrub_stamp":"2024-06-28T16:15:08.170703+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.170703+0000","objects_scrubbed":6,"log_size":104,"log_dups_size":200,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:57:06.513866+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53726,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":505,"num_read_kb":1165,"num_write":406,"num_write_kb":309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":509,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.204716+0000","last_change":"2024-06-28T16:15:00.204716+0000","last_active":"2024-06-28T16:15:00.204716+0000","last_peered":"2024-06-28T16:15:00.204716+0000","last_clean":"2024-06-28T16:15:00.204716+0000","last_became_active":"2024-06-28T15:00:23.417390+0000","last_became_peered":"2024-06-28T15:00:23.417390+0000","last_unstale":"2024-06-28T16:15:00.204716+0000","last_undegraded":"2024-06-28T16:15:00.204716+0000","last_fullsized":"2024-06-28T16:15:00.204716+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:00.204463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:00.204463+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.204463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:41:34.935690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"260'101","reported_seq":617,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.114583+0000","last_change":"2024-06-28T16:14:56.114642+0000","last_active":"2024-06-28T16:14:56.114583+0000","last_peered":"2024-06-28T16:14:56.114583+0000","last_clean":"2024-06-28T16:14:56.114583+0000","last_became_active":"2024-06-28T15:00:25.412379+0000","last_became_peered":"2024-06-28T15:00:25.412379+0000","last_unstale":"2024-06-28T16:14:56.114583+0000","last_undegraded":"2024-06-28T16:14:56.114583+0000","last_fullsized":"2024-06-28T16:14:56.114583+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'101","last_scrub_stamp":"2024-06-28T16:14:56.114398+0000","last_deep_scrub":"260'101","last_deep_scrub_stamp":"2024-06-28T16:14:56.114398+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.114398+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:50:04.756947+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":88,"num_write":238,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"260'189","reported_seq":832,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.954505+0000","last_change":"2024-06-28T16:14:57.954505+0000","last_active":"2024-06-28T16:14:57.954505+0000","last_peered":"2024-06-28T16:14:57.954505+0000","last_clean":"2024-06-28T16:14:57.954505+0000","last_became_active":"2024-06-28T15:00:17.406647+0000","last_became_peered":"2024-06-28T15:00:17.406647+0000","last_unstale":"2024-06-28T16:14:57.954505+0000","last_undegraded":"2024-06-28T16:14:57.954505+0000","last_fullsized":"2024-06-28T16:14:57.954505+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'189","last_scrub_stamp":"2024-06-28T16:14:57.954211+0000","last_deep_scrub":"260'189","last_deep_scrub_stamp":"2024-06-28T16:14:57.954211+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.954211+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":100,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:15:57.967237+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":748,"num_read_kb":601,"num_write":301,"num_write_kb":145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"260'334","reported_seq":1011,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:27.040044+0000","last_change":"2024-06-28T16:15:27.040348+0000","last_active":"2024-06-28T16:15:27.040044+0000","last_peered":"2024-06-28T16:15:27.040044+0000","last_clean":"2024-06-28T16:15:27.040044+0000","last_became_active":"2024-06-28T15:00:15.402945+0000","last_became_peered":"2024-06-28T15:00:15.402945+0000","last_unstale":"2024-06-28T16:15:27.040044+0000","last_undegraded":"2024-06-28T16:15:27.040044+0000","last_fullsized":"2024-06-28T16:15:27.040044+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'334","last_scrub_stamp":"2024-06-28T16:15:27.039840+0000","last_deep_scrub":"260'334","last_deep_scrub_stamp":"2024-06-28T16:15:27.039840+0000","last_clean_scrub_stamp":"2024-06-28T16:15:27.039840+0000","objects_scrubbed":9,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:02:38.767341+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340417,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":534,"num_read_kb":20609,"num_write":1305,"num_write_kb":45266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0,4],"acting":[7,3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":517,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:21.320178+0000","last_change":"2024-06-28T16:15:21.320284+0000","last_active":"2024-06-28T16:15:21.320178+0000","last_peered":"2024-06-28T16:15:21.320178+0000","last_clean":"2024-06-28T16:15:21.320178+0000","last_became_active":"2024-06-28T15:00:19.382839+0000","last_became_peered":"2024-06-28T15:00:19.382839+0000","last_unstale":"2024-06-28T16:15:21.320178+0000","last_undegraded":"2024-06-28T16:15:21.320178+0000","last_fullsized":"2024-06-28T16:15:21.320178+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:21.319924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:21.319924+0000","last_clean_scrub_stamp":"2024-06-28T16:15:21.319924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:57:39.016603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"260'55","reported_seq":586,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.212479+0000","last_change":"2024-06-28T16:15:07.212638+0000","last_active":"2024-06-28T16:15:07.212479+0000","last_peered":"2024-06-28T16:15:07.212479+0000","last_clean":"2024-06-28T16:15:07.212479+0000","last_became_active":"2024-06-28T15:00:21.404727+0000","last_became_peered":"2024-06-28T15:00:21.404727+0000","last_unstale":"2024-06-28T16:15:07.212479+0000","last_undegraded":"2024-06-28T16:15:07.212479+0000","last_fullsized":"2024-06-28T16:15:07.212479+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'55","last_scrub_stamp":"2024-06-28T16:15:07.212261+0000","last_deep_scrub":"260'55","last_deep_scrub_stamp":"2024-06-28T16:15:07.212261+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.212261+0000","objects_scrubbed":3,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:08:38.234431+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":48,"num_write":36,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.776013+0000","last_change":"2024-06-28T16:15:01.776013+0000","last_active":"2024-06-28T16:15:01.776013+0000","last_peered":"2024-06-28T16:15:01.776013+0000","last_clean":"2024-06-28T16:15:01.776013+0000","last_became_active":"2024-06-28T15:00:23.419115+0000","last_became_peered":"2024-06-28T15:00:23.419115+0000","last_unstale":"2024-06-28T16:15:01.776013+0000","last_undegraded":"2024-06-28T16:15:01.776013+0000","last_fullsized":"2024-06-28T16:15:01.776013+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:01.775641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:01.775641+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.775641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:56:43.293391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"260'191","reported_seq":730,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.032091+0000","last_change":"2024-06-28T16:14:55.032091+0000","last_active":"2024-06-28T16:14:55.032091+0000","last_peered":"2024-06-28T16:14:55.032091+0000","last_clean":"2024-06-28T16:14:55.032091+0000","last_became_active":"2024-06-28T15:00:25.426046+0000","last_became_peered":"2024-06-28T15:00:25.426046+0000","last_unstale":"2024-06-28T16:14:55.032091+0000","last_undegraded":"2024-06-28T16:14:55.032091+0000","last_fullsized":"2024-06-28T16:14:55.032091+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'191","last_scrub_stamp":"2024-06-28T16:14:55.031799+0000","last_deep_scrub":"260'191","last_deep_scrub_stamp":"2024-06-28T16:14:55.031799+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.031799+0000","objects_scrubbed":1,"log_size":91,"log_dups_size":100,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:06:38.230720+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":286,"num_read_kb":283,"num_write":434,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":398,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"260'423","reported_seq":1180,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.153858+0000","last_change":"2024-06-28T16:15:13.154003+0000","last_active":"2024-06-28T16:15:13.153858+0000","last_peered":"2024-06-28T16:15:13.153858+0000","last_clean":"2024-06-28T16:15:13.153858+0000","last_became_active":"2024-06-28T15:00:17.389992+0000","last_became_peered":"2024-06-28T15:00:17.389992+0000","last_unstale":"2024-06-28T16:15:13.153858+0000","last_undegraded":"2024-06-28T16:15:13.153858+0000","last_fullsized":"2024-06-28T16:15:13.153858+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'423","last_scrub_stamp":"2024-06-28T16:15:13.153624+0000","last_deep_scrub":"260'423","last_deep_scrub_stamp":"2024-06-28T16:15:13.153624+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.153624+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":400,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:59:03.430418+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1786,"num_read_kb":1490,"num_write":759,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"260'328","reported_seq":981,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.730674+0000","last_change":"2024-06-28T16:15:13.730884+0000","last_active":"2024-06-28T16:15:13.730674+0000","last_peered":"2024-06-28T16:15:13.730674+0000","last_clean":"2024-06-28T16:15:13.730674+0000","last_became_active":"2024-06-28T15:00:15.419267+0000","last_became_peered":"2024-06-28T15:00:15.419267+0000","last_unstale":"2024-06-28T16:15:13.730674+0000","last_undegraded":"2024-06-28T16:15:13.730674+0000","last_fullsized":"2024-06-28T16:15:13.730674+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'328","last_scrub_stamp":"2024-06-28T16:15:13.730424+0000","last_deep_scrub":"260'328","last_deep_scrub_stamp":"2024-06-28T16:15:13.730424+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.730424+0000","objects_scrubbed":5,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:57:21.223073+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5413475,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":487,"num_read_kb":27459,"num_write":1288,"num_write_kb":48933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,3],"acting":[5,7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":512,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.747129+0000","last_change":"2024-06-28T16:15:06.747183+0000","last_active":"2024-06-28T16:15:06.747129+0000","last_peered":"2024-06-28T16:15:06.747129+0000","last_clean":"2024-06-28T16:15:06.747129+0000","last_became_active":"2024-06-28T15:00:19.385990+0000","last_became_peered":"2024-06-28T15:00:19.385990+0000","last_unstale":"2024-06-28T16:15:06.747129+0000","last_undegraded":"2024-06-28T16:15:06.747129+0000","last_fullsized":"2024-06-28T16:15:06.747129+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:06.746938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:06.746938+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.746938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:07:08.302586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"260'323","reported_seq":946,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.152026+0000","last_change":"2024-06-28T16:15:07.152103+0000","last_active":"2024-06-28T16:15:07.152026+0000","last_peered":"2024-06-28T16:15:07.152026+0000","last_clean":"2024-06-28T16:15:07.152026+0000","last_became_active":"2024-06-28T15:00:21.405045+0000","last_became_peered":"2024-06-28T15:00:21.405045+0000","last_unstale":"2024-06-28T16:15:07.152026+0000","last_undegraded":"2024-06-28T16:15:07.152026+0000","last_fullsized":"2024-06-28T16:15:07.152026+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'323","last_scrub_stamp":"2024-06-28T16:15:07.151832+0000","last_deep_scrub":"260'323","last_deep_scrub_stamp":"2024-06-28T16:15:07.151832+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.151832+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":300,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:20:40.489212+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":48456,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":530,"num_read_kb":1162,"num_write":414,"num_write_kb":289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":502,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.144381+0000","last_change":"2024-06-28T16:14:50.144381+0000","last_active":"2024-06-28T16:14:50.144381+0000","last_peered":"2024-06-28T16:14:50.144381+0000","last_clean":"2024-06-28T16:14:50.144381+0000","last_became_active":"2024-06-28T15:00:23.411264+0000","last_became_peered":"2024-06-28T15:00:23.411264+0000","last_unstale":"2024-06-28T16:14:50.144381+0000","last_undegraded":"2024-06-28T16:14:50.144381+0000","last_fullsized":"2024-06-28T16:14:50.144381+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:50.144047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:50.144047+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.144047+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:56:31.669987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"260'113","reported_seq":615,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.683376+0000","last_change":"2024-06-28T16:14:43.683376+0000","last_active":"2024-06-28T16:14:43.683376+0000","last_peered":"2024-06-28T16:14:43.683376+0000","last_clean":"2024-06-28T16:14:43.683376+0000","last_became_active":"2024-06-28T15:00:25.442010+0000","last_became_peered":"2024-06-28T15:00:25.442010+0000","last_unstale":"2024-06-28T16:14:43.683376+0000","last_undegraded":"2024-06-28T16:14:43.683376+0000","last_fullsized":"2024-06-28T16:14:43.683376+0000","mapping_epoch":29,"log_start":"260'101","ondisk_log_start":"260'101","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'113","last_scrub_stamp":"2024-06-28T16:14:43.682925+0000","last_deep_scrub":"260'113","last_deep_scrub_stamp":"2024-06-28T16:14:43.682925+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.682925+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":101,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:07:47.348656+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":638,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":85,"num_write":260,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"260'105","reported_seq":679,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:20.277565+0000","last_change":"2024-06-28T16:15:20.277626+0000","last_active":"2024-06-28T16:15:20.277565+0000","last_peered":"2024-06-28T16:15:20.277565+0000","last_clean":"2024-06-28T16:15:20.277565+0000","last_became_active":"2024-06-28T15:00:17.373396+0000","last_became_peered":"2024-06-28T15:00:17.373396+0000","last_unstale":"2024-06-28T16:15:20.277565+0000","last_undegraded":"2024-06-28T16:15:20.277565+0000","last_fullsized":"2024-06-28T16:15:20.277565+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'105","last_scrub_stamp":"2024-06-28T16:15:20.277371+0000","last_deep_scrub":"260'105","last_deep_scrub_stamp":"2024-06-28T16:15:20.277371+0000","last_clean_scrub_stamp":"2024-06-28T16:15:20.277371+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:05:20.707326+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":424,"num_read_kb":340,"num_write":155,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"260'307","reported_seq":964,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:30.286189+0000","last_change":"2024-06-28T16:15:30.286346+0000","last_active":"2024-06-28T16:15:30.286189+0000","last_peered":"2024-06-28T16:15:30.286189+0000","last_clean":"2024-06-28T16:15:30.286189+0000","last_became_active":"2024-06-28T15:00:15.412333+0000","last_became_peered":"2024-06-28T15:00:15.412333+0000","last_unstale":"2024-06-28T16:15:30.286189+0000","last_undegraded":"2024-06-28T16:15:30.286189+0000","last_fullsized":"2024-06-28T16:15:30.286189+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'307","last_scrub_stamp":"2024-06-28T16:15:30.285983+0000","last_deep_scrub":"260'307","last_deep_scrub_stamp":"2024-06-28T16:15:30.285983+0000","last_clean_scrub_stamp":"2024-06-28T16:15:30.285983+0000","objects_scrubbed":8,"log_size":107,"log_dups_size":200,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:20:34.060427+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1451455,"num_objects":8,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":458,"num_read_kb":31119,"num_write":1276,"num_write_kb":43155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6,3],"acting":[0,5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.188245+0000","last_change":"2024-06-28T16:14:51.188245+0000","last_active":"2024-06-28T16:14:51.188245+0000","last_peered":"2024-06-28T16:14:51.188245+0000","last_clean":"2024-06-28T16:14:51.188245+0000","last_became_active":"2024-06-28T15:00:19.386645+0000","last_became_peered":"2024-06-28T15:00:19.386645+0000","last_unstale":"2024-06-28T16:14:51.188245+0000","last_undegraded":"2024-06-28T16:14:51.188245+0000","last_fullsized":"2024-06-28T16:14:51.188245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.187878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.187878+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.187878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:53:35.500317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"260'293","reported_seq":887,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.836441+0000","last_change":"2024-06-28T16:15:16.836538+0000","last_active":"2024-06-28T16:15:16.836441+0000","last_peered":"2024-06-28T16:15:16.836441+0000","last_clean":"2024-06-28T16:15:16.836441+0000","last_became_active":"2024-06-28T15:00:21.394357+0000","last_became_peered":"2024-06-28T15:00:21.394357+0000","last_unstale":"2024-06-28T16:15:16.836441+0000","last_undegraded":"2024-06-28T16:15:16.836441+0000","last_fullsized":"2024-06-28T16:15:16.836441+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'293","last_scrub_stamp":"2024-06-28T16:15:16.836265+0000","last_deep_scrub":"260'293","last_deep_scrub_stamp":"2024-06-28T16:15:16.836265+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.836265+0000","objects_scrubbed":4,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:39:45.845728+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53697,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":496,"num_read_kb":1194,"num_write":404,"num_write_kb":327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":511,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.135809+0000","last_change":"2024-06-28T16:14:59.135809+0000","last_active":"2024-06-28T16:14:59.135809+0000","last_peered":"2024-06-28T16:14:59.135809+0000","last_clean":"2024-06-28T16:14:59.135809+0000","last_became_active":"2024-06-28T15:00:23.406296+0000","last_became_peered":"2024-06-28T15:00:23.406296+0000","last_unstale":"2024-06-28T16:14:59.135809+0000","last_undegraded":"2024-06-28T16:14:59.135809+0000","last_fullsized":"2024-06-28T16:14:59.135809+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:59.135545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:59.135545+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.135545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:33:44.459217+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"260'68","reported_seq":646,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.449222+0000","last_change":"2024-06-28T16:15:01.449222+0000","last_active":"2024-06-28T16:15:01.449222+0000","last_peered":"2024-06-28T16:15:01.449222+0000","last_clean":"2024-06-28T16:15:01.449222+0000","last_became_active":"2024-06-28T15:00:17.396861+0000","last_became_peered":"2024-06-28T15:00:17.396861+0000","last_unstale":"2024-06-28T16:15:01.449222+0000","last_undegraded":"2024-06-28T16:15:01.449222+0000","last_fullsized":"2024-06-28T16:15:01.449222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'68","last_scrub_stamp":"2024-06-28T16:15:01.448976+0000","last_deep_scrub":"260'68","last_deep_scrub_stamp":"2024-06-28T16:15:01.448976+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.448976+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:55:04.325605+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":279,"num_read_kb":229,"num_write":93,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"260'309","reported_seq":954,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:28.303700+0000","last_change":"2024-06-28T16:15:28.303812+0000","last_active":"2024-06-28T16:15:28.303700+0000","last_peered":"2024-06-28T16:15:28.303700+0000","last_clean":"2024-06-28T16:15:28.303700+0000","last_became_active":"2024-06-28T15:00:15.383008+0000","last_became_peered":"2024-06-28T15:00:15.383008+0000","last_unstale":"2024-06-28T16:15:28.303700+0000","last_undegraded":"2024-06-28T16:15:28.303700+0000","last_fullsized":"2024-06-28T16:15:28.303700+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'309","last_scrub_stamp":"2024-06-28T16:15:28.303480+0000","last_deep_scrub":"260'309","last_deep_scrub_stamp":"2024-06-28T16:15:28.303480+0000","last_clean_scrub_stamp":"2024-06-28T16:15:28.303480+0000","objects_scrubbed":6,"log_size":109,"log_dups_size":200,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:25:28.517303+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2156401,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":459,"num_read_kb":6724,"num_write":1144,"num_write_kb":24207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":524,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.216032+0000","last_change":"2024-06-28T16:15:18.216084+0000","last_active":"2024-06-28T16:15:18.216032+0000","last_peered":"2024-06-28T16:15:18.216032+0000","last_clean":"2024-06-28T16:15:18.216032+0000","last_became_active":"2024-06-28T15:00:19.390111+0000","last_became_peered":"2024-06-28T15:00:19.390111+0000","last_unstale":"2024-06-28T16:15:18.216032+0000","last_undegraded":"2024-06-28T16:15:18.216032+0000","last_fullsized":"2024-06-28T16:15:18.216032+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:18.215836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:18.215836+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.215836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:01:06.524192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"260'476","reported_seq":1136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:54.999846+0000","last_change":"2024-06-28T16:14:54.999846+0000","last_active":"2024-06-28T16:14:54.999846+0000","last_peered":"2024-06-28T16:14:54.999846+0000","last_clean":"2024-06-28T16:14:54.999846+0000","last_became_active":"2024-06-28T15:00:21.398262+0000","last_became_peered":"2024-06-28T15:00:21.398262+0000","last_unstale":"2024-06-28T16:14:54.999846+0000","last_undegraded":"2024-06-28T16:14:54.999846+0000","last_fullsized":"2024-06-28T16:14:54.999846+0000","mapping_epoch":25,"log_start":"260'400","ondisk_log_start":"260'400","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'476","last_scrub_stamp":"2024-06-28T16:14:54.999556+0000","last_deep_scrub":"260'476","last_deep_scrub_stamp":"2024-06-28T16:14:54.999556+0000","last_clean_scrub_stamp":"2024-06-28T16:14:54.999556+0000","objects_scrubbed":6,"log_size":76,"log_dups_size":400,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:46:08.548671+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86979,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":863,"num_read_kb":2070,"num_write":734,"num_write_kb":637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":24966,"num_omap_keys":48,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":503,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.607103+0000","last_change":"2024-06-28T16:14:53.607103+0000","last_active":"2024-06-28T16:14:53.607103+0000","last_peered":"2024-06-28T16:14:53.607103+0000","last_clean":"2024-06-28T16:14:53.607103+0000","last_became_active":"2024-06-28T15:00:23.410519+0000","last_became_peered":"2024-06-28T15:00:23.410519+0000","last_unstale":"2024-06-28T16:14:53.607103+0000","last_undegraded":"2024-06-28T16:14:53.607103+0000","last_fullsized":"2024-06-28T16:14:53.607103+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.606849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.606849+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.606849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:00:33.451749+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"260'170","reported_seq":697,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.126982+0000","last_change":"2024-06-28T16:14:50.126982+0000","last_active":"2024-06-28T16:14:50.126982+0000","last_peered":"2024-06-28T16:14:50.126982+0000","last_clean":"2024-06-28T16:14:50.126982+0000","last_became_active":"2024-06-28T15:00:25.558426+0000","last_became_peered":"2024-06-28T15:00:25.558426+0000","last_unstale":"2024-06-28T16:14:50.126982+0000","last_undegraded":"2024-06-28T16:14:50.126982+0000","last_fullsized":"2024-06-28T16:14:50.126982+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'170","last_scrub_stamp":"2024-06-28T16:14:50.126565+0000","last_deep_scrub":"260'170","last_deep_scrub_stamp":"2024-06-28T16:14:50.126565+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.126565+0000","objects_scrubbed":3,"log_size":70,"log_dups_size":100,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:28:56.261088+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":845,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":215,"num_read_kb":219,"num_write":377,"num_write_kb":95,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"260'159","reported_seq":797,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:08.243927+0000","last_change":"2024-06-28T16:15:08.243996+0000","last_active":"2024-06-28T16:15:08.243927+0000","last_peered":"2024-06-28T16:15:08.243927+0000","last_clean":"2024-06-28T16:15:08.243927+0000","last_became_active":"2024-06-28T15:00:17.395643+0000","last_became_peered":"2024-06-28T15:00:17.395643+0000","last_unstale":"2024-06-28T16:15:08.243927+0000","last_undegraded":"2024-06-28T16:15:08.243927+0000","last_fullsized":"2024-06-28T16:15:08.243927+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'159","last_scrub_stamp":"2024-06-28T16:15:08.243717+0000","last_deep_scrub":"260'159","last_deep_scrub_stamp":"2024-06-28T16:15:08.243717+0000","last_clean_scrub_stamp":"2024-06-28T16:15:08.243717+0000","objects_scrubbed":1,"log_size":59,"log_dups_size":100,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:49:47.952363+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":654,"num_read_kb":561,"num_write":257,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2876,"num_omap_keys":12,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"260'290","reported_seq":940,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.075499+0000","last_change":"2024-06-28T16:14:44.075499+0000","last_active":"2024-06-28T16:14:44.075499+0000","last_peered":"2024-06-28T16:14:44.075499+0000","last_clean":"2024-06-28T16:14:44.075499+0000","last_became_active":"2024-06-28T15:00:59.939485+0000","last_became_peered":"2024-06-28T15:00:59.939485+0000","last_unstale":"2024-06-28T16:14:44.075499+0000","last_undegraded":"2024-06-28T16:14:44.075499+0000","last_fullsized":"2024-06-28T16:14:44.075499+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'290","last_scrub_stamp":"2024-06-28T16:14:44.075189+0000","last_deep_scrub":"260'290","last_deep_scrub_stamp":"2024-06-28T16:14:44.075189+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.075189+0000","objects_scrubbed":3,"log_size":90,"log_dups_size":200,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:23:04.302226+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":430,"num_read_kb":21907,"num_write":1034,"num_write_kb":35159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,6],"acting":[1,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:53.274570+0000","last_change":"2024-06-28T16:14:53.274570+0000","last_active":"2024-06-28T16:14:53.274570+0000","last_peered":"2024-06-28T16:14:53.274570+0000","last_clean":"2024-06-28T16:14:53.274570+0000","last_became_active":"2024-06-28T15:00:19.385068+0000","last_became_peered":"2024-06-28T15:00:19.385068+0000","last_unstale":"2024-06-28T16:14:53.274570+0000","last_undegraded":"2024-06-28T16:14:53.274570+0000","last_fullsized":"2024-06-28T16:14:53.274570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:53.274324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:53.274324+0000","last_clean_scrub_stamp":"2024-06-28T16:14:53.274324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:11:33.949188+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"260'356","reported_seq":972,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:18.835932+0000","last_change":"2024-06-28T16:15:18.835932+0000","last_active":"2024-06-28T16:15:18.835932+0000","last_peered":"2024-06-28T16:15:18.835932+0000","last_clean":"2024-06-28T16:15:18.835932+0000","last_became_active":"2024-06-28T15:00:21.400480+0000","last_became_peered":"2024-06-28T15:00:21.400480+0000","last_unstale":"2024-06-28T16:15:18.835932+0000","last_undegraded":"2024-06-28T16:15:18.835932+0000","last_fullsized":"2024-06-28T16:15:18.835932+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'356","last_scrub_stamp":"2024-06-28T16:15:18.835637+0000","last_deep_scrub":"260'356","last_deep_scrub_stamp":"2024-06-28T16:15:18.835637+0000","last_clean_scrub_stamp":"2024-06-28T16:15:18.835637+0000","objects_scrubbed":8,"log_size":56,"log_dups_size":300,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:15:40.539768+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":47708,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":581,"num_read_kb":1249,"num_write":480,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":11460,"num_omap_keys":24,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":508,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:05.168340+0000","last_change":"2024-06-28T16:15:05.168340+0000","last_active":"2024-06-28T16:15:05.168340+0000","last_peered":"2024-06-28T16:15:05.168340+0000","last_clean":"2024-06-28T16:15:05.168340+0000","last_became_active":"2024-06-28T15:00:23.417580+0000","last_became_peered":"2024-06-28T15:00:23.417580+0000","last_unstale":"2024-06-28T16:15:05.168340+0000","last_undegraded":"2024-06-28T16:15:05.168340+0000","last_fullsized":"2024-06-28T16:15:05.168340+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:05.168092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:05.168092+0000","last_clean_scrub_stamp":"2024-06-28T16:15:05.168092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:54:08.480636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"260'176","reported_seq":693,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.494904+0000","last_change":"2024-06-28T16:15:02.494904+0000","last_active":"2024-06-28T16:15:02.494904+0000","last_peered":"2024-06-28T16:15:02.494904+0000","last_clean":"2024-06-28T16:15:02.494904+0000","last_became_active":"2024-06-28T15:00:25.430299+0000","last_became_peered":"2024-06-28T15:00:25.430299+0000","last_unstale":"2024-06-28T16:15:02.494904+0000","last_undegraded":"2024-06-28T16:15:02.494904+0000","last_fullsized":"2024-06-28T16:15:02.494904+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'176","last_scrub_stamp":"2024-06-28T16:15:02.494614+0000","last_deep_scrub":"260'176","last_deep_scrub_stamp":"2024-06-28T16:15:02.494614+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.494614+0000","objects_scrubbed":6,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:35:42.752972+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2035,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":193,"num_read_kb":191,"num_write":477,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1d","version":"260'315","reported_seq":1067,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.121432+0000","last_change":"2024-06-28T16:15:12.121432+0000","last_active":"2024-06-28T16:15:12.121432+0000","last_peered":"2024-06-28T16:15:12.121432+0000","last_clean":"2024-06-28T16:15:12.121432+0000","last_became_active":"2024-06-28T15:00:17.383284+0000","last_became_peered":"2024-06-28T15:00:17.383284+0000","last_unstale":"2024-06-28T16:15:12.121432+0000","last_undegraded":"2024-06-28T16:15:12.121432+0000","last_fullsized":"2024-06-28T16:15:12.121432+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'315","last_scrub_stamp":"2024-06-28T16:15:12.121164+0000","last_deep_scrub":"260'315","last_deep_scrub_stamp":"2024-06-28T16:15:12.121164+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.121164+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":300,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:19:12.129939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1515,"num_read_kb":1335,"num_write":548,"num_write_kb":317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"260'347","reported_seq":1038,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.960069+0000","last_change":"2024-06-28T16:15:16.960532+0000","last_active":"2024-06-28T16:15:16.960069+0000","last_peered":"2024-06-28T16:15:16.960069+0000","last_clean":"2024-06-28T16:15:16.960069+0000","last_became_active":"2024-06-28T15:00:15.405499+0000","last_became_peered":"2024-06-28T15:00:15.405499+0000","last_unstale":"2024-06-28T16:15:16.960069+0000","last_undegraded":"2024-06-28T16:15:16.960069+0000","last_fullsized":"2024-06-28T16:15:16.960069+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'347","last_scrub_stamp":"2024-06-28T16:15:16.959776+0000","last_deep_scrub":"260'347","last_deep_scrub_stamp":"2024-06-28T16:15:16.959776+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.959776+0000","objects_scrubbed":13,"log_size":47,"log_dups_size":300,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:25:44.951698+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17426850,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":585,"num_read_kb":18043,"num_write":1303,"num_write_kb":33783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7,2],"acting":[4,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":512,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.087242+0000","last_change":"2024-06-28T16:15:11.087242+0000","last_active":"2024-06-28T16:15:11.087242+0000","last_peered":"2024-06-28T16:15:11.087242+0000","last_clean":"2024-06-28T16:15:11.087242+0000","last_became_active":"2024-06-28T15:00:19.378852+0000","last_became_peered":"2024-06-28T15:00:19.378852+0000","last_unstale":"2024-06-28T16:15:11.087242+0000","last_undegraded":"2024-06-28T16:15:11.087242+0000","last_fullsized":"2024-06-28T16:15:11.087242+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:11.086992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:11.086992+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.086992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:18:25.029287+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"260'285","reported_seq":896,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.193472+0000","last_change":"2024-06-28T16:15:06.193472+0000","last_active":"2024-06-28T16:15:06.193472+0000","last_peered":"2024-06-28T16:15:06.193472+0000","last_clean":"2024-06-28T16:15:06.193472+0000","last_became_active":"2024-06-28T15:00:21.480868+0000","last_became_peered":"2024-06-28T15:00:21.480868+0000","last_unstale":"2024-06-28T16:15:06.193472+0000","last_undegraded":"2024-06-28T16:15:06.193472+0000","last_fullsized":"2024-06-28T16:15:06.193472+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'285","last_scrub_stamp":"2024-06-28T16:15:06.193211+0000","last_deep_scrub":"260'285","last_deep_scrub_stamp":"2024-06-28T16:15:06.193211+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.193211+0000","objects_scrubbed":6,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:07:51.229985+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40850,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":446,"num_read_kb":956,"num_write":348,"num_write_kb":241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":515,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.011327+0000","last_change":"2024-06-28T16:15:01.011381+0000","last_active":"2024-06-28T16:15:01.011327+0000","last_peered":"2024-06-28T16:15:01.011327+0000","last_clean":"2024-06-28T16:15:01.011327+0000","last_became_active":"2024-06-28T15:00:23.400417+0000","last_became_peered":"2024-06-28T15:00:23.400417+0000","last_unstale":"2024-06-28T16:15:01.011327+0000","last_undegraded":"2024-06-28T16:15:01.011327+0000","last_fullsized":"2024-06-28T16:15:01.011327+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:01.011100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:01.011100+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.011100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:29:13.769857+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"260'121","reported_seq":631,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:50.687682+0000","last_change":"2024-06-28T16:14:50.687682+0000","last_active":"2024-06-28T16:14:50.687682+0000","last_peered":"2024-06-28T16:14:50.687682+0000","last_clean":"2024-06-28T16:14:50.687682+0000","last_became_active":"2024-06-28T15:00:25.438176+0000","last_became_peered":"2024-06-28T15:00:25.438176+0000","last_unstale":"2024-06-28T16:14:50.687682+0000","last_undegraded":"2024-06-28T16:14:50.687682+0000","last_fullsized":"2024-06-28T16:14:50.687682+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'121","last_scrub_stamp":"2024-06-28T16:14:50.687377+0000","last_deep_scrub":"260'121","last_deep_scrub_stamp":"2024-06-28T16:14:50.687377+0000","last_clean_scrub_stamp":"2024-06-28T16:14:50.687377+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":100,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:06:30.493792+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":120,"num_read_kb":111,"num_write":277,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"260'74","reported_seq":665,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:02.334685+0000","last_change":"2024-06-28T16:15:02.334685+0000","last_active":"2024-06-28T16:15:02.334685+0000","last_peered":"2024-06-28T16:15:02.334685+0000","last_clean":"2024-06-28T16:15:02.334685+0000","last_became_active":"2024-06-28T15:00:17.402735+0000","last_became_peered":"2024-06-28T15:00:17.402735+0000","last_unstale":"2024-06-28T16:15:02.334685+0000","last_undegraded":"2024-06-28T16:15:02.334685+0000","last_fullsized":"2024-06-28T16:15:02.334685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'74","last_scrub_stamp":"2024-06-28T16:15:02.334423+0000","last_deep_scrub":"260'74","last_deep_scrub_stamp":"2024-06-28T16:15:02.334423+0000","last_clean_scrub_stamp":"2024-06-28T16:15:02.334423+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:29:13.249710+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":299,"num_read_kb":249,"num_write":100,"num_write_kb":39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"260'336","reported_seq":1068,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:23.105933+0000","last_change":"2024-06-28T16:15:23.106192+0000","last_active":"2024-06-28T16:15:23.105933+0000","last_peered":"2024-06-28T16:15:23.105933+0000","last_clean":"2024-06-28T16:15:23.105933+0000","last_became_active":"2024-06-28T15:00:15.415197+0000","last_became_peered":"2024-06-28T15:00:15.415197+0000","last_unstale":"2024-06-28T16:15:23.105933+0000","last_undegraded":"2024-06-28T16:15:23.105933+0000","last_fullsized":"2024-06-28T16:15:23.105933+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'336","last_scrub_stamp":"2024-06-28T16:15:23.105650+0000","last_deep_scrub":"260'336","last_deep_scrub_stamp":"2024-06-28T16:15:23.105650+0000","last_clean_scrub_stamp":"2024-06-28T16:15:23.105650+0000","objects_scrubbed":5,"log_size":36,"log_dups_size":300,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:23:46.778415+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3732560,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":648,"num_read_kb":6210,"num_write":1163,"num_write_kb":17828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7,1],"acting":[0,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":516,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.967053+0000","last_change":"2024-06-28T16:14:51.967053+0000","last_active":"2024-06-28T16:14:51.967053+0000","last_peered":"2024-06-28T16:14:51.967053+0000","last_clean":"2024-06-28T16:14:51.967053+0000","last_became_active":"2024-06-28T15:00:19.378270+0000","last_became_peered":"2024-06-28T15:00:19.378270+0000","last_unstale":"2024-06-28T16:14:51.967053+0000","last_undegraded":"2024-06-28T16:14:51.967053+0000","last_fullsized":"2024-06-28T16:14:51.967053+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:51.966801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:51.966801+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.966801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:16:26.209301+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"260'107","reported_seq":662,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:51.993632+0000","last_change":"2024-06-28T16:14:51.993706+0000","last_active":"2024-06-28T16:14:51.993632+0000","last_peered":"2024-06-28T16:14:51.993632+0000","last_clean":"2024-06-28T16:14:51.993632+0000","last_became_active":"2024-06-28T15:00:21.484352+0000","last_became_peered":"2024-06-28T15:00:21.484352+0000","last_unstale":"2024-06-28T16:14:51.993632+0000","last_undegraded":"2024-06-28T16:14:51.993632+0000","last_fullsized":"2024-06-28T16:14:51.993632+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'107","last_scrub_stamp":"2024-06-28T16:14:51.993438+0000","last_deep_scrub":"260'107","last_deep_scrub_stamp":"2024-06-28T16:14:51.993438+0000","last_clean_scrub_stamp":"2024-06-28T16:14:51.993438+0000","objects_scrubbed":6,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:28:06.528863+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":124,"num_read_kb":119,"num_write":88,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1780,"num_omap_keys":6,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1f","version":"260'515","reported_seq":2255,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.146256+0000","last_change":"2024-06-28T16:14:48.146256+0000","last_active":"2024-06-28T16:14:48.146256+0000","last_peered":"2024-06-28T16:14:48.146256+0000","last_clean":"2024-06-28T16:14:48.146256+0000","last_became_active":"2024-06-28T15:00:23.407490+0000","last_became_peered":"2024-06-28T15:00:23.407490+0000","last_unstale":"2024-06-28T16:14:48.146256+0000","last_undegraded":"2024-06-28T16:14:48.146256+0000","last_fullsized":"2024-06-28T16:14:48.146256+0000","mapping_epoch":27,"log_start":"260'500","ondisk_log_start":"260'500","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'515","last_scrub_stamp":"2024-06-28T16:14:48.145991+0000","last_deep_scrub":"260'515","last_deep_scrub_stamp":"2024-06-28T16:14:48.145991+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.145991+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":500,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:10:57.232491+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.10","version":"260'103","reported_seq":617,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.637768+0000","last_change":"2024-06-28T16:14:55.637768+0000","last_active":"2024-06-28T16:14:55.637768+0000","last_peered":"2024-06-28T16:14:55.637768+0000","last_clean":"2024-06-28T16:14:55.637768+0000","last_became_active":"2024-06-28T15:00:25.445018+0000","last_became_peered":"2024-06-28T15:00:25.445018+0000","last_unstale":"2024-06-28T16:14:55.637768+0000","last_undegraded":"2024-06-28T16:14:55.637768+0000","last_fullsized":"2024-06-28T16:14:55.637768+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'103","last_scrub_stamp":"2024-06-28T16:14:55.637507+0000","last_deep_scrub":"260'103","last_deep_scrub_stamp":"2024-06-28T16:14:55.637507+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.637507+0000","objects_scrubbed":1,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:19:26.856020+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":117,"num_read_kb":111,"num_write":241,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"260'78","reported_seq":664,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:20.971945+0000","last_change":"2024-06-28T16:15:20.972078+0000","last_active":"2024-06-28T16:15:20.971945+0000","last_peered":"2024-06-28T16:15:20.971945+0000","last_clean":"2024-06-28T16:15:20.971945+0000","last_became_active":"2024-06-28T15:00:17.387086+0000","last_became_peered":"2024-06-28T15:00:17.387086+0000","last_unstale":"2024-06-28T16:15:20.971945+0000","last_undegraded":"2024-06-28T16:15:20.971945+0000","last_fullsized":"2024-06-28T16:15:20.971945+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'78","last_scrub_stamp":"2024-06-28T16:15:20.971625+0000","last_deep_scrub":"260'78","last_deep_scrub_stamp":"2024-06-28T16:15:20.971625+0000","last_clean_scrub_stamp":"2024-06-28T16:15:20.971625+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:39:40.485714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":333,"num_read_kb":275,"num_write":113,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"260'328","reported_seq":970,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:44.118252+0000","last_change":"2024-06-28T16:14:44.118252+0000","last_active":"2024-06-28T16:14:44.118252+0000","last_peered":"2024-06-28T16:14:44.118252+0000","last_clean":"2024-06-28T16:14:44.118252+0000","last_became_active":"2024-06-28T15:00:59.941915+0000","last_became_peered":"2024-06-28T15:00:59.941915+0000","last_unstale":"2024-06-28T16:14:44.118252+0000","last_undegraded":"2024-06-28T16:14:44.118252+0000","last_fullsized":"2024-06-28T16:14:44.118252+0000","mapping_epoch":32,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'328","last_scrub_stamp":"2024-06-28T16:14:44.117934+0000","last_deep_scrub":"260'328","last_deep_scrub_stamp":"2024-06-28T16:14:44.117934+0000","last_clean_scrub_stamp":"2024-06-28T16:14:44.117934+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":300,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:54:20.388393+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5506084,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":497,"num_read_kb":57892,"num_write":1218,"num_write_kb":47404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:12.747922+0000","last_change":"2024-06-28T16:15:12.747922+0000","last_active":"2024-06-28T16:15:12.747922+0000","last_peered":"2024-06-28T16:15:12.747922+0000","last_clean":"2024-06-28T16:15:12.747922+0000","last_became_active":"2024-06-28T15:00:19.390613+0000","last_became_peered":"2024-06-28T15:00:19.390613+0000","last_unstale":"2024-06-28T16:15:12.747922+0000","last_undegraded":"2024-06-28T16:15:12.747922+0000","last_fullsized":"2024-06-28T16:15:12.747922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:12.747667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:12.747667+0000","last_clean_scrub_stamp":"2024-06-28T16:15:12.747667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:15:33.069119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"260'331","reported_seq":949,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:22.930866+0000","last_change":"2024-06-28T16:15:22.930866+0000","last_active":"2024-06-28T16:15:22.930866+0000","last_peered":"2024-06-28T16:15:22.930866+0000","last_clean":"2024-06-28T16:15:22.930866+0000","last_became_active":"2024-06-28T15:00:21.391631+0000","last_became_peered":"2024-06-28T15:00:21.391631+0000","last_unstale":"2024-06-28T16:15:22.930866+0000","last_undegraded":"2024-06-28T16:15:22.930866+0000","last_fullsized":"2024-06-28T16:15:22.930866+0000","mapping_epoch":25,"log_start":"260'300","ondisk_log_start":"260'300","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'331","last_scrub_stamp":"2024-06-28T16:15:22.930580+0000","last_deep_scrub":"260'331","last_deep_scrub_stamp":"2024-06-28T16:15:22.930580+0000","last_clean_scrub_stamp":"2024-06-28T16:15:22.930580+0000","objects_scrubbed":7,"log_size":31,"log_dups_size":300,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:45:49.759442+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43414,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":516,"num_read_kb":1105,"num_write":400,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":510,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.214355+0000","last_change":"2024-06-28T16:15:01.214355+0000","last_active":"2024-06-28T16:15:01.214355+0000","last_peered":"2024-06-28T16:15:01.214355+0000","last_clean":"2024-06-28T16:15:01.214355+0000","last_became_active":"2024-06-28T15:00:23.404312+0000","last_became_peered":"2024-06-28T15:00:23.404312+0000","last_unstale":"2024-06-28T16:15:01.214355+0000","last_undegraded":"2024-06-28T16:15:01.214355+0000","last_fullsized":"2024-06-28T16:15:01.214355+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:01.214103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:01.214103+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.214103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:21:36.177910+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"260'136","reported_seq":657,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.687390+0000","last_change":"2024-06-28T16:14:57.687390+0000","last_active":"2024-06-28T16:14:57.687390+0000","last_peered":"2024-06-28T16:14:57.687390+0000","last_clean":"2024-06-28T16:14:57.687390+0000","last_became_active":"2024-06-28T15:00:25.424826+0000","last_became_peered":"2024-06-28T15:00:25.424826+0000","last_unstale":"2024-06-28T16:14:57.687390+0000","last_undegraded":"2024-06-28T16:14:57.687390+0000","last_fullsized":"2024-06-28T16:14:57.687390+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'136","last_scrub_stamp":"2024-06-28T16:14:57.687124+0000","last_deep_scrub":"260'136","last_deep_scrub_stamp":"2024-06-28T16:14:57.687124+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.687124+0000","objects_scrubbed":5,"log_size":36,"log_dups_size":100,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:05:03.591761+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1205,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":159,"num_read_kb":144,"num_write":309,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"260'118","reported_seq":701,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.933025+0000","last_change":"2024-06-28T16:15:19.933025+0000","last_active":"2024-06-28T16:15:19.933025+0000","last_peered":"2024-06-28T16:15:19.933025+0000","last_clean":"2024-06-28T16:15:19.933025+0000","last_became_active":"2024-06-28T15:00:17.386756+0000","last_became_peered":"2024-06-28T15:00:17.386756+0000","last_unstale":"2024-06-28T16:15:19.933025+0000","last_undegraded":"2024-06-28T16:15:19.933025+0000","last_fullsized":"2024-06-28T16:15:19.933025+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'118","last_scrub_stamp":"2024-06-28T16:15:19.932659+0000","last_deep_scrub":"260'118","last_deep_scrub_stamp":"2024-06-28T16:15:19.932659+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.932659+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":100,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:41:38.014558+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":469,"num_read_kb":378,"num_write":166,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"260'293","reported_seq":974,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:48.180832+0000","last_change":"2024-06-28T16:14:48.180832+0000","last_active":"2024-06-28T16:14:48.180832+0000","last_peered":"2024-06-28T16:14:48.180832+0000","last_clean":"2024-06-28T16:14:48.180832+0000","last_became_active":"2024-06-28T15:00:59.943670+0000","last_became_peered":"2024-06-28T15:00:59.943670+0000","last_unstale":"2024-06-28T16:14:48.180832+0000","last_undegraded":"2024-06-28T16:14:48.180832+0000","last_fullsized":"2024-06-28T16:14:48.180832+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'293","last_scrub_stamp":"2024-06-28T16:14:48.180366+0000","last_deep_scrub":"260'293","last_deep_scrub_stamp":"2024-06-28T16:14:48.180366+0000","last_clean_scrub_stamp":"2024-06-28T16:14:48.180366+0000","objects_scrubbed":3,"log_size":93,"log_dups_size":200,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:11:28.293389+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306233,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":513,"num_read_kb":66858,"num_write":1091,"num_write_kb":26064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,1,6],"acting":[2,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":511,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.644404+0000","last_change":"2024-06-28T16:15:04.644404+0000","last_active":"2024-06-28T16:15:04.644404+0000","last_peered":"2024-06-28T16:15:04.644404+0000","last_clean":"2024-06-28T16:15:04.644404+0000","last_became_active":"2024-06-28T15:00:19.398245+0000","last_became_peered":"2024-06-28T15:00:19.398245+0000","last_unstale":"2024-06-28T16:15:04.644404+0000","last_undegraded":"2024-06-28T16:15:04.644404+0000","last_fullsized":"2024-06-28T16:15:04.644404+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:04.644148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:04.644148+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.644148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:23:05.800918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"260'301","reported_seq":901,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:32.343362+0000","last_change":"2024-06-28T16:15:32.343498+0000","last_active":"2024-06-28T16:15:32.343362+0000","last_peered":"2024-06-28T16:15:32.343362+0000","last_clean":"2024-06-28T16:15:32.343362+0000","last_became_active":"2024-06-28T15:00:21.404701+0000","last_became_peered":"2024-06-28T15:00:21.404701+0000","last_unstale":"2024-06-28T16:15:32.343362+0000","last_undegraded":"2024-06-28T16:15:32.343362+0000","last_fullsized":"2024-06-28T16:15:32.343362+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'301","last_scrub_stamp":"2024-06-28T16:15:32.343097+0000","last_deep_scrub":"260'301","last_deep_scrub_stamp":"2024-06-28T16:15:32.343097+0000","last_clean_scrub_stamp":"2024-06-28T16:15:32.343097+0000","objects_scrubbed":4,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:19:20.550850+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44354,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":486,"num_read_kb":1147,"num_write":394,"num_write_kb":305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":501,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.234419+0000","last_change":"2024-06-28T16:14:52.234419+0000","last_active":"2024-06-28T16:14:52.234419+0000","last_peered":"2024-06-28T16:14:52.234419+0000","last_clean":"2024-06-28T16:14:52.234419+0000","last_became_active":"2024-06-28T15:00:23.415531+0000","last_became_peered":"2024-06-28T15:00:23.415531+0000","last_unstale":"2024-06-28T16:14:52.234419+0000","last_undegraded":"2024-06-28T16:14:52.234419+0000","last_fullsized":"2024-06-28T16:14:52.234419+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:52.234179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:52.234179+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.234179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:09:52.052832+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.16","version":"260'168","reported_seq":722,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.154699+0000","last_change":"2024-06-28T16:14:52.154699+0000","last_active":"2024-06-28T16:14:52.154699+0000","last_peered":"2024-06-28T16:14:52.154699+0000","last_clean":"2024-06-28T16:14:52.154699+0000","last_became_active":"2024-06-28T15:00:25.553724+0000","last_became_peered":"2024-06-28T15:00:25.553724+0000","last_unstale":"2024-06-28T16:14:52.154699+0000","last_undegraded":"2024-06-28T16:14:52.154699+0000","last_fullsized":"2024-06-28T16:14:52.154699+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'168","last_scrub_stamp":"2024-06-28T16:14:52.154408+0000","last_deep_scrub":"260'168","last_deep_scrub_stamp":"2024-06-28T16:14:52.154408+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.154408+0000","objects_scrubbed":2,"log_size":68,"log_dups_size":100,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:26:53.092282+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":555,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":239,"num_read_kb":217,"num_write":397,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"260'143","reported_seq":816,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.153578+0000","last_change":"2024-06-28T16:15:11.153578+0000","last_active":"2024-06-28T16:15:11.153578+0000","last_peered":"2024-06-28T16:15:11.153578+0000","last_clean":"2024-06-28T16:15:11.153578+0000","last_became_active":"2024-06-28T15:00:17.383224+0000","last_became_peered":"2024-06-28T15:00:17.383224+0000","last_unstale":"2024-06-28T16:15:11.153578+0000","last_undegraded":"2024-06-28T16:15:11.153578+0000","last_fullsized":"2024-06-28T16:15:11.153578+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'143","last_scrub_stamp":"2024-06-28T16:15:11.153330+0000","last_deep_scrub":"260'143","last_deep_scrub_stamp":"2024-06-28T16:15:11.153330+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.153330+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":100,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:59:15.862953+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":649,"num_read_kb":545,"num_write":214,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"260'301","reported_seq":1075,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:24.365485+0000","last_change":"2024-06-28T16:15:24.365603+0000","last_active":"2024-06-28T16:15:24.365485+0000","last_peered":"2024-06-28T16:15:24.365485+0000","last_clean":"2024-06-28T16:15:24.365485+0000","last_became_active":"2024-06-28T15:00:15.423813+0000","last_became_peered":"2024-06-28T15:00:15.423813+0000","last_unstale":"2024-06-28T16:15:24.365485+0000","last_undegraded":"2024-06-28T16:15:24.365485+0000","last_fullsized":"2024-06-28T16:15:24.365485+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'301","last_scrub_stamp":"2024-06-28T16:15:24.365252+0000","last_deep_scrub":"260'301","last_deep_scrub_stamp":"2024-06-28T16:15:24.365252+0000","last_clean_scrub_stamp":"2024-06-28T16:15:24.365252+0000","objects_scrubbed":6,"log_size":101,"log_dups_size":200,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:53:47.692150+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5724260,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":754,"num_read_kb":37474,"num_write":1107,"num_write_kb":31583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7,2],"acting":[3,6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":513,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.014004+0000","last_change":"2024-06-28T16:15:13.014004+0000","last_active":"2024-06-28T16:15:13.014004+0000","last_peered":"2024-06-28T16:15:13.014004+0000","last_clean":"2024-06-28T16:15:13.014004+0000","last_became_active":"2024-06-28T15:00:19.383307+0000","last_became_peered":"2024-06-28T15:00:19.383307+0000","last_unstale":"2024-06-28T16:15:13.014004+0000","last_undegraded":"2024-06-28T16:15:13.014004+0000","last_fullsized":"2024-06-28T16:15:13.014004+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:15:13.013754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:15:13.013754+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.013754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:17:26.792620+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"260'289","reported_seq":870,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:35.247053+0000","last_change":"2024-06-28T16:15:35.247132+0000","last_active":"2024-06-28T16:15:35.247053+0000","last_peered":"2024-06-28T16:15:35.247053+0000","last_clean":"2024-06-28T16:15:35.247053+0000","last_became_active":"2024-06-28T15:00:21.396257+0000","last_became_peered":"2024-06-28T15:00:21.396257+0000","last_unstale":"2024-06-28T16:15:35.247053+0000","last_undegraded":"2024-06-28T16:15:35.247053+0000","last_fullsized":"2024-06-28T16:15:35.247053+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'289","last_scrub_stamp":"2024-06-28T16:15:35.246851+0000","last_deep_scrub":"260'289","last_deep_scrub_stamp":"2024-06-28T16:15:35.246851+0000","last_clean_scrub_stamp":"2024-06-28T16:15:35.246851+0000","objects_scrubbed":3,"log_size":89,"log_dups_size":200,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:29:48.575801+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43992,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":439,"num_read_kb":1078,"num_write":366,"num_write_kb":297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:47.973645+0000","last_change":"2024-06-28T16:14:47.973645+0000","last_active":"2024-06-28T16:14:47.973645+0000","last_peered":"2024-06-28T16:14:47.973645+0000","last_clean":"2024-06-28T16:14:47.973645+0000","last_became_active":"2024-06-28T15:00:23.409107+0000","last_became_peered":"2024-06-28T15:00:23.409107+0000","last_unstale":"2024-06-28T16:14:47.973645+0000","last_undegraded":"2024-06-28T16:14:47.973645+0000","last_fullsized":"2024-06-28T16:14:47.973645+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:47.973365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:47.973365+0000","last_clean_scrub_stamp":"2024-06-28T16:14:47.973365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:58:08.598920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"260'149","reported_seq":891,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.717817+0000","last_change":"2024-06-28T16:14:58.717817+0000","last_active":"2024-06-28T16:14:58.717817+0000","last_peered":"2024-06-28T16:14:58.717817+0000","last_clean":"2024-06-28T16:14:58.717817+0000","last_became_active":"2024-06-28T15:00:25.424376+0000","last_became_peered":"2024-06-28T15:00:25.424376+0000","last_unstale":"2024-06-28T16:14:58.717817+0000","last_undegraded":"2024-06-28T16:14:58.717817+0000","last_fullsized":"2024-06-28T16:14:58.717817+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'149","last_scrub_stamp":"2024-06-28T16:14:58.717559+0000","last_deep_scrub":"260'149","last_deep_scrub_stamp":"2024-06-28T16:14:58.717559+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.717559+0000","objects_scrubbed":5,"log_size":49,"log_dups_size":100,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:45:24.602655+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":398,"num_read_kb":429,"num_write":339,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1200,"num_omap_keys":6,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.18","version":"260'112","reported_seq":709,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.936863+0000","last_change":"2024-06-28T16:15:01.936863+0000","last_active":"2024-06-28T16:15:01.936863+0000","last_peered":"2024-06-28T16:15:01.936863+0000","last_clean":"2024-06-28T16:15:01.936863+0000","last_became_active":"2024-06-28T15:00:17.403007+0000","last_became_peered":"2024-06-28T15:00:17.403007+0000","last_unstale":"2024-06-28T16:15:01.936863+0000","last_undegraded":"2024-06-28T16:15:01.936863+0000","last_fullsized":"2024-06-28T16:15:01.936863+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'112","last_scrub_stamp":"2024-06-28T16:15:01.936598+0000","last_deep_scrub":"260'112","last_deep_scrub_stamp":"2024-06-28T16:15:01.936598+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.936598+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":100,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:50:31.559519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":478,"num_read_kb":386,"num_write":182,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"260'379","reported_seq":1056,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:21.180633+0000","last_change":"2024-06-28T16:15:21.180834+0000","last_active":"2024-06-28T16:15:21.180633+0000","last_peered":"2024-06-28T16:15:21.180633+0000","last_clean":"2024-06-28T16:15:21.180633+0000","last_became_active":"2024-06-28T15:00:15.414969+0000","last_became_peered":"2024-06-28T15:00:15.414969+0000","last_unstale":"2024-06-28T16:15:21.180633+0000","last_undegraded":"2024-06-28T16:15:21.180633+0000","last_fullsized":"2024-06-28T16:15:21.180633+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'379","last_scrub_stamp":"2024-06-28T16:15:21.180331+0000","last_deep_scrub":"260'379","last_deep_scrub_stamp":"2024-06-28T16:15:21.180331+0000","last_clean_scrub_stamp":"2024-06-28T16:15:21.180331+0000","objects_scrubbed":9,"log_size":79,"log_dups_size":300,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:59:23.438744+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1604505,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":539,"num_read_kb":3182,"num_write":1321,"num_write_kb":13350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,4,7],"acting":[0,5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"260'2","reported_seq":519,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:52.947126+0000","last_change":"2024-06-28T16:14:52.947126+0000","last_active":"2024-06-28T16:14:52.947126+0000","last_peered":"2024-06-28T16:14:52.947126+0000","last_clean":"2024-06-28T16:14:52.947126+0000","last_became_active":"2024-06-28T15:00:19.403344+0000","last_became_peered":"2024-06-28T15:00:19.403344+0000","last_unstale":"2024-06-28T16:14:52.947126+0000","last_undegraded":"2024-06-28T16:14:52.947126+0000","last_fullsized":"2024-06-28T16:14:52.947126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-06-28T16:14:52.946822+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-06-28T16:14:52.946822+0000","last_clean_scrub_stamp":"2024-06-28T16:14:52.946822+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:53:37.784932+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"260'297","reported_seq":907,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:55.943802+0000","last_change":"2024-06-28T16:14:55.943802+0000","last_active":"2024-06-28T16:14:55.943802+0000","last_peered":"2024-06-28T16:14:55.943802+0000","last_clean":"2024-06-28T16:14:55.943802+0000","last_became_active":"2024-06-28T15:00:21.387539+0000","last_became_peered":"2024-06-28T15:00:21.387539+0000","last_unstale":"2024-06-28T16:14:55.943802+0000","last_undegraded":"2024-06-28T16:14:55.943802+0000","last_fullsized":"2024-06-28T16:14:55.943802+0000","mapping_epoch":25,"log_start":"260'200","ondisk_log_start":"260'200","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'297","last_scrub_stamp":"2024-06-28T16:14:55.943503+0000","last_deep_scrub":"260'297","last_deep_scrub_stamp":"2024-06-28T16:14:55.943503+0000","last_clean_scrub_stamp":"2024-06-28T16:14:55.943503+0000","objects_scrubbed":5,"log_size":97,"log_dups_size":200,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:01:20.919128+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44825,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":494,"num_read_kb":1086,"num_write":388,"num_write_kb":274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":509,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.070879+0000","last_change":"2024-06-28T16:14:56.070879+0000","last_active":"2024-06-28T16:14:56.070879+0000","last_peered":"2024-06-28T16:14:56.070879+0000","last_clean":"2024-06-28T16:14:56.070879+0000","last_became_active":"2024-06-28T15:00:23.416363+0000","last_became_peered":"2024-06-28T15:00:23.416363+0000","last_unstale":"2024-06-28T16:14:56.070879+0000","last_undegraded":"2024-06-28T16:14:56.070879+0000","last_fullsized":"2024-06-28T16:14:56.070879+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-28T16:14:56.070609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-28T16:14:56.070609+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.070609+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:44:31.199646+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"260'129","reported_seq":636,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:43.940870+0000","last_change":"2024-06-28T16:14:43.940870+0000","last_active":"2024-06-28T16:14:43.940870+0000","last_peered":"2024-06-28T16:14:43.940870+0000","last_clean":"2024-06-28T16:14:43.940870+0000","last_became_active":"2024-06-28T15:00:25.428959+0000","last_became_peered":"2024-06-28T15:00:25.428959+0000","last_unstale":"2024-06-28T16:14:43.940870+0000","last_undegraded":"2024-06-28T16:14:43.940870+0000","last_fullsized":"2024-06-28T16:14:43.940870+0000","mapping_epoch":29,"log_start":"260'100","ondisk_log_start":"260'100","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'129","last_scrub_stamp":"2024-06-28T16:14:43.940462+0000","last_deep_scrub":"260'129","last_deep_scrub_stamp":"2024-06-28T16:14:43.940462+0000","last_clean_scrub_stamp":"2024-06-28T16:14:43.940462+0000","objects_scrubbed":3,"log_size":29,"log_dups_size":100,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:17:21.534335+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":814,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":97,"num_write":299,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.27","version":"260'175","reported_seq":771,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:56.005191+0000","last_change":"2024-06-28T16:14:56.005265+0000","last_active":"2024-06-28T16:14:56.005191+0000","last_peered":"2024-06-28T16:14:56.005191+0000","last_clean":"2024-06-28T16:14:56.005191+0000","last_became_active":"2024-06-28T15:00:17.406872+0000","last_became_peered":"2024-06-28T15:00:17.406872+0000","last_unstale":"2024-06-28T16:14:56.005191+0000","last_undegraded":"2024-06-28T16:14:56.005191+0000","last_fullsized":"2024-06-28T16:14:56.005191+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'175","last_scrub_stamp":"2024-06-28T16:14:56.004971+0000","last_deep_scrub":"260'175","last_deep_scrub_stamp":"2024-06-28T16:14:56.004971+0000","last_clean_scrub_stamp":"2024-06-28T16:14:56.004971+0000","objects_scrubbed":1,"log_size":75,"log_dups_size":100,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:44:14.471029+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":660,"num_read_kb":511,"num_write":250,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7564,"num_omap_keys":32,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"260'285","reported_seq":919,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:07.394078+0000","last_change":"2024-06-28T16:15:07.394078+0000","last_active":"2024-06-28T16:15:07.394078+0000","last_peered":"2024-06-28T16:15:07.394078+0000","last_clean":"2024-06-28T16:15:07.394078+0000","last_became_active":"2024-06-28T15:00:15.408337+0000","last_became_peered":"2024-06-28T15:00:15.408337+0000","last_unstale":"2024-06-28T16:15:07.394078+0000","last_undegraded":"2024-06-28T16:15:07.394078+0000","last_fullsized":"2024-06-28T16:15:07.394078+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'285","last_scrub_stamp":"2024-06-28T16:15:07.393622+0000","last_deep_scrub":"260'285","last_deep_scrub_stamp":"2024-06-28T16:15:07.393622+0000","last_clean_scrub_stamp":"2024-06-28T16:15:07.393622+0000","objects_scrubbed":5,"log_size":85,"log_dups_size":200,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:14:07.466937+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":404,"num_read_kb":33775,"num_write":1000,"num_write_kb":10945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4,5],"acting":[6,0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"260'72","reported_seq":683,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:59.969785+0000","last_change":"2024-06-28T16:14:59.969851+0000","last_active":"2024-06-28T16:14:59.969785+0000","last_peered":"2024-06-28T16:14:59.969785+0000","last_clean":"2024-06-28T16:14:59.969785+0000","last_became_active":"2024-06-28T15:00:17.403938+0000","last_became_peered":"2024-06-28T15:00:17.403938+0000","last_unstale":"2024-06-28T16:14:59.969785+0000","last_undegraded":"2024-06-28T16:14:59.969785+0000","last_fullsized":"2024-06-28T16:14:59.969785+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'72","last_scrub_stamp":"2024-06-28T16:14:59.969594+0000","last_deep_scrub":"260'72","last_deep_scrub_stamp":"2024-06-28T16:14:59.969594+0000","last_clean_scrub_stamp":"2024-06-28T16:14:59.969594+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:54:37.592959+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":319,"num_read_kb":277,"num_write":96,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"260'322","reported_seq":1005,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:57.267065+0000","last_change":"2024-06-28T16:14:57.267065+0000","last_active":"2024-06-28T16:14:57.267065+0000","last_peered":"2024-06-28T16:14:57.267065+0000","last_clean":"2024-06-28T16:14:57.267065+0000","last_became_active":"2024-06-28T15:00:15.420261+0000","last_became_peered":"2024-06-28T15:00:15.420261+0000","last_unstale":"2024-06-28T16:14:57.267065+0000","last_undegraded":"2024-06-28T16:14:57.267065+0000","last_fullsized":"2024-06-28T16:14:57.267065+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'322","last_scrub_stamp":"2024-06-28T16:14:57.266750+0000","last_deep_scrub":"260'322","last_deep_scrub_stamp":"2024-06-28T16:14:57.266750+0000","last_clean_scrub_stamp":"2024-06-28T16:14:57.266750+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":300,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:20:49.339058+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262189,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":529,"num_read_kb":23662,"num_write":1259,"num_write_kb":28174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6,7],"acting":[2,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"260'38","reported_seq":605,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:00.589352+0000","last_change":"2024-06-28T16:15:00.589416+0000","last_active":"2024-06-28T16:15:00.589352+0000","last_peered":"2024-06-28T16:15:00.589352+0000","last_clean":"2024-06-28T16:15:00.589352+0000","last_became_active":"2024-06-28T15:00:17.393889+0000","last_became_peered":"2024-06-28T15:00:17.393889+0000","last_unstale":"2024-06-28T16:15:00.589352+0000","last_undegraded":"2024-06-28T16:15:00.589352+0000","last_fullsized":"2024-06-28T16:15:00.589352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'38","last_scrub_stamp":"2024-06-28T16:15:00.589144+0000","last_deep_scrub":"260'38","last_deep_scrub_stamp":"2024-06-28T16:15:00.589144+0000","last_clean_scrub_stamp":"2024-06-28T16:15:00.589144+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:04:36.901823+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":178,"num_read_kb":152,"num_write":58,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"260'358","reported_seq":1009,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:20.874114+0000","last_change":"2024-06-28T16:15:20.874504+0000","last_active":"2024-06-28T16:15:20.874114+0000","last_peered":"2024-06-28T16:15:20.874114+0000","last_clean":"2024-06-28T16:15:20.874114+0000","last_became_active":"2024-06-28T15:00:15.418773+0000","last_became_peered":"2024-06-28T15:00:15.418773+0000","last_unstale":"2024-06-28T16:15:20.874114+0000","last_undegraded":"2024-06-28T16:15:20.874114+0000","last_fullsized":"2024-06-28T16:15:20.874114+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'358","last_scrub_stamp":"2024-06-28T16:15:20.873913+0000","last_deep_scrub":"260'358","last_deep_scrub_stamp":"2024-06-28T16:15:20.873913+0000","last_clean_scrub_stamp":"2024-06-28T16:15:20.873913+0000","objects_scrubbed":9,"log_size":58,"log_dups_size":300,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T20:50:36.816524+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8714448,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":553,"num_read_kb":16492,"num_write":1257,"num_write_kb":37641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,0,6],"acting":[5,3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"260'142","reported_seq":796,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.147070+0000","last_change":"2024-06-28T16:15:06.147193+0000","last_active":"2024-06-28T16:15:06.147070+0000","last_peered":"2024-06-28T16:15:06.147070+0000","last_clean":"2024-06-28T16:15:06.147070+0000","last_became_active":"2024-06-28T15:00:17.398357+0000","last_became_peered":"2024-06-28T15:00:17.398357+0000","last_unstale":"2024-06-28T16:15:06.147070+0000","last_undegraded":"2024-06-28T16:15:06.147070+0000","last_fullsized":"2024-06-28T16:15:06.147070+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'142","last_scrub_stamp":"2024-06-28T16:15:06.146782+0000","last_deep_scrub":"260'142","last_deep_scrub_stamp":"2024-06-28T16:15:06.146782+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.146782+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":100,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:02:03.196875+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":623,"num_read_kb":529,"num_write":221,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"260'332","reported_seq":986,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:24.148018+0000","last_change":"2024-06-28T16:15:24.148018+0000","last_active":"2024-06-28T16:15:24.148018+0000","last_peered":"2024-06-28T16:15:24.148018+0000","last_clean":"2024-06-28T16:15:24.148018+0000","last_became_active":"2024-06-28T15:00:15.391150+0000","last_became_peered":"2024-06-28T15:00:15.391150+0000","last_unstale":"2024-06-28T16:15:24.148018+0000","last_undegraded":"2024-06-28T16:15:24.148018+0000","last_fullsized":"2024-06-28T16:15:24.148018+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'332","last_scrub_stamp":"2024-06-28T16:15:24.147482+0000","last_deep_scrub":"260'332","last_deep_scrub_stamp":"2024-06-28T16:15:24.147482+0000","last_clean_scrub_stamp":"2024-06-28T16:15:24.147482+0000","objects_scrubbed":9,"log_size":32,"log_dups_size":300,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:55:18.493042+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1605357,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":453,"num_read_kb":84462,"num_write":1265,"num_write_kb":41774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4,1],"acting":[0,3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"260'192","reported_seq":798,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:42.132066+0000","last_change":"2024-06-28T16:14:42.132111+0000","last_active":"2024-06-28T16:14:42.132066+0000","last_peered":"2024-06-28T16:14:42.132066+0000","last_clean":"2024-06-28T16:14:42.132066+0000","last_became_active":"2024-06-28T15:00:59.806725+0000","last_became_peered":"2024-06-28T15:00:59.806725+0000","last_unstale":"2024-06-28T16:14:42.132066+0000","last_undegraded":"2024-06-28T16:14:42.132066+0000","last_fullsized":"2024-06-28T16:14:42.132066+0000","mapping_epoch":32,"log_start":"257'100","ondisk_log_start":"257'100","created":21,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'192","last_scrub_stamp":"2024-06-28T16:14:42.131976+0000","last_deep_scrub":"260'192","last_deep_scrub_stamp":"2024-06-28T16:14:42.131976+0000","last_clean_scrub_stamp":"2024-06-28T16:14:42.131976+0000","objects_scrubbed":1,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:22:19.905274+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":759,"num_read_kb":602,"num_write":307,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.24","version":"260'334","reported_seq":1007,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:29.347623+0000","last_change":"2024-06-28T16:15:29.347949+0000","last_active":"2024-06-28T16:15:29.347623+0000","last_peered":"2024-06-28T16:15:29.347623+0000","last_clean":"2024-06-28T16:15:29.347623+0000","last_became_active":"2024-06-28T15:00:15.383350+0000","last_became_peered":"2024-06-28T15:00:15.383350+0000","last_unstale":"2024-06-28T16:15:29.347623+0000","last_undegraded":"2024-06-28T16:15:29.347623+0000","last_fullsized":"2024-06-28T16:15:29.347623+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'334","last_scrub_stamp":"2024-06-28T16:15:29.347354+0000","last_deep_scrub":"260'334","last_deep_scrub_stamp":"2024-06-28T16:15:29.347354+0000","last_clean_scrub_stamp":"2024-06-28T16:15:29.347354+0000","objects_scrubbed":9,"log_size":34,"log_dups_size":300,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T22:10:43.722259+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263219,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":502,"num_read_kb":41336,"num_write":1201,"num_write_kb":38478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0,7],"acting":[3,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"260'398","reported_seq":1008,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.938126+0000","last_change":"2024-06-28T16:14:58.938126+0000","last_active":"2024-06-28T16:14:58.938126+0000","last_peered":"2024-06-28T16:14:58.938126+0000","last_clean":"2024-06-28T16:14:58.938126+0000","last_became_active":"2024-06-28T15:00:17.404844+0000","last_became_peered":"2024-06-28T15:00:17.404844+0000","last_unstale":"2024-06-28T16:14:58.938126+0000","last_undegraded":"2024-06-28T16:14:58.938126+0000","last_fullsized":"2024-06-28T16:14:58.938126+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'398","last_scrub_stamp":"2024-06-28T16:14:58.937867+0000","last_deep_scrub":"260'398","last_deep_scrub_stamp":"2024-06-28T16:14:58.937867+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.937867+0000","objects_scrubbed":1,"log_size":98,"log_dups_size":300,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:18:27.056087+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1527,"num_read_kb":1156,"num_write":658,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":268,"num_omap_keys":1,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"260'343","reported_seq":1036,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.134211+0000","last_change":"2024-06-28T16:15:09.134294+0000","last_active":"2024-06-28T16:15:09.134211+0000","last_peered":"2024-06-28T16:15:09.134211+0000","last_clean":"2024-06-28T16:15:09.134211+0000","last_became_active":"2024-06-28T15:00:15.401275+0000","last_became_peered":"2024-06-28T15:00:15.401275+0000","last_unstale":"2024-06-28T16:15:09.134211+0000","last_undegraded":"2024-06-28T16:15:09.134211+0000","last_fullsized":"2024-06-28T16:15:09.134211+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'343","last_scrub_stamp":"2024-06-28T16:15:09.133999+0000","last_deep_scrub":"260'343","last_deep_scrub_stamp":"2024-06-28T16:15:09.133999+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.133999+0000","objects_scrubbed":3,"log_size":43,"log_dups_size":300,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:37:29.220333+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":559,"num_read_kb":45669,"num_write":1313,"num_write_kb":19663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,5],"acting":[1,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"260'221","reported_seq":851,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:14.135956+0000","last_change":"2024-06-28T16:15:14.135956+0000","last_active":"2024-06-28T16:15:14.135956+0000","last_peered":"2024-06-28T16:15:14.135956+0000","last_clean":"2024-06-28T16:15:14.135956+0000","last_became_active":"2024-06-28T15:00:17.382739+0000","last_became_peered":"2024-06-28T15:00:17.382739+0000","last_unstale":"2024-06-28T16:15:14.135956+0000","last_undegraded":"2024-06-28T16:15:14.135956+0000","last_fullsized":"2024-06-28T16:15:14.135956+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'221","last_scrub_stamp":"2024-06-28T16:15:14.135703+0000","last_deep_scrub":"260'221","last_deep_scrub_stamp":"2024-06-28T16:15:14.135703+0000","last_clean_scrub_stamp":"2024-06-28T16:15:14.135703+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":200,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:03:26.946056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":866,"num_read_kb":690,"num_write":335,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"260'298","reported_seq":937,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.436658+0000","last_change":"2024-06-28T16:15:10.436794+0000","last_active":"2024-06-28T16:15:10.436658+0000","last_peered":"2024-06-28T16:15:10.436658+0000","last_clean":"2024-06-28T16:15:10.436658+0000","last_became_active":"2024-06-28T15:00:15.421189+0000","last_became_peered":"2024-06-28T15:00:15.421189+0000","last_unstale":"2024-06-28T16:15:10.436658+0000","last_undegraded":"2024-06-28T16:15:10.436658+0000","last_fullsized":"2024-06-28T16:15:10.436658+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'298","last_scrub_stamp":"2024-06-28T16:15:10.436402+0000","last_deep_scrub":"260'298","last_deep_scrub_stamp":"2024-06-28T16:15:10.436402+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.436402+0000","objects_scrubbed":7,"log_size":98,"log_dups_size":200,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T00:19:16.974922+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7105903,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":419,"num_read_kb":19450,"num_write":1148,"num_write_kb":38199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4,5],"acting":[6,7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"260'80","reported_seq":650,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.448105+0000","last_change":"2024-06-28T16:15:03.448105+0000","last_active":"2024-06-28T16:15:03.448105+0000","last_peered":"2024-06-28T16:15:03.448105+0000","last_clean":"2024-06-28T16:15:03.448105+0000","last_became_active":"2024-06-28T15:00:17.396125+0000","last_became_peered":"2024-06-28T15:00:17.396125+0000","last_unstale":"2024-06-28T16:15:03.448105+0000","last_undegraded":"2024-06-28T16:15:03.448105+0000","last_fullsized":"2024-06-28T16:15:03.448105+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'80","last_scrub_stamp":"2024-06-28T16:15:03.447853+0000","last_deep_scrub":"260'80","last_deep_scrub_stamp":"2024-06-28T16:15:03.447853+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.447853+0000","objects_scrubbed":1,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:57:53.065744+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":321,"num_read_kb":257,"num_write":118,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"260'270","reported_seq":936,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:26.330845+0000","last_change":"2024-06-28T16:15:26.331055+0000","last_active":"2024-06-28T16:15:26.330845+0000","last_peered":"2024-06-28T16:15:26.330845+0000","last_clean":"2024-06-28T16:15:26.330845+0000","last_became_active":"2024-06-28T15:00:15.408375+0000","last_became_peered":"2024-06-28T15:00:15.408375+0000","last_unstale":"2024-06-28T16:15:26.330845+0000","last_undegraded":"2024-06-28T16:15:26.330845+0000","last_fullsized":"2024-06-28T16:15:26.330845+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'270","last_scrub_stamp":"2024-06-28T16:15:26.330596+0000","last_deep_scrub":"260'270","last_deep_scrub_stamp":"2024-06-28T16:15:26.330596+0000","last_clean_scrub_stamp":"2024-06-28T16:15:26.330596+0000","objects_scrubbed":5,"log_size":70,"log_dups_size":200,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:26:19.674184+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":484,"num_read_kb":6587,"num_write":1005,"num_write_kb":17135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,5],"acting":[3,4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"260'62","reported_seq":646,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:17.240437+0000","last_change":"2024-06-28T16:15:17.240573+0000","last_active":"2024-06-28T16:15:17.240437+0000","last_peered":"2024-06-28T16:15:17.240437+0000","last_clean":"2024-06-28T16:15:17.240437+0000","last_became_active":"2024-06-28T15:00:17.379374+0000","last_became_peered":"2024-06-28T15:00:17.379374+0000","last_unstale":"2024-06-28T16:15:17.240437+0000","last_undegraded":"2024-06-28T16:15:17.240437+0000","last_fullsized":"2024-06-28T16:15:17.240437+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'62","last_scrub_stamp":"2024-06-28T16:15:17.240048+0000","last_deep_scrub":"260'62","last_deep_scrub_stamp":"2024-06-28T16:15:17.240048+0000","last_clean_scrub_stamp":"2024-06-28T16:15:17.240048+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:17:53.458911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":216,"num_write":84,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"260'367","reported_seq":1020,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.676929+0000","last_change":"2024-06-28T16:15:11.677022+0000","last_active":"2024-06-28T16:15:11.676929+0000","last_peered":"2024-06-28T16:15:11.676929+0000","last_clean":"2024-06-28T16:15:11.676929+0000","last_became_active":"2024-06-28T15:00:15.427171+0000","last_became_peered":"2024-06-28T15:00:15.427171+0000","last_unstale":"2024-06-28T16:15:11.676929+0000","last_undegraded":"2024-06-28T16:15:11.676929+0000","last_fullsized":"2024-06-28T16:15:11.676929+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'367","last_scrub_stamp":"2024-06-28T16:15:11.676722+0000","last_deep_scrub":"260'367","last_deep_scrub_stamp":"2024-06-28T16:15:11.676722+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.676722+0000","objects_scrubbed":4,"log_size":67,"log_dups_size":300,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:32:35.302039+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":723921,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":536,"num_read_kb":26554,"num_write":1152,"num_write_kb":52254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2,6],"acting":[5,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"260'230","reported_seq":861,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:04.989876+0000","last_change":"2024-06-28T16:15:04.989943+0000","last_active":"2024-06-28T16:15:04.989876+0000","last_peered":"2024-06-28T16:15:04.989876+0000","last_clean":"2024-06-28T16:15:04.989876+0000","last_became_active":"2024-06-28T15:00:17.403760+0000","last_became_peered":"2024-06-28T15:00:17.403760+0000","last_unstale":"2024-06-28T16:15:04.989876+0000","last_undegraded":"2024-06-28T16:15:04.989876+0000","last_fullsized":"2024-06-28T16:15:04.989876+0000","mapping_epoch":21,"log_start":"260'200","ondisk_log_start":"260'200","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'230","last_scrub_stamp":"2024-06-28T16:15:04.989667+0000","last_deep_scrub":"260'230","last_deep_scrub_stamp":"2024-06-28T16:15:04.989667+0000","last_clean_scrub_stamp":"2024-06-28T16:15:04.989667+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":200,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:46:58.817243+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":902,"num_read_kb":724,"num_write":338,"num_write_kb":163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"260'282","reported_seq":938,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:27.318871+0000","last_change":"2024-06-28T16:15:27.319004+0000","last_active":"2024-06-28T16:15:27.318871+0000","last_peered":"2024-06-28T16:15:27.318871+0000","last_clean":"2024-06-28T16:15:27.318871+0000","last_became_active":"2024-06-28T15:00:15.376014+0000","last_became_peered":"2024-06-28T15:00:15.376014+0000","last_unstale":"2024-06-28T16:15:27.318871+0000","last_undegraded":"2024-06-28T16:15:27.318871+0000","last_fullsized":"2024-06-28T16:15:27.318871+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'282","last_scrub_stamp":"2024-06-28T16:15:27.318626+0000","last_deep_scrub":"260'282","last_deep_scrub_stamp":"2024-06-28T16:15:27.318626+0000","last_clean_scrub_stamp":"2024-06-28T16:15:27.318626+0000","objects_scrubbed":7,"log_size":82,"log_dups_size":200,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T16:54:14.907511+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462789,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":409,"num_read_kb":32504,"num_write":910,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2,1],"acting":[3,4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"260'70","reported_seq":668,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:01.378898+0000","last_change":"2024-06-28T16:15:01.378961+0000","last_active":"2024-06-28T16:15:01.378898+0000","last_peered":"2024-06-28T16:15:01.378898+0000","last_clean":"2024-06-28T16:15:01.378898+0000","last_became_active":"2024-06-28T15:00:17.401917+0000","last_became_peered":"2024-06-28T15:00:17.401917+0000","last_unstale":"2024-06-28T16:15:01.378898+0000","last_undegraded":"2024-06-28T16:15:01.378898+0000","last_fullsized":"2024-06-28T16:15:01.378898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'70","last_scrub_stamp":"2024-06-28T16:15:01.378703+0000","last_deep_scrub":"260'70","last_deep_scrub_stamp":"2024-06-28T16:15:01.378703+0000","last_clean_scrub_stamp":"2024-06-28T16:15:01.378703+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:54:33.487185+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":295,"num_read_kb":247,"num_write":98,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"260'337","reported_seq":982,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.634962+0000","last_change":"2024-06-28T16:15:09.635188+0000","last_active":"2024-06-28T16:15:09.634962+0000","last_peered":"2024-06-28T16:15:09.634962+0000","last_clean":"2024-06-28T16:15:09.634962+0000","last_became_active":"2024-06-28T15:00:15.431444+0000","last_became_peered":"2024-06-28T15:00:15.431444+0000","last_unstale":"2024-06-28T16:15:09.634962+0000","last_undegraded":"2024-06-28T16:15:09.634962+0000","last_fullsized":"2024-06-28T16:15:09.634962+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'337","last_scrub_stamp":"2024-06-28T16:15:09.634716+0000","last_deep_scrub":"260'337","last_deep_scrub_stamp":"2024-06-28T16:15:09.634716+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.634716+0000","objects_scrubbed":6,"log_size":37,"log_dups_size":300,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:58:38.266578+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":461,"num_read_kb":557,"num_write":1266,"num_write_kb":33580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,3],"acting":[5,7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"260'424","reported_seq":1187,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.016891+0000","last_change":"2024-06-28T16:15:06.016973+0000","last_active":"2024-06-28T16:15:06.016891+0000","last_peered":"2024-06-28T16:15:06.016891+0000","last_clean":"2024-06-28T16:15:06.016891+0000","last_became_active":"2024-06-28T15:00:17.402842+0000","last_became_peered":"2024-06-28T15:00:17.402842+0000","last_unstale":"2024-06-28T16:15:06.016891+0000","last_undegraded":"2024-06-28T16:15:06.016891+0000","last_fullsized":"2024-06-28T16:15:06.016891+0000","mapping_epoch":21,"log_start":"260'400","ondisk_log_start":"260'400","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'424","last_scrub_stamp":"2024-06-28T16:15:06.016665+0000","last_deep_scrub":"260'424","last_deep_scrub_stamp":"2024-06-28T16:15:06.016665+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.016665+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":400,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T18:27:00.891642+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1825,"num_read_kb":1465,"num_write":755,"num_write_kb":468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"260'319","reported_seq":973,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:31.310383+0000","last_change":"2024-06-28T16:15:31.310482+0000","last_active":"2024-06-28T16:15:31.310383+0000","last_peered":"2024-06-28T16:15:31.310383+0000","last_clean":"2024-06-28T16:15:31.310383+0000","last_became_active":"2024-06-28T15:00:15.415009+0000","last_became_peered":"2024-06-28T15:00:15.415009+0000","last_unstale":"2024-06-28T16:15:31.310383+0000","last_undegraded":"2024-06-28T16:15:31.310383+0000","last_fullsized":"2024-06-28T16:15:31.310383+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'319","last_scrub_stamp":"2024-06-28T16:15:31.310182+0000","last_deep_scrub":"260'319","last_deep_scrub_stamp":"2024-06-28T16:15:31.310182+0000","last_clean_scrub_stamp":"2024-06-28T16:15:31.310182+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":300,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:45:14.627771+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5242944,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":479,"num_read_kb":2084,"num_write":1202,"num_write_kb":29565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,6],"acting":[0,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"260'26","reported_seq":583,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:11.742665+0000","last_change":"2024-06-28T16:15:11.742725+0000","last_active":"2024-06-28T16:15:11.742665+0000","last_peered":"2024-06-28T16:15:11.742665+0000","last_clean":"2024-06-28T16:15:11.742665+0000","last_became_active":"2024-06-28T15:00:17.389844+0000","last_became_peered":"2024-06-28T15:00:17.389844+0000","last_unstale":"2024-06-28T16:15:11.742665+0000","last_undegraded":"2024-06-28T16:15:11.742665+0000","last_fullsized":"2024-06-28T16:15:11.742665+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'26","last_scrub_stamp":"2024-06-28T16:15:11.742459+0000","last_deep_scrub":"260'26","last_deep_scrub_stamp":"2024-06-28T16:15:11.742459+0000","last_clean_scrub_stamp":"2024-06-28T16:15:11.742459+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T02:47:41.122636+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":112,"num_write":34,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"260'305","reported_seq":945,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:06.433717+0000","last_change":"2024-06-28T16:15:06.433887+0000","last_active":"2024-06-28T16:15:06.433717+0000","last_peered":"2024-06-28T16:15:06.433717+0000","last_clean":"2024-06-28T16:15:06.433717+0000","last_became_active":"2024-06-28T15:00:15.397986+0000","last_became_peered":"2024-06-28T15:00:15.397986+0000","last_unstale":"2024-06-28T16:15:06.433717+0000","last_undegraded":"2024-06-28T16:15:06.433717+0000","last_fullsized":"2024-06-28T16:15:06.433717+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'305","last_scrub_stamp":"2024-06-28T16:15:06.433513+0000","last_deep_scrub":"260'305","last_deep_scrub_stamp":"2024-06-28T16:15:06.433513+0000","last_clean_scrub_stamp":"2024-06-28T16:15:06.433513+0000","objects_scrubbed":7,"log_size":105,"log_dups_size":200,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T19:04:10.170914+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079926,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":433,"num_read_kb":10373,"num_write":1047,"num_write_kb":25421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"260'192","reported_seq":866,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:09.290301+0000","last_change":"2024-06-28T16:15:09.290366+0000","last_active":"2024-06-28T16:15:09.290301+0000","last_peered":"2024-06-28T16:15:09.290301+0000","last_clean":"2024-06-28T16:15:09.290301+0000","last_became_active":"2024-06-28T15:00:17.394783+0000","last_became_peered":"2024-06-28T15:00:17.394783+0000","last_unstale":"2024-06-28T16:15:09.290301+0000","last_undegraded":"2024-06-28T16:15:09.290301+0000","last_fullsized":"2024-06-28T16:15:09.290301+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'192","last_scrub_stamp":"2024-06-28T16:15:09.290105+0000","last_deep_scrub":"260'192","last_deep_scrub_stamp":"2024-06-28T16:15:09.290105+0000","last_clean_scrub_stamp":"2024-06-28T16:15:09.290105+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":100,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:44:41.337242+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":852,"num_read_kb":715,"num_write":335,"num_write_kb":202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"260'302","reported_seq":923,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:46.051125+0000","last_change":"2024-06-28T16:14:46.051125+0000","last_active":"2024-06-28T16:14:46.051125+0000","last_peered":"2024-06-28T16:14:46.051125+0000","last_clean":"2024-06-28T16:14:46.051125+0000","last_became_active":"2024-06-28T15:00:59.938004+0000","last_became_peered":"2024-06-28T15:00:59.938004+0000","last_unstale":"2024-06-28T16:14:46.051125+0000","last_undegraded":"2024-06-28T16:14:46.051125+0000","last_fullsized":"2024-06-28T16:14:46.051125+0000","mapping_epoch":32,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'302","last_scrub_stamp":"2024-06-28T16:14:46.050757+0000","last_deep_scrub":"260'302","last_deep_scrub_stamp":"2024-06-28T16:14:46.050757+0000","last_clean_scrub_stamp":"2024-06-28T16:14:46.050757+0000","objects_scrubbed":13,"log_size":102,"log_dups_size":200,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T23:23:03.771050+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6542288,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":437,"num_read_kb":25077,"num_write":1246,"num_write_kb":38037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6,3],"acting":[1,2,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"260'392","reported_seq":1071,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:22.358463+0000","last_change":"2024-06-28T16:15:22.358526+0000","last_active":"2024-06-28T16:15:22.358463+0000","last_peered":"2024-06-28T16:15:22.358463+0000","last_clean":"2024-06-28T16:15:22.358463+0000","last_became_active":"2024-06-28T15:00:17.374143+0000","last_became_peered":"2024-06-28T15:00:17.374143+0000","last_unstale":"2024-06-28T16:15:22.358463+0000","last_undegraded":"2024-06-28T16:15:22.358463+0000","last_fullsized":"2024-06-28T16:15:22.358463+0000","mapping_epoch":21,"log_start":"260'300","ondisk_log_start":"260'300","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'392","last_scrub_stamp":"2024-06-28T16:15:22.358270+0000","last_deep_scrub":"260'392","last_deep_scrub_stamp":"2024-06-28T16:15:22.358270+0000","last_clean_scrub_stamp":"2024-06-28T16:15:22.358270+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":300,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:17:15.649882+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1520,"num_read_kb":1191,"num_write":653,"num_write_kb":382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"260'255","reported_seq":928,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:14:58.309302+0000","last_change":"2024-06-28T16:14:58.309458+0000","last_active":"2024-06-28T16:14:58.309302+0000","last_peered":"2024-06-28T16:14:58.309302+0000","last_clean":"2024-06-28T16:14:58.309302+0000","last_became_active":"2024-06-28T15:00:15.430228+0000","last_became_peered":"2024-06-28T15:00:15.430228+0000","last_unstale":"2024-06-28T16:14:58.309302+0000","last_undegraded":"2024-06-28T16:14:58.309302+0000","last_fullsized":"2024-06-28T16:14:58.309302+0000","mapping_epoch":19,"log_start":"260'200","ondisk_log_start":"260'200","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'255","last_scrub_stamp":"2024-06-28T16:14:58.309078+0000","last_deep_scrub":"260'255","last_deep_scrub_stamp":"2024-06-28T16:14:58.309078+0000","last_clean_scrub_stamp":"2024-06-28T16:14:58.309078+0000","objects_scrubbed":6,"log_size":55,"log_dups_size":200,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:49:38.745040+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170399,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":449,"num_read_kb":6008,"num_write":895,"num_write_kb":11243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,1,7],"acting":[2,6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"260'134","reported_seq":787,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:13.792855+0000","last_change":"2024-06-28T16:15:13.793011+0000","last_active":"2024-06-28T16:15:13.792855+0000","last_peered":"2024-06-28T16:15:13.792855+0000","last_clean":"2024-06-28T16:15:13.792855+0000","last_became_active":"2024-06-28T15:00:17.381562+0000","last_became_peered":"2024-06-28T15:00:17.381562+0000","last_unstale":"2024-06-28T16:15:13.792855+0000","last_undegraded":"2024-06-28T16:15:13.792855+0000","last_fullsized":"2024-06-28T16:15:13.792855+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'134","last_scrub_stamp":"2024-06-28T16:15:13.792604+0000","last_deep_scrub":"260'134","last_deep_scrub_stamp":"2024-06-28T16:15:13.792604+0000","last_clean_scrub_stamp":"2024-06-28T16:15:13.792604+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":100,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:32:09.953292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":570,"num_read_kb":479,"num_write":190,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"260'496","reported_seq":1195,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:19.851543+0000","last_change":"2024-06-28T16:15:19.851688+0000","last_active":"2024-06-28T16:15:19.851543+0000","last_peered":"2024-06-28T16:15:19.851543+0000","last_clean":"2024-06-28T16:15:19.851543+0000","last_became_active":"2024-06-28T15:00:15.418850+0000","last_became_peered":"2024-06-28T15:00:15.418850+0000","last_unstale":"2024-06-28T16:15:19.851543+0000","last_undegraded":"2024-06-28T16:15:19.851543+0000","last_fullsized":"2024-06-28T16:15:19.851543+0000","mapping_epoch":19,"log_start":"260'400","ondisk_log_start":"260'400","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'496","last_scrub_stamp":"2024-06-28T16:15:19.851317+0000","last_deep_scrub":"260'496","last_deep_scrub_stamp":"2024-06-28T16:15:19.851317+0000","last_clean_scrub_stamp":"2024-06-28T16:15:19.851317+0000","objects_scrubbed":1,"log_size":96,"log_dups_size":400,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T01:36:52.931988+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":907,"num_read_kb":23220,"num_write":1415,"num_write_kb":30715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,4,6],"acting":[5,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"260'338","reported_seq":1032,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:16.827560+0000","last_change":"2024-06-28T16:15:16.827810+0000","last_active":"2024-06-28T16:15:16.827560+0000","last_peered":"2024-06-28T16:15:16.827560+0000","last_clean":"2024-06-28T16:15:16.827560+0000","last_became_active":"2024-06-28T15:00:15.408158+0000","last_became_peered":"2024-06-28T15:00:15.408158+0000","last_unstale":"2024-06-28T16:15:16.827560+0000","last_undegraded":"2024-06-28T16:15:16.827560+0000","last_fullsized":"2024-06-28T16:15:16.827560+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'338","last_scrub_stamp":"2024-06-28T16:15:16.827294+0000","last_deep_scrub":"260'338","last_deep_scrub_stamp":"2024-06-28T16:15:16.827294+0000","last_clean_scrub_stamp":"2024-06-28T16:15:16.827294+0000","objects_scrubbed":9,"log_size":38,"log_dups_size":300,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T21:01:49.282108+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6690738,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":568,"num_read_kb":38558,"num_write":1259,"num_write_kb":46809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,7],"acting":[5,1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"260'77","reported_seq":655,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:15.306055+0000","last_change":"2024-06-28T16:15:15.306055+0000","last_active":"2024-06-28T16:15:15.306055+0000","last_peered":"2024-06-28T16:15:15.306055+0000","last_clean":"2024-06-28T16:15:15.306055+0000","last_became_active":"2024-06-28T15:00:17.379771+0000","last_became_peered":"2024-06-28T15:00:17.379771+0000","last_unstale":"2024-06-28T16:15:15.306055+0000","last_undegraded":"2024-06-28T16:15:15.306055+0000","last_fullsized":"2024-06-28T16:15:15.306055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'77","last_scrub_stamp":"2024-06-28T16:15:15.305668+0000","last_deep_scrub":"260'77","last_deep_scrub_stamp":"2024-06-28T16:15:15.305668+0000","last_clean_scrub_stamp":"2024-06-28T16:15:15.305668+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T03:33:25.492967+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":308,"num_read_kb":249,"num_write":109,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"260'387","reported_seq":1111,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:10.144751+0000","last_change":"2024-06-28T16:15:10.144995+0000","last_active":"2024-06-28T16:15:10.144751+0000","last_peered":"2024-06-28T16:15:10.144751+0000","last_clean":"2024-06-28T16:15:10.144751+0000","last_became_active":"2024-06-28T15:00:15.401324+0000","last_became_peered":"2024-06-28T15:00:15.401324+0000","last_unstale":"2024-06-28T16:15:10.144751+0000","last_undegraded":"2024-06-28T16:15:10.144751+0000","last_fullsized":"2024-06-28T16:15:10.144751+0000","mapping_epoch":19,"log_start":"260'300","ondisk_log_start":"260'300","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'387","last_scrub_stamp":"2024-06-28T16:15:10.144446+0000","last_deep_scrub":"260'387","last_deep_scrub_stamp":"2024-06-28T16:15:10.144446+0000","last_clean_scrub_stamp":"2024-06-28T16:15:10.144446+0000","objects_scrubbed":9,"log_size":87,"log_dups_size":300,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-30T04:10:22.329753+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1079973,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":661,"num_read_kb":1946,"num_write":1429,"num_write_kb":27589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,5,0],"acting":[1,4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.36","version":"260'176","reported_seq":787,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-06-28T16:15:03.646038+0000","last_change":"2024-06-28T16:15:03.646038+0000","last_active":"2024-06-28T16:15:03.646038+0000","last_peered":"2024-06-28T16:15:03.646038+0000","last_clean":"2024-06-28T16:15:03.646038+0000","last_became_active":"2024-06-28T15:00:17.382774+0000","last_became_peered":"2024-06-28T15:00:17.382774+0000","last_unstale":"2024-06-28T16:15:03.646038+0000","last_undegraded":"2024-06-28T16:15:03.646038+0000","last_fullsized":"2024-06-28T16:15:03.646038+0000","mapping_epoch":21,"log_start":"260'100","ondisk_log_start":"260'100","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'176","last_scrub_stamp":"2024-06-28T16:15:03.645757+0000","last_deep_scrub":"260'176","last_deep_scrub_stamp":"2024-06-28T16:15:03.645757+0000","last_clean_scrub_stamp":"2024-06-28T16:15:03.645757+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":100,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-06-29T17:35:27.591692+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":719,"num_read_kb":570,"num_write":272,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":65,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5652,"num_read_kb":3987,"num_write":2073,"num_write_kb":817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7430,"num_omap_keys":15,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1080,"ondisk_log_size":1080,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":19748,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":13201,"num_read_kb":13000,"num_write":15927,"num_write_kb":4660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4270,"num_omap_keys":25,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":679936,"data_stored":39496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1835,"ondisk_log_size":1835,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1424454,"num_objects":198,"num_object_clones":0,"num_object_copies":396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":18162,"num_read_kb":37754,"num_write":15149,"num_write_kb":11215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":21,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":655336,"num_omap_keys":1530,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2056,"ondisk_log_size":2056,"up":64,"acting":64,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1793,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1644,"num_read_kb":1954,"num_write":22,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":87319,"num_read_kb":70822,"num_write":35298,"num_write_kb":17822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":43558,"num_omap_keys":182,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3548,"ondisk_log_size":3548,"up":128,"acting":128,"num_store_stats":8},{"poolid":68,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1128,"num_omap_keys":12,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":149605345,"num_objects":433,"num_object_clones":0,"num_object_copies":1732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":433,"num_whiteouts":0,"num_read":34839,"num_read_kb":1786510,"num_write":77776,"num_write_kb":1982911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202506240,"data_stored":202506240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3438,"ondisk_log_size":3438,"up":256,"acting":256,"num_store_stats":8},{"poolid":67,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":129,"num_read_kb":165927,"num_write":154,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":5242880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":66,"num_pg":32,"stat_sum":{"num_bytes":16777216,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":137,"num_read_kb":186344,"num_write":154,"num_write_kb":127968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33554432,"data_stored":33554432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575762,"num_pgs":115,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":356188,"kb_used_data":36244,"kb_used_omap":1,"kb_used_meta":319934,"kb_avail":94015652,"statfs":{"total":96636764160,"available":96272027648,"internally_reserved":0,"allocated":37113856,"data_stored":35129654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":327612874},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575762,"num_pgs":108,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323068,"kb_used_data":43380,"kb_used_omap":1,"kb_used_meta":279678,"kb_avail":94048772,"statfs":{"total":96636764160,"available":96305942528,"internally_reserved":0,"allocated":44421120,"data_stored":42495981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":286390729},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575764,"num_pgs":114,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201360,"kb_used_data":35592,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94170480,"statfs":{"total":96636764160,"available":96430571520,"internally_reserved":0,"allocated":36446208,"data_stored":34448502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575764,"num_pgs":113,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":146836,"kb_used_data":43468,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":94225004,"statfs":{"total":96636764160,"available":96486404096,"internally_reserved":0,"allocated":44511232,"data_stored":42596897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575767,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303252,"kb_used_data":29196,"kb_used_omap":1,"kb_used_meta":274046,"kb_avail":94068588,"statfs":{"total":96636764160,"available":96326234112,"internally_reserved":0,"allocated":29896704,"data_stored":27928906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":280623559},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575765,"num_pgs":99,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137284,"kb_used_data":29820,"kb_used_omap":1,"kb_used_meta":107454,"kb_avail":94234556,"statfs":{"total":96636764160,"available":96496185344,"internally_reserved":0,"allocated":30535680,"data_stored":28610104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110033349},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575762,"num_pgs":120,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":201576,"kb_used_data":39968,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94170264,"statfs":{"total":96636764160,"available":96430350336,"internally_reserved":0,"allocated":40927232,"data_stored":38982339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165476805},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575765,"num_pgs":116,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287396,"kb_used_data":39644,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":94084444,"statfs":{"total":96636764160,"available":96342470656,"internally_reserved":0,"allocated":40595456,"data_stored":38600327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":253688261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":25272320,"data_stored":25272320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":25198592,"data_stored":25198592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":21118976,"data_stored":21118976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19386368,"data_stored":19386368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":29863936,"data_stored":29863936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20664320,"data_stored":20664320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28712960,"data_stored":28712960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32288768,"data_stored":32288768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":7701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":4574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":4965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":4436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":5746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":65,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11534336,"data_stored":11534336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":66,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10485760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5242880,"data_stored":5242880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":67,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":68,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-28T16:16:03.847 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-06-28T16:16:04.206 INFO:teuthology.misc:Shutting down mds daemons... 2024-06-28T16:16:04.206 INFO:teuthology.misc:Shutting down osd daemons... 2024-06-28T16:16:04.207 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-06-28T16:16:04.207 INFO:teuthology.orchestra.run:waiting for 300 2024-06-28T16:16:06.798 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-06-28T16:16:10.309 INFO:tasks.ceph.osd.0:Stopped 2024-06-28T16:16:10.309 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-06-28T16:16:10.309 INFO:teuthology.orchestra.run:waiting for 300 2024-06-28T16:16:10.396 INFO:tasks.ceph.osd.1:Stopped 2024-06-28T16:16:10.397 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-06-28T16:16:10.397 INFO:teuthology.orchestra.run:waiting for 300 2024-06-28T16:16:12.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-06-28T16:16:12.807 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-06-28T16:16:16.499 INFO:tasks.ceph.osd.2:Stopped 2024-06-28T16:16:16.499 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-06-28T16:16:16.499 INFO:teuthology.orchestra.run:waiting for 300 2024-06-28T16:16:16.592 INFO:tasks.ceph.osd.3:Stopped 2024-06-28T16:16:16.592 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-06-28T16:16:16.592 INFO:teuthology.orchestra.run:waiting for 300 2024-06-28T16:16:18.616 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-06-28T16:16:18.616 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-06-28T16:16:22.695 INFO:tasks.ceph.osd.4:Stopped 2024-06-28T16:16:22.696 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-06-28T16:16:22.696 INFO:teuthology.orchestra.run:waiting for 300 2024-06-28T16:16:24.322 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-06-28T16:16:24.323 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-06-28T16:16:28.798 INFO:tasks.ceph.osd.5:Stopped 2024-06-28T16:16:28.799 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-06-28T16:16:28.799 INFO:teuthology.orchestra.run:waiting for 300 2024-06-28T16:16:29.928 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-06-28T16:16:29.929 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-06-28T16:16:34.901 INFO:tasks.ceph.osd.6:Stopped 2024-06-28T16:16:34.901 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-06-28T16:16:34.902 INFO:teuthology.orchestra.run:waiting for 300 2024-06-28T16:16:34.996 INFO:tasks.ceph.osd.7:Stopped 2024-06-28T16:16:34.997 INFO:teuthology.misc:Shutting down mgr daemons... 2024-06-28T16:16:34.997 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-06-28T16:16:34.997 INFO:teuthology.orchestra.run:waiting for 300 2024-06-28T16:16:35.028 INFO:tasks.ceph.mgr.y:Stopped 2024-06-28T16:16:35.029 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-06-28T16:16:35.029 INFO:teuthology.orchestra.run:waiting for 300 2024-06-28T16:16:35.057 INFO:tasks.ceph.mgr.x:Stopped 2024-06-28T16:16:35.057 INFO:teuthology.misc:Shutting down mon daemons... 2024-06-28T16:16:35.058 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-06-28T16:16:35.058 INFO:teuthology.orchestra.run:waiting for 300 2024-06-28T16:16:35.093 INFO:tasks.ceph.mon.a:Stopped 2024-06-28T16:16:35.093 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-06-28T16:16:35.094 INFO:teuthology.orchestra.run:waiting for 300 2024-06-28T16:16:35.151 INFO:tasks.ceph.mon.c:Stopped 2024-06-28T16:16:35.152 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-06-28T16:16:35.152 INFO:teuthology.orchestra.run:waiting for 300 2024-06-28T16:16:35.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-06-28T16:16:35.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-06-28T16:16:35.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-06-28T16:16:35.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-06-28T16:16:35.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-06-28T16:16:35.177 INFO:tasks.ceph.mon.b:Stopped 2024-06-28T16:16:35.177 INFO:tasks.ceph:Checking cluster log for badness... 2024-06-28T16:16:35.177 DEBUG:teuthology.orchestra.run.smithi178:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2024-06-28T16:16:35.211 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi178.front.sepia.ceph.com 2024-06-28T16:16:35.211 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-06-28T16:16:35.707 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi178.front.sepia.ceph.com 2024-06-28T16:16:35.707 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-06-28T16:16:35.960 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi178.front.sepia.ceph.com 2024-06-28T16:16:35.961 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-06-28T16:16:36.153 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi178.front.sepia.ceph.com 2024-06-28T16:16:36.153 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-06-28T16:16:36.365 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi195.front.sepia.ceph.com 2024-06-28T16:16:36.365 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-06-28T16:16:36.711 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi195.front.sepia.ceph.com 2024-06-28T16:16:36.711 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-06-28T16:16:36.957 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi195.front.sepia.ceph.com 2024-06-28T16:16:36.958 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-06-28T16:16:37.154 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi195.front.sepia.ceph.com 2024-06-28T16:16:37.155 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-06-28T16:16:37.359 INFO:tasks.ceph:Archiving mon data... 2024-06-28T16:16:37.361 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037/data/mon.a.tgz 2024-06-28T16:16:37.361 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-06-28T16:16:37.367 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.vqSSoRST1Q 2024-06-28T16:16:37.367 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.vqSSoRST1Q 2024-06-28T16:16:37.532 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.vqSSoRST1Q 2024-06-28T16:16:37.596 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.vqSSoRST1Q is 475KB 2024-06-28T16:16:37.658 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.vqSSoRST1Q 2024-06-28T16:16:37.663 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037/data/mon.c.tgz 2024-06-28T16:16:37.663 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-06-28T16:16:37.712 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.zmT4UbJv8S 2024-06-28T16:16:37.712 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.zmT4UbJv8S 2024-06-28T16:16:37.869 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.zmT4UbJv8S 2024-06-28T16:16:37.936 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.zmT4UbJv8S is 476KB 2024-06-28T16:16:37.998 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.zmT4UbJv8S 2024-06-28T16:16:38.003 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037/data/mon.b.tgz 2024-06-28T16:16:38.003 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-06-28T16:16:38.009 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.DHqrJY6hsR 2024-06-28T16:16:38.010 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.DHqrJY6hsR 2024-06-28T16:16:38.174 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.DHqrJY6hsR 2024-06-28T16:16:38.245 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.DHqrJY6hsR is 476KB 2024-06-28T16:16:38.453 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.DHqrJY6hsR 2024-06-28T16:16:38.460 INFO:tasks.ceph:Cleaning ceph cluster... 2024-06-28T16:16:38.460 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-06-28T16:16:38.463 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-06-28T16:16:38.591 DEBUG:tasks.ceph:valgrind exception message: valgrind error: MismatchedFree operator delete[](void*, unsigned long) MonClient::get_monmap_and_config() 2024-06-28T16:16:38.597 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-06-28T16:16:38.597 INFO:tasks.ceph:Archiving crash dumps... 2024-06-28T16:16:38.599 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/crash to /home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037/remote/smithi178/crash 2024-06-28T16:16:38.600 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-06-28T16:16:38.614 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/crash to /home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037/remote/smithi195/crash 2024-06-28T16:16:38.615 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-06-28T16:16:38.654 INFO:tasks.ceph:Compressing logs... 2024-06-28T16:16:38.655 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-28T16:16:38.661 DEBUG:teuthology.orchestra.run.smithi195:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-28T16:16:38.683 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97251.log 2024-06-28T16:16:38.684 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31200.log 2024-06-28T16:16:38.685 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.97251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63015.log 2024-06-28T16:16:38.685 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31200.log: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.97251.log.gz 2024-06-28T16:16:38.685 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31200.log.gz 2024-06-28T16:16:38.685 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30998.log 2024-06-28T16:16:38.686 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50519.log 2024-06-28T16:16:38.687 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.63015.log: /var/log/ceph/ceph-client.admin.30998.log: gzip -5 --verbose -- /var/log/ceph/rgw.ceph.client.0.log 2024-06-28T16:16:38.687 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30998.log.gz 2024-06-28T16:16:38.687 INFO:teuthology.orchestra.run.smithi178.stderr:gzip/var/log/ceph/ceph-client.admin.50519.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.86631.log 2024-06-28T16:16:38.687 INFO:teuthology.orchestra.run.smithi178.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.50519.log.gz 2024-06-28T16:16:38.687 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/rgw.ceph.client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67507.log 2024-06-28T16:16:38.687 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.86631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.114796.log 2024-06-28T16:16:38.690 INFO:teuthology.orchestra.run.smithi178.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.86631.log.gz 2024-06-28T16:16:38.690 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.67507.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-06-28T16:16:38.690 INFO:teuthology.orchestra.run.smithi178.stderr: 25.5% -- replaced with /var/log/ceph/ceph-client.admin.67507.log.gz 2024-06-28T16:16:38.690 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.114796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43954.log 2024-06-28T16:16:38.690 INFO:teuthology.orchestra.run.smithi178.stderr: 84.3% -- replaced with /var/log/ceph/ceph-client.0.114796.log.gz 2024-06-28T16:16:38.690 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31700.log 2024-06-28T16:16:38.691 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55290.log 2024-06-28T16:16:38.691 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43954.log.gz 2024-06-28T16:16:38.691 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31700.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31700.log.gz 2024-06-28T16:16:38.691 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108489.log 2024-06-28T16:16:38.691 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68790.log 2024-06-28T16:16:38.691 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.55290.log: 20.3% -- replaced with /var/log/ceph/ceph-client.admin.55290.log.gz 2024-06-28T16:16:38.691 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.108489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87221.log 2024-06-28T16:16:38.697 INFO:teuthology.orchestra.run.smithi178.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.108489.log.gz 2024-06-28T16:16:38.698 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.68790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30799.log 2024-06-28T16:16:38.698 INFO:teuthology.orchestra.run.smithi178.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.63015.log.gz 2024-06-28T16:16:38.698 INFO:teuthology.orchestra.run.smithi178.stderr: 48.6% -- replaced with /var/log/ceph/ceph-client.admin.68790.log.gz 2024-06-28T16:16:38.698 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.87221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46339.log 2024-06-28T16:16:38.698 INFO:teuthology.orchestra.run.smithi178.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.87221.log.gz 2024-06-28T16:16:38.698 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41544.log 2024-06-28T16:16:38.699 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30799.log.gz 2024-06-28T16:16:38.699 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.46339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65142.log 2024-06-28T16:16:38.699 INFO:teuthology.orchestra.run.smithi178.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.46339.log.gz 2024-06-28T16:16:38.699 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.123728.log 2024-06-28T16:16:38.699 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41544.log.gz 2024-06-28T16:16:38.699 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.65142.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-06-28T16:16:38.699 INFO:teuthology.orchestra.run.smithi178.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.65142.log.gz 2024-06-28T16:16:38.699 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.123728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76600.log 2024-06-28T16:16:38.699 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.0.log: 84.3% -- replaced with /var/log/ceph/ceph-client.0.123728.log.gz 2024-06-28T16:16:38.700 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31600.log 2024-06-28T16:16:38.700 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36488.log 2024-06-28T16:16:38.700 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.76600.log: /var/log/ceph/ceph-client.admin.31600.log: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.76600.log.gzgzip 2024-06-28T16:16:38.700 INFO:teuthology.orchestra.run.smithi178.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99614.log 2024-06-28T16:16:38.700 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31600.log.gz 2024-06-28T16:16:38.700 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.36488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75420.log 2024-06-28T16:16:38.700 INFO:teuthology.orchestra.run.smithi178.stderr: 43.9% -- replaced with /var/log/ceph/ceph-client.0.36488.log.gz 2024-06-28T16:16:38.700 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.99614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49904.log 2024-06-28T16:16:38.701 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.75420.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.99614.log.gz 2024-06-28T16:16:38.701 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105517.log 2024-06-28T16:16:38.701 INFO:teuthology.orchestra.run.smithi178.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.75420.log.gz 2024-06-28T16:16:38.701 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.49904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49929.log 2024-06-28T16:16:38.701 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49904.log.gz 2024-06-28T16:16:38.701 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.105517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54204.log 2024-06-28T16:16:38.701 INFO:teuthology.orchestra.run.smithi178.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.105517.log.gz 2024-06-28T16:16:38.701 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.49929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.130808.log 2024-06-28T16:16:38.702 INFO:teuthology.orchestra.run.smithi178.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.49929.log.gz 2024-06-28T16:16:38.702 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.54204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80140.log 2024-06-28T16:16:38.711 INFO:teuthology.orchestra.run.smithi178.stderr: 63.2% -- replaced with /var/log/ceph/ceph-client.admin.54204.log.gz 2024-06-28T16:16:38.712 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.130808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59141.log 2024-06-28T16:16:38.712 INFO:teuthology.orchestra.run.smithi178.stderr: 80.6% -- replaced with /var/log/ceph/ceph-client.0.130808.log.gz 2024-06-28T16:16:38.712 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.80140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41519.log 2024-06-28T16:16:38.712 INFO:teuthology.orchestra.run.smithi178.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.80140.log.gz 2024-06-28T16:16:38.712 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.59141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115389.log 2024-06-28T16:16:38.712 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123127.log 2024-06-28T16:16:38.712 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41519.log.gz 2024-06-28T16:16:38.712 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.115389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134701.log 2024-06-28T16:16:38.713 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.123127.log: gzip 92.5% -- replaced with /var/log/ceph/ceph-client.admin.123127.log.gz 2024-06-28T16:16:38.713 INFO:teuthology.orchestra.run.smithi178.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.110259.log 2024-06-28T16:16:38.713 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134701.log.gz 2024-06-28T16:16:38.713 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110849.log 2024-06-28T16:16:38.713 INFO:teuthology.orchestra.run.smithi178.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.59141.log.gz 2024-06-28T16:16:38.713 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.110259.log: 85.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.110259.log.gz 2024-06-28T16:16:38.713 INFO:teuthology.orchestra.run.smithi178.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.124318.log 2024-06-28T16:16:38.713 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.110849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110849.log.gz 2024-06-28T16:16:38.714 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66325.log 2024-06-28T16:16:38.714 INFO:teuthology.orchestra.run.smithi178.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.115389.log.gz 2024-06-28T16:16:38.714 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.124318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40339.log 2024-06-28T16:16:38.714 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.66325.log: 80.4% -- replaced with /var/log/ceph/ceph-client.0.124318.log.gz 2024-06-28T16:16:38.714 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31409.log 2024-06-28T16:16:38.714 INFO:teuthology.orchestra.run.smithi178.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.66325.log.gz 2024-06-28T16:16:38.714 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42159.log 2024-06-28T16:16:38.714 INFO:teuthology.orchestra.run.smithi178.stderr: 82.6% -- replaced with /var/log/ceph/ceph-client.admin.40339.log.gz 2024-06-28T16:16:38.714 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.31409.log: -- /var/log/ceph/ceph-client.admin.95481.log 2024-06-28T16:16:38.715 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31409.log.gz 2024-06-28T16:16:38.715 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134961.log 2024-06-28T16:16:38.715 INFO:teuthology.orchestra.run.smithi178.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.42159.log.gz 2024-06-28T16:16:38.715 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.95481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122538.log 2024-06-28T16:16:38.715 INFO:teuthology.orchestra.run.smithi178.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.95481.log.gz 2024-06-28T16:16:38.715 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134961.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-06-28T16:16:38.715 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134961.log.gz 2024-06-28T16:16:38.715 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.122538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107287.log 2024-06-28T16:16:38.716 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134805.log 2024-06-28T16:16:38.716 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.107287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104927.log 2024-06-28T16:16:38.716 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31001.log 2024-06-28T16:16:38.716 INFO:teuthology.orchestra.run.smithi178.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.107287.log.gz 2024-06-28T16:16:38.716 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134805.log.gz 2024-06-28T16:16:38.716 INFO:teuthology.orchestra.run.smithi178.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.122538.log.gz 2024-06-28T16:16:38.716 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.104927.log: 82.6% -- replaced with /var/log/ceph/ceph-client.admin.104927.log.gz 2024-06-28T16:16:38.716 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107899.log 2024-06-28T16:16:38.717 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26037.log 2024-06-28T16:16:38.717 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31001.log.gz 2024-06-28T16:16:38.717 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54750.log 2024-06-28T16:16:38.717 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.107899.log: /var/log/ceph/ceph-client.admin.26037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96071.log 2024-06-28T16:16:38.717 INFO:teuthology.orchestra.run.smithi178.stderr: 82.7% -- replaced with /var/log/ceph/ceph-client.admin.107899.log.gz 2024-06-28T16:16:38.717 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26037.log.gz 2024-06-28T16:16:38.717 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.54750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68824.log 2024-06-28T16:16:38.717 INFO:teuthology.orchestra.run.smithi178.stderr: 52.5% -- replaced with /var/log/ceph/ceph-client.admin.54750.log.gz 2024-06-28T16:16:38.717 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.96071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39749.log 2024-06-28T16:16:38.718 INFO:teuthology.orchestra.run.smithi178.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.96071.log.gz 2024-06-28T16:16:38.718 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.68824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61077.log 2024-06-28T16:16:38.724 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55324.log 2024-06-28T16:16:38.725 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.39749.log: /var/log/ceph/ceph-client.admin.61077.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-06-28T16:16:38.725 INFO:teuthology.orchestra.run.smithi178.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.39749.log.gz 2024-06-28T16:16:38.725 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.55324.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.55324.log.gz 2024-06-28T16:16:38.725 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70700.log 2024-06-28T16:16:38.725 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31212.log 2024-06-28T16:16:38.725 INFO:teuthology.orchestra.run.smithi178.stderr: 92.8% 92.1% -- replaced with /var/log/ceph/ceph-client.0.68824.log.gz 2024-06-28T16:16:38.725 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61077.log.gz 2024-06-28T16:16:38.725 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.70700.log: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.70700.log.gz 2024-06-28T16:16:38.726 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31430.log 2024-06-28T16:16:38.726 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110874.log 2024-06-28T16:16:38.726 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31212.log.gz 2024-06-28T16:16:38.726 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71290.log 2024-06-28T16:16:38.726 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31430.log.gz 2024-06-28T16:16:38.726 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.110874.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110874.log.gz 2024-06-28T16:16:38.726 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39159.log 2024-06-28T16:16:38.727 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.71290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63667.log 2024-06-28T16:16:38.727 INFO:teuthology.orchestra.run.smithi178.stderr: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.71290.log.gz/var/log/ceph/ceph-client.admin.39159.log: 2024-06-28T16:16:38.727 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134831.log 2024-06-28T16:16:38.727 INFO:teuthology.orchestra.run.smithi178.stderr: 83.5% -- replaced with /var/log/ceph/ceph-client.admin.39159.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74830.log 2024-06-28T16:16:38.727 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-06-28T16:16:38.727 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.63667.log: /var/log/ceph/ceph-client.admin.134831.log: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.63667.log.gz 2024-06-28T16:16:38.727 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134831.log.gz 2024-06-28T16:16:38.727 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115978.log 2024-06-28T16:16:38.727 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.74830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31676.log 2024-06-28T16:16:38.728 INFO:teuthology.orchestra.run.smithi178.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.74830.log.gz 2024-06-28T16:16:38.728 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.115978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91941.log 2024-06-28T16:16:38.728 INFO:teuthology.orchestra.run.smithi178.stderr: 92.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107877.log 2024-06-28T16:16:38.728 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.115978.log.gz/var/log/ceph/ceph-client.admin.31676.log: 2024-06-28T16:16:38.728 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31676.log.gz 2024-06-28T16:16:38.728 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.91941.log: 83.7% -- replaced with /var/log/ceph/ceph-client.admin.91941.log.gz 2024-06-28T16:16:38.728 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42749.log 2024-06-28T16:16:38.728 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.107877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134649.log 2024-06-28T16:16:38.729 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107877.log.gz 2024-06-28T16:16:38.729 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.42749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30826.log 2024-06-28T16:16:38.729 INFO:teuthology.orchestra.run.smithi178.stderr: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.42749.log.gz 2024-06-28T16:16:38.729 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31652.log 2024-06-28T16:16:38.729 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134649.log.gz 2024-06-28T16:16:38.729 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30826.log.gz 2024-06-28T16:16:38.729 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85451.log 2024-06-28T16:16:38.729 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94301.log 2024-06-28T16:16:38.729 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31652.log.gz 2024-06-28T16:16:38.730 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.85451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31194.log 2024-06-28T16:16:38.734 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25933.log 2024-06-28T16:16:38.735 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-06-28T16:16:38.735 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.25933.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-06-28T16:16:38.735 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25933.log.gz 2024-06-28T16:16:38.735 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25885.log 2024-06-28T16:16:38.735 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25837.log 2024-06-28T16:16:38.735 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.25885.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25885.log.gz -5 2024-06-28T16:16:38.736 INFO:teuthology.orchestra.run.smithi195.stderr: --verbose -- /var/log/ceph/ceph-osd.5.log 2024-06-28T16:16:38.736 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.25837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25837.log.gz 2024-06-28T16:16:38.736 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25861.log 2024-06-28T16:16:38.736 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-06-28T16:16:38.736 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.25861.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25861.log.gz 2024-06-28T16:16:38.736 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-06-28T16:16:38.736 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-06-28T16:16:38.736 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25909.log 2024-06-28T16:16:38.740 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25773.log 2024-06-28T16:16:38.740 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.25909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25909.log.gz 2024-06-28T16:16:38.740 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25957.log 2024-06-28T16:16:38.740 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-06-28T16:16:38.740 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.25773.log: /var/log/ceph/ceph-client.admin.25957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25773.log.gz 2024-06-28T16:16:38.740 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25957.log.gz 2024-06-28T16:16:38.740 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-06-28T16:16:38.741 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25797.log 2024-06-28T16:16:38.741 INFO:teuthology.orchestra.run.smithi195.stderr: 90.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-06-28T16:16:38.741 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.25797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25797.log.gz 2024-06-28T16:16:38.748 INFO:teuthology.orchestra.run.smithi178.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.85451.log.gz 2024-06-28T16:16:38.748 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43339.log 2024-06-28T16:16:38.748 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.94301.log: 82.7% -- replaced with /var/log/ceph/ceph-client.admin.94301.log.gz 2024-06-28T16:16:38.748 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31194.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31194.log.gz 2024-06-28T16:16:38.748 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52339.log 2024-06-28T16:16:38.749 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135226.log 2024-06-28T16:16:38.749 INFO:teuthology.orchestra.run.smithi178.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.43339.log.gz 2024-06-28T16:16:38.749 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.52339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96661.log 2024-06-28T16:16:38.749 INFO:teuthology.orchestra.run.smithi178.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.52339.log.gz 2024-06-28T16:16:38.749 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43929.log 2024-06-28T16:16:38.749 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.135226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135226.log.gz 2024-06-28T16:16:38.749 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.96661.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.96661.log.gz 2024-06-28T16:16:38.749 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100204.log 2024-06-28T16:16:38.750 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70110.log 2024-06-28T16:16:38.750 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43929.log.gz 2024-06-28T16:16:38.750 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.100204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77780.log 2024-06-28T16:16:38.750 INFO:teuthology.orchestra.run.smithi178.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.100204.log.gz 2024-06-28T16:16:38.750 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134935.log 2024-06-28T16:16:38.750 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.70110.log: 82.2%/var/log/ceph/ceph-client.admin.77780.log: -- replaced with /var/log/ceph/ceph-client.admin.70110.log.gz 2024-06-28T16:16:38.750 INFO:teuthology.orchestra.run.smithi178.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.77780.log.gz 2024-06-28T16:16:38.751 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84861.log 2024-06-28T16:16:38.751 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74240.log 2024-06-28T16:16:38.751 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134935.log.gz 2024-06-28T16:16:38.751 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.84861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109669.log 2024-06-28T16:16:38.751 INFO:teuthology.orchestra.run.smithi178.stderr: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.84861.log.gz 2024-06-28T16:16:38.751 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120256.log 2024-06-28T16:16:38.751 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.74240.log: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.74240.log.gz 2024-06-28T16:16:38.752 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.109669.log: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.109669.log.gz 2024-06-28T16:16:38.752 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91351.log 2024-06-28T16:16:38.752 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.120256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30924.log 2024-06-28T16:16:38.752 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.91351.log: 92.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.120256.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.45749.log 2024-06-28T16:16:38.752 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-06-28T16:16:38.752 INFO:teuthology.orchestra.run.smithi178.stderr: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.91351.log.gz 2024-06-28T16:16:38.752 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30924.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30924.log.gz 2024-06-28T16:16:38.753 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31004.log 2024-06-28T16:16:38.753 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.45749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34736.log 2024-06-28T16:16:38.753 INFO:teuthology.orchestra.run.smithi178.stderr: 81.2% -- replaced with /var/log/ceph/ceph-client.admin.45749.log.gz 2024-06-28T16:16:38.753 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52289.log 2024-06-28T16:16:38.753 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31004.log.gz 2024-06-28T16:16:38.753 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134675.log 2024-06-28T16:16:38.753 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.34736.log: 43.5% -- replaced with /var/log/ceph/ceph-client.0.34736.log.gz/var/log/ceph/ceph-client.admin.52289.log: 2024-06-28T16:16:38.754 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31410.log 2024-06-28T16:16:38.754 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52289.log.gz 2024-06-28T16:16:38.754 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134675.log.gz 2024-06-28T16:16:38.754 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113309.log 2024-06-28T16:16:38.754 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32969.log 2024-06-28T16:16:38.754 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31410.log: /var/log/ceph/ceph-client.admin.113309.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-06-28T16:16:38.755 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31410.log.gz 2024-06-28T16:16:38.755 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.32969.log: gzip -5 87.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.113309.log.gz /var/log/ceph/ceph-client.admin.80730.log 2024-06-28T16:16:38.755 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-06-28T16:16:38.755 INFO:teuthology.orchestra.run.smithi178.stderr: 80.8% -- replaced with /var/log/ceph/ceph-client.0.32969.log.gz 2024-06-28T16:16:38.755 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119667.log 2024-06-28T16:16:38.755 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.80730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57851.log 2024-06-28T16:16:38.755 INFO:teuthology.orchestra.run.smithi178.stderr: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.80730.log.gz 2024-06-28T16:16:38.756 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.119667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71880.log 2024-06-28T16:16:38.756 INFO:teuthology.orchestra.run.smithi178.stderr: 91.6%/var/log/ceph/ceph-client.admin.57851.log: -- replaced with /var/log/ceph/ceph-client.admin.119667.log.gz 2024-06-28T16:16:38.756 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49289.log 2024-06-28T16:16:38.756 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.71880.log: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.71880.log.gz 2024-06-28T16:16:38.756 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21003.log 2024-06-28T16:16:38.756 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.49289.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.49289.log.gz 2024-06-28T16:16:38.756 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120845.log 2024-06-28T16:16:38.757 INFO:teuthology.orchestra.run.smithi195.stderr: 88.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-06-28T16:16:38.762 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph.tmp-client.admin.21003.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21003.log.gz 2024-06-28T16:16:38.763 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31003.log 2024-06-28T16:16:38.763 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.120845.log: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.57851.log.gz 2024-06-28T16:16:38.763 INFO:teuthology.orchestra.run.smithi178.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.120845.log.gz 2024-06-28T16:16:38.763 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134727.log 2024-06-28T16:16:38.763 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81320.log 2024-06-28T16:16:38.763 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31003.log.gz 2024-06-28T16:16:38.763 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89581.log 2024-06-28T16:16:38.763 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134727.log.gz 2024-06-28T16:16:38.764 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.81320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109079.log 2024-06-28T16:16:38.764 INFO:teuthology.orchestra.run.smithi178.stderr: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.81320.log.gz 2024-06-28T16:16:38.764 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.89581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57206.log 2024-06-28T16:16:38.764 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.109079.log: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.89581.log.gz 2024-06-28T16:16:38.764 INFO:teuthology.orchestra.run.smithi178.stderr: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.109079.log.gz 2024-06-28T16:16:38.765 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101384.log 2024-06-28T16:16:38.765 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135200.log 2024-06-28T16:16:38.765 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.57206.log: /var/log/ceph/ceph-client.admin.101384.log: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.101384.log.gz 2024-06-28T16:16:38.765 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.134003.log 2024-06-28T16:16:38.765 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.135200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135200.log.gz 2024-06-28T16:16:38.765 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46929.log 2024-06-28T16:16:38.765 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.134003.log: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.57206.log.gz 2024-06-28T16:16:38.765 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30989.log 2024-06-28T16:16:38.765 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.46929.log: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.46929.log.gz 2024-06-28T16:16:38.767 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30750.log 2024-06-28T16:16:38.767 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30989.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30989.log.gz 2024-06-28T16:16:38.768 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135250.log 2024-06-28T16:16:38.768 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30750.log.gz 2024-06-28T16:16:38.768 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.admin.log 2024-06-28T16:16:38.768 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.135250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135250.log.gz 2024-06-28T16:16:38.769 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93121.log 2024-06-28T16:16:38.771 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ops-log-ceph-client.admin.log: 94.2% -- replaced with /var/log/ceph/ops-log-ceph-client.admin.log.gz 2024-06-28T16:16:38.771 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45159.log 2024-06-28T16:16:38.772 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.93121.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.93121.log.gz 2024-06-28T16:16:38.773 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-06-28T16:16:38.774 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.45159.log: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.45159.log.gz 2024-06-28T16:16:38.774 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34146.log 2024-06-28T16:16:38.787 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.1.log: 86.7% -- replaced with /var/log/ceph/ceph-client.0.134003.log.gz 2024-06-28T16:16:38.787 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31748.log 2024-06-28T16:16:38.788 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.34146.log: 81.6% -- replaced with /var/log/ceph/ceph-client.0.34146.log.gz 2024-06-28T16:16:38.789 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90171.log 2024-06-28T16:16:38.789 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31748.log.gz 2024-06-28T16:16:38.790 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67559.log 2024-06-28T16:16:38.790 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.90171.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.90171.log.gz 2024-06-28T16:16:38.790 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135013.log 2024-06-28T16:16:38.791 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.67559.log: 84.1% -- replaced with /var/log/ceph/ceph-client.0.67559.log.gz 2024-06-28T16:16:38.792 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112079.log 2024-06-28T16:16:38.792 INFO:teuthology.orchestra.run.smithi195.stderr: 95.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-06-28T16:16:38.793 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.135013.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135013.log.gz 2024-06-28T16:16:38.793 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37848.log 2024-06-28T16:16:38.794 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.112079.log: 81.7% -- replaced with /var/log/ceph/ceph-client.admin.112079.log.gz 2024-06-28T16:16:38.794 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54495.log 2024-06-28T16:16:38.795 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.37848.log: 83.3% -- replaced with /var/log/ceph/ceph-client.admin.37848.log.gz 2024-06-28T16:16:38.795 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98434.log 2024-06-28T16:16:38.796 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.54495.log: 55.7% -- replaced with /var/log/ceph/ceph-client.admin.54495.log.gz 2024-06-28T16:16:38.796 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134857.log 2024-06-28T16:16:38.797 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.98434.log: 82.7% -- replaced with /var/log/ceph/ceph-client.admin.98434.log.gz 2024-06-28T16:16:38.797 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110899.log 2024-06-28T16:16:38.798 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134857.log.gz 2024-06-28T16:16:38.799 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31204.log 2024-06-28T16:16:38.799 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.110899.log: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.110899.log.gz 2024-06-28T16:16:38.800 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135146.log 2024-06-28T16:16:38.800 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31204.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31204.log.gz 2024-06-28T16:16:38.801 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.126678.log 2024-06-28T16:16:38.801 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.135146.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135146.log.gz 2024-06-28T16:16:38.802 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35904.log 2024-06-28T16:16:38.803 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.126678.log: 84.6% -- replaced with /var/log/ceph/ceph-client.0.126678.log.gz 2024-06-28T16:16:38.803 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83091.log 2024-06-28T16:16:38.804 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.35904.log: 43.9% -- replaced with /var/log/ceph/ceph-client.0.35904.log.gz 2024-06-28T16:16:38.804 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104337.log 2024-06-28T16:16:38.805 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.83091.log: 81.3% -- replaced with /var/log/ceph/ceph-client.admin.83091.log.gz 2024-06-28T16:16:38.805 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31412.log 2024-06-28T16:16:38.806 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.104337.log: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.104337.log.gz 2024-06-28T16:16:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97844.log 2024-06-28T16:16:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31412.log.gz 2024-06-28T16:16:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112719.log 2024-06-28T16:16:38.812 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.97844.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.97844.log.gz 2024-06-28T16:16:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134779.log 2024-06-28T16:16:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.112719.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112719.log.gz 2024-06-28T16:16:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82500.log 2024-06-28T16:16:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134779.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134779.log.gz 2024-06-28T16:16:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61725.log 2024-06-28T16:16:38.813 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.82500.log: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.82500.log.gz 2024-06-28T16:16:38.814 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93711.log 2024-06-28T16:16:38.814 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.61725.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61725.log.gz 2024-06-28T16:16:38.815 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55265.log 2024-06-28T16:16:38.816 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.93711.log: 82.6% -- replaced with /var/log/ceph/ceph-client.admin.93711.log.gz 2024-06-28T16:16:38.816 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118354.log 2024-06-28T16:16:38.817 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.55265.log: 21.6% -- replaced with /var/log/ceph/ceph-client.admin.55265.log.gz 2024-06-28T16:16:38.817 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134753.log 2024-06-28T16:16:38.818 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.118354.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.118354.log.gz 2024-06-28T16:16:38.819 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113899.log 2024-06-28T16:16:38.819 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134753.log.gz 2024-06-28T16:16:38.820 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86041.log 2024-06-28T16:16:38.820 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.113899.log: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.113899.log.gz 2024-06-28T16:16:38.821 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-06-28T16:16:38.822 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.86041.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.86041.log.gz 2024-06-28T16:16:38.822 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47519.log 2024-06-28T16:16:38.836 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134596.log 2024-06-28T16:16:38.837 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.47519.log: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.47519.log.gz 2024-06-28T16:16:38.839 INFO:teuthology.orchestra.run.smithi178.stderr: 93.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-06-28T16:16:38.839 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117176.log 2024-06-28T16:16:38.840 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134596.log.gz 2024-06-28T16:16:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81910.log 2024-06-28T16:16:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.117176.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.117176.log.gz 2024-06-28T16:16:38.843 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52314.log 2024-06-28T16:16:38.845 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.81910.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.81910.log.gz 2024-06-28T16:16:38.845 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32385.log 2024-06-28T16:16:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.52314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52314.log.gz 2024-06-28T16:16:38.846 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.127268.log 2024-06-28T16:16:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.32385.log: 43.9% -- replaced with /var/log/ceph/ceph-client.0.32385.log.gz 2024-06-28T16:16:38.847 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54424.log 2024-06-28T16:16:38.848 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.127268.log: 81.2% -- replaced with /var/log/ceph/ceph-client.0.127268.log.gz 2024-06-28T16:16:38.848 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31396.log 2024-06-28T16:16:38.849 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.54424.log: 57.5% -- replaced with /var/log/ceph/ceph-client.admin.54424.log.gz 2024-06-28T16:16:38.849 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33559.log 2024-06-28T16:16:38.850 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31396.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31396.log.gz 2024-06-28T16:16:38.850 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48699.log 2024-06-28T16:16:38.851 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.33559.log: 44.2% -- replaced with /var/log/ceph/ceph-client.0.33559.log.gz 2024-06-28T16:16:38.852 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94891.log 2024-06-28T16:16:38.853 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.48699.log: 80.8% -- replaced with /var/log/ceph/ceph-client.admin.48699.log.gz 2024-06-28T16:16:38.853 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30775.log 2024-06-28T16:16:38.854 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.94891.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.94891.log.gz 2024-06-28T16:16:38.854 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72470.log 2024-06-28T16:16:38.855 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30775.log.gz 2024-06-28T16:16:38.855 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31772.log 2024-06-28T16:16:38.856 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.72470.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.72470.log.gz 2024-06-28T16:16:38.857 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60432.log 2024-06-28T16:16:38.857 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31772.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31772.log.gz 2024-06-28T16:16:38.858 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31192.log 2024-06-28T16:16:38.872 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.60432.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-06-28T16:16:38.873 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31192.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31192.log.gz 2024-06-28T16:16:38.886 INFO:teuthology.orchestra.run.smithi178.stderr: 95.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125498.log 2024-06-28T16:16:38.893 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mon.c.log: -- replaced with /var/log/ceph/ceph-client.admin.60432.log.gz 2024-06-28T16:16:38.896 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.126088.log 2024-06-28T16:16:38.896 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.125498.log: 80.5% -- replaced with /var/log/ceph/ceph-client.0.125498.log.gz 2024-06-28T16:16:38.912 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117765.log 2024-06-28T16:16:38.912 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.126088.log: 84.1% -- replaced with /var/log/ceph/ceph-client.0.126088.log.gz 2024-06-28T16:16:38.913 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53519.log 2024-06-28T16:16:38.914 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.117765.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.117765.log.gz 2024-06-28T16:16:38.932 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31626.log 2024-06-28T16:16:38.933 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.53519.log: 82.2% -- replaced with /var/log/ceph/ceph-client.admin.53519.log.gz 2024-06-28T16:16:38.944 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112694.log 2024-06-28T16:16:38.944 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31626.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31626.log.gz 2024-06-28T16:16:38.960 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31199.log 2024-06-28T16:16:38.961 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.112694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112694.log.gz 2024-06-28T16:16:38.968 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31392.log 2024-06-28T16:16:38.968 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31199.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31199.log.gz 2024-06-28T16:16:38.976 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103747.log 2024-06-28T16:16:38.976 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31392.log.gz 2024-06-28T16:16:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25927.log 2024-06-28T16:16:38.984 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.103747.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.103747.log.gz 2024-06-28T16:16:38.992 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35320.log 2024-06-28T16:16:38.992 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.25927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25927.log.gz 2024-06-28T16:16:39.000 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135117.log 2024-06-28T16:16:39.000 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.35320.log: 43.9% -- replaced with /var/log/ceph/ceph-client.0.35320.log.gz 2024-06-28T16:16:39.008 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51699.log 2024-06-28T16:16:39.008 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.135117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135117.log.gz 2024-06-28T16:16:39.016 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.0.log 2024-06-28T16:16:39.016 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.51699.log: 80.6% -- replaced with /var/log/ceph/ceph-client.admin.51699.log.gz 2024-06-28T16:16:39.024 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135065.log 2024-06-28T16:16:39.032 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ops-log-ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135039.log 2024-06-28T16:16:39.032 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.135065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135065.log.gz 2024-06-28T16:16:39.040 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-06-28T16:16:39.040 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.135039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135039.log.gz 2024-06-28T16:16:39.048 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99024.log 2024-06-28T16:16:39.051 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph.audit.log: 94.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-06-28T16:16:39.060 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106107.log 2024-06-28T16:16:39.060 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.99024.log: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.99024.log.gz 2024-06-28T16:16:39.068 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.129628.log 2024-06-28T16:16:39.068 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.106107.log: 82.7% -- replaced with /var/log/ceph/ceph-client.admin.106107.log.gz 2024-06-28T16:16:39.076 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.130218.log 2024-06-28T16:16:39.076 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.129628.log: 84.5% -- replaced with /var/log/ceph/ceph-client.0.129628.log.gz 2024-06-28T16:16:39.084 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135091.log 2024-06-28T16:16:39.085 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.130218.log: 84.5% -- replaced with /var/log/ceph/ceph-client.0.130218.log.gz 2024-06-28T16:16:39.092 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31182.log 2024-06-28T16:16:39.093 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.135091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135091.log.gz 2024-06-28T16:16:39.100 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31002.log 2024-06-28T16:16:39.100 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31182.log.gz 2024-06-28T16:16:39.108 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134623.log 2024-06-28T16:16:39.109 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31002.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31002.log.gz 2024-06-28T16:16:39.116 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43979.log 2024-06-28T16:16:39.117 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134623.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134623.log.gz 2024-06-28T16:16:39.124 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83681.log 2024-06-28T16:16:39.125 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.43979.log: 81.8% -- replaced with /var/log/ceph/ceph-client.admin.43979.log.gz 2024-06-28T16:16:39.132 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38450.log 2024-06-28T16:16:39.133 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.83681.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.83681.log.gz 2024-06-28T16:16:39.140 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.127858.log 2024-06-28T16:16:39.141 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.38450.log: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.38450.log.gz 2024-06-28T16:16:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.124908.log 2024-06-28T16:16:39.148 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.127858.log: 83.9% -- replaced with /var/log/ceph/ceph-client.0.127858.log.gz 2024-06-28T16:16:39.156 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25779.log 2024-06-28T16:16:39.156 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.124908.log: 84.7% -- replaced with /var/log/ceph/ceph-client.0.124908.log.gz 2024-06-28T16:16:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106697.log 2024-06-28T16:16:39.164 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.25779.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25779.log.gz 2024-06-28T16:16:39.172 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135173.log 2024-06-28T16:16:39.172 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.106697.log: 82.1% -- replaced with /var/log/ceph/ceph-client.admin.106697.log.gz 2024-06-28T16:16:39.180 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59787.log 2024-06-28T16:16:39.180 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.135173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.135173.log.gz 2024-06-28T16:16:39.186 INFO:teuthology.orchestra.run.smithi178.stderr: 94.3% -- replaced with /var/log/ceph/ops-log-ceph-client.0.log.gz 2024-06-28T16:16:39.191 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84271.log 2024-06-28T16:16:39.194 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.59787.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59787.log.gz 2024-06-28T16:16:39.208 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/ceph.client.0.log 2024-06-28T16:16:39.208 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.84271.log: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.84271.log.gz 2024-06-28T16:16:39.224 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51109.log 2024-06-28T16:16:39.225 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/valgrind/ceph.client.0.log: 94.7% -- replaced with /var/log/ceph/valgrind/ceph.client.0.log.gz 2024-06-28T16:16:39.240 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.31795.log 2024-06-28T16:16:39.240 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.51109.log: 80.9% -- replaced with /var/log/ceph/ceph-client.admin.51109.log.gz 2024-06-28T16:16:39.252 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121434.log 2024-06-28T16:16:39.253 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.31795.log: 95.9% -- replaced with /var/log/ceph/ceph-client.0.31795.log.gz 2024-06-28T16:16:39.268 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101974.log 2024-06-28T16:16:39.269 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.121434.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.121434.log.gz 2024-06-28T16:16:39.280 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66916.log 2024-06-28T16:16:39.280 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.101974.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.101974.log.gz 2024-06-28T16:16:39.296 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30994.log 2024-06-28T16:16:39.297 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.66916.log: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.66916.log.gz 2024-06-28T16:16:39.308 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44569.log 2024-06-28T16:16:39.308 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30994.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30994.log.gz 2024-06-28T16:16:39.320 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76010.log 2024-06-28T16:16:39.321 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.44569.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.44569.log.gz 2024-06-28T16:16:39.332 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30950.log 2024-06-28T16:16:39.332 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.76010.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.76010.log.gz 2024-06-28T16:16:39.348 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111489.log 2024-06-28T16:16:39.348 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30950.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30950.log.gz 2024-06-28T16:16:39.364 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68149.log 2024-06-28T16:16:39.364 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.111489.log: 82.4% -- replaced with /var/log/ceph/ceph-client.admin.111489.log.gz 2024-06-28T16:16:39.380 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54663.log 2024-06-28T16:16:39.381 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.68149.log: 84.5% -- replaced with /var/log/ceph/ceph-client.0.68149.log.gz 2024-06-28T16:16:39.396 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54702.log 2024-06-28T16:16:39.396 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.54663.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54663.log.gz 2024-06-28T16:16:39.416 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78960.log 2024-06-28T16:16:39.417 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.54702.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54702.log.gz 2024-06-28T16:16:39.417 INFO:teuthology.orchestra.run.smithi178.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-06-28T16:16:39.418 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103154.log 2024-06-28T16:16:39.418 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.78960.log: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.78960.log.gz 2024-06-28T16:16:39.419 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112669.log 2024-06-28T16:16:39.420 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.103154.log: 82.3% -- replaced with /var/log/ceph/ceph-client.admin.103154.log.gz 2024-06-28T16:16:39.420 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90761.log 2024-06-28T16:16:39.420 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.112669.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112669.log.gz 2024-06-28T16:16:39.426 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118943.log 2024-06-28T16:16:39.426 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.90761.log: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.90761.log.gz 2024-06-28T16:16:39.426 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31724.log 2024-06-28T16:16:39.427 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.118943.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.118943.log.gz 2024-06-28T16:16:39.427 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69414.log 2024-06-28T16:16:39.429 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31724.log.gz 2024-06-28T16:16:39.430 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31422.log 2024-06-28T16:16:39.430 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.69414.log: 80.4% -- replaced with /var/log/ceph/ceph-client.0.69414.log.gz 2024-06-28T16:16:39.430 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88991.log 2024-06-28T16:16:39.430 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31422.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31422.log.gz 2024-06-28T16:16:39.430 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30886.log 2024-06-28T16:16:39.430 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.88991.log: 82.5% -- replaced with /var/log/ceph/ceph-client.admin.88991.log.gz 2024-06-28T16:16:39.430 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54725.log 2024-06-28T16:16:39.431 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30886.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30886.log.gz 2024-06-28T16:16:39.431 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.131988.log 2024-06-28T16:16:39.440 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.54725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54773.log 2024-06-28T16:16:39.441 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.131988.log: 84.6% -- replaced with /var/log/ceph/ceph-client.0.131988.log.gz 2024-06-28T16:16:39.443 INFO:teuthology.orchestra.run.smithi178.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54725.log.gz 2024-06-28T16:16:39.444 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41569.log 2024-06-28T16:16:39.444 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.54773.log: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.54773.log.gz 2024-06-28T16:16:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48109.log 2024-06-28T16:16:39.445 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.41569.log: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.41569.log.gz 2024-06-28T16:16:39.446 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73060.log 2024-06-28T16:16:39.447 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.48109.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.48109.log.gz 2024-06-28T16:16:39.447 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65735.log 2024-06-28T16:16:39.448 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.73060.log: 82.6% -- replaced with /var/log/ceph/ceph-client.admin.73060.log.gz 2024-06-28T16:16:39.448 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58496.log 2024-06-28T16:16:39.450 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.65735.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65735.log.gz 2024-06-28T16:16:39.450 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64553.log 2024-06-28T16:16:39.456 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.58496.log: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.58496.log.gz 2024-06-28T16:16:39.456 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52929.log 2024-06-28T16:16:39.457 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.64553.log: 81.1% -- replaced with /var/log/ceph/ceph-client.admin.64553.log.gz 2024-06-28T16:16:39.457 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78370.log 2024-06-28T16:16:39.458 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.52929.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.52929.log.gz 2024-06-28T16:16:39.458 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134883.log 2024-06-28T16:16:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.78370.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.78370.log.gz 2024-06-28T16:16:39.459 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.132578.log 2024-06-28T16:16:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134883.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134883.log.gz 2024-06-28T16:16:39.460 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100794.log 2024-06-28T16:16:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.132578.log: 80.9% -- replaced with /var/log/ceph/ceph-client.0.132578.log.gz 2024-06-28T16:16:39.461 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31411.log 2024-06-28T16:16:39.462 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.100794.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.100794.log.gz 2024-06-28T16:16:39.463 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.131398.log 2024-06-28T16:16:39.463 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31411.log.gz 2024-06-28T16:16:39.464 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134909.log 2024-06-28T16:16:39.464 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.131398.log: 84.3% -- replaced with /var/log/ceph/ceph-client.0.131398.log.gz 2024-06-28T16:16:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62370.log 2024-06-28T16:16:39.465 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134909.log.gz 2024-06-28T16:16:39.466 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87811.log 2024-06-28T16:16:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.62370.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62370.log.gz 2024-06-28T16:16:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40929.log 2024-06-28T16:16:39.469 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.87811.log: 82.0% -- replaced with /var/log/ceph/ceph-client.admin.87811.log.gz 2024-06-28T16:16:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134987.log 2024-06-28T16:16:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.40929.log: 81.6% -- replaced with /var/log/ceph/ceph-client.admin.40929.log.gz 2024-06-28T16:16:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77190.log 2024-06-28T16:16:39.473 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.134987.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.134987.log.gz 2024-06-28T16:16:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73650.log 2024-06-28T16:16:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.77190.log: 81.4% -- replaced with /var/log/ceph/ceph-client.admin.77190.log.gz 2024-06-28T16:16:39.474 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30997.log 2024-06-28T16:16:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.73650.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.73650.log.gz 2024-06-28T16:16:39.482 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49879.log 2024-06-28T16:16:39.483 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.30997.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30997.log.gz 2024-06-28T16:16:39.483 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.129038.log 2024-06-28T16:16:39.483 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.49879.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49879.log.gz 2024-06-28T16:16:39.483 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102564.log 2024-06-28T16:16:39.484 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.129038.log: 81.1% -- replaced with /var/log/ceph/ceph-client.0.129038.log.gz 2024-06-28T16:16:39.484 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79550.log 2024-06-28T16:16:39.484 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.102564.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.102564.log.gz 2024-06-28T16:16:39.484 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.128448.log 2024-06-28T16:16:39.485 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.79550.log: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.79550.log.gz 2024-06-28T16:16:39.485 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31211.log 2024-06-28T16:16:39.485 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.128448.log: 80.8% -- replaced with /var/log/ceph/ceph-client.0.128448.log.gz 2024-06-28T16:16:39.485 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55913.log 2024-06-28T16:16:39.486 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.31211.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31211.log.gz 2024-06-28T16:16:39.486 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.133407.log 2024-06-28T16:16:39.497 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.55913.log: 94.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88401.log 2024-06-28T16:16:39.498 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.133407.log: 80.5% -- replaced with /var/log/ceph/ceph-client.0.133407.log.gz 2024-06-28T16:16:39.498 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55913.log.gz 2024-06-28T16:16:39.499 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92531.log 2024-06-28T16:16:39.500 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.88401.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.88401.log.gz 2024-06-28T16:16:39.501 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56559.log 2024-06-28T16:16:39.502 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.92531.log: 82.6% -- replaced with /var/log/ceph/ceph-client.admin.92531.log.gz 2024-06-28T16:16:39.503 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.56559.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.56559.log.gz 2024-06-28T16:16:39.998 INFO:teuthology.orchestra.run.smithi195.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-06-28T16:16:40.177 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-06-28T16:16:40.178 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-06-28T16:16:40.178 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-06-28T16:16:40.178 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-06-28T16:16:40.248 INFO:teuthology.orchestra.run.smithi178.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-06-28T16:16:42.275 INFO:teuthology.orchestra.run.smithi178.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-06-28T16:16:42.832 INFO:teuthology.orchestra.run.smithi178.stderr: 92.5% -- replaced with /var/log/ceph/rgw.ceph.client.0.log.gz 2024-06-28T16:16:46.150 INFO:teuthology.orchestra.run.smithi178.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-06-28T16:16:46.965 INFO:teuthology.orchestra.run.smithi195.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-06-28T16:16:49.465 INFO:teuthology.orchestra.run.smithi178.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-06-28T16:16:49.477 INFO:teuthology.orchestra.run.smithi195.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-06-28T16:16:50.671 INFO:teuthology.orchestra.run.smithi178.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-06-28T16:16:51.122 INFO:teuthology.orchestra.run.smithi195.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-06-28T16:16:51.156 INFO:teuthology.orchestra.run.smithi195.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-06-28T16:16:51.158 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-06-28T16:16:51.158 INFO:teuthology.orchestra.run.smithi195.stderr:real 0m12.452s 2024-06-28T16:16:51.158 INFO:teuthology.orchestra.run.smithi195.stderr:user 0m43.587s 2024-06-28T16:16:51.158 INFO:teuthology.orchestra.run.smithi195.stderr:sys 0m1.571s 2024-06-28T16:16:57.757 INFO:teuthology.orchestra.run.smithi178.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-06-28T16:16:57.758 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-06-28T16:16:57.758 INFO:teuthology.orchestra.run.smithi178.stderr:real 0m19.095s 2024-06-28T16:16:57.758 INFO:teuthology.orchestra.run.smithi178.stderr:user 0m57.118s 2024-06-28T16:16:57.758 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m2.188s 2024-06-28T16:16:57.759 INFO:tasks.ceph:Archiving logs... 2024-06-28T16:16:57.760 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037/remote/smithi178/log 2024-06-28T16:16:57.761 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-28T16:17:04.381 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037/remote/smithi195/log 2024-06-28T16:17:04.382 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-28T16:17:09.672 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/ceph.py", line 1978, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: MismatchedFree operator delete[](void*, unsigned long) MonClient::get_monmap_and_config() 2024-06-28T16:17:09.686 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-28T16:17:09.700 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/task/install/__init__.py", line 641, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/ceph.py", line 1978, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ec2a38ba1b69b5357de8f0e8f96b00b4a577169d/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: MismatchedFree operator delete[](void*, unsigned long) MonClient::get_monmap_and_config() 2024-06-28T16:17:09.703 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-28T16:17:09.703 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-28T16:17:09.719 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-28T16:17:09.782 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-28T16:17:09.796 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-28T16:17:09.797 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-28T16:17:09.800 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-28T16:17:09.815 INFO:teuthology.orchestra.run.smithi178.stdout: remote refid st t when poll reach delay offset jitter 2024-06-28T16:17:09.815 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================== 2024-06-28T16:17:09.816 INFO:teuthology.orchestra.run.smithi178.stdout:+hv01.front.sepi 67.205.162.81 3 u 24 128 377 0.107 -4.100 0.232 2024-06-28T16:17:09.816 INFO:teuthology.orchestra.run.smithi178.stdout:+hv02.front.sepi 63.231.80.2 3 u 25 128 377 0.090 +1.678 0.176 2024-06-28T16:17:09.816 INFO:teuthology.orchestra.run.smithi178.stdout:*hv03.front.sepi 74.6.168.72 3 u 8 128 377 0.089 -0.211 0.095 2024-06-28T16:17:09.816 INFO:teuthology.orchestra.run.smithi178.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-06-28T16:17:09.817 INFO:teuthology.orchestra.run.smithi195.stdout: remote refid st t when poll reach delay offset jitter 2024-06-28T16:17:09.818 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================== 2024-06-28T16:17:09.818 INFO:teuthology.orchestra.run.smithi195.stdout:+hv01.front.sepi 67.205.162.81 3 u 158 256 377 0.182 -6.687 0.406 2024-06-28T16:17:09.818 INFO:teuthology.orchestra.run.smithi195.stdout:+hv02.front.sepi 63.231.80.2 3 u 7 256 377 0.187 -1.023 0.670 2024-06-28T16:17:09.818 INFO:teuthology.orchestra.run.smithi195.stdout:*hv03.front.sepi 74.6.168.72 3 u 157 256 377 0.196 -2.228 0.928 2024-06-28T16:17:09.818 INFO:teuthology.orchestra.run.smithi195.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-06-28T16:17:09.819 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-28T16:17:09.837 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-28T16:17:09.837 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-28T16:17:09.863 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-28T16:17:09.896 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-28T16:17:09.942 INFO:teuthology.task.internal:Duration was 5235.246566 seconds 2024-06-28T16:17:09.942 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-28T16:17:09.978 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-28T16:17:09.979 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-28T16:17:09.981 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-28T16:17:10.020 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-28T16:17:10.020 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-06-28T16:17:10.020 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-28T16:17:10.069 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2024-06-28T16:17:10.069 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-28T16:17:10.079 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-28T16:17:10.082 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-28T16:17:10.113 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-28T16:17:10.147 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-28T16:17:10.147 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-28T16:17:10.181 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-28T16:17:10.244 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-28T16:17:10.259 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-28T16:17:10.260 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-28T16:17:10.273 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-28T16:17:10.284 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-28T16:17:10.305 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-28T16:17:10.317 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-28T16:17:10.325 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-06-28T16:17:10.336 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2024-06-28T16:17:10.464 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-28T16:17:10.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T16:17:10.470 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-28T16:17:10.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-28T16:17:10.510 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-28T16:17:10.526 INFO:teuthology.task.internal:Transferring archived files... 2024-06-28T16:17:10.528 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037/remote/smithi178 2024-06-28T16:17:10.529 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-28T16:17:10.554 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/cbodley-2024-06-28_14:14:22-rgw-wip-cbodley-testing-distro-default-smithi/7778037/remote/smithi195 2024-06-28T16:17:10.554 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-28T16:17:10.577 INFO:teuthology.task.internal:Removing archive directory... 2024-06-28T16:17:10.578 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-28T16:17:10.590 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-28T16:17:10.626 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-28T16:17:10.643 INFO:teuthology.task.internal:Not uploading archives. 2024-06-28T16:17:10.643 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-28T16:17:10.670 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-28T16:17:10.670 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-28T16:17:10.674 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-28T16:17:10.677 INFO:teuthology.orchestra.run.smithi178.stdout: 655475 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 28 16:17 /home/ubuntu/cephtest 2024-06-28T16:17:10.679 INFO:teuthology.orchestra.run.smithi195.stdout: 658467 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 28 16:17 /home/ubuntu/cephtest 2024-06-28T16:17:10.681 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-28T16:17:10.702 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-28T16:17:10.737 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: Exception: valgrind error: MismatchedFree operator delete[](void*, unsigned long) MonClient::get_monmap_and_config() 2024-06-28T16:17:10.742 INFO:teuthology.run:Summary data: description: rgw/verify/{0-install accounts$/{tenant} clusters/fixed-2 datacache/rgw-datacache frontend/beast ignore-pg-availability inline-data$/{off} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec-profile s3tests-branch sharding$/{single} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} duration: 5235.246565580368 failure_reason: 'Command failed (s3 tests against rgw) on smithi178 with status 1: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m ''not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3''"' flavor: default owner: scheduled_cbodley@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b33761ec7fb144c6b333ea70d845b0c7 status: fail success: false 2024-06-28T16:17:10.742 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-28T16:17:10.835 INFO:teuthology.run:FAIL